[ECOS] gdb stubs problem

[ECOS] gdb stubs problem

[Bob Cagle]

I'm having an issue with gdb/redboot/eCos when my application hits a breakpoint.

I'm using an ecos-3.0 [+] port from a vendor (rather not say who at
the moment, but the cpu is a mips32 24kec).
I added a network driver to Redboot, and am running gdb stubs there.
For a small test eCos application, everything works great. I can load,
run, interrupt system, break-point, continue, etc. No problems.

When I try debugging a larger eCos application (30Mbyte elf image, ~70
threads), gdb works okay until the app starts spawning lots of threads
( which are a mixture of eCos threads and pthreads). After that point,
I can start/pause the application repeatedly without issue, but if I
try to look at another thread's stack, or hit a breakpoint in one of
the new threads, things go badly.

By badly I mean something in eCos seems to get into a tight loop and
the application console becomes very laggy. gdb starts printing
'ignoring protocol error' repeatedly. From wireshark, it appears that
gdb and stubs are exchanging only ack and nack messages (+ and -).
Not sure why the target is sending multiple T05thread messages, or why
the acks seem to get out of order with the requests. I can still ping
the Redboot IP, but if I try to ping the eCos IP it is very laggy.

Wireshark analysis of tcp stream from when breakpoint is hit:
$T05thread:0000000a;25:1c870e80;1d:b02a5781;#37+
$g#67
$T05thread:0000000a;25:1c870e80;1d:b02a5781;#37+
$T05thread:0000000a;25:1c870e80;1d:b02a5781;#37+
$T05thread:0000000a;25:1c870e80;1d:b02a5781;#37+
$g#67
$g#67
$g#67
---+---+--  [this sequence repeats forever]

A similar trace from my test application when successfully hitting a breakpoint:

$T02thread:00000005;25:48260480;1d:a82f1280;#aa+
$g#67+
$00000000feffffff0100000000001380000000000200000000000380000003800104001000000380140b
0000c8c82780000000000d0005050e0005050f00050510000505110005051200050513000505140005051
50005051600050517000505180005051900050588301280a8b01180901d1280a82f1280a82f1280482604
8003040010c3c300000068000000000000248000504826048000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000#a4+
$X800401d8,4:���}#a6+
$OK#9a+
$X8004f5ac,4:s=^L#da+
$OK#9a+
$X8004f9d8,4:
$OK#9a+
$T5#89+
$OK#9a+
$T4#88+
$OK#9a+
..., everything works fine here.

My current hypothesis is that I'm hitting some kind of limit in
gdb/stubs/ecos related to using lots of threads (or a mixture of posix
and eCos threads), but not sure where to start looking. I'm hoping
somebody has seen something like this before or has some ideas for
debugging this problem.

Regards,
Bob Cagle
bobcagle@gmail.com

--
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] GDB stubs problem

[Gary Thomas]

You are definitely having trouble with the serial port not being able to
keep up with GDB (serial overrun).

The first, and indeed simplest, thing to try would be to rebuild the GDB stubs
for the board to run at 9600 baud instead of 19200.  Then try it.

On 29-May-2001 xavier wrote:
> 
> My target is a MIPS R3000 architecture, not x86, so I can't run the Linux's GDB.
> 
> I turned on the remote debug in GDB as following:
> 
> (gdb) set remotebaud 19200
> (gdb) set debug remote 1
> (gdb) target remote com1
> Remote debugging using com1
> Sending packet: $Hc-1#09...Sending packet: $Hc-1#09...Sending packet: $Hc-1#09..
> .Sending packet: $Hc-1#09...Timed out.
> Timed out.
> Timed out.
> Ignoring packet error, continuing...
> Sending packet: $qC#b4...Sending packet: $qC#b4...Sending packet: $qC#b4...Sendi
> ng packet: $qC#b4...Timed out.
> Timed out.
> Timed out.
> Ignoring packet error, continuing...
> Sending packet: $qOffsets#4b...Sending packet: $qOffsets#4b...Sending packet: $q
> Offsets#4b...Sending packet: $qOffsets#4b...Timed out.
> Timed out.
> Timed out.
> Ignoring packet error, continuing...
> Couldn't establish connection to remote target
> Malformed response to offset query, timeout
> (gdb)
> 
> Then I logged every characters came from serial port in function hal_jmr_get_char()
> in packages/hal/mips/jmr3904/v1_3_1/src/plf_stub.c. The results show that when
> the GDB tried to connect to the target, it sent the following packets:
> 
> +$H9$Hc9$Hc9$Hc9---+$4$qC4$qC4$qC4---+$b$qOb$qOb$qOb---+
> 
> But according the debug messages shown on GDB's console above, the packets are:
> 
> $Hc-1#09$Hc-1#09$Hc-1#09$Hc-1#09$qC#b4$qC#b4$qC#b4$qC#b4
> $qOffsets#4b$qOffsets#4b$qOffsets#4b$qOffsets#4b
> 
> Why aren't they equal? I guess it's the problem that causes the connection failed.
> Could someone help me?
> 
> Thanks.
> 
>> I've had the same problem.  Are you, by chance, trying to use gdb5.0
>> for x86?  If so, try the gdb that comes with your Linux distrubution.
>> I found that gdb5.0 didn't work.
>> 
>> On Mon, May 28, 2001 at 08:13:22PM +0800, xavier wrote:
>> > 
>> > Hi,
>> > 
>> > I'm trying GDB stubs support in eCos. I build both gdb_module.img and
>> > a test program with GDB stubs included. When the gdb_module.img
>> > starts in the target, it sends the following packet:
>> > 
>> > ++$T0525:a0307a34;1d:80005420;#2
>> > 
>> > It seems OK. The gdb_module is waiting for a connection. Then I connect
>> > GDB to the target. The target receives the following packet:
>> > 
>> > +$H9$Hc9$Hc9$Hc9---+$4$qC4$qC4$qC4---+$b$qOb$qOb$qOb---+
>> > 
>> > (gdb)  target remote com1
>> > Remote debugging using com1
>> > Ignoring packet error, continuing...
>> > Ignoring packet error, continuing...
>> > Ignoring packet error, continuing...
>> > Couldn't establish connection to remote target
>> > Malformed response to offset query, timeout
>> > (gdb)
>> > 
>> > I looked into the source code and found that in getpacket() in generic-stubs.c,
>> > __add_char_to_packet() expects a '#' character to finish a packet but there is
>> > no '#' character sent from GDB. So getpacket() ran forever.
>> > 
>> > Any suggestion?
>> > 
>> 

Re: [ECOS] GDB stubs problem

[xavier]

My target is a MIPS R3000 architecture, not x86, so I can't run the Linux's GDB.

I turned on the remote debug in GDB as following:

(gdb) set remotebaud 19200
(gdb) set debug remote 1
(gdb) target remote com1
Remote debugging using com1
Sending packet: $Hc-1#09...Sending packet: $Hc-1#09...Sending packet: $Hc-1#09..
.Sending packet: $Hc-1#09...Timed out.
Timed out.
Timed out.
Ignoring packet error, continuing...
Sending packet: $qC#b4...Sending packet: $qC#b4...Sending packet: $qC#b4...Sendi
ng packet: $qC#b4...Timed out.
Timed out.
Timed out.
Ignoring packet error, continuing...
Sending packet: $qOffsets#4b...Sending packet: $qOffsets#4b...Sending packet: $q
Offsets#4b...Sending packet: $qOffsets#4b...Timed out.
Timed out.
Timed out.
Ignoring packet error, continuing...
Couldn't establish connection to remote target
Malformed response to offset query, timeout
(gdb)

Then I logged every characters came from serial port in function hal_jmr_get_char()
in packages/hal/mips/jmr3904/v1_3_1/src/plf_stub.c. The results show that when
the GDB tried to connect to the target, it sent the following packets:

+$H9$Hc9$Hc9$Hc9---+$4$qC4$qC4$qC4---+$b$qOb$qOb$qOb---+

But according the debug messages shown on GDB's console above, the packets are:

$Hc-1#09$Hc-1#09$Hc-1#09$Hc-1#09$qC#b4$qC#b4$qC#b4$qC#b4
$qOffsets#4b$qOffsets#4b$qOffsets#4b$qOffsets#4b

Why aren't they equal? I guess it's the problem that causes the connection failed.
Could someone help me?

Thanks.

> I've had the same problem.  Are you, by chance, trying to use gdb5.0
> for x86?  If so, try the gdb that comes with your Linux distrubution.
> I found that gdb5.0 didn't work.
> 
> On Mon, May 28, 2001 at 08:13:22PM +0800, xavier wrote:
> > 
> > Hi,
> > 
> > I'm trying GDB stubs support in eCos. I build both gdb_module.img and
> > a test program with GDB stubs included. When the gdb_module.img
> > starts in the target, it sends the following packet:
> > 
> > ++$T0525:a0307a34;1d:80005420;#2
> > 
> > It seems OK. The gdb_module is waiting for a connection. Then I connect
> > GDB to the target. The target receives the following packet:
> > 
> > +$H9$Hc9$Hc9$Hc9---+$4$qC4$qC4$qC4---+$b$qOb$qOb$qOb---+
> > 
> > (gdb)  target remote com1
> > Remote debugging using com1
> > Ignoring packet error, continuing...
> > Ignoring packet error, continuing...
> > Ignoring packet error, continuing...
> > Couldn't establish connection to remote target
> > Malformed response to offset query, timeout
> > (gdb)
> > 
> > I looked into the source code and found that in getpacket() in generic-stubs.c,
> > __add_char_to_packet() expects a '#' character to finish a packet but there is
> > no '#' character sent from GDB. So getpacket() ran forever.
> > 
> > Any suggestion?
> > 
> 

Re: [ECOS] GDB stubs problem

[elf]

I've had the same problem.  Are you, by chance, trying to use gdb5.0
for x86?  If so, try the gdb that comes with your Linux distrubution.
I found that gdb5.0 didn't work.

On Mon, May 28, 2001 at 08:13:22PM +0800, xavier wrote:
> 
> Hi,
> 
> I'm trying GDB stubs support in eCos. I build both gdb_module.img and
> a test program with GDB stubs included. When the gdb_module.img
> starts in the target, it sends the following packet:
> 
> ++$T0525:a0307a34;1d:80005420;#2
> 
> It seems OK. The gdb_module is waiting for a connection. Then I connect
> GDB to the target. The target receives the following packet:
> 
> +$H9$Hc9$Hc9$Hc9---+$4$qC4$qC4$qC4---+$b$qOb$qOb$qOb---+
> 
> (gdb)  target remote com1
> Remote debugging using com1
> Ignoring packet error, continuing...
> Ignoring packet error, continuing...
> Ignoring packet error, continuing...
> Couldn't establish connection to remote target
> Malformed response to offset query, timeout
> (gdb)
> 
> I looked into the source code and found that in getpacket() in generic-stubs.c,
> __add_char_to_packet() expects a '#' character to finish a packet but there is
> no '#' character sent from GDB. So getpacket() ran forever.
> 
> Any suggestion?
> 

[ECOS] GDB stubs problem

[xavier]

Hi,

I'm trying GDB stubs support in eCos. I build both gdb_module.img and
a test program with GDB stubs included. When the gdb_module.img
starts in the target, it sends the following packet:

++$T0525:a0307a34;1d:80005420;#2

It seems OK. The gdb_module is waiting for a connection. Then I connect
GDB to the target. The target receives the following packet:

+$H9$Hc9$Hc9$Hc9---+$4$qC4$qC4$qC4---+$b$qOb$qOb$qOb---+

(gdb)  target remote com1
Remote debugging using com1
Ignoring packet error, continuing...
Ignoring packet error, continuing...
Ignoring packet error, continuing...
Couldn't establish connection to remote target
Malformed response to offset query, timeout
(gdb)

I looked into the source code and found that in getpacket() in generic-stubs.c,
__add_char_to_packet() expects a '#' character to finish a packet but there is
no '#' character sent from GDB. So getpacket() ran forever.

Any suggestion?