Stepping of multithreaded application

Stepping of multithreaded application

[Tomas Martinec]

Hello,

I am developing an Eclipse plugin that allows debugging of a MIPS
virtual machine. The GDB is used as a debugger backend. I have
difficulties with debugging multiprocessor machines (processors are
represented as threads in the GDB).

Stepping of different threads does not seem to work well. When I
change the current thread in the eclipse and apply the step command,
the debugger is stopped in the previous thread. I expect that the
debugger should be stopped at the next line of the newly set thread.
Is my expectation right?

I minimalized the scenario to the GDB/virtual machine only and I got
an example gdb session. The structure of the session is basically
this:

tomas@tomas-laptop:~/workspaceGDB/gdb-7.2/gdb$ ./gdb --interpreter mi2
// use the MI
// connect to the machine
target remote :10001
// bsp_start is the first C function of the bootstrap processor
break bsp_start
// go to the C code
continue
// get information about the other processors/threads
info threads
// do a step with the third processor
-exec-next --thread 3 1
*stopped,reason="signal-received",signal-name="SIGTRAP",signal-meaning="Trace/breakpoint
trap",frame={addr="0x80004354",func="print",args=[{name="text",value="0x0"}],file="drivers/printer.c",fullname="/home/tomas/runtime-EclipseApplication/kalisto/kernel/drivers/printer.c",line="27"},thread-id="1",stopped-threads="all"
// the first processor was stepped instead the third one

Here is the complete session log (without the remote debugging messages):

tomas@tomas-laptop:~/workspaceGDB/gdb-7.2/gdb$ ./gdb --interpreter mi2
=thread-group-added,id="i1"
~"GNU gdb (GDB) 7.2\n"
~"Copyright (C) 2010 Free Software Foundation, Inc.\n"
~"License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>\nThis is free software: you are
free to change and redistribute it.\nThere is NO WARRANTY, to the
extent permitted by law.  Type \"show copying\"\nand \"show warranty\"
for details.\n"
~"This GDB was configured as \"--host=i686-pc-linux-gnu
--target=mips\".\nFor bug reporting instructions, please see:\n"
~"<http://www.gnu.org/software/gdb/bugs/>.\n"
~"The target is assumed to be little endian\n"
(gdb)
set debug remote 1
^done
(gdb)
target remote :10001
~"Remote debugging using :10001\n"
=thread-group-started,id="i1",pid="42000"
=thread-created,id="1",group-id="i1"
~"0xbfc00000 in ?? ()\n"
*stopped,frame={addr="0xbfc00000",func="??",args=[]},thread-id="1",stopped-threads="all"
^done
(gdb)
break bsp_start
~"Breakpoint 1 at 0x80002540: file main.c, line 69.\n"
^done
(gdb)
continue
~"Continuing.\n"
^running
*running,thread-id="all"
(gdb)
~"\nBreakpoint "
~"1, bsp_start () at main.c:69\n"
~"69\t\tprint (\"Initializing Kalisto\\n\");\n"
*stopped,frame={addr="0x80002540",func="bsp_start",args=[],file="main.c",fullname="/home/tomas/runtime-EclipseApplication/kalisto/kernel/main.c",line="69"},thread-id="1",stopped-threads="all"
(gdb)
info threads
=thread-created,id="2",group-id="i1"
~"[New Thread 2]\n"
=thread-created,id="3",group-id="i1"
~"[New Thread 3]\n"
~"  3 Thread 3 (CPU 3)  0x80002654 in ap_start () at main.c:129\n"
~"  2 Thread 2 (CPU 2)  0x80002654 in ap_start () at main.c:129\n"
~"* 1 Thread 1 (CPU 1)  bsp_start () at main.c:69\n"
^done
(gdb)
-exec-next --thread 3 1
^running
*running,thread-id="1"
(gdb)
*running,thread-id="all"
~"[Switching to Thread 1]\n"
*stopped,reason="signal-received",signal-name="SIGTRAP",signal-meaning="Trace/breakpoint
trap",frame={addr="0x80004354",func="print",args=[{name="text",value="0x0"}],file="drivers/printer.c",fullname="/home/tomas/runtime-EclipseApplication/kalisto/kernel/drivers/printer.c",line="27"},thread-id="1",stopped-threads="all"
=thread-selected,id="1"
(gdb)

I have also the same log but with the remote debugging messages. It is
quite long, so I paste only the relevant part:

-exec-next --thread 3 1
&"Sending packet: $Hg3#e2..."
&"Ack\n"
&"Packet received: OK\n"
&"Sending packet: $g#67..."
&"Ack\n"
&"Packet received:
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000200000000000080d8060080000000004c030080040040000000000000000000000000000000000054260080\n"
&"Sending packet: $Hg1#e0..."
&"Ack\n"
&"Packet received: OK\n"
&"Sending packet: $g#67..."
&"Ack\n"
&"Packet received:
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000080d8040080d804008038030080040040000000000000000000000000000000000040250080\n"
&"Sending packet: $g#67..."
&"Ack\n"
&"Packet received:
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000080d8040080d804008038030080040040000000000000000000000000000000000040250080\n"
&"Sending packet: $Hc1#dc..."
&"Ack\n"
&"Packet received: OK\n"
&"Sending packet: $s#73..."
&"Ack\n"
^running
*running,thread-id="1"
(gdb)
&"Packet received: T05thread:00000001;25:44250080;\n"
&"Sending packet: $Hg3#e2..."
&"Ack\n"
&"Packet received: OK\n"
&"Sending packet: $g#67..."
&"Ack\n"
&"Packet received:
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000200000000000080d8060080000000004c030080040040000000000000000000000000000000000058260080\n"
&"Sending packet: $Hc0#db..."
&"Ack\n"
&"Packet received: \n"
&"Sending packet: $s#73..."
&"Ack\n"
*running,thread-id="all"
&"Packet received: T05thread:00000001;25:48250080;\n"
&"Sending packet: $g#67..."
&"Ack\n"
&"Packet received:
0000000000000000000000800000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000080d8040080d80400804c250080040040000000000000000000000000000000000048250080\n"
&"Sending packet: $T3#87..."
&"Ack\n"
&"Packet received: OK\n"
&"Sending packet: $Hg3#e2..."
&"Ack\n"
&"Packet received: OK\n"
&"Sending packet: $g#67..."
&"Ack\n"
&"Packet received:
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000200000000000080d8060080d80600804c03008004004000000000000000000000000000000000005c260080\n"
&"Sending packet: $Hg1#e0..."
&"Ack\n"
&"Packet received: OK\n"
&"Sending packet: $g#67..."
&"Ack\n"
&"Packet received:
0000000000000000000000800000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000080d8040080d80400804c250080040040000000000000000000000000000000000048250080\n"
&"Sending packet: $Z0,80002540,4#a9..."
&"Ack\n"
&"Packet received: OK\n"
&"Sending packet: $Hg3#e2..."
&"Ack\n"
&"Packet received: OK\n"
&"Sending packet: $g#67..."
&"Ack\n"
&"Packet received:
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000200000000000080d8060080d80600804c03008004004000000000000000000000000000000000005c260080\n"
&"Sending packet: $s#73..."
&"Ack\n"
&"Packet received: T05thread:00000001;25:54430080;\n"
&"Sending packet: $g#67..."
&"Ack\n"
&"Packet received:
00000000000000000000008000000000ec430080000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000080d8040080d80400804c250080040040000000000000000000000000000000000054430080\n"
~"[Switching to Thread 1]\n"
&"Sending packet: $z0,80002540,4#c9..."
&"Ack\n"
&"Packet received: OK\n"
&"Sending packet: $m800004c0,40#bc..."
&"Ack\n"
&"Packet received:
00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000038030080\n"
*stopped,reason="signal-received",signal-name="SIGTRAP",signal-meaning="Trace/breakpoint
trap",frame={addr="0x80004354",func="print",args=[{name="text",value="0x0"}],file="drivers/printer.c",fullname="/home/tomas/runtime-EclipseApplication/kalisto/kernel/drivers/printer.c",line="27"},thread-id="1",stopped-threads="all"
=thread-selected,id="1"

From the debugging messages I can see, that the GDB steps twice. The
first step is for the first thread - maybe because the first thread
has just raised the breakpoint. Before the second step the GDB tries
to change the current thread in the virtual machine to 0 (means ANY
according to the doc). That is not supported in the virtual machine
and the machine responds so. I believe not supporting the Hc0 command
is not wrong for the virtual machine. Anyway the second step is aimed
for ANY thread and it is applied for the first thread, which both
leads to the unexpected behaviour. I think that the GDB should send
Hc3 command to the simulator before the second step.

I have also tried to find out why the Hc3 command is not send from the
GDB sources. The thread id that will be step is determined in resume
function of target.c. In my case the RESUME_ALL ptid is used. That
ptid leads to the Hc0 command. So far I have not understood why the
resume_ptid is kept in the default RESUME_ALL.

Sorry for quite a story. I have been solving it for some time and
advice would be appreciated. Any idea why the GDB does not send the
Hc3 command? Maybe I just cannot see something simple.

Regards,
Tomas

Re: Stepping of multithreaded application

[Pedro Alves]

On Tuesday 10 May 2011 17:27:20, Tomas Martinec wrote:
> Sorry for quite a story. I have been solving it for some time and
> advice would be appreciated. Any idea why the GDB does not send the
> Hc3 command? Maybe I just cannot see something simple.

Can you paste the relevant logs again, but with "set debug infrun 1"
enabled in addition to "set debug remote 1" ?  It's probably possible to
infer what's going on from the remote log alone, but the infrun
log should make it easier.

-- 
Pedro Alves

Re: Stepping of multithreaded application

[Tomas Martinec]

> Can you paste the relevant logs again, but with "set debug infrun 1"
> enabled in addition to "set debug remote 1" ?  It's probably possible to
> infer what's going on from the remote log alone, but the infrun
> log should make it easier.

Here is the log with the infrun debug. I cannot see much from it on
the first sight.

-exec-next --thread 3 1
&"Sending packet: $Hg3#e2..."
&"Ack\n"
&"Packet received: OK\n"
&"Sending packet: $g#67..."
&"Ack\n"
&"Packet received:
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000200000000000080d8060080000000004c030080040040000000000000000000000000000000000054260080\n"
&"infrun: clear_proceed_status_thread (Thread 3)\n"
&"infrun: clear_proceed_status_thread (Thread 2)\n"
&"infrun: clear_proceed_status_thread (Thread 1)\n"
&"infrun: proceed (addr=0xffffffff, signal=144, step=1)\n"
&"Sending packet: $Hg1#e0..."
&"Ack\n"
&"Packet received: OK\n"
&"Sending packet: $g#67..."
&"Ack\n"
&"Packet received:
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000080d8040080d804008038030080040040000000000000000000000000000000000040250080\n"
&"Sending packet: $g#67..."
&"Ack\n"
&"Packet received:
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000080d8040080d804008038030080040040000000000000000000000000000000000040250080\n"
&"infrun: resume (step=1, signal=0), trap_expected=1\n"
&"Sending packet: $Hc1#dc..."
&"Ack\n"
&"Packet received: OK\n"
&"Sending packet: $s#73..."
&"Ack\n"
^running
*running,thread-id="1"
(gdb)
&"infrun: wait_for_inferior (treat_exec_as_sigtrap=0)\n"
&"Packet received: T05thread:00000001;25:44250080;\n"
&"infrun: target_wait (-1, status) =\ninfrun:   42000 [Thread
1],\ninfrun:   status->kind = stopped, signal = SIGTRAP\n"
&"infrun: infwait_normal_state\n"
&"infrun: TARGET_WAITKIND_STOPPED\n"
&"infrun: stop_pc = 0x80002544\n"
&"infrun: handling deferred step\n"
&"Sending packet: $Hg3#e2..."
&"Ack\n"
&"Packet received: OK\n"
&"Sending packet: $g#67..."
&"Ack\n"
&"Packet received:
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000200000000000080d8060080000000004c030080040040000000000000000000000000000000000058260080\n"
&"infrun: resume (step=1, signal=0), trap_expected=0\n"
&"Sending packet: $Hc0#db..."
&"Ack\n"
&"Packet received: \n"
&"Sending packet: $s#73..."
&"Ack\n"
*running,thread-id="all"
&"infrun: prepare_to_wait\n"
&"Packet received: T05thread:00000001;25:48250080;\n"
&"infrun: target_wait (-1, status) =\ninfrun:   42000 [Thread
1],\ninfrun:   status->kind = stopped, signal = SIGTRAP\n"
&"infrun: infwait_normal_state\n"
&"infrun: TARGET_WAITKIND_STOPPED\n"
&"infrun: stop_pc = 0x80002548\n"
&"infrun: context switch\n"
&"infrun: Switching context from Thread 3 to Thread 1\n"
&"Sending packet: $g#67..."
&"Ack\n"
&"Packet received:
0000000000000000000000800000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000080d8040080d80400804c250080040040000000000000000000000000000000000048250080\n"
&"Sending packet: $T3#87..."
&"Ack\n"
&"Packet received: OK\n"
&"infrun: switching back to stepped thread\n"
&"infrun: Switching context from Thread 1 to Thread 3\n"
&"Sending packet: $Hg3#e2..."
&"Ack\n"
&"Packet received: OK\n"
&"Sending packet: $g#67..."
&"Ack\n"
&"Packet received:
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000200000000000080d8060080d80600804c03008004004000000000000000000000000000000000005c260080\n"
&"Sending packet: $Hg1#e0..."
&"Ack\n"
&"Packet received: OK\n"
&"Sending packet: $g#67..."
&"Ack\n"
&"Packet received:
0000000000000000000000800000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000080d8040080d80400804c250080040040000000000000000000000000000000000048250080\n"
&"Sending packet: $Z0,80002540,4#a9..."
&"Ack\n"
&"Packet received: OK\n"
&"Sending packet: $Hg3#e2..."
&"Ack\n"
&"Packet received: OK\n"
&"Sending packet: $g#67..."
&"Ack\n"
&"Packet received:
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000200000000000080d8060080d80600804c03008004004000000000000000000000000000000000005c260080\n"
&"infrun: resume (step=1, signal=0), trap_expected=0\n"
&"Sending packet: $s#73..."
&"Ack\n"
&"infrun: prepare_to_wait\n"
&"Packet received: T05thread:00000001;25:54430080;\n"
&"infrun: target_wait (-1, status) =\ninfrun:   42000 [Thread
1],\ninfrun:   status->kind = stopped, signal = SIGTRAP\n"
&"infrun: infwait_normal_state\n"
&"infrun: TARGET_WAITKIND_STOPPED\n"
&"infrun: stop_pc = 0x80004354\n"
&"infrun: context switch\n"
&"infrun: Switching context from Thread 3 to Thread 1\n"
&"Sending packet: $g#67..."
&"Ack\n"
&"Packet received:
00000000000000000000008000000000ec430080000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000080d8040080d80400804c250080040040000000000000000000000000000000000054430080\n"
&"infrun: random signal 5\n"
&"infrun: stop_stepping\n"
~"[Switching to Thread 1]\n"
&"Sending packet: $z0,80002540,4#c9..."
&"Ack\n"
&"Packet received: OK\n"
&"Sending packet: $m800004c0,40#bc..."
&"Ack\n"
&"Packet received:
00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000038030080\n"
*stopped,reason="signal-received",signal-name="SIGTRAP",signal-meaning="Trace/breakpoint
trap",frame={addr="0x80004354",func="print",args=[{name="text",value="0x0"}],file="drivers/printer.c",fullname="/home/tomas/runtime-EclipseApplication/kalisto/kernel/drivers/printer.c",line="27"},thread-id="1",stopped-threads="all"
=thread-selected,id="1"
(gdb)

Tomas

Re: Stepping of multithreaded application

[Pedro Alves]

On Tuesday 10 May 2011 17:27:20, Tomas Martinec wrote:
> From the debugging messages I can see, that the GDB steps twice. The
> first step is for the first thread - maybe because the first thread
> has just raised the breakpoint. Before the second step the GDB tries
> to change the current thread in the virtual machine to 0 (means ANY
> according to the doc). 

Right.

> That is not supported in the virtual machine
> and the machine responds so. I believe not supporting the Hc0 command
> is not wrong for the virtual machine. 

IMO, it's wrong.  But I don't think it'd make a difference, unless
the target gives preference to the already/previously selected
Hg/c thread.

In any case, you should really teach your VM about the vCont
packet instead of 'Hc'/'s'/'c'.

> Anyway the second step is aimed
> for ANY thread and it is applied for the first thread, which both
> leads to the unexpected behaviour. I think that the GDB should send
> Hc3 command to the simulator before the second step.

I think so too.

> I have also tried to find out why the Hc3 command is not send from the
> GDB sources. The thread id that will be step is determined in resume
> function of target.c. In my case the RESUME_ALL ptid is used. That
> ptid leads to the Hc0 command. So far I have not understood why the
> resume_ptid is kept in the default RESUME_ALL.

That's due to the "set scheduler-locking" setting.  Even though
you're "next"ing thread 3, by default, the other threads run
a bit too.  See patch below.

> -exec-next --thread 3 1
> &"Sending packet: $Hg3#e2..."
> &"Ack\n"
> &"Packet received: OK\n"
> &"Sending packet: $g#67..."
> &"Ack\n"
> &"Packet received:
> 0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000200000000000080d8060080000000004c030080040040000000000000000000000000000000000054260080\n"
> &"infrun: clear_proceed_status_thread (Thread 3)\n"
> &"infrun: clear_proceed_status_thread (Thread 2)\n"
> &"infrun: clear_proceed_status_thread (Thread 1)\n"
> &"infrun: proceed (addr=0xffffffff, signal=144, step=1)\n"
> &"Sending packet: $Hg1#e0..."
> &"Ack\n"
> &"Packet received: OK\n"
> &"Sending packet: $g#67..."
> &"Ack\n"
> &"Packet received:
> 0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000080d8040080d804008038030080040040000000000000000000000000000000000040250080\n"
> &"Sending packet: $g#67..."
> &"Ack\n"
> &"Packet received:
> 0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000080d8040080d804008038030080040040000000000000000000000000000000000040250080\n"
> &"infrun: resume (step=1, signal=0), trap_expected=1\n"
> &"Sending packet: $Hc1#dc..."
> &"Ack\n"
> &"Packet received: OK\n"
> &"Sending packet: $s#73..."

We want to "next" thread 3 (which resumes all other
threads as well, see the manual for "set scheduler-locking"),
but thread 1 was stopped at a breakpoint, so we need to move
it past the breakpoint first (infrun.c:prepare_to_proceed).

> &"Ack\n"
> ^running
> *running,thread-id="1"
> (gdb)
> &"infrun: wait_for_inferior (treat_exec_as_sigtrap=0)\n"
> &"Packet received: T05thread:00000001;25:44250080;\n"
> &"infrun: target_wait (-1, status) =\ninfrun:   42000 [Thread
> 1],\ninfrun:   status->kind = stopped, signal = SIGTRAP\n"
> &"infrun: infwait_normal_state\n"
> &"infrun: TARGET_WAITKIND_STOPPED\n"
> &"infrun: stop_pc = 0x80002544\n"
> &"infrun: handling deferred step\n"

Done.  Go back to stepping thread 3 and resuming all
others...

> &"Sending packet: $Hg3#e2..."
> &"Ack\n"
> &"Packet received: OK\n"
> &"Sending packet: $g#67..."
> &"Ack\n"
> &"Packet received:
> 0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000200000000000080d8060080000000004c030080040040000000000000000000000000000000000058260080\n"
> &"infrun: resume (step=1, signal=0), trap_expected=0\n"
> &"Sending packet: $Hc0#db..."

Select any thread in the target...  (Indeed strange.)

> &"Ack\n"
> &"Packet received: \n"
> &"Sending packet: $s#73..."

Step the target...

> &"Ack\n"
> *running,thread-id="all"
> &"infrun: prepare_to_wait\n"
> &"Packet received: T05thread:00000001;25:48250080;\n"

The target finished stepping thread 1.  But we wanted
thread 3 to be stepped...

Please try the patch below.  It's "any_thread_ptid" that causes "Hc0".
If we're stepping, obviously we need to step the correct thread, not
a random one.  I don't really understand how this hasn't been a
problem before.

As explained above and in the comment in the patch, you
should really teach your target to support the vCont packet.
Hc/s are deprecated for multi-threading.

Pedro Alves

2011-05-11  Pedro Alves  <pedro@codesourcery.com>

	gdb/
	* remote.c (remote_resume): Set the continue thread to
	inferior_ptid if the core passed in a wildcard resume, instead of
	setting it to a random thread.

---
 gdb/remote.c |   13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

Index: src/gdb/remote.c
===================================================================
--- src.orig/gdb/remote.c	2011-05-11 15:17:48.658167999 +0100
+++ src/gdb/remote.c	2011-05-11 18:00:13.988168000 +0100
@@ -4542,8 +4542,17 @@ remote_resume (struct target_ops *ops,
 
   /* All other supported resume packets do use Hc, so set the continue
      thread.  */
-  if (ptid_equal (ptid, minus_one_ptid))
-    set_continue_thread (any_thread_ptid);
+
+  /* A wildcard PTID means either "step the current thread and
+     continue others", or "continue all threads".
+
+     A specific PTID means `step only this process id'.  Problem is
+     the 's' packet is ambiguous: "step just this thread and leave
+     others stopped", or "step this thread and continue all others"?
+     vCont doesn't have this problem.  */
+
+  if (ptid_equal (minus_one_ptid, ptid) || ptid_is_pid (ptid))
+    set_continue_thread (inferior_ptid);
   else
     set_continue_thread (ptid);
 

Re: Stepping of multithreaded application

[Tomas Martinec]

>> That is not supported in the virtual machine
>> and the machine responds so. I believe not supporting the Hc0 command
>> is not wrong for the virtual machine.
>
> IMO, it's wrong.  But I don't think it'd make a difference, unless
> the target gives preference to the already/previously selected
> Hg/c thread.

> In any case, you should really teach your VM about the vCont
> packet instead of 'Hc'/'s'/'c'.

I will implement both the Hc0 and vCont in the virtual machine. But at
first I will try to make just the Hc/s/c commands working.

> Please try the patch below.  It's "any_thread_ptid" that causes "Hc0".
> If we're stepping, obviously we need to step the correct thread, not
> a random one.  I don't really understand how this hasn't been a
> problem before.

There was another problem after applying the patch. The GDB resumed
the thread 1 after the thread 3 stepped. I think I have made a fix:

--- infrun_old.c	2010-08-31 21:31:23.000000000 +0200
+++ infrun.c	2011-05-12 12:20:15.000000000 +0200
@@ -3513,6 +3513,8 @@
 	      singlestep_breakpoints_inserted_p = 0;
 	    }

+	  ecs->event_thread->trap_expected = 0;
+
 	  /* Note: We do not call context_switch at this point, as the
 	     context is already set up for stepping the original thread.  */
 	  switch_to_thread (deferred_step_ptid);

The trap_expected flag of the thread 1 was not cleared after the step
of thread 1. Therefore the code has run into the following branch
during step handling of the thread 3:

infrun.c: line 4182

  /* In all-stop mode, if we're currently stepping but have stopped in
     some other thread, we need to switch back to the stepped thread.  */
  if (!non_stop)
    {
      struct thread_info *tp;

      tp = iterate_over_threads (currently_stepping_or_nexting_callback,
				 ecs->event_thread);
      if (tp)
	{
              // this branch is done, which is wrong; it is done,
because the thread 1
              // has trap_expected set and therefore the thread is
considered to be stepping
              //
              // the thread 1 is resumed in this branch; in my case
the thread 3 never finishes stepping

The debugging seems to be fine after applying my fix. Still I think
somebody more familiar with the GDB should review it.

Tomas

Re: Stepping of multithreaded application

[Pedro Alves]

On Thursday 12 May 2011 11:44:50, Tomas Martinec wrote:
> >> That is not supported in the virtual machine
> >> and the machine responds so. I believe not supporting the Hc0 command
> >> is not wrong for the virtual machine.
> >
> > IMO, it's wrong.  But I don't think it'd make a difference, unless
> > the target gives preference to the already/previously selected
> > Hg/c thread.
> 
> > In any case, you should really teach your VM about the vCont
> > packet instead of 'Hc'/'s'/'c'.
> 
> I will implement both the Hc0 and vCont in the virtual machine. But at
> first I will try to make just the Hc/s/c commands working.
> 
> > Please try the patch below.  It's "any_thread_ptid" that causes "Hc0".
> > If we're stepping, obviously we need to step the correct thread, not
> > a random one.  I don't really understand how this hasn't been a
> > problem before.
> 
> There was another problem after applying the patch. The GDB resumed
> the thread 1 after the thread 3 stepped. I think I have made a fix:
> 
> --- infrun_old.c	2010-08-31 21:31:23.000000000 +0200
> +++ infrun.c	2011-05-12 12:20:15.000000000 +0200
> @@ -3513,6 +3513,8 @@
>  	      singlestep_breakpoints_inserted_p = 0;
>  	    }
> 
> +	  ecs->event_thread->trap_expected = 0;
> +
>  	  /* Note: We do not call context_switch at this point, as the
>  	     context is already set up for stepping the original thread.  */
>  	  switch_to_thread (deferred_step_ptid);
> 
> The trap_expected flag of the thread 1 was not cleared after the step
> of thread 1. Therefore the code has run into the following branch
> during step handling of the thread 3:

This makes sense.  However ...

> 
> infrun.c: line 4182
> 
>   /* In all-stop mode, if we're currently stepping but have stopped in
>      some other thread, we need to switch back to the stepped thread.  */
>   if (!non_stop)
>     {
>       struct thread_info *tp;
> 
>       tp = iterate_over_threads (currently_stepping_or_nexting_callback,
> 				 ecs->event_thread);
>       if (tp)
> 	{
>               // this branch is done, which is wrong; it is done,
> because the thread 1
>               // has trap_expected set and therefore the thread is
> considered to be stepping
>               //
>               // the thread 1 is resumed in this branch; in my case
> the thread 3 never finishes stepping

When I try this (without your patch on x86_64-linux native, I see
is:

Breakpoint 1, thread0 (arg=0x0) at loop.c:39
39              (*myp) ++;
(gdb) info threads 
  Id   Target Id         Frame 
  3    Thread 0x7ffff703a700 (LWP 27787) "loop" 0x00007ffff78e620d in nanosleep () from /lib/libc.so.6
* 2    Thread 0x7ffff783b700 (LWP 27786) "loop" thread0 (arg=0x0) at loop.c:39
  1    Thread 0x7ffff7fd5700 (LWP 27783) "loop" 0x00007ffff7bc7285 in pthread_join () from /lib/libpthread.so.0
(gdb) t 3
[Switching to thread 3 (Thread 0x7ffff703a700 (LWP 27787))]
#0  0x00007ffff78e620d in nanosleep () from /lib/libc.so.6
(gdb) n
infrun: clear_proceed_status_thread (Thread 0x7ffff703a700 (LWP 27787))
infrun: clear_proceed_status_thread (Thread 0x7ffff783b700 (LWP 27786))
infrun: clear_proceed_status_thread (Thread 0x7ffff7fd5700 (LWP 27783))
Single stepping until exit from function nanosleep,
which has no line number information.
infrun: proceed (addr=0xffffffffffffffff, signal=144, step=1)
thread_function1: 8
thread_function0: 10
infrun: resume (step=1, signal=0), trap_expected=1
infrun: wait_for_inferior ()
infrun: target_wait (-1, status) =
infrun:   27783 [Thread 0x7ffff783b700 (LWP 27786)],
infrun:   status->kind = stopped, signal = SIGTRAP
infrun: infwait_normal_state
infrun: TARGET_WAITKIND_STOPPED
infrun: stop_pc = 0x400741
infrun: handling deferred step
infrun: resume (step=1, signal=0), trap_expected=0
infrun: prepare_to_wait
infrun: target_wait (-1, status) =
infrun:   27783 [Thread 0x7ffff703a700 (LWP 27787)],
infrun:   status->kind = stopped, signal = SIGTRAP
infrun: infwait_normal_state
infrun: TARGET_WAITKIND_STOPPED
infrun: stop_pc = 0x7ffff78e620d
infrun: switching back to stepped thread
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

This is indeed wrong.

infrun: Switching context from Thread 0x7ffff703a700 (LWP 27787) to Thread 0x7ffff783b700 (LWP 27786)
infrun: resume (step=0, signal=0), trap_expected=0

We end up continuing all threads (turn on 'set debug lin-lwp'
to get more debug detail), not stepping or continuing
a single thread.  This `resume' call clears "trap_expected",
and everything happens to work.

I imagine you're seeing a 'Hc1', 'c', and only continuing
thread 1 instead of all threads?  This is part of the
ambiguity with Hc/c/s that vCont fixes...

> The debugging seems to be fine after applying my fix. Still I think
> somebody more familiar with the GDB should review it.

I'll test and apply your fix if nothing turns up regardless.

-- 
Pedro Alves

Re: Stepping of multithreaded application

[Tomas Martinec]

> I imagine you're seeing a 'Hc1', 'c', and only continuing
> thread 1 instead of all threads?

Yes, that is right.

> I'll test and apply your fix if nothing turns up regardless.

Please let me know if you find a problem with the fix.

Tomas

Re: Stepping of multithreaded application

[Pedro Alves]

On Monday 16 May 2011 16:22:49, Tomas Martinec wrote:
> > I imagine you're seeing a 'Hc1', 'c', and only continuing
> > thread 1 instead of all threads?
> 
> Yes, that is right.

Hmm.  This is making me very nervous on changing GDB's
age old behavior...  Who knows which old targets out
there are assuming:

- "Hc 0" to mean continue all threads
- "Hc $FOO" to continue/step only thread $FOO

and picking the 's' (step) thread from the previous Hg ?

I fear we'd be switching from one broken behavior to
another.  Since new stubs can just vCont, I'm now
inclined to leave things as is.

> > I'll test and apply your fix if nothing turns up regardless.
> 
> Please let me know if you find a problem with the fix.

I've applied this fix.  Thanks.

-- 
Pedro Alves

Re: Stepping of multithreaded application

[Tomas Martinec]

> Hmm.  This is making me very nervous on changing GDB's
> age old behavior...  Who knows which old targets out
> there are assuming:
>
> - "Hc 0" to mean continue all threads
> - "Hc $FOO" to continue/step only thread $FOO
>
> and picking the 's' (step) thread from the previous Hg ?
>
> I fear we'd be switching from one broken behavior to
> another.  Since new stubs can just vCont, I'm now
> inclined to leave things as is.

Ok, I am fine with that.

Tomas