gdb arbitrarily starting threads

gdb arbitrarily starting threads

[William M. (Mike) Miller]

I have a single-threaded executable that has a number of debug-print
routines that can be called interactively in the debugger to display
data structures to stderr.  Something has changed in gdb/gcc/Cygwin
during the last year or so (I don't update my installation very often)
so that when I call one of these functions, _sometimes_ gdb will start
a new thread and put my current thread into "running" status, so that
I can't step, etc. (although the thread isn't actually running).
Here's the scenario:

1) I say "gdb xxx.exe", set a breakpoint, and run.  When I hit the
breakpoint, "info threads" shows me two threads, one for my executable
and one with some _cygtls/cygthread/ntdll frames; both are stopped.

2) I say "up" a couple of times, then "p db_foo(p)" (where db_foo is
one of these debug-print routines that displays to stderr the data
structure pointed to by "p").  Sometimes, but not always, gdb will
respond with

    [New Thread 1436.0x1f00]

and although a "bt" will show the stack as still stopped at the
breakpoint, "info threads" will show all threads as "(running)", and
typing "n" is met with the error,

    Cannot execute this command while the selected thread is running.

(Actually, for the first time just now, while running my scenario to
make sure I got all the details right for this email, I saw gdb start
a new thread in response to a "next" command --

    (gdb) n
    [New Thread 1436.0x1a0c]

after I had done a "p db_foo(p)" a couple of times without seeing a
new thread created, so apparently it's not just invoking one of these
debug-print functions that can result in this behavior.)

Changing to one of the newly-created threads reveals this stack trace:

#0  0x7733df6c in ntdll!ZwWaitForWorkViaWorkerFactory () from
/c/WINDOWS/SYSTEM32/ntdll.dll
#1  0x77320153 in ntdll!RtlRunOnceBeginInitialize () from
/c/WINDOWS/SYSTEM32/ntdll.dll
#2  0x6100626b in _cygtls::call2(unsigned long (*)(void*, void*),
void*, void*) () from /usr/bin/cygwin1.dll
#3  0x61006306 in _cygtls::call(unsigned long (*)(void*, void*),
void*) () from /usr/bin/cygwin1.dll
#4  0x610828d8 in threadfunc_fe(void*) () from /usr/bin/cygwin1.dll
#5  0x74be7c04 in KERNEL32!BaseThreadInitThunk () from
/c/WINDOWS/SYSTEM32/KERNEL32.DLL
#6  0x7735ad1f in ntdll!RtlInitializeExceptionChain () from
/c/WINDOWS/SYSTEM32/ntdll.dll
#7  0x7735acea in ntdll!RtlInitializeExceptionChain () from
/c/WINDOWS/SYSTEM32/ntdll.dll
#8  0x00000000 in ?? ()

Any ideas on what I can do to avoid this annoying behavior or to
regain control of my debugging session after it occurs?  I'm running
32-bit Cygwin on Windows 8.1.

Thanks.

-- 
William M. (Mike) Miller | Edison Design Group
william.m.miller@gmail.com

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: gdb arbitrarily starting threads

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2308 bytes --]

On Jun 20 09:29, William M. (Mike) Miller wrote:
> I have a single-threaded executable that has a number of debug-print
> routines that can be called interactively in the debugger to display
> data structures to stderr.  Something has changed in gdb/gcc/Cygwin
> during the last year or so (I don't update my installation very often)
> so that when I call one of these functions, _sometimes_ gdb will start
> a new thread and put my current thread into "running" status, so that
> I can't step, etc. (although the thread isn't actually running).
> Here's the scenario:
> 
> 1) I say "gdb xxx.exe", set a breakpoint, and run.  When I hit the
> breakpoint, "info threads" shows me two threads, one for my executable
> and one with some _cygtls/cygthread/ntdll frames; both are stopped.
> 
> 2) I say "up" a couple of times, then "p db_foo(p)" (where db_foo is
> one of these debug-print routines that displays to stderr the data
> structure pointed to by "p").  Sometimes, but not always, gdb will
> respond with
> 
>     [New Thread 1436.0x1f00]

You don't have this under control.  Nor does Cygwin, most of the time.

E.g, a debugged Windows executable has always at least two threads, one
is your process thread, one is the thread created by the OS when the
debugger attaches to a process.

Every Cygwin process has at least two threads, your process thread, as
well as a so-called "signal thread", the thread handling POSIX signals,
created at every process startup by the Cygwin DLL.

Certain Windows system calls create threads in your process to perform
tasks "under the hood".

Cygwin is doing the same, for instance to implement interruptible system
calls where the underlying OS call is non-interuptible.  select(2) may
start multiple threads at once.  Etc.

There's no way around that.  Windows process handling is centered around
cheap thread creation and termination.  Have a look into the "Details"
view in Windows Task Manager.  Activate the "Threads" column.  Note that
almost all processes are running with more than one thread.  Just don't
worry about it, it's kind of system inherent.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: gdb arbitrarily starting threads

[William M. (Mike) Miller]

On Sat, Jun 20, 2015 at 2:47 PM, Corinna Vinschen
<corinna-cygwin@cygwin.com> wrote:
> On Jun 20 09:29, William M. (Mike) Miller wrote:
>> I have a single-threaded executable that has a number of debug-print
>> routines that can be called interactively in the debugger to display
>> data structures to stderr.  Something has changed in gdb/gcc/Cygwin
>> during the last year or so (I don't update my installation very often)
>> so that when I call one of these functions, _sometimes_ gdb will start
>> a new thread and put my current thread into "running" status, so that
>> I can't step, etc. (although the thread isn't actually running).
>> Here's the scenario:
>>
>> 1) I say "gdb xxx.exe", set a breakpoint, and run.  When I hit the
>> breakpoint, "info threads" shows me two threads, one for my executable
>> and one with some _cygtls/cygthread/ntdll frames; both are stopped.
>>
>> 2) I say "up" a couple of times, then "p db_foo(p)" (where db_foo is
>> one of these debug-print routines that displays to stderr the data
>> structure pointed to by "p").  Sometimes, but not always, gdb will
>> respond with
>>
>>     [New Thread 1436.0x1f00]
>
> You don't have this under control.  Nor does Cygwin, most of the time.
>
> E.g, a debugged Windows executable has always at least two threads, one
> is your process thread, one is the thread created by the OS when the
> debugger attaches to a process.
>
> Every Cygwin process has at least two threads, your process thread, as
> well as a so-called "signal thread", the thread handling POSIX signals,
> created at every process startup by the Cygwin DLL.
>
> Certain Windows system calls create threads in your process to perform
> tasks "under the hood".
>
> Cygwin is doing the same, for instance to implement interruptible system
> calls where the underlying OS call is non-interuptible.  select(2) may
> start multiple threads at once.  Etc.
>
> There's no way around that.  Windows process handling is centered around
> cheap thread creation and termination.  Have a look into the "Details"
> view in Windows Task Manager.  Activate the "Threads" column.  Note that
> almost all processes are running with more than one thread.  Just don't
> worry about it, it's kind of system inherent.

Thanks for the reply.  My concern, however, isn't that new threads are
being created; it's that when gdb does this, it sets the state of all
threads, including my main thread, to "(running)" so that I can't
single-step through my program any more.  If I try to say "next", I
get the error:

    Cannot execute this command while the selected thread is running.

But I did nothing overt to start my thread.

It never used to do this, but it happens regularly now.

-- 
William M. (Mike) Miller | Edison Design Group
william.m.miller@gmail.com

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: gdb arbitrarily starting threads

[Ken Brown]

On 6/20/2015 4:02 PM, William M. (Mike) Miller wrote:
> Thanks for the reply.  My concern, however, isn't that new threads are
> being created; it's that when gdb does this, it sets the state of all
> threads, including my main thread, to "(running)" so that I can't
> single-step through my program any more.  If I try to say "next", I
> get the error:
>
>      Cannot execute this command while the selected thread is running.
>
> But I did nothing overt to start my thread.
>
> It never used to do this, but it happens regularly now.

I'm not seeing this on my system (Windows 7, both 32-bit and 64-bit 
Cygwin).  I debugged the emacs source, set a breakpoint, and stepped 
through the program when the breakpoint was hit.

Could this be caused by BLODA interference 
(https://cygwin.com/faq/faq.html#faq.using.bloda)?

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: gdb arbitrarily starting threads

[William M. (Mike) Miller]

On Sat, Jun 20, 2015 at 4:42 PM, Ken Brown <kbrown@cornell.edu> wrote:
> On 6/20/2015 4:02 PM, William M. (Mike) Miller wrote:
>>
>> Thanks for the reply.  My concern, however, isn't that new threads are
>> being created; it's that when gdb does this, it sets the state of all
>> threads, including my main thread, to "(running)" so that I can't
>> single-step through my program any more.  If I try to say "next", I
>> get the error:
>>
>>      Cannot execute this command while the selected thread is running.
>>
>> But I did nothing overt to start my thread.
>>
>> It never used to do this, but it happens regularly now.
>
>
> I'm not seeing this on my system (Windows 7, both 32-bit and 64-bit Cygwin).

I'm running 8.1, so that could well be the difference there.

> I debugged the emacs source, set a breakpoint, and stepped through the
> program when the breakpoint was hit.
>
> Could this be caused by BLODA interference
> (https://cygwin.com/faq/faq.html#faq.using.bloda)?

I certainly won't say that it's impossible, but I think it's unlikely.
The symptoms mentioned there don't seem related.  Also, the only one
of those programs I'm using is Windows Defender, and I've been using
it for over two years, but this problem only started within the last
few months.  (Of course, I suppose it's possible that something in
Cygwin/gdb/gcc changed to make it more sensitive, or an update to
Windows Defender made it more intrusive...)

Failing a way to keep gdb from doing this, is there a way in gdb to
stop a thread that's in "(running)" state?  (Even though it's sitting
at a breakpoint.)

-- 
William M. (Mike) Miller | Edison Design Group
william.m.miller@gmail.com

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: gdb arbitrarily starting threads

[Ken Brown]

On 6/20/2015 4:51 PM, William M. (Mike) Miller wrote:
> On Sat, Jun 20, 2015 at 4:42 PM, Ken Brown <kbrown@cornell.edu> wrote:
>> On 6/20/2015 4:02 PM, William M. (Mike) Miller wrote:
>>>
>>> Thanks for the reply.  My concern, however, isn't that new threads are
>>> being created; it's that when gdb does this, it sets the state of all
>>> threads, including my main thread, to "(running)" so that I can't
>>> single-step through my program any more.  If I try to say "next", I
>>> get the error:
>>>
>>>       Cannot execute this command while the selected thread is running.
>>>
>>> But I did nothing overt to start my thread.
>>>
>>> It never used to do this, but it happens regularly now.

Eli Zaretskii has asked me to pass on the following information:

On 6/21/2015 2:43 PM, Eli Zaretskii wrote:
 > Please point the people discussing this on the Cygwin list to these 2
 > threads on GDB mailing lists:
 >
 >    https://sourceware.org/ml/gdb/2015-03/msg00024.html
 >    https://sourceware.org/ml/gdb/2015-06/msg00005.html
 >    https://sourceware.org/ml/gdb-patches/2015-06/msg00215.html
 >
 > Bottom line: (1) it happens on Windows 7, not just on 8.1; (2) it
 > happens in the MinGW build of GDB, and so is not specific to threads
 > started by Cygwin for its own purposes; (3) it will most probably be
 > solved soon, since it is currently blocking the release of the
 > upcoming GDB 7.10.

Please keep Eli in the CC on any replies since he is not subscribed to the 
Cygwin list.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: gdb arbitrarily starting threads

[William M. (Mike) Miller]

On Sun, Jun 21, 2015 at 3:29 PM, Ken Brown <kbrown@cornell.edu> wrote:
> On 6/20/2015 4:51 PM, William M. (Mike) Miller wrote:
>>
>> On Sat, Jun 20, 2015 at 4:42 PM, Ken Brown <kbrown@cornell.edu> wrote:
>>>
>>> On 6/20/2015 4:02 PM, William M. (Mike) Miller wrote:
>>>>
>>>>
>>>> Thanks for the reply.  My concern, however, isn't that new threads are
>>>> being created; it's that when gdb does this, it sets the state of all
>>>> threads, including my main thread, to "(running)" so that I can't
>>>> single-step through my program any more.  If I try to say "next", I
>>>> get the error:
>>>>
>>>>       Cannot execute this command while the selected thread is running.
>>>>
>>>> But I did nothing overt to start my thread.
>>>>
>>>> It never used to do this, but it happens regularly now.
>
>
> Eli Zaretskii has asked me to pass on the following information:
>
> On 6/21/2015 2:43 PM, Eli Zaretskii wrote:
>> Please point the people discussing this on the Cygwin list to these 2
>> threads on GDB mailing lists:
>>
>>    https://sourceware.org/ml/gdb/2015-03/msg00024.html
>>    https://sourceware.org/ml/gdb/2015-06/msg00005.html
>>    https://sourceware.org/ml/gdb-patches/2015-06/msg00215.html
>>
>> Bottom line: (1) it happens on Windows 7, not just on 8.1; (2) it
>> happens in the MinGW build of GDB, and so is not specific to threads
>> started by Cygwin for its own purposes; (3) it will most probably be
>> solved soon, since it is currently blocking the release of the
>> upcoming GDB 7.10.
>
> Please keep Eli in the CC on any replies since he is not subscribed to the
> Cygwin list.

Thanks to Eli for the work on tracking this down!  It certainly is
annoying, so I'll definitely keep my eye out for a 7.10 announcement.
Thanks for the info.

-- 
William M. (Mike) Miller | Edison Design Group
william.m.miller@gmail.com

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: gdb arbitrarily starting threads

[Eli Zaretskii]

People who can reproduce this problem and can also build their own GDB
are encouraged to try the patch posted here:

  https://sourceware.org/ml/gdb/2015-06/msg00071.html

and report whether it solves the problem on Cygwin.  (I already
verified that the native MinGW debugging is fixed by that patch.)

Thanks.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: gdb arbitrarily starting threads

[William M. (Mike) Miller]

On Wed, Jun 24, 2015 at 10:33 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> People who can reproduce this problem and can also build their own GDB
> are encouraged to try the patch posted here:
>
>   https://sourceware.org/ml/gdb/2015-06/msg00071.html
>
> and report whether it solves the problem on Cygwin.  (I already
> verified that the native MinGW debugging is fixed by that patch.)

Any prognosis for if/when a version of gdb containing this patch might
make it to Cygwin?

-- 
William M. (Mike) Miller | Edison Design Group
william.m.miller@gmail.com

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: gdb arbitrarily starting threads

[William M. (Mike) Miller]

On Wed, Jun 24, 2015 at 10:33 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> People who can reproduce this problem and can also build their own GDB
> are encouraged to try the patch posted here:
>
>   https://sourceware.org/ml/gdb/2015-06/msg00071.html
>
> and report whether it solves the problem on Cygwin.  (I already
> verified that the native MinGW debugging is fixed by that patch.)

Any estimate on when a gdb release containing this patch might make it
to Cygwin?  It's not in the most recent version in setup (7.9.1-1).

-- 
William M. (Mike) Miller | Edison Design Group
william.m.miller@gmail.com

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple