pthread_kill: signals remain pending after target thread exits

pthread_kill: signals remain pending after target thread exits

[John Carey]

There seems to be a problem with pthread_kill: a pending signal targeting a particular thread prevents other threads from receiving signals sharing the same signal number--even after the original target thread exits and is joined.

To reproduce the issue:

  1. Block signal number S.

  2. Create thread T.

  3. Send a signal with signal number S to thread T in particular (as opposed to the process in general).

  4. After that signal has been sent, allow T to terminate without unblocking S or calling sigwait().

  5. Join T.

  6. Create thread N.

  7. Have N call sigwait() with a signal set that contains S.

  8. Send to N a new signal with signal number S.

  9. N never receives the new signal--instead, the new signal is discarded because the earlier signal remains pending.

BUT: It seems possible that N might inadvertently inherit the pending signal if its _cygtls instance happens to be allocated at the same address as the _cygtls instance of T.  It would be hard to predict when that would happen.  See the discussion of the source code, below.

For comparison, note that when performing the same steps on Linux (Ubuntu 14.04.3), N does in fact receive the second signal.

Here is the relevant Cygwin source code, if I am understanding things correctly:

  - sigproc.cc : wait_sig : calls pending_signals::add, then tries to process signals in the queue, but leaves queued any signal that failed to process

  - exceptions.cc : sigpacket::process : signal processing fails if it cannot find the target thread using init_cygheap::find_tls

  - sigproc.cc : pending_signals::add : discards new signals whose signal number matches that of a pending signal--regardless of target thread

  - cygheap.cc : init_cygheap::find_tls : looks for a thread by the address of its _cygtls instance, but a thread that has been joined might happen to have had the same _cygtls address as a thread that was subsequently created, and therefore pending signals for terminated threads might sometimes be inherited by unrelated new threads (or so it seems to me--as yet I have not managed to trigger such a scenario)

In my view it would be desirable if:

  - Pending signals targeting a particular thread would not outlast that thread.

  - Multiple pending signals targeting different threads could coexist, even if they shared the same signal number.  This happens on Linux (Ubuntu 14.04.3), where I can generate two signals for two different threads, then sleep for a bit in each target thread, and finally have each thread receive its signal with sigwait()--neither signal is lost during the sleeping period.

-- John Carey

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: pthread_kill: signals remain pending after target thread exits

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 3413 bytes --]

Hi John,

On Sep 11 18:11, John Carey wrote:
> There seems to be a problem with pthread_kill: a pending signal
> targeting a particular thread prevents other threads from receiving
> signals sharing the same signal number--even after the original target
> thread exits and is joined.
> 
> To reproduce the issue:
> 
>   1. Block signal number S.
> 
>   2. Create thread T.
> 
>   3. Send a signal with signal number S to thread T in particular (as
>   opposed to the process in general).
> 
>   4. After that signal has been sent, allow T to terminate without
>   unblocking S or calling sigwait().
> 
>   5. Join T.
> 
>   6. Create thread N.
> 
>   7. Have N call sigwait() with a signal set that contains S.
> 
>   8. Send to N a new signal with signal number S.
> 
>   9. N never receives the new signal--instead, the new signal is
>   discarded because the earlier signal remains pending.

Yuk.

> BUT: It seems possible that N might inadvertently inherit the pending
> signal if its _cygtls instance happens to be allocated at the same
> address as the _cygtls instance of T.  It would be hard to predict
> when that would happen.  See the discussion of the source code, below.

The important thing here is to get rid of the pending signal.

> For comparison, note that when performing the same steps on Linux
> (Ubuntu 14.04.3), N does in fact receive the second signal.
> 
> Here is the relevant Cygwin source code, if I am understanding things
> correctly:
> 
>   - sigproc.cc : wait_sig : calls pending_signals::add, then tries to
>   process signals in the queue, but leaves queued any signal that
>   failed to process
> 
>   - exceptions.cc : sigpacket::process : signal processing fails if it
>   cannot find the target thread using init_cygheap::find_tls
> 
>   - sigproc.cc : pending_signals::add : discards new signals whose
>   signal number matches that of a pending signal--regardless of target
>   thread
> 
>   - cygheap.cc : init_cygheap::find_tls : looks for a thread by the
>   address of its _cygtls instance, but a thread that has been joined
>   might happen to have had the same _cygtls address as a thread that
>   was subsequently created, and therefore pending signals for
>   terminated threads might sometimes be inherited by unrelated new
>   threads (or so it seems to me--as yet I have not managed to trigger
>   such a scenario)
> 
> In my view it would be desirable if:
> 
>   - Pending signals targeting a particular thread would not outlast
>   that thread.

Since you looked into the code anyway, do you have an idea how to
implement that?  For a start, do you have a simple testcase, only
the bare code needed to reproduce the issue?

>   - Multiple pending signals targeting different threads could
>   coexist, even if they shared the same signal number.  This happens
>   on Linux (Ubuntu 14.04.3), where I can generate two signals for two
>   different threads, then sleep for a bit in each target thread, and
>   finally have each thread receive its signal with sigwait()--neither
>   signal is lost during the sleeping period.

That requires to extend the handling for pending signals.  That's
a rather bigger task...


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

RE: pthread_kill: signals remain pending after target thread exits

[John Carey]

[-- Attachment #1: Type: text/plain, Size: 3320 bytes --]

> From: Corinna Vinschen [corinna-cygwin@cygwin.com]
> Sent: Wednesday, October 21, 2015 4:48 AM
> Subject: Re: pthread_kill: signals remain pending after target thread exits
...
> > On Sep 11 18:11, John Carey wrote:
> > There seems to be a problem with pthread_kill: a pending signal
> > targeting a particular thread prevents other threads from receiving
> > signals sharing the same signal number--even after the original target
> > thread exits and is joined.
...
> The important thing here is to get rid of the pending signal.

Yes, I agree that is the most important thing.

> > In my view it would be desirable if:
> >
> >   - Pending signals targeting a particular thread would not outlast
> >   that thread.
> 
> Since you looked into the code anyway, do you have an idea how to
> implement that?  For a start, do you have a simple testcase, only
> the bare code needed to reproduce the issue?

I've attached a test case that I *think* gets into the right spot, at least for 64-bit Cygwin 2.0.4.  That is, it hangs trying to receive the signal, instead of terminating.  (This test passes (terminates) in 32-bit Cygwin 1.7.9 and 64-bit Ubuntu 14.04.3 LTS.)

As to a fix: sorry, but though I looked at the code, I am not sufficiently confident to suggest a specific change.  I think that the internal signal handling thread has exclusive access to the pending signal collection, which is one difficulty.  And I'm not sure how the race is resolved between something trying to use the cygtls and the cygtls being destroyed.  At a guess, there are at least two general approaches to a fix:

1. Somehow prevent new signals from being sent to the terminating thread, then notify the internal signal handling thread of the need to purge pending signals targeting the doomed thread, then delay cygtls destruction until confirmation that that purge is complete.

2. In the pending signal representation, replace the direct cygtls address with a pointer to some small reference-counted object associated with the cygtls.  That small object could live on for a while, even after the original cygtls has been destroyed and its memory reused for a new cygtls, so that the signal processing thread can take its time purging references.  But there has to be some way to atomically do two things: 1) check whether this small object still points to a valid cygtls, and 2) if it does, delay destruction of that cygtls until some task has been performed (such as processing a signal).  Perhaps this small object could contain an invalidation flag and some synchronization objects (mutex, condition variable, etc.) in addition to the raw cygtls pointer.

> >   - Multiple pending signals targeting different threads could
> >   coexist, even if they shared the same signal number.  This happens
> >   on Linux (Ubuntu 14.04.3), where I can generate two signals for two
> >   different threads, then sleep for a bit in each target thread, and
> >   finally have each thread receive its signal with sigwait()--neither
> >   signal is lost during the sleeping period.
> 
> That requires to extend the handling for pending signals.  That's
> a rather bigger task...

Yeah.  It's nice if threads don't interfere with each other, but this part would indeed be harder to change.

-- John Carey

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: test_pending_signal.c --]
[-- Type: text/x-csrc; name="test_pending_signal.c", Size: 6688 bytes --]

/* Copyright (c) 2015, Electric Cloud, Inc.
 * All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 *
 *   - Redistributions of source code must retain the above copyright
 *     notice, this list of conditions and the following disclaimer.
 *
 *   - Redistributions in binary form must reproduce the above copyright
 *     notice, this list of conditions and the following disclaimer in the
 *     documentation and/or other materials provided with the distribution.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
 * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
 * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
 * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
 * HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
 * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
 * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
 * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
 * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 */

/* This test program demonstates a Cygwin bug in which a signal sent
 * to a particular thread remains pending after the thread terminates.
 */

#include <unistd.h>
#include <pthread.h>
#include <errno.h>
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

static pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
static pthread_cond_t cond = PTHREAD_COND_INITIALIZER;
static int fully_up = 0;
static int allow_exit = 0;
static int about_to_sigwait = 0;
static int signal_received = -1;

static void check_syscall(char const *context, int result)
{
    if (result == -1) {
        fprintf(stderr, "%s: %s\n", context, strerror(errno));
        exit(EXIT_FAILURE);
    }
}

static void check_threadcall(char const *context, int error_number)
{
    if (error_number) {
        fprintf(stderr, "%s: %s\n", context, strerror(error_number));
        exit(EXIT_FAILURE);
    }
}

static void *thread1(void *arg)
{
    check_threadcall("pthread_mutex_lock",
            pthread_mutex_lock(&mutex));

    fully_up = 1;

    check_threadcall("pthread_cond_broadcast",
            pthread_cond_broadcast(&cond));

    while (! allow_exit) {
        pthread_cond_wait(&cond, &mutex);
    }

    check_threadcall("pthread_mutex_unlock",
            pthread_mutex_unlock(&mutex));
    return NULL;
}

static void *thread2(void *arg)
{
    sigset_t set;
    int sig;

    check_syscall("sigemptyset",
            sigemptyset(&set));

    check_syscall("sigaddset",
            sigaddset(&set, SIGTERM));

    check_threadcall("pthread_mutex_lock",
            pthread_mutex_lock(&mutex));

    about_to_sigwait = 1;

    check_threadcall("pthread_cond_broadcast",
            pthread_cond_broadcast(&cond));

    check_threadcall("pthread_mutex_unlock",
            pthread_mutex_unlock(&mutex));

    /* Try to receive a signal. */

    sig = 0;
    check_threadcall("sigwait",
            sigwait(&set, &sig));

    check_threadcall("pthread_mutex_lock",
            pthread_mutex_lock(&mutex));

    signal_received = sig;

    check_threadcall("pthread_cond_broadcast",
            pthread_cond_broadcast(&cond));

    check_threadcall("pthread_mutex_unlock",
            pthread_mutex_unlock(&mutex));

    return NULL;
}

int main(int argc, char **argv)
{
    sigset_t set;
    pthread_t tid1, tid2;
    int sig;

    /* Block SIGTERM for this thread and all threads it creates. */

    check_syscall("sigemptyset",
            sigemptyset(&set));

    check_syscall("sigaddset",
            sigaddset(&set, SIGTERM));

    check_threadcall("pthread_sigmask",
            pthread_sigmask(SIG_BLOCK, &set, NULL));

    check_threadcall("pthread_create",
            pthread_create(&tid1, NULL, thread1, NULL));

    /* Just be sure, wait until a known point in the other thread. */

    check_threadcall("pthread_mutex_lock",
            pthread_mutex_lock(&mutex));

    while (! fully_up) {
        pthread_cond_wait(&cond, &mutex);
    }

    /* Send a signal specifically to that other thread
     * (as opposed to the process as a whole). */

    check_threadcall("pthread_kill",
            pthread_kill(tid1, SIGTERM));

    /* Allow the other thread to terminate. */

    allow_exit = 1;

    check_threadcall("pthread_cond_broadcast",
            pthread_cond_broadcast(&cond));

    check_threadcall("pthread_mutex_unlock",
            pthread_mutex_unlock(&mutex));

    /* Join the other thread. */

    check_threadcall("pthread_join",
            pthread_join(tid1, NULL));

    /* At this point the other thread is gone, but because of the bug
     * there is still a SIGTERM that cannot be delivered unless and
     * until a new thread obtains the same cygtls instance. */

    /* Start a new signal-handling thread. */

    check_threadcall("pthread_create",
            pthread_create(&tid2, NULL, thread2, NULL));

    check_threadcall("pthread_mutex_lock",
            pthread_mutex_lock(&mutex));

    while (! about_to_sigwait) {
        pthread_cond_wait(&cond, &mutex);
    }

    check_threadcall("pthread_mutex_unlock",
            pthread_mutex_unlock(&mutex));

    /* Unfortunately, we have no way to be sure that the
     * other thread really is blocked in sigwait.  Sleep
     * a second to give it time to get into sigwait. */

    sleep(1);

    /* Send the same signal to the process as a whole;
     * if things were working, the new signal handling
     * thread would receive it inside of sigwait().
     *
     * We avoid raise(sig) because POSIX says it means
     * pthread_kill(pthread_self(), sig), meaning that
     * this thread would have to receive the signal. */

    check_syscall("kill",
            kill(getpid(), SIGTERM));

    printf("starting to await signal\n");
    fflush(stdout);

    /* Wait for that thread to receive that signal. */

    check_threadcall("pthread_mutex_lock",
            pthread_mutex_lock(&mutex));

    while ((sig = signal_received) == -1) {
        pthread_cond_wait(&cond, &mutex);
    }

    check_threadcall("pthread_mutex_unlock",
            pthread_mutex_unlock(&mutex));

    printf("received signal: %d (\"%s\")\n", sig, strsignal(sig));
    fflush(stdout);

    /* Join the other thread. */

    check_threadcall("pthread_join",
            pthread_join(tid2, NULL));

    return 0;
}

[-- Attachment #3: Type: text/plain, Size: 218 bytes --]

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: pthread_kill: signals remain pending after target thread exits

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2620 bytes --]

On Oct 22 02:08, John Carey wrote:
> > From: Corinna Vinschen [corinna-cygwin@cygwin.com]
> > Sent: Wednesday, October 21, 2015 4:48 AM
> > Subject: Re: pthread_kill: signals remain pending after target thread exits
> ...
> > > On Sep 11 18:11, John Carey wrote:
> > > There seems to be a problem with pthread_kill: a pending signal
> > > targeting a particular thread prevents other threads from receiving
> > > signals sharing the same signal number--even after the original target
> > > thread exits and is joined.
> ...
> > The important thing here is to get rid of the pending signal.
> 
> Yes, I agree that is the most important thing.
> 
> > > In my view it would be desirable if:
> > >
> > >   - Pending signals targeting a particular thread would not outlast
> > >   that thread.
> > 
> > Since you looked into the code anyway, do you have an idea how to
> > implement that?  For a start, do you have a simple testcase, only
> > the bare code needed to reproduce the issue?
> 
> I've attached a test case that I *think* gets into the right spot, at
> least for 64-bit Cygwin 2.0.4.  That is, it hangs trying to receive
> the signal, instead of terminating.  (This test passes (terminates) in
> 32-bit Cygwin 1.7.9 and 64-bit Ubuntu 14.04.3 LTS.)

Thanks for the testcase.  I applied a patch which hopefully works as
desired, at least to fix the immediate problem of the remaining pending
signal when a thread exits.  I uploaded a new developer snapshot to
https://cygwin.com/snapshots.  Please give it a try.

Note that the today's snapshot does *NOT* contain the changes concerning
the new ACL handling, so people testing that stuff should skip this
snapshot.

> > >   - Multiple pending signals targeting different threads could
> > >   coexist, even if they shared the same signal number.  This happens
> > >   on Linux (Ubuntu 14.04.3), where I can generate two signals for two
> > >   different threads, then sleep for a bit in each target thread, and
> > >   finally have each thread receive its signal with sigwait()--neither
> > >   signal is lost during the sleeping period.
> > 
> > That requires to extend the handling for pending signals.  That's
> > a rather bigger task...
> 
> Yeah.  It's nice if threads don't interfere with each other, but this
> part would indeed be harder to change.

I added that to my neverending TODO list.  Maybe I get around to it at
one point.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: pthread_kill: signals remain pending after target thread exits

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2794 bytes --]


John?  Ping?

On Oct 23 14:55, Corinna Vinschen wrote:
> On Oct 22 02:08, John Carey wrote:
> > > From: Corinna Vinschen [corinna-cygwin@cygwin.com]
> > > Sent: Wednesday, October 21, 2015 4:48 AM
> > > Subject: Re: pthread_kill: signals remain pending after target thread exits
> > ...
> > > > On Sep 11 18:11, John Carey wrote:
> > > > There seems to be a problem with pthread_kill: a pending signal
> > > > targeting a particular thread prevents other threads from receiving
> > > > signals sharing the same signal number--even after the original target
> > > > thread exits and is joined.
> > ...
> > > The important thing here is to get rid of the pending signal.
> > 
> > Yes, I agree that is the most important thing.
> > 
> > > > In my view it would be desirable if:
> > > >
> > > >   - Pending signals targeting a particular thread would not outlast
> > > >   that thread.
> > > 
> > > Since you looked into the code anyway, do you have an idea how to
> > > implement that?  For a start, do you have a simple testcase, only
> > > the bare code needed to reproduce the issue?
> > 
> > I've attached a test case that I *think* gets into the right spot, at
> > least for 64-bit Cygwin 2.0.4.  That is, it hangs trying to receive
> > the signal, instead of terminating.  (This test passes (terminates) in
> > 32-bit Cygwin 1.7.9 and 64-bit Ubuntu 14.04.3 LTS.)
> 
> Thanks for the testcase.  I applied a patch which hopefully works as
> desired, at least to fix the immediate problem of the remaining pending
> signal when a thread exits.  I uploaded a new developer snapshot to
> https://cygwin.com/snapshots.  Please give it a try.
> 
> Note that the today's snapshot does *NOT* contain the changes concerning
> the new ACL handling, so people testing that stuff should skip this
> snapshot.
> 
> > > >   - Multiple pending signals targeting different threads could
> > > >   coexist, even if they shared the same signal number.  This happens
> > > >   on Linux (Ubuntu 14.04.3), where I can generate two signals for two
> > > >   different threads, then sleep for a bit in each target thread, and
> > > >   finally have each thread receive its signal with sigwait()--neither
> > > >   signal is lost during the sleeping period.
> > > 
> > > That requires to extend the handling for pending signals.  That's
> > > a rather bigger task...
> > 
> > Yeah.  It's nice if threads don't interfere with each other, but this
> > part would indeed be harder to change.
> 
> I added that to my neverending TODO list.  Maybe I get around to it at
> one point.
> 
> 
> Thanks,
> Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

RE: pthread_kill: signals remain pending after target thread exits

[John Carey]

[-- Attachment #1: Type: text/plain, Size: 2091 bytes --]

Sorry for the delay.

From: Corinna Vinschen [corinna-cygwin@cygwin.com]
Sent: Friday, October 23, 2015 5:55 AM
> > I've attached a test case that I *think* gets into the right spot, at
> > least for 64-bit Cygwin 2.0.4.  That is, it hangs trying to receive
> > the signal, instead of terminating.  (This test passes (terminates) in
> > 32-bit Cygwin 1.7.9 and 64-bit Ubuntu 14.04.3 LTS.)
> 
> Thanks for the testcase.  I applied a patch which hopefully works as
> desired, at least to fix the immediate problem of the remaining pending
> signal when a thread exits.  I uploaded a new developer snapshot to
> https://cygwin.com/snapshots.  Please give it a try.

Thanks; that was fast!  I tried replacing cygwin1.dll with cygwin1-20151023.dll .

The original test case now works.  I checked some of my other tests,
and unfortunately some of them failed, so I extracted out a new test
case, which is attached.  My guess is that something is subtly different
about the timing on this test.

This new test hangs about half the time, and I have to use Windows Task Manager
to kill it (not Cygwin "kill"). Sometimes I see "pthread_kill: Unknown error -1" while
it hangs (meaning that pthread_kill returned -1 instead of an error number).

> > > >   - Multiple pending signals targeting different threads could
> > > >   coexist, even if they shared the same signal number.  This happens
> > > >   on Linux (Ubuntu 14.04.3), where I can generate two signals for two
> > > >   different threads, then sleep for a bit in each target thread, and
> > > >   finally have each thread receive its signal with sigwait()--neither
> > > >   signal is lost during the sleeping period.
> > >
> > > That requires to extend the handling for pending signals.  That's
> > > a rather bigger task...
> >
> > Yeah.  It's nice if threads don't interfere with each other, but this
> > part would indeed be harder to change.
> 
> I added that to my neverending TODO list.  Maybe I get around to it at
> one point.

I know the feeling.  No worries, and thanks.

-- John Carey

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: test_pending_signal2.c --]
[-- Type: text/x-csrc; name="test_pending_signal2.c", Size: 6295 bytes --]

/* Copyright (c) 2015, Electric Cloud, Inc.
 * All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 *
 *   - Redistributions of source code must retain the above copyright
 *     notice, this list of conditions and the following disclaimer.
 *
 *   - Redistributions in binary form must reproduce the above copyright
 *     notice, this list of conditions and the following disclaimer in the
 *     documentation and/or other materials provided with the distribution.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
 * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
 * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
 * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
 * HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
 * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
 * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
 * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
 * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 */

/* This test program demonstates a Cygwin bug in which a signal sent
 * to a particular thread remains pending after the thread terminates.
 *
 * Somehow even though the original fix worked for test_pending_signal.c,
 * about half the time this test case triggers a hang that must be killed
 * using the Windows Task Manager instead of Cygwin "kill".  Sometimes we
 * see "pthread_kill: Unknown error -1" during the hang.
 */

#include <unistd.h>
#include <pthread.h>
#include <errno.h>
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>


static void check_syscall(char const *context, int result)
{
    if (result == -1) {
        fprintf(stderr, "%s: %s\n", context, strerror(errno));
        exit(EXIT_FAILURE);
    }
}

static void check_threadcall(char const *context, int error_number)
{
    if (error_number) {
        fprintf(stderr, "%s: %s\n", context, strerror(error_number));
        exit(EXIT_FAILURE);
    }
}


typedef struct shared_struct {
    sigset_t signal_mask;  /* signals to block */
    pthread_mutex_t mutex;
    pthread_cond_t cond;
    int eat;
    int done;
} shared_t;


static void *phage_thread (void *arg)
{
    shared_t *shared = (shared_t *)arg;

    check_threadcall("pthread_mutex_lock",
            pthread_mutex_lock(&shared->mutex));

    if (shared->eat <= 1) {
        shared->eat = 1;
        check_threadcall("pthread_cond_broadcast",
                pthread_cond_broadcast(&shared->cond));

        while (shared->eat <= 1) {
            check_threadcall("pthread_cond_wait",
                    pthread_cond_wait(&shared->cond, &shared->mutex));
        }
    }

    check_threadcall("pthread_mutex_unlock",
            pthread_mutex_unlock(&shared->mutex));
    return NULL;
}

static void *signal_thread (void *arg)
{
    shared_t *shared = (shared_t *)arg;
    int sig_caught;
    int ern;

    check_threadcall("sigwait",
            sigwait(&shared->signal_mask, &sig_caught));

    if (sig_caught == SIGUSR2) {
        puts("Received SIGUSR2");
        fflush(stdout);
    } else {
        fprintf (stderr, "\nUnexpected signal %d\n", sig_caught);
    }

    check_threadcall("pthread_mutex_lock",
            pthread_mutex_lock(&shared->mutex));

    shared->done = 1;

    check_threadcall("pthread_cond_broadcast",
            pthread_cond_broadcast(&shared->cond));

    check_threadcall("pthread_mutex_unlock",
            pthread_mutex_unlock(&shared->mutex));

    return NULL;
}


int main(int argc, char **argv)
{
    shared_t *shared = (shared_t *)malloc(sizeof(shared_t));
    pthread_t phage_tid, signal_tid;
    struct timespec ts;
    int ern;

    sigemptyset(&shared->signal_mask);
    sigaddset(&shared->signal_mask, SIGUSR2);

    check_threadcall("pthread_mutex_init",
            pthread_mutex_init(&shared->mutex, NULL));

    check_threadcall("pthread_cond_init",
            pthread_cond_init(&shared->cond, NULL));

    check_threadcall("pthread_sigmask",
            pthread_sigmask(SIG_BLOCK, &shared->signal_mask, NULL));

    shared->eat = 0;
    shared->done = 0;

    check_threadcall("pthread_create",
            pthread_create(&phage_tid, NULL, phage_thread, shared));

    check_threadcall("pthread_mutex_lock",
            pthread_mutex_lock(&shared->mutex));

    while (shared->eat == 0) {
        check_threadcall("pthread_cond_wait",
                pthread_cond_wait(&shared->cond, &shared->mutex));
    }

    check_threadcall("pthread_kill",
            pthread_kill(phage_tid, SIGUSR2));

    shared->eat = 2;

    check_threadcall("pthread_cond_broadcast",
            pthread_cond_broadcast(&shared->cond));

    check_threadcall("pthread_mutex_unlock",
            pthread_mutex_unlock(&shared->mutex));

    check_threadcall("pthread_join",
            pthread_join(phage_tid, NULL));

    check_threadcall("pthread_create",
            pthread_create(&signal_tid, NULL, signal_thread, shared));

    check_threadcall("pthread_mutex_lock",
            pthread_mutex_lock(&shared->mutex));

    for (;;) {
        if (shared->done) {
            check_threadcall("pthread_join",
                    pthread_join(signal_tid, NULL));
            break;
        } {
            ern = pthread_kill(signal_tid, SIGUSR2);
            if (ern != ESRCH) {
                check_threadcall("pthread_kill", ern);
            }

            check_syscall("clock_gettime",
                    clock_gettime(CLOCK_REALTIME, &ts));

            ts.tv_nsec += 100000000;
            if (ts.tv_nsec >= 1000000000) {
                ts.tv_nsec -= 1000000000;
                ts.tv_sec += 1;
            }

            check_threadcall("pthread_cond_timedwait",
                pthread_cond_timedwait(&shared->cond, &shared->mutex, &ts));
        }
    }

    check_threadcall("pthread_mutex_unlock",
            pthread_mutex_unlock(&shared->mutex));

    return EXIT_SUCCESS;
}

[-- Attachment #3: Type: text/plain, Size: 218 bytes --]

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: pthread_kill: signals remain pending after target thread exits

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1297 bytes --]

On Oct 27 19:06, John Carey wrote:
> Sorry for the delay.
> 
> From: Corinna Vinschen [corinna-cygwin@cygwin.com]
> Sent: Friday, October 23, 2015 5:55 AM
> > > I've attached a test case that I *think* gets into the right spot, at
> > > least for 64-bit Cygwin 2.0.4.  That is, it hangs trying to receive
> > > the signal, instead of terminating.  (This test passes (terminates) in
> > > 32-bit Cygwin 1.7.9 and 64-bit Ubuntu 14.04.3 LTS.)
> > 
> > Thanks for the testcase.  I applied a patch which hopefully works as
> > desired, at least to fix the immediate problem of the remaining pending
> > signal when a thread exits.  I uploaded a new developer snapshot to
> > https://cygwin.com/snapshots.  Please give it a try.
> 
> Thanks; that was fast!  I tried replacing cygwin1.dll with cygwin1-20151023.dll .
> 
> The original test case now works.  I checked some of my other tests,
> and unfortunately some of them failed, so I extracted out a new test
> case, which is attached.  My guess is that something is subtly different
> about the timing on this test.

Is this a regression?  Did it work with 2.2.1?


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: pthread_kill: signals remain pending after target thread exits

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1682 bytes --]

On Oct 28 10:14, Corinna Vinschen wrote:
> On Oct 27 19:06, John Carey wrote:
> > Sorry for the delay.
> > 
> > From: Corinna Vinschen [corinna-cygwin@cygwin.com]
> > Sent: Friday, October 23, 2015 5:55 AM
> > > > I've attached a test case that I *think* gets into the right spot, at
> > > > least for 64-bit Cygwin 2.0.4.  That is, it hangs trying to receive
> > > > the signal, instead of terminating.  (This test passes (terminates) in
> > > > 32-bit Cygwin 1.7.9 and 64-bit Ubuntu 14.04.3 LTS.)
> > > 
> > > Thanks for the testcase.  I applied a patch which hopefully works as
> > > desired, at least to fix the immediate problem of the remaining pending
> > > signal when a thread exits.  I uploaded a new developer snapshot to
> > > https://cygwin.com/snapshots.  Please give it a try.
> > 
> > Thanks; that was fast!  I tried replacing cygwin1.dll with cygwin1-20151023.dll .
> > 
> > The original test case now works.  I checked some of my other tests,
> > and unfortunately some of them failed, so I extracted out a new test
> > case, which is attached.  My guess is that something is subtly different
> > about the timing on this test.
> 
> Is this a regression?  Did it work with 2.2.1?

Answering myself here, this didn't work at all in 2.2.1.  I can
reproduce the problem and I'm going to take a look.  Not sure if
there's a quick solution, though.  This looks like a deadlock
situation.  The signal definitely got send, I just don't see yet
why it's not handled in wait_sig.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

RE: pthread_kill: signals remain pending after target thread exits

[John Carey]

> From: Corinna Vinschen [corinna-cygwin@cygwin.com]
> Sent: Thursday, October 29, 2015 1:02 AM
> On Oct 28 10:14, Corinna Vinschen wrote:
...
> > > Thanks; that was fast!  I tried replacing cygwin1.dll with cygwin1-20151023.dll .
> > >
> > > The original test case now works.  I checked some of my other tests,
> > > and unfortunately some of them failed, so I extracted out a new test
> > > case, which is attached.  My guess is that something is subtly different
> > > about the timing on this test.
> >
> > Is this a regression?  Did it work with 2.2.1?
> 
> Answering myself here, this didn't work at all in 2.2.1.  I can
> reproduce the problem and I'm going to take a look.  Not sure if
> there's a quick solution, though.  This looks like a deadlock
> situation.  The signal definitely got send, I just don't see yet
> why it's not handled in wait_sig.

Yes, test_pending_signal2.c fails in 2.2.1, though the symptoms
differ: there is no Cygwin process left hanging that cannot be
killed by a Cygwin signal.

But both tests appear to pass in Cygwin 1.7.9, so in that sense
it is a regression, just not a recent one.

-- John Carey

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

RE: pthread_kill: signals remain pending after target thread exits

[John Carey]

> From: Corinna Vinschen [corinna-cygwin@cygwin.com]
> Sent: Thursday, October 29, 2015 1:02 AM
> On Oct 28 10:14, Corinna Vinschen wrote:
...
> > > Thanks; that was fast!  I tried replacing cygwin1.dll with cygwin1-20151023.dll .
> > >
> > > The original test case now works.  I checked some of my other tests,
> > > and unfortunately some of them failed, so I extracted out a new test
> > > case, which is attached.  My guess is that something is subtly different
> > > about the timing on this test.
> >
> > Is this a regression?  Did it work with 2.2.1?
> 
> Answering myself here, this didn't work at all in 2.2.1.  I can
> reproduce the problem and I'm going to take a look.  Not sure if
> there's a quick solution, though.  This looks like a deadlock
> situation.  The signal definitely got send, I just don't see yet
> why it's not handled in wait_sig.

Yes, test_pending_signal2.c fails in 2.2.1, though the symptoms
differ: there is no Cygwin process left hanging that cannot be
killed by a Cygwin signal.

But both tests appear to pass in Cygwin 1.7.9, so in that sense
it is a regression, just not a recent one.

-- John Carey

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: pthread_kill: signals remain pending after target thread exits

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1736 bytes --]

On Oct 29 16:16, John Carey wrote:
> > From: Corinna Vinschen [corinna-cygwin@cygwin.com]
> > Sent: Thursday, October 29, 2015 1:02 AM
> > On Oct 28 10:14, Corinna Vinschen wrote:
> ...
> > > > Thanks; that was fast!  I tried replacing cygwin1.dll with cygwin1-20151023.dll .
> > > >
> > > > The original test case now works.  I checked some of my other tests,
> > > > and unfortunately some of them failed, so I extracted out a new test
> > > > case, which is attached.  My guess is that something is subtly different
> > > > about the timing on this test.
> > >
> > > Is this a regression?  Did it work with 2.2.1?
> > 
> > Answering myself here, this didn't work at all in 2.2.1.  I can
> > reproduce the problem and I'm going to take a look.  Not sure if
> > there's a quick solution, though.  This looks like a deadlock
> > situation.  The signal definitely got send, I just don't see yet
> > why it's not handled in wait_sig.
> 
> Yes, test_pending_signal2.c fails in 2.2.1, though the symptoms
> differ: there is no Cygwin process left hanging that cannot be
> killed by a Cygwin signal.
> 
> But both tests appear to pass in Cygwin 1.7.9, so in that sense
> it is a regression, just not a recent one.

The regression here was my original fix.  It didn't clear pending
signals for an exiting thread thorougly enough.  I applied a patch now
which should fix this issue.  I'll upload a new developer snapshot to
https://cygwin.com/snapshots/ later today or tomorrow (still looking
into the PrlFS issue).  Please give it a try.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: pthread_kill: signals remain pending after target thread exits

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1948 bytes --]

On Nov  2 15:18, Corinna Vinschen wrote:
> On Oct 29 16:16, John Carey wrote:
> > > From: Corinna Vinschen [corinna-cygwin@cygwin.com]
> > > Sent: Thursday, October 29, 2015 1:02 AM
> > > On Oct 28 10:14, Corinna Vinschen wrote:
> > ...
> > > > > Thanks; that was fast!  I tried replacing cygwin1.dll with cygwin1-20151023.dll .
> > > > >
> > > > > The original test case now works.  I checked some of my other tests,
> > > > > and unfortunately some of them failed, so I extracted out a new test
> > > > > case, which is attached.  My guess is that something is subtly different
> > > > > about the timing on this test.
> > > >
> > > > Is this a regression?  Did it work with 2.2.1?
> > > 
> > > Answering myself here, this didn't work at all in 2.2.1.  I can
> > > reproduce the problem and I'm going to take a look.  Not sure if
> > > there's a quick solution, though.  This looks like a deadlock
> > > situation.  The signal definitely got send, I just don't see yet
> > > why it's not handled in wait_sig.
> > 
> > Yes, test_pending_signal2.c fails in 2.2.1, though the symptoms
> > differ: there is no Cygwin process left hanging that cannot be
> > killed by a Cygwin signal.
> > 
> > But both tests appear to pass in Cygwin 1.7.9, so in that sense
> > it is a regression, just not a recent one.
> 
> The regression here was my original fix.  It didn't clear pending
> signals for an exiting thread thorougly enough.  I applied a patch now
> which should fix this issue.  I'll upload a new developer snapshot to
> https://cygwin.com/snapshots/ later today or tomorrow (still looking
> into the PrlFS issue).  Please give it a try.

Snapshot is up.  The fix is in the 2.3.0-0.5 test release as well.
Please give any one of them a try.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

RE: pthread_kill: signals remain pending after target thread exits

[John Carey]

> From: Corinna Vinschen [corinna-cygwin@cygwin.com]
> Sent: Monday, November 02, 2015 9:58 AM
> On Nov  2 15:18, Corinna Vinschen wrote:
> > On Oct 29 16:16, John Carey wrote:
> > > > From: Corinna Vinschen [corinna-cygwin@cygwin.com]
> > > > Sent: Thursday, October 29, 2015 1:02 AM
> > > > On Oct 28 10:14, Corinna Vinschen wrote:
> > > ...
> > > > > > Thanks; that was fast!  I tried replacing cygwin1.dll with cygwin1-20151023.dll .
> > > > > >
> > > > > > The original test case now works.  I checked some of my other tests,
> > > > > > and unfortunately some of them failed, so I extracted out a new test
> > > > > > case, which is attached.  My guess is that something is subtly different
> > > > > > about the timing on this test.
> > > > >
> > > > > Is this a regression?  Did it work with 2.2.1?
> > > >
> > > > Answering myself here, this didn't work at all in 2.2.1.  I can
> > > > reproduce the problem and I'm going to take a look.  Not sure if
> > > > there's a quick solution, though.  This looks like a deadlock
> > > > situation.  The signal definitely got send, I just don't see yet
> > > > why it's not handled in wait_sig.
> > >
> > > Yes, test_pending_signal2.c fails in 2.2.1, though the symptoms
> > > differ: there is no Cygwin process left hanging that cannot be
> > > killed by a Cygwin signal.
> > >
> > > But both tests appear to pass in Cygwin 1.7.9, so in that sense
> > > it is a regression, just not a recent one.
> >
> > The regression here was my original fix.  It didn't clear pending
> > signals for an exiting thread thorougly enough.  I applied a patch now
> > which should fix this issue.  I'll upload a new developer snapshot to
> > https://cygwin.com/snapshots/ later today or tomorrow (still looking
> > into the PrlFS issue).  Please give it a try.
> 
> Snapshot is up.  The fix is in the 2.3.0-0.5 test release as well.
> Please give any one of them a try.

I tested snapshot cygwin1-20151102.dll, and it works in all of my tests.  Nice!

I'm glad you figured out how to delete those pending signals.

Thanks,
John Carey

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: pthread_kill: signals remain pending after target thread exits

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2528 bytes --]

On Nov  2 23:54, John Carey wrote:
> > From: Corinna Vinschen [corinna-cygwin@cygwin.com]
> > Sent: Monday, November 02, 2015 9:58 AM
> > On Nov  2 15:18, Corinna Vinschen wrote:
> > > On Oct 29 16:16, John Carey wrote:
> > > > > From: Corinna Vinschen [corinna-cygwin@cygwin.com]
> > > > > Sent: Thursday, October 29, 2015 1:02 AM
> > > > > On Oct 28 10:14, Corinna Vinschen wrote:
> > > > ...
> > > > > > > Thanks; that was fast!  I tried replacing cygwin1.dll with cygwin1-20151023.dll .
> > > > > > >
> > > > > > > The original test case now works.  I checked some of my other tests,
> > > > > > > and unfortunately some of them failed, so I extracted out a new test
> > > > > > > case, which is attached.  My guess is that something is subtly different
> > > > > > > about the timing on this test.
> > > > > >
> > > > > > Is this a regression?  Did it work with 2.2.1?
> > > > >
> > > > > Answering myself here, this didn't work at all in 2.2.1.  I can
> > > > > reproduce the problem and I'm going to take a look.  Not sure if
> > > > > there's a quick solution, though.  This looks like a deadlock
> > > > > situation.  The signal definitely got send, I just don't see yet
> > > > > why it's not handled in wait_sig.
> > > >
> > > > Yes, test_pending_signal2.c fails in 2.2.1, though the symptoms
> > > > differ: there is no Cygwin process left hanging that cannot be
> > > > killed by a Cygwin signal.
> > > >
> > > > But both tests appear to pass in Cygwin 1.7.9, so in that sense
> > > > it is a regression, just not a recent one.
> > >
> > > The regression here was my original fix.  It didn't clear pending
> > > signals for an exiting thread thorougly enough.  I applied a patch now
> > > which should fix this issue.  I'll upload a new developer snapshot to
> > > https://cygwin.com/snapshots/ later today or tomorrow (still looking
> > > into the PrlFS issue).  Please give it a try.
> > 
> > Snapshot is up.  The fix is in the 2.3.0-0.5 test release as well.
> > Please give any one of them a try.
> 
> I tested snapshot cygwin1-20151102.dll, and it works in all of my tests.  Nice!

One good news, thanks!

> I'm glad you figured out how to delete those pending signals.

Given that I introduced the regression in the first place, it's a bit
embarrassing that it took me so long to understand it 8-}


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]