RE: Threaded Cygwin Python Import Problem

RE: Threaded Cygwin Python Import Problem

[Robert Collins]

Do WIN32 handles need to be specially opened to allow them to be
inherited into the child process created by fork?

Thats the only thing that needs fixing AFAICT.

Rob
 

> -----Original Message-----
> From: Christopher Faylor [ mailto:cgf@redhat.com ]
> Sent: Thursday, July 19, 2001 10:09 AM
> To: cygwin@cygwin.com
> Subject: Re: Threaded Cygwin Python Import Problem
> 
> 
> On Wed, Jul 18, 2001 at 07:47:38PM -0400, Greg Smith wrote:
> >
> >I think I've recreated Jason's threaded Python problem after fork(),
> >see the testcase below.  And this is on my w2k sp2 dual pIII 850
> >machine, where I have been unsuccessful in recreating Jason's
> >original Python testcase.
> >
> >I certainly do not claim Cygwin expertise, nor Windows API expertise,
> >and not as much *ix as I'd like (ask me about os/390, however, and
> >I'll give you a different story ;-).
> >
> >But, it looks like fork_copy() blindly copies everything from the
> >original process address space to the new address space, including
> >any pthread_mutex_t structures.  And neither __pthread_mutex_lock
> >nor WaitForSingleObject() recognize that the mutex is not properly
> >initialized.  If the comments in the testcase are removed (ie clear
> >the lock and initialize it), then the testcase runs as expected.
> 
> If that is the case then a "fixup_after_fork" routine is probably
> required.  You can see several of those in fork.cc.
> 
> cgf
> 
> --
> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
> Bug reporting:         http://cygwin.com/bugs.html
> Documentation:         http://cygwin.com/docs.html
> FAQ:                   http://cygwin.com/faq/
> 
> 

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: Threaded Cygwin Python Import Problem

[Robert Collins]

> -----Original Message-----
> From: Greg Smith [ mailto:gsmith@nc.rr.com ]
> Sent: Thursday, July 19, 2001 10:49 AM
> To: cygwin@cygwin.com
> Subject: Re: Threaded Cygwin Python Import Problem
> 
> 
> I quite agree.  I suppose this will require a linked structure
> pointing to locks/conditions that need to be re-initialized
> (attributes, threads/lock/condition/whatever, are probably ok?).

Yup. Which I helpfully got rid of in my rewrite :-/. Actually there was
no fixup-after-fork routine at that point in tie, but there was a fixed
size array. Sounds like we'll need a thread safe linked list like the
MTInterface::destructors list I created for pthread_atfork().
 
> But thinking of forking a threaded process makes my head hurt ;-)
> So, can I be so bold as to ask a few questions ?? I know I can
> discern the answers by RTFS, but it would help to hear the answer
> from an authoritative source, and maybe save a little research
> along the way...

RTFS would help :]. But RTFSuSv2 would help more :].
 
> In a threaded process, when fork() is issued, are all the threads
> created in the new process, or just the thread that did the fork ?

The thread that forked. No other threads are duplicated.

> I'd guess the latter; what if a separate thread owned a lock and
> was doing it's thing ? then you wouldn't want the new process to
> do the same thing thinking it owned the lock too ?  In our 
> application,

pthread_atfork() is a fork syncronisation routine that allows pthreaded
applications to prepare for fork() (ie wait for all locks to be
cleared); and then run after-fork-in-parent and after-fork-in-child
code.

> that we discussed the other day, we had a problem in 98/me where
> fork failed issued by a non-main thread, but all it wanted to do
> was asynchrously issue a shell command; it might be catastrophic if
> the threaded environment was replicated...

It's not - so no problemo. 

Rob
 
> Anyway, I'll start reading the source.
> 
> Thanks,
> 
> Greg
> 
> Christopher Faylor wrote:
> > >But, it looks like fork_copy() blindly copies everything from the
> > >original process address space to the new address space, including
> > >any pthread_mutex_t structures.
> > If that is the case then a "fixup_after_fork" routine is probably
> > required.  You can see several of those in fork.cc.
> >
> 
> --
> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
> Bug reporting:         http://cygwin.com/bugs.html
> Documentation:         http://cygwin.com/docs.html
> FAQ:                   http://cygwin.com/faq/
> 
> 

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Threaded Cygwin Python Import Problem

[Greg Smith]

I quite agree.  I suppose this will require a linked structure
pointing to locks/conditions that need to be re-initialized
(attributes, threads/lock/condition/whatever, are probably ok?).

But thinking of forking a threaded process makes my head hurt ;-)
So, can I be so bold as to ask a few questions ?? I know I can
discern the answers by RTFS, but it would help to hear the answer
from an authoritative source, and maybe save a little research
along the way...

In a threaded process, when fork() is issued, are all the threads
created in the new process, or just the thread that did the fork ?
I'd guess the latter; what if a separate thread owned a lock and
was doing it's thing ? then you wouldn't want the new process to
do the same thing thinking it owned the lock too ?  In our application,
that we discussed the other day, we had a problem in 98/me where
fork failed issued by a non-main thread, but all it wanted to do
was asynchrously issue a shell command; it might be catastrophic if
the threaded environment was replicated...

Anyway, I'll start reading the source.

Thanks,

Greg

Christopher Faylor wrote:
> >But, it looks like fork_copy() blindly copies everything from the
> >original process address space to the new address space, including
> >any pthread_mutex_t structures.
> If that is the case then a "fixup_after_fork" routine is probably
> required.  You can see several of those in fork.cc.
>

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Threaded Cygwin Python Import Problem

[Christopher Faylor]

On Wed, Jul 18, 2001 at 07:47:38PM -0400, Greg Smith wrote:
>
>I think I've recreated Jason's threaded Python problem after fork(),
>see the testcase below.  And this is on my w2k sp2 dual pIII 850
>machine, where I have been unsuccessful in recreating Jason's
>original Python testcase.
>
>I certainly do not claim Cygwin expertise, nor Windows API expertise,
>and not as much *ix as I'd like (ask me about os/390, however, and
>I'll give you a different story ;-).
>
>But, it looks like fork_copy() blindly copies everything from the
>original process address space to the new address space, including
>any pthread_mutex_t structures.  And neither __pthread_mutex_lock
>nor WaitForSingleObject() recognize that the mutex is not properly
>initialized.  If the comments in the testcase are removed (ie clear
>the lock and initialize it), then the testcase runs as expected.

If that is the case then a "fixup_after_fork" routine is probably
required.  You can see several of those in fork.cc.

cgf

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Threaded Cygwin Python Import Problem

[Greg Smith]

I think I've recreated Jason's threaded Python problem after fork(),
see the testcase below.  And this is on my w2k sp2 dual pIII 850
machine, where I have been unsuccessful in recreating Jason's
original Python testcase.

I certainly do not claim Cygwin expertise, nor Windows API expertise,
and not as much *ix as I'd like (ask me about os/390, however, and
I'll give you a different story ;-).

But, it looks like fork_copy() blindly copies everything from the
original process address space to the new address space, including
any pthread_mutex_t structures.  And neither __pthread_mutex_lock
nor WaitForSingleObject() recognize that the mutex is not properly
initialized.  If the comments in the testcase are removed (ie clear
the lock and initialize it), then the testcase runs as expected.

I can sympathize with the solution `Don't DO that'; forking a
multi-threaded process is murky at best anyways.  A humble suggestion
would be to include the pid number with the check to see if the lock
is properly initialized.

Greg

gsmith@SMITHS1006 ~
$ cat ftest.c
#include <pthread.h>
#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>

int main (int argc, char *argv[])
{
pthread_mutex_t l;
pid_t           p;

   if (pthread_mutex_init(&l, NULL) == -1) {
      perror ("pthread_mutex_init"); abort(1); }

   if (pthread_mutex_lock(&l) == -1) {
      perror ("pthread_mutex_lock"); abort(2); }

   if (pthread_mutex_unlock(&l) == -1) {
      perror ("pthread_mutex_unlock"); abort(3); }

   if (p = fork()) {
      printf("parent waiting for %d\n",p);
      waitpid(p, NULL, 0);
   }
   else {
      sleep(1);

//    memset(&l, 0, sizeof(l));
//    if (pthread_mutex_init(&l, NULL) == -1) {
//       perror ("pthread_mutex_init"); abort(1); }

      printf("child locking\n",p);
      if (pthread_mutex_lock(&l) == -1) {
         perror ("pthread_mutex_unlock"); abort(4); }

      printf("child unlocking\n",p);
      if (pthread_mutex_lock(&l) == -1) {
         perror ("pthread_mutex_unlock"); abort(5); }
   }
}

gsmith@SMITHS1006 ~
$ gcc -o ftest ftest.c

gsmith@SMITHS1006 ~
$ ./ftest
parent waiting for 1224
child locking

gdb `bt' for thread[1] in pid 1224 shows:

(gdb) bt
#0  0x77f827e8 in _libkernel32_a_iname ()
#1  0x77e86a15 in _libkernel32_a_iname ()
#2  0x77e86a3d in _libkernel32_a_iname ()
#3  0x6105f1aa in pthread_mutex::Lock (this=0xa0100c8)
    at /src/winsup/cygwin/thread.cc:644
#4  0x61060c66 in __pthread_mutex_lock (mutex=0x240feb0)
    at /src/winsup/cygwin/thread.cc:1917
#5  0x6103b32e in pthread_mutex_lock (mutex=0x240feb0)
    at /src/winsup/cygwin/pthread.cc:240
#6  0x004011c9 in ?? ()
#7  0x61003b0a in dll_crt0_1 () at /src/winsup/cygwin/dcrt0.cc:859
#8  0x61003cdd in _dll_crt0 () at /src/winsup/cygwin/dcrt0.cc:925
#9  0x61003d1c in dll_crt0 (uptr=0x0) at /src/winsup/cygwin/dcrt0.cc:937
#10 0x00401287 in ?? ()
#11 0x0040103d in ?? ()
#12 0x77e97d08 in _libkernel32_a_iname ()

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/