Bug in fork() while in a thread

Bug in fork() while in a thread

[Jason Curl]

[-- Attachment #1: Type: text/plain, Size: 1097 bytes --]

Hello,

Is it allowed to issue the fork() system call while not in the main 
thread? When I read the OpenGroup specifications I don't seem to find 
anything against allowing this.

In particular, if I create a thread, then issue a fork(), data that 
exists on the stack is corrupted after the fork() is in the child. Using 
data on the heap doesn't show any issues (and is currently my 
workaround, in case this is a bug).

I've created a minimal example:
$ gcc -g -O0 forktestcase.c -o forktestcase.exe
$ ./forktest
Pointer BEFORE THE FORK env is 0x28cd00
This is the child
Pointer env is 0x28cd00
  env[0] is
  env[1] is
Child exited with 0

Further, a stackdump is now present, probably while iterating through 
the null terminated array at 0x28cd00 after the fork().

If you have a look at the attached code, I believe the expected output is
$ ./forktest
Pointer BEFORE THE FORK env is 0x28cd00
This is the child
Pointer env is 0x28cd00
Child exited with 0

This bug is not recreatable if we execute fork() in the main thread and 
not a separate thread.

Please find attached the test case.



[-- Attachment #2: forktestcase.c --]
[-- Type: text/plain, Size: 1335 bytes --]

#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <pthread.h>

struct mydat {
	pthread_t pth;
	char    **env;
	pid_t     pid;
	int       status;
};

extern char **environ;

void *callfunc(void *data)
{
	struct mydat *dat = data;
	int pid;
	int i;
	
	printf("Pointer BEFORE THE FORK env is %p\n", dat->env);
	i = 0;
	while (dat->env[i]) {
		printf(" env[%d] is %s\n", i, dat->env[i]);
		i++;
	}


	pid = fork();
	if (pid == 0) {
		int i;
		printf("This is the child\n");
		
		printf("Pointer env is %p\n", dat->env);
		i = 0;
		while (dat->env[i]) {
			printf(" env[%d] is %s\n", i, dat->env[i]);
			i++;
		}
		
		exit(0);
	} else {
		int status;
		
		dat->pid = pid;
		waitpid(pid, &status, 0);
		dat->status = status;
		printf("Child exited with %d\n", WEXITSTATUS(status));
	}
}

void newthread(struct mydat *dat)
{
	if (pthread_create(&dat->pth, NULL, callfunc, dat)) {
		perror("Couldn't create thread\n");
		return;
	}
	
	pthread_join(dat->pth, NULL);
}

int main(int argc, char **argv)
{
	struct mydat *dat;
	int   value1;
	int   valueb1, valueb2;
	int   fd1, fd2;
	char *env2[] = { "APPTESTRESULT=42", (char *)0 };
	char *env[] = { (char *)0 };
	
	dat = malloc(sizeof(struct mydat));

	dat->env = env;
	newthread(dat);
	
	free(dat);
}

[-- Attachment #3: Type: text/plain, Size: 218 bytes --]

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Bug in fork() while in a thread

[Christopher Faylor]

On Sun, Aug 15, 2010 at 07:42:01PM +0200, Jason Curl wrote:
>Is it allowed to issue the fork() system call while not in the main 
>thread? When I read the OpenGroup specifications I don't seem to find 
>anything against allowing this.
>
>In particular, if I create a thread, then issue a fork(), data that 
>exists on the stack is corrupted after the fork() is in the child. Using 
>data on the heap doesn't show any issues (and is currently my 
>workaround, in case this is a bug).

If I'm reading this correctly then "the stack" in this case is the stack
associated with the main thread.  Cygwin only duplicates the stack in
the executing thread.  In your example, env (or presumably env2) from
the main thread is passed to another thread which then calls fork.  In
that scenario, the forked process is going to see garbage in env since
the array has never been initialized.

It is theoretically possible to duplicate the stack of the main thread
and other threads in the forked process but this isn't something that I,
personally, would want to take on.  I'm the guy who wrote the code that
duplicates the state of the stack when a thread forks.  It was done at
the request of a customer and it was very tricky to get right.  This
isn't an experience I'd willingly take on again since it would be a lot
of work, would require testing on every Windows OS from NT4 on, and
would potentially slow down an already slow down Cygwin's already slow
fork() implementation.

However, if this is something that you're willing to do, I'll happily
review a patch to the cygwin1.dll.  You'd need to modify fork related
code in dcrt0.cc.

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Bug in fork() while in a thread

[Jason Curl]

On 15/08/2010 20:53, Christopher Faylor wrote:
> On Sun, Aug 15, 2010 at 07:42:01PM +0200, Jason Curl wrote:
>> In particular, if I create a thread, then issue a fork(), data that
>> exists on the stack is corrupted after the fork() is in the child. Using
>> data on the heap doesn't show any issues (and is currently my
>> workaround, in case this is a bug).
>
> If I'm reading this correctly then "the stack" in this case is the stack
> associated with the main thread.  Cygwin only duplicates the stack in
> the executing thread.  In your example, env (or presumably env2) from
> the main thread is passed to another thread which then calls fork.  In
> that scenario, the forked process is going to see garbage in env since
> the array has never been initialized.
>
> It is theoretically possible to duplicate the stack of the main thread
> and other threads in the forked process but this isn't something that I,
> personally, would want to take on.  I'm the guy who wrote the code that
> duplicates the state of the stack when a thread forks.  It was done at
> the request of a customer and it was very tricky to get right.  This
> isn't an experience I'd willingly take on again since it would be a lot
> of work, would require testing on every Windows OS from NT4 on, and
> would potentially slow down an already slow down Cygwin's already slow
> fork() implementation.
>
> However, if this is something that you're willing to do, I'll happily
> review a patch to the cygwin1.dll.  You'd need to modify fork related
> code in dcrt0.cc.

Thanks Chris for the detailed reply. I've got a workaround and you've 
confirmed it is correct with your explanation. I don't get much time, my 
C++ needs improving, let's see what I can do.

>
> cgf
>



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Bug in fork() while in a thread

[Corinna Vinschen]

On Aug 15 14:53, Christopher Faylor wrote:
> On Sun, Aug 15, 2010 at 07:42:01PM +0200, Jason Curl wrote:
> >Is it allowed to issue the fork() system call while not in the main 
> >thread? When I read the OpenGroup specifications I don't seem to find 
> >anything against allowing this.
> >
> >In particular, if I create a thread, then issue a fork(), data that 
> >exists on the stack is corrupted after the fork() is in the child. Using 
> >data on the heap doesn't show any issues (and is currently my 
> >workaround, in case this is a bug).
> 
> If I'm reading this correctly then "the stack" in this case is the stack
> associated with the main thread.  Cygwin only duplicates the stack in
> the executing thread.  In your example, env (or presumably env2) from
> the main thread is passed to another thread which then calls fork.  In
> that scenario, the forked process is going to see garbage in env since
> the array has never been initialized.
> 
> It is theoretically possible to duplicate the stack of the main thread
> and other threads in the forked process [...]

I guess I'm missing something here.  Here's an excerpt from the SUSv4
fork man page:

  The fork() function shall create a new process. The new process (child
  process) shall be an exact copy of the calling process (parent process)
  except as detailed below:

  [...]

  o A process shall be created with a single thread. If a
    multi-threaded process calls fork(), the new process shall contain
    a replica of the calling thread and its entire address space,
    possibly including the states of mutexes and other resources.
    [...]

Is the stack of another thread, which is not executed in the forked
process anymore, a part of the calling's thread "entire address space"?


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Bug in fork() while in a thread

[Christopher Faylor]

On Mon, Aug 16, 2010 at 11:05:52AM +0200, Corinna Vinschen wrote:
>On Aug 15 14:53, Christopher Faylor wrote:
>> On Sun, Aug 15, 2010 at 07:42:01PM +0200, Jason Curl wrote:
>> >Is it allowed to issue the fork() system call while not in the main 
>> >thread? When I read the OpenGroup specifications I don't seem to find 
>> >anything against allowing this.
>> >
>> >In particular, if I create a thread, then issue a fork(), data that 
>> >exists on the stack is corrupted after the fork() is in the child. Using 
>> >data on the heap doesn't show any issues (and is currently my 
>> >workaround, in case this is a bug).
>> 
>> If I'm reading this correctly then "the stack" in this case is the stack
>> associated with the main thread.  Cygwin only duplicates the stack in
>> the executing thread.  In your example, env (or presumably env2) from
>> the main thread is passed to another thread which then calls fork.  In
>> that scenario, the forked process is going to see garbage in env since
>> the array has never been initialized.
>> 
>> It is theoretically possible to duplicate the stack of the main thread
>> and other threads in the forked process [...]
>
>I guess I'm missing something here.  Here's an excerpt from the SUSv4
>fork man page:
>
>  The fork() function shall create a new process. The new process (child
>  process) shall be an exact copy of the calling process (parent process)
>  except as detailed below:
>
>  [...]
>
>  o A process shall be created with a single thread. If a
>    multi-threaded process calls fork(), the new process shall contain
>    a replica of the calling thread and its entire address space,
>    possibly including the states of mutexes and other resources.
>    [...]
>
>Is the stack of another thread, which is not executed in the forked
>process anymore, a part of the calling's thread "entire address space"?

I saw that too but I can confirm that linux duplicates the main thread.
That isn't too surprising since it probably just replicates the entire
address space of the process including all of the stacks of all of the
threads.

Cygwin is more "efficient" than that.

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Bug in fork() while in a thread

[Jason Curl]

Corinna Vinschen <corinna-cygwin <at> cygwin.com> writes:
> On Aug 15 14:53, Christopher Faylor wrote:
> > On Sun, Aug 15, 2010 at 07:42:01PM +0200, Jason Curl wrote:
> > >Is it allowed to issue the fork() system call while not in the main 
> > >thread? When I read the OpenGroup specifications I don't seem to find 
> > >anything against allowing this.
> > If I'm reading this correctly then "the stack" in this case is the stack
> > associated with the main thread.  Cygwin only duplicates the stack in
> > the executing thread.  In your example, env (or presumably env2) from
> > the main thread is passed to another thread which then calls fork.  In
> > that scenario, the forked process is going to see garbage in env since
> > the array has never been initialized.
> I guess I'm missing something here.  Here's an excerpt from the SUSv4
> fork man page:
> 
>   The fork() function shall create a new process. The new process (child
>   process) shall be an exact copy of the calling process (parent process)
>   except as detailed below:
> 
>   [...]
> 
>   o A process shall be created with a single thread. If a
>     multi-threaded process calls fork(), the new process shall contain
>     a replica of the calling thread and its entire address space,
>     possibly including the states of mutexes and other resources.
>     [...]
> 
> Is the stack of another thread, which is not executed in the forked
> process anymore, a part of the calling's thread "entire address space"?

Open Group Posix.1c-2004 mentions only a "signal stack" doesn't need to be 
copied for XSI.

Linux & FreeBSD 7.0 work OK. QNX641 returns ENOSYS if it even sniffs a thread 
call. I haven't tested Solaris Sparc. 

Which standard is Cygwin "closest" to, that I can use as a reference when I 
trip up on similar problems? I have the hint the SuSv4 implementation of Posix.

Thanks for your wonderful support.
Jason.




--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Bug in fork() while in a thread

[Corinna Vinschen]

On Aug 21 08:50, Jason Curl wrote:
> Open Group Posix.1c-2004 mentions only a "signal stack" doesn't need to be 
> copied for XSI.
> 
> Linux & FreeBSD 7.0 work OK. QNX641 returns ENOSYS if it even sniffs a thread 
> call. I haven't tested Solaris Sparc. 
> 
> Which standard is Cygwin "closest" to, that I can use as a reference when I 
> trip up on similar problems? I have the hint the SuSv4 implementation of Posix.

We're trying for Linux in the first place, with POSIX/SusV4 as fallback.
Due to a lot of Windows quirks and the limited number of Cygwin "kernel'
hackers, it's not exactly simple.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple