public inbox for gcc@gcc.gnu.org
 help / color / mirror / Atom feed
* RE: libiberty/pex-unix vfork abuse?
@ 2007-12-07 17:24 J.C. Pizarro
  2007-12-07 17:42 ` Dave Korn
  0 siblings, 1 reply; 26+ messages in thread
From: J.C. Pizarro @ 2007-12-07 17:24 UTC (permalink / raw)
  To: Dave Korn, Ian Lance Taylor, gcc

On 2007/12/07, "Dave Korn" <dave.korn@artimi.com> wrote:
> > On the other hand, the setting of environ is very dubious and is
> > likely to break on real systems.  The code should be changed to call
> > execve instead.  Unfortunately there is no standard execvpe function.
> > Fortunately gcc never uses the variant which sets environ.  Offhand
> > I'm not sure what does.
>
>   Perhaps we could work around this case by setting environ in the parent
> before the vfork call and restoring it afterward, but we'd need kind of
> serialisation there, and I don't know how to do a critical section using
> pthreads/posix.

You can do a critical section mainly between processes using system calls of
IPC synchronization like filelocks, RPCs, shared memory with mmap and
mutexes/semaphores, messages passing through pipes as tunnels, MPI, etc.

Now well, a critical section between multithreaded processes are complicated.

   J.C.Pizarro

^ permalink raw reply	[flat|nested] 26+ messages in thread

* RE: libiberty/pex-unix vfork abuse?
  2007-12-07 17:24 libiberty/pex-unix vfork abuse? J.C. Pizarro
@ 2007-12-07 17:42 ` Dave Korn
  2007-12-07 17:50   ` Joe Buck
  0 siblings, 1 reply; 26+ messages in thread
From: Dave Korn @ 2007-12-07 17:42 UTC (permalink / raw)
  To: 'J.C. Pizarro', 'Ian Lance Taylor', gcc

On 07 December 2007 17:24, J.C. Pizarro wrote:

> You can do a critical section mainly between processes 

  Thanks for your well-meaning attempt to help, but you don't understand what
we're talking about, and sending a generic list of synchronisation techniques
without regard to their relevance or applicability in this situation doesn't
actually tell either me or Ian anything we didn't already know nor advance the
discussion any.

    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: libiberty/pex-unix vfork abuse?
  2007-12-07 17:42 ` Dave Korn
@ 2007-12-07 17:50   ` Joe Buck
  2007-12-07 18:09     ` J.C. Pizarro
  0 siblings, 1 reply; 26+ messages in thread
From: Joe Buck @ 2007-12-07 17:50 UTC (permalink / raw)
  To: Dave Korn; +Cc: 'J.C. Pizarro', 'Ian Lance Taylor', gcc

On Fri, Dec 07, 2007 at 05:41:50PM -0000, Dave Korn wrote:
> On 07 December 2007 17:24, J.C. Pizarro wrote:
> 
> > You can do a critical section mainly between processes 
> 
>   Thanks for your well-meaning attempt to help, but you don't understand what
> we're talking about, and sending a generic list of synchronisation techniques
> without regard to their relevance or applicability in this situation doesn't
> actually tell either me or Ian anything we didn't already know nor advance the
> discussion any.

And this is hardly an isolated case.  J.C., please stop responding to
every issue with a grab-bag of buzzwords and generic solutions.  As
Dave says, you aren't telling people anything they didn't know.

If you actually experiment with a specific idea and have data, by all
means submit that.  But random suggestions based on something you read in
school are useless, and many developers on this list already took those
classes and read those papers.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: libiberty/pex-unix vfork abuse?
  2007-12-07 17:50   ` Joe Buck
@ 2007-12-07 18:09     ` J.C. Pizarro
  2007-12-07 18:14       ` Andrew Haley
                         ` (3 more replies)
  0 siblings, 4 replies; 26+ messages in thread
From: J.C. Pizarro @ 2007-12-07 18:09 UTC (permalink / raw)
  To: Joe Buck, gcc

2007/12/7, Joe Buck <Joe.Buck@synopsys.com> wrote:
> On Fri, Dec 07, 2007 at 05:41:50PM -0000, Dave Korn wrote:
> > On 07 December 2007 17:24, J.C. Pizarro wrote:
> >
> > > You can do a critical section mainly between processes
> >
> >   Thanks for your well-meaning attempt to help, but you don't understand what
> > we're talking about, and sending a generic list of synchronisation techniques
> > without regard to their relevance or applicability in this situation doesn't
> > actually tell either me or Ian anything we didn't already know nor advance the
> > discussion any.
>
> And this is hardly an isolated case.  J.C., please stop responding to
> every issue with a grab-bag of buzzwords and generic solutions.  As
> Dave says, you aren't telling people anything they didn't know.
>
> If you actually experiment with a specific idea and have data, by all
> means submit that.  But random suggestions based on something you read in
> school are useless, and many developers on this list already took those
> classes and read those papers.

> But random suggestions based on something you read in school are useless

You're wrong. My suggestions are not based from school and are not useless.
My suggestions are based from university, books, papers and internet, and
i did put those by a same reason, my freedom.

Do you permit me a question for you?

   "Are important the suggestions?"

   J.C.Pizarro

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: libiberty/pex-unix vfork abuse?
  2007-12-07 18:09     ` J.C. Pizarro
@ 2007-12-07 18:14       ` Andrew Haley
  2007-12-07 18:28       ` Dave Korn
                         ` (2 subsequent siblings)
  3 siblings, 0 replies; 26+ messages in thread
From: Andrew Haley @ 2007-12-07 18:14 UTC (permalink / raw)
  To: J.C. Pizarro; +Cc: Joe Buck, gcc

J.C. Pizarro writes:

 > You're wrong. My suggestions are not based from school and are not useless.
 > My suggestions are based from university, books, papers and internet, and
 > i did put those by a same reason, my freedom.

You have the freedom to make useless postings to this list, just as we
have freedom to ask you to stop.  Please stop.

Andrew.

-- 
Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, UK
Registered in England and Wales No. 3798903

^ permalink raw reply	[flat|nested] 26+ messages in thread

* RE: libiberty/pex-unix vfork abuse?
  2007-12-07 18:09     ` J.C. Pizarro
  2007-12-07 18:14       ` Andrew Haley
@ 2007-12-07 18:28       ` Dave Korn
  2007-12-07 18:47         ` J.C. Pizarro
  2007-12-07 18:31       ` Diego Novillo
  2007-12-07 22:14       ` Gabriel Dos Reis
  3 siblings, 1 reply; 26+ messages in thread
From: Dave Korn @ 2007-12-07 18:28 UTC (permalink / raw)
  To: 'J.C. Pizarro', 'Joe Buck', gcc

On 07 December 2007 18:09, J.C. Pizarro wrote:

> You're wrong. My suggestions are not based from school and are not useless.

  Now /you're/ wrong: your suggestions *are* useless.  You suggested using
inter-process communications to try and resolve a potential data-access race
condition between multiple threads within the *same* process.  This can NOT be
done by any of "filelocks, RPCs, shared memory with mmap and
mutexes/semaphores, messages passing through pipes as tunnels, MPI".

 - A filelock cannot be used to prevent a second thread within the same
process from reading from or writing to the environ[] array (or any other
variable) or calling setenv/getenv while a first thread temporarily swaps the
value in it.

 - A remote procedure call cannot stop a second thread in the same process
from reading the environ[] array (or any other ... (etc).

 - An mmap'd shared memory block cannot stop a second thread  ... (etc).

 - Passing a message through a pipe cannot stop a second thread ... (etc).

 - A mutex/semaphore *could* do this, but only if every call to getenv/setenv
and every direct access to the environ array were *also* wrapped in mutex lock
calls.  This is not something you can work around in a library implementation
of pexecute, it would require every author of every program that uses
libiberty to modify their entire code.

  The fact that you have suggested all those useless suggestions proves that
you have not understood what we are discussing, which was how to temporarily
alter the value of a global variable in one thread of a program without any
other thread seeing the altered value, when we do not control the other
threads because we are just a library function, not the application itself.
The generic solution for this problem, which I mentioned in my original post,
is a "critical section", which is a stretch of code that locks out the
scheduler from scheduling any other threads of the process to run during the
period the lock is held.

> Do you permit me a question for you?
> 
>    "Are important the suggestions?"

  Their importance is proportional to the product of their relevance
multiplied by their feasibility.  Your suggestions above were all irrelevant,
except for the unfeasible one.  0*(anything) == (anything)*0 == 0*0 == 0.

    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: libiberty/pex-unix vfork abuse?
  2007-12-07 18:09     ` J.C. Pizarro
  2007-12-07 18:14       ` Andrew Haley
  2007-12-07 18:28       ` Dave Korn
@ 2007-12-07 18:31       ` Diego Novillo
  2007-12-07 22:14       ` Gabriel Dos Reis
  3 siblings, 0 replies; 26+ messages in thread
From: Diego Novillo @ 2007-12-07 18:31 UTC (permalink / raw)
  To: J.C. Pizarro, Joe.Buck, gcc

On 12/7/07 1:09 PM, J.C. Pizarro wrote:

> Do you permit me a question for you?
> 
>    "Are important the suggestions?"
> 
>    J.C.Pizarro

JC,

The problem that many of us have with your responses is that they are 
almost always content-free.  You do not seem to grasp the basic 
principles of the issues that you write about.  Your responses are 
purely an amalgamation of very basic terms that you seem to have gotten 
from books/papers.

We *all* know those things.  We generally take that for granted.  So, 
when you offer only that information, you are not contributing anything 
to the discussion.  You are just adding noise.

The problem is that over time, people stop listening to you and then 
when you eventually do have an actual contribution, nobody will listen 
to it, because people will already be too tired of arguing with you.

Just a suggestion.  Maybe you *do* have something useful to contribute, 
it's just hard to see what it might be.


Diego.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: libiberty/pex-unix vfork abuse?
  2007-12-07 18:28       ` Dave Korn
@ 2007-12-07 18:47         ` J.C. Pizarro
  0 siblings, 0 replies; 26+ messages in thread
From: J.C. Pizarro @ 2007-12-07 18:47 UTC (permalink / raw)
  To: Dave Korn, gcc

On 2007/12/7, Dave Korn <dave.korn@artimi.com> wrote:
> On 07 December 2007 18:09, J.C. Pizarro wrote:
>
> > You're wrong. My suggestions are not based from school and are not useless.
>
>   Now /you're/ wrong: your suggestions *are* useless.  You suggested using
> inter-process communications to try and resolve a potential data-access race
> condition between multiple threads within the *same* process.  This can NOT be
> done by any of "filelocks, RPCs, shared memory with mmap and
> mutexes/semaphores, messages passing through pipes as tunnels, MPI".
>
>  - A filelock cannot be used to prevent a second thread within the same
> process from reading from or writing to the environ[] array (or any other
> variable) or calling setenv/getenv while a first thread temporarily swaps the
> value in it.
>
>  - A remote procedure call cannot stop a second thread in the same process
> from reading the environ[] array (or any other ... (etc).
>
>  - An mmap'd shared memory block cannot stop a second thread  ... (etc).
>
>  - Passing a message through a pipe cannot stop a second thread ... (etc).
>
>  - A mutex/semaphore *could* do this, but only if every call to getenv/setenv
> and every direct access to the environ array were *also* wrapped in mutex lock
> calls.  This is not something you can work around in a library implementation
> of pexecute, it would require every author of every program that uses
> libiberty to modify their entire code.
>
>   The fact that you have suggested all those useless suggestions proves that
> you have not understood what we are discussing, which was how to temporarily
> alter the value of a global variable in one thread of a program without any
> other thread seeing the altered value, when we do not control the other
> threads because we are just a library function, not the application itself.
> The generic solution for this problem, which I mentioned in my original post,
> is a "critical section", which is a stretch of code that locks out the
> scheduler from scheduling any other threads of the process to run during the
> period the lock is held.
>
> > Do you permit me a question for you?
> >
> >    "Are important the suggestions?"
>
>   Their importance is proportional to the product of their relevance
> multiplied by their feasibility.  Your suggestions above were all irrelevant,
> except for the unfeasible one.  0*(anything) == (anything)*0 == 0*0 == 0.
>
>     cheers,
>       DaveK
> --
> Can't think of a witty .sigline today....

and you wrote too at start of this topic:

> "The vfork() function shall be equivalent to fork(), except that the behavior
> is undefined if the process created by vfork() either modifies any data other
> than a variable of type pid_t used to store the return value from vfork(), or
> returns from the function in which vfork() was called, or calls any other
> function before successfully calling _exit() or one of the exec family of
> functions."

Briefly, it's fork-like. It's about processes, not threads.
It's about parent and children processes.
And the another man talked about threads (critical section for threads).

   J.C.Pizarro "the noiser" (theirs freedoms that they want to stop me)

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: libiberty/pex-unix vfork abuse?
  2007-12-07 18:09     ` J.C. Pizarro
                         ` (2 preceding siblings ...)
  2007-12-07 18:31       ` Diego Novillo
@ 2007-12-07 22:14       ` Gabriel Dos Reis
  3 siblings, 0 replies; 26+ messages in thread
From: Gabriel Dos Reis @ 2007-12-07 22:14 UTC (permalink / raw)
  To: J.C. Pizarro; +Cc: Joe Buck, gcc

"J.C. Pizarro" <jcpiza@gmail.com> writes:

| > But random suggestions based on something you read in school are useless
| 
| You're wrong. My suggestions are not based from school and are not useless.
| My suggestions are based from university, books, papers and internet

What is the difference?

-- Gaby

^ permalink raw reply	[flat|nested] 26+ messages in thread

* RE: libiberty/pex-unix vfork abuse?
  2007-12-10 20:06           ` Brian Dessent
@ 2007-12-11  3:31             ` Dave Korn
  0 siblings, 0 replies; 26+ messages in thread
From: Dave Korn @ 2007-12-11  3:31 UTC (permalink / raw)
  To: gcc, 'Andreas Schwab'; +Cc: 'Ian Lance Taylor'

On 10 December 2007 20:01, Brian Dessent wrote:

> Andreas Schwab wrote:
> 
>> Typically in a multithreaded environment vfork is mapped to fork anyway.
> 
> ...which is what I don't understand about this whole thread.  It seems
> Dave is seeing some strange behavior in Cygwin, but Cygwin's vfork =
> fork, there is no difference.  There used to be a vfork specialization
> in Cygwin, but it is broken and has not been enabled in quite a long
> time.

  Yes, I've noticed it's #ifdef'd out now I've been through the code more
thoroughly, and it's also the case that in the old version of gcc/libiberty I'm
using the manipulation of the environ variable isn't there either, so I don't
have a current problem to solve myself, but some of the embedded guys might want
to keep an eye out for it.

    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: libiberty/pex-unix vfork abuse?
  2007-12-10 22:44             ` Andreas Schwab
@ 2007-12-10 23:10               ` Daniel Jacobowitz
  0 siblings, 0 replies; 26+ messages in thread
From: Daniel Jacobowitz @ 2007-12-10 23:10 UTC (permalink / raw)
  To: gcc

On Mon, Dec 10, 2007 at 11:35:15PM +0100, Andreas Schwab wrote:
> Glibc will map vfork to fork in a multithreaded environment.

LinuxThreads used to.  NPTL does not; this caused various trouble for
GDB at the time.

-- 
Daniel Jacobowitz
CodeSourcery

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: libiberty/pex-unix vfork abuse?
  2007-12-10 19:22         ` Joe Buck
  2007-12-10 19:37           ` Daniel Jacobowitz
  2007-12-10 21:05           ` Ian Lance Taylor
@ 2007-12-10 22:56           ` Andreas Schwab
  2 siblings, 0 replies; 26+ messages in thread
From: Andreas Schwab @ 2007-12-10 22:56 UTC (permalink / raw)
  To: Joe Buck; +Cc: Dave Korn, 'Ian Lance Taylor', gcc

Joe Buck <Joe.Buck@synopsys.COM> writes:

> While the standard's wording might need fixing, with every implementation
> of vfork I know of, there are no threads.  It's a mechanism for systems
> that don't support fork (or that can only do fork in a horribly
> inefficient way, say because there's no MMU, and no support for copy on
> write), but that support the creation of new processes.  It's just a
> cheat to support the traditional fork-followed-by-exec, an aid for porting
> Unix code to non-Unix systems.

vfork has been invented by Unix, it has nothing to do with porting.  It
was needed to avoid the overhead of copying the address space just to
throw the copy away during execve.  On modern systems the overhead is
almost nonexistent.

> The reason vfork blocks the parent until the child makes a new process
> or quits is because that's the only supported behavior on systems that
> support it; it is not really "blocking the parent" at all as there is
> only one process.

The only reason the parent needs to be suspended is because it shares
the stack with the child.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: libiberty/pex-unix vfork abuse?
  2007-12-10 19:37           ` Daniel Jacobowitz
  2007-12-10 20:24             ` Joe Buck
@ 2007-12-10 22:44             ` Andreas Schwab
  2007-12-10 23:10               ` Daniel Jacobowitz
  1 sibling, 1 reply; 26+ messages in thread
From: Andreas Schwab @ 2007-12-10 22:44 UTC (permalink / raw)
  To: gcc

Daniel Jacobowitz <drow@false.org> writes:

> On Mon, Dec 10, 2007 at 11:18:57AM -0800, Joe Buck wrote:
>> While the standard's wording might need fixing, with every implementation
>> of vfork I know of, there are no threads.  It's a mechanism for systems
>> that don't support fork (or that can only do fork in a horribly
>> inefficient way, say because there's no MMU, and no support for copy on
>> write), but that support the creation of new processes.
>
> No, Dave's right.  On GNU/Linux you can have two threads running on
> different processors simultaneously calling vfork.  And even with an
> MMU it is considerably more efficient than requiring setup of a new
> copy-on-write page table.

Glibc will map vfork to fork in a multithreaded environment.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: libiberty/pex-unix vfork abuse?
  2007-12-10 19:22         ` Joe Buck
  2007-12-10 19:37           ` Daniel Jacobowitz
@ 2007-12-10 21:05           ` Ian Lance Taylor
  2007-12-10 22:56           ` Andreas Schwab
  2 siblings, 0 replies; 26+ messages in thread
From: Ian Lance Taylor @ 2007-12-10 21:05 UTC (permalink / raw)
  To: Joe Buck; +Cc: Dave Korn, 'Andreas Schwab', gcc

Joe Buck <Joe.Buck@synopsys.COM> writes:

> I don't think it's wise to waste time fixing theoretical bugs
> exposed by close reading of the standard.  Now, messing with environ
> with vfork will mess up the parent process, and if that happens it's a
> bug.  But getting around it by using fork will harm portability, as the
> only reason to bother with vfork at all is that fork might not be
> available.

That's not the only reason.  I used vfork because I measured
performance improvements with vfork over fork.  I can't remember
whether I did the measurements on GNU/Linux or on NetBSD.

Performance improvements are not particularly surprising.  Despite the
fact that, as the Open Group specification suggests, vfork is a broken
interface, it was implemented to be faster than fork/exec, and it is
faster.

Ian

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: libiberty/pex-unix vfork abuse?
  2007-12-10 19:37           ` Daniel Jacobowitz
@ 2007-12-10 20:24             ` Joe Buck
  2007-12-10 22:44             ` Andreas Schwab
  1 sibling, 0 replies; 26+ messages in thread
From: Joe Buck @ 2007-12-10 20:24 UTC (permalink / raw)
  To: gcc

On Mon, Dec 10, 2007 at 02:22:45PM -0500, Daniel Jacobowitz wrote:
> On Mon, Dec 10, 2007 at 11:18:57AM -0800, Joe Buck wrote:
> > While the standard's wording might need fixing, with every implementation
> > of vfork I know of, there are no threads.  It's a mechanism for systems
> > that don't support fork (or that can only do fork in a horribly
> > inefficient way, say because there's no MMU, and no support for copy on
> > write), but that support the creation of new processes.
> 
> No, Dave's right.  On GNU/Linux you can have two threads running on
> different processors simultaneously calling vfork.  And even with an
> MMU it is considerably more efficient than requiring setup of a new
> copy-on-write page table.

Yes, that's true.  And of course some implementation might map vfork to
fork, which complies with standards.  I only meant to talk about races
between parent and child "process".


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: libiberty/pex-unix vfork abuse?
  2007-12-10 20:01         ` Andreas Schwab
@ 2007-12-10 20:06           ` Brian Dessent
  2007-12-11  3:31             ` Dave Korn
  0 siblings, 1 reply; 26+ messages in thread
From: Brian Dessent @ 2007-12-10 20:06 UTC (permalink / raw)
  To: Andreas Schwab; +Cc: Dave Korn, 'Ian Lance Taylor', gcc

Andreas Schwab wrote:

> Typically in a multithreaded environment vfork is mapped to fork anyway.

...which is what I don't understand about this whole thread.  It seems
Dave is seeing some strange behavior in Cygwin, but Cygwin's vfork =
fork, there is no difference.  There used to be a vfork specialization
in Cygwin, but it is broken and has not been enabled in quite a long
time.

Brian

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: libiberty/pex-unix vfork abuse?
  2007-12-10 19:19       ` Dave Korn
  2007-12-10 19:22         ` Joe Buck
@ 2007-12-10 20:01         ` Andreas Schwab
  2007-12-10 20:06           ` Brian Dessent
  1 sibling, 1 reply; 26+ messages in thread
From: Andreas Schwab @ 2007-12-10 20:01 UTC (permalink / raw)
  To: Dave Korn; +Cc: 'Ian Lance Taylor', gcc

"Dave Korn" <dave.korn@artimi.com> writes:

> On 07 December 2007 20:52, Andreas Schwab wrote:
>
>> "Dave Korn" <dave.korn@artimi.com> writes:
>> 
>>>   Perhaps we could work around this case by setting environ in the parent
>>> before the vfork call and restoring it afterward, but we'd need kind of
>>> serialisation there,
>> 
>> Do we?  vfork should block the parent until the child calls execve or
>> exit.
>
>   I don't see anything in posix that suggests that?

That is true, but technically it is rather difficult to implement a true
vfork without blocking the parent between vfork and exec/exit, since
that these are the only synchronisation points.

> I'm worrying in this case about races between multiple threads in the
> parent vfork'ing multiple children,

Typically in a multithreaded environment vfork is mapped to fork anyway.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: libiberty/pex-unix vfork abuse?
  2007-12-10 19:22         ` Joe Buck
@ 2007-12-10 19:37           ` Daniel Jacobowitz
  2007-12-10 20:24             ` Joe Buck
  2007-12-10 22:44             ` Andreas Schwab
  2007-12-10 21:05           ` Ian Lance Taylor
  2007-12-10 22:56           ` Andreas Schwab
  2 siblings, 2 replies; 26+ messages in thread
From: Daniel Jacobowitz @ 2007-12-10 19:37 UTC (permalink / raw)
  To: gcc

On Mon, Dec 10, 2007 at 11:18:57AM -0800, Joe Buck wrote:
> While the standard's wording might need fixing, with every implementation
> of vfork I know of, there are no threads.  It's a mechanism for systems
> that don't support fork (or that can only do fork in a horribly
> inefficient way, say because there's no MMU, and no support for copy on
> write), but that support the creation of new processes.

No, Dave's right.  On GNU/Linux you can have two threads running on
different processors simultaneously calling vfork.  And even with an
MMU it is considerably more efficient than requiring setup of a new
copy-on-write page table.

-- 
Daniel Jacobowitz
CodeSourcery

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: libiberty/pex-unix vfork abuse?
  2007-12-10 19:19       ` Dave Korn
@ 2007-12-10 19:22         ` Joe Buck
  2007-12-10 19:37           ` Daniel Jacobowitz
                             ` (2 more replies)
  2007-12-10 20:01         ` Andreas Schwab
  1 sibling, 3 replies; 26+ messages in thread
From: Joe Buck @ 2007-12-10 19:22 UTC (permalink / raw)
  To: Dave Korn; +Cc: 'Andreas Schwab', 'Ian Lance Taylor', gcc

On Mon, Dec 10, 2007 at 06:32:08PM -0000, Dave Korn wrote:
> On 07 December 2007 20:52, Andreas Schwab wrote:
> 
> > "Dave Korn" <dave.korn@artimi.com> writes:
> > 
> >>   Perhaps we could work around this case by setting environ in the parent
> >> before the vfork call and restoring it afterward, but we'd need kind of
> >> serialisation there,
> > 
> > Do we?  vfork should block the parent until the child calls execve or
> > exit.
> 
>   I don't see anything in posix that suggests that?  I'm worrying in this case
> about races between multiple threads in the parent vfork'ing multiple children,
> not about the child-parent interaction, which this suggestion was a workaround
> for.  (But in any case, the subsequent suggestion by Ross to just fall back on
> fork instead of vfork when the environment needs setting is probably the
> simplest and most robust, obsoleting this earlier suggestion of mine).

While the standard's wording might need fixing, with every implementation
of vfork I know of, there are no threads.  It's a mechanism for systems
that don't support fork (or that can only do fork in a horribly
inefficient way, say because there's no MMU, and no support for copy on
write), but that support the creation of new processes.  It's just a
cheat to support the traditional fork-followed-by-exec, an aid for porting
Unix code to non-Unix systems.  The reason vfork blocks the parent until
the child makes a new process or quits is because that's the only
supported behavior on systems that support it; it is not really "blocking
the parent" at all as there is only one process.

I don't think it's wise to waste time fixing theoretical bugs
exposed by close reading of the standard.  Now, messing with environ
with vfork will mess up the parent process, and if that happens it's a
bug.  But getting around it by using fork will harm portability, as the
only reason to bother with vfork at all is that fork might not be
available.


^ permalink raw reply	[flat|nested] 26+ messages in thread

* RE: libiberty/pex-unix vfork abuse?
  2007-12-07 20:55     ` Andreas Schwab
@ 2007-12-10 19:19       ` Dave Korn
  2007-12-10 19:22         ` Joe Buck
  2007-12-10 20:01         ` Andreas Schwab
  0 siblings, 2 replies; 26+ messages in thread
From: Dave Korn @ 2007-12-10 19:19 UTC (permalink / raw)
  To: 'Andreas Schwab'; +Cc: 'Ian Lance Taylor', gcc

On 07 December 2007 20:52, Andreas Schwab wrote:

> "Dave Korn" <dave.korn@artimi.com> writes:
> 
>>   Perhaps we could work around this case by setting environ in the parent
>> before the vfork call and restoring it afterward, but we'd need kind of
>> serialisation there,
> 
> Do we?  vfork should block the parent until the child calls execve or
> exit.

  I don't see anything in posix that suggests that?  I'm worrying in this case
about races between multiple threads in the parent vfork'ing multiple children,
not about the child-parent interaction, which this suggestion was a workaround
for.  (But in any case, the subsequent suggestion by Ross to just fall back on
fork instead of vfork when the environment needs setting is probably the
simplest and most robust, obsoleting this earlier suggestion of mine).

    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: libiberty/pex-unix vfork abuse?
  2007-12-07 17:09   ` Dave Korn
  2007-12-07 18:40     ` Ian Lance Taylor
@ 2007-12-07 20:55     ` Andreas Schwab
  2007-12-10 19:19       ` Dave Korn
  1 sibling, 1 reply; 26+ messages in thread
From: Andreas Schwab @ 2007-12-07 20:55 UTC (permalink / raw)
  To: Dave Korn; +Cc: 'Ian Lance Taylor', gcc

"Dave Korn" <dave.korn@artimi.com> writes:

>   Perhaps we could work around this case by setting environ in the parent
> before the vfork call and restoring it afterward, but we'd need kind of
> serialisation there,

Do we?  vfork should block the parent until the child calls execve or
exit.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: libiberty/pex-unix vfork abuse?
@ 2007-12-07 19:38 Ross Ridge
  0 siblings, 0 replies; 26+ messages in thread
From: Ross Ridge @ 2007-12-07 19:38 UTC (permalink / raw)
  To: gcc

Dave Korn writes:
>  Perhaps we could work around this case by setting environ in the parent
> before the vfork call and restoring it afterward, but we'd need kind of
> serialisation there, and I don't know how to do a critical section using
> pthreads/posix.

A simple solution would be to call fork() instead of vfork() when changing
the environment.

					Ross Ridge

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: libiberty/pex-unix vfork abuse?
  2007-12-07 17:09   ` Dave Korn
@ 2007-12-07 18:40     ` Ian Lance Taylor
  2007-12-07 20:55     ` Andreas Schwab
  1 sibling, 0 replies; 26+ messages in thread
From: Ian Lance Taylor @ 2007-12-07 18:40 UTC (permalink / raw)
  To: Dave Korn, shinwell; +Cc: gcc

"Dave Korn" <dave.korn@artimi.com> writes:

> > On the other hand, the setting of environ is very dubious and is
> > likely to break on real systems.  The code should be changed to call
> > execve instead.  Unfortunately there is no standard execvpe function.
> > Fortunately gcc never uses the variant which sets environ.  Offhand
> > I'm not sure what does.
> 
>   Perhaps we could work around this case by setting environ in the parent
> before the vfork call and restoring it afterward, but we'd need kind of
> serialisation there, and I don't know how to do a critical section using
> pthreads/posix.

The setting of environ came in here:

http://gcc.gnu.org/ml/gcc-patches/2006-05/msg00377.html

Mark, setting the global environ variable won't work correctly on Unix
when using vfork.  Your e-mail refers to the prelinker.  The prelinker
sources that I have don't use the pex routines.  Do your prelinker
sources require setting the environment when using the PEX_SEARCH flag
while setting the environment?  If not, I think the simple approach is
to disallow that flag for pex_run_in_environment, to not set environ,
and to use execve instead.

Ian

^ permalink raw reply	[flat|nested] 26+ messages in thread

* RE: libiberty/pex-unix vfork abuse?
  2007-12-07 16:59 ` Ian Lance Taylor
@ 2007-12-07 17:09   ` Dave Korn
  2007-12-07 18:40     ` Ian Lance Taylor
  2007-12-07 20:55     ` Andreas Schwab
  0 siblings, 2 replies; 26+ messages in thread
From: Dave Korn @ 2007-12-07 17:09 UTC (permalink / raw)
  To: 'Ian Lance Taylor'; +Cc: gcc

On 07 December 2007 16:59, Ian Lance Taylor wrote:

> "Dave Korn" <dave.korn@artimi.com> writes:
> 
>>   Note the several calls to dup2() and close(), which seem to me to be
>> "calls [to] any other function", and the setting of environ, which seem to
>> me to be modification of "any data other than a variable of type pid_t
>> used to store the return value from vfork()".
> 
> Despite the standardese, vfork was invented to support calling
> dup/dup2 before calling exec.  Without that feature, it would be
> nearly useless.  Any actual implementation of vfork must support
> calling dup/dup2, or it will break all real programs which use vfork.

  Ah, right, hence the 'defensive coding' relating to the fdtab in cygwin.  I
can see how hard it would be to do the standard unix fork-and-fd-swap dance
without that.  (Should possibly Cc. the austin group ml and suggest a revision
to the wording, assuming they aren't deciding to remove it altogether).

> On the other hand, the setting of environ is very dubious and is
> likely to break on real systems.  The code should be changed to call
> execve instead.  Unfortunately there is no standard execvpe function.
> Fortunately gcc never uses the variant which sets environ.  Offhand
> I'm not sure what does.

  Perhaps we could work around this case by setting environ in the parent
before the vfork call and restoring it afterward, but we'd need kind of
serialisation there, and I don't know how to do a critical section using
pthreads/posix.

    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: libiberty/pex-unix vfork abuse?
  2007-12-07 15:08 Dave Korn
@ 2007-12-07 16:59 ` Ian Lance Taylor
  2007-12-07 17:09   ` Dave Korn
  0 siblings, 1 reply; 26+ messages in thread
From: Ian Lance Taylor @ 2007-12-07 16:59 UTC (permalink / raw)
  To: Dave Korn; +Cc: gcc

"Dave Korn" <dave.korn@artimi.com> writes:

>   Note the several calls to dup2() and close(), which seem to me to be "calls
> [to] any other function", and the setting of environ, which seem to me to be
> modification of "any data other than a variable of type pid_t used to store
> the return value from vfork()".

Despite the standardese, vfork was invented to support calling
dup/dup2 before calling exec.  Without that feature, it would be
nearly useless.  Any actual implementation of vfork must support
calling dup/dup2, or it will break all real programs which use vfork.

On the other hand, the setting of environ is very dubious and is
likely to break on real systems.  The code should be changed to call
execve instead.  Unfortunately there is no standard execvpe function.
Fortunately gcc never uses the variant which sets environ.  Offhand
I'm not sure what does.

Ian

^ permalink raw reply	[flat|nested] 26+ messages in thread

* libiberty/pex-unix vfork abuse?
@ 2007-12-07 15:08 Dave Korn
  2007-12-07 16:59 ` Ian Lance Taylor
  0 siblings, 1 reply; 26+ messages in thread
From: Dave Korn @ 2007-12-07 15:08 UTC (permalink / raw)
  To: gcc


    Hey all,

  This is what posix says about vfork:

http://www.opengroup.org/onlinepubs/000095399/functions/vfork.html
"The vfork() function shall be equivalent to fork(), except that the behavior
is undefined if the process created by vfork() either modifies any data other
than a variable of type pid_t used to store the return value from vfork(), or
returns from the function in which vfork() was called, or calls any other
function before successfully calling _exit() or one of the exec family of
functions."

  This is how pex-unix.c uses vfork:

static long
pex_unix_exec_child (struct pex_obj *obj, int flags, const char *executable,
		     char * const * argv, char * const * env,
                     int in, int out, int errdes,
		     int toclose, const char **errmsg, int *err)
{
  pid_t pid;

  /* We declare these to be volatile to avoid warnings from gcc about
     them being clobbered by vfork.  */
  volatile int sleep_interval;
  volatile int retries;

  sleep_interval = 1;
  pid = -1;
  for (retries = 0; retries < 4; ++retries)
    {
      pid = vfork ();
      if (pid >= 0)
	break;
      sleep (sleep_interval);
      sleep_interval *= 2;
    }

  switch (pid)
    {
    case -1:
      *err = errno;
      *errmsg = VFORK_STRING;
      return -1;

    case 0:
      /* Child process.  */
      if (in != STDIN_FILE_NO)
	{
	  if (dup2 (in, STDIN_FILE_NO) < 0)
	    pex_child_error (obj, executable, "dup2", errno);
	  if (close (in) < 0)
	    pex_child_error (obj, executable, "close", errno);
	}
      if (out != STDOUT_FILE_NO)
	{
	  if (dup2 (out, STDOUT_FILE_NO) < 0)
	    pex_child_error (obj, executable, "dup2", errno);
	  if (close (out) < 0)
	    pex_child_error (obj, executable, "close", errno);
	}
      if (errdes != STDERR_FILE_NO)
	{
	  if (dup2 (errdes, STDERR_FILE_NO) < 0)
	    pex_child_error (obj, executable, "dup2", errno);
	  if (close (errdes) < 0)
	    pex_child_error (obj, executable, "close", errno);
	}
      if (toclose >= 0)
	{
	  if (close (toclose) < 0)
	    pex_child_error (obj, executable, "close", errno);
	}
      if ((flags & PEX_STDERR_TO_STDOUT) != 0)
	{
	  if (dup2 (STDOUT_FILE_NO, STDERR_FILE_NO) < 0)
	    pex_child_error (obj, executable, "dup2", errno);
	}

      if (env)
        environ = (char**) env;

      if ((flags & PEX_SEARCH) != 0)
	{
	  execvp (executable, argv);
	  pex_child_error (obj, executable, "execvp", errno);
	}
      else
	{
	  execv (executable, argv);
	  pex_child_error (obj, executable, "execv", errno);
	}



  Note the several calls to dup2() and close(), which seem to me to be "calls
[to] any other function", and the setting of environ, which seem to me to be
modification of "any data other than a variable of type pid_t used to store
the return value from vfork()".  The comment on pex_child_error (which uses
write() to do output) gives a hint at the thinking here:


/* Report an error from a child process.  We don't use stdio routines,
   because we might be here due to a vfork call.  */

static void
pex_child_error (struct pex_obj *obj, const char *executable,
		 const char *errmsg, int err)
{
#define writeerr(s) (void) write (STDERR_FILE_NO, s, strlen (s))
  writeerr (obj->pname);


  But I don't see any reason to assume the restriction only applies to f*()
stdio functions, in fact by my reading I don't think you're [technically] even
allowed to call a pure const inline function that's part of your own code.  (I
assume that that would in fact work ok in practice at least most of the time).

  Are we ok here?  This code seems like it's doing the wrong thing to me.  As
far as I can tell, we only get away with this in cygwin because of paranoid
defensive programming that backs up the fd table before running the vfork'd
child's code in the parent's context up to the first exec*() call, and then
restores it afterward, but I'm fairly sure that this implementation will still
overwrite the parent's environment.... which could well be Not A Good Thing!


    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....

^ permalink raw reply	[flat|nested] 26+ messages in thread

end of thread, other threads:[~2007-12-10 23:28 UTC | newest]

Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2007-12-07 17:24 libiberty/pex-unix vfork abuse? J.C. Pizarro
2007-12-07 17:42 ` Dave Korn
2007-12-07 17:50   ` Joe Buck
2007-12-07 18:09     ` J.C. Pizarro
2007-12-07 18:14       ` Andrew Haley
2007-12-07 18:28       ` Dave Korn
2007-12-07 18:47         ` J.C. Pizarro
2007-12-07 18:31       ` Diego Novillo
2007-12-07 22:14       ` Gabriel Dos Reis
  -- strict thread matches above, loose matches on Subject: below --
2007-12-07 19:38 Ross Ridge
2007-12-07 15:08 Dave Korn
2007-12-07 16:59 ` Ian Lance Taylor
2007-12-07 17:09   ` Dave Korn
2007-12-07 18:40     ` Ian Lance Taylor
2007-12-07 20:55     ` Andreas Schwab
2007-12-10 19:19       ` Dave Korn
2007-12-10 19:22         ` Joe Buck
2007-12-10 19:37           ` Daniel Jacobowitz
2007-12-10 20:24             ` Joe Buck
2007-12-10 22:44             ` Andreas Schwab
2007-12-10 23:10               ` Daniel Jacobowitz
2007-12-10 21:05           ` Ian Lance Taylor
2007-12-10 22:56           ` Andreas Schwab
2007-12-10 20:01         ` Andreas Schwab
2007-12-10 20:06           ` Brian Dessent
2007-12-11  3:31             ` Dave Korn

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).