Copy-on-write fork

Copy-on-write fork

[Chris January]

This is mainly a question aimed at Christopher Faylor, but maybe someone
else knows the answer.
My question is, with regard to Chris's post "Re: copy-on-write (oh well)"
[http://www.cygwin.com/ml/cygwin-developers/2000-07/msg00026.html], does
anyone know why a copy-on-write implementation of fork takes longer than the
current Cygwin version??
BTW, I've not had any problems forking beyond the first level using the
example code from 'Window NT/2000 Native API reference'. What problems did
you encounter Chris? My test case is probably not rigorous enough.

A test program and statistics are shown below which clearly show Cygwin's
fork implementation in the lead.

#include <unistd.h>
#include <stdio.h>
#include <windows.h>

int main(void) {
    int pid, i;
    DWORD dwStartTicks, dwEndTicks;
    dwStartTicks = GetTickCount();
    for (i=0;i<100;i++) {
        pid = fork();
        if (!pid)
            exit(0);
    }
    dwEndTicks = GetTickCount();
    printf("average fork time = %g ms\n", (dwEndTicks -
dwStartTicks)/1000.0);
}

test uses the copy-on-write implementation
test2 uses Cygwin's implementation

$ time ./test
average fork time = 3.345 ms

real    0m3.391s
user    0m0.020s
sys     0m0.030s

$ time ./test2
average fork time = 1.972 ms

real    0m2.043s
user    0m0.060s
sys     0m0.220s

Regards
Chris



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Copy-on-write fork

[Christopher Faylor]

On Sat, Apr 20, 2002 at 03:06:55AM +0100, Chris January wrote:
>This is mainly a question aimed at Christopher Faylor, but maybe someone
>else knows the answer.
>My question is, with regard to Chris's post "Re: copy-on-write (oh well)"
>[http://www.cygwin.com/ml/cygwin-developers/2000-07/msg00026.html], does
>anyone know why a copy-on-write implementation of fork takes longer than the
>current Cygwin version??
>BTW, I've not had any problems forking beyond the first level using the
>example code from 'Window NT/2000 Native API reference'. What problems did
>you encounter Chris? My test case is probably not rigorous enough.

I don't know.  I've remarked on this in the past.  My benchmarks showed
the same thing.  I was excited about doing this when I was first hired
by Cygnus since I wanted to contribute to making cygwin faster.

I implemented a fork using Windows API copy-on-write (for NT) and I
believe I also tried to use the low-level NT technique.  Neither showed
any noticeable performance gain and, of course, both suffered from
being NT-only.

I assume that one possible reason is that the copy-on-write fork may be
somehow bypassing normal in-memory sharing of text segments but I never
knew for sure.

The problems with forking beyond the first level are for the Windows
API method, not with the low-level NT calls.

cgf

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: Copy-on-write fork

[Gary R. Van Sickle]

cgf wrote:

> I assume that one possible reason is that the copy-on-write fork may be
> somehow bypassing normal in-memory sharing of text segments but I never
> knew for sure.
>

Have either of you tried this comparison on XP, to see if it's any different
there?  I'm running XP here, if Chris J. wants to put the c-o-w DLL up somewhere
so I can download it and try it.

--
Gary R. Van Sickle
Brewer.  Patriot.


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Copy-on-write fork

[Sven Köhler]

> A test program and statistics are shown below which clearly show Cygwin's
> fork implementation in the lead.

how much memory did your programs allocate prior to fork()ing?
copy-on-write might only apply to applications with high memory-usage.

another thing i didn't understand was, why you took a the real-time for 
mesuring?

i can do "time mc" and exit mc after 10sec and time will show real-time 
10sec but the _real cpu-time_ is below <1sec.
the CPU-times consumed by the copy-on-write-implemention is _lower_ than 
cygwin-implementation's CPU-time.

so why are you guys worried?




--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: Copy-on-write fork

[Chris January]

> > A test program and statistics are shown below which clearly
> show Cygwin's
> > fork implementation in the lead.
>
> how much memory did your programs allocate prior to fork()ing?
> copy-on-write might only apply to applications with high memory-usage.
>
> another thing i didn't understand was, why you took a the real-time for
> mesuring?
>
> i can do "time mc" and exit mc after 10sec and time will show real-time
> 10sec but the _real cpu-time_ is below <1sec.
> the CPU-times consumed by the copy-on-write-implemention is _lower_ than
> cygwin-implementation's CPU-time.
>
> so why are you guys worried?

Repeat after me: don't open old threads.
However I'll let you off this once, because you are using a newsreader and
I've made the same mistake before.

If you think copy on write is faster, then feel free to do some tests. A
website with some nice pretty graphs, and source code would be great. I can
send you some code for starters, I'm sure Chris Faylor has some around too.
It may well be that both of us were simply not doing the right tests.
My tests were based on timing a single process that allocated a large region
of memory, then forked in a loop. Each forked process touched the memory
allocated earlier by overwriting it with a random value.

Chris


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: Copy-on-write fork

[Gary R. Van Sickle]

> Repeat after me: don't open old threads.
> However I'll let you off this once, because you are using a newsreader and
> I've made the same mistake before.

BTW:  As long as it's open, I did try to compile and link with the libfork.a
you(?) sent me, to try it on XP, and got nowhere.  I couldn't link for reasons I
no longer remember.

--
Gary R. Van Sickle
Brewer.  Patriot.


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Copy-on-write fork

[Sven Köhler]

> Repeat after me: don't open old threads.


oops ...
didn't know that rule
after which time is a thread an old thread?

> However I'll let you off this once, because you are using a newsreader and
> I've made the same mistake before.

Thx

> If you think copy on write is faster, then feel free to do some tests. A
> website with some nice pretty graphs, and source code would be great. 
> I can
> send you some code for starters, I'm sure Chris Faylor has some around 
> too.
> It may well be that both of us were simply not doing the right tests.


i'm a developer too, but i've haven't got much time for this, and i'm 
not much into C-programming anymore.
if i had the time, i would be pleased to do it,
but for now, i'm just a cygwin-"user".

> My tests were based on timing a single process that allocated a large 
> region
> of memory, then forked in a loop. Each forked process touched the memory
> allocated earlier by overwriting it with a random value.

OK, now one would need statistics, how much of the fork()ed-memory is 
overwritten usually etc.

i just wondered, how you compared the results of the time-command?
it might be the case, that a program consumes more real-time, but less 
cpu-time. less cpu-time could be preferred, but real-time is more 
important from the user's point-of-view.




--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: Copy-on-write fork

[Robert Collins]

> -----Original Message-----
> From: Chris January [mailto:chris@atomice.net] 
> Sent: Saturday, April 20, 2002 12:07 PM
> To: cygwin@cygwin.com
> Subject: Copy-on-write fork
> 
> 
> This is mainly a question aimed at Christopher Faylor, but 
> maybe someone else knows the answer. My question is, with 
> regard to Chris's post "Re: copy-on-write (oh well)" 
> [http://www.cygwin.com/ml/cygwin-developers/2000-07/msg00026.h
> tml], does anyone know why a copy-on-write implementation of 
> fork takes longer than the current Cygwin version?? 

What would be interesting would be to closely monitor the relevant
perfmon items and see if you can determine the thing causing the delay.
You'd probably need a custom program to get enough detail though :[.

Rob

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Copy-on-write fork

[Chris January]

> > I assume that one possible reason is that the copy-on-write fork may be
> > somehow bypassing normal in-memory sharing of text segments but I never
> > knew for sure.
> >
>
> Have either of you tried this comparison on XP, to see if it's any
different
> there?  I'm running XP here, if Chris J. wants to put the c-o-w DLL up
somewhere
> so I can download it and try it.
Please do - I would be interested to see how it runs on XP - you can
download a static library containing just fork() here:
http://www.doc.ic.ac.uk/~ccj00/atomice.net/show.cgi?article=1019312635446.68
4814058244

Regards
Chris



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/