* fix off-by-one in dup2
@ 2013-09-25 23:26 Eric Blake
2013-10-15 14:06 ` Christopher Faylor
2013-12-04 9:32 ` Corinna Vinschen
0 siblings, 2 replies; 15+ messages in thread
From: Eric Blake @ 2013-09-25 23:26 UTC (permalink / raw)
To: cygwin-patches
[-- Attachment #1: Type: text/plain, Size: 3083 bytes --]
Solves the segfault here: http://cygwin.com/ml/cygwin/2013-09/msg00397.html
but does not address the fact that we are still screwy with regards to
rlimit.
======
Ultimately, based on my understanding of POSIX and glibc, my goal is to
have a number of changes (this patch only scratches the surface; there's
more to go):
dtable.h tracks soft and hard limits, inherited over fork and preserved
across exec
hard limit starts at OPEN_MAX_MAX and can only be reduced
soft limit starts at hard limit, and can be reduced to _POSIX_OPEN_MAX (8)
dtable.size starts at MAX(32, fork/exec size)
getdtablesize() and sysconf(_SC_OPEN_MAX) always returns the soft limit,
as in glibc and permitted by POSIX (_SC_OPEN_MAX is the only sysconf
variable that can be runtime dynamic)
dtable.size is decoupled from soft limit, and is guaranteed to be <=
hard limit. It can grow up to current soft limit; but soft limit can
later be reduced lower than dtable.size (glibc does this); on fork and
exec, we are careful to still allow fds beyond the current soft limit.
getrlimit(RLIMIT_NOFILE, &r) => returns soft and hard limits from dtable
rather than hard limit as a constant and soft limit as current dtable.size
setrlimit(RLIMIT_NOFILE, &r) => cannot set hard limit to unlimited; soft
limit of unlimited is translated to current hard limit; hard limit
cannot be increased (EPERM) or reduced below dtable.size (EINVAL); soft
limit can be reduced arbitrarily (including below OPEN_MAX of 256)
setdtablesize() => guarantees that dtable.size is at least that large
(must be <= soft limit), but does not lower dtable.size or change limits
=====
2013-09-25 Eric Blake <eblake@redhat.com>
dup2: fix off-by-one crash
* dtable.cc (dup3): Fix off-by-one.
(find_unused_handle): Reduce time spent expanding during dup.
* syscalls.cc (setdtablesize): Report error on invalid value.
diff --git i/winsup/cygwin/dtable.cc w/winsup/cygwin/dtable.cc
index 2501a26..c2982a8 100644
--- i/winsup/cygwin/dtable.cc
+++ w/winsup/cygwin/dtable.cc
@@ -233,7 +233,7 @@ dtable::find_unused_handle (int start)
if (fds[i] == NULL)
return i;
}
- while (extend (NOFILE_INCR));
+ while (extend (MAX (NOFILE_INCR, start - size)));
return -1;
}
@@ -754,7 +754,7 @@ dtable::dup3 (int oldfd, int newfd, int flags)
if (!not_open (newfd))
close (newfd);
- else if ((size_t) newfd > size
+ else if ((size_t) newfd >= size
&& find_unused_handle (newfd) < 0)
/* couldn't extend fdtab */
{
diff --git i/winsup/cygwin/syscalls.cc w/winsup/cygwin/syscalls.cc
index e1886e6..8c1c70a 100644
--- i/winsup/cygwin/syscalls.cc
+++ w/winsup/cygwin/syscalls.cc
@@ -2578,6 +2578,9 @@ system (const char *cmdstring)
extern "C" int
setdtablesize (int size)
{
+ if (size < 0)
+ return -1;
+
if (size <= (int)cygheap->fdtab.size || cygheap->fdtab.extend (size -
cygheap->fdtab.size))
return 0;
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 621 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: fix off-by-one in dup2
2013-09-25 23:26 fix off-by-one in dup2 Eric Blake
@ 2013-10-15 14:06 ` Christopher Faylor
2013-10-15 20:45 ` Yaakov (Cygwin/X)
2013-11-23 13:19 ` Eric Blake
2013-12-04 9:32 ` Corinna Vinschen
1 sibling, 2 replies; 15+ messages in thread
From: Christopher Faylor @ 2013-10-15 14:06 UTC (permalink / raw)
To: cygwin-patches
On Wed, Sep 25, 2013 at 05:26:25PM -0600, Eric Blake wrote:
>Solves the segfault here: http://cygwin.com/ml/cygwin/2013-09/msg00397.html
>but does not address the fact that we are still screwy with regards to
>rlimit.
Corinna reminded me about this.
Sorry for the delay in responding. I was investigating if setdtablesize
should set an errno on error but it is difficult to say if it should
since it seems not to be a POSIX or Linux. So, I guess we can just say
that it should set EINVAL. Would you mind making that minor change and
checking this in?
cgf
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: fix off-by-one in dup2
2013-10-15 14:06 ` Christopher Faylor
@ 2013-10-15 20:45 ` Yaakov (Cygwin/X)
2013-10-15 22:34 ` Christopher Faylor
2013-11-23 13:19 ` Eric Blake
1 sibling, 1 reply; 15+ messages in thread
From: Yaakov (Cygwin/X) @ 2013-10-15 20:45 UTC (permalink / raw)
To: cygwin-patches
On 2013-10-15 09:06, Christopher Faylor wrote:
> On Wed, Sep 25, 2013 at 05:26:25PM -0600, Eric Blake wrote:
>> Solves the segfault here: http://cygwin.com/ml/cygwin/2013-09/msg00397.html
>> but does not address the fact that we are still screwy with regards to
>> rlimit.
>
> Sorry for the delay in responding. I was investigating if setdtablesize
> should set an errno on error but it is difficult to say if it should
> since it seems not to be a POSIX or Linux.
Did you see <http://man7.org/linux/man-pages/man2/getdtablesize.2.html>?
Yaakov
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: fix off-by-one in dup2
2013-10-15 20:45 ` Yaakov (Cygwin/X)
@ 2013-10-15 22:34 ` Christopher Faylor
2013-10-16 7:40 ` Yaakov (Cygwin/X)
0 siblings, 1 reply; 15+ messages in thread
From: Christopher Faylor @ 2013-10-15 22:34 UTC (permalink / raw)
To: cygwin-patches
On Tue, Oct 15, 2013 at 03:45:08PM -0500, Yaakov (Cygwin/X) wrote:
>On 2013-10-15 09:06, Christopher Faylor wrote:
>> On Wed, Sep 25, 2013 at 05:26:25PM -0600, Eric Blake wrote:
>>> Solves the segfault here: http://cygwin.com/ml/cygwin/2013-09/msg00397.html
>>> but does not address the fact that we are still screwy with regards to
>>> rlimit.
>>
>> Sorry for the delay in responding. I was investigating if setdtablesize
>> should set an errno on error but it is difficult to say if it should
>> since it seems not to be a POSIX or Linux.
>
>Did you see <http://man7.org/linux/man-pages/man2/getdtablesize.2.html>?
How does that help with setdtablesize?
cgf
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: fix off-by-one in dup2
2013-10-15 22:34 ` Christopher Faylor
@ 2013-10-16 7:40 ` Yaakov (Cygwin/X)
0 siblings, 0 replies; 15+ messages in thread
From: Yaakov (Cygwin/X) @ 2013-10-16 7:40 UTC (permalink / raw)
To: cygwin-patches
On 2013-10-15 17:34, Christopher Faylor wrote:
> On Tue, Oct 15, 2013 at 03:45:08PM -0500, Yaakov (Cygwin/X) wrote:
>> On 2013-10-15 09:06, Christopher Faylor wrote:
>>> Sorry for the delay in responding. I was investigating if setdtablesize
>>> should set an errno on error but it is difficult to say if it should
>>> since it seems not to be a POSIX or Linux.
>>
>> Did you see <http://man7.org/linux/man-pages/man2/getdtablesize.2.html>?
>
> How does that help with setdtablesize?
Never mind, it seems I misread your message.
Yaakov
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: fix off-by-one in dup2
2013-10-15 14:06 ` Christopher Faylor
2013-10-15 20:45 ` Yaakov (Cygwin/X)
@ 2013-11-23 13:19 ` Eric Blake
1 sibling, 0 replies; 15+ messages in thread
From: Eric Blake @ 2013-11-23 13:19 UTC (permalink / raw)
To: cygwin-patches
[-- Attachment #1: Type: text/plain, Size: 884 bytes --]
On 10/15/2013 08:06 AM, Christopher Faylor wrote:
> On Wed, Sep 25, 2013 at 05:26:25PM -0600, Eric Blake wrote:
>> Solves the segfault here: http://cygwin.com/ml/cygwin/2013-09/msg00397.html
>> but does not address the fact that we are still screwy with regards to
>> rlimit.
>
> Corinna reminded me about this.
>
> Sorry for the delay in responding. I was investigating if setdtablesize
> should set an errno on error but it is difficult to say if it should
> since it seems not to be a POSIX or Linux. So, I guess we can just say
> that it should set EINVAL. Would you mind making that minor change and
> checking this in?
Yikes, I still haven't done this (and was reminded by today's
announcement to test snapshots). I'll try to get to it pronto.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 621 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: fix off-by-one in dup2
2013-09-25 23:26 fix off-by-one in dup2 Eric Blake
2013-10-15 14:06 ` Christopher Faylor
@ 2013-12-04 9:32 ` Corinna Vinschen
2013-12-04 11:36 ` Corinna Vinschen
1 sibling, 1 reply; 15+ messages in thread
From: Corinna Vinschen @ 2013-12-04 9:32 UTC (permalink / raw)
To: cygwin-patches
[-- Attachment #1: Type: text/plain, Size: 1942 bytes --]
Hi guys,
I'm not quite sure yet *why* this happens, but this change in
dtable::find_unused_handle...
On Sep 25 17:26, Eric Blake wrote:
> [...]
> diff --git i/winsup/cygwin/dtable.cc w/winsup/cygwin/dtable.cc
> index 2501a26..c2982a8 100644
> --- i/winsup/cygwin/dtable.cc
> +++ w/winsup/cygwin/dtable.cc
> @@ -233,7 +233,7 @@ dtable::find_unused_handle (int start)
> if (fds[i] == NULL)
> return i;
> }
> - while (extend (NOFILE_INCR));
> + while (extend (MAX (NOFILE_INCR, start - size)));
> return -1;
> }
...introduced the problem reported in
http://cygwin.com/ml/cygwin/2013-12/msg00072.html
The problem is still present in the current sources.
If I apply this change...
Index: dtable.cc
===================================================================
RCS file: /cvs/src/src/winsup/cygwin/dtable.cc,v
retrieving revision 1.275
diff -u -p -r1.275 dtable.cc
--- dtable.cc 1 Dec 2013 19:17:56 -0000 1.275
+++ dtable.cc 4 Dec 2013 09:26:01 -0000
@@ -223,7 +223,8 @@ dtable::delete_archetype (fhandler_base
int
dtable::find_unused_handle (size_t start)
{
- size_t extendby = (start >= size) ? 1 + start - size : NOFILE_INCR;
+ //size_t extendby = (start >= size) ? 1 + start - size : NOFILE_INCR;
+ size_t extendby = NOFILE_INCR;
/* This do loop should only ever execute twice. */
int res = -1;
..., which essentially reverts the original change from Eric, the
problem is fixed.
Off the top of my head I don't understand why Eric's as well as cgf's
solution (which are not equivalent) both introduce this problem, but
always using NOFILE_INCR works, so I publish it here for discussion.
I'm off for a doc appointment now, maybe I have some clue while sitting
in the anteroom.
Corinna
--
Corinna Vinschen Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat
[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: fix off-by-one in dup2
2013-12-04 9:32 ` Corinna Vinschen
@ 2013-12-04 11:36 ` Corinna Vinschen
2013-12-04 12:04 ` Corinna Vinschen
0 siblings, 1 reply; 15+ messages in thread
From: Corinna Vinschen @ 2013-12-04 11:36 UTC (permalink / raw)
To: cygwin-patches
[-- Attachment #1: Type: text/plain, Size: 2258 bytes --]
On Dec 4 10:32, Corinna Vinschen wrote:
> Hi guys,
>
>
> I'm not quite sure yet *why* this happens, but this change in
> dtable::find_unused_handle...
>
> On Sep 25 17:26, Eric Blake wrote:
> > [...]
> > diff --git i/winsup/cygwin/dtable.cc w/winsup/cygwin/dtable.cc
> > index 2501a26..c2982a8 100644
> > --- i/winsup/cygwin/dtable.cc
> > +++ w/winsup/cygwin/dtable.cc
> > @@ -233,7 +233,7 @@ dtable::find_unused_handle (int start)
> > if (fds[i] == NULL)
> > return i;
> > }
> > - while (extend (NOFILE_INCR));
> > + while (extend (MAX (NOFILE_INCR, start - size)));
> > return -1;
> > }
>
> ...introduced the problem reported in
> http://cygwin.com/ml/cygwin/2013-12/msg00072.html
>
> The problem is still present in the current sources.
>
> If I apply this change...
>
> Index: dtable.cc
> ===================================================================
> RCS file: /cvs/src/src/winsup/cygwin/dtable.cc,v
> retrieving revision 1.275
> diff -u -p -r1.275 dtable.cc
> --- dtable.cc 1 Dec 2013 19:17:56 -0000 1.275
> +++ dtable.cc 4 Dec 2013 09:26:01 -0000
> @@ -223,7 +223,8 @@ dtable::delete_archetype (fhandler_base
> int
> dtable::find_unused_handle (size_t start)
> {
> - size_t extendby = (start >= size) ? 1 + start - size : NOFILE_INCR;
> + //size_t extendby = (start >= size) ? 1 + start - size : NOFILE_INCR;
> + size_t extendby = NOFILE_INCR;
>
> /* This do loop should only ever execute twice. */
> int res = -1;
>
>
> ..., which essentially reverts the original change from Eric, the
> problem is fixed.
>
> Off the top of my head I don't understand why Eric's as well as cgf's
> solution (which are not equivalent) both introduce this problem, but
> always using NOFILE_INCR works, so I publish it here for discussion.
>
> I'm off for a doc appointment now, maybe I have some clue while sitting
> in the anteroom.
Not really. Btw., this helps to fix the problem as well:
size_t extendby = (start >= size) ? MAX (1 + start - size, NOFILE_INCR)
: NOFILE_INCR;
Corinna
--
Corinna Vinschen Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat
[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: fix off-by-one in dup2
2013-12-04 11:36 ` Corinna Vinschen
@ 2013-12-04 12:04 ` Corinna Vinschen
2013-12-04 17:00 ` Christopher Faylor
0 siblings, 1 reply; 15+ messages in thread
From: Corinna Vinschen @ 2013-12-04 12:04 UTC (permalink / raw)
To: cygwin-patches
[-- Attachment #1: Type: text/plain, Size: 981 bytes --]
On Dec 4 12:36, Corinna Vinschen wrote:
> On Dec 4 10:32, Corinna Vinschen wrote:
> > Hi guys,
> > [...etc...]
> > The problem is still present in the current sources.
> > [...]
Ouch, ouch, ouch! I tested the wrong DLL. Actually current CVS fixes
this problem. Duh. Sorry for the confusion.
One question, though. Assuming start is == size, then the current code
in CVS extends the fd table by only 1. If that happens often, the
current code would have to call ccalloc/memcpy/cfree a lot. Wouldn't
it in fact be better to extend always by at least NOFILE_INCR, and to
extend by (1 + start - size) only if start is > size + NOFILE_INCR?
Something like
size_t extendby = (start >= size + NOFILE_INCR) ? 1 + start - size : NOFILE_INCR;
?
Sorry again. Fortunately it's my WJM week...
Corinna
--
Corinna Vinschen Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat
[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: fix off-by-one in dup2
2013-12-04 12:04 ` Corinna Vinschen
@ 2013-12-04 17:00 ` Christopher Faylor
2013-12-04 17:23 ` Corinna Vinschen
0 siblings, 1 reply; 15+ messages in thread
From: Christopher Faylor @ 2013-12-04 17:00 UTC (permalink / raw)
To: cygwin-patches
On Wed, Dec 04, 2013 at 01:04:08PM +0100, Corinna Vinschen wrote:
>On Dec 4 12:36, Corinna Vinschen wrote:
>> On Dec 4 10:32, Corinna Vinschen wrote:
>> > Hi guys,
>> > [...etc...]
>> > The problem is still present in the current sources.
>> > [...]
>
>Ouch, ouch, ouch! I tested the wrong DLL. Actually current CVS fixes
>this problem. Duh. Sorry for the confusion.
>
>One question, though. Assuming start is == size, then the current code
>in CVS extends the fd table by only 1. If that happens often, the
>current code would have to call ccalloc/memcpy/cfree a lot. Wouldn't
>it in fact be better to extend always by at least NOFILE_INCR, and to
>extend by (1 + start - size) only if start is > size + NOFILE_INCR?
>Something like
>
> size_t extendby = (start >= size + NOFILE_INCR) ? 1 + start - size : NOFILE_INCR;
>
>?
>
>Sorry again. Fortunately it's my WJM week...
I don't think it is a common occurrence for start >= size. It is
usually done when something like bash dup2's stdin/stdout/stderr to a
high fd. Howeer, I'll check in something which guarantees that there is
always a NOFILE_INCR entries free after start.
cgf
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: fix off-by-one in dup2
2013-12-04 17:00 ` Christopher Faylor
@ 2013-12-04 17:23 ` Corinna Vinschen
2013-12-04 17:51 ` Christopher Faylor
0 siblings, 1 reply; 15+ messages in thread
From: Corinna Vinschen @ 2013-12-04 17:23 UTC (permalink / raw)
To: cygwin-patches
[-- Attachment #1: Type: text/plain, Size: 1605 bytes --]
On Dec 4 12:00, Christopher Faylor wrote:
> On Wed, Dec 04, 2013 at 01:04:08PM +0100, Corinna Vinschen wrote:
> >On Dec 4 12:36, Corinna Vinschen wrote:
> >> On Dec 4 10:32, Corinna Vinschen wrote:
> >> > Hi guys,
> >> > [...etc...]
> >> > The problem is still present in the current sources.
> >> > [...]
> >
> >Ouch, ouch, ouch! I tested the wrong DLL. Actually current CVS fixes
> >this problem. Duh. Sorry for the confusion.
> >
> >One question, though. Assuming start is == size, then the current code
> >in CVS extends the fd table by only 1. If that happens often, the
> >current code would have to call ccalloc/memcpy/cfree a lot. Wouldn't
> >it in fact be better to extend always by at least NOFILE_INCR, and to
> >extend by (1 + start - size) only if start is > size + NOFILE_INCR?
> >Something like
> >
> > size_t extendby = (start >= size + NOFILE_INCR) ? 1 + start - size : NOFILE_INCR;
> >
> >?
> >
> >Sorry again. Fortunately it's my WJM week...
>
> I don't think it is a common occurrence for start >= size. It is
> usually done when something like bash dup2's stdin/stdout/stderr to a
> high fd. Howeer, I'll check in something which guarantees that there is
> always a NOFILE_INCR entries free after start.
That might be helpful. Tcsh, for instance, always dup's it's std
descriptors to the new fds 15-19. If it does so in this order, it would
have to call extend 5 times.
Corinna
--
Corinna Vinschen Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat
[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: fix off-by-one in dup2
2013-12-04 17:23 ` Corinna Vinschen
@ 2013-12-04 17:51 ` Christopher Faylor
2013-12-04 19:44 ` Corinna Vinschen
2013-12-05 13:45 ` Eric Blake
0 siblings, 2 replies; 15+ messages in thread
From: Christopher Faylor @ 2013-12-04 17:51 UTC (permalink / raw)
To: cygwin-patches
On Wed, Dec 04, 2013 at 06:23:24PM +0100, Corinna Vinschen wrote:
>On Dec 4 12:00, Christopher Faylor wrote:
>> On Wed, Dec 04, 2013 at 01:04:08PM +0100, Corinna Vinschen wrote:
>> >On Dec 4 12:36, Corinna Vinschen wrote:
>> >> On Dec 4 10:32, Corinna Vinschen wrote:
>> >> > Hi guys,
>> >> > [...etc...]
>> >> > The problem is still present in the current sources.
>> >> > [...]
>> >
>> >Ouch, ouch, ouch! I tested the wrong DLL. Actually current CVS fixes
>> >this problem. Duh. Sorry for the confusion.
>> >
>> >One question, though. Assuming start is == size, then the current code
>> >in CVS extends the fd table by only 1. If that happens often, the
>> >current code would have to call ccalloc/memcpy/cfree a lot. Wouldn't
>> >it in fact be better to extend always by at least NOFILE_INCR, and to
>> >extend by (1 + start - size) only if start is > size + NOFILE_INCR?
>> >Something like
>> >
>> > size_t extendby = (start >= size + NOFILE_INCR) ? 1 + start - size : NOFILE_INCR;
>> >
>> >?
>> >
>> >Sorry again. Fortunately it's my WJM week...
>>
>> I don't think it is a common occurrence for start >= size. It is
>> usually done when something like bash dup2's stdin/stdout/stderr to a
>> high fd. Howeer, I'll check in something which guarantees that there is
>> always a NOFILE_INCR entries free after start.
>
>That might be helpful. Tcsh, for instance, always dup's it's std
>descriptors to the new fds 15-19. If it does so in this order, it would
>have to call extend 5 times.
dtable.h:#define NOFILE_INCR 32
It shouldn't extend in that scenario. The table starts with 32
elements.
cgf
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: fix off-by-one in dup2
2013-12-04 17:51 ` Christopher Faylor
@ 2013-12-04 19:44 ` Corinna Vinschen
2013-12-05 13:45 ` Eric Blake
1 sibling, 0 replies; 15+ messages in thread
From: Corinna Vinschen @ 2013-12-04 19:44 UTC (permalink / raw)
To: cygwin-patches
[-- Attachment #1: Type: text/plain, Size: 1990 bytes --]
On Dec 4 12:51, Christopher Faylor wrote:
> On Wed, Dec 04, 2013 at 06:23:24PM +0100, Corinna Vinschen wrote:
> >On Dec 4 12:00, Christopher Faylor wrote:
> >> On Wed, Dec 04, 2013 at 01:04:08PM +0100, Corinna Vinschen wrote:
> >> >On Dec 4 12:36, Corinna Vinschen wrote:
> >> >> On Dec 4 10:32, Corinna Vinschen wrote:
> >> >> > Hi guys,
> >> >> > [...etc...]
> >> >> > The problem is still present in the current sources.
> >> >> > [...]
> >> >
> >> >Ouch, ouch, ouch! I tested the wrong DLL. Actually current CVS fixes
> >> >this problem. Duh. Sorry for the confusion.
> >> >
> >> >One question, though. Assuming start is == size, then the current code
> >> >in CVS extends the fd table by only 1. If that happens often, the
> >> >current code would have to call ccalloc/memcpy/cfree a lot. Wouldn't
> >> >it in fact be better to extend always by at least NOFILE_INCR, and to
> >> >extend by (1 + start - size) only if start is > size + NOFILE_INCR?
> >> >Something like
> >> >
> >> > size_t extendby = (start >= size + NOFILE_INCR) ? 1 + start - size : NOFILE_INCR;
> >> >
> >> >?
> >> >
> >> >Sorry again. Fortunately it's my WJM week...
> >>
> >> I don't think it is a common occurrence for start >= size. It is
> >> usually done when something like bash dup2's stdin/stdout/stderr to a
> >> high fd. Howeer, I'll check in something which guarantees that there is
> >> always a NOFILE_INCR entries free after start.
> >
> >That might be helpful. Tcsh, for instance, always dup's it's std
> >descriptors to the new fds 15-19. If it does so in this order, it would
> >have to call extend 5 times.
>
> dtable.h:#define NOFILE_INCR 32
>
> It shouldn't extend in that scenario. The table starts with 32
> elements.
Right. I just thought it's a good example.
Corinna
--
Corinna Vinschen Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat
[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: fix off-by-one in dup2
2013-12-04 17:51 ` Christopher Faylor
2013-12-04 19:44 ` Corinna Vinschen
@ 2013-12-05 13:45 ` Eric Blake
2013-12-05 19:56 ` Christopher Faylor
1 sibling, 1 reply; 15+ messages in thread
From: Eric Blake @ 2013-12-05 13:45 UTC (permalink / raw)
To: cygwin-patches
[-- Attachment #1: Type: text/plain, Size: 2102 bytes --]
On 12/04/2013 10:51 AM, Christopher Faylor wrote:
>>>> One question, though. Assuming start is == size, then the current code
>>>> in CVS extends the fd table by only 1. If that happens often, the
>>>> current code would have to call ccalloc/memcpy/cfree a lot. Wouldn't
>>>> it in fact be better to extend always by at least NOFILE_INCR, and to
>>>> extend by (1 + start - size) only if start is > size + NOFILE_INCR?
>>>> Something like
>>>>
>>>> size_t extendby = (start >= size + NOFILE_INCR) ? 1 + start - size : NOFILE_INCR;
>>>>
Always increasing by a minimum of NOFILE_INCR is wrong in one case - we
should never increase beyond OPEN_MAX_MAX (currently 3200). dup2(0,
3199) should succeed (unless it fails with EMFILE due to rlimit, but we
already know that our handling of setrlimit(RLIMIT_NOFILE) is still a
bit awkward); but dup2(0, 3200) must always fail with EBADF. I think
the code in CVS is still wrong: we want to increase to the larger of the
value specified by the user or NOFILE_INCR to minimize repeated calloc,
but we also need to cap the increase to be at most OPEN_MAX_MAX
descriptors, to avoid having a table larger than what the rest of our
code base will support.
Not having NOFILE_INCR free slots after a user allocation is not fatal;
it means that the first allocation to a large number will not have tail
padding, but the next allocation to fd+1 will allocate NOFILE_INCR slots
rather than just one. My original idea of MAX(NOFILE_INCR, start -
size) expresses that.
>>
>> That might be helpful. Tcsh, for instance, always dup's it's std
>> descriptors to the new fds 15-19. If it does so in this order, it would
>> have to call extend 5 times.
>
> dtable.h:#define NOFILE_INCR 32
>
> It shouldn't extend in that scenario. The table starts with 32
> elements.
Rather, the table starts with 256 elements; which is why dup2 wouldn't
crash until dup'ing to 256 or greater before I started touching this.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 621 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: fix off-by-one in dup2
2013-12-05 13:45 ` Eric Blake
@ 2013-12-05 19:56 ` Christopher Faylor
0 siblings, 0 replies; 15+ messages in thread
From: Christopher Faylor @ 2013-12-05 19:56 UTC (permalink / raw)
To: cygwin-patches
On Thu, Dec 05, 2013 at 06:45:22AM -0700, Eric Blake wrote:
>On 12/04/2013 10:51 AM, Christopher Faylor wrote:
>
>>>>> One question, though. Assuming start is == size, then the current code
>>>>> in CVS extends the fd table by only 1. If that happens often, the
>>>>> current code would have to call ccalloc/memcpy/cfree a lot. Wouldn't
>>>>> it in fact be better to extend always by at least NOFILE_INCR, and to
>>>>> extend by (1 + start - size) only if start is > size + NOFILE_INCR?
>>>>> Something like
>>>>>
>>>>> size_t extendby = (start >= size + NOFILE_INCR) ? 1 + start - size : NOFILE_INCR;
>>>>>
>
>Always increasing by a minimum of NOFILE_INCR is wrong in one case - we
>should never increase beyond OPEN_MAX_MAX (currently 3200). dup2(0,
>3199) should succeed (unless it fails with EMFILE due to rlimit, but we
>already know that our handling of setrlimit(RLIMIT_NOFILE) is still a
>bit awkward); but dup2(0, 3200) must always fail with EBADF. I think
>the code in CVS is still wrong: we want to increase to the larger of the
>value specified by the user or NOFILE_INCR to minimize repeated calloc,
>but we also need to cap the increase to be at most OPEN_MAX_MAX
>descriptors, to avoid having a table larger than what the rest of our
>code base will support.
I made some more changes to CVS. Incidentally did you catch the fact
that you broke how this worked in 1.7.26? You were taking a MAX of a
signed and unsigned quantity so the signed quantity was promoted to a
huge positive number.
>Not having NOFILE_INCR free slots after a user allocation is not fatal;
No one implied it was.
>it means that the first allocation to a large number will not have tail
>padding, but the next allocation to fd+1 will allocate NOFILE_INCR slots
>rather than just one. My original idea of MAX(NOFILE_INCR, start -
>size) expresses that.
That wasn't Corinna's concern. My replacement code would have called
calloc for every one of:
dup2(0, 32);
dup2(1, 33);
dup2(2, 34);
Obviously there are different ways to avoid this and I chose to extend
the table after the "start" location.
>>> That might be helpful. Tcsh, for instance, always dup's it's std
>>> descriptors to the new fds 15-19. If it does so in this order, it would
>>> have to call extend 5 times.
>>
>> dtable.h:#define NOFILE_INCR 32
>>
>> It shouldn't extend in that scenario. The table starts with 32
>> elements.
>
>Rather, the table starts with 256 elements; which is why dup2 wouldn't
>crash until dup'ing to 256 or greater before I started touching this.
The table is initialized in dtable_init() with 32 elements. When it
enters main, it is still 32 elements, at least according to
cygheap->fdtab.size. I just checked this with gdb.
cgf
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2013-12-05 19:56 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-09-25 23:26 fix off-by-one in dup2 Eric Blake
2013-10-15 14:06 ` Christopher Faylor
2013-10-15 20:45 ` Yaakov (Cygwin/X)
2013-10-15 22:34 ` Christopher Faylor
2013-10-16 7:40 ` Yaakov (Cygwin/X)
2013-11-23 13:19 ` Eric Blake
2013-12-04 9:32 ` Corinna Vinschen
2013-12-04 11:36 ` Corinna Vinschen
2013-12-04 12:04 ` Corinna Vinschen
2013-12-04 17:00 ` Christopher Faylor
2013-12-04 17:23 ` Corinna Vinschen
2013-12-04 17:51 ` Christopher Faylor
2013-12-04 19:44 ` Corinna Vinschen
2013-12-05 13:45 ` Eric Blake
2013-12-05 19:56 ` Christopher Faylor
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).