Re: Typo in <sys/select.h>?

Re: Typo in <sys/select.h>?

[Lavrentiev, Anton (NIH/NLM/NCBI) [C]]

> Remember that 64 is MAXIMUM_WAIT_OBJECTS for WaitForMultipleObjects(),
> the underlying Win32 API used to implement select(), so using more than
> 64 hits some complex code to work around that...

True but the complex code (that involves thread spawning, if that's what you're referring to)
will only be activated if the actual number of objects inquired in those sets really exceed 64,
regardless of how big the sets passed to select() were...

So basically, FD_SETSIZE, as Corinna pointed out, just refers to how much "ballast" the arguments
carry -- with larger default FD_SETSIZE they can become unnecessarily heavy for just a few small
file descriptors.

And since FD_SETSIZE is allowed to be overridden by the user, the complex processing at some
point is unavoidable, since there is no limitation for how large FD_SETSIZE can be, and the sets
are densely populated.

Anton Lavrentiev
Contractor NIH/NLM/NCBI

Re: Typo in <sys/select.h>?

[Lavrentiev, Anton (NIH/NLM/NCBI) [C]]

> However, discussing this shows how irrelevant the actual default value
> of FD_SETSIZE is.

Correct, yet the comments in the header files (along with used values) should be consistent :-)

> [...] to define FD_SETSIZE according to its requirements anyway.

That'd work for CYGWIN right away;  but won't for glibc (because FD_SETSIZE
is not used in the actual type definition, but __FD_SETSIZE), so there comes
the array trick shown earlier.

Anton Lavrentiev
Contractor NIH/NLM/NCBI

Re: Typo in <sys/select.h>?

[Ken Brown]

On 7/6/2022 11:57 AM, Lavrentiev, Anton (NIH/NLM/NCBI) [C] via Cygwin wrote:
>> However, discussing this shows how irrelevant the actual default value
>> of FD_SETSIZE is.
> 
> Correct, yet the comments in the header files (along with used values) should be consistent :-)

I'll make the changes that Corinna suggested.  They'll take effect starting with 
Cygwin 3.4.0.

Ken

Re: Typo in <sys/select.h>?

[Lavrentiev, Anton (NIH/NLM/NCBI) [C]]

> DESCRIPTION
>        WARNING: select() can monitor only file descriptors  numbers  that  are
>        less  than  FD_SETSIZE (1024)-an unreasonably low limit for many modern

Whoever wrote this, was wrong (they might have never consulted the actual kernel code, or were just
blindsided by FD_SETSIZE being a predefined constant).  You can take a look at the kernel source code,
if you don't believe me.

This select() trick has been like that from the earliest days of Linux, and the behavior is carefully carried over.

https://github.com/torvalds/linux/blob/master/fs/select.c#L625

It's just an FYI :-)

Anton Lavrentiev
Contractor NIH/NLM/NCBI

Re: Typo in <sys/select.h>?

[Corinna Vinschen]

On Jul  6 14:26, Lavrentiev, Anton (NIH/NLM/NCBI) [C] via Cygwin wrote:
> > DESCRIPTION
> >        WARNING: select() can monitor only file descriptors  numbers  that  are
> >        less  than  FD_SETSIZE (1024)-an unreasonably low limit for many modern
> 
> Whoever wrote this, was wrong (they might have never consulted the actual kernel code, or were just
> blindsided by FD_SETSIZE being a predefined constant).  You can take a look at the kernel source code,
> if you don't believe me.
> 
> This select() trick has been like that from the earliest days of Linux, and the behavior is carefully carried over.
> 
> https://github.com/torvalds/linux/blob/master/fs/select.c#L625
> 
> It's just an FYI :-)

:+1:

You're right as far as the kernel is concerned.  The problem is apparently
in glibc.  From the same man page:

  POSIX allows an implementation to define an upper limit, advertised via
  the constant FD_SETSIZE, on the range of file descriptors that  can  be
  specified  in a file descriptor set.  The Linux kernel imposes no fixed
  limit, but the glibc implementation makes  fd_set  a  fixed-size  type,
  with  FD_SETSIZE  defined  as 1024, and the FD_*() macros operating ac‐
  cording to that limit.  To monitor file descriptors greater than  1023,
  use poll(2) or epoll(7) instead.

And that's still the case with glibc from current git master:

bits/typesizes.h:

  /* Number of descriptors that can fit in an `fd_set'.  */
  #define __FD_SETSIZE            1024

That's defined unconditionally, just as this stuff from sys/select.h:

misc/sys/select.h:

  /* fd_set for select and pselect.  */
  typedef struct
    { 
      /* XPG4.2 requires this member name.  Otherwise avoid the name
	 from the global namespace.  */
  #ifdef __USE_XOPEN
      __fd_mask fds_bits[__FD_SETSIZE / __NFDBITS];
  # define __FDS_BITS(set) ((set)->fds_bits)
  #else
      __fd_mask __fds_bits[__FD_SETSIZE / __NFDBITS];
  # define __FDS_BITS(set) ((set)->__fds_bits)
  #endif
    } fd_set;

  /* Maximum number of file descriptors in `fd_set'.  */
  #define FD_SETSIZE              __FD_SETSIZE

However, discussing this shows how irrelevant the actual default value
of FD_SETSIZE is.  Per POSIX, it's just that, a default value.  In
interested process which thinks it needs higher fd numbers should make
sure to define FD_SETSIZE according to its requirements anyway.


Corinna

Re: Typo in <sys/select.h>?

[Lavrentiev, Anton (NIH/NLM/NCBI) [C]]

> On Linux, select(2) is really only capable to
> handle file descriptors numbers up to descriptor number 1023,

That is not true.  While FD_SETSIZE is defined as a fixed constant,
Linux kernel does not actually "know" (or care) about it.

So you can have an array of fd_sets, like this, in your code:

fd_set r_fds[NFDS];

and then if "fd" is a file descriptor in question, you'd do

FD_SET(fd % FD_SETSIZE, &r_fds[fd / FD_SETSIZE]);

and then

n = select(maxfd + 1, r_fds, ...);

The Linux kernel is guided by the maxfd parameter and assumes the sets are as
large as required to cover that number of file descriptors (obviously checking
that those sets are still within the process reach).

NFDS above is chosen such a way that "NFDS * FD_SETSIZE" covers all your required
file descriptors, if there are more than just FD_SETSIZE.

Anton Lavrentiev
Contractor NIH/NLM/NCBI

Re: Typo in <sys/select.h>?

[Corinna Vinschen]

On Jul  6 13:19, Lavrentiev, Anton (NIH/NLM/NCBI) [C] via Cygwin wrote:
> > On Linux, select(2) is really only capable to
> > handle file descriptors numbers up to descriptor number 1023,
> 
> That is not true.  While FD_SETSIZE is defined as a fixed constant,
> Linux kernel does not actually "know" (or care) about it.

$ rpm -q man-pages
man-pages-5.12-2.fc35.noarch
$ man 2 select
[...]
DESCRIPTION
       WARNING: select() can monitor only file descriptors  numbers  that  are
       less  than  FD_SETSIZE (1024)—an unreasonably low limit for many modern
       applications—and this limitation will not change.  All modern  applica‐
       tions  should instead use poll(2) or epoll(7), which do not suffer this
       limitation.
[...]


Corinna

RE: Typo in <sys/select.h>?

[Lavrentiev, Anton (NIH/NLM/NCBI) [C]]

> I'm no expert, but it seems the FD_SETSIZE should have been 128.
> Long is 8 bytes, 64 bits. One bit if one open file, 64 * 64 = 4096.

FD_SETSIZE is the file descriptor bitset capacity in _BITS_.

64 (as currently in there) means 1 int (on 64 bit platforms, or 2 ints on 32 bit
platforms, respectively), and is capable of representing 64 file descriptors by
default.

Anton Lavrentiev
Contractor NIH/NLM/NCBI

Typo in <sys/select.h>?

[Lavrentiev, Anton (NIH/NLM/NCBI) [C]]

Hi,

There's some inconsistency between <sys/select.h> and <sys/param.h>:

sys/select.h has this:
-----------------------
/*
 * Select uses bit masks of file descriptors in longs.
 * These macros manipulate such bit fields (the filesystem macros use chars).
 * FD_SETSIZE may be defined by the user, but the default here
 * should be >= NOFILE (param.h).
 */
#ifndef FD_SETSIZE
#define FD_SETSIZE      64
#endif
----------------------

Now, this is the relevant part of sys/param.h looks like this:
----------------------
/* Max number of open files.  The Posix version is OPEN_MAX.  */
/* Number of fds is virtually unlimited in cygwin, but we must provide
   some reasonable value for Posix conformance */
#define NOFILE          8192
----------------------

So it's either "<= NOFILE" that was actually meant to be there in the comment (or,
an equivalent "should NOT be > NOFILE"), or FD_SETSIZE should have been defined as 8192,
if the comment is actually correct.  Or maybe I'm missing something here :-)

I understand that if I redefined FD_SETSIZE in my code before including <sys/select.h>,
it'd work with whatever large (or small) fd_set I need, but that's not what I'm after.

Anton Lavrentiev
Contractor NIH/NLM/NCBI

Re: Typo in <sys/select.h>?

[Andrey Repin]

Greetings, Lavrentiev, Anton (NIH/NLM/NCBI) [C]!

> There's some inconsistency between <sys/select.h> and <sys/param.h>:

> sys/select.h has this:
> -----------------------
> /*
>  * Select uses bit masks of file descriptors in longs.
>  * These macros manipulate such bit fields (the filesystem macros use chars).
>  * FD_SETSIZE may be defined by the user, but the default here
>  * should be >= NOFILE (param.h).
>  */
> #ifndef FD_SETSIZE
> #define FD_SETSIZE      64
> #endif
> ----------------------

> Now, this is the relevant part of sys/param.h looks like this:
> ----------------------
> /* Max number of open files.  The Posix version is OPEN_MAX.  */
> /* Number of fds is virtually unlimited in cygwin, but we must provide
>    some reasonable value for Posix conformance */
> #define NOFILE          8192
> ----------------------

> So it's either "<= NOFILE" that was actually meant to be there in the comment (or,
> an equivalent "should NOT be > NOFILE"), or FD_SETSIZE should have been defined as 8192,
> if the comment is actually correct.  Or maybe I'm missing something here :-)

> I understand that if I redefined FD_SETSIZE in my code before including <sys/select.h>,
> it'd work with whatever large (or small) fd_set I need, but that's not what I'm after.

I'm no expert, but it seems the FD_SETSIZE should have been 128.
Long is 8 bytes, 64 bits. One bit if one open file, 64 * 64 = 4096.


-- 
With best regards,
Andrey Repin
Tuesday, July 5, 2022 17:52:26

Sorry for my terrible english...

Re: Typo in <sys/select.h>?

[Ken Brown]

On 7/5/2022 10:13 AM, Lavrentiev, Anton (NIH/NLM/NCBI) [C] via Cygwin wrote:
> Hi,
> 
> There's some inconsistency between <sys/select.h> and <sys/param.h>:
> 
> sys/select.h has this:
> -----------------------
> /*
>   * Select uses bit masks of file descriptors in longs.
>   * These macros manipulate such bit fields (the filesystem macros use chars).
>   * FD_SETSIZE may be defined by the user, but the default here
>   * should be >= NOFILE (param.h).
>   */
> #ifndef FD_SETSIZE
> #define FD_SETSIZE      64
> #endif
> ----------------------

I think Cygwin's FD macros are based on FreeBSD.  The most recent <sys/select.h> 
on FreeBSD says:

---------------------------------------------------------------------
/*
  * Select uses bit masks of file descriptors in longs.  These macros
  * manipulate such bit fields (the filesystem macros use chars).
  * FD_SETSIZE may be defined by the user, but the default here should
  * be enough for most uses.
  */
#ifndef	FD_SETSIZE
#define	FD_SETSIZE	1024
#endif
---------------------------------------------------------------------

NOFILE isn't mentioned.  Maybe Cygwin (or really newlib) should also remove the 
reference to NOFILE and, perhaps, change the default FD_SETSIZE to 1024.  But 
Cygwin isn't the only newlib target, so there might be good reasons to keep it 
at 64.

Corinna, WDYT?  Or should the discussion be moved to the newlib list?

> Now, this is the relevant part of sys/param.h looks like this:
> ----------------------
> /* Max number of open files.  The Posix version is OPEN_MAX.  */
> /* Number of fds is virtually unlimited in cygwin, but we must provide
>     some reasonable value for Posix conformance */
> #define NOFILE          8192
> ----------------------
> 
> So it's either "<= NOFILE" that was actually meant to be there in the comment (or,
> an equivalent "should NOT be > NOFILE"), or FD_SETSIZE should have been defined as 8192,
> if the comment is actually correct.  Or maybe I'm missing something here :-)

I suspect that the comment was actually meant as written.  On FreeBSD, NOFILE is 
defined to be OPEN_MAX, which is 64.  On Cygwin, however, it wouldn't make sense 
to follow that comment since, as noted in your quote from sys/param.h, there is 
effectively no limit on open files on Cygwin.  (Cygwin maintains a dynamically 
growing table of file descriptors.)

Ken

Re: Typo in <sys/select.h>?

[Corinna Vinschen]

On Jul  5 17:51, Ken Brown wrote:
> On 7/5/2022 10:13 AM, Lavrentiev, Anton (NIH/NLM/NCBI) [C] via Cygwin wrote:
> > Hi,
> > 
> > There's some inconsistency between <sys/select.h> and <sys/param.h>:
> > 
> > sys/select.h has this:
> > -----------------------
> > /*
> >   * Select uses bit masks of file descriptors in longs.
> >   * These macros manipulate such bit fields (the filesystem macros use chars).
> >   * FD_SETSIZE may be defined by the user, but the default here
> >   * should be >= NOFILE (param.h).
> >   */
> > #ifndef FD_SETSIZE
> > #define FD_SETSIZE      64
> > #endif
> > ----------------------
> 
> I think Cygwin's FD macros are based on FreeBSD.  The most recent
> <sys/select.h> on FreeBSD says:
> 
> ---------------------------------------------------------------------
> /*
>  * Select uses bit masks of file descriptors in longs.  These macros
>  * manipulate such bit fields (the filesystem macros use chars).
>  * FD_SETSIZE may be defined by the user, but the default here should
>  * be enough for most uses.
>  */
> #ifndef	FD_SETSIZE
> #define	FD_SETSIZE	1024
> #endif
> ---------------------------------------------------------------------
> 
> NOFILE isn't mentioned.  Maybe Cygwin (or really newlib) should also remove
> the reference to NOFILE and, perhaps, change the default FD_SETSIZE to 1024.

NOFILE is the old BSD variant of OPEN_MAX.  In Cygwin it's defined a bit
weird because the values of NOFILE and OPEN_MAX don't correspond. While
NOFILE is the arbitrary value 8192 (whereever I took that from back in
2003), OPEN_MAX is __OPEN_MAX which is the historical arbitrary value
3200.

Linux doesn't define OPEN_MAX (X11 does for some reason) and NOFILE is
based on OPEN_AX, i. e.

  #if !defined NOFILE && defined OPEN_MAX
  # define NOFILE         OPEN_MAX
  #endif

We should probably do the same on the master branch.  In theory there
should be no source anymore actually requiring on of these macros.

> But Cygwin isn't the only newlib target, so there might be good reasons to
> keep it at 64.
> 
> Corinna, WDYT?  Or should the discussion be moved to the newlib list?

I guess we can change FD_SETSIZE to 1024 as on Linux, albeit this has no
real meaning on Cygwin.  On Linux, select(2) is really only capable to
handle file descriptors numbers up to descriptor number 1023, but Cygwin
doesn't have this problem.  FD_SETSIZE == 64 was only something to save
space.  The bigger FD_SETSIZE, the bigger are the default fd_sets,
something you don't want on small targets.

So, yeah, something like

  #ifndef FD_SETSIZE
  # ifdef __CYGWIN__ 
  #  define FD_SETSIZE      1024
  # else
  #  define FD_SETSIZE      64
  # endif
  #endif

would be ok for master.


Corinna

Re: Typo in <sys/select.h>?

[Jon Turney]

On 06/07/2022 08:42, Corinna Vinschen wrote:
> On Jul  5 17:51, Ken Brown wrote:
>> On 7/5/2022 10:13 AM, Lavrentiev, Anton (NIH/NLM/NCBI) [C] via Cygwin wrote:
> 
> I guess we can change FD_SETSIZE to 1024 as on Linux, albeit this has no
> real meaning on Cygwin.  On Linux, select(2) is really only capable to
> handle file descriptors numbers up to descriptor number 1023, but Cygwin
> doesn't have this problem.  FD_SETSIZE == 64 was only something to save
> space.  The bigger FD_SETSIZE, the bigger are the default fd_sets,
> something you don't want on small targets.
> 
> So, yeah, something like
> 
>    #ifndef FD_SETSIZE
>    # ifdef __CYGWIN__
>    #  define FD_SETSIZE      1024
>    # else
>    #  define FD_SETSIZE      64
>    # endif
>    #endif

Remember that 64 is MAXIMUM_WAIT_OBJECTS for WaitForMultipleObjects(), 
the underlying Win32 API used to implement select(), so using more than 
64 hits some complex code to work around that...

Re: Typo in <sys/select.h>?

[Corinna Vinschen]

On Jul  6 15:01, Jon Turney wrote:
> On 06/07/2022 08:42, Corinna Vinschen wrote:
> > On Jul  5 17:51, Ken Brown wrote:
> > > On 7/5/2022 10:13 AM, Lavrentiev, Anton (NIH/NLM/NCBI) [C] via Cygwin wrote:
> > 
> > I guess we can change FD_SETSIZE to 1024 as on Linux, albeit this has no
> > real meaning on Cygwin.  On Linux, select(2) is really only capable to
> > handle file descriptors numbers up to descriptor number 1023, but Cygwin
> > doesn't have this problem.  FD_SETSIZE == 64 was only something to save
> > space.  The bigger FD_SETSIZE, the bigger are the default fd_sets,
> > something you don't want on small targets.
> > 
> > So, yeah, something like
> > 
> >    #ifndef FD_SETSIZE
> >    # ifdef __CYGWIN__
> >    #  define FD_SETSIZE      1024
> >    # else
> >    #  define FD_SETSIZE      64
> >    # endif
> >    #endif
> 
> Remember that 64 is MAXIMUM_WAIT_OBJECTS for WaitForMultipleObjects(), the
> underlying Win32 API used to implement select(), so using more than 64 hits
> some complex code to work around that...

This isn't what FD_SETSIZE is about.  FD_SETSIZE does *NOT* define the
maximum count of fd's in an fd_set.

It defines the maximum fd number usable in an fd_set.

So if FD_SETSIZE is defined low enough:

  #define FD_SETSIZE 3
  #include <sys/select.h>

  /* Only fd's 0, 1, and 2 will be allowed in this fd_set */
  fd_set set;

  FD_ZERO (&set);
  /* This will probaly set fd to 3 */
  fd = open ("foo", O_RDONLY); 
  /* So this will (hopefully) fail */
  FD_SET (fd, &set);


Corinna

Re: Typo in <sys/select.h>?

[Corinna Vinschen]

On Jul  6 16:15, Corinna Vinschen wrote:
> On Jul  6 15:01, Jon Turney wrote:
> > Remember that 64 is MAXIMUM_WAIT_OBJECTS for WaitForMultipleObjects(), the
> > underlying Win32 API used to implement select(), so using more than 64 hits
> > some complex code to work around that...
> 
> This isn't what FD_SETSIZE is about.  FD_SETSIZE does *NOT* define the
> maximum count of fd's in an fd_set.
> 
> It defines the maximum fd number usable in an fd_set.
> 
> So if FD_SETSIZE is defined low enough:
> 
>   #define FD_SETSIZE 3
>   #include <sys/select.h>
> 
>   /* Only fd's 0, 1, and 2 will be allowed in this fd_set */
>   fd_set set;
> 
>   FD_ZERO (&set);
>   /* This will probaly set fd to 3 */
>   fd = open ("foo", O_RDONLY); 
>   /* So this will (hopefully) fail */
>   FD_SET (fd, &set);

Right, this isn't quite how it works, given fd_set is a bitfield using
unsigned long as basetype under the hood.  This should just outline
the idea behind FD_SETSIZE.


Corinna