Mismatch between newlib and glibc regarding fileno

Mismatch between newlib and glibc regarding fileno

[Torbjorn SVENSSON]

Hi all,

I've been trying to run tests for arm-none-eabi on GCC14 tree.
What I've seen is that fileno() is not available. If you look though the 
header files, I see that this is guarded by __POSIX_VISIBLE and due to a 
recent change in the GCC testsuite (part of PR96395), they moved the 
test case to gcc/testsuite/c-c++-common/analyzer/fileno-1.c and then 
invoke it with g++ and -std=c++98. With this change, strict ANSI is 
defined, but not __POSIX_VISIBLE.

If I run the same test on the native g++ tool in Ubuntu, I instead get 
that __USE_POSIX is set (the guard for fileno() in glibc), so this 
differs from the behavior noticed with newlib.


Is it correct that newlib does not define __POSIX_VISIBLE when g++ is 
invoked with -std=c++98 (or any other standard that does not use the GNU 
extensions)?
It can also be seen using gcc with -std=c11 for example, so it's not 
strictly a C vs C++ issue.


The expression used in glibc: 
https://github.com/bminor/glibc/blob/master/include/features.h#L335

The expression used in newlib: 
https://sourceware.org/git/?p=newlib-cygwin.git;a=blob;f=newlib/libc/include/sys/features.h;h=6a925c87e9ec333fc51538201aa7b52d24b3ca5b;hb=refs/heads/main#l148


Kind regards,
Torbjörn

Re: Mismatch between newlib and glibc regarding fileno

[Andrew Pinski]

On Fri, Feb 9, 2024 at 8:30 AM Torbjorn SVENSSON
<torbjorn.svensson@foss.st.com> wrote:
>
> Hi all,
>
> I've been trying to run tests for arm-none-eabi on GCC14 tree.
> What I've seen is that fileno() is not available. If you look though the
> header files, I see that this is guarded by __POSIX_VISIBLE and due to a
> recent change in the GCC testsuite (part of PR96395), they moved the
> test case to gcc/testsuite/c-c++-common/analyzer/fileno-1.c and then
> invoke it with g++ and -std=c++98. With this change, strict ANSI is
> defined, but not __POSIX_VISIBLE.

Note the testcase failure is recorded as https://gcc.gnu.org/PR113278 .

>
> If I run the same test on the native g++ tool in Ubuntu, I instead get
> that __USE_POSIX is set (the guard for fileno() in glibc), so this
> differs from the behavior noticed with newlib.

That is also due to _GNU_SOURCE being defined for C++ for Linux/g++. I
think this is just a GCC testcase issue rather than something needing
to be fixed in newlib even.

Thanks,
Andrew

>
>
> Is it correct that newlib does not define __POSIX_VISIBLE when g++ is
> invoked with -std=c++98 (or any other standard that does not use the GNU
> extensions)?
> It can also be seen using gcc with -std=c11 for example, so it's not
> strictly a C vs C++ issue.
>
>
> The expression used in glibc:
> https://github.com/bminor/glibc/blob/master/include/features.h#L335
>
> The expression used in newlib:
> https://sourceware.org/git/?p=newlib-cygwin.git;a=blob;f=newlib/libc/include/sys/features.h;h=6a925c87e9ec333fc51538201aa7b52d24b3ca5b;hb=refs/heads/main#l148
>
>
> Kind regards,
> Torbjörn

Re: Mismatch between newlib and glibc regarding fileno

[Corinna Vinschen]

On Feb  9 08:40, Andrew Pinski wrote:
> On Fri, Feb 9, 2024 at 8:30 AM Torbjorn SVENSSON
> <torbjorn.svensson@foss.st.com> wrote:
> >
> > Hi all,
> >
> > I've been trying to run tests for arm-none-eabi on GCC14 tree.
> > What I've seen is that fileno() is not available. If you look though the
> > header files, I see that this is guarded by __POSIX_VISIBLE and due to a
> > recent change in the GCC testsuite (part of PR96395), they moved the
> > test case to gcc/testsuite/c-c++-common/analyzer/fileno-1.c and then
> > invoke it with g++ and -std=c++98. With this change, strict ANSI is
> > defined, but not __POSIX_VISIBLE.
> 
> Note the testcase failure is recorded as https://gcc.gnu.org/PR113278 .
> 
> >
> > If I run the same test on the native g++ tool in Ubuntu, I instead get
> > that __USE_POSIX is set (the guard for fileno() in glibc), so this
> > differs from the behavior noticed with newlib.
> 
> That is also due to _GNU_SOURCE being defined for C++ for Linux/g++. I
> think this is just a GCC testcase issue rather than something needing
> to be fixed in newlib even.

Along these lines, note the Linux man page for fileno:

    STANDARDS
       POSIX.1-2008.

    HISTORY
      POSIX.1-2001.

and the feature test in GLibc's stdio.h:

    #ifdef  __USE_POSIX
    /* Return the system file descriptor for STREAM.  */
    extern int fileno (FILE *__stream) __THROW __wur;
    #endif /* Use POSIX.  */


Corinna

Re: Mismatch between newlib and glibc regarding fileno

[Torbjorn SVENSSON]

On 2024-02-09 17:54, Corinna Vinschen wrote:
> On Feb  9 08:40, Andrew Pinski wrote:
>> On Fri, Feb 9, 2024 at 8:30 AM Torbjorn SVENSSON
>> <torbjorn.svensson@foss.st.com> wrote:
>>>
>>> Hi all,
>>>
>>> I've been trying to run tests for arm-none-eabi on GCC14 tree.
>>> What I've seen is that fileno() is not available. If you look though the
>>> header files, I see that this is guarded by __POSIX_VISIBLE and due to a
>>> recent change in the GCC testsuite (part of PR96395), they moved the
>>> test case to gcc/testsuite/c-c++-common/analyzer/fileno-1.c and then
>>> invoke it with g++ and -std=c++98. With this change, strict ANSI is
>>> defined, but not __POSIX_VISIBLE.
>>
>> Note the testcase failure is recorded as https://gcc.gnu.org/PR113278 .
>>
>>>
>>> If I run the same test on the native g++ tool in Ubuntu, I instead get
>>> that __USE_POSIX is set (the guard for fileno() in glibc), so this
>>> differs from the behavior noticed with newlib.
>>
>> That is also due to _GNU_SOURCE being defined for C++ for Linux/g++. I
>> think this is just a GCC testcase issue rather than something needing
>> to be fixed in newlib even.
> 
> Along these lines, note the Linux man page for fileno:
> 
>      STANDARDS
>         POSIX.1-2008.
> 
>      HISTORY
>        POSIX.1-2001.
> 
> and the feature test in GLibc's stdio.h:
> 
>      #ifdef  __USE_POSIX
>      /* Return the system file descriptor for STREAM.  */
>      extern int fileno (FILE *__stream) __THROW __wur;
>      #endif /* Use POSIX.  */
> 
> 
> Corinna
> 


Okay, so newlib is more restrictive than glibc on this topic.
I will prepare a patch for test cases in GCC with defining _POSIX_SOURCE 
  so that the test cases succeed for newlib.

Kind regards,
Torbjörn

Re: Mismatch between newlib and glibc regarding fileno

[Corinna Vinschen]

On Feb 12 16:36, Torbjorn SVENSSON wrote:
> On 2024-02-09 17:54, Corinna Vinschen wrote:
> > On Feb  9 08:40, Andrew Pinski wrote:
> > > On Fri, Feb 9, 2024 at 8:30 AM Torbjorn SVENSSON
> > > <torbjorn.svensson@foss.st.com> wrote:
> > > > 
> > > > Hi all,
> > > > 
> > > > I've been trying to run tests for arm-none-eabi on GCC14 tree.
> > > > What I've seen is that fileno() is not available. If you look though the
> > > > header files, I see that this is guarded by __POSIX_VISIBLE and due to a
> > > > recent change in the GCC testsuite (part of PR96395), they moved the
> > > > test case to gcc/testsuite/c-c++-common/analyzer/fileno-1.c and then
> > > > invoke it with g++ and -std=c++98. With this change, strict ANSI is
> > > > defined, but not __POSIX_VISIBLE.
> > > 
> > > Note the testcase failure is recorded as https://gcc.gnu.org/PR113278 .
> > > 
> > > > 
> > > > If I run the same test on the native g++ tool in Ubuntu, I instead get
> > > > that __USE_POSIX is set (the guard for fileno() in glibc), so this
> > > > differs from the behavior noticed with newlib.
> > > 
> > > That is also due to _GNU_SOURCE being defined for C++ for Linux/g++. I
> > > think this is just a GCC testcase issue rather than something needing
> > > to be fixed in newlib even.
> > 
> > Along these lines, note the Linux man page for fileno:
> > 
> >      STANDARDS
> >         POSIX.1-2008.
> > 
> >      HISTORY
> >        POSIX.1-2001.
> > 
> > and the feature test in GLibc's stdio.h:
> > 
> >      #ifdef  __USE_POSIX
> >      /* Return the system file descriptor for STREAM.  */
> >      extern int fileno (FILE *__stream) __THROW __wur;
> >      #endif /* Use POSIX.  */
> > 
> > 
> > Corinna
> > 
> Okay, so newlib is more restrictive than glibc on this topic.
> I will prepare a patch for test cases in GCC with defining _POSIX_SOURCE  so
> that the test cases succeed for newlib.

It looks like it.  But I do wonder if that's really intended by glibc.
I ran a quick test, first under newlibL

  $ g++ -std=c++98 -E -dM /usr/include/features.h | grep VISIBLE
  #define __LARGEFILE_VISIBLE 0
  #define __ISO_C_VISIBLE 1999
  #define __XSI_VISIBLE 0
  #define __GNU_VISIBLE 0
  #define __BSD_VISIBLE 0
  #define __POSIX_VISIBLE 0
  #define __SVID_VISIBLE 0
  #define __ATFILE_VISIBLE 0
  #define __MISC_VISIBLE 0

then under glibc:

  $ g++ -std=c++98 -E -dM x.cc | grep '#define __USE'
  #define __USE_UNIX98 1
  #define __USE_FORTIFY_LEVEL 0
  #define __USE_ISOC11 1
  #define __USE_ISOC95 1
  #define __USE_ISOC99 1
  #define __USE_XOPEN 1
  #define __USE_XOPEN2K 1
  #define __USE_POSIX199506 1
  #define __USE_GNU 1
  #define __USE_XOPEN2KXSI 1
  #define __USE_XOPEN2K8 1
  #define __USE_POSIX 1
  #define __USER_LABEL_PREFIX__ 
  #define __USE_MISC 1
  #define __USE_POSIX2 1
  #define __USE_LARGEFILE64 1
  #define __USE_POSIX199309 1
  #define __USE_XOPEN2K8XSI 1
  #define __USE_LARGEFILE 1
  #define __USE_XOPEN_EXTENDED 1
  #define __USE_DYNAMIC_STACK_SIZE 1
  #define __USE_ATFILE 1

How is it possible that with -std=c++98, everything and the kitchen sink
is enabled?  Is that really correct?!?


Corinna

Re: Mismatch between newlib and glibc regarding fileno

[Corinna Vinschen]

On Feb 12 17:33, Corinna Vinschen wrote:
> On Feb 12 16:36, Torbjorn SVENSSON wrote:
> > Okay, so newlib is more restrictive than glibc on this topic.
> > I will prepare a patch for test cases in GCC with defining _POSIX_SOURCE  so
> > that the test cases succeed for newlib.
> 
> It looks like it.  But I do wonder if that's really intended by glibc.
> I ran a quick test, first under newlibL
> 
>   $ g++ -std=c++98 -E -dM /usr/include/features.h | grep VISIBLE
>   #define __LARGEFILE_VISIBLE 0
>   #define __ISO_C_VISIBLE 1999
>   #define __XSI_VISIBLE 0
>   #define __GNU_VISIBLE 0
>   #define __BSD_VISIBLE 0
>   #define __POSIX_VISIBLE 0
>   #define __SVID_VISIBLE 0
>   #define __ATFILE_VISIBLE 0
>   #define __MISC_VISIBLE 0
> 
> then under glibc:
> 
>   $ g++ -std=c++98 -E -dM x.cc | grep '#define __USE'
>   #define __USE_UNIX98 1
>   #define __USE_FORTIFY_LEVEL 0
>   #define __USE_ISOC11 1
>   #define __USE_ISOC95 1
>   #define __USE_ISOC99 1
>   #define __USE_XOPEN 1
>   #define __USE_XOPEN2K 1
>   #define __USE_POSIX199506 1
>   #define __USE_GNU 1
>   #define __USE_XOPEN2KXSI 1
>   #define __USE_XOPEN2K8 1
>   #define __USE_POSIX 1
>   #define __USER_LABEL_PREFIX__ 
>   #define __USE_MISC 1
>   #define __USE_POSIX2 1
>   #define __USE_LARGEFILE64 1
>   #define __USE_POSIX199309 1
>   #define __USE_XOPEN2K8XSI 1
>   #define __USE_LARGEFILE 1
>   #define __USE_XOPEN_EXTENDED 1
>   #define __USE_DYNAMIC_STACK_SIZE 1
>   #define __USE_ATFILE 1
> 
> How is it possible that with -std=c++98, everything and the kitchen sink
> is enabled?  Is that really correct?!?

...especially since __STRICT_ANSI__ is defined to 1 in this scenario.


Corinna

Re: Mismatch between newlib and glibc regarding fileno

[Torbjorn SVENSSON]

On 2024-02-12 17:40, Corinna Vinschen wrote:
> On Feb 12 17:33, Corinna Vinschen wrote:
>> On Feb 12 16:36, Torbjorn SVENSSON wrote:
>>> Okay, so newlib is more restrictive than glibc on this topic.
>>> I will prepare a patch for test cases in GCC with defining _POSIX_SOURCE  so
>>> that the test cases succeed for newlib.
>>
>> It looks like it.  But I do wonder if that's really intended by glibc.
>> I ran a quick test, first under newlibL
>>
>>    $ g++ -std=c++98 -E -dM /usr/include/features.h | grep VISIBLE
>>    #define __LARGEFILE_VISIBLE 0
>>    #define __ISO_C_VISIBLE 1999
>>    #define __XSI_VISIBLE 0
>>    #define __GNU_VISIBLE 0
>>    #define __BSD_VISIBLE 0
>>    #define __POSIX_VISIBLE 0
>>    #define __SVID_VISIBLE 0
>>    #define __ATFILE_VISIBLE 0
>>    #define __MISC_VISIBLE 0
>>
>> then under glibc:
>>
>>    $ g++ -std=c++98 -E -dM x.cc | grep '#define __USE'
>>    #define __USE_UNIX98 1
>>    #define __USE_FORTIFY_LEVEL 0
>>    #define __USE_ISOC11 1
>>    #define __USE_ISOC95 1
>>    #define __USE_ISOC99 1
>>    #define __USE_XOPEN 1
>>    #define __USE_XOPEN2K 1
>>    #define __USE_POSIX199506 1
>>    #define __USE_GNU 1
>>    #define __USE_XOPEN2KXSI 1
>>    #define __USE_XOPEN2K8 1
>>    #define __USE_POSIX 1
>>    #define __USER_LABEL_PREFIX__
>>    #define __USE_MISC 1
>>    #define __USE_POSIX2 1
>>    #define __USE_LARGEFILE64 1
>>    #define __USE_POSIX199309 1
>>    #define __USE_XOPEN2K8XSI 1
>>    #define __USE_LARGEFILE 1
>>    #define __USE_XOPEN_EXTENDED 1
>>    #define __USE_DYNAMIC_STACK_SIZE 1
>>    #define __USE_ATFILE 1
>>
>> How is it possible that with -std=c++98, everything and the kitchen sink
>> is enabled?  Is that really correct?!?
> 
> ...especially since __STRICT_ANSI__ is defined to 1 in this scenario.

Do you know any way to identify if this is the intended behavior or if 
it's an overlook in the glibc end?

Regardless if glibc is doing this deliberately or not, I suppose the 
correct thing is to manually define _POSIX_VISIBLE in the test case, right?

Re: Mismatch between newlib and glibc regarding fileno

[Corinna Vinschen]

On Feb 12 18:11, Torbjorn SVENSSON wrote:
> On 2024-02-12 17:40, Corinna Vinschen wrote:
> > On Feb 12 17:33, Corinna Vinschen wrote:
> > > On Feb 12 16:36, Torbjorn SVENSSON wrote:
> > > > Okay, so newlib is more restrictive than glibc on this topic.
> > > > I will prepare a patch for test cases in GCC with defining _POSIX_SOURCE  so
> > > > that the test cases succeed for newlib.
> > > 
> > > It looks like it.  But I do wonder if that's really intended by glibc.
> > > I ran a quick test, first under newlibL
> > > [...]
> > > How is it possible that with -std=c++98, everything and the kitchen sink
> > > is enabled?  Is that really correct?!?
> > 
> > ...especially since __STRICT_ANSI__ is defined to 1 in this scenario.
> 
> Do you know any way to identify if this is the intended behavior or if it's
> an overlook in the glibc end?

Except for asking on the glibc mailing list, no.

> Regardless if glibc is doing this deliberately or not, I suppose the correct
> thing is to manually define _POSIX_VISIBLE in the test case, right?

Yes, I would suggest that.  It can't be wrong to be very exact there.


Corinna

Re: Mismatch between newlib and glibc regarding fileno

[Joseph Myers]

On Mon, 12 Feb 2024, Corinna Vinschen wrote:

> How is it possible that with -std=c++98, everything and the kitchen sink
> is enabled?  Is that really correct?!?

See <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=11196> (though 
<https://gcc.gnu.org/bugzilla/show_bug.cgi?id=2082> is older).  Much of 
the standard C++ library implementation is in the headers, which makes it 
tricky to implement without _GNU_SOURCE when the C++ library wants to use 
(internally) lots of features not in ISO C.

-- 
Joseph S. Myers
josmyers@redhat.com

Re: Mismatch between newlib and glibc regarding fileno

[Torbjorn SVENSSON]

On 2024-02-12 19:14, Joseph Myers wrote:
> On Mon, 12 Feb 2024, Corinna Vinschen wrote:
> 
>> How is it possible that with -std=c++98, everything and the kitchen sink
>> is enabled?  Is that really correct?!?
> 
> See <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=11196> (though
> <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=2082> is older).  Much of
> the standard C++ library implementation is in the headers, which makes it
> tricky to implement without _GNU_SOURCE when the C++ library wants to use
> (internally) lots of features not in ISO C.
> 

So, based on this, is it correct that newlib includes the check for 
__STRICT_ANSI__?

I.e., should GCC tests be updated to add -D_POSIX_SOURCE or should the 
check for __STRICT_ANSI__ be removed from the newlib features.h file? Or 
maybe both should be done?

Kind regards,
Torbjörn

Re: Mismatch between newlib and glibc regarding fileno

[Corinna Vinschen]

On Feb 12 20:27, Torbjorn SVENSSON wrote:
> 
> 
> On 2024-02-12 19:14, Joseph Myers wrote:
> > On Mon, 12 Feb 2024, Corinna Vinschen wrote:
> > 
> > > How is it possible that with -std=c++98, everything and the kitchen sink
> > > is enabled?  Is that really correct?!?
> > 
> > See <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=11196> (though
> > <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=2082> is older).  Much of
> > the standard C++ library implementation is in the headers, which makes it
> > tricky to implement without _GNU_SOURCE when the C++ library wants to use
> > (internally) lots of features not in ISO C.
> > 
> 
> So, based on this, is it correct that newlib includes the check for
> __STRICT_ANSI__?
> 
> I.e., should GCC tests be updated to add -D_POSIX_SOURCE or should the check
> for __STRICT_ANSI__ be removed from the newlib features.h file? Or maybe
> both should be done?

glibc's feature.h performs the same __STRICT_ANSI__ checks, afaics.


Corinna

Re: Mismatch between newlib and glibc regarding fileno

[Torbjorn SVENSSON]

On 2024-02-12 20:40, Corinna Vinschen wrote:
> On Feb 12 20:27, Torbjorn SVENSSON wrote:
>>
>>
>> On 2024-02-12 19:14, Joseph Myers wrote:
>>> On Mon, 12 Feb 2024, Corinna Vinschen wrote:
>>>
>>>> How is it possible that with -std=c++98, everything and the kitchen sink
>>>> is enabled?  Is that really correct?!?
>>>
>>> See <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=11196> (though
>>> <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=2082> is older).  Much of
>>> the standard C++ library implementation is in the headers, which makes it
>>> tricky to implement without _GNU_SOURCE when the C++ library wants to use
>>> (internally) lots of features not in ISO C.
>>>
>>
>> So, based on this, is it correct that newlib includes the check for
>> __STRICT_ANSI__?
>>
>> I.e., should GCC tests be updated to add -D_POSIX_SOURCE or should the check
>> for __STRICT_ANSI__ be removed from the newlib features.h file? Or maybe
>> both should be done?
> 
> glibc's feature.h performs the same __STRICT_ANSI__ checks, afaics.

I've pushed a patch on GCC14 that defines _POSIX_SOURCE.
https://gcc.gnu.org/pipermail/gcc-patches/2024-February/645731.html

Don't know if someone will look more into why newlib has a different set 
of defines than newlib, but I'm happy with that the test cases no longer 
produces fails in the log.

Kind regards,
Torbjörn