[PATCH] newlib: libc: Fix bugs in the commit 3d94e07c49b5.

[PATCH] newlib: libc: Fix bugs in the commit 3d94e07c49b5.

[Takashi Yano]

The commit 3d94e07c49b5 has a few bugs which cause testsuite failure
in libstdc++. This is due to excess orientation check in __srefill_r()
and _ungetc_r(). This patch fixes them.

Fixes: 3d94e07c49b5 ("newlib: libc: Fix crash on fprintf to a wide-oriented stream.")
Reported-by: Christophe Lyon <christophe.lyon@linaro.org>
Reviewed-by:
Signed-off-by: Takashi Yano <takashi.yano@nifty.ne.jp>
---
 newlib/libc/stdio/refill.c | 3 ---
 newlib/libc/stdio/ungetc.c | 8 ++------
 2 files changed, 2 insertions(+), 9 deletions(-)

diff --git a/newlib/libc/stdio/refill.c b/newlib/libc/stdio/refill.c
index c1ef7e120..cd71ed152 100644
--- a/newlib/libc/stdio/refill.c
+++ b/newlib/libc/stdio/refill.c
@@ -43,9 +43,6 @@ __srefill_r (struct _reent * ptr,
 
   CHECK_INIT (ptr, fp);
 
-  if (ORIENT (fp, -1) != -1)
-    return EOF;
-
   fp->_r = 0;			/* largely a convenience for callers */
 
   /* SysV does not make this test; take it out for compatibility */
diff --git a/newlib/libc/stdio/ungetc.c b/newlib/libc/stdio/ungetc.c
index 79914af08..5053fd6c4 100644
--- a/newlib/libc/stdio/ungetc.c
+++ b/newlib/libc/stdio/ungetc.c
@@ -125,12 +125,6 @@ _ungetc_r (struct _reent *rptr,
 
   _newlib_flockfile_start (fp);
 
-  if (ORIENT (fp, -1) != -1)
-    {
-      _newlib_flockfile_exit (fp);
-      return EOF;
-    }
-
   /* After ungetc, we won't be at eof anymore */
   fp->_flags &= ~__SEOF;
 
@@ -213,6 +207,8 @@ int
 ungetc (int c,
        register FILE *fp)
 {
+  if (ORIENT (fp, -1) != -1)
+    return EOF;
   return _ungetc_r (_REENT, c, fp);
 }
 #endif /* !_REENT_ONLY */
-- 
2.39.0

Re: [PATCH] newlib: libc: Fix bugs in the commit 3d94e07c49b5.

[Corinna Vinschen]

On Nov 10 20:34, Takashi Yano wrote:
> The commit 3d94e07c49b5 has a few bugs which cause testsuite failure
> in libstdc++. This is due to excess orientation check in __srefill_r()
> and _ungetc_r(). This patch fixes them.

Oops, _ungetc_r is used from ungetwc as well, that's not good. 
Sorry that I missed that yesterday!

> Fixes: 3d94e07c49b5 ("newlib: libc: Fix crash on fprintf to a wide-oriented stream.")
> Reported-by: Christophe Lyon <christophe.lyon@linaro.org>
> Reviewed-by:
> Signed-off-by: Takashi Yano <takashi.yano@nifty.ne.jp>
> ---
>  newlib/libc/stdio/refill.c | 3 ---
>  newlib/libc/stdio/ungetc.c | 8 ++------
>  2 files changed, 2 insertions(+), 9 deletions(-)
> 
> diff --git a/newlib/libc/stdio/refill.c b/newlib/libc/stdio/refill.c
> index c1ef7e120..cd71ed152 100644
> --- a/newlib/libc/stdio/refill.c
> +++ b/newlib/libc/stdio/refill.c
> @@ -43,9 +43,6 @@ __srefill_r (struct _reent * ptr,
>  
>    CHECK_INIT (ptr, fp);
>  
> -  if (ORIENT (fp, -1) != -1)
> -    return EOF;
> -
>    fp->_r = 0;			/* largely a convenience for callers */
>  
>    /* SysV does not make this test; take it out for compatibility */
> diff --git a/newlib/libc/stdio/ungetc.c b/newlib/libc/stdio/ungetc.c
> index 79914af08..5053fd6c4 100644
> --- a/newlib/libc/stdio/ungetc.c
> +++ b/newlib/libc/stdio/ungetc.c
> @@ -125,12 +125,6 @@ _ungetc_r (struct _reent *rptr,
>  
>    _newlib_flockfile_start (fp);
>  
> -  if (ORIENT (fp, -1) != -1)
> -    {
> -      _newlib_flockfile_exit (fp);
> -      return EOF;
> -    }
> -
>    /* After ungetc, we won't be at eof anymore */
>    fp->_flags &= ~__SEOF;
>  
> @@ -213,6 +207,8 @@ int
>  ungetc (int c,
>         register FILE *fp)
>  {
> +  if (ORIENT (fp, -1) != -1)
> +    return EOF;
>    return _ungetc_r (_REENT, c, fp);
>  }
>  #endif /* !_REENT_ONLY */
> -- 
> 2.39.0

Didn't you forget ungetwc?

But then again, I checked GLibC, and there's something weird:

ungetc does not at all set or test the orientation.

ungetwc sets the orientation to 1, but doesn't check it.

Puzzeling.  I wonder about the reasoning behind this.


Corinna

Re: [PATCH] newlib: libc: Fix bugs in the commit 3d94e07c49b5.

[Takashi Yano]

On Fri, 10 Nov 2023 16:17:45 +0100
Corinna Vinschen wrote:
> On Nov 10 20:34, Takashi Yano wrote:
> > The commit 3d94e07c49b5 has a few bugs which cause testsuite failure
> > in libstdc++. This is due to excess orientation check in __srefill_r()
> > and _ungetc_r(). This patch fixes them.
> 
> Oops, _ungetc_r is used from ungetwc as well, that's not good. 
> Sorry that I missed that yesterday!
> 
> > Fixes: 3d94e07c49b5 ("newlib: libc: Fix crash on fprintf to a wide-oriented stream.")
> > Reported-by: Christophe Lyon <christophe.lyon@linaro.org>
> > Reviewed-by:
> > Signed-off-by: Takashi Yano <takashi.yano@nifty.ne.jp>
> > ---
> >  newlib/libc/stdio/refill.c | 3 ---
> >  newlib/libc/stdio/ungetc.c | 8 ++------
> >  2 files changed, 2 insertions(+), 9 deletions(-)
> > 
> > diff --git a/newlib/libc/stdio/refill.c b/newlib/libc/stdio/refill.c
> > index c1ef7e120..cd71ed152 100644
> > --- a/newlib/libc/stdio/refill.c
> > +++ b/newlib/libc/stdio/refill.c
> > @@ -43,9 +43,6 @@ __srefill_r (struct _reent * ptr,
> >  
> >    CHECK_INIT (ptr, fp);
> >  
> > -  if (ORIENT (fp, -1) != -1)
> > -    return EOF;
> > -
> >    fp->_r = 0;			/* largely a convenience for callers */
> >  
> >    /* SysV does not make this test; take it out for compatibility */
> > diff --git a/newlib/libc/stdio/ungetc.c b/newlib/libc/stdio/ungetc.c
> > index 79914af08..5053fd6c4 100644
> > --- a/newlib/libc/stdio/ungetc.c
> > +++ b/newlib/libc/stdio/ungetc.c
> > @@ -125,12 +125,6 @@ _ungetc_r (struct _reent *rptr,
> >  
> >    _newlib_flockfile_start (fp);
> >  
> > -  if (ORIENT (fp, -1) != -1)
> > -    {
> > -      _newlib_flockfile_exit (fp);
> > -      return EOF;
> > -    }
> > -
> >    /* After ungetc, we won't be at eof anymore */
> >    fp->_flags &= ~__SEOF;
> >  
> > @@ -213,6 +207,8 @@ int
> >  ungetc (int c,
> >         register FILE *fp)
> >  {
> > +  if (ORIENT (fp, -1) != -1)
> > +    return EOF;
> >    return _ungetc_r (_REENT, c, fp);
> >  }
> >  #endif /* !_REENT_ONLY */
> > -- 
> > 2.39.0
> 
> Didn't you forget ungetwc?

Indeed. Thanks! I'll submit v2 patch.

> But then again, I checked GLibC, and there's something weird:
> ungetc does not at all set or test the orientation.
> ungetwc sets the orientation to 1, but doesn't check it.
> Puzzeling.  I wonder about the reasoning behind this.

Hmm, I'll also check that behaviour.

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

Re: [PATCH] newlib: libc: Fix bugs in the commit 3d94e07c49b5.

[Corinna Vinschen]

On Nov 10 16:17, Corinna Vinschen wrote:
> Didn't you forget ungetwc?
> 
> But then again, I checked GLibC, and there's something weird:
> 
> ungetc does not at all set or test the orientation.
> 
> ungetwc sets the orientation to 1, but doesn't check it.
> 
> Puzzeling.  I wonder about the reasoning behind this.

Apparently, ungetwc has been added only later.  ungetc, OTOH, was
defined in a way which allowed to use it on wide-char oriented streams
as well.

So the stance in GLibC is, for backward compatibility reasons, ungetc
can't and must not check or set the orientation at all.

The fact that ungetwc doesn't test the orientation might be a bug in
glibc.  We can follow suit (being "bug-compatible" :)), or we can test
the orientation.  Given that using the correctly oriented functions is
ultimately the responsibility of the application, both ways to handle
this should be fine.


Corinna

Re: [PATCH] newlib: libc: Fix bugs in the commit 3d94e07c49b5.

[Takashi Yano]

On Fri, 10 Nov 2023 17:08:40 +0100
Corinna Vinschen wrote:
> On Nov 10 16:17, Corinna Vinschen wrote:
> > Didn't you forget ungetwc?
> > 
> > But then again, I checked GLibC, and there's something weird:
> > 
> > ungetc does not at all set or test the orientation.
> > 
> > ungetwc sets the orientation to 1, but doesn't check it.
> > 
> > Puzzeling.  I wonder about the reasoning behind this.
> 
> Apparently, ungetwc has been added only later.  ungetc, OTOH, was
> defined in a way which allowed to use it on wide-char oriented streams
> as well.
> 
> So the stance in GLibC is, for backward compatibility reasons, ungetc
> can't and must not check or set the orientation at all.
> 
> The fact that ungetwc doesn't test the orientation might be a bug in
> glibc.  We can follow suit (being "bug-compatible" :)), or we can test
> the orientation.  Given that using the correctly oriented functions is
> ultimately the responsibility of the application, both ways to handle
> this should be fine.

I noticed that getchar/getwchar/gets etc. in newlib do not set
orientation. For example, both getwchar() and getchar() success
in the following code.

#include <stdio.h>
#include <wchar.h>

int main()
{
  wchar_t w;
  char c;
  w = getwchar();
  ungetwc(w, stdin);
  c = getchar();
  printf("%lc,%c\n", w, c);
  return 0;
}

Hmmmmm...

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

Re: [PATCH] newlib: libc: Fix bugs in the commit 3d94e07c49b5.

[Corinna Vinschen]

On Nov 11 01:33, Takashi Yano wrote:
> On Fri, 10 Nov 2023 17:08:40 +0100
> Corinna Vinschen wrote:
> > On Nov 10 16:17, Corinna Vinschen wrote:
> > > Didn't you forget ungetwc?
> > > 
> > > But then again, I checked GLibC, and there's something weird:
> > > 
> > > ungetc does not at all set or test the orientation.
> > > 
> > > ungetwc sets the orientation to 1, but doesn't check it.
> > > 
> > > Puzzeling.  I wonder about the reasoning behind this.
> > 
> > Apparently, ungetwc has been added only later.  ungetc, OTOH, was
> > defined in a way which allowed to use it on wide-char oriented streams
> > as well.
> > 
> > So the stance in GLibC is, for backward compatibility reasons, ungetc
> > can't and must not check or set the orientation at all.
> > 
> > The fact that ungetwc doesn't test the orientation might be a bug in
> > glibc.  We can follow suit (being "bug-compatible" :)), or we can test
> > the orientation.  Given that using the correctly oriented functions is
> > ultimately the responsibility of the application, both ways to handle
> > this should be fine.
> 
> I noticed that getchar/getwchar/gets etc. in newlib do not set
> orientation. For example, both getwchar() and getchar() success
> in the following code.
> 
> #include <stdio.h>
> #include <wchar.h>
> 
> int main()
> {
>   wchar_t w;
>   char c;
>   w = getwchar();
>   ungetwc(w, stdin);
>   c = getchar();
>   printf("%lc,%c\n", w, c);
>   return 0;
> }

Just for the sake of clarity, this requires your not yet applied patch
"newlib: libc: Fix bugs in the commit 3d94e07c49b5".  Otherwise
getwchar() always returns WEOF.

> Hmmmmm...

In GLibC, getchar returns EOF in this case.  Their macros never check
for the orientation, only the underlying functions do that. In case of
getchar that's the __uflow function, which is hidden under a pile of
other macros and functions.

Our underlying functions of getchar are _getc_r, __srget_r and
__srefill_r.  As already noticed, __srefill_r is called from wide-char
code as well, but __srget_r isn't, so that sounds like a good place to
set and check the orientation.

I tried exactly that, and getchar() still returns the ungetwc'ed character.

Our pile of macros and function is BSD based and works slightly
differently than GLibC's.  __srget_r is only called if no char is in the
buffer, but ungetwc filled something into the buffer.

So we either let this slip, or we have to add the ORIENT checks to
getc and _getc_r:

diff --git a/newlib/libc/stdio/getc.c b/newlib/libc/stdio/getc.c
index c8781320e603..12bffaf8f79a 100644
--- a/newlib/libc/stdio/getc.c
+++ b/newlib/libc/stdio/getc.c
@@ -81,6 +81,9 @@ _getc_r (struct _reent *ptr,
 {
   int result;
   CHECK_INIT (ptr, fp);
+  if (ORIENT (fp, -1) != -1)
+    return EOF;
+
   _newlib_flockfile_start (fp);
   result = __sgetc_r (ptr, fp);
   _newlib_flockfile_end (fp);
@@ -96,6 +99,9 @@ getc (register FILE *fp)
   struct _reent *reent = _REENT;
 
   CHECK_INIT (reent, fp);
+  if (ORIENT (fp, -1) != -1)
+    return EOF;
+
   _newlib_flockfile_start (fp);
   result = __sgetc_r (reent, fp);
   _newlib_flockfile_end (fp);


Corinna