[PATCH] avoid setting __SLBF and __SNBF on a file descriptor at the same time

[PATCH] avoid setting __SLBF and __SNBF on a file descriptor at the same time

[Giuseppe Musumeci]

Hi all,
I think I found an issue in __smakebuf_r which could lead to memory trashing.

__sinit initialises some common file descriptors as line buffered and
relies on the first users of such FDs to call __smakebuf_r. If
__smakebuf_r realises there's no space for a buffer (malloc returns
NULL), it makes them unbuffered. However, while setting the __SNBF
bit, it doesn't clear the __SLBF bit in the flags. Depending on the
order in which functions check buffering flags in the FD, sometime
they assume it's line buffered (e.g. __sfvwrite_r), trashing
application memory that's not really been allocated to them.

The following patch should solve the problem:


diff --git a/newlib/libc/stdio/makebuf.c b/newlib/libc/stdio/makebuf.c
index c8a50c5..e4cfd36 100644
--- a/newlib/libc/stdio/makebuf.c
+++ b/newlib/libc/stdio/makebuf.c
@@ -100,7 +100,7 @@ _DEFUN(__smakebuf_r, (ptr, fp),
     {
       if (!(fp->_flags & __SSTR))
        {
-         fp->_flags |= __SNBF;
+         fp->_flags = (fp->_flags & ~__SLBF) | __SNBF;
          fp->_bf._base = fp->_p = fp->_nbuf;
          fp->_bf._size = 1;
        }
@@ -115,6 +115,6 @@ _DEFUN(__smakebuf_r, (ptr, fp),
        * even when not outputting to a terminal to ease debugging. */
       if (fp->_file == 1 || fp->_file == 2 ||
           (couldbetty && _isatty_r (ptr, fp->_file)))
-       fp->_flags |= __SLBF;
+       fp->_flags = (fp->_flags & ~__SNBF) | __SLBF;
     }
 }


Thanks and regards,
Giuseppe Musumeci

Re: [PATCH] avoid setting __SLBF and __SNBF on a file descriptor at the same time

[Jeff Johnston]

Makes sense.  Patch applied.

Thanks,

-- Jeff J.

----- Original Message -----
> Hi all,
> I think I found an issue in __smakebuf_r which could lead to memory trashing.
> 
> __sinit initialises some common file descriptors as line buffered and
> relies on the first users of such FDs to call __smakebuf_r. If
> __smakebuf_r realises there's no space for a buffer (malloc returns
> NULL), it makes them unbuffered. However, while setting the __SNBF
> bit, it doesn't clear the __SLBF bit in the flags. Depending on the
> order in which functions check buffering flags in the FD, sometime
> they assume it's line buffered (e.g. __sfvwrite_r), trashing
> application memory that's not really been allocated to them.
> 
> The following patch should solve the problem:
> 
> 
> diff --git a/newlib/libc/stdio/makebuf.c b/newlib/libc/stdio/makebuf.c
> index c8a50c5..e4cfd36 100644
> --- a/newlib/libc/stdio/makebuf.c
> +++ b/newlib/libc/stdio/makebuf.c
> @@ -100,7 +100,7 @@ _DEFUN(__smakebuf_r, (ptr, fp),
>      {
>        if (!(fp->_flags & __SSTR))
>         {
> -         fp->_flags |= __SNBF;
> +         fp->_flags = (fp->_flags & ~__SLBF) | __SNBF;
>           fp->_bf._base = fp->_p = fp->_nbuf;
>           fp->_bf._size = 1;
>         }
> @@ -115,6 +115,6 @@ _DEFUN(__smakebuf_r, (ptr, fp),
>         * even when not outputting to a terminal to ease debugging. */
>        if (fp->_file == 1 || fp->_file == 2 ||
>            (couldbetty && _isatty_r (ptr, fp->_file)))
> -       fp->_flags |= __SLBF;
> +       fp->_flags = (fp->_flags & ~__SNBF) | __SLBF;
>      }
>  }
> 
> 
> Thanks and regards,
> Giuseppe Musumeci
>