[Bug libc/13151] New: fmemopen streams fail to read eof

[Bug libc/13151] New: fmemopen streams fail to read eof

[bugdal at aerifal dot cx]

http://sourceware.org/bugzilla/show_bug.cgi?id=13151

             Bug #: 13151
           Summary: fmemopen streams fail to read eof
           Product: glibc
           Version: unspecified
            Status: NEW
          Severity: normal
          Priority: P2
         Component: libc
        AssignedTo: drepper.fsp@gmail.com
        ReportedBy: bugdal@aerifal.cx
    Classification: Unclassified


Streams obtained by fmemopen fail to yield EOF when attempting to read past the
"current size" of the buffer. Minimal failure case:

FILE *f = fmemopen((char[10]){"hello"}, 10, "a+");
assert(getc(f)==EOF);

(The "current size" established by opening in append mode is the offset of the
null byte, 5.)

Reference:

http://pubs.opengroup.org/onlinepubs/9699919799/functions/fmemopen.html

I understand that the GNU fmemopen predated the standardized function, and I'm
not sure what politics were involved in writing the standard such that the
existing GNU version did not conform. Nonetheless, this should be fixed. If you
want to keep the original GNU behavior, there should be a separate
__posix_fmemopen that gets used when _POSIX_C_SOURCE >= 200809L.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug stdio/13151] fmemopen streams fail to read eof

[jsm28 at gcc dot gnu.org]

http://sourceware.org/bugzilla/show_bug.cgi?id=13151

Joseph Myers <jsm28 at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|libc                        |stdio

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug stdio/13151] fmemopen streams fail to read eof

[mtk.manpages at gmail dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=13151

Michael Kerrisk <mtk.manpages at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |mtk.manpages at gmail dot
                   |                            |com

--- Comment #1 from Michael Kerrisk <mtk.manpages at gmail dot com> 2012-04-27 20:15:20 UTC ---
I think your example isn't quite right, but there is a real problem here.

See the fopen(3) man page:

       a+     Open for reading and appending (writing at end  of
              file).   The file is created if it does not exist.
              The initial file position for reading  is  at  the
              beginning  of  the  file,  but  output  is  always
              appended to the end of the file.

So, "a+" should set the file pointer to byte 0.

It looks like the problem is that the file position is not correctly reset to 0
when mode is "a+". Without a command-line argument, the program below does an
fmemopen(), and then tries reading characters, and we see the problem:

$ ./a.out
Initial ftell(): -1
Segmentation fault (core dumped)

With a command-line argument, the program does an initial fseek() to explicitly
place the file pointer at byte 0, and then all is well:

./a.out x
Initial ftell(): -1
Resetting file position
New ftell(): 0
 0: h 104; feof()=0; ftell() = 1
 1: e 101; feof()=0; ftell() = 2
 2: l 108; feof()=0; ftell() = 3
 3: l 108; feof()=0; ftell() = 4
 4: o 111; feof()=0; ftell() = 5
 5: � -1; feof()=1; ftell() = 5

So, it seems that the fix would be that when fmemopen() mode is "a+", the glibc
implementation should set the file position to 0.

=========
#define _XOPEN_SOURCE 700
#include <stdio.h>
#include <string.h>
#include <assert.h>
#include <stdlib.h>

#define errExit(msg)     do { perror(msg); exit(EXIT_FAILURE); \
            } while (0)

int
main(int argc, char **argv)
{
    FILE *fp = fmemopen((char[20]){"hello, world"}, 5, "a+");
    char c;
    int j;

    printf("Initial ftell(): %ld\n", ftell(fp));

    if (argc > 1) {
    printf("Resetting file position\n");
        fseek(fp, 0, SEEK_SET);
        printf("New ftell(): %ld\n", ftell(fp));
    }

    for (j = 0; j < 20 && !feof(fp); j++) {
        c = fgetc(fp);
    printf("%2d: %c %d; feof()=%d; ftell() = %ld\n",
        j, c, c, feof(fp), ftell(fp));
    }
}

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug stdio/13151] fmemopen streams fail to read eof

[mtk.manpages at gmail dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=13151

--- Comment #2 from Michael Kerrisk <mtk.manpages at gmail dot com> 2012-04-27 20:31:07 UTC ---
Rich,

I spoke a little soon. I see that that the fmemopen() POSIX spec has a
definition of "a+" that is more consistent with your explanation. Anyway, we
both agree there's a problem here ;-).

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug stdio/13151] fmemopen streams fail to read eof

[mtk.manpages at gmail dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=13151

--- Comment #3 from Michael Kerrisk <mtk.manpages at gmail dot com> 2012-04-28 02:20:55 UTC ---
I tested a little more. See the code below. 

The problem seems to occur only when the specified size of the buffer does not
include a null byte. If the specified size does include a null byte, then the
file position is correctly set to the first null byte. The standard does
clearly cover both cases:

[[
The stream maintains a current position in the buffer. This position is
initially set to either the beginning of the buffer (for r and w modes) or to
the first null byte in the buffer (for a modes). If no null byte is found in
append mode, the initial position is set to one byte after the end of the
buffer.
]]

Thus, glibc's fmemopen() should be setting the file position to the byte after
the end of the supplied string. Instead, it sets the file position to -1.

Using the test program below, here's what happens when the specified size
includes the null byte:

$ ./a.out 15
Initial ftell(): 12

And here's what happens if the specified size doesn't include the null byte:

$ ./a.out  10
Initial ftell(): -1



/* fmemopen_a+_file_pos_bug.c
*/
#define _XOPEN_SOURCE 700
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <ctype.h>

#define errExit(msg)     do { perror(msg); exit(EXIT_FAILURE); \
            } while (0)

#define BSIZE 20
int
main(int argc, char **argv)
{
    char buf[BSIZE] = {"hello, world"};
    FILE *fp;

    if (argc < 2) {
    fprintf(stderr, "Usage: %s <nchars> [fseek-pos]\n", argv[0]);
    exit(EXIT_FAILURE);
    }

    fp = fmemopen(buf, atoi(argv[1]), "a+");
    if (fp == NULL)
    errExit("fmemopen");

    printf("Initial ftell(): %ld\n", ftell(fp));

    if (argc > 2) {
    printf("Resetting file position\n");
        fseek(fp, atoi(argv[2]), SEEK_SET);
        printf("New ftell(): %ld\n", ftell(fp));
    }
}

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug stdio/13151] fmemopen streams fail to read eof

[mtk.manpages at gmail dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=13151

--- Comment #4 from Michael Kerrisk <mtk.manpages at gmail dot com> 2012-04-28 03:27:37 UTC ---
I've added the following text to the man page under BUGS

[[
If the
.I size
argument given to
.BR fmemopen ()
does not cover a null byte in
.IR buf
then, according to POSIX.1-2008,
the file position should be set to the next byte after the end of the buffer.
However, in this case, the glibc
.\" FIXME http://sourceware.org/bugzilla/show_bug.cgi?id=13151
.BR fmemopen ()
sets the file position to \-1.
]]

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug stdio/13151] fmemopen streams fail to read eof

[mtk.manpages at gmail dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=13151

--- Comment #5 from Michael Kerrisk <mtk.manpages at gmail dot com> 2012-04-28 04:26:34 UTC ---
Twaeked that man page text. Now it is:

If the
.I mode
argument to
.BR fmemopen ()
specifies append ("a" or "a+"), and the
.I size
argument does not cover a null byte in
.IR buf
then, according to POSIX.1-2008,
the initial file position should be set to
the next byte after the end of the buffer.
However, in this case the glibc
.\" FIXME http://sourceware.org/bugzilla/show_bug.cgi?id=13151
.BR fmemopen ()
sets the file position to \-1.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug stdio/13151] fmemopen streams fail to read eof

[schwab@linux-m68k.org]

http://sourceware.org/bugzilla/show_bug.cgi?id=13151

Andreas Schwab <schwab@linux-m68k.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         AssignedTo|drepper.fsp at gmail dot    |unassigned at sourceware
                   |com                         |dot org

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug stdio/13151] fmemopen streams fail to read eof

[ondra at iuuk dot mff.cuni.cz]

http://sourceware.org/bugzilla/show_bug.cgi?id=13151

OndrejBilka <ondra at iuuk dot mff.cuni.cz> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |ondra at iuuk dot
                   |                            |mff.cuni.cz

--- Comment #6 from OndrejBilka <ondra at iuuk dot mff.cuni.cz> 2013-05-20 15:24:11 UTC ---
Michael,
I am refactoring fmemopen. It should handle a+ mode like fopen does. See
http://sourceware.org/ml/libc-alpha/2013-05/msg00729.html

If I read correctly should be also resolved by this patch.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug stdio/13151] fmemopen streams fail to read eof

[jackie.rosen at hushmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=13151

Jackie Rosen <jackie.rosen at hushmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jackie.rosen at hushmail dot com

--- Comment #7 from Jackie Rosen <jackie.rosen at hushmail dot com> ---
*** Bug 260998 has been marked as a duplicate of this bug. ***
Seen from the domain http://volichat.com
Page where seen: http://volichat.com/adult-chat-rooms
Marked for reference. Resolved as fixed @bugzilla.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug stdio/13151] fmemopen streams fail to read eof

[schwab at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=13151

Andreas Schwab <schwab at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|jackie.rosen at hushmail dot com   |

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug stdio/13151] fmemopen streams fail to read eof

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=13151

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
              Flags|                            |security-

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug stdio/13151] fmemopen streams fail to read eof

[adhemerval.zanella at linaro dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=13151

Adhemerval Zanella <adhemerval.zanella at linaro dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
                 CC|                            |adhemerval.zanella at linaro dot o
                   |                            |rg
         Resolution|---                         |FIXED

--- Comment #8 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> ---
This is fixed by fdb7d390dd0d96e4a8239c46f3aa64598b90842b.  Closing bug.

-- 
You are receiving this mail because:
You are on the CC list for the bug.