[Bug libc/14433] New: setvbuf _IOLBF doesn't honor size

[Bug libc/14433] New: setvbuf _IOLBF doesn't honor size

[P at draigBrady dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=14433

             Bug #: 14433
           Summary: setvbuf _IOLBF doesn't honor size
           Product: glibc
           Version: 2.14
            Status: NEW
          Severity: normal
          Priority: P2
         Component: libc
        AssignedTo: unassigned@sourceware.org
        ReportedBy: P@draigBrady.com
                CC: drepper.fsp@gmail.com
    Classification: Unclassified


POSIX says _IOLBF shall enable line buffering.
However there are two aspects to line buffering.

1. Atomic output of lines, so they're not split
2. Immediate output of lines.

Currently _IOLBF does both of the above.
I would like to propose a change so that if size is specified with _IOLBF,
then 2. is not done, and instead output is only done when the buffer is full.
This would allow programs to distinguish their requirements and improve
performance.

For example, the GNU coreutils checksum utilities would like
behavior 1 above, but not 2, so that output is generated efficiently,
when many small files are being processed.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/14433] setvbuf _IOLBF doesn't honor size

[bugdal at aerifal dot cx]

http://sourceware.org/bugzilla/show_bug.cgi?id=14433

Rich Felker <bugdal at aerifal dot cx> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
                 CC|                            |bugdal at aerifal dot cx
         Resolution|                            |INVALID

--- Comment #1 from Rich Felker <bugdal at aerifal dot cx> 2012-08-06 11:35:55 UTC ---
The whole point (and REQUIREMENT) of line buffering is that lines appear as
output to the underlying file descriptor whenever a newline is seen. Your
proposed change is non-conformant and defeats the whole purpose of line
buffering. Moreover, there is no "line atomicity" requirement of line buffering
and in fact it would be impossible to guarantee. What would you want

   printf("%.*d\n", 2000000000, 0)

to do?

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/14433] setvbuf _IOLBF doesn't honor size

[P at draigBrady dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=14433

--- Comment #2 from Pádraig Brady <P at draigBrady dot com> 2012-08-06 13:55:20 UTC ---
If the buffer is full you output it obviously.

I meant atomic output of lines < the buffer size.
This can very useful and can take advantage of this from POSIX:

"Atomic/non-atomic: A write is atomic if the whole amount written in one
operation is not interleaved with data from any other process. This is useful
when there are multiple writers sending data to a single reader. Applications
need to know how large a write request can be expected to be performed
atomically.This maximum is called {PIPE_BUF}. This volume of IEEE Std
1003.1-2001 does not say whether write requests for more than {PIPE_BUF} bytes
are atomic, but requires that writes of {PIPE_BUF}or fewer bytes shall be
atomic."

So as a performance hint, for programs that output lines,
but don't care when they're output, they could just do:

  setvbuf (stdout, NULL, _IOLBF, PIPE_BUF);

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/14433] setvbuf _IOLBF doesn't honor size

[bugdal at aerifal dot cx]

http://sourceware.org/bugzilla/show_bug.cgi?id=14433

--- Comment #3 from Rich Felker <bugdal at aerifal dot cx> 2012-08-06 16:06:27 UTC ---
Atomicity is only guaranteed when making a direct write() to a pipe. Nowhere is
stdio specified to have any atomicity properties with respect to pipes. With
that said, I think glibc and most if not all existing implementations will give
you the atomicity when the underlying fd is a pipe as long as the conditions
are met, but it would be foolish to rely on this.

Anyway, your proposed change to setvbuf contradicts the requirements of POSIX
and ISO C. Adding such a mode would require a new buffering mode constant so as
not to conflict with existing required semantics.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/14433] setvbuf _IOLBF doesn't honor size

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=14433

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
              Flags|                            |security-

-- 
You are receiving this mail because:
You are on the CC list for the bug.