[Bug stdio/15142] New: Missing locking in _IO_cleanup

[Bug stdio/15142] New: Missing locking in _IO_cleanup

[schwab@linux-m68k.org]

http://sourceware.org/bugzilla/show_bug.cgi?id=15142

             Bug #: 15142
           Summary: Missing locking in _IO_cleanup
           Product: glibc
           Version: 2.3.4
            Status: NEW
          Severity: normal
          Priority: P2
         Component: stdio
        AssignedTo: unassigned@sourceware.org
        ReportedBy: schwab@linux-m68k.org
    Classification: Unclassified


Created attachment 6870
  --> http://sourceware.org/bugzilla/attachment.cgi?id=6870
Testcase

When _IO_flush_all_lockp is called from _IO_cleanup it doesn't do any locking
on _IO_list_all, which races with fopen/fclose from other threads.  This can
result in heap corruption.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug stdio/15142] Missing locking in _IO_cleanup

[bugdal at aerifal dot cx]

http://sourceware.org/bugzilla/show_bug.cgi?id=15142

Rich Felker <bugdal at aerifal dot cx> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |bugdal at aerifal dot cx

--- Comment #1 from Rich Felker <bugdal at aerifal dot cx> 2013-02-14 20:12:54 UTC ---
I have two related issues open on the Austin Group bug tracker:

http://austingroupbugs.net/view.php?id=610
http://austingroupbugs.net/view.php?id=611

Unfortunately, I believe the current glibc behavior of not performing
appropriate locking is intentional, so that exit works even when locks
would/should block exit. This is contrary to the requirements of the standard
and harmful to applications that have expectations on the atomicity/integrity
of stdio operations performed under lock.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug stdio/15142] Missing locking in _IO_cleanup

[schwab@linux-m68k.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=15142

--- Comment #2 from Andreas Schwab <schwab@linux-m68k.org> ---
Doesn't seem any recent progress on the issues.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug stdio/15142] Missing locking in _IO_cleanup

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=15142

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |fweimer at redhat dot com

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug stdio/15142] Missing locking in _IO_cleanup

[ppluzhnikov at google dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=15142

Paul Pluzhnikov <ppluzhnikov at google dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |ppluzhnikov at google dot com

--- Comment #4 from Paul Pluzhnikov <ppluzhnikov at google dot com> ---
*** Bug 30510 has been marked as a duplicate of this bug. ***

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug stdio/15142] Missing locking in _IO_cleanup

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=15142

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
              Flags|security?                   |security-
         Resolution|---                         |FIXED
   Target Milestone|---                         |2.38

--- Comment #5 from Florian Weimer <fweimer at redhat dot com> ---
Fixed for 2.38 via:

commit af130d27099651e0d27b2cf2cfb44dafd6fe8a26
Author: Andreas Schwab <schwab@suse.de>
Date:   Tue Jan 30 10:16:00 2018 +0100

    Always do locking when accessing streams (bug 15142, bug 14697)

    Now that abort no longer calls fflush there is no reason to avoid locking
    the stdio streams anywhere.  This fixes a conformance issue and potential
    heap corruption during exit.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug stdio/15142] Missing locking in _IO_cleanup

[dvyukov at google dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=15142

Dmitry Vyukov <dvyukov at google dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |dvyukov at google dot com

--- Comment #6 from Dmitry Vyukov <dvyukov at google dot com> ---
We started getting hangs on the following program:

https://github.com/llvm/llvm-project/blob/995d1d114e4e4ff708a03cdb0a975209c6197f9f/compiler-rt/test/tsan/getline_nohang.cpp#L28

Basically just calls a blocking getline in one thread and another thread tries
to exit.

Does it mean it's illegal to exit while there any blocking stream calls
anywhere in the program?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug stdio/15142] Missing locking in _IO_cleanup

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=15142

--- Comment #7 from Florian Weimer <fweimer at redhat dot com> ---
(In reply to Dmitry Vyukov from comment #6)
> We started getting hangs on the following program:
> 
> https://github.com/llvm/llvm-project/blob/
> 995d1d114e4e4ff708a03cdb0a975209c6197f9f/compiler-rt/test/tsan/
> getline_nohang.cpp#L28
> 
> Basically just calls a blocking getline in one thread and another thread
> tries to exit.

It's blocking on this:

  FILE *stream = fdopen(fd[0], "r");
  while (1) {
    volatile int res = getline(&line, &size, stream);
    (void)res;
  }

It's not a writable stream, so we could avoid the blocking with a more complex
handshake between stdio streams and exit. I'm not sure if it's worth doing
that.  We could perhaps add another flag to fopen/fdopen that indicates that
the stream should not participate in fflush (NULL) or exit flushing.

For streams which are blocked in writing, POSIX does not really give us a way
to make forward progress because we have to flush the unwritten data before
exiting.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug stdio/15142] Missing locking in _IO_cleanup

[dvyukov at google dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=15142

--- Comment #8 from Dmitry Vyukov <dvyukov at google dot com> ---
> For streams which are blocked in writing, POSIX does not really give us a way to make forward progress because we have to flush the unwritten data before exiting.

Is it really the case for this program?

If a write does not happen before exit (which is the case in any such
blocking), then program cannot potentially know the write has even started
before fflush/exit, so it cannot possibly expect the write side-effects to be
flushed.

What am I missing?

> We could perhaps add another flag to fopen/fdopen that indicates that the stream should not participate in fflush (NULL) or exit flushing.

Should we worry about all of the existing programs that will start hanging?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug stdio/15142] Missing locking in _IO_cleanup

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=15142

--- Comment #9 from Florian Weimer <fweimer at redhat dot com> ---
(In reply to Dmitry Vyukov from comment #8)
> > For streams which are blocked in writing, POSIX does not really give us a way to make forward progress because we have to flush the unwritten data before exiting.
> 
> Is it really the case for this program?

No, this program does not have any unflushed data to be written, hence my
comment about a more complex locking protocol avoiding the issue.

Exit flushing is special and not specified as equivalent to fflush (NULL), so
maybe it's sufficient to put read-only streams on a separate list, and flush
only writable streams on exit. But it's not clear to me if it's worth making
changes here if that only fixes this LLVM test case, and the real-world issues
are with applications exiting with pending unwritten data.

> If a write does not happen before exit (which is the case in any such
> blocking), then program cannot potentially know the write has even started
> before fflush/exit, so it cannot possibly expect the write side-effects to
> be flushed.
> 
> What am I missing?

There are cases where we must block according to POSIX. Lack of blocking is
observable by another process.

> > We could perhaps add another flag to fopen/fdopen that indicates that the stream should not participate in fflush (NULL) or exit flushing.
> 
> Should we worry about all of the existing programs that will start hanging?

Andreas Schwab wrote this:

“
This has been part of SUSE/openSUSE for several years, and I have not
seen any complaints so far.  It's more likely that you get a crash
during the unlocked access to the streams.
”

<https://inbox.sourceware.org/libc-alpha/mvmr0pptpmm.fsf@suse.de/>

This reduced my worries considerably.

-- 
You are receiving this mail because:
You are on the CC list for the bug.