libgcj/1138: Blocking I/O calls cause other threads to lock

libgcj/1138: Blocking I/O calls cause other threads to lock

[bryce]

>Number:         1138
>Category:       libgcj
>Synopsis:       Blocking I/O calls cause other threads to lock
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    tromey
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Dec 20 12:06:31 PST 2000
>Closed-Date:    Fri Nov 12 16:32:40 PST 1999
>Last-Modified:  Thu Mar  2 14:40:00 PST 2000
>Originator:     Bryce McKinlay
>Release:        libgcj 99-07-13 snapshot
>Organization:
>Environment:
Redhat Linux 6.0, kernel 2.2.7, glibc 2.1.1-6
>Description:
If a thread enters a blocking IO call [anything that 
eventually calls FileDescription.read(), for example], and
that call blocks, ALL other threads are prevented from 
executing properly (!). Among other things, new threads do
not get created until the blocking read() call returns. 
>How-To-Repeat:
This code demonstrates the problem. If things are working 
right, it should print "this is okay" and exit. If not, it
will just hang until a key is pressed because the second
thread will not be created until the first unblocks. 

(Sorry that this example is not more concise!)

import java.io.*;

public class BlockThread implements Runnable
{
  private InputStream is;
  
  public static void main(String args[])
  {
    try
      {
	BlockThread t = new BlockThread( System.in );
	// give the BlockThread lots of time to start up
	Thread.sleep(100);
	BlockThread t2 = new BlockThread( null );
      }
    catch (Exception x) {};
  }

  public BlockThread (InputStream is)
  {
    // create a thread that blocks on reading the given input stream
    this.is = is;
    Thread t = new Thread(this);
    t.start();
  }
  
  public void run()
  {
    if (is == null)
      {
        System.out.println("This is okay. Exiting.");
	System.exit(0);
      }
      
    try
      {
	InputStreamReader isr = new InputStreamReader(is);
	PrintWriter out = new PrintWriter(System.out);
	char[] buffer = new char[100];
	while (true)
	  {
	    System.out.println(this + " is about to block");
	    int len = isr.read(buffer, 0, buffer.length);
	    System.out.println(this + " has unblocked");
	    if (len < 0) 
	      break;
	    out.write(buffer, 0, len);
	  }  
      }
    catch (Exception x) {}  
  }
}
>Fix:
Replacing the calls to read() in natFileDescriptorPosix.cc with
calls to "__read()" fixes the problem. I'm not sure what the
differences between read() and __read() are supposed to be,
but I'm guessing that using __read() forces the 
non-thread-aware version of the function to be used. Is this
another case of LinuxThreads being broken??

99/7/16 -
This is a problem with gc/linuxthreads interraction. 

Possible workarounds: 

1) linking pthreads before gcjgc (in libgcj.spec)
2) commenting out "#define MPROTECT_VDB" at line 593 in
boehm-gc/config.h (from Jeff Sturm)

We need a better fix for this - newwer version of gc 
perhaps?
>Release-Note:

>Audit-Trail:

Formerly PR libgcj/13

Responsible-Changed-From-To: apbianco->tromey
Responsible-Changed-By: bryce
Responsible-Changed-When: Thu Jul 15 18:47:40 1999
Responsible-Changed-Why:
Incorrectly filed the first time
State-Changed-From-To: open->feedback
State-Changed-By: tromey
State-Changed-When: Mon Jul 19 12:28:01 1999
State-Changed-Why:
I've fixed this by commenting out the MPROTECT_VDB
define.  Please verify that this works and get back
to me.  Thanks.
I did this on both the 2.95 branch and the trunk.

From: Tom Tromey <tromey@cygnus.com>
To: Gene Zilberstein <zilber@jps.net>
Cc: java-gnats@sourceware.cygnus.com, tromey@cygnus.com, bryce@albatross.co.nz,
        green@cygnus.com
Subject: Re: libgcj/13
Date: Sun, 26 Sep 1999 12:26:58 -0700

 >>>>> "Gene" == Gene Zilberstein <zilber@jps.net> writes:
 
 Gene> This is broken on freebsd 3.x as well...
 
 I didn't know we had a freebsd port of libgcj.
 Without seeing the port I can't really comment on why this bug might
 occur there.  On Linux the bug was in the GC.
 
 T
 
State-Changed-From-To: feedback->closed
State-Changed-By: tromey
State-Changed-When: Fri Nov 12 16:32:39 1999
State-Changed-Why:
    This has been in feedback for months.
    I assume it is fixed.

From: tromey@cygnus.com
To: bryce@albatross.co.nz, java-gnats@sourceware.cygnus.com, tromey@cygnus.com
Cc:  
Subject: Re: libgcj/13
Date: 13 Nov 1999 00:32:40 -0000

 Synopsis: Blocking I/O calls cause other threads to lock
 
 State-Changed-From-To: feedback->closed
 State-Changed-By: tromey
 State-Changed-When: Fri Nov 12 16:32:39 1999
 State-Changed-Why:
     This has been in feedback for months.
     I assume it is fixed.
 
 http://sourceware.cygnus.com/cgi-bin/gnatsweb.pl?cmd=view&database=java&pr=13

From: John Stracke <francis@thibault.org>
To: java-gnats@sourceware.cygnus.com, tromey@cygnus.com, bryce@albatross.co.nz
Cc:  
Subject: Re: libgcj/13
Date: Thu, 02 Mar 2000 14:17:28 -0500

 http://sourceware.cygnus.com/cgi-bin/gnatsweb.pl?cmd=view&pr=13&database=java
 
 I tried the suggested workaround of linking in pthreads before gcjgc
 (in libgcj.spec), and it didn't work; my test program (below) got a
 coredump.
 
 import java.io.DataInputStream;
 import java.io.IOException;
 
 public class Thread1 implements Runnable {
     public void run()
     {
  System.err.println("Thread1");
  DataInputStream in=new DataInputStream(System.in);
  try {
      String line=in.readLine();
  } catch (IOException io) {
      System.err.println(io);
  }
     }
 
     public static void main(String args[])
     {
  Thread t=new Thread(new Thread1());
  t.start();
  System.err.println("Thread2");
     }
 };
 
 --
 /================================================================\
 |John Stracke        | http://www.thibault.org |S/MIME, HTML OK  |
 |Francois Thibault   |===========================================|
 |Bhakail, East       |"The Elliot 903 was the first portable I   |
 |francis@thibault.org|ever used, though it needed a furniture van|
 |                    |to *make* it portable." --Will Rose        |
 \================================================================/
 
 
 

From: Bryce McKinlay <bryce@albatross.co.nz>
To: John Stracke <francis@thibault.org>
Cc: tromey@cygnus.com, java-gnats@sourceware.cygnus.com
Subject: Re: libgcj/13
Date: Fri, 03 Mar 2000 09:12:35 +1300

 John,
 
 What version of gcj & libgcj are you using? And what platform?
 
 Your test program works fine for me on Redhat 6.1 (x86) using recent snapshots.
 
 PR libgcj/13 was fixed a long time ago, and I doubt its the cause of your
 problem.
 
 regards
 
   [ bryce ]
 
 
 John Stracke wrote:
 
 >  I tried the suggested workaround of linking in pthreads before gcjgc
 >  (in libgcj.spec), and it didn't work; my test program (below) got a
 >  coredump.
 >
 >  import java.io.DataInputStream;
 >  import java.io.IOException;
 >
 >  public class Thread1 implements Runnable {
 >      public void run()
 >      {
 >   System.err.println("Thread1");
 >   DataInputStream in=new DataInputStream(System.in);
 >   try {
 >       String line=in.readLine();
 >   } catch (IOException io) {
 >       System.err.println(io);
 >   }
 >      }
 >
 >      public static void main(String args[])
 >      {
 >   Thread t=new Thread(new Thread1());
 >   t.start();
 >   System.err.println("Thread2");
 >      }
 >  };
 

From: John Stracke <francis@thibault.org>
To: Bryce McKinlay <bryce@albatross.co.nz>
Cc: tromey@cygnus.com, java-gnats@sourceware.cygnus.com
Subject: Re: libgcj/13
Date: Thu, 02 Mar 2000 15:30:34 -0500

 Bryce McKinlay wrote:
 
 > What version of gcj & libgcj are you using? And what platform?
 
 gcc 2.95.2, libgcj 2.95.1 (the most recent I could find), on RedHat 6.1, with no
 particular flags to configure except for --prefix (didn't want to make a mistake
 and blow away the bundled gcc).  Oh, and --enable-shared (for gcc, not for
 libgcj).
 
 I also tried it with --enable-threads=posix.
 
 > PR libgcj/13 was fixed a long time ago, and I doubt its the cause of your
 > problem.
 
 You're right; it seems to be a more general problem.  Here's a test case where the
 parent thread hangs until the child terminates, even though the child isn't doing
 any I/O.
 
 public class Thread2 implements Runnable {
     public void run()
     {
  try {
      Thread.sleep(100000);
  } catch (java.lang.InterruptedException ie) {
      System.err.println(ie);
  }
     }
 
     public static void main(String args[])
     {
  Thread t=new Thread(new Thread2());
  t.start();
  System.err.println("main thread");
     }
 };
 
 --
 /===============================================================\
 |John Stracke        | http://www.thibault.org |S/MIME, HTML OK |
 |Francois Thibault   |==========================================|
 |Bhakail, East       |"Your reality, sir, is lies & balderdash, |
 |francis@thibault.org|and I am pleased to say I have no grasp on|
 |                    |it whatsoever!" --Baron Munchausen        |
 \===============================================================/
 
 
 

From: Bryce McKinlay <bryce@albatross.co.nz>
To: John Stracke <francis@thibault.org>
Cc: tromey@cygnus.com, java-gnats@sourceware.cygnus.com
Subject: Re: libgcj/13
Date: Fri, 03 Mar 2000 09:42:48 +1300

 John Stracke wrote:
 
 > Bryce McKinlay wrote:
 >
 > > What version of gcj & libgcj are you using? And what platform?
 >
 > gcc 2.95.2, libgcj 2.95.1 (the most recent I could find), on RedHat 6.1, with no
 > particular flags to configure except for --prefix (didn't want to make a mistake
 > and blow away the bundled gcc).  Oh, and --enable-shared (for gcc, not for
 > libgcj).
 >
 > I also tried it with --enable-threads=posix.
 
 Even though 2.95.1 is our most recent release, it is very outdated. There are many
 fixes relating to threads and gc in libgcj cvs and snapshot releases. So far we
 havn't been able to do a new release because it depends on changes in the compiler.
 
 Try a snapshot from ftp://sourceware.cygnus.com/pub/java/snapshot
 
 You'll also need a snapshot gcc to compile it. If you don't want to use an unstable
 compiler, you can try my java patch for gcc-2.95.2 which is at
 http://waitaki.otago.ac.nz/~bryce/gcj
 
 
 > > PR libgcj/13 was fixed a long time ago, and I doubt its the cause of your
 > > problem.
 >
 > You're right; it seems to be a more general problem.  Here's a test case where the
 > parent thread hangs until the child terminates, even though the child isn't doing
 > any I/O.
 
 I don't see what the problem is here. Are you expecting the application to shut down
 when main() returns? The app should not exit until all non-daemon threads have died.
 
 regards
 
   [ bryce ]
 
 

From: John Stracke <francis@thibault.org>
To: Bryce McKinlay <bryce@albatross.co.nz>, tromey@cygnus.com,
        java-gnats@sourceware.cygnus.com
Cc:  
Subject: Re: libgcj/13
Date: Thu, 02 Mar 2000 15:47:12 -0500

 John Stracke wrote:
 
 > You're right; it seems to be a more general problem.  Here's a test case where the
 > parent thread hangs until the child terminates, even though the child isn't doing
 > any I/O.
 
 It doesn't seem to be because of the parent's I/O, either.  I ran it under gdb, and
 the call to t.start() blocked for the 100 seconds until run() returned:
 
 (gdb) break main__7Thread2Pt6JArray1ZPQ34java4lang6String
 Breakpoint 1 at 0x804b315: file Thread2.java, line 13.
 (gdb) run
 Starting program: /home/francis/work/experiments/gcj/threads/Thread2
 
 Breakpoint 1, Thread2.main (args=@806fff0) at Thread2.java:13
 13              Thread t=new Thread(new Thread2());
 Current language:  auto; currently java
 (gdb) next
 14              t.start();
 (gdb) shell date
 Thu Mar  2 15:44:25 EST 2000
 (gdb) next
 15              System.err.println("main thread");
 (gdb) shell date
 Thu Mar  2 15:46:07 EST 2000
 
 --
 /===============================================================\
 |John Stracke        | http://www.thibault.org |S/MIME, HTML OK |
 |Francois Thibault   |==========================================|
 |Bhakail, East       |The problem with any unwritten law is that|
 |francis@thibault.org|you don't know where to go to erase it.   |
 \===============================================================/
 
 
 

From: John Stracke <francis@thibault.org>
To: Bryce McKinlay <bryce@albatross.co.nz>
Cc: tromey@cygnus.com, java-gnats@sourceware.cygnus.com
Subject: Re: libgcj/13
Date: Thu, 02 Mar 2000 15:51:16 -0500

 Bryce McKinlay wrote:
 
 > Even though 2.95.1 is our most recent release, it is very outdated. There are many
 > fixes relating to threads and gc in libgcj cvs and snapshot releases. So far we
 > havn't been able to do a new release because it depends on changes in the compiler.
 >
 > Try a snapshot from ftp://sourceware.cygnus.com/pub/java/snapshot
 
 Thanks, I will.
 
 > I don't see what the problem is here. Are you expecting the application to shut down
 > when main() returns?
 
 No, the problem is that the println() doesn't execute until after run() finishes.  The
 order of operations should be "create thread t, t goes to sleep for 100 seconds, main
 thread does println(), eventually t wakes up and the program exits"; instead, it's
 "create thread t, t goes to sleep for 100 seconds, t wakes up and terminates, main
 thread does println() and the program exits".  In essence, there's no multithreading
 going on at all.
 
 --
 /==============================================================\
 |John Stracke        | http://www.thibault.org |S/MIME, HTML OK|
 |Francois Thibault   |=========================================|
 |Bhakail, East       |"We can't duplicate the bug." "Have you  |
 |francis@thibault.org|tried the Xerox machine?"                |
 \==============================================================/
 
 
 

From: Bryce McKinlay <bryce@albatross.co.nz>
To: John Stracke <francis@thibault.org>
Cc: tromey@cygnus.com, java-gnats@sourceware.cygnus.com
Subject: Re: libgcj/13
Date: Fri, 03 Mar 2000 10:24:06 +1300

 John Stracke wrote:
 
 > > I don't see what the problem is here. Are you expecting the application to shut down
 > > when main() returns?
 >
 > No, the problem is that the println() doesn't execute until after run() finishes.  The
 > order of operations should be "create thread t, t goes to sleep for 100 seconds, main
 > thread does println(), eventually t wakes up and the program exits"; instead, it's
 > "create thread t, t goes to sleep for 100 seconds, t wakes up and terminates, main
 > thread does println() and the program exits".  In essence, there's no multithreading
 > going on at all.
 
 OK. Make sure you configure both libgcj and gcc with "--enable-threads=posix". If that
 doesn't work then it must be a bug in 2.95.2. It works fine for me here.
 
 regards
 
   [ bryce ]
 
 
 

From: John Stracke <francis@thibault.org>
To: Bryce McKinlay <bryce@albatross.co.nz>
Cc: tromey@cygnus.com, java-gnats@sourceware.cygnus.com
Subject: Re: libgcj/13
Date: Thu, 02 Mar 2000 17:34:24 -0500

 Bryce McKinlay wrote:
 
 > OK. Make sure you configure both libgcj and gcc with "--enable-threads=posix".
 
 Yes, that works.  Sorry to trouble you.  (The gcc docs said that --enable-threads was only
 for Objective C, so I didn't try it.)
 
 Thanks!
 
 --
 /================================================================\
 |John Stracke        | http://www.thibault.org |S/MIME, HTML OK  |
 |Francois Thibault   |===========================================|
 |Bhakail, East       |"Hastur was paranoid, which was simply a   |
 |francis@thibault.org|sensible...well-adjusted reaction to living|
 |                    |in Hell." --_Good Omens_                   |
 \================================================================/
 
 
 
>Unformatted: