libgcj/5417: Illegal memory access in _Jv_IsAssignableFrom

libgcj/5417: Illegal memory access in _Jv_IsAssignableFrom

[msmith]

>Number:         5417
>Category:       libgcj
>Synopsis:       Illegal memory access in _Jv_IsAssignableFrom
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    unassigned
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Jan 17 14:26:01 PST 2002
>Closed-Date:
>Last-Modified:
>Originator:     Michael Smith
>Release:        gcc version 3.1 20010911 (experimental)
>Organization:
>Environment:
Linux msmith 2.4.7-10 #1 Thu Sep 6 17:27:27 EDT 2001 i686 unknown

host: i686-pc-linux-gnu
build: i686-pc-linux-gnu
target: i686-pc-linux-gnu

configured with: ../gcc/configure --host=i686-pc-linux-gnu --prefix=/home/msmith/GCC-3.1 --enable-shared --enable-threads=posix --enable-languages=c,c++,java --enable-version-specific-runtime-libs
>Description:
The compiler translates the "instanceof" operator into a call to _Jv_IsInstanceOf which gets the class object for the left hand side and checks to see if it is assignable to the right hand side using _Jv_IsAssignableFrom.  In neither method is there a check to ensure that the class has actually been initialized.  In circumstances where the class specified as the right hand operand of instanceof has not been initialized, the _Jv_IsAssignableFrom method will access parts of the class structure that have not been initialized.  Using electric fence exposes an access to unallocated memory.

It is not just enough to patch _Jv_IsInstanceOf to add a check to force the initialization of the class.  The problem would occur any place that _Jv_IsAssignableFrom is called with an uninitialized class.  Another such example is a cast. The compiler generates a _Jv_CheckCast, which also does not force an initialization of the class.
>How-To-Repeat:
The following demonstrates the illegal access using electric fence to generate a segv (which is turned into a NullPointerException) when _Jv_IsAssignableFrom accesses the uninitialized class.  In the second (1) and fourth (3) test cases, the class goes into an infinite loop generating segvs (segv causes exception to be constructed which causes segv, etc.).  I don't know why the casts should have such a different effect than the instanceof operator.  

// BEGIN FILE - Main.java

public class Main {
    public static void main(String[] args) throws Exception {
	int testNum = 0;
	if(args != null && args.length > 0) {
	    testNum = Integer.parseInt(args[0]);
	}
	switch(testNum) {
	case 0:
	    try {
		Test.testInstanceOf();
	    } catch (NullPointerException exception) {
		System.err.println("Illegal memory access for instanceof");
	    }
	    break;
	case 1:
	    try {
		Test.testCast();
	    } catch (NullPointerException exception) {
		System.err.println("Illegal memory access for cast");
	    }
	    break;
	case 2:
	    try {
		Test.testInstanceOf();
	    } catch (NullPointerException exception) {
		System.err.println
		    ("Illegal memory access for instanceof (interface)");
	    }
	    break;
	case 3:
	    try {
		Test.testCast();
	    } catch (NullPointerException exception) {
		System.err.println
		    ("Illegal memory access for interface cast");
	    }
	    break;
	}
    }
}

class Test {
    public static void testInstanceOf() {
	Object t = new Test();
	if(t instanceof Foo1) {
	    System.err.println(".");
	}
    }
    public static void testCast() {
	Object t = new Test();
	try {
	    if(((Foo2)t).equals(".")) {
		System.err.println(".");
	    }
	} catch (ClassCastException exception) {
	    // expected
	}
    }
    public static void testInstanceOfI() {
	Object t = new Test();
	if(t instanceof Bar2) {
	    System.err.println(".");
	}
    }
    public static void testCastI() {
	Object t = new Test();
	try {
	    if(((Bar2)t).equals(".")) {
		System.err.println(".");
	    }
	} catch (ClassCastException exception) {
	    // expected
	}
    }
}
class Foo1 {}
class Foo2 {}
interface Bar1 {}
interface Bar2 {}

// END FILE - Main.java

bash$ /home/msmith/GCC-3.1/bin/gcj --main=Main -o Test \
   Main.java
bash$ ./Test 0
bash$ ./Test 1
bash$ ./Test 2
bash$ ./Test 3
bash$ export LD_PRELOAD=/home/msmith/lib/libefence.so.0.0
bash$ ./Test 0

  Electric Fence 2.2.0 Copyright (C) 1987-1999 Bruce Perens <bruce@perens.com>
Illegal memory access for instanceof
bash$ ./Test 1

  Electric Fence 2.2.0 Copyright (C) 1987-1999 Bruce Perens 
^C
bash$ ./Test 2

  Electric Fence 2.2.0 Copyright (C) 1987-1999 Bruce Perens <bruce@perens.com>
Illegal memory access for instanceof (interface)
bash$ ./Test 3

  Electric Fence 2.2.0 Copyright (C) 1987-1999 Bruce Perens 
^C
>Fix:
Force initialization of the classes passed into _Jv_IsAssignableFrom:

Index: libjava/java/lang/natClass.cc
===================================================================
RCS file: /cvs/gcc/gcc/libjava/java/lang/natClass.cc,v
retrieving revision 1.44
diff -r1.44 natClass.cc
915a916,918
>   _Jv_InitClass(target);
>   _Jv_InitClass(source);
> 

OR, force initialization of the class passed into _Jv_IsInstanceOf and _Jv_CheckCast (the class associated with the object must have been initialized in order for the object to have been created.  Although this patch probably has better performance, I'm not convinced that other/future callers of _Jv_IsAssignableFrom will ensure the classes are initialized before calling that function.  Thereforce, the first patch (adding the initialization into _Jv_IsAssignableFrom) is probably preferrable.

Index: libjava/java/lang/natClass.cc
===================================================================
RCS file: /cvs/gcc/gcc/libjava/java/lang/natClass.cc,v
retrieving revision 1.44
diff -r1.44 natClass.cc
993a994,996
> 
>   _Jv_InitClass(cl);
> 
999a1003,1004
>   _Jv_InitClass(c);
> 

>Release-Note:
>Audit-Trail:
>Unformatted:

Re: libgcj/5417: Illegal memory access in _Jv_IsAssignableFrom

[rodrigc]

Synopsis: Illegal memory access in _Jv_IsAssignableFrom

State-Changed-From-To: open->closed
State-Changed-By: rodrigc
State-Changed-When: Sat Jan 19 10:29:03 2002
State-Changed-Why:
    Fixed in CVS.

http://gcc.gnu.org/cgi-bin/gnatsweb.pl?cmd=view%20audit-trail&database=gcc&pr=5417

Re: libgcj/5417: Illegal memory access in _Jv_IsAssignableFrom

[Bryce McKinlay]

The following reply was made to PR libgcj/5417; it has been noted by GNATS.

From: Bryce McKinlay <bryce@waitaki.otago.ac.nz>
To: msmith@spinnakernet.com, gcc-gnats@gcc.gnu.org, gcc-prs@gcc.gnu.org,
   gcc-bugs@gcc.gnu.org, nobody@gcc.gnu.org
Cc:  
Subject: Re: libgcj/5417: Illegal memory access in _Jv_IsAssignableFrom
Date: Fri, 18 Jan 2002 13:12:09 +1300

 http://gcc.gnu.org/cgi-bin/gnatsweb.pl?cmd=view%20audit-trail&database=gcc&pr=5417
 
 >
 > *Synopsis:* Illegal memory access in _Jv_IsAssignableFrom
 >
 > *Release:* gcc version 3.1 20010911 (experimental)
 >
 
 Please try this again with a more recent GCC from CVS - this should have 
 been fixed some time ago by:
 
 2001-09-30  Bryce McKinlay  <bryce@waitaki.otago.ac.nz>
 
     * java/lang/natClass.cc (_Jv_IsAssignableFrom): Handle the case of
     an uninitialized target class.
 
 regards
 
 Bryce.