libgcj/6390: java.lang.reflect.Array.newInstance doesn't deal out of memory issues correctly

libgcj/6390: java.lang.reflect.Array.newInstance doesn't deal out of memory issues correctly

[mark]

>Number:         6390
>Category:       libgcj
>Synopsis:       java.lang.reflect.Array.newInstance doesn't deal out of memory issues correctly
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    unassigned
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sun Apr 21 02:16:02 PDT 2002
>Closed-Date:
>Last-Modified:
>Originator:     mark@klomp.org
>Release:        gcj 3.1
>Organization:
>Environment:

>Description:
This comes from the following thread
http://gcc.gnu.org/ml/java/2002-04/msg00051.html

The following program (extracted from the mauve test) run under gdb
gives:

import java.lang.reflect.Array;
public class Big
{
  public static void main(String[] args)
  {
    String[][] t = (String[][]) Array.newInstance(String.class,
                    new int[] {Integer.MAX_VALUE, Integer.MAX_VALUE});
    System.out.println(t.length);
  }
}

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1024 (LWP 23056)]
0x40250020 in java::lang::Class::isPrimitive() (this=0x0)
    at ../../../gcc/libjava/java/lang/Class.h:208
208           return vtable == JV_PRIMITIVE_VTABLE;
Current language:  auto; currently c++
(gdb) bt
#0  0x40250020 in java::lang::Class::isPrimitive() (this=0x0)
    at ../../../gcc/libjava/java/lang/Class.h:208
#1  0x40222e89 in _Jv_NewMultiArrayUnchecked (type=0x80a2af0,
dimensions=1, 
    sizes=0x8085e7c) at ../../../gcc/libjava/prims.cc:541
#2  0x40222f09 in _Jv_NewMultiArrayUnchecked (type=0x80a2a10,
dimensions=2, 
    sizes=0x8085e78) at ../../../gcc/libjava/prims.cc:552
#3  0x40222fad in _Jv_NewMultiArray(java::lang::Class*, int, int*) (
    type=0x80a2a10, dimensions=2, sizes=0x8085e78)
    at ../../../gcc/libjava/prims.cc:566
#4  0x4025af14 in
java::lang::reflect::Array::newInstance(java::lang::Class*,
JArray<int>*) (componentType=0x8049350, dimensions=0x8085e70)
    at ../../../gcc/libjava/java/lang/reflect/natArray.cc:63
#5  0x08048ab7 in Big.main(java.lang.String[]) (args=0x8089fe8) at
Big.java:6

[...]

when I change the first value of the dimension array to
Integer.MAX_VALUE it gives the same result (silent run or SEGV under
gdb). Replacing the first value with Integer.MAX_VALUE-1 always gives a
SEGV (with or without gdb). And using just a huge value like 2000000000
actually does give OutOfMemoryError!

Small numbers {100,200} -> OK.
Big numbers {10000, 16000} -> Out of swap space, kernel kill.
Huge numbers {2000000000, 1000} -> OutOfMemoryError
Almost MAXINT {Integer.MAX_VALUE-1, Integer.MAX_VALUE} -> SEGV.
MAXINT number {Integer.MAX_VALUE, Intger.MAX_VALUE} -> Silent failure.
>How-To-Repeat:

>Fix:

>Release-Note:
>Audit-Trail:
>Unformatted: