libgcj missing symbol __data_start

libgcj missing symbol __data_start

[Andreas Krebbel]

Hi Andrew,

I see a lot of libjava fails on s390 when the -findirect-dispatch
switch is used. The testcases fail with:

/gcc-build/s390x-ibm-linux-gnu/libjava/testsuite/anonarray.exe: symbol
lookup error:
/gcc-build/s390x-ibm-linux-gnu/libjava/.libs/libgcj.so.11: undefined
symbol: __data_start

This symbol used to be in the .dynsym section of the main executable
after linking. I think it was exported since libgcj.so requires
it. But with your change libgcj_bc.so does not depend on libgcj.so
anymore and since the link step does not directly use libgcj.so it is
not loaded while linking the main executable what seems to lead to the
__data_start symbol to be ommitted from the .dynsym section.

rev 150852:
ldd libgcj_bc.so
        libc.so.6 => /lib64/libc.so.6 (0x0000020000020000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x0000020000182000)
        /lib/ld64.so.1 (0x000002aaaaaaa000)

rev 150853:
ldd libgcj_bc.so
        libgcj.so.11 => /home/krebbel/build/gcc-build/s390x-ibm-linux-gnu/libjava/.libs/libgcj.so.11 (0x0000020000005000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00000200033bc000)
        librt.so.1 => /lib64/librt.so.1 (0x00000200033d7000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00000200033e2000)
        libz.so.1 => /lib64/libz.so.1 (0x00000200033e7000)
        libc.so.6 => /lib64/libc.so.6 (0x00000200033ff000)
        libgcc_s.so.1 => /home/krebbel/build/gcc-install/lib/../lib64/libgcc_s.so.1 (0x0000020003562000)
        /lib/ld64.so.1 (0x000002aaaaaaa000)

Does that make sense to you?  Could you please have a look if so?

I've verified that revision 150853 introduced the failures.

Bye,

-Andreas-

Re: libgcj missing symbol __data_start

[Andrew Haley]

Andreas Krebbel wrote:
> Hi Andrew,
> 
> I see a lot of libjava fails on s390 when the -findirect-dispatch
> switch is used. The testcases fail with:
> 
> /gcc-build/s390x-ibm-linux-gnu/libjava/testsuite/anonarray.exe: symbol
> lookup error:
> /gcc-build/s390x-ibm-linux-gnu/libjava/.libs/libgcj.so.11: undefined
> symbol: __data_start
> 
> This symbol used to be in the .dynsym section of the main executable
> after linking. I think it was exported since libgcj.so requires
> it. But with your change libgcj_bc.so does not depend on libgcj.so
> anymore and since the link step does not directly use libgcj.so it is
> not loaded while linking the main executable what seems to lead to the
> __data_start symbol to be ommitted from the .dynsym section.
> 
> rev 150852:
> ldd libgcj_bc.so
>         libc.so.6 => /lib64/libc.so.6 (0x0000020000020000)
>         libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x0000020000182000)
>         /lib/ld64.so.1 (0x000002aaaaaaa000)
> 
> rev 150853:
> ldd libgcj_bc.so
>         libgcj.so.11 => /home/krebbel/build/gcc-build/s390x-ibm-linux-gnu/libjava/.libs/libgcj.so.11 (0x0000020000005000)
>         libpthread.so.0 => /lib64/libpthread.so.0 (0x00000200033bc000)
>         librt.so.1 => /lib64/librt.so.1 (0x00000200033d7000)
>         libdl.so.2 => /lib64/libdl.so.2 (0x00000200033e2000)
>         libz.so.1 => /lib64/libz.so.1 (0x00000200033e7000)
>         libc.so.6 => /lib64/libc.so.6 (0x00000200033ff000)
>         libgcc_s.so.1 => /home/krebbel/build/gcc-install/lib/../lib64/libgcc_s.so.1 (0x0000020003562000)
>         /lib/ld64.so.1 (0x000002aaaaaaa000)
> 
> Does that make sense to you?  Could you please have a look if so?
> 
> I've verified that revision 150853 introduced the failures.

Hard to say; I have no idea why s/390 should be any different from any
of the other systems.  We have a s/390 here at Red Hat but it might be
easier if there's a way I could log onto your system and have a look.

Andrew.

Re: libgcj missing symbol __data_start

[Andreas Krebbel]

> Hard to say; I have no idea why s/390 should be any different from any
> of the other systems.  We have a s/390 here at Red Hat but it might be
> easier if there's a way I could log onto your system and have a look.
No. It is unfortunately impossible to give you access to one of our 
systems for various reasons. But I tried to track it down:

On x86_64 the __data_start is declared weak:
[boehm-gc]$ readelf -s .libs/libgcjgc_convenience.a | grep __data_start
     80: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND __data_start

On s390x the symbol is declared global. The difference seem to come from
boehm-gc/include/private/gcconfig.h which uses pragma weak for x86 but 
not for S/390.

Since the symbol is weak on x86 the __data_start symbol missing in the 
main executable does not lead to an error.

The following patch fixes all the failures on s390x. But I'm not sure if 
that's the right thing to do:

Index: boehm-gc/include/private/gcconfig.h
===================================================================
*** boehm-gc/include/private/gcconfig.h.orig    2008-12-02 
14:08:42.000000000 +0100
--- boehm-gc/include/private/gcconfig.h 2009-08-26 17:07:12.000000000 +0200
***************
*** 1826,1831 ****
--- 1826,1832 ----
   #       define OS_TYPE "LINUX"
   #       define LINUX_STACKBOTTOM
   #       define DYNAMIC_LOADING
+ #       pragma weak __data_start
         extern int __data_start[];
   #       define DATASTART ((ptr_t)(__data_start))
       extern int _end[];


Bye,

-Andreas-

RE: libgcj missing symbol __data_start

[Boehm, Hans]

[Adding the gc mailing list, since this still applies to current gc versions.] 

> -----Original Message-----
> From: Andreas Krebbel
> Sent: Wednesday, August 26, 2009 9:33 AM
> 
> > Hard to say; I have no idea why s/390 should be any 
> different from any 
> > of the other systems.  We have a s/390 here at Red Hat but 
> it might be 
> > easier if there's a way I could log onto your system and 
> have a look.
> No. It is unfortunately impossible to give you access to one 
> of our systems for various reasons. But I tried to track it down:
> 
> On x86_64 the __data_start is declared weak:
> [boehm-gc]$ readelf -s .libs/libgcjgc_convenience.a | grep 
> __data_start
>      80: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND 
> __data_start
> 
> On s390x the symbol is declared global. The difference seem 
> to come from boehm-gc/include/private/gcconfig.h which uses 
> pragma weak for x86 but not for S/390.
> 
> Since the symbol is weak on x86 the __data_start symbol 
> missing in the main executable does not lead to an error.
> 
> The following patch fixes all the failures on s390x. But I'm 
> not sure if that's the right thing to do:
> 
> Index: boehm-gc/include/private/gcconfig.h
> ===================================================================
> *** boehm-gc/include/private/gcconfig.h.orig    2008-12-02 
> 14:08:42.000000000 +0100
> --- boehm-gc/include/private/gcconfig.h 2009-08-26 
> 17:07:12.000000000 +0200
> ***************
> *** 1826,1831 ****
> --- 1826,1832 ----
>    #       define OS_TYPE "LINUX"
>    #       define LINUX_STACKBOTTOM
>    #       define DYNAMIC_LOADING
> + #       pragma weak __data_start
>          extern int __data_start[];
>    #       define DATASTART ((ptr_t)(__data_start))
>        extern int _end[];
> 
> 
> Bye,
> 
> -Andreas-
> 
Unfortunately, I don't think this is the right solution.  If I understand correctly, DATASTART will end up as zero, which will cause the collector to start tracing at location zero, which seems unlikely to work, if the value is needed.  I suspect the value isn't actually used when dl_iterate_phdr is available and the application is dynamically linked, but still ...

Another popular option on other architectures is to define SEARCH_FOR_DATA_START instead of defining DATASTART to have a specific value.  This creates a weak reference to __data_start.  If that ends up being zero, the collector will walk the address space backwards starting at __end until an access faults.  It will assume that that's the beginning of the main data segment.  That might make debugging a bit harder because of the extra SIGSEGV at startup, but I still think it's a much better solution, assuming it actually works.  If it doesn't, this at least needs a conspicuous FIXME comment, since it's pretty clearly doing the wrong thing.  In any case, I think this should also go in the upstream source.

Hans

Re: libgcj missing symbol __data_start

[BGB]

----- Original Message ----- 
From: "Boehm, Hans" <hans.boehm@hp.com>
To: "Andreas Krebbel" <krebbel@linux.vnet.ibm.com>; "Andrew Haley" 
<aph@redhat.com>
Cc: <gcc-patches@gcc.gnu.org>; <java@gcc.gnu.org>; <gc@linux.hpl.hp.com>
Sent: Wednesday, August 26, 2009 10:41 AM
Subject: RE: libgcj missing symbol __data_start


> [Adding the gc mailing list, since this still applies to current gc 
> versions.]

<snip>

<--

Unfortunately, I don't think this is the right solution.  If I understand 
correctly, DATASTART will end up as zero, which will cause the collector to 
start tracing at location zero, which seems unlikely to work, if the value 
is needed.  I suspect the value isn't actually used when dl_iterate_phdr is 
available and the application is dynamically linked, but still ...

Another popular option on other architectures is to define 
SEARCH_FOR_DATA_START instead of defining DATASTART to have a specific 
value.  This creates a weak reference to __data_start.  If that ends up 
being zero, the collector will walk the address space backwards starting at 
__end until an access faults.  It will assume that that's the beginning of 
the main data segment.  That might make debugging a bit harder because of 
the extra SIGSEGV at startup, but I still think it's a much better solution, 
assuming it actually works.  If it doesn't, this at least needs a 
conspicuous FIXME comment, since it's pretty clearly doing the wrong thing. 
In any case, I think this should also go in the upstream source.

Hans

-->


although not directly related, I will note that for my GC on Windows, I 
ended up switching to a strategy of using the ToolHelp library to iterate 
over the loaded modules and find their used address ranges...

it also makes the GC work more reliably as well...

I am not sure if something similar can be done on Linux (I have not looked 
into this).

Re: libgcj missing symbol __data_start

[Andrew Haley]

Andreas Krebbel wrote:
> Hi Andrew,
> 
> I see a lot of libjava fails on s390 when the -findirect-dispatch
> switch is used. The testcases fail with:
> 
> /gcc-build/s390x-ibm-linux-gnu/libjava/testsuite/anonarray.exe: symbol
> lookup error:
> /gcc-build/s390x-ibm-linux-gnu/libjava/.libs/libgcj.so.11: undefined
> symbol: __data_start
> 
> This symbol used to be in the .dynsym section of the main executable
> after linking. I think it was exported since libgcj.so requires
> it. But with your change libgcj_bc.so does not depend on libgcj.so
> anymore and since the link step does not directly use libgcj.so it is
> not loaded while linking the main executable

That's the idea, yes.

> what seems to lead to the
> __data_start symbol to be ommitted from the .dynsym section.
> 
> rev 150852:
> ldd libgcj_bc.so
>         libc.so.6 => /lib64/libc.so.6 (0x0000020000020000)
>         libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x0000020000182000)
>         /lib/ld64.so.1 (0x000002aaaaaaa000)
> 
> rev 150853:
> ldd libgcj_bc.so
>         libgcj.so.11 => /home/krebbel/build/gcc-build/s390x-ibm-linux-gnu/libjava/.libs/libgcj.so.11 (0x0000020000005000)
>         libpthread.so.0 => /lib64/libpthread.so.0 (0x00000200033bc000)
>         librt.so.1 => /lib64/librt.so.1 (0x00000200033d7000)
>         libdl.so.2 => /lib64/libdl.so.2 (0x00000200033e2000)
>         libz.so.1 => /lib64/libz.so.1 (0x00000200033e7000)
>         libc.so.6 => /lib64/libc.so.6 (0x00000200033ff000)
>         libgcc_s.so.1 => /home/krebbel/build/gcc-install/lib/../lib64/libgcc_s.so.1 (0x0000020003562000)
>         /lib/ld64.so.1 (0x000002aaaaaaa000)
> 
> Does that make sense to you?  Could you please have a look if so?

I can make no sense of this at all.  When I build a simple executable,
before and after my patch, I get:

happy:~ $ /home/aph/gcc/trunk/install/bin/gcj Hello.java --main=Hello -Wl,-Map,map -findirect-dispatch
happy:~ $ readelf -D -s a.out | grep __data_start
    2   2: 00000000006013e0     0  NOTYPE GLOBAL DEFAULT  24 __data_start

Looking in the linker map, I see

.data           0x00000000006013e0      0x198
 *(.data .data.* .gnu.linkonce.d.*)
 .data          0x00000000006013e0        0x4 /usr/lib/../lib64/crt1.o
                0x00000000006013e0                __data_start
                0x00000000006013e0                data_start

So, someone is using __data_start, but I don't know who.

Please try this.

Andrew.


2009-09-01  Andrew Haley  <aph@redhat.com>

	* libgcj_bc.c (__data_start, data_start, _end): Add dummy usage.

Index: libgcj_bc.c
===================================================================
--- libgcj_bc.c	(revision 150834)
+++ libgcj_bc.c	(working copy)
@@ -112,3 +112,14 @@
 DECLARE_PRIM_TYPE(float)
 DECLARE_PRIM_TYPE(double)
 DECLARE_PRIM_TYPE(void)
+
+
+/* Force executable to export __data_start et al.  */
+
+#pragma weak __data_start
+extern int __data_start[];
+#pragma weak data_start
+extern int data_start[];
+#pragma weak _end
+extern int _end[];
+static void *dummy[] __attribute__((used)) = {__data_start, data_start, _end};

Re: libgcj missing symbol __data_start

[Andreas Krebbel]

Hi Andrew,

your patch fixes the failures on S/390.

Thanks.

Bye,

-Andreas-


> 2009-09-01  Andrew Haley  <aph@redhat.com>
> 
> 	* libgcj_bc.c (__data_start, data_start, _end): Add dummy usage.
> 
> Index: libgcj_bc.c
> ===================================================================
> --- libgcj_bc.c	(revision 150834)
> +++ libgcj_bc.c	(working copy)
> @@ -112,3 +112,14 @@
>  DECLARE_PRIM_TYPE(float)
>  DECLARE_PRIM_TYPE(double)
>  DECLARE_PRIM_TYPE(void)
> +
> +
> +/* Force executable to export __data_start et al.  */
> +
> +#pragma weak __data_start
> +extern int __data_start[];
> +#pragma weak data_start
> +extern int data_start[];
> +#pragma weak _end
> +extern int _end[];
> +static void *dummy[] __attribute__((used)) = {__data_start, data_start, _end};
> 

Re: libgcj missing symbol __data_start

[Andrew Haley]

Andrew Haley wrote:

> 
> 2009-09-01  Andrew Haley  <aph@redhat.com>
> 
> 	* libgcj_bc.c (__data_start, data_start, _end): Add dummy usage.
> 
> Index: libgcj_bc.c
> ===================================================================
> --- libgcj_bc.c	(revision 150834)
> +++ libgcj_bc.c	(working copy)
> @@ -112,3 +112,14 @@
>  DECLARE_PRIM_TYPE(float)
>  DECLARE_PRIM_TYPE(double)
>  DECLARE_PRIM_TYPE(void)
> +
> +
> +/* Force executable to export __data_start et al.  */
> +
> +#pragma weak __data_start
> +extern int __data_start[];
> +#pragma weak data_start
> +extern int data_start[];
> +#pragma weak _end
> +extern int _end[];
> +static void *dummy[] __attribute__((used)) = {__data_start, data_start, _end};

I committed this.

Andrew.