egcs on AIX: native or gnu ld?

egcs on AIX: native or gnu ld?

[Andrey Slepuhin]

Hi,

Last week I had intensive testing of two builds of egcs-971008 on AIX
4.2.1 -
with gnu ld (from gas-970915) and with native ld (both with
--enable-shared option).
All my examples were compiled and executed fine with both installation
except
some questions.

1) what means gnu ld warning 

ld: warning: attempt to export undefined symbol `global constructors
keyed to f(void)'

when I tried to test -frepo option? Without this option I didn't get any
warnings.
And what means `global constructors keyed to f' in context of non-member
function f?
It seems that this warning does not affect program execution

2) It seems that I cannot use native linker with option -frepo, because
collect normally reads .rpo files but collect2 exits due to return code
8 of linker
(undefined symbols)

3) When using native linker to build shared libraries I permanently
receive
warnings about the following duplicate symbols (which are automatically
generated
during link stage):

ld: 0711-224 WARNING: Duplicate symbol: ._GLOBAL__DI
ld: 0711-224 WARNING: Duplicate symbol: ._GLOBAL__DD
ld: 0711-224 WARNING: Duplicate symbol: _GLOBAL__DI
ld: 0711-224 WARNING: Duplicate symbol: _GLOBAL__DD

What these symbols mean? I tried to look into source of automatically
generated
.c files, and it seems that these functions call something like module
initialization
functions (am I right?), so they differ in each shared object. I didn't
saw any
effect of such duplication, but I'm not sure that this is right thing.

4) I successfully build omniORB2 2.0 with both installations of egcs,
but in case
of using gnu linker the most of examples fail (segmentation fault). I
tried to
debug these examples but without success :-(. May be this is a bug of
omniORB,
but I'm not sure, because all works fine with native ld (both dynamic
and static
libraries) and with gnu ld (static libraries). The only idea I have is
that these
segmentation faults are in reference with using shared libraries in
multithreaded environment.

5) What about including support of shared libraries with native ld to
the future
snapshots of egcs? Now I am using hacked version of system script
makeC++SharedLib,
but it only needed for generating correct exports list. At least, I have
no
problems yet with shared libraries built with native ld.

6) Does anybody has an experience of building and using egcs on AIX? I
didn't saw
any messages in mailing lists (but may be they have no problems? :-))

Regards,

Andrey Slepuhin,
Moscow State University

Re: egcs on AIX: native or gnu ld?

[David Edelsohn]

	Most of your questions deal with G++ implementation issues which I
cannot address.  I really do not know how -frepo works or about additional
files it creates and how linkers other than GNU ld are suppose to handle
them.  It sounds like at least some of the features of G++ are relying
upon GNU ld or SVR4/ELF-like linkers.

	The _GLOBAL__DI/_GLOBAL_DD symbols invoke the global constructors
and destructors for a shared object and are exported for AIX.  Because
each shared object will contain those same entry points, they will be
duplicated and the AIX linker will warn about them.  In AIX 4, the linker
now announced a lot more warnings about these type of issues which always
were produced by GCC.

>>>>> Andrey Slepuhin writes:

| 5) What about including support of shared libraries with native ld to the
| future snapshots of egcs? Now I am using hacked version of system script
| makeC++SharedLib, but it only needed for generating correct exports
| list. At least, I have no problems yet with shared libraries built with
| native ld.

	To what type of support of shared libraries are you refering?
Specifically creating an export list?  Because AIX's XCOFF object file
format has finer granularity about the visibility of symbols than ELF, it
takes more work to make it behave like other Unix systems where all global
symbols are visible outside the module.

	I am not sure that the FSF would be allowed to include a modified
version of AIX XLC's makeC++SharedLib script, but if you were to
reimplement the functionality for G++ as makeG++SharedLib, it probably
could be included in the egcs distributions and installed automatically.

	AIX 4.2 also has features like specifying symbols automatically to
be invoked as constructors and destructors upon loading, but it uses a
special header section populated from the commandline instead of ctor/dtor
or init/fini object file sections of SVR4/ELF.  Also, the ordering of
invocation in the case of dependencies leaves something to be desired.  I
am not sure if it is worth the implementation / compatibility headaches.

David

Re: egcs on AIX: native or gnu ld?

[Joe Buck]

> 	Most of your questions deal with G++ implementation issues which I
> cannot address.  I really do not know how -frepo works or about additional
> files it creates and how linkers other than GNU ld are suppose to handle
> them.

-frepo is handled by the collect2 program, which then calls either the
GNU or the "native" linker.

>  It sounds like at least some of the features of G++ are relying
> upon GNU ld or SVR4/ELF-like linkers.

No, the collect program handles the constructor problems on other systems.
See the g++ FAQ for a discussion ( http://www.cygnus.com/misc/g++FAQ_toc.html ).
It may not be able to deal with global constructors in shared libraries,
though.

> | 5) What about including support of shared libraries with native ld to the
> | future snapshots of egcs?

g++ never worked with only a native ld on non-ELF systems.  Something
has to resolve the constructors.

If, for a particular platform other than ELF, there is a way to make
shared libraries work with the native linker, and someone wants to
contribute it, fine, but it shouldn't be a priority when the workaround
"use the GNU linker" exists.

Re: egcs on AIX: native or gnu ld?

[Andrey Slepuhin]

David Edelsohn wrote:
> 
>         Most of your questions deal with G++ implementation issues which I
> cannot address.  I really do not know how -frepo works or about additional
> files it creates and how linkers other than GNU ld are suppose to handle
> them.  It sounds like at least some of the features of G++ are relying
> upon GNU ld or SVR4/ELF-like linkers.

-frepo option applies also -fno-implicit-templates option and cause g++
to generate for each source file .rpo file with a list of needed templates.
The linker (or linker driver in the case of native linker) looks at this list
and at the result of link stage, marks what templates really must be
instantiated
in each module, then recompiles source modules and repeat the process until
success (i.e. until there will be no undefined symbols).
Well, I discover that passing -bnoquiet option to AIX linker I'll get better
results. The only problem is that I receive messages like the following:

  ld: 0711-319 WARNING: Exported symbol not defined: _GLOBAL_.I.f__Fv
  ld: 0711-318 ERROR: Undefined symbols were found.
          The following symbols are in error:
  Symbol                    Inpndx  TY CL Source-File(Object-File) OR      
Import-File{Shared-object}
                                RLD: Address  Section  Rld-type Referencing
Symbol
    
--------------------------------------------------------------------------------
  _GLOBAL_.I.f__Fv          [2]     ER DS ccMav3Ma.c(/tmp/ccMav3Ma.o)
                                     00000004 .data    R_POS    [16]   
__CTOR_LIST__
  ER: The return code is 8.

Manually relinking files once more I obtain correct exexutable (If extra pass
is needed why not to do it automatically?). Unfortunately, -frepo option is
bad documented :-(

>         The _GLOBAL__DI/_GLOBAL_DD symbols invoke the global constructors
> and destructors for a shared object and are exported for AIX.  Because
> each shared object will contain those same entry points, they will be
> duplicated and the AIX linker will warn about them.  In AIX 4, the linker
> now announced a lot more warnings about these type of issues which always
> were produced by GCC.

Well, I undestand this, but if I link my program with several shared objects,
can I be sure that global constructors/destructors code will be called for
each object?

> | 5) What about including support of shared libraries with native ld to the
> | future snapshots of egcs? Now I am using hacked version of system script
> | makeC++SharedLib, but it only needed for generating correct exports
> | list. At least, I have no problems yet with shared libraries built with
> | native ld.
> 
>         To what type of support of shared libraries are you refering?
> Specifically creating an export list?  Because AIX's XCOFF object file
> format has finer granularity about the visibility of symbols than ELF, it
> takes more work to make it behave like other Unix systems where all global
> symbols are visible outside the module.
> 
>         I am not sure that the FSF would be allowed to include a modified
> version of AIX XLC's makeC++SharedLib script, but if you were to
> reimplement the functionality for G++ as makeG++SharedLib, it probably
> could be included in the egcs distributions and installed automatically.

As I understand, the main goal of makeC++SharedLib script is to provide
correct calling of global constructors/destructors. In egcs this task is done
by linker driver, which generates extra code to do this. So, the only problem
is to generate correct export list, and makeC++SharedLib do this simply by
piping nm output to awk. This can be easy included in egcs linker driver.
Moreover, we do not need to omit some special symbol names (but may be we must
omit others? linker driver for AIX automatically exports some symbols,
but I don't know what rules it uses to do this).

But I am seriously worried about strange segmentation faults of omniORB linked
with shared libs built with gnu ld. For example, while debugging, I find
the following strange situation: in the statement

pd_rx_begin = (void*)((omni::ptr_arith_t )pd_rx_begin + total );
// omni::ptr_arith_t=int, total is int, pd_rx_begin is void*

(It doesn't look like as a good C++ code, but I'm not author of it :-)) after
a number of normal executions once the high byte of pd_rx_begin become zero
(all the time total=0) - so later this cause segmentation fault. This may be
caused by another thread, but effect is stable, and all works fine with static
libraries and also with shared libraries built with native linker. Any ideas?
And what about mixing shared libraries built with native ld (always libc,
for example) and shared libraries built with gnu ld?

Andrey.

Re: egcs on AIX: native or gnu ld?

[David Edelsohn]

>>>>> Joe Buck writes:

>> | 5) What about including support of shared libraries with native ld to the
>> | future snapshots of egcs?

Joe> g++ never worked with only a native ld on non-ELF systems.  Something
Joe> has to resolve the constructors.

	I don't think he meant native-ld alone.  I assumed that he meant
collect2+native-ld instead of gnu-ld.  Unfortunately, gnu-ld does not work
very well with AIX, particularly newer releases.  This is partially due to
the difficult that I have encountered trying to extract the design
documents from IBM's RS/6000 Division to ease Cygnus's engineering effort.
The internal details are not very well documented in AIX's online
documentation.

David

These are my own opinions; I am not speaking as an official representative
of IBM Corporation.
===============================================================================
David Edelsohn                                      T.J. Watson Research Center
dje@watson.ibm.com                                  P.O. Box 218
+1 914 945 4364 (TL 862)                            Yorktown Heights, NY 10598

Re: egcs on AIX: native or gnu ld?

[David Edelsohn]

>>>>> Andrey Slepuhin writes:

Andrey> ld: 0711-319 WARNING: Exported symbol not defined: _GLOBAL_.I.f__Fv
Andrey> ld: 0711-318 ERROR: Undefined symbols were found.
Andrey> _GLOBAL_.I.f__Fv          [2]     ER DS ccMav3Ma.c(/tmp/ccMav3Ma.o)

	I think that this is an individual constructor.

Andrey> Manually relinking files once more I obtain correct exexutable (If extra pass
Andrey> is needed why not to do it automatically?). Unfortunately, -frepo option is
Andrey> bad documented :-(

	XLC has a twolink option performs the link phase twice.  This
allows the IBM garbage-collecting linker to perform a pass and then
re-examine the global symbols needing export.  This frequently is useful
for static contructors.  What may happen is that the initial examination
recognizes the symbol as a candidate for export but the linker discovers
that the symbol is not referenced and removes it altogether which then
causes an error because the original pass included it in the list of
exports.  The two-pass version only exports those symbols present after
symbol garbage-collection.

>> The _GLOBAL__DI/_GLOBAL_DD symbols invoke the global constructors
>> and destructors for a shared object and are exported for AIX.  Because
>> each shared object will contain those same entry points, they will be
>> duplicated and the AIX linker will warn about them.  In AIX 4, the linker
>> now announced a lot more warnings about these type of issues which always
>> were produced by GCC.

Andrey> Well, I undestand this, but if I link my program with several shared objects,
Andrey> can I be sure that global constructors/destructors code will be called for
Andrey> each object?

	These GLOBAL__DI/DD symbols exist for each library.  collect2 and
gnu ld collect all of the static constructors to be run at program startup
-- both symbols within the program and symbols within any libraries.  The
GLOBAL__DI/DD symbols are present and duplicated in the executable because
they are present in each library as a defining instance, but they never
are referenced because collect2 has created a separate list of all of the
constructor symbols for the executable.

	Wilfried Philips and someone from IBM Austin previously posted
makeC++SharedLib-like scripts to create an export file.  I do not know if
you can use it (appended) combined with your own hacks and any necessary
multiple link phases automation to create a single patch for improved
shared library support by GCC/EGCS that you would like to submit for
inclusion in future releases.

David

----- Begin Included Text -----

From: Wilfried Philips <philips@ibmmbv2.elis.rug.ac.be>
Newsgroups: comp.unix.aix
Subject: Re: Dynamic Loading...
Date: 17 Feb 1997 18:40:40 +0100
Organization: ELIS-RUG (University of Gent, Belgium)

%%%%%%%%%%%mkexps%%%%%%%%%%%%%%%%%%%%%%
#!/bin/ksh
#
# mkexps - make export list
# This program creates an export list by combining all the "." and normal names
# into one list.
#
if [[ "$#" -ne 1 ]]
then
    print "Usage: mkexps ArchiveFile"
    exit -2
fi
if [[ ! -f $1 ]]
then
    print "mkexps: Cannot open file \"$1\""
    exit -1
fi

dump -g $1 | awk '
BEGIN {
        top = 1
}
/^[ ]*[0-9][0-9]*/ {
    if ( (n = index( $2, "." )) > 0 ) {
        export_array[ top++ ] = substr( $2, n+1, length( $2 ))
    }
    else {
	export_array[ top++ ] = $2
    }
}

END {
    for ( i = 1; i < top; i++ )
    {
	print export_array[ i ]
    }

}' | sort | uniq

%%%%%%%%%%%mkexps%%%%%%%%%%%%%%%%%%%%%%

*** OR ***

From: dosha@austin.ibm.com
Subject: Re: Dynamic Loading...
Date: Thu, 20 Feb 1997 17:30:29 -0600
Organization: IBM Austin

Try the following:

nm libtest.a | grep ' [BAD] ' | cut -f1 -d ' '  > test.exp

----- End Incuded Text -----
===============================================================================
David Edelsohn                                      T.J. Watson Research Center
dje@watson.ibm.com                                  P.O. Box 218
+1 914 945 4364 (TL 862)                            Yorktown Heights, NY 10598

Re: egcs on AIX: native or gnu ld?

[Andrey Slepuhin]

David Edelsohn wrote:
>         These GLOBAL__DI/DD symbols exist for each library.  collect2 and
> gnu ld collect all of the static constructors to be run at program startup
> -- both symbols within the program and symbols within any libraries.  The
> GLOBAL__DI/DD symbols are present and duplicated in the executable because
> they are present in each library as a defining instance, but they never
> are referenced because collect2 has created a separate list of all of the
> constructor symbols for the executable.

Well, GLOBAL__DI/DD symbols are automatically exported by collect2. But
what
about __eh_<something> symbols, which are defined for each object file
and
are global bss symbols? Should I export them or they can be omitted?

I'm now looking into collect2 source to find appropriate place for
patching.
I see that collect2 exports some symbols by direct parsing of object
files.
As I don't know XCOFF file format I'll try to create separate export
list by calling a script. This is not a good solution so it should be
fixed
later. Well, when I obtain working version I'll send a patch.

Andrey.

Re: egcs on AIX: native or gnu ld?

[Andrey Slepuhin]

David Edelsohn wrote:
> 
> >>>>> Andrey Slepuhin writes:
> 
> Andrey> ld: 0711-319 WARNING: Exported symbol not defined: _GLOBAL_.I.f__Fv
> Andrey> ld: 0711-318 ERROR: Undefined symbols were found.
> Andrey> _GLOBAL_.I.f__Fv          [2]     ER DS ccMav3Ma.c(/tmp/ccMav3Ma.o)
> 
>         I think that this is an individual constructor.
> 
> Andrey> Manually relinking files once more I obtain correct exexutable (If extra pass
> Andrey> is needed why not to do it automatically?). Unfortunately, -frepo option is
> Andrey> bad documented :-(
> 
>         XLC has a twolink option performs the link phase twice.  This
> allows the IBM garbage-collecting linker to perform a pass and then
> re-examine the global symbols needing export.  This frequently is useful
> for static contructors.  What may happen is that the initial examination
> recognizes the symbol as a candidate for export but the linker discovers
> that the symbol is not referenced and removes it altogether which then
> causes an error because the original pass included it in the list of
> exports.  The two-pass version only exports those symbols present after
> symbol garbage-collection.

Indeed, the error was caused by a fact that _GLOBAL_.I.f__Fv symbol is
defined
in my object file if I use default g++ flags and is undefined if I use
-frepo.
This symbol is referenced from __CTOR_LIST__ in automatically generated
by collect2
code. I don't understand why _GLOBAL_.I.f__Fv is absent in object file
when
I use -frepo, because this symbol is not connected with template
instantiation.

Andrey.

Re: egcs on AIX: native or gnu ld?

[Andrey Slepuhin]

In addition to my previous letter:
It seems that I find what's wrong with _GLOBAL_.I.f__Fv and -frepo.
I have the following in my test file:

#include ... //class ABCD efinition
static ABCD<int> x;
void f() {...}

Removing static or adding any other non-static global template object
solves the
problem (and in this case I have _GLOBAL_.I.<name of non-static global
template object>
instead of _GLOBAL_.I.f__Fv). I really do not understand the situation.

Andrey.

Re: egcs on AIX: native or gnu ld?

[David Edelsohn]

>>>>> Andrey Slepuhin writes:

Andrey> Well, GLOBAL__DI/DD symbols are automatically exported by collect2. But
Andrey> what
Andrey> about __eh_<something> symbols, which are defined for each object file
Andrey> and
Andrey> are global bss symbols? Should I export them or they can be omitted?

	You do not need to know the details of XCOFF objects, that already
is implemented in collect2.  I guess that I do not understand exception
handling and how __eh_* symbols are handled well enough to know if they
need to be exported.  Someone who understand the C++ implementation needs
to examine this.  A symbol needs to be exported if it will be referenced
by name outside its module.  If one module somehow gets a pointer to a
symbol in another loaded module through its own list of transfer vectors,
for example, export is irrelevant.  Only if the linker/loader needs to
resolve named symbols across modules does export become an issue.

	If C++ applications have the names of __eh_* symbols in other
modules compiled in, then it would seem that exports are necessary.  In
that case, one needs to expand collect2.c:scan_prog_file() to add __eh_*
symbols to the export list along with constructors and destructors.

	Again, I am not an expert on C++ internals and I do not know how
the __eh_* symbols interact with the other exception handling
implementations that have evolved.  PowerOpen/XCOFF ABI functions already
have traceback tables at the end of each function and IBM's XLC uses that
to walk the stack for exceptions.  This could be used in the same way as
the DWARF 2 frame unwind information for non-sjlj exception handling.  I
do not forsee having time to work on this at any time in the near future,
but it is an open project to improve AIX support.

David

Re: egcs on AIX: native or gnu ld?

[Mike Stump]

> To: mrs@wrs.com (Mike Stump)
> Cc: pooh@msu.ru, egcs@cygnus.com
> Date: Mon, 20 Oct 97 18:22:36 -0400
> From: David Edelsohn <dje@watson.ibm.com>

> >>>>> Mike Stump writes:

> [ ... ] I guess I do not understand your no regexs request.  [ ... ]
> However, the names are found essentially using a prefix regex.  How
> else can one determine that a symbol is an exception handler other
> than looking for __eh_*?

This isn't what these symbols are.  There are 6 or so symbols that
have specific names that were being referred to, (__eh_type,
__eh_pc...).  They are BSS common things.  They are also defined more
recently in libgcc.a.

Re: egcs on AIX: native or gnu ld?

[David Edelsohn]

>>>>> Mike Stump writes:

>> A symbol needs to be exported if it will be referenced by name
>> outside its module.
>> 
>> If C++ applications have the names of __eh_* symbols in other
>> modules compiled in, then it would seem that exports are necessary.  In
>> that case, one needs to expand collect2.c:scan_prog_file() to add __eh_*
>> symbols to the export list along with constructors and destructors.

Mike> Given the above, I say Add them (by exact name, no .* regexs please),
Mike> they need to be exported.

	I guess I do not understand your no regexs request.  Current
export are determined using is_ctor_dtor() function which looks for all
symbols starting with "GLOBAL_.I." or some variant.  Each ctor or dtor is
exported individually by name for a particular file, and one can use
multiple linking passes to ensure that only those symbols truly present in
the final shared library are exported.  However, the names are found
essentially using a prefix regex.  How else can one determine that a
symbol is an exception handler other than looking for __eh_*?

	Also, to further clarify my comment above so that we do not
unnecessarily go down this path (although there is not much harm in
implementing this and exporting more symbols), only those symbols which
have compiled references *by name* in another module need to be exported.
If you somehow have access to a root symbol for your own private list,
transfer vectors, or otherwise obtain pointers to the functions using your
own scheme to grovel for the symbol names, they do not need to be
exported.  Export simply determines if the symbol is externally visible
outside the module for the AIX linker or loader to handle the resolution.
If the library is loaded for another reason and already mapped in to the
user's address space, the symbols are present and are callable through
pointers if the process has its own private way to determin the addresses.

David

Re: egcs on AIX: native or gnu ld?

[Mike Stump]

> To: Andrey Slepuhin <pooh@msu.ru>
> Cc: egcs@cygnus.com
> Date: Mon, 20 Oct 97 13:36:49 -0400
> From: David Edelsohn <dje@watson.ibm.com>

> Andrey> Well, GLOBAL__DI/DD symbols are automatically exported by
> Andrey> collect2. But what about __eh_<something> symbols, which are
> Andrey> defined for each object file and are global bss symbols?
> Andrey> Should I export them or they can be omitted?

> I guess that I do not understand exception handling and how __eh_*
> symbols are handled well enough to know if they need to be exported.
> Someone who understand the C++ implementation needs to examine this.
> A symbol needs to be exported if it will be referenced by name
> outside its module.
>
> 	If C++ applications have the names of __eh_* symbols in other
> modules compiled in, then it would seem that exports are necessary.  In
> that case, one needs to expand collect2.c:scan_prog_file() to add __eh_*
> symbols to the export list along with constructors and destructors.

Given the above, I say Add them (by exact name, no .* regexs please),
they need to be exported.