libgcc: On AIX, increase chances to find landing pads for exceptions

libgcc: On AIX, increase chances to find landing pads for exceptions

[Michael Haubenwallner]

[-- Attachment #1: Type: text/plain, Size: 917 bytes --]

Hi David,

still experiencing exception-not-caught problems with gcc-4.2.4 on AIX
leads me to some patch proposed in http://gcc.gnu.org/PR13878 back in
2004 already, ought to be fixed by some different commit since 3.4.0.

As long as build systems (even libtool right now) on AIX do export these
_GLOBAL__* symbols from shared libraries, overlapping frame-base address
ranges may become registered, even if newer gcc (seen with 4.8) does name
the FDE symbols more complex to reduce these chances.

But still, just think of linking some static library into multiple shared
libraries and/or the main executable. Or sometimes there is just need for
some hackery to override a shared object's implementation detail and rely
on runtime linking to do the override at runtime.

Agreed both is "wrong" to some degree, but the larger an application is,
the higher is the chance for this to happen.

Thoughts?

Thanks!
/haubi/

[-- Attachment #2: libgcc-On-AIX-increase-chances-to-find-landing-pads-.patch --]
[-- Type: text/x-patch, Size: 1579 bytes --]

2016-02-08  Michael Haubenwallner  <michael.haubenwallner@ssi-schaefer.com>

	On AIX, increase chances to find landing pads for exceptions.
	* unwind-dw2-fde.c (_Unwind_Find_FDE): Stop assuming registered
	object's address ranges to not overlap.

--- libgcc/unwind-dw2-fde.c
+++ libgcc/unwind-dw2-fde.c
@@ -1013,7 +1013,25 @@ _Unwind_Find_FDE (void *pc, struct dwarf_eh_bases *bases)
 	f = search_object (ob, pc);
 	if (f)
 	  goto fini;
+	/*
 	break;
+	   In an ideal world, even on AIX, we could break here because
+	   objects would not overlap. But the larger an application is,
+	   the more likely an "overlap" may happen (on AIX) because of:
+	   - Shared libraries do export the FDE symbols ("_GLOBAL__F*"),
+	     which is a bug in their build system, but out of gcc's control.
+	   - Other shared libraries, or the main executable, do contain
+	     identical or similar object files - which is suboptimal,
+	     but may be intentional. However, exporting their FDE symbols,
+	     which may have identical names as seen in the former shared
+	     libraries, again is a bug in their build system, but still
+	     out of gcc's control.
+	   - Finally, run time linking is enabled, redirecting adresses of
+	     identically named exported symbols from their original shared
+	     library's address range into another shared library's or the
+	     main executable's address range.
+	   This results in address ranges being registered by different
+	   objects to potentially overlap.  */
       }
 
   /* Classify and search the objects we've not yet processed.  */

Re: libgcc: On AIX, increase chances to find landing pads for exceptions

[David Edelsohn]

Runtime linking is disabled by default on AIX, and I disabled it for libstdc++.

There are two remaining issues:

1) FDEs with overlapping ranges causing problems with exceptions.  I'm
not sure of the best way to work around this.  Your patch is one
possible solution.

2) AIX linker garbage collection conflicting with scanning for
symbols.  collect2 scanning needs to better emulate SVR4 linker
semantics for object files and archives.

Thanks, David


On Mon, Feb 8, 2016 at 7:14 AM, Michael Haubenwallner
<michael.haubenwallner@ssi-schaefer.com> wrote:
> Hi David,
>
> still experiencing exception-not-caught problems with gcc-4.2.4 on AIX
> leads me to some patch proposed in http://gcc.gnu.org/PR13878 back in
> 2004 already, ought to be fixed by some different commit since 3.4.0.
>
> As long as build systems (even libtool right now) on AIX do export these
> _GLOBAL__* symbols from shared libraries, overlapping frame-base address
> ranges may become registered, even if newer gcc (seen with 4.8) does name
> the FDE symbols more complex to reduce these chances.
>
> But still, just think of linking some static library into multiple shared
> libraries and/or the main executable. Or sometimes there is just need for
> some hackery to override a shared object's implementation detail and rely
> on runtime linking to do the override at runtime.
>
> Agreed both is "wrong" to some degree, but the larger an application is,
> the higher is the chance for this to happen.
>
> Thoughts?
>
> Thanks!
> /haubi/

Re: libgcc: On AIX, increase chances to find landing pads for exceptions

[Michael Haubenwallner]

On 02/08/2016 02:59 PM, David Edelsohn wrote:
> Runtime linking is disabled by default on AIX, and I disabled it for libstdc++.

For large applications mainly developed on/for Linux I do prefer/need
runtime linking even on AIX. Still I do believe there is no AIX-based
reason to leave runtime linking disabled, but build-/linktime issues
instead that cause things to fail with runtime linking enabled.

> There are two remaining issues:
> 
> 1) FDEs with overlapping ranges causing problems with exceptions.  I'm
> not sure of the best way to work around this.  Your patch is one
> possible solution.

This patch is not meant as a final solution, but to improve current
situation with broken build systems exporting even _GLOBAL__ symbols.
I'm about to prepare another libtool patch to fix that one.

> 2) AIX linker garbage collection conflicting with scanning for
> symbols.  collect2 scanning needs to better emulate SVR4 linker
> semantics for object files and archives.

Probably collect2 should filter the symbol list originating in either
an explicit -bexport:file or the -bexpall/-bexpfull flags and pass the
resulting symbol list as explicit -bexport:file only to the AIX linker?

/haubi/

> 
> Thanks, David
> 
> 
> On Mon, Feb 8, 2016 at 7:14 AM, Michael Haubenwallner
> <michael.haubenwallner@ssi-schaefer.com> wrote:
>> Hi David,
>>
>> still experiencing exception-not-caught problems with gcc-4.2.4 on AIX
>> leads me to some patch proposed in http://gcc.gnu.org/PR13878 back in
>> 2004 already, ought to be fixed by some different commit since 3.4.0.
>>
>> As long as build systems (even libtool right now) on AIX do export these
>> _GLOBAL__* symbols from shared libraries, overlapping frame-base address
>> ranges may become registered, even if newer gcc (seen with 4.8) does name
>> the FDE symbols more complex to reduce these chances.
>>
>> But still, just think of linking some static library into multiple shared
>> libraries and/or the main executable. Or sometimes there is just need for
>> some hackery to override a shared object's implementation detail and rely
>> on runtime linking to do the override at runtime.
>>
>> Agreed both is "wrong" to some degree, but the larger an application is,
>> the higher is the chance for this to happen.
>>
>> Thoughts?
>>
>> Thanks!
>> /haubi/

Re: libgcc: On AIX, increase chances to find landing pads for exceptions

[David Edelsohn]

On Wed, Feb 10, 2016 at 1:52 AM, Michael Haubenwallner
<michael.haubenwallner@ssi-schaefer.com> wrote:
>
> On 02/08/2016 02:59 PM, David Edelsohn wrote:
>> Runtime linking is disabled by default on AIX, and I disabled it for libstdc++.
>
> For large applications mainly developed on/for Linux I do prefer/need
> runtime linking even on AIX. Still I do believe there is no AIX-based
> reason to leave runtime linking disabled, but build-/linktime issues
> instead that cause things to fail with runtime linking enabled.

What do you mean by the term "runtime linking"?  Runtime linking means
runtime overloading of symbols -- preloading -- not dynamic linking
and loading.  dlopen does not require runtime linking.  There also are
issues with searching for shared objects with .a or .so file
extension, but that can be addressed separately.

Runtime linking causes every global, exported function call to be
invoked through indirect glue code.  And each function must be
inserted into the TOC.  The indirect call overhead is very expensive,
and potential TOC overflow can cause even more performance
degradation.

Your statement of no AIX-based reason to leave runtime linking
disabled is fundamentally flawed.

>
>> There are two remaining issues:
>>
>> 1) FDEs with overlapping ranges causing problems with exceptions.  I'm
>> not sure of the best way to work around this.  Your patch is one
>> possible solution.
>
> This patch is not meant as a final solution, but to improve current
> situation with broken build systems exporting even _GLOBAL__ symbols.
> I'm about to prepare another libtool patch to fix that one.
>
>> 2) AIX linker garbage collection conflicting with scanning for
>> symbols.  collect2 scanning needs to better emulate SVR4 linker
>> semantics for object files and archives.
>
> Probably collect2 should filter the symbol list originating in either
> an explicit -bexport:file or the -bexpall/-bexpfull flags and pass the
> resulting symbol list as explicit -bexport:file only to the AIX linker?

-bexpall and -bexpfull cause numerous problem by re-exporting symbols.

All of the suggestions will produce programs that function, but have
severe performance impacts and unintended consequences that you seem
to be ignoring.

- David

>
> /haubi/
>
>>
>> Thanks, David
>>
>>
>> On Mon, Feb 8, 2016 at 7:14 AM, Michael Haubenwallner
>> <michael.haubenwallner@ssi-schaefer.com> wrote:
>>> Hi David,
>>>
>>> still experiencing exception-not-caught problems with gcc-4.2.4 on AIX
>>> leads me to some patch proposed in http://gcc.gnu.org/PR13878 back in
>>> 2004 already, ought to be fixed by some different commit since 3.4.0.
>>>
>>> As long as build systems (even libtool right now) on AIX do export these
>>> _GLOBAL__* symbols from shared libraries, overlapping frame-base address
>>> ranges may become registered, even if newer gcc (seen with 4.8) does name
>>> the FDE symbols more complex to reduce these chances.
>>>
>>> But still, just think of linking some static library into multiple shared
>>> libraries and/or the main executable. Or sometimes there is just need for
>>> some hackery to override a shared object's implementation detail and rely
>>> on runtime linking to do the override at runtime.
>>>
>>> Agreed both is "wrong" to some degree, but the larger an application is,
>>> the higher is the chance for this to happen.
>>>
>>> Thoughts?
>>>
>>> Thanks!
>>> /haubi/

Re: libgcc: On AIX, increase chances to find landing pads for exceptions

[Michael Haubenwallner]

On 02/10/2016 02:27 PM, David Edelsohn wrote:
> On Wed, Feb 10, 2016 at 1:52 AM, Michael Haubenwallner
> <michael.haubenwallner@ssi-schaefer.com> wrote:
>>
>> On 02/08/2016 02:59 PM, David Edelsohn wrote:
>>> Runtime linking is disabled by default on AIX, and I disabled it for libstdc++.
>>
>> For large applications mainly developed on/for Linux I do prefer/need
>> runtime linking even on AIX. Still I do believe there is no AIX-based
>> reason to leave runtime linking disabled, but build-/linktime issues
>> instead that cause things to fail with runtime linking enabled.
> 
> What do you mean by the term "runtime linking"?  Runtime linking means
> runtime overloading of symbols -- preloading -- not dynamic linking
> and loading.  dlopen does not require runtime linking.  There also are
> issues with searching for shared objects with .a or .so file
> extension, but that can be addressed separately.

We're actually talking about the same "runtime linking" understanding.

> Runtime linking causes every global, exported function call to be
> invoked through indirect glue code.  And each function must be
> inserted into the TOC.  The indirect call overhead is very expensive,
> and potential TOC overflow can cause even more performance
> degradation.

Well, I'm in the lucky situation to not need to care for performance
that much. Don't get me wrong - I don't want to change any default
settings, but keep symbol-overloading at runtime just working if
requested for the final executable (by linking with -brtl).

What I indeed do not know: Is the performance overhead still active
even when the final executable is _not_ linked with -brtl, but the
shared libraries only are linked with -G ?

> Your statement of no AIX-based reason to leave runtime linking
> disabled is fundamentally flawed.

Agreed performance wise. But still I fail to see a functional reason.

>>> There are two remaining issues:
>>>
>>> 1) FDEs with overlapping ranges causing problems with exceptions.  I'm
>>> not sure of the best way to work around this.  Your patch is one
>>> possible solution.
>>
>> This patch is not meant as a final solution, but to improve current
>> situation with broken build systems exporting even _GLOBAL__ symbols.
>> I'm about to prepare another libtool patch to fix that one.
>>
>>> 2) AIX linker garbage collection conflicting with scanning for
>>> symbols.  collect2 scanning needs to better emulate SVR4 linker
>>> semantics for object files and archives.
>>
>> Probably collect2 should filter the symbol list originating in either
>> an explicit -bexport:file or the -bexpall/-bexpfull flags and pass the
>> resulting symbol list as explicit -bexport:file only to the AIX linker?
> 
> -bexpall and -bexpfull cause numerous problem by re-exporting symbols.
> 
> All of the suggestions will produce programs that function, but have
> severe performance impacts and unintended consequences that you seem
> to be ignoring.

Erm... I don't think so - what I mean is: Probably collect2 should _not_
passthrough the -bexpall/-bexpfull/-bexport:file flags to the linker,
but itself create an export file and pass this one _instead_.
However, this feels like a workaround for broken build systems - which
do provide the breaking -bexp* flags to gcc (collect2) already.

Thanks!
/haubi/

> 
> - David
> 
>>
>> /haubi/
>>
>>>
>>> Thanks, David
>>>
>>>
>>> On Mon, Feb 8, 2016 at 7:14 AM, Michael Haubenwallner
>>> <michael.haubenwallner@ssi-schaefer.com> wrote:
>>>> Hi David,
>>>>
>>>> still experiencing exception-not-caught problems with gcc-4.2.4 on AIX
>>>> leads me to some patch proposed in http://gcc.gnu.org/PR13878 back in
>>>> 2004 already, ought to be fixed by some different commit since 3.4.0.
>>>>
>>>> As long as build systems (even libtool right now) on AIX do export these
>>>> _GLOBAL__* symbols from shared libraries, overlapping frame-base address
>>>> ranges may become registered, even if newer gcc (seen with 4.8) does name
>>>> the FDE symbols more complex to reduce these chances.
>>>>
>>>> But still, just think of linking some static library into multiple shared
>>>> libraries and/or the main executable. Or sometimes there is just need for
>>>> some hackery to override a shared object's implementation detail and rely
>>>> on runtime linking to do the override at runtime.
>>>>
>>>> Agreed both is "wrong" to some degree, but the larger an application is,
>>>> the higher is the chance for this to happen.
>>>>
>>>> Thoughts?
>>>>
>>>> Thanks!
>>>> /haubi/

Re: libgcc: On AIX, increase chances to find landing pads for exceptions

[Michael Haubenwallner]

[-- Attachment #1: Type: text/plain, Size: 580 bytes --]

Hi David,

On 02/10/2016 10:52 AM, Michael Haubenwallner wrote:
<snip>
>> There are two remaining issues:
>>
>> 1) FDEs with overlapping ranges causing problems with exceptions.  I'm
>> not sure of the best way to work around this.  Your patch is one
>> possible solution.
> 
> This patch is not meant as a final solution, but to improve current
> situation with broken build systems exporting even _GLOBAL__ symbols.
> I'm about to prepare another libtool patch to fix that one.

so this is the libtool patch I'm about to submit.

What do you think? Reasonable?

Thanks!
/haubi/

[-- Attachment #2: 0001-AIX-stop-exporting-_GLOBAL__-syms.patch --]
[-- Type: text/x-patch, Size: 4223 bytes --]

From 18daacef170d0b0ad67869173cd521dc991d2264 Mon Sep 17 00:00:00 2001
From: Michael Haubenwallner <michael.haubenwallner@ssi-schaefer.com>
Date: Mon, 29 Feb 2016 16:07:32 +0100
Subject: [PATCH] AIX: stop exporting _GLOBAL__ syms

* m4/libtool.m4 (_LT_LINKER_SHLIBS): On AIX, GNU g++ generates
_GLOBAL__* symbols as landing pads for C++ exceptions.  These symbols
must not be exported from shared libraries, or exception handling may
break for applications with runtime linking enabled.
---
 m4/libtool.m4 | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/m4/libtool.m4 b/m4/libtool.m4
index ee292af..656d2cd 100644
--- a/m4/libtool.m4
+++ b/m4/libtool.m4
@@ -4939,9 +4939,9 @@ m4_if([$1], [CXX], [
     # it in the Import File for the 'aix-soname' feature, so we have
     # to replace the "-B" option with "-P" for AIX nm.
     if $NM -V 2>&1 | $GREP 'GNU' > /dev/null; then
-      _LT_TAGVAR(export_symbols_cmds, $1)='$NM -Bpg $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "W")) && ([substr](\$ 3,1,1) != ".")) { if (\$ 2 == "W") { print \$ 3 " weak" } else { print \$ 3 } } }'\'' | sort -u > $export_symbols'
+      _LT_TAGVAR(export_symbols_cmds, $1)='$NM -Bpg $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "W")) && ([substr](\$ 3,1,1) != ".") && ([substr](\$ 3,1,9) != "_GLOBAL__")) { if (\$ 2 == "W") { print \$ 3 " weak" } else { print \$ 3 } } }'\'' | sort -u > $export_symbols'
     else
-      _LT_TAGVAR(export_symbols_cmds, $1)='`func_echo_all $NM | $SED -e '\''s/B\([[^B]]*\)$/P\1/'\''` -PCpgl $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "L") || (\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) && ([substr](\$ 1,1,1) != ".")) { if ((\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) { print \$ 1 " weak" } else { print \$ 1 } } }'\'' | sort -u > $export_symbols'
+      _LT_TAGVAR(export_symbols_cmds, $1)='`func_echo_all $NM | $SED -e '\''s/B\([[^B]]*\)$/P\1/'\''` -PCpgl $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "L") || (\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) && ([substr](\$ 1,1,1) != ".") && ([substr](\$ 1,1,9) != "_GLOBAL__")) { if ((\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) { print \$ 1 " weak" } else { print \$ 1 } } }'\'' | sort -u > $export_symbols'
     fi
     ;;
   pw32*)
@@ -5394,9 +5394,9 @@ _LT_EOF
 	# it in the Import File for the 'aix-soname' feature, so we have
 	# to replace the "-B" option with "-P" for AIX nm.
 	if $NM -V 2>&1 | $GREP 'GNU' > /dev/null; then
-	  _LT_TAGVAR(export_symbols_cmds, $1)='$NM -Bpg $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "W")) && ([substr](\$ 3,1,1) != ".")) { if (\$ 2 == "W") { print \$ 3 " weak" } else { print \$ 3 } } }'\'' | sort -u > $export_symbols'
+	  _LT_TAGVAR(export_symbols_cmds, $1)='$NM -Bpg $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "W")) && ([substr](\$ 3,1,1) != ".") && ([substr](\$ 3,1,9) != "_GLOBAL__")) { if (\$ 2 == "W") { print \$ 3 " weak" } else { print \$ 3 } } }'\'' | sort -u > $export_symbols'
 	else
-	  _LT_TAGVAR(export_symbols_cmds, $1)='`func_echo_all $NM | $SED -e '\''s/B\([[^B]]*\)$/P\1/'\''` -PCpgl $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "L") || (\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) && ([substr](\$ 1,1,1) != ".")) { if ((\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) { print \$ 1 " weak" } else { print \$ 1 } } }'\'' | sort -u > $export_symbols'
+	  _LT_TAGVAR(export_symbols_cmds, $1)='`func_echo_all $NM | $SED -e '\''s/B\([[^B]]*\)$/P\1/'\''` -PCpgl $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "L") || (\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) && ([substr](\$ 1,1,1) != ".") && ([substr](\$ 1,1,9) != "_GLOBAL__")) { if ((\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) { print \$ 1 " weak" } else { print \$ 1 } } }'\'' | sort -u > $export_symbols'
 	fi
 	aix_use_runtimelinking=no
 
-- 
2.4.6

Re: libgcc: On AIX, increase chances to find landing pads for exceptions

[David Edelsohn]

On Tue, Mar 1, 2016 at 7:09 AM, Michael Haubenwallner
<michael.haubenwallner@ssi-schaefer.com> wrote:
> Hi David,
>
> On 02/10/2016 10:52 AM, Michael Haubenwallner wrote:
> <snip>
>>> There are two remaining issues:
>>>
>>> 1) FDEs with overlapping ranges causing problems with exceptions.  I'm
>>> not sure of the best way to work around this.  Your patch is one
>>> possible solution.
>>
>> This patch is not meant as a final solution, but to improve current
>> situation with broken build systems exporting even _GLOBAL__ symbols.
>> I'm about to prepare another libtool patch to fix that one.
>
> so this is the libtool patch I'm about to submit.
>
> What do you think? Reasonable?

I don't think that Ian really cares about this.

I guess that the patch is reasonable, but the libtool command is
becoming extremely complicated.

- David