[Bug] DWARF-5 section names in PE/PEP and weak symbols

[Bug] DWARF-5 section names in PE/PEP and weak symbols

[Achim Gratz]

The commit

--8<---------------cut here---------------start------------->8---
commit ba6eb62ff0ea9843a018cfd7cd06777bd66ae0a0
Author: Nick Clifton <nickc@redhat.com>
Date:   Mon Mar 1 16:25:06 2021 +0000

    Add DWARF-5 section names to PE and PEP linker scripts.
    
            PR 27268
            * scripttempl/pe.sc: Add DWARF-5 section names.
            * scripttempl/pep.sc: Likewise.
--8<---------------cut here---------------end--------------->8---

changed the behaviour of weak symbols in Cygwin.  The following test
from gnulib that should fail.  In other words fputs should have a symbol
value of 0x0 and the resulting executable should not be able to run
since the Windows loader does not resolve weak symbols):

--8<--weak.c-------cut here---------------start------------->8---
#include <stdio.h>
#pragma weak fputs
int main ()
{
  return (fputs == NULL);
}
--8<---------------cut here---------------end--------------->8---

$ gcc -o weak weak.c # w/ binutils 2.36.1
$ ./weak ; echo $?
./weak: ./weak: cannot execute binary file
126

Instead, starting with the above commit, the test succeeds since fputs
has the (expected) value from cygwin1.dll.  This then leads to
horrendous crashes in applications that attempt to actually use "real"
weak symbols based on the (probably too simplistic) test from gnulib:

$ gcc -o weak weak.c # w/ binutils 2.37
$ ./weak ; echo $?
0

I've narrowed it down to just one of the new DWARF5 sections and
removing them from the linker scripts restores the correct behaviour of
weak symbols in Cygwin (I have no idea if that triggers problems
elsewhere):

--8<---------------cut here---------------start------------->8---
diff --git a/ld/scripttempl/pe.sc b/ld/scripttempl/pe.sc
index e9d1f78cb32..2e64a681850 100644
--- a/ld/scripttempl/pe.sc
+++ b/ld/scripttempl/pe.sc
@@ -447,10 +447,12 @@ SECTIONS
   {
     *(.zdebug_line_str)
   }
+/* ===
   .debug_loclists ${RELOCATING+BLOCK(__section_alignment__)} ${RELOCATING+(NOLOAD)} :
   {
     *(.debug_loclists)
   }
+=== */
   .zdebug_loclists ${RELOCATING+BLOCK(__section_alignment__)} ${RELOCATING+(NOLOAD)} :
   {
     *(.zdebug_loclists)
diff --git a/ld/scripttempl/pep.sc b/ld/scripttempl/pep.sc
index afa2d23871c..dbb657c2090 100644
--- a/ld/scripttempl/pep.sc
+++ b/ld/scripttempl/pep.sc
@@ -452,10 +452,12 @@ SECTIONS
   {
     *(.zdebug_line_str)
   }
+/* ===
   .debug_loclists ${RELOCATING+BLOCK(__section_alignment__)} ${RELOCATING+(NOLOAD)} :
   {
     *(.debug_loclists)
   }
+=== */
   .zdebug_loclists ${RELOCATING+BLOCK(__section_alignment__)} ${RELOCATING+(NOLOAD)} :
   {
     *(.zdebug_loclists)
--8<---------------cut here---------------end--------------->8---


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Waldorf MIDI Implementation & additional documentation:
http://Synth.Stromeko.net/Downloads.html#WaldorfDocs

Re: [Bug] DWARF-5 section names in PE/PEP and weak symbols

[Nick Clifton]

Hi Achim,

   Please could you file a bug report for this problem here:

https://sourceware.org/bugzilla/enter_bug.cgi?product=binutils


> changed the behaviour of weak symbols in Cygwin.  The following test
> from gnulib that should fail.  In other words fputs should have a symbol
> value of 0x0 and the resulting executable should not be able to run
> since the Windows loader does not resolve weak symbols):
> 
> --8<--weak.c-------cut here---------------start------------->8---
> #include <stdio.h>
> #pragma weak fputs
> int main ()
> {
>    return (fputs == NULL);
> }
> --8<---------------cut here---------------end--------------->8---
> 
> $ gcc -o weak weak.c # w/ binutils 2.36.1
> $ ./weak ; echo $?
> ./weak: ./weak: cannot execute binary file
> 126
> 
> Instead, starting with the above commit, the test succeeds since fputs
> has the (expected) value from cygwin1.dll.  

So you are saying that because the linker script now includes specific
references to DWARF-5 debug sections, the fputs symbol is being pulled
in to the link from the cygwin1.dll and hence it is no longer undefined ?
That seems to be most strange.


> I've narrowed it down to just one of the new DWARF5 sections and
> removing them from the linker scripts restores the correct behaviour of
> weak symbols in Cygwin (I have no idea if that triggers problems
> elsewhere):

> +/* ===
>     .debug_loclists ${RELOCATING+BLOCK(__section_alignment__)} ${RELOCATING+(NOLOAD)} :
>     {
>       *(.debug_loclists)
>     }
> +=== */

Hmm, I suppose that the location lists will refer to real locations in
the code space, and so including the debug information could also pull
in the code itself.  But really you would only want to have the debug
information for code that is already part of the final link.  So maybe
the underlying problem is that the debug information has not been pruned
to match discarded code.

Quick question - if instead of deleting the references to the .debug_loclists
section you move them into the /DISCARD/ section earlier in the pe.se file,
does this also solve the problem.  (The point being that if the script
explitcitly discards this information we also have room to add a comment
explaining why.  Plus it will prevent future changes to the pe.sc file
from adding the section back in).  (Also the change would be needed in
the pep.sc file as well, obviously.  Plus the .zdebug_loclists section
would need similar treatment).

Cheers
   Nick

Re: [Bug] DWARF-5 section names in PE/PEP and weak symbols

[Achim Gratz]

Nick Clifton via Binutils writes:
>   Please could you file a bug report for this problem here:

https://sourceware.org/bugzilla/show_bug.cgi?id=28719

> So you are saying that because the linker script now includes specific
> references to DWARF-5 debug sections, the fputs symbol is being pulled
> in to the link from the cygwin1.dll and hence it is no longer undefined ?
> That seems to be most strange.

It is much more strange: looking at the working and non-working version
(going from 2.36 to 2.37) with objdump, everything but the debug section
(differently named and now at the beginning of the object file instead
of the end), there is absolutely no other difference in the other
sections that I've been able to find.

> Hmm, I suppose that the location lists will refer to real locations in
> the code space, and so including the debug information could also pull
> in the code itself.  But really you would only want to have the debug
> information for code that is already part of the final link.  So maybe
> the underlying problem is that the debug information has not been pruned
> to match discarded code.

I have no idea where and what to look for.

> Quick question - if instead of deleting the references to the .debug_loclists
> section you move them into the /DISCARD/ section earlier in the pe.se file,
> does this also solve the problem.  (The point being that if the script
> explitcitly discards this information we also have room to add a comment
> explaining why.  Plus it will prevent future changes to the pe.sc file
> from adding the section back in).  (Also the change would be needed in
> the pep.sc file as well, obviously.  Plus the .zdebug_loclists section
> would need similar treatment).

How do I do that (and no, I'll not be able to check until the beginning
of next year).


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: [Bug] DWARF-5 section names in PE/PEP and weak symbols

[Achim Gratz]

Achim Gratz writes:
> Nick Clifton via Binutils writes:
>>   Please could you file a bug report for this problem here:
>
> https://sourceware.org/bugzilla/show_bug.cgi?id=28719

Any chance of getting a fix for this into 2.38?


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada

Re: [Bug] DWARF-5 section names in PE/PEP and weak symbols

[Achim Gratz]

Nick Clifton via Binutils writes:
> Quick question - if instead of deleting the references to the .debug_loclists
> section you move them into the /DISCARD/ section earlier in the pe.se file,
> does this also solve the problem.  (The point being that if the script
> explitcitly discards this information we also have room to add a comment
> explaining why.  Plus it will prevent future changes to the pe.sc file
> from adding the section back in).  (Also the change would be needed in
> the pep.sc file as well, obviously.  Plus the .zdebug_loclists section
> would need similar treatment).

Just moving *(.debug_loclists) to the DISCARD section still results in
the erroneous resolution of the fputs symbol.  Also moving
*(.zdebug_loclists) recovers the correct behaviour.  I have no idea what
that means in terms of actual attempt at debugging, though.

--8<---------------cut here---------------start------------->8---
diff --git a/ld/scripttempl/pe.sc b/ld/scripttempl/pe.sc
index e9d1f78cb32..bbe3f341906 100644
--- a/ld/scripttempl/pe.sc
+++ b/ld/scripttempl/pe.sc
@@ -203,6 +203,8 @@ SECTIONS
     *(.debug\$S)
     *(.debug\$T)
     *(.debug\$F)
+    *(.debug_loclists)
+    *(.zdebug_loclists)
     *(.drectve)
     ${RELOCATING+ *(.note.GNU-stack)}
     ${RELOCATING+ *(.gnu.lto_*)}
@@ -447,6 +449,7 @@ SECTIONS
   {
     *(.zdebug_line_str)
   }
+/* ===
   .debug_loclists ${RELOCATING+BLOCK(__section_alignment__)} ${RELOCATING+(NOLOAD)} :
   {
     *(.debug_loclists)
@@ -455,6 +458,7 @@ SECTIONS
   {
     *(.zdebug_loclists)
   }
+=== */
   .debug_macro ${RELOCATING+BLOCK(__section_alignment__)} ${RELOCATING+(NOLOAD)} :
   {
     *(.debug_macro)
diff --git a/ld/scripttempl/pep.sc b/ld/scripttempl/pep.sc
index afa2d23871c..cd8a2defe4e 100644
--- a/ld/scripttempl/pep.sc
+++ b/ld/scripttempl/pep.sc
@@ -208,6 +208,8 @@ SECTIONS
     *(.debug\$S)
     *(.debug\$T)
     *(.debug\$F)
+    *(.debug_loclists)
+    *(.zdebug_loclists)
     *(.drectve)
     ${RELOCATING+ *(.note.GNU-stack)}
     ${RELOCATING+ *(.gnu.lto_*)}
@@ -452,6 +454,7 @@ SECTIONS
   {
     *(.zdebug_line_str)
   }
+/* ===
   .debug_loclists ${RELOCATING+BLOCK(__section_alignment__)} ${RELOCATING+(NOLOAD)} :
   {
     *(.debug_loclists)
@@ -460,6 +463,7 @@ SECTIONS
   {
     *(.zdebug_loclists)
   }
+=== */
   .debug_macro ${RELOCATING+BLOCK(__section_alignment__)} ${RELOCATING+(NOLOAD)} :
   {
     *(.debug_macro)
--8<---------------cut here---------------end--------------->8---


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [Bug] DWARF-5 section names in PE/PEP and weak symbols

[Achim Gratz]

Nick Clifton via Binutils writes:
> So you are saying that because the linker script now includes specific
> references to DWARF-5 debug sections, the fputs symbol is being pulled
> in to the link from the cygwin1.dll and hence it is no longer undefined ?
> That seems to be most strange.

Looking further into this I'm beginning to think that the previous
behaviour was a bug that has since been fixed (and that probably has
nothing to do with the DWARF5 sections other than this change for some
reason interacted with the pre-existing bug to trip up a poorly written
autoconf test in gnulib).

In fact it appears that with 2.38 weak symbols are in fact working
correctly for the handful examples that I've come up with.  I'll have to
go back to the original example that produced buggy behaviour (grep-3.7
w/o patched gnulib threadlib.m4) and see if that's now working as well
(it may not, since the libraries have all been produced by earlier ld,
but it seems worth trying).


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf rackAttack V1.04R1:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [Bug] DWARF-5 section names in PE/PEP and weak symbols

[ASSI]

Achim Gratz writes:
> In fact it appears that with 2.38 weak symbols are in fact working
> correctly for the handful examples that I've come up with.  I'll have to
> go back to the original example that produced buggy behaviour (grep-3.7
> w/o patched gnulib threadlib.m4) and see if that's now working as well
> (it may not, since the libraries have all been produced by earlier ld,
> but it seems worth trying).

So, grep-3.7 does still not work when compiled with weak symbols
enabled.  It seems that it does not properly resoves the symbol using the
DLL at runtime and/or misses to take the actual base address of the DLL
into account (which differs from the one that it had when it was built
due to rebasing).


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf rackAttack V1.04R1:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [Bug] DWARF-5 section names in PE/PEP and weak symbols

[Achim Gratz]

So here's another attempt at getting to the bottom of this.  The failing
gnulib test was using fputs, the actually failing gnulib module was
calling thrd_exit if the weak symbol resolved non-NULL and
serendipitously I've stumbled upon snprintf behaving differently…

--8<---------------cut here---------------start------------->8---
#include <stdio.h>
#include <pthread.h>
#include <threads.h>
#pragma weak fputs
#pragma weak thrd_exit
#pragma weak snprintf
#pragma weak asnprintf
#pragma weak vsnprintf
int main ()
{
  char buf[256] = "snprintf was not called.\n";
  printf( "snprintf  => %014p\tis %sequal NULL\n", snprintf,  ((snprintf  == NULL) ? "" : "not ") );
  printf( "fputs     => %014p\tis %sequal NULL\n", fputs,     ((fputs     == NULL) ? "" : "not ") );
  printf( "thrd_exit => %014p\tis %sequal NULL\n", thrd_exit, ((thrd_exit == NULL) ? "" : "not ") );
  if (snprintf != NULL)
    snprintf(buf, sizeof(buf), "%s\n", "sprintf called through symbol");
  printf( "%s", buf );
  return (fputs == NULL);
}
--8<---------------cut here---------------end--------------->8---

This is what happens:

--8<---------------cut here---------------start------------->8---
...packages/binutils-gdb/build (2130)# gcc -Wl,--trace-symbol={fputs,snprintf,thrd_exit} -c -o weak_test.o weak_test.c
...packages/binutils-gdb/build (2131)# gcc -Wl,--trace-symbol={fputs,{a,v,}snprintf,thrd_exit} -o weak_test weak_test.o
/usr/lib/gcc/x86_64-pc-cygwin/11/../../../../x86_64-pc-cygwin/bin/ld: weak_test.o: reference to snprintf
/usr/lib/gcc/x86_64-pc-cygwin/11/../../../../x86_64-pc-cygwin/bin/ld: weak_test.o: reference to thrd_exit
/usr/lib/gcc/x86_64-pc-cygwin/11/../../../../x86_64-pc-cygwin/bin/ld: weak_test.o: reference to fputs
/usr/lib/gcc/x86_64-pc-cygwin/11/../../../../x86_64-pc-cygwin/bin/ld: /usr/lib/gcc/x86_64-pc-cygwin/11/../../../../lib/libcygwin.a(t-d000578.o): definition of fputs
/usr/lib/gcc/x86_64-pc-cygwin/11/../../../../x86_64-pc-cygwin/bin/ld: /usr/lib/gcc/x86_64-pc-cygwin/11/../../../../lib/libcygwin.a(t-d001527.o): definition of thrd_exit
...packages/binutils-gdb/build (2132)# ./weak_test ; echo $?
snprintf  => 0x000000000000     is equal NULL
fputs     => 0x0001004012e0     is not equal NULL
thrd_exit => 0x000100401300     is not equal NULL
snprintf was not called.
0
...packages/binutils-gdb/build (2133)# nm weak_test.o /bin/cygwin1.dll | egrep ' (fputs|[av]?snprintf|thrd_exit)$'
                 U fputs
                 w snprintf
                 U thrd_exit
00000001801abb10 T asnprintf
00000001801b1610 T fputs
00000001801c0380 T snprintf
00000001801849a0 T thrd_exit
00000001801eec30 T vsnprintf
--8<---------------cut here---------------end--------------->8---

So, snprintf seems to behave as expected (it resolves NULL) and the two
other symbols are resolved even though they should not.  If they had
resolved to the actual address things might even work, but the actual
values explain nicely why a program calling through the pointer crashes
in various horrible ways (depending on what actually is mapped there).


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf rackAttack V1.04R1:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [Bug] DWARF-5 section names in PE/PEP and weak symbols

[Achim Gratz]

Achim Gratz writes:
> So, snprintf seems to behave as expected (it resolves NULL) and the two
> other symbols are resolved even though they should not.  If they had
> resolved to the actual address things might even work, but the actual
> values explain nicely why a program calling through the pointer crashes
> in various horrible ways (depending on what actually is mapped there).

So I've removed the "weak" attribute from snprintf so that it links:

--8<---------------cut here---------------start------------->8---
...packages/binutils-gdb/build (2139)# 
...packages/binutils-gdb/build (2139)# objdump -x /usr/lib/libcygwin.a |less
...packages/binutils-gdb/build (2140)# gcc -Wl,--trace-symbol={fputs,snprintf,thrd_exit} -c -o weak_test.o weak_test.c
...packages/binutils-gdb/build (2141)# gcc -Wl,--trace-symbol={fputs,{a,v,}snprintf,thrd_exit} -o weak_test weak_test.o
/usr/lib/gcc/x86_64-pc-cygwin/11/../../../../x86_64-pc-cygwin/bin/ld: weak_test.o: reference to snprintf
/usr/lib/gcc/x86_64-pc-cygwin/11/../../../../x86_64-pc-cygwin/bin/ld: weak_test.o: reference to thrd_exit
/usr/lib/gcc/x86_64-pc-cygwin/11/../../../../x86_64-pc-cygwin/bin/ld: weak_test.o: reference to fputs
/usr/lib/gcc/x86_64-pc-cygwin/11/../../../../x86_64-pc-cygwin/bin/ld: /usr/lib/gcc/x86_64-pc-cygwin/11/../../../../lib/libcygwin.a(t-d000578.o): definition of fputs
/usr/lib/gcc/x86_64-pc-cygwin/11/../../../../x86_64-pc-cygwin/bin/ld: /usr/lib/gcc/x86_64-pc-cygwin/11/../../../../lib/libcygwin.a(t-d001396.o): definition of snprintf
/usr/lib/gcc/x86_64-pc-cygwin/11/../../../../x86_64-pc-cygwin/bin/ld: /usr/lib/gcc/x86_64-pc-cygwin/11/../../../../lib/libcygwin.a(t-d001527.o): definition of thrd_exit
...packages/binutils-gdb/build (2142)# ./weak_test ; echo $?
snprintf  => 0x0001004012e0     is not equal NULL
fputs     => 0x0001004012c0     is not equal NULL
thrd_exit => 0x0001004012f0     is not equal NULL
sprintf called through symbol
0
...packages/binutils-gdb/build (2143)# nm weak_test.o /bin/cygwin1.dll | egrep ' (fputs|[av]?snprintf|thrd_exit)$'
                 U fputs
                 U snprintf
                 U thrd_exit
00000001801abb10 T asnprintf
00000001801b1610 T fputs
00000001801c0380 T snprintf
00000001801849a0 T thrd_exit
00000001801eec30 T vsnprintf
--8<---------------cut here---------------end--------------->8---


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada

Re: [Bug] DWARF-5 section names in PE/PEP and weak symbols

[Achim Gratz]

Achim Gratz writes:
> So, snprintf seems to behave as expected (it resolves NULL) and the two
> other symbols are resolved even though they should not.  If they had
> resolved to the actual address things might even work, but the actual
> values explain nicely why a program calling through the pointer crashes
> in various horrible ways (depending on what actually is mapped there).

I've verified that when I add potential call sites and not just printf
output to the other two functions, they no longer get resolved at link
time and show up as NULL in the resulting executable.  That part is
quite likely as intended.  However, if the linker doesn't see a call site
it resolves something that apparently doesn't work later on.  I have not
yet tracked down how the grep-3.7 binary ends up crashing, but the most
likely path is that theweak symbol gets resolved to something
nonsensical during an intermediate link step and then transferred into
the actual executable that doesn't resolve the symbol at all (using some
data in the object instead of the library) or not relocating correctly.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada