[Bug libc/26386] New: libc_nonshared.a 32-bit relocations

[Bug libc/26386] New: libc_nonshared.a 32-bit relocations

[bbatliner at ocient dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=26386

            Bug ID: 26386
           Summary: libc_nonshared.a 32-bit relocations
           Product: glibc
           Version: 2.28
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: libc
          Assignee: unassigned at sourceware dot org
          Reporter: bbatliner at ocient dot com
                CC: drepper.fsp at gmail dot com
  Target Milestone: ---

Referred by Florian Weimer to create a bug report for this behavior: 
https://sourceware.org/pipermail/libc-help/2020-August/005397.html

glibc-devel-2.28-101.el8.x86_64 from the BaseOS repo is causing
linking errors for our C++ builds, likely due to 32-bit relocations
compiled into libc_nonshared.a.

Here's a minimal C program to reproduce the error:

```
#include <stdlib.h>

void foo() {}

int main() {
        atexit(&foo);
        return 0;
}
```

On CentOS 8.2.2004 with glibc 2.28 and gcc 8.2.0 compiled with
mcmodel=large (some paths truncated with ...):

$ gcc -S -o foo.s foo.c
$ as --64 -o foo.o foo.s
$ ld -Tfoo.lds --eh-frame-hdr -m elf_x86_64 -dynamic-linker
/lib64/ld-linux-x86-64.so.2 -rpath /.../gcc/stage-8.2.0/bin/lib64
-L/.../gcc/stage-8.2.0/bin/lib64 -o a.out /lib/../lib64/crt1.o
/lib/../lib64/crti.o
/.../gcc/stage-8.2.0/bin/lib/gcc/x86_64-pc-linux-gnu/8.2.0/crtbegin.o
-L/.../gcc/stage-8.2.0/bin/lib/gcc/x86_64-pc-linux-gnu/8.2.0
-L/.../gcc/stage-8.2.0/bin/lib/gcc/x86_64-pc-linux-gnu/8.2.0/../../../../lib64
-L/lib/../lib64 -L/usr/lib/../lib64
-L/.../gcc/stage-8.2.0/bin/lib/gcc/x86_64-pc-linux-gnu/8.2.0/../../.. foo.o
-lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s
--no-as-needed
/.../gcc/stage-8.2.0/bin/lib/gcc/x86_64-pc-linux-gnu/8.2.0/crtend.o
/lib/../lib64/crtn.o
ld: /usr/lib64/libc_nonshared.a(atexit.oS): in function `atexit':
(.text+0x7): relocation truncated to fit: R_X86_64_PC32 against symbol
`__dso_handle' defined in .data section in
/.../gcc/stage-8.2.0/bin/lib/gcc/x86_64-pc-linux-gnu/8.2.0/crtbegin.o

The linker script foo.lds is the default ld linker script, with
`. = 0x200000000000;` inserted just before `.data` to force the
data to be beyond the 32-bit address space.

On CentOS 7.8.2003 with glibc 2.17 and gcc 8.2.0 compiled with
mcmodel=large:

$ gcc -S -o foo.s foo.c
$ as --64 -o foo.o foo.s
$ ld -Tfoo.lds --eh-frame-hdr -m elf_x86_64 -dynamic-linker
/lib64/ld-linux-x86-64.so.2 -rpath /.../gcc/stage-8.2.0/bin/lib64
-L/.../gcc/stage-8.2.0/bin/lib64 -o a.out /lib/../lib64/crt1.o
/lib/../lib64/crti.o
/.../gcc/stage-8.2.0/bin/lib/gcc/x86_64-pc-linux-gnu/8.2.0/crtbegin.o
-L/.../gcc/stage-8.2.0/bin/lib/gcc/x86_64-pc-linux-gnu/8.2.0
-L/.../gcc/stage-8.2.0/bin/lib/gcc/x86_64-pc-linux-gnu/8.2.0/../../../../lib64
-L/lib/../lib64 -L/usr/lib/../lib64
-L/.../gcc/stage-8.2.0/bin/lib/gcc/x86_64-pc-linux-gnu/8.2.0/../../.. foo.o
-lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s
--no-as-needed
/.../gcc/stage-8.2.0/bin/lib/gcc/x86_64-pc-linux-gnu/8.2.0/crtend.o
/lib/../lib64/crtn.o
ld: failed to convert GOTPCREL relocation; relink with --no-relax

The GOTPCREL relocation comes from glibc 2.17's libc_nonshared.a:

$ readelf -r /lib64/libc_nonshared.a | grep __dso_handle
000000000003  000900000009 R_X86_64_GOTPCREL 0000000000000000 __dso_handle - 4
000000000003  000900000009 R_X86_64_GOTPCREL 0000000000000000 __dso_handle - 4

Linking with --no-relax resolves the issue on CentOS 7 / glibc 2.17.

However, on CentOS 8 / glibc 2.28:

$ readelf -r /lib64/libc_nonshared.a | grep __dso_handle
000000000007  003200000002 R_X86_64_PC32     0000000000000000 __dso_handle - 4
000000000007  003200000002 R_X86_64_PC32     0000000000000000 __dso_handle - 4
000000000007  003200000002 R_X86_64_PC32     0000000000000000 __dso_handle - 4

The relocations are PC32 and are always truncated, failing to link.
Florian believes this to be a regression introduced in commit
825adeeed1e95990fd1efb70d9ac3eb7f1ea802a.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/26386] libc_nonshared.a 32-bit relocations

[bbatliner at ocient dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=26386

--- Comment #1 from Brendan Batliner <bbatliner at ocient dot com> ---
Our current workaround is to take the libc_nonshared.a from the
CentOS 7 glibc-devel package, and statically link against that
archive specifically, even though we dynamically link to glibc
2.28 for the dynamically loaded symbols.

Is this safe/compatible? What do you recommend?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/26386] libc_nonshared.a 32-bit relocations

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=26386

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |fweimer at redhat dot com

--- Comment #2 from Florian Weimer <fweimer at redhat dot com> ---
(In reply to Brendan Batliner from comment #1)
> Our current workaround is to take the libc_nonshared.a from the
> CentOS 7 glibc-devel package, and statically link against that
> archive specifically, even though we dynamically link to glibc
> 2.28 for the dynamically loaded symbols.

Does this really work?  I think the pristine CentOS 7 builds also fail to link.
(It explains why this hasn't been reported as a regression.)

I'm not sure if -mcmodel=large is actually supported in the toolchain. I doubt
few distributions make it work.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/26386] libc_nonshared.a 32-bit relocations

[bbatliner at ocient dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=26386

--- Comment #3 from Brendan Batliner <bbatliner at ocient dot com> ---
Hi Florian, apologies for the late response, we de-prioritized
building on CentOS 8 for the time being.

Here are some steps to show you that CentOS 7's glibc-devel-2.17
package is built with 64-bit linker relocations. Output is edited
for brevity.

$ docker run -it --rm centos:7
[root@9bf6f62b0281 /]# cat /etc/system-release
CentOS Linux release 7.8.2003 (Core)
[root@9bf6f62b0281 /]# yum install -y glibc-devel
[root@9bf6f62b0281 /]# readelf -r /lib64/libc_nonshared.a | grep __dso_handle
000000000003  000900000009 R_X86_64_GOTPCREL 0000000000000000 __dso_handle - 4
000000000003  000900000009 R_X86_64_GOTPCREL 0000000000000000 __dso_handle - 4
[root@9bf6f62b0281 /]# rpm -qf /lib64/libc_nonshared.a
glibc-devel-2.17-307.el7.1.x86_64


And here is the corresponding sequence of steps on the latest
CentOS 8 Docker image.

$ docker run -it --rm centos:8
[root@78d2c175012a /]# cat /etc/system-release
CentOS Linux release 8.1.1911 (Core)
[root@78d2c175012a /]# dnf install -y glibc-devel
[root@78d2c175012a /]# readelf -r /lib64/libc_nonshared.a | grep __dso_handle
000000000007  003200000002 R_X86_64_PC32     0000000000000000 __dso_handle - 4
000000000007  003200000002 R_X86_64_PC32     0000000000000000 __dso_handle - 4
000000000007  003200000002 R_X86_64_PC32     0000000000000000 __dso_handle - 4
[root@78d2c175012a /]# rpm -qf /lib64/libc_nonshared.a
glibc-devel-2.28-101.el8.x86_64

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/26386] libc_nonshared.a 32-bit relocations

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=26386

--- Comment #4 from Florian Weimer <fweimer at redhat dot com> ---
My point is that this slightly extended example:

#include <stdio.h>

int x[1000 * 1000 * 1000] = { 1 };

static void
handler (void)
{
  printf ("%d\n", x[0]);
}

int
main (void)
{
  atexit (handler);
}

also fails to link on Red Hat Enterprise Linux 7:

/usr/lib/gcc/x86_64-redhat-linux/4.8.5/crtbegin.o: In function
`deregister_tm_clones':
crtstuff.c:(.text+0x8): relocation truncated to fit: R_X86_64_32S against
`.tm_clone_table'
/usr/lib/gcc/x86_64-redhat-linux/4.8.5/crtbegin.o: In function
`register_tm_clones':
crtstuff.c:(.text+0x38): relocation truncated to fit: R_X86_64_32S against
`.tm_clone_table'
/usr/lib/gcc/x86_64-redhat-linux/4.8.5/crtbegin.o: In function
`__do_global_dtors_aux':
crtstuff.c:(.text+0x72): relocation truncated to fit: R_X86_64_PC32 against
`.bss'
crtstuff.c:(.text+0x85): relocation truncated to fit: R_X86_64_PC32 against
`.bss'
collect2: error: ld returned 1 exit status

Doesn't this mean that -mcmodel=large is broken for most uses?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/26386] libc_nonshared.a 32-bit relocations

[bbatliner at ocient dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=26386

--- Comment #5 from Brendan Batliner <bbatliner at ocient dot com> ---
I reproduced that failure on RHEL 7 and gcc-4.8.5-39.el7.x86_64.rpm.

I mentioned that we build gcc 8.2.0 ourselves with -mcmodel=large,
and additionally patch it to build crt{begin,end}.o with the same
option:

 # Extra flags to use when compiling crt{begin,end}.o.
-CRTSTUFF_T_CFLAGS =
+CRTSTUFF_T_CFLAGS := -mcmodel=large

In our gcc, I can compile the test program successfully:

$ gcc test.c
test.c: In function ‘main’:
test.c:14:3: warning: implicit declaration of function ‘atexit’
[-Wimplicit-function-declaration]
   atexit (handler);
   ^~~~~~
$ ./a.out
1

This being said, I would agree with you that -mcmodel=large is broken
for most users, unless they're willing to recompile some components
of their toolchain.

We're able to build gcc with the correct options, but the 32-bit
relocation present in libc_nonshared.a comes from glibc, not gcc,
so I'm wondering what to do in this case.

-- 
You are receiving this mail because:
You are on the CC list for the bug.