[Bug dynamic-link/27318] New: glibc fails to load binaries when built with -march=sandybridge: CPU ISA level is lower than required

[Bug dynamic-link/27318] New: glibc fails to load binaries when built with -march=sandybridge: CPU ISA level is lower than required

[slyfox at inbox dot ru]

https://sourceware.org/bugzilla/show_bug.cgi?id=27318

            Bug ID: 27318
           Summary: glibc fails to load binaries when built with
                    -march=sandybridge:  CPU ISA level is lower than
                    required
           Product: glibc
           Version: 2.33
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: dynamic-link
          Assignee: unassigned at sourceware dot org
          Reporter: slyfox at inbox dot ru
                CC: hjl.tools at gmail dot com
  Target Milestone: ---

The system uses binutils-2.36 and gcc-11 (master). Configure glibc as:

$ ../glibc/configure --host=x86_64-pc-linux-gnu CC="x86_64-pc-linux-gnu-gcc
-march=sandybridge" --prefix=/usr --disable-werror

$ ./elf/ld.so ./libc.so
./libc.so: (p) CPU ISA level is lower than required: needed: 7; got: 3

The CPU is `Intel(R) Core(TM) i7-2700K CPU @ 3.50GHz` (sandbybridge + AES).

After adding a bit of debug glibc contains ISA_1_V3 level (too high) while CPU
contains ISA_1_V2 level (ok).

I patched glibc slightly to get ISA levels:

--- a/sysdeps/x86/dl-prop.h
+++ b/sysdeps/x86/dl-prop.h
@@ -54,8 +54,8 @@ dl_isa_level_check (struct link_map *m, const char *program)
          != l->l_x86_isa_1_needed)
        {
          if (program)
-           _dl_fatal_printf ("%s: CPU ISA level is lower than required\n",
-                             *l->l_name != '\0' ? l->l_name : program);
+           _dl_fatal_printf ("%s: (p) CPU ISA level is lower than required:
needed: %d; got: %d\n",
+                             *l->l_name != '\0' ? l->l_name : program,
l->l_x86_isa_1_needed, cpu_features->isa_1);
          else
            _dl_signal_error (0, l->l_name, "dlopen",
                              N_("CPU ISA level is lower than required"));

I suspect that binutils somehow managed to infer too high ISA level for final
binary. 

$ readelf -a elf/ld.so | tail -n 5
Displaying notes found in: .note.gnu.property
  Owner                Data size        Description
  GNU                  0x00000020       NT_GNU_PROPERTY_TYPE_0
      Properties: x86 feature used: x86, x87, XMM, YMM, ZMM, FXSR, XSAVE,
XSAVEC
        x86 ISA used: x86-64-baseline, x86-64-v3, x86-64-v4

$ LANG=C h readelf -a libc.so | tail -n 5

Displaying notes found in: .note.ABI-tag
  Owner                Data size        Description
  GNU                  0x00000010       NT_GNU_ABI_TAG (ABI version tag)
    OS: Linux, ABI: 3.2.0

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27318] glibc fails to load binaries when built with -march=sandybridge: CPU ISA level is lower than required

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27318

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |fweimer at redhat dot com
              Flags|                            |security-

--- Comment #1 from Florian Weimer <fweimer at redhat dot com> ---
Something similar has been reported on libc-help:
https://sourceware.org/pipermail/libc-help/2021-January/005648.html

If your GCC hasn't been configured with something like --with-arch=cascadelake,
then I really do believe this is a binutils issue. It really looks like it's
not computing the ISA used property correctly.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27318] glibc fails to load binaries when built with -march=sandybridge: CPU ISA level is lower than required

[slyfox at inbox dot ru]

https://sourceware.org/bugzilla/show_bug.cgi?id=27318

--- Comment #2 from Sergei Trofimovich <slyfox at inbox dot ru> ---
Aha, that makes sense. I'll try to find where v3 leaks out. So far it looks
like some rtld files need newer ISA:

$ LANG=C h readelf -a $(find -name 'rtld-*.o*') |& egrep 'File|v3' | fgrep -B1
v3
File: ./string/rtld-strcspn-c.os
      Properties: x86 ISA used: x86-64-baseline, x86-64-v3
--
File: ./gmon/rtld-profil.os
      Properties: x86 ISA used: x86-64-baseline, x86-64-v3
--
File: ./signal/rtld-sigaction.os
      Properties: x86 ISA used: x86-64-baseline, x86-64-v3

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27318] glibc fails to load binaries when built with -march=sandybridge: CPU ISA level is lower than required

[hjl.tools at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27318

H.J. Lu <hjl.tools at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|unassigned at sourceware dot org   |hjl.tools at gmail dot com

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27318] glibc fails to load binaries when built with -march=sandybridge: CPU ISA level is lower than required

[hjl.tools at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27318

H.J. Lu <hjl.tools at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |WAITING
   Last reconfirmed|                            |2021-02-02
     Ever confirmed|0                           |1

--- Comment #3 from H.J. Lu <hjl.tools at gmail dot com> ---
sysdeps/x86/isa-level.c has

# if defined __AVX__ || defined __AVX2__ || defined __F16C__ \
     || defined __FMA__ || defined __LZCNT__ || defined __MOVBE__ \
     || defined __XSAVE__
#  define ISA_V3        GNU_PROPERTY_X86_ISA_1_V3
# else
#  define ISA_V3        0
# endif

Since -march=sandybridge enables parts of v3 ISAs, x86-64-v3 is set.
We couldn't set x86 ISA needed to x86-64-v2 since this libc won't run on
v2 machines.  Technically, the v3 marker is correct.  We could issue
an error in this case.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27318] glibc fails to load binaries when built with -march=sandybridge: CPU ISA level is lower than required

[hjl.tools at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27318

H.J. Lu <hjl.tools at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |NEW

--- Comment #4 from H.J. Lu <hjl.tools at gmail dot com> ---
A patch is posted at

https://sourceware.org/pipermail/libc-alpha/2021-February/122268.html

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27318] glibc fails to load binaries when built with -march=sandybridge: CPU ISA level is lower than required

[xry111 at mengyan1223 dot wang]

https://sourceware.org/bugzilla/show_bug.cgi?id=27318

Xi Ruoyao <xry111 at mengyan1223 dot wang> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |xry111 at mengyan1223 dot wang

--- Comment #5 from Xi Ruoyao <xry111 at mengyan1223 dot wang> ---
Should we move this to binutils, add a binutils configure option to disable ISA
markers completely?  If CFLAGS contains -march=something we'll have "too high"
ISA markers all over the system.  It severly affects Gentoo or LFS users using
their custom -march= settings.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27318] glibc fails to load binaries when built with -march=sandybridge: CPU ISA level is lower than required

[xry111 at mengyan1223 dot wang]

https://sourceware.org/bugzilla/show_bug.cgi?id=27318

--- Comment #6 from Xi Ruoyao <xry111 at mengyan1223 dot wang> ---
Sorry, I misunderstood the problem.  Please ignore my previous message.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27318] glibc fails to load binaries when built with -march=sandybridge: CPU ISA level is lower than required

[slyfox at inbox dot ru]

https://sourceware.org/bugzilla/show_bug.cgi?id=27318

--- Comment #7 from Sergei Trofimovich <slyfox at inbox dot ru> ---
(In reply to Xi Ruoyao from comment #5)
> Should we move this to binutils, add a binutils configure option to disable
> ISA markers completely?  If CFLAGS contains -march=something we'll have "too
> high" ISA markers all over the system.  It severly affects Gentoo or LFS
> users using their custom -march= settings.

Gentoo should not be affected after
https://gitweb.gentoo.org/repo/gentoo.git/commit/sys-libs/glibc?id=5dbd6a821ff753e3b41324c4fb7c58cf65eeea33
workaround.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27318] glibc fails to load binaries when built with -march=sandybridge: CPU ISA level is lower than required

[schwab@linux-m68k.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27318

Andreas Schwab <schwab@linux-m68k.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jengelh at inai dot de

--- Comment #8 from Andreas Schwab <schwab@linux-m68k.org> ---
*** Bug 27446 has been marked as a duplicate of this bug. ***

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27318] glibc fails to load binaries when built with -march=sandybridge: CPU ISA level is lower than required

[hjl.tools at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27318

H.J. Lu <hjl.tools at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED
   Target Milestone|---                         |2.34

--- Comment #9 from H.J. Lu <hjl.tools at gmail dot com> ---
Fixed on master and 2.33 branch.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27318] glibc fails to load binaries when built with -march=sandybridge: CPU ISA level is lower than required

[sam at gentoo dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27318

Sam James <sam at gentoo dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |sam at gentoo dot org

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27318] glibc fails to load binaries when built with -march=sandybridge: CPU ISA level is lower than required

[sourceware at mattwhitlock dot name]

https://sourceware.org/bugzilla/show_bug.cgi?id=27318

Matt Whitlock <sourceware at mattwhitlock dot name> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |sourceware at mattwhitlock dot nam
                   |                            |e

--- Comment #10 from Matt Whitlock <sourceware at mattwhitlock dot name> ---
The fix is deficient. The arguments to 'test' are missing needed quoting:

 121 if test $libc_cv_have_x86_lahf_sahf = yes; then
 122   AC_DEFINE(HAVE_X86_LAHF_SAHF)
 123 fi
 124 if test $libc_cv_have_x86_movbe = yes; then
 125   AC_DEFINE(HAVE_X86_MOVBE)
 126 fi

Those variables can be unset, resulting in 'test' throwing errors, as reported
at https://bugs.gentoo.org/785091.

/var/tmp/portage/sys-libs/glibc-2.33/work/glibc-2.33/configure: 168: test: =:
unexpected operator
/var/tmp/portage/sys-libs/glibc-2.33/work/glibc-2.33/configure: 172: test: =:
unexpected operator

A fix is to put double-quotation marks around the shell variable references.

Do I need to open a new bug to report this, or can it be handled in the context
of this bug?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27318] glibc fails to load binaries when built with -march=sandybridge: CPU ISA level is lower than required

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27318

--- Comment #11 from Florian Weimer <fweimer at redhat dot com> ---
Please file a new bug for this. Thanks.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27318] glibc fails to load binaries when built with -march=sandybridge: CPU ISA level is lower than required

[sourceware at mattwhitlock dot name]

https://sourceware.org/bugzilla/show_bug.cgi?id=27318

--- Comment #12 from Matt Whitlock <sourceware at mattwhitlock dot name> ---
Never mind. The problem on Gentoo is that the Gentoo ebuild is forcing
libc_cv_include_x86_isa_level=no on the configure command line (as a vestigial
workaround to this bug), and that's causing the configure test to be skipped,
thereby skipping over the initialization of libc_cv_have_x86_lahf_sahf and
libc_cv_have_x86_movbe. I've reported that problem to Gentoo. Apologies for the
noise here.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27318] glibc fails to load binaries when built with -march=sandybridge: CPU ISA level is lower than required

[sam at gentoo dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27318

Sam James <sam at gentoo dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           See Also|                            |https://sourceware.org/bugz
                   |                            |illa/show_bug.cgi?id=30722

-- 
You are receiving this mail because:
You are on the CC list for the bug.