GNU hash-style compatibility problem on x86_64

GNU hash-style compatibility problem on x86_64

[Greg Schafer]

Hi,

Host platform is current x86_64:

  HJL binutils-2.17.50.0.18
  gcc-4.2.1
  glibc-2.6.1 (glibc defaults to compiling with --hash-style=both)

The problem strikes on this host when attempting to natively bootstrap an
older toolchain (eg: with FSF binutils-2.17):

$ echo 'main(){}' | stage1/xgcc -Bstage1/ -xc -o /dev/null -
/lib64/libc.so.6: file not recognized: File format not recognized
collect2: ld returned 1 exit status

Essentially, the older ld chokes on the newer libc ie: the one compiled with
--hash-style=both (it affects any dso with --hash-style=gnu actually).

The weird thing is, it only happens on x86_64. I've tested the exact same
scenario on x86 and ppc and those arches both work fine. Why should x86_64
behave any differently here? This is all above my head so I would really
appreciate your help in understanding the root cause.

You should be able to easily reproduce this if your x86_64 host has a libc
that was compiled with --hash-style=both (latest Fedora, Debian lenny etc),
just build binutils-2.17 then try to link using its ld. Note: you might have
to fiddle with gcc specs if your host gcc is hardwired for --hash-style=gnu

Thanks
Greg

(yes, I could just use a newer binutils, or even cross compile, but that's
not the point)

Re: GNU hash-style compatibility problem on x86_64

[H.J. Lu]

On Tue, Aug 14, 2007 at 05:22:31PM +1000, Greg Schafer wrote:
> Hi,
> 
> Host platform is current x86_64:
> 
>   HJL binutils-2.17.50.0.18
>   gcc-4.2.1
>   glibc-2.6.1 (glibc defaults to compiling with --hash-style=both)
> 
> The problem strikes on this host when attempting to natively bootstrap an
> older toolchain (eg: with FSF binutils-2.17):
> 
> $ echo 'main(){}' | stage1/xgcc -Bstage1/ -xc -o /dev/null -
> /lib64/libc.so.6: file not recognized: File format not recognized
> collect2: ld returned 1 exit status
> 
> Essentially, the older ld chokes on the newer libc ie: the one compiled with
> --hash-style=both (it affects any dso with --hash-style=gnu actually).
> 
> The weird thing is, it only happens on x86_64. I've tested the exact same
> scenario on x86 and ppc and those arches both work fine. Why should x86_64
> behave any differently here? This is all above my head so I would really
> appreciate your help in understanding the root cause.
> 
> You should be able to easily reproduce this if your x86_64 host has a libc
> that was compiled with --hash-style=both (latest Fedora, Debian lenny etc),
> just build binutils-2.17 then try to link using its ld. Note: you might have
> to fiddle with gcc specs if your host gcc is hardwired for --hash-style=gnu
> 

Well, it sounds like a bug in binutils 2.17. There are so many of them
and they have been fixed in the current binutils. I don't want to spend
time on it unless it is reproducible in the current binutils.


H.J.

Re: GNU hash-style compatibility problem on x86_64

[Daniel Jacobowitz]

On Tue, Aug 14, 2007 at 05:47:47AM -0700, H.J. Lu wrote:
> Well, it sounds like a bug in binutils 2.17. There are so many of them
> and they have been fixed in the current binutils. I don't want to spend
> time on it unless it is reproducible in the current binutils.

But wasn't the point of --hash-style=both to be compatible with older
tools?

At a guess this has to do with the unrecognized shdr type.

-- 
Daniel Jacobowitz
CodeSourcery

Re: GNU hash-style compatibility problem on x86_64

[H.J. Lu]

On Tue, Aug 14, 2007 at 08:56:38AM -0400, Daniel Jacobowitz wrote:
> On Tue, Aug 14, 2007 at 05:47:47AM -0700, H.J. Lu wrote:
> > Well, it sounds like a bug in binutils 2.17. There are so many of them
> > and they have been fixed in the current binutils. I don't want to spend
> > time on it unless it is reproducible in the current binutils.
> 
> But wasn't the point of --hash-style=both to be compatible with older
> tools?
> 

I don't recommend older tools on Linux. If it forces users to use
the current binutils on Linux, it is even better.


H.J.

Re: GNU hash-style compatibility problem on x86_64

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 774 bytes --]

On Tuesday 14 August 2007, H.J. Lu wrote:
> On Tue, Aug 14, 2007 at 08:56:38AM -0400, Daniel Jacobowitz wrote:
> > On Tue, Aug 14, 2007 at 05:47:47AM -0700, H.J. Lu wrote:
> > > Well, it sounds like a bug in binutils 2.17. There are so many of them
> > > and they have been fixed in the current binutils. I don't want to spend
> > > time on it unless it is reproducible in the current binutils.
> >
> > But wasn't the point of --hash-style=both to be compatible with older
> > tools?
>
> I don't recommend older tools on Linux. If it forces users to use
> the current binutils on Linux, it is even better.

except that binutils-2.17 is "current binutils" ... not everyone recognizes 
the snapshots as real releases (since they arent real GNU releases)
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 827 bytes --]

Re: GNU hash-style compatibility problem on x86_64

[Jakub Jelinek]

On Tue, Aug 14, 2007 at 09:07:29AM -0400, Mike Frysinger wrote:
> On Tuesday 14 August 2007, H.J. Lu wrote:
> > On Tue, Aug 14, 2007 at 08:56:38AM -0400, Daniel Jacobowitz wrote:
> > > On Tue, Aug 14, 2007 at 05:47:47AM -0700, H.J. Lu wrote:
> > > > Well, it sounds like a bug in binutils 2.17. There are so many of them
> > > > and they have been fixed in the current binutils. I don't want to spend
> > > > time on it unless it is reproducible in the current binutils.
> > >
> > > But wasn't the point of --hash-style=both to be compatible with older
> > > tools?
> >
> > I don't recommend older tools on Linux. If it forces users to use
> > the current binutils on Linux, it is even better.
> 
> except that binutils-2.17 is "current binutils" ... not everyone recognizes 
> the snapshots as real releases (since they arent real GNU releases)

Well, if you build your glibc with newer binutils than that
(otherwise it wouldn't be built with --hash-style=both), then you already
weren't recognizing 2.17 as "current binutils".

	Jakub

Re: GNU hash-style compatibility problem on x86_64

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 1310 bytes --]

On Tuesday 14 August 2007, Jakub Jelinek wrote:
> On Tue, Aug 14, 2007 at 09:07:29AM -0400, Mike Frysinger wrote:
> > On Tuesday 14 August 2007, H.J. Lu wrote:
> > > On Tue, Aug 14, 2007 at 08:56:38AM -0400, Daniel Jacobowitz wrote:
> > > > On Tue, Aug 14, 2007 at 05:47:47AM -0700, H.J. Lu wrote:
> > > > > Well, it sounds like a bug in binutils 2.17. There are so many of
> > > > > them and they have been fixed in the current binutils. I don't want
> > > > > to spend time on it unless it is reproducible in the current
> > > > > binutils.
> > > >
> > > > But wasn't the point of --hash-style=both to be compatible with older
> > > > tools?
> > >
> > > I don't recommend older tools on Linux. If it forces users to use
> > > the current binutils on Linux, it is even better.
> >
> > except that binutils-2.17 is "current binutils" ... not everyone
> > recognizes the snapshots as real releases (since they arent real GNU
> > releases)
>
> Well, if you build your glibc with newer binutils than that
> (otherwise it wouldn't be built with --hash-style=both), then you already
> weren't recognizing 2.17 as "current binutils".

or you were simply testing the latest snapshot and didnt realize you just 
boned yourself with an option that's supposed to be backwards compatible
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 827 bytes --]

Re: GNU hash-style compatibility problem on x86_64

[Daniel Jacobowitz]

On Tue, Aug 14, 2007 at 03:29:26PM +0200, Jakub Jelinek wrote:
> Well, if you build your glibc with newer binutils than that
> (otherwise it wouldn't be built with --hash-style=both), then you already
> weren't recognizing 2.17 as "current binutils".

Your distributor probably did it for you.  Or the distributor of some
third party library, not necessarily glibc.

If there's something we can fix that would make --hash-style=both
libraries work with older binutils on x86-64, we should fix it.  I
don't know if it's possible, though.

-- 
Daniel Jacobowitz
CodeSourcery

Re: GNU hash-style compatibility problem on x86_64

[Greg Schafer]

On Tue, Aug 14, 2007 at 08:56:38AM -0400, Daniel Jacobowitz wrote:
> On Tue, Aug 14, 2007 at 05:47:47AM -0700, H.J. Lu wrote:
> > Well, it sounds like a bug in binutils 2.17. There are so many of them
> > and they have been fixed in the current binutils. I don't want to spend
> > time on it unless it is reproducible in the current binutils.
> 
> But wasn't the point of --hash-style=both to be compatible with older
> tools?

Yes, that was my understanding too and the main reason for my report. Roland
mentioned back-compat issues in this LKML posting:

http://lkml.org/lkml/2006/7/26/262

but admittedly he only touches on dynamic linkers, and of course he was
talking in the context of kernel vDSO.

> At a guess this has to do with the unrecognized shdr type.

I can understand HJ's response in that there isn't much incentive to fix
bugs in an old binutils release, but I was kind of hoping one of the experts
here might be able to pinpoint the problem and produce a minimal patch.

Regards
Greg

Re: GNU hash-style compatibility problem on x86_64

[DJ Delorie]

Greg Schafer <gschafer@zip.com.au> writes:
> I can understand HJ's response in that there isn't much incentive to fix
> bugs in an old binutils release, but I was kind of hoping one of the experts
> here might be able to pinpoint the problem and produce a minimal patch.

Try this one:

--- bfd/elf.c       3 Aug 2006 21:41:05 -0000
+++ bfd/elf.c       3 Aug 2007 01:13:57 -0000
@@ -1822,6 +1822,7 @@ bfd_section_from_shdr (bfd *abfd, unsign
     case SHT_FINI_ARRAY:       /* .fini_array section.  */
     case SHT_PREINIT_ARRAY:    /* .preinit_array section.  */
     case SHT_GNU_LIBLIST:      /* .gnu.liblist section.  */
+    case SHT_GNU_HASH:         /* .gnu.hash section.  */
       return _bfd_elf_make_section_from_shdr (abfd, hdr, name, shindex);
 
     case SHT_DYNAMIC:  /* Dynamic linking information.  */

Re: GNU hash-style compatibility problem on x86_64

[Greg Schafer]

On Tue, Aug 14, 2007 at 07:35:03PM -0400, DJ Delorie wrote:
> Greg Schafer <gschafer@zip.com.au> writes:
> > I can understand HJ's response in that there isn't much incentive to fix
> > bugs in an old binutils release, but I was kind of hoping one of the experts
> > here might be able to pinpoint the problem and produce a minimal patch.
> 
> Try this one:
> 
> --- bfd/elf.c       3 Aug 2006 21:41:05 -0000
> +++ bfd/elf.c       3 Aug 2007 01:13:57 -0000
> @@ -1822,6 +1822,7 @@ bfd_section_from_shdr (bfd *abfd, unsign
>      case SHT_FINI_ARRAY:       /* .fini_array section.  */
>      case SHT_PREINIT_ARRAY:    /* .preinit_array section.  */
>      case SHT_GNU_LIBLIST:      /* .gnu.liblist section.  */
> +    case SHT_GNU_HASH:         /* .gnu.hash section.  */
>        return _bfd_elf_make_section_from_shdr (abfd, hdr, name, shindex);
>  
>      case SHT_DYNAMIC:  /* Dynamic linking information.  */
> 

Heh, it works. Thanks a lot!

In addition to the above, I just needed the corresponding define in
include/elf/common.h to make it build.

But it's still a mystery why x86 and ppc don't need this.. oh well.

Regards
Greg

Re: GNU hash-style compatibility problem on x86_64

[DJ Delorie]

> In addition to the above, I just needed the corresponding define in
> include/elf/common.h to make it build.

Doh, I forgot that part of the patch internally once, too.

Re: GNU hash-style compatibility problem on x86_64

[Eric Botcazou]

> But it's still a mystery why x86 and ppc don't need this.. oh well.

Their BFD backend is more liberal and accepts sections with unknown IDs.

--
Eric Botcazou

Re: GNU hash-style compatibility problem on x86_64

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 263 bytes --]

On Friday 24 August 2007, Eric Botcazou wrote:
> > But it's still a mystery why x86 and ppc don't need this.. oh well.
>
> Their BFD backend is more liberal and accepts sections with unknown IDs.

looks like ia64 falls into the "very anal" category as well
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 827 bytes --]