ld 2.18 --gc-sections bug? (with testcase)

ld 2.18 --gc-sections bug? (with testcase)

[Denys Vlasenko]

Hi,

I think I have a testcase where ld --gc-sections erroneously
discards a section. I'd appreciate if someone will take
a look at it.

You can download it from
http://busybox.net/~vda/ksymtab_bug.tar.bz2

This testcase is adapted from Linux kernel build.

Kernel image is linked using custom linker script. It must contain some
special sections needed for dynamic loading of external modules at runtime.
One such section is __ksymtab.

It seems that in this testcase ld does not follow instructions
given to it in custom linker script (arch_x86_64_kernel_vmlinux.lds).

For some reason __ksymtab___const_udelay and __ksymtab_synchronize_irq
symbols are not included into zz_vmlinux_bad, while other __ksymtab_*
symbols are included, as intended (for example, __ksymtab___udelay).

--print-gc-sections does not report them as discarded,
and they shouldn't be, because they are coming from __ksymtab input
sections, and arch_x86_64_kernel_vmlinux.lds has KEEP directive for those:
__ksymtab : {  KEEP(*(__ksymtab)) }

[in fact, zz_discarded.lst does not indicate any of __ksymtab sections as discarded]

__ksymtab___const_udelay is in arch_x86_64_lib_lib.a,
__ksymtab_synchronize_irq is in kernel_built-in.o,
so it doesn't seem to be caused by .a versus .o difference.

__ksymtab___const_udelay and __ksymtab_synchronize_irq
are specific to this particular set of .o/.a files.
With slightly different kernel .config, other __ksymtab_XXX sections
will be erroneously discarded. There is no apparent pattern
in which ones will be affected.

# x86_64-pc-linux-gnu-ld -v
GNU ld (GNU Binutils) 2.18

# Bad link:
x86_64-pc-linux-gnu-ld -m elf_x86_64 --gc-sections --print-gc-sections \
--build-id \
-o zz_vmlinux_bad \
-Map zz_link_bad.map -T arch_x86_64_kernel_vmlinux.lds \
\
arch_x86_64_kernel_head.o arch_x86_64_kernel_head64.o \
arch_x86_64_kernel_init_task.o init_built-in.o \
--start-group \
usr_built-in.o arch_x86_64_kernel_built-in.o arch_x86_64_mm_built-in.o \
arch_x86_64_crypto_built-in.o arch_x86_64_vdso_built-in.o \
arch_x86_64_ia32_built-in.o kernel_built-in.o mm_built-in.o \
fs_built-in.o ipc_built-in.o security_built-in.o crypto_built-in.o \
block_built-in.o lib_lib.a arch_x86_64_lib_lib.a lib_built-in.o \
arch_x86_64_lib_built-in.o drivers_built-in.o sound_built-in.o \
arch_x86_64_pci_built-in.o net_built-in.o \
--end-group \
.tmp_kallsyms3.o \
  2>zz_discarded.lst

# Here you will see __ksymtab___udelay but no __ksymtab___const_udelay
objdump -x zz_vmlinux_bad | grep 'ksymtab[a-z_]*udelay'
echo
objdump -x zz_vmlinux_bad | grep '_synchronize_irq'
echo

# If one removes --gc-sections, link works:
x86_64-pc-linux-gnu-ld -m elf_x86_64 \
--build-id \
-o zz_vmlinux_good \
-Map zz_link_good.map -T arch_x86_64_kernel_vmlinux.lds \
\
arch_x86_64_kernel_head.o arch_x86_64_kernel_head64.o \
arch_x86_64_kernel_init_task.o init_built-in.o \
--start-group \
usr_built-in.o arch_x86_64_kernel_built-in.o arch_x86_64_mm_built-in.o \
arch_x86_64_crypto_built-in.o arch_x86_64_vdso_built-in.o \
arch_x86_64_ia32_built-in.o kernel_built-in.o mm_built-in.o \
fs_built-in.o ipc_built-in.o security_built-in.o crypto_built-in.o \
block_built-in.o lib_lib.a arch_x86_64_lib_lib.a lib_built-in.o \
arch_x86_64_lib_built-in.o drivers_built-in.o sound_built-in.o \
arch_x86_64_pci_built-in.o net_built-in.o \
--end-group \
.tmp_kallsyms3.o

# Here you will see both __ksymtab___udelay and __ksymtab___const_udelay
objdump -x zz_vmlinux_good | grep 'ksymtab[a-z_]*udelay'
echo
objdump -x zz_vmlinux_good | grep '_synchronize_irq'
echo



Sample output:

ffffffff806e0f00 l     O __ksymtab      0000000000000010 __ksymtab___udelay

ffffffff806eb590 l     O __kcrctab      0000000000000008 __kcrctab_synchronize_irq
ffffffff806f4b20 l     O __ksymtab_strings      0000000000000010 __kstrtab_synchronize_irq
00000000e523ad75 g       *ABS*  0000000000000000 __crc_synchronize_irq

ffffffff806e7050 l     O __ksymtab      0000000000000010 __ksymtab___const_udelay
ffffffff806e7060 l     O __ksymtab      0000000000000010 __ksymtab___udelay

ffffffff806f16f0 l     O __kcrctab      0000000000000008 __kcrctab_synchronize_irq
ffffffff806e3b90 l     O __ksymtab      0000000000000010 __ksymtab_synchronize_irq
ffffffff806fac80 l     O __ksymtab_strings      0000000000000010 __kstrtab_synchronize_irq
00000000e523ad75 g       *ABS*  0000000000000000 __crc_synchronize_irq

--
vda

Re: ld 2.18 --gc-sections bug? (with testcase)

[H.J. Lu]

On Fri, Sep 07, 2007 at 01:01:32PM +0100, Denys Vlasenko wrote:
> Hi,
> 
> I think I have a testcase where ld --gc-sections erroneously
> discards a section. I'd appreciate if someone will take
> a look at it.
> 
> You can download it from
> http://busybox.net/~vda/ksymtab_bug.tar.bz2
> 
> This testcase is adapted from Linux kernel build.
> 

Please open a bug report at

http://www.sourceware.org/bugzilla/


H.J.

Re: ld 2.18 --gc-sections bug? (with testcase)

[Denys Vlasenko]

On Friday 07 September 2007 14:12, H.J. Lu wrote:
> On Fri, Sep 07, 2007 at 01:01:32PM +0100, Denys Vlasenko wrote:
> > I think I have a testcase where ld --gc-sections erroneously
> > discards a section. I'd appreciate if someone will take
> > a look at it.
> > 
> > You can download it from
> > http://busybox.net/~vda/ksymtab_bug.tar.bz2
> > 
> > This testcase is adapted from Linux kernel build.
> > 
> 
> Please open a bug report at
> 
> http://www.sourceware.org/bugzilla/

Done:

http://www.sourceware.org/bugzilla/show_bug.cgi?id=5006
--
vda

Re: ld 2.18 --gc-sections bug? (with testcase)

[Alan Modra]

This one turns out to be nothing to do with --gc-sections, but a
rather more serious bug.  If an input section with contents is linked
in to a bss output section, the ELF linker currently leaves the output
section as NOBITS.  File space is thus not allocated for the output
section, but a number of places write to the file regardless,
typically trashing the symbol table, string table, or debug sections.

bfd/
	PR ld/2864, ld/5006
	* elf.c (special_sections): Comment typo.
	(elf_fake_sections): Force SHT_PROGBITS for sections that are
	SHT_NOBITS if BFD section flags say they have contents.
ld/
	* ldwrite.c (build_link_order <lang_padding_statement_enum>): Correct
	condition under which we build a bfd_data_link_order.

Index: bfd/elf.c
===================================================================
RCS file: /cvs/src/src/bfd/elf.c,v
retrieving revision 1.413
diff -u -p -r1.413 elf.c
--- bfd/elf.c	25 Aug 2007 13:20:41 -0000	1.413
+++ bfd/elf.c	8 Sep 2007 08:07:42 -0000
@@ -2084,7 +2084,7 @@ static const struct bfd_elf_special_sect
 static const struct bfd_elf_special_section *special_sections[] =
 {
   special_sections_b,		/* 'b' */
-  special_sections_c,		/* 'b' */
+  special_sections_c,		/* 'c' */
   special_sections_d,		/* 'd' */
   NULL,				/* 'e' */
   special_sections_f,		/* 'f' */
@@ -2475,16 +2475,28 @@ elf_fake_sections (bfd *abfd, asection *
 
   /* If the section type is unspecified, we set it based on
      asect->flags.  */
+  if ((asect->flags & SEC_GROUP) != 0)
+    sh_type = SHT_GROUP;
+  else if ((asect->flags & SEC_ALLOC) != 0
+	   && (((asect->flags & (SEC_LOAD | SEC_HAS_CONTENTS)) == 0)
+	       || (asect->flags & SEC_NEVER_LOAD) != 0))
+    sh_type = SHT_NOBITS;
+  else
+    sh_type = SHT_PROGBITS;
+
   if (this_hdr->sh_type == SHT_NULL)
-    {
-      if ((asect->flags & SEC_GROUP) != 0)
-	this_hdr->sh_type = SHT_GROUP;
-      else if ((asect->flags & SEC_ALLOC) != 0
-	       && (((asect->flags & (SEC_LOAD | SEC_HAS_CONTENTS)) == 0)
-		   || (asect->flags & SEC_NEVER_LOAD) != 0))
-	this_hdr->sh_type = SHT_NOBITS;
-      else
-	this_hdr->sh_type = SHT_PROGBITS;
+    this_hdr->sh_type = sh_type;
+  else if (this_hdr->sh_type == SHT_NOBITS
+	   && sh_type == SHT_PROGBITS
+	   && (asect->flags & SEC_ALLOC) != 0)
+    {
+      /* Warn if we are changing a NOBITS section to PROGBITS, but
+	 allow the link to proceed.  This can happen when users link
+	 non-bss input sections to bss output sections, or emit data
+	 to a bss output section via a linker script.  */
+      (*_bfd_error_handler)
+	(_("section `%A' type changed to PROGBITS"), asect);
+      this_hdr->sh_type = sh_type;
     }
 
   switch (this_hdr->sh_type)
Index: ld/ldwrite.c
===================================================================
RCS file: /cvs/src/src/ld/ldwrite.c,v
retrieving revision 1.26
diff -u -p -r1.26 ldwrite.c
--- ld/ldwrite.c	6 Jul 2007 14:09:41 -0000	1.26
+++ ld/ldwrite.c	8 Sep 2007 08:08:37 -0000
@@ -270,7 +270,10 @@ build_link_order (lang_statement_union_t
 	output_section = statement->padding_statement.output_section;
 	ASSERT (statement->padding_statement.output_section->owner
 		== output_bfd);
-	if ((output_section->flags & SEC_HAS_CONTENTS) != 0)
+	if (((output_section->flags & SEC_HAS_CONTENTS) != 0
+	     || ((output_section->flags & SEC_LOAD) != 0
+		 && (output_section->flags & SEC_THREAD_LOCAL)))
+	    && (output_section->flags & SEC_NEVER_LOAD) == 0)
 	  {
 	    link_order = bfd_new_link_order (output_bfd, output_section);
 	    link_order->type = bfd_data_link_order;

-- 
Alan Modra
Australia Development Lab, IBM

Re: ld 2.18 --gc-sections bug? (with testcase)

[Rask Ingemann Lambertsen]

On Sat, Sep 08, 2007 at 08:34:39PM +0930, Alan Modra wrote:
> This one turns out to be nothing to do with --gc-sections, but a
> rather more serious bug.  If an input section with contents is linked
> in to a bss output section, the ELF linker currently leaves the output
> section as NOBITS.  File space is thus not allocated for the output
> section, but a number of places write to the file regardless,
> typically trashing the symbol table, string table, or debug sections.
[snip]
> 
> Index: bfd/elf.c
> ===================================================================
> RCS file: /cvs/src/src/bfd/elf.c,v
> retrieving revision 1.413
> diff -u -p -r1.413 elf.c
> --- bfd/elf.c	25 Aug 2007 13:20:41 -0000	1.413
> +++ bfd/elf.c	8 Sep 2007 08:07:42 -0000
> @@ -2084,7 +2084,7 @@ static const struct bfd_elf_special_sect
[...]
> +    this_hdr->sh_type = sh_type;
> +  else if (this_hdr->sh_type == SHT_NOBITS
> +	   && sh_type == SHT_PROGBITS
> +	   && (asect->flags & SEC_ALLOC) != 0)
> +    {
> +      /* Warn if we are changing a NOBITS section to PROGBITS, but
> +	 allow the link to proceed.  This can happen when users link
> +	 non-bss input sections to bss output sections, or emit data
> +	 to a bss output section via a linker script.  */
> +      (*_bfd_error_handler)
> +	(_("section `%A' type changed to PROGBITS"), asect);
> +      this_hdr->sh_type = sh_type;
>      }

   This triggers all the time on m32r-unknown-elf (as least with GCC
compiled programs):
ld-new: section `.sbss' type changed to PROGBITS

1) Is it something which should be fixed? If so, pointers appreciated. For
example, I can't even tell which file is causing it in spite of trying
"--verbose --print-map".

2) If it is really a warning rather than an error message, shouldn't it be
prefixed with "warning: " or so? Dejagnu currently counts it as an error.

-- 
Rask Ingemann Lambertsen
Danish law requires addresses in e-mail to be logged and stored for a year

section `.sbss' type changed to PROGBITS (Was: ld 2.18 --gc-sections bug? (with testcase))

[Rask Ingemann Lambertsen]

On Mon, Dec 10, 2007 at 05:11:23PM +0100, Rask Ingemann Lambertsen wrote:
> 
>    This triggers all the time on m32r-unknown-elf (as least with GCC
> compiled programs):
> ld-new: section `.sbss' type changed to PROGBITS
> 
> 1) Is it something which should be fixed? If so, pointers appreciated. For
> example, I can't even tell which file is causing it in spite of trying
> "--verbose --print-map".

   I found that GCC generates this when compiling newlib:

        .global __malloc_top_pad
        .section        .sbss,"aw",@nobits
        .balign 4
        .type   __malloc_top_pad, @object
        .size   __malloc_top_pad, 4
__malloc_top_pad:
        .zero   4
...
        .global __malloc_max_sbrked_mem
        .section        .sbss
        .balign 4
        .type   __malloc_max_sbrked_mem, @object
        .size   __malloc_max_sbrked_mem, 4
__malloc_max_sbrked_mem:
        .zero   4
        .global __malloc_max_total_mem
        .balign 4
        .type   __malloc_max_total_mem, @object
        .size   __malloc_max_total_mem, 4
__malloc_max_total_mem:
        .zero   4

   The resulting object file has this:

Sections:
Idx Name          Size      VMA       LMA       File off  Algn
...
  3 .sbss         0000000c  00000000  00000000  00000a94  2**2
                  CONTENTS, ALLOC, LOAD, DATA

   It is enough to assemble just the __malloc_top_pad part above to
reproduce the problem:

$ as-new ~/nobits.s -o ~/nobits.o
as-new: section `.sbss' type changed to PROGBITS

-- 
Rask Ingemann Lambertsen
Danish law requires addresses in e-mail to be logged and stored for a year

Re: section `.sbss' type changed to PROGBITS (Was: ld 2.18 --gc-sections bug? (with testcase))

[Alan Modra]

On Mon, Dec 10, 2007 at 07:23:17PM +0100, Rask Ingemann Lambertsen wrote:
> $ as-new ~/nobits.s -o ~/nobits.o
> as-new: section `.sbss' type changed to PROGBITS

It's an m32r gas bug.

	* config/tc-m32r.c (md_begin): Mark .sbss as being bss style section.

Index: gas/config/tc-m32r.c
===================================================================
RCS file: /cvs/src/src/gas/config/tc-m32r.c,v
retrieving revision 1.51
diff -u -p -r1.51 tc-m32r.c
--- gas/config/tc-m32r.c	17 Oct 2007 16:45:55 -0000	1.51
+++ gas/config/tc-m32r.c	10 Dec 2007 23:27:02 -0000
@@ -713,6 +713,7 @@ md_begin (void)
 
   /* The sbss section is for local .scomm symbols.  */
   sbss_section = subseg_new (".sbss", 0);
+  seg_info (sbss_section)->bss = 1;
 
   /* This is copied from perform_an_assembly_pass.  */
   applicable = bfd_applicable_section_flags (stdoutput);

-- 
Alan Modra
Australia Development Lab, IBM

Re: ld 2.18 --gc-sections bug? (with testcase)

[Alan Modra]

On Mon, Dec 10, 2007 at 05:11:23PM +0100, Rask Ingemann Lambertsen wrote:
> 2) If it is really a warning rather than an error message, shouldn't it be
> prefixed with "warning: " or so? Dejagnu currently counts it as an error.

	* elf.c (elf_fake_sections): Add "warning:" to "..changed to PROGBITS"
	message.

Index: bfd/elf.c
===================================================================
RCS file: /cvs/src/src/bfd/elf.c,v
retrieving revision 1.424
diff -u -p -r1.424 elf.c
--- bfd/elf.c	13 Nov 2007 05:56:10 -0000	1.424
+++ bfd/elf.c	10 Dec 2007 23:35:36 -0000
@@ -2497,7 +2497,7 @@ elf_fake_sections (bfd *abfd, asection *
 	 non-bss input sections to bss output sections, or emit data
 	 to a bss output section via a linker script.  */
       (*_bfd_error_handler)
-	(_("section `%A' type changed to PROGBITS"), asect);
+	(_("warning: section `%A' type changed to PROGBITS"), asect);
       this_hdr->sh_type = sh_type;
     }
 

-- 
Alan Modra
Australia Development Lab, IBM

Re: section `.sbss' type changed to PROGBITS (Was: ld 2.18 --gc-sections bug? (with testcase))

[Rask Ingemann Lambertsen]

On Tue, Dec 11, 2007 at 09:58:13AM +1030, Alan Modra wrote:
> On Mon, Dec 10, 2007 at 07:23:17PM +0100, Rask Ingemann Lambertsen wrote:
> > $ as-new ~/nobits.s -o ~/nobits.o
> > as-new: section `.sbss' type changed to PROGBITS
> 
> It's an m32r gas bug.
> 
> 	* config/tc-m32r.c (md_begin): Mark .sbss as being bss style section.

   That fixes it. Thanks.

-- 
Rask Ingemann Lambertsen
Danish law requires addresses in e-mail to be logged and stored for a year