[PATCH] libiberty: Fix up debug.temp.o creation if *.o has 64K+ sections [PR104617]

[PATCH] libiberty: Fix up debug.temp.o creation if *.o has 64K+ sections [PR104617]

[Jakub Jelinek]

Hi!

On
#define A(n) int foo1##n(void) { return 1##n; }
#define B(n) A(n##0) A(n##1) A(n##2) A(n##3) A(n##4) A(n##5) A(n##6) A(n##7) A(n##8) A(n##9)
#define C(n) B(n##0) B(n##1) B(n##2) B(n##3) B(n##4) B(n##5) B(n##6) B(n##7) B(n##8) B(n##9)
#define D(n) C(n##0) C(n##1) C(n##2) C(n##3) C(n##4) C(n##5) C(n##6) C(n##7) C(n##8) C(n##9)
#define E(n) D(n##0) D(n##1) D(n##2) D(n##3) D(n##4) D(n##5) D(n##6) D(n##7) D(n##8) D(n##9)
E(0) E(1) E(2) D(30) D(31) C(320) C(321) C(322) C(323) C(324) C(325)
B(3260) B(3261) B(3262) B(3263) A(32640) A(32641) A(32642)
testcase with
./xgcc -B ./ -c -g -fpic -ffat-lto-objects -flto  -O0 -o foo1.o foo1.c -ffunction-sections
./xgcc -B ./ -shared -g -fpic -flto -O0 -o foo1.so foo1.o
/tmp/ccTW8mBm.debug.temp.o: file not recognized: file format not recognized
(testcase too slow to be included into testsuite).
The problem is clearly reported by readelf:
readelf: foo1.o.debug.temp.o: Warning: Section 2 has an out of range sh_link value of 65321
readelf: foo1.o.debug.temp.o: Warning: Section 5 has an out of range sh_link value of 65321
readelf: foo1.o.debug.temp.o: Warning: Section 10 has an out of range sh_link value of 65323
readelf: foo1.o.debug.temp.o: Warning: [ 2]: Link field (65321) should index a symtab section.
readelf: foo1.o.debug.temp.o: Warning: [ 5]: Link field (65321) should index a symtab section.
readelf: foo1.o.debug.temp.o: Warning: [10]: Link field (65323) should index a string section.
because simple_object_elf_copy_lto_debug_sections doesn't adjust sh_info and
sh_link fields in ElfNN_Shdr if they are in between SHN_{LO,HI}RESERVE
inclusive.  Not adjusting those is incorrect though, SHN_{LO,HI}RESERVE
range is only relevant to the 16-bit fields, mainly st_shndx in ElfNN_Sym
where if one needs >= SHN_LORESERVE section number, SHN_XINDEX should be
used instead and .symtab_shndx section should contain the real section
index, and in ElfNN_Ehdr e_shnum and e_shstrndx fields, where if >=
SHN_LORESERVE value is needed it should put those into
Shdr[0].sh_{size,link}.  But, sh_{link,info} are 32-bit fields which can
contain any section index.

Note, as simple-object-elf.c mentions, binutils from 2.12 to 2.18 (so before
2011) used to mishandle the > 63.75K sections case and assumed there is a
hole in between the sections, but what
simple_object_elf_copy_lto_debug_sections does wouldn't help in that case
for the debug temp object creation, we'd need to detect the case also in
that routine and take it into account in the remapping etc.  I think
it is not worth it given that it is over 10 years, if somebody needs
63.75K or more sections, better use more recent binutils.

Bootstrapped/regtested on x86_64-linux and i686-linux, ok for trunk?

2022-02-22  Jakub Jelinek  <jakub@redhat.com>

	PR lto/104617
	* simple-object-elf.c (simple_object_elf_match): Fix up URL
	in comment.
	(simple_object_elf_copy_lto_debug_sections): Remap sh_info and
	sh_link even if they are in the SHN_LORESERVE .. SHN_HIRESERVE
	range (inclusive).

--- libiberty/simple-object-elf.c.jj	2022-01-11 23:11:23.967267993 +0100
+++ libiberty/simple-object-elf.c	2022-02-21 20:37:12.815202845 +0100
@@ -528,7 +528,7 @@ simple_object_elf_match (unsigned char h
 	     not handle objects with more than SHN_LORESERVE sections
 	     correctly.  All large section indexes were offset by
 	     0x100.  There is more information at
-	     http://sourceware.org/bugzilla/show_bug.cgi?id-5900 .
+	     https://sourceware.org/PR5900 .
 	     Fortunately these object files are easy to detect, as the
 	     GNU binutils always put the section header string table
 	     near the end of the list of sections.  Thus if the
@@ -1559,17 +1559,13 @@ simple_object_elf_copy_lto_debug_section
 	  {
 	    sh_info = ELF_FETCH_FIELD (type_functions, ei_class, Shdr,
 				       shdr, sh_info, Elf_Word);
-	    if (sh_info < SHN_LORESERVE
-		|| sh_info > SHN_HIRESERVE)
-	      sh_info = sh_map[sh_info];
+	    sh_info = sh_map[sh_info];
 	    ELF_SET_FIELD (type_functions, ei_class, Shdr,
 			   shdr, sh_info, Elf_Word, sh_info);
 	  }
 	sh_link = ELF_FETCH_FIELD (type_functions, ei_class, Shdr,
 				   shdr, sh_link, Elf_Word);
-	if (sh_link < SHN_LORESERVE
-	    || sh_link > SHN_HIRESERVE)
-	  sh_link = sh_map[sh_link];
+	sh_link = sh_map[sh_link];
 	ELF_SET_FIELD (type_functions, ei_class, Shdr,
 		       shdr, sh_link, Elf_Word, sh_link);
       }

	Jakub

Re: [PATCH] libiberty: Fix up debug.temp.o creation if *.o has 64K+ sections [PR104617]

[Richard Biener]

On Tue, 22 Feb 2022, Jakub Jelinek wrote:

> Hi!
> 
> On
> #define A(n) int foo1##n(void) { return 1##n; }
> #define B(n) A(n##0) A(n##1) A(n##2) A(n##3) A(n##4) A(n##5) A(n##6) A(n##7) A(n##8) A(n##9)
> #define C(n) B(n##0) B(n##1) B(n##2) B(n##3) B(n##4) B(n##5) B(n##6) B(n##7) B(n##8) B(n##9)
> #define D(n) C(n##0) C(n##1) C(n##2) C(n##3) C(n##4) C(n##5) C(n##6) C(n##7) C(n##8) C(n##9)
> #define E(n) D(n##0) D(n##1) D(n##2) D(n##3) D(n##4) D(n##5) D(n##6) D(n##7) D(n##8) D(n##9)
> E(0) E(1) E(2) D(30) D(31) C(320) C(321) C(322) C(323) C(324) C(325)
> B(3260) B(3261) B(3262) B(3263) A(32640) A(32641) A(32642)
> testcase with
> ./xgcc -B ./ -c -g -fpic -ffat-lto-objects -flto  -O0 -o foo1.o foo1.c -ffunction-sections
> ./xgcc -B ./ -shared -g -fpic -flto -O0 -o foo1.so foo1.o
> /tmp/ccTW8mBm.debug.temp.o: file not recognized: file format not recognized
> (testcase too slow to be included into testsuite).
> The problem is clearly reported by readelf:
> readelf: foo1.o.debug.temp.o: Warning: Section 2 has an out of range sh_link value of 65321
> readelf: foo1.o.debug.temp.o: Warning: Section 5 has an out of range sh_link value of 65321
> readelf: foo1.o.debug.temp.o: Warning: Section 10 has an out of range sh_link value of 65323
> readelf: foo1.o.debug.temp.o: Warning: [ 2]: Link field (65321) should index a symtab section.
> readelf: foo1.o.debug.temp.o: Warning: [ 5]: Link field (65321) should index a symtab section.
> readelf: foo1.o.debug.temp.o: Warning: [10]: Link field (65323) should index a string section.
> because simple_object_elf_copy_lto_debug_sections doesn't adjust sh_info and
> sh_link fields in ElfNN_Shdr if they are in between SHN_{LO,HI}RESERVE
> inclusive.  Not adjusting those is incorrect though, SHN_{LO,HI}RESERVE
> range is only relevant to the 16-bit fields, mainly st_shndx in ElfNN_Sym
> where if one needs >= SHN_LORESERVE section number, SHN_XINDEX should be
> used instead and .symtab_shndx section should contain the real section
> index, and in ElfNN_Ehdr e_shnum and e_shstrndx fields, where if >=
> SHN_LORESERVE value is needed it should put those into
> Shdr[0].sh_{size,link}.  But, sh_{link,info} are 32-bit fields which can
> contain any section index.
> 
> Note, as simple-object-elf.c mentions, binutils from 2.12 to 2.18 (so before
> 2011) used to mishandle the > 63.75K sections case and assumed there is a
> hole in between the sections, but what
> simple_object_elf_copy_lto_debug_sections does wouldn't help in that case
> for the debug temp object creation, we'd need to detect the case also in
> that routine and take it into account in the remapping etc.  I think
> it is not worth it given that it is over 10 years, if somebody needs
> 63.75K or more sections, better use more recent binutils.
> 
> Bootstrapped/regtested on x86_64-linux and i686-linux, ok for trunk?

OK.  I suppose this also qualifies for backports?

Thanks,
Richard.

> 2022-02-22  Jakub Jelinek  <jakub@redhat.com>
> 
> 	PR lto/104617
> 	* simple-object-elf.c (simple_object_elf_match): Fix up URL
> 	in comment.
> 	(simple_object_elf_copy_lto_debug_sections): Remap sh_info and
> 	sh_link even if they are in the SHN_LORESERVE .. SHN_HIRESERVE
> 	range (inclusive).
> 
> --- libiberty/simple-object-elf.c.jj	2022-01-11 23:11:23.967267993 +0100
> +++ libiberty/simple-object-elf.c	2022-02-21 20:37:12.815202845 +0100
> @@ -528,7 +528,7 @@ simple_object_elf_match (unsigned char h
>  	     not handle objects with more than SHN_LORESERVE sections
>  	     correctly.  All large section indexes were offset by
>  	     0x100.  There is more information at
> -	     http://sourceware.org/bugzilla/show_bug.cgi?id-5900 .
> +	     https://sourceware.org/PR5900 .
>  	     Fortunately these object files are easy to detect, as the
>  	     GNU binutils always put the section header string table
>  	     near the end of the list of sections.  Thus if the
> @@ -1559,17 +1559,13 @@ simple_object_elf_copy_lto_debug_section
>  	  {
>  	    sh_info = ELF_FETCH_FIELD (type_functions, ei_class, Shdr,
>  				       shdr, sh_info, Elf_Word);
> -	    if (sh_info < SHN_LORESERVE
> -		|| sh_info > SHN_HIRESERVE)
> -	      sh_info = sh_map[sh_info];
> +	    sh_info = sh_map[sh_info];
>  	    ELF_SET_FIELD (type_functions, ei_class, Shdr,
>  			   shdr, sh_info, Elf_Word, sh_info);
>  	  }
>  	sh_link = ELF_FETCH_FIELD (type_functions, ei_class, Shdr,
>  				   shdr, sh_link, Elf_Word);
> -	if (sh_link < SHN_LORESERVE
> -	    || sh_link > SHN_HIRESERVE)
> -	  sh_link = sh_map[sh_link];
> +	sh_link = sh_map[sh_link];
>  	ELF_SET_FIELD (type_functions, ei_class, Shdr,
>  		       shdr, sh_link, Elf_Word, sh_link);
>        }
> 
> 	Jakub
> 
> 

-- 
Richard Biener <rguenther@suse.de>
SUSE Software Solutions Germany GmbH, Maxfeldstrasse 5, 90409 Nuernberg,
Germany; GF: Ivo Totev; HRB 36809 (AG Nuernberg)