VMA section overlap warnings for overlays

VMA section overlap warnings for overlays

[David Stubbs]

Hi,

I have an issue with overlays that seems to have come about as a result
of this patch (http://sourceware.org/ml/binutils/2008-05/msg00235.html):

---
2008-05-28  Jan Kratochvil  <jan.kratochvil@redhat.com>

	* elf.c (assign_file_positions_for_load_sections): Adjust
pre-section
	gaps based on VMA and P_VADDR instead of LMA and P_PADDR
addresses.
---

Consider the following simplified code:

---
brsbs01 0>cat over.S
        .section SEC_NAME,"ax",@progbits
        nop
        
brsbs01 0>cat start.s
        .section        .text,"ax",@progbits
        .globl  _start
        .type   _start, @function
_start:
        nop

        .section        .imem,"ax",@progbits
        nop

brsbs01 0>cat ldscript.x 
OUTPUT_FORMAT("elf32-i386")
OUTPUT_ARCH(i386)

ENTRY(_start)

PHDRS
{
  mem PT_LOAD;
  imem PT_LOAD;
}

SECTIONS
{
  .text 0x1000:
  {
    *(.text)
  } :mem

  .imem 0x2000 : AT (LOADADDR (.text) + SIZEOF (.text))
  {
    *(.imem)
  } :imem

  .overlay1 ADDR (.imem) + SIZEOF (.imem) 
    : AT (LOADADDR (.imem) + SIZEOF (.imem))
  {
    *(.overlay.1)
  } :imem
       
  .overlay2 ADDR (.imem) + SIZEOF (.imem) 
    : AT (LOADADDR (.overlay1) + SIZEOF (.overlay1))
  {
    *(.overlay.2)
  } :imem

  .overlay3 ADDR (.imem) + SIZEOF (.imem) 
    : AT (LOADADDR (.overlay2) + SIZEOF (.overlay2))
  {
    *(.overlay.3)
  } :imem
}
---

Built with:

---
gcc over.S -DSEC_NAME=".overlay.1" -c -o over1.o
gcc over.S -DSEC_NAME=".overlay.2" -c -o over2.o
gcc over.S -DSEC_NAME=".overlay.3" -c -o over3.o
gcc start.s -c -o start.o
ld over1.o over2.o over3.o start.o -T ldscript.x -o test.exe 
---

Using an old version of GNU ld (2.17) I don't get any linker warnings,
and the executable looks as I'd expect.

When using the latest code from CVS, I get the following warnings when
linking:

---
brsbs01 0>ld-new over1.o over2.o over3.o start.o -T ldscript.x -o
test.exe
ld-new: test.exe: section .overlay2 vma 0x2001 overlaps previous
sections
ld-new: test.exe: section .overlay3 vma 0x2001 overlaps previous
sections
---

As it happens, the executable actually looks correct in this case, but
the warnings can't be a good thing. Also, I have a larger case where
we're getting these warnings and our overlays aren't working (whereas
they used to with an older linker) - I just haven't managed to reproduce
this problem in a smaller test yet...

Am I doing something wrong or unusual in my linker script, or does it
look like there's a problem in the linker?

Thanks for any advice,
Dave.

Re: VMA section overlap warnings for overlays

[Alan Modra]

On Fri, Apr 16, 2010 at 02:29:00PM +0100, David Stubbs wrote:
> Am I doing something wrong or unusual in my linker script, or does it
> look like there's a problem in the linker?

Your script is fine.  I'm looking at reverting Jan's patch.

-- 
Alan Modra
Australia Development Lab, IBM

RE: VMA section overlap warnings for overlays

[David Stubbs]

> > Am I doing something wrong or unusual in my linker script, or
> does it
> > look like there's a problem in the linker?
> 
> Your script is fine.  I'm looking at reverting Jan's patch.

On a related note, I've noticed that a slightly 'enhanced' version of
the linker script ends up with incorrect LMAs being used. Here's the
script:

---
UTPUT_FORMAT("elf32-i386")
OUTPUT_ARCH(i386)

ENTRY(_start)

PHDRS
{
  mem PT_LOAD;
  info PT_LOAD;
  imem PT_LOAD;
}

SECTIONS
{
  .text 0x1000:
  {
    *(.text)
  } :mem

  .overlays 0x9000:
  {
    LONG (LOADADDR (.overlay1))
    LONG (LOADADDR (.overlay2))
    LONG (LOADADDR (.overlay3))
  } :info

  .imem 0x2000 : AT (LOADADDR (.text) + SIZEOF (.text))
  {
    *(.imem)
  } :imem

  .overlay1 ALIGN (ADDR (.imem) + SIZEOF (.imem), 8K) 
    : AT (LOADADDR (.imem) + SIZEOF (.imem))
  {
    *(.overlay.1)
  } :imem
       
  .overlay2 ALIGN (ADDR (.imem) + SIZEOF (.imem), 8K)
    : AT (LOADADDR (.overlay1) + SIZEOF (.overlay1))
  {
    *(.overlay.2)
  } :imem

  .overlay3 ALIGN (ADDR (.imem) + SIZEOF (.imem), 8k)
    : AT (LOADADDR (.overlay2) + SIZEOF (.overlay2))
  {
    *(.overlay.3)
  } :imem
}
---

The .overlays section reports the load addresses as:

---
Hex dump of section '.overlays':
  0x00009000          00001004 00001003 00001002 ............
---

Whereas the section headers report:

---
Idx Name          Size      VMA       LMA       File off  Algn
  1 .overlay1     00000001  00004000  00003001  00005000  2**0
  4 .overlay2     00000001  00004000  00003002  00005001  2**0
  5 .overlay3     00000001  00004000  00003003  00005002  2**0
---

So the LMAs in the .overlays section do not match the LMAs in the
section headers. And for completeness, the program headers looks like:

---
Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg
Align
  LOAD           0x001000 0x00001000 0x00001000 0x00001 0x00001 R E
0x1000
  LOAD           0x002000 0x00009000 0x00009000 0x0000c 0x0000c RW
0x1000
  LOAD           0x003000 0x00002000 0x00001001 0x02003 0x02003 R E
0x1000

 Section to Segment mapping:
  Segment Sections...
   00     .text 
   01     .overlays 
   02     .overlay1 .imem .overlay2 .overlay3
---

I think it's the contents of the .overlays section that is actually
incorrect in this case. 

I don't think reverting that patch alone fixes it though. In fact, from
a very quick test I get this error:

---
/home/stubbs/local_disc/work/gnu_live_sources/binutils/build/ld/ld-new:
test.exe: section `.overlay1' can't be allocated in segment 2
LOAD: .imem .overlay1 .overlay2 .overlay3
---

Which definitely doesn't look correct to me. There must be some other
associated change that I've not yet found...

I was planning on taking another look into this, but would obviously
appreciate any help you can provide!

Thanks again,
Dave.

Re: VMA section overlap warnings for overlays

[Alan Modra]

On Wed, Apr 21, 2010 at 03:44:08PM +0100, David Stubbs wrote:
>   .imem 0x2000 : AT (LOADADDR (.text) + SIZEOF (.text))
>   {
>     *(.imem)
>   } :imem

So this goes to imem with a vma of 0x2000 and lma x, say

>   .overlay1 ALIGN (ADDR (.imem) + SIZEOF (.imem), 8K) 
>     : AT (LOADADDR (.imem) + SIZEOF (.imem))
>   {
>     *(.overlay.1)
>   } :imem

and this to imem with vma 0x4000 and lma x+1

Now we have a problem.  If we put .overlay1 in imem at p_offset+1 then
the execution model lma (calculated from p_paddr+p_offset+1) for
.overlay1 will be correct, but the vma (from p_vaddr+p_offset+1
ie. 0x2001) will be wrong.  Of course, with multiple overlays packed
into one header you can't possibly get the execution model vma correct
for all the overlays, so you probably don't care.  However, the
ELF_IS_SECTION_IN_SEGMENT_FILE test in
elf.c:assign_file_positions_for_load_sections fails, which is why you
get a linker error.

Conversely, putting .overlay1 at p_offset+0x2000 will give the correct
vma but the wrong lma, and of course insert a whole lot of padding.
This is what Jan's patch did, and is quite wrong for overlays..

-- 
Alan Modra
Australia Development Lab, IBM

Re: VMA section overlap warnings for overlays

[Alan Modra]

On Thu, Apr 22, 2010 at 10:41:06AM +0930, Alan Modra wrote:
> Now we have a problem.  If we put .overlay1 in imem at p_offset+1 then
> the execution model lma (calculated from p_paddr+p_offset+1) for
> .overlay1 will be correct, but the vma (from p_vaddr+p_offset+1
> ie. 0x2001) will be wrong.  Of course, with multiple overlays packed
> into one header you can't possibly get the execution model vma correct
> for all the overlays, so you probably don't care.  However, the
> ELF_IS_SECTION_IN_SEGMENT_FILE test in
> elf.c:assign_file_positions_for_load_sections fails, which is why you
> get a linker error.
> 
> Conversely, putting .overlay1 at p_offset+0x2000 will give the correct
> vma but the wrong lma, and of course insert a whole lot of padding.
> This is what Jan's patch did, and is quite wrong for overlays..

Committed.  I didn't see any occurrence of the problem Jan originally
reported in http://sourceware.org/ml/binutils/2008-05/msg00235.html on
i386 or powerpc using recent compilers so perhaps some other change
cured it.

bfd/
	* elf.c (assign_file_positions_for_load_sections): Revert 2008-05-29
	change.  Tidy.  Don't error on sections not allocated in segment.
ld/testsuite/
	* ld-elf/extract-symbol-1sec.d: Update lma.
	* ld-i386/alloc.d: Expect a warning, not an error.

Index: bfd/elf.c
===================================================================
RCS file: /cvs/src/src/bfd/elf.c,v
retrieving revision 1.508
diff -u -p -r1.508 elf.c
--- bfd/elf.c	10 Apr 2010 22:21:31 -0000	1.508
+++ bfd/elf.c	22 Apr 2010 01:10:22 -0000
@@ -4453,32 +4453,17 @@ assign_file_positions_for_load_sections 
 		      && ((this_hdr->sh_flags & SHF_TLS) == 0
 			  || p->p_type == PT_TLS))))
 	    {
-	      bfd_signed_vma adjust = sec->vma - (p->p_vaddr + p->p_memsz);
+	      bfd_vma adjust = sec->lma - (p->p_paddr + p->p_memsz);
 
-	      if (sec->vma < p->p_vaddr + p->p_memsz)
+	      if (sec->lma < p->p_paddr + p->p_memsz)
 		{
 		  (*_bfd_error_handler)
-		    (_("%B: section %A vma 0x%lx overlaps previous sections"),
-		     abfd, sec, (unsigned long) sec->vma);
+		    (_("%B: section %A lma 0x%lx overlaps previous sections"),
+		     abfd, sec, (unsigned long) sec->lma);
 		  adjust = 0;
-		}
-	      p->p_memsz += adjust;
-
-	      if (p->p_paddr + p->p_memsz != sec->lma)
-		{
-		  /* This behavior is a compromise--ld has long
-		     silently changed the lma of sections when
-		     lma - vma is not equal for every section in a
-		     pheader--but only in the internal elf structures.
-		     Silently changing the lma is probably a bug, but
-		     changing it would have subtle and unknown
-		     consequences for existing scripts.
-
-		     Instead modify the bfd data structure to reflect
-		     what happened.  This at least fixes the values
-		     for the lma in the mapfile.  */
 		  sec->lma = p->p_paddr + p->p_memsz;
 		}
+	      p->p_memsz += adjust;
 
 	      if (this_hdr->sh_type != SHT_NOBITS)
 		{
@@ -4581,8 +4566,6 @@ assign_file_positions_for_load_sections 
 		  (_("%B: section `%A' can't be allocated in segment %d"),
 		   abfd, sec, j);
 		print_segment_map (m);
-		bfd_set_error (bfd_error_bad_value);
-		return FALSE;
 	      }
 	  }
     }
Index: ld/testsuite/ld-elf/extract-symbol-1sec.d
===================================================================
RCS file: /cvs/src/src/ld/testsuite/ld-elf/extract-symbol-1sec.d,v
retrieving revision 1.5
diff -u -p -r1.5 extract-symbol-1sec.d
--- ld/testsuite/ld-elf/extract-symbol-1sec.d	18 Nov 2009 16:37:55 -0000	1.5
+++ ld/testsuite/ld-elf/extract-symbol-1sec.d	22 Apr 2010 01:10:22 -0000
@@ -8,7 +8,7 @@
 #...
 Sections:
  *Idx +Name +Size +VMA +LMA .*
- *0 +\.foo +0+ +0+10000 +0+ .*
+ *0 +\.foo +0+ +0+10000 +0+10000 .*
  *CONTENTS, ALLOC, LOAD, CODE
  *1 +\.bar +0+ +0+20000 +0+10000 .*
  *ALLOC, READONLY, CODE
Index: ld/testsuite/ld-i386/alloc.d
===================================================================
RCS file: /cvs/src/src/ld/testsuite/ld-i386/alloc.d,v
retrieving revision 1.1
diff -u -p -r1.1 alloc.d
--- ld/testsuite/ld-i386/alloc.d	22 Mar 2007 15:50:38 -0000	1.1
+++ ld/testsuite/ld-i386/alloc.d	22 Apr 2010 01:33:24 -0000
@@ -1,4 +1,4 @@
 #name: Invalid allocated section
 #as: --32
 #ld: -melf_i386 -T alloc.t
-#error: .*section `.foo' can't be allocated in segment 0.*
+#warning: .*section `.foo' can't be allocated in segment 0.*

-- 
Alan Modra
Australia Development Lab, IBM

RE: VMA section overlap warnings for overlays

[David Stubbs]

> Committed.  I didn't see any occurrence of the problem Jan
> originally
> reported in http://sourceware.org/ml/binutils/2008-
> 05/msg00235.html on
> i386 or powerpc using recent compilers so perhaps some other
> change
> cured it.

Thanks Alan. I've tested this and it fixes the problems we were having.
I've modified the warning slightly though (in our sources) to only emit
it if the section VMA != LMA - as in there's a chance that this is an
overlay. 

Dave.

Re: VMA section overlap warnings for overlays

[Alan Modra]

This patch silences the "section can't be allocated in segment"
warning in cases where people pack overlays into one segment.

I also took the opportunity to tidy associated SECTION_IN_SEGMENT
macros.

include/elf/
	* internal.h (ELF_SECTION_SIZE): Protect macro args with parentheses.
	Invert logic to clarify test for .tbss.
	(ELF_IS_SECTION_IN_SEGMENT): Rename to..
	(ELF_SECTION_IN_SEGMENT_1): ..this.  Add check_vma param.  Protect
	macro args with parentheses.
	(ELF_SECTION_IN_SEGMENT): Define.
	(ELF_IS_SECTION_IN_SEGMENT_FILE): Delete.
	(ELF_IS_SECTION_IN_SEGMENT_MEMORY): Delete.
bfd/
	* elf.c: Replace use of ELF_IS_SECTION_IN_SEGMENT and
	ELF_IS_SECTION_IN_SEGMENT_FILE with ELF_SECTION_IN_SEGMENT
	throughout file.
	(assign_file_positions_for_load_sections): Modify section in
	segment warning to ignore overlay vmas.
	* elf32-spu.c (spu_elf_object_p): Replace use of
	ELF_IS_SECTION_IN_SEGMENT_MEMORY with ELF_SECTION_IN_SEGMENT.
binutils/
	* readelf.c (process_program_headers): Replace use of
	ELF_IS_SECTION_IN_SEGMENT_MEMORY with ELF_SECTION_IN_SEGMENT.

Index: include/elf/internal.h
===================================================================
RCS file: /cvs/src/src/include/elf/internal.h,v
retrieving revision 1.24
diff -u -p -r1.24 internal.h
--- include/elf/internal.h	15 Apr 2010 10:26:08 -0000	1.24
+++ include/elf/internal.h	24 Apr 2010 01:03:03 -0000
@@ -292,42 +292,36 @@ struct elf_segment_map
 /* .tbss is special.  It doesn't contribute memory space to normal
    segments and it doesn't take file space in normal segments.  */
 #define ELF_SECTION_SIZE(sec_hdr, segment)			\
-   (((sec_hdr->sh_flags & SHF_TLS) == 0				\
-     || sec_hdr->sh_type != SHT_NOBITS				\
-     || segment->p_type == PT_TLS) ? sec_hdr->sh_size : 0)
+  ((!(((sec_hdr)->sh_flags & SHF_TLS) != 0			\
+      && (sec_hdr)->sh_type == SHT_NOBITS)			\
+    || (segment)->p_type == PT_TLS) ? (sec_hdr)->sh_size : 0)
 
 /* Decide if the given sec_hdr is in the given segment.  PT_TLS segment
    contains only SHF_TLS sections.  Only PT_LOAD, PT_GNU_RELRO and
    and PT_TLS segments can contain SHF_TLS sections.  */
-#define ELF_IS_SECTION_IN_SEGMENT(sec_hdr, segment)			\
-  (((((sec_hdr->sh_flags & SHF_TLS) != 0)				\
-     && (segment->p_type == PT_TLS					\
-	 || segment->p_type == PT_GNU_RELRO				\
-	 || segment->p_type == PT_LOAD))				\
-    || ((sec_hdr->sh_flags & SHF_TLS) == 0				\
-	&& segment->p_type != PT_TLS					\
-	&& segment->p_type != PT_PHDR))					\
+#define ELF_SECTION_IN_SEGMENT_1(sec_hdr, segment, check_vma)		\
+  ((((((sec_hdr)->sh_flags & SHF_TLS) != 0)				\
+     && ((segment)->p_type == PT_TLS					\
+	 || (segment)->p_type == PT_GNU_RELRO				\
+	 || (segment)->p_type == PT_LOAD))				\
+    || (((sec_hdr)->sh_flags & SHF_TLS) == 0				\
+	&& (segment)->p_type != PT_TLS					\
+	&& (segment)->p_type != PT_PHDR))				\
    /* Any section besides one of type SHT_NOBITS must have a file	\
       offset within the segment.  */					\
-   && (sec_hdr->sh_type == SHT_NOBITS					\
-       || ((bfd_vma) sec_hdr->sh_offset >= segment->p_offset		\
-	   && (sec_hdr->sh_offset + ELF_SECTION_SIZE(sec_hdr, segment)	\
-	       <= segment->p_offset + segment->p_filesz)))		\
+   && ((sec_hdr)->sh_type == SHT_NOBITS					\
+       || ((bfd_vma) (sec_hdr)->sh_offset >= (segment)->p_offset	\
+	   && ((sec_hdr)->sh_offset + ELF_SECTION_SIZE(sec_hdr, segment) \
+	       <= (segment)->p_offset + (segment)->p_filesz)))		\
    /* SHF_ALLOC sections must have VMAs within the segment.  Be		\
       careful about segments right at the end of memory.  */		\
-   && ((sec_hdr->sh_flags & SHF_ALLOC) == 0				\
-       || (sec_hdr->sh_addr >= segment->p_vaddr				\
-	   && (sec_hdr->sh_addr - segment->p_vaddr			\
-	       + ELF_SECTION_SIZE(sec_hdr, segment) <= segment->p_memsz))))
+   && (!(check_vma)							\
+       || ((sec_hdr)->sh_flags & SHF_ALLOC) == 0			\
+       || ((sec_hdr)->sh_addr >= (segment)->p_vaddr			\
+	   && ((sec_hdr)->sh_addr - (segment)->p_vaddr			\
+	       + ELF_SECTION_SIZE(sec_hdr, segment) <= (segment)->p_memsz))))
 
-/* Decide if the given sec_hdr is in the given segment in file.  */
-#define ELF_IS_SECTION_IN_SEGMENT_FILE(sec_hdr, segment)	\
-  (sec_hdr->sh_size > 0						\
-   && ELF_IS_SECTION_IN_SEGMENT (sec_hdr, segment))
-
-/* Decide if the given sec_hdr is in the given segment in memory.  */
-#define ELF_IS_SECTION_IN_SEGMENT_MEMORY(sec_hdr, segment)	\
-  (ELF_SECTION_SIZE(sec_hdr, segment) > 0			\
-   && ELF_IS_SECTION_IN_SEGMENT (sec_hdr, segment))
+#define ELF_SECTION_IN_SEGMENT(sec_hdr, segment)			\
+  (ELF_SECTION_IN_SEGMENT_1 (sec_hdr, segment, 1))
 
 #endif /* _ELF_INTERNAL_H */
Index: bfd/elf.c
===================================================================
RCS file: /cvs/src/src/bfd/elf.c,v
retrieving revision 1.509
diff -u -p -r1.509 elf.c
--- bfd/elf.c	22 Apr 2010 01:51:00 -0000	1.509
+++ bfd/elf.c	24 Apr 2010 01:03:06 -0000
@@ -978,7 +978,7 @@ _bfd_elf_make_section_from_shdr (bfd *ab
       for (i = 0; i < elf_elfheader (abfd)->e_phnum; i++, phdr++)
 	{
 	  if (phdr->p_type == PT_LOAD
-	      && ELF_IS_SECTION_IN_SEGMENT (hdr, phdr))
+	      && ELF_SECTION_IN_SEGMENT (hdr, phdr))
 	    {
 	      if ((flags & SEC_LOAD) == 0)
 		newsect->lma = (phdr->p_paddr
@@ -4552,22 +4552,38 @@ assign_file_positions_for_load_sections 
       /* Check that all sections are in a PT_LOAD segment.
 	 Don't check funky gdb generated core files.  */
       if (p->p_type == PT_LOAD && bfd_get_format (abfd) != bfd_core)
-	for (i = 0, secpp = m->sections; i < m->count; i++, secpp++)
-	  {
-	    Elf_Internal_Shdr *this_hdr;
-	    asection *sec;
+	{
+	  bfd_boolean check_vma = TRUE;
 
-	    sec = *secpp;
-	    this_hdr = &(elf_section_data(sec)->this_hdr);
-	    if (this_hdr->sh_size != 0
-		&& !ELF_IS_SECTION_IN_SEGMENT_FILE (this_hdr, p))
+	  for (i = 1; i < m->count; i++)
+	    if (m->sections[i]->vma == m->sections[i - 1]->vma
+		&& ELF_SECTION_SIZE (&(elf_section_data (m->sections[i])
+				       ->this_hdr), p) != 0
+		&& ELF_SECTION_SIZE (&(elf_section_data (m->sections[i - 1])
+				       ->this_hdr), p) != 0)
 	      {
-		(*_bfd_error_handler)
-		  (_("%B: section `%A' can't be allocated in segment %d"),
-		   abfd, sec, j);
-		print_segment_map (m);
+		/* Looks like we have overlays packed into the segment.  */
+		check_vma = FALSE;
+		break;
 	      }
-	  }
+
+	  for (i = 0; i < m->count; i++)
+	    {
+	      Elf_Internal_Shdr *this_hdr;
+	      asection *sec;
+
+	      sec = m->sections[i];
+	      this_hdr = &(elf_section_data(sec)->this_hdr);
+	      if (this_hdr->sh_size != 0
+		  && !ELF_SECTION_IN_SEGMENT_1 (this_hdr, p, check_vma))
+		{
+		  (*_bfd_error_handler)
+		    (_("%B: section `%A' can't be allocated in segment %d"),
+		     abfd, sec, j);
+		  print_segment_map (m);
+		}
+	    }
+	}
     }
 
   elf_tdata (abfd)->next_file_pos = off;
@@ -5837,7 +5853,8 @@ copy_elf_program_header (bfd *ibfd, bfd 
 	   section = section->next)
 	{
 	  this_hdr = &(elf_section_data(section)->this_hdr);
-	  if (ELF_IS_SECTION_IN_SEGMENT_FILE (this_hdr, segment))
+	  if (this_hdr->sh_size != 0
+	      && ELF_SECTION_IN_SEGMENT (this_hdr, segment))
 	    {
 	      if (!first_section)
 		first_section = lowest_section = section;
@@ -5916,7 +5933,8 @@ copy_elf_program_header (bfd *ibfd, bfd 
 	       section = section->next)
 	    {
 	      this_hdr = &(elf_section_data(section)->this_hdr);
-	      if (ELF_IS_SECTION_IN_SEGMENT_FILE (this_hdr, segment))
+	      if (this_hdr->sh_size != 0
+		  && ELF_SECTION_IN_SEGMENT (this_hdr, segment))
 		{
 		  map->sections[isec++] = section->output_section;
 		  if (isec == section_count)
@@ -5993,7 +6011,8 @@ copy_private_bfd_data (bfd *ibfd, bfd *o
 
 	      /* Check if this section is covered by the segment.  */
 	      this_hdr = &(elf_section_data(section)->this_hdr);
-	      if (ELF_IS_SECTION_IN_SEGMENT_FILE (this_hdr, segment))
+	      if (this_hdr->sh_size != 0
+		  && ELF_SECTION_IN_SEGMENT (this_hdr, segment))
 		{
 		  /* FIXME: Check if its output section is changed or
 		     removed.  What else do we need to check?  */
Index: bfd/elf32-spu.c
===================================================================
RCS file: /cvs/src/src/bfd/elf32-spu.c,v
retrieving revision 1.94
diff -u -p -r1.94 elf32-spu.c
--- bfd/elf32-spu.c	4 Feb 2010 09:16:40 -0000	1.94
+++ bfd/elf32-spu.c	24 Apr 2010 01:03:08 -0000
@@ -272,7 +272,8 @@ spu_elf_object_p (bfd *abfd)
 	      {
 		Elf_Internal_Shdr *shdr = elf_elfsections (abfd)[j];
 
-		if (ELF_IS_SECTION_IN_SEGMENT_MEMORY (shdr, phdr))
+		if (ELF_SECTION_SIZE (shdr, phdr) != 0
+		    && ELF_SECTION_IN_SEGMENT (shdr, phdr))
 		  {
 		    asection *sec = shdr->bfd_section;
 		    spu_elf_section_data (sec)->u.o.ovl_index = num_ovl;
Index: binutils/readelf.c
===================================================================
RCS file: /cvs/src/src/binutils/readelf.c,v
retrieving revision 1.494
diff -u -p -r1.494 readelf.c
--- binutils/readelf.c	20 Apr 2010 08:21:54 -0000	1.494
+++ binutils/readelf.c	24 Apr 2010 01:03:12 -0000
@@ -3850,7 +3850,8 @@ process_program_headers (FILE * file)
 
 	  for (j = 1; j < elf_header.e_shnum; j++, section++)
 	    {
-	      if (ELF_IS_SECTION_IN_SEGMENT_MEMORY (section, segment))
+	      if (ELF_SECTION_SIZE (section, segment) != 0
+		  && ELF_SECTION_IN_SEGMENT (section, segment))
 		printf ("%s ", SECTION_NAME (section));
 	    }
 

-- 
Alan Modra
Australia Development Lab, IBM

RE: VMA section overlap warnings for overlays

[David Stubbs]

Hi,

If we have a single segment containing a BSS section followed by
overlays then we still get the overlap warning. 

Does the following patch look like the correct fix? I believe it fixes
my original test case, and I've also tested it on i686-pc-linux-gnu
without any problems.

Thanks,
Dave.

Bfd/ChangeLog:

2010-05-07  David Stubbs  <stubbs@icerasemi.com>

	* elf.c (assign_file_positions_for_load_sections): Calculate
adjust
	based on file size instead of memory size. Increment p->p_memsz
by 
	vma based calculation instead of adjust.

Index: bfd/elf.c
===================================================================
RCS file: /cvs/src/src/bfd/elf.c,v
retrieving revision 1.510
diff -u -p -r1.510 elf.c
--- bfd/elf.c	24 Apr 2010 01:05:24 -0000	1.510
+++ bfd/elf.c	7 May 2010 15:41:09 -0000
@@ -4453,17 +4453,17 @@ assign_file_positions_for_load_sections 
 		      && ((this_hdr->sh_flags & SHF_TLS) == 0
 			  || p->p_type == PT_TLS))))
 	    {
-	      bfd_vma adjust = sec->lma - (p->p_paddr + p->p_memsz);
+	      bfd_vma adjust = sec->lma - (p->p_paddr + p->p_filesz);
 
-	      if (sec->lma < p->p_paddr + p->p_memsz)
+	      if (sec->lma < p->p_paddr + p->p_filesz)
 		{
 		  (*_bfd_error_handler)
 		    (_("%B: section %A lma 0x%lx overlaps previous
sections"),
 		     abfd, sec, (unsigned long) sec->lma);
 		  adjust = 0;
-		  sec->lma = p->p_paddr + p->p_memsz;
+		  sec->lma = p->p_paddr + p->p_filesz;
 		}
-	      p->p_memsz += adjust;
+	      p->p_memsz += sec->vma - (p->p_vaddr + p->p_memsz);
 
 	      if (this_hdr->sh_type != SHT_NOBITS)
 		{

Re: VMA section overlap warnings for overlays

[Alan Modra]

On Fri, May 07, 2010 at 05:14:06PM +0100, David Stubbs wrote:
> Does the following patch look like the correct fix?

No, it doesn't look correct.  I think your problem is more likely due
to a ldlang.c bug in setting lma.  Do you have a testcase handy?

-- 
Alan Modra
Australia Development Lab, IBM

RE: VMA section overlap warnings for overlays

[David Stubbs]

> On Fri, May 07, 2010 at 05:14:06PM +0100, David Stubbs wrote:
> > Does the following patch look like the correct fix?
> 
> No, it doesn't look correct.  I think your problem is more likely
> due
> to a ldlang.c bug in setting lma.  Do you have a testcase handy?

Thanks for the response. I didn't realize / think that you needed to
allocate filespace for bss sections when they're followed by non BSS
sections in the same segment. 

I wasn't able to reproduce the problem, but that turned out to be
because I'd somehow lost the bit of code that calls write_zeros in our
copy of the source.

I have however found another potential problem with strip. Here's the
test:

---
brsbs01 0>cat start.s
  	.section	.text,"ax",@progbits
        .globl  _start
        .type   _start, @function
_start:
        nop
       
        .section .dmem.bss,"aw",@nobits
        .zero 0x10
brsbs01 0>cat over.S
        .section SEC_NAME,"ax",@progbits
        .zero 0x10

        .section SEC_NAME.bss,"aw",@nobits
        .zero 0x10
brsbs01 0>cat ldscript.x
OUTPUT_FORMAT("elf32-i386")
OUTPUT_ARCH(i386)

ENTRY(_start)

PHDRS
{
  mem PT_LOAD;
  imem PT_LOAD;
}

SECTIONS
{
  .text 0x1000:
  {
    *(.text)
  } :mem

  .dmem.bss 0x2000
    : AT (LOADADDR (.text) + SIZEOF (.text))
  {
    *(.dmem.bss)
  } :imem

  .overlay1 ADDR (.dmem.bss) + SIZEOF (.dmem.bss)
    : AT (LOADADDR (.dmem.bss) + SIZEOF (.dmem.bss))
  {
    *(.overlay1)
  } :imem
      
  .overlay2 ADDR (.dmem.bss) + SIZEOF (.dmem.bss)
    : AT (LOADADDR (.overlay1) + SIZEOF (.overlay1))
  {
    *(.overlay2)
  } :imem

  .overlay1.bss ADDR (.overlay1) + SIZEOF (.overlay1)
    : AT (LOADADDR (.overlay2) + SIZEOF (.overlay2))
  {
    *(.overlay1.bss)
  } :imem

  .overlay2.bss ADDR (.overlay2) + SIZEOF (.overlay2)
    : AT (LOADADDR (.overlay1.bss) + SIZEOF (.overlay1.bss))
  {
    *(.overlay1.bss)
  } :imem

}
brsbs01 0>gcc over.S -DSEC_NAME=.overlay1 -c -o over1.o
brsbs01 0>gcc over.S -DSEC_NAME=.overlay2 -c -o over2.o
brsbs01 0>gcc start.s -c -o start.o
brsbs01 0>ld over1.o over2.o start.o -T ldscript.x -o test.exe 
brsbs01 1>strip test.exe
BFD: stHVOxcK: section .overlay1.bss lma 0x1021 overlaps previous
sections
BFD: stHVOxcK: section .overlay2.bss lma 0x1021 overlaps previous
sections
---

The overlap warning is being emitted because 'adjust' is being
calculated as a negative number. I don't yet know why.

Thanks again,
Dave.

Re: VMA section overlap warnings for overlays

[Alan Modra]

On Wed, May 12, 2010 at 02:19:24PM +0100, David Stubbs wrote:
> I have however found another potential problem with strip. Here's the
> test:
[snip]
> brsbs01 1>strip test.exe
> BFD: stHVOxcK: section .overlay1.bss lma 0x1021 overlaps previous
> sections
> BFD: stHVOxcK: section .overlay2.bss lma 0x1021 overlaps previous
> sections

This is caused by the heuristics we use to set LMA for SHT_NOBITS
sections (bfd/elf.c:984).  I don't think we can make any changes here
though: sh_offset isn't really valid for a SHT_NOBITS section.

-- 
Alan Modra
Australia Development Lab, IBM

Re: VMA section overlap warnings for overlays

[Andreas Schwab]

Alan Modra <amodra@gmail.com> writes:

> @@ -5837,7 +5853,8 @@ copy_elf_program_header (bfd *ibfd, bfd 
>  	   section = section->next)
>  	{
>  	  this_hdr = &(elf_section_data(section)->this_hdr);
> -	  if (ELF_IS_SECTION_IN_SEGMENT_FILE (this_hdr, segment))
> +	  if (this_hdr->sh_size != 0
> +	      && ELF_SECTION_IN_SEGMENT (this_hdr, segment))

Why can't an empty section never be part of a segment?

Andreas.

-- 
Andreas Schwab, schwab@redhat.com
GPG Key fingerprint = D4E8 DBE3 3813 BB5D FA84  5EC7 45C6 250E 6F00 984E
"And now for something completely different."

Re: VMA section overlap warnings for overlays

[Alan Modra]

On Thu, Jul 15, 2010 at 10:11:03AM +0200, Andreas Schwab wrote:
> Alan Modra <amodra@gmail.com> writes:
> 
> > @@ -5837,7 +5853,8 @@ copy_elf_program_header (bfd *ibfd, bfd 
> >  	   section = section->next)
> >  	{
> >  	  this_hdr = &(elf_section_data(section)->this_hdr);
> > -	  if (ELF_IS_SECTION_IN_SEGMENT_FILE (this_hdr, segment))
> > +	  if (this_hdr->sh_size != 0
> > +	      && ELF_SECTION_IN_SEGMENT (this_hdr, segment))
> 
> Why can't an empty section never be part of a segment?

Good question.  I didn't change anything with that patch.
HJ's ELF_IS_SECTION_IN_SEGMENT_FILE included the sh_size check.

-- 
Alan Modra
Australia Development Lab, IBM

Re: VMA section overlap warnings for overlays

[Andreas Schwab]

Alan Modra <amodra@gmail.com> writes:

> On Thu, Jul 15, 2010 at 10:11:03AM +0200, Andreas Schwab wrote:
>> Alan Modra <amodra@gmail.com> writes:
>> 
>> > @@ -5837,7 +5853,8 @@ copy_elf_program_header (bfd *ibfd, bfd 
>> >  	   section = section->next)
>> >  	{
>> >  	  this_hdr = &(elf_section_data(section)->this_hdr);
>> > -	  if (ELF_IS_SECTION_IN_SEGMENT_FILE (this_hdr, segment))
>> > +	  if (this_hdr->sh_size != 0
>> > +	      && ELF_SECTION_IN_SEGMENT (this_hdr, segment))
>> 
>> Why can't an empty section never be part of a segment?
>
> Good question.  I didn't change anything with that patch.
> HJ's ELF_IS_SECTION_IN_SEGMENT_FILE included the sh_size check.

The problem is that it clobbers the file offset of such an empty section
during objcopy/strip.

$ binutils/readelf -WSl libfreebl3.so 
There are 41 section headers, starting at offset 0xe25b8:

Section Headers:
  [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            0000000000000000 000000 000000 00      0   0  0
  [ 1] .note.gnu.build-id NOTE            0000000000000190 000190 000024 00   A  0   0  4
  [ 2] .gnu.hash         GNU_HASH        00000000000001b8 0001b8 000050 00   A  3   0  8
  [ 3] .dynsym           DYNSYM          0000000000000208 000208 000798 18   A  4   3  8
  [ 4] .dynstr           STRTAB          00000000000009a0 0009a0 000336 00   A  0   0  1
  [ 5] .gnu.version      VERSYM          0000000000000cd6 000cd6 0000a2 02   A  3   0  2
  [ 6] .gnu.version_d    VERDEF          0000000000000d78 000d78 000054 00   A  4   3  8
  [ 7] .gnu.version_r    VERNEED         0000000000000dd0 000dd0 000070 00   A  4   2  8
  [ 8] .rela.dyn         RELA            0000000000000e40 000e40 007bd8 18   A  3   0  8
  [ 9] .rela.plt         RELA            0000000000008a18 008a18 0005b8 18   A  3  26  8
  [10] .init             PROGBITS        0000000000008fd0 008fd0 000058 00  AX  0   0  8
  [11] .text             PROGBITS        0000000000009030 009030 03b1a8 00  AX  0   0 16
  [12] .fini             PROGBITS        00000000000441d8 0441d8 000028 00  AX  0   0  8
  [13] .rodata           PROGBITS        0000000000044200 044200 013bb0 00   A  0   0  8
  [14] .eh_frame_hdr     PROGBITS        0000000000057db0 057db0 000e5c 00   A  0   0  4
  [15] .eh_frame         PROGBITS        0000000000058c10 058c10 004f04 00   A  0   0  8
  [16] .ctors            PROGBITS        0000000000060000 060000 000010 00  WA  0   0  8
  [17] .dtors            PROGBITS        0000000000060010 060010 000018 00  WA  0   0  8
  [18] .jcr              PROGBITS        0000000000060028 060028 000008 00  WA  0   0  8
  [19] .data.rel.ro      PROGBITS        0000000000060030 060030 0007a8 00  WA  0   0  8
  [20] .dynamic          DYNAMIC         00000000000607d8 0607d8 0001e0 10  WA  4   0  8
  [21] .data             PROGBITS        00000000000609b8 0609b8 000024 00  WA  0   0  8
  [22] .toc1             PROGBITS        00000000000609e0 0609e0 000418 00  WA  0   0  8
  [23] .opd              PROGBITS        0000000000060df8 060df8 001cf8 00  WA  0   0  8
  [24] .branch_lt        PROGBITS        0000000000062af0 062af0 000000 00  WA  0   0  8
  [25] .got              PROGBITS        0000000000062af0 062af0 000130 08  WA  0   0  8
  [26] .plt              NOBITS          0000000000062c20 062c20 0005d0 18  WA  0   0  8
  [27] .bss              NOBITS          00000000000631f0 062c20 004490 00  WA  0   0  8
  [28] .comment          PROGBITS        0000000000000000 062c20 000059 01  MS  0   0  1
  [29] .debug_aranges    PROGBITS        0000000000000000 062c79 000630 00      0   0  1
  [30] .debug_pubnames   PROGBITS        0000000000000000 0632a9 002088 00      0   0  1
  [31] .debug_info       PROGBITS        0000000000000000 065331 0246a2 00      0   0  1
  [32] .debug_abbrev     PROGBITS        0000000000000000 0899d3 004c0b 00      0   0  1
  [33] .debug_line       PROGBITS        0000000000000000 08e5de 00b8cf 00      0   0  1
  [34] .debug_str        PROGBITS        0000000000000000 099ead 006261 01  MS  0   0  1
  [35] .debug_loc        PROGBITS        0000000000000000 0a010e 034c46 00      0   0  1
  [36] .debug_pubtypes   PROGBITS        0000000000000000 0d4d54 002f3a 00      0   0  1
  [37] .debug_ranges     PROGBITS        0000000000000000 0d7c8e 00a7a0 00      0   0  1
  [38] .shstrtab         STRTAB          0000000000000000 0e242e 00018a 00      0   0  1
  [39] .symtab           SYMTAB          0000000000000000 0e2ff8 004a10 18     40 713  8
  [40] .strtab           STRTAB          0000000000000000 0e7a08 00316d 00      0   0  1
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings)
  I (info), L (link order), G (group), x (unknown)
  O (extra OS processing required) o (OS specific), p (processor specific)

Elf file type is DYN (Shared object file)
Entry point 0x60df8
There are 6 program headers, starting at offset 64

Program Headers:
  Type           Offset   VirtAddr           PhysAddr           FileSiz  MemSiz   Flg Align
  LOAD           0x000000 0x0000000000000000 0x0000000000000000 0x05db14 0x05db14 R E 0x10000
  LOAD           0x060000 0x0000000000060000 0x0000000000060000 0x002c20 0x007680 RW  0x10000
  DYNAMIC        0x0607d8 0x00000000000607d8 0x00000000000607d8 0x0001e0 0x0001e0 RW  0x8
  NOTE           0x000190 0x0000000000000190 0x0000000000000190 0x000024 0x000024 R   0x4
  GNU_EH_FRAME   0x057db0 0x0000000000057db0 0x0000000000057db0 0x000e5c 0x000e5c R   0x4
  GNU_STACK      0x000000 0x0000000000000000 0x0000000000000000 0x000000 0x000000 RW  0x8

 Section to Segment mapping:
  Segment Sections...
   00     .note.gnu.build-id .gnu.hash .dynsym .dynstr .gnu.version .gnu.version_d .gnu.version_r .rela.dyn .rela.plt .init .text .fini .rodata .eh_frame_hdr .eh_frame 
   01     .ctors .dtors .jcr .data.rel.ro .dynamic .data .toc1 .opd .got .plt .bss 
   02     .dynamic 
   03     .note.gnu.build-id 
   04     .eh_frame_hdr 
   05     
$ binutils/objcopy libfreebl3.so libfreebl3.so-copy
$ diff -u <(binutils/readelf -WSl libfreebl3.so) <(binutils/readelf -WSl libfreebl3.so-copy)
--- /proc/self/fd/63	2010-07-15 15:37:37.162244967 +0200
+++ /proc/self/fd/62	2010-07-15 15:37:37.163244682 +0200
@@ -26,7 +26,7 @@
   [21] .data             PROGBITS        00000000000609b8 0609b8 000024 00  WA  0   0  8
   [22] .toc1             PROGBITS        00000000000609e0 0609e0 000418 00  WA  0   0  8
   [23] .opd              PROGBITS        0000000000060df8 060df8 001cf8 00  WA  0   0  8
-  [24] .branch_lt        PROGBITS        0000000000062af0 062af0 000000 00  WA  0   0  8
+  [24] .branch_lt        PROGBITS        0000000000062af0 062c20 000000 00  WA  0   0  8
   [25] .got              PROGBITS        0000000000062af0 062af0 000130 08  WA  0   0  8
   [26] .plt              NOBITS          0000000000062c20 062c20 0005d0 18  WA  0   0  8
   [27] .bss              NOBITS          00000000000631f0 062c20 004490 00  WA  0   0  8

The sh_offset of .branch_lt no longer agrees with sh_addr modulo segment
alignment.

Andreas.

-- 
Andreas Schwab, schwab@redhat.com
GPG Key fingerprint = D4E8 DBE3 3813 BB5D FA84  5EC7 45C6 250E 6F00 984E
"And now for something completely different."

Re: VMA section overlap warnings for overlays

[H.J. Lu]

On Thu, Jul 15, 2010 at 6:46 AM, Andreas Schwab <schwab@redhat.com> wrote:
> Alan Modra <amodra@gmail.com> writes:
>
>> On Thu, Jul 15, 2010 at 10:11:03AM +0200, Andreas Schwab wrote:
>>> Alan Modra <amodra@gmail.com> writes:
>>>
>>> > @@ -5837,7 +5853,8 @@ copy_elf_program_header (bfd *ibfd, bfd
>>> >       section = section->next)
>>> >    {
>>> >      this_hdr = &(elf_section_data(section)->this_hdr);
>>> > -    if (ELF_IS_SECTION_IN_SEGMENT_FILE (this_hdr, segment))
>>> > +    if (this_hdr->sh_size != 0
>>> > +        && ELF_SECTION_IN_SEGMENT (this_hdr, segment))
>>>
>>> Why can't an empty section never be part of a segment?
>>
>> Good question.  I didn't change anything with that patch.
>> HJ's ELF_IS_SECTION_IN_SEGMENT_FILE included the sh_size check.
>
> The problem is that it clobbers the file offset of such an empty section
> during objcopy/strip.
>
> $ binutils/readelf -WSl libfreebl3.so
> There are 41 section headers, starting at offset 0xe25b8:
>
> Section Headers:
>  [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
>  [ 0]                   NULL            0000000000000000 000000 000000 00      0   0  0
>  [ 1] .note.gnu.build-id NOTE            0000000000000190 000190 000024 00   A  0   0  4
>  [ 2] .gnu.hash         GNU_HASH        00000000000001b8 0001b8 000050 00   A  3   0  8
>  [ 3] .dynsym           DYNSYM          0000000000000208 000208 000798 18   A  4   3  8
>  [ 4] .dynstr           STRTAB          00000000000009a0 0009a0 000336 00   A  0   0  1
>  [ 5] .gnu.version      VERSYM          0000000000000cd6 000cd6 0000a2 02   A  3   0  2
>  [ 6] .gnu.version_d    VERDEF          0000000000000d78 000d78 000054 00   A  4   3  8
>  [ 7] .gnu.version_r    VERNEED         0000000000000dd0 000dd0 000070 00   A  4   2  8
>  [ 8] .rela.dyn         RELA            0000000000000e40 000e40 007bd8 18   A  3   0  8
>  [ 9] .rela.plt         RELA            0000000000008a18 008a18 0005b8 18   A  3  26  8
>  [10] .init             PROGBITS        0000000000008fd0 008fd0 000058 00  AX  0   0  8
>  [11] .text             PROGBITS        0000000000009030 009030 03b1a8 00  AX  0   0 16
>  [12] .fini             PROGBITS        00000000000441d8 0441d8 000028 00  AX  0   0  8
>  [13] .rodata           PROGBITS        0000000000044200 044200 013bb0 00   A  0   0  8
>  [14] .eh_frame_hdr     PROGBITS        0000000000057db0 057db0 000e5c 00   A  0   0  4
>  [15] .eh_frame         PROGBITS        0000000000058c10 058c10 004f04 00   A  0   0  8
>  [16] .ctors            PROGBITS        0000000000060000 060000 000010 00  WA  0   0  8
>  [17] .dtors            PROGBITS        0000000000060010 060010 000018 00  WA  0   0  8
>  [18] .jcr              PROGBITS        0000000000060028 060028 000008 00  WA  0   0  8
>  [19] .data.rel.ro      PROGBITS        0000000000060030 060030 0007a8 00  WA  0   0  8
>  [20] .dynamic          DYNAMIC         00000000000607d8 0607d8 0001e0 10  WA  4   0  8
>  [21] .data             PROGBITS        00000000000609b8 0609b8 000024 00  WA  0   0  8
>  [22] .toc1             PROGBITS        00000000000609e0 0609e0 000418 00  WA  0   0  8
>  [23] .opd              PROGBITS        0000000000060df8 060df8 001cf8 00  WA  0   0  8
>  [24] .branch_lt        PROGBITS        0000000000062af0 062af0 000000 00  WA  0   0  8
>  [25] .got              PROGBITS        0000000000062af0 062af0 000130 08  WA  0   0  8
>  [26] .plt              NOBITS          0000000000062c20 062c20 0005d0 18  WA  0   0  8
>  [27] .bss              NOBITS          00000000000631f0 062c20 004490 00  WA  0   0  8
>  [28] .comment          PROGBITS        0000000000000000 062c20 000059 01  MS  0   0  1
>  [29] .debug_aranges    PROGBITS        0000000000000000 062c79 000630 00      0   0  1
>  [30] .debug_pubnames   PROGBITS        0000000000000000 0632a9 002088 00      0   0  1
>  [31] .debug_info       PROGBITS        0000000000000000 065331 0246a2 00      0   0  1
>  [32] .debug_abbrev     PROGBITS        0000000000000000 0899d3 004c0b 00      0   0  1
>  [33] .debug_line       PROGBITS        0000000000000000 08e5de 00b8cf 00      0   0  1
>  [34] .debug_str        PROGBITS        0000000000000000 099ead 006261 01  MS  0   0  1
>  [35] .debug_loc        PROGBITS        0000000000000000 0a010e 034c46 00      0   0  1
>  [36] .debug_pubtypes   PROGBITS        0000000000000000 0d4d54 002f3a 00      0   0  1
>  [37] .debug_ranges     PROGBITS        0000000000000000 0d7c8e 00a7a0 00      0   0  1
>  [38] .shstrtab         STRTAB          0000000000000000 0e242e 00018a 00      0   0  1
>  [39] .symtab           SYMTAB          0000000000000000 0e2ff8 004a10 18     40 713  8
>  [40] .strtab           STRTAB          0000000000000000 0e7a08 00316d 00      0   0  1
> Key to Flags:
>  W (write), A (alloc), X (execute), M (merge), S (strings)
>  I (info), L (link order), G (group), x (unknown)
>  O (extra OS processing required) o (OS specific), p (processor specific)
>
> Elf file type is DYN (Shared object file)
> Entry point 0x60df8
> There are 6 program headers, starting at offset 64
>
> Program Headers:
>  Type           Offset   VirtAddr           PhysAddr           FileSiz  MemSiz   Flg Align
>  LOAD           0x000000 0x0000000000000000 0x0000000000000000 0x05db14 0x05db14 R E 0x10000
>  LOAD           0x060000 0x0000000000060000 0x0000000000060000 0x002c20 0x007680 RW  0x10000
>  DYNAMIC        0x0607d8 0x00000000000607d8 0x00000000000607d8 0x0001e0 0x0001e0 RW  0x8
>  NOTE           0x000190 0x0000000000000190 0x0000000000000190 0x000024 0x000024 R   0x4
>  GNU_EH_FRAME   0x057db0 0x0000000000057db0 0x0000000000057db0 0x000e5c 0x000e5c R   0x4
>  GNU_STACK      0x000000 0x0000000000000000 0x0000000000000000 0x000000 0x000000 RW  0x8
>
>  Section to Segment mapping:
>  Segment Sections...
>   00     .note.gnu.build-id .gnu.hash .dynsym .dynstr .gnu.version .gnu.version_d .gnu.version_r .rela.dyn .rela.plt .init .text .fini .rodata .eh_frame_hdr .eh_frame
>   01     .ctors .dtors .jcr .data.rel.ro .dynamic .data .toc1 .opd .got .plt .bss
>   02     .dynamic
>   03     .note.gnu.build-id
>   04     .eh_frame_hdr
>   05
> $ binutils/objcopy libfreebl3.so libfreebl3.so-copy
> $ diff -u <(binutils/readelf -WSl libfreebl3.so) <(binutils/readelf -WSl libfreebl3.so-copy)
> --- /proc/self/fd/63    2010-07-15 15:37:37.162244967 +0200
> +++ /proc/self/fd/62    2010-07-15 15:37:37.163244682 +0200
> @@ -26,7 +26,7 @@
>   [21] .data             PROGBITS        00000000000609b8 0609b8 000024 00  WA  0   0  8
>   [22] .toc1             PROGBITS        00000000000609e0 0609e0 000418 00  WA  0   0  8
>   [23] .opd              PROGBITS        0000000000060df8 060df8 001cf8 00  WA  0   0  8
> -  [24] .branch_lt        PROGBITS        0000000000062af0 062af0 000000 00  WA  0   0  8
> +  [24] .branch_lt        PROGBITS        0000000000062af0 062c20 000000 00  WA  0   0  8
>   [25] .got              PROGBITS        0000000000062af0 062af0 000130 08  WA  0   0  8
>   [26] .plt              NOBITS          0000000000062c20 062c20 0005d0 18  WA  0   0  8
>   [27] .bss              NOBITS          00000000000631f0 062c20 004490 00  WA  0   0  8
>
> The sh_offset of .branch_lt no longer agrees with sh_addr modulo segment
> alignment.
>

What is an empty section used for?

-- 
H.J.

Re: VMA section overlap warnings for overlays

[Andreas Schwab]

"H.J. Lu" <hjl.tools@gmail.com> writes:

> What is an empty section used for?

It is present.

Andreas.

-- 
Andreas Schwab, schwab@redhat.com
GPG Key fingerprint = D4E8 DBE3 3813 BB5D FA84  5EC7 45C6 250E 6F00 984E
"And now for something completely different."

Re: VMA section overlap warnings for overlays

[H.J. Lu]

On Thu, Jul 15, 2010 at 7:12 AM, Andreas Schwab <schwab@redhat.com> wrote:
> "H.J. Lu" <hjl.tools@gmail.com> writes:
>
>> What is an empty section used for?
>
> It is present.
>

Should it be there?


-- 
H.J.

Re: VMA section overlap warnings for overlays

[Alan Modra]

On Thu, Jul 15, 2010 at 06:53:39AM -0700, H.J. Lu wrote:
> On Thu, Jul 15, 2010 at 6:46 AM, Andreas Schwab <schwab@redhat.com> wrote:
> > The sh_offset of .branch_lt no longer agrees with sh_addr modulo segment
> > alignment.
> 
> What is an empty section used for?

This particular section stores addresses used by long branch stubs.
I'd be happier if it was eliminated when no long branch stubs were
needed, but that turns out to be difficult.  So unfortunately ppc64
binaries get these zero sized sections.

-- 
Alan Modra
Australia Development Lab, IBM

Re: VMA section overlap warnings for overlays

[H.J. Lu]

On Thu, Jul 15, 2010 at 7:17 AM, Alan Modra <amodra@gmail.com> wrote:
> On Thu, Jul 15, 2010 at 06:53:39AM -0700, H.J. Lu wrote:
>> On Thu, Jul 15, 2010 at 6:46 AM, Andreas Schwab <schwab@redhat.com> wrote:
>> > The sh_offset of .branch_lt no longer agrees with sh_addr modulo segment
>> > alignment.
>>
>> What is an empty section used for?
>
> This particular section stores addresses used by long branch stubs.
> I'd be happier if it was eliminated when no long branch stubs were
> needed, but that turns out to be difficult.  So unfortunately ppc64
> binaries get these zero sized sections.
>

Is that a problem when it isn't included in any segment by
objcopy/strip?


-- 
H.J.

Re: VMA section overlap warnings for overlays

[Andreas Schwab]

"H.J. Lu" <hjl.tools@gmail.com> writes:

> Is that a problem when it isn't included in any segment by
> objcopy/strip?

See <https://bugzilla.redhat.com/show_bug.cgi?id=614382>.

Andreas.

-- 
Andreas Schwab, schwab@redhat.com
GPG Key fingerprint = D4E8 DBE3 3813 BB5D FA84  5EC7 45C6 250E 6F00 984E
"And now for something completely different."

Re: VMA section overlap warnings for overlays

[H.J. Lu]

On Thu, Jul 15, 2010 at 7:36 AM, Andreas Schwab <schwab@redhat.com> wrote:
> "H.J. Lu" <hjl.tools@gmail.com> writes:
>
>> Is that a problem when it isn't included in any segment by
>> objcopy/strip?
>
> See <https://bugzilla.redhat.com/show_bug.cgi?id=614382>.
>

Why does an empty section affect section layout?

-- 
H.J.

Re: VMA section overlap warnings for overlays

[Andreas Schwab]

"H.J. Lu" <hjl.tools@gmail.com> writes:

> Why does an empty section affect section layout?

??? That's the bug.

Andreas.

-- 
Andreas Schwab, schwab@redhat.com
GPG Key fingerprint = D4E8 DBE3 3813 BB5D FA84  5EC7 45C6 250E 6F00 984E
"And now for something completely different."

Re: VMA section overlap warnings for overlays

[Alan Modra]

On Fri, Jul 16, 2010 at 09:39:16AM +0200, Andreas Schwab wrote:
> "H.J. Lu" <hjl.tools@gmail.com> writes:
> 
> > Why does an empty section affect section layout?
> 
> ??? That's the bug.

I think we could probably remove all the sh_size != 0 tests associated
with ELF_SECTION_IN_SEGMENT in elf.c.  I'm not sure why they were
there in the first place, possibly to cover bugs in the precursors to
ELF_SECTION_IN_SEGMENT.

If we do that, then any zero size load/alloc sections will be added to
a segment map by copy_elf_program_header for objcopy and strip, just
as they are for ld.  Then the zero size sections will be handled by
assign_file_positions_for_load_sections and placed with other load
sections, rather than being shunted off to
assign_file_positions_for_non_load_sections.

-- 
Alan Modra
Australia Development Lab, IBM

Re: VMA section overlap warnings for overlays

[Andreas Schwab]

Alan Modra <amodra@gmail.com> writes:

> On Fri, Jul 16, 2010 at 09:39:16AM +0200, Andreas Schwab wrote:
>> "H.J. Lu" <hjl.tools@gmail.com> writes:
>> 
>> > Why does an empty section affect section layout?
>> 
>> ??? That's the bug.
>
> I think we could probably remove all the sh_size != 0 tests associated
> with ELF_SECTION_IN_SEGMENT in elf.c.  I'm not sure why they were
> there in the first place, possibly to cover bugs in the precursors to
> ELF_SECTION_IN_SEGMENT.

ELF_SECTION_SIZE is also weird.  It returns 0 for a .tbss section
outside a TLS segment, but that makes it impossible to distinguish it
from a genuine empty section.  There are two uses of ELF_SECTION_SIZE
together with ELF_SECTION_IN_SEGMENT, which are in readelf.c end
elf32-spu.c.  Thus readelf -l never considers an empty section be part
of any segment, but removing the ELF_SECTION_SIZE check causes the .tbss
section to be mishandled.  I think ELF_SECTION_IN_SEGMENT_1 should be
modified to handle the special case of .tbss directly.

Andreas.

-- 
Andreas Schwab, schwab@redhat.com
GPG Key fingerprint = D4E8 DBE3 3813 BB5D FA84  5EC7 45C6 250E 6F00 984E
"And now for something completely different."

Re: VMA section overlap warnings for overlays

[Alan Modra]

On Mon, Jul 19, 2010 at 02:42:58PM +0200, Andreas Schwab wrote:
> ELF_SECTION_SIZE is also weird.  It returns 0 for a .tbss section
> outside a TLS segment, but that makes it impossible to distinguish it
> from a genuine empty section.  There are two uses of ELF_SECTION_SIZE
> together with ELF_SECTION_IN_SEGMENT, which are in readelf.c end
> elf32-spu.c.

I think the one in elf32-spu.c can be removed.

>  Thus readelf -l never considers an empty section be part
> of any segment, but removing the ELF_SECTION_SIZE check causes the .tbss
> section to be mishandled.  I think ELF_SECTION_IN_SEGMENT_1 should be
> modified to handle the special case of .tbss directly.

It will also need changing to properly handle zero size sections at
the end of a segment.  Currently, zero size sections may wrongly match
at the end of a segment, or match multiple segments.  For instance, if
you just keep the special handling for .tbss in readelf but allow zero
size sections, then a number of powerpc tests fail as follows:

regexp_diff match failure
regexp "^ +03 +\.tdata \.dynamic \.got \.plt $"
line   "   03     .tdata .dynamic .branch_lt .got .plt "
regexp_diff match failure
regexp "^ +04 +\.dynamic $"
line   "   04     .dynamic .branch_lt "
FAIL: TLSTOC dynamic exec

The first difference is expected;  We now include the zero size
.branch_lt in a PT_LOAD header.  The second difference is
problematic.  .branch_lt doesn't really belong in PT_DYNAMIC.

-- 
Alan Modra
Australia Development Lab, IBM

Re: VMA section overlap warnings for overlays

[Alan Modra]

This removes all the special treatment for zero size sections when
using ELF_SECTION_IN_SEGMENT in elf.c, and makes readelf display
section to segment mapping for zero size sections.  I chose to not
include a zero size section in readelf's mapping if the section was at
the end of a segment.  eg. if the segment covers 0x1000 to 0x1fff,
then a zero sized section located at 0x2000 is not shown as part of
the segment.  That seems reasonable to me, but I'm open to
suggestions.  I'll note that omitting sections like this in elf.c, ie.
using ELF_SECTION_IN_SEGMENT_STRICT there, causes testsuite failures
on x86.  A zero length .got.plt sometimes ends up as the last section
mapped to a load segment.

include/elf/
	* internal.h (ELF_TBSS_SPECIAL): New macro, extracted from..
	(ELF_SECTION_SIZE): ..here.
	(ELF_SECTION_IN_SEGMENT_1): Add "strict" arg.
	(ELF_SECTION_IN_SEGMENT_STRICT): New macro.
bfd/
	* elf.c (assign_file_positions_for_load_sections): Check that
	zero size sections are allocated in segments too.
	(assign_file_positions_for_non_load_sections): Warn if zero
	size alloc sections are found here.
	(copy_elf_program_header): Don't drop zero size sections from
	segment map.
	(copy_private_bfd_data): Check for changes in zero size sections.
binutils/
	* readelf.c (process_program_headers): Don't ignore all zero size
	sections.
ld/testsuite/
	* ld-powerpc/tlsexe.r: Update.
	* ld-powerpc/tlsexetoc.r: Update.
	* ld-powerpc/tlsso.r: Update.
	* ld-powerpc/tlstocso.r: Update.

Index: include/elf/internal.h
===================================================================
RCS file: /cvs/src/src/include/elf/internal.h,v
retrieving revision 1.25
diff -u -p -r1.25 internal.h
--- include/elf/internal.h	24 Apr 2010 01:05:22 -0000	1.25
+++ include/elf/internal.h	20 Jul 2010 13:53:26 -0000
@@ -291,37 +291,55 @@ struct elf_segment_map
 
 /* .tbss is special.  It doesn't contribute memory space to normal
    segments and it doesn't take file space in normal segments.  */
+#define ELF_TBSS_SPECIAL(sec_hdr, segment)			\
+  (((sec_hdr)->sh_flags & SHF_TLS) != 0				\
+   && (sec_hdr)->sh_type == SHT_NOBITS				\
+   && (segment)->p_type != PT_TLS)
+
 #define ELF_SECTION_SIZE(sec_hdr, segment)			\
-  ((!(((sec_hdr)->sh_flags & SHF_TLS) != 0			\
-      && (sec_hdr)->sh_type == SHT_NOBITS)			\
-    || (segment)->p_type == PT_TLS) ? (sec_hdr)->sh_size : 0)
+  (ELF_TBSS_SPECIAL(sec_hdr, segment) ? 0 : (sec_hdr)->sh_size)
 
-/* Decide if the given sec_hdr is in the given segment.  PT_TLS segment
-   contains only SHF_TLS sections.  Only PT_LOAD, PT_GNU_RELRO and
-   and PT_TLS segments can contain SHF_TLS sections.  */
-#define ELF_SECTION_IN_SEGMENT_1(sec_hdr, segment, check_vma)		\
-  ((((((sec_hdr)->sh_flags & SHF_TLS) != 0)				\
+/* Decide if the section SEC_HDR is in SEGMENT.  If CHECK_VMA, then
+   VMAs are checked for alloc sections.  If STRICT, then a zero size
+   section won't match at the end of a segment, unless the segment
+   is also zero size.  */
+#define ELF_SECTION_IN_SEGMENT_1(sec_hdr, segment, check_vma, strict)	\
+  ((/* Only PT_LOAD, PT_GNU_RELRO and PT_TLS segments can contain	\
+       SHF_TLS sections.  */						\
+    ((((sec_hdr)->sh_flags & SHF_TLS) != 0)				\
      && ((segment)->p_type == PT_TLS					\
 	 || (segment)->p_type == PT_GNU_RELRO				\
 	 || (segment)->p_type == PT_LOAD))				\
+    /* PT_TLS segment contains only SHF_TLS sections, PT_PHDR no	\
+       sections at all.  */						\
     || (((sec_hdr)->sh_flags & SHF_TLS) == 0				\
 	&& (segment)->p_type != PT_TLS					\
 	&& (segment)->p_type != PT_PHDR))				\
-   /* Any section besides one of type SHT_NOBITS must have a file	\
-      offset within the segment.  */					\
+   /* Any section besides one of type SHT_NOBITS must have file		\
+      offsets within the segment.  */					\
    && ((sec_hdr)->sh_type == SHT_NOBITS					\
        || ((bfd_vma) (sec_hdr)->sh_offset >= (segment)->p_offset	\
-	   && ((sec_hdr)->sh_offset + ELF_SECTION_SIZE(sec_hdr, segment) \
-	       <= (segment)->p_offset + (segment)->p_filesz)))		\
-   /* SHF_ALLOC sections must have VMAs within the segment.  Be		\
-      careful about segments right at the end of memory.  */		\
+	   && (!(strict)						\
+	       || ((sec_hdr)->sh_offset - (segment)->p_offset		\
+		   <= (segment)->p_filesz - 1))				\
+	   && (((sec_hdr)->sh_offset - (segment)->p_offset		\
+		+ ELF_SECTION_SIZE(sec_hdr, segment))			\
+	       <= (segment)->p_filesz)))				\
+   /* SHF_ALLOC sections must have VMAs within the segment.  */		\
    && (!(check_vma)							\
        || ((sec_hdr)->sh_flags & SHF_ALLOC) == 0			\
        || ((sec_hdr)->sh_addr >= (segment)->p_vaddr			\
-	   && ((sec_hdr)->sh_addr - (segment)->p_vaddr			\
-	       + ELF_SECTION_SIZE(sec_hdr, segment) <= (segment)->p_memsz))))
+	   && (!(strict)						\
+	       || ((sec_hdr)->sh_addr - (segment)->p_vaddr		\
+		   <= (segment)->p_memsz - 1))				\
+	   && (((sec_hdr)->sh_addr - (segment)->p_vaddr			\
+		+ ELF_SECTION_SIZE(sec_hdr, segment))			\
+	       <= (segment)->p_memsz))))
 
 #define ELF_SECTION_IN_SEGMENT(sec_hdr, segment)			\
-  (ELF_SECTION_IN_SEGMENT_1 (sec_hdr, segment, 1))
+  (ELF_SECTION_IN_SEGMENT_1 (sec_hdr, segment, 1, 0))
+
+#define ELF_SECTION_IN_SEGMENT_STRICT(sec_hdr, segment)			\
+  (ELF_SECTION_IN_SEGMENT_1 (sec_hdr, segment, 1, 1))
 
 #endif /* _ELF_INTERNAL_H */
Index: bfd/elf.c
===================================================================
RCS file: /cvs/src/src/bfd/elf.c,v
retrieving revision 1.512
diff -u -p -r1.512 elf.c
--- bfd/elf.c	17 Jul 2010 03:10:50 -0000	1.512
+++ bfd/elf.c	20 Jul 2010 13:53:29 -0000
@@ -4589,8 +4589,7 @@ assign_file_positions_for_load_sections 
 
 	      sec = m->sections[i];
 	      this_hdr = &(elf_section_data(sec)->this_hdr);
-	      if (this_hdr->sh_size != 0
-		  && !ELF_SECTION_IN_SEGMENT_1 (this_hdr, p, check_vma))
+	      if (!ELF_SECTION_IN_SEGMENT_1 (this_hdr, p, check_vma, 0))
 		{
 		  (*_bfd_error_handler)
 		    (_("%B: section `%A' can't be allocated in segment %d"),
@@ -4640,13 +4639,12 @@ assign_file_positions_for_non_load_secti
 	BFD_ASSERT (hdr->sh_offset == hdr->bfd_section->filepos);
       else if ((hdr->sh_flags & SHF_ALLOC) != 0)
 	{
-	  if (hdr->sh_size != 0)
-	    ((*_bfd_error_handler)
-	     (_("%B: warning: allocated section `%s' not in segment"),
-	      abfd,
-	      (hdr->bfd_section == NULL
-	       ? "*unknown*"
-	       : hdr->bfd_section->name)));
+	  (*_bfd_error_handler)
+	    (_("%B: warning: allocated section `%s' not in segment"),
+	     abfd,
+	     (hdr->bfd_section == NULL
+	      ? "*unknown*"
+	      : hdr->bfd_section->name));
 	  /* We don't need to page align empty sections.  */
 	  if ((abfd->flags & D_PAGED) != 0 && hdr->sh_size != 0)
 	    off += vma_page_aligned_bias (hdr->sh_addr, off,
@@ -5868,8 +5866,7 @@ copy_elf_program_header (bfd *ibfd, bfd 
 	   section = section->next)
 	{
 	  this_hdr = &(elf_section_data(section)->this_hdr);
-	  if (this_hdr->sh_size != 0
-	      && ELF_SECTION_IN_SEGMENT (this_hdr, segment))
+	  if (ELF_SECTION_IN_SEGMENT (this_hdr, segment))
 	    {
 	      if (!first_section)
 		first_section = lowest_section = section;
@@ -5948,8 +5945,7 @@ copy_elf_program_header (bfd *ibfd, bfd 
 	       section = section->next)
 	    {
 	      this_hdr = &(elf_section_data(section)->this_hdr);
-	      if (this_hdr->sh_size != 0
-		  && ELF_SECTION_IN_SEGMENT (this_hdr, segment))
+	      if (ELF_SECTION_IN_SEGMENT (this_hdr, segment))
 		{
 		  map->sections[isec++] = section->output_section;
 		  if (isec == section_count)
@@ -6026,8 +6022,7 @@ copy_private_bfd_data (bfd *ibfd, bfd *o
 
 	      /* Check if this section is covered by the segment.  */
 	      this_hdr = &(elf_section_data(section)->this_hdr);
-	      if (this_hdr->sh_size != 0
-		  && ELF_SECTION_IN_SEGMENT (this_hdr, segment))
+	      if (ELF_SECTION_IN_SEGMENT (this_hdr, segment))
 		{
 		  /* FIXME: Check if its output section is changed or
 		     removed.  What else do we need to check?  */
Index: binutils/readelf.c
===================================================================
RCS file: /cvs/src/src/binutils/readelf.c,v
retrieving revision 1.506
diff -u -p -r1.506 readelf.c
--- binutils/readelf.c	19 Jul 2010 12:46:01 -0000	1.506
+++ binutils/readelf.c	20 Jul 2010 13:53:33 -0000
@@ -3889,8 +3889,8 @@ process_program_headers (FILE * file)
 
 	  for (j = 1; j < elf_header.e_shnum; j++, section++)
 	    {
-	      if (ELF_SECTION_SIZE (section, segment) != 0
-		  && ELF_SECTION_IN_SEGMENT (section, segment))
+	      if (!ELF_TBSS_SPECIAL (section, segment)
+		  && ELF_SECTION_IN_SEGMENT_STRICT (section, segment))
 		printf ("%s ", SECTION_NAME (section));
 	    }
 
Index: ld/testsuite/ld-powerpc/tlsexe.r
===================================================================
RCS file: /cvs/src/src/ld/testsuite/ld-powerpc/tlsexe.r,v
retrieving revision 1.25
diff -u -p -r1.25 tlsexe.r
--- ld/testsuite/ld-powerpc/tlsexe.r	2 Oct 2009 15:00:30 -0000	1.25
+++ ld/testsuite/ld-powerpc/tlsexe.r	20 Jul 2010 07:05:25 -0000
@@ -47,7 +47,7 @@ Program Headers:
  +0+ +
  +01 +\.interp 
  +02 +\.interp \.hash \.dynsym \.dynstr \.rela\.dyn \.rela\.plt \.text 
- +03 +\.tdata \.dynamic \.got \.plt 
+ +03 +\.tdata \.dynamic (\.branch_lt |)\.got \.plt 
  +04 +\.dynamic 
  +05 +\.tdata \.tbss 
 
Index: ld/testsuite/ld-powerpc/tlsexetoc.r
===================================================================
RCS file: /cvs/src/src/ld/testsuite/ld-powerpc/tlsexetoc.r,v
retrieving revision 1.26
diff -u -p -r1.26 tlsexetoc.r
--- ld/testsuite/ld-powerpc/tlsexetoc.r	2 Oct 2009 15:00:30 -0000	1.26
+++ ld/testsuite/ld-powerpc/tlsexetoc.r	20 Jul 2010 07:05:25 -0000
@@ -47,7 +47,7 @@ Program Headers:
  +0+ +
  +01 +\.interp 
  +02 +\.interp \.hash \.dynsym \.dynstr \.rela\.dyn \.rela\.plt \.text 
- +03 +\.tdata \.dynamic \.got \.plt 
+ +03 +\.tdata \.dynamic (\.branch_lt |)\.got \.plt 
  +04 +\.dynamic 
  +05 +\.tdata \.tbss 
 
Index: ld/testsuite/ld-powerpc/tlsso.r
===================================================================
RCS file: /cvs/src/src/ld/testsuite/ld-powerpc/tlsso.r,v
retrieving revision 1.26
diff -u -p -r1.26 tlsso.r
--- ld/testsuite/ld-powerpc/tlsso.r	2 Oct 2009 15:00:30 -0000	1.26
+++ ld/testsuite/ld-powerpc/tlsso.r	20 Jul 2010 07:05:25 -0000
@@ -40,7 +40,7 @@ Program Headers:
  Section to Segment mapping:
  +Segment Sections\.\.\.
  +0+ +\.hash \.dynsym \.dynstr \.rela\.dyn \.rela\.plt \.text 
- +01 +\.tdata \.dynamic \.got \.plt 
+ +01 +\.tdata \.dynamic (\.branch_lt |)\.got \.plt 
  +02 +\.dynamic 
  +03 +\.tdata \.tbss 
 
Index: ld/testsuite/ld-powerpc/tlstocso.r
===================================================================
RCS file: /cvs/src/src/ld/testsuite/ld-powerpc/tlstocso.r,v
retrieving revision 1.24
diff -u -p -r1.24 tlstocso.r
--- ld/testsuite/ld-powerpc/tlstocso.r	2 Oct 2009 15:00:30 -0000	1.24
+++ ld/testsuite/ld-powerpc/tlstocso.r	20 Jul 2010 07:05:25 -0000
@@ -40,7 +40,7 @@ Program Headers:
  Section to Segment mapping:
  +Segment Sections\.\.\.
  +0+ +\.hash \.dynsym \.dynstr \.rela\.dyn \.rela\.plt \.text 
- +01 +\.tdata \.dynamic \.got \.plt 
+ +01 +\.tdata \.dynamic (\.branch_lt |)\.got \.plt 
  +02 +\.dynamic 
  +03 +\.tdata \.tbss 
 

-- 
Alan Modra
Australia Development Lab, IBM

Re: VMA section overlap warnings for overlays

[H.J. Lu]

On Wed, Apr 21, 2010 at 6:53 PM, Alan Modra <amodra@gmail.com> wrote:
> On Thu, Apr 22, 2010 at 10:41:06AM +0930, Alan Modra wrote:
>> Now we have a problem.  If we put .overlay1 in imem at p_offset+1 then
>> the execution model lma (calculated from p_paddr+p_offset+1) for
>> .overlay1 will be correct, but the vma (from p_vaddr+p_offset+1
>> ie. 0x2001) will be wrong.  Of course, with multiple overlays packed
>> into one header you can't possibly get the execution model vma correct
>> for all the overlays, so you probably don't care.  However, the
>> ELF_IS_SECTION_IN_SEGMENT_FILE test in
>> elf.c:assign_file_positions_for_load_sections fails, which is why you
>> get a linker error.
>>
>> Conversely, putting .overlay1 at p_offset+0x2000 will give the correct
>> vma but the wrong lma, and of course insert a whole lot of padding.
>> This is what Jan's patch did, and is quite wrong for overlays..
>
> Committed.  I didn't see any occurrence of the problem Jan originally
> reported in http://sourceware.org/ml/binutils/2008-05/msg00235.html on
> i386 or powerpc using recent compilers so perhaps some other change
> cured it.
>
> bfd/
>        * elf.c (assign_file_positions_for_load_sections): Revert 2008-05-29
>        change.  Tidy.  Don't error on sections not allocated in segment.
> ld/testsuite/
>        * ld-elf/extract-symbol-1sec.d: Update lma.
>        * ld-i386/alloc.d: Expect a warning, not an error.
>

This patch caused:

http://www.sourceware.org/bugzilla/show_bug.cgi?id=11953


-- 
H.J.

Re: VMA section overlap warnings for overlays

[Alan Modra]

On Fri, Aug 27, 2010 at 06:01:25PM -0700, H.J. Lu wrote:
> On Wed, Apr 21, 2010 at 6:53 PM, Alan Modra <amodra@gmail.com> wrote:
> >        * elf.c (assign_file_positions_for_load_sections): Revert 2008-05-29
> >        change.  Tidy.  Don't error on sections not allocated in segment.
> > ld/testsuite/
> >        * ld-elf/extract-symbol-1sec.d: Update lma.
> >        * ld-i386/alloc.d: Expect a warning, not an error.
> >
> 
> This patch caused:
> 
> http://www.sourceware.org/bugzilla/show_bug.cgi?id=11953

I don't think it did.  I'm looking into it now, and will put any
futher comments into bugzilla.

-- 
Alan Modra
Australia Development Lab, IBM

Re: VMA section overlap warnings for overlays

[Alan Modra]

The real bug was that copy_elf_program_header calculated header_size
from the first section found to be in the segment rather than the
section with the lowest lma.  So a one line fix.  The rest of this
patch is to cope with (and fix) invalid program header p_paddr values.
I won't commit this immediately as I'd like to run some more tests.

	PR binutils/11953
	* elf.c (copy_elf_program_header): Calculate map->header_size
	from lowest_section, not first_section.  Validate program
	header p_paddr against section lma.  Find lowest_section in
	second loop over headers.

Index: bfd/elf.c
===================================================================
RCS file: /cvs/src/src/bfd/elf.c,v
retrieving revision 1.517
diff -u -p -r1.517 elf.c
--- bfd/elf.c	25 Aug 2010 14:53:40 -0000	1.517
+++ bfd/elf.c	28 Aug 2010 12:57:00 -0000
@@ -5870,7 +5870,7 @@ copy_elf_program_header (bfd *ibfd, bfd 
       bfd_size_type amt;
       Elf_Internal_Shdr *this_hdr;
       asection *first_section = NULL;
-      asection *lowest_section = NULL;
+      asection *lowest_section;
 
       /* Compute how many sections are in this segment.  */
       for (section = ibfd->sections, section_count = 0;
@@ -5880,10 +5880,8 @@ copy_elf_program_header (bfd *ibfd, bfd 
 	  this_hdr = &(elf_section_data(section)->this_hdr);
 	  if (ELF_SECTION_IN_SEGMENT (this_hdr, segment))
 	    {
-	      if (!first_section)
-		first_section = lowest_section = section;
-	      if (section->lma < lowest_section->lma)
-		lowest_section = section;
+	      if (first_section == NULL)
+		first_section = section;
 	      section_count++;
 	    }
 	}
@@ -5937,17 +5935,7 @@ copy_elf_program_header (bfd *ibfd, bfd 
 	    phdr_included = TRUE;
 	}
 
-      if (map->includes_filehdr && first_section)
-	/* We need to keep the space used by the headers fixed.  */
-	map->header_size = first_section->vma - segment->p_vaddr;
-      
-      if (!map->includes_phdrs
-	  && !map->includes_filehdr
-	  && map->p_paddr_valid)
-	/* There is some other padding before the first section.  */
-	map->p_vaddr_offset = ((lowest_section ? lowest_section->lma : 0)
-			       - segment->p_paddr);
-
+      lowest_section = first_section;
       if (section_count != 0)
 	{
 	  unsigned int isec = 0;
@@ -5960,12 +5948,41 @@ copy_elf_program_header (bfd *ibfd, bfd 
 	      if (ELF_SECTION_IN_SEGMENT (this_hdr, segment))
 		{
 		  map->sections[isec++] = section->output_section;
+		  if (section->lma < lowest_section->lma)
+		    lowest_section = section;
+		  if ((section->flags & SEC_ALLOC) != 0)
+		    {
+		      bfd_vma seg_off;
+
+		      /* Section lmas are set up from PT_LOAD header
+			 p_paddr in _bfd_elf_make_section_from_shdr.
+			 If this header has a p_paddr that disagrees
+			 with the section lma, flag the p_paddr as
+			 invalid.  */
+		      if ((section->flags & SEC_LOAD) != 0)
+			seg_off = this_hdr->sh_offset - segment->p_offset;
+		      else
+			seg_off = this_hdr->sh_addr - segment->p_vaddr;
+		      if (section->lma - segment->p_paddr != seg_off)
+			map->p_paddr_valid = FALSE;
+		    }
 		  if (isec == section_count)
 		    break;
 		}
 	    }
 	}
 
+      if (map->includes_filehdr && lowest_section != NULL)
+	/* We need to keep the space used by the headers fixed.  */
+	map->header_size = lowest_section->vma - segment->p_vaddr;
+      
+      if (!map->includes_phdrs
+	  && !map->includes_filehdr
+	  && map->p_paddr_valid)
+	/* There is some other padding before the first section.  */
+	map->p_vaddr_offset = ((lowest_section ? lowest_section->lma : 0)
+			       - segment->p_paddr);
+
       map->count = section_count;
       *pointer_to_map = map;
       pointer_to_map = &map->next;

-- 
Alan Modra
Australia Development Lab, IBM

Re: VMA section overlap warnings for overlays

[H.J. Lu]

On Sat, Aug 28, 2010 at 6:28 AM, Alan Modra <amodra@gmail.com> wrote:
> The real bug was that copy_elf_program_header calculated header_size
> from the first section found to be in the segment rather than the
> section with the lowest lma.  So a one line fix.  The rest of this
> patch is to cope with (and fix) invalid program header p_paddr values.

There is no such a thing as "invalid program header p_paddr values"
since ELF spec says it has unspecified contents. You can't depend
on contents in p_addr in objcopy.

> I won't commit this immediately as I'd like to run some more tests.
>
>        PR binutils/11953
>        * elf.c (copy_elf_program_header): Calculate map->header_size
>        from lowest_section, not first_section.  Validate program
>        header p_paddr against section lma.  Find lowest_section in
>        second loop over headers.
>

Can we not look at p_addr here?

-- 
H.J.

Re: VMA section overlap warnings for overlays

[Alan Modra]

On Sat, Aug 28, 2010 at 06:39:36AM -0700, H.J. Lu wrote:
> On Sat, Aug 28, 2010 at 6:28 AM, Alan Modra <amodra@gmail.com> wrote:
> > The real bug was that copy_elf_program_header calculated header_size
> > from the first section found to be in the segment rather than the
> > section with the lowest lma.  So a one line fix.  The rest of this
> > patch is to cope with (and fix) invalid program header p_paddr values.
> 
> There is no such a thing as "invalid program header p_paddr values"
> since ELF spec says it has unspecified contents.

Yes, that's technically true.  My language wasn't precise.  I should
have said inconsistent p_paddr values.

> You can't depend on contents in p_addr in objcopy.

Are you saying that objcopy should throw away all p_paddr values?
That doesn't make sense at all.  Just because the ELF spec doesn't
specify p_paddr, doesn't mean that GNU binutils cannot use p_paddr.
We use p_paddr to record section LMAs, since ELF section headers only
have a field for VMAs.

> > I won't commit this immediately as I'd like to run some more tests.
> >
> >        PR binutils/11953
> >        * elf.c (copy_elf_program_header): Calculate map->header_size
> >        from lowest_section, not first_section.  Validate program
> >        header p_paddr against section lma.  Find lowest_section in
> >        second loop over headers.
> >
> 
> Can we not look at p_addr here?

I don't know what you mean by that question.

-- 
Alan Modra
Australia Development Lab, IBM

Re: VMA section overlap warnings for overlays

[H.J. Lu]

On Sun, Aug 29, 2010 at 8:20 PM, Alan Modra <amodra@gmail.com> wrote:
> On Sat, Aug 28, 2010 at 06:39:36AM -0700, H.J. Lu wrote:
>> On Sat, Aug 28, 2010 at 6:28 AM, Alan Modra <amodra@gmail.com> wrote:
>> > The real bug was that copy_elf_program_header calculated header_size
>> > from the first section found to be in the segment rather than the
>> > section with the lowest lma.  So a one line fix.  The rest of this
>> > patch is to cope with (and fix) invalid program header p_paddr values.
>>
>> There is no such a thing as "invalid program header p_paddr values"
>> since ELF spec says it has unspecified contents.
>
> Yes, that's technically true.  My language wasn't precise.  I should
> have said inconsistent p_paddr values.
>
>> You can't depend on contents in p_addr in objcopy.
>
> Are you saying that objcopy should throw away all p_paddr values?
> That doesn't make sense at all.  Just because the ELF spec doesn't
> specify p_paddr, doesn't mean that GNU binutils cannot use p_paddr.
> We use p_paddr to record section LMAs, since ELF section headers only
> have a field for VMAs.
>
>> > I won't commit this immediately as I'd like to run some more tests.
>> >
>> >        PR binutils/11953
>> >        * elf.c (copy_elf_program_header): Calculate map->header_size
>> >        from lowest_section, not first_section.  Validate program
>> >        header p_paddr against section lma.  Find lowest_section in
>> >        second loop over headers.
>> >
>>
>> Can we not look at p_addr here?
>
> I don't know what you mean by that question.
>

Can we compute p_addr from other fields and don't check if
it matches the original value?


-- 
H.J.

Re: VMA section overlap warnings for overlays

[Alan Modra]

On Sun, Aug 29, 2010 at 09:46:13PM -0700, H.J. Lu wrote:
> Can we compute p_addr from other fields

No.  An ELF executable may have no section headers.

-- 
Alan Modra
Australia Development Lab, IBM

Re: VMA section overlap warnings for overlays

[H.J. Lu]

On Tue, Jul 20, 2010 at 7:10 AM, Alan Modra <amodra@gmail.com> wrote:
> This removes all the special treatment for zero size sections when
> using ELF_SECTION_IN_SEGMENT in elf.c, and makes readelf display
> section to segment mapping for zero size sections.  I chose to not
> include a zero size section in readelf's mapping if the section was at
> the end of a segment.  eg. if the segment covers 0x1000 to 0x1fff,
> then a zero sized section located at 0x2000 is not shown as part of
> the segment.  That seems reasonable to me, but I'm open to
> suggestions.  I'll note that omitting sections like this in elf.c, ie.
> using ELF_SECTION_IN_SEGMENT_STRICT there, causes testsuite failures
> on x86.  A zero length .got.plt sometimes ends up as the last section
> mapped to a load segment.
>
> include/elf/
>        * internal.h (ELF_TBSS_SPECIAL): New macro, extracted from..
>        (ELF_SECTION_SIZE): ..here.
>        (ELF_SECTION_IN_SEGMENT_1): Add "strict" arg.
>        (ELF_SECTION_IN_SEGMENT_STRICT): New macro.
> bfd/
>        * elf.c (assign_file_positions_for_load_sections): Check that
>        zero size sections are allocated in segments too.
>        (assign_file_positions_for_non_load_sections): Warn if zero
>        size alloc sections are found here.
>        (copy_elf_program_header): Don't drop zero size sections from
>        segment map.
>        (copy_private_bfd_data): Check for changes in zero size sections.
> binutils/
>        * readelf.c (process_program_headers): Don't ignore all zero size
>        sections.
> ld/testsuite/
>        * ld-powerpc/tlsexe.r: Update.
>        * ld-powerpc/tlsexetoc.r: Update.
>        * ld-powerpc/tlsso.r: Update.
>        * ld-powerpc/tlstocso.r: Update.
>

This may have caused:

http://sourceware.org/bugzilla/show_bug.cgi?id=12516

-- 
H.J.

Re: VMA section overlap warnings for overlays

[Alan Modra]

On Thu, Feb 24, 2011 at 03:49:25PM -0800, H.J. Lu wrote:
> On Tue, Jul 20, 2010 at 7:10 AM, Alan Modra <amodra@gmail.com> wrote:
> > This removes all the special treatment for zero size sections when
> > using ELF_SECTION_IN_SEGMENT in elf.c, and makes readelf display
[snip]
> This may have caused:
> 
> http://sourceware.org/bugzilla/show_bug.cgi?id=12516

Yes, looks like it.

	PR 12519
	* internal.h (ELF_SECTION_IN_SEGMENT_1): Don't match zero size
	sections at start or end of PT_DYNAMIC.

Index: include/elf/internal.h
===================================================================
RCS file: /cvs/src/src/include/elf/internal.h,v
retrieving revision 1.26
diff -u -p -r1.26 internal.h
--- include/elf/internal.h	20 Jul 2010 14:08:30 -0000	1.26
+++ include/elf/internal.h	25 Feb 2011 04:28:30 -0000
@@ -302,7 +302,9 @@ struct elf_segment_map
 /* Decide if the section SEC_HDR is in SEGMENT.  If CHECK_VMA, then
    VMAs are checked for alloc sections.  If STRICT, then a zero size
    section won't match at the end of a segment, unless the segment
-   is also zero size.  */
+   is also zero size.  Regardless of STRICT and CHECK_VMA, zero size
+   sections won't match at the start or end of PT_DYNAMIC, unless
+   PT_DYNAMIC is itself zero sized.  */
 #define ELF_SECTION_IN_SEGMENT_1(sec_hdr, segment, check_vma, strict)	\
   ((/* Only PT_LOAD, PT_GNU_RELRO and PT_TLS segments can contain	\
        SHF_TLS sections.  */						\
@@ -334,7 +336,19 @@ struct elf_segment_map
 		   <= (segment)->p_memsz - 1))				\
 	   && (((sec_hdr)->sh_addr - (segment)->p_vaddr			\
 		+ ELF_SECTION_SIZE(sec_hdr, segment))			\
-	       <= (segment)->p_memsz))))
+	       <= (segment)->p_memsz)))					\
+   /* No zero size sections at start or end of PT_DYNAMIC.  */		\
+   && ((segment)->p_type != PT_DYNAMIC					\
+       || (sec_hdr)->sh_size != 0					\
+       || (segment)->p_memsz == 0					\
+       || (((sec_hdr)->sh_type == SHT_NOBITS				\
+	    || ((bfd_vma) (sec_hdr)->sh_offset > (segment)->p_offset	\
+	        && ((sec_hdr)->sh_offset - (segment)->p_offset		\
+		    < (segment)->p_filesz)))				\
+	   && (((sec_hdr)->sh_flags & SHF_ALLOC) == 0			\
+	       || ((sec_hdr)->sh_addr > (segment)->p_vaddr		\
+		   && ((sec_hdr)->sh_addr - (segment)->p_vaddr		\
+		       < (segment)->p_memsz))))))
 
 #define ELF_SECTION_IN_SEGMENT(sec_hdr, segment)			\
   (ELF_SECTION_IN_SEGMENT_1 (sec_hdr, segment, 1, 0))

-- 
Alan Modra
Australia Development Lab, IBM

Re: VMA section overlap warnings for overlays

[Andreas Schwab]

Alan Modra <amodra@gmail.com> writes:

> 	* internal.h (ELF_SECTION_IN_SEGMENT_1): Don't match zero size
> 	sections at start or end of PT_DYNAMIC.

That fixes the original test case, too.

Thanks, Andreas.

-- 
Andreas Schwab, schwab@redhat.com
GPG Key fingerprint = D4E8 DBE3 3813 BB5D FA84  5EC7 45C6 250E 6F00 984E
"And now for something completely different."

Re: VMA section overlap warnings for overlays

[H.J. Lu]

On Thu, Feb 24, 2011 at 11:49 PM, Alan Modra <amodra@gmail.com> wrote:
> On Thu, Feb 24, 2011 at 03:49:25PM -0800, H.J. Lu wrote:
>> On Tue, Jul 20, 2010 at 7:10 AM, Alan Modra <amodra@gmail.com> wrote:
>> > This removes all the special treatment for zero size sections when
>> > using ELF_SECTION_IN_SEGMENT in elf.c, and makes readelf display
> [snip]
>> This may have caused:
>>
>> http://sourceware.org/bugzilla/show_bug.cgi?id=12516
>
> Yes, looks like it.
>
>        PR 12519
>        * internal.h (ELF_SECTION_IN_SEGMENT_1): Don't match zero size
>        sections at start or end of PT_DYNAMIC.
>

Please add a testcase.

Thanks.

-- 
H.J.

Re: VMA section overlap warnings for overlays

[H.J. Lu]

On Thu, Feb 24, 2011 at 11:49 PM, Alan Modra <amodra@gmail.com> wrote:
> On Thu, Feb 24, 2011 at 03:49:25PM -0800, H.J. Lu wrote:
>> On Tue, Jul 20, 2010 at 7:10 AM, Alan Modra <amodra@gmail.com> wrote:
>> > This removes all the special treatment for zero size sections when
>> > using ELF_SECTION_IN_SEGMENT in elf.c, and makes readelf display
> [snip]
>> This may have caused:
>>
>> http://sourceware.org/bugzilla/show_bug.cgi?id=12516
>
> Yes, looks like it.
>
>        PR 12519
>        * internal.h (ELF_SECTION_IN_SEGMENT_1): Don't match zero size
>        sections at start or end of PT_DYNAMIC.

I checked in this.


H.J.
---
diff --git a/ld/testsuite/ChangeLog b/ld/testsuite/ChangeLog
index b7de27d..4ff663f 100644
--- a/ld/testsuite/ChangeLog
+++ b/ld/testsuite/ChangeLog
@@ -1,3 +1,10 @@
+2011-02-25  H.J. Lu  <hongjiu.lu@intel.com>
+
+	PR ld/12516
+	* ld-elf/dynamic1.d: New.
+	* ld-elf/dynamic1.ld: Likewise.
+	* ld-elf/dynamic1.s: Likewise.
+
 2011-02-25  Alan Modra  <amodra@gmail.com>

 	* ld-mn10300/i135409-3.s: Correct .size label reference.
diff --git a/ld/testsuite/ld-elf/dynamic1.d b/ld/testsuite/ld-elf/dynamic1.d
new file mode 100644
index 0000000..6a8ba55
--- /dev/null
+++ b/ld/testsuite/ld-elf/dynamic1.d
@@ -0,0 +1,10 @@
+#ld: -shared -T dynamic1.ld
+#readelf: -l --wide
+#target: *-*-linux* *-*-gnu*
+
+#...
+ Section to Segment mapping:
+  Segment Sections...
+#...
+   0[1-9]     .dynamic[ 	]*
+#pass
diff --git a/ld/testsuite/ld-elf/dynamic1.ld b/ld/testsuite/ld-elf/dynamic1.ld
new file mode 100644
index 0000000..d110bf7
--- /dev/null
+++ b/ld/testsuite/ld-elf/dynamic1.ld
@@ -0,0 +1,8 @@
+SECTIONS
+{
+  . = SIZEOF_HEADERS;
+  .text : { *(.text) }
+  .data : { *(.data) }
+  .data1 : { KEEP (*(.data1)) }
+  .dynamic : { *(.dynamic) }
+}
diff --git a/ld/testsuite/ld-elf/dynamic1.s b/ld/testsuite/ld-elf/dynamic1.s
new file mode 100644
index 0000000..f1caf0e
--- /dev/null
+++ b/ld/testsuite/ld-elf/dynamic1.s
@@ -0,0 +1,2 @@
+.section .data1,"aw"
+.balign 16

Re: VMA section overlap warnings for overlays

[Andreas Schwab]

"H.J. Lu" <hjl.tools@gmail.com> writes:

> diff --git a/ld/testsuite/ld-elf/dynamic1.d b/ld/testsuite/ld-elf/dynamic1.d
> new file mode 100644
> index 0000000..6a8ba55
> --- /dev/null
> +++ b/ld/testsuite/ld-elf/dynamic1.d
> @@ -0,0 +1,10 @@
> +#ld: -shared -T dynamic1.ld
> +#readelf: -l --wide
> +#target: *-*-linux* *-*-gnu*
> +
> +#...
> + Section to Segment mapping:
> +  Segment Sections...
> +#...
> +   0[1-9]     .dynamic[ 	]*
> +#pass

I don't see how that tests anything, this bug is about objcopy.

Andreas.

-- 
Andreas Schwab, schwab@redhat.com
GPG Key fingerprint = D4E8 DBE3 3813 BB5D FA84  5EC7 45C6 250E 6F00 984E
"And now for something completely different."

Re: VMA section overlap warnings for overlays

[H.J. Lu]

On Fri, Feb 25, 2011 at 8:22 AM, Andreas Schwab <schwab@redhat.com> wrote:
> "H.J. Lu" <hjl.tools@gmail.com> writes:
>
>> diff --git a/ld/testsuite/ld-elf/dynamic1.d b/ld/testsuite/ld-elf/dynamic1.d
>> new file mode 100644
>> index 0000000..6a8ba55
>> --- /dev/null
>> +++ b/ld/testsuite/ld-elf/dynamic1.d
>> @@ -0,0 +1,10 @@
>> +#ld: -shared -T dynamic1.ld
>> +#readelf: -l --wide
>> +#target: *-*-linux* *-*-gnu*
>> +
>> +#...
>> + Section to Segment mapping:
>> +  Segment Sections...
>> +#...
>> +   0[1-9]     .dynamic[      ]*
>> +#pass
>
> I don't see how that tests anything, this bug is about objcopy.
>

Both readelf and objcpy use ELF_SECTION_IN_SEGMENT_1.
Before the fix:


readelf -l x.so

Elf file type is DYN (Shared object file)
Entry point 0xb0
There are 2 program headers, starting at offset 64

Program Headers:
  Type           Offset             VirtAddr           PhysAddr
                 FileSiz            MemSiz              Flags  Align
  LOAD           0x0000000000000000 0x0000000000000000 0x0000000000000000
                 0x00000000000001b0 0x00000000000001b0  RW     200000
  DYNAMIC        0x0000000000000100 0x0000000000000100 0x0000000000000100
                 0x00000000000000b0 0x00000000000000b0  RW     8

 Section to Segment mapping:
  Segment Sections...
   00     .dynsym .dynstr .hash .data1 .dynamic
   01     .data1 .dynamic
            ^^^^^^^

That is equivalent to "The first section in the PT_DYNAMIC segment is
not the .dynamic section".



-- 
H.J.