Bad behavior with explicit phdrs and loadable orphans

Bad behavior with explicit phdrs and loadable orphans

[David Heine]

When explicit PHDRs and memory regions are used with orphan input 
sections, the default placement of these sections can generate either 
very large files or a SEEK overflow.

Here is a small example of a seek overflow for an x86 target.  I would 
send an example of the large file generation, but it doesn't seem very 
neighborly.

 > cat small-x86.s
         .section .foo, "ax"
         # nothing here.  Builds a allocatable section of 0 size
         .align 4
         .section .text
fn:
         pushl   %ebp
         movl    %esp, %ebp
         movl    $0, %eax
         popl    %ebp
         ret

 > cat link-x86.t
MEMORY
{
   normal_seg : ORIGIN = 0xd0000000, LENGTH = 0x10000
}

PHDRS
{
   normal_phdr PT_LOAD;
}

SECTIONS
{

   .text :
   {
     *(.text)
   } >normal_seg :normal_phdr

   .data :
   {
     *(.data)
   } >normal_seg :normal_phdr

   .bss :
   {
     *(.bss)
   } >normal_seg :normal_phdr

}

 > gcc -c small-x86.s
 > ld -Tlink-x86.t -o small-x86.exe small-x86.o
ld: warning: no memory region specified for section `.foo'
ld: final link failed: File truncated

This actual rather cryptic error is caused by a call to fseek with a 
negative argument.  The default placement of the ".foo" section is in 
the "*default*" memory at address 0, the PHDR where it is placed has 
memory that starts at 0xd0000000.  The system fails to build an output 
file large enough to map addresses from 0x0 to 0xd0000000.

If the start address of the "normal_seg" is set to 0x40000000, no 
overflow will occur, but the linker will build a VERY large file (and 
crash some machines with less robust filesystems).

This behavior seems a little extreme for misnaming a loadable section, 
but I'm not sure what a good solution would be.  The ld orphan placement 
algorithm seems to be looking for a good place to put the section in the 
emulation statements.  Perhaps it just needs to look for a good memory 
region while placing an orphan as well.  Alternatively, when explicit 
PHDRS are used, the linker could do a better job of placing the default 
memory section so that it was reasonably contiguous with an existing 
PHDR.   Or perhaps orphan loadable sections should generate an error 
when using explicit PHDRs before the attempt is made to build a 
multi-gigabyte file.

Any thoughts on a solution to this problem?

-David

Re: Bad behavior with explicit phdrs and loadable orphans

[Nick Clifton]

Hi David,

> When explicit PHDRs and memory regions are used with orphan input
> sections, the default placement of these sections can generate either
> very large files or a SEEK overflow.

Note - your problem could also be overcome by fixing the linker script
to provide a better definition for the memory regions.  For example in
your script if you change:

> MEMORY
> {
>    normal_seg : ORIGIN = 0xd0000000, LENGTH = 0x10000
> }

To:

  MEMORY
  {
     normal_seg (wax) : ORIGIN = 0xd0000000, LENGTH = 0x10000
  }

The link will work.  But anyway, this is only a workaround, not a full
solution.
 
An alternative is the patch below will also work.  It "fixes"
lang_memory_default() so that if a memory region does not have any
flags set, then it will accept any input section.  This seems to be a
better interpretation of how a memory region without attributes is
supposed to behave.


> This behavior seems a little extreme for misnaming a loadable
> section, but I'm not sure what a good solution would be.  The ld
> orphan placement algorithm seems to be looking for a good place to
> put the section in the emulation statements.  Perhaps it just needs
> to look for a good memory region while placing an orphan as well.
> Alternatively, when explicit PHDRS are used, the linker could do a
> better job of placing the default memory section so that it was
> reasonably contiguous with an existing PHDR.   Or perhaps orphan
> loadable sections should generate an error when using explicit PHDRs
> before the attempt is made to build a multi-gigabyte file.

I think that the linker ought to try to be a little bit better about
choosing memory regions for orphan sections.  Ideally it ought to
attempt to discover if the section it has chosen to inherit the orphan
has a memory region associated with it, and if so, use that region.
(Assuming that there is room of course).

Let me know what you think

Cheers
        Nick

2003-04-24  Nick Clifton  <nickc@redhat.com>

	* ldlang.c (lang_memory_default): If a region has not
	attributes do not check for matching section flags.
        * ld.texinfo: Document that a region without attributes will
	accept any input section.

Index: ld/ldlang.c
===================================================================
RCS file: /cvs/src/src/ld/ldlang.c,v
retrieving revision 1.111
diff -c -3 -p -w -r1.111 ldlang.c
*** ld/ldlang.c	31 Mar 2003 18:12:52 -0000	1.111
--- ld/ldlang.c	24 Apr 2003 14:08:39 -0000
*************** lang_memory_default (section)
*** 736,746 ****
         p != (lang_memory_region_type *) NULL;
         p = p->next)
      {
!       if ((p->flags & sec_flags) != 0
! 	  && (p->not_flags & sec_flags) == 0)
! 	{
  	  return p;
- 	}
      }
    return lang_memory_region_lookup ("*default*");
  }
--- 736,746 ----
         p != (lang_memory_region_type *) NULL;
         p = p->next)
      {
!       /* Only check for a positive match if the region has attributes set.  */
!       if (p->flags && ((p->flags & sec_flags) == 0))
! 	continue;
!       if ((p->not_flags & sec_flags) == 0)
  	return p;
      }
    return lang_memory_region_lookup ("*default*");
  }

Index: ld/ld.texinfo
===================================================================
RCS file: /cvs/src/src/ld/ld.texinfo,v
retrieving revision 1.93
diff -c -3 -p -w -r1.93 ld.texinfo
*** ld/ld.texinfo	15 Apr 2003 08:51:54 -0000	1.93
--- ld/ld.texinfo	24 Apr 2003 14:08:56 -0000
*************** If a unmapped section matches any of the
*** 3563,3569 ****
  @samp{!}, it will be placed in the memory region.  The @samp{!}
  attribute reverses this test, so that an unmapped section will be placed
  in the memory region only if it does not match any of the listed
! attributes.
  
  @kindex ORIGIN =
  @kindex o =
--- 3563,3570 ----
  @samp{!}, it will be placed in the memory region.  The @samp{!}
  attribute reverses this test, so that an unmapped section will be placed
  in the memory region only if it does not match any of the listed
! attributes.  If not attributes are specified then the region will
! accept any input section.
  
  @kindex ORIGIN =
  @kindex o =

Re: Bad behavior with explicit phdrs and loadable orphans

[David Heine]

Nick Clifton wrote:
> Hi David,
> 
> 
>>When explicit PHDRs and memory regions are used with orphan input
>>sections, the default placement of these sections can generate either
>>very large files or a SEEK overflow.
> 
> 
> Note - your problem could also be overcome by fixing the linker script
> to provide a better definition for the memory regions.  For example in
> your script if you change:
> 
> 
>>MEMORY
>>{
>>   normal_seg : ORIGIN = 0xd0000000, LENGTH = 0x10000
>>}
> 
> 
> To:
> 
>   MEMORY
>   {
>      normal_seg (wax) : ORIGIN = 0xd0000000, LENGTH = 0x10000
>   }
> 
> The link will work.  But anyway, this is only a workaround, not a full
> solution.
>  
> An alternative is the patch below will also work.  It "fixes"
> lang_memory_default() so that if a memory region does not have any
> flags set, then it will accept any input section.  This seems to be a
> better interpretation of how a memory region without attributes is
> supposed to behave.
> 
> 
> 
>>This behavior seems a little extreme for misnaming a loadable
>>section, but I'm not sure what a good solution would be.  The ld
>>orphan placement algorithm seems to be looking for a good place to
>>put the section in the emulation statements.  Perhaps it just needs
>>to look for a good memory region while placing an orphan as well.
>>Alternatively, when explicit PHDRS are used, the linker could do a
>>better job of placing the default memory section so that it was
>>reasonably contiguous with an existing PHDR.   Or perhaps orphan
>>loadable sections should generate an error when using explicit PHDRs
>>before the attempt is made to build a multi-gigabyte file.
> 
> 
> I think that the linker ought to try to be a little bit better about
> choosing memory regions for orphan sections.  Ideally it ought to
> attempt to discover if the section it has chosen to inherit the orphan
> has a memory region associated with it, and if so, use that region.
> (Assuming that there is room of course).
> 
> Let me know what you think
> 
> Cheers
>         Nick
> 

Nick,

Thanks for taking a look.  My initial thoughts on this were to find a 
better memory region to put the orphan section like you've implemented. 
However, the problem that caused the initial bad behavior was that an 
assembly file had mis-spelled a section name.  While the proposed patch 
keeps the system from generating a large file or causing the SEEK 
overflow in this case, the problem would recur with any orphan that can 
not be placed by this default placement algorithm for some linker script.
   Because the patch manages to place the orphan section, it also 
removes the warning that indicated the actual problem:

	warning: no memory region specified for section

For our target when using explicit memory regions, it is usually an 
error to have orphan sections that are loadable and allocatable for 
non-relocatable links.   I'm not sure if this is generally true for 
other targets, but the existence of the original warning indicates that 
it might be.

   It might be better to keep the old placement algorithm and convert 
the warning: "no memory region specified for section" into a failure 
that will avoid the massive file generation.

If you think the new placement algorithm makes more sense, it would 
probably be less intrusive to other targets if it were modified to first 
look for regions that match one of the positive attributes, then look 
for ones with no attributes so that the behavior only changes for 
sections that were previously unplaced.  Also, adding an interface in 
ldlang.c that determines whether explicit memory regions are used (like 
the test guarding the "no memory region specified for section" warning) 
would allow target-specific code to generate a warning about orphan 
loadable,allocatable sections when using explicit memory regions.

-David

Re: Bad behavior with explicit phdrs and loadable orphans

[Nick Clifton]

Hi David,

> Thanks for taking a look.  My initial thoughts on this were to find a
> better memory region to put the orphan section like you've
> implemented. However, the problem that caused the initial bad behavior
> was that an assembly file had mis-spelled a section name.  While the
> proposed patch keeps the system from generating a large file or
> causing the SEEK overflow in this case, the problem would recur with
> any orphan that can not be placed by this default placement algorithm
> for some linker script.
>    Because the patch manages to place the orphan section, it also
> removes the warning that indicated the actual problem:
> 
> 	warning: no memory region specified for section
> 
> For our target when using explicit memory regions, it is usually an
> error to have orphan sections that are loadable and allocatable for
> non-relocatable links.   I'm not sure if this is generally true for
> other targets, but the existence of the original warning indicates
> that it might be.

Yes, I think that in general a loadable/allocatable section is intended
to be used and hence ought to be explicitly handled by the linker
script.

>    It might be better to keep the old placement algorithm and convert
> the warning: "no memory region specified for section" into a failure
> that will avoid the massive file generation.

Well I have not applied my suggested patch, since I was waiting for
feedback from you, so there is no problem with keeping the old
algorithm.

> If you think the new placement algorithm makes more sense, it would
> probably be less intrusive to other targets if it were modified to
> first look for regions that match one of the positive attributes, then
> look for ones with no attributes so that the behavior only changes for
> sections that were previously unplaced.

Basically I am in favour of the "if it ain't broke, don't fix it"
approach.  So if we can leave the current placement algorithm in
place, and just update the warning to an error, then I am in favour of
doing that.  Please could you try out this alternative patch and let
me know what you think.

Cheers
        Nick

2003-05-02  Nick Clifton  <nickc@redhat.com>

	* ldlang.c (lang_size_sections_1): Default to generating a
	fatal error message if a loadable section is not allocated
	to a memory region when regions are defined.

Index: ld/ldlang.c
===================================================================
RCS file: /cvs/src/src/ld/ldlang.c,v
retrieving revision 1.112
diff -c -3 -p -w -r1.112 ldlang.c
*** ld/ldlang.c	29 Apr 2003 01:53:46 -0000	1.112
--- ld/ldlang.c	2 May 2003 09:22:46 -0000
*************** lang_size_sections_1 (s, output_section_
*** 3050,3056 ****
  
  		    /* If a loadable section is using the default memory
  		       region, and some non default memory regions were
! 		       defined, issue a warning.  */
  		    if ((bfd_get_section_flags (output_bfd, os->bfd_section)
  			 & (SEC_ALLOC | SEC_LOAD)) != 0
  			&& (bfd_get_section_flags (output_bfd, os->bfd_section)
--- 3050,3056 ----
  
  		    /* If a loadable section is using the default memory
  		       region, and some non default memory regions were
! 		       defined, issue an error message.  */
  		    if ((bfd_get_section_flags (output_bfd, os->bfd_section)
  			 & (SEC_ALLOC | SEC_LOAD)) != 0
  			&& (bfd_get_section_flags (output_bfd, os->bfd_section)
*************** lang_size_sections_1 (s, output_section_
*** 3062,3070 ****
  			&& (strcmp (lang_memory_region_list->name,
  				    "*default*") != 0
  			    || lang_memory_region_list->next != NULL))
! 		      einfo (_("%P: warning: no memory region specified for section `%s'\n"),
  			     bfd_get_section_name (output_bfd,
  						   os->bfd_section));
  
  		    dot = os->region->current;
  
--- 3062,3086 ----
  			&& (strcmp (lang_memory_region_list->name,
  				    "*default*") != 0
  			    || lang_memory_region_list->next != NULL))
! 		      {
! 			/* By default this is an error rather than just a
! 			   warning because if we allocate the section to the
! 			   default memory region we can end up creating an
! 			   excessivly large binary, or even seg faulting when
! 			   attmepting to perform a negative seek.  See
! 			     http://sources.redhat.com/ml/binutils/2003-04/msg00423.html			 
! 			   for an example of this.  This behaviour can be
! 			   overridden by the using the --no-check-sections
! 			   switch.  */
! 			if (command_line.check_section_addresses)
! 			  einfo (_("%P%F: error: no memory region specified for loadable section `%s'\n"),
  				 bfd_get_section_name (output_bfd,
  						       os->bfd_section));
+ 			else
+ 			  einfo (_("%P: warning: no memory region specified for loadable section `%s'\n"),
+ 				 bfd_get_section_name (output_bfd,
+ 						       os->bfd_section));
+ 		      }
  
  		    dot = os->region->current;
  

Re: Bad behavior with explicit phdrs and loadable orphans

[David Heine]

Nick Clifton wrote:
> Hi David,
> 
> 
>>Thanks for taking a look.  My initial thoughts on this were to find a
>>better memory region to put the orphan section like you've
>>implemented. However, the problem that caused the initial bad behavior
>>was that an assembly file had mis-spelled a section name.  While the
>>proposed patch keeps the system from generating a large file or
>>causing the SEEK overflow in this case, the problem would recur with
>>any orphan that can not be placed by this default placement algorithm
>>for some linker script.
>>   Because the patch manages to place the orphan section, it also
>>removes the warning that indicated the actual problem:
>>
>>	warning: no memory region specified for section
>>
>>For our target when using explicit memory regions, it is usually an
>>error to have orphan sections that are loadable and allocatable for
>>non-relocatable links.   I'm not sure if this is generally true for
>>other targets, but the existence of the original warning indicates
>>that it might be.
> 
> 
> Yes, I think that in general a loadable/allocatable section is intended
> to be used and hence ought to be explicitly handled by the linker
> script.
> 
> 
>>   It might be better to keep the old placement algorithm and convert
>>the warning: "no memory region specified for section" into a failure
>>that will avoid the massive file generation.
> 
> 
> Well I have not applied my suggested patch, since I was waiting for
> feedback from you, so there is no problem with keeping the old
> algorithm.
> 
> 
>>If you think the new placement algorithm makes more sense, it would
>>probably be less intrusive to other targets if it were modified to
>>first look for regions that match one of the positive attributes, then
>>look for ones with no attributes so that the behavior only changes for
>>sections that were previously unplaced.
> 
> 
> Basically I am in favour of the "if it ain't broke, don't fix it"
> approach.  So if we can leave the current placement algorithm in
> place, and just update the warning to an error, then I am in favour of
> doing that.  Please could you try out this alternative patch and let
> me know what you think.

This alternate patch works well.

-David

> Cheers
>         Nick
> 
> 2003-05-02  Nick Clifton  <nickc@redhat.com>
> 
> 	* ldlang.c (lang_size_sections_1): Default to generating a
> 	fatal error message if a loadable section is not allocated
> 	to a memory region when regions are defined.
> 
> Index: ld/ldlang.c
> ===================================================================
> RCS file: /cvs/src/src/ld/ldlang.c,v
> retrieving revision 1.112
> diff -c -3 -p -w -r1.112 ldlang.c
> *** ld/ldlang.c	29 Apr 2003 01:53:46 -0000	1.112
> --- ld/ldlang.c	2 May 2003 09:22:46 -0000
> *************** lang_size_sections_1 (s, output_section_
> *** 3050,3056 ****
>   
>   		    /* If a loadable section is using the default memory
>   		       region, and some non default memory regions were
> ! 		       defined, issue a warning.  */
>   		    if ((bfd_get_section_flags (output_bfd, os->bfd_section)
>   			 & (SEC_ALLOC | SEC_LOAD)) != 0
>   			&& (bfd_get_section_flags (output_bfd, os->bfd_section)
> --- 3050,3056 ----
>   
>   		    /* If a loadable section is using the default memory
>   		       region, and some non default memory regions were
> ! 		       defined, issue an error message.  */
>   		    if ((bfd_get_section_flags (output_bfd, os->bfd_section)
>   			 & (SEC_ALLOC | SEC_LOAD)) != 0
>   			&& (bfd_get_section_flags (output_bfd, os->bfd_section)
> *************** lang_size_sections_1 (s, output_section_
> *** 3062,3070 ****
>   			&& (strcmp (lang_memory_region_list->name,
>   				    "*default*") != 0
>   			    || lang_memory_region_list->next != NULL))
> ! 		      einfo (_("%P: warning: no memory region specified for section `%s'\n"),
>   			     bfd_get_section_name (output_bfd,
>   						   os->bfd_section));
>   
>   		    dot = os->region->current;
>   
> --- 3062,3086 ----
>   			&& (strcmp (lang_memory_region_list->name,
>   				    "*default*") != 0
>   			    || lang_memory_region_list->next != NULL))
> ! 		      {
> ! 			/* By default this is an error rather than just a
> ! 			   warning because if we allocate the section to the
> ! 			   default memory region we can end up creating an
> ! 			   excessivly large binary, or even seg faulting when
> ! 			   attmepting to perform a negative seek.  See
> ! 			     http://sources.redhat.com/ml/binutils/2003-04/msg00423.html			 
> ! 			   for an example of this.  This behaviour can be
> ! 			   overridden by the using the --no-check-sections
> ! 			   switch.  */
> ! 			if (command_line.check_section_addresses)
> ! 			  einfo (_("%P%F: error: no memory region specified for loadable section `%s'\n"),
>   				 bfd_get_section_name (output_bfd,
>   						       os->bfd_section));
> + 			else
> + 			  einfo (_("%P: warning: no memory region specified for loadable section `%s'\n"),
> + 				 bfd_get_section_name (output_bfd,
> + 						       os->bfd_section));
> + 		      }
>   
>   		    dot = os->region->current;
>   
> 

Re: Bad behavior with explicit phdrs and loadable orphans

[Nick Clifton]

Hi David,

> This alternate patch works well.

Excellent, in which case I shall apply it.

>> 2003-05-02  Nick Clifton  <nickc@redhat.com>
>> 	* ldlang.c (lang_size_sections_1): Default to generating a
>> 	fatal error message if a loadable section is not allocated
>> 	to a memory region when regions are defined.

Applied.

Cheers
        Nick