Help needed - defining the output section order

Help needed - defining the output section order

[Simon Kissel]

Hi List,

I'm working on a replaceable resource compiler for the FreePascal
(www.freepascal.org) team. I'm mentioning this to clarify that any
"upgrade your gcc", "pascal sucks anyway, use c" kind of answers
won't really help me ;)

After fiddling with this for a few nights, I really need some help
from you binutils gurus out there.

The situation is the following:

I need to link an object file into an ELF executable. This is how the
object file looks like:

Section Headers:
  [Nr] Name              Type            Addr     Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            00000000 000000 000000 00      0   0  0
  [ 1] .text             PROGBITS        00000000 000034 000000 00  AX  0   0  4
  [ 2] .data             PROGBITS        00000000 000034 000000 00  WA  0   0  4
  [ 3] .bss              NOBITS          00000000 000034 000000 00  WA  0   0  4
  [ 4] fpc.ressym        PROGBITS        00000000 000034 00000f 00   A  0   0  4
  [ 5] fpc.resstr        PROGBITS        00000000 000044 000000 00   A  0   0  4
  [ 6] fpc.reshash       PROGBITS        00000000 000044 000040 00   A  0   0  4
  [ 7] fpc.resdata       PROGBITS        00000000 000084 0005b8 00   A  0   0  4
  [ 8] fpc.resspare      NOBITS          00000000 00063c 0005b8 00   A  0   0  4
  [ 9] .shstrtab         STRTAB          00000000 00063c 000067 00      0   0  1
  [10] .symtab           SYMTAB          00000000 000884 000090 10     11   9  4
  [11] .strtab           STRTAB          00000000 000914 000001 00      0   0  1

What I need is to have the fpc.* sections in exactly the same order
(or at least have fpc.resspare directly after fpc.resdata) in the
output. Instead I get this:

Section Headers:
  [Nr] Name              Type            Addr     Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            00000000 000000 000000 00      0   0  0
  [ 1] .interp           PROGBITS        080480d4 0000d4 000013 00   A  0   0  1
  [ 2] .hash             HASH            080480e8 0000e8 000038 04   A  3   0  4
  [ 3] .dynsym           DYNSYM          08048120 000120 000090 10   A  4   1  4
  [ 4] .dynstr           STRTAB          080481b0 0001b0 00007f 00   A  0   0  1
  [ 5] .gnu.version      VERSYM          08048230 000230 000012 02   A  3   0  2
  [ 6] .gnu.version_r    VERNEED         08048244 000244 000020 00   A  4   1  4
  [ 7] .rel.dyn          REL             08048264 000264 000008 08   A  3   0  4
  [ 8] .rel.plt          REL             0804826c 00026c 000040 08   A  3   a  4
  [ 9] .init             PROGBITS        080482ac 0002ac 000034 00  AX  0   0  4
  [10] .plt              PROGBITS        080482e0 0002e0 000090 04  AX  0   0  4
  [11] .text             PROGBITS        08048370 000370 028ec4 00  AX  0   0  4
  [12] .fini             PROGBITS        08071234 029234 00001f 00  AX  0   0  4
  [13] fpc.ressym        PROGBITS        08071254 029254 00000f 00   A  0   0  4
  [14] fpc.resstr        PROGBITS        08071264 029264 000000 00   A  0   0  4
  [15] fpc.reshash       PROGBITS        08071264 029264 000040 00   A  0   0  4
  [16] fpc.resdata       PROGBITS        080712a4 0292a4 0005b8 00   A  0   0  4
  [17] .eh_frame         PROGBITS        08072000 02a000 000004 00  WA  0   0  4
  [18] .ctors            PROGBITS        08072004 02a004 000008 00  WA  0   0  4
  [19] .dtors            PROGBITS        0807200c 02a00c 000008 00  WA  0   0  4
  [20] .dynamic          DYNAMIC         08072014 02a014 0000c8 08  WA  4   0  4
  [21] .got              PROGBITS        080720dc 02a0dc 000004 04  WA  0   0  4
  [22] .got.plt          PROGBITS        080720e0 02a0e0 00002c 04  WA  0   0  4
  [23] .data             PROGBITS        0807210c 02a10c 00b450 00  WA  0   0  4
  [24] .bss              NOBITS          0807d560 03555c 001dd8 00  WA  0   0 16
  [25] fpc.resspare      NOBITS          0807f338 03555c 0005b8 00   A  0   0  4
  [26] .comment          PROGBITS        00000000 03555c 000098 00      0   0  1
  [27] .note             NOTE            00000000 0355f4 000050 00      0   0  1
  [28] .shstrtab         STRTAB          00000000 035644 0000ed 00      0   0  1

As you can see, fpc.resspare got moved away, probably because it's of
NOBITS type.

Now the big questions is: Is there any way to force ld to put the fpc.resspare
section after fpc.resdata?

From reading the docs it appeared to me that possibly using a SECTIONS
command in the linker script might do the trick - so added the
following to the end of the linker script:

SECTIONS{
  fpc.ressym    : { *(fpc.ressym) }
  fpc.resstr    : { *(fpc.resstr) }
  fpc.reshash   : { *(fpc.reshash) }
  fpc.resdata   : { *(fpc.resdata) }
  fpc.resspare  : { *(fpc.resspare) }
}

Sadly this only results in
"Not enough room for program headers (allocated 5, need 6)" and "final
link failed: Bad value".

Here's the really simple linker script. I know it's ugly, that's
because it's auto-generated by the compiler. Don't be shocked of the
c:\crossfpc\... stuff, this is GNU ld version 2.15.94 20050118 used
for cross-compiling/linking ELF under Windows) ;)

SEARCH_DIR(C:\crossfpc\lib\)
SEARCH_DIR(.\)
SEARCH_DIR(C:\crossfpc\fpcunits\i386-linux\rtl\)
SEARCH_DIR(C:\crossfpc\kylixcompat\)
INPUT(
C:\crossfpc\fpcunits\i386-linux\rtl\cprt21.o
C:\crossfpc\lib\crtbegin.o
C:\crossfpc\lib\crti.o
res.o
C:\crossfpc\fpcunits\i386-linux\rtl\system.o
C:\crossfpc\fpcunits\i386-linux\rtl\objpas.o
C:\crossfpc\fpcunits\i386-linux\rtl\cmem.o
C:\crossfpc\fpcunits\i386-linux\rtl\sysutils.o
C:\crossfpc\fpcunits\i386-linux\rtl\unix.o
C:\crossfpc\fpcunits\i386-linux\rtl\errors.o
C:\crossfpc\fpcunits\i386-linux\rtl\sysconst.o
C:\crossfpc\fpcunits\i386-linux\rtl\unixtype.o
C:\crossfpc\fpcunits\i386-linux\rtl\baseunix.o
C:\crossfpc\fpcunits\i386-linux\rtl\strings.o
C:\crossfpc\fpcunits\i386-linux\rtl\syscall.o
)
GROUP(
C:\crossfpc\examples\resources\libptestres.a
)
INPUT(
-lc
)
INPUT(
C:\crossfpc\lib\crtend.o
C:\crossfpc\lib\crtn.o
)

Using this script works, but will change the section order of res.o as
explained above. Adding the SECTIONS stuff listed above to the end of
the script gives the error message above.

Any pointers on how to achieve my goal would be very welcome. If you
instead wish me to explain "the bigger picture" (WHY I need this),
I'll be happy to do that, too.

Thank you,

Simon

Re: Help needed - defining the output section order

[Nick Clifton]

Hi Simon,

> Now the big questions is: Is there any way to force ld to put the fpc.resspare
> section after fpc.resdata?

Yes - use the SECTIONS command in the linker script...

> From reading the docs it appeared to me that possibly using a SECTIONS
> command in the linker script might do the trick - so added the
> following to the end of the linker script:
> 
> SECTIONS{
>   fpc.ressym    : { *(fpc.ressym) }
>   fpc.resstr    : { *(fpc.resstr) }
>   fpc.reshash   : { *(fpc.reshash) }
>   fpc.resdata   : { *(fpc.resdata) }
>   fpc.resspare  : { *(fpc.resspare) }
> }
> 
> Sadly this only results in
> "Not enough room for program headers (allocated 5, need 6)" and "final
> link failed: Bad value".

Ah well, that is because your new linker script is too simple.  There is 
a section in the linker documentation that talks about this problem:

   When producing an ELF output file, if the linker script uses
   the SIZEOF_HEADERS builtin function, the linker must compute
   the number of program headers before it has determined all
   the section addresses and sizes.  If the linker later
   discovers that it needs additional program headers, it will
   report an error "not enough room for program headers".  To
   avoid this error, you must avoid using the SIZEOF_HEADERS
   function, or you must rework your linker script to avoid
   forcing the linker to use additional program headers, or
   you must define the program headers yourself using the
   PHDRS command.

So what you probably need to do is to add a PHDRS command to your linker 
script.

Plus what you probably really want to do is to find out the default 
script used by the linker (by running ld --verbose) and then take the 
SECTIONS command from there and merge in your fpc.* sections.  The 
SECTIONS command in the default script will probably be a lot more 
comprehensive than the one that you are currently using.

Cheers
   Nick

Re: Help needed - defining the output section order

[Andreas Schwab]

Simon Kissel <scamp@untergrund.net> writes:

> From reading the docs it appeared to me that possibly using a SECTIONS
> command in the linker script might do the trick - so added the
> following to the end of the linker script:
>
> SECTIONS{
>   fpc.ressym    : { *(fpc.ressym) }
>   fpc.resstr    : { *(fpc.resstr) }
>   fpc.reshash   : { *(fpc.reshash) }
>   fpc.resdata   : { *(fpc.resdata) }
>   fpc.resspare  : { *(fpc.resspare) }
> }

You could try to put all fpc.* sections in a single output section.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re[2]: Help needed - defining the output section order

[Simon Kissel]

Hi Nick,

NC> Yes - use the SECTIONS command in the linker script...

In this case it would keep it in the order as the sections are listed
in the SECTIONS command? That would be cool.

NC> Ah well, that is because your new linker script is too simple.  There is
NC> a section in the linker documentation that talks about this problem:

NC>    When producing an ELF output file, if the linker script uses
NC>    the SIZEOF_HEADERS builtin function, the linker must compute
NC>    the number of program headers before it has determined all
NC>    the section addresses and sizes.  If the linker later
NC>    discovers that it needs additional program headers, it will
NC>    report an error "not enough room for program headers".  To
NC>    avoid this error, you must avoid using the SIZEOF_HEADERS
NC>    function, or you must rework your linker script to avoid
NC>    forcing the linker to use additional program headers, or
NC>    you must define the program headers yourself using the
NC>    PHDRS command.

...well, that part of the docs doesn't say "If you use the SECTIONS
command, generation of program headers will fail" ;)

(But this is a completely different topic)

NC> So what you probably need to do is to add a PHDRS command to your linker
NC> script.

Ok, that looks really complex. But if there is no other way, I'll try
to go and survive that route.

NC> Plus what you probably really want to do is to find out the default
NC> script used by the linker (by running ld --verbose) and then take the
NC> SECTIONS command from there and merge in your fpc.* sections. The
NC> SECTIONS command in the default script will probably be a lot more
NC> comprehensive than the one that you are currently using.

I'll try that first and report back.

Thank you,

Simon

Re[2]: Help needed - defining the output section order

[Simon Kissel]

Hello Andreas,

>> From reading the docs it appeared to me that possibly using a SECTIONS
>> command in the linker script might do the trick - so added the
>> following to the end of the linker script:
>>
>> SECTIONS{
>>   fpc.ressym    : { *(fpc.ressym) }
>>   fpc.resstr    : { *(fpc.resstr) }
>>   fpc.reshash   : { *(fpc.reshash) }
>>   fpc.resdata   : { *(fpc.resdata) }
>>   fpc.resspare  : { *(fpc.resspare) }
>> }

AS> You could try to put all fpc.* sections in a single output section.

No, this doesn't help. They need to be in separated sections.

Here's why:

First of all: The goal was to implement a replaceable resource system
for FPC's ELF flavors that is similar to Kylix ones.

The goals in particular were:

- Resources can be embedded into the executable by the compiler
- Resources can be accessed from inside the executable at runtime
- It's possible to add, extract and change resources in a compiled
  executable.

The ELF binary format doesn't really provide support for all this.
Most importantly it does not support resizing of sections in the
binary.

This is solved in Kylix (and now for FPC) the following way: Inside
the ELF binary there are a bunch of new sections, holding the
resources, their names, pointers to the sections - and, here is the
clever trick: A spare section that has 0 bytes saved in the exe, but
is marked as the same size as the currently embedded resources are
(this is possible in ELF). It directly follows the section that has
the resource data. Now, if after compiling the resources are changed,
both the resource data section and the spare section get resized. This
means that at maxmimum, you can double the size of the resources in an
exisiting exe. This always should be enough headroom. It's important
to point out, that the spare section does not take up any space. It's
just that empty address room of the same size as the current resource
data get's reserved, which later can be re-used.

Simon

Re[2]: Help needed - defining the output section order

[Simon Kissel]

Hi Nick,

NC> Plus what you probably really want to do is to find out the default
NC> script used by the linker (by running ld --verbose) and then take the
NC> SECTIONS command from there and merge in your fpc.* sections. The
NC> SECTIONS command in the default script will probably be a lot more
NC> comprehensive than the one that you are currently using.

What I now tried was to simply take the internal linker script
displayed using --verbose and put it at the end (also tried at the
beginning) of my linker script.

The interesting part is now that ld chokes on the exact same script
it uses internally (at least it says so), when it gets fed with it
from the outside:

SECTIONS
{
  /* Read-only sections, merged into text segment: */
  PROVIDE (__executable_start = 0x08048000); . = 0x08048000 + SIZEOF_HEADERS;
  .interp         : { *(.interp) }
  .hash           : { *(.hash) }
  .dynsym         : { *(.dynsym) }
  .dynstr         : { *(.dynstr) }
  .gnu.version    : { *(.gnu.version) }
  .gnu.version_d  : { *(.gnu.version_d) }
  .gnu.version_r  : { *(.gnu.version_r) }
  .rel.dyn        :
    {
      *(.rel.init)
      *(.rel.text .rel.text.* .rel.gnu.linkonce.t.*)
      *(.rel.fini)
      *(.rel.rodata .rel.rodata.* .rel.gnu.linkonce.r.*)
      *(.rel.data.rel.ro*)
      *(.rel.data .rel.data.* .rel.gnu.linkonce.d.*)
      *(.rel.tdata .rel.tdata.* .rel.gnu.linkonce.td.*)
      *(.rel.tbss .rel.tbss.* .rel.gnu.linkonce.tb.*)
      *(.rel.ctors)
      *(.rel.dtors)
      *(.rel.got)
      *(.rel.bss .rel.bss.* .rel.gnu.linkonce.b.*)
    }
  .rela.dyn       :
    {
      *(.rela.init)
      *(.rela.text .rela.text.* .rela.gnu.linkonce.t.*)
      *(.rela.fini)
      *(.rela.rodata .rela.rodata.* .rela.gnu.linkonce.r.*)
      *(.rela.data .rela.data.* .rela.gnu.linkonce.d.*)
      *(.rela.tdata .rela.tdata.* .rela.gnu.linkonce.td.*)
      *(.rela.tbss .rela.tbss.* .rela.gnu.linkonce.tb.*)
      *(.rela.ctors)
      *(.rela.dtors)
      *(.rela.got)
      *(.rela.bss .rela.bss.* .rela.gnu.linkonce.b.*)
    }
  .rel.plt        : { *(.rel.plt) }
  .rela.plt       : { *(.rela.plt) }
  .init           :
  {
    KEEP (*(.init))
  } =0x90909090
  .plt            : { *(.plt) }
  .text           :
  {
    *(.text .stub .text.* .gnu.linkonce.t.*)
    KEEP (*(.text.*personality*))
    /* .gnu.warning sections are handled specially by elf32.em.  */
    *(.gnu.warning)
  } =0x90909090
  .fini           :
  {
    KEEP (*(.fini))
  } =0x90909090
  PROVIDE (__etext = .);
  PROVIDE (_etext = .);
  PROVIDE (etext = .);
  .rodata         : { *(.rodata .rodata.* .gnu.linkonce.r.*) }
  .rodata1        : { *(.rodata1) }
  .eh_frame_hdr : { *(.eh_frame_hdr) }
  .eh_frame       : ONLY_IF_RO { KEEP (*(.eh_frame)) }
  .gcc_except_table   : ONLY_IF_RO { KEEP (*(.gcc_except_table)) *(.gcc_except_table.*) }
  /* Adjust the address for the data segment.  We want to adjust up to
     the same address within the page on the next page up.  */
  . = ALIGN (0x1000) - ((0x1000 - .) & (0x1000 - 1)); . = DATA_SEGMENT_ALIGN (0x1000, 0x1000);
  /* Exception handling  */
  .eh_frame       : ONLY_IF_RW { KEEP (*(.eh_frame)) }
  .gcc_except_table   : ONLY_IF_RW { KEEP (*(.gcc_except_table)) *(.gcc_except_table.*) }
  /* Thread Local Storage sections  */
  .tdata          : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
  .tbss           : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
  /* Ensure the __preinit_array_start label is properly aligned.  We
     could instead move the label definition inside the section, but
     the linker would then create the section even if it turns out to
     be empty, which isn't pretty.  */
  . = ALIGN(32 / 8);
  PROVIDE (__preinit_array_start = .);
  .preinit_array     : { KEEP (*(.preinit_array)) }
  PROVIDE (__preinit_array_end = .);
  PROVIDE (__init_array_start = .);
  .init_array     : { KEEP (*(.init_array)) }
  PROVIDE (__init_array_end = .);
  PROVIDE (__fini_array_start = .);
  .fini_array     : { KEEP (*(.fini_array)) }
  PROVIDE (__fini_array_end = .);
  .ctors          :
  {
    /* gcc uses crtbegin.o to find the start of
       the constructors, so we make sure it is
       first.  Because this is a wildcard, it
       doesn't matter if the user does not
       actually link against crtbegin.o; the
       linker won't look for a file to match a
       wildcard.  The wildcard also means that it
       doesn't matter which directory crtbegin.o
       is in.  */
    KEEP (*crtbegin*.o(.ctors))
    /* We don't want to include the .ctor section from
       from the crtend.o file until after the sorted ctors.
       The .ctor section from the crtend file contains the
       end of ctors marker and it must be last */
    KEEP (*(EXCLUDE_FILE (*crtend*.o ) .ctors))
    KEEP (*(SORT(.ctors.*)))
    KEEP (*(.ctors))
  }
  .dtors          :
  {
    KEEP (*crtbegin*.o(.dtors))
    KEEP (*(EXCLUDE_FILE (*crtend*.o ) .dtors))
    KEEP (*(SORT(.dtors.*)))
    KEEP (*(.dtors))
  }
  .jcr            : { KEEP (*(.jcr)) }
  .data.rel.ro : { *(.data.rel.ro.local) *(.data.rel.ro*) }
  .dynamic        : { *(.dynamic) }
  .got            : { *(.got) }
  . = DATA_SEGMENT_RELRO_END (12, .);
  .got.plt        : { *(.got.plt) }
  .data           :
  {
    *(.data .data.* .gnu.linkonce.d.*)
    KEEP (*(.gnu.linkonce.d.*personality*))
    SORT(CONSTRUCTORS)
  }
  .data1          : { *(.data1) }
  _edata = .;
  PROVIDE (edata = .);
  __bss_start = .;
  .bss            :
  {
   *(.dynbss)
   *(.bss .bss.* .gnu.linkonce.b.*)
   *(COMMON)
   /* Align here to ensure that the .bss section occupies space up to
      _end.  Align after .bss to ensure correct alignment even if the
      .bss section disappears because there are no input sections.  */
   . = ALIGN(32 / 8);
  }
  . = ALIGN(32 / 8);
  _end = .;
  PROVIDE (end = .);
  . = DATA_SEGMENT_END (.);
  /* Stabs debugging sections.  */
  .stab          0 : { *(.stab) }
  .stabstr       0 : { *(.stabstr) }
  .stab.excl     0 : { *(.stab.excl) }
  .stab.exclstr  0 : { *(.stab.exclstr) }
  .stab.index    0 : { *(.stab.index) }
  .stab.indexstr 0 : { *(.stab.indexstr) }
  .comment       0 : { *(.comment) }
  /* DWARF debug sections.
     Symbols in the DWARF debugging sections are relative to the beginning
     of the section so we begin them at 0.  */
  /* DWARF 1 */
  .debug          0 : { *(.debug) }
  .line           0 : { *(.line) }
  /* GNU DWARF 1 extensions */
  .debug_srcinfo  0 : { *(.debug_srcinfo) }
  .debug_sfnames  0 : { *(.debug_sfnames) }
  /* DWARF 1.1 and DWARF 2 */
  .debug_aranges  0 : { *(.debug_aranges) }
  .debug_pubnames 0 : { *(.debug_pubnames) }
  /* DWARF 2 */
  .debug_info     0 : { *(.debug_info .gnu.linkonce.wi.*) }
  .debug_abbrev   0 : { *(.debug_abbrev) }
  .debug_line     0 : { *(.debug_line) }
  .debug_frame    0 : { *(.debug_frame) }
  .debug_str      0 : { *(.debug_str) }
  .debug_loc      0 : { *(.debug_loc) }
  .debug_macinfo  0 : { *(.debug_macinfo) }
  /* SGI/MIPS DWARF 2 extensions */
  .debug_weaknames 0 : { *(.debug_weaknames) }
  .debug_funcnames 0 : { *(.debug_funcnames) }
  .debug_typenames 0 : { *(.debug_typenames) }
  .debug_varnames  0 : { *(.debug_varnames) }
  /DISCARD/ : { *(.note.GNU-stack) }
}

And LD reports "invalid assignment to location counter" on the line
that follows the last }.

I think I'm in need for a little more help here :)

(Also, I still would love to hear an alternative solution that does
not require the compiler to generate all LD default scripts for each
platform itself - this sounds like the direct way to version hell).

Simon

Re: Help needed - defining the output section order

[Nick Clifton]

Hi Simon,

> (Also, I still would love to hear an alternative solution that does
> not require the compiler to generate all LD default scripts for each
> platform itself - this sounds like the direct way to version hell).

Ok - how about using objcopy and the --add-section and --remove-section 
switches ?  When you need to resize the fpc.* sections you could use 
--remove-section to strip the old ones of the executable and then 
--add-section to add in the new ones.  Of course this assumes that you 
have some way of recreating the contents of these sections since the 
--add-section switch just reads in a file and dumps its contents into 
the newly created section.

Cheers
   Nick

Re: Help needed - defining the output section order

[Nick Clifton]

Hi Simon,

> What I now tried was to simply take the internal linker script
> displayed using --verbose and put it at the end (also tried at the
> beginning) of my linker script.

Are you using the -T linker command line switch so that your new script 
*overrides* the old linker script or are you just putting the name of 
the file containing the script on the linker command line ?  In the 
latter case the linker will treat it as an *additional* linker script to 
be used in addition to the builtin script which will cause all kinds of 
problems.

Cheers
   Nick

Re[2]: Help needed - defining the output section order

[Simon Kissel]

Hi Nick,

>> What I now tried was to simply take the internal linker script
>> displayed using --verbose and put it at the end (also tried at the
>> beginning) of my linker script.

NC> Are you using the -T linker command line switch so that your new script
NC> *overrides* the old linker script or are you just putting the name of
NC> the file containing the script on the linker command line ?  In the
NC> latter case the linker will treat it as an *additional* linker script to
NC> be used in addition to the builtin script which will cause all kinds of
NC> problems.

Oops. That was indeed the problem. With -T it now eats it's script
just fine.

Sadly if I add my
  fpc.ressym    : { *(fpc.ressym) }
  fpc.resstr    : { *(fpc.resstr) }
  fpc.reshash   : { *(fpc.reshash) }
  fpc.resdata   : { *(fpc.resdata) }
  fpc.resspare  : { *(fpc.resspare) }
block to it, I'm back at "Not enough room for program headers
(allocated 5, need 6)". Darn ;)

Simon

Re[2]: Help needed - defining the output section order

[Simon Kissel]

[Forgot the copy for the list, sorry]

Hi Nick,

>> (Also, I still would love to hear an alternative solution that does
>> not require the compiler to generate all LD default scripts for each
>> platform itself - this sounds like the direct way to version hell).

NC> Ok - how about using objcopy and the --add-section and --remove-section
NC> switches ?  When you need to resize the fpc.* sections you could use
NC> --remove-section to strip the old ones of the executable and then 
NC> --add-section to add in the new ones.  Of course this assumes that you
NC> have some way of recreating the contents of these sections since the
NC> --add-section switch just reads in a file and dumps its contents into
NC> the newly created section.

I have to admit I've got next to no knowledge about objcopy. But
looking at it from the ELF perspective, my logic says to me this can't
work: If one would change the size of a section when replacing it,
sections after it in the ELF would need be relocated. And this again
could only theoretically work if the symbols would still be present
and not stripped.

That's the whole concept behind what I'm trying to achieve - as we
know that it's impossible to resize sections in an ELF executable, we
use a NOBITS section directly following the section containing the
resource data. This way we can make sure there is address space
available in case we later wish to increase the size of the section
containing the resources. Changing the size now simply is a matter of
increasing the size of the resource data section, decreasing the size of the
NOBITS section, and changing the file offsets of the sections after
it - the load addresses of all sections remain unchanged, and
therefore relocation isn't needed.

That's what Borland has implemented on Linux for Kylix. However, they
wrote their own ELF linker to do that. And I'm now trying to achieve
the same thing WITHOUT having to write an own linker ;)

Simon

Re: Help needed - defining the output section order

[Nick Clifton]

Hi Simon,

> Sadly if I add my
>   fpc.ressym    : { *(fpc.ressym) }
>   fpc.resstr    : { *(fpc.resstr) }
>   fpc.reshash   : { *(fpc.reshash) }
>   fpc.resdata   : { *(fpc.resdata) }
>   fpc.resspare  : { *(fpc.resspare) }
> block to it, I'm back at "Not enough room for program headers
> (allocated 5, need 6)". Darn ;)

Right, but now you have a single script which contains a big SECTIONS 
command which handles everything and to which you can add a PHDR command.

Creating the PHDRS command should not be too bad.  Just take the linked 
  executable with the fpc.* sections in the wrong place and run "readelf 
-l" on it.  This should display the program headers for the executable. 
  Now all you have to do is to translate this information into a PHDRS 
command and add an extra header.

Here's an example from running this command on my local system:

   $ readelf -l `which readelf`
Program Headers:
   Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
   PHDR           0x000034 0x08048034 0x08048034 0x00100 0x00100 R E 0x4
   INTERP         0x000134 0x08048134 0x08048134 0x00013 0x00013 R   0x1
       [Requesting program interpreter: /lib/ld-linux.so.2]
   LOAD           0x000000 0x08048000 0x08048000 0x2fbdc 0x2fbdc R E 0x1000
   LOAD           0x02fbdc 0x08078bdc 0x08078bdc 0x005b8 0x01218 RW  0x1000
   DYNAMIC        0x02fbf0 0x08078bf0 0x08078bf0 0x000c8 0x000c8 RW  0x4
   NOTE           0x000148 0x08048148 0x08048148 0x00020 0x00020 R   0x4
   GNU_EH_FRAME   0x02fb64 0x08077b64 0x08077b64 0x0001c 0x0001c R   0x4
   GNU_STACK      0x000000 0x00000000 0x00000000 0x00000 0x00000 RW  0x4

Which turns into:

PHDRS
{
   headers PT_PHDR PHDRS ;
   interp PT_INTERP ;
   text PT_LOAD FILEHDR ;
   data PT_LOAD ;
   dynamic PT_DYNAMIC ;
   note PT_NOTE ;
   gnu_eh_frame  0xa74e550 ;
   gnu_stack 0xa74e551 ;
   extra PT_LOAD ;
}

Note - I had to use numbers for the header types not supported by the 
linker script syntax.  You will probably not need to do this.

You may then find that you need to add ":<phdr-name>" to various parts 
of the SECTIONS command,  especially the fpc.* assignments.  eg:

    fpc.ressym    : { *(fpc.ressym) } :extra
    fpc.resstr    : { *(fpc.resstr) } :extra
    fpc.reshash   : { *(fpc.reshash) } :extra
    fpc.resdata   : { *(fpc.resdata) } :extra
    fpc.resspare  : { *(fpc.resspare) } :extra

Cheers
   Nick

Re[2]: Help needed - defining the output section order

[Simon Kissel]

Hi Nick,

NC> Which turns into:

NC> PHDRS
NC> {
NC>    headers PT_PHDR PHDRS ;
NC>    interp PT_INTERP ;
NC>    text PT_LOAD FILEHDR ;
NC>    data PT_LOAD ;
NC>    dynamic PT_DYNAMIC ;
NC>    note PT_NOTE ;
NC>    gnu_eh_frame  0xa74e550 ;
NC>    gnu_stack 0xa74e551 ;
NC>    extra PT_LOAD ;
NC> }

Ok, so I tried this.

Here is the script:

SECTIONS
{
  /* Read-only sections, merged into text segment: */
  PROVIDE (__executable_start = 0x08048000); . = 0x08048000 + SIZEOF_HEADERS;
[...etc, the copied default ld script for elf_i386]
  /* SGI/MIPS DWARF 2 extensions */
  .debug_weaknames 0 : { *(.debug_weaknames) }
  .debug_funcnames 0 : { *(.debug_funcnames) }
  .debug_typenames 0 : { *(.debug_typenames) }
  .debug_varnames  0 : { *(.debug_varnames) }
  /DISCARD/ : { *(.note.GNU-stack) }
  fpc.ressym    : { *(fpc.ressym) }   :extra
  fpc.resstr    : { *(fpc.resstr) }   :extra
  fpc.reshash   : { *(fpc.reshash) }  :extra
  fpc.resdata   : { *(fpc.resdata) }  :extra
  fpc.resspare  : { *(fpc.resspare) } :extra
}

PHDRS
{
   headers PT_PHDR PHDRS ;
   interp PT_INTERP ;
   text PT_LOAD FILEHDR ;
   data PT_LOAD ;
   dynamic PT_DYNAMIC ;
   extra PT_LOAD ;
}
SEARCH_DIR(.\)
[...and so on]
INPUT(
[...and so on]
)

Well, it works. Sort of. The resulting executable now is >130 MB in
size, and completely broken:

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  PHDR           0x000034 0x00001034 0x00000000 0x000c0 0x000c0 R   0x4
  INTERP         0x000000 0x00000000 0x00000000 0x00000 0x00000     0x4
      [Requesting program interpreter: ¦ELF???]
  LOAD           0x000000 0x00000000 0x00000000 0x000f4 0x000f4 R   0x1000
  LOAD           0x0000f4 0x00000000 0x00000000 0x00000 0x00000     0x1000
  DYNAMIC        0x000000 0x00000000 0x00000000 0x00000 0x00000     0x4readelf:
Warning: the .dynamic section is not contained within the dynamic segment
  LOAD           0x001098 0x00000098 0x00000098 0x807d738 0x8080510 RWE 0x1000

Hrmm?! ;)

Simon

Re: Help needed - defining the output section order

[Nick Clifton]

Hi Simon,

> Here is the script:

> PHDRS
> {
>    headers PT_PHDR PHDRS ;
>    interp PT_INTERP ;
>    text PT_LOAD FILEHDR ;
>    data PT_LOAD ;
>    dynamic PT_DYNAMIC ;
>    extra PT_LOAD ;
> }

> Well, it works. Sort of. The resulting executable now is >130 MB in
> size, and completely broken:

Oh well.  It looks like you will need to sprinkle :text and :data 
liberally through the SECTIONS command then to make sure that sections 
are placed into the right segments.

Cheers
   Nick

PS.  Yes this is difficult but then you are trying to make the linker do 
something that it was not designed to do. :-(

Re: Help needed - defining the output section order

[Segher Boessenkool]

>> Sadly this only results in
>> "Not enough room for program headers (allocated 5, need 6)" and "final
>> link failed: Bad value".
>
> Ah well, that is because your new linker script is too simple.  There 
> is a section in the linker documentation that talks about this 
> problem:

[snip]

> So what you probably need to do is to add a PHDRS command to your 
> linker script.

It might be enough if you prevent ld from creating the magic stack
segment behind its own back, e.g., I have this in my Makefiles:

%.o: %.c
         $(CC) $(CFLAGS) -c $< -o $@ && $(OBJCOPY) -R .note.GNU-stack $@

(using /DISCARD/ in your linker script is _not_ enough).


Segher

Re[2]: Help needed - defining the output section order

[Simon Kissel]

Hello Segher,

Wednesday, August 24, 2005, 9:30:08 AM, you wrote:

SB> [snip]

>> So what you probably need to do is to add a PHDRS command to your 
>> linker script.

SB> It might be enough if you prevent ld from creating the magic stack
SB> segment behind its own back, e.g., I have this in my Makefiles:

SB> %.o: %.c
SB>          $(CC) $(CFLAGS) -c $< -o $@ && $(OBJCOPY) -R .note.GNU-stack $@

SB> (using /DISCARD/ in your linker script is _not_ enough).

This sounds very interesting - however, I have to admit I don't know
much about makefile syntax and/or gcc options - I'm a pascal guy,
after all ;). To which ld options would this translate to?

Thanks,

Simon

Re: Re[2]: Help needed - defining the output section order

[Segher Boessenkool]

> SB> It might be enough if you prevent ld from creating the magic stack
> SB> segment behind its own back, e.g., I have this in my Makefiles:
>
> SB> %.o: %.c
> SB>          $(CC) $(CFLAGS) -c $< -o $@ && $(OBJCOPY) -R 
> .note.GNU-stack $@
>
> SB> (using /DISCARD/ in your linker script is _not_ enough).
>
> This sounds very interesting - however, I have to admit I don't know
> much about makefile syntax and/or gcc options - I'm a pascal guy,
> after all ;). To which ld options would this translate to?

Not an ld option; instead, you run an objcopy command on every
object file, before you link them together.  An example is

	objcopy -R .note.GNU-stack example.o

Do this on _every_ object file that is used in the link.

Some background: if ld sees a .note.GNU-stack note section in
all of the input files, it puts a GNU_STACK segment into the
output file automagically.  Linux uses this information to make
the stack non-executable.  But this automatic generation of a
PHDR seems to screw up the precomputed count of PHDRS.  Maybe
we can call this a bug, even?


Segher