RE: bug in RedBoot ELF loader?

RE: bug in RedBoot ELF loader?

[Daly, Jeffrey]

Uh, are you confusing the difference between the disk image of a program
(which has the headers) and the the memory image of the program (which
only includes the instructions/data)?

-----Original Message-----
From: ecos-devel-owner@ecos.sourceware.org
[mailto:ecos-devel-owner@ecos.sourceware.org] On Behalf Of Bert Thomas
Sent: Wednesday, June 07, 2006 5:38 AM
To: Gary Thomas
Cc: eCos development
Subject: Re: bug in RedBoot ELF loader?

> Of course - the headers are just that; descriptions of stuff to come
> _later_ in the file.  A sane ELF file (files created by GNU ld behave
> this way) will have the various section headers first, followed by
> the actual program segments.  There is no need for a loader (like
> RedBoot which is what started this discussion) to ever load the
> headers as part of the image, rather only process them to figure
> out what needs to be loaded and where.  For example, a RAM program

I aggree with you on the sanity part. However, I bet that most if not 
all Linux executables have a segment that include the headers. I assume 
you are working on a Linux machine. Could you try readelf on ls for 
example?

Again, it is not that I disaggree with you. It is just that I observe it

isn't the way you and I expected it to be.

I suspect the reason that your example doesn't have a segment that 
includes the headers has something to do with the linker script you 
wrote to link that program.

Bert

Re: bug in RedBoot ELF loader?

[Gary Thomas]

On Wed, 2006-06-07 at 10:38 +0100, Bert Thomas wrote:
> > Of course - the headers are just that; descriptions of stuff to come
> > _later_ in the file.  A sane ELF file (files created by GNU ld behave
> > this way) will have the various section headers first, followed by
> > the actual program segments.  There is no need for a loader (like
> > RedBoot which is what started this discussion) to ever load the
> > headers as part of the image, rather only process them to figure
> > out what needs to be loaded and where.  For example, a RAM program
> 
> I aggree with you on the sanity part. However, I bet that most if not 
> all Linux executables have a segment that include the headers. I assume 
> you are working on a Linux machine. Could you try readelf on ls for 
> example?
> 
> Again, it is not that I disaggree with you. It is just that I observe it 
> isn't the way you and I expected it to be.
> 
> I suspect the reason that your example doesn't have a segment that 
> includes the headers has something to do with the linker script you 
> wrote to link that program.

Fair enough, but I don't care about Linux executables (at least,
not in this context).  RedBoot loads programs to run on embedded
systems (including Linux kernels) and every one to date was
organized in this [sane] fashion.

All of that aside, if the program segments are not laid out in
strictly one-direction (i.e. can be processed from the start
of the file to the end), then RedBoot's loader can't handle them
anyway since it has virtually no way to position randomly within
the "file".  Remember that the most common "file" is actually a
stream of data (X-Modem, TFTP, etc) and RedBoot cannot afford to
have an entire copy of it in memory, hence no random jumping about.

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: bug in RedBoot ELF loader?

[Bert Thomas]

> Of course - the headers are just that; descriptions of stuff to come
> _later_ in the file.  A sane ELF file (files created by GNU ld behave
> this way) will have the various section headers first, followed by
> the actual program segments.  There is no need for a loader (like
> RedBoot which is what started this discussion) to ever load the
> headers as part of the image, rather only process them to figure
> out what needs to be loaded and where.  For example, a RAM program

I aggree with you on the sanity part. However, I bet that most if not 
all Linux executables have a segment that include the headers. I assume 
you are working on a Linux machine. Could you try readelf on ls for 
example?

Again, it is not that I disaggree with you. It is just that I observe it 
isn't the way you and I expected it to be.

I suspect the reason that your example doesn't have a segment that 
includes the headers has something to do with the linker script you 
wrote to link that program.

Bert

Re: bug in RedBoot ELF loader?

[Gary Thomas]

On Tue, 2006-06-06 at 22:32 +0100, Bert Thomas wrote:
> > 
> > It means nonsense.
> > 
> > I do not agree that it's common for programs to include their ELF
> > header within the code.  A *long* time ago, things were structured
> > that way, but we've progressed quite a lot (since the 70's :-)
> 
> Up until now I have been unable to find a program that, according to 
> readelf, does not include the headers. If you do readelf on something on 
> your system, do the program segment headers specify an offset that skips 
> the headers themselves?

Of course - the headers are just that; descriptions of stuff to come
_later_ in the file.  A sane ELF file (files created by GNU ld behave
this way) will have the various section headers first, followed by
the actual program segments.  There is no need for a loader (like
RedBoot which is what started this discussion) to ever load the
headers as part of the image, rather only process them to figure
out what needs to be loaded and where.  For example, a RAM program
on my PowerPC system (suitable for loading with RedBoot) looks like
this:
[gthomas@hermes RAM]$ readelf -e install/bin/redboot.elf 
ELF Header:
  Magic:   7f 45 4c 46 01 02 01 00 00 00 00 00 00 00 00 00 
  Class:                             ELF32
  Data:                              2's complement, big endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              EXEC (Executable file)
  Machine:                           PowerPC
  Version:                           0x1
  Entry point address:               0x80100
  Start of program headers:          52 (bytes into file)
  Start of section headers:          503664 (bytes into file)
  Flags:                             0x0
  Size of this header:               52 (bytes)
  Size of program headers:           32 (bytes)
  Number of program headers:         1
  Size of section headers:           40 (bytes)
  Number of section headers:         23
  Section header string table index: 20

Section Headers:
  [Nr] Name              Type            Addr     Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            00000000 000000 000000 00      0   0  0
  [ 1] .vectors          PROGBITS        00080000 010000 001500 00  AX  0   0 256
  [ 2] .text             PROGBITS        00081500 011500 01be48 00  AX  0   0  4
  [ 3] .fini             PROGBITS        0009d348 034640 000000 00   W  0   0  1
  [ 4] .rodata1          PROGBITS        0009d348 034640 000000 00   W  0   0  1
  [ 5] .rodata           PROGBITS        0009d348 02d348 0058f0 00   A  0   0  4
  [ 6] .fixup            PROGBITS        000a2c38 034640 000000 00   W  0   0  1
  [ 7] .gcc_except_table PROGBITS        000a2c38 034640 000000 00   W  0   0  1
  [ 8] .data             PROGBITS        000a2c38 032c38 001a08 04 WAX  0   0  8
  [ 9] .sbss             NOBITS          000a4640 034640 0000a0 00  WA  0   0  4
  [10] .bss              NOBITS          000a46e0 034640 01db88 00  WA  0   0 32
  [11] .debug_line       PROGBITS        00000000 034640 00cf33 00      0   0  1
  [12] .debug_info       PROGBITS        00000000 041573 01fedb 00      0   0  1
  [13] .debug_abbrev     PROGBITS        00000000 06144e 007bc5 00      0   0  1
  [14] .debug_aranges    PROGBITS        00000000 069018 001a90 00      0   0  8
  [15] .debug_pubnames   PROGBITS        00000000 06aaa8 003323 00      0   0  1
  [16] .debug_str        PROGBITS        00000000 06ddcb 007688 01  MS  0   0  1
  [17] .comment          PROGBITS        00000000 075453 0005a0 00      0   0  1
  [18] .debug_frame      PROGBITS        00000000 0759f4 003cf4 00      0   0  4
  [19] .debug_ranges     PROGBITS        00000000 0796e8 0017a8 00      0   0  1
  [20] .shstrtab         STRTAB          00000000 07ae90 0000df 00      0   0  1
  [21] .symtab           SYMTAB          00000000 07b308 005e60 10     22 922  4
  [22] .strtab           STRTAB          00000000 081168 003e87 00      0   0  1
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings)
  I (info), L (link order), G (group), x (unknown)
  O (extra OS processing required) o (OS specific), p (processor specific)

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  LOAD           0x010000 0x00080000 0x00080000 0x24640 0x42268 RWE 0x10000

 Section to Segment mapping:
  Segment Sections...
   00     .vectors .text .rodata .data .sbss .bss 

All of the program and section headers are in the first 0x10000 bytes of the
file (actually much less than that).  The real data, which is all that RedBoot
needs to load into RAM starts at offset 0x10000 in the file.

> 
> If my analysis is wrong, can you explain what's wrong?
> 
> Do linker scripts have an influence on this? (I mean: other then 
> obviously the number and size of segments)
> 
> Bert
-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: bug in RedBoot ELF loader?

[Gary Thomas]

On Tue, 2006-06-06 at 19:05 +0100, Bert Thomas wrote:
> David Vrabel wrote:
> > Bert Thomas wrote:
> > 
> >>Example:
> >>
> >>An ELF file with 2 segments:
> >>segment 0: offset in file = 0, length = 0xac
> >>segment 1: offset in file = 0xac, length = 15
> > 
> > 
> > An ELF with a section having a file offset of 0 looks bogus to me.  Why
> > would you need to load the ELF header into RAM?
> 
> Example: readelf -a /bin/ls
> 
> <deletia>
> Program Headers:
>    Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
>    PHDR           0x000034 0x08048034 0x08048034 0x000e0 0x000e0 R E 0x4
>    INTERP         0x000114 0x08048114 0x08048114 0x00013 0x00013 R   0x1
>        [Requesting program interpreter: /lib/ld-linux.so.2]
>    LOAD           0x000000 0x08048000 0x08048000 0x0e994 0x0e994 R E 0x1000
>    LOAD           0x00f000 0x08057000 0x08057000 0x003c0 0x0074c RW  0x1000
>    DYNAMIC        0x00f188 0x08057188 0x08057188 0x000d0 0x000d0 RW  0x4
>    NOTE           0x000128 0x08048128 0x08048128 0x00020 0x00020 R   0x4
>    GNU_EH_FRAME   0x00e968 0x08056968 0x08056968 0x0002c 0x0002c R   0x4
> <deletia>
> 
> Read:
> http://www.muppetlabs.com/~breadbox/software/tiny/teensy.html
> 
> "(it's normal behavior for a program to include its ELF header and 
> program header table in its memory image)"

Bunk :-(  Shades of old COFF files...

> 
> I'd aggree with you however...
> 
> Bert
-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: bug in RedBoot ELF loader?

[Bert Thomas]

David Vrabel wrote:
> Bert Thomas wrote:
> 
>>Example:
>>
>>An ELF file with 2 segments:
>>segment 0: offset in file = 0, length = 0xac
>>segment 1: offset in file = 0xac, length = 15
> 
> 
> An ELF with a section having a file offset of 0 looks bogus to me.  Why
> would you need to load the ELF header into RAM?

Example: readelf -a /bin/ls

<deletia>
Program Headers:
   Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
   PHDR           0x000034 0x08048034 0x08048034 0x000e0 0x000e0 R E 0x4
   INTERP         0x000114 0x08048114 0x08048114 0x00013 0x00013 R   0x1
       [Requesting program interpreter: /lib/ld-linux.so.2]
   LOAD           0x000000 0x08048000 0x08048000 0x0e994 0x0e994 R E 0x1000
   LOAD           0x00f000 0x08057000 0x08057000 0x003c0 0x0074c RW  0x1000
   DYNAMIC        0x00f188 0x08057188 0x08057188 0x000d0 0x000d0 RW  0x4
   NOTE           0x000128 0x08048128 0x08048128 0x00020 0x00020 R   0x4
   GNU_EH_FRAME   0x00e968 0x08056968 0x08056968 0x0002c 0x0002c R   0x4
<deletia>

Read:
http://www.muppetlabs.com/~breadbox/software/tiny/teensy.html

"(it's normal behavior for a program to include its ELF header and 
program header table in its memory image)"

I'd aggree with you however...

Bert

Re: bug in RedBoot ELF loader?

[David Vrabel]

Bert Thomas wrote:
> 
> Example:
> 
> An ELF file with 2 segments:
> segment 0: offset in file = 0, length = 0xac
> segment 1: offset in file = 0xac, length = 15

An ELF with a section having a file offset of 0 looks bogus to me.  Why
would you need to load the ELF header into RAM?

David Vrabel
-- 
David Vrabel, Design Engineer

Arcom, Clifton Road           Tel: +44 (0)1223 411200 ext. 3233
Cambridge CB1 7EA, UK         Web: http://www.arcom.com/

bug in RedBoot ELF loader?

[Bert Thomas]

Hi all,

I think I may have found a bug in the RedBoot ELF loader, but it may be 
just as well that I don't understand the code well enough. Here's the part:

*************
    for (phx = 0;  phx < ehdr.e_phnum;  phx++) {
         if (phdr[phx].p_type == PT_LOAD) {
             // Loadable segment
             addr = (unsigned char *)phdr[phx].p_vaddr;
             len = phdr[phx].p_filesz;
             if ((unsigned long)addr < lowest_address) {
                 lowest_address = (unsigned long)addr;
             }
             addr += addr_offset;
             if (offset > phdr[phx].p_offset) {
                 if ((phdr[phx].p_offset + len) < offset) {
                     diag_printf("Can't load ELF file - program headers 
out of order\n");
                     return 0;
                 }
                 addr += offset - phdr[phx].p_offset;
             } else {
                 while (offset < phdr[phx].p_offset) {
                     if ((*getc)() < 0) {
                         diag_printf(SHORT_DATA);
                         return 0;
                     }
                     offset++;
                 }
             }
*************

It appears that normally the first program segment header contains a 
file offset of 0. This is on purpose, to tell the loader to load the ELF 
header and the program segment headers as well I am told. However, this 
is where the redboot loader fails. It can't seek back to the beginning 
of the file. The destination address in memory is compensated for that 
by this line:

addr += offset - phdr[phx].p_offset;

(At least, that's what I think).

However, I know for sure that it is incorrect to read 'len' bytes from 
the current offset instead of the real offset. This results in trouble 
when the next segment to be loaded is smaller then the "offset error". 
In some of my experiments the next segment was a "comment" segment that 
I didn't really need to have in memory. However, the loader failed 
loading my segment, saying "Can't load ELF file - program headers out of 
order". They weren't out of order, but I can see why it comes to that 
conclusion.

Example:

An ELF file with 2 segments:
segment 0: offset in file = 0, length = 0xac
segment 1: offset in file = 0xac, length = 15

First iteration, to load segment 0, offset=84
len = 0xac (172 bytes), so start reading 172 bytes from offset 84
Second iteration,to load segment 1, offset=256 (84+172)
len = 15, expected offset in file=172 (0xac), expected end in file = 
172+15=187
The current offset is 256 and thus larger then 187 -> error!

My suggested fix would be to adjust 'len' to the remaining bytes, say:

len -= offset;

Regards,
Bert