RE: [ECOS] binutils wizardry

RE: [ECOS] binutils wizardry

[Doyle, Patrick]

> -----Original Message-----
> From: Sergei Organov [mailto:osv@topconrd.ru] 
> Sent: Friday, July 30, 2004 9:48 AM
> To: Doyle, Patrick
> Cc: 'Gary Thomas'; ecos discuss list
> Subject: Re: [ECOS] binutils wizardry
> 
> $ echo | as -o elf_file
> $ objcopy --add-section=section_name=binary_file elf_file
> 

Thanks Sergei, that's exactly what I was looking for.  Now I'll go play with
"ld" to figure out how to paste that onto my (ELF) executable at address
0x80000.

--wpd


Patrick Doyle
DSP Design Engineer
(603) 546-2179

 

This communication is from DTC Communications, Inc. and is intended to be
confidential and solely for the use of the persons or entities addressed
above.  If you are not an intended recipient, be aware that the information
contained herein may be protected from unauthorized use by privilege or law,
and any copying, distribution, disclosure, or other use of this information
is prohibited.  If you have received this communication in error, please
contact the sender by return e-mail or telephone the above number
immediately and delete or destroy all copies.  Thank you for your
cooperation.

 



-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

RE: [ECOS] binutils wizardry

[Doyle, Patrick]

Thanks to Gary, Sergei, and Andrew for the pointers... they got me started
in the right direction.  Since I don't maintain a web page, much less a
blog, and I might want to find this again, I thought I would summarize what
I learned and what I did.  Perhaps somebody else might even find this useful
in some bizarre set of circumstances.

Anyway, my original problem was that I have an existing ELF executable that
loads into memory starting at address 0x20000 and some binary data that I
want associated with that executable that wants to be loaded into memory at
address 0x80000.  I guess I forgot to mention that I don't really want to
relink my executable every time them binary data changes.

Anyway, Sergei suggested

$ echo | as -o elf_file
$ objcopy --add-section=section_name=binary_file elf_file

which transforms the binary file into a section called "section_name" in an
ELF file.

Andrew suggested 

$ objcopy -I binary -O elf32-littlemips \
  --rename-section .data=.rodata,alloc,load,readonly,data,contents \
  binary_file elf_file

which does the same thing, but doesn't require the assembler to create a
blank ELF file first, and which allows one to manipulate the flags in the
section header.

Once I had the ELF version of my binary file, I found that I could append it
to my existing ELF image using objcopy.  (Actually, using Sergei's "objcopy"
command with a "--set-section-flags" argument could be used as well.
Unfortunately, no matter how hard I tried, I couldn't convince RedBoot to
load my newly appended section.  (Also, the objcopy command produced a
strange error when I tried it ("BFD: stHmId0k: warning: allocation section
`blah' not in segment") but objdump showed that it did append the section to
my file.

Interestingly, if I converted my file to an S-Record file (using "objcopy -O
srec") or to a raw binary file (using "objcopy -O binary"), my binary data
showed up -- I just couldn't get RedBoot to load it directly from the ELF
file.  I didn't like the way that the S-Record file doubled my output file
size.  The binary file also increased significantly in size due to the
padding of zeros between the end of my executable image (at something like
0x3a240) and the start of my data (at address 0x80000).  Also, it lost the
entry point information.

Looking at the source code for the "load" command in RedBoot, I learned that
it loaded memory based on the contents of the "program headers" rather than
the contents of the "section headers".  Now the error message from objcopy
started to make sense (except for that "stHmId0k" part).  Apparently,
objcopy can manipulate section headers, but not program headers.  The tool
that I've always used to manipulate program headers is "ld".

So, to make a long boring email even longer, what I did was to convert my
executable image into a binary file (keeping track of its start address and
entry point), and linked that with my data to produce a new executable with
two separate program headers.  RedBoot is happy to load this file.

If anybody is interested (is anybody still reading at this point?) I have
attached the perl script I created to perform this process.  Please don't
laugh too hard as you read it -- remember, you were just starting with perl
once too :-)

--wpd

=============================================================
#!/usr/bin/perl -w

use strict;

my $image   = "prog.img";
my $binfile = "wavefiles";
my $binaddr = "0x80000";
my $outfile = "combo";

my $tc     = "arm-elf-";	# toolchain
my $target = "elf32-littlearm";
my $arch   = "arm";

my $bin2elf = "${tc}objcopy -I binary -O $target -B $arch";

# I wonder how I would do the "grep" part in perl.  All I want to do is
# to grab the one line that starts with whitespace followed by the
# word "LOAD" out of the output of the "readelf" command.  Would it be
# more or less efficient to do it in perl?
# Of course, I also need the one line that starts with "Entry point"
# as well.  I really should figure out how to do this idiomatically.
my $image_info_line = `${tc}readelf -l $image | egrep '^[[:space:]]*LOAD'`;
my @image_info = split(' ', $image_info_line);
my $image_addr = $image_info[2];

my $entry_info_line = `${tc}readelf -l $image | egrep '^Entry point'`;
my @entry_info = split(' ', $entry_info_line);
my $entry_point = $entry_info[2];

system("${tc}objcopy -O binary $image $image.bin") == 0
    or die "error creating binary image of $image";
system("$bin2elf --rename-section .data=image $image.bin $image.o") == 0
    or die "error creating relocatable version of $image";

system("$bin2elf --rename-section .data=data $binfile $binfile.o") == 0
    or die "error creating relocatable version of $binfile";

open TARGET, ">", "target.ld" 
    or die "Unable to open \"target.ld\" for writing\n";
print TARGET <<EOF;
SECTIONS
{
image $image_addr : { *(image) }
data  $binaddr : { *(data) }
}
EOF
close TARGET;

system("${tc}ld -e $entry_point -o $outfile -T target.ld $image.o
$binfile.o") == 0
    or die "error creating output file $outfile";

====================================================================

Patrick Doyle
DSP Design Engineer
(603) 546-2179

 

This communication is from DTC Communications, Inc. and is intended to be
confidential and solely for the use of the persons or entities addressed
above.  If you are not an intended recipient, be aware that the information
contained herein may be protected from unauthorized use by privilege or law,
and any copying, distribution, disclosure, or other use of this information
is prohibited.  If you have received this communication in error, please
contact the sender by return e-mail or telephone the above number
immediately and delete or destroy all copies.  Thank you for your
cooperation.

 



-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

RE: [ECOS] binutils wizardry

[Andrew Dyer]

> -----Original Message-----
> From: ecos-discuss-owner@ecos.sourceware.org
> [mailto:ecos-discuss-owner@ecos.sourceware.org]On Behalf Of Doyle,
> Patrick
> Sent: Friday, July 30, 2004 7:42 AM
> To: ecos discuss list
> Subject: [ECOS] binutils wizardry
> 
> 
> I have an existing application that I load onto my device 
> with RedBoot.  I
> have a raw binary data file that I also load onto my device that my
> application expects to find in memory at address 0x80000.  I 
> would like to
> figure out some method, preferably _after_ I've linked my 
> main application,
> to glue these two pieces together into a single ELF file so 
> that I could
> download both pieces in one swell foop.
> 
> I'm reasonably sure I can do this with some subset of the programs in
> binutils, but I'm not too sure where to start.  I vaguely 
> remember looking
> at how Linux gets gzip'ed for booting (10 or more years ago), so I'll
> probably start there, but, in the mean time, I thought I 
> would ask for some
> pointers.
> 
> Any pointers?
> 


For building redboot on our custom platform I include binary data like
this:

In the .cdl file
(packages/hal/mips/rhtvp100/current/cdl/hal_mips_vr5500_rhtvp100.cdl
for me) I add the following build rule.

	# custom build rule to add the fpga configuration data into the
	# libtarget.a library after its built (in the .xildata section)
      # The data will either be included in the final image or not based
	# on section directives in the include/pkgconf/*.ldi files
	# 
	# FIXME - this is mostly just pure evil
	#
	# objcopy is forced to run from the source dir
	# because it creates symbols for the start and end of the data
	# (which we use) with the source filename (including path) 
	# embedded in it.  By running it in the source directory
	# we force the symbols to always be named the same no
	# matter where the source tree is
	#
	# $(shell pwd) gives a non-zero return code under cygwin
	# for some reason, so it's prefixed with a '-'
	#
	# objcopy doesn't like to overwrite it's output file so we
	# remove it by force
	#
	make -priority 201 {
        v2pro_top_flip.o : <PACKAGE>/src/v2pro_top_flip.bin
		-CURDIR = $(shell pwd)
		@rm -f $@
		cd $(<D) ; \
	      $(OBJCOPY) -I binary -O elf32-littlemips \
	          --rename-section
.data=.xildata,alloc,load,readonly,data,contents \
              $(<F) $(CURDIR)/$@
		$(AR) rcs $(PREFIX)/lib/libtarget.a $@
    }




in the linker include  
(packages/hal/mips/rhtvp100/current/include/pkgconf/mlt_mips_vr5500_rhtv
p100_romram.ldi
for me) file I include a section at the end 

SECTIONS
{
	(blah, blah, blah deleted)

	// bring in xilinx pointers and data
	.xil_data_p 0xbfc70000  :
  	{	
		LONG(ABSOLUTE(_binary_v2pro_top_flip_bin_start));
		LONG(ABSOLUTE(_binary_v2pro_top_flip_bin_end));
	    *(.xildata)
  	} > rom

    SECTIONS_END
}


this puts two word pointers to the start and end of the data
at virtual address 0xbfc70000 (necessary for our application,
not required otherwise) and immediately follows it with anything
in any of the files assigned to the .xildata section.

--
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] binutils wizardry

[Sergei Organov]

"Doyle, Patrick" <WPD@dtccom.com> writes:
> > From: Gary Thomas [mailto:gary@mlbassoc.com]
> >
> > Arrange to have your special data in it's own ELF section.  
> > Then you can
> > use objcopy to adjust the address of that section (and leave the rest 
> > alone).  You should be able to work out the details to get 
> > your program
> > to load one place and the data in a totally different one.
> > 
> It's the "Arrange to have your special data in it's own ELF section" bit
> that I'm looking to get started on.  Starting with the raw binary file, can
> I convince objcopy or ld to wrap the appropriate gizmos around it to turn it
> into an ELF section starting at address 0x80000?

That's how I do an elf file containing section called "section_name"
that contains data from a binary:

$ echo | as -o elf_file
$ objcopy --add-section=section_name=binary_file elf_file

BR,
Sergei.


-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

RE: [ECOS] binutils wizardry

[Gary Thomas]

On Fri, 2004-07-30 at 06:59, Doyle, Patrick wrote:
> > -----Original Message-----
> > From: Gary Thomas [mailto:gary@mlbassoc.com] 
> > Sent: Friday, July 30, 2004 8:50 AM
> > To: Doyle, Patrick
> > Cc: ecos discuss list
> > Subject: Re: [ECOS] binutils wizardry
> > 
> > 
> > 
> > Arrange to have your special data in it's own ELF section.  
> > Then you can
> > use objcopy to adjust the address of that section (and leave the rest 
> > alone).  You should be able to work out the details to get 
> > your program
> > to load one place and the data in a totally different one.
> > 
> It's the "Arrange to have your special data in it's own ELF section" bit
> that I'm looking to get started on.  Starting with the raw binary file, can
> I convince objcopy or ld to wrap the appropriate gizmos around it to turn it
> into an ELF section starting at address 0x80000?
> 
> Once I have that, how can I glue that ELF section to the existing ELF
> application?
> 
> While I'm looking for the "teach me how to fish" answer here, I'd be glad
> for the "give me a fish" solution as well -- I'll just reverse engineer the
> "teach me how to fish" from that :-)

I've had little luck convincing objcopy to turn raw binary data into
any usable format, although it is purported to work.  Others may have 
been more successful.

What I normally do in situations like this is to turn the binary data 
into something that I can assemble/compile.  Look at how I did it for
the uE250 (PXA250) platform:
  ../packages/hal/arm/xscale/uE250/current/src/uE250_pci_bitstream.h

Getting that data into it's own ELF section can then be done using
"attribute" - see how it's done for the FLASH drivers.  You will need
to add special handling in your linker script as well, but using the
FLASH drivers (.2ram sections) you should be able to figure it out.

I hope this helps.  Beyond this, I'd end up tinkering for a while until
I got it right - I'm sure you can do the same.

-- 
Gary Thomas <gary@mlbassoc.com>
MLB Associates


-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

RE: [ECOS] binutils wizardry

[Doyle, Patrick]

> -----Original Message-----
> From: Gary Thomas [mailto:gary@mlbassoc.com] 
> Sent: Friday, July 30, 2004 8:50 AM
> To: Doyle, Patrick
> Cc: ecos discuss list
> Subject: Re: [ECOS] binutils wizardry
> 
> 
> 
> Arrange to have your special data in it's own ELF section.  
> Then you can
> use objcopy to adjust the address of that section (and leave the rest 
> alone).  You should be able to work out the details to get 
> your program
> to load one place and the data in a totally different one.
> 
It's the "Arrange to have your special data in it's own ELF section" bit
that I'm looking to get started on.  Starting with the raw binary file, can
I convince objcopy or ld to wrap the appropriate gizmos around it to turn it
into an ELF section starting at address 0x80000?

Once I have that, how can I glue that ELF section to the existing ELF
application?

While I'm looking for the "teach me how to fish" answer here, I'd be glad
for the "give me a fish" solution as well -- I'll just reverse engineer the
"teach me how to fish" from that :-)

--wpd

Patrick Doyle
DSP Design Engineer
(603) 546-2179

 

This communication is from DTC Communications, Inc. and is intended to be
confidential and solely for the use of the persons or entities addressed
above.  If you are not an intended recipient, be aware that the information
contained herein may be protected from unauthorized use by privilege or law,
and any copying, distribution, disclosure, or other use of this information
is prohibited.  If you have received this communication in error, please
contact the sender by return e-mail or telephone the above number
immediately and delete or destroy all copies.  Thank you for your
cooperation.

 



-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] binutils wizardry

[Gary Thomas]

On Fri, 2004-07-30 at 06:42, Doyle, Patrick wrote:
> I have an existing application that I load onto my device with RedBoot.  I
> have a raw binary data file that I also load onto my device that my
> application expects to find in memory at address 0x80000.  I would like to
> figure out some method, preferably _after_ I've linked my main application,
> to glue these two pieces together into a single ELF file so that I could
> download both pieces in one swell foop.
> 
> I'm reasonably sure I can do this with some subset of the programs in
> binutils, but I'm not too sure where to start.  I vaguely remember looking
> at how Linux gets gzip'ed for booting (10 or more years ago), so I'll
> probably start there, but, in the mean time, I thought I would ask for some
> pointers.

This probably wouldn't help much with your situation.

> 
> Any pointers?

Arrange to have your special data in it's own ELF section.  Then you can
use objcopy to adjust the address of that section (and leave the rest 
alone).  You should be able to work out the details to get your program
to load one place and the data in a totally different one.

With some magic spells, you could probably even make this happen 
directly at link time (by editing the linker scripts).

-- 
Gary Thomas <gary@mlbassoc.com>
MLB Associates


-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

[ECOS] binutils wizardry

[Doyle, Patrick]

I have an existing application that I load onto my device with RedBoot.  I
have a raw binary data file that I also load onto my device that my
application expects to find in memory at address 0x80000.  I would like to
figure out some method, preferably _after_ I've linked my main application,
to glue these two pieces together into a single ELF file so that I could
download both pieces in one swell foop.

I'm reasonably sure I can do this with some subset of the programs in
binutils, but I'm not too sure where to start.  I vaguely remember looking
at how Linux gets gzip'ed for booting (10 or more years ago), so I'll
probably start there, but, in the mean time, I thought I would ask for some
pointers.

Any pointers?

--wpd


Patrick Doyle
DSP Design Engineer
(603) 546-2179

 

This communication is from DTC Communications, Inc. and is intended to be
confidential and solely for the use of the persons or entities addressed
above.  If you are not an intended recipient, be aware that the information
contained herein may be protected from unauthorized use by privilege or law,
and any copying, distribution, disclosure, or other use of this information
is prohibited.  If you have received this communication in error, please
contact the sender by return e-mail or telephone the above number
immediately and delete or destroy all copies.  Thank you for your
cooperation.

 


-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss