New bfd file format bfd_image ....

New bfd file format bfd_image ....

[Andrew Cagney]

Hello,

Not so recently the bfd open method:

> /* Create a new BFD as if by bfd_openr.  Rather than opening a file,
>    reconstruct an ELF file by reading the segments out of remote memory
>    based on the ELF file header at EHDR_VMA and the ELF program headers it
>    points to.  If not null, *LOADBASEP is filled in with the difference
>    between the VMAs from which the segments were read, and the VMAs the
>    file headers (and hence BFD's idea of each section's VMA) put them at.
> 
>    The function TARGET_READ_MEMORY is called to copy LEN bytes from the
>    remote memory at target address VMA into the local buffer at MYADDR; it
>    should return zero on success or an `errno' code on failure.  TEMPL must
>    be a BFD for an ELF target with the word size and byte order found in
>    the remote memory.  */
> extern bfd *bfd_elf_bfd_from_remote_memory
>   (bfd *templ, bfd_vma ehdr_vma, bfd_vma *loadbasep,
>    int (*target_read_memory) (bfd_vma vma, char *myaddr, int len));

was added.  It's there to let GDB create a BFD corresponding to an 
in-memory image of an object file.  This in turn being needed by GDB to 
open/extract GNU/Linux's vsyscall page.

Now that GDB's started trying to use it, it enountered an itegration 
problem - the code leads to a hard-wired assumption that there's elf in 
bfd vis:
http://sources.redhat.com/ml/gdb/2004-04/msg00140.html

I'd like to propose that BFD add a new ``file format'' ``image'' (better 
name welcome) that more closely reflects what is going on here.

Given an ``image'', it would then be possible to open a bfd using the 
more traditional sequence:

	abfd = bfd_open... (memory image)
	bfd_check_format (abfd, bfd_image)

Rather than talk theory, I've prototyped the idea and found the exact 
sequence is like:

+  nbfd = bfd_openr_iovec ("shared object read from target memory",
+                         gnutarget, inf_open, NULL,
+                         inf_pread, inf_close);
+  /* Use start_address to tunnel the address of the ELF header through
+     to the check-format code.  */
+  bfd_set_start_address (nbfd, elf_header_addr);
+  if (!bfd_check_format (nbfd, bfd_image))

creating NBFD with what looks like a normal elf BFD object.

The prototype identified the following issues:

- the theory appears sound

It should be possible to feed libbfd a memory image and the address of 
an in-memory executable header and have it return a BFD.  It would, for 
instance, open the possability of doing things like:
	objdump --flags -h /proc/$$/mem
to get a dump of whats in part of a memory image.

- there's a problem tunneling the address of the in-memory elf header

The inital bfd_open results in a BFD that has access to the entire 
in-memory image.  For check_format (bfd_elf_image_p) to parse the elf 
header it needs that header's address (ok header's file offset).  In my 
prototype I tunneled the address of the elf header by storing it in 
start_address.  Something else is needed.  One thought was:
	abfd = bfd_open
	bfd_seek (abfd, header_address, SEEK_SET)
	bfd_check_format (....)
only bfd_check_format does a seek(0) before probing the format.  Another 
thought was to use a filed like:
	abfd->origin
comments?

- there's a problem passing back LMA vs VMA

The image, as with a shared library, can be re-located by a constant 
offset.  I returned this information using section.lma, not sure if its 
a good idea (but it's not a bad idea).

- the final BFD looks like a file not memory

I (as did roland before me) convert the memory image into something that 
more closely resembles the original file.  This was easier then trying 
to reproduce the code needed to create BFD that retains the old memory 
image (leave that as a very long term goal).

comments, thoughts, suggestions, worth trying to integrate?

Andrew

Re: New bfd file format bfd_image ....

[Ian Lance Taylor]

Andrew Cagney <cagney@gnu.org> writes:

> - there's a problem tunneling the address of the in-memory elf header

I don't really understand what this is used for, so please pardon the
obvious questions.  The ELF header in an ELF file is always at offset
zero.  Why can't you use that?  Why do you need to specify the address
somehow?  Assuming some obvious answer to that, could you pass the
information to the iovec somehow?

> Another thought was to use a filed like:
> 	abfd->origin
> comments?

I suppose that I do think it would be reasonable to consistently use
abfd->origin as a file offset.  The current code only uses it when
abfd->my_archive is not NULL, but I don't see why we need to keep that
requirement.  We just set abfd->origin to 0 in the normal case.

Ian

Re: New bfd file format bfd_image ....

[Andrew Cagney]

> Andrew Cagney <cagney@gnu.org> writes:
> 
> 
>>> - there's a problem tunneling the address of the in-memory elf header
> 
> 
> I don't really understand what this is used for, so please pardon the
> obvious questions.  The ELF header in an ELF file is always at offset
> zero.  Why can't you use that?  Why do you need to specify the address
> somehow?  Assuming some obvious answer to that, could you pass the
> information to the iovec somehow?

It is the runtime address of the elf header as loaded into the 
executable.  Perhaphs ``bfd_runtime'' is a better name for this ``file 
format''.

>>> Another thought was to use a filed like:
>>> 	abfd->origin
>>> comments?
> 
> 
> I suppose that I do think it would be reasonable to consistently use
> abfd->origin as a file offset.  The current code only uses it when
> abfd->my_archive is not NULL, but I don't see why we need to keep that
> requirement.  We just set abfd->origin to 0 in the normal case.

ok.

Andrew

Re: New bfd file format bfd_image ....

[Andrew Cagney]

[-- Attachment #1: Type: text/plain, Size: 339 bytes --]

Hello,

How should I proceed with this?  The first step would be to add 
bfd_runtime to the list of formats.  Anyone got an easy way to check 
that I've added an extra entry to all bfds?  Compiling with -Werror 
didn't seem to help much here :-?

(wonders out loud if BFD_SEND_FMT should be replaced with corresponding 
functions)

Andrew

[-- Attachment #2: Attached Message --]
[-- Type: message/rfc822, Size: 6744 bytes --]

From: Andrew Cagney <cagney@gnu.org>
To: binutils@sources.redhat.com
Subject: New bfd file format bfd_image ....
Date: Mon, 03 May 2004 18:32:26 -0400
Message-ID: <4096C87A.1080504@gnu.org>

Hello,

Not so recently the bfd open method:

> /* Create a new BFD as if by bfd_openr.  Rather than opening a file,
>    reconstruct an ELF file by reading the segments out of remote memory
>    based on the ELF file header at EHDR_VMA and the ELF program headers it
>    points to.  If not null, *LOADBASEP is filled in with the difference
>    between the VMAs from which the segments were read, and the VMAs the
>    file headers (and hence BFD's idea of each section's VMA) put them at.
> 
>    The function TARGET_READ_MEMORY is called to copy LEN bytes from the
>    remote memory at target address VMA into the local buffer at MYADDR; it
>    should return zero on success or an `errno' code on failure.  TEMPL must
>    be a BFD for an ELF target with the word size and byte order found in
>    the remote memory.  */
> extern bfd *bfd_elf_bfd_from_remote_memory
>   (bfd *templ, bfd_vma ehdr_vma, bfd_vma *loadbasep,
>    int (*target_read_memory) (bfd_vma vma, char *myaddr, int len));

was added.  It's there to let GDB create a BFD corresponding to an 
in-memory image of an object file.  This in turn being needed by GDB to 
open/extract GNU/Linux's vsyscall page.

Now that GDB's started trying to use it, it enountered an itegration 
problem - the code leads to a hard-wired assumption that there's elf in 
bfd vis:
http://sources.redhat.com/ml/gdb/2004-04/msg00140.html

I'd like to propose that BFD add a new ``file format'' ``image'' (better 
name welcome) that more closely reflects what is going on here.

Given an ``image'', it would then be possible to open a bfd using the 
more traditional sequence:

	abfd = bfd_open... (memory image)
	bfd_check_format (abfd, bfd_image)

Rather than talk theory, I've prototyped the idea and found the exact 
sequence is like:

+  nbfd = bfd_openr_iovec ("shared object read from target memory",
+                         gnutarget, inf_open, NULL,
+                         inf_pread, inf_close);
+  /* Use start_address to tunnel the address of the ELF header through
+     to the check-format code.  */
+  bfd_set_start_address (nbfd, elf_header_addr);
+  if (!bfd_check_format (nbfd, bfd_image))

creating NBFD with what looks like a normal elf BFD object.

The prototype identified the following issues:

- the theory appears sound

It should be possible to feed libbfd a memory image and the address of 
an in-memory executable header and have it return a BFD.  It would, for 
instance, open the possability of doing things like:
	objdump --flags -h /proc/$$/mem
to get a dump of whats in part of a memory image.

- there's a problem tunneling the address of the in-memory elf header

The inital bfd_open results in a BFD that has access to the entire 
in-memory image.  For check_format (bfd_elf_image_p) to parse the elf 
header it needs that header's address (ok header's file offset).  In my 
prototype I tunneled the address of the elf header by storing it in 
start_address.  Something else is needed.  One thought was:
	abfd = bfd_open
	bfd_seek (abfd, header_address, SEEK_SET)
	bfd_check_format (....)
only bfd_check_format does a seek(0) before probing the format.  Another 
thought was to use a filed like:
	abfd->origin
comments?

- there's a problem passing back LMA vs VMA

The image, as with a shared library, can be re-located by a constant 
offset.  I returned this information using section.lma, not sure if its 
a good idea (but it's not a bad idea).

- the final BFD looks like a file not memory

I (as did roland before me) convert the memory image into something that 
more closely resembles the original file.  This was easier then trying 
to reproduce the code needed to create BFD that retains the old memory 
image (leave that as a very long term goal).

comments, thoughts, suggestions, worth trying to integrate?

Andrew

Re: New bfd file format bfd_image ....

[Nick Clifton]

Hi Andrew,

> How should I proceed with this?  The first step would be to add 
> bfd_runtime to the list of formats.  Anyone got an easy way to check 
> that I've added an extra entry to all bfds?

Compile a bfd library with --enable-targets=all --enable-64-bit-bfd and 
then link it with a small application that runs over the known vectors 
printing out a field from the end of the bfd ?

Cheers
   Nick

Re: New bfd file format bfd_image ....

[Andrew Cagney]

> Hi Andrew,
> 
>> How should I proceed with this?  The first step would be to add bfd_runtime to the list of formats.  Anyone got an easy way to check that I've added an extra entry to all bfds?
> 
> 
> Compile a bfd library with --enable-targets=all --enable-64-bit-bfd and then link it with a small application that runs over the known vectors printing out a field from the end of the bfd ?

That's just what I was hopeing you wouldn't say :-)  Will do (mumble 
something about adding it as a testcase :-).

Andrew