Working with ELF already loaded in memory

Working with ELF already loaded in memory

[Sonal Santan]

Hello,

We are exploring using libdw in XRT, https://github.com/Xilinx/XRT. In the use case we have, the ELF (a shared library) is already loaded in memory as data by an XRT application. In XRT we would like to walk through the DWARF/ELF information embedded in this shared library. 

Going through the libdw it appears that all APIs require either a file handle or a file name of the ELF object to create a session. Since we do not have access to the ELF file -- but rather the ELF file contents are already loaded in memory -- is there any other mechanism to create a session for extracting DWARF information using libdw? 

Thanks,
-Sonal 

Re: Working with ELF already loaded in memory

[Mark Wielaard]

Hi Sinal,

On Thu, 2021-07-08 at 05:02 +0000, Sonal Santan via Elfutils-devel
wrote:
> Going through the libdw it appears that all APIs require either a
> file handle or a file name of the ELF object to create a session.
> Since we do not have access to the ELF file -- but rather the ELF
> file contents are already loaded in memory -- is there any other
> mechanism to create a session for extracting DWARF information using
> libdw? 

Yes, if you just need the information already loaded into memory and
you know where the library is mapped you can use:

/* Create descriptor for memory region.  */
extern Elf *elf_memory (char *__image, size_t __size);

You can then use that Elf handle to extract the information that has
been mapped in. Which often is not the actual debug information though.

If you have a Elf handle you can use:

/* Returns the build ID as found in a NT_GNU_BUILD_ID note from either
   a SHT_NOTE section or from a PT_NOTE segment if the ELF file
   doesn't contain any section headers.  On success a pointer to the
   build ID is written to *BUILDID_P, and the positive length of the
   build ID is returned.  Returns 0 if the ELF lacks a NT_GNU_BUILD_ID
   note.  Returns -1 in case of malformed data or other errors.  */
extern ssize_t dwelf_elf_gnu_build_id (Elf *elf, const void **build_idp);

You can then use the build_id to lookup the debug information (file).

You can also use libdwfl (part of libdw) to do some of the above
automagically. See for example:

/* Call dwfl_report_module for each file mapped into the address space of PID.
   Returns zero on success, -1 if dwfl_report_module failed,
   or an errno code if opening the proc files failed.  */
extern int dwfl_linux_proc_report (Dwfl *dwfl, pid_t pid);

The Dwfl will then be a representation of the modules (executable and
shared libraries) of that particular process. You can then iterate
through those modules using dwfl_getmodules and get a Dwarf handle
using dwfl_module_getdwarf (or for all with dwfl_getdwarf). libdw will
then try to extract that build-id from each module and try various
lookups to get the (separate on disk) debuginfo.

Hope that helps,

Mark

RE: Working with ELF already loaded in memory

[Sonal Santan]

Hello Mark,

> -----Original Message-----
> From: Mark Wielaard <mark@klomp.org>
> Sent: Thursday, July 8, 2021 8:02 AM
> To: Sonal Santan <sonals@xilinx.com>; elfutils-devel@sourceware.org
> Subject: Re: Working with ELF already loaded in memory
> 
> Hi Sinal,
> 
> On Thu, 2021-07-08 at 05:02 +0000, Sonal Santan via Elfutils-devel
> wrote:
> > Going through the libdw it appears that all APIs require either a file
> > handle or a file name of the ELF object to create a session.
> > Since we do not have access to the ELF file -- but rather the ELF file
> > contents are already loaded in memory -- is there any other mechanism
> > to create a session for extracting DWARF information using libdw?
> 
> Yes, if you just need the information already loaded into memory and you know
> where the library is mapped you can use:
> 
> /* Create descriptor for memory region.  */ extern Elf *elf_memory (char
> *__image, size_t __size);
> 
> You can then use that Elf handle to extract the information that has been
> mapped in. Which often is not the actual debug information though.
> 

Thanks for the pointer. I was looking for APIs inside /usr/include/elfutils directory and
did not realize elfutils also places header files under /usr/include directory. I am working 
with this API now.

> If you have a Elf handle you can use:
> 
> /* Returns the build ID as found in a NT_GNU_BUILD_ID note from either
>    a SHT_NOTE section or from a PT_NOTE segment if the ELF file
>    doesn't contain any section headers.  On success a pointer to the
>    build ID is written to *BUILDID_P, and the positive length of the
>    build ID is returned.  Returns 0 if the ELF lacks a NT_GNU_BUILD_ID
>    note.  Returns -1 in case of malformed data or other errors.  */ extern ssize_t
> dwelf_elf_gnu_build_id (Elf *elf, const void **build_idp);
> 
> You can then use the build_id to lookup the debug information (file).
> 
> You can also use libdwfl (part of libdw) to do some of the above automagically.
> See for example:
> 
> /* Call dwfl_report_module for each file mapped into the address space of PID.
>    Returns zero on success, -1 if dwfl_report_module failed,
>    or an errno code if opening the proc files failed.  */ extern int
> dwfl_linux_proc_report (Dwfl *dwfl, pid_t pid);

Does this also work for ELF files which are loaded into heap as blob -- not really
_mapped_ into the address space? I guess I will have to provide some hints to dwfl
so it can locate the loaded blob when walking through the process map?

Thanks,
-Sonal

> 
> The Dwfl will then be a representation of the modules (executable and shared
> libraries) of that particular process. You can then iterate through those
> modules using dwfl_getmodules and get a Dwarf handle using
> dwfl_module_getdwarf (or for all with dwfl_getdwarf). libdw will then try to
> extract that build-id from each module and try various lookups to get the
> (separate on disk) debuginfo.
> 
> Hope that helps,
> 
> Mark

Re: Working with ELF already loaded in memory

[Mark Wielaard]

Hi Sonal,

On Fri, 2021-07-09 at 05:34 +0000, Sonal Santan wrote:
> 
> Thanks for the pointer. I was looking for APIs inside
> /usr/include/elfutils directory and
> did not realize elfutils also places header files under /usr/include
> directory. I am working 
> with this API now.

Ah, yes, the libelf library is kind of "standard" (it is also available
on BSDs, AIX, Solaris, etc.) so it goes under /usr/include. The libdw
library places its headers under
/usr/include/elfutils/{libdw.h,libdwelf.h,libdwfl.h}.

> > You can also use libdwfl (part of libdw) to do some of the above automagically.
> > See for example:
> > 
> > /* Call dwfl_report_module for each file mapped into the address space of PID.
> >    Returns zero on success, -1 if dwfl_report_module failed,
> >    or an errno code if opening the proc files failed.  */ extern int
> > dwfl_linux_proc_report (Dwfl *dwfl, pid_t pid);
> 
> Does this also work for ELF files which are loaded into heap as blob -- not really
> _mapped_ into the address space? I guess I will have to provide some hints to dwfl
> so it can locate the loaded blob when walking through the process map?

It probably wouldn't work if the the library is not normally mapped
into memory since it relies on the content of /proc/<pid>/map to locate
what is mapped where. You can also create your own and use:

/* Similar, but reads an input stream in the format of Linux
/proc/PID/maps
   files giving module layout, not the file for a live process.  */
extern int dwfl_linux_proc_maps_report (Dwfl *dwfl, FILE *);

But there are also more specific interfaces to map segments to modules
in libdwfl.h. They don't have to be file mapped, but can also come for
memory or parts of a file (see for example the core file mapper). But
these interfaces are a little obscure with minimal documentation.
Please feel free to ask for more background if you get stuck.

Cheers,

Mark