objdump -d --line-numbers performance question

objdump -d --line-numbers performance question

[Emile Snyder]

[-- Attachment #1: Type: text/plain, Size: 1581 bytes --]

Hello all,

In one project I work on there's some scripts which we use to do
debugging that use objdump to get at debugging symbol info to decode
backtrace dumps from the program.  The application binary is ~50MB when
built with debugging info, and when we moved from developing on RedHat
7.3 to RH 9 or later, the objdump time went through the roof.

I grabbed a binutils-2.14 tarball, and spent a little time digging. 
oprofile reports that some huge proportion of the time is spent in the
bfd/elf.c:elf_find_function() call, which is being called (with some
intermediates) by binutils/objdump.c:show_line().

Looking at elf_find_function() it jumps out that it's doing a linear
search through the entire set of symbols every time it's called.  But
show_line() has access to find_symbol_for_address which does a binary
search on a sorted set of symbols.

So.  The attached patch uses find_symbol_for_address() in show_line(),
and then passes down a fake asymbol** list consisting of just the found
symbol and a null end marker.  So far it always produces identical
'objdump -d --line-numbers' output for me as the stock objdump, but my
run time on one representative file has gone from ~2 hours to ~4
minutes.

Does this look right?  If not, can anyone give me some pointers on the
right way to try to optimize things?

Thanks,
-emile


+----------------------------------------------------------------------
this wine is particularly heavy, and is mostly recommended for
hand-to-hand combat. -- Eric Idle 
+----------------------------------------------------------------------

[-- Attachment #2: binutils-2.14.objdumppatch --]
[-- Type: text/x-patch, Size: 1027 bytes --]

diff -Naur binutils-2.14/binutils/objdump.c binutils-2.14-modified/binutils/objdump.c
--- binutils-2.14/binutils/objdump.c	2003-03-31 16:32:47.000000000 -0800
+++ binutils-2.14-modified/binutils/objdump.c	2005-02-25 12:02:43.617595584 -0800
@@ -980,13 +980,22 @@
   const char *filename;
   const char *functionname;
   unsigned int line;
+  asymbol *foundsym;
+  asymbol *mocksyms[2];
+  long symplace;
 
   if (! with_line_numbers && ! with_source_code)
     return;
 
-  if (! bfd_find_nearest_line (abfd, section, syms, addr_offset, &filename,
-			       &functionname, &line))
-    return;
+  foundsym = find_symbol_for_address(abfd, section, 
+                                     section->vma + addr_offset, 
+                                     FALSE, &symplace);
+  mocksyms[0] = foundsym;
+  mocksyms[1] = NULL;
+
+  if (! bfd_find_nearest_line (abfd, section, mocksyms, addr_offset, &filename,
+  			       &functionname, &line))
+      return;
 
   if (filename != NULL && *filename == '\0')
     filename = NULL;

Re: objdump -d --line-numbers performance question

[Nick Clifton]

[-- Attachment #1: Type: text/plain, Size: 1458 bytes --]

Hi Emile,

> Looking at elf_find_function() it jumps out that it's doing a linear
> search through the entire set of symbols every time it's called.  But
> show_line() has access to find_symbol_for_address which does a binary
> search on a sorted set of symbols.
> 
> So.  The attached patch uses find_symbol_for_address() in show_line(),
> and then passes down a fake asymbol** list consisting of just the found
> symbol and a null end marker.  So far it always produces identical
> 'objdump -d --line-numbers' output for me as the stock objdump, but my
> run time on one representative file has gone from ~2 hours to ~4
> minutes.
> 
> Does this look right?  If not, can anyone give me some pointers on the
> right way to try to optimize things?

Actually this looks like a very good idea.  I only have two improvements 
to suggest:

   1. Be paranoid.  If the call to bfd_find_nearest_line() fails to find 
the desired information with the abbreviated symbol table then it should 
be re-called with the full symbol table.

   2. Use the latest sources.  The 2.15 release is not going to have any 
new features added to it, so it is best to produce a patch against the 
latest sources in the binutils CVS repository.

To that end, please could you try out the attached patch which is meant 
to address both of these points.  If you can confirm that it does still 
work (and reduce the search times) then I will commit it to the sources.

Cheers
   Nick




[-- Attachment #2: objdump.c.patch --]
[-- Type: text/plain, Size: 2837 bytes --]

Index: binutils/objdump.c
===================================================================
RCS file: /cvs/src/src/binutils/objdump.c,v
retrieving revision 1.100
diff -c -3 -p -r1.100 objdump.c
*** binutils/objdump.c	23 Feb 2005 12:25:57 -0000	1.100
--- binutils/objdump.c	1 Mar 2005 11:05:40 -0000
*************** skip_to_line (struct print_file_list *p,
*** 1026,1042 ****
     listing.  */
  
  static void
! show_line (bfd *abfd, asection *section, bfd_vma addr_offset)
  {
!   const char *filename;
!   const char *functionname;
!   unsigned int line;
  
    if (! with_line_numbers && ! with_source_code)
      return;
  
!   if (! bfd_find_nearest_line (abfd, section, syms, addr_offset, &filename,
! 			       &functionname, &line))
      return;
  
    if (filename != NULL && *filename == '\0')
--- 1026,1065 ----
     listing.  */
  
  static void
! show_line (bfd *abfd, asection *section, bfd_vma addr_offset,
! 	   struct disassemble_info * info)
  {
!   const char *filename = NULL;
!   const char *functionname = NULL;
!   unsigned int line = 0;
!   asymbol *foundsym;
!   long symplace;
  
    if (! with_line_numbers && ! with_source_code)
      return;
  
!   /* bfd_find_nearest_line will perform a linear search of the symbol table
!      for a matching address, but we already have a sorted symbol table and
!      a binary search function, so try creating a fake symbol table first.  */
!   foundsym = find_symbol_for_address (section->vma + addr_offset, 
! 				      info, & symplace);
!   if (foundsym)
!     {
!       asymbol *mocksyms[2];
! 
!       mocksyms[0] = foundsym;
!       mocksyms[1] = NULL;
! 
!       /* We do not need to check the return code here since if it
! 	 is zero then functionname will not have been initialised.  */
!       (void) bfd_find_nearest_line (abfd, section, mocksyms, addr_offset,
! 				    & filename, & functionname, & line);
!     }
! 
!   if ((functionname == NULL || filename == NULL)
!       /* Ourt shortcut failed.  Try a full search.  */
!       && (! bfd_find_nearest_line (abfd, section, syms, addr_offset,
! 				   & filename, & functionname, & line)))
      return;
  
    if (filename != NULL && *filename == '\0')
*************** disassemble_bytes (struct disassemble_in
*** 1320,1326 ****
  	    /* The line number tables will refer to unadjusted
  	       section VMAs, so we must undo any VMA modifications
  	       when calling show_line.  */
! 	    show_line (aux->abfd, section, addr_offset - adjust_section_vma);
  
  	  if (! prefix_addresses)
  	    {
--- 1343,1349 ----
  	    /* The line number tables will refer to unadjusted
  	       section VMAs, so we must undo any VMA modifications
  	       when calling show_line.  */
! 	    show_line (aux->abfd, section, addr_offset - adjust_section_vma, info);
  
  	  if (! prefix_addresses)
  	    {

Re: objdump -d --line-numbers performance question

[Emile Snyder]

Thanks for the feedback.  I couldn't get cvs to let me log in for some
reason, so I tried this with binutils-050228.tar.bz2  Unfortunately it
doesn't work there (although the similar idea -- defaulting to the old
linear search of the whole list -- does work on 2.14).

I'll keep digging and see if I can figure out what's going in.

thanks,
-emile

On Tue, 2005-03-01 at 03:21, Nick Clifton wrote:
> > Does this look right?  If not, can anyone give me some pointers on the
> > right way to try to optimize things?
> 
> Actually this looks like a very good idea.  I only have two improvements 
> to suggest:
> 
>    1. Be paranoid.  If the call to bfd_find_nearest_line() fails to find 
> the desired information with the abbreviated symbol table then it should 
> be re-called with the full symbol table.
> 
>    2. Use the latest sources.  The 2.15 release is not going to have any 
> new features added to it, so it is best to produce a patch against the 
> latest sources in the binutils CVS repository.
> 
> To that end, please could you try out the attached patch which is meant 
> to address both of these points.  If you can confirm that it does still 
> work (and reduce the search times) then I will commit it to the sources.
> 
> Cheers
>    Nick
> 
> 
> 
> 
> ______________________________________________________________________
> Index: binutils/objdump.c
> ===================================================================
> RCS file: /cvs/src/src/binutils/objdump.c,v
> retrieving revision 1.100
> diff -c -3 -p -r1.100 objdump.c
> *** binutils/objdump.c	23 Feb 2005 12:25:57 -0000	1.100
> --- binutils/objdump.c	1 Mar 2005 11:05:40 -0000
> *************** skip_to_line (struct print_file_list *p,
> *** 1026,1042 ****
>      listing.  */
>   
>   static void
> ! show_line (bfd *abfd, asection *section, bfd_vma addr_offset)
>   {
> !   const char *filename;
> !   const char *functionname;
> !   unsigned int line;
>   
>     if (! with_line_numbers && ! with_source_code)
>       return;
>   
> !   if (! bfd_find_nearest_line (abfd, section, syms, addr_offset, &filename,
> ! 			       &functionname, &line))
>       return;
>   
>     if (filename != NULL && *filename == '\0')
> --- 1026,1065 ----
>      listing.  */
>   
>   static void
> ! show_line (bfd *abfd, asection *section, bfd_vma addr_offset,
> ! 	   struct disassemble_info * info)
>   {
> !   const char *filename = NULL;
> !   const char *functionname = NULL;
> !   unsigned int line = 0;
> !   asymbol *foundsym;
> !   long symplace;
>   
>     if (! with_line_numbers && ! with_source_code)
>       return;
>   
> !   /* bfd_find_nearest_line will perform a linear search of the symbol table
> !      for a matching address, but we already have a sorted symbol table and
> !      a binary search function, so try creating a fake symbol table first.  */
> !   foundsym = find_symbol_for_address (section->vma + addr_offset, 
> ! 				      info, & symplace);
> !   if (foundsym)
> !     {
> !       asymbol *mocksyms[2];
> ! 
> !       mocksyms[0] = foundsym;
> !       mocksyms[1] = NULL;
> ! 
> !       /* We do not need to check the return code here since if it
> ! 	 is zero then functionname will not have been initialised.  */
> !       (void) bfd_find_nearest_line (abfd, section, mocksyms, addr_offset,
> ! 				    & filename, & functionname, & line);
> !     }
> ! 
> !   if ((functionname == NULL || filename == NULL)
> !       /* Ourt shortcut failed.  Try a full search.  */
> !       && (! bfd_find_nearest_line (abfd, section, syms, addr_offset,
> ! 				   & filename, & functionname, & line)))
>       return;
>   
>     if (filename != NULL && *filename == '\0')
> *************** disassemble_bytes (struct disassemble_in
> *** 1320,1326 ****
>   	    /* The line number tables will refer to unadjusted
>   	       section VMAs, so we must undo any VMA modifications
>   	       when calling show_line.  */
> ! 	    show_line (aux->abfd, section, addr_offset - adjust_section_vma);
>   
>   	  if (! prefix_addresses)
>   	    {
> --- 1343,1349 ----
>   	    /* The line number tables will refer to unadjusted
>   	       section VMAs, so we must undo any VMA modifications
>   	       when calling show_line.  */
> ! 	    show_line (aux->abfd, section, addr_offset - adjust_section_vma, info);
>   
>   	  if (! prefix_addresses)
>   	    {

+----------------------------------------------------------------------
On two occasions I have been asked [by members of Parliament], 'Pray,
Mr. Babbage, if you put into the machine wrong figures, will the right
answers come out?' I am not able rightly to apprehend the kind of
confusion of ideas that could provoke such a question. -- Charles
Babbage 
+----------------------------------------------------------------------