[Bug gdb/12332] New: Provide efficient source lookup for use by IDE

[Bug gdb/12332] New: Provide efficient source lookup for use by IDE

[oss.sprigogin at gmail dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=12332

           Summary: Provide efficient source lookup for use by IDE
           Product: gdb
           Version: 7.2
            Status: NEW
          Severity: enhancement
          Priority: P2
         Component: gdb
        AssignedTo: unassigned@sourceware.org
        ReportedBy: oss.sprigogin@gmail.com


One feature that would be very useful for using gdb as a debugging backend for
Eclipse CDT is a new type of source lookup. Consider a scenario when a source
file is represented in debug information by its relative path, lets say
relative to directory D1. Currently it is possible to insert a line breakpoint
by either using a relative path exactly as it's represented in the debug
information, or by using an absolute path to the file and supplying D1 as part
of environment-directory. In the latter case gdb checks the file system to
verify that the source file exists. In a complex project source files my be
compiled relative to many different directories and keeping track of which
files were compiled relative to which directories becomes challenging. When
environment-directory contains a large number of directories, breakpoint
insertion by absolute path involves a lot of disk IO and becomes very slow,
especially when the underlying file system is slow.

The situation could be improved if gdb supported a different type of absolute
path source lookup. With this type of lookup an absolute path to a file is
effectively treated as a set of relative paths, of which gdb selects the
longest one matching the path in the debug information. In this mode an
absolute path received by GDB should be first matched against the debug
information as is, then, if there is no match, matched without the first
segment, then without the first two segments, and so on. Source lookup in this
mode should not depend on environment-directory and should not involve any IO.

Please see the Eclipse CDT bug
https://bugs.eclipse.org/bugs/show_bug.cgi?id=225805 for a longer discussion on
source lookup issues.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug gdb/12332] Provide efficient source lookup for use by IDE

[jan.kratochvil at redhat dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=12332

Jan Kratochvil <jan.kratochvil at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jan.kratochvil at redhat
                   |                            |dot com

--- Comment #1 from Jan Kratochvil <jan.kratochvil at redhat dot com> 2011-02-25 22:56:20 UTC ---
I (briefly) read through the Eclipse Bug but I do not see any indication the
filesystem lookup would be slow.
In fact since Linux kernel dcache (linux-2.1.x or so) any pathname lookups are
blazingly fast:
find $PWD -type f|tr -cd /|wc -c
 = 279751 = directory components for lookup
find $PWD -type f|wc -l
 = 26239 = files = number of breakpoints
time sh -c 'find $PWD -type f|xargs /bin/ls|wc -l'
real 0m0.903s = the overhead of any lookups is really negligible
[ kernel-debug-2.6.35.11-83.fc14.x86_64, i7-920, ext3 fs, second run ]

If you mean slowness on the first run (and on the second run it is already
fast) then it is by the disk access.  But not looking up the leading
directories would not help it as those are already in dcache even in such case.

I would rather bet the slowness is in GDB itself, it has problems handling
1000+ breakpoints even on high-end hardware.  I rather did not expect anyone
would use so many breakpoints.

I already did one breakpoints performance improvement which made the 1000
breakpoints case acceptable again (although it is still far from perfect):
http://sourceware.org/ml/gdb-patches/2009-10/msg00289.html

I would like to see some oprofile or gprof results first before deciding which
part to fix.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug gdb/12332] Provide efficient source lookup for use by IDE

[sprigogin at google dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=12332

Sergey Prigogin <sprigogin at google dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |sprigogin at google dot com

--- Comment #2 from Sergey Prigogin <sprigogin at google dot com> 2011-02-28 04:23:49 UTC ---
(In reply to comment #1)
First of all file systems are not created equal. Here is directory listing
timing on NFS:
$ find $PWD -type f|tr -cd /|wc -c
755813
$ find $PWD -type f|wc -l
81702
$ time sh -c 'find $PWD -type f|xargs /bin/ls|wc -l'
81702
real    0m49.537s

50 seconds is not negligible. In my GDB experiment some source directories were
on an even slower file system that represented a view of a version control
system. GDB was over 5 minutes to set just a few breakpoints. Almost all time
was spent in file system IO.

When instead of passing environment-directory and absolute paths of the files
containing the breakpoints, the breakpoints were set up using relative paths
matching the ones in debug info in the binary, there was no IO and the
breakpoints were set instantly.

> I would rather bet the slowness is in GDB itself, it has problems handling
> 1000+ breakpoints even on high-end hardware.  I rather did not expect anyone
> would use so many breakpoints.

The slowness manifests itself with just a few breakpoints. I don't remember how
many breakpoints I had in my experiment, but it was definitely less than 5.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug gdb/12332] Provide efficient source lookup for use by IDE

[marc.khouzam at ericsson dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=12332

Marc Khouzam <marc.khouzam at ericsson dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |marc.khouzam at ericsson
                   |                            |dot com

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug gdb/12332] Provide efficient source lookup for use by IDE

[marc.khouzam at ericsson dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=12332

--- Comment #3 from Marc Khouzam <marc.khouzam at ericsson dot com> 2011-03-11 21:07:36 UTC ---
(In reply to comment #2)
> The slowness manifests itself with just a few breakpoints. I don't remember how
> many breakpoints I had in my experiment, but it was definitely less than 5.

That is right.  The slowness is when there are many paths given to
-environment-directory.  I was told that in a particular case, there were 8800
paths!  With this, setting a single breakpoint takes so much time that the user
things Eclipse/GDB is hung.

My guess is that GDB has to run through all the directories of
-environment-directory and look for the file where the breakpoint has been set.
 That probably takes a lot of time.

Note that this happens even if the breakpoint is set using a absolute path.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug gdb/12332] Provide efficient source lookup for use by IDE

[dje at google dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=12332

dje at google dot com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |dje at google dot com,
                   |                            |tromey at redhat dot com

--- Comment #4 from dje at google dot com 2012-04-16 17:49:24 UTC ---
With the new trailing path lookup, can we mark this as closed?
I'm not sure if the feature request described here sufficiently matches this
patch:

2012-01-16  Tom Tromey  <tromey@redhat.com>

        * NEWS: Add item.
        * symtab.h (compare_filenames_for_search): Declare.
        * symtab.c (compare_filenames_for_search): New function.
        (iterate_over_some_symtabs): Use it.
        * symfile.h (struct quick_symbol_functions)
        <map_symtabs_matching_filename>: Change spec.
        * psymtab.c (partial_map_symtabs_matching_filename): Use
        compare_filenames_for_search.  Update for new spec.
        * dwarf2read.c (dw2_map_symtabs_matching_filename): Use
        compare_filenames_for_search.  Update for new spec.
        * breakpoint.c (clear_command): Use compare_filenames_for_search.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.