[Bug libc/26179] New: _dl_map_object_deps re-walking transitive deps of already loaded DSOs

[Bug libc/26179] New: _dl_map_object_deps re-walking transitive deps of already loaded DSOs

[andrewjcg at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=26179

            Bug ID: 26179
           Summary: _dl_map_object_deps re-walking transitive deps of
                    already loaded DSOs
           Product: glibc
           Version: 2.33
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: libc
          Assignee: unassigned at sourceware dot org
          Reporter: andrewjcg at gmail dot com
                CC: drepper.fsp at gmail dot com
  Target Milestone: ---

Created attachment 12665
  --> https://sourceware.org/bugzilla/attachment.cgi?id=12665&action=edit
ccccccccccccfrheljkkugnelnetrrjlvlctrflrejtv

In workflows with lots of DSOs which are independently `dlopen`d but which have
several common dependency DSOs, each `_dl_map_object_deps` call will re-walk
these common transitive deps, doing O(N) `strcmp`s in `_dl_map_object` for each
one, which can become costly.

I noticed this while looking into some workflows where Python code `dlopen`s a
lot of DSOs, each of which shares a lot of transitive deps with the others. 
Profiling shows a significant time is spent in `strcmp`. 

It looks like much of this work can be avoided having each call to
`_dl_map_object` also return whether that DSO was already loaded, and terminate
the recursion if so.  An example of an attempt at this is attached which
removed most of this overhead (but I'm not entirely sure it's safe).

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/26179] _dl_map_object_deps re-walking transitive deps of already loaded DSOs

[andrewjcg at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=26179

Andrew <andrewjcg at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #12665|ccccccccccccfrheljkkugnelne |fix_dl_map_object_deps
        description|trrjlvlctrflrejtv           |

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/26179] _dl_map_object_deps re-walking transitive deps of already loaded DSOs

[P at draigBrady dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=26179

Pádraig Brady <P at draigBrady dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |P at draigBrady dot com

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/26179] _dl_map_object_deps re-walking transitive deps of already loaded DSOs

[carlos at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=26179

Carlos O'Donell <carlos at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |carlos at redhat dot com
         Resolution|---                         |DUPLICATE
             Status|UNCONFIRMED                 |RESOLVED

--- Comment #1 from Carlos O'Donell <carlos at redhat dot com> ---
(In reply to Andrew from comment #0)
> Created attachment 12665 [details]
> fix_dl_map_object_deps
> 
> In workflows with lots of DSOs which are independently `dlopen`d but which
> have several common dependency DSOs, each `_dl_map_object_deps` call will
> re-walk these common transitive deps, doing O(N) `strcmp`s in
> `_dl_map_object` for each one, which can become costly.
> 
> I noticed this while looking into some workflows where Python code `dlopen`s
> a lot of DSOs, each of which shares a lot of transitive deps with the
> others.  Profiling shows a significant time is spent in `strcmp`. 
> 
> It looks like much of this work can be avoided having each call to
> `_dl_map_object` also return whether that DSO was already loaded, and
> terminate the recursion if so.  An example of an attempt at this is attached
> which removed most of this overhead (but I'm not entirely sure it's safe).

We are already working on resolving this as part of bug 17645.

We are only v2.1 of the fix that switches to a full DFS RPO sort with "use"
tracking per link map to avoid visiting nodes again, and terminating the sort
early if the list was already sorted.

I'm marking this as a duplicate of bug 17645.

*** This bug has been marked as a duplicate of bug 17645 ***

-- 
You are receiving this mail because:
You are on the CC list for the bug.