Re: Long linking time

Re: Long linking time

[Nicolas Savoire]

I confirm that. 
I saw no improvement when using  binutils-2.17.50.0.3 from  
ftp.kernel.org/pub/linux/devel/binutils/ . 
Though I found a patch 
(http://sourceware.org/ml/binutils/2006-04/msg00329.html) which reduces 
linking time from 6 minutes to 20 seconds!

nicolas

On Monday 28 August 2006 15:02, Talbot, George wrote:
> Do you mean that the patch is already in the source distribution for
> binutils-2.17.50.0.3?
>
> The caching patch is not in binutils-2.17.50.0.3, from what I can tell.
> I don't mean to be dense, but I did have to put it in there myself, and
> it wasn't in the README at the top-level of the source tree.
>
> Let me know what I'm missing here.
>
> --
>
> George T. Talbot
> <gtalbot@locuspharma.com>
>
>
> -----Original Message-----
> From: H. J. Lu [mailto:hjl@lucon.org]
> Sent: Friday, August 25, 2006 10:44 AM
> To: Talbot, George
> Cc: Alan Modra; nicos@maunakeatech.com; binutils@sourceware.org
> Subject: Re: Re: Long linking time
>
> On Fri, Aug 25, 2006 at 08:17:37AM -0400, Talbot, George wrote:
> > I tried the latest version.  It didn't help that much.  I then
> > hand-applied the patch that caches the value of the kept symbols, and
> > that helped a dramatic amount.  I would strongly argue for the
>
> inclusion
>
> If you follow README for the latest Linux binutils, that patch should
> be applied.
>
>
> H.J.

Re: Long linking time

[Alan Modra]

On Thu, Aug 03, 2006 at 11:06:18AM +0200, Nicolas Savoire wrote:
> I don't know if this is a gcc or ld issue, but since I updated gcc to 4.1.1 
> version, the linking times of the project I work on have skyrocketted. One 
> particular shared library, which used to link in less than a second now takes  
> 6 minutes to link!!!! This library instantiates a lot of templates (use of 
> itk image library and boost).
> After further investigation, it appears that disabling debug info generation 
> during compilation or asking ld to strip debug info (-S flag) leads to normal 
> linking time (~1s). 
> I use binutils 2.17, but versions 2.15 and 2.16 exhibit the same behaviour.
> A quick profiling show that almost all the execution time is spent in qsort, 
> the call stack at that point looks like:
> #0  elf_sort_elf_symbol (arg1=0x99e49d0, arg2=0x99eb618) at elf.c:8524
> #1  0xb7eb5ddb in bsearch () from /lib/libc.so.6
> #2  0xb7eb5f3a in qsort () from /lib/libc.so.6
> #3  0x0809fbfc in bfd_elf_match_symbols_in_sections (sec1=0x81889e4, 
> sec2=0x83aa974) at elf.c:8628
> #4  0x080ac0a1 in match_group_member (sec=0x83aa974, group=0x817ad2c) at 
> elflink.c:6778
> #5  0x080ac113 in _bfd_elf_check_kept_section (sec=0x83aa974) at 
> elflink.c:6802
> #6  0x080acca0 in elf_link_input_bfd (finfo=0xbffe0684, input_bfd=0x81528c0) 
> at elflink.c:7155
> #7  0x080af013 in bfd_elf_final_link (abfd=0x8133128, info=0x8126920) at 
> elflink.c:8266
> #8  0x0805f39f in ldwrite () at ldwrite.c:557
> #9  0x0805c6ec in main (argc=82, argv=0xbffe0914) at .././ld/ldmain.c:468
> 
> I had already posted a message about this issue, but it remains without 
> answer. I hope this more descriptive version will help.

This is a ld problem.  We are paying a high price for some checks that
HJ added to validate linkonce/comdat sections.  I approved at least some
of the patches so I share some blame too.  HJ has posted some patches to
improve time spent in _bfd_elf_check_kept_section, but they haven't been
applied yet to FSF binutils.  I'm not too keen on approving them because
I think they are just bandaids, and the real fix is to revert the symbol
matching code.

You might like to try compiling HJ's latest release from
ftp.kernel.org/pub/linux/devel/binutils/ which I assume has HJ's
bandaids applied.  Please report the link times vs mainline binutils
back here, with and without -g.

-- 
Alan Modra
IBM OzLabs - Linux Technology Centre

Long linking time

[Nicolas Savoire]

I don't know if this is a gcc or ld issue, but since I updated gcc to 4.1.1 
version, the linking times of the project I work on have skyrocketted. One 
particular shared library, which used to link in less than a second now takes  
6 minutes to link!!!! This library instantiates a lot of templates (use of 
itk image library and boost).
After further investigation, it appears that disabling debug info generation 
during compilation or asking ld to strip debug info (-S flag) leads to normal 
linking time (~1s). 
I use binutils 2.17, but versions 2.15 and 2.16 exhibit the same behaviour.
A quick profiling show that almost all the execution time is spent in qsort, 
the call stack at that point looks like:
#0  elf_sort_elf_symbol (arg1=0x99e49d0, arg2=0x99eb618) at elf.c:8524
#1  0xb7eb5ddb in bsearch () from /lib/libc.so.6
#2  0xb7eb5f3a in qsort () from /lib/libc.so.6
#3  0x0809fbfc in bfd_elf_match_symbols_in_sections (sec1=0x81889e4, 
sec2=0x83aa974) at elf.c:8628
#4  0x080ac0a1 in match_group_member (sec=0x83aa974, group=0x817ad2c) at 
elflink.c:6778
#5  0x080ac113 in _bfd_elf_check_kept_section (sec=0x83aa974) at 
elflink.c:6802
#6  0x080acca0 in elf_link_input_bfd (finfo=0xbffe0684, input_bfd=0x81528c0) 
at elflink.c:7155
#7  0x080af013 in bfd_elf_final_link (abfd=0x8133128, info=0x8126920) at 
elflink.c:8266
#8  0x0805f39f in ldwrite () at ldwrite.c:557
#9  0x0805c6ec in main (argc=82, argv=0xbffe0914) at .././ld/ldmain.c:468

I had already posted a message about this issue, but it remains without 
answer. I hope this more descriptive version will help.

Regards,
nicolas