dlltool import library build optimization

dlltool import library build optimization

[Mumit Khan]

When building large DLLs, especially C++ ones, dlltool can take
inordinately long time to and take the file system to its knees
while it's creating those thousands of small files, one per
symbol. For those who actually have to use a native windows
machine, this can make it almost impossible to use dlltool to
create import libraries with say 10k symbols thanks to all the
files it needs to create first.

I know zilch about bfd backend, so perhaps someone can help me
here. Is there anyway we can bunch up say 100 symbols, create
the archive, and move to the next? The current scheme uses
the following technique:

  
  open a new bfd for writing (output_bfd)
  
  make head bfd
  make tail bfd

  for each exports
    make a new bfd and add it to the chain
  done

  Reorganize the chain of bfd's

  set the output_bfd archive head to the created chain

  close output_bfd


Is there a way for creating the output bfd a few at a time? Would
bfd_openw kill the current contents and start from scratch?

Regards,
Mumit

Re: dlltool import library build optimization

[Ian Lance Taylor]

   Date: Wed, 22 Dec 1999 13:27:33 -0600 (CST)
   From: Mumit Khan <khan@NanoTech.Wisc.EDU>

   When building large DLLs, especially C++ ones, dlltool can take
   inordinately long time to and take the file system to its knees
   while it's creating those thousands of small files, one per
   symbol. For those who actually have to use a native windows
   machine, this can make it almost impossible to use dlltool to
   create import libraries with say 10k symbols thanks to all the
   files it needs to create first.

dlltool doesn't have to create any files at all.  That's a horrible
hack.  It should just build the BFD's in memory.

Ian

Re: dlltool import library build optimization

[Mumit Khan]

Ian Lance Taylor <ian@zembu.com> writes:
> 
> dlltool doesn't have to create any files at all.  That's a horrible
> hack.  It should just build the BFD's in memory.
> 

I agree. DJ has already done it in pe-dll, and it's a matter of reusing
that in dlltool. I'll give it a shot when I get back in a week or so.

Regards,
Mumit

Re: dlltool import library build optimization

[Mumit Khan]

Ian Lance Taylor <ian@zembu.com> writes:
> 
> dlltool doesn't have to create any files at all.  That's a horrible
> hack.  It should just build the BFD's in memory.
> 

I'm about to submit patches for dlltool that will build the BFD's in
memory. However, I have no clue how to do this for non-i386pe targets,
so I'll just submit what I have and hope that Nick and others can fill
in the rest. My changes are essentially DJ's pe-dll retrofitted into
dlltool, with a few tweaks. Needs testing and cleanup.

Ideas, Nick?

Regards,
Mumit

Re: dlltool import library build optimization

[Mumit Khan]

Ian Lance Taylor <ian@zembu.com> writes:
> 
> dlltool doesn't have to create any files at all.  That's a horrible
> hack.  It should just build the BFD's in memory.
> 

I'm about to submit patches for dlltool that will build the BFD's in
memory. However, I have no clue how to do this for non-i386pe targets,
so I'll just submit what I have and hope that Nick and others can fill
in the rest. My changes are essentially DJ's pe-dll retrofitted into
dlltool, with a few tweaks. Needs testing and cleanup.

Ideas, Nick?

Regards,
Mumit

Re: dlltool import library build optimization

[Nick Clifton]

Hi Mumit,

: > dlltool doesn't have to create any files at all.  That's a horrible
: > hack.  It should just build the BFD's in memory.
: > 
: 
: I'm about to submit patches for dlltool that will build the BFD's in
: memory. However, I have no clue how to do this for non-i386pe targets,
: so I'll just submit what I have and hope that Nick and others can fill
: in the rest. My changes are essentially DJ's pe-dll retrofitted into
: dlltool, with a few tweaks. Needs testing and cleanup.
: 
: Ideas, Nick?

Non i386-pe DLLs are pretty much the same as i386-pe ones.  I will be
happy to check your patches against eg arm and mcore and fix any
problems that may arise.

Cheers
	Nick

Re: dlltool import library build optimization

[Mumit Khan]

Nick Clifton <nickc@cygnus.com> writes:
> 
> Non i386-pe DLLs are pretty much the same as i386-pe ones.  I will be
> happy to check your patches against eg arm and mcore and fix any
> problems that may arise.
> 

Thanks. I'll dig out, clean, test and submit later this week. In my
initial tests, I could build a large implib in < 1/10th the time 
needed normally on Linux; it's almost impossible to build the particular 
implib on a FAT FS since it pretty much chokes the OS with about ~12k
tiny files.

Regards,
Mumit

Re: dlltool import library build optimization

[Mumit Khan]

Nick Clifton <nickc@cygnus.com> writes:
> 
> Non i386-pe DLLs are pretty much the same as i386-pe ones.  I will be
> happy to check your patches against eg arm and mcore and fix any
> problems that may arise.
> 

Thanks. I'll dig out, clean, test and submit later this week. In my
initial tests, I could build a large implib in < 1/10th the time 
needed normally on Linux; it's almost impossible to build the particular 
implib on a FAT FS since it pretty much chokes the OS with about ~12k
tiny files.

Regards,
Mumit

Re: dlltool import library build optimization

[Nick Clifton]

Hi Mumit,

: > dlltool doesn't have to create any files at all.  That's a horrible
: > hack.  It should just build the BFD's in memory.
: > 
: 
: I'm about to submit patches for dlltool that will build the BFD's in
: memory. However, I have no clue how to do this for non-i386pe targets,
: so I'll just submit what I have and hope that Nick and others can fill
: in the rest. My changes are essentially DJ's pe-dll retrofitted into
: dlltool, with a few tweaks. Needs testing and cleanup.
: 
: Ideas, Nick?

Non i386-pe DLLs are pretty much the same as i386-pe ones.  I will be
happy to check your patches against eg arm and mcore and fix any
problems that may arise.

Cheers
	Nick