[avail for test] libtool-devel-20030121-1

[avail for test] libtool-devel-20030121-1

[Charles Wilson]

I've updated libtool-devel to the 20030121 CVS, plus added a major 
change of my own (inspired by Earnie Boyd and others on the libtool 
mailing list) that "fixes" the 'relink exe's over and over and over' 
problem on both mingw and cygwin.

[Skip to the last two paragraphs for the real important part]

A (not-so-short) description of the problem and the solution approach 
taken: if you have a package that builds a shared library (.dll) AND an 
exe which relies on that library, then libtool puts the actual 
executable into a .lib/ subdirectory under your current build directory, 
and NOT in the build directory itself.

This is also done on "real" unix platforms.  The reason is that 
uninstalled shared libraries cannot easily be found by the runtime 
loader on most platforms, unix and cygwin/windows included.

So, in order to run the uninstalled executable, you must first set the 
environment appropriately so that the shared library (.dll) will be 
found.  This means setting LD_LIBRARY_PATH on some unices, or setting 
PATH on cygwin so that the Windows Runtime Loader will find the .dll.

libtool does this by creating a shell script in the actual build 
directory.  The shell script sets the variables and then exec's 
./lib/my-real-executable.

If your application is "foo", this works fine on unix.  The makefile 
wants to see 'foo' -- and it does; only the 'foo' that make sees is 
actually a shell script:

   <builddir>/foo       : a shell script
   <builddir>/.lib/foo  : the real executable

And it only gets built once.  However, on cygwin/windows, you have
   <builddir>/foo           : a shell script
   <builddir>/.lib/foo.exe  : the real executable
and the makefile *wants* 'foo.exe' -- but only sees 'foo'.  Therefore, 
make assumes that the executable hasn't been created, and builds it 
again.  EVERY time you run 'make <anything>'.  Sometimes *multiple* 
times if there are cross-dependencies.

There were several solutions:
   1) Teach 'make' to only want 'foo' instead of 'foo.exe'.  There are 
problems here -- this requires mucking with automake, which has the 
potential to break non-libtool builds if not done carefully.
   2) Name the script 'foo.exe' -- bad idea.  cygwin and mingw take one 
look at the extension, and simply exec it directly *without* parsing the 
first line for an interpreter.  Doing this would actually require mods 
to the cygwin and MSYS kernel -- and would slow cygwin down on *every* 
executable invokation.

<note: the following is true only for cygwin/mingw builds>

   3) What I did:  create a binary wrapper -- an actual executable -- 
named 'foo.exe' in the main build directory.  It is NOT the real 
foo.exe.  It simply exec's the shell script, which in turn sets up the 
environment and exec's the real .lib/foo.exe.  Eventually, the 'set up 
the environment' part could be moved into the binary wrapper itself, at 
least on cygwin/mingw -- but there are problems with that; libtool 
itself *sources* and parses the shell wrapper -- it can't do that with a 
binary wrapper.  So that's for later.  This works now, even if it is a 
bit kludgey.

Unfortunately, it might lead to a FAQ: the shell wrapper contains a 
banner at the top that says "This is not the real foo.exe.  It is a 
wrapper" blah blah blah.  Plus, it's named 'foo' instead of 'foo.exe' -- 
a tipoff for us windows denizens.  Now, you have:

<builddir>/foo           : shell wrapper
<builddir>/foo.exe       : binary wrapper
<builddir>/.lib/foo.exe  : the real executable

There's no easy way to 'label' <builddir>/foo.exe as a wrapper -- so 
some folks may be tempted to install it, with disappointing results. 
Perhaps if the binary wrapper doesn't find .libs/foo.exe, it can report 
an error of the form:
   "You must have installed foo.exe by hand, because I am not the
real foo.exe.  I am simply a wrapper used during the build process.  Go 
back to your build directory, and look in the .lib subdirectory for the 
real foo.exe.  And next time, use 'make install' -- don't try to install 
libtoolized packages by hand."

In any case, my patch has made it into libtool CVS as of 2003-01-30 
(without the friendly error message), but I haven't updated this test 
distribution to that level; I'm simply releasing my patched version of 
2002-01-21.

So, test and enjoy; I will probably make this the official cygwin 
libtool-devel very soon.  It has been up on my off-site cygutils testing 
area for almost two weeks, and now that Robert Boehne has put this 
iteration of my patch into CVS I wanted to release it to a wider testing 
audience.

Unless there are significant bugs reported, I do not expect to update 
our libtool-devel again until libtool-1.5 is released, which may be in a 
month -- or it may be in June.  For now, I am happy with the status of 
libtool-devel (the friendly error message is just a simple cosmetic fix; 
I'm too swamped right now to worry about cosmetics; functionality only.)

[IMPORTANT]
Translation: this will be the basic behavior of official libtool-1.5 on 
cygwin.  If you don't like it, now is the time to yell.  So TEST, you 
huskies, TEST!

Anyway, you know the drill: setup.exe, choose 'Experimental', etc.

--Chuck


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: [avail for test] libtool-devel-20030121-1

[Ralf Habacker]

Hi Chuck,
>    3) What I did:  create a binary wrapper -- an actual executable --
> named 'foo.exe' in the main build directory.  It is NOT the real
> foo.exe.  It simply exec's the shell script, which in turn sets up the
> environment and exec's the real .lib/foo.exe.  Eventually, the 'set up
> the environment' part could be moved into the binary wrapper itself, at
> least on cygwin/mingw -- but there are problems with that; libtool
> itself *sources* and parses the shell wrapper -- it can't do that with a
> binary wrapper.  So that's for later.  This works now, even if it is a
> bit kludgey.
>
> Unfortunately, it might lead to a FAQ: the shell wrapper contains a
> banner at the top that says "This is not the real foo.exe.  It is a
> wrapper" blah blah blah.  Plus, it's named 'foo' instead of 'foo.exe' --
> a tipoff for us windows denizens.  Now, you have:
>
> <builddir>/foo           : shell wrapper
> <builddir>/foo.exe       : binary wrapper
> <builddir>/.lib/foo.exe  : the real executable
>
i've played a little with this stuff and have seen, that at least for cygwin
there is an easier way to deal with this. Create a simple link from 'foo' to
'foo.exe' and Makes need are fullfilled.

See ltmain.sh:
<snip>
    # The program doesn't exist.
    \$echo \"\$0: error: \$progdir/\$program does not exist\" 1>&2
    \$echo \"This script is just a wrapper for \$program.\" 1>&2
    echo \"See the $PACKAGE documentation for more information.\" 1>&2
    exit 1
  fi
fi\
"
	chmod +x $output
+	ln -s $output $output$exeext
	fi

The single added line fix this.

Ralf


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Charles Wilson]

Ralf Habacker wrote:

> i've played a little with this stuff and have seen, that at least for cygwin
> there is an easier way to deal with this. Create a simple link from 'foo' to
> 'foo.exe' and Makes need are fullfilled.
> 
> See ltmain.sh:
> <snip>
>     # The program doesn't exist.
>     \$echo \"\$0: error: \$progdir/\$program does not exist\" 1>&2
>     \$echo \"This script is just a wrapper for \$program.\" 1>&2
>     echo \"See the $PACKAGE documentation for more information.\" 1>&2
>     exit 1
>   fi
> fi\
> "
> 	chmod +x $output
> +	ln -s $output $output$exeext
> 	fi
> 
> The single added line fix this.

It sure would have been nice if you'd mentioned this idea at some point 
over the past two months that I've been working on this, and CC:ing the 
cygwin list on the progress or lack of it.  (I also tried this at one 
point, and ran into problems; I don't recall what they were.  This was 
so easy to do that it didn't take long to implement or test -- and it 
failed, so I dropped it and it left very little impression on me about 
WHY it failed. Sorry.)

But, in any case, *even if it worked in all cases*, it's too late.  At 
this point, this IS the mechanism that libtool will use; it's been 
through the wringer with the folks on the automake list, libtool list, 
and has been tested on cygwin and mingw.  The symlink idea has not 
(plus, there's my earlier experience in which the symlink idea failed 
somehow).

the question now is, does the current implementation work, or not.  I'm 
not interested at this point in finding another mechanism.  I'm 
interested in working any kinks out of THIS one -- if there are any 
left.  I have beaten on it pretty hard, ya know.

But, I'm not being completely arbitrary here; there's more in store for 
the executable wrapper idea.  Eventually, I'd like to subsume all of the 
script wrapper's functionality into the executable wrapper, and do away 
with the script entirely.

--Chuck



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Christopher Faylor]

On Mon, Feb 03, 2003 at 10:04:33PM -0500, Charles Wilson wrote:
>But, in any case, *even if it worked in all cases*, it's too late.  At 
>this point, this IS the mechanism that libtool will use; it's been 
>through the wringer with the folks on the automake list, libtool list, 
>and has been tested on cygwin and mingw.  The symlink idea has not 
>(plus, there's my earlier experience in which the symlink idea failed 
>somehow).

I don't suppose that the idea was squashed because symlinks don't work
for mingw.  Although I guess they do work courtesy of cygw, er, msys.

cgf

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Charles Wilson]

Christopher Faylor wrote:

> I don't suppose that the idea was squashed because symlinks don't work
> for mingw.  Although I guess they do work courtesy of cygw, er, msys.

Yeah, all of the autotools on mingw depend on MSYS==cygwin for basic 
functionality.  So they get symlink support "for free".

--Chuck


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: [avail for test] libtool-devel-20030121-1

[Ralf Habacker]

>
> > Thanks for this additional note.
> >
> >>from a previous mail:
> >
> >>>So, it's important, Ralf, that your 'file' changes NEVER generate a
> >>>false positive (e.g. saying something is an import lib when it is not).
> >>> If your code generates a false negative (saying something is static
> >>>when it's actually an import) -- because for false negatives, my slower
> >>>code will catch it, and mark it "import".
> >
> >
> > There is only one open issue : 'file' reads the first 16K into a buffer.
> >
> > My currently proposal is to search for the string _dll_iname starting at the
> > position of the first object file until the end of the archive (if
> it is less
> > than 16KB) or at least the first 16 KB, if the archive is longer.
>
> How do you get `file` to do that??? "if filesize < 16kb, then...' etc?

I'm not sure, ifc I have understand you right.

file provides a variable nbytes, which will be HOWMANY, if the archive is bigger
or the archive len otherwise.
file.h:# define HOWMANY 16384           /* how much of the file to look at */

Currently I'm discussing the implementation with the 'file' author.

> > That means, if the above mentioned string is found in that area,
> this archive
> > will be identified as import library otherwise as static archive.
>
> That sounds okay.  You'd get false negatives (very few), and no false
> positives.  That's good -- false positives are really bad.  False
> negatives, we let the slower code deal with.
>
> > Because ar puts an offset table to each object file after it's ar
> header and a
> > symbol cache, (the first symbol is a dll_iname symbol, there are some
> > limitations.
> > I've recognized, that the limit is by about 4070 object files.
> > One can estimate, that this proposal will fail with import libs bigger than
> > about 1-1.5 MB. For cygwin I've found this true for only one library:
> > libcrypto.dll.a with 1.6 MB file size.
>
> Hmm...but, if the `file`-based test misses an import lib, the slower
> could could be used to catch it.  And, that slower could *does*
> successfully detect that libcrypto.dll.a is an import lib, even though
> it only looks at the first 100 lines of the `nm` output. So, I see no
> problems there.
>
me too
> >
> > For kde I have found four import libraries which exceeds this limit
> (the biggest
> > import library is about 4.5 MB), but this is no problem, because
> definitly there
> > will not be any further kde releases with import libraries.
>
> I assume you mean because you're going to link directly to the DLL.
> A request: in your packaged releases, include a symlink with the
> "official" import lib name:
>    lib/libkdefoo.dll.a  --> ../bin/cygkdefooX.dll

Yes, i will use chucks-great-directly-linking-to-dll-compatibility-trick :-)

BTW: Currently I'm using a hacked libtool with directly-linking-to-dll support,
which works for mostly cases. It works great. May be you can benefit for a
future libtool support.


Ralf




--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Christopher Faylor]

On Wed, Feb 19, 2003 at 08:54:20PM -0500, Charles Wilson wrote:
>>2.  This could be fixed by using the new direct-linking-to-dll
>>functionality for libraries with many symbols rsp.  big import
>>libraries, which I hope will be available soon.
>
>right.  What's the holdup there?  IIRC, there are two outstanding
>patches to binutils with cygwin impact; one of yours, I think, plus
>another one from Fabrizio Gennari (as modified by Danny Smith).  Last I
>heard, we were still waiting on Fabrizio's copyright assignment; it
>doesn't appear that the patch has been added.  Are we still waiting?
>Anybody know what's up with that?

AFAIK, we're still waiting on Fabrizio's assignment.  I don't recall
that there are any other outstanding binutils patches that impact
cygwin.

Oh wait, yes.  There is a windres patch still waiting approval, I think.
But that isn't really pertinent to this discussion, I suspect.

cgf

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Charles Wilson]

Ralf Habacker wrote:

>>Now, the relative *offset* of that symbol might move around.  But the
>>symname is not likely to change.
>>
> 
> Thanks for this additional note.
> 
>>from a previous mail:
> 
>>>So, it's important, Ralf, that your 'file' changes NEVER generate a
>>>false positive (e.g. saying something is an import lib when it is not).
>>> If your code generates a false negative (saying something is static
>>>when it's actually an import) -- because for false negatives, my slower
>>>code will catch it, and mark it "import".
> 
> 
> There is only one open issue : 'file' reads the first 16K into a buffer.
> 
> My currently proposal is to search for the string _dll_iname starting at the
> position of the first object file until the end of the archive (if it is less
> than 16KB) or at least the first 16 KB, if the archive is longer.

How do you get `file` to do that??? "if filesize < 16kb, then...' etc?

> That means, if the above mentioned string is found in that area, this archive
> will be identified as import library otherwise as static archive.

That sounds okay.  You'd get false negatives (very few), and no false 
positives.  That's good -- false positives are really bad.  False 
negatives, we let the slower code deal with.

> Because ar puts an offset table to each object file after it's ar header and a
> symbol cache, (the first symbol is a dll_iname symbol, there are some
> limitations.
> I've recognized, that the limit is by about 4070 object files.
> One can estimate, that this proposal will fail with import libs bigger than
> about 1-1.5 MB. For cygwin I've found this true for only one library:
> libcrypto.dll.a with 1.6 MB file size.

Hmm...but, if the `file`-based test misses an import lib, the slower 
could could be used to catch it.  And, that slower could *does* 
successfully detect that libcrypto.dll.a is an import lib, even though 
it only looks at the first 100 lines of the `nm` output. So, I see no 
problems there.

> 
> For kde I have found four import libraries which exceeds this limit (the biggest
> import library is about 4.5 MB), but this is no problem, because definitly there
> will not be any further kde releases with import libraries.

I assume you mean because you're going to link directly to the DLL.  A 
request: in your packaged releases, include a symlink with the 
"official" import lib name:
   lib/libkdefoo.dll.a  --> ../bin/cygkdefooX.dll

> 
> So as result I see the following:
> 
> 1. the file patch would generate some false negative in case of very big import
> libraries created  with recent binutils.

this is okay.  See above.

> 2. This could be fixed by using the new direct-linking-to-dll functionality for
> libraries with many symbols rsp. big import libraries, which I hope will be
> available soon.

right.  What's the holdup there?  IIRC, there are two outstanding 
patches to binutils with cygwin impact;  one of yours, I think, plus 
another one from Fabrizio Gennari (as modified by Danny Smith).  Last I 
heard, we were still waiting on Fabrizio's copyright assignment; it 
doesn't appear that the patch has been added. Are we still waiting? 
Anybody know what's up with that?

--Chuck





--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Charles Wilson]

Igor Pechtchanski wrote:

> I'm not libtool-savvy, so this may seem like a stupid question -- if it
> is, let me know.  However, I've been kinda following this discussion, and
> haven't seen it answered.  So here goes:
> - How hard is it to add the capability to pass linker arguments through
>   the LD variable?  What's involved?  It seems (to me, anyway) like this
>   would be the least painful solution all around...
> This is probably totally irrelevant and out of line.  I'll just shut up
> now.

You could -- but that runs counter to the "libtool way'.  Libtool 
doesn't even let the compiler determine which system libraries get 
linked -- at least, not directly.  Instead, libtool parses the output of 
'gcc <args> -v' and 'gcc -nostdlibs <args> -v', and stores the 'crt.o' 
and 'libgcc.a' miscellany.

Then, when it's time to link a library, libtool uses ld directly, and 
adds the 'crt.0' and friends itself.

The idea is that libtool gets maximal information; nothing is hidden 
from libtool (e.g. gcc's spec file adding extra system libs, etc).  Now, 
some have argued [ahem, ahem] that libtool is trying to do too much. 
Maybe so.  But this is really such a minor issue (and my time is so 
limited) that I don't want to tackle that beast -- the effort is too 
high for the possible benefit.

Anyway, passing args thru LD -- kinda defeats the "libtool controls 
everything" idea.  In fact, I was surprised that the patch that let 
CC='gcc -mno-cygwin' work in libtool was accepted.  I was FLOORED when 
the "let CC contain any flag whatsoever" patch made it in.

Now, this may be an argument that "okay, the rules have been loosened; 
submit the patch and hope...". However, I'd rather play it conservative 
this close to a release.  I want the most common 4 cases, 
cygwinBuild/cygiwnTarget to work.  mingwBuild/mingwTarget.  And 
linuxbuild/cygiwnTarget, linuxBuild/mingwTarget.

those all work now.  Let's not break anything before 1.5 comes out; too 
many projects are holding off until libtool-1.5 is available before 
accepting cygiwn patches (esp. cygwin patches that *require* libtool-devel).

so that's the argument against a broad "LD can contain any random flag" 
patch.  what about a focused patch, that specifically allows 
--enable-psuedo-relocs?

Ugh.  How hackish.  I'd rather push for that to become the default 
behavior in binutils, than kludge up the (already nasty) libtool code 
with YASPE (yet another special purpose exception).

--Chuck


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: [avail for test] libtool-devel-20030121-1

[Ralf Habacker]

> > You may be right in this issue, but couldn't the symbol "_dll_iname" doesn't
> > change in future implementation too ? The important question seems
> to me like
> > this: [1] Which is the mostly stable identifier we build on ?
>
> filenames change because evil twisted pesky end-users change them, at
> any time, for any reason.  The _dll_iname symbol has been in every
> DLL/implib I've built in the four (five?) years I've been using cygwin.
>   The only way that symbol is going to change is if somebody
> deliberately changes that particular piece of binutils.
>
> Now, the relative *offset* of that symbol might move around.  But the
> symname is not likely to change.
>
Thanks for this additional note.

>from a previous mail:
>>So, it's important, Ralf, that your 'file' changes NEVER generate a
>>false positive (e.g. saying something is an import lib when it is not).
>>  If your code generates a false negative (saying something is static
>>when it's actually an import) -- because for false negatives, my slower
>>code will catch it, and mark it "import".

There is only one open issue : 'file' reads the first 16K into a buffer.

My currently proposal is to search for the string _dll_iname starting at the
position of the first object file until the end of the archive (if it is less
than 16KB) or at least the first 16 KB, if the archive is longer.
That means, if the above mentioned string is found in that area, this archive
will be identified as import library otherwise as static archive.

Because ar puts an offset table to each object file after it's ar header and a
symbol cache, (the first symbol is a dll_iname symbol, there are some
limitations.
I've recognized, that the limit is by about 4070 object files.
One can estimate, that this proposal will fail with import libs bigger than
about 1-1.5 MB. For cygwin I've found this true for only one library:
libcrypto.dll.a with 1.6 MB file size.

For kde I have found four import libraries which exceeds this limit (the biggest
import library is about 4.5 MB), but this is no problem, because definitly there
will not be any further kde releases with import libraries.

So as result I see the following:

1. the file patch would generate some false negative in case of very big import
libraries created  with recent binutils.

2. This could be fixed by using the new direct-linking-to-dll functionality for
libraries with many symbols rsp. big import libraries, which I hope will be
available soon.

Ralf


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Igor Pechtchanski]

On Sun, 16 Feb 2003, Charles Wilson wrote:

> Max Bowsher wrote:
>
> >>>But, that's neither here nor there.  IF these crossbreed implibs are
> >                                                ^^^^^^^^^^^^^^^^^^
> > libuuid.a at least is static *only* - not crossbreed. So there really is no
> > way for libtool know to allow it except by name.
>
> Ugh.  You're right -- I was confused by the implicit rules in
> winsup/w32api/lib/Makefile.in.  libkernel32.a and libshell32.a are
> crossbreeds, but the others (libscrnsave.a, libscrnsavw.a,
> liblargeint.a, and libuuid.a) are pure static. [The rest are pure import].
>
> I really really REALLY don't want to special case this. I was able to
> avoid special-casing the compiler runtimes, by using libtool's existing
> ability to parse the output of gcc -nostdlib, and compare it to the
> "normal" link command to determine what libraries are, in fact, compiler
> runtimes.
>
> Newer libtools (e.g. newer than the 20030121 release) allow compiler
> args to be passed in the CC variable, but not linker args thru the LD
> variable.  This means there can be no easy solution ("turn libuuid.a
> into a DLL and import lib; set LD='ld --enable-runtime-pseudo-reloc';
> and then run libtool").  Sigh.
>
> The only alternatives for this particular problem seem to be:
>
> 1) punt.  If you want -luuid, grab the single source file
> winsup/w32api/lib/uuid.c, and add it to your project's source.  Or,
> messier, in your own makefile locate the system's libuuid.a, ar -x it,
> and include the resulting .o directly into your dependencies. You could
> be clever by (basically) recreating libuuid.a as a convenience (static)
> library within your project; then libtool wouldn't complain and you
> could still say -luuid (only, it would have -L<srcdir> in front of it).
>
> 2) delay.  --enable-runtime-pseudo-relocs will be the default **on
> cygwin** someday (never on mingw).  Wait until then, or push it now.  In
> any case, once it is the default, then we can simply dllize libuuid, and
> then -luuid will grab the import library, and libtool will be happy.
> But this can never be the solution for mingw.
>
> 3) kludge.  Put a special-case exception for -luuid and libuuid.a -- and
> the other four !@#$!@# static libs in w32api -- into the libtool code.
>
> 4) revoke the libtool policy; DLLs with static dependencies are just dandy.
>
> All four alternatives suck.  #4 is the worst; it won't happen.  #2 won't
> help mingw.  That leaves #1 and #3 -- and I hate kludges.  How important
> is this?  Is "punting" really such a bad idea?
>
> --Chuck

Chuck,

I'm not libtool-savvy, so this may seem like a stupid question -- if it
is, let me know.  However, I've been kinda following this discussion, and
haven't seen it answered.  So here goes:
- How hard is it to add the capability to pass linker arguments through
  the LD variable?  What's involved?  It seems (to me, anyway) like this
  would be the least painful solution all around...
This is probably totally irrelevant and out of line.  I'll just shut up
now.
	Igor
-- 
				http://cs.nyu.edu/~pechtcha/
      |\      _,,,---,,_		pechtcha@cs.nyu.edu
ZZZzz /,`.-'`'    -.  ;-;;,_		igor@watson.ibm.com
     |,4-  ) )-,_. ,\ (  `'-'		Igor Pechtchanski
    '---''(_/--'  `-'\_) fL	a.k.a JaguaR-R-R-r-r-r-.-.-.  Meow!

Oh, boy, virtual memory! Now I'm gonna make myself a really *big* RAMdisk!
  -- /usr/games/fortune


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Charles Wilson]

Max Bowsher wrote:

> Besides, this means altering the platform to suit libtool. Talk about the
> tail wagging the dog!

see below

>>The only alternatives for this particular problem seem to be:
>>
>>1) punt.
> 
> 
> Well, it's not like we've got many complaints about this.
> 
> 
>>2) delay.  --enable-runtime-pseudo-relocs will be the default **on
>>cygwin** someday (never on mingw).  Wait until then, or push it now.
>>In any case, once it is the default, then we can simply dllize libuuid,
>>and then -luuid will grab the import library, and libtool will be
>>happy. But this can never be the solution for mingw.
> 
> 
> And, as I say above, is ridiculous. Libtool is supposed to assist
> portability - no force platforms to redesign themselves.

No, not really. You still have to conform to certain basic rules -- 
libtool is not a magic black box ("dump all .c, .f, .C, and random .o or 
.a files here, turn crank, and I'll do the right thing").

Fringe platforms -- like cygwin -- have always had to do strange things. 
  In the 1.4.x libtools, you had to have special declarations in your 
header files. You had to entirely avoid backlinking. You had to include 
a special flag (LIBTOOL_DLL_WIN32. I think) in configure.in.  And 
"normal" platforms still have to follow certain rules.

We've done a LOT of work over the past two years to hide these things 
from you.  There have been countless changes in the cygwin kernel, 
libtool, autoconf, automake, binutils, and gcc to enable the present 
functionality -- and now, building dlls with the -devel autotools 
framework is so much like unix, that you think it should be *exactly* 
like unix.  Even when accessing *windows-specific libraries* like 
w32api/libuuid.a!

There's one big non-unixism I'd like to get rid of, but the solution is 
cygwin-specific.  It won't be supported on mingw (or -mno-cygwin).  And 
that non-unixism is the "difficulties" with data imports from a DLL, 
where the datatype has multiword storage.  --pseudo-relocs solves that 
problem, but (in reality) it's only a viable solution for libtool 
purposes once it becomes the default setting in ld.exe.  But, since 
pseudo-reloc exe's require runtime support...it is unwise to enable it 
as the default on mingw, since mingw DLLs are supposed to be usable by 
standard windows tools (e.g. MSVC).  DLLs which require pseudo-reloc 
support in the linker and runtime are NOT portable to standard windows 
tools.  Since on a cygwin system, we *know* we will always have 
cygwin1.dll -- and because we don't support any other compiler than 
gcc/binutils -- we're safe in making our DLLs pseudo-reloc dependant, on 
the rare occasions where that becomes necessary.  Thus, this is a 
cygwin-only, not mingw, solution.

Anyway, I was talking about "rules" you must follow -- in any build 
environment (like libtool).  One of the old, and still current, rules is 
that on windows/cygwin/mingw, you must use -no-undefined [and it better 
be true!] -- because you can't make a DLL with undefined symbols.  Now, 
one of the NEW rules is "ensure that all dependencies are dynamic."  If 
the are not, fine -- libtool won't *fail*.  It will simply build a 
static lib.  But because you haven't followed the rules for shared libs, 
libtool won't build a shared lib.

the choice is simple: (1) figure out how to *modify the cygwin platform* 
to ensure that ALL system libs are available in shared form, or (2) 
modify libtool and weaken the existing policy.

that's it.

In the past, we've done either -- I modified libtool so that it wouldn't 
complain about compiler runtime libs [dllized compiler libs, while a 
laudable goal, are WAY too hard without MASSIVE changes in the gcc 
sourcecode.  Fortunately, Zack and Nathanael are gluttons for 
punishment.  Their goal is converting gcc/binutils to use modern 
autotools -- and that's 95% -- 100% of the work, given the -devel 
autotools on cygwin]  At other times, we dllized certain existing add-on 
libraries so that a new port of a different library could be added to 
the cygwin system in dllized form, straight from its very beginning.

I hope that once Zack and Nathanael's work in the gcc and binutils tree, 
updating them to modern autotools, is done, that our gcc builds will 
suddenly include shared compiler runtimes...but I'm not holding my 
breath on that score.  And *hopefully* that change won't require any 
work on cgf's part: it'll just "happen".

Most of cygwin's runtime support libs are shared.  [e.g. ALL of mine, 
and I provide a lot of 'em; other maintainers have been just as 
proactive].  Most of the w32api libs -- since they are derived from MS 
DLL's in the first place -- are shared.  There are only a few -- six -- 
exceptions.

And you are arguing *against* improving our platform by making all libs 
available as shared, simply because it's driven by a libtool 
requirement?  sheesh.

The task may be difficult -- given that the static items in question are 
"bad" datatypes leading to pseudo-reloc problems -- but you seem to be 
saying that the goal itself is unworthy, given the motivation.

>>3) kludge.  Put a special-case exception for -luuid and libuuid.a --
>>and the other four !@#$!@# static libs in w32api -- into the libtool code.
> 
> 
> Messy.

Yes, quite.  But it's still better than...

>>4) revoke the libtool policy; DLLs with static dependencies are just
>>dandy.
> 
> 
> I like it. 

You may like it, but IMNSHO, you're wrong.  Think about libpng, for 
instance -- which depends on libz.

Now, suppose I ship a libpng.dll which was linked with a static libz.a, 
and further suppose that there's some global static symbol in libz -- 
call it, "encryption_key".  Now, libpng.dll doesn't RE-EXPORT that 
symbol; why should it?  "encryption_key" belongs to libz.  But if 
libpng.dll were linked dynamically to libz.dll -- and my application 
were linked dynamically to both libz.dll and libpng.dll -- then my app 
would ALSO have access to the *public symbol* exported by libz.dll, 
which libpng.dll uses.

However, I then write a problem, chuck.exe, which links against 
libpng.dll.  Now, if I don't use any libz routines *myself*, I don't 
need to link chuck.exe against libz, because libpng.dll has had its 
dependencies satisfied *statically*.  (In real life, cygpng.dll is 
linked dynamically against cygz.dll, so chuck.exe DOES need to add '-lz' 
even if chuck.exe doesn't access zlib functions itself.  But that's real 
life.  Back to the fairytale:) However, suppose further that chuck.exe 
DOES use zlib directly.  But now, zlib has changed.  There's a new 
version available...in which the "encryption_key" is double, instead of int.

Now what?  (a) which version of encryption_key gets used by chuck.exe? 
double, or int?  (b) since it's a "global static' you would think that 
if chuck.exe changed its value, chuck.exe could reasonable expect that 
change to affect how the libpng routines work.  but it won't -- because 
cygpng has its own private copy.

For instance: chuck.exe sets encryption_key to "4.234123" -- but libpng 
saves an encrypted file using some other, random key [that I don't know 
the value of, and which in this context is a CONSTANT probably equal to 
ZERO, because I have no ability, from chuck.exe, to change libpng's 
private copy of that variable!!]

Okay, so maybe we let libpng re-export "encryption_key".  Fine -- if 
that's the ONLY thing I need from libz, all is well.  I link against 
only -lpng, I can access the re-exported (private static copy of) 
"encryption_key".  But what if I also need to compress/uncompress other 
zlib streams?  Then I need to link against -lz -- and I'll get a symbol 
conflict. [I'm glossing over some things here like "static symbols 
override dynamic imports" -- but then you'd be right back in the 
situation described above.]

Okay, so libpng's re-exported version of encryption_key is exported 
under a different name.  Now, there's no symbol conflict.  It'll work -- 
but look what you've done; you've basically forked the zlib interface. 
You might as well snarf all of the zlib code into libpng itself, and 
drop any dependency whatsoever on the "real" zlib.  This is not progress.

It's a silly example, of course -- but you could probably find a real 
life example if you looked hard enough.  And that's why I think the 
libtool policy change is a GOOD one.  Shared libs, in general, should 
not have static dependencies.

> But it's not going to happen. So:
> How about a flag, like -no-undefined ?
> For example: -i-know-what-i-want-to-link-dont-interfere-please :-)

--shoot-self-in-foot-with-unpredictable-consequences ?

I've noticed that most of the time (but not always) when folks complain 
about libtool being too smart for its own good, they don't understand 
WHY libtool makes the decisions it does.  Sometimes, most of the time, 
libtool *really does know best*.  Or at least, libtool knows better than 
the wet-behind-the-ears developer who can't get it to act like she wants 
it to.

I trust that you will not bring up those numerous occasions where *I* 
have complained about libtool being too smart.  Thanks. <g>

>>All four alternatives suck.  #4 is the worst; it won't happen.  #2
>>won't help mingw.  That leaves #1 and #3 -- and I hate kludges.  How
>>important is this?  Is "punting" really such a bad idea?
> 
> 
> Punting is acceptable, if necessary. What do you think about my flag idea?

It's just another way around the rules -- and these are rules I happen 
to agree with. I'd rather put special-case in there specifically for a 
small set of [4!] known libs, because I know those won't be abused. 
"Gee it's too hard to build libsilly as a dll, I'll just link my 
cygsillier.dll against libsilly.a, and use Max's --shoot-self-in-foot 
flag..."

--Chuck


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Charles Wilson]

Max Bowsher wrote:

>>>
>>>But, that's neither here nor there.  IF these crossbreed implibs are
> 
>                                                  ^^^^^^^^^^^^^^^^^^
> libuuid.a at least is static *only* - not crossbreed. So there really is no
> way for libtool know to allow it except by name.

Ugh.  You're right -- I was confused by the implicit rules in 
winsup/w32api/lib/Makefile.in.  libkernel32.a and libshell32.a are 
crossbreeds, but the others (libscrnsave.a, libscrnsavw.a, 
liblargeint.a, and libuuid.a) are pure static. [The rest are pure import].

I really really REALLY don't want to special case this. I was able to 
avoid special-casing the compiler runtimes, by using libtool's existing 
ability to parse the output of gcc -nostdlib, and compare it to the 
"normal" link command to determine what libraries are, in fact, compiler 
runtimes.

Newer libtools (e.g. newer than the 20030121 release) allow compiler 
args to be passed in the CC variable, but not linker args thru the LD 
variable.  This means there can be no easy solution ("turn libuuid.a 
into a DLL and import lib; set LD='ld --enable-runtime-pseudo-reloc'; 
and then run libtool").  Sigh.

The only alternatives for this particular problem seem to be:

1) punt.  If you want -luuid, grab the single source file 
winsup/w32api/lib/uuid.c, and add it to your project's source.  Or, 
messier, in your own makefile locate the system's libuuid.a, ar -x it, 
and include the resulting .o directly into your dependencies. You could 
be clever by (basically) recreating libuuid.a as a convenience (static) 
library within your project; then libtool wouldn't complain and you 
could still say -luuid (only, it would have -L<srcdir> in front of it).

2) delay.  --enable-runtime-pseudo-relocs will be the default **on 
cygwin** someday (never on mingw).  Wait until then, or push it now.  In 
any case, once it is the default, then we can simply dllize libuuid, and 
then -luuid will grab the import library, and libtool will be happy. 
But this can never be the solution for mingw.

3) kludge.  Put a special-case exception for -luuid and libuuid.a -- and 
the other four !@#$!@# static libs in w32api -- into the libtool code.

4) revoke the libtool policy; DLLs with static dependencies are just dandy.

All four alternatives suck.  #4 is the worst; it won't happen.  #2 won't 
help mingw.  That leaves #1 and #3 -- and I hate kludges.  How important 
is this?  Is "punting" really such a bad idea?

--Chuck


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Charles Wilson]

Ralf Habacker wrote:

>>the negative: Ralf, you keep trying to assume things based on filenames.
>>  Filenames LIE.  Whether it is the name of the archive (foo.dll.a) or
>>the name of an object in the archive (dxxxxxx.o), it's gonna fail -- and
>>it will fail in EXTREMELY hard-to-track-down ways.
> 
> 
> You may be right in this issue, but couldn't the symbol "_dll_iname" doesn't
> change in future implementation too ? The important question seems to me like
> this: [1] Which is the mostly stable identifier we build on ?

filenames change because evil twisted pesky end-users change them, at 
any time, for any reason.  The _dll_iname symbol has been in every 
DLL/implib I've built in the four (five?) years I've been using cygwin. 
  The only way that symbol is going to change is if somebody 
deliberately changes that particular piece of binutils.

Now, the relative *offset* of that symbol might move around.  But the 
symname is not likely to change.


> This sounds good.

Great, because it's already implemented, posted to libtool-patches, and 
accepted into CVS.

> I have no problem with this.

great. There will be a new test release of libtool posted as soon as I 
can get it uploaded.

It'll work faster than the current -test release without any user 
changes -- and you should be able to see an additional speedup on your 
system, if you've got your hacked file magic ready to go.

--Chuck


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: [avail for test] libtool-devel-20030121-1

[Ralf Habacker]

> > Thats very bad.
> >
>
> Yeah.  I don't know why these implibs need to declare these static
> structures; it's possible that w32api is just following the lead of the
> corresponding .lib files in the MSVC distribution.
>
> But, that's neither here nor there.  IF these crossbreed implibs are
> detected as "import libraries" then libtool will let you link a
> sharedlib using them as dependencies.  this could eventually lead to a
> data mismatch problem, if the static info is changed -- but since your
> DLL has a private copy of the old version...
>
> Fortunately, this isn't likely.  The static data consists of GUID
> structures for well-known services/programs/etc on windows.  MS would be
> insane to change those definitions at this point.
>
> >
> > There might be another (easier) way for identifiying. There are
> some resources
> > in the internet relating to the ar format for examples
<snip>
> the negative: Ralf, you keep trying to assume things based on filenames.
>   Filenames LIE.  Whether it is the name of the archive (foo.dll.a) or
> the name of an object in the archive (dxxxxxx.o), it's gonna fail -- and
> it will fail in EXTREMELY hard-to-track-down ways.

You may be right in this issue, but couldn't the symbol "_dll_iname" doesn't
change in future implementation too ? The important question seems to me like
this: [1] Which is the mostly stable identifier we build on ?

> the positive: I'm glad you're following thru on this.  Here's a plan: I
> can make arrangements so that WHEN we find the Right Thing to do in
> file, libtool can take advantage of it immediately.  thus, we can get my
> speedups into libtool now along with hooks for your super-speedup.
>
> Flowchart:
>
> .  'file -L $fn' # e.g. if symlink, check the target
> .  if file output has "DLL"
> .    exit "x86 DLL"
> .  elif file output has "executable"
> .    if "MS Windows PE Intel" # to screen out "executable" shell scripts
> .      exit "x86 DLL"
> .    endif
> .  elif file output has "ar archive import library"
> .    exit "x86 archive import"
> .  elif file output has "ar archive" # both static and import libs
> .    if objdump output is good       # make sure it's an ar of x86 objs
> .      if nm output has " I "        # fallback test for import libs
> .        exit "x86 archive import"
> .      else
> .        exit "x86 archive static"
> .      endif
> .    endif
> .  endif
> .  exit "unknown"
>
> This way, we get my speedups right away.  The "elif file output has "ar
> archive import library" isn't used right now (because stock 'file' won't
> ever generate that output string).  Thus, right now we use the "ar
> archive" section for both static libs and import libs.  No probs.
>
> Later, your code gets into 'file' -- and suddenly we see big speedups
> for most linking against import libs.  Even better, YOU can see those
> speedups immediately using (the new upcoming 'stock' libtool), simply by
> changing your magic file.  Thus, it helps testing.  Even better,
> anything that FAILS your new file test, will get checked again using the
> slower code!

This sounds good.

> So, it's important, Ralf, that your 'file' changes NEVER generate a
> false positive (e.g. saying something is an import lib when it is not).
>   If your code generates a false negative (saying something is static
> when it's actually an import) -- because for false negatives, my slower
> code will catch it, and mark it "import".

see [1]

> Oh, and we MUST agree on the id string before hand.  You've suggested
> that import libs add 'import library' to the existing 'ar archive';
> thus: 'ar archive import library'.  sounds good to me -- I'll go with
> that.  But don't change it.
>
> sound good?

I have no problem with this.

Ralf




--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Max Bowsher]

Charles Wilson wrote:
> Charles Wilson wrote:
>
>>> I think that in some cases, libtool tries to do too much. I've run
>>> into a number of cases which would have just worked if libtool had
>>> passed the arguments to gcc without modification, but it insists on
>>> filtering
>>> stuff in
>>> complex ways. For example, gcc -print-search-dirs doesn't admit to
>>> searching
>>> /usr/lib/w32api, so any attempt to link with a w32api library when
>>> cross-compiling with -mno-cygwin (which means the Cygwin-specific
>>> hack in libtool.m4 doesn't get activated) fails. Now, admittedly,
>>> gcc is probably wrong in not showing all its search dirs correctly,
>>> but sometimes I wish libtool trusted the compiler more.
>>
>>
>> mebbe. But that's another discussion.
>
> Your larger point is still subject for a (different) discussion, but
> one small issue: a *very* recent patch to libtool CVS allows passing
> arguments to gcc.  so, CC='gcc -mno-cygwin -L/usr/lib/w32api' is okay
> now.  [there was an earlier patch that specifically allowed -mXXXXX
> options; the newer one lets anything go]
>
> However, I tend to just add -L/usr/lib/w32api directly to my spec file
> anyway (although a cross compiler would be subtly different, I know).

Neither will work. The compiler is already perfectly happy. libtool decides
it knows better, though, and intervenes, breaking the build. I wouldn't
worry too much about this now. There's already a hardcoded
sys_lib_search_path_spec for cygwin. It only affects building for mingw in a
cygwin environment.

About the only solution I can think of is to test $CC for -mno-cygwin, and
use the cygwin sys_lib_search_path_spec if found.

Max.


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Max Bowsher]

Charles Wilson wrote:
> Max Bowsher wrote:
>>>> But, that's neither here nor there.  IF these crossbreed implibs
>>                                                 ^^^^^^^^^^^^^^^^^^
>> libuuid.a at least is static *only* - not crossbreed. So there
>> really is no way for libtool know to allow it except by name.
>
> Ugh.  You're right -- I was confused by the implicit rules in
> winsup/w32api/lib/Makefile.in.  libkernel32.a and libshell32.a are
> crossbreeds, but the others (libscrnsave.a, libscrnsavw.a,
> liblargeint.a, and libuuid.a) are pure static. [The rest are pure
> import].
>
> I really really REALLY don't want to special case this. I was able to
> avoid special-casing the compiler runtimes, by using libtool's
> existing ability to parse the output of gcc -nostdlib, and compare it
> to the "normal" link command to determine what libraries are, in
> fact, compiler runtimes.
>
> Newer libtools (e.g. newer than the 20030121 release) allow compiler
> args to be passed in the CC variable, but not linker args thru the LD
> variable.  This means there can be no easy solution ("turn libuuid.a
> into a DLL and import lib; set LD='ld --enable-runtime-pseudo-reloc';
> and then run libtool").  Sigh.

Besides, this means altering the platform to suit libtool. Talk about the
tail wagging the dog!

> The only alternatives for this particular problem seem to be:
>
> 1) punt.

Well, it's not like we've got many complaints about this.

> 2) delay.  --enable-runtime-pseudo-relocs will be the default **on
> cygwin** someday (never on mingw).  Wait until then, or push it now.
> In any case, once it is the default, then we can simply dllize libuuid,
> and then -luuid will grab the import library, and libtool will be
> happy. But this can never be the solution for mingw.

And, as I say above, is ridiculous. Libtool is supposed to assist
portability - no force platforms to redesign themselves.

> 3) kludge.  Put a special-case exception for -luuid and libuuid.a --
> and the other four !@#$!@# static libs in w32api -- into the libtool code.

Messy.

> 4) revoke the libtool policy; DLLs with static dependencies are just
> dandy.

I like it. But it's not going to happen. So:
How about a flag, like -no-undefined ?
For example: -i-know-what-i-want-to-link-dont-interfere-please :-)

> All four alternatives suck.  #4 is the worst; it won't happen.  #2
> won't help mingw.  That leaves #1 and #3 -- and I hate kludges.  How
> important is this?  Is "punting" really such a bad idea?

Punting is acceptable, if necessary. What do you think about my flag idea?

Max.


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Charles Wilson]

Charles Wilson wrote:

>> I think that in some cases, libtool tries to do too much. I've run into a
>> number of cases which would have just worked if libtool had passed the
>> arguments to gcc without modification, but it insists on filtering 
>> stuff in
>> complex ways. For example, gcc -print-search-dirs doesn't admit to 
>> searching
>> /usr/lib/w32api, so any attempt to link with a w32api library when
>> cross-compiling with -mno-cygwin (which means the Cygwin-specific hack in
>> libtool.m4 doesn't get activated) fails. Now, admittedly, gcc is probably
>> wrong in not showing all its search dirs correctly, but sometimes I wish
>> libtool trusted the compiler more.
> 
> 
> mebbe. But that's another discussion.

Your larger point is still subject for a (different) discussion, but one 
small issue: a *very* recent patch to libtool CVS allows passing 
arguments to gcc.  so, CC='gcc -mno-cygwin -L/usr/lib/w32api' is okay 
now.  [there was an earlier patch that specifically allowed -mXXXXX 
options; the newer one lets anything go]

However, I tend to just add -L/usr/lib/w32api directly to my spec file 
anyway (although a cross compiler would be subtly different, I know).

--Chuck


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Charles Wilson]

Max Bowsher wrote:

> 
> Neither will work. The compiler is already perfectly happy. libtool decides
> it knows better, though, and intervenes, breaking the build. I wouldn't
> worry too much about this now. There's already a hardcoded
> sys_lib_search_path_spec for cygwin. It only affects building for mingw in a
> cygwin environment.

-mno-cygwin, in other words.  So "native" (e.g. MSYS) mingw builds are 
not affected by this?

> About the only solution I can think of is to test $CC for -mno-cygwin, and
> use the cygwin sys_lib_search_path_spec if found.

That's not so bad. Although, it should NOT use cygwin's 
sys_lib_search_path_spec -- you'd pull in all of the cygwin-dependent 
libz's and libncurses's.  What you want is to ADD /usr/lib/w32api to the 
"standard" 'gcc -mno-cygwin' search path.  That "standard" path already 
includes the right gcc runtime directory, and already includes 
/usr/lib/mingw via some symlinks.  You just need to add 
-L/usr/lib/w32api -- we know that nothing in there is cygwin-dependent.

I believe there are a few checks for -mno-cygwin already sprinkled 
around in the code.  Wanna submit a patch as outlined above?

--Chuck



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Max Bowsher]

On Mon, 17 Feb 2003, Charles Wilson wrote:

> Max Bowsher wrote:
>
> > Besides, this means altering the platform to suit libtool. Talk about the
> > tail wagging the dog!
>
> see below
>
> >>The only alternatives for this particular problem seem to be:
> >>
> >>1) punt.
> >
> > Well, it's not like we've got many complaints about this.
> >
> >>2) delay.  --enable-runtime-pseudo-relocs will be the default **on
> >>cygwin** someday (never on mingw).  Wait until then, or push it now.
> >>In any case, once it is the default, then we can simply dllize libuuid,
> >>and then -luuid will grab the import library, and libtool will be
> >>happy. But this can never be the solution for mingw.
> >
> > And, as I say above, is ridiculous. Libtool is supposed to assist
> > portability - no force platforms to redesign themselves.
>
> No, not really. You still have to conform to certain basic rules --
> libtool is not a magic black box ("dump all .c, .f, .C, and random .o or
> .a files here, turn crank, and I'll do the right thing").
>
> Fringe platforms -- like cygwin -- have always had to do strange things.
>   In the 1.4.x libtools, you had to have special declarations in your
> header files. You had to entirely avoid backlinking. You had to include
> a special flag (LIBTOOL_DLL_WIN32. I think) in configure.in.  And
> "normal" platforms still have to follow certain rules.
>
> We've done a LOT of work over the past two years to hide these things
> from you.  There have been countless changes in the cygwin kernel,
> libtool, autoconf, automake, binutils, and gcc to enable the present
> functionality -- and now, building dlls with the -devel autotools
> framework is so much like unix, that you think it should be *exactly*
> like unix.  Even when accessing *windows-specific libraries* like
> w32api/libuuid.a!

Hmm. It seems us relative newcomers to Cygwin are fortunate to have never
experienced the bad old days!

> There's one big non-unixism I'd like to get rid of, but the solution is
> cygwin-specific.  It won't be supported on mingw (or -mno-cygwin).  And
> that non-unixism is the "difficulties" with data imports from a DLL,
> where the datatype has multiword storage.  --pseudo-relocs solves that
> problem, but (in reality) it's only a viable solution for libtool
> purposes once it becomes the default setting in ld.exe.  But, since
> pseudo-reloc exe's require runtime support...it is unwise to enable it
> as the default on mingw, since mingw DLLs are supposed to be usable by
> standard windows tools (e.g. MSVC).  DLLs which require pseudo-reloc
> support in the linker and runtime are NOT portable to standard windows
> tools.  Since on a cygwin system, we *know* we will always have
> cygwin1.dll -- and because we don't support any other compiler than
> gcc/binutils -- we're safe in making our DLLs pseudo-reloc dependant, on
> the rare occasions where that becomes necessary.  Thus, this is a
> cygwin-only, not mingw, solution.
>
> Anyway, I was talking about "rules" you must follow -- in any build
> environment (like libtool).  One of the old, and still current, rules is
> that on windows/cygwin/mingw, you must use -no-undefined [and it better
> be true!] -- because you can't make a DLL with undefined symbols.  Now,
> one of the NEW rules is "ensure that all dependencies are dynamic."  If
> the are not, fine -- libtool won't *fail*.  It will simply build a
> static lib.  But because you haven't followed the rules for shared libs,
> libtool won't build a shared lib.

Yesss.. but, when building glib, if one of its components gets staticized,
the next one fails to link.

> the choice is simple: (1) figure out how to *modify the cygwin platform*
> to ensure that ALL system libs are available in shared form, or (2)
> modify libtool and weaken the existing policy.
>
> that's it.

(1) is not really an option, since w32api is an alternative implementation
of something essentially defined by Microsoft.

> In the past, we've done either -- I modified libtool so that it wouldn't
> complain about compiler runtime libs [dllized compiler libs, while a
> laudable goal, are WAY too hard without MASSIVE changes in the gcc
> sourcecode.  Fortunately, Zack and Nathanael are gluttons for
> punishment.  Their goal is converting gcc/binutils to use modern
> autotools -- and that's 95% -- 100% of the work, given the -devel
> autotools on cygwin]  At other times, we dllized certain existing add-on
> libraries so that a new port of a different library could be added to
> the cygwin system in dllized form, straight from its very beginning.
>
> I hope that once Zack and Nathanael's work in the gcc and binutils tree,
> updating them to modern autotools, is done, that our gcc builds will
> suddenly include shared compiler runtimes...but I'm not holding my
> breath on that score.  And *hopefully* that change won't require any
> work on cgf's part: it'll just "happen".
>
> Most of cygwin's runtime support libs are shared.  [e.g. ALL of mine,
> and I provide a lot of 'em; other maintainers have been just as
> proactive].  Most of the w32api libs -- since they are derived from MS
> DLL's in the first place -- are shared.  There are only a few -- six --
> exceptions.
>
> And you are arguing *against* improving our platform by making all libs
> available as shared, simply because it's driven by a libtool
> requirement?  sheesh.

Not exactly. w32api currently more-or-less follow the file layout of the
Platform SDK. That's what I'm arguing against changing.

> >>3) kludge.  Put a special-case exception for -luuid and libuuid.a --
> >>and the other four !@#$!@# static libs in w32api -- into the libtool code.
> >
> > Messy.
>
> Yes, quite.  But it's still better than...
>
> >>4) revoke the libtool policy; DLLs with static dependencies are just
> >>dandy.
> >
> > I like it.
>
> You may like it, but IMNSHO, you're wrong.  Think about libpng, for
> instance -- which depends on libz.
>
> Now, suppose I ship a libpng.dll which was linked with a static libz.a,
> and further suppose that there's some global static symbol in libz --
> call it, "encryption_key".  Now, libpng.dll doesn't RE-EXPORT that
> symbol; why should it?  "encryption_key" belongs to libz.  But if
> libpng.dll were linked dynamically to libz.dll -- and my application
> were linked dynamically to both libz.dll and libpng.dll -- then my app
> would ALSO have access to the *public symbol* exported by libz.dll,
> which libpng.dll uses.
>
> However, I then write a problem, chuck.exe, which links against
> libpng.dll.  Now, if I don't use any libz routines *myself*, I don't
> need to link chuck.exe against libz, because libpng.dll has had its
> dependencies satisfied *statically*.  (In real life, cygpng.dll is
> linked dynamically against cygz.dll, so chuck.exe DOES need to add '-lz'
> even if chuck.exe doesn't access zlib functions itself.  But that's real
> life.  Back to the fairytale:) However, suppose further that chuck.exe
> DOES use zlib directly.  But now, zlib has changed.  There's a new
> version available...in which the "encryption_key" is double, instead of int.
>
> Now what?  (a) which version of encryption_key gets used by chuck.exe?
> double, or int?  (b) since it's a "global static' you would think that
> if chuck.exe changed its value, chuck.exe could reasonable expect that
> change to affect how the libpng routines work.  but it won't -- because
> cygpng has its own private copy.
>
> For instance: chuck.exe sets encryption_key to "4.234123" -- but libpng
> saves an encrypted file using some other, random key [that I don't know
> the value of, and which in this context is a CONSTANT probably equal to
> ZERO, because I have no ability, from chuck.exe, to change libpng's
> private copy of that variable!!]
>
> Okay, so maybe we let libpng re-export "encryption_key".  Fine -- if
> that's the ONLY thing I need from libz, all is well.  I link against
> only -lpng, I can access the re-exported (private static copy of)
> "encryption_key".  But what if I also need to compress/uncompress other
> zlib streams?  Then I need to link against -lz -- and I'll get a symbol
> conflict. [I'm glossing over some things here like "static symbols
> override dynamic imports" -- but then you'd be right back in the
> situation described above.]
>
> Okay, so libpng's re-exported version of encryption_key is exported
> under a different name.  Now, there's no symbol conflict.  It'll work --
> but look what you've done; you've basically forked the zlib interface.
> You might as well snarf all of the zlib code into libpng itself, and
> drop any dependency whatsoever on the "real" zlib.  This is not progress.
>
> It's a silly example, of course -- but you could probably find a real
> life example if you looked hard enough.  And that's why I think the
> libtool policy change is a GOOD one.  Shared libs, in general, should
> not have static dependencies.

Aargh! At least I now have a little more insight into *why* this policy
exists.

> > But it's not going to happen. So:
> > How about a flag, like -no-undefined ?
> > For example: -i-know-what-i-want-to-link-dont-interfere-please :-)
>
> --shoot-self-in-foot-with-unpredictable-consequences ?
>
> I've noticed that most of the time (but not always) when folks complain
> about libtool being too smart for its own good, they don't understand
> WHY libtool makes the decisions it does.  Sometimes, most of the time,
> libtool *really does know best*.  Or at least, libtool knows better than
> the wet-behind-the-ears developer who can't get it to act like she wants
> it to.

Ok - I concede the point that, most of the time, when libtool does
something unwanted, it's because no-one has bothered to teach it about the
peculiarities of the platform.

> I trust that you will not bring up those numerous occasions where *I*
> have complained about libtool being too smart.  Thanks. <g>
>
> >>All four alternatives suck.  #4 is the worst; it won't happen.  #2
> >>won't help mingw.  That leaves #1 and #3 -- and I hate kludges.  How
> >>important is this?  Is "punting" really such a bad idea?
> >
> > Punting is acceptable, if necessary. What do you think about my flag idea?
>
> It's just another way around the rules -- and these are rules I happen
> to agree with. I'd rather put special-case in there specifically for a
> small set of [4!] known libs, because I know those won't be abused.
> "Gee it's too hard to build libsilly as a dll, I'll just link my
> cygsillier.dll against libsilly.a, and use Max's --shoot-self-in-foot
> flag..."

Ok. Makes sense. Maybe I'll get round to making patch for this.

Max.



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Max Bowsher]

On Mon, 17 Feb 2003, Charles Wilson wrote:

> Max Bowsher wrote:
>
> >
> > Neither will work. The compiler is already perfectly happy. libtool decides
> > it knows better, though, and intervenes, breaking the build. I wouldn't
> > worry too much about this now. There's already a hardcoded
> > sys_lib_search_path_spec for cygwin. It only affects building for mingw in a
> > cygwin environment.
>
> -mno-cygwin, in other words.  So "native" (e.g. MSYS) mingw builds are
> not affected by this?

I wouldn't know. I'm too happy with the Cygwin environment to bother
trying!

> > About the only solution I can think of is to test $CC for -mno-cygwin, and
> > use the cygwin sys_lib_search_path_spec if found.
>
> That's not so bad. Although, it should NOT use cygwin's
> sys_lib_search_path_spec -- you'd pull in all of the cygwin-dependent
> libz's and libncurses's.  What you want is to ADD /usr/lib/w32api to the
> "standard" 'gcc -mno-cygwin' search path.  That "standard" path already
> includes the right gcc runtime directory, and already includes
> /usr/lib/mingw via some symlinks.  You just need to add
> -L/usr/lib/w32api -- we know that nothing in there is cygwin-dependent.
>
> I believe there are a few checks for -mno-cygwin already sprinkled
> around in the code.  Wanna submit a patch as outlined above?

Why not? As soon as I have some free time :-)
(Maybe in a couple of weeks or so)


Max.



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Max Bowsher]

Charles Wilson wrote:
>> Ralf Habacker wrote:
>>>>> BTW: Do you know which libraries are also hybrid execpt of
>>>>> cygwin1.dll ?
>>>
>>>
>>>> There are about a half-dozen in /usr/lib/w32api -- and worse, the
>>>> static
>>>
>>> members are "bad" variable  types; if you make the static members
>>> part of the DLL, then these vars can't be auto-imported without
>>> using pseudo-relocs. Of course, since the DLLs are provided by MS,
>>> we can't really modify what is in them.
>>>
>>> Thats very bad.
>>>
>>
>> Yeah.  I don't know why these implibs need to declare these static
>> structures; it's possible that w32api is just following the lead of
>> the corresponding .lib files in the MSVC distribution.
>>
>> But, that's neither here nor there.  IF these crossbreed implibs are
                                                 ^^^^^^^^^^^^^^^^^^
libuuid.a at least is static *only* - not crossbreed. So there really is no
way for libtool know to allow it except by name.

>> detected as "import libraries" then libtool will let you link a
>> sharedlib using them as dependencies.

Max.


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Charles Wilson]

[-- Attachment #1: Type: text/plain, Size: 5011 bytes --]

Ralf Habacker wrote:
>>>BTW: Do you know which libraries are also hybrid execpt of cygwin1.dll ?
> 
> 
>>There are about a half-dozen in /usr/lib/w32api -- and worse, the static
> 
> members are "bad" variable  types; if you make the static members part of the
> DLL, then these vars can't be auto-imported without using pseudo-relocs. Of
> course, since the DLLs are provided by MS, we can't really modify what is in
> them.
> 
> Thats very bad.
> 

Yeah.  I don't know why these implibs need to declare these static 
structures; it's possible that w32api is just following the lead of the 
corresponding .lib files in the MSVC distribution.

But, that's neither here nor there.  IF these crossbreed implibs are 
detected as "import libraries" then libtool will let you link a 
sharedlib using them as dependencies.  this could eventually lead to a 
data mismatch problem, if the static info is changed -- but since your 
DLL has a private copy of the old version...

Fortunately, this isn't likely.  The static data consists of GUID 
structures for well-known services/programs/etc on windows.  MS would be 
insane to change those definitions at this point.

> 
> There might be another (easier) way for identifiying. There are some resources
> in the internet relating to the ar format for examples
> http://publibn.boulder.ibm.com/doc_link/en_US/a_doc_lib/files/aixfiles/ar_IA64.h
> tm
> 
> After the ar header, there is a vector with pointers to each file. The first
> points to the first objectfile. The first bytes at the offset position is a
> string of the object file name, which is build
> for import library like d<6-digit-num>.o, which could be easily identified with
> one indirect file magic rule line and two addition simple lines.

[...]

>>(0x48l)  byte 0x64
>>
>>>(0x48l+7) leword 0x2e6f	import libray
> 
> 
> What dou you think ?

If I understand correctly, you look at the first object file in the 
archive, and check that the first letter of the name is 'd', and that 
the 8th and ninth letters are '.o'   I don't think that is sufficient...

> 
> The problem is, that I have tried this indirecting mode in several way, but
> without success. It seems to me that this functionality is broken in recent file
> release. Do anyone has got same experiences ?

...even if it works properly in well-controlled circumstances -- which 
apparently it doesn't.  And in poorly controlled circumstances, I could 
have an object file in my archive named "delayby.orig.o" and you 
couldn't distinguish it from "d000000.o".

the negative: Ralf, you keep trying to assume things based on filenames. 
  Filenames LIE.  Whether it is the name of the archive (foo.dll.a) or 
the name of an object in the archive (dxxxxxx.o), it's gonna fail -- and 
it will fail in EXTREMELY hard-to-track-down ways.

the positive: I'm glad you're following thru on this.  Here's a plan: I 
can make arrangements so that WHEN we find the Right Thing to do in 
file, libtool can take advantage of it immediately.  thus, we can get my 
speedups into libtool now along with hooks for your super-speedup.

Flowchart:

.  'file -L $fn' # e.g. if symlink, check the target
.  if file output has "DLL"
.    exit "x86 DLL"
.  elif file output has "executable"
.    if "MS Windows PE Intel" # to screen out "executable" shell scripts
.      exit "x86 DLL"
.    endif
.  elif file output has "ar archive import library"
.    exit "x86 archive import"
.  elif file output has "ar archive" # both static and import libs
.    if objdump output is good       # make sure it's an ar of x86 objs
.      if nm output has " I "        # fallback test for import libs
.        exit "x86 archive import"
.      else
.        exit "x86 archive static"
.      endif
.    endif
.  endif
.  exit "unknown"

This way, we get my speedups right away.  The "elif file output has "ar 
archive import library" isn't used right now (because stock 'file' won't 
ever generate that output string).  Thus, right now we use the "ar 
archive" section for both static libs and import libs.  No probs.

Later, your code gets into 'file' -- and suddenly we see big speedups 
for most linking against import libs.  Even better, YOU can see those 
speedups immediately using (the new upcoming 'stock' libtool), simply by 
changing your magic file.  Thus, it helps testing.  Even better, 
anything that FAILS your new file test, will get checked again using the 
slower code!

So, it's important, Ralf, that your 'file' changes NEVER generate a 
false positive (e.g. saying something is an import lib when it is not). 
  If your code generates a false negative (saying something is static 
when it's actually an import) -- because for false negatives, my slower 
code will catch it, and mark it "import".

Oh, and we MUST agree on the id string before hand.  You've suggested 
that import libs add 'import library' to the existing 'ar archive'; 
thus: 'ar archive import library'.  sounds good to me -- I'll go with 
that.  But don't change it.

sound good?

--Chuck

[-- Attachment #2: cygwin_libid --]
[-- Type: text/plain, Size: 993 bytes --]

#!/bin/sh
win32_libid () {
  win32_libid_type="unknown"
  win32_fileres=`file -L $1 2>/dev/null`
  case $win32_fileres in
  *ar\ archive\ import\ library*) # definitely import
    win32_libid_type="x86 archive import"
    ;;
  *ar\ archive*) # could be an import, or static
    if eval $OBJDUMP -f $1 | head -n 10 2>/dev/null | \
      grep -E 'file format pe-i386(.*architecture: i386)?' >/dev/null ; then
      win32_nmres=`eval $NM -f posix -A $1 | \
        sed -n -e '1,100{/ I /{x;/import/!{s/^/import/;h;p;};x;}}'`
      if [ "x$win32_nmres" = "ximport" ] ; then
        win32_libid_type="x86 archive import"
      else
        win32_libid_type="x86 archive static"
      fi
    fi
    ;;
  *DLL*) 
    win32_libid_type="x86 DLL"
    ;;
  *executable*) # but shell scripts are "executable" too...
    case $win32_fileres in
    *MS\ Windows\ PE\ Intel*)
      win32_libid_type="x86 DLL"
      ;;
    esac
    ;;
  esac
  echo $win32_libid_type
}

OBJDUMP=objdump
NM=nm
win32_libid $1



[-- Attachment #3: Type: text/plain, Size: 214 bytes --]

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Ralf Habacker]

>convenience libs do not count. You can still link a DLL with convenience libs,
because it is assumed that a true convenience lib is built by your project, for
your project, and only for your project -- it is not available to "outside
users" and therefore there can never be any mismatch between the symbols
provided by (part of) the DLL and those provided by the "real" static library.

>The prohibition is on OUTSIDE static dependencies. For instance, suppose you
only have libz.a. Now, you >build cygkde.dll (or libkde.so on some unixoid
platform) which depends on libz.a. Now, if I build chuckclient.exe which depends
on the kde shared lib, and on -lz, I could possibly get a symbol conflict.
>[This is actually more of an issue if I were trying to build chucklib.dll]

> So, the libtool folks prohibited this behavior (for this reason, and also
because it plays havoc with libtool's attempt to keep track of, via libfoo.la,
the dependencies of each created sharedlib).

> But don't worry about convenience libs; those are fine.

Thanks for this hints.
After some analyse afterwards I recognized, that the original kde libtool stuff
contains a bug in building convenience libraries. It does not include any object
files in special cases of using libtool flags, which let me go on a wrong way.

Regards
Ralf


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Ralf Habacker]

>>BTW: Do you know which libraries are also hybrid execpt of cygwin1.dll ?

>There are about a half-dozen in /usr/lib/w32api -- and worse, the static
members are "bad" variable  types; if you make the static members part of the
DLL, then these vars can't be auto-imported without using pseudo-relocs. Of
course, since the DLLs are provided by MS, we can't really modify what is in
them.

Thats very bad.

>So these "extra bits" probably need to *stay* static, and appended to the end
of the import lib...but, because they are (currently) appended to the end of the
importlib portion, your code will get it "right'.


>>Question: for "normal" import libs (that is, excluding the hybrids like
>>libcygwin.a), does your version work always? Or does _dll_iname 'float
>>around' even within otherwise normal import libs?

<snip>


>>That mean, we onbly have to figure out the relative pointer to the 'dll_iname'
>>string.
>>When I have time, I will look into the coff file format or is someone else
here,
>>who can give a pointer to this ?

There might be another (easier) way for identifiying. There are some resources
in the internet relating to the ar format for examples
http://publibn.boulder.ibm.com/doc_link/en_US/a_doc_lib/files/aixfiles/ar_IA64.h
tm

After the ar header, there is a vector with pointers to each file. The first
points to the first objectfile. The first bytes at the offset position is a
string of the object file name, which is build
for import library like d<6-digit-num>.o, which could be easily identified with
one indirect file magic rule line and two addition simple lines.

$ dump /usr/lib/libz.dll.a
/usr/lib/libz.dll.a:

| ar header
|  Addr     0 1  2 3  4 5  6 7  8 9  A B  C D  E F 0 2 4 6 8 A C E
|--------  ---- ---- ---- ---- ---- ---- ---- ---- ----------------
|00000000  213c 6172 6368 3e0a 2f20 2020 2020 2020 !<arch>./
|00000010  2020 2020 2020 2020 3130 3135 3930 3739         10159079
|00000020  3138 2020 3020 2020 2020 3020 2020 2020 18  0     0
|00000030  3020 2020 2020 2020 3534 3132 2020 2020 0       5412
|00000040  2020 600a 0000 00fa


vector with objectfile offsets

                              0000 1568 0000 16a0   `....z...h...
                              ^^^^^^^^^ points to first object file.

00000050  0000 1820 0000 1820 0000 1820 0000 1a6c ... ... ... ...l
00000060  0000 1a6c 0000 1a6c 0000 1c9e 0000 1c9e ...l...l........


00001560  745f 636f 6465 0000 6430 3030 3039 342e t_code..d000094.
                              ^^^^^^^^^^^^^^^^^
00001570  6f2f 2020 2020 2020 3130 3135 3930 3739 o/      10159079

MagDir/archives
<snip>

>(0x48l)  byte 0x64
>>(0x48l+7) leword 0x2e6f	import libray

What dou you think ?

The problem is, that I have tried this indirecting mode in several way, but
without success. It seems to me that this functionality is broken in recent file
release. Do anyone has got same experiences ?


Regards
Ralf


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Charles Wilson]

Ralf Habacker wrote:

> This depends on how the hybrid lib is created. If it starts with the static
> object files, it would be identified as static, if it starts with the import
> library as import library.

right.

> BTW: Do you know which libraries are also hybrid execpt of cygwin1.dll ?

There are about a half-dozen in /usr/lib/w32api -- and worse, the static 
members are "bad" variable types; if you make the static members part of 
the DLL, then these vars can't be auto-imported without using 
pseudo-relocs.  Of course, since the DLLs are provided by MS, we can't 
really modify what is in them.

So these "extra bits" probably need to *stay* static, and appended to 
the end of the import lib...but, because they are (currently) appended 
to the end of the importlib portion, your code will get it "right'.

>>Question: for "normal" import libs (that is, excluding the hybrids like
>>libcygwin.a), does your version work always? Or does _dll_iname 'float
>>around' even within otherwise normal import libs?
>>

<snip>

> That mean, we onbly have to figure out the relative pointer to the 'dll_iname'
> string.
> When I have time, I will look into the coff file format or is someone else here,
> who can give a pointer to this ?

Can't help you there.  I'd hunt around in the binutils/bfd 
sources...because you're looking in an 'ar archive' for the first bfd, 
and the in that bfd for the string "_dll_iname"

--Chuck


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: [avail for test] libtool-devel-20030121-1

[Ralf Habacker]

> But, you'd still need to check all "static libs" to see if they are
> really import libs after all.  The good news it that we expect this to
> happen only rarely: when an import lib is a "hybrid" lib with static
> objs "in front" --> the modified 'file' test incorrectly identifies it
> as a static archive.

This depends on how the hybrid lib is created. If it starts with the static
object files, it would be identified as static, if it starts with the import
library as import library.

BTW: Do you know which libraries are also hybrid execpt of cygwin1.dll ?


> Question: for "normal" import libs (that is, excluding the hybrids like
> libcygwin.a), does your version work always? Or does _dll_iname 'float
> around' even within otherwise normal import libs?
>
'file' supports indirect offset generation. See $ man magic

       Indirect  offsets  are  of  the  form:
       ((x[.[bslBSL]][+-][y]).  The value of x is used as  an  offset  in  the
       file.  A  byte,  short  or long is read at that offset depending on the
       [bslBSL] type specifier. The capitalized types interpret the number  as
       a big endian value, whereas the small letter versions interpet the num-
       ber as a little endian value. To that number the value of  y  is  added
       and  the  result  is used as an offset in the file. The default type if
       one is not specified is long.

       Sometimes you do not know the exact  offset  as  this  depends  on  the
       length  of  preceding fields. You can specify an offset relative to the
       end of the last uplevel field (of course this may only be done for sub-
       level  tests,  i.e.  test beginning with > ). Such a relative offset is
       specified using & as a prefix to the offset.

That mean, we onbly have to figure out the relative pointer to the 'dll_iname'
string.
When I have time, I will look into the coff file format or is someone else here,
who can give a pointer to this ?

Ralf


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Charles Wilson]

Max Bowsher wrote:
>>>ARGH.  This defeats the whole purpose of the policy change -- and it
>>>is a policy change driven by the libtool development
> 
> 
> I've been hunting for the relevant threads on the libtool list - can't find
> them. Can anyone offer URLs or search terms that would help?

I think it was actually described in the code/comments/changelogs/NEWS 
or somesuch.  But if you're really curious, ask on the libtool list. 
I'm sure one of the PowersThatBe will clue us all in.

> I think that in some cases, libtool tries to do too much. I've run into a
> number of cases which would have just worked if libtool had passed the
> arguments to gcc without modification, but it insists on filtering stuff in
> complex ways. For example, gcc -print-search-dirs doesn't admit to searching
> /usr/lib/w32api, so any attempt to link with a w32api library when
> cross-compiling with -mno-cygwin (which means the Cygwin-specific hack in
> libtool.m4 doesn't get activated) fails. Now, admittedly, gcc is probably
> wrong in not showing all its search dirs correctly, but sometimes I wish
> libtool trusted the compiler more.

mebbe. But that's another discussion.

--Chuck



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Charles Wilson]

Ralf Habacker wrote:
>>ARGH.  This defeats the whole purpose of the policy change -- and it is
>>a policy change driven by the libtool development.  I don't want to
>>support a forked version of libtool that differs from mainline on a
>>basic policy issue.
>>
> 
> May be, but like Max has stated, I don't like to be forced to make every static
> lib as shared lib. This would break the whole kde build system, because often
> convenience librarys are build and assembled together into a dll. 

convenience libs do not count.  You can still link a DLL with 
convenience libs, because it is assumed that a true convenience lib is 
built by your project, for your project, and only for your project -- it 
is not available to "outside users" and therefore there can never be any 
mismatch between the symbols provided by (part of) the DLL and those 
provided by the "real" static library.

The prohibition is on OUTSIDE static dependencies.  For instance, 
suppose you only have libz.a.  Now, you build cygkde.dll (or libkde.so 
on some unixoid platform)  which depends on libz.a.  Now, if I build 
chuckclient.exe which depends on the kde shared lib, and on -lz, I could 
possibly get a symbol conflict.  [This is actually more of an issue if I 
were trying to build chucklib.dll]

So, the libtool folks prohibited this behavior (for this reason, and 
also because it plays havoc with libtool's attempt to keep track of, via 
libfoo.la, the dependencies of each created sharedlib).

But don't worry about convenience libs; those are fine.

--Chuck


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Max Bowsher]

Christopher Faylor wrote:
> gcc doesn't search /usr/lib/w32api.  ld does.  Since it's impossible
> for ld to *not* search /usr/lib/w32api, I don't know why this is an issue
> unless you've got a mismatch between gcc and ld.

It's an issue when cross-compiling cygwin -> mingw, using -mno-cygwin, since
libtool runs gcc -print-search-dirs, doesn't see /usr/lib/w32api there, so
when you try to link with a w32api library, libtool petulantly refuses to
link properly, because *it* thinks the w32api library won't be found.
Whereas, if it just trusted gcc/ld to do their jobs, there would be no
problem.

It's only not an issue when compiling cygwin -> cygwin because
sys_lib_search_path_spec is hardcoded in libtool for this case.


Max.


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Christopher Faylor]

On Mon, Feb 10, 2003 at 08:02:17PM -0000, Max Bowsher wrote:
>>> ARGH.  This defeats the whole purpose of the policy change -- and it
>>> is a policy change driven by the libtool development.
>
>I've been hunting for the relevant threads on the libtool list - can't find
>them. Can anyone offer URLs or search terms that would help?
>
>Ralf Habacker wrote:
>> May be, but like Max has stated, I don't like to be forced to make
>> every static lib as shared lib. This would break the whole kde build
>> system, because often convenience librarys are build and assembled
>> together into a dll. May be, that's the reason, why on linux and
>> other systems "pass_all" is used, because it overrides this police.
>
>I think that in some cases, libtool tries to do too much. I've run into a
>number of cases which would have just worked if libtool had passed the
>arguments to gcc without modification, but it insists on filtering stuff in
>complex ways. For example, gcc -print-search-dirs doesn't admit to searching
>/usr/lib/w32api, so any attempt to link with a w32api library when
>cross-compiling with -mno-cygwin (which means the Cygwin-specific hack in
>libtool.m4 doesn't get activated) fails. Now, admittedly, gcc is probably
>wrong in not showing all its search dirs correctly, but sometimes I wish
>libtool trusted the compiler more.

gcc doesn't search /usr/lib/w32api.  ld does.  Since it's impossible for
ld to *not* search /usr/lib/w32api, I don't know why this is an issue
unless you've got a mismatch between gcc and ld.

cgf

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Max Bowsher]

>> ARGH.  This defeats the whole purpose of the policy change -- and it
>> is a policy change driven by the libtool development.

I've been hunting for the relevant threads on the libtool list - can't find
them. Can anyone offer URLs or search terms that would help?

Ralf Habacker wrote:
> May be, but like Max has stated, I don't like to be forced to make
> every static lib as shared lib. This would break the whole kde build
> system, because often convenience librarys are build and assembled
> together into a dll. May be, that's the reason, why on linux and
> other systems "pass_all" is used, because it overrides this police.

I think that in some cases, libtool tries to do too much. I've run into a
number of cases which would have just worked if libtool had passed the
arguments to gcc without modification, but it insists on filtering stuff in
complex ways. For example, gcc -print-search-dirs doesn't admit to searching
/usr/lib/w32api, so any attempt to link with a w32api library when
cross-compiling with -mno-cygwin (which means the Cygwin-specific hack in
libtool.m4 doesn't get activated) fails. Now, admittedly, gcc is probably
wrong in not showing all its search dirs correctly, but sometimes I wish
libtool trusted the compiler more.



Max.


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: [avail for test] libtool-devel-20030121-1

[Ralf Habacker]

> ARGH.  This defeats the whole purpose of the policy change -- and it is
> a policy change driven by the libtool development.  I don't want to
> support a forked version of libtool that differs from mainline on a
> basic policy issue.
>
May be, but like Max has stated, I don't like to be forced to make every static
lib as shared lib. This would break the whole kde build system, because often
convenience librarys are build and assembled together into a dll. May be, that's
the reason, why on linux and other systems "pass_all" is used, because it
overrides this police.

	  case $linkmode in
	  lib)
	    if test "$deplibs_check_method" != pass_all; then
	      echo
	      echo "*** Warning: Trying to link with static lib archive $deplib."
	      echo "*** I have the capability to make that library automatically link
in when"
	      echo "*** you link to this library.  But I can only do this if you have
a"
	      echo "*** shared version of the library, which you do not appear to have"
	      echo "*** because the file extensions .$libext of this argument makes me
believe"
	      echo "*** that it is just a static archive that I should not used here."

Ralf



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Charles Wilson]

Ralf Habacker wrote:

>>1) x86 DLL (and EXE)
>>2) x86 archive static
>>3) x86 archive import
>>4) other
>>
>>Do NOT try to combine 2) and 3) -- which is what all of your suggestions
>>in this email would do.
>>
> 
> [1]
> What about extending the 'file' tool, which supports already 1, (2 combined with
> 3) and 4 ?
> You have written (and I have checked this too) that 'file's execution time is
> independend from the object file size, so this would speedup this detecion.

Now that's an idea, but we'd need to get it into the "real" file 
distribution -- otherwise, cross-compile builds won't work.

> I've taken a look into the file source and have seen the only thing I don't know
> is how to define a rule for a string with a variable fileoffset, because
> '_dll_iname' is, as far as i can see, one of major identifier of import
> libraries.

Yep. Believe it or not, I *did* look at this idea early on -- and you've 
hit on the reason I didn't pursue it.

> For an example how to add support with fixed file offset
> /usr/lib/libintl.dll.a):
> 
> $ diff -ubB archive.orig archive
> --- MagDir/archive.orig        2003-02-10 09:01:41.000000000 +0100
> +++ MagDir/archive     2003-02-10 09:00:36.000000000 +0100
> @@ -79,6 +79,8 @@
>  >0     beshort         3               - shared library module
>  >0     beshort         4               - debug break-pointed module
>  >0     beshort         5               - absolute code program module
> +>866   string          _dll_iname      import library
> +
>  0      string          \<ar>           System V Release 1 ar archive
>  0      string          =<ar>           archive
>  #

Sure -- but as you say, that only works for fixed offsets.  Now, 
**most** import libs will need only the fixed offset, especially if you 
hunt for _dll_iname instead of " I " in $NM's output as I was doing. 
This is a very good idea.

But, you'd still need to check all "static libs" to see if they are 
really import libs after all.  The good news it that we expect this to 
happen only rarely: when an import lib is a "hybrid" lib with static 
objs "in front" --> the modified 'file' test incorrectly identifies it 
as a static archive.

Question: for "normal" import libs (that is, excluding the hybrids like 
libcygwin.a), does your version work always? Or does _dll_iname 'float 
around' even within otherwise normal import libs?

Downside: we can't use this idea in libtool-1.5.  We have to wait until 
after your change (after suitable debugging, of course) makes it into 
not just OUR file distribution, but the "real" file distribution.

Summary: sounds good.  Please pursue, and once your modification has 
been accepted into 'file' and the upgrade has reasonable penetration on 
non-cygwin platforms, then we can *really* speed up win32_libid() -- 
maybe by libtool-1.5.1 or .2.  But for now, I think we'll have to stick 
with the moderate speedup that my proposal allows.

--Chuck



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: [avail for test] libtool-devel-20030121-1

[Ralf Habacker]

> Ain't gonna happen.  Find me a faster way to distinguish between x86
> import libs and x86 static libs (and random-other-architecture libs of
> any sort), and I'll thank you.
>
see [1]

> But telling me that I must support a fundamental philosophical and not
> merely technical fork of libtool for the foreseable future is NOT an option.
>
> As I see it, you have two problems:
>    1) my detection code is too slow for your taste, and
>    2) that very detection sometimes causes your build to fail, because
> you are trying to build dynlibs with static dependencies.

right,

> 1) x86 DLL (and EXE)
> 2) x86 archive static
> 3) x86 archive import
> 4) other
>
> Do NOT try to combine 2) and 3) -- which is what all of your suggestions
> in this email would do.
>
[1]
What about extending the 'file' tool, which supports already 1, (2 combined with
3) and 4 ?
You have written (and I have checked this too) that 'file's execution time is
independend from the object file size, so this would speedup this detecion.

I've taken a look into the file source and have seen the only thing I don't know
is how to define a rule for a string with a variable fileoffset, because
'_dll_iname' is, as far as i can see, one of major identifier of import
libraries.

For an example how to add support with fixed file offset
/usr/lib/libintl.dll.a):

$ diff -ubB archive.orig archive
--- MagDir/archive.orig        2003-02-10 09:01:41.000000000 +0100
+++ MagDir/archive     2003-02-10 09:00:36.000000000 +0100
@@ -79,6 +79,8 @@
 >0     beshort         3               - shared library module
 >0     beshort         4               - debug break-pointed module
 >0     beshort         5               - absolute code program module
+>866   string          _dll_iname      import library
+
 0      string          \<ar>           System V Release 1 ar archive
 0      string          =<ar>           archive
 #

the relating example:

$ file -m ./magic /usr/lib/libintl.dll.a
/usr/lib/libintl.dll.a: current ar archive import library

$ file -m ./magic /usr/lib/libintl.a
/usr/lib/libintl.a: current ar archive


Ralf


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Max Bowsher]

Charles Wilson wrote:
> Max Bowsher wrote:
>> This seems like a good time to mention that I ran into this problem
>> building gtk+ (or glib), I forget. It wanted -luuid, but -luuid is a
>> static archive, which libtool doesn't currently like. I had to hack
>> libtool as Ralf mentions above to get it to work.
>
> ARGH.  That defeats the whole purpose of the *POLICY* change in
>   libtool. I do NOT want to be in the business of supporting a forked
> version of
> libtool that disagrees with mainline on a *fundamental* policy issue.
>
> ** you can't build shared libraries that depend on static libraries,
> when using a "modern" libtool (e.g. HEAD CVS, pre-1.5) **

Nor would I expect you to. However, I might try to change the mainline's
mind on this at some point, once I've studied the issue more thoughroughly.

> If you have a problem with the policy, the place to fix it is NOT to
> hack up libtool.  Instead, get shared versions of your dependencies.

Even when making a library shared is 10 parts overhead for 1 part
usefulness? It's not libtool's place to force platforms to change. However,
neither is it the libtool mainline maintainer's place to go out and write
code to deal with every little foible of every platform.

I will research the policy descision in the libtool ML archives, and see if
I can come up with a good solution.



Max.


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Charles Wilson]

Ralf Habacker wrote:
>>Ralf -- please drop my 'final' script into one of your generated
>>libtools and run your tests (what? rebuilding KDE?) on it, and let me
>>know what you think.  (Oh, and since (a) 'file' execution speed is
>>invariant with target size, and (b) we match *DLL* and *executable*
>>separately, if you are linking directly to DLLs -- as I believe Ralf's
>>KDE build does -- then my version is almost as fast (<1% difference) as
>>Ralf's "check the name of the file only" version)
>>
> 
> Chuck, this script does not work with original libtool 1.4e, which is provided
> with kde.
> It hangs on the last line, see below:

> + grep -E ^x86 archive import|^x86 DLL

"grep" hangs?  That's truly bizarre.

But, you'd need to relibtoolize the whole KDE tree with a modern version 
of libtool, as I describe in the other message.  This particular test 
you have done is not helpful (but that's my fault -- I'm sorry I 
suggested "kde" as a test base.  I had thought you were *already* 
relibtoolizing with modern libtool; I did not realize you were building 
KDE using the KDE-supplied, 1 year old version of libtool.)

> Then I've taken a recent libtool from www.cygwin.com, build my profiler lib with
> this (cvs dir tools/profiler from  http://kde-cygwin.sf.net) and copied this
> libtool into the kde source tree.

Now that's better, but...not quite.  See, there have also been changes 
in libtool.m4 -- which means that after you run aclocal and autoconf, 
your configure script is different.  It sets different variables, it 
sets the same variables to possibly different values, etc etc.

You really have to re-libtoolize the *actual* tree you are building.

> The results for makeing for example kdecore:
> 
> pass_all: 40 sec from make start until the compile command line comes up.
> 
> file_magic (win32_libid): 50 sec from make start until the ar(!) command line
> comes up. The problem I've got with this is that I can't build a shared library.
> Instead I've got some errors.

I believe this is because of the libtool.m4 --> configure script changes 
that you did not pick up, using your method of snarfing a different 
project's prebuilt libtool.

> The only way I see to fix it is to add static archives to deplibs_check_method:
>  deplibs_check_method="file_magic ^x86 archive import|^x86 DLL|^x86 archive
> static"

ARGH.  This defeats the whole purpose of the policy change -- and it is 
a policy change driven by the libtool development.  I don't want to 
support a forked version of libtool that differs from mainline on a 
basic policy issue.

Not gonna happen.  See my reply to Max.

> but
> 1b. this can be reached with a much easier way using the 'file' command:
> 
> deplibs_check_method="file_magic DLL|archive"
> file_magic_cmd="file"> 
> This needs equal time as "pass_all" (40 sec from make start until the link
> command)

Again, you're just saying "pass_all" by another name.  You avoid the 
(untested) codepaths within libtool this way, but you're still reverting 
the official libtool policy.

And if you think about it long enough, you'll probably agree that the 
libtool folks' decision to prevent dynlibs depending on statlibs is a 
GOOD thing.

> Chuck, what kind of advantage do you see in win32_libid against this.
> win32_libid makes this stuff more complicated as it is (see 1.), so I vote for
> skipping the win32_libid command complety and using the 'file' command. It seems
> obsolate to me.
> I'm sorry, that you may be frustrated about the work which is already done, we
> can learn from this: Do not make things complexer as they are. :-)

"For every complex problem there is an answer that is clear, simple, and 
wrong." -H.L. Mencken

The big slowdown in win32_libid() is using objdump and nm to help 
determine that a given "foo.a" file is (1) an archive, (2) an archive of 
x86 objects, and (3a) an archive of x86 IMPORT objects, or (3b) and 
archive of x86 STATIC objects.

You are trying to argue that we don't really need to distinguish between 
(3a) and (3b) -- so just drop the whole $objdump and $nm thing.  BUT 
THAT IS NOT POSSIBLE -- unless we want to VIOLATE the policy: Thou shalt 
not create dynamic libraries that have static dependencies.

Ain't gonna happen.  Find me a faster way to distinguish between x86 
import libs and x86 static libs (and random-other-architecture libs of 
any sort), and I'll thank you.

But telling me that I must support a fundamental philosophical and not 
merely technical fork of libtool for the foreseable future is NOT an option.

As I see it, you have two problems:
   1) my detection code is too slow for your taste, and
   2) that very detection sometimes causes your build to fail, because 
you are trying to build dynlibs with static dependencies.

So, you have two reasons to try to get win32_libid() OUT, or replace 
file_magic with pass_all, or whatever.  Unfortunately, (2) is NOT my 
problem.  You want to build dynamic libraries?  Make sure all of your 
dependencies are dynamic.  You want win32_libid() to go faster?  Great, 
me too -- but don't optimize it into a no-op.  It has a purpose.  It 
needs to perform that purpose -- which is classifying objects into these 
four categories (not three, not two, FOUR):

1) x86 DLL (and EXE)
2) x86 archive static
3) x86 archive import
4) other

Do NOT try to combine 2) and 3) -- which is what all of your suggestions 
in this email would do.

--Chuck


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Charles Wilson]

Ralf Habacker wrote:
>>1.4e isn't a specific version. It just means "some cvs checkout after the
>>1.4d release"
> 
> 
> but this could be a long period: :-)
> 
> The recent devel libtool tells:
> $ libtool --version
> ltmain.sh (GNU libtool) 1.4e (1.1175 2003/01/01 01:57:46)
> 
> ltmain of recent kde from anoncvs.kde.org: 
> VERSION=1.4e
> TIMESTAMP=" (1.1090 2002/02/07 19:54:36)"

So, it's from CVS HEAD -- but doesn't include these cygwin changes 
(duplicates indicate multiple same-day commits):

2002-02-09  Gary V. Vaughan  <gary@gnu.org>
   From Robert Collins  <robert.collins@itdomain.com.au>  < * >
2002-05-30  Charles Wilson  <cwilson@ece.gatech.edu>
2002-05-31  Charles Wilson  <cwilson@ece.gatech.edu>
2002-10-15  Charles Wilson  <cwilson@ece.gatech.edu>
2002-10-30  Charles Wilson  <cwilson@ece.gatech.edu>  < ** >
2002-11-03  Charles Wilson  <cwilson@ece.gatech.edu>
2002-11-03  Charles Wilson  <cwilson@ece.gatech.edu>
2002-11-18  Charles Wilson  <cwilson@ece.gatech.edu>
2002-12-30  Charles Wilson  <cwilson@ece.gatech.edu>
2002-12-30  Charles Wilson  <cwilson@ece.gatech.edu>
2003-01-28  Charles Wilson  <cwilson@ece.gatech.edu>

< * > This is when the auto-import support was first added.
< ** > this is when the shell function win32_libid() was first introduced.

And you wonder why simply dropping a new *version* of that shell 
function doesn't work in KDE?  You need to relibtoolize the whole tree 
with a more recent version of libtool, and THEN you can drop in my 
replacement test.

But, see my reply to your other message.

--Chuck



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Charles Wilson]

Max Bowsher wrote:
> 1.4e isn't a specific version. It just means "some cvs checkout after the
> 1.4d release"

Right.  But when?  And on which branch?  1.4-branch CVS, or HEAD CVS 
(which is to say, pre-1.5)?  Granted, libtool's CVS organization and 
branch naming could be better, but AFAIK KDE still uses a 1.4.x-style 
"1.4e" from the 1.4 branch, and not a 1.5-style "1.4e" from the HEAD branch.

> 
> This seems like a good time to mention that I ran into this problem building
> gtk+ (or glib), I forget. It wanted -luuid, but -luuid is a static archive,
> which libtool doesn't currently like. I had to hack libtool as Ralf mentions
> above to get it to work.

ARGH.  That defeats the whole purpose of the *POLICY* change in libtool. 
  I do NOT want to be in the business of supporting a forked version of 
libtool that disagrees with mainline on a *fundamental* policy issue.

** you can't build shared libraries that depend on static libraries, 
when using a "modern" libtool (e.g. HEAD CVS, pre-1.5) **

the only exception to this rule are the compiler runtimes, such as 
libgcc, libg++, libsupc++, libstdc++, libg2c, etc.

If you have a problem with the policy, the place to fix it is NOT to 
hack up libtool.  Instead, get shared versions of your dependencies. 
Here are all of the w32api libs that currently build as static archives, 
and are not simply import libs for Win32 dlls, and do not build shared:

libdxguid.a
liblargeint.a
libscrnsave.a
libscrnsavw.a
libuuid.a
libdinput.a

e.g. the EXTRA_LIBS in winsup/w32api/Makefile.in

EXTRA_LIBS=libuuid.a libscrnsave.a libscrnsavw.a libdxguid.a liblargeint.a

Now, scratch your itch: you have a problem with libuuid.a; figure out 
how to build it as an dynamic library (hint:

gcc -shared -Wl,--export-all -Wl,--out-implib=libuuid.dll.a -o 
cyguuid-0.dll uuid.o

works) but you'll have to link your client using 
--enable-runtime-pseudo-reloc.  Eventually this will become the default 
for ld on cygwin I hope, but isn't yet -- and currently it is hard to 
pass compiler and linker flags like this thru libtool.

I'm not sure what the best workaround is right now; perhaps it is time 
to push for --enable-runtime-pseudo-reloc as the default on cygwin ( but 
  not mingw )?

--Chuck

P.S. the problem must have been in gtk, because I didn't have that 
problem compiling glib-2.2.0.

P.P.S. cursory inspection shows that

   largeint.c is the complete contents of liblargeint.a, and includes 
"bad" data types -- if you build this as a DLL, you'll need to use 
--pseudo-relocs to link against it.

   dxguid.c is the complete contents of libxguid.a.  --pseudo-relocs needed.

   ditto dinput.c, libdinput.a

   scrnsave.c is the complete contents of BOTH libscrnsave.a AND 
libscrnsavw.a.  However, I believe that all public symbols in scrnsave.c 
are "good" data types; this should be easily dll-izable.

  shell32.c contains "bad" data types, but it is not the entire contents 
of libshell32.a.  libshell32.a is created as an import lib for the 
MS-provide shell32.dll, but then shell32.o (created from shell32.c) is 
appended.  So, libshell32.a is a "hybrid" like libcygwin.a -- but it 
will be detected as an import lib.  So there's no need to "dllize" this. 
   ('course, philosophically, I dunno if this structure is a good idea. 
  Every program linked against -lshell32 will get its own private copy 
of the data provided by shell32.c -- what if it changes? But making 
those objects shared is bad -- because they are "bad" data types, 
leading to the --psuedo-reloc problem)

   kernel32.c provides functions only, and is appended to libkernel32.a. 
  No probs here.


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: [avail for test] libtool-devel-20030121-1

[Ralf Habacker]

> 1.4e isn't a specific version. It just means "some cvs checkout after the
> 1.4d release"

but this could be a long period: :-)

The recent devel libtool tells:
$ libtool --version
ltmain.sh (GNU libtool) 1.4e (1.1175 2003/01/01 01:57:46)

ltmain of recent kde from anoncvs.kde.org: 
VERSION=1.4e
TIMESTAMP=" (1.1090 2002/02/07 19:54:36)"

Ralf 




--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Max Bowsher]

Ralf Habacker wrote:
> Chuck, this script does not work with original libtool 1.4e

1.4e isn't a specific version. It just means "some cvs checkout after the
1.4d release"

> file_magic (win32_libid): 50 sec from make start until the ar(!)
> command line comes up. The problem I've got with this is that I can't
> build a shared library. Instead I've got some errors.
>
> 1.
> *** Warning: linker path does not have real file for library -lutil.
> *** I have the capability to make that library automatically link in
> when
> *** you link to this library.  But I can only do this if you have a
> *** shared version of the library, which you do not appear to have
> *** because I did check the linker path looking for a file starting
> *** with libutil and none of the candidates passed a file format test
> *** using a file magic. Last file checked: /usr/lib/libutil.a
> *** The inter-library dependencies that have been dropped here will be
> *** automatically added whenever a program is linked with this library
> *** or is declared to -dlopen it.
> /usr/lib/libutil.a is a nonlibtool static archive, which isn't
> catched by your script. This results into a linker fail with an
> "undefined reference" error, because a function of this lib is needed.
>
> The only way I see to fix it is to add static archives to
>  deplibs_check_method: deplibs_check_method="file_magic ^x86 archive
> import|^x86 DLL|^x86 archive static"

This seems like a good time to mention that I ran into this problem building
gtk+ (or glib), I forget. It wanted -luuid, but -luuid is a static archive,
which libtool doesn't currently like. I had to hack libtool as Ralf mentions
above to get it to work.


Max.


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: [avail for test] libtool-devel-20030121-1

[Ralf Habacker]

>  From 'info libtool':
>
>    `pass_all'
>       will pass everything without any checking.  This may work on
>       platforms in which code is position-independent by default and
>       inter-library dependencies are properly supported by the dynamic
>       linker, for example, on DEC OSF/1 3 and 4.
>
>  From ltmain.sh:
>
>     echo "*** Warning: Linking the shared library $output against the"
>     echo "*** static library $deplib is not portable!"
>
> and
>
>            if test "$alldeplibs" = yes &&
>               { test "$deplibs_check_method" = pass_all ||
>                 { test "$build_libtool_libs" = yes &&
>                   test -n "$library_names"; }; }; then
>              # We only need to search for static libraries
>              continue
>            fi
>
> which seems to imply that when building a shlib, we won't add the
> dependencies -lfoo -lbar to the link command, if libfoo and libbar are
> shared libs. But on cygwin/windows, all dependencies must be satisfied
> at link time...unlike ELF.
>
> In our case "position independent" is technically true for all object
> files -- -fpic does nothing.  However, it is STILL possible that one
> would compile code one way for inclusion within a shared library (e.g.
> -DBUILDING_DLL) and differently when making a static library.  There are
> still extant cases in current cygwin libraries where the DLL and the .a
> are *not* interchangable, since the client code must be compiled with
> -DBUILDING_STATIC or some such.  A relic of the "old way" when we had to
> define __declspec(xxxxxx).  So static libs and shared libs are not
> necessarily drop-in replacements for each other.
>
Thanks for this pointer. It seems to me, that you are much more deeper in the
libtool stuff as I am.

> So, interlibrary dependencies are not automatic.
>
> > cygwin* | mingw* )  	lt_cv_deplibs_check_method='pass_all'   # RH
> > cygwin* | mingw* )   	lt_cv_deplibs_check_method='file_magic'  # CW
>
> Which is not to say that it won't work to do it your way  -- IF and only
> IF you are (a) linking only to new-style, non-declspec()-using
> libraries, or (b) are linking only to libraries built (new-style) as
> part of your package.  E.g. KDE.
>
> In the future, it might be possible to use 'pass_all' on cygwin
> (assuming the point about 'dropping -lfoo -lbar' mentioned above doesn't
> apply) -- but I doubt that we'll ever get rid of some of the special cases.
>
For the answer see my next mail, which describes a way solving this all without
pass_all. :-)


> Worse, using pass_all means that a lot of different (read: untested on
> cygwin) codepaths are used, for all of the esoteric features of libtool
> like staticlinked -dlopen'ed modules, or dynamically linked modules that
> depend on other sharedlibs that are part of the same package, etc etc.
> As complex as KDE is, I doubt it really exercises ALL of the possibile
> permutations and uses of libtool.
>
> I presume that you have run the entire libtool test suite with your
> proposed change, and it passed all tests except for build_relink2 and
> quote?  If not, then, well...I don't have time to do your homework, and
> the current mechanism has had a three month shakedown period.  I expect
> libtool-1.5 within *days*.
>
> > Also in case you tell me that this is to late, see this for
> information purpose.
>
> Understood, but...
>
> > I can see from this, that major unix platforms use 'pass_all', so
> there couldn't
> > be so much issues.
>
> Ha ha ha ha ha ha hee hee hee hee.  Oh, it is to laugh.
>
> Ralf, cygwin is not unix.  cygwin is not linux.  cygwin is not solaris.
>   cygwin is not irix.  cygwin is cygwin.  Our runtime loader is crap.
> We don't have ld.so.conf.  We don't have ld.so.  We don't use ELF format
> for our shared libraries or object code.   We have Microsoft's
> bastardized implementation of a runtime loader, that has led to an
> industry curse-word: DLL HELL.  We have PE/COFF format object code.
>
> cygwin is different.  Just because it works one way on linux -- and
> *may* work the same way, in a narrow range of usages expected by KDE, on
> cygwin, doesn't mean that cygwin and linux/solaris/irix/etc are
> interchangable.
>
> I note that once again, rather than trying to help me speed up the
> function you complained about, by testing my various proposals, you are
> instead chasing down rabbit trails and wandering in the weeds -- and not
> presenting evidence that you HAVE, in fact, actually tested your own
> ideas, much less mine. (actually, I do assume that you have tested your
> ideas, but you haven't *told* me that you have done so.)
>
I can only say it again. See the next mail.

Ralf


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: [avail for test] libtool-devel-20030121-1

[Ralf Habacker]

> Ralf -- please drop my 'final' script into one of your generated
> libtools and run your tests (what? rebuilding KDE?) on it, and let me
> know what you think.  (Oh, and since (a) 'file' execution speed is
> invariant with target size, and (b) we match *DLL* and *executable*
> separately, if you are linking directly to DLLs -- as I believe Ralf's
> KDE build does -- then my version is almost as fast (<1% difference) as
> Ralf's "check the name of the file only" version)
>
Chuck, this script does not work with original libtool 1.4e, which is provided
with kde.
It hangs on the last line, see below:

+ newdeplibs= -L/usr/lib/w32api
+ expr -lbz2 : -l\(.*\)
+ name=bz2
+ test bz2 !=
+ test bz2 != 0
+ test Xyes = Xyes
+ test -n -lbz2
+ eval $echo "lib$name"
+ echo libbz2
+ libname=libbz2
+ ls /usr/lib/w32api/libbz2[.-]*
+ potential_libs=
+ ls /usr/lib/libbz2.a /usr/lib/libbz2.dll.a
+ potential_libs=/usr/lib/libbz2.a
/usr/lib/libbz2.dll.a
+ ls -lLd /usr/lib/libbz2.a
+ grep  ->
+ potlib=/usr/lib/libbz2.a
+ test -h /usr/lib/libbz2.a
+ eval win32_libid "$potlib"
+ /usr/bin/sed 10q
+ grep -E ^x86 archive import|^x86 DLL

Then I've taken a recent libtool from www.cygwin.com, build my profiler lib with
this (cvs dir tools/profiler from  http://kde-cygwin.sf.net) and copied this
libtool into the kde source tree.
The results for makeing for example kdecore:

pass_all: 40 sec from make start until the compile command line comes up.

file_magic (win32_libid): 50 sec from make start until the ar(!) command line
comes up. The problem I've got with this is that I can't build a shared library.
Instead I've got some errors.

1.
*** Warning: linker path does not have real file for library -lutil.
*** I have the capability to make that library automatically link in when
*** you link to this library.  But I can only do this if you have a
*** shared version of the library, which you do not appear to have
*** because I did check the linker path looking for a file starting
*** with libutil and none of the candidates passed a file format test
*** using a file magic. Last file checked: /usr/lib/libutil.a
*** The inter-library dependencies that have been dropped here will be
*** automatically added whenever a program is linked with this library
*** or is declared to -dlopen it.
/usr/lib/libutil.a is a nonlibtool static archive, which isn't catched by your
script. This results into a linker fail with an "undefined reference" error,
because a function of this lib is needed.

The only way I see to fix it is to add static archives to deplibs_check_method:
 deplibs_check_method="file_magic ^x86 archive import|^x86 DLL|^x86 archive
static"

but
1b. this can be reached with a much easier way using the 'file' command:

deplibs_check_method="file_magic DLL|archive"
file_magic_cmd="file"

This needs equal time as "pass_all" (40 sec from make start until the link
command)

If you need executables too use deplibs_check_method="file_magic
executable|DLL|archive"

Chuck, what kind of advantage do you see in win32_libid against this.
win32_libid makes this stuff more complicated as it is (see 1.), so I vote for
skipping the win32_libid command complety and using the 'file' command. It seems
obsolate to me.
I'm sorry, that you may be frustrated about the work which is already done, we
can learn from this: Do not make things complexer as they are. :-)

Ralf


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Charles Wilson]

 From 'info libtool':

   `pass_all'
      will pass everything without any checking.  This may work on
      platforms in which code is position-independent by default and
      inter-library dependencies are properly supported by the dynamic
      linker, for example, on DEC OSF/1 3 and 4.

 From ltmain.sh:

    echo "*** Warning: Linking the shared library $output against the"
    echo "*** static library $deplib is not portable!"

and

           if test "$alldeplibs" = yes &&
              { test "$deplibs_check_method" = pass_all ||
                { test "$build_libtool_libs" = yes &&
                  test -n "$library_names"; }; }; then
             # We only need to search for static libraries
             continue
           fi

which seems to imply that when building a shlib, we won't add the 
dependencies -lfoo -lbar to the link command, if libfoo and libbar are 
shared libs. But on cygwin/windows, all dependencies must be satisfied 
at link time...unlike ELF.

In our case "position independent" is technically true for all object 
files -- -fpic does nothing.  However, it is STILL possible that one 
would compile code one way for inclusion within a shared library (e.g. 
-DBUILDING_DLL) and differently when making a static library.  There are 
still extant cases in current cygwin libraries where the DLL and the .a 
are *not* interchangable, since the client code must be compiled with 
-DBUILDING_STATIC or some such.  A relic of the "old way" when we had to 
define __declspec(xxxxxx).  So static libs and shared libs are not 
necessarily drop-in replacements for each other.

So, interlibrary dependencies are not automatic.

> cygwin* | mingw* )  	lt_cv_deplibs_check_method='pass_all'   # RH
> cygwin* | mingw* )   	lt_cv_deplibs_check_method='file_magic'  # CW

Which is not to say that it won't work to do it your way  -- IF and only 
IF you are (a) linking only to new-style, non-declspec()-using 
libraries, or (b) are linking only to libraries built (new-style) as 
part of your package.  E.g. KDE.

In the future, it might be possible to use 'pass_all' on cygwin 
(assuming the point about 'dropping -lfoo -lbar' mentioned above doesn't 
apply) -- but I doubt that we'll ever get rid of some of the special cases.

Worse, using pass_all means that a lot of different (read: untested on 
cygwin) codepaths are used, for all of the esoteric features of libtool 
like staticlinked -dlopen'ed modules, or dynamically linked modules that 
depend on other sharedlibs that are part of the same package, etc etc. 
As complex as KDE is, I doubt it really exercises ALL of the possibile 
permutations and uses of libtool.

I presume that you have run the entire libtool test suite with your 
proposed change, and it passed all tests except for build_relink2 and 
quote?  If not, then, well...I don't have time to do your homework, and 
the current mechanism has had a three month shakedown period.  I expect 
libtool-1.5 within *days*.

> Also in case you tell me that this is to late, see this for information purpose.

Understood, but...

> I can see from this, that major unix platforms use 'pass_all', so there couldn't
> be so much issues.

Ha ha ha ha ha ha hee hee hee hee.  Oh, it is to laugh.

Ralf, cygwin is not unix.  cygwin is not linux.  cygwin is not solaris. 
  cygwin is not irix.  cygwin is cygwin.  Our runtime loader is crap. 
We don't have ld.so.conf.  We don't have ld.so.  We don't use ELF format 
for our shared libraries or object code.   We have Microsoft's 
bastardized implementation of a runtime loader, that has led to an 
industry curse-word: DLL HELL.  We have PE/COFF format object code.

cygwin is different.  Just because it works one way on linux -- and 
*may* work the same way, in a narrow range of usages expected by KDE, on 
cygwin, doesn't mean that cygwin and linux/solaris/irix/etc are 
interchangable.

I note that once again, rather than trying to help me speed up the 
function you complained about, by testing my various proposals, you are 
instead chasing down rabbit trails and wandering in the weeds -- and not 
presenting evidence that you HAVE, in fact, actually tested your own 
ideas, much less mine. (actually, I do assume that you have tested your 
ideas, but you haven't *told* me that you have done so.)

--Chuck


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: [avail for test] libtool-devel-20030121-1

[Ralf Habacker]

Hi Charles,

I've taken a deeper look into the libtool sources and found a hint relating to
the way other os do this checking, for example

libtool.m4.in

<snip>
# AC_DEPLIBS_CHECK_METHOD
<snip>

pass_all
========
gnu*)
irix5* | irix6* | nonstopux*)
linux*) (most hosts)
osf3* | osf4* | osf5*)
sco3.2v5*)
solaris*)
sysv5OpenUNIX8* | sysv5UnixWare7* | sysv5uw[[78]]* | unixware7* | sysv4*uw2*)
aix4* | aix5*)
beos)

file_magic
==========
bsdi4*)
pw32*)
darwin* | rhapsody*)
freebsd*)
hpux10.20* | hpux11*)
newos6*)
openbsd*)
sysv4 | sysv4.2uw2* | sysv4.3* | sysv5*)

match_pattern
netbsd*)

unknown
nto-qnx)


cygwin* | mingw* )  	lt_cv_deplibs_check_method='pass_all'   # RH
cygwin* | mingw* )   	lt_cv_deplibs_check_method='file_magic'  # CW

Also in case you tell me that this is to late, see this for information purpose.

I can see from this, that major unix platforms use 'pass_all', so there couldn't
be so much issues.

Ralf




--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Charles Wilson]

[-- Attachment #1: Type: text/plain, Size: 2049 bytes --]

Okay, using a more 'fair' metric on speed (don't throw a lot of 
'unknowns' at the libid() function; libtool presumably will only present 
things that IT believes are libraries or DLLs for identification. 
Unless the user mis-specifies something.  So, using THIS test routine, 
which only grabs .dll.a, .a, .dll, and .exe files:

time for fn in /usr/bin/*.exe /usr/bin/*.dll /usr/lib/*.a ; do
   echo $fn : `./cygwin_libid $fn`
done

I get the following results:

Current win32_libid():
real    3m25.480s
user    2m0.436s
sys     1m24.073s


best case, tweaked Ralf script:
real    1m59.535s
user    1m11.151s
sys     0m56.012s


best case, paranoid Chuck script:
real    2m13.261s
user    1m12.385s
sys     0m54.422s


That is, if current is a "100%", then Ralf's is a "58%" and my version 
is a "65%".

Here's what the extra 7% execution speed buys you: my version is never 
fooled by merely renaming the file.  It ALWAYS looks at the actual 
content and format of the file itself (or it will follow a symlink and 
look at the contents of the target).

How'd I speed it up, while still being paranoid like the original?  Try 
to spawn as few subsidiary processes as possible.  When possible, use 
sh's case command for simple regex matching, not grep -E.  Reduce one 
long chain of pipes and spawned programs to a single (complicated) sed 
command.  Use 'head' to discard (and not wait for) long extraneous outputs.

Some of these ideas were originally proposed by Ralf, but this version 
is sufficiently "mine" that I have no problem sumitting to the libtool 
folks.

Ralf -- please drop my 'final' script into one of your generated 
libtools and run your tests (what? rebuilding KDE?) on it, and let me 
know what you think.  (Oh, and since (a) 'file' execution speed is 
invariant with target size, and (b) we match *DLL* and *executable* 
separately, if you are linking directly to DLLs -- as I believe Ralf's 
KDE build does -- then my version is almost as fast (<1% difference) as 
Ralf's "check the name of the file only" version)

--Chuck

[-- Attachment #2: cygwin_libid_orig --]
[-- Type: application/x-java-applet, Size: 804 bytes --]

[-- Attachment #3: cygwin_libid_ralf_tweaked --]
[-- Type: application/x-java-applet, Size: 639 bytes --]

[-- Attachment #4: cygwin_libid_chuck_final --]
[-- Type: application/x-java-applet, Size: 847 bytes --]

[-- Attachment #5: Type: text/plain, Size: 214 bytes --]

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: [avail for test] libtool-devel-20030121-1

[Ralf Habacker]

> You never followed up on that.

Rebooting every time need very much time (about 10 minutes for me) and there is
an easier way to emulate that. I've written an applications which's allocate any
available memory. This removes all cached files' dll's and so one. I had similar
problems while inspecting the kde loading time and have read this tips somewhere
in the internet.

See http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/kde-cygwin/tools/fillmem/ for
further informations.

See in the task manager how many virtual memory you have and call fillmem with a
value near this limit (in Bytes).

PS: Does anyone have an idea, how to get the available virtual memory through an
api function ?

Ralf


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Charles Wilson]

Ralf, this is the last message in the thread you mentioned.

Charles Wilson said:
 > You've shown that win32_libid() as it is currently can take up to 5
 > seconds on a single "library", while most "libraries" can be
 > identified in 0.3-0.6 seconds.  But that doesn't address how much
 > longer win32_libid() adds to the total link time when building a
 > library.
 >
 > e.g. try this:
 >
 > You're linking libkdetext or somesuch.  Halt the build right as the
 > link command (/bin/sh ../libtool -mode=link ....) is printed.  Save
 > the command in a text file somewhere.
 >
 > ****REBOOT**** in order to clear your machine's memory/cache
 > whatever. Run the libtool -mode=link command (saved earlier) by hand,
 > with the current win32_libid() function -- and time it.
 >
 > ****REBOOT**** again.  Then, time the link command again (but edit
 > your prebuilt libtool script and replace win32_libid() with this:
 >
 > win32_libid () { echo="x86 archive import" }
 >
 > (e.g. a no-op.)  This will tell us exactly how much extra time is
 > added to a typical kde libtool-link command by the current definition
 > of win32_libid()  AND what percentage of the total link time that
 > reflects.
 >
 > What I'm getting at is: is this 5 seconds for the absolute worst case
 > dependency (compared to 0.3 for most of them) really that bad,
 > compared to the total time involved in the link.  How much does
 > absolute correctness (e.g. no prone-to-failure shortcuts) cost us?

You never followed up on that.  However,

I'm betting that we can get most of the speedup we need by adopting the 
  function below.  It's not quite as agressive as yours, but it makes 
fewer assumptions.  Results using the following snippet:

time for fn in /usr/lib/* /usr/bin/* ; do
   echo $fn : `cygwin_libid $fn`
done

are as follows.  I ran the test twice with each script and took the 
faster score.  Hopefully that accounts for cacheing issues.

old script:
real    4m59.249s
user    3m1.315s
sys     2m8.683s

Chuck's new script:
real    3m40.841s
user    2m1.821s
sys     1m36.813s

Ralf's proposal:
real    4m24.808s
user    2m38.784s
sys     1m52.503s

Ralf's proposal (with .exe fix):
real    3m17.747s
user    1m55.214s
sys     1m24.283s

Also, Ralf's could probably be sped up by using some of the 'head -n 
XXX' tricks in my version.

1801 files, 139MB.

Note that .exe's are marked as 'x86 DLL' in both scripts, because soon 
the patch which allows exe's to have export tables will go into 
binutils, and we ought to allow linking against them. And really, there 
is very little difference between .dlls and .exes.

The difference between this script and Ralf's proposal is that Ralf 
assumed that every file passed to the function was x86 and didn't check 
it.  Also, any file named *.dll.a was assumed to be an import lib 
without checking.  I believe that this loophole would tempt people to 
rename .a's as .dll.a's which would defeat the purpose of the entire 
system -- and would probably end up breaking tertiary programs.  (**)  I 
*don't* think folks will be tempted to rename .a's as .dll's, however. 
Also, Ralf's version (as proposed) marks .exe's as 'unknown' -- but 
that's easy to fix, and would have the side effect of improving Ralf's 
scores on my "benchmark".

The downside to my version is that this symlink:
   libfoo.dll.a -> ../bin/cygfoo.dll
will be marked "unknown".(*)  Ralf's version will mark it as 'x86 
archive import' -- which is acceptable given how the result is used, but 
isn't really correct.  The original script would correctly identify it 
as 'x86 DLL'.

(*) This could be fixed with some low-impact special case code (detect 
symlink, follow it, check the filename of the target), but I'll leave 
that as an excercise...

(**) The way libtool works, if cygA.dll depend on cygB.dll, then the 
flag '-lB' is automatically appended to the link command whenever I try 
to build C.exe which depends on cygA.dll.  But, if somebody broke the 
rules which building A.dll because they only had a static libB.a (and 
renamed it to libB.dll.a to fool libtool), the A.dll will re-export the 
symobls in libB.a.  And you don't want that.  But perhaps this simply 
falls under the heading of "the user is allowed to shoot themselves in 
the foot(head)".  But this is a mighty shiny and tempting bullet..."Hmm, 
I can't link my dll because I only have a static version of this 
dependency.  Hmmm...let me rename it!"

--Chuck

P.S. I'll be out of contact for a while, so "discuss". :-)

Chuck's new version.
##############################################

#!/bin/sh
# changes:
#   0) use extension to identify .dlls (user can then shoot
#      self in foot, but it is their own fault).  Upside:
#      much faster AND allows successful link against a
#      *symlink* to a dll; original code did not.
#   1) only call objdump and nm once, and only when necessary
#   2) only need the first 3 lines of $OBJDUMP output; keep 10
#      and throw away the rest.
#   3) don't call awk
#   4) only check to distinguish between static and import libs
#      within the first 100 lines of $NM output.  This seems safe.
#      Normal import libs have an " I " symbol on line 5--8.
#      libcygwin.a [a hybrid import/static lib] has it on line 32.
#      only truly pathological cases would appear static for 100
#      symbols before an import symbol appears...
#   5) 'file' is fast; use it first to decide that $1 is an archive
#      before using $OBJDUMP to insure x86 and / or $NM to distinguish
#      between static / import.
#   6) both .dll's and .exe's are marked as 'x86 DLL' -- because
#      they are, and because .exe's will soon be able to have export
#      tables, so we should allow linking to them.
win32_libid () {
   win32_libid_type="unknown"
   if echo $1 | grep -E "\.dll$|\.exe$" >/dev/null; then
     win32_libid_type="x86 DLL"
   else
     if eval file $1 2>/dev/null | \
       grep -E 'ar archive' >/dev/null; then
       if eval $OBJDUMP -f $1 | head -n 10 2>/dev/null | \
         grep -E 'file format pe-i386(.*architecture: i386)?' >/dev/null 
; then
         if eval $NM -f posix -A $1 | head -n 100 | grep " I " 
 >/dev/null ; then
           win32_libid_type="x86 archive import"
         else
           win32_libid_type="x86 archive static"
         fi
       fi
     fi
     echo $win32_libid_type
}

OBJDUMP=objdump
NM=nm
win32_libid $1
##############################################

Ralf's proposal
##############################################
#!/bin/sh
# changes:
#   1) only use nm for .a files
#
win32_libid () {
   win32_libid_type="unknown"
   if echo $1 | grep -E "\.dll$" >/dev/null; then
     win32_libid_type="x86 DLL"
   else
     if echo $1 | grep -E "\.dll.a$" >/dev/null; then
       win32_libid_type="x86 archive import"
     else
       if echo $1 | grep -E "\.a$" >/dev/null; then
         if eval $NM -f posix -A $1 | grep " I " >/dev/null ; then
           win32_libid_type="x86 archive import"
         else
           win32_libid_type="x86 archive static"
       	fi
       fi
     fi
   fi
   echo $win32_libid_type
}

OBJDUMP=objdump
NM=nm
win32_libid $1
##############################################




--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Charles Wilson]

Christopher Faylor wrote:

> 
> How hard could this all be??????????
> 
> Maybe I'll work on this in my "copious spare time".  Chuckle.  Snort.
> That's a good one.
> 
> Oh, and the GPL sucks.  Have I mentioned that?  Have I?  Huh?  Should
> I mention it again in case you missed it?

I can't decide whether to laugh or cry. :-P

--Chuck


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Christopher Faylor]

On Wed, Feb 05, 2003 at 10:07:45PM -0500, Charles Wilson wrote:
>It's not that you didn't chime in with your speedup patches.  Its that
>you apparently didn't bother to read the emails in the discussion, so
>that when you DID chime in, it was with (slightly) offtopic and
>(mostly) unhelpful at this late date.

But then there's a lot of that going around here this week...

It seems to me that libtool should be able to pregenerate a non-gpl,
statically linked cygwin dll version of tcl which will be buildable by
anyone and should automatically correct flaws in the top level configury
while modifying setup.exe to install only the dlls that you want.
libtool seems like the perfect tool for stabilizing cygwin releases
similar to Debian or fetchmail, too.

How hard could this all be??????????

Maybe I'll work on this in my "copious spare time".  Chuckle.  Snort.
That's a good one.

Oh, and the GPL sucks.  Have I mentioned that?  Have I?  Huh?  Should
I mention it again in case you missed it?

cgf

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Charles Wilson]

Ralf Habacker wrote:
> Chuck, about two month ago we had a private discussion about the file_magic
> stuff, where i have investigated about 7 hours to figure out a faster approach
> for identifing import library file types, because the implementation you have
> mentioned would increase the kde compiling time more than about 70 % (that means
> 6,8 hours instead 4 hours for a full recompile, which is inacceptable  for me).
> Probably you remember this stuff, if not I can send you the thread. So it seems
> to me, that you only like to get my confirmation about your solution and nothing
> else.

Not true.  As I remember, I was focused on fixing *brokenness* in CVS 
libtool.  I believe the thread to which you refer was concerned with the 
"cygwin has only static gcc/g++ runtime libs" bug, not the "libtool on 
cygwin rebuilds exe's unnecessarily" bug.  There were lots of bugs to 
choose from.

So, as I said, I wanted to confirm (back then) that my fix corrected the 
static runtime problem. You were complaining about speed -- and 
suggested improvements, for which I am grateful but judged the time was 
not right.  I wanted to squash all of the known bugs first, before 
introducing speedups -- especially speedups like yours that were 
deliberate "speed over pedantic correctness" optimizations.

Since iteration #6 of my (et. al.) fix for the rebuild-exe bug has 
finally been absorbed into CVS, I now consider all *major* outstanding 
bugs in cygwin libtool-devel squashed.  That means *now* is the time for 
speed improvements -- now we can introduce new exciting bugs.  And yes, 
Ralf, I did expect that you would raise that issue now.  I'm glad you 
did -- I want some sort of speedup fix to go into CVS -- but you did the 
coding.  Therefore you'll have to submit it and assign copyright, etc 
etc.  I can't.

Of course, I'll argue against your most aggressive version (I believe 
you had an aggressive one, and a moderate one) because IMO the 
aggressive patch was far too cavalier about false positives/negatives.

But now's the time (two months ago was not).  Post 'em to libtool-patches.

[snip]

> the auto-import-from-dll-stuff (probably you don't know this
> cause), so this may not be wath you have expected, but think about if this isn't
> also a good contribution.

Sure, I liked this.  I advocated heavily for its eventual inclusion in 
binutils.  It's possible that post-1.5, libtool might use the 
symlink/copy-of-dll as an import lib, in most cases.  But 1.5 is 
**imminent**.  There's not enough time for a major change like that to 
stabilize; besides, it's a platform issue as well as a libtool issue. 
That sort of thing should be decided not just by you, or me, or the 
libtool folks, but should be thoroughly hashed out on the cygwin and 
mingw lists.

> next month. not earlier, sorry. This is reason for the
> contribution right now.

That's fine.  It won't make it into 1.5, but 1.5.1 and friends will be 
coming along afterwards.  And hopefully without the two year delay like 
we saw between actively developed 1.4.x and 1.5...

> (kde3 for example libtool 1.4e, automake 1.5/1.6) release and so following the

I understand sticking with "real" libtool (but a CVS version of the 
1.4.x branch?  That's odd...) But I don't understand staying with the 
increasingly obsolete am-1.5 and am-1.6 releases.  Those crazy kde 
people...even gcc/binutils are finally coming into the 21st century...

> If you would have asked, I would have said:
> Please wait until I'm going to the kde3 build fix and than I can help. Sorry.

It's not that you didn't chime in with your speedup patches.  Its that 
you apparently didn't bother to read the emails in the discussion, so 
that when you DID chime in, it was with (slightly) offtopic and (mostly) 
unhelpful at this late date.

> which is named "overload". :-)

You got that right.  I'm in major overload mode right now.  Big time.

--Chuck


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: [avail for test] libtool-devel-20030121-1

[Ralf Habacker]

> Yes, Ralf, I know.  This is like the sixth iteration trying to solve
> this problem.  The very first attempt did what you suggest (only it is
> much much more complicated than you imply; you're overlooking a lot).
> However, that fix was unacceptable to the automake and libtool people.
> Hence, tries #2, #3, #4, #5, and this one.
>
> Please don't go over old ground, since you've been ignoring the onging
> discussion which I have CC:ed to the cygwin and mingw lists throughout.
>   I am not interested in doing it another way -- I've already done it
> three different ways (with various refinements, leading to a total of
> six attempts).  I am interested in bug reports and fixes for this method.
>
> And if I sound annoyed, I am: I expected better from you.  I would have
> thought that you, given your KDE work, would have been very interested
> in the *on-going* development -- and not content merely to sit back and
> ignore the whole thread, and then chime in after the fact with old ideas.
>

Chuck, about two month ago we had a private discussion about the file_magic
stuff, where i have investigated about 7 hours to figure out a faster approach
for identifing import library file types, because the implementation you have
mentioned would increase the kde compiling time more than about 70 % (that means
6,8 hours instead 4 hours for a full recompile, which is inacceptable  for me).
Probably you remember this stuff, if not I can send you the thread. So it seems
to me, that you only like to get my confirmation about your solution and nothing
else.

For example:
>I realize that ANY slowdown is painful, but at least these checks are
>not performed when linking (and relinking and relinking and ...)
>executables.  They only apply when building sharedlibs, and those happen
>only once per build.  Unlike exe's.  Sigh.

This may be true for small applications and with small number of dll's but not
for kde as it
contains about 200 dll's and 100 exe's. (see timing example above)

May be you'r right with your point of view, but this would throw back me very
much, so I had decided to stop this thread. (I'm using pass_all currently, it
works for me)

Instead I begun to think about how to solve this problem in a general way and
the result is the auto-import-from-dll-stuff (probably you don't know this
cause), so this may not be wath you have expected, but think about if this isn't
also a good contribution.

Additional you know that the libtool is very complicate and someone need very
deep knowledge about this stuff to handle with this. At that time I couldn't
investigate more time to enter the same knowledge level as you to give a
qualified answer.
In the last two weeks I've started fixing the kde3 build system and have to go
very deeply into libtool and friends, so now I'm feeling comfortable again with
this stuff and so I could give the above mentioned input and that probably for a
time windows of about the next month. not earlier, sorry. This is reason for the
contribution right now.

Third, I have stated more than one time, that kde uses an older libtool/automake
(kde3 for example libtool 1.4e, automake 1.5/1.6) release and so following the
on-going process makes not very much sense for me (because of limited time).
Instead I have to fix bugs in that releases and from the growing knowledge I can
give input.

Fourth: Chuck, the problem in the last two month was, you haven't asked, if I
have time for handling this, you only have started some threads and requested
like "... need input" and you have expected, that I have the libtool knowledge
all the days, which isn't true. If you would have asked, I would have said:
Please wait until I'm going to the kde3 build fix and than I can help. Sorry.

Fifth: I have never seen a man like you, who is able to write so very detailed
and deals so fast with such complicated stuff, that sometime a lamp goes on,
which is named "overload". :-)

Regards

Ralf



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[Charles Wilson]

ralf.habacker@freenet.de wrote:
> 
> There were several solutions:
>    1) Teach 'make' to only want 'foo' instead of 'foo.exe'.  There are 
> problems here -- this requires mucking with automake, which has the 
> potential to break non-libtool builds if not done carefully.
> 
> Do you have seen thatg ltmain.sh defines for cygwin the .exe extension ? Automake creates rules like 
> 
> <application>$(EXEEXT): ... 
> 
> so setting this exeext to an empty string catch this. 
> 
> 	case $host in
> 	  *cygwin*)
> 	    exeext=.exe
> 	    outputname=`echo $outputname|${SED} 's,.exe$,,'` ;;
> 	  *) exeext= ;;
> 	esac


Yes, Ralf, I know.  This is like the sixth iteration trying to solve 
this problem.  The very first attempt did what you suggest (only it is 
much much more complicated than you imply; you're overlooking a lot). 
However, that fix was unacceptable to the automake and libtool people. 
Hence, tries #2, #3, #4, #5, and this one.

Please don't go over old ground, since you've been ignoring the onging 
discussion which I have CC:ed to the cygwin and mingw lists throughout. 
  I am not interested in doing it another way -- I've already done it 
three different ways (with various refinements, leading to a total of 
six attempts).  I am interested in bug reports and fixes for this method.

And if I sound annoyed, I am: I expected better from you.  I would have 
thought that you, given your KDE work, would have been very interested 
in the *on-going* development -- and not content merely to sit back and 
ignore the whole thread, and then chime in after the fact with old ideas.

--Chuck



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [avail for test] libtool-devel-20030121-1

[ralf.habacker]

I've updated libtool-devel to the 20030121 CVS, plus added a major 
change of my own (inspired by Earnie Boyd and others on the libtool 
mailing list) that "fixes" the 'relink exe's over and over and over' 
problem on both mingw and cygwin.

[Skip to the last two paragraphs for the real important part]

A (not-so-short) description of the problem and the solution approach 
taken: if you have a package that builds a shared library (.dll) AND an 
exe which relies on that library, then libtool puts the actual 
executable into a .lib/ subdirectory under your current build directory, 
and NOT in the build directory itself.

This is also done on "real" unix platforms.  The reason is that 
uninstalled shared libraries cannot easily be found by the runtime 
loader on most platforms, unix and cygwin/windows included.

So, in order to run the uninstalled executable, you must first set the 
environment appropriately so that the shared library (.dll) will be 
found.  This means setting LD_LIBRARY_PATH on some unices, or setting 
PATH on cygwin so that the Windows Runtime Loader will find the .dll.

libtool does this by creating a shell script in the actual build 
directory.  The shell script sets the variables and then exec's 
./lib/my-real-executable.

If your application is "foo", this works fine on unix.  The makefile 
wants to see 'foo' -- and it does; only the 'foo' that make sees is 
actually a shell script:

   <builddir>/foo       : a shell script
   <builddir>/.lib/foo  : the real executable

And it only gets built once.  However, on cygwin/windows, you have
   <builddir>/foo           : a shell script
   <builddir>/.lib/foo.exe  : the real executable
and the makefile *wants* 'foo.exe' -- but only sees 'foo'.  Therefore, 
make assumes that the executable hasn't been created, and builds it 
again.  EVERY time you run 'make <anything>'.  Sometimes *multiple* 
times if there are cross-dependencies.

There were several solutions:
   1) Teach 'make' to only want 'foo' instead of 'foo.exe'.  There are 
problems here -- this requires mucking with automake, which has the 
potential to break non-libtool builds if not done carefully.

Do you have seen thatg ltmain.sh defines for cygwin the .exe extension ? Automake creates rules like 

<application>$(EXEEXT): ... 

so setting this exeext to an empty string catch this. 

	case $host in
	  *cygwin*)
	    exeext=.exe
	    outputname=`echo $outputname|${SED} 's,.exe$,,'` ;;
	  *) exeext= ;;
	esac

Ralf 





--
Singles aufgepasst!
Spielend in Kontakt kommen und neue Freunde finden in der freenet.de Community!
Jetzt durchstarten unter: http://www.freenet.de/tipp/community/