Re: ld --auto-imports

Re: ld --auto-imports

[Robert Collins]

----- Original Message -----
From: "Ralf Habacker" <Ralf.Habacker@saght.tessag.com>
To: "Robert Collins" <robert.collins@itdomain.com.au>
Sent: Monday, June 11, 2001 5:24 PM
Subject: AW: ld --auto-imports


> I have tried to mark this with ^^^^, but there are tabs in my mail, so
they
> don't fit the correct columns.
>
> > However it works with -lcygwin first in the link line. Do you know
why
> > that fixes it? Maybe more symbols need to be stripped... I'll roll
my
> > own and have a look and see why -lcygwin is needed (it's not needed
with
> > an unaltered ld).
> The reason is, that hello.dll uses _cygwin_dll_entry@12 from greet.dll
and
> not from cygwin1.dll. This doesn't work, but I don't know why.

Yes - I tracked that down just now myself :]. Use
-Wl,--exclude-symbols=_cygwin_dll_entry@12
 on the .dll link line, and you will no longer need -lcygwin in the link
command.

Thats fixed the segfaulting for me, and I'm back to libtool. I suspect
that there are a number of symbols that will need to be excluded from
automatic exporting, and I'll generate a patch to ld that will include
Pauls' patch, and exclude those extra symbols.

Thanks for the help,

Rob

>
> Ralf
>
> >
> > Rob
> >
> >
> >
> >
>
>

AW: ld --auto-imports

[Ralf Habacker]

> > I have tried to mark this with ^^^^, but there are tabs in my mail, so
> they
> > don't fit the correct columns.
> >
> > > However it works with -lcygwin first in the link line. Do you know
> why
> > > that fixes it? Maybe more symbols need to be stripped... I'll roll
> my
> > > own and have a look and see why -lcygwin is needed (it's not needed
> with
> > > an unaltered ld).
> > The reason is, that hello.dll uses _cygwin_dll_entry@12 from greet.dll
> and
> > not from cygwin1.dll. This doesn't work, but I don't know why.
>
> Yes - I tracked that down just now myself :]. Use
> -Wl,--exclude-symbols=_cygwin_dll_entry@12
>  on the .dll link line, and you will no longer need -lcygwin in the link
> command.
>
> Thats fixed the segfaulting for me, and I'm back to libtool. I suspect
> that there are a number of symbols that will need to be excluded from
> automatic exporting, and I'll generate a patch to ld that will include
> Pauls' patch, and exclude those extra symbols.

I have seens, that impure_ptr is although used in this way. Should this be
excluded too ? It seems, that is doesn't matter, but I'm not familiar with
the internal of cygwin1.dll

Ralf


>
> Thanks for the help,
>
> Rob
>
> >
> > Ralf
> >
> > >
> > > Rob
> > >
> > >
> > >
> > >
> >
> >
>
>

Re: ld --auto-imports

[Robert Collins]

----- Original Message -----
From: "Ralf Habacker" <Ralf.Habacker@saght.tessag.com>
To: "Binutils" <binutils@sources.redhat.com>
Cc: "Robert Collins" <robert.collins@itdomain.com.au>
Sent: Monday, June 11, 2001 6:07 PM
Subject: AW: ld --auto-imports


> > > I have tried to mark this with ^^^^, but there are tabs in my
mail, so
> > they
> > > don't fit the correct columns.
> > >
> > > > However it works with -lcygwin first in the link line. Do you
know
> > why
> > > > that fixes it? Maybe more symbols need to be stripped... I'll
roll
> > my
> > > > own and have a look and see why -lcygwin is needed (it's not
needed
> > with
> > > > an unaltered ld).
> > > The reason is, that hello.dll uses _cygwin_dll_entry@12 from
greet.dll
> > and
> > > not from cygwin1.dll. This doesn't work, but I don't know why.
> >
> > Yes - I tracked that down just now myself :]. Use
> > -Wl,--exclude-symbols=_cygwin_dll_entry@12
> >  on the .dll link line, and you will no longer need -lcygwin in the
link
> > command.
> >
> > Thats fixed the segfaulting for me, and I'm back to libtool. I
suspect
> > that there are a number of symbols that will need to be excluded
from
> > automatic exporting, and I'll generate a patch to ld that will
include
> > Pauls' patch, and exclude those extra symbols.
>
> I have seens, that impure_ptr is although used in this way. Should
this be
> excluded too ? It seems, that is doesn't matter, but I'm not familiar
with
> the internal of cygwin1.dll
>
> Ralf
>

_cygwin_crt0_common@8
_cygwin_noncygwin_dll_entry@12
_fmode
_impure_ptr
cygwin_attach_dll
cygwin_premain0
cygwin_premain1
cygwin_premain2
cygwin_premain3
environ

are also candidates for not being automatically exported.

I haven't looked into this with any depth yet. Once I've got libtool
passing the entire test suite, I'll be looking at this part of it more.

Rob

Re: ld --auto-imports

[Robert Collins]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 5121 bytes --]

----- Original Message -----
From: "Ralf Habacker" <Ralf.Habacker@saght.tessag.com>
To: "Binutils" <binutils@sources.redhat.com>
Cc: "Paul Sokolovsky" <paul-ml@is.lg.ua>; "Robert Collins"
<robert.collins@itdomain.com.au>
Sent: Monday, June 11, 2001 5:06 PM
Subject: AW: ld --auto-imports


> Hi Robert,
>
> > Hello Robert,
> >
> > âîñêðåñåíüå, 10 èþíÿ 2001 ã., you wrote to me:
> >
> > RC> Hi Paul,
> > RC>     You may not recognise my name - I hack on the cygwin1.dll
> > internals
> > RC> among other things. Ralf Habacker put me onto your auto-imports
> > RC> patch/hack for ld 2.11. I hope you don't mind me writing you
direct, I
> > RC> wasn't sure hwat list was appropriate. Feel free to redirect
> > me, or copy
> > RC> any reply to a list of your choice.
> >
> >     I guess binutils list is a place to discuss this, and it fact
I'd
> > appreciate such mails appearing there - likely, it won't be too easy
> > to get that patch included, so it would be nice if messages showing
> > that community reviews patch were sent there. I'm not subscribed to
> > it, however, so I'd appreciate cc: to me.
> >
> > RC> Your ld patch looks very nice, and I have a modified
libtool-HEAD here
> > RC> that using it rather than a 5 stage dllwrap process.
> >
> >     You may want to know that I did it quite some time ago, along
with
> > other useful changes. It is available at
> > http://sourceforge.net/projects/pw32/ . It worked OOB for me for
stuff
> > I tried it on (glib/gtk+ libs, for example). Ralf Habacker reported
it
> > worked ok for him with cygwin (even though package on the link above
> > built for my PW32).
> >
> > RC> I've a small bit of feedback/a question, and a more serious
question.
> >
> > RC> Firstly the feedback. I'm using a pre-compiled version from Ralf
> > RC> habacker. That version (and AFAIK it's a stock build) creates
the .dll
> > RC> base image at 0x610c0000 which collides with the cygwin1.dll.
Now the
> > RC> cygwin1.dll is theoretically relocatable, but we've just
> > found out that
> > RC> it doesn't like relocating :/. According to MSDN, the default
> > image base
>
> Why does the auto-import sample works under cygwin 1.1.8 and not under
> cygwin 1.3.2 ?
>
> > RC> for .dll's should be 0x10000000. Is there any reason your
modified ld
> > RC> couldn't set that as the default base address for .dlls? That
would
> > RC> remove the need to add --base-address=0x10000000 to the link
line in
> > RC> libtool.
> >
> >     Well:
> >
> > 1. As you understand, it is not connected with auto-import stuff in
> > any way. That's how Ralf Habacker built it.
> >
> > 2. Having specific base hardcoded is bad anyway - you'll get
> > collisions (automatically resolved, of course). Far better to
> > --enable-auto-image-base switch to ld and let it choose unique base
> > itself (based on hashing of dll name, etc.)
> >
> > 3. With my personal attitudes, ld on win32 should work as close to
> > normal unix version as possible. Mundane unix system doesn't have
any
> > idiosynacrasies around "relocatability" -
so, --enable-auto-image-base
> > should be the default as it offers most seamless solution to win32
> > strangeness. The same for --enable-auto-import . That's how I tried
to
> > do while maintaining binutild fow Mingw32. Well, new maintainer
> > doesn't share these ideas, so having that all in libtool is not
worst
> > solution ;-)
> >
> > RC> Now for the more serious question. Does your patch in theory
support
> > RC> dll's linking to other .dll's with auto-imports?
> >
> >     Of course, why not? I wouldn't claim I tested that, however.
> >
> > RC> I have a trivial hello
> > RC> world case here (built with my libtool, but I'm very sure the
symptoms
> > RC> will be the same without libtool) that segfaults _every time_.
That
> > RC> testcase has 3 source files, 2 libraries.
> > RC> hello.exe - depends on cyghello-0.dll
> > RC> cyghello-0.dll - depends on cyggreet-0.dll
> >
> > RC> cyghello-0.dll reads a string from cyggreet-0.dll, and writes it
+ ,
> > RC> World! to stdout.
>
> >
> > RC> Thanks for any suggestions you can offer. I'm happy to drop you
the
> > RC> trivial testcase, and/or prepare a non-libtool testcase that
shows the
> > RC> same symptoms - if that would help.
>
> I have provided another example with nested libs und
>
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/kde-cygwin/ld/auto-import
-nes
> ted-libs-sample/
> >
> >     Please provide one. I'd appeciate if it were not just mere
gcc/ld
> > based, but also wouldn't depend on cygwin (i.e. were built with
> > -mno-cygwin or standalone mingw).
> >
>
> There is a mistake in the makefile. Note the -lcygwin in the line
creating
> hello.dll. It must be, otherwise dll initialisation doesn*t work.

Thank you!.

I had other problems in the makefile I've just realised:
the --out-implib statements are wrong.

However it works with -lcygwin first in the link line. Do you know why
that fixes it? Maybe more symbols need to be stripped... I'll roll my
own and have a look and see why -lcygwin is needed (it's not needed with
an unaltered ld).

Rob

Re: ld --auto-imports

[Robert Collins]

----- Original Message -----
From: "Christopher Faylor" <cgf@redhat.com>
To: <binutils@sources.redhat.com>; <libtool@gnu.org>
Sent: Monday, June 11, 2001 6:11 AM
Subject: Re: ld --auto-imports


> On Sun, Jun 10, 2001 at 08:35:05PM +0300, Paul Sokolovsky wrote:
> >RC> Firstly the feedback. I'm using a pre-compiled version from Ralf
> >RC> habacker. That version (and AFAIK it's a stock build) creates the
.dll
> >RC> base image at 0x610c0000 which collides with the cygwin1.dll. Now
the
> >RC> cygwin1.dll is theoretically relocatable, but we've just found
out that
> >RC> it doesn't like relocating :/. According to MSDN, the default
image base
> >RC> for .dll's should be 0x10000000. Is there any reason your
modified ld
> >RC> couldn't set that as the default base address for .dlls? That
would
> >RC> remove the need to add --base-address=0x10000000 to the link line
in
> >RC> libtool.
>
> Before it makes it into the popular consciousness that "the cygwin DLL
is
> not relocatable", I'd like to point out that this is not strictly
true.
>
> The Cygwin DLL assumes that it will be located in the same place in
both
> the parent and the child when forking or execing.  Cygwin assumes that
> it will be able to allocate space for its heaps at the same location
in
> the parent and the child.
>
> This relies on undocumented, and potentially fragile behavior on
Windows.


It's spawn() that caused the behaviour I saw, and AFAICT will occur with
fork() as well. I've gone into this in more detail in the cygwin list.

And this behaviour _is fragile_: specifically there are two cases that
will break cygwin's assumption every single time.

1) The process calling spawn() has cygwin1.dll able to load at it's
preferred base address. The child process image has a different .dll
listed earlier in it's import table, this will cause the child process's
cygwin1.dll base to be different.
2) The process calling spawn() has cygwin1.dll relocated, and the child
process image loads cygwin1.dll at it's preferred base address.

> However, AFAIK, there is no reason why Cygwin couldn't be located at
any
> base besides 0x61000000 as long as it is loaded into the same location
> in the parent and the child.
>
> If it isn't the fork or exec behavior that is the issue here then this
> may be a bug.  If it is a bug then we can fix the bug.

So for the record:
"To the best of my knowledge Cygwin1.dll is 100% relocatable for any
single process."

However, I'm looking for a way to mark cygwin1.dll as non-relocatable to
prevent this issue occuring at all - does anyone here know the
appropriate flag to ld, or a .def file entry to achieve this?

There's more detail on this in the thread "dll base address" on the
cygwin developers list, if anyone wants the full story :]

Rob

Re: ld --auto-imports

[Robert Collins]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 4868 bytes --]

----- Original Message -----
From: "Paul Sokolovsky" <Paul.Sokolovsky@technologist.com>
To: "Robert Collins" <robert.collins@itdomain.com.au>
Cc: <binutils@sources.redhat.com>; <libtool@gnu.org>
Sent: Monday, June 11, 2001 3:35 AM
Subject: Re: ld --auto-imports


> Hello Robert,
>
> âîñêðåñåíüå, 10 èþíÿ 2001 ã., you wrote to me:
>
> RC> Hi Paul,
> RC>     You may not recognise my name - I hack on the cygwin1.dll
internals
> RC> among other things. Ralf Habacker put me onto your auto-imports
> RC> patch/hack for ld 2.11. I hope you don't mind me writing you
direct, I
> RC> wasn't sure hwat list was appropriate. Feel free to redirect me,
or copy
> RC> any reply to a list of your choice.
>
>     I guess binutils list is a place to discuss this, and it fact I'd
> appreciate such mails appearing there - likely, it won't be too easy
> to get that patch included, so it would be nice if messages showing
> that community reviews patch were sent there. I'm not subscribed to
> it, however, so I'd appreciate cc: to me.

I'm not subscribed to binutils either, so I too would appreciate that
courtesy from other repliers.

> RC> Your ld patch looks very nice, and I have a modified libtool-HEAD
here
> RC> that using it rather than a 5 stage dllwrap process.
>
>     You may want to know that I did it quite some time ago, along with
> other useful changes. It is available at
> http://sourceforge.net/projects/pw32/ . It worked OOB for me for stuff
> I tried it on (glib/gtk+ libs, for example). Ralf Habacker reported it
> worked ok for him with cygwin (even though package on the link above
> built for my PW32).

I wanted to push it's limits, because introducing it into libtool for
example requires that it has at least the same level of functionality
that libtool does. In fact it was Ralf's reporting his success on the
cygwin-xfree list that got me motivated :].

<SNIP background>
> RC> for .dll's should be 0x10000000. Is there any reason your modified
ld
> RC> couldn't set that as the default base address for .dlls? That
would
> RC> remove the need to add --base-address=0x10000000 to the link line
in
> RC> libtool.
>
>     Well:
>
> 1. As you understand, it is not connected with auto-import stuff in
> any way. That's how Ralf Habacker built it.

Is it? Ahh. Sounds like it's time for me to roll-my-own.

> 2. Having specific base hardcoded is bad anyway - you'll get
> collisions (automatically resolved, of course). Far better to
> --enable-auto-image-base switch to ld and let it choose unique base
> itself (based on hashing of dll name, etc.)

I tried that. ld still created an image base of 0x610c0000.

> 3. With my personal attitudes, ld on win32 should work as close to
> normal unix version as possible. Mundane unix system doesn't have any
> idiosynacrasies around "relocatability" - so, --enable-auto-image-base
> should be the default as it offers most seamless solution to win32
> strangeness. The same for --enable-auto-import . That's how I tried to
> do while maintaining binutild fow Mingw32. Well, new maintainer
> doesn't share these ideas, so having that all in libtool is not worst
> solution ;-)

I agree with you, please see me answer to 2 for a potential problem :].

> RC> Now for the more serious question. Does your patch in theory
support
> RC> dll's linking to other .dll's with auto-imports?
>
>     Of course, why not? I wouldn't claim I tested that, however.
>
> RC> I have a trivial hello
> RC> world case here (built with my libtool, but I'm very sure the
symptoms
> RC> will be the same without libtool) that segfaults _every time_.
That
> RC> testcase has 3 source files, 2 libraries.
> RC> hello.exe - depends on cyghello-0.dll
> RC> cyghello-0.dll - depends on cyggreet-0.dll
>
> RC> cyghello-0.dll reads a string from cyggreet-0.dll, and writes it +
,
> RC> World! to stdout.
>
> RC> Thanks for any suggestions you can offer. I'm happy to drop you
the
> RC> trivial testcase, and/or prepare a non-libtool testcase that shows
the
> RC> same symptoms - if that would help.
>
>     Please provide one. I'd appeciate if it were not just mere gcc/ld
> based, but also wouldn't depend on cygwin (i.e. were built with
> -mno-cygwin or standalone mingw).

It's 200 kb, because I've included the compiled version, in case you get
different results when building and or linking. As such, I've put it
http://users.bigpond.net.au/robertc/testcase.tgz so that anyone involved
in this discussion can grab it for review. It's derived from the goat
book's hello test case.

The testcase is bare bones: no libtool, autoconf, automake. If you can
identify the issue, and a workaround, I'm happy to put in the elbow
grease to get libtool doing that.

Thanks!
Rob

>
> RC> Rob
>
>
>
>
> --
> Paul Sokolovsky, IT Specialist
> http://www.brainbench.com/transcript.jsp?pid=11135
>
>
>

Re: ld --auto-imports

[Christopher Faylor]

On Sun, Jun 10, 2001 at 08:35:05PM +0300, Paul Sokolovsky wrote:
>RC> Firstly the feedback. I'm using a pre-compiled version from Ralf
>RC> habacker. That version (and AFAIK it's a stock build) creates the .dll
>RC> base image at 0x610c0000 which collides with the cygwin1.dll. Now the
>RC> cygwin1.dll is theoretically relocatable, but we've just found out that
>RC> it doesn't like relocating :/. According to MSDN, the default image base
>RC> for .dll's should be 0x10000000. Is there any reason your modified ld
>RC> couldn't set that as the default base address for .dlls? That would
>RC> remove the need to add --base-address=0x10000000 to the link line in
>RC> libtool.

Before it makes it into the popular consciousness that "the cygwin DLL is
not relocatable", I'd like to point out that this is not strictly true.

The Cygwin DLL assumes that it will be located in the same place in both
the parent and the child when forking or execing.  Cygwin assumes that
it will be able to allocate space for its heaps at the same location in
the parent and the child.

This relies on undocumented, and potentially fragile behavior on Windows.

However, AFAIK, there is no reason why Cygwin couldn't be located at any
base besides 0x61000000 as long as it is loaded into the same location
in the parent and the child.

If it isn't the fork or exec behavior that is the issue here then this
may be a bug.  If it is a bug then we can fix the bug.

I know that this is a fruitless email and that now everyone is nodding
their heads saying "Yup.  Cygwin not relocatable." I guess I'm just
sharpening my skills for the inevitable responses that I will probably
be making for the next several years.

Btw, whether it is a bug in Cygwin or not, I'm obviously not going to
release a binary version of binutils for Cygwin which incorporates
behavior that breaks the use of Cygwin.  In case anyone was wondering...

cgf

Re: ld --auto-imports

[Paul Sokolovsky]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 3843 bytes --]

Hello Robert,

âîñêðåñåíüå, 10 èþíÿ 2001 ã., you wrote to me:

RC> Hi Paul,
RC>     You may not recognise my name - I hack on the cygwin1.dll internals
RC> among other things. Ralf Habacker put me onto your auto-imports
RC> patch/hack for ld 2.11. I hope you don't mind me writing you direct, I
RC> wasn't sure hwat list was appropriate. Feel free to redirect me, or copy
RC> any reply to a list of your choice.

    I guess binutils list is a place to discuss this, and it fact I'd
appreciate such mails appearing there - likely, it won't be too easy
to get that patch included, so it would be nice if messages showing
that community reviews patch were sent there. I'm not subscribed to
it, however, so I'd appreciate cc: to me.

RC> Your ld patch looks very nice, and I have a modified libtool-HEAD here
RC> that using it rather than a 5 stage dllwrap process.

    You may want to know that I did it quite some time ago, along with
other useful changes. It is available at
http://sourceforge.net/projects/pw32/ . It worked OOB for me for stuff
I tried it on (glib/gtk+ libs, for example). Ralf Habacker reported it
worked ok for him with cygwin (even though package on the link above
built for my PW32).

RC> I've a small bit of feedback/a question, and a more serious question.

RC> Firstly the feedback. I'm using a pre-compiled version from Ralf
RC> habacker. That version (and AFAIK it's a stock build) creates the .dll
RC> base image at 0x610c0000 which collides with the cygwin1.dll. Now the
RC> cygwin1.dll is theoretically relocatable, but we've just found out that
RC> it doesn't like relocating :/. According to MSDN, the default image base
RC> for .dll's should be 0x10000000. Is there any reason your modified ld
RC> couldn't set that as the default base address for .dlls? That would
RC> remove the need to add --base-address=0x10000000 to the link line in
RC> libtool.

    Well:

1. As you understand, it is not connected with auto-import stuff in
any way. That's how Ralf Habacker built it.

2. Having specific base hardcoded is bad anyway - you'll get
collisions (automatically resolved, of course). Far better to
--enable-auto-image-base switch to ld and let it choose unique base
itself (based on hashing of dll name, etc.)

3. With my personal attitudes, ld on win32 should work as close to
normal unix version as possible. Mundane unix system doesn't have any
idiosynacrasies around "relocatability" - so, --enable-auto-image-base
should be the default as it offers most seamless solution to win32
strangeness. The same for --enable-auto-import . That's how I tried to
do while maintaining binutild fow Mingw32. Well, new maintainer
doesn't share these ideas, so having that all in libtool is not worst
solution ;-)

RC> Now for the more serious question. Does your patch in theory support
RC> dll's linking to other .dll's with auto-imports?

    Of course, why not? I wouldn't claim I tested that, however.

RC> I have a trivial hello
RC> world case here (built with my libtool, but I'm very sure the symptoms
RC> will be the same without libtool) that segfaults _every time_. That
RC> testcase has 3 source files, 2 libraries.
RC> hello.exe - depends on cyghello-0.dll
RC> cyghello-0.dll - depends on cyggreet-0.dll

RC> cyghello-0.dll reads a string from cyggreet-0.dll, and writes it + ,
RC> World! to stdout.

RC> Thanks for any suggestions you can offer. I'm happy to drop you the
RC> trivial testcase, and/or prepare a non-libtool testcase that shows the
RC> same symptoms - if that would help.

    Please provide one. I'd appeciate if it were not just mere gcc/ld
based, but also wouldn't depend on cygwin (i.e. were built with
-mno-cygwin or standalone mingw).

RC> Rob




--
Paul Sokolovsky, IT Specialist
http://www.brainbench.com/transcript.jsp?pid=11135