ld --base-file option, dlltool generated exp files

ld --base-file option, dlltool generated exp files

[Tom Kacvinsky]

[-- Attachment #1: Type: text/plain, Size: 2316 bytes --]

HI,

I have an interesting issue I am facing.  On Windows 10, using a custom
built
MinGW-w64 toolchain, based on binutils 2.38, the mingw-w64-crt v10.0.0 with
UCRT support, and GCC 12.1.0.  This also happens with binutils 2.40 and the
MinGW-w64 run time version v11.0.0 (recently released).

Of note, binutils 2.30 (ancient, I know), doesn't have this problem.

We use the GNAT portion of GCC for compiling our Ada code, and use gnatdll
for producing the DLL of our Ada code.

What I found through lots of experimentation is that either the base file
generated
by ld or the exp file generated by dlltool is off and is making a DLL that
causes our
applications to crash when run with Microsoft's Application Verifier.  The
steps I've
been following are laid out in the gnatdll user guide

https://gcc.gnu.org/onlinedocs/gnat_ugn/Using-gnatdll.html

Except I pass "-g -Wl,-v" to the -largs option of gnatdll so that gnatlink
keeps the
binder file around and gives me the exact ld invocation.  I then manually
follow the
steps, but by invoking the three linker commands and two dlltool
invocations as
as they would be if I ran everything though gnatdll.

The reason I say it's either the base file or the exp file is that I can
take my export
definition file (a .def file) and generate an import library and exp file
using Microsoft's
lib tool, and that exp file makes the final link produced a DLL that does
not have an
issue.  So I _suspect_, but am not sure, that either the base file
generated by ld using
the --base-file option is not right, and so dlltool produces an exp file
that isn't quite
right, or perhaps the base file is right, but dlltool is still generating
an exp file that has
problems.

I looked for commits in bfd/cofflink.c (where the .base file would be
generated) and also
looked at commits to dlltool, but nothing stood out.  But then I am not a
binutils expert.

I have a way around the problem without using a base file (just pass the
.def file directly
to ld so that an export table is generated), but I wanted to report this
issue.

What would be most useful for a reproducer?  I think I am going to have a
difficult time
paring our code down to an MRE, and I'm reluctant to release IP object code
(I can run
that up the flagpole, tough).

Any help would be much appreciated.

Regards,

Tom

Re: ld --base-file option, dlltool generated exp files

[Nick Clifton]

Hi Tom,

> What I found through lots of experimentation is that either the base file
> generated by ld or the exp file generated by dlltool is off and is making 
> a DLL that causes our applications to crash 

> The reason I say it's either the base file or the exp file is that I can
> take my export definition file (a .def file) and generate an import library 
> and exp file using Microsoft's lib tool, and that exp file makes the final 
> link produce a DLL that does not have an issue.

> I have a way around the problem without using a base file (just pass the
> .def file directly to ld so that an export table is generated), but I 
> wanted to report this issue.

Thank you for doing this.  It always helps when problems are reported, even
if we do not have a solution available.

Please could you file a bug report here:

   https://sourceware.org/bugzilla/enter_bug.cgi?product=binutils


> What would be most useful for a reproducer?  I think I am going to have a
> difficult time paring our code down to an MRE, and I'm reluctant to 
> release IP object code

Understood.  A stand alone reproducer would be ideal, but if that is not
possible then some further details on the problem would definitely help.
For example - are you able to identify what is wrong the the DLLs being
produced by the linker ?  Do any of the various PE-file checker programs
that are out there report any problems ?

Also - do you know if they problem happens with older versions of the
binutils, eg 2.35 or 2.36 ?  Possibly the problem is due to a (relatively)
recent change to the linker.

I cannot make any guarantees, but I will definitely look at any bug reports
you file.

Cheers
   Nick

Re: ld --base-file option, dlltool generated exp files

[Tom Kacvinsky]

[-- Attachment #1: Type: text/plain, Size: 3088 bytes --]

Hi NIck,

On Wed, May 10, 2023 at 7:38 AM Nick Clifton <nickc@redhat.com> wrote:

> Hi Tom,
>
> > What I found through lots of experimentation is that either the base file
> > generated by ld or the exp file generated by dlltool is off and is
> making
> > a DLL that causes our applications to crash
>
> > The reason I say it's either the base file or the exp file is that I can
> > take my export definition file (a .def file) and generate an import
> library
> > and exp file using Microsoft's lib tool, and that exp file makes the
> final
> > link produce a DLL that does not have an issue.
>
> > I have a way around the problem without using a base file (just pass the
> > .def file directly to ld so that an export table is generated), but I
> > wanted to report this issue.
>
> Thank you for doing this.  It always helps when problems are reported, even
> if we do not have a solution available.
>
> Please could you file a bug report here:
>
>    https://sourceware.org/bugzilla/enter_bug.cgi?product=binutils
>
>
I haven't yet done so as I am fighting getting at least the major versions
of
binutils between which things broke.  I think that would help to have in the
bug report.

But it appears it is going to be more involved than I thought.  I thought I
was
going to get away with building all of our code with one binutils (using the
version of binutils I know works) and then just swapping versions of
binutils
used for making the DLL until I find the version that broke.  But that
process
ends up producing a DLL Windows does not like.  :-(. So, I have to build the
entire GCC + binutils toolchain, with the binutils version changing and GCC
remaining fixed.  This will take a while.

It also doesn't help that my MSYS2 + MinGW-w64 installation doesn't like
building 2.34 and 2.35 (before and  after those versions, things are
fine).  It
is a hang generating the files from pep.em and associated input files and
scripts.


>
> > What would be most useful for a reproducer?  I think I am going to have a
> > difficult time paring our code down to an MRE, and I'm reluctant to
> > release IP object code
>
> Understood.  A stand alone reproducer would be ideal, but if that is not
> possible then some further details on the problem would definitely help.
> For example - are you able to identify what is wrong the the DLLs being
> produced by the linker ?  Do any of the various PE-file checker programs
> that are out there report any problems ?
>

I have reason to believe the problem is with relocations based on the .base
and .exp files, but I don't know if a PE checking program can validate
relocation
information.  I'll have to look into this.


> Also - do you know if they problem happens with older versions of the
> binutils, eg 2.35 or 2.36 ?  Possibly the problem is due to a (relatively)
> recent change to the linker.
>

See above - I am trying to figure this out.


> I cannot make any guarantees, but I will definitely look at any bug reports
> you file.


Thanks,

Tom

Re: ld --base-file option, dlltool generated exp files

[Tom Kacvinsky]

[-- Attachment #1: Type: text/plain, Size: 2095 bytes --]

Hi,

On Sun, May 14, 2023 at 2:44 PM Tom Kacvinsky <tkacvins@gmail.com> wrote:

> Hi NIck,
>
> On Wed, May 10, 2023 at 7:38 AM Nick Clifton <nickc@redhat.com> wrote:
>
>> Hi Tom,
>>
>> > What I found through lots of experimentation is that either the base
>> file
>> > generated by ld or the exp file generated by dlltool is off and is
>> making
>> > a DLL that causes our applications to crash
>>
>> > The reason I say it's either the base file or the exp file is that I can
>> > take my export definition file (a .def file) and generate an import
>> library
>> > and exp file using Microsoft's lib tool, and that exp file makes the
>> final
>> > link produce a DLL that does not have an issue.
>>
>> > I have a way around the problem without using a base file (just pass the
>> > .def file directly to ld so that an export table is generated), but I
>> > wanted to report this issue.
>>
>> Thank you for doing this.  It always helps when problems are reported,
>> even
>> if we do not have a solution available.
>>
>> Please could you file a bug report here:
>>
>>    https://sourceware.org/bugzilla/enter_bug.cgi?product=binutils
>>
>>
> I haven't yet done so as I am fighting getting at least the major versions
> of
> binutils between which things broke.  I think that would help to have in
> the
> bug report.
>
> But it appears it is going to be more involved than I thought.  I thought
> I was
> going to get away with building all of our code with one binutils (using
> the
> version of binutils I know works) and then just swapping versions of
> binutils
> used for making the DLL until I find the version that broke.  But that
> process
> ends up producing a DLL Windows does not like.  :-(. So, I have to build
> the
> entire GCC + binutils toolchain, with the binutils version changing and GCC
> remaining fixed.  This will take a while.
>

Filed bug https://sourceware.org/bugzilla/show_bug.cgi?id=30448 as I found
the versions between which things broke.  I think I even have a problematic
commit.

Regards,

Tom