[Bug target/44139] New: Exporting emutls symbols from a DLL broken on w32 targets

[Bug target/44139] New: Exporting emutls symbols from a DLL broken on w32 targets

[dougsemler at gmail dot com]

Windows targets that use emutls add a "." character as a separator from the
_emutls_{t,v} and the true symbol name.  However, exporting these symbol names
from a DLL is problematic (i.e. a thread local data variable), because the "."
in the symbol name tells the Windows loader to treat the symbol as a forwarding
RVA to a second DLL (in this case, it will look for the symbol "varname" in
__emutls_v.dll, the symbol exported is __emutls_v.varname).  Note that this is
a runtime issue, not a compile/link issue.

This can be worked around by

#define TARGET_EMUTLS_VAR_PREFIX "__emutls_v_"
#define TARGET_EMUTLS_TMPL_PREFIX "__emutls_t_"

in the config header (e.g. config/i386/cygming.h or config/i386/mingw-w64.h,
etc).

However, this breaks backward compatibility with object files that were built
with previous versions of the compiler (e.g libold.a would have a global name
of __emutls_v.varname, while the patch would cause gcc to look for
__emutls_v_varname, which would cause linker errors).


-- 
           Summary: Exporting emutls symbols from a DLL broken on w32
                    targets
           Product: gcc
           Version: 4.5.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: target
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: dougsemler at gmail dot com
GCC target triplet: i686-pc-mingw32 x86_64-*-mingw32 i686-pc-cygwin


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44139

[Bug target/44139] Exporting emutls symbols from a DLL broken on w32 targets

[davek at gcc dot gnu dot org]

------- Comment #1 from davek at gcc dot gnu dot org  2010-05-15 09:06 -------
(In reply to comment #0)
> Windows targets that use emutls add a "." character as a separator from the
> _emutls_{t,v} and the true symbol name.  However, exporting these symbol names
> from a DLL is problematic (i.e. a thread local data variable), because the "."
> in the symbol name tells the Windows loader to treat the symbol as a forwarding
> RVA to a second DLL (in this case, it will look for the symbol "varname" in
> __emutls_v.dll, the symbol exported is __emutls_v.varname).  Note that this is
> a runtime issue, not a compile/link issue.

  I think it's actually a link time issue after all, and we could be going to
dodge a bullet re: having to break backward compatibility here.  The PE spec
has this to say about forwarded exports (pe coff spec rev. 8, 6.3.2):

" Each entry in the export address table is a field that uses one of two
formats in the following table. If the address specified is not within the
export section (as defined by the address and length that are indicated in the
optional header), the field is an export RVA, which is an actual address in
code or data. Otherwise, the field is a forwarder RVA, which names a symbol in
another DLL. "

In other words, I don't think the runtime loader actually keys off the presence
of a dot in the exported symbol, but where the EAT entry points to.  If we can
persuade ld that sometimes a dot in a .DEF file symbol listing is indeed just a
dot, and not a forwarder definition, it should be able to generate a regular
EAT entry pointing at the exported symbol, rather than placing an ASCII string
in the export section and pointing the EAT entry at it, and hopefully
everything should "just work".

I'll test if this works by crudely hacking ld about a bit before thinking about
how we should implement this in terms of interface.  We might be able to get
away without changing the compiler.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44139

[Bug target/44139] Exporting emutls symbols from a DLL broken on w32 targets

[davek at gcc dot gnu dot org]

------- Comment #2 from davek at gcc dot gnu dot org  2010-05-15 09:34 -------
(In reply to comment #1)

> In other words, I don't think the runtime loader actually keys off the presence
> of a dot in the exported symbol, but where the EAT entry points to.  If we can
> persuade ld that sometimes a dot in a .DEF file symbol listing is indeed just a
> dot, and not a forwarder definition, it should be able to generate a regular
> EAT entry pointing at the exported symbol, rather than placing an ASCII string
> in the export section and pointing the EAT entry at it, and hopefully
> everything should "just work".

Theory confirmed by applying the following gross hack to src/ld/pe-dll.c:
Index: ld/pe-dll.c
===================================================================
RCS file: /cvs/src/src/ld/pe-dll.c,v
retrieving revision 1.129
diff -p -u -r1.129 pe-dll.c
--- ld/pe-dll.c 28 Apr 2010 15:24:22 -0000      1.129
+++ ld/pe-dll.c 15 May 2010 09:32:37 -0000
@@ -867,7 +867,7 @@ process_def_file_and_drectve (bfd *abfd
       char *name;

       /* Check for forward exports */
-      if (strchr (pe_def_file->exports[i].internal_name, '.'))
+      if (0 && strchr (pe_def_file->exports[i].internal_name, '.'))
        {
          count_exported++;
          if (!pe_def_file->exports[i].flag_noname)

Will attach the trivial testcase I tried it against.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44139

[Bug target/44139] Exporting emutls symbols from a DLL broken on w32 targets

[davek at gcc dot gnu dot org]

------- Comment #3 from davek at gcc dot gnu dot org  2010-05-15 09:37 -------
Created an attachment (id=20665)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=20665&action=view)
testcase: main executable source


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44139

[Bug target/44139] Exporting emutls symbols from a DLL broken on w32 targets

[davek at gcc dot gnu dot org]

------- Comment #6 from davek at gcc dot gnu dot org  2010-05-15 09:38 -------
Created an attachment (id=20668)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=20668&action=view)
testcase: makefile


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44139

[Bug target/44139] Exporting emutls symbols from a DLL broken on w32 targets

[davek at gcc dot gnu dot org]

------- Comment #5 from davek at gcc dot gnu dot org  2010-05-15 09:38 -------
Created an attachment (id=20667)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=20667&action=view)
testcase: dll header


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44139

[Bug target/44139] Exporting emutls symbols from a DLL broken on w32 targets

[davek at gcc dot gnu dot org]

------- Comment #4 from davek at gcc dot gnu dot org  2010-05-15 09:37 -------
Created an attachment (id=20666)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=20666&action=view)
testcase: dll source 


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44139

[Bug target/44139] Exporting emutls symbols from a DLL broken on w32 targets

[davek at gcc dot gnu dot org]

------- Comment #7 from davek at gcc dot gnu dot org  2010-05-15 09:45 -------
So... I think now we just need to figure out how to tell LD that some export
directives contain dots because they're exporting a symbol containing a dot. 
Actually, that's probably all we need to do: when we find an export directive,
before checking if there's a dot in it, check if it matches the name of an
existing symbol and if so assume that that's the one we actually want to
export.

I'm going to close this bug and mark it as invalid, not because the report is
incorrect in any way, but because I think this should be considered a binutils
rather than GCC problem.  Doug, you can open a PR in the sourceware.org
bugzilla against binutils and assign it to me if you like, but I'll try not to
forget about fixing this problem even if you don't!


-- 

davek at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|                            |INVALID


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44139

[Bug target/44139] Exporting emutls symbols from a DLL broken on w32 targets

[dougsemler at gmail dot com]

------- Comment #8 from dougsemler at gmail dot com  2010-05-15 13:03 -------
Done

cf. http://sourceware.org/bugzilla/show_bug.cgi?id=11603

(note to self, pay more attention to specs and less attention to de facto
behavior of toolsets ;-))


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44139

[Bug target/44139] Exporting emutls symbols from a DLL broken on w32 targets

[davek at gcc dot gnu dot org]

------- Comment #9 from davek at gcc dot gnu dot org  2010-05-15 13:48 -------
FTR: Patch posted at http://sourceware.org/ml/binutils/2010-05/msg00171.html


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44139

[Bug target/44139] Exporting emutls symbols from a DLL broken on w32 targets

[davek at gcc dot gnu dot org]

-- 

davek at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         AssignedTo|unassigned at gcc dot gnu   |davek at gcc dot gnu dot org
                   |dot org                     |
             Status|UNCONFIRMED                 |ASSIGNED
     Ever Confirmed|0                           |1
   Last reconfirmed|0000-00-00 00:00:00         |2010-05-17 18:25:49
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44139

[Bug target/44139] Exporting emutls symbols from a DLL broken on w32 targets

[davek at gcc dot gnu dot org]

------- Comment #10 from davek at gcc dot gnu dot org  2010-05-17 18:25 -------
Re-opening.  It turns out that GCC fails to actually apply the dllexport
attribute to TLS control vars.  So solving the binutils problem allows
auto-export of a TLS variable to work, but as auto-export gets disabled in the
presence of any explicit dllexport directives, this isn't an effective
solution.  I believe the problem needs to be addressed in
config/i386/winnt.c#i386_pe_encode_section_info()


-- 

davek at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |UNCONFIRMED
         Resolution|INVALID                     |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44139

[Bug target/44139] Exporting emutls symbols from a DLL broken on w32 targets

[ktietz at gcc dot gnu dot org]

------- Comment #11 from ktietz at gcc dot gnu dot org  2010-05-18 14:22 -------
(In reply to comment #10)
> Re-opening.  It turns out that GCC fails to actually apply the dllexport
> attribute to TLS control vars.  So solving the binutils problem allows
> auto-export of a TLS variable to work, but as auto-export gets disabled in the
> presence of any explicit dllexport directives, this isn't an effective
> solution.  I believe the problem needs to be addressed in
> config/i386/winnt.c#i386_pe_encode_section_info()
> 

I have doubts about the approach in winnt.c, but well maybe I am wrong here. I
investigated for this issue and see the real issue in declaration cloning in
varasm.c's emutls_decl- function, which simply doesn't copy attributes of the
delaration, which then leads to issues about name-decoration.
Also the DECL_DLLIMPORT_P has to be copied here, too (for import).


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44139

[Bug target/44139] Exporting emutls symbols from a DLL broken on w32 targets

[davek at gcc dot gnu dot org]

------- Comment #12 from davek at gcc dot gnu dot org  2010-05-18 14:29 -------
(In reply to comment #11)

> I have doubts about the approach in winnt.c, but well maybe I am wrong here. I
> investigated for this issue and see the real issue in declaration cloning in
> varasm.c's emutls_decl- function, which simply doesn't copy attributes of the
> delaration, which then leads to issues about name-decoration.
> Also the DECL_DLLIMPORT_P has to be copied here, too (for import).

Well, what I was thinking when I wrote that is that we could recognize the
TARGET_EMUTLS_xxx_PREFIX in winnt.c, look up the original decl, and copy
whatever necessary attributes over at that time.  However it does look like the
TARGET_EMUTLS_VAR_INIT hook might be what we're actually looking for here; I'll
check the code.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44139

[Bug target/44139] Exporting emutls symbols from a DLL broken on w32 targets

[davek at gcc dot gnu dot org]

------- Comment #13 from davek at gcc dot gnu dot org  2010-05-18 14:33 -------
(In reply to comment #12)
> TARGET_EMUTLS_VAR_INIT 

Nah, scratch that, it's not really sensible.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44139

[Bug target/44139] Exporting emutls symbols from a DLL broken on w32 targets

[ktietz at gcc dot gnu dot org]

------- Comment #14 from ktietz at gcc dot gnu dot org  2010-05-18 15:18 -------
Hi Dave,

following patch solves the issue for me pretty well.

ChangeLog

      * varasm.c (emutls_decl): Clone attributes for new decl.

Index: gcc/gcc/varasm.c
===================================================================
--- gcc.orig/gcc/varasm.c       2010-05-18 13:19:20.000000000 +0200
+++ gcc/gcc/varasm.c    2010-05-18 17:10:11.385445300 +0200
@@ -403,6 +403,8 @@ emutls_decl (tree decl)
        int foo() { return i; }
        __thread int i = 1;
      in which I goes from external to locally defined and initialized.  */
+  DECL_DLLIMPORT_P (to) = DECL_DLLIMPORT_P (decl);
+  DECL_ATTRIBUTES (to) = targetm.merge_decl_attributes (decl, to);

   TREE_STATIC (to) = TREE_STATIC (decl);
   TREE_USED (to) = TREE_USED (decl);


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44139

[Bug target/44139] Exporting emutls symbols from a DLL broken on w32 targets

[davek at gcc dot gnu dot org]

------- Comment #15 from davek at gcc dot gnu dot org  2010-05-18 15:26 -------
(In reply to comment #14)
> Hi Dave,
> 
> following patch solves the issue for me pretty well.

That looks good to me to, although doing it in the middle-end makes me worry
that some other target might /not/ be expecting attributes to be merged from
one to the other!  That's the slight advantage of doing it in the encode
section hook, even though string-matching the symbols is a bit kludgey.

I can't test it right now owing to parallel make check being thoroughly borked
on cygwin :(  Could be a few days before it gets fixed and I can do anything
productive gcc-wise.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44139

[Bug target/44139] Exporting emutls symbols from a DLL broken on w32 targets

[dominiq at lps dot ens dot fr]

------- Comment #16 from dominiq at lps dot ens dot fr  2010-05-18 16:28 -------
You may be interested by pr 44132.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44139

[Bug target/44139] Exporting emutls symbols from a DLL broken on w32 targets

[dannysmith at users dot sourceforge dot net]

------- Comment #17 from dannysmith at users dot sourceforge dot net  2010-05-18 22:43 -------
(In reply to comment #14)
> Index: gcc/gcc/varasm.c
> ===================================================================
> --- gcc.orig/gcc/varasm.c       2010-05-18 13:19:20.000000000 +0200
> +++ gcc/gcc/varasm.c    2010-05-18 17:10:11.385445300 +0200
> @@ -403,6 +403,8 @@ emutls_decl (tree decl)
>         int foo() { return i; }
>         __thread int i = 1;
>       in which I goes from external to locally defined and initialized.  */
> +  DECL_DLLIMPORT_P (to) = DECL_DLLIMPORT_P (decl);
> +  DECL_ATTRIBUTES (to) = targetm.merge_decl_attributes (decl, to);
> 
>    TREE_STATIC (to) = TREE_STATIC (decl);
>    TREE_USED (to) = TREE_USED (decl);
> 

I like this approach better too.  It would be even cleaner (here and elswhere)
if we had a decl_with_vis.dllexport_flag and DECL_DLLEXPORT_P.  14 spare bits
left in decl_with_vis.  Are they too precious??
Danny


-- 

dannysmith at users dot sourceforge dot net changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |dannysmith at users dot
                   |                            |sourceforge dot net


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44139

[Bug target/44139] Exporting emutls symbols from a DLL broken on w32 targets

[ktietz at gcc dot gnu dot org]

------- Comment #18 from ktietz at gcc dot gnu dot org  2010-05-19 09:15 -------
Hi David,

Could you test the suggested patch for AIX? Richard told me that the patch is
sensible, but the attribute merging is something to be tested for AIX.

Thanks in advance,
Kai


-- 

ktietz at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |dje at gcc dot gnu dot org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44139

[Bug target/44139] Exporting emutls symbols from a DLL broken on w32 targets

[dje at gcc dot gnu dot org]

------- Comment #19 from dje at gcc dot gnu dot org  2010-05-19 15:40 -------
What is the relationship between this bug and PR 44132?  Richi and Honza seem
to prefer the DECL_PRESERVE_P hack.  We will see if Iain's lowering works.  I
don't think both the decl attribute merging patch and DECL_PRESERVE_P patch are
needed.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44139

[Bug target/44139] Exporting emutls symbols from a DLL broken on w32 targets

[ktietz at gcc dot gnu dot org]

------- Comment #20 from ktietz at gcc dot gnu dot org  2010-05-19 16:18 -------
(In reply to comment #19)
> What is the relationship between this bug and PR 44132?  Richi and Honza seem
> to prefer the DECL_PRESERVE_P hack.  We will see if Iain's lowering works.  I
> don't think both the decl attribute merging patch and DECL_PRESERVE_P patch are
> needed.
> 

I don't see here any relations AFAICS (beside both patches are touching
emutls).  This declaration preserve flag is fine, but AFAICS don't address this
issue.
The point here is that w32 targets dependent on original decl-attributes for
name-decoration. So Richi asked me, that I ask you, if you could test this
patch for AIX to avoid side-effects.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44139

[Bug target/44139] Exporting emutls symbols from a DLL broken on w32 targets

[dje at gcc dot gnu dot org]

------- Comment #21 from dje at gcc dot gnu dot org  2010-05-20 17:52 -------
I tested the patch from comment #14 on top of the DECL_PRESERVE_P patch from PR
44132, which was necessary to get back to a sane level of C++ and libgomp
failures on AIX.  The DECL_DLLIMPORT_P and DECL_ATTRIBUTES patch did not have
any further affect on AIX testsuite results.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44139

[Bug target/44139] Exporting emutls symbols from a DLL broken on w32 targets

[ktietz at gcc dot gnu dot org]

------- Comment #22 from ktietz at gcc dot gnu dot org  2010-05-21 11:28 -------
Patch fron comment #14 applied to trunk.
Back-port won't be done as there is a risc of emutls-fallout (as Richard
mentioned in his approval).

Committed at revision 159658.


-- 

ktietz at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|                            |FIXED


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44139