static vs. shared linking

static vs. shared linking

[David Stacey]

I've been having difficulty building poco-1.6.0 for Cygwin for some
time. I've managed to produce a test case that shows the problem:

https://dl.dropboxusercontent.com/u/119453582/Cygwin/crashtest.tar.xz

This archive contains source files that produce a very simple library.
When linked statically, the code works fine. However, when linked as a
shared DLL, the test crashes with a core dump. The behaviour is
identical on x86 and x86_64 architectures.

Have I made a stupid error in the compilation of the shared case, or is
something more interesting going on?

Dave.




--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: static vs. shared linking

[David Stacey]

On 24/03/2015 00:02, David Stacey wrote:
> I've been having difficulty building poco-1.6.0 for Cygwin for some
> time. I've managed to produce a test case that shows the problem:
>
> https://dl.dropboxusercontent.com/u/119453582/Cygwin/crashtest.tar.xz
>
> This archive contains source files that produce a very simple library.
> When linked statically, the code works fine. However, when linked as a
> shared DLL, the test crashes with a core dump. The behaviour is
> identical on x86 and x86_64 architectures.
>
> Have I made a stupid error in the compilation of the shared case, or is
> something more interesting going on?

I don't know if anyone has managed to look at this. I haven't had a 
deluge of e-mails telling me that I've done something silly (which is a 
shame, because then I could fix it quickly and move on). For the sake of 
a straw to clutch at, I tried compiling with clang++ rather than g++, 
and got the same result:

     $ ./go.sh
     Running test (static link)...
     Done.

     Running test (shared link)...
     ./go.sh: line 19:  3744 Aborted                 (core dumped) 
./shared_test
     Done.

Any help or hints would be greatly appreciated.

Dave.



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: static vs. shared linking

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 3087 bytes --]

On Mar 24 18:39, David Stacey wrote:
> On 24/03/2015 00:02, David Stacey wrote:
> >I've been having difficulty building poco-1.6.0 for Cygwin for some
> >time. I've managed to produce a test case that shows the problem:
> >
> >https://dl.dropboxusercontent.com/u/119453582/Cygwin/crashtest.tar.xz
> >
> >This archive contains source files that produce a very simple library.
> >When linked statically, the code works fine. However, when linked as a
> >shared DLL, the test crashes with a core dump. The behaviour is
> >identical on x86 and x86_64 architectures.
> >
> >Have I made a stupid error in the compilation of the shared case, or is
> >something more interesting going on?
> 
> I don't know if anyone has managed to look at this. I haven't had a deluge
> of e-mails telling me that I've done something silly (which is a shame,
> because then I could fix it quickly and move on). For the sake of a straw to
> clutch at, I tried compiling with clang++ rather than g++, and got the same
> result:
> 
>     $ ./go.sh
>     Running test (static link)...
>     Done.
> 
>     Running test (shared link)...
>     ./go.sh: line 19:  3744 Aborted                 (core dumped)
> ./shared_test
>     Done.
> 
> Any help or hints would be greatly appreciated.

For a start, you should contemplate to build your test with -g to allow
debugging.  Then you can run the testcase under GDB and get (more or less)
useful output.  The crash occurs in a delete call, afaics.  If you
install the cygwin-debuginfo package, addr2line returns something like this
as the call stack (non-required path components removed):

[...]/cygwin/exceptions.cc:1247
[...]/cygwin/exceptions.cc:1501
[...]/cygwin/sigproc.cc:717
[...]/cygwin/signal.cc:252
[...]/cygwin/signal.cc:303
[...]/cygwin/signal.cc:313
[...]/cygwin/signal.cc:289
[...]/cygwin/signal.cc:375

Everything up to here you can ignore, they are the result of the
abort() call in free(), which occurs here:

[...]/cygwin/malloc.cc:4779
[...]/cygwin/malloc_wrapper.cc:47
[...]/cygwin/sigfe.s:43
[...]/cygwin/libstdcxx_wrapper.cc:69

That's the actual call to the delete method.

/usr/lib/gcc/x86_64-pc-cygwin/4.9.2/include/c++/ext/new_allocator.h:110
/usr/lib/gcc/x86_64-pc-cygwin/4.9.2/include/c++/bits/basic_string.tcc:449
/usr/lib/gcc/x86_64-pc-cygwin/4.9.2/include/c++/bits/basic_string.h:249
/usr/lib/gcc/x86_64-pc-cygwin/4.9.2/include/c++/bits/basic_string.tcc:512

And this is where it comes from.  It's a call to

  void basic_string<_CharT, _Traits, _Alloc>::reserve(size_type __res)

But, really, I have no idea how this stuff is correlated.  I'm not
a user of libstdc++.  What's noticable is the fact that the crash does
*not* occur because the shared lib is unloaded or something like that.
However, building with -g allows you to step through the code and see
what happens, maybe that gives a clue.


HTH,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: static vs. shared linking

[Warren Young]

On Mar 25, 2015, at 3:04 AM, Corinna Vinschen <corinna-cygwin@cygwin.com> wrote:
> 
> And this is where it comes from.  It's a call to
> 
>  void basic_string<_CharT, _Traits, _Alloc>::reserve(size_type __res)

David, what happens if you say 

    wtext.reserve(1);

inside runTests() before the call to crash()?

If that makes the symptom disappear, I wonder if there’s some problem with a Cygwin *.exe owning a std::string that gets resized by a Cygwin *.dll.  If so, that probably *is* a memory ownership coordination problem that affects Cygwin proper.

I’ve run both versions under valgrind on a Linux box here, and they can find no fault with your code.  On manual inspection, I, too find it to be perfectly cromulent.
--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: static vs. shared linking

[David Stacey]

On 25/03/2015 16:59, Warren Young wrote:
> On Mar 25, 2015, at 3:04 AM, Corinna Vinschen<corinna-cygwin@cygwin.com>  wrote:
>>
>> And this is where it comes from.  It's a call to
>>
>>     void basic_string<_CharT, _Traits, _Alloc>::reserve(size_type __res)
> David, what happens if you say
>
>      wtext.reserve(1);
>
> inside runTests() before the call to crash()?
>
> If that makes the symptom disappear, I wonder if there’s some problem with a Cygwin *.exe owning a std::string that gets resized by a Cygwin *.dll.  If so, that probably*is*  a memory ownership coordination problem that affects Cygwin proper.

Thank you for taking the time to look at this. You are quite correct. 
When I reserve() space on the string in runTests() then the problem goes 
away.

In order to test your hypothesis about memory ownership, I'll create a 
test that malloc(3)s some memory in the .exe and free(3)s it in a shared 
library; Corinna showed that the crash was coming from an abort() in 
free(). However, I can't believe it's that simple - you'd think there 
would be dozens of programmes crashing for this reason.

> I’ve run both versions under valgrind on a Linux box here, and they can find no fault with your code.

Thanks. Obviously, when coming across problems like this, my first 
thought is that I've managed to do something silly. But if neither you, 
me nor Corinna can spot an obvious blunder then that opens the 
possibility that there's something fruity going on.

> On manual inspection, I, too find it to be perfectly cromulent.

Personally, I was thoroughly discombobulated.

Dave.



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: static vs. shared linking

[David Stacey]

On 25/03/2015 22:11, David Stacey wrote:
> On 25/03/2015 16:59, Warren Young wrote:
>> If that makes the symptom disappear, I wonder if there’s some problem 
>> with a Cygwin *.exe owning a std::string that gets resized by a 
>> Cygwin *.dll.  If so, that probably*is*  a memory ownership 
>> coordination problem that affects Cygwin proper.
>
> In order to test your hypothesis about memory ownership, I'll create a 
> test that malloc(3)s some memory in the .exe and free(3)s it in a 
> shared library; Corinna showed that the crash was coming from an 
> abort() in free(). However, I can't believe it's that simple - you'd 
> think there would be dozens of programmes crashing for this reason.

Indeed. I created a very simple test that alloc(3)s memory in main() and 
then free(3)s it in a shared library. That works fine. So there's more 
to it than that :-(

Dave.



// crash_library.h
#ifndef CRASH_LIBRARY_H
#define CRASH_LIBRARY_H

extern void Crash(unsigned short *ptr);

#endif // CRASH_LIBRARY_H



// crash_library.cpp
#include "crash_library.h"
#include <stdlib.h>

void Crash(unsigned short *ptr)
{
   free(ptr);
}



// main.cpp
#include "crash_library.h"
#include <stdlib.h>

int main()
{
   unsigned short*ptr = (unsigned short*)malloc(sizeof(unsigned short));
   Crash(ptr);
   return 0;
}



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: static vs. shared linking

[David Stacey]

On 25/03/2015 09:04, Corinna Vinschen wrote:
> On Mar 24 18:39, David Stacey wrote:
>> On 24/03/2015 00:02, David Stacey wrote:
>>> I've been having difficulty building poco-1.6.0 for Cygwin for some
>>> time. I've managed to produce a test case that shows the problem:
>>>
>>> https://dl.dropboxusercontent.com/u/119453582/Cygwin/crashtest.tar.xz
>>>
>>> This archive contains source files that produce a very simple library.
>>> When linked statically, the code works fine. However, when linked as a
>>> shared DLL, the test crashes with a core dump. The behaviour is
>>> identical on x86 and x86_64 architectures.
>>>
>>> Have I made a stupid error in the compilation of the shared case, or is
>>> something more interesting going on?
>>   
>> I don't know if anyone has managed to look at this. I haven't had a deluge
>> of e-mails telling me that I've done something silly (which is a shame,
>> because then I could fix it quickly and move on). For the sake of a straw to
>> clutch at, I tried compiling with clang++ rather than g++, and got the same
>> result:
>>   
>>        $ ./go.sh
>>        Running test (static link)...
>>        Done.
>>   
>>        Running test (shared link)...
>>        ./go.sh: line 19:  3744 Aborted                 (core dumped)
>> ./shared_test
>>        Done.
>>   
>> Any help or hints would be greatly appreciated.
> For a start, you should contemplate to build your test with -g to allow
> debugging.  Then you can run the testcase under GDB and get (more or less)
> useful output.  The crash occurs in a delete call, afaics.  If you
> install the cygwin-debuginfo package, addr2line returns something like this
> as the call stack (non-required path components removed):
>
> [...]/cygwin/exceptions.cc:1247
> [...]/cygwin/exceptions.cc:1501
> [...]/cygwin/sigproc.cc:717
> [...]/cygwin/signal.cc:252
> [...]/cygwin/signal.cc:303
> [...]/cygwin/signal.cc:313
> [...]/cygwin/signal.cc:289
> [...]/cygwin/signal.cc:375

Thank you for your comments - they were really helpful. Yes, I should 
have specified '-g' on the command line - that was an omission on my 
part - sorry.

I've never had much joy out of addr2line before, and I'm struggling to 
recreate what you've done. I've added '-g' to the command line, run 
'go.sh' again. This generates a fresh stackdump file, and then I do:

awk '/^[0-9]/{print $2}' shared_test.exe.stackdump | addr2line -f -e 
shared_test.exe

but I just see question marks. Please could you show the exact lines 
you're using.

Many thanks,

Dave.



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: static vs. shared linking

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2309 bytes --]

On Mar 25 22:42, David Stacey wrote:
> On 25/03/2015 09:04, Corinna Vinschen wrote:
> >  If you
> >install the cygwin-debuginfo package, addr2line returns something like this
> >as the call stack (non-required path components removed):
> >
> >[...]/cygwin/exceptions.cc:1247
> >[...]/cygwin/exceptions.cc:1501
> >[...]/cygwin/sigproc.cc:717
> >[...]/cygwin/signal.cc:252
> >[...]/cygwin/signal.cc:303
> >[...]/cygwin/signal.cc:313
> >[...]/cygwin/signal.cc:289
> >[...]/cygwin/signal.cc:375
> 
> Thank you for your comments - they were really helpful. Yes, I should have
> specified '-g' on the command line - that was an omission on my part -
> sorry.
> 
> I've never had much joy out of addr2line before, and I'm struggling to
> recreate what you've done. I've added '-g' to the command line, run 'go.sh'
> again. This generates a fresh stackdump file, and then I do:
> 
> awk '/^[0-9]/{print $2}' shared_test.exe.stackdump | addr2line -f -e
> shared_test.exe
> 
> but I just see question marks. Please could you show the exact lines you're
> using.

addr2line is a bit dumb and needs help.  What I do is to cat the
stackdump file and look at the addresses.  They usually show where
the stuff comes from:

  $ gawk '/^0/{print $2}' mkgroup.exe.stackdump
  7FFBDC82DDB6
  001800FEC36
  001800FE188
  001800CF471
  001800CF53D
  0018007EAC1
  00100402DE6
  00180049411
  00180046369
  00180046180
  00180049488
  00100401351
  00100401010
  7FFBD9FE13D2
  7FFBDC85EB64

The 7f addresses are from OS DLLs you can't read with addr2line.
0018xxx is the Cygwin DLL, 0010xxx is the application itself.  Other
addresses are other DLLs.  Just check the addresses against
/etc/rebase.db.x86_64.

Then call addr2line for each object file, e.g.:

  $ addr2line -e /usr/bin/cygwin1.dll 001800FEC36 001800FE188 001800CF471
  [...]/cygwin/passwd.cc:576
  [...]/cygwin/passwd.cc:353
  [...]/cygwin/grp.cc:413
  $ addr2line -e /usr/bin/mkgroup.exe 00100401351 00100401010
  /usr/src/debug/cygwin-1.7.35-1/winsup/cygwin/lib/cygwin_crt0.c:22
  /usr/src/debug/cygwin-1.7.35-1/winsup/cygwin/crt0.c:34


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: static vs. shared linking

[David Stacey]

On 30/03/15 11:55, Corinna Vinschen wrote:
> On Mar 25 22:42, David Stacey wrote:
>> I've never had much joy out of addr2line before, and I'm struggling to
>> recreate what you've done. I've added '-g' to the command line, run 'go.sh'
>> again. This generates a fresh stackdump file, and then I do:
>>   
>> awk '/^[0-9]/{print $2}' shared_test.exe.stackdump | addr2line -f -e
>> shared_test.exe
>>   
>> but I just see question marks. Please could you show the exact lines you're
>> using.
> addr2line is a bit dumb and needs help.  What I do is to cat the
> stackdump file and look at the addresses.  They usually show where
> the stuff comes from:
>
>    $ gawk '/^0/{print $2}' mkgroup.exe.stackdump
>    7FFBDC82DDB6
>    001800FEC36
>    001800FE188
>    001800CF471
>    001800CF53D
>    0018007EAC1
>    00100402DE6
>    00180049411
>    00180046369
>    00180046180
>    00180049488
>    00100401351
>    00100401010
>    7FFBD9FE13D2
>    7FFBDC85EB64
>
> The 7f addresses are from OS DLLs you can't read with addr2line.
> 0018xxx is the Cygwin DLL, 0010xxx is the application itself.  Other
> addresses are other DLLs.  Just check the addresses against
> /etc/rebase.db.x86_64.
>
> Then call addr2line for each object file, e.g.:
>
>    $ addr2line -e /usr/bin/cygwin1.dll 001800FEC36 001800FE188 001800CF471
>    [...]/cygwin/passwd.cc:576
>    [...]/cygwin/passwd.cc:353
>    [...]/cygwin/grp.cc:413
>    $ addr2line -e /usr/bin/mkgroup.exe 00100401351 00100401010
>    /usr/src/debug/cygwin-1.7.35-1/winsup/cygwin/lib/cygwin_crt0.c:22
>    /usr/src/debug/cygwin-1.7.35-1/winsup/cygwin/crt0.c:34

Thank you for your reply and the explanation. That requires quite a bit 
of knowledge before addr2line is usable - no wonder I've never had 
anything sensible out of it before!

Back to the matter in hand - I don't suppose you had thoughts on why my 
simple application crashes when linked as shared, but works fine when 
linked statically?

Dave.


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: static vs. shared linking

[Andrey Repin]

Greetings, David Stacey!

> Back to the matter in hand - I don't suppose you had thoughts on why my 
> simple application crashes when linked as shared, but works fine when 
> linked statically?

Probably I've missed this bit before, forgive me if I did, but have you
rebased your library after linking?


-- 
With best regards,
Andrey Repin
Monday, March 30, 2015 23:29:13

Sorry for my terrible english...


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: static vs. shared linking

[David Stacey]

On 30/03/2015 21:30, Andrey Repin wrote:
> Greetings, David Stacey!
>
>> Back to the matter in hand - I don't suppose you had thoughts on why my
>> simple application crashes when linked as shared, but works fine when
>> linked statically?
> Probably I've missed this bit before, forgive me if I did, but have you
> rebased your library after linking?

Thank you for your reply. I tried this two different ways:

   - Running 'rebase -s' on cygcrash_library.dll;
   - Moving 'cygcrash_library.dll' into /usr/bin and triggering a full 
rebase using 'rebase-trigger full' and then running setup-x86_64.exe.

Sadly, neither of these made any difference - the application still 
crashed :-(

Thanks for the suggestion anyway,

Dave.



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: static vs. shared linking

[Andrey Repin]

Greetings, David Stacey!

>>> Back to the matter in hand - I don't suppose you had thoughts on why my
>>> simple application crashes when linked as shared, but works fine when
>>> linked statically?
>> Probably I've missed this bit before, forgive me if I did, but have you
>> rebased your library after linking?

> Thank you for your reply. I tried this two different ways:

>    - Running 'rebase -s' on cygcrash_library.dll;
>    - Moving 'cygcrash_library.dll' into /usr/bin and triggering a full 
> rebase using 'rebase-trigger full' and then running setup-x86_64.exe.

I think you need to rebase both your app and library (assuming they both
Cygwin dependent).
Simply moving it around wouldn't help, rebase only tend to apps in its list(s).

> Sadly, neither of these made any difference - the application still 
> crashed :-(

Well, shoot... was worth a try, at the very least.


-- 
With best regards,
Andrey Repin
Tuesday, March 31, 2015 03:22:01

Sorry for my terrible english...


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: static vs. shared linking

[Achim Gratz]

David Stacey writes:
> Thank you for your reply. I tried this two different ways:
>
>   - Running 'rebase -s' on cygcrash_library.dll;

Better try 'rebase -O' if you're just experimenting.

>   - Moving 'cygcrash_library.dll' into /usr/bin and triggering a full
> rebase using 'rebase-trigger full' and then running setup-x86_64.exe.

As long as it isn't mentioned in /etc/setup/*.lst.gz that won't do
anything.  You could put the location for your library into a file in
/var/lib/rebase/user.d to have it picked up by the autorebase.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf rackAttack:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: static vs. shared linking

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1013 bytes --]

On Mar 31 08:04, Achim Gratz wrote:
> David Stacey writes:
> > Thank you for your reply. I tried this two different ways:
> >
> >   - Running 'rebase -s' on cygcrash_library.dll;
> 
> Better try 'rebase -O' if you're just experimenting.
> 
> >   - Moving 'cygcrash_library.dll' into /usr/bin and triggering a full
> > rebase using 'rebase-trigger full' and then running setup-x86_64.exe.
> 
> As long as it isn't mentioned in /etc/setup/*.lst.gz that won't do
> anything.  You could put the location for your library into a file in
> /var/lib/rebase/user.d to have it picked up by the autorebase.

Rebasing shouldn't matter much in this situation.  The effect doesn't
apply for a rebase problem, and, assuming 64 bit, given the system DLLs
are usually rebased, they are in a completely different memory area
than non-rebased DLLs.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: static vs. shared linking

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1654 bytes --]

On Mar 30 20:15, David Stacey wrote:
> On 30/03/15 11:55, Corinna Vinschen wrote:
> >On Mar 25 22:42, David Stacey wrote:
> >>I've never had much joy out of addr2line before, and I'm struggling to
> >>recreate what you've done. I've added '-g' to the command line, run 'go.sh'
> >>again. This generates a fresh stackdump file, and then I do:
> >>awk '/^[0-9]/{print $2}' shared_test.exe.stackdump | addr2line -f -e
> >>shared_test.exe
> >>but I just see question marks. Please could you show the exact lines you're
> >>using.
> >addr2line is a bit dumb and needs help.  What I do is to cat the
> >stackdump file and look at the addresses.  They usually show where
> >the stuff comes from:
> >[...]
> 
> Thank you for your reply and the explanation. That requires quite a bit of
> knowledge before addr2line is usable - no wonder I've never had anything
> sensible out of it before!
> 
> Back to the matter in hand - I don't suppose you had thoughts on why my
> simple application crashes when linked as shared, but works fine when linked
> statically?

No, sorry.  This may be a c++11 thingy which requires "something" in
libstdc++ and Cygwin, but I don't know what that could be.  It's
especially weird that free() aborts.  This points to some malloc/free
inconsistency, as if the malloc (or new) call used another
implementation of malloc than the aborting free call.  It may also
be a memory overflow issue but that would show up on other platforms
as well.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: static vs. shared linking

[David Stacey]

On 31/03/15 10:05, Corinna Vinschen wrote:
> On Mar 30 20:15, David Stacey wrote:
>> Back to the matter in hand - I don't suppose you had thoughts on why my
>> simple application crashes when linked as shared, but works fine when linked
>> statically?
> No, sorry.  This may be a c++11 thingy which requires "something" in
> libstdc++ and Cygwin, but I don't know what that could be.  It's
> especially weird that free() aborts.  This points to some malloc/free
> inconsistency, as if the malloc (or new) call used another
> implementation of malloc than the aborting free call.  It may also
> be a memory overflow issue but that would show up on other platforms
> as well.

Thanks for your reply. Ah well, worth a try. I'm going to keep plugging 
away at this. My next goal is to produce a C++ example that doesn't 
require the STL, and after that maybe an example in plain C. Reducing 
the amount of code and eliminating number dependencies should help us 
see what's going on more clearly.

I'll post back here if and when I make more progress.

Dave.


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: static vs. shared linking

[David Stacey]

On 31/03/2015 17:35, David Stacey wrote:
> I'll post back here if and when I make more progress.

tl;dr: The problem was caused by a template being instantiated twice 
(one in the shared DLL and one in the main executable). This was fixed 
by compiling with '-frepo'. I do wonder if g++ should have resolved this 
automatically, as it does on Linux.

Longer version: Finally, I think I understand what's going on. 
std::basic_string<> contains a static array of bytes that represent an 
empty string [1]. If your string happens to be empty, the internals of 
std::basic_string<> point at this byte array rather than dynamically 
creating storage. At various points in the std::basic_string<> code, it 
tests to see if the address of the internal storage matches this byte 
array and acts accordingly.

There is supposed to be exactly one of these byte arrays for each 
instantiation of std::basic_string<>. However, in my example code (and 
also poco-1.6.0) there were two - one in the shared DLL and one in the 
main executable. Hence testing the pointer failed (the internal storage 
was pointing at the 'wrong' static byte array), and the 
std::basic_string<> code tried to 'delete' and area of memory that was 
never 'new'ed. Bang!

Reading the gcc documentation [2], it appears that on Linux the compiler 
resolves this automatically by following the 'Borland' model, but on 
Cygwin it does not. Is this a gcc issue - should we expect g++ on Cygwin 
to behave as per Linux here?

The solution is to compile with '-frepo', which works for both my test 
code and also poco-1.6.0 - although it has quite an impact on the 
compilation time (it trebles what was already a fairly lengthy 
compilation). Do you think this is the correct way to proceed, or should 
I look to explicitly export an instantiation of the std::basic_string<>s 
that Poco creates?

I can't believe that I'm the first person to fall foul of this - any 
library that relies heavily on templates risks falling into the same 
trap. For instance, how is this issue resolved in Boost? I've looked at 
'boost.cygport' and it isn't using '-frepo'...

Finally, many thanks to all those who have taken the time to help 
resolve this matter - you've (just about) managed to keep me sane! I 
have one more failing test to investigate, but hopefully I can get 
poco-1.6.0 released soon.

Dave.

[1] - /usr/lib/gcc/i686-pc-cygwin/4.9.2/include/c++/bits/basic_string.h 
line 178, member '_S_empty_rep_storage'.
[2] - 
https://gcc.gnu.org/onlinedocs/gcc-4.9.2/gcc/Template-Instantiation.html



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: static vs. shared linking

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 3105 bytes --]

On Apr  9 09:15, David Stacey wrote:
> On 31/03/2015 17:35, David Stacey wrote:
> >I'll post back here if and when I make more progress.
> 
> tl;dr: The problem was caused by a template being instantiated twice (one in
> the shared DLL and one in the main executable). This was fixed by compiling
> with '-frepo'. I do wonder if g++ should have resolved this automatically,
> as it does on Linux.
> 
> Longer version: Finally, I think I understand what's going on.
> std::basic_string<> contains a static array of bytes that represent an empty
> string [1]. If your string happens to be empty, the internals of
> std::basic_string<> point at this byte array rather than dynamically
> creating storage. At various points in the std::basic_string<> code, it
> tests to see if the address of the internal storage matches this byte array
> and acts accordingly.
> 
> There is supposed to be exactly one of these byte arrays for each
> instantiation of std::basic_string<>. However, in my example code (and also
> poco-1.6.0) there were two - one in the shared DLL and one in the main
> executable. Hence testing the pointer failed (the internal storage was
> pointing at the 'wrong' static byte array), and the std::basic_string<> code
> tried to 'delete' and area of memory that was never 'new'ed. Bang!
> 
> Reading the gcc documentation [2], it appears that on Linux the compiler
> resolves this automatically by following the 'Borland' model, but on Cygwin
> it does not. Is this a gcc issue - should we expect g++ on Cygwin to behave
> as per Linux here?

The documentation just claims that the Borland mode is supported on
ELF and Windows systems, it does not state what's the default behaviour
is in terms of -frepo.

It would sure be nice if Cygwin's g++ behaves the same as the Linux g++,
so if the Linux g++ sets -frepo automatically, we should do the same.

> The solution is to compile with '-frepo', which works for both my test code
> and also poco-1.6.0 - although it has quite an impact on the compilation
> time (it trebles what was already a fairly lengthy compilation). Do you
> think this is the correct way to proceed, or should I look to explicitly
> export an instantiation of the std::basic_string<>s that Poco creates?

Sorry, I'm not an expert on this template stuff.  But if -frepo works
for you it sounds like the right thing to go forward.

> I can't believe that I'm the first person to fall foul of this - any library
> that relies heavily on templates risks falling into the same trap. For
> instance, how is this issue resolved in Boost? I've looked at
> 'boost.cygport' and it isn't using '-frepo'...
> 
> Finally, many thanks to all those who have taken the time to help resolve
> this matter - you've (just about) managed to keep me sane! I have one more
> failing test to investigate, but hopefully I can get poco-1.6.0 released
> soon.

Nice.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: static vs. shared linking

[David Stacey]

On 09/04/15 18:24, Corinna Vinschen wrote:
> On Apr  9 09:15, David Stacey wrote:
>>   The solution is to compile with '-frepo', which works for both my test code
>>   and also poco-1.6.0 - although it has quite an impact on the compilation
>>   time (it trebles what was already a fairly lengthy compilation). Do you
>>   think this is the correct way to proceed, or should I look to explicitly
>>   export an instantiation of the std::basic_string<>s that Poco creates?
> Sorry, I'm not an expert on this template stuff.

Seriously, you're missing all the fun ;-)

> But if -frepo works
> for you it sounds like the right thing to go forward.

Thanks for your reply. OK, I'll stick with '-frepo'. I suspect that 
users of the Poco library will also be forced to use '-frepo' (otherwise 
they'll find themselves with the same problem I had). That shouldn't be 
too much of a hardship. I'll be sure to include that on the announcement.

Dave.


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: static vs. shared linking

[Larry Hall (Cygwin)]

On 04/09/2015 04:15 AM, David Stacey wrote:

<snip>

> I can't believe that I'm the first person to fall foul of this - any library
> that relies heavily on templates risks falling into the same trap.

<snip>

It's true that someone using STL strings has the potential to see this bug
but I doubt there are allot of template libraries out there pulling the same
memory trick or doing so with the same catastrophic results.  In addition,
this is not the first time this has come up as an issue (for Cygwin or other
platforms) in one form or another.  Here's a good reference:

<https://gcc.gnu.org/bugzilla/show_bug.cgi?id=16612>

There they recommend the "--enable-fully-dynamic-string" flag as a solution
for this particular problem but I agree if the "-frepo" will solve this as
well, it's better because it manages templates better overall and aligns
with Linux behavior.

-- 
Larry

_____________________________________________________________________

A: Yes.
 > Q: Are you sure?
 >> A: Because it reverses the logical flow of conversation.
 >>> Q: Why is top posting annoying in email?

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: static vs. shared linking

[David Stacey]

On 09/04/15 22:32, Larry Hall (Cygwin) wrote:
> On 04/09/2015 04:15 AM, David Stacey wrote:
>
> <snip>
>
>> I can't believe that I'm the first person to fall foul of this - any 
>> library
>> that relies heavily on templates risks falling into the same trap.
>
> <snip>
>
> It's true that someone using STL strings has the potential to see this 
> bug
> but I doubt there are allot of template libraries out there pulling 
> the same
> memory trick or doing so with the same catastrophic results. 

It's going to affect all templates that have a static member variable, 
where an instantiation of said template is passed across a DLL boundary. 
Maybe there aren't too many of those.

> In addition,
> this is not the first time this has come up as an issue (for Cygwin or 
> other
> platforms) in one form or another.  Here's a good reference:
>
> <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=16612>

Wow - that was an interesting read, thank you! Obviously a different 
context, but most definitely the same problem. It looks as though I 
stumbled into a bear trap that's been around for at least ten years.

> There they recommend the "--enable-fully-dynamic-string" flag as a 
> solution
> for this particular problem but I agree if the "-frepo" will solve 
> this as
> well, it's better because it manages templates better overall and aligns
> with Linux behavior.

I'm not sure I like '--enable-fully-dynamic-string', because it changes 
the API of std::string, and the two APIs are not interchangeable. So a 
library compiled with '--enable-fully-dynamic-string' can't be used with 
code that omitted that compiler option. You have to use it everywhere or 
not at all.

Poco is a network-centric library for other programmes to use. If I were 
to use '--enable-fully-dynamic-string' in compiling Poco then that would 
force users of the Poco library to use that compiler option too. And if, 
at the same time, they tried to link against another C++ API that wasn't 
built that way then the code wouldn't link - or if it did link it would 
crash when run.

So I either have to use '-frepo' or explicitly export the templates that 
Poco uses. '-frepo' might take longer to compile, and I suspect that it 
will force users of Poco to compile with '-frepo' as well. But it's 
guaranteed to catch other templates that I might have missed, and it 
will work with other C++ libraries that weren't built with 
'--enable-fully-dynamic-string'. So I'll stick with '-frepo'.

Anyway, thanks again for the link - interesting stuff.

Dave.


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: static vs. shared linking

[David Stacey]

On 24/03/2015 00:02, David Stacey wrote:
> I've been having difficulty building poco-1.6.0 for Cygwin for some
> time. I've managed to produce a test case that shows the problem:
>
> https://dl.dropboxusercontent.com/u/119453582/Cygwin/crashtest.tar.xz
>
> This archive contains source files that produce a very simple library.
> When linked statically, the code works fine. However, when linked as a
> shared DLL, the test crashes with a core dump. The behaviour is
> identical on x86 and x86_64 architectures.

As a quick test, I moved all of the inlined code from crash_library.h 
into the cpp file, as I wanted to rule out some crazy inlining problem. 
As expected, the shared library version still crashed.

Dave.



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple