PING Jan Nieuwenhuizen re libguile17

PING Jan Nieuwenhuizen re libguile17

[Dave Korn]

    Hi Jan, if you're currently around,


  It looks to me as if cygguile-17.dll needs to be rebuilt with an up-to-date
gcc with the fix for throwing strings across dll boundaries.  As we discovered
in a recent thread[*], autogen crashes on exit[**], and it appears to be due
to a problem in libguile[***].  I tried rebuilding it from source with the
latest gcc, and it fixes the problem, at least for me; here's what happens
when you run an OOTB build of autogen against the current shipping version of
libguile17:

/tmp/autogen/autogen-5.9.4 $ cygcheck ./agen5/autogen.exe
agen5/autogen.exe
  G:\cygwin\bin\cygwin1.dll
    G:\WINNT\System32\ADVAPI32.DLL
      G:\WINNT\System32\NTDLL.DLL
      G:\WINNT\System32\KERNEL32.DLL
      G:\WINNT\System32\RPCRT4.DLL
  G:\cygwin\bin\cygguile-17.dll
    G:\cygwin\bin\cygcrypt-0.dll
    G:\cygwin\bin\cyggmp-3.dll
    G:\cygwin\bin\cygintl-8.dll
      G:\cygwin\bin\cygiconv-2.dll
    G:\cygwin\bin\cygltdl-3.dll
@_______. .
(       /"\
 ||--||(___)
 '"  '"'---'
/tmp/autogen/autogen-5.9.4 $ ./agen5/autogen.exe  < /dev/null
No definition data were read
AutoGen aborting on signal 11 (Segmentation fault) in state ABORTING
@_______. .
(       /"\
 ||--||(___)
 '"  '"'---'
/tmp/autogen/autogen-5.9.4 $

  And here's what happens when I try it with the rebuilt version of
libguile-17:


/tmp/autogen/autogen-5.9.4 $ cygcheck ./agen5/autogen.exe
agen5/autogen.exe
  G:\cygwin\bin\cygwin1.dll
    G:\WINNT\System32\ADVAPI32.DLL
      G:\WINNT\System32\NTDLL.DLL
      G:\WINNT\System32\KERNEL32.DLL
      G:\WINNT\System32\RPCRT4.DLL
  T:\cygwin\build\install\bin\cygguile-17.dll
    G:\cygwin\bin\cygcrypt-0.dll
    G:\cygwin\bin\cyggmp-3.dll
    G:\cygwin\bin\cygintl-8.dll
      G:\cygwin\bin\cygiconv-2.dll
    G:\cygwin\bin\cygltdl-3.dll
@_______. .
(       /"\
 ||--||(___)
 '"  '"'---'
/tmp/autogen/autogen-5.9.4 $ ./agen5/autogen.exe  < /dev/null
No definition data were read
@_______. .
(       /"\
 ||--||(___)
 '"  '"'---'
/tmp/autogen/autogen-5.9.4 $


  So I wonder if you could verify which gcc package you built the libguile
release with.  If you wanted to test against autogen, I uploaded a tarball of
the build I was using[****], or it builds OOTB - if you have an updated
cygguile-17.dll, that is ...


    cheers,
      DaveK

[*] - http://www.cygwin.com/ml/cygwin/2008-01/msg00244.html
[**] - http://www.cygwin.com/ml/cygwin/2008-01/msg00247.html
[***] -
http://www.mail-archive.com/autogen-users@lists.sourceforge.net/msg00077.html
[****] -
http://rapidshare.com/files/85998958/autogen-5.9.4-0-strip.tar.bz2.html
-- 
Can't think of a witty .sigline today....

Re: PING Jan Nieuwenhuizen re libguile17

[Jan Nieuwenhuizen]

Dave Korn:

Hi Dave,

> Hi Jan, if you're currently around,

Thanks for the ping, I've been away.

>   It looks to me as if cygguile-17.dll needs to be rebuilt with an up-to-date
> gcc with the fix for throwing strings across dll boundaries.

Hmm.  It seems that gcc is at

    install: release/gcc/gcc-core/gcc-core-3.4.4-3.tar.bz2 3704219
f50239b82e65821f5

I have been using gcc-4.1.1.  Have cygwin specific fixes been made?  Are
these fixes in upstream gcc 4.x?  Do I need a configure flag?

Greetings,
Jan.

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond - The music
typesetter
http://www.xs4all.nl/~jantien       | http://www.lilypond.org

RE: PING Jan Nieuwenhuizen re libguile17

[Dave Korn]

On 31 January 2008 16:05, Jan Nieuwenhuizen wrote:

> Dave Korn:
> 
> Hi Dave,
> 
>> Hi Jan, if you're currently around,
> 
> Thanks for the ping, I've been away.

  Thanks for getting back to me!
 
>>   It looks to me as if cygguile-17.dll needs to be rebuilt with an
>> up-to-date gcc with the fix for throwing strings across dll boundaries.
> 
> Hmm.  It seems that gcc is at
> 
>     install: release/gcc/gcc-core/gcc-core-3.4.4-3.tar.bz2 3704219
> f50239b82e65821f5
> 
> I have been using gcc-4.1.1.  Have cygwin specific fixes been made?  Are
> these fixes in upstream gcc 4.x?  Do I need a configure flag?

  Ah, if you used 4.1.1 to compile libguile, that would certainly explain it.

  There is a cygwin-local fix in 3.4.4-3 that is essential for correctness
when throwing exceptions or passing std::string objects across DLL boundaries.
This is a bug in upstream gcc: see 

http://gcc.gnu.org/PR24196

  The fix is almost - but not quite - equivalent to applying the
"--enable-fully-dynamic-string" configure option, but hopefully has the
advantage of being ABI-compatible.


  As a bit of generic advice (to all package maintainers, not just you), I
really wouldn't recommend using 4.x as a production compiler for cygwin
packages for distribution just yet.  

    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....

RE: PING Jan Nieuwenhuizen re libguile17

[Jan Nieuwenhuizen]

Dave Korn writes:

Hi Dave,

>   There is a cygwin-local fix in 3.4.4-3 that is essential for correctness
> when throwing exceptions or passing std::string objects across DLL boundaries.
> This is a bug in upstream gcc: see 
> 
> http://gcc.gnu.org/PR24196

I see...

>  The fix is almost - but not quite - equivalent to applying the
> "--enable-fully-dynamic-string" configure option, but hopefully has the
> advantage of being ABI-compatible.

>  As a bit of generic advice (to all package maintainers, not just you), I
> really wouldn't recommend using 4.x as a production compiler for cygwin
> packages for distribution just yet.  

LilyPond does not support gcc 3.x anymore, would it be of any use to
try rebuilding guile with the --enable-fully-dynamic string option?

Jan.

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond - The music typesetter
http://www.xs4all.nl/~jantien       | http://www.lilypond.org

RE: PING Jan Nieuwenhuizen re libguile17

[Dave Korn]

On 04 February 2008 14:10, Jan Nieuwenhuizen wrote:

>> http://gcc.gnu.org/PR24196
> 
> I see...
> 
>>  The fix is almost - but not quite - equivalent to applying the
>> "--enable-fully-dynamic-string" configure option, but hopefully has the
>> advantage of being ABI-compatible.
> 
>>  As a bit of generic advice (to all package maintainers, not just you), I
>> really wouldn't recommend using 4.x as a production compiler for cygwin
>> packages for distribution just yet.
> 
> LilyPond does not support gcc 3.x anymore, would it be of any use to
> try rebuilding guile with the --enable-fully-dynamic string option?

  You need to build the /compiler/ using that option, not the target
application, but apart from that yes: it should then fix your builds of
lilypond, libguile, et al.


    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....

RE: PING Jan Nieuwenhuizen re libguile17

[Jan Nieuwenhuizen]

Dave Korn:

Hi Dave,

> > LilyPond does not support gcc 3.x anymore, would it be of any use to
> > try rebuilding guile with the --enable-fully-dynamic string option?
> 
>   You need to build the /compiler/ using that option, not the target
> application, but apart from that yes: it should then fix your builds of
> lilypond, libguile, et al.

I rebuilt gcc (4.1.1) with --enable-fully-dynamic-string*) and rebuilt
guile
(http://lilypond.org/cygwin/release/guile/libguile17/libguile17-1.8.2-3.tar.bz2)
and today finally got round to testing it.  It does not seem to help.

I unpacked the autogen you provided
(http://rs323.rapidshare.com/files/85998958/autogen-5.9.4-0-strip.tar.bz2)

and ran

   15:52:49 root@Abbicci:~/autogen-5.9.4
   $ usr/bin/autogen.exe --help
   autogen (GNU AutoGen) - The Automated Program Generator - Ver. 5.9.4
   USAGE:  autogen [ -<flag> [<val>] | --<name>[{=| }<val>] ]... [ <def-file> ]
   ...
   please send bug reports to:  autogen-users@lists.sourceforge.net
   AutoGen aborting on signal 11 (Segmentation fault) in state OPTIONS
   Hangup
   [129]15:53:00 root@Abbicci:~/autogen-5.9.4

Greetings,
Jan.

*) full configure command:
! cd /home/janneke/vc/gub-cygwin/target/cygwin/build/cross/gcc-4.1.1 && /home/janneke/vc/gub-cygwin/target/cygwin/src/cross/gcc-4.1.1/configure --prefix=/home/janneke/vc/gub-cygwin/target/cygwin/install/cross/gcc-4.1.1-root/usr --program-prefix=i686-cygwin- --prefix=/home/janneke/vc/gub-cygwin/target/cygwin/root/usr/cross --with-slibdir=/usr/lib --target=i686-cygwin --with-sysroot=/home/janneke/vc/gub-cygwin/target/cygwin/root --disable-multilib  --with-as=/home/janneke/vc/gub-cygwin/target/cygwin/root/usr/cross/bin/i686-cygwin-as --with-ld=/home/janneke/vc/gub-cygwin/target/cygwin/root/usr/cross/bin/i686-cygwin-ld --enable-static --enable-shared  --enable-languages=c,c++  --enable-libstdcxx-debug  --with-newlib --enable-threads --enable-fully-dynamic-string
-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond - The music typesetter
http://www.xs4all.nl/~jantien       | http://www.lilypond.org

Re: PING Jan Nieuwenhuizen re libguile17

[Brian Dessent]

Jan Nieuwenhuizen wrote:

> I rebuilt gcc (4.1.1) with --enable-fully-dynamic-string*) and rebuilt
> guile
> (http://lilypond.org/cygwin/release/guile/libguile17/libguile17-1.8.2-3.tar.bz2)
> and today finally got round to testing it.  It does not seem to help.

Note that the Cygwin gcc is not using --enable-fully-dynamic-string, it
is using the patch in PR24196 which is a compromise between the
pessimization of assuming fully dynamic strings and the optimization of
assuming one global instance of _S_empty_rep_storage.

Also, I'm confused on another issue: if libguile exposes a C++ ABI then
mixing 4.1 and 3.4 should be incompatible anyway, regardless of PR24196,
by the fact that there are so many g++ changes between those major
versions.  However looking at the exports of the DLL I see no C++
symbols.  But if libguile does not expose a C++ ABI then why does the
PR24196-patched gcc cause it to work again?  Or is it that it uses C++
internally and that is where the std::string-across-DLL-boundaries
problem occurs?

Brian

Re: PING Jan Nieuwenhuizen re libguile17

[Jan Nieuwenhuizen]

Brian Dessent:

> Note that the Cygwin gcc is not using --enable-fully-dynamic-string, it
> is using the patch in PR24196 which is a compromise between the
> pessimization of assuming fully dynamic strings and the optimization of
> assuming one global instance of _S_empty_rep_storage.

Hmm, so I should try a forward-patch of that for 4.1?  Can someone give
me some pointers about this?

> Also, I'm confused on another issue: if libguile exposes a C++ ABI then
> mixing 4.1 and 3.4 should be incompatible anyway

AFAIK, 3.4 broke its abi with earlier 3.x exactly because of providing a
3.x version that was abi-compatible with the (much stricter and
sometimes problematic) 4.x series.

> , regardless of PR24196,
> by the fact that there are so many g++ changes between those major
> versions.  However looking at the exports of the DLL I see no C++
> symbols.  But if libguile does not expose a C++ ABI then why does the
> PR24196-patched gcc cause it to work again?  Or is it that it uses C++
> internally and that is where the std::string-across-DLL-boundaries
> problem occurs?

libguile does not use C++.  As far as I understand, it is a problem that
is introduced while linking the dlls.

Jan.

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond - The music typesetter
http://www.xs4all.nl/~jantien       | http://www.lilypond.org

Re: PING Jan Nieuwenhuizen re libguile17

[Brian Dessent]

Jan Nieuwenhuizen wrote:

> libguile does not use C++.  As far as I understand, it is a problem that
> is introduced while linking the dlls.

The problem addressed by PR24196 and --enable-fully-dynamic-string is
entirely centered around an implementation detail of std::string in
libstdc++.  Specifically, libstdc++ has an optimization where empty
std::string objects don't get any memory allocated to them; their
internal storage pointers are instead set equal to a reference to some
static _S_empty_rep_storage object.  This allows them to be created
efficiently as well as allowing comparing for equality with other empty
strings trivially and efficiently.  This works great when libstdc++ is a
shared library as this ensures that there will only be one instance of
_S_empty_rep_storage for the entire process, so libstdc++ can always
recognise an empty std::string by comparing its pointer to the address
of _S_empty_rep_storage.

However Cygwin has never had the luxury of shared gcc target libraries
so every module that links with libstdc++ does so statically.  If you
have a process consisting of an .exe and a .dll which are both linked to
libstdc++, both modules will each have their own instance of
_S_empty_rep_storage.  If a function in one module tries to pass an
empty std::string argument to a function in the other, there will be a
crash because that module will not see that this object is an empty
string because its pointer does not point to that module's version of
_S_empty_rep_storage.  It will see it as a regular, fully initialized
string when in fact it is not.  The fix is to simply pessimize this
optimization by initializing empty std::objects with their own storage
instead of relying on having a global singleton.

The issue should have no effect on code that does not pass std::string
objects across DLLs, and by extension it should have no effect on C.  It
has nothing to do with DLLs in general or linking, and I'm struggling
more and more to see how it would be relevant in this case.

However, I have in fact verified myself that rebuilding libguile with
Cygwin gcc 3.4.4 with the PR24196 patch does fix the problem.  I'm now
wonderding if PR24196 is a red herring and the real issue is an ABI
incompatibility between 3.4 and 4.1.  To verify this hypothesis, I will
try rebuilding libguile with the older version of Cygwin gcc 3.4.4
without the PR24196 patch.

Brian

RE: PING Jan Nieuwenhuizen re libguile17

[Dave Korn]

Brian Dessent wrote on 22 March 2008 16:50:

> However, I have in fact verified myself that rebuilding libguile with
> Cygwin gcc 3.4.4 with the PR24196 patch does fix the problem.  

  Thanks for the confirmation.

> I'm now
> wonderding if PR24196 is a red herring and the real issue is an ABI
> incompatibility between 3.4 and 4.1.  To verify this
> hypothesis, I will
> try rebuilding libguile with the older version of Cygwin gcc 3.4.4
> without the PR24196 patch. 

  I don't think I actually tried this at the time, I used a self-built
libguile vs. the shipped distro version.  It'll be interesting to see your
results.


[  busy playing with gcc 4 myself this weekend, trying out a shared libgcc.
]

    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....

Re: PING Jan Nieuwenhuizen re libguile17

[Brian Dessent]

Dave Korn wrote:

>   I don't think I actually tried this at the time, I used a self-built
> libguile vs. the shipped distro version.  It'll be interesting to see your
> results.

I rebuilt libguile with gcc 3.4.4-1 (without any PR24196 patches) and an
autogen using it passes all tests.  So it seems this is simply an ABI
incompatibility between 4.1 and 3.4.

Brian

RE: PING Jan Nieuwenhuizen re libguile17

[Dave Korn]

Jan Nieuwenhuizen wrote on 25 March 2008 09:50:

> Brian Dessent:
> 
> > I rebuilt libguile with gcc 3.4.4-1 (without any PR24196 patches) and an
> > autogen using it passes all tests.  So it seems this is simply an ABI
> > incompatibility between 4.1 and 3.4.
> 
> Thanks.  I'll start a test using gcc-4.0.4; with some luck that's new
> enough for LilyPond and still binary compatible Cygwin.
> 
> Jan.

  The C ABI is meant to be the same between 3.x and 4.x, so since there's
apparently no C++ involved, I don't think this is necessarily "simply an ABI
incompatibility", I think it's a real regression.

  Jan, please just to clarify: did you build the actual release version of
libguile with gcc4?  I couldn't tell from the context whether you were only
using it to try the --enable-fully-dynamic-string experiment.

  I don't recommend using gcc 4 series for production releases yet.  There
was a lot of instability after the change to tree-ssa infrastructure and
both 4.0 and 4.1 series had a big bunch of regressions.  (I think it's
starting to get there with the 4.2 and 4.3 series and that's why I'm 

  I would *strongly* recommend all maintainers stick with the official
release of the compiler when building packages for the distro.  It's the
same version the DLL is built with; we should use it for the apps too.  C is
/supposed/ to be ABI-compatible, but as we've discovered, it's not always as
simple as that.  Using the exact same compiler for the DLL and the packages
means we've got one less thing that can go wrong.

  Is there some specific problem in building lilypond that is resolved by
using gcc 4?  I *am* willing to do maintenance releases of the 3.4 compiler
if there are important bugs to fix, doubly so if it's needed to support a
package in the official distro.  (I'm also working in the background on an
experimental gcc 4 package).

    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....

RE: PING Jan Nieuwenhuizen re libguile17

[Jan Nieuwenhuizen]

Dave Korn writes:

>   The C ABI is meant to be the same between 3.x and 4.x, so since there's
> apparently no C++ involved, I don't think this is necessarily "simply an ABI
> incompatibility", I think it's a real regression.

I tested compiling libguile for Cygwin using gcc-4.0.0, gcc-4.0.4, and
gcc-4.2.3 but all have the same segfault in autogen.

>   Jan, please just to clarify: did you build the actual release version of
> libguile with gcc4?

Yes, plain 1.8.2.

>   I don't recommend using gcc 4 series for production releases yet.

I know, but for all my packages it seemed to work for years now...

>   Is there some specific problem in building lilypond that is resolved by
> using gcc 4?

I'm investigating this.  It's quite tricky, currently LilyPond itself
does not compile using gcc-4.2.3, but we do have success reports with
4.3.0.

>   I *am* willing to do maintenance releases of the 3.4 compiler
> if there are important bugs to fix, doubly so if it's needed to support a
> package in the official distro.

What is the best gcc 3.4 to use for Cygwin?  Unpacking Cygwin's gcc/g++
source archive in a python script is a bit of a pain (that's an
understatement :-).

>   (I'm also working in the background on an
> experimental gcc 4 package).

That would be nice.
Jan.

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond - The music typesetter
http://www.xs4all.nl/~jantien       | http://www.lilypond.org

RE: PING Jan Nieuwenhuizen re libguile17

[Dave Korn]

Jan Nieuwenhuizen wrote on 26 March 2008 11:22:

> >   I *am* willing to do maintenance releases of the 3.4 compiler
> > if there are important bugs to fix, doubly so if it's needed to support
> > a package in the official distro.
> 
> What is the best gcc 3.4 to use for Cygwin?  

  /usr/bin/gcc, i.e. the latest release in the distro.

> Unpacking Cygwin's gcc/g++
> source archive in a python script is a bit of a pain (that's an
> understatement :-). 

  I don't understand why that would be necessary?

> >   (I'm also working in the background on an
> > experimental gcc 4 package).
> 
> That would be nice.

  Dwarf2 debug and exceptions, shared libgcc.  Seems to mostly work but the
testsuite's still running (since sunday!).  Two libstdc++ regressions showed
up so far, g++ clean, gcc still running.


    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....

RE: PING Jan Nieuwenhuizen re libguile17

[Jan Nieuwenhuizen]

Dave Korn writes:

>   /usr/bin/gcc, i.e. the latest release in the distro.

Ah, I meant source-wise, I do not run Cygwin.

> > Unpacking Cygwin's gcc/g++
> > source archive in a python script is a bit of a pain (that's an
> > understatement :-). 
> 
>   I don't understand why that would be necessary?

Releases are not so often that it is annoying, I manually repacked them.
But still.  When working with computers, people find it handy to
automate repetitive tasks, such as cross-building a whole
dependency-tree of packages for a number of platforms.  Python is a
handy tool to automate such a task, which is what we have done in GUB
(LilyPond's Grand Unified Builder).  Tracking upstream's sources
currently works easily with tarballs, CVS, GIT, bazaar and subversion.
Only Cygwin's [gcc] source archives are a bit of a challenge ;-) 

Greetings,
Jan.

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond - The music typesetter
http://www.xs4all.nl/~jantien       | http://www.lilypond.org

RE: PING Jan Nieuwenhuizen re libguile17

[Dave Korn]

Jan Nieuwenhuizen wrote on 26 March 2008 15:20:

> Dave Korn writes:
> 
> >   /usr/bin/gcc, i.e. the latest release in the distro.
> 
> Ah, I meant source-wise, I do not run Cygwin.

  It comes in a source distribution as well, same as all cygwin packages.

> > > Unpacking Cygwin's gcc/g++
> > > source archive in a python script is a bit of a pain (that's an
> > > understatement :-).
> > 
> >   I don't understand why that would be necessary?
> 
> Releases are not so often that it is annoying, I manually
> repacked them.
> But still.  When working with computers, people find it handy to
> automate repetitive tasks

  Is this meant to be as offensively patronising as it came out?  It reads
like it was aimed at a two-year old child.  If I want the Janet and John
guide to computer fundamentals, I'll ask for it.

    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....

RE: PING Jan Nieuwenhuizen re libguile17

[Jan Nieuwenhuizen]

Dave Korn:

>   Is this meant to be as offensively patronising as it came out?

Sorry, I should have added at least a smiley.  Having said that, I've
torn quite some hair over trying to unpack gcc's source packages.  They
seem unnecessary difficult to unpack automatically, and as you asked why
one would want to do that, I decided to flip.

Jan.

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond - The music typesetter
http://www.xs4all.nl/~jantien       | http://www.lilypond.org

RE: PING Jan Nieuwenhuizen re libguile17

[Dave Korn]

Jan Nieuwenhuizen wrote on 27 March 2008 11:10:

> Dave Korn:
> 
> >   Is this meant to be as offensively patronising as it came out?
> 
> Sorry, I should have added at least a smiley.  Having said that, I've
> torn quite some hair over trying to unpack gcc's source
> packages.  They
> seem unnecessary difficult to unpack automatically, and as
> you asked why
> one would want to do that, I decided to flip.

  What exactly is the nature of the problem?  It all seems fairly
straightforward to me, but that's not to say it can't be changed if there's
a reason to.

    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....

RE: PING Jan Nieuwenhuizen re libguile17

[Jan Nieuwenhuizen]

Dave Korn writes:

>   What exactly is the nature of the problem?  It all seems fairly
> straightforward to me, but that's not to say it can't be changed if there's
> a reason to.

As I remember the problematic thing is that there are so many options
for this second variant of Cygwin's source package.

I went as far as the sorry burp of python code below before I gave up,
this works for some packages, but not for all.  The problem becomes more
clear when you imagine the different [optional stuff] mentioned in the
function's docstring.

Jan.

def untar_cygwin_src_package_variant2 (self, file_name, split=False):
    '''Unpack this unbelievably *interesting* version of Cygwin source packages.

foo[version][-split]-x.y.z-b.tar.bz2 contains
foo[-split]-x.y.z.tar.[bz2|gz] and foo[version]-x.y.z-b.patch
(and optionally foo[version]-x.y.z-b.patch2 ...).
foo-x.y.z.tar.[bz2|gz] contains foo-x.y.z.  The patch contains patches
against all foo split source balls, so applying it may fail partly and
complain about missing files.'''

    file_name = self.expand (file_name)
    unpackdir = os.path.dirname (self.expand (self.srcdir ()))
    t = misc.split_ball (file_name)
    print 'split: ' + `t`
    no_src = re.sub ('-src', '', file_name)
    base = re.sub ('\.tar\..*', '', no_src)
    # FIXME: use split iso custom ball_re macramee
    ball_re = '^([a-z]+)([.0-9]+)?(-[a-z+]+)?(.*)(-[0-9]+)'
    m = re.match (ball_re, base)
    if m.group (3):
        second_tarball = re.sub (ball_re, '\\1\\3\\4', base)
    else:
        second_tarball = re.sub (ball_re, '\\1\\4', base)
    print 'second_tarball: ' + second_tarball
    if split and m.group (3):
        second_tarball_contents = re.sub (ball_re, '\\1\\3\\4', base)
    else:
        second_tarball_contents = re.sub (ball_re, '\\1\\4', base)
    print 'second_tarball_contents: ' + second_tarball_contents
    flags = '-jxf'
    self.system ('''
rm -rf %(unpackdir)s/%(base)s
tar -C %(unpackdir)s %(flags)s %(downloads)s/%(file_name)s
''',
                 locals ())
    tgz = 'tar.bz2'
    if not os.path.exists (unpackdir + '/' + second_tarball + '.' + tgz):
        flags = '-zxf'
        tgz = 'tar.gz'
    self.system ('''
tar -C %(unpackdir)s %(flags)s %(unpackdir)s/%(second_tarball)s.%(tgz)s
''',
                 locals ())
    if split:
        return
    if m.group (2):
        patch = re.sub (ball_re, '\\1\\2\\4\\5.patch', base)
    else:
        patch = re.sub (ball_re, '\\1\\4\\5.patch', base)
    print 'patch: ' + patch
    print 'scrdir: ', self.expand ('%(srcdir)s')
    self.system ('''
cd %(unpackdir)s && mv %(second_tarball_contents)s %(base)s
cd %(srcdir)s && patch -p1 -f < %(unpackdir)s/%(patch)s || true
''',
                 locals ())
-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond - The music typesetter
http://www.xs4all.nl/~jantien       | http://www.lilypond.org

RE: PING Jan Nieuwenhuizen re libguile17

[Dave Korn]

[-- Attachment #1: Type: text/plain, Size: 1971 bytes --]

Jan Nieuwenhuizen wrote on 22 March 2008 13:03:

> Brian Dessent:
> 
> > Note that the Cygwin gcc is not using
> --enable-fully-dynamic-string, it
> > is using the patch in PR24196 which is a compromise between the
> > pessimization of assuming fully dynamic strings and the optimization of
> > assuming one global instance of _S_empty_rep_storage.
> 
> Hmm, so I should try a forward-patch of that for 4.1?  Can someone give
> me some pointers about this?

  Well, here is an untested port of it to 4.1 for you to play with.  We're
removing the _S_empty_rep optimisation by 1) removing conditionals so it
gets treated the same as all other strings and 2) useing _M_refcopy instead
of _M_refdata so it gets the same reference counting behaviour as other
strings.

> > Also, I'm confused on another issue: if libguile exposes a C++ ABI then
> > mixing 4.1 and 3.4 should be incompatible anyway
> 
> AFAIK, 3.4 broke its abi with earlier 3.x exactly because of providing a
> 3.x version that was abi-compatible with the (much stricter and
> sometimes problematic) 4.x series.

  In general, you can't even expect compatibility even between X.y and X.z,
although most of those abi breaks are smaller.

> > , regardless of PR24196,
> > by the fact that there are so many g++ changes between those major
> > versions.  However looking at the exports of the DLL I see no C++
> > symbols.  But if libguile does not expose a C++ ABI then why does the
> > PR24196-patched gcc cause it to work again?  Or is it that it uses C++
> > internally and that is where the std::string-across-DLL-boundaries
> > problem occurs?
> 
> libguile does not use C++.  As far as I understand, it is a problem that
> is introduced while linking the dlls.

  Hmm, if it's not that then I don't know what, nor why autogen started
working for me when I recompiled libguile, but the crashes definitely seemed
to unwind through it.


    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....

[-- Attachment #2: br1.diff --]
[-- Type: application/octet-stream, Size: 3275 bytes --]

Index: basic_string.h
===================================================================
--- basic_string.h	(revision 133447)
+++ basic_string.h	(working copy)
@@ -226,11 +226,8 @@ namespace std
 	void
 	_M_dispose(const _Alloc& __a)
 	{
-#ifndef _GLIBCXX_FULLY_DYNAMIC_STRING
-	  if (__builtin_expect(this != &_S_empty_rep(), false))
-#endif
-	    if (__gnu_cxx::__exchange_and_add(&this->_M_refcount, -1) <= 0)
-	      _M_destroy(__a);
+	  if (__gnu_cxx::__exchange_and_add(&this->_M_refcount, -1) <= 0)
+	    _M_destroy(__a);
 	}  // XXX MT
 
 	void
@@ -239,10 +236,7 @@ namespace std
 	_CharT*
 	_M_refcopy() throw()
 	{
-#ifndef _GLIBCXX_FULLY_DYNAMIC_STRING
-	  if (__builtin_expect(this != &_S_empty_rep(), false))
-#endif
-            __gnu_cxx::__atomic_add(&this->_M_refcount, 1);
+	  __gnu_cxx::__atomic_add(&this->_M_refcount, 1);
 	  return _M_refdata();
 	}  // XXX MT
 
@@ -2048,11 +2042,7 @@ namespace std
   template<typename _CharT, typename _Traits, typename _Alloc>
     inline basic_string<_CharT, _Traits, _Alloc>::
     basic_string()
-#ifndef _GLIBCXX_FULLY_DYNAMIC_STRING
-    : _M_dataplus(_S_empty_rep()._M_refdata(), _Alloc()) { }
-#else
-    : _M_dataplus(_S_construct(size_type(), _CharT(), _Alloc()), _Alloc()) { }
-#endif
+    : _M_dataplus(_S_empty_rep()._M_refcopy(), _Alloc()) { }
 
   // operator+
   /**
Index: basic_string.tcc
===================================================================
--- basic_string.tcc	(revision 133447)
+++ basic_string.tcc	(working copy)
@@ -90,10 +90,9 @@ namespace std
       _S_construct(_InIterator __beg, _InIterator __end, const _Alloc& __a,
 		   input_iterator_tag)
       {
-#ifndef _GLIBCXX_FULLY_DYNAMIC_STRING
 	if (__beg == __end && __a == _Alloc())
-	  return _S_empty_rep()._M_refdata();
-#endif
+	  return _S_empty_rep()._M_refcopy();
+
 	// Avoid reallocation for common case.
 	_CharT __buf[128];
 	size_type __len = 0;
@@ -136,10 +135,9 @@ namespace std
       _S_construct(_InIterator __beg, _InIterator __end, const _Alloc& __a,
 		   forward_iterator_tag)
       {
-#ifndef _GLIBCXX_FULLY_DYNAMIC_STRING
 	if (__beg == __end && __a == _Alloc())
-	  return _S_empty_rep()._M_refdata();
-#endif
+	  return _S_empty_rep()._M_refcopy();
+
 	// NB: Not required, but considered best practice.
 	if (__builtin_expect(__is_null_pointer(__beg) && __beg != __end, 0))
 	  __throw_logic_error(__N("basic_string::_S_construct NULL not valid"));
@@ -164,10 +162,9 @@ namespace std
     basic_string<_CharT, _Traits, _Alloc>::
     _S_construct(size_type __n, _CharT __c, const _Alloc& __a)
     {
-#ifndef _GLIBCXX_FULLY_DYNAMIC_STRING
       if (__n == 0 && __a == _Alloc())
-	return _S_empty_rep()._M_refdata();
-#endif
+	return _S_empty_rep()._M_refcopy();
+
       // Check for out_of_range and length_error exceptions.
       _Rep* __r = _Rep::_S_create(__n, size_type(0), __a);
       if (__n)
@@ -435,10 +432,6 @@ namespace std
     basic_string<_CharT, _Traits, _Alloc>::
     _M_leak_hard()
     {
-#ifndef _GLIBCXX_FULLY_DYNAMIC_STRING
-      if (_M_rep() == &_S_empty_rep())
-	return;
-#endif
       if (_M_rep()->_M_is_shared())
 	_M_mutate(0, 0, 0);
       _M_rep()->_M_set_leaked();

Re: PING Jan Nieuwenhuizen re libguile17

[Jan Nieuwenhuizen]

Dave Korn writes:

Hi Dave,

>   It looks to me as if cygguile-17.dll needs to be rebuilt with an up-to-date
> gcc with the fix for throwing strings across dll boundaries.  As we discovered
> in a recent thread[*], autogen crashes on exit[**], and it appears to be due
> to a problem in libguile[***].  I tried rebuilding it from source with the
> latest gcc, and it fixes the problem, at least for me;

I cannot reproduce this fix.  After [cross]compiling Guile with Cygwin's
gcc-3.4.4-3 I still got the autogen crash, so I built Guile on a freshly
installed cygwin, and autogen still crashes for me.  There must be
something else going on here.  It would not surprise me if the current
guile, cross-compiled with gcc-4.1 turned out to be fine...

Jan.

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond - The music typesetter
http://www.xs4all.nl/~jantien       | http://www.lilypond.org