sigsegv on aix with gcc

sigsegv on aix with gcc

[Jędrzej Dudkiewicz]

Hi,

I maintain an application (rather large) for AIX (among others). When
compiled with any level of optimization, it always crashes, always in
the same place. What's funny, the same thing happens when I compile on
AIX 4.3 with g++ 3.3.2 and on AIX 5.2 with g++ 4.5.3. Here are stack
traces, both from our AIX 5.3 machine:

Binary created with g++ 4.5.3:
#0  0x1006bb58 in std::locale::operator= (this=<error reading
variable>, __other=<error reading variable>) at  _start_ :178
#1  0x10072110 in std::ios_base::_M_init (this=0x215a3bdc)
    at /jd/gccbuild/aix-4.5.2/gcc-4.5.2/libstdc++-v3/src/ios_locale.cc:44
#2  0x10072014 in std::basic_ios<char, std::char_traits<char> >::init
(this=<error reading variable>,
    __sb=<error reading variable>)
#3  0x10219db0 in CCertificateFile::GetValue (UID=..., section=..., key=...)
    at /opt/freeware/era-gcc-4.5.2/lib/gcc/powerpc-ibm-aix5.3.0.0/4.5.2/include/c++/istream:584
#4  0x1021ac08 in CCertificateFile::GivePrivateKey (ID=...) at
Shared/FileIO/CertificateFile.cpp:290
#5  0x102d47a8 in CSslSocket::send_messages_to_server (ssl_sock=0x217ad428)
    at Shared/NetworkIO/SslSockets/SslSockets.cpp:3047
#6  0x102d83e0 in CSslSocket::perform_protocol_as_client (ssl_sock=0x217ad428)
    at Shared/NetworkIO/SslSockets/SslSockets.cpp:632
#7  0x102da514 in CSslSocket::ssl_connect (ssl_sock=0x217ad428,
sServer=..., nPort=9035)
    at Shared/NetworkIO/SslSockets/SslSockets.cpp:736
#8  0x1016312c in CESSocket::Connect (this=0x2157f3c8, szIP=0x2157d024
<error reading variable>, nPort=9035)
    at Shared/NetworkIO/ESSockets/ESSocket.cpp:42
#9  0x10422224 in CPoolableSocket::Connect (this=0x2157f3c8,
szIP=0x2157d024 <error reading variable>, nPort=9035)
    at Shared/NetworkIO/ESSockets/PoolableSocket.cpp:68
#10 0x10132330 in CConnectionManager::AcquireParentConnection ()
    at Shared/NetworkIO/ConnectionManager/ConnectionManager.cpp:925
#11 0x1000349c in CServiceModule::StartStopThread
(pEracentServiceControl=0x20549128) at
Shared/Service/UnixService.cpp:260
#12 0xd010d644 in ?? ()
#13 0x00000000 in ?? ()

Binary created with g++ 3.3.2:
#0  0xd03e2f88 in ?? ()
#1  0x100174b8 in std::locale::operator= (this=<error reading
variable>, __other=<error reading variable>) at  _start_ :360
#2  0x10038878 in std::ios_base::_M_init (this=<error reading
variable>) at  _start_ :299
#3  0x10038770 in std::basic_ios<char, std::char_traits<char> >::init
(this=<error reading variable>,
    __sb=<error reading variable>) at  _start_ :150
#4  0x10160a0c in CCertificateFile::GetValue (UID=..., section=...,
key=...) at /opt/freeware/include/c++/3.3.2/istream:106
Backtrace stopped: frame did not save the PC

Executable file is statically linked with libstdc++.

GCC 3.3.2:
Reading specs from /opt/freeware/lib/gcc-lib/powerpc-ibm-aix4.3.3.0/3.3.2/specs
Configured with: ../gcc-3.3.2/configure --with-as=/usr/bin/as
--with-ld=/usr/bin/ld --disable-nls --enable-languages=c,c++
--prefix=/opt/freeware --enable-threads --disable-shared
Thread model: aix
gcc version 3.3.2

GCC 4.5.2:
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/opt/freeware/era-gcc-4.5.2/libexec/gcc/powerpc-ibm-aix5.3.0.0/4.5.2/lto-wrapper
Target: powerpc-ibm-aix5.3.0.0
Configured with: /jd/gccbuild/aix-4.5.2/gcc-4.5.2/configure
--with-as=/usr/bin/as --with-ld=/usr/bin/ld --enable-languages=c,c++
--prefix=/opt/freeware/era-gcc-4.5.2/ --enable-threads
--enable-version-specific-runtime-libs --disable-shared
--program-suffix=-4.5.2 --disable-nls
Thread model: aix
gcc version 4.5.2 (GCC)

If memory serves me well, 3.3.2 is taken from bullfreeware.com and
4.5.2 is compiled by me.

Juding by results (same compiler, same place) an error must be quite
obvious and in my code, but I have no idea what I should check. I
tried increasing stack size, adding -fno-strict-aliasing and few more,
but to no avail.

Can you give me ANY pointers regarding further steps? Thanks in advance.
-- 
Jędrzej Dudkiewicz

I really hate this damn machine, I wish that they would sell it.
It never does just what I want, but only what I tell it.

Re: sigsegv on aix with gcc

[Andrew Haley]

On 06/06/11 11:43, Jędrzej Dudkiewicz wrote:

> I maintain an application (rather large) for AIX (among others). When
> compiled with any level of optimization, it always crashes, always in
> the same place. What's funny, the same thing happens when I compile on
> AIX 4.3 with g++ 3.3.2 and on AIX 5.2 with g++ 4.5.3. Here are stack
> traces, both from our AIX 5.3 machine:
> 
> Juding by results (same compiler, same place) an error must be quite
> obvious and in my code, but I have no idea what I should check. I
> tried increasing stack size, adding -fno-strict-aliasing and few more,
> but to no avail.
> 
> Can you give me ANY pointers regarding further steps? Thanks in advance.

You're just going to have to debug it.  Assuming you've done the obvious
checking compile-time warning messages, it's time to break out the
debugger.

Andrew.

Re: sigsegv on aix with gcc

[Jędrzej Dudkiewicz]

2011/6/6 Andrew Haley <aph@redhat.com>:
> On 06/06/11 11:43, Jędrzej Dudkiewicz wrote:
>
>> Can you give me ANY pointers regarding further steps? Thanks in advance.
>
> You're just going to have to debug it.  Assuming you've done the obvious
> checking compile-time warning messages, it's time to break out the
> debugger.

Well, that's the first thing I did, but I just can't figure out what
is happening - I compiled with -O (as without optimization everything
"works") so source code isn't exactly the same as binary code, and
many variables are inlined, so I can't print their values.
Additionally - program core dumps in libstdc++, in locale's operator=
- that's not really deep, but I'm not completely aware of situation,
which makes things even harder. Also, same code works fine when
compiled on Linux, HPUX, MacOSX and SunOS with different versions of
gcc. Valgrind under Linux and MacOSX says nothing at all - so my guess
is that there's something "odd" about AIX. I was hoping for some
pointers, like "check this" or "link against that" or "define A".
-- 
Jędrzej Dudkiewicz

I really hate this damn machine, I wish that they would sell it.
It never does just what I want, but only what I tell it.

Re: sigsegv on aix with gcc

[Jonathan Wakely]

2011/6/7 Jędrzej Dudkiewicz :
>
> Well, that's the first thing I did, but I just can't figure out what
> is happening - I compiled with -O (as without optimization everything
> "works") so source code isn't exactly the same as binary code, and
> many variables are inlined, so I can't print their values.
> Additionally - program core dumps in libstdc++, in locale's operator=
> - that's not really deep, but I'm not completely aware of situation,
> which makes things even harder. Also, same code works fine when
> compiled on Linux, HPUX, MacOSX and SunOS with different versions of
> gcc. Valgrind under Linux and MacOSX says nothing at all - so my guess
> is that there's something "odd" about AIX. I was hoping for some
> pointers, like "check this" or "link against that" or "define A".

Build gcc with --enable-libstdcxx-debug so you get a version of
libstdc++.so built with debugging symbols and can step into the
locale's assignment operator.