[Bug libstdc++/31368] New: basic_string and unsigned short leads to memory fault

[Bug libstdc++/31368] New: basic_string and unsigned short leads to memory fault

[gregoryk at edifecs dot com]

Main application dynamically loads shared library and uses function from
library to fill out string. If basic_string<char> or basic_string<wchar_t> are
used then no problem observed. In case of basic_string<unsigned short>
application aborts (MALLOC_CHECK_=2) with message 

*** glibc detected *** free(): invalid pointer: 0x00000000005015c0 ***
Aborted

gdb shows stack trace:

#0  0x0000003a79d2e2ed in raise () from /lib64/tls/libc.so.6
#1  0x0000003a79d2fa3e in abort () from /lib64/tls/libc.so.6
#2  0x0000003a79d62db1 in __libc_message () from /lib64/tls/libc.so.6
#3  0x0000003a79d6888e in _int_free () from /lib64/tls/libc.so.6
#4  0x0000003a79d68bd6 in free () from /lib64/tls/libc.so.6
#5  0x0000003a7ceae19e in operator delete () from /usr/lib64/libstdc++.so.6
#6  0x0000002a95579665 in __gnu_cxx::new_allocator<char>::deallocate () from
./libloader.so
#7  0x0000002a9557954b in std::basic_string<unsigned short,
std::char_traits<unsigned short>, std::allocator<unsigned short>
>::_Rep::_M_destroy () from ./libloader.so
#8  0x0000002a9557908c in std::basic_string<unsigned short,
std::char_traits<unsigned short>, std::allocator<unsigned short>
>::_Rep::_M_dispose () from ./libloader.so
#9  0x0000002a95578e3e in std::basic_string<unsigned short,
std::char_traits<unsigned short>, std::allocator<unsigned short> >::reserve ()
   from ./libloader.so
#10 0x0000002a95578cef in std::basic_string<unsigned short,
std::char_traits<unsigned short>, std::allocator<unsigned short> >::append ()
   from ./libloader.so
#11 0x0000002a95578c63 in std::basic_string<unsigned short,
std::char_traits<unsigned short>, std::allocator<unsigned short> >::append ()
   from ./libloader.so
#12 0x0000002a95578c38 in read_string () from ./libloader.so
#13 0x0000000000400860 in main ()

The "workaround" is to add reserve(1) before passing string to function. In
this case there is no abort. 

Why the behavior is different? Where is the problem?

Source code:

main.cpp
//---------------------------------------------------------
#include <dlfcn.h>
#include <string>
#include "loader.h"

typedef void *pfLoader(std::ustring&);

int main(int argc, char* argv[])
{
        void * libHandle = dlopen("libloader.so", RTLD_NOW|RTLD_GLOBAL);
        pfLoader* pF = (pfLoader*)dlsym(libHandle, "read_string");

        std::ustring s;
//      s.reserve(1);   // uncomment this line to prevent abort
        pF(s);
        return 0;
}
//---------------------------------------------------------

loader.h
//---------------------------------------------------------
#ifndef _LOADER_INCLUDED_
#define _LOADER_INCLUDED_

typedef unsigned short USHORT;

namespace std
{
        typedef std::basic_string<USHORT, std::char_traits<USHORT>,
std::allocator<USHORT> > ustring;
}

extern "C" void read_string(std::ustring& sValue);

#endif // _RUNNER_INCLUDED_
//---------------------------------------------------------

loader.cpp
//---------------------------------------------------------
#include <string>
#include "loader.h"

const USHORT _sV [] = {'T','e','s','t','\0'};

void read_string(std::ustring& sValue)
{
        sValue.append(_sV);
}
//---------------------------------------------------------

Build commands:
main:  g++ -o main -I. -ldl main.cpp
lib:   g++ -Wl,-E -fPIC -I. -shared -o libloader.so loader.cpp

Version infos:
OS: 
Red Hat Enterprise Linux AS release 4 (Nahant Update 3)
Kernel \r on an \m

g++:
Reading specs from /usr/lib/gcc/x86_64-redhat-linux/3.4.5/specs
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man
--infodir=/usr/share/info --enable-shared --enable-threads=posix
--disable-checking --with-system-zlib --enable-__cxa_atexit
--disable-libunwind-exceptions --enable-java-awt=gtk --host=x86_64-redhat-linux
Thread model: posix
gcc version 3.4.5 20051201 (Red Hat 3.4.5-2)

glibc: 3.4


-- 
           Summary: basic_string and unsigned short leads to memory fault
           Product: gcc
           Version: 3.4.5
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: libstdc++
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: gregoryk at edifecs dot com


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31368

[Bug libstdc++/31368] basic_string and unsigned short leads to memory fault

[pcarlini at suse dot de]

------- Comment #1 from pcarlini at suse dot de  2007-03-26 22:10 -------
Seems the same as libstdc++/25956. Can you try whether instantiating the string
class at global scope works for you too:

  http://gcc.gnu.org/ml/libstdc++/2006-01/msg00162.html

???


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31368

[Bug libstdc++/31368] basic_string and unsigned short leads to memory fault

[gregoryk at edifecs dot com]

------- Comment #2 from gregoryk at edifecs dot com  2007-03-26 22:28 -------
Though the problem from bug 25956 looks same the instantiating didn't help.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31368

[Bug libstdc++/31368] basic_string and unsigned short leads to memory fault

[pcarlini at suse dot de]

------- Comment #3 from pcarlini at suse dot de  2007-03-26 22:52 -------
Ok, but before we take any further action we badly need a better reproduce: on
any up to date x86 and x86-64 linux machine I tried, your testcase segfault
immediately both for unsigned short and for char (and the workaround doesn't
work)


-- 

pcarlini at suse dot de changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |WAITING


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31368

[Bug libstdc++/31368] basic_string and unsigned short leads to memory fault

[gregoryk at edifecs dot com]

------- Comment #4 from gregoryk at edifecs dot com  2007-03-26 23:13 -------
Ok. Can I help you somehow?


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31368

[Bug libstdc++/31368] basic_string and unsigned short leads to memory fault

[pcarlini at suse dot de]

------- Comment #5 from pcarlini at suse dot de  2007-03-26 23:24 -------
Yes, as I said, we need a better reproducer, that is a testcase that on most of
the current x86-linux or x86-64-linux machines behaves as you are reporting: if
you want to help you should look for a recent machine (e.g., glibc2.4 / 2.5,
system compiler gcc 4.1.x) and refine your testcase to behave as you expect.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31368

[Bug libstdc++/31368] basic_string and unsigned short leads to memory fault

[pcarlini at suse dot de]

------- Comment #6 from pcarlini at suse dot de  2007-03-27 18:07 -------
To restate my point: if, with a recent compiler, I change the testcase to use
<ext/vstring.h>, which has completely different memory management, instead of
<string>, even for plain char I get an immediate Segmentation Fault without any
debugging information available. Thus, I strongly suspect the testcase is
somehow broken and must be fixed first: I can do that eventually, but feedback
from the original submitter is strongly encouraged, of course.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31368

[Bug libstdc++/31368] basic_string and unsigned short leads to memory fault

[gregoryk at edifecs dot com]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 396 bytes --]



------- Comment #7 from gregoryk at edifecs dot com  2007-03-27 18:20 -------
Unfortunately I do not have possibility to run my test case using latest GCC
compiler. I’m limited in hardware and software choice. Besides I’m not sure
what do you mean “to fix testcase”. If you could fix it then it will be
accepted in any way by me. 


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31368

[Bug libstdc++/31368] basic_string and unsigned short leads to memory fault

[pcarlini at suse dot de]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 824 bytes --]



------- Comment #8 from pcarlini at suse dot de  2007-03-27 18:24 -------
(In reply to comment #7)
> Unfortunately I do not have possibility to run my test case using latest GCC
> compiler. I’m limited in hardware and software choice. Besides I’m not sure
> what do you mean “to fix testcase”.

As I said, the testcase Segfaults badly on any recent machine, for <string> but
also for <ext/vstring.h> and also for plain char. This is unfortunate, this is
not the behavior you expected when you submitted the PR, and if we cannot
reproduce the issue that you are maintaining to see we cannot understand it and
we cannot take any further action. Can you please double check you are building
the so, dlopening, etc, in your testcase in the correct, portable way?


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31368

[Bug libstdc++/31368] basic_string and unsigned short leads to memory fault

[pcarlini at suse dot de]

------- Comment #9 from pcarlini at suse dot de  2007-03-27 18:36 -------
I get an immediate Segmentation fault even if I change read_string to do
nothing..


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31368

[Bug libstdc++/31368] basic_string and unsigned short leads to memory fault

[gregoryk at edifecs dot com]

------- Comment #10 from gregoryk at edifecs dot com  2007-03-27 18:39 -------
Got it, thanks. In may original test I was relaying on LD_LIBRARY_PATH to have
current folder in the values. And there was no checking of dlopen result for
simplicity. Here is updated code. It will search for libloader.so in current
folder.

#include <dlfcn.h>
#include <string>
#include "loader.h"

typedef void *pfLoader(std::ustring&);

int main(int argc, char* argv[])
{
        void * libHandle = dlopen("./libloader.so", RTLD_NOW|RTLD_GLOBAL);

        if (0 == libHandle)
        {
                printf("Can not load \"./libloader.so\".\n");
                return 1;
        }

        pfLoader* pF = (pfLoader*)dlsym(libHandle, "read_string");

        if (0 == pF)
        {
                printf("Can not find function \"read_string\" in
libloader.so\n");
                return 2;
        }

        std::ustring s;
//      s.reserve(1);   // uncomment this line to prevent abort
        pF(s);
        return 0;
}


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31368

[Bug libstdc++/31368] basic_string and unsigned short leads to memory fault

[gregoryk at edifecs dot com]

------- Comment #11 from gregoryk at edifecs dot com  2007-03-27 18:40 -------
Forgot to mention, that was main.cpp code.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31368

[Bug libstdc++/31368] basic_string and unsigned short leads to memory fault

[pcarlini at suse dot de]

------- Comment #12 from pcarlini at suse dot de  2007-03-27 19:22 -------
Ok, now I see. The kind of issue is unfortunately known, akin to 24196 for
example, and ultimately due to the special, optimized way we are dealing with
empty strings, not allocating dynamic memory at all. I don't think we can
really solve the problem without breaking the binary compatibility of the
entire library, therefore we suggest various options:
1- Make sure to never deal with empty strings (which means a string not holding
a memory buffer, that's why reserve(1) works, for example)
2- Rebuild the library passing --enable-fully-dynamic-string, the option has
been added exactly to help users in this case.
3- Switch to a different implementation of the string class: in recent releases
we are offering one under <ext/vstring.h> (will become standard when we decide
to break binary compatibility)


-- 

pcarlini at suse dot de changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         AssignedTo|unassigned at gcc dot gnu   |pcarlini at suse dot de
                   |dot org                     |
             Status|WAITING                     |ASSIGNED
     Ever Confirmed|0                           |1
   Last reconfirmed|0000-00-00 00:00:00         |2007-03-27 19:22:05
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31368

[Bug libstdc++/31368] basic_string and unsigned short leads to memory fault

[pcarlini at suse dot de]

------- Comment #13 from pcarlini at suse dot de  2007-03-27 19:25 -------
By the way, as a matter of portability, isn't a good idea to use the string
class with anything != char and wchar_t: things usually work in rather recent
releases of GCC only because we are delivering a "generic" implementation of
char_traits, which cannot be assumed, in general (and that's why probably your
specific issue with empty strings has not been reported earlier).


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31368

[Bug libstdc++/31368] basic_string and unsigned short leads to memory fault

[pcarlini at suse dot de]

-- 

pcarlini at suse dot de changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         AssignedTo|pcarlini at suse dot de     |unassigned at gcc dot gnu
                   |                            |dot org
             Status|ASSIGNED                    |NEW


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31368

[Bug libstdc++/31368] basic_string and unsigned short leads to memory fault

[gregoryk at edifecs dot com]

------- Comment #14 from gregoryk at edifecs dot com  2007-03-27 19:51 -------
Thank you for info. The sample is working now with
_GLIBCXX_FULLY_DYNAMIC_STRING turned on. What is the next procedure with bug
status? For me bug can be marked as FIXED. 

Regarding portability. We use unsigned short strings on more then 6 platforms
and would like to continue this way. We use custom implementation of
char_traits<unsigned short> where it is not available. And looking on migration
on Xerces STL as a common solution for all platforms (yes, it is not easy but
worth to try).


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31368

[Bug libstdc++/31368] basic_string and unsigned short leads to memory fault

[pcarlini at suse dot de]

------- Comment #15 from pcarlini at suse dot de  2007-03-27 19:53 -------
Ok, I will mark it as suspended, because when we break the binary compatibility
things will always work fine.


-- 

pcarlini at suse dot de changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |SUSPENDED


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31368