gcc5 and std::string

gcc5 and std::string

[PICCA Frederic-Emmanuel]

Hello,

First, I am maintaining the tango and pytango package in the Debian distribution.
So during the gcc5 rebuilt I got affected by something strange, first in the tango library[1].
Then a problem with pytango[2] which depends on tango. you will find a interesting comment in [3].
And a small test program whcih shows the problem in [4].

So I would like your opinion about this pytango FTBFS.
It seem that gcc5.2 generate different symbols in tango (abi-cxx11)  than in pytango (no abi-cxx11).
I do not understand why, so I am wondering if this is a bug in pytango, tango or in gcc.

thanks for your help

Frédéric

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=797285
[2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=797284#57
[3] https://lists.debian.org/debian-python/2015/09/msg00031.html
[4] https://lists.debian.org/debian-python/2015/09/msg00032.html

Re: gcc5 and std::string

[Avi Kivity]

On 09/11/2015 04:05 PM, PICCA Frederic-Emmanuel wrote:
> Hello,
>
> First, I am maintaining the tango and pytango package in the Debian distribution.
> So during the gcc5 rebuilt I got affected by something strange, first in the tango library[1].
> Then a problem with pytango[2] which depends on tango. you will find a interesting comment in [3].
> And a small test program whcih shows the problem in [4].
>
> So I would like your opinion about this pytango FTBFS.
> It seem that gcc5.2 generate different symbols in tango (abi-cxx11)  than in pytango (no abi-cxx11).
> I do not understand why, so I am wondering if this is a bug in pytango, tango or in gcc.

This post explains it:

http://developerblog.redhat.com/2015/02/05/gcc5-and-the-c11-abi/

You need to add the define mentioned there to pytango, or remove it from 
tango, depending on your distribution's guidelines.


> thanks for your help
>
> Frédéric
>
> [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=797285
> [2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=797284#57
> [3] https://lists.debian.org/debian-python/2015/09/msg00031.html
> [4] https://lists.debian.org/debian-python/2015/09/msg00032.html

Re: gcc5 and std::string

[Jonathan Wakely]

 On 11 September 2015 at 14:24, Avi Kivity wrote:
>
> This post explains it:
>
> http://developerblog.redhat.com/2015/02/05/gcc5-and-the-c11-abi/

Also
http://fedoramagazine.org/gcc-5-in-fedora-whats-an-abi-and-what-happens-when-we-change-it/
and
https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_dual_abi.html

RE:gcc5 and std::string

[PICCA Frederic-Emmanuel]

just to be sure to understand,

I rebuilt both projects with gcc 5.2 but in one case I got a std::string with the old API
and in another a symbol with the new CXX11 abi.

I do not try to link a project compiled with the new ABI and a library with old version.

What I understood is that std::string is special because two symbols are emitted on in the normal name space but with the cxx11 abi and one in the __cxx11 namespace.

In pytango the symbol emited by gcc5.2  is the once without the __cxx11 namespace

_ZN5Tango17ranges_type2constIsE3strE -> Tango::ranges_type2const<short>::str

but if I understand correctly this string is cxx11 abi with the same old name mangled.

BUT

in tango the symbols emited by gcc are


:/usr/lib/i386-linux-gnu$ nm -D libtango.so.8.1.2  | grep ranges_type | c++filt 
0045f258 D Tango::ranges_type2const<bool>::enu
00460dfc B Tango::ranges_type2const<bool>::str[abi:cxx11]
0045f280 D Tango::ranges_type2const<double>::enu
00460fdc B Tango::ranges_type2const<double>::str[abi:cxx11]
0045f27c D Tango::ranges_type2const<float>::enu
00460fac B Tango::ranges_type2const<float>::str[abi:cxx11]
0045f26c D Tango::ranges_type2const<unsigned char>::enu
00460eec B Tango::ranges_type2const<unsigned char>::str[abi:cxx11]
0045f278 D Tango::ranges_type2const<long>::enu
00460f7c B Tango::ranges_type2const<long>::str[abi:cxx11]
0045f268 D Tango::ranges_type2const<unsigned long>::enu
00460ebc B Tango::ranges_type2const<unsigned long>::str[abi:cxx11]
0045f25c D Tango::ranges_type2const<Tango::DevEncoded>::enu
00460e2c B Tango::ranges_type2const<Tango::DevEncoded>::str[abi:cxx11]
0045f250 D Tango::ranges_type2const<Tango::DevState>::enu
00460d9c B Tango::ranges_type2const<Tango::DevState>::str[abi:cxx11]
0045f254 D Tango::ranges_type2const<char*>::enu
00460dcc B Tango::ranges_type2const<char*>::str[abi:cxx11]
0045f270 D Tango::ranges_type2const<short>::enu
00460f1c B Tango::ranges_type2const<short>::str[abi:cxx11]
0045f260 D Tango::ranges_type2const<unsigned short>::enu
00460e5c B Tango::ranges_type2const<unsigned short>::str[abi:cxx11]
0045f274 D Tango::ranges_type2const<long long>::enu
00460f4c B Tango::ranges_type2const<long long>::str[abi:cxx11]
0045f264 D Tango::ranges_type2const<unsigned long long>::enu
00460e8c B Tango::ranges_type2const<unsigned long long>::str[abi:cxx11]


So my question now is why in one case the [abi:cxx11] is present (tango)  and not in the other (pytango)
I use the same compiler in both cases. Except that for pytango I am building a python extension and it calls gcc instead of g++.
But I guess thath gcc is clever enought to understand that this is a c++ file.

Sorry if this is trivial for you.

But I try to understand clearly where is the problem.

Thanks


________________________________________
De : Jonathan Wakely [jwakely.gcc@gmail.com]
Envoyé : vendredi 11 septembre 2015 15:27
À : PICCA Frederic-Emmanuel
Cc : gcc-help; taurel@esrf.eu; Avi Kivity
Objet : Re: gcc5 and std::string

 On 11 September 2015 at 14:24, Avi Kivity wrote:
>
> This post explains it:
>
> http://developerblog.redhat.com/2015/02/05/gcc5-and-the-c11-abi/

Also
http://fedoramagazine.org/gcc-5-in-fedora-whats-an-abi-and-what-happens-when-we-change-it/
and
https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_dual_abi.html

Re: gcc5 and std::string

[David Abdurachmanov]

> On 11 Sep 2015, at 15:51, PICCA Frederic-Emmanuel <frederic-emmanuel.picca@synchrotron-soleil.fr> wrote:
> 
> In pytango the symbol emited by gcc5.2  is the once without the __cxx11 namespace
> 
> _ZN5Tango17ranges_type2constIsE3strE -> Tango::ranges_type2const<short>::str
> 
> but if I understand correctly this string is cxx11 abi with the same old name mangled.

Are both compiled with -std=c++11 (or above)?

I see that tango specifies -std=c++11

libtool: link: g++ -g -O2 -fstack-protector-strong -Wformat
-Werror=format-security -std=c++11 -D_REENTRANT -DOMNI_UNLOADABLE_STUBS -Wl,-z
-Wl,relro -o .libs/notifd2db notifd2db.o

david

RE:gcc5 and std::string

[PICCA Frederic-Emmanuel]

Here the build log of pytango [1] when you can see that 

i586-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -DPYTANGO_NUMPY_VERSION="1.9.2" -DPYTANGO_HAS_UNIQUE_PTR=1 -I/usr/lib/python3/dist-packages/numpy/core/include -I/usr/include/tango -I/«PKGBUILDDIR»/src/boost/cpp -I/«PKGBUILDDIR»/src/boost/cpp/server -I/usr/include/python3.4m -c /«PKGBUILDDIR»/src/boost/cpp/group.cpp -o build/temp.linux-i686-3.4/«PKGBUILDDIR»/src/boost/cpp/group.o -std=c++0x

it contains c++0x which is the old name for c++11 (right ?).

[1] https://buildd.debian.org/status/fetch.php?pkg=pytango&arch=i386&ver=8.1.5-1%2Bb1&stamp=1441753396

Re: gcc5 and std::string

[Jonathan Wakely]

On 11 September 2015 at 14:55, David Abdurachmanov wrote:
>
> Are both compiled with -std=c++11 (or above)?

That's irrelevant, the -std options don't affect the ABI.

Re: gcc5 and std::string

[Jonathan Wakely]

On 11 September 2015 at 14:51, PICCA Frederic-Emmanuel wrote:
> just to be sure to understand,
>
> I rebuilt both projects with gcc 5.2 but in one case I got a std::string with the old API
> and in another a symbol with the new CXX11 abi.
>
> I do not try to link a project compiled with the new ABI and a library with old version.
>
> What I understood is that std::string is special because two symbols are emitted on in the normal name space but with the cxx11 abi and one in the __cxx11 namespace.
>
> In pytango the symbol emited by gcc5.2  is the once without the __cxx11 namespace
>
> _ZN5Tango17ranges_type2constIsE3strE -> Tango::ranges_type2const<short>::str
>
> but if I understand correctly this string is cxx11 abi with the same old name mangled.

The symbol shown above doesn't mention std::string so you can't really
say it "is cxx11 abi" because it isn't using any std::string, new or
old. As far as we can tell from the symbol it doesn't depend on
std::string at all.


> So my question now is why in one case the [abi:cxx11] is present (tango)  and not in the other (pytango)

Without seeing how it is declared I can only guess why it doesn't have
the abi-tag.

Is it only forward-declared in pytango and properly defined in tango?

The abi-tag can only be added when the compiler knows that it depends
on another type with the tag, so if pytango only sees a
forward-declaration it can't know that it should have the tag.

You can compile tango with -Wabi-tag to get warnings about entities
that could cause this problem, and to resolve it you can add the tag
explicitly to the declarations e.g.

#ifdef _GLIBCXX_USE_CXX11_ABI
#define TANGO_CXX11_ABI __attribute((abi_tag("cxx11")))
#else
#define TANGO_CXX11_ABI
#endif

namespace Tango
{
  template<class T>
    struct ranges_type2const
    {
      struct TANGO_CXX11_ABI str;
    };
}




> I use the same compiler in both cases. Except that for pytango I am building a python extension and it calls gcc instead of g++.
> But I guess thath gcc is clever enought to understand that this is a c++ file.

Yes.

RE:gcc5 and std::string

[PICCA Frederic-Emmanuel]

first thanks a lot for your really helpfull comment.

> The symbol shown above doesn't mention std::string so you can't really
> say it "is cxx11 abi" because it isn't using any std::string, new or
> old. As far as we can tell from the symbol it doesn't depend on
> std::string at all.

sorry, I saw str and my brean tought it was std::string.


I recompiled the tango project with the -Wabi-tag

now I have thousands of line like this.

../../../../../lib/cpp/log4tango/include/log4tango/AppenderAttachable.hh:58:24: warning: 'log4tango::AppenderAttachable' does not have the "cxx11" ABI tag that 'log4tango::AppenderMap {aka std::map<std::__cxx11::basic_string<char>, log4tango::Appender*>}' (used in the type of 'log4tango::AppenderAttachable::_appenders') has [-Wabi-tag]
../../../../../lib/cpp/log4tango/include/log4tango/LoggerStream.hh:58:7: warning: 'log4tango::LoggerStream' does not have the "cxx11" ABI tag that 'std::ostringstream {aka std::__cxx11::basic_ostringstream<char>}' (used in the type of 'log4tango::LoggerStream::_buffer') has [-Wabi-tag]
../../../../../lib/cpp/log4tango/include/log4tango/Logger.hh:56:24: warning: 'log4tango::Logger' does not have the "cxx11" ABI tag that 'const string {aka const std::__cxx11::basic_string<char>}' (used in the type of 'log4tango::Logger::_name') has [-Wabi-tag]
../../../../../lib/cpp/log4tango/include/log4tango/LoggingEvent.hh:50:25: warning: 'log4tango::LoggingEvent' does not have the "cxx11" ABI tag that 'const string {aka const std::__cxx11::basic_string<char>}' (used in the type of 'log4tango::LoggingEvent::logger_name') has [-Wabi-tag]
../../../../../lib/cpp/log4tango/include/log4tango/LoggingEvent.hh:50:25: warning: 'log4tango::LoggingEvent' does not have the "cxx11" ABI tag that 'const string {aka const std::__cxx11::basic_string<char>}' (used in the type of 'log4tango::LoggingEvent::message') has [-Wabi-tag]
../../../../../lib/cpp/log4tango/include/log4tango/LoggingEvent.hh:50:25: warning: 'log4tango::LoggingEvent' does not have the "cxx11" ABI tag that 'std::__cxx11::string {aka std::__cxx11::basic_string<char>}' (used in the type of 'log4tango::LoggingEvent::thread_name') has [-Wabi-tag]
../../../../../lib/cpp/log4tango/include/log4tango/Layout.hh:41:24: warning: 'log4tango::Layout' does not have the "cxx11" ABI tag that 'std::__cxx11::string {aka std::__cxx11::basic_string<char>}' (used in the type of 'virtual std::__cxx11::string log4tango::Layout::format(const log4tango::LoggingEvent&)') has [-Wabi-tag]
../../../../../lib/cpp/log4tango/include/log4tango/Appender.hh:45:24: warning: 'log4tango::Appender' does not have the "cxx11" ABI tag that 'const string {aka const std::__cxx11::basic_string<char>}' (used in the type of 'log4tango::Appender::_name') has [-Wabi-tag]
../../../../../lib/cpp/log4tango/include/log4tango/PatternLayout.hh:99:26: warning: 'log4tango::PatternLayout::PatternComponent' does not have the "cxx11" ABI tag that 'std::ostringstream {aka std::__cxx11::basic_ostringstream<char>}' (used in the type of 'virtual void log4tango::PatternLayout::PatternComponent::append(std::ostringstream&, const log4tango::LoggingEvent&)') has [-Wabi-tag]
../../../../../lib/cpp/log4tango/include/log4tango/PatternLayout.hh:43:24: warning: 'log4tango::PatternLayout' does not have the "cxx11" ABI tag that 'std::__cxx11::string {aka std::__cxx11::basic_string<char>}' (used in the type of 'log4tango::PatternLayout::_conversionPattern') has [-Wabi-tag]
../../../../../lib/cpp/log4tango/include/log4tango/PatternLayout.hh:43:24: warning: 'log4tango::PatternLayout' does not have the "cxx11" ABI tag that 'const string {aka const std::__cxx11::basic_string<char>}' (used in the type of 'virtual int log4tango::PatternLayout::set_conversion_pattern(const string&)') has [-Wabi-tag]
../../../../../lib/cpp/log4tango/include/log4tango/PatternLayout.hh:43:24: warning: 'log4tango::PatternLayout' does not have the "cxx11" ABI tag that 'std::__cxx11::string {aka std::__cxx11::basic_string<char>}' (used in the type of 'virtual std::__cxx11::string log4tango::PatternLayout::get_conversion_pattern() const') has [-Wabi-tag]
../../../../../lib/cpp/log4tango/include/log4tango/LoggingEvent.hh:50:25: warning: 'log4tango::LoggingEvent' does not have the "cxx11" ABI tag that 'const string {aka const std::__cxx11::basic_string<char>}' (used in the type of 'log4tango::LoggingEvent::logger_name') has [-Wabi-tag]
../../../../../lib/cpp/log4tango/include/log4tango/LoggingEvent.hh:50:25: warning: 'log4tango::LoggingEvent' does not have the "cxx11" ABI tag that 'const string {aka const std::__cxx11::basic_string<char>}' (used in the type of 'log4tango::LoggingEvent::message') has [-Wabi-tag]
../../../../../lib/cpp/log4tango/include/log4tango/LoggingEvent.hh:50:25: warning: 'log4tango::LoggingEvent' does not have the "cxx11" ABI tag that 'std::__cxx11::string {aka std::__cxx11::basic_string<char>}' (used in the type of 'log4tango::LoggingEvent::thread_name') has [-Wabi-tag

I filtered the line which explain explicitely that the symbol inherit from the cxx11 ABI.

If I understand you correctly I need to fix each of these symbols by adding your #define snipset. (right ?)

It seems to me that this is a HUGE task and it "polute" the code with this snipset everywhere.
I mean is it something really specific to gcc ?, what about other c++ compiler, I especially think about msvc abd clang.
What i understand is that this is just a waning saying that a problem can occure when someone use one of these symbols in the incriminate .h file.

I imagine also that not all libraries where instrumented like that.
since the problem show up for me at runtime. I have the filling that I have a damocles sword above my head. Maybe there is something that I do not understand correctly :)

In another hand only a few of these symbols cause troubles practcally (the used one, so only the one in the public API).
Do you have some practical advise in order to fix this problem with a minimal impact on all the code.

thanks a lot for your help


Frederic

Re: gcc5 and std::string

[Jonathan Wakely]

On 12 September 2015 at 10:12, PICCA Frederic-Emmanuel wrote:
> If I understand you correctly I need to fix each of these symbols by adding your #define snipset. (right ?)

No, only the ones which are given the tag in tango but not in pytango,
to make them consistent.

RE:gcc5 and std::string

[PICCA Frederic-Emmanuel]

> No, only the ones which are given the tag in tango but not in pytango,
> to make them consistent.


ok, but if I were the uptream of tango, I need to fix all these symbols in order to be consistant for all its downstream (users).

RE:gcc5 and std::string

[PICCA Frederic-Emmanuel]

> No, only the ones which are given the tag in tango but not in pytango,
> to make them consistent.


Ok, so I am trying to fix only the problematic symbols.
In order o do this I would like to know for a symbols the source file and the line number at the origin of it.
This _Z... was produced due to this souce.hpp:125:5:


Is it possible ?

thanks

Fred

Re: gcc5 and std::string

[Mason]

On 11/09/2015 15:24, Avi Kivity wrote:

> This post explains it:
> http://developerblog.redhat.com/2015/02/05/gcc5-and-the-c11-abi/

It is quite unfortunate that both C11 and C++11 were published in 2011.
As it is, I thought the above link discussed a change to the C ABI...
"GCC5 and the C++11 ABI"

The author should use c++11 or cxx11 in the URL name.

Regards.