libstdc++ ABI update rule? (baseline_symbols.txt)

libstdc++ ABI update rule? (baseline_symbols.txt)

[YumeYao]

Hi All,

I'm trying to do some experimental optimization to libstdc++, which
involves heavy use of __builtin_constant_p(), this, however,
eventually impacts how gcc calculate inlining or not, hence I got
error when testing libstdc++ about ABI changes.

Specifically, I met most of such issues when optimizing basic_string.

Then I tried fix the ABI changes by templatizing the functions, adding
__attribute__((always_inline)) and even gnu_inline(sometimes adding
only always_inline breaks the optimization, I noticed this issue very
long ago...) to the failed parts, which in turn makes other existing
functions in libstdc++ got uninlined and present in libstdc++.so as
ABI changes of added symbols/functions.

What really confuses me is the following fact:

without any attribute notation and only by templatizing functions, I
managed to get a version working on some old gcc version without any
abi change. But it then failed when I migrated it to a newer version.

I have some basic knowledge about extern template and inlining and
symbols, etc. and I tried to find the answers from gcc source, but
obviously there's something beyond the source.

I know inline decision is calculated by pseudo "cost", therefore not
only the change in library could make an ABI change, but also the
logic change in gcc inline calculator could make such change, even
more, how the user uses the code can also make a difference on inline
decision because the times or frequency of using a library function
can also impact the inline decision.

So I want to ask:

1. how the ABI of libstdc++ is maintained in gcc, just update
baseline_symbols.txt unconditionally when preparing for a release?

2. How does gcc decide to put which symbol to libstdc++.so? I can see
files named xxxx-inst(antiation).cc but obviously some symbols not
belonging to it get instantiated and put in libstdc++.so

Thanks in advance,
yumeyao

Re: libstdc++ ABI update rule? (baseline_symbols.txt)

[Jonathan Wakely]

On Tue, 9 Aug 2022, 17:10 YumeYao via Libstdc++, <libstdc++@gcc.gnu.org>
wrote:

> Hi All,
>
> I'm trying to do some experimental optimization to libstdc++, which
> involves heavy use of __builtin_constant_p(), this, however,
> eventually impacts how gcc calculate inlining or not, hence I got
> error when testing libstdc++ about ABI changes.
>


What errors do you get?



> Specifically, I met most of such issues when optimizing basic_string.
>
> Then I tried fix the ABI changes by templatizing the functions,


That certainly isn't going to work, that's explicitly introducing an ABI
change which seems a strange way to try to fix ABI breakage.


adding
> __attribute__((always_inline)) and even gnu_inline(sometimes adding
> only always_inline breaks the optimization, I noticed this issue very
> long ago...) to the failed parts, which in turn makes other existing
> functions in libstdc++ got uninlined and present in libstdc++.so as
> ABI changes of added symbols/functions.
>
> What really confuses me is the following fact:
>
> without any attribute notation and only by templatizing functions, I
> managed to get a version working on some old gcc version without any
> abi change.



I don't understand how.



But it then failed when I migrated it to a newer version.
>
> I have some basic knowledge about extern template and inlining and
> symbols, etc. and I tried to find the answers from gcc source, but
> obviously there's something beyond the source.
>
> I know inline decision is calculated by pseudo "cost", therefore not
> only the change in library could make an ABI change, but also the
> logic change in gcc inline calculator could make such change, even
> more, how the user uses the code can also make a difference on inline
> decision because the times or frequency of using a library function
> can also impact the inline decision.
>
> So I want to ask:
>
> 1. how the ABI of libstdc++ is maintained in gcc, just update
> baseline_symbols.txt unconditionally when preparing for a release?
>
> 2. How does gcc decide to put which symbol to libstdc++.so? I can see
> files named xxxx-inst(antiation).cc but obviously some symbols not
> belonging to it get instantiated and put in libstdc++.so


See
https://gcc.gnu.org/onlinedocs/libstdc++/manual/appendix_porting.html#build_hacking.configure.version

Re: libstdc++ ABI update rule? (baseline_symbols.txt)

[YumeYao]

On Wed, Aug 10, 2022 at 9:56 PM Jonathan Wakely <jwakely.gcc@gmail.com> wrote:
>
>
>
> On Tue, 9 Aug 2022, 17:10 YumeYao via Libstdc++, <libstdc++@gcc.gnu.org> wrote:
>>
>> Hi All,
>>
>> I'm trying to do some experimental optimization to libstdc++, which
>> involves heavy use of __builtin_constant_p(), this, however,
>> eventually impacts how gcc calculate inlining or not, hence I got
>> error when testing libstdc++ about ABI changes.
>
>
>
> What errors do you get?

added symbols / incompatible symbols. For example I got some
std::basic_string<_Char>::function instantiations, because I touched
string headers.

>>
>> Specifically, I met most of such issues when optimizing basic_string.
>>
>> Then I tried fix the ABI changes by templatizing the functions,
>
>
> That certainly isn't going to work, that's explicitly introducing an ABI change which seems a strange way to try to fix ABI breakage.

Actually it did work. I introduced some helper functions in
std::basic_string, and somehow some of them didn't get inlined and
appeared in the ABI check as added symbols as a result of template
class member function instantiation.
I then changed the helper function into a (template class member)
template function and it didn't appear in the ABI check any more.

>> adding
>> __attribute__((always_inline)) and even gnu_inline(sometimes adding
>> only always_inline breaks the optimization, I noticed this issue very
>> long ago...) to the failed parts, which in turn makes other existing
>> functions in libstdc++ got uninlined and present in libstdc++.so as
>> ABI changes of added symbols/functions.
>>
>> What really confuses me is the following fact:
>>
>> without any attribute notation and only by templatizing functions, I
>> managed to get a version working on some old gcc version without any
>> abi change.
>
>
>
> I don't understand how.
>

I don't understand either.

Initially on gcc7 I didn't use template class member template
function, then ABI check complained about added symbols.
Then like I said above, I turned them into template class member
template functions and got ABI fixed.

Then I migrated the code change to gcc8, only getting the issue again.
So I manually fixed the ABI break by always_inline'ing the functions
corresponding to the added symbols. Initially it resulted the outer
(caller) functions got un-inlined, resulting in the outer functions
appearing as added symbols in ABI check, so I had to add always_inline
level by level until I met a caller function that was in the existing
ABI list already.

>
>> But it then failed when I migrated it to a newer version.
>>
>> I have some basic knowledge about extern template and inlining and
>> symbols, etc. and I tried to find the answers from gcc source, but
>> obviously there's something beyond the source.
>>
>> I know inline decision is calculated by pseudo "cost", therefore not
>> only the change in library could make an ABI change, but also the
>> logic change in gcc inline calculator could make such change, even
>> more, how the user uses the code can also make a difference on inline
>> decision because the times or frequency of using a library function
>> can also impact the inline decision.
>>
>> So I want to ask:
>>
>> 1. how the ABI of libstdc++ is maintained in gcc, just update
>> baseline_symbols.txt unconditionally when preparing for a release?
>>
>> 2. How does gcc decide to put which symbol to libstdc++.so? I can see
>> files named xxxx-inst(antiation).cc but obviously some symbols not
>> belonging to it get instantiated and put in libstdc++.so
>
>
> See https://gcc.gnu.org/onlinedocs/libstdc++/manual/appendix_porting.html#build_hacking.configure.version
>
>

This seems to only explain ABI versioning, which is something I'm
already (somewhat) aware of. But here in question 2 I didn't mean gcc
community, I actually meant the compiler itself.

Well, to me, the Makefiles in gcc are just some dinosaurs I don't want
to read over. So... just let me guess.
I guess it's when compiling some source files in libstdc++ the default
visibility is on for that compile unit (for exporting symbols). So
when some functions don't get inlined while compiling such files,
these functions appear in libstdc++.so as added symbols.
Fix me if it totally doesn't work in this way...