GCC 8.3.0, -flto and violation of C++ One Definition Rule

GCC 8.3.0, -flto and violation of C++ One Definition Rule

[Tom Kacvinsky]

Hi,

First, using GCC 8.3.0 and binutils 2.37.I am trying to increase
performance of linking our product, so I thought I'd give LTO a try.  So
I am compiling all object files with -flto, and passing -flto to g++
(which we use as our link driver).  However, what I have found is that
some of our code violates the C++ One Definition Rule (-Werror=odr). This
only happens when building with LTO - without LTO, the C++ rule is
not violated.  The problem exists with LTO using both the BFD and gold
linkers.

So, my question is, since the LTO object files are now such that one
needs to use gcc-nm to examine them (which I know is a wrapper around nm,
and passes an option to load the LTO plugin). how can I leverage that to
see if there are other translation units that define the class that ODR
violation is complaining about?  I did do a fairly thorough analysis of
the object files and did not see there the particular class and methods
would be multiply defined, but that was just based on symbol names from
gcc-nm output.  I suspect there is more to this since the object files
have LTO information now, and that is what I'd like to examine.

Any hints on how to move forward with diagnosing LTO link errors?

Thanks,

Tom

Re: GCC 8.3.0, -flto and violation of C++ One Definition Rule

[Xi Ruoyao]

On Wed, 2021-12-29 at 06:44 -0500, Tom Kacvinsky via Gcc-help wrote:
> Hi,
> 
> First, using GCC 8.3.0 and binutils 2.37.I am trying to increase
> performance of linking our product, so I thought I'd give LTO a try.  So
> I am compiling all object files with -flto, and passing -flto to g++
> (which we use as our link driver).  However, what I have found is that
> some of our code violates the C++ One Definition Rule (-Werror=odr). This
> only happens when building with LTO - without LTO, the C++ rule is
> not violated.

No, it's violated all the time.  -Wodr only works with LTO because it's
impossible to detect ODR volations (across multiple TUs) just by linking
normal (non-LTO) object files.

> The problem exists with LTO using both the BFD and gold
> linkers.
> 
> So, my question is, since the LTO object files are now such that one
> needs to use gcc-nm to examine them (which I know is a wrapper around nm,
> and passes an option to load the LTO plugin). how can I leverage that to
> see if there are other translation units that define the class that ODR
> violation is complaining about?
> I did do a fairly thorough analysis of
> the object files and did not see there the particular class and methods
> would be multiply defined, but that was just based on symbol names from
> gcc-nm output.

Generally you can't find ODR violations this way.  ODR violation does
not mean multiple definitions of a symbol.  For example:

// in 1.cpp

enum A
{
	a = 1,
};

// in 2.cpp

enum A
{
	a = 2,
};

If you link the objects compiled from these TUs together, you will
violate ODR.  But these TUs won't produce any symbols for A or A::a at
all.

> I suspect there is more to this since the object files
> have LTO information now, and that is what I'd like to examine.
> 
> Any hints on how to move forward with diagnosing LTO link errors?

With GCC-11, the diagnostic shows the exact location of ODR violation,
like:

> t-1.cc:1:6: warning: type 'A' violates the C++ One Definition Rule [-Wodr]
>     1 | enum A
>       |      ^
> t-2.cc:3:6: note: an enum with different value name is defined in another translation unit
>     3 | enum A
>       |      ^
> t-1.cc:3:9: note: name 'a' is defined to 1 while another translation unit defines it as 2
>     3 |         a = 1,
>       |         ^
> t-2.cc:5:9: note: mismatching definition
>     5 |         a = 2,
>       |         ^
> 

I'm not sure how GCC 8 behaves.

-- 
Xi Ruoyao <xry111@mengyan1223.wang>
School of Aerospace Science and Technology, Xidian University

Re: GCC 8.3.0, -flto and violation of C++ One Definition Rule

[Tom Kacvinsky]

On Wed, Dec 29, 2021 at 7:55 AM Xi Ruoyao <xry111@mengyan1223.wang> wrote:
>
> On Wed, 2021-12-29 at 06:44 -0500, Tom Kacvinsky via Gcc-help wrote:
> > Hi,
> >
> > First, using GCC 8.3.0 and binutils 2.37.I am trying to increase
> > performance of linking our product, so I thought I'd give LTO a try.  So
> > I am compiling all object files with -flto, and passing -flto to g++
> > (which we use as our link driver).  However, what I have found is that
> > some of our code violates the C++ One Definition Rule (-Werror=odr). This
> > only happens when building with LTO - without LTO, the C++ rule is
> > not violated.
>
> No, it's violated all the time.  -Wodr only works with LTO because it's
> impossible to detect ODR volations (across multiple TUs) just by linking
> normal (non-LTO) object files.
>
> > The problem exists with LTO using both the BFD and gold
> > linkers.
> >
> > So, my question is, since the LTO object files are now such that one
> > needs to use gcc-nm to examine them (which I know is a wrapper around nm,
> > and passes an option to load the LTO plugin). how can I leverage that to
> > see if there are other translation units that define the class that ODR
> > violation is complaining about?
> > I did do a fairly thorough analysis of
> > the object files and did not see there the particular class and methods
> > would be multiply defined, but that was just based on symbol names from
> > gcc-nm output.
>
> Generally you can't find ODR violations this way.  ODR violation does
> not mean multiple definitions of a symbol.  For example:
>
> // in 1.cpp
>
> enum A
> {
>         a = 1,
> };
>
> // in 2.cpp
>
> enum A
> {
>         a = 2,
> };
>
> If you link the objects compiled from these TUs together, you will
> violate ODR.  But these TUs won't produce any symbols for A or A::a at
> all.
>
> > I suspect there is more to this since the object files
> > have LTO information now, and that is what I'd like to examine.
> >
> > Any hints on how to move forward with diagnosing LTO link errors?
>
> With GCC-11, the diagnostic shows the exact location of ODR violation,
> like:
>
> > t-1.cc:1:6: warning: type 'A' violates the C++ One Definition Rule [-Wodr]
> >     1 | enum A
> >       |      ^
> > t-2.cc:3:6: note: an enum with different value name is defined in another translation unit
> >     3 | enum A
> >       |      ^
> > t-1.cc:3:9: note: name 'a' is defined to 1 while another translation unit defines it as 2
> >     3 |         a = 1,
> >       |         ^
> > t-2.cc:5:9: note: mismatching definition
> >     5 |         a = 2,
> >       |         ^
> >
>
> I'm not sure how GCC 8 behaves.
>

that's the problem.  It is _not_ defined in another translation unit.
Only defined in one
C++ file

Re: GCC 8.3.0, -flto and violation of C++ One Definition Rule

[Jonathan Wakely]

On Wed, 29 Dec 2021, 11:45 Tom Kacvinsky via Gcc-help, <gcc-help@gcc.gnu.org>
wrote:

> Hi,
>
> First, using GCC 8.3.0 and binutils 2.37.I am trying to increase
> performance of linking our product, so I thought I'd give LTO a try.  So
> I am compiling all object files with -flto, and passing -flto to g++
> (which we use as our link driver).  However, what I have found is that
> some of our code violates the C++ One Definition Rule (-Werror=odr). This
> only happens when building with LTO - without LTO, the C++ rule is
> not violated.


As already explained, this is almost certainly wrong. It is more likely
that the LTO violation is always present, but only detected when using LTO.


  The problem exists with LTO using both the BFD and gold
> linkers.
>
> So, my question is, since the LTO object files are now such that one
> needs to use gcc-nm to examine them (which I know is a wrapper around nm,
> and passes an option to load the LTO plugin). how can I leverage that to
> see if there are other translation units that define the class that ODR
> violation is complaining about?  I did do a fairly thorough analysis of
> the object files and did not see there the particular class and methods
> would be multiply defined,


It would help if you tell us the actual error/warning you get. -Wodr can
warn about various different things. It does not warn about multiple
definitions, it warns about *inconsistent* definitions.


but that was just based on symbol names from
> gcc-nm output.  I suspect there is more to this since the object files
> have LTO information now, and that is what I'd like to examine.
>
> Any hints on how to move forward with diagnosing LTO link errors?
>
> Thanks,
>
> Tom
>

Re: GCC 8.3.0, -flto and violation of C++ One Definition Rule

[Tom Kacvinsky]

On Wed, Dec 29, 2021 at 10:39 AM Jonathan Wakely <jwakely.gcc@gmail.com> wrote:
>
>
>
> On Wed, 29 Dec 2021, 11:45 Tom Kacvinsky via Gcc-help, <gcc-help@gcc.gnu.org> wrote:
>>
>> Hi,
>>
>> First, using GCC 8.3.0 and binutils 2.37.I am trying to increase
>> performance of linking our product, so I thought I'd give LTO a try.  So
>> I am compiling all object files with -flto, and passing -flto to g++
>> (which we use as our link driver).  However, what I have found is that
>> some of our code violates the C++ One Definition Rule (-Werror=odr). This
>> only happens when building with LTO - without LTO, the C++ rule is
>> not violated.
>
>
> As already explained, this is almost certainly wrong. It is more likely that the LTO violation is always present, but only detected when using LTO.
>
>
>>   The problem exists with LTO using both the BFD and gold
>> linkers.
>>
>> So, my question is, since the LTO object files are now such that one
>> needs to use gcc-nm to examine them (which I know is a wrapper around nm,
>> and passes an option to load the LTO plugin). how can I leverage that to
>> see if there are other translation units that define the class that ODR
>> violation is complaining about?  I did do a fairly thorough analysis of
>> the object files and did not see there the particular class and methods
>> would be multiply defined,
>
>
> It would help if you tell us the actual error/warning you get. -Wodr can warn about various different things. It does not warn about multiple definitions, it warns about *inconsistent* definitions.
>

This is long.  Not sure of the attachment fule for this, so I am
pasting it in email.  Ib obfuscated the actual source file
names, but this is the general gist of the link error.  I wonder if
the error is coming from boost::python::api::object.

/home/home/tkacvins/project/libbar/include/Bar.h:38:7: error: type
‘struct Bar’ violates the C++ One Definition Rule [-Werror=odr]
 class Bar {
       ^
/home/home/tkacvins/project/libbar/include/Bar.h:38:7: note: a
different type is defined in another translation unit
 class Bar {
       ^
/home/home/tkacvins/project/libbar/include/Bar.h:40:32: note: the
first difference of corresponding definitions is field ‘api’
         boost::python::object* api;
                                ^
/home/home/tkacvins/project/libbar/include/Bar.h:40:32: note: a field
of same name but different type is defined in another translation unit
         boost::python::object* api;
                                ^
/home/home/tkacvins/project/libbar/include/Bar.h:15:11: note: type
name ‘boost::python::object’ should match type name
‘boost::python::api::object’
     class object;
           ^
/home/BUILD64/lib/boost-1.69.0-py39-1/include/boost/python/object_core.hpp:238:9:
note: the incompatible type is defined here
   class object : public object_base
         ^
/home/tkacvins/libfoo/include/foo.h:21:8: error: type ‘struct
C_Function_Table’ violates the C++ One Definition Rule [-Werror=odr]
 struct C_Function_Table
        ^
/home/tkacvins/project/libfoo/include/gen_foo.h:14: note: a different
type is defined in another translation unit
 struct C_Function_Table

/home/tkacvins/libfoo/include/foo.h:25:52: note: the first difference
of corresponding definitions is field ‘create_quick_script_for_c’
                                     int is_whitebox);
                                                    ^
/home/tkacvins/project/libfoo/include/gen_foo.h:16: note: a field with
different name is defined in another translation unit
  void (*create_quick_script_for_ada)(const char* param0, const char*
param1, const char* param2, int param3);

/home/home/tkacvins/project/libbar/include/Bar.h:82: error:
‘a2e_guard’ violates the C++ One Definition Rule  [-Werror=odr]
         void a2e_guard(bool enabled);

/home/home/tkacvins/project/libbar/src/src/Bar.cpp:333: note: implicit
this pointer type mismatch
 void Bar::a2e_guard(bool enabled){

/home/home/tkacvins/project/libbar/include/Bar.h:38:7: note: type
‘struct Bar’ itself violates the C++ One Definition Rule
 class Bar {
       ^
/home/home/tkacvins/project/libbar/src/src/Bar.cpp:333: note:
‘a2e_guard’ was previously declared here
 void Bar::a2e_guard(bool enabled){

/home/home/tkacvins/project/libbar/include/Bar.h:73:36: error:
‘instance’ violates the C++ One Definition Rule  [-Werror=odr]
         static Bar * instance();
                                    ^
/home/home/tkacvins/project/libbar/src/src/Bar.cpp:329: note: return
value type mismatch
 Bar * Bar::instance(){

/home/home/tkacvins/project/libbar/include/Bar.h:38:7: note: type
‘struct Bar’ itself violates the C++ One Definition Rule
 class Bar {
       ^
/home/home/tkacvins/project/libbar/src/src/Bar.cpp:329: note:
‘instance’ was previously declared here
 Bar * Bar::instance(){

/home/home/tkacvins/project/libbar/include/Bar.h:172:21: error:
‘get_source_file_name’ violates the C++ One Definition Rule
[-Werror=odr]
         std::string get_source_file_name(int source_file_index);
                     ^
/home/home/tkacvins/project/libbar/src/src/Bar.cpp:1901: note:
implicit this pointer type mismatch
 std::string Bar::get_source_file_name(int source_file_index){

/home/home/tkacvins/project/libbar/include/Bar.h:38:7: note: type
‘struct Bar’ itself violates the C++ One Definition Rule
 class Bar {
       ^
/home/home/tkacvins/project/libbar/src/src/Bar.cpp:1901: note:
‘get_source_file_name’ was previously declared here
 std::string Bar::get_source_file_name(int source_file_index){

/home/home/tkacvins/project/libbar/include/Bar.h:79:14: error:
‘get_enable_commits’ violates the C++ One Definition Rule
[-Werror=odr]
         bool get_enable_commits();
              ^
/home/home/tkacvins/project/libbar/src/src/Bar.cpp:447: note: implicit
this pointer type mismatch
 bool Bar::get_enable_commits(){

/home/home/tkacvins/project/libbar/include/Bar.h:38:7: note: type
‘struct Bar’ itself violates the C++ One Definition Rule
 class Bar {
       ^
/home/home/tkacvins/project/libbar/src/src/Bar.cpp:447: note:
‘get_enable_commits’ was previously declared here
 bool Bar::get_enable_commits(){

/home/home/tkacvins/project/libbar/include/Bar.h:78:14: error:
‘set_enable_commits’ violates the C++ One Definition Rule
[-Werror=odr]
         void set_enable_commits(bool enabled);
              ^
/home/home/tkacvins/project/libbar/src/src/Bar.cpp:443: note: implicit
this pointer type mismatch
 void Bar::set_enable_commits(bool enabled){

/home/home/tkacvins/project/libbar/include/Bar.h:38:7: note: type
‘struct Bar’ itself violates the C++ One Definition Rule
 class Bar {
       ^
/home/home/tkacvins/project/libbar/src/src/Bar.cpp:443: note:
‘set_enable_commits’ was previously declared here
 void Bar::set_enable_commits(bool enabled){

/home/home/tkacvins/project/libbar/include/Bar.h:77:14: error:
‘commit’ violates the C++ One Definition Rule  [-Werror=odr]
         void commit();
              ^
/home/home/tkacvins/project/libbar/src/src/Bar.cpp:432: note: implicit
this pointer type mismatch
 void Bar::commit(){

/home/home/tkacvins/project/libbar/include/Bar.h:38:7: note: type
‘struct Bar’ itself violates the C++ One Definition Rule
 class Bar {
       ^
/home/home/tkacvins/project/libbar/src/src/Bar.cpp:432: note: ‘commit’
was previously declared here
 void Bar::commit(){

/home/home/tkacvins/project/libbar/include/Bar.h:92:14: error:
‘ada_clean_slots’ violates the C++ One Definition Rule  [-Werror=odr]
         void ada_clean_slots            (int parent_id, int max_index);
              ^
/home/home/tkacvins/project/libbar/src/src/Bar.cpp:963: note: implicit
this pointer type mismatch
 void Bar::ada_clean_slots(int parent_id, int max_index){

/home/home/tkacvins/project/libbar/include/Bar.h:38:7: note: type
‘struct Bar’ itself violates the C++ One Definition Rule
 class Bar {
       ^
/home/home/tkacvins/project/libbar/src/src/Bar.cpp:963: note:
‘ada_clean_slots’ was previously declared here
 void Bar::ada_clean_slots(int parent_id, int max_index){

/home/home/tkacvins/project/libbar/include/Bar.h:91:14: error:
‘ada_add_slot’ violates the C++ One Definition Rule  [-Werror=odr]
         void ada_add_slot               (int parent_id, int
testcase_id, int index, int iterations, const char* delay, int print);
              ^
/home/home/tkacvins/project/libbar/src/src/Bar.cpp:951: note: implicit
this pointer type mismatch
 void Bar::ada_add_slot(int parent_id, int testcase_id, int index, int
iterations, const char * delay, int print) {

/home/home/tkacvins/project/libbar/include/Bar.h:38:7: note: type
‘struct Bar’ itself violates the C++ One Definition Rule
 class Bar {
       ^
/home/home/tkacvins/project/libbar/src/src/Bar.cpp:951: note:
‘ada_add_slot’ was previously declared here
 void Bar::ada_add_slot(int parent_id, int testcase_id, int index, int
iterations, const char * delay, int print) {

/home/home/tkacvins/project/libbar/include/Bar.h:90:14: error:
‘delete_testcase_results’ violates the C++ One Definition Rule
[-Werror=odr]
         void delete_testcase_results    (int unique_id );
              ^
/home/home/tkacvins/project/libbar/src/src/Bar.cpp:486: note: implicit
this pointer type mismatch
 void Bar::delete_testcase_results(int unique_id){

/home/home/tkacvins/project/libbar/include/Bar.h:38:7: note: type
‘struct Bar’ itself violates the C++ One Definition Rule
 class Bar {
       ^
/home/home/tkacvins/project/libbar/src/src/Bar.cpp:486: note:
‘delete_testcase_results’ was previously declared here
 void Bar::delete_testcase_results(int unique_id){

/home/home/tkacvins/project/libbar/include/Bar.h:89:14: error:
‘delete_testcase’ violates the C++ One Definition Rule  [-Werror=odr]
         void delete_testcase            (int unique_id );
              ^
/home/home/tkacvins/project/libbar/src/src/Bar.cpp:477: note: implicit
this pointer type mismatch
 void Bar::delete_testcase(int unique_id){

/home/home/tkacvins/project/libbar/include/Bar.h:38:7: note: type
‘struct Bar’ itself violates the C++ One Definition Rule
 class Bar {
       ^
/home/home/tkacvins/project/libbar/src/src/Bar.cpp:477: note:
‘delete_testcase’ was previously declared here
 void Bar::delete_testcase(int unique_id){

/home/home/tkacvins/project/libbar/include/Bar.h:88:14: error:
‘replace_testcase’ violates the C++ One Definition Rule  [-Werror=odr]
         void replace_testcase           (int uut_id, int
subprogram_id, int testcase_index);
              ^
/home/home/tkacvins/project/libbar/src/src/Bar.cpp:676: note: implicit
this pointer type mismatch
 void Bar::replace_testcase(int uutID, int subprogID, int tcID){

/home/home/tkacvins/project/libbar/include/Bar.h:38:7: note: type
‘struct Bar’ itself violates the C++ One Definition Rule
 class Bar {
       ^
/home/home/tkacvins/project/libbar/src/src/Bar.cpp:676: note:
‘replace_testcase’ was previously declared here
 void Bar::replace_testcase(int uutID, int subprogID, int tcID){

/home/home/tkacvins/project/libbar/include/Bar.h:58:9: error:
‘__dt_comp ’ violates the C++ One Definition Rule  [-Werror=odr]
         ~Bar();
         ^
/home/home/tkacvins/project/libbar/src/src/Bar.cpp:295: note: implicit
this pointer type mismatch
 Bar::~Bar() {

/home/home/tkacvins/project/libbar/include/Bar.h:38:7: note: type
‘struct Bar’ itself violates the C++ One Definition Rule
 class Bar {
       ^
/home/home/tkacvins/project/libbar/src/src/Bar.cpp:295: note:
‘__dt_comp ’ was previously declared here
 Bar::~Bar() {

/home/home/tkacvins/project/libbar/include/Bar.h:80:14: error: ‘flush’
violates the C++ One Definition Rule  [-Werror=odr]
         void flush();
              ^
/home/home/tkacvins/project/libbar/src/src/Bar.cpp:421: note: implicit
this pointer type mismatch
 void Bar::flush(){

/home/home/tkacvins/project/libbar/include/Bar.h:38:7: note: type
‘struct Bar’ itself violates the C++ One Definition Rule
 class Bar {
       ^
/home/home/tkacvins/project/libbar/src/src/Bar.cpp:421: note: ‘flush’
was previously declared here
 void Bar::flush(){

/home/home/tkacvins/project/libbar/include/Bar.h:57:9: error:
‘__ct_comp ’ violates the C++ One Definition Rule  [-Werror=odr]
         Bar(const char* path, bool force_create=false);
         ^
/home/home/tkacvins/project/libbar/src/src/Bar.cpp:231: note: implicit
this pointer type mismatch
 Bar::Bar(const char* path, bool force_create) {

/home/home/tkacvins/project/libbar/include/Bar.h:38:7: note: type
‘struct Bar’ itself violates the C++ One Definition Rule
 class Bar {
       ^
/home/home/tkacvins/project/libbar/src/src/Bar.cpp:231: note:
‘__ct_comp ’ was previously declared here
 Bar::Bar(const char* path, bool force_create) {

Re: GCC 8.3.0, -flto and violation of C++ One Definition Rule

[Jonathan Wakely]

On Wed, 29 Dec 2021 at 16:16, Tom Kacvinsky <tkacvins@gmail.com> wrote:
>
> On Wed, Dec 29, 2021 at 10:39 AM Jonathan Wakely <jwakely.gcc@gmail.com> wrote:
> >
> >
> >
> > On Wed, 29 Dec 2021, 11:45 Tom Kacvinsky via Gcc-help, <gcc-help@gcc.gnu.org> wrote:
> >>
> >> Hi,
> >>
> >> First, using GCC 8.3.0 and binutils 2.37.I am trying to increase
> >> performance of linking our product, so I thought I'd give LTO a try.  So
> >> I am compiling all object files with -flto, and passing -flto to g++
> >> (which we use as our link driver).  However, what I have found is that
> >> some of our code violates the C++ One Definition Rule (-Werror=odr). This
> >> only happens when building with LTO - without LTO, the C++ rule is
> >> not violated.
> >
> >
> > As already explained, this is almost certainly wrong. It is more likely that the LTO violation is always present, but only detected when using LTO.
> >
> >
> >>   The problem exists with LTO using both the BFD and gold
> >> linkers.
> >>
> >> So, my question is, since the LTO object files are now such that one
> >> needs to use gcc-nm to examine them (which I know is a wrapper around nm,
> >> and passes an option to load the LTO plugin). how can I leverage that to
> >> see if there are other translation units that define the class that ODR
> >> violation is complaining about?  I did do a fairly thorough analysis of
> >> the object files and did not see there the particular class and methods
> >> would be multiply defined,
> >
> >
> > It would help if you tell us the actual error/warning you get. -Wodr can warn about various different things. It does not warn about multiple definitions, it warns about *inconsistent* definitions.
> >
>
> This is long.  Not sure of the attachment fule for this, so I am
> pasting it in email.  Ib obfuscated the actual source file
> names, but this is the general gist of the link error.  I wonder if
> the error is coming from boost::python::api::object.
>
> /home/home/tkacvins/project/libbar/include/Bar.h:38:7: error: type
> ‘struct Bar’ violates the C++ One Definition Rule [-Werror=odr]

You said it was defined in one C++ file, but it's clearly defined in a
header. So the problem is that the definition is different in
different translation units.



>  class Bar {
>        ^
> /home/home/tkacvins/project/libbar/include/Bar.h:38:7: note: a
> different type is defined in another translation unit
>  class Bar {
>        ^
> /home/home/tkacvins/project/libbar/include/Bar.h:40:32: note: the
> first difference of corresponding definitions is field ‘api’
>          boost::python::object* api;
>                                 ^
> /home/home/tkacvins/project/libbar/include/Bar.h:40:32: note: a field
> of same name but different type is defined in another translation unit
>          boost::python::object* api;
>                                 ^
> /home/home/tkacvins/project/libbar/include/Bar.h:15:11: note: type
> name ‘boost::python::object’ should match type name
> ‘boost::python::api::object’
>      class object;
>            ^
> /home/BUILD64/lib/boost-1.69.0-py39-1/include/boost/python/object_core.hpp:238:9:
> note: the incompatible type is defined here
>    class object : public object_base
>          ^

As it says, one definition has a member of type
‘boost::python::object’ and another has a member of type
‘boost::python::api::object’. I have two guesses how that could
happen: either you're compiling with two different versions of boost
(which seems unlikely because I think boost::python::api::object has
been in that namespace for 20 years), or you are using a forward
declaration of boost::python::object in your own files, instead of
including the correct boost header to define it properly.

The most likely explanation is that somebody tried to "optimize" the
build by cheating, and not including the right boost header for the
type.

Re: GCC 8.3.0, -flto and violation of C++ One Definition Rule

[Jonathan Wakely]

On Wed, 29 Dec 2021 at 17:01, Jonathan Wakely wrote:
>
> On Wed, 29 Dec 2021 at 16:16, Tom Kacvinsky <tkacvins@gmail.com> wrote:
> >
> > On Wed, Dec 29, 2021 at 10:39 AM Jonathan Wakely <jwakely.gcc@gmail.com> wrote:
> > >
> > >
> > >
> > > On Wed, 29 Dec 2021, 11:45 Tom Kacvinsky via Gcc-help, <gcc-help@gcc.gnu.org> wrote:
> > >>
> > >> Hi,
> > >>
> > >> First, using GCC 8.3.0 and binutils 2.37.I am trying to increase
> > >> performance of linking our product, so I thought I'd give LTO a try.  So
> > >> I am compiling all object files with -flto, and passing -flto to g++
> > >> (which we use as our link driver).  However, what I have found is that
> > >> some of our code violates the C++ One Definition Rule (-Werror=odr). This
> > >> only happens when building with LTO - without LTO, the C++ rule is
> > >> not violated.
> > >
> > >
> > > As already explained, this is almost certainly wrong. It is more likely that the LTO violation is always present, but only detected when using LTO.
> > >
> > >
> > >>   The problem exists with LTO using both the BFD and gold
> > >> linkers.
> > >>
> > >> So, my question is, since the LTO object files are now such that one
> > >> needs to use gcc-nm to examine them (which I know is a wrapper around nm,
> > >> and passes an option to load the LTO plugin). how can I leverage that to
> > >> see if there are other translation units that define the class that ODR
> > >> violation is complaining about?  I did do a fairly thorough analysis of
> > >> the object files and did not see there the particular class and methods
> > >> would be multiply defined,
> > >
> > >
> > > It would help if you tell us the actual error/warning you get. -Wodr can warn about various different things. It does not warn about multiple definitions, it warns about *inconsistent* definitions.
> > >
> >
> > This is long.  Not sure of the attachment fule for this, so I am
> > pasting it in email.  Ib obfuscated the actual source file
> > names, but this is the general gist of the link error.  I wonder if
> > the error is coming from boost::python::api::object.
> >
> > /home/home/tkacvins/project/libbar/include/Bar.h:38:7: error: type
> > ‘struct Bar’ violates the C++ One Definition Rule [-Werror=odr]
>
> You said it was defined in one C++ file, but it's clearly defined in a
> header. So the problem is that the definition is different in
> different translation units.
>
>
>
> >  class Bar {
> >        ^
> > /home/home/tkacvins/project/libbar/include/Bar.h:38:7: note: a
> > different type is defined in another translation unit
> >  class Bar {
> >        ^
> > /home/home/tkacvins/project/libbar/include/Bar.h:40:32: note: the
> > first difference of corresponding definitions is field ‘api’
> >          boost::python::object* api;
> >                                 ^
> > /home/home/tkacvins/project/libbar/include/Bar.h:40:32: note: a field
> > of same name but different type is defined in another translation unit
> >          boost::python::object* api;
> >                                 ^
> > /home/home/tkacvins/project/libbar/include/Bar.h:15:11: note: type
> > name ‘boost::python::object’ should match type name
> > ‘boost::python::api::object’
> >      class object;
> >            ^
> > /home/BUILD64/lib/boost-1.69.0-py39-1/include/boost/python/object_core.hpp:238:9:
> > note: the incompatible type is defined here
> >    class object : public object_base
> >          ^
>
> As it says, one definition has a member of type
> ‘boost::python::object’ and another has a member of type
> ‘boost::python::api::object’. I have two guesses how that could
> happen: either you're compiling with two different versions of boost
> (which seems unlikely because I think boost::python::api::object has
> been in that namespace for 20 years), or you are using a forward
> declaration of boost::python::object in your own files, instead of
> including the correct boost header to define it properly.
>
> The most likely explanation is that somebody tried to "optimize" the
> build by cheating, and not including the right boost header for the
> type.

Including <boost/python/object_fwd.hpp> would be the correct way to do that.

Re: GCC 8.3.0, -flto and violation of C++ One Definition Rule

[Tom Kacvinsky]

On Wed, Dec 29, 2021 at 12:04 PM Jonathan Wakely <jwakely.gcc@gmail.com> wrote:
>
> On Wed, 29 Dec 2021 at 17:01, Jonathan Wakely wrote:
> >
> > On Wed, 29 Dec 2021 at 16:16, Tom Kacvinsky <tkacvins@gmail.com> wrote:
> > >
> > > On Wed, Dec 29, 2021 at 10:39 AM Jonathan Wakely <jwakely.gcc@gmail.com> wrote:
> > > >
> > > >
> > > >
> > > > On Wed, 29 Dec 2021, 11:45 Tom Kacvinsky via Gcc-help, <gcc-help@gcc.gnu.org> wrote:
> > > >>
> > > >> Hi,
> > > >>
> > > >> First, using GCC 8.3.0 and binutils 2.37.I am trying to increase
> > > >> performance of linking our product, so I thought I'd give LTO a try.  So
> > > >> I am compiling all object files with -flto, and passing -flto to g++
> > > >> (which we use as our link driver).  However, what I have found is that
> > > >> some of our code violates the C++ One Definition Rule (-Werror=odr). This
> > > >> only happens when building with LTO - without LTO, the C++ rule is
> > > >> not violated.
> > > >
> > > >
> > > > As already explained, this is almost certainly wrong. It is more likely that the LTO violation is always present, but only detected when using LTO.
> > > >
> > > >
> > > >>   The problem exists with LTO using both the BFD and gold
> > > >> linkers.
> > > >>
> > > >> So, my question is, since the LTO object files are now such that one
> > > >> needs to use gcc-nm to examine them (which I know is a wrapper around nm,
> > > >> and passes an option to load the LTO plugin). how can I leverage that to
> > > >> see if there are other translation units that define the class that ODR
> > > >> violation is complaining about?  I did do a fairly thorough analysis of
> > > >> the object files and did not see there the particular class and methods
> > > >> would be multiply defined,
> > > >
> > > >
> > > > It would help if you tell us the actual error/warning you get. -Wodr can warn about various different things. It does not warn about multiple definitions, it warns about *inconsistent* definitions.
> > > >
> > >
> > > This is long.  Not sure of the attachment fule for this, so I am
> > > pasting it in email.  Ib obfuscated the actual source file
> > > names, but this is the general gist of the link error.  I wonder if
> > > the error is coming from boost::python::api::object.
> > >
> > > /home/home/tkacvins/project/libbar/include/Bar.h:38:7: error: type
> > > ‘struct Bar’ violates the C++ One Definition Rule [-Werror=odr]
> >
> > You said it was defined in one C++ file, but it's clearly defined in a
> > header. So the problem is that the definition is different in
> > different translation units.
> >
> >
> >
> > >  class Bar {
> > >        ^
> > > /home/home/tkacvins/project/libbar/include/Bar.h:38:7: note: a
> > > different type is defined in another translation unit
> > >  class Bar {
> > >        ^
> > > /home/home/tkacvins/project/libbar/include/Bar.h:40:32: note: the
> > > first difference of corresponding definitions is field ‘api’
> > >          boost::python::object* api;
> > >                                 ^
> > > /home/home/tkacvins/project/libbar/include/Bar.h:40:32: note: a field
> > > of same name but different type is defined in another translation unit
> > >          boost::python::object* api;
> > >                                 ^
> > > /home/home/tkacvins/project/libbar/include/Bar.h:15:11: note: type
> > > name ‘boost::python::object’ should match type name
> > > ‘boost::python::api::object’
> > >      class object;
> > >            ^
> > > /home/BUILD64/lib/boost-1.69.0-py39-1/include/boost/python/object_core.hpp:238:9:
> > > note: the incompatible type is defined here
> > >    class object : public object_base
> > >          ^
> >
> > As it says, one definition has a member of type
> > ‘boost::python::object’ and another has a member of type
> > ‘boost::python::api::object’. I have two guesses how that could
> > happen: either you're compiling with two different versions of boost
> > (which seems unlikely because I think boost::python::api::object has
> > been in that namespace for 20 years), or you are using a forward
> > declaration of boost::python::object in your own files, instead of
> > including the correct boost header to define it properly.
> >
> > The most likely explanation is that somebody tried to "optimize" the
> > build by cheating, and not including the right boost header for the
> > type.
>
> Including <boost/python/object_fwd.hpp> would be the correct way to do that.

OK, I'll try that.  What we had done is some hackery with boost/python.hpp being
include in the C++ file and not the header (with the header in essence "forward
declaring" the boost namespace.  We did this because of a conflict between Qt
having an "#undef slots" but Python's object.h using slots in a struct
definition.
I worked around that, but I can try your suggestion - it just might work around
the issue with having to play the #undef game before including the header that
pulls in boost/python.hpp

Thanks for the help with this, it is much appreciated!

Tom

Re: GCC 8.3.0, -flto and violation of C++ One Definition Rule

[Jonathan Wakely]

On Wed, 29 Dec 2021, 18:18 Tom Kacvinsky, <tkacvins@gmail.com> wrote:

> On Wed, Dec 29, 2021 at 12:04 PM Jonathan Wakely <jwakely.gcc@gmail.com>
> wrote:
> >
> > On Wed, 29 Dec 2021 at 17:01, Jonathan Wakely wrote:
> > >
> > > On Wed, 29 Dec 2021 at 16:16, Tom Kacvinsky <tkacvins@gmail.com>
> wrote:
> > > >
> > > > On Wed, Dec 29, 2021 at 10:39 AM Jonathan Wakely <
> jwakely.gcc@gmail.com> wrote:
> > > > >
> > > > >
> > > > >
> > > > > On Wed, 29 Dec 2021, 11:45 Tom Kacvinsky via Gcc-help, <
> gcc-help@gcc.gnu.org> wrote:
> > > > >>
> > > > >> Hi,
> > > > >>
> > > > >> First, using GCC 8.3.0 and binutils 2.37.I am trying to increase
> > > > >> performance of linking our product, so I thought I'd give LTO a
> try.  So
> > > > >> I am compiling all object files with -flto, and passing -flto to
> g++
> > > > >> (which we use as our link driver).  However, what I have found is
> that
> > > > >> some of our code violates the C++ One Definition Rule
> (-Werror=odr). This
> > > > >> only happens when building with LTO - without LTO, the C++ rule is
> > > > >> not violated.
> > > > >
> > > > >
> > > > > As already explained, this is almost certainly wrong. It is more
> likely that the LTO violation is always present, but only detected when
> using LTO.
> > > > >
> > > > >
> > > > >>   The problem exists with LTO using both the BFD and gold
> > > > >> linkers.
> > > > >>
> > > > >> So, my question is, since the LTO object files are now such that
> one
> > > > >> needs to use gcc-nm to examine them (which I know is a wrapper
> around nm,
> > > > >> and passes an option to load the LTO plugin). how can I leverage
> that to
> > > > >> see if there are other translation units that define the class
> that ODR
> > > > >> violation is complaining about?  I did do a fairly thorough
> analysis of
> > > > >> the object files and did not see there the particular class and
> methods
> > > > >> would be multiply defined,
> > > > >
> > > > >
> > > > > It would help if you tell us the actual error/warning you get.
> -Wodr can warn about various different things. It does not warn about
> multiple definitions, it warns about *inconsistent* definitions.
> > > > >
> > > >
> > > > This is long.  Not sure of the attachment fule for this, so I am
> > > > pasting it in email.  Ib obfuscated the actual source file
> > > > names, but this is the general gist of the link error.  I wonder if
> > > > the error is coming from boost::python::api::object.
> > > >
> > > > /home/home/tkacvins/project/libbar/include/Bar.h:38:7: error: type
> > > > ‘struct Bar’ violates the C++ One Definition Rule [-Werror=odr]
> > >
> > > You said it was defined in one C++ file, but it's clearly defined in a
> > > header. So the problem is that the definition is different in
> > > different translation units.
> > >
> > >
> > >
> > > >  class Bar {
> > > >        ^
> > > > /home/home/tkacvins/project/libbar/include/Bar.h:38:7: note: a
> > > > different type is defined in another translation unit
> > > >  class Bar {
> > > >        ^
> > > > /home/home/tkacvins/project/libbar/include/Bar.h:40:32: note: the
> > > > first difference of corresponding definitions is field ‘api’
> > > >          boost::python::object* api;
> > > >                                 ^
> > > > /home/home/tkacvins/project/libbar/include/Bar.h:40:32: note: a field
> > > > of same name but different type is defined in another translation
> unit
> > > >          boost::python::object* api;
> > > >                                 ^
> > > > /home/home/tkacvins/project/libbar/include/Bar.h:15:11: note: type
> > > > name ‘boost::python::object’ should match type name
> > > > ‘boost::python::api::object’
> > > >      class object;
> > > >            ^
> > > >
> /home/BUILD64/lib/boost-1.69.0-py39-1/include/boost/python/object_core.hpp:238:9:
> > > > note: the incompatible type is defined here
> > > >    class object : public object_base
> > > >          ^
> > >
> > > As it says, one definition has a member of type
> > > ‘boost::python::object’ and another has a member of type
> > > ‘boost::python::api::object’. I have two guesses how that could
> > > happen: either you're compiling with two different versions of boost
> > > (which seems unlikely because I think boost::python::api::object has
> > > been in that namespace for 20 years), or you are using a forward
> > > declaration of boost::python::object in your own files, instead of
> > > including the correct boost header to define it properly.
> > >
> > > The most likely explanation is that somebody tried to "optimize" the
> > > build by cheating, and not including the right boost header for the
> > > type.
> >
> > Including <boost/python/object_fwd.hpp> would be the correct way to do
> that.
>
> OK, I'll try that.  What we had done is some hackery with boost/python.hpp
> being
> include in the C++ file and not the header (with the header in essence
> "forward
> declaring" the boost namespace.



So exactly what I guessed from the error messages :-)

There is no need to debug the LTO objects, just read the errors.

If you really *must* play games like this, you need to do it correctly.
Declare the type in the correct namespace (as the object_fwd.hpp header
does).