Moving C++ code to a different ELF section

Moving C++ code to a different ELF section

[Saul Tamari]

Hi,

I'm trying to move some C++ code to a different ELF section and am
facing some errors which I don't understand. I'm using g++ v4.8.1 on
x86.

When compiling the following code I'm getting these errors:
/tmp/ccpp2AkE.s: Assembler messages:
/tmp/ccpp2AkE.s:63: Error: CFI instruction used without previous .cfi_startproc
/tmp/ccpp2AkE.s:64: Error: CFI instruction used without previous .cfi_startproc
/tmp/ccpp2AkE.s:66: Error: .cfi_endproc without corresponding .cfi_startproc
/tmp/ccpp2AkE.s: Error: open CFI at the end of file; missing
.cfi_endproc directive


The source is:
#include <iostream>
#include <stdlib.h>

int qqq;

int main(int argc, char* argv[])
{
        std::cout << "hey " << std::endl;

        qqq = rand();
        if (qqq > 0x1000000) {
                asm volatile ("jmp 1f \n\t  .pushsection
__kuku,\"ax\",@progbits \n\t 1:");
                std::cout << "0x123456" << std::endl;
                throw 12345;
                asm volatile("jmp 3f \n\t .popsection  \n\t 3:");
        }

        return 0;
}


What do these errors mean? Is there a way to fix them? Is there an
alternate method to move similar code to a different section?


Thanks,
Saul

Re: Moving C++ code to a different ELF section

[Ian Lance Taylor]

On Wed, Feb 19, 2014 at 5:57 AM, Saul Tamari <stamari@gmail.com> wrote:
>
> I'm trying to move some C++ code to a different ELF section and am
> facing some errors which I don't understand. I'm using g++ v4.8.1 on
> x86.
>
> When compiling the following code I'm getting these errors:
> /tmp/ccpp2AkE.s: Assembler messages:
> /tmp/ccpp2AkE.s:63: Error: CFI instruction used without previous .cfi_startproc
> /tmp/ccpp2AkE.s:64: Error: CFI instruction used without previous .cfi_startproc
> /tmp/ccpp2AkE.s:66: Error: .cfi_endproc without corresponding .cfi_startproc
> /tmp/ccpp2AkE.s: Error: open CFI at the end of file; missing
> .cfi_endproc directive
>
>
> The source is:
> #include <iostream>
> #include <stdlib.h>
>
> int qqq;
>
> int main(int argc, char* argv[])
> {
>         std::cout << "hey " << std::endl;
>
>         qqq = rand();
>         if (qqq > 0x1000000) {
>                 asm volatile ("jmp 1f \n\t  .pushsection
> __kuku,\"ax\",@progbits \n\t 1:");
>                 std::cout << "0x123456" << std::endl;
>                 throw 12345;
>                 asm volatile("jmp 3f \n\t .popsection  \n\t 3:");
>         }
>
>         return 0;
> }
>
>
> What do these errors mean? Is there a way to fix them? Is there an
> alternate method to move similar code to a different section?

The assembler errors occur because GCC emits debug info in the
assembler stream using CFI pseudo-ops, and you are moving the
pseudo-ops to a different section in a way that GCC does not
understand.  The assembler is seeing CFI pseudo-ops that make no
sense, so it is giving errors about them.

The approach you are using can not work.  The compiler is not an
assembler.  It does not issue instructions in precise sequence.  It
copies and duplicates and rearranges instructions as it sees fit.
This is so even though you are using asm volatile.  All the asm
volatile promises is that the string will appear at the right point in
execution sequence.  Your strings can only work if they appear at the
right point in the assembler output.  That is a different matter that
the compiler does not guarantee.

You didn't see what you are trying to do, but at a guess you should
look at the -freorder-blocks-and-partition option.

Ian

Re: Moving C++ code to a different ELF section

[Saul Tamari]

Hi,

I am trying to see if moving mostly unused code (e.g. conditional
debug print statements) to a different section (and to different
pages) would impact performance in a large application.

Thanks,
Saul


On Wed, Feb 19, 2014 at 9:11 AM, Ian Lance Taylor <iant@google.com> wrote:
> On Wed, Feb 19, 2014 at 5:57 AM, Saul Tamari <stamari@gmail.com> wrote:
>>
>> I'm trying to move some C++ code to a different ELF section and am
>> facing some errors which I don't understand. I'm using g++ v4.8.1 on
>> x86.
>>
>> When compiling the following code I'm getting these errors:
>> /tmp/ccpp2AkE.s: Assembler messages:
>> /tmp/ccpp2AkE.s:63: Error: CFI instruction used without previous .cfi_startproc
>> /tmp/ccpp2AkE.s:64: Error: CFI instruction used without previous .cfi_startproc
>> /tmp/ccpp2AkE.s:66: Error: .cfi_endproc without corresponding .cfi_startproc
>> /tmp/ccpp2AkE.s: Error: open CFI at the end of file; missing
>> .cfi_endproc directive
>>
>>
>> The source is:
>> #include <iostream>
>> #include <stdlib.h>
>>
>> int qqq;
>>
>> int main(int argc, char* argv[])
>> {
>>         std::cout << "hey " << std::endl;
>>
>>         qqq = rand();
>>         if (qqq > 0x1000000) {
>>                 asm volatile ("jmp 1f \n\t  .pushsection
>> __kuku,\"ax\",@progbits \n\t 1:");
>>                 std::cout << "0x123456" << std::endl;
>>                 throw 12345;
>>                 asm volatile("jmp 3f \n\t .popsection  \n\t 3:");
>>         }
>>
>>         return 0;
>> }
>>
>>
>> What do these errors mean? Is there a way to fix them? Is there an
>> alternate method to move similar code to a different section?
>
> The assembler errors occur because GCC emits debug info in the
> assembler stream using CFI pseudo-ops, and you are moving the
> pseudo-ops to a different section in a way that GCC does not
> understand.  The assembler is seeing CFI pseudo-ops that make no
> sense, so it is giving errors about them.
>
> The approach you are using can not work.  The compiler is not an
> assembler.  It does not issue instructions in precise sequence.  It
> copies and duplicates and rearranges instructions as it sees fit.
> This is so even though you are using asm volatile.  All the asm
> volatile promises is that the string will appear at the right point in
> execution sequence.  Your strings can only work if they appear at the
> right point in the assembler output.  That is a different matter that
> the compiler does not guarantee.
>
> You didn't see what you are trying to do, but at a guess you should
> look at the -freorder-blocks-and-partition option.
>
> Ian

Re: Moving C++ code to a different ELF section

[Ian Lance Taylor]

On Wed, Feb 19, 2014 at 7:15 AM, Saul Tamari <stamari@gmail.com> wrote:
>
> I am trying to see if moving mostly unused code (e.g. conditional
> debug print statements) to a different section (and to different
> pages) would impact performance in a large application.

So, you want -freorder-blocks-and-partition.

You can't do it using asm statements that change the section in ways
that the compiler doesn't know about.

Ian


> On Wed, Feb 19, 2014 at 9:11 AM, Ian Lance Taylor <iant@google.com> wrote:
>> On Wed, Feb 19, 2014 at 5:57 AM, Saul Tamari <stamari@gmail.com> wrote:
>>>
>>> I'm trying to move some C++ code to a different ELF section and am
>>> facing some errors which I don't understand. I'm using g++ v4.8.1 on
>>> x86.
>>>
>>> When compiling the following code I'm getting these errors:
>>> /tmp/ccpp2AkE.s: Assembler messages:
>>> /tmp/ccpp2AkE.s:63: Error: CFI instruction used without previous .cfi_startproc
>>> /tmp/ccpp2AkE.s:64: Error: CFI instruction used without previous .cfi_startproc
>>> /tmp/ccpp2AkE.s:66: Error: .cfi_endproc without corresponding .cfi_startproc
>>> /tmp/ccpp2AkE.s: Error: open CFI at the end of file; missing
>>> .cfi_endproc directive
>>>
>>>
>>> The source is:
>>> #include <iostream>
>>> #include <stdlib.h>
>>>
>>> int qqq;
>>>
>>> int main(int argc, char* argv[])
>>> {
>>>         std::cout << "hey " << std::endl;
>>>
>>>         qqq = rand();
>>>         if (qqq > 0x1000000) {
>>>                 asm volatile ("jmp 1f \n\t  .pushsection
>>> __kuku,\"ax\",@progbits \n\t 1:");
>>>                 std::cout << "0x123456" << std::endl;
>>>                 throw 12345;
>>>                 asm volatile("jmp 3f \n\t .popsection  \n\t 3:");
>>>         }
>>>
>>>         return 0;
>>> }
>>>
>>>
>>> What do these errors mean? Is there a way to fix them? Is there an
>>> alternate method to move similar code to a different section?
>>
>> The assembler errors occur because GCC emits debug info in the
>> assembler stream using CFI pseudo-ops, and you are moving the
>> pseudo-ops to a different section in a way that GCC does not
>> understand.  The assembler is seeing CFI pseudo-ops that make no
>> sense, so it is giving errors about them.
>>
>> The approach you are using can not work.  The compiler is not an
>> assembler.  It does not issue instructions in precise sequence.  It
>> copies and duplicates and rearranges instructions as it sees fit.
>> This is so even though you are using asm volatile.  All the asm
>> volatile promises is that the string will appear at the right point in
>> execution sequence.  Your strings can only work if they appear at the
>> right point in the assembler output.  That is a different matter that
>> the compiler does not guarantee.
>>
>> You didn't see what you are trying to do, but at a guess you should
>> look at the -freorder-blocks-and-partition option.
>>
>> Ian

Re: Moving C++ code to a different ELF section

[Saul Tamari]

In the gcc manual the -freorder-blocks-and-partition description
includes the following:
"...
This optimization is automatically turned off in the presence of
exception handling, for linkonce sections, for functions with a
user-defined section attribute and on any architecture that does not
support named sections."

I also tried compiling my application (which uses exceptions) with
this flag and I don't see any new sections in the generated code.


On Wed, Feb 19, 2014 at 12:01 PM, Ian Lance Taylor <iant@google.com> wrote:
> On Wed, Feb 19, 2014 at 7:15 AM, Saul Tamari <stamari@gmail.com> wrote:
>>
>> I am trying to see if moving mostly unused code (e.g. conditional
>> debug print statements) to a different section (and to different
>> pages) would impact performance in a large application.
>
> So, you want -freorder-blocks-and-partition.
>
> You can't do it using asm statements that change the section in ways
> that the compiler doesn't know about.
>
> Ian
>
>
>> On Wed, Feb 19, 2014 at 9:11 AM, Ian Lance Taylor <iant@google.com> wrote:
>>> On Wed, Feb 19, 2014 at 5:57 AM, Saul Tamari <stamari@gmail.com> wrote:
>>>>
>>>> I'm trying to move some C++ code to a different ELF section and am
>>>> facing some errors which I don't understand. I'm using g++ v4.8.1 on
>>>> x86.
>>>>
>>>> When compiling the following code I'm getting these errors:
>>>> /tmp/ccpp2AkE.s: Assembler messages:
>>>> /tmp/ccpp2AkE.s:63: Error: CFI instruction used without previous .cfi_startproc
>>>> /tmp/ccpp2AkE.s:64: Error: CFI instruction used without previous .cfi_startproc
>>>> /tmp/ccpp2AkE.s:66: Error: .cfi_endproc without corresponding .cfi_startproc
>>>> /tmp/ccpp2AkE.s: Error: open CFI at the end of file; missing
>>>> .cfi_endproc directive
>>>>
>>>>
>>>> The source is:
>>>> #include <iostream>
>>>> #include <stdlib.h>
>>>>
>>>> int qqq;
>>>>
>>>> int main(int argc, char* argv[])
>>>> {
>>>>         std::cout << "hey " << std::endl;
>>>>
>>>>         qqq = rand();
>>>>         if (qqq > 0x1000000) {
>>>>                 asm volatile ("jmp 1f \n\t  .pushsection
>>>> __kuku,\"ax\",@progbits \n\t 1:");
>>>>                 std::cout << "0x123456" << std::endl;
>>>>                 throw 12345;
>>>>                 asm volatile("jmp 3f \n\t .popsection  \n\t 3:");
>>>>         }
>>>>
>>>>         return 0;
>>>> }
>>>>
>>>>
>>>> What do these errors mean? Is there a way to fix them? Is there an
>>>> alternate method to move similar code to a different section?
>>>
>>> The assembler errors occur because GCC emits debug info in the
>>> assembler stream using CFI pseudo-ops, and you are moving the
>>> pseudo-ops to a different section in a way that GCC does not
>>> understand.  The assembler is seeing CFI pseudo-ops that make no
>>> sense, so it is giving errors about them.
>>>
>>> The approach you are using can not work.  The compiler is not an
>>> assembler.  It does not issue instructions in precise sequence.  It
>>> copies and duplicates and rearranges instructions as it sees fit.
>>> This is so even though you are using asm volatile.  All the asm
>>> volatile promises is that the string will appear at the right point in
>>> execution sequence.  Your strings can only work if they appear at the
>>> right point in the assembler output.  That is a different matter that
>>> the compiler does not guarantee.
>>>
>>> You didn't see what you are trying to do, but at a guess you should
>>> look at the -freorder-blocks-and-partition option.
>>>
>>> Ian

Re: Moving C++ code to a different ELF section

[Ian Lance Taylor]

On Wed, Feb 19, 2014 at 9:26 AM, Saul Tamari <stamari@gmail.com> wrote:
> In the gcc manual the -freorder-blocks-and-partition description
> includes the following:
> "...
> This optimization is automatically turned off in the presence of
> exception handling, for linkonce sections, for functions with a
> user-defined section attribute and on any architecture that does not
> support named sections."
>
> I also tried compiling my application (which uses exceptions) with
> this flag and I don't see any new sections in the generated code.

OK, then I think your best path forward will be to modify the compiler
to enhance that option to work with exception handling.  I don't know
of any fundamental reason why exception handling is incompatible with
separate sections for hot and cold code, though it will likely require
a modification to the way the exception frame information is generated
(this is, as it happens, the same CFI information that you are getting
errors on with your asm approach).

Or, of course, you can use different functions for the cold code,
using the section attribute to put them in a different section, if
that is workable.

Ian


> On Wed, Feb 19, 2014 at 12:01 PM, Ian Lance Taylor <iant@google.com> wrote:
>> On Wed, Feb 19, 2014 at 7:15 AM, Saul Tamari <stamari@gmail.com> wrote:
>>>
>>> I am trying to see if moving mostly unused code (e.g. conditional
>>> debug print statements) to a different section (and to different
>>> pages) would impact performance in a large application.
>>
>> So, you want -freorder-blocks-and-partition.
>>
>> You can't do it using asm statements that change the section in ways
>> that the compiler doesn't know about.
>>
>> Ian
>>
>>
>>> On Wed, Feb 19, 2014 at 9:11 AM, Ian Lance Taylor <iant@google.com> wrote:
>>>> On Wed, Feb 19, 2014 at 5:57 AM, Saul Tamari <stamari@gmail.com> wrote:
>>>>>
>>>>> I'm trying to move some C++ code to a different ELF section and am
>>>>> facing some errors which I don't understand. I'm using g++ v4.8.1 on
>>>>> x86.
>>>>>
>>>>> When compiling the following code I'm getting these errors:
>>>>> /tmp/ccpp2AkE.s: Assembler messages:
>>>>> /tmp/ccpp2AkE.s:63: Error: CFI instruction used without previous .cfi_startproc
>>>>> /tmp/ccpp2AkE.s:64: Error: CFI instruction used without previous .cfi_startproc
>>>>> /tmp/ccpp2AkE.s:66: Error: .cfi_endproc without corresponding .cfi_startproc
>>>>> /tmp/ccpp2AkE.s: Error: open CFI at the end of file; missing
>>>>> .cfi_endproc directive
>>>>>
>>>>>
>>>>> The source is:
>>>>> #include <iostream>
>>>>> #include <stdlib.h>
>>>>>
>>>>> int qqq;
>>>>>
>>>>> int main(int argc, char* argv[])
>>>>> {
>>>>>         std::cout << "hey " << std::endl;
>>>>>
>>>>>         qqq = rand();
>>>>>         if (qqq > 0x1000000) {
>>>>>                 asm volatile ("jmp 1f \n\t  .pushsection
>>>>> __kuku,\"ax\",@progbits \n\t 1:");
>>>>>                 std::cout << "0x123456" << std::endl;
>>>>>                 throw 12345;
>>>>>                 asm volatile("jmp 3f \n\t .popsection  \n\t 3:");
>>>>>         }
>>>>>
>>>>>         return 0;
>>>>> }
>>>>>
>>>>>
>>>>> What do these errors mean? Is there a way to fix them? Is there an
>>>>> alternate method to move similar code to a different section?
>>>>
>>>> The assembler errors occur because GCC emits debug info in the
>>>> assembler stream using CFI pseudo-ops, and you are moving the
>>>> pseudo-ops to a different section in a way that GCC does not
>>>> understand.  The assembler is seeing CFI pseudo-ops that make no
>>>> sense, so it is giving errors about them.
>>>>
>>>> The approach you are using can not work.  The compiler is not an
>>>> assembler.  It does not issue instructions in precise sequence.  It
>>>> copies and duplicates and rearranges instructions as it sees fit.
>>>> This is so even though you are using asm volatile.  All the asm
>>>> volatile promises is that the string will appear at the right point in
>>>> execution sequence.  Your strings can only work if they appear at the
>>>> right point in the assembler output.  That is a different matter that
>>>> the compiler does not guarantee.
>>>>
>>>> You didn't see what you are trying to do, but at a guess you should
>>>> look at the -freorder-blocks-and-partition option.
>>>>
>>>> Ian

Re: Moving C++ code to a different ELF section

[Saul Tamari]

On Wed, Feb 19, 2014 at 12:45 PM, Ian Lance Taylor <iant@google.com> wrote:
[snip]
> OK, then I think your best path forward will be to modify the compiler
> to enhance that option to work with exception handling.  I don't know
> of any fundamental reason why exception handling is incompatible with
> separate sections for hot and cold code, though it will likely require
> a modification to the way the exception frame information is generated
> (this is, as it happens, the same CFI information that you are getting
> errors on with your asm approach).

That would be quite a challenge for me as I have zero experience with
the gcc codebase.


> Or, of course, you can use different functions for the cold code,
> using the section attribute to put them in a different section, if
> that is workable.

That won't enable me to use it without extensive modifications to the
current code base.


Thanks,
Saul