[Android] -fpic default option

[Android] -fpic default option

[Alexander Ivchenko]

By default in Android we always compile with -fpic or -fPIC, even when
compiling executable. Because of that we have some test fails on
Android:

For example:
gcc/testsuite/gcc.target/i386/pr47312.c
/* { dg-do run } */
/* { dg-options "-O2" } */

void exit (int);
void noreturn_autodetection_failed ();
__attribute__ ((noinline))
detect_noreturn ()
{
  exit (0);
}
int
main (void)
{
  detect_noreturn ();
  noreturn_autodetection_failed ();
  return 0;
}

If gcc knew, that we are not going to make a shared library (which is
implied if we are using pic option), then it would delete the call of
noreturn_autodetection_failed as a dead code. But in case of pic we
cannot rely on the fact that detect_noreturn () will not be
overwritten on runtime, eventhough we are compiling executable and
that will never happen! So in addition to several testfails, it seems
that we are losing an optimization opportunity here also.
I'm wondering, maybe we can use -fpie instead of -fpic by default in
the compiler (of course if there are no -shared or -c options; those
cases will mean that we cannot rely on the fact that we will have an
executable after linking)? It seems that it would solve the problem..
Of course we always can add something like { target nonpic } to all
tests that fail on Android, but probably the problem is deeper than
that and that would be only hiding our head in the sand.

Re: [Android] -fpic default option

[Maxim Kuvyrkov]

On 15/11/2012, at 2:26 AM, Alexander Ivchenko wrote:

> By default in Android we always compile with -fpic or -fPIC, even when
> compiling executable. Because of that we have some test fails on
> Android:
> 
> For example:
> gcc/testsuite/gcc.target/i386/pr47312.c
> /* { dg-do run } */
> /* { dg-options "-O2" } */
> 
> void exit (int);
> void noreturn_autodetection_failed ();
> __attribute__ ((noinline))
> detect_noreturn ()
> {
>  exit (0);
> }
> int
> main (void)
> {
>  detect_noreturn ();
>  noreturn_autodetection_failed ();
>  return 0;
> }
> 
> If gcc knew, that we are not going to make a shared library (which is
> implied if we are using pic option), then it would delete the call of
> noreturn_autodetection_failed as a dead code. But in case of pic we
> cannot rely on the fact that detect_noreturn () will not be
> overwritten on runtime, eventhough we are compiling executable and
> that will never happen! So in addition to several testfails, it seems
> that we are losing an optimization opportunity here also.
> I'm wondering, maybe we can use -fpie instead of -fpic by default in
> the compiler (of course if there are no -shared or -c options; those
> cases will mean that we cannot rely on the fact that we will have an
> executable after linking)? It seems that it would solve the problem..
> Of course we always can add something like { target nonpic } to all
> tests that fail on Android, but probably the problem is deeper than
> that and that would be only hiding our head in the sand.

The canonical way of building native applications for Android is to add -fno-pic to the compiler flags.  The intent was to make the most common behavior the default, and for Android that happens to be building shared libraries that can then be called through JNI.

I think the right fix here is to add "-fno-pic" to dg-options for affected tests in gcc.target/i386.  For architecture-independent test, it may be more prudent to use { target nonpic }, as you suggested, since we can't be sure that a given target / multilib can handle non-pic code.

Thank you,

--
Maxim Kuvyrkov
CodeSourcery / Mentor Graphics

Re: [Android] -fpic default option

[H.J. Lu]

On Wed, Nov 14, 2012 at 5:26 AM, Alexander Ivchenko <aivchenk@gmail.com> wrote:
> By default in Android we always compile with -fpic or -fPIC, even when
> compiling executable. Because of that we have some test fails on
> Android:
>
> For example:
> gcc/testsuite/gcc.target/i386/pr47312.c
> /* { dg-do run } */
> /* { dg-options "-O2" } */
>
> void exit (int);
> void noreturn_autodetection_failed ();
> __attribute__ ((noinline))
> detect_noreturn ()
> {
>   exit (0);
> }
> int
> main (void)
> {
>   detect_noreturn ();
>   noreturn_autodetection_failed ();
>   return 0;
> }
>
> If gcc knew, that we are not going to make a shared library (which is
> implied if we are using pic option), then it would delete the call of
> noreturn_autodetection_failed as a dead code. But in case of pic we
> cannot rely on the fact that detect_noreturn () will not be
> overwritten on runtime, eventhough we are compiling executable and
> that will never happen! So in addition to several testfails, it seems
> that we are losing an optimization opportunity here also.
> I'm wondering, maybe we can use -fpie instead of -fpic by default in
> the compiler (of course if there are no -shared or -c options; those
> cases will mean that we cannot rely on the fact that we will have an
> executable after linking)? It seems that it would solve the problem..
> Of course we always can add something like { target nonpic } to all

We should add  { target nonpic } independent of Android.

> tests that fail on Android, but probably the problem is deeper than
> that and that would be only hiding our head in the sand.

Using -fPIE to compile executables is a good idea.

-- 
H.J.

Re: [Android] -fpic default option

[Alexander Ivchenko]

>> The canonical way of building native applications for Android is to add -fno-pic to the compiler flags.
That’s true for programs in userspace, but for Android system-level
programs (standalone executables) like system tools, console apps or
tests it seems -fpic doesn’t make much sense.

>> The intent was to make the most common behavior the default, and for Android that
>> happens to be building shared libraries that can then be called through JNI.
Okay, but for building shared libs we either provide -shared to the
driver, or -c, right? In these cases
we can go with -fPIC by default. In other cases we can safely assume
that the executable will be created and
in such case it would be a good idea to use -fPIE.
That would solve the problem with a lot of tests that failed on
Android and also with native applications (executables, not libs) that
are built
with -fPIC now. What do you think?


2012/11/15 H.J. Lu <hjl.tools@gmail.com>:
> On Wed, Nov 14, 2012 at 5:26 AM, Alexander Ivchenko <aivchenk@gmail.com> wrote:
>> By default in Android we always compile with -fpic or -fPIC, even when
>> compiling executable. Because of that we have some test fails on
>> Android:
>>
>> For example:
>> gcc/testsuite/gcc.target/i386/pr47312.c
>> /* { dg-do run } */
>> /* { dg-options "-O2" } */
>>
>> void exit (int);
>> void noreturn_autodetection_failed ();
>> __attribute__ ((noinline))
>> detect_noreturn ()
>> {
>>   exit (0);
>> }
>> int
>> main (void)
>> {
>>   detect_noreturn ();
>>   noreturn_autodetection_failed ();
>>   return 0;
>> }
>>
>> If gcc knew, that we are not going to make a shared library (which is
>> implied if we are using pic option), then it would delete the call of
>> noreturn_autodetection_failed as a dead code. But in case of pic we
>> cannot rely on the fact that detect_noreturn () will not be
>> overwritten on runtime, eventhough we are compiling executable and
>> that will never happen! So in addition to several testfails, it seems
>> that we are losing an optimization opportunity here also.
>> I'm wondering, maybe we can use -fpie instead of -fpic by default in
>> the compiler (of course if there are no -shared or -c options; those
>> cases will mean that we cannot rely on the fact that we will have an
>> executable after linking)? It seems that it would solve the problem..
>> Of course we always can add something like { target nonpic } to all
>
> We should add  { target nonpic } independent of Android.
>
>> tests that fail on Android, but probably the problem is deeper than
>> that and that would be only hiding our head in the sand.
>
> Using -fPIE to compile executables is a good idea.
>
> --
> H.J.

Re: [Android] -fpic default option

[Maxim Kuvyrkov]

On 15/11/2012, at 10:39 PM, Alexander Ivchenko wrote:

>>> The canonical way of building native applications for Android is to add -fno-pic to the compiler flags.
> That’s true for programs in userspace, but for Android system-level
> programs (standalone executables) like system tools, console apps or
> tests it seems -fpic doesn’t make much sense.

We seem to be saying the same thing, but it sounds like a disagreement.  Native (as in non-java) applications should be compiled with -fno-pic as -fpic (and -fPIE) is unnecessary.

> 
>>> The intent was to make the most common behavior the default, and for Android that
>>> happens to be building shared libraries that can then be called through JNI.
> Okay, but for building shared libs we either provide -shared to the
> driver, or -c, right? In these cases
> we can go with -fPIC by default.

Assuming normal build process (i.e., when one doesn't compile and link application in one command), by the time you provide -shared to the driver all the object files are already compiled.  They /need/ to be compiled as PIC for use in shared library or /can/ be compiled as non-PIC for use in native executable.  And there are also various games with partial linking.

> In other cases we can safely assume
> that the executable will be created and
> in such case it would be a good idea to use -fPIE.

I don't see why it would be a good idea to use -fPIE for most normal user-space applications when -fno-pic would suffice and provide better code.

Am I missing something?

Thank you,

--
Maxim Kuvyrkov
CodeSourcery / Mentor Graphics

Re: [Android] -fpic default option

[Alexander Ivchenko]

You are right, we are talking about the same things. The only open
question I see is regarding:

>>>> In other cases we can safely assume
>>>> that the executable will be created and
>>>> in such case it would be a good idea to use -fPIE.

>> I don't see why it would be a good idea to use -fPIE for most normal user-space applications when -fno-pic would
>> suffice and provide better code.

Do you agree that if the driver knows that the executable will be at
the end of the build
we can safely use -fno-pic or -fPIE to override -fpic default? I'm not
quite sure which one is better for Android; -fPIE will
give us all the security advantages of the position independent code
and probably is a better option.


2012/11/16 Maxim Kuvyrkov <maxim@codesourcery.com>:
> On 15/11/2012, at 10:39 PM, Alexander Ivchenko wrote:
>
>>>> The canonical way of building native applications for Android is to add -fno-pic to the compiler flags.
>> That’s true for programs in userspace, but for Android system-level
>> programs (standalone executables) like system tools, console apps or
>> tests it seems -fpic doesn’t make much sense.
>
> We seem to be saying the same thing, but it sounds like a disagreement.  Native (as in non-java) applications should be compiled with -fno-pic as -fpic (and -fPIE) is unnecessary.
>
>>
>>>> The intent was to make the most common behavior the default, and for Android that
>>>> happens to be building shared libraries that can then be called through JNI.
>> Okay, but for building shared libs we either provide -shared to the
>> driver, or -c, right? In these cases
>> we can go with -fPIC by default.
>
> Assuming normal build process (i.e., when one doesn't compile and link application in one command), by the time you provide -shared to the driver all the object files are already compiled.  They /need/ to be compiled as PIC for use in shared library or /can/ be compiled as non-PIC for use in native executable.  And there are also various games with partial linking.
>
>> In other cases we can safely assume
>> that the executable will be created and
>> in such case it would be a good idea to use -fPIE.
>
> I don't see why it would be a good idea to use -fPIE for most normal user-space applications when -fno-pic would suffice and provide better code.
>
> Am I missing something?
>
> Thank you,
>
> --
> Maxim Kuvyrkov
> CodeSourcery / Mentor Graphics
>

Re: [Android] -fpic default option

[Maxim Kuvyrkov]

On 18/11/2012, at 7:50 AM, Alexander Ivchenko wrote:

> You are right, we are talking about the same things. The only open
> question I see is regarding:
> 
>>>>> In other cases we can safely assume
>>>>> that the executable will be created and
>>>>> in such case it would be a good idea to use -fPIE.
> 
>>> I don't see why it would be a good idea to use -fPIE for most normal user-space applications when -fno-pic would
>>> suffice and provide better code.
> 
> Do you agree that if the driver knows that the executable will be at
> the end of the build
> we can safely use -fno-pic or -fPIE to override -fpic default?

If the driver can be sure that an executable is being compiled (which is a challenge in general case, but simple cases -- compilation of GCC testsuites -- can be accommodated) then it is OK to revert to default for a particular <arch>-linux-gnu target.  For most architectures that would mean reverting to -fno-pic, but at least mips-linux-gnu uses -fpic by default in certain configurations.  This is me not picking on your proposal, just alerting that -fno-pic may not be the right choice for executables for all targets.

> I'm not
> quite sure which one is better for Android; -fPIE will
> give us all the security advantages of the position independent code
> and probably is a better option.

Using -fPIE will also bring all the performance drawbacks of PIC, there is a reason why -fPIE is not the most common way of building applications.  I don't think Android is too security-concious a platform to warrant -fPIE (does Android even use address-space randomization?).  Performance is a more serious consideration for its target market.

Alexander, I appreciate you looking into -fpic vs -fno-pic compilation for Android.

Thank you,

--
Maxim Kuvyrkov
CodeSourcery / Mentor Graphics

Re: [Android] -fpic default option

[Alexander Ivchenko]

> If the driver can be sure that an executable is being compiled
> (which is a challenge in general case, but simple cases --
> compilation of GCC testsuites -- can be accommodated)

I totally agree here that it could be a challenge. Also it could be
quite confusing: if the user wants to get assembler code we must
conservatively generate it with -fpic; However producing executable
with default options and then objdumping it will give us different
assembly with -fno-pic..
And although it could fix those testcases that are failing right now
on android that would indeed be hiding our head in the sand.
Considering all that, I believe that we are left with only one
solution: to carefully add "-fno-pic" or
 "{ target nonpic }" to the affected tests as we discussed above.

Thank you very much for your help!

2012/11/18 Maxim Kuvyrkov <maxim@codesourcery.com>:
> On 18/11/2012, at 7:50 AM, Alexander Ivchenko wrote:
>
>> You are right, we are talking about the same things. The only open
>> question I see is regarding:
>>
>>>>>> In other cases we can safely assume
>>>>>> that the executable will be created and
>>>>>> in such case it would be a good idea to use -fPIE.
>>
>>>> I don't see why it would be a good idea to use -fPIE for most normal user-space applications when -fno-pic would
>>>> suffice and provide better code.
>>
>> Do you agree that if the driver knows that the executable will be at
>> the end of the build
>> we can safely use -fno-pic or -fPIE to override -fpic default?
>
> If the driver can be sure that an executable is being compiled (which is a challenge in general case, but simple cases -- compilation of GCC testsuites -- can be accommodated) then it is OK to revert to default for a particular <arch>-linux-gnu target.  For most architectures that would mean reverting to -fno-pic, but at least mips-linux-gnu uses -fpic by default in certain configurations.  This is me not picking on your proposal, just alerting that -fno-pic may not be the right choice for executables for all targets.
>
>> I'm not
>> quite sure which one is better for Android; -fPIE will
>> give us all the security advantages of the position independent code
>> and probably is a better option.
>
> Using -fPIE will also bring all the performance drawbacks of PIC, there is a reason why -fPIE is not the most common way of building applications.  I don't think Android is too security-concious a platform to warrant -fPIE (does Android even use address-space randomization?).  Performance is a more serious consideration for its target market.
>
> Alexander, I appreciate you looking into -fpic vs -fno-pic compilation for Android.
>
> Thank you,
>
> --
> Maxim Kuvyrkov
> CodeSourcery / Mentor Graphics
>