PCH and exec-shield...

PCH and exec-shield...

[David Daney]

Using a recent 3.4.0 snapshot, I found that most of the PCH testsuite 
has random failures when run on a machine that has the exec-shield 
"feature" (Fedora Core 1 and probably RH9 also).

It was pointed out to me that putting the following in my 
/etc/sysctl.conf file would fix the problems:

# Turning off exec shield.
kernel.exec-shield = 0
kernel.exec-shield-randomize = 0

I searched through the info pages as well as 
http://gcc.gnu.org/gcc-3.4/changes.html and found no mention of this.

Is this documented anywhere?

If not, should it be?

David Daney

Re: PCH and exec-shield...

[Eric Christopher]

On Tue, 2004-03-02 at 11:01, David Daney wrote:
> Using a recent 3.4.0 snapshot, I found that most of the PCH testsuite 
> has random failures when run on a machine that has the exec-shield 
> "feature" (Fedora Core 1 and probably RH9 also).
> 
> It was pointed out to me that putting the following in my 
> /etc/sysctl.conf file would fix the problems:
> 
> # Turning off exec shield.
> kernel.exec-shield = 0
> kernel.exec-shield-randomize = 0
> 
> I searched through the info pages as well as 
> http://gcc.gnu.org/gcc-3.4/changes.html and found no mention of this.
> 
> Is this documented anywhere?

I think I've seen it documented on the fedora release notes, but i could
be wrong. If it isn't, it probably should be there as well. I think a
change to changes.html for 3.4 would be good for this - perhaps in the
PCH section.

-eric

-- 
Eric Christopher <echristo@redhat.com>

Re: PCH and exec-shield...

[law]

In message <1078254320.3736.26.camel@dzur.sfbay.redhat.com>, Eric Christopher w
rites:
 >On Tue, 2004-03-02 at 11:01, David Daney wrote:
 >> Using a recent 3.4.0 snapshot, I found that most of the PCH testsuite 
 >> has random failures when run on a machine that has the exec-shield 
 >> "feature" (Fedora Core 1 and probably RH9 also).
 >> 
 >> It was pointed out to me that putting the following in my 
 >> /etc/sysctl.conf file would fix the problems:
 >> 
 >> # Turning off exec shield.
 >> kernel.exec-shield = 0
 >> kernel.exec-shield-randomize = 0
 >> 
 >> I searched through the info pages as well as 
 >> http://gcc.gnu.org/gcc-3.4/changes.html and found no mention of this.
 >> 
 >> Is this documented anywhere?
 >
 >I think I've seen it documented on the fedora release notes, but i could
 >be wrong.
That matches my recollection as well.

 >If it isn't, it probably should be there as well. I think a
 >change to changes.html for 3.4 would be good for this - perhaps in the
 >PCH section.
Yes, that would probably be wise.

Alternately, I believe we could have the build process mark cc1plus and
friends as not needing/wanting the exec-shield functionality.  IIRC there
is a program you can run which twiddles bits in the ELF headers/sections
which allow you to control exec-shield on a per-binary basis.

jeff

Re: PCH and exec-shield...

[Giovanni Bajo]

David Daney wrote:

[PCH failure with exec-shield]

> I searched through the info pages as well as
> http://gcc.gnu.org/gcc-3.4/changes.html and found no mention of this.
> Is this documented anywhere?
> If not, should it be?

No it's not documented, and yes, it badly needs to be documented. It's tracked
in PR14206. If you feel helpful, you could provide some patches :)

Giovanni Bajo

Re: PCH and exec-shield...

[Jakub Jelinek]

On Tue, Mar 02, 2004 at 12:22:22PM -0700, law@redhat.com wrote:
>  >If it isn't, it probably should be there as well. I think a
>  >change to changes.html for 3.4 would be good for this - perhaps in the
>  >PCH section.
> Yes, that would probably be wise.
> 
> Alternately, I believe we could have the build process mark cc1plus and
> friends as not needing/wanting the exec-shield functionality.  IIRC there
> is a program you can run which twiddles bits in the ELF headers/sections
> which allow you to control exec-shield on a per-binary basis.

I assume the problem is not with exec-shield itself, but with address space
randomization.  There is a way to turn this on/off for all
children spawned from a process via personality, but it is on IA-32 only
ATM.  Once there is some way to turn randomization off on all architectures,
IMHO the GCC driver and GDB should turn randomization off (in the latter
case only by default, with possibility to override if needed).

	Jakub

Re: PCH and exec-shield...

[Ian Lance Taylor]

law@redhat.com writes:

> Alternately, I believe we could have the build process mark cc1plus and
> friends as not needing/wanting the exec-shield functionality.  IIRC there
> is a program you can run which twiddles bits in the ELF headers/sections
> which allow you to control exec-shield on a per-binary basis.

Not to get too far out, but is there any way that PCH could work even
when exec-shield is enabled?

Or is it just the testsuite which is an issue here?

It doesn't seem desirable to disable exec-shield on the whole system
just to make PCH work.

With the default kernel setting, cc1 will only use exec-shield if it
has a PT_GNU_STACK program segment.  Unfortunately objcopy can't fix
that for you.  Also unfortunately I don't think anybody has added a
linker option to disable creating such a segment.  Also unfortunately
the linker options which do exist are not documented.  Grrrr.

Ian

Re: PCH and exec-shield...

[Jakub Jelinek]

On Tue, Mar 02, 2004 at 02:59:04PM -0500, Ian Lance Taylor wrote:
> law@redhat.com writes:
> 
> > Alternately, I believe we could have the build process mark cc1plus and
> > friends as not needing/wanting the exec-shield functionality.  IIRC there
> > is a program you can run which twiddles bits in the ELF headers/sections
> > which allow you to control exec-shield on a per-binary basis.
> 
> Not to get too far out, but is there any way that PCH could work even
> when exec-shield is enabled?
> 
> Or is it just the testsuite which is an issue here?
> 
> It doesn't seem desirable to disable exec-shield on the whole system
> just to make PCH work.
> 
> With the default kernel setting, cc1 will only use exec-shield if it
> has a PT_GNU_STACK program segment.  Unfortunately objcopy can't fix
> that for you.  Also unfortunately I don't think anybody has added a

There is execstack(8) utility which allows to tweak this.

> linker option to disable creating such a segment.  Also unfortunately
> the linker options which do exist are not documented.  Grrrr.

ld --help | grep stack
  -z execstack          Mark executable as requiring executable stack
  -z noexecstack        Mark executable as not requiring executable stack
as --help | grep stack
  --execstack             require executable stack for this object
  --noexecstack           don't require executable stack for this object

Still, I really think PCH has problems with randomization, not
non-executable stack, and randomization at least ought to be orthogonal
to whether stack is executable or not.

	Jakub

Re: PCH and exec-shield...

[Ian Lance Taylor]

Jakub Jelinek <jakub@redhat.com> writes:

> > With the default kernel setting, cc1 will only use exec-shield if it
> > has a PT_GNU_STACK program segment.  Unfortunately objcopy can't fix
> > that for you.  Also unfortunately I don't think anybody has added a
> 
> There is execstack(8) utility which allows to tweak this.

Hmmm.  I don't seem to have it on my Fedora Core 1 system.

> > linker option to disable creating such a segment.  Also unfortunately
> > the linker options which do exist are not documented.  Grrrr.
> 
> ld --help | grep stack
>   -z execstack          Mark executable as requiring executable stack
>   -z noexecstack        Mark executable as not requiring executable stack
> as --help | grep stack
>   --execstack             require executable stack for this object
>   --noexecstack           don't require executable stack for this object

OK, I should not have implied that they are not documented at all.
However, the linker and assembler have manuals.  These options are not
documented there, and they should be.  Also, all new options for these
tools should be mentioned in the NEWS file.  Also, I think there
should be a way to say ``do not mark executable stack requirements;''
that might be the default in the assembler, but it is not in the
linker.

> Still, I really think PCH has problems with randomization, not
> non-executable stack, and randomization at least ought to be orthogonal
> to whether stack is executable or not.

Yes.  However, at least in the Fedora Core 1 kernel sources, it
appears that, by default, randomization is tied to whether the program
is marked as requiring or not requiring an executable stack.

Ian

Re: PCH and exec-shield...

[Geoff Keating]

Ian Lance Taylor <ian@wasabisystems.com> writes:

> law@redhat.com writes:
> 
> > Alternately, I believe we could have the build process mark cc1plus and
> > friends as not needing/wanting the exec-shield functionality.  IIRC there
> > is a program you can run which twiddles bits in the ELF headers/sections
> > which allow you to control exec-shield on a per-binary basis.
> 
> Not to get too far out, but is there any way that PCH could work even
> when exec-shield is enabled?

I believe this is now a FAQ on this list.

Yes, this can be done.  All you have to do is to find a chunk of
address space that is safe, and write HOST_HOOKS_GT_PCH_GET_ADDRESS and
HOST_HOOKS_GT_PCH_USE_ADDRESS routines.  For an example, look at
config/rs6000/host-darwin.c; the hooks are also documented in texinfo.

How to find 'safe' address space?  The approach in host-darwin.c is
probably not a good idea on x86 linux, but there are regions of
address space that are unused; you might need to write a linker script
fragment to reserve them, or maybe they'll be free by default.

-- 
- Geoffrey Keating <geoffk@geoffk.org>

Re: PCH and exec-shield...

[Geoff Keating]

Eric Christopher <echristo@redhat.com> writes:

> On Tue, 2004-03-02 at 11:01, David Daney wrote:
> > Using a recent 3.4.0 snapshot, I found that most of the PCH testsuite 
> > has random failures when run on a machine that has the exec-shield 
> > "feature" (Fedora Core 1 and probably RH9 also).
> > 
> > It was pointed out to me that putting the following in my 
> > /etc/sysctl.conf file would fix the problems:
> > 
> > # Turning off exec shield.
> > kernel.exec-shield = 0
> > kernel.exec-shield-randomize = 0
> > 
> > I searched through the info pages as well as 
> > http://gcc.gnu.org/gcc-3.4/changes.html and found no mention of this.
> > 
> > Is this documented anywhere?
> 
> I think I've seen it documented on the fedora release notes, but i could
> be wrong. If it isn't, it probably should be there as well. I think a
> change to changes.html for 3.4 would be good for this - perhaps in the
> PCH section.

In the installation notes for linux, perhaps?

-- 
- Geoffrey Keating <geoffk@geoffk.org>

Re: PCH and exec-shield...

[Ian Lance Taylor]

David Daney <ddaney@avtrex.com> writes:

> It was pointed out to me that putting the following in my
> /etc/sysctl.conf file would fix the problems:
> 
> # Turning off exec shield.
> kernel.exec-shield = 0
> kernel.exec-shield-randomize = 0

Using the gcc PCH testsuite, I just confirmed that, as expected, you
only need to set kernel.exec-shield-randomize to 0 to get the tests to
pass.  The value of kernel.exec-shield is not relevant.

Ian

Re: PCH and exec-shield...

[Mark Mitchell]

Ian Lance Taylor wrote:

>law@redhat.com writes:
>
>  
>
>>Alternately, I believe we could have the build process mark cc1plus and
>>friends as not needing/wanting the exec-shield functionality.  IIRC there
>>is a program you can run which twiddles bits in the ELF headers/sections
>>which allow you to control exec-shield on a per-binary basis.
>>    
>>
>
>Not to get too far out, but is there any way that PCH could work even
>when exec-shield is enabled?
>  
>
Geoff has already given you the current, platform-specific, approach.

There is a more general approach used by other PCH implementations which 
involves walking the loaded PCH and swizzling pointers.  That would be a 
superior approach as it would make PCH more robust on all platforms, and 
would permit people to use exec-shield randomization (and similar 
features on other operating systems) with PCH.

Until that is implemented, many people will probably complain that PCH 
is broken.

-- 
Mark Mitchell
CodeSourcery, LLC
(916) 791-8304
mark@codesourcery.com

Re: PCH and exec-shield...

[Ian Lance Taylor]

Mark Mitchell <mark@codesourcery.com> writes:

> >Not to get too far out, but is there any way that PCH could work even
> >when exec-shield is enabled?
> >
> Geoff has already given you the current, platform-specific, approach.
> 
> There is a more general approach used by other PCH implementations
> which involves walking the loaded PCH and swizzling pointers.  That
> would be a superior approach as it would make PCH more robust on all
> platforms, and would permit people to use exec-shield randomization
> (and similar features on other operating systems) with PCH.
> 
> Until that is implemented, many people will probably complain that PCH
> is broken.

Well, I do take Geoff's point that it's faster to use a PCH if you
don't have to swizzle pointers when you load it.  And Richard did
manage to work out approaches which work using exec-shield on Linux
(and so did I, for that matter, although Richard's are better).

You just mentioned pointer hashing--I don't think that pointer hashing
requires pointer swizzling when loading a PCH with the current
approach; I think it just requires pointer swizzling when the PCH is
created.

On the other hand, I agree with you that it would be a good idea to
implement PCH pointer swizzling on load as a fallback mechanism for
cases in which the PCH can not be loaded at the predicted memory
address.

Ian

Re: PCH and exec-shield...

[Mark Mitchell]

Ian Lance Taylor wrote:

>Mark Mitchell <mark@codesourcery.com> writes:
>
>  
>
>>>Not to get too far out, but is there any way that PCH could work even
>>>when exec-shield is enabled?
>>>
>>>      
>>>
>>Geoff has already given you the current, platform-specific, approach.
>>
>>There is a more general approach used by other PCH implementations
>>which involves walking the loaded PCH and swizzling pointers.  That
>>would be a superior approach as it would make PCH more robust on all
>>platforms, and would permit people to use exec-shield randomization
>>(and similar features on other operating systems) with PCH.
>>
>>Until that is implemented, many people will probably complain that PCH
>>is broken.
>>    
>>
>
>Well, I do take Geoff's point that it's faster to use a PCH if you
>don't have to swizzle pointers when you load it.  And Richard did
>manage to work out approaches which work using exec-shield on Linux
>(and so did I, for that matter, although Richard's are better).
>  
>
The typical approach is:

(1) Try to reaload the PCH at the right address.

(2) If it ended up at a different address, swizzle.

So, there's no cost unless the VM system didn't put it in the expected 
place.  It's true that there's a cost if you end up at the wrong place, 
but right now we just throw out the PCH altogether in that case.

-- 
Mark Mitchell
CodeSourcery, LLC
(916) 791-8304
mark@codesourcery.com

Re: PCH and exec-shield...

[Mike Stump]

On Monday, March 8, 2004, at 01:04 PM, Mark Mitchell wrote:
> There is a more general approach used by other PCH implementations 
> which involves walking the loaded PCH and swizzling pointers.  That 
> would be a superior approach as it would make PCH more robust on all 
> platforms, and would permit people to use exec-shield randomization 
> (and similar features on other operating systems) with PCH.
>
> Until that is implemented, many people will probably complain that PCH 
> is broken.

Using a feature that serves no purpose other than to slow down 
compilations can reasonably be considered a bug.  By having PCH swizzle 
on load, we permit the introductions of bugs into the compiler.  This 
is bad.  This is why it hasn't been done.

For robustness we should fall back to the normal .h file and pretend 
the PCH file doesn't exist.  I think we might just give a sorry now, 
that might be wrong.

We may one day have a reason to swizzle on load, but I don't think 
we've found it yet.

Re: PCH and exec-shield...

[David Edelsohn]

>>>>> Mike Stump writes:

Mike> Using a feature that serves no purpose other than to slow down 
Mike> compilations can reasonably be considered a bug.  By having PCH swizzle 
Mike> on load, we permit the introductions of bugs into the compiler.  This 
Mike> is bad.  This is why it hasn't been done.

	How does swizzling pointers on load only when the PCH file could
not be loaded at its original location slow down compilations and serve no
purpose?  Please make sure that you are considering the entire context of
the proposal.

David

Re: PCH and exec-shield...

[Mark Mitchell]

Mike Stump wrote:

> On Monday, March 8, 2004, at 01:04 PM, Mark Mitchell wrote:
> 
>> There is a more general approach used by other PCH implementations 
>> which involves walking the loaded PCH and swizzling pointers.  That 
>> would be a superior approach as it would make PCH more robust on all 
>> platforms, and would permit people to use exec-shield randomization 
>> (and similar features on other operating systems) with PCH.
>>
>> Until that is implemented, many people will probably complain that PCH 
>> is broken.
> 
> 
> Using a feature that serves no purpose other than to slow down 
> compilations can reasonably be considered a bug.  By having PCH swizzle 
> on load, we permit the introductions of bugs into the compiler.  This is 
> bad.  This is why it hasn't been done.
> 
> For robustness we should fall back to the normal .h file and pretend the 
> PCH file doesn't exist.  I think we might just give a sorry now, that 
> might be wrong.

I agree that falling back to the normal .h file would be much better 
than giving an error message.

But, there is good evidence (EDG's implementation) that swizzling is 
often much faster than reparsing.  And, on some systems it's the only 
option; the system does not reliably place things at the same addresses 
on multiple executions.  An approach that tries to put things at the 
same addresses, but then falls back to swizzling if necessary, would 
provide a better user experience than your approach, all other things 
being equal.

-- 
Mark Mitchell
CodeSourcery, LLC
mark@codesourcery.com

Re: PCH and exec-shield...

[Mike Stump]

On Monday, March 15, 2004, at 08:14 PM, David Edelsohn wrote:
> 	How does swizzling pointers on load only when the PCH file could
> not be loaded at its original location slow down compilations and 
> serve no
> purpose?

If you accept as true that it could not be loaded at the desired 
address, then yes, swizzling is best, we can agree on this.

On Monday, March 15, 2004, at 08:34 PM, Mark Mitchell wrote:
> And, on some systems it's the only option;

If it is, then, yes, swizzling is fine.

I'm merely suggesting that it is best not to leap to swizzle on load 
prematurely.  For example, on darwin, we had problems with this, and 
had to invent the host-darwin.c scheme to avoid adding the swizzle 
code.  If we can, it is best to talk with the kernel and runtime people 
to find ways to get an address allocated, in all cases, adding OS or 
runtime features as necessary to achieve this.

I'm merely trying to discourage swizzle on load as the first solution, 
when other solutions may exist.

Take for example the OS feature of load at random address by default.  
This has the effect of making PCH unreliable otherwise.  Now, suppose 
there is a bit that is inherited across exec that can turn the feature 
off, instead of SOL, we should instead just set that bit in gcc.c so 
that the invoked compilers are deterministic.  Or, maybe there is a bit 
in the file-format somewhere that we can set to get determinism, if so, 
we should arrange to get that bit set.

For OSes that have just added the feature, but have yet to provide the 
don't do this feature, we could politely ask them for such a feature.


So, out of curiosity, on what system is it the only option?

Re: PCH and exec-shield...

[Mark Mitchell]

Mike Stump wrote:

>> And, on some systems it's the only option;
>
> So, out of curiosity, on what system is it the only option?
>
HP-UX is a system where I, at least, do not know how to make this 
reliably, especially on all variants of HP-UX.  There are enough 
variants and sub-variants that it would be hard to even really test them 
all.

On other systems, it's extremely difficult/impossible to get a single 
PCH to be loaded at the same address reliably across versions of the OS, 
or even on machines running the same version of the OS but configured 
differently.  In the past, we've had this problem on Solaris, for 
example.  In situations where you have NFS-mounted filesytems shared by 
lots of developers, it's often advantageous to be able to use the same 
PCH file for all of the developers, even if it is not maximally fast for 
all of them.

-- 
Mark Mitchell
CodeSourcery, LLC
(916) 791-8304
mark@codesourcery.com

Re: PCH and exec-shield...

[Zack Weinberg]

To ask a tangential question, if (as presently implemented) PCH has to
reload data structures at the original address, why is there a problem
with hashing pointers?

zw

Re: PCH and exec-shield...

[Mike Stump]

On Tuesday, March 16, 2004, at 03:50 PM, Zack Weinberg wrote:
> To ask a tangential question, if (as presently implemented) PCH has to
> reload data structures at the original address, why is there a problem
> with hashing pointers?

We swizzle on PCH write.  PCH write can be amortized across multiple 
uses better, also, during writing, memory is hot.  During load, we 
could have to bring in (from disk/network) and COW it.

Re: PCH and exec-shield...

[Ian Lance Taylor]

Mike Stump <mrs@apple.com> writes:

> On Tuesday, March 16, 2004, at 03:50 PM, Zack Weinberg wrote:
> > To ask a tangential question, if (as presently implemented) PCH has to
> > reload data structures at the original address, why is there a problem
> > with hashing pointers?
> 
> We swizzle on PCH write.  PCH write can be amortized across multiple
> uses better, also, during writing, memory is hot.  During load, we
> could have to bring in (from disk/network) and COW it.

On PCH write, gcc swizzles pointers and also copies the hash tables to
the PCH address.  It seems to me that gcc could rehash the pointer
values while copying the hash tables.

So I still don't understand why we can't hash on pointer values.

Ian

Re: PCH and exec-shield...

[Mike Stump]

On Tuesday, March 16, 2004, at 06:51 PM, Ian Lance Taylor wrote:
> On PCH write, gcc swizzles pointers and also copies the hash tables to
> the PCH address.  It seems to me that gcc could rehash the pointer
> values while copying the hash tables.

Minor nit, I don't believe we copy.  We merely edit in place and write. 
  But yes, I suspect that could be done.

> So I still don't understand why we can't hash on pointer values.

I don't think it was meant that it is impossible, but that more code 
would need to be written.  Merely changing to use address, would be 
wrong.  Doing this and fixing it so that it would work fully, would be 
fine.

Re: PCH and exec-shield...

[Hans-Peter Nilsson]

On Tue, 16 Mar 2004, Zack Weinberg wrote:
> To ask a tangential question, if (as presently implemented) PCH has to
> reload data structures at the original address, why is there a problem
> with hashing pointers?

Maybe I'm answering that question out of its context, but it
seems you abandoned context by the "tangential" qualifier:
Because it's too easy to get unrepeatable behavior, from one
machine to another or from one run to another, including
seemingly harmlessly different code and different bug behavior.

brgds, H-P

Re: PCH and exec-shield...

[Zack Weinberg]

Hans-Peter Nilsson <hp@bitrange.com> writes:
> On Tue, 16 Mar 2004, Zack Weinberg wrote:
>> To ask a tangential question, if (as presently implemented) PCH has
>> to reload data structures at the original address, why is there a
>> problem with hashing pointers?
>
> Maybe I'm answering that question out of its context, but it seems
> you abandoned context by the "tangential" qualifier: Because it's
> too easy to get unrepeatable behavior, from one machine to another
> or from one run to another, including seemingly harmlessly different
> code and different bug behavior.

I think you're conflating two phenomena.  The variation across
machines is an effect of iterating over pointer hashtables and using
the order to influence code generation (for instance, changing which
hard register is assigned to which pseudo).  That's independent of PCH
and everyone agrees this shouldn't be done

PCH goes farther, and (for reasons which I still don't fully
understand) forces one not to use pointer hashing at all, even if the
table is never iterated over.  This is what I don't like.  In
particular, the current places where DECL_UID and TYPE_UID are used
(ignoring tree-ssa) could all be replaced by pointer hashing without
iteration if not for PCH.

zw