PCH bit me

PCH bit me

[Alan Modra]

Yesterday I ran into lots of errors in the libstdc++ testsuite, all due
to segfaults.

The scenario was:
1) full bootstrap in clean dir, run testsuite.
2) patch a few things, rebuild, run testsuite looking for regressions.
I forget now exactly how I did the rebuild, probably just with plain
"make".  It certainly wasn't in a clean dir, as the changes I made were
relatively minor and I didn't want to wait hours for a bootstrap to
complete.  (Actually, the "patch a few things" was reverting back to
a clean tree.)

The segfaults were due to size_htab->hash_f being set to some location
inside size_htab_hash rather than to the start of the function.  As you
can imagine, this messed up the stack.  A little snooping with gdb
revealed hash_f being changed:

0x0ff249ac in memcpy () from /lib/libc.so.6
(gdb) bt
#0  0x0ff249ac in memcpy () from /lib/libc.so.6
#1  0x0ff16358 in _IO_file_xsputn () from /lib/libc.so.6
#2  0x0ff17258 in _IO_sgetn () from /lib/libc.so.6
#3  0x0ff09838 in fread () from /lib/libc.so.6
#4  0x10259fe0 in gt_pch_restore (f=0x1)
    at /src/gcc-ppc64-34/gcc/ggc-common.c:576
[snip]

So, the question is, am I just a sorry loser in expecting this scenario
to work, or is pch restoring things it shouldn't?  (or both!)

-- 
Alan Modra
IBM OzLabs - Linux Technology Centre

Re: PCH bit me

[Daniel Jacobowitz]

On Thu, Nov 27, 2003 at 02:53:36PM +1030, Alan Modra wrote:
> Yesterday I ran into lots of errors in the libstdc++ testsuite, all due
> to segfaults.
> 
> The scenario was:
> 1) full bootstrap in clean dir, run testsuite.
> 2) patch a few things, rebuild, run testsuite looking for regressions.
> I forget now exactly how I did the rebuild, probably just with plain
> "make".  It certainly wasn't in a clean dir, as the changes I made were
> relatively minor and I didn't want to wait hours for a bootstrap to
> complete.  (Actually, the "patch a few things" was reverting back to
> a clean tree.)
> 
> The segfaults were due to size_htab->hash_f being set to some location
> inside size_htab_hash rather than to the start of the function.  As you
> can imagine, this messed up the stack.  A little snooping with gdb
> revealed hash_f being changed:
> 
> 0x0ff249ac in memcpy () from /lib/libc.so.6
> (gdb) bt
> #0  0x0ff249ac in memcpy () from /lib/libc.so.6
> #1  0x0ff16358 in _IO_file_xsputn () from /lib/libc.so.6
> #2  0x0ff17258 in _IO_sgetn () from /lib/libc.so.6
> #3  0x0ff09838 in fread () from /lib/libc.so.6
> #4  0x10259fe0 in gt_pch_restore (f=0x1)
>     at /src/gcc-ppc64-34/gcc/ggc-common.c:576
> [snip]
> 
> So, the question is, am I just a sorry loser in expecting this scenario
> to work, or is pch restoring things it shouldn't?  (or both!)

Some weeks ago, there was a discussion about tracking more properties
of the compiler in the PCH file for validation.  I don't remember what
the outcome of it was, though.  What should happen in your case is the
PCH file being rejected because the layout of global variables has
changed.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: PCH bit me

[Alan Modra]

On Wed, Nov 26, 2003 at 11:29:50PM -0500, Daniel Jacobowitz wrote:
> On Thu, Nov 27, 2003 at 02:53:36PM +1030, Alan Modra wrote:
> [about size_htab->hash_f being trashed by pch]
> 
> What should happen in your case is the
> PCH file being rejected because the layout of global variables has
> changed.

I don't think the layout of global vars changed with the patches I made,
just code in .text.  A simple test that _etext and _end were unchanged
would catch most cases..

-- 
Alan Modra
IBM OzLabs - Linux Technology Centre

Re: PCH bit me

[Geoff Keating]

Alan Modra <amodra@bigpond.net.au> writes:

> On Wed, Nov 26, 2003 at 11:29:50PM -0500, Daniel Jacobowitz wrote:
> > On Thu, Nov 27, 2003 at 02:53:36PM +1030, Alan Modra wrote:
> > [about size_htab->hash_f being trashed by pch]
> > 
> > What should happen in your case is the
> > PCH file being rejected because the layout of global variables has
> > changed.
> 
> I don't think the layout of global vars changed with the patches I made,
> just code in .text.  A simple test that _etext and _end were unchanged
> would catch most cases..

This way lies madness (and we're already some distance down this path).

We already have a simple check for this in c-pch.c.  If anything more
complex is desired, I would suggest one of two things:

a) the Linux kernel uses a 'build number' to distinguish cases like
  these.  The first time you type 'make', you get a kernel with
  version number like '2.4.20 #1'; if you do 'make' again in the same
  directory, even with nothing changed, it rebuilds and relinks and
  you get '2.4.20 #2'.

b) compute a proper checksum (maybe using MD5) of every file that goes
  into the link of cc1, except of course the file containing the
  checksum, and use that for the 'same executable' part of the PCH
  verification.

There is an option (c), if you're willing to do much more work, but
you'll want (b) first anyway.

-- 
- Geoffrey Keating <geoffk@geoffk.org>

Re: PCH bit me

[Geoff Keating]

Alan Modra <amodra@bigpond.net.au> writes:

> Yesterday I ran into lots of errors in the libstdc++ testsuite, all due
> to segfaults.
> 
> The scenario was:
> 1) full bootstrap in clean dir, run testsuite.
> 2) patch a few things, rebuild, run testsuite looking for regressions.
> I forget now exactly how I did the rebuild, probably just with plain
> "make".  It certainly wasn't in a clean dir, as the changes I made were
> relatively minor and I didn't want to wait hours for a bootstrap to
> complete.  (Actually, the "patch a few things" was reverting back to
> a clean tree.)
> 
> The segfaults were due to size_htab->hash_f being set to some location
> inside size_htab_hash rather than to the start of the function.  As you
> can imagine, this messed up the stack.  A little snooping with gdb
> revealed hash_f being changed:
> 
> 0x0ff249ac in memcpy () from /lib/libc.so.6
> (gdb) bt
> #0  0x0ff249ac in memcpy () from /lib/libc.so.6
> #1  0x0ff16358 in _IO_file_xsputn () from /lib/libc.so.6
> #2  0x0ff17258 in _IO_sgetn () from /lib/libc.so.6
> #3  0x0ff09838 in fread () from /lib/libc.so.6
> #4  0x10259fe0 in gt_pch_restore (f=0x1)
>     at /src/gcc-ppc64-34/gcc/ggc-common.c:576
> [snip]
> 
> So, the question is, am I just a sorry loser in expecting this scenario
> to work, or is pch restoring things it shouldn't?  (or both!)

Sounds like a missing dependency in the libstdc++ Makefiles; the PCH
files depend on the compiler executable.

(Of course, every built object depends on the compiler executable; but
for PCH files, it's a bit more critical.)

It'd be possible to make PCH more robust against this sort of thing,
but it'd only really be useful for GCC maintainers, and it would
surely take more time than it would save.

-- 
- Geoffrey Keating <geoffk@geoffk.org>