Re: [Bug pch/14400] Cannot compile qt-x11-free-3.3.0

Re: [Bug pch/14400] Cannot compile qt-x11-free-3.3.0

[Ian Lance Taylor]

"geoffk at gcc dot gnu dot org" <gcc-bugzilla@gcc.gnu.org> writes:

> I believe I saw this on Darwin before implementing the scheme it uses now.  I don't believe there's any 
> solution to the general problem that doesn't know more about the host's memory map; the current 
> generic solution is just a heuristic and is not reliable.  For reliable operation, *especially* on x86-linux-
> gnu, I recommend implementing a solution like Darwin's.  That would also fix 14206.

PR 14206 is fixed on mainline by RTH's patch.  For 3.4 I put in a doc
fix.

> This is not a problem that you can solve with a 'quick fix'.

Perhaps for 3.4 we should disable PCH for any target other than
Darwin.  It seems to me that it is a bad idea to include an unreliable
feature.  It's particularly unfortunate that it causes gcc 3.4 to fail
to build a popular package (qt-x11-free) on a popular platform
(GNU/Linux).  It's even more unfortunate that the failure currently
shows up as a random compiler crash.

I will investigate further on mainline.

Ian

Re: [Bug pch/14400] Cannot compile qt-x11-free-3.3.0

[Geoff Keating]

On Mar 29, 2004, at 11:06 PM, Ian Lance Taylor wrote:

> "geoffk at gcc dot gnu dot org" <gcc-bugzilla@gcc.gnu.org> writes:
>
>> I believe I saw this on Darwin before implementing the scheme it uses 
>> now.  I don't believe there's any
>> solution to the general problem that doesn't know more about the 
>> host's memory map; the current
>> generic solution is just a heuristic and is not reliable.  For 
>> reliable operation, *especially* on x86-linux-
>> gnu, I recommend implementing a solution like Darwin's.  That would 
>> also fix 14206.
>
> PR 14206 is fixed on mainline by RTH's patch.  For 3.4 I put in a doc
> fix.

I've now caught up enough that I read RTH's patch (I've been on 
vacation).  Yes, that patch should fix this problem too.  I'd suggest 
backporting it to 3.4.

-- 
Geoff Keating <geoffk@apple.com>

Re: [Bug pch/14400] Cannot compile qt-x11-free-3.3.0

[Ian Lance Taylor]

"geoffk at apple dot com" <gcc-bugzilla@gcc.gnu.org> writes:

> On Mar 29, 2004, at 11:06 PM, Ian Lance Taylor wrote:

> > PR 14206 is fixed on mainline by RTH's patch.  For 3.4 I put in a doc
> > fix.

> I've now caught up enough that I read RTH's patch (I've been on 
> vacation).  Yes, that patch should fix this problem too.  I'd suggest 
> backporting it to 3.4.

We can do that.  However, I am still troubled by the fact that the
default fallback for gt_pch_get_address/gt_pch_use_address is
unreliable.  RTH's patch may well fix the problem on GNU/Linux for all
but strange cases.  But it seems to me that with that patch PCH will
work reasonably on GNU/Linux and on Darwin, but will not work reliably
on any other system.

That is, I don't think that any system can reliably use
mmap_gt_pch_get_address/mmap_gt_pch_use_address, although they are the
default functions on a system which supports mmap.

Is it better to have PCH which unpredictably fails or PCH which
reliably fails?  Right now, in mainline, I think we have the former,
except on GNU/Linux and Darwin.

Ian

Re: [Bug pch/14400] Cannot compile qt-x11-free-3.3.0

[Geoffrey Keating]

On 02/04/2004, at 7:17 AM, Ian Lance Taylor wrote:

> "geoffk at apple dot com" <gcc-bugzilla@gcc.gnu.org> writes:
>
>> On Mar 29, 2004, at 11:06 PM, Ian Lance Taylor wrote:
>
>>> PR 14206 is fixed on mainline by RTH's patch.  For 3.4 I put in a doc
>>> fix.
>
>> I've now caught up enough that I read RTH's patch (I've been on
>> vacation).  Yes, that patch should fix this problem too.  I'd suggest
>> backporting it to 3.4.
>
> We can do that.  However, I am still troubled by the fact that the
> default fallback for gt_pch_get_address/gt_pch_use_address is
> unreliable.  RTH's patch may well fix the problem on GNU/Linux for all
> but strange cases.  But it seems to me that with that patch PCH will
> work reasonably on GNU/Linux and on Darwin, but will not work reliably
> on any other system.
>
> That is, I don't think that any system can reliably use
> mmap_gt_pch_get_address/mmap_gt_pch_use_address, although they are the
> default functions on a system which supports mmap.
>
> Is it better to have PCH which unpredictably fails or PCH which
> reliably fails?  Right now, in mainline, I think we have the former,
> except on GNU/Linux and Darwin.

I think that PCH that fails on one case out of a thousand is better 
than PCH that fails in one thousand cases out of one thousand.  
However, it would be better if we could make failure even less frequent 
by tweaking the heuristic.  Ian, you've studied the problem, do you 
have any suggestions for a better heuristic?  Maybe gt_pch_get_address 
should map, say, 32M of unused memory before trying to allocate space 
for the PCH.

Re: [Bug pch/14400] Cannot compile qt-x11-free-3.3.0

[Ian Lance Taylor]

Geoffrey Keating <geoffk@apple.com> writes:

> I think that PCH that fails on one case out of a thousand is better
> than PCH that fails in one thousand cases out of one thousand.

Any thoughts on how difficult it would be to, when a PCH can not be
loaded at the correct address, skip the PCH and try for the original
header files?  We'd want to issue a warning, but the result will still
be better then a compiler crash, particularly since the crash will be
awkward to work around without manually removing the PCH.  Of course,
the result will not necessarily be equivalent--the original headers
might not be present, or might have been modified since the PCH was
created.

Ian

Re: [Bug pch/14400] Cannot compile qt-x11-free-3.3.0

[Mike Stump]

On Saturday, April 3, 2004, at 07:38 PM, Ian Lance Taylor wrote:
> Any thoughts on how difficult it would be to, when a PCH can not be
> loaded at the correct address, skip the PCH and try for the original
> header files?

Just move the call that tries the mmap of the pch file to the 
validation routine.  The issue there is we'd need to think about trying 
to mmap twice.  Currently, we only ever do it once.  We would try and 
mmap more than once when there are multiple PCH files for the same 
source, as we run thought all of them, trying to validate each one.

> Of course, the result will not necessarily be equivalent--the original 
> headers
> might not be present, or might have been modified since the PCH was 
> created.

This issue is known and expected and not an issue.

Re: [Bug pch/14400] Cannot compile qt-x11-free-3.3.0

[Geoff Keating]

On Apr 3, 2004, at 7:38 PM, Ian Lance Taylor wrote:

> Geoffrey Keating <geoffk@apple.com> writes:
>
>> I think that PCH that fails on one case out of a thousand is better
>> than PCH that fails in one thousand cases out of one thousand.
>
> Any thoughts on how difficult it would be to, when a PCH can not be
> loaded at the correct address, skip the PCH and try for the original
> header files?  We'd want to issue a warning, but the result will still
> be better then a compiler crash, particularly since the crash will be
> awkward to work around without manually removing the PCH.  Of course,
> the result will not necessarily be equivalent--the original headers
> might not be present, or might have been modified since the PCH was
> created.

The primary difficulty that I see would be in documenting this, that is 
that the result is not convenient to use.  At the moment, the 
documentation for PCH describes everything that you need to do to use a 
PCH, and it's agreed that cases where it does not work are bugs.  
Adding an extra clause that says "oh, and even if you follow these 
rules, on some platforms the compiler might just decide it doesn't like 
your PCH" does not seem very helpful.  Especially if it doesn't come 
with a list of those platforms.

The other difficulty is that I'm not sure that you can always back out 
of the address-space allocation.  I would probably have to see a patch 
before I could say how difficult that might be.

Again, my experience is that on all unix-like platforms, you *can* 
always get a suitable chunk of address space in some fashion, if you 
try hard enough.  For example, HP-UX was quoted as an OS on which it is 
difficult to do this, yet with less than ten minutes' searching the 
web, I found 
<http://docs.hp.com/hpux/onlinedocs/5965-4642/5965-4642.html> which 
seems to give sufficient information under 'Space apportioned to a 
Process"; apparently on HP-UX 10.x in 32-bit mode, data lives beween 
0x40000000 and 0x7fffffff, with a default stack size of 80Mb, so 
probably mapping the PCH so that its end was at 0x6fffffff would work.

-- 
Geoff Keating <geoffk@apple.com>

Re: [Bug pch/14400] Cannot compile qt-x11-free-3.3.0

[Ian Lance Taylor]

Geoff Keating <geoffk@apple.com> writes:

> On Apr 3, 2004, at 7:38 PM, Ian Lance Taylor wrote:
> 
> > Any thoughts on how difficult it would be to, when a PCH can not be
> > loaded at the correct address, skip the PCH and try for the original
> > header files?  We'd want to issue a warning, but the result will still
> > be better then a compiler crash, particularly since the crash will be
> > awkward to work around without manually removing the PCH.  Of course,
> > the result will not necessarily be equivalent--the original headers
> > might not be present, or might have been modified since the PCH was
> > created.

...

> Again, my experience is that on all unix-like platforms, you *can*
> always get a suitable chunk of address space in some fashion, if you
> try hard enough.

Even if that turns out to be true, it doesn't really help us with a
gcc release when in actual fact that work has only been done on two
platforms, namely Darwin and GNU/Linux.  Nobody is working to fix this
on mainline, much less for 3.4.

You didn't like the idea of disabling PCH on other platforms, so we
have to consider the case in which there is no platform specific
implementation, and the default implementation of PCH, using mmap,
fails.  Right now, in that case, the compiler simply fails, either
with a segmentation violation or a call to sorry, and it is rather
difficult to work around the crash without simply discarding the PCH.

Ian