Best way to remove cpp0

Best way to remove cpp0

[Neil Booth]

I'm considering (gradually) moving the functionality of cppmain.c into
the main library, and then dropping cpp0.  Then cc1 or cc1plus etc.
would contian the only preprocessor, including the stand-alone case.

The cc1 binary could then act in three states:

1) Stand-alone preprocessor (invoked with "cpp")
2) Compiler-integrated preprocessor ("gcc", "g++" etc.)
3) Both (something like "gcc -save-temps")

This would remove a few issues with the existing codebase, and allow
for more efficient preprocessed output, since that code can be part of
cpplib without destroying encapsulation.  We could also simplify some
nasty specs in gcc.c, I imagine.

But how does 3) work?  It's easy to insert in a location, outside
cpp_get_token(), at the level of what is currently c_lex() but would
probably become part of cpplib, something like

   token = cpp_get_token (pfile);
   if (pfile->preprocessed_output_p)
	output_token (pfile, token);

to simultaneously generate preprocessed output, and have hooks to do
the line markers like we do now in cppmain.c.  This is what I'm
leaning towards.  1) would be a separate function that is similar to
the current scan_translation_unit() in cppmain.c, just getting tokens
and outputting them.

However, if there's a bug in the compiler during 2), say a segfault,
then the preprocessed output with -save-temps would only proceed as
far as the compiler bug.  We have asked users to submit the
preprocessed output obtained with -save-temps.

What's the best way forwards for this case?

Neil.

Re: Best way to remove cpp0

[Daniel Jacobowitz]

On Fri, Oct 12, 2001 at 10:35:09PM +0100, Neil Booth wrote:
> However, if there's a bug in the compiler during 2), say a segfault,
> then the preprocessed output with -save-temps would only proceed as
> far as the compiler bug.  We have asked users to submit the
> preprocessed output obtained with -save-temps.
> 
> What's the best way forwards for this case?

Is there a real problem with this?  It would not be the complete
preprocessed output, but it should be enough to trigger the bug.
I'd actually find this a little more intuitive.

-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer

Re: Best way to remove cpp0

[Neil Booth]

Daniel Jacobowitz wrote:-

> Is there a real problem with this?  It would not be the complete
> preprocessed output, but it should be enough to trigger the bug.
> I'd actually find this a little more intuitive.

True, I'd thought of that, but, um....  We'd need to be able to
complete any in-progress output, at the very least.  I'd need to be
convinced it's robust in general.

Neil.

Re: Best way to remove cpp0

[Joseph S. Myers]

On Fri, 12 Oct 2001, Neil Booth wrote:

> However, if there's a bug in the compiler during 2), say a segfault,
> then the preprocessed output with -save-temps would only proceed as
> far as the compiler bug.  We have asked users to submit the
> preprocessed output obtained with -save-temps.
> 
> What's the best way forwards for this case?

When an ICE occurs, the driver should run the back end again in
preprocessor mode to generate the .i file, then run it on that .i file in
compiler mode to check whether the ICE occurs with the .i file.  If it
occurs with the .i file, the compiler should state the name of the .i file
it has created that should be included in the bug report (along with
details of host/target/compilation options); if it does not (e.g., ICE in
the preprocessor), it should list all the source files that should be
sent.

-- 
Joseph S. Myers
jsm28@cam.ac.uk

Re: Best way to remove cpp0

[Stan Shebs]

Neil Booth wrote:
> 
> However, if there's a bug in the compiler during 2), say a segfault,
> then the preprocessed output with -save-temps would only proceed as
> far as the compiler bug.  We have asked users to submit the
> preprocessed output obtained with -save-temps.
> 
> What's the best way forwards for this case?

How much overhead would it be to buffer all the way to the end of
a file, and write out before proceeding to compilation proper?
When a segfault occurs, things are going to be pretty scrambled,
you'll want your tokens safe on disk.

Or you could have -save-temps read ahead by a half-million tokens
or so, then dump before proceeding into compilation proper.  You
can prevent bloat (although -save-temps is something you use when
you're desperate to debug and not caring much about size/speed),
but still most .i files will be complete and well-formed.

Stan

Re: Best way to remove cpp0

[Neil Booth]

Stan Shebs wrote:-

> How much overhead would it be to buffer all the way to the end of
> a file, and write out before proceeding to compilation proper?

Loadsaoverhead 8-) I really don't think we want to go down that route;
I think the current scheme is no worse.

> When a segfault occurs, things are going to be pretty scrambled,
> you'll want your tokens safe on disk.

Right; instead I think a possible way is something along the lines
Joseph mentions, so that if the binary returns an exit code indicating
ICE (how do we do that?), then it re-invokes with -E to just generate
preprocessed output, and then tries to compile that with
-fpreprocessed.

Though I don't really want to implement the driver bits, it sounds
quite nasty.  Maybe Joseph could chip in there :-)

Alternatively, maybe cc1 can intercept all ICEs and segfaults, and try
to restart doing preprocessing only.  This might not be realistic,
though.

Neil.

Re: Best way to remove cpp0

[Joern Rennecke]

Neil Booth wrote:
> 
> I'm considering (gradually) moving the functionality of cppmain.c into
> the main library, and then dropping cpp0.  Then cc1 or cc1plus etc.
> would contian the only preprocessor, including the stand-alone case.
> 
> The cc1 binary could then act in three states:
> 
> 1) Stand-alone preprocessor (invoked with "cpp")
> 2) Compiler-integrated preprocessor ("gcc", "g++" etc.)

The Gnu coding standards say you shouldn't make the program behaviour
dependent on the name the program is invoked with.

> 3) Both (something like "gcc -save-temps")

This is just asking for Heisenbugs.  You want to be able to feed the .i
file into cc1 with the same result as when you invoke the driver with
--save-temps, so you better actually feed the .i file into cc1 the first
time too.
Whether that .i file is generated by a binary called cpp0 or cc1 with some
fancy options is not material for this issue, of course.

On the other hand, you are going to use a binary that has extra functionality
that is seldom used (stand-alone preprocessing) for another task that
happens very often.  We currently don't have a sufficiently advanced
profile based feedback infrastructure to avoid increasing the working
set size by interleaving used code with unused code from the extra
functionality.

> This would remove a few issues with the existing codebase, and allow
> for more efficient preprocessed output, since that code can be part of
> cpplib without destroying encapsulation.  We could also simplify some
> nasty specs in gcc.c, I imagine.

I can't follow you here.  Why is it posible to make a cc1-cum-cpp binary
with more efficient preprocessed output more easily than a single-purpose
cpp binary?

Re: Best way to remove cpp0

[Neil Booth]

Joern Rennecke wrote:-

> The Gnu coding standards say you shouldn't make the program behaviour
> dependent on the name the program is invoked with.

I don't believe that was my proposal - gcc and g++ have always done
different things.

> This is just asking for Heisenbugs.  You want to be able to feed the .i
> file into cc1 with the same result as when you invoke the driver with
> --save-temps, so you better actually feed the .i file into cc1 the first
> time too.

Right; but that is precisely Joseph's suggestion, no?

> On the other hand, you are going to use a binary that has extra functionality
> that is seldom used (stand-alone preprocessing) for another task that
> happens very often.  We currently don't have a sufficiently advanced
> profile based feedback infrastructure to avoid increasing the working
> set size by interleaving used code with unused code from the extra
> functionality.

It's very small.  I wouldn't call the amount of code in cppmain.c
significant.

> I can't follow you here.  Why is it posible to make a cc1-cum-cpp binary
> with more efficient preprocessed output more easily than a single-purpose
> cpp binary?

Because it is in cppmain.c, which uses the library's external
interface.  If moved into the library, things that are currently done
with function calls etc. could be done by derefencing pfile, for
example.

Neil.

Re: Best way to remove cpp0

[Joern Rennecke]

> Right; but that is precisely Joseph's suggestion, no?

No, he proposed to have a cc1 that writes the .i file while it is
simultanously - maybe with X tokens delay - compiling the preprocessing
tokens to assembler (or at least syntax check it with -fsyntax-only).

> > I can't follow you here.  Why is it posible to make a cc1-cum-cpp binary
> > with more efficient preprocessed output more easily than a single-purpose
> > cpp binary?
> 
> Because it is in cppmain.c, which uses the library's external
> interface.  If moved into the library, things that are currently done
> with function calls etc. could be done by derefencing pfile, for
> example.

Well, you could move the functionality into cpplib and have a separate cpp0
executable which just invokes that functionality.