Re: Ada, treelang needs to be converted to --enable-mapped-location

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Richard Kenner]

    Do you have the same complaint, then, about hashtab.c?  What about
    xmalloc?

I miss your point.  There is no requirement that a GCC front end use those.

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Per Bothner]

Robert Dewar wrote:

> Note that even the initial creation of source locations in the GNAT
> front end does not meet the requirement of creating locations in
> increasing order, because of scan backup.

This message discusses that issue.  Adding a small cache could
make source_location work ok with increasing order.

http://gcc.gnu.org/ml/gcc/2004-10/msg00248.html

I prefer the solution (1a) from that message.  It "generalizes"
source_location to work with "random" allocation, with a small
amount of code in libcpp, a smaller amount of code in gcc/ada,
and no need for a callback.  However, it is admittedly more code
than the callback solution.
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Florian Weimer]

* Daniel Jacobowitz:

> On Thu, Oct 07, 2004 at 10:33:05AM +0200, Florian Weimer wrote:
>> * Steven Bosscher:
>> 
>> >> If you can do this in a way that does not make the non-Ada case go
>> >> measuably slower, that seems OK.  If the indirection required is costly,
>> >> then your proposal is lessa ttractive.
>> >
>> > I humbly disagree, it is not OK.
>> >
>> > With this idea, we lock ourselves in with the Ada front end, so if
>> > we at one point, for whatever reason, decide that source_location
>> > should be something other than a "32-bit opaque data type", we first
>> > have to go through all this again to get the Ada representation
>> > changed as well.
>> 
>> How can the back end develop *any* dependency on the representation of
>> the data type?  I'd assume that in order to generate source location
>> information from scratch, you need at least a rough understanding of
>> the source code in question.  Such knowledge probably doesn't belong
>> to the back end.
>
> We've got all these other little "front ends", like "C++" and "C"...

Huh?  I don't think that with C or C++, the back end suddenly needs to
generate an error message five lines away from a given source
location.  And only such operations could break the abstraction
boundary.

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Richard Kenner]

    Usually, we do try to operate by some kind of consensus, so I'm not 
    trying to overrule other maintainers here.  If other people still object 
    to your plan, then we'll have to work that out.

Certainly.

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Mark Mitchell]

Richard Kenner wrote:

>    If you can do this in a way that does not make the non-Ada case go 
>    measuably slower, that seems OK.  If the indirection required is costly, 
>    then your proposal is lessa ttractive.
>
>It seems highly unlikely that changing a direct call to expand_location into
>an indirect call via a lang hook would be a measurable difference given the
>relative rarity of those calls.
>  
>
I agree. 

So, unless someone were to measure the compiler performance and prove 
that this was a real problem, I'd be OK with your plan.  The fallback 
would be to do the Gigi transformation that you described earlier, even 
though you'd be making essentially an isomporphic transformation.

Usually, we do try to operate by some kind of consensus, so I'm not 
trying to overrule other maintainers here.  If other people still object 
to your plan, then we'll have to work that out.

-- 
Mark Mitchell
CodeSourcery, LLC
(916) 791-8304
mark@codesourcery.com

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Daniel Jacobowitz]

On Thu, Oct 07, 2004 at 10:33:05AM +0200, Florian Weimer wrote:
> * Steven Bosscher:
> 
> >> If you can do this in a way that does not make the non-Ada case go
> >> measuably slower, that seems OK.  If the indirection required is costly,
> >> then your proposal is lessa ttractive.
> >
> > I humbly disagree, it is not OK.
> >
> > With this idea, we lock ourselves in with the Ada front end, so if
> > we at one point, for whatever reason, decide that source_location
> > should be something other than a "32-bit opaque data type", we first
> > have to go through all this again to get the Ada representation
> > changed as well.
> 
> How can the back end develop *any* dependency on the representation of
> the data type?  I'd assume that in order to generate source location
> information from scratch, you need at least a rough understanding of
> the source code in question.  Such knowledge probably doesn't belong
> to the back end.

We've got all these other little "front ends", like "C++" and "C"...

-- 
Daniel Jacobowitz

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Richard Kenner]

    It requires *lots* of changes all over the place in all
    other front ends.

Certainly not.  The default value for the langhook would be expand_location,
so the other front ends would work exactly as they do now.

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Richard Kenner]

    If you can do this in a way that does not make the non-Ada case go 
    measuably slower, that seems OK.  If the indirection required is costly, 
    then your proposal is lessa ttractive.

It seems highly unlikely that changing a direct call to expand_location into
an indirect call via a lang hook would be a measurable difference given the
relative rarity of those calls.

    I think that if Ada wants to have its own source location machinery
    and its own diagnostic machinery, that's your business, so long as it
    doesn't impact the rest of the compiler adversely.  In the long run,
    having code that duplicates stuff that is common to the other front
    ends may lead to other problems for Ada, but I think that's up to the
    Ada maintainership to evaluate.

Right.  In my mind, the tradeoffs relate to the specialization of the code vs
the amount of code being shared and the issues are somewhat different between
diagnostic and source location machinery.  For the latter, there's simply not
enough code sharing going on to justify complicating the large existing
source file mechanisms in the Ada front end.

For diagnostic output, the major difference is in the other direction:
it is worth adding Ada-specific functionality to the existing
diagnostic code in order to enable it to be shared?  That's more a question
for the maintainers of the diagnostic machinery, though there is still the
issue of using code outside of the Ada tree in those cases when GCC is not
the backend since the current diagnostic code includes significantly more GCC
code than the source location machinery, as I understand it.

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Florian Weimer]

* Steven Bosscher:

>> If you can do this in a way that does not make the non-Ada case go
>> measuably slower, that seems OK.  If the indirection required is costly,
>> then your proposal is lessa ttractive.
>
> I humbly disagree, it is not OK.
>
> With this idea, we lock ourselves in with the Ada front end, so if
> we at one point, for whatever reason, decide that source_location
> should be something other than a "32-bit opaque data type", we first
> have to go through all this again to get the Ada representation
> changed as well.

How can the back end develop *any* dependency on the representation of
the data type?  I'd assume that in order to generate source location
information from scratch, you need at least a rough understanding of
the source code in question.  Such knowledge probably doesn't belong
to the back end.

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Laurent GUERBY]

Has anyone looked at wether the -gnatD switch (where
all sloc will be pointing to virtual files generated
by the front-end after scanning and not real source files), 
is relevant to this design discussion?

Laurent

On Thu, 2004-10-07 at 07:15, Robert Dewar wrote:
> Richard Kenner wrote:
> 
> > (2) If we do it in Gigi, the issue is that the line-map code expects to see
> > locations created in increasing order, but there's no guarantee that the
> > nodes will be processed in that order.
> 
> Note that even the initial creation of source locations in the GNAT
> front end does not meet the requirement of creating locations in
> increasing order, because of scan backup.
> 
> 
> > My proposal is basically to take the view that source_location is an unsigned
> > int that's an opaque datatype used to encode a (file, line column) triple.
> > This datatype can either be defined in the front end (in which case it
> > provides a language hook to produce that triple from the encoding) or it can
> > use routines provided in line-map.c to generate the encoding.
> 
> That really seems easier to me, given especially that the concept
> of locations is language dependent wrt things like generic instantiation.

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Robert Dewar]

I did a bit more rummaging in the GNAT front end to see
how source locations are used, and as I rummage, more comes
to light (and to memory).

Basically an Sloc value in GNAT encodes a triple
(file-with-instantiation-history, line, column)

where line is unlimited and column is 0-32767, though
that limit on columns could trivially be increased if
necessary.

There is a strong assumption (really subsuming my
assumptions 1 and 2 in the previous note), that a
source location can be used as a direct index into
the source buffer for a given file, e.g.

    Source_Buffer (Sloc)

where Source_Buffer is a pointer to a source buffer
appropriately biased, allows direct access to the
character in the source buffer corresponding to Sloc,

This assumption is made throughout the front end, but only
in a relatively few places outside the lexical scanner. The
lexical scanner however is fundamentally dependent on this
assumption, since for example Token_Ptr, a pointer to the
current token *is* a Sloc value and that identity is assumed
througout the scanner.

Trying to abstract this out would require a lot of
source changes, and run the risk of introducing substantial
overhead into the lexical scanner, where speed is really
important.

My basic conclusion here is that trying to make any significant
change to the Sloc data structure in GNAT would be really major
work, and require substantial rewriting of the scanner.

Of course, it is still possible to do a conversion from one
form to the other in gigi, not trivial, but certainly possible.
None of my observations here speak to that possibility.

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Mark Mitchell]

Steven Bosscher wrote:

>On Thursday 07 October 2004 04:31, Mark Mitchell wrote:
>  
>
>>Richard Kenner wrote:
>>    
>>
>>>My proposal is basically to take the view that source_location is an
>>>unsigned int that's an opaque datatype used to encode a (file, line
>>>column) triple. This datatype can either be defined in the front end (in
>>>which case it provides a language hook to produce that triple from the
>>>encoding) or it can use routines provided in line-map.c to generate the
>>>encoding.
>>>      
>>>
>>If you can do this in a way that does not make the non-Ada case go
>>measuably slower, that seems OK.  If the indirection required is costly,
>>then your proposal is lessa ttractive.
>>    
>>
>
>I humbly disagree, it is not OK.
>
>With this idea, we lock ourselves in with the Ada front end, so if
>we at one point, for whatever reason, decide that source_location
>should be something other than a "32-bit opaque data type", we first
>have to go through all this again to get the Ada representation
>changed as well.
>  
>
The reason that I disagree with your position is that I would be willing 
to make that change, even if that means that Ada were to be 
(temporarily) broken.

If we decide that we have a compelling reason to use a representation 
other than a 32-bit integer, then I would be willing to (at that point) 
simply make the change.  The Ada folks would then eventually make the 
necessary changes.  On the one hand, I don't think we should try to 
force Ada to change *now*; on the other, I don't think we should avoid 
making compelling improvements to avoid making changes in Ada.

-- 
Mark Mitchell
CodeSourcery, LLC
(916) 791-8304
mark@codesourcery.com

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Steven Bosscher]

On Thursday 07 October 2004 04:31, Mark Mitchell wrote:
> Richard Kenner wrote:
> >My proposal is basically to take the view that source_location is an
> > unsigned int that's an opaque datatype used to encode a (file, line
> > column) triple. This datatype can either be defined in the front end (in
> > which case it provides a language hook to produce that triple from the
> > encoding) or it can use routines provided in line-map.c to generate the
> > encoding.
>
> If you can do this in a way that does not make the non-Ada case go
> measuably slower, that seems OK.  If the indirection required is costly,
> then your proposal is lessa ttractive.

I humbly disagree, it is not OK.

With this idea, we lock ourselves in with the Ada front end, so if
we at one point, for whatever reason, decide that source_location
should be something other than a "32-bit opaque data type", we first
have to go through all this again to get the Ada representation
changed as well.

IMHO the proper solution is to do the translation in gigi.

Gr.
Steven

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Steven Bosscher]

On Thursday 07 October 2004 03:45, Richard Kenner wrote:
>     It can certainly be made to work.  I am not in favor of this change,
>     though, as it requires adding call-back hooks, which are generally to
>     be avoided, and which in this case are unnecessary, even if they make
>     it easier in the short term.
>
> Hook, singular.  It means that there's a language-dependent version of
> expand_location, that's all.  The advantage is that it requires *no other
> change* anywhere except to make Sloc_to_locus a nop in the Ada front end.

Hah.
Think outside the box for once please.

It requires *lots* of changes all over the place in all
other front ends.

Gr.
Steven

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Robert Dewar]

Robert Dewar wrote:

>> My proposal is basically to take the view that source_location is an 
>> unsigned
>> int that's an opaque datatype used to encode a (file, line column) 
>> triple.
>> This datatype can either be defined in the front end (in which case it
>> provides a language hook to produce that triple from the encoding) or 
>> it can
>> use routines provided in line-map.c to generate the encoding.

I did a bit more rummaging in the Ada front end. It is certainly not the
case that the Ada front end regards this type as opaque. The following
assumptions are fairly strongly embedded.

1. The values representing the column positions of a single line are
consecutive integer values, so it is fine to do arithmetic, e.g. to
compute the right column location for an error message that is to be
issued in the middle of a token, e.g. a character in a string literal
that is not in the subtype:

      1. package q is
      2.    type r is ('a', 'b', 'c');
      3.    type m is array (integer range <>) of r;
      4.    s : m:= "abababdcbcbcb";
                           |
         >>> character not defined for type "r" defined at line 2

      5. end q;

2. If two values represent source locations in the same source file, then
they are strictly ordered, so that a comparison of source locations
indicates ordering in the file.

3. The Sloc values in GNAT are signed. THe negative values are used to
indicate special locations (No_Location, Standard_Location,
Standard_ASCII_Location, System_Location), and the code uses
range tests to test for special locations.

There may be other assumptions, but these are the ones that are for
sure present.

It would be possible to eliminate all such non-opaque
assumptions, and indeed it might be a good idea to do so, but it
is not a trivial job.

Of course the back end could regard it as opaque, but that's only
part of the story. The above issues would definitely have to be
addressed if there was a consideration of changing the representation
in GNAT. The fairly elaborate representation of source locations
was part of the original design of GNAT 14 years ago since error
message placement was one of the original areas we concentrated
on, and I fear it has at this stage assumptions about the basic
form of these values, which has not changed significantly in
this time period, are fairly strongly embedded.

At the least I will investigate this further, and

   a) document these assumptions that are being made

   b) at least gradually try to eliminate them

Making Source_Ptr a private type of package Sinput would
be desirable, but that is for sure quite a bit of work!

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Robert Dewar]

Richard Kenner wrote:

> (2) If we do it in Gigi, the issue is that the line-map code expects to see
> locations created in increasing order, but there's no guarantee that the
> nodes will be processed in that order.

Note that even the initial creation of source locations in the GNAT
front end does not meet the requirement of creating locations in
increasing order, because of scan backup.


> My proposal is basically to take the view that source_location is an unsigned
> int that's an opaque datatype used to encode a (file, line column) triple.
> This datatype can either be defined in the front end (in which case it
> provides a language hook to produce that triple from the encoding) or it can
> use routines provided in line-map.c to generate the encoding.

That really seems easier to me, given especially that the concept
of locations is language dependent wrt things like generic instantiation.

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Mark Mitchell]

Richard Kenner wrote:

>My proposal is basically to take the view that source_location is an unsigned
>int that's an opaque datatype used to encode a (file, line column) triple.
>This datatype can either be defined in the front end (in which case it
>provides a language hook to produce that triple from the encoding) or it can
>use routines provided in line-map.c to generate the encoding.
>  
>
If you can do this in a way that does not make the non-Ada case go 
measuably slower, that seems OK.  If the indirection required is costly, 
then your proposal is lessa ttractive.

I think that if Ada wants to have its own source location machinery and 
its own diagnostic machinery, that's your business, so long as it 
doesn't impact the rest of the compiler adversely.  In the long run, 
having code that duplicates stuff that is common to the other front ends 
may lead to other problems for Ada, but I think that's up to the Ada 
maintainership to evaluate.

-- 
Mark Mitchell
CodeSourcery, LLC
(916) 791-8304
mark@codesourcery.com

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Richard Kenner]

    The problem here is that we cannot make a decision for Ada independently 
    of the rest of GCC because the data structures (*_DECL tree nodes) in 
    question are shared among all GCC  front ends.  There is a clear benefit 
    to the other front ends from using this new approach: the compiler 
    itself goes faster with no loss in functionality.

But that's not the issue here. We aren't talking about the tree nodes, but
about the meaning of what a "source location" is: as far as the tree nodes
are concerned it's an opaque data type.

    If that means translating from the current Ada representation to the
    GCC representation at the last minute, when the Ada front end
    translates all its other data structures, that's fine.  I don't
    understand how that can be impossible.  It might be difficult for the
    Ada maintainers, but that may be the price paid for being independent
    of the core GCC data structures, and thereby providing the ability to
    reuse the Ada front end with other back ends.

The issues are the following:

(1) Source location handling is done very early in the Ada front end and with
a lot of Ada-specific processing.  Using line-map at this point creates two
problems:

(1a) Linking to that file in a context where the rest of GCC isn't used and
it's the only file from that tree and also the only C file creates a
build/packaging issue.

(1b) Because there's so much additional file and line handling, superimposing
that on top of the existing line-map routines adds significant complexity to
the existing code and more than wipes out the benefits of sharing that small
piece of code.

(2) If we do it in Gigi, the issue is that the line-map code expects to see
locations created in increasing order, but there's no guarantee that the
nodes will be processed in that order.  What we can do is add a pre-pass that
walks the the tree and converts each Ada Sloc into a (file, line, column)
triple and then sorts all the triples.  We can then call the line-map code
for each such triple and make a hash table to be used later to map an Ada
Sloc to a GCC source_location.

However, what the above code does is make a mapping from one 32-bit opaque
datatype that encodes a (file, line, column) triple to a different 32-bit
opaque datatype that encodes *exactly the same thing* in a different way.
This seems pointless.


My proposal is basically to take the view that source_location is an unsigned
int that's an opaque datatype used to encode a (file, line column) triple.
This datatype can either be defined in the front end (in which case it
provides a language hook to produce that triple from the encoding) or it can
use routines provided in line-map.c to generate the encoding.

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Mark Mitchell]

Richard Kenner wrote:

>Linking Ada code to C is not hard, but has a cost, both in clarity and in
>losing the ability to inline the code being called.  It's worth doing that
>linkage if (and only if!) there is some benefit in code sharing obtained from
>that linkage.  
>
The problem here is that we cannot make a decision for Ada independently 
of the rest of GCC because the data structures (*_DECL tree nodes) in 
question are shared among all GCC  front ends.  There is a clear benefit 
to the other front ends from using this new approach: the compiler 
itself goes faster with no loss in functionality.

I think that it's purely up to the Ada maintainers to decide how to 
interface the Ada front end to the core GCC data structures in an 
effective way.  If that means translating from the current Ada 
representation to the GCC representation at the last minute, when the 
Ada front end translates all its other data structures, that's fine.  I 
don't understand how that can be impossible.  It might be difficult for 
the Ada maintainers, but that may be the price paid for being 
independent of the core GCC data structures, and thereby providing the 
ability to reuse the Ada front end with other back ends.

-- 
Mark Mitchell
CodeSourcery, LLC
(916) 791-8304
mark@codesourcery.com

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Richard Kenner]

    It can certainly be made to work.  I am not in favor of this change,
    though, as it requires adding call-back hooks, which are generally to
    be avoided, and which in this case are unnecessary, even if they make
    it easier in the short term.

Hook, singular.  It means that there's a language-dependent version of
expand_location, that's all.  The advantage is that it requires *no other
change* anywhere except to make Sloc_to_locus a nop in the Ada front end.

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Per Bothner]

Robert Dewar wrote:

> Can someone answer whether the gcc mapped location approach accomodates
> column numbers,

Yes.

> for both the template and instantiations. 

Not sure what you mean by this.
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Per Bothner]

Richard Kenner wrote:

>     > In another thread I proposed a very simple solution.  Can you please
>     > comment on whether it's workable or not?
> 
> No.  What I'm talking about is the proposal to use Ada's Sloc as the
> source_location and make a language hook that calls expand_location by
> default but for Ada, calls a function that's essentially Sloc_to_locus
> and expands out the Sloc into (file, line, column).  That way there's
> exactly one representation of source location in use and one file name
> table.

It can certainly be made to work.  I am not in favor of this change,
though, as it requires adding call-back hooks, which are generally to
be avoided, and which in this case are unnecessary, even if they make
it easier in the short term.

However, if you can persuade Zack or some other global write
maintainer I won't put up much of a fight.  I do think the
linemap_find_location api extension I suggested is a better
solution: it's a modest localized change for Ada, and doesn't
effect the rest of the compiler.  There is a slight performance
penalty, but I doubt it would be measurable, and it would be
partly compensate by reduced call-back indirection.

The ideal solution is to change Source_Ptr to be the same as
source_location.  We're certainly open to moidfying the latter
if needed, assuming there is no measurable performance degradation.
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Robert Dewar]

Richard Kenner wrote:
>     Are you thinking of some other file?  sinfo.ad[sb] barely mentions
>     Source_Ptr at all:  Only in two functions to get/set the End_Span,
>     neither of looks like they would need to be changed when converting
>     to source_location!
> 
> Sorry, indeed I was: sinput.ad[bs].
> 
>     The gain is that alternative solutions require mapping between two
>     encodings for source locations.  Plus better integration between Ada
>     and the rest of the compilers, which *we* think is desirable.  Plus
>     using source_location is probably faster, considering what you have to
>     go through in Sloc_to_locus (even *before* switching to
>     source_location.

Sloc's are used to convey semantic information, in particular they are
used to determine unit membership, and source position, so you need to
look at lib.adb as well.

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Robert Dewar]

Can someone answer whether the gcc mapped location approach accomodates
column numbers, for both the template and instantiations. We definitely
need this in GNAT.

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Richard Kenner]

    Are you thinking of some other file?  sinfo.ad[sb] barely mentions
    Source_Ptr at all:  Only in two functions to get/set the End_Span,
    neither of looks like they would need to be changed when converting
    to source_location!

Sorry, indeed I was: sinput.ad[bs].

    The gain is that alternative solutions require mapping between two
    encodings for source locations.  Plus better integration between Ada
    and the rest of the compilers, which *we* think is desirable.  Plus
    using source_location is probably faster, considering what you have to
    go through in Sloc_to_locus (even *before* switching to
    source_location.

But that seems the advantage of what I proposed (see below).

    > In another thread I proposed a very simple solution.  Can you please
    > comment on whether it's workable or not?

    IRC you proposed making a localized change in Sloc_to_locus.  

No.  What I'm talking about is the proposal to use Ada's Sloc as the
source_location and make a language hook that calls expand_location by
default but for Ada, calls a function that's essentially Sloc_to_locus
and expands out the Sloc into (file, line, column).  That way there's
exactly one representation of source location in use and one file name
table.

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Per Bothner]

Richard Kenner wrote:

> The point is that sinfo.ads and sinfo.adb is a huge and complex piece of
> code, far larger than linemap.c.

Are you thinking of some other file?  sinfo.ad[sb] barely mentions
Source_Ptr at all:  Only in two functions to get/set the End_Span,
neither of looks like they would need to be changed when converting
to source_location!

> It's going to be a tremendous amout
> of work to do the type of surgery you suggest

I'm not convinced.  And I'm willing to bet on it:  I'll do it
for USD 5k.  If I reluctantly come to the conclusion you're right,
then no charge.

But see below.

> and I don't see *any* gain in doing do.

The gain is that alternative solutions require mapping between two
encodings for source locations.  Plus better integration between
Ada and the rest of the compilers, which *we* think is desirable.
Plus using source_location is probably faster, considering what
you have to go through in Sloc_to_locus (even *before* switching
to source_location.

> In another thread I proposed a very simple solution.  Can you please
> comment on whether it's workable or not?

IRC you proposed making a localized change in Sloc_to_locus.  That is
certainly possible.  There are two basic approaches:

(1) simple-minded - just create source_location values as requested,
without any "planning".  If the requested location does not map
acceptably into the current line_map, just allocate a new one with
linemap_add.  This to some extent happens automatically anyway.  The
risk/downside is you may get too many line_map entries if the Source_Ptr
jumps around too much, but in practice it may not matter too much.

(1a) like (1) but with some caching.  Keep a cache of say the 5 most
recently used line_map entries.  If the requested Source_Ptr matches
one of them, use it.  If not, create a new line_map entry using
linemap_add.  The downside is that this ties you to the line-map
internals.  The way around that is to define an API and place it
in a new file in libcpp.  E.g. in libcpp/line-map-cache.[ch]:
   static (struct line_map*) cache[5];
   source_location
   linemap_get_location (struct line_table*,
     const char *filename, int line, int column);

(2) Build dome data structure to map Source_Ptr to source_location
efficiently and "compactly".  E.g. such that the mapping is monotonic.
This is probably not worth it. at least not to begin with.

I suggest starting with (1), but defining the interfaces so you can
switch easily to (1a).  Or just go for (1a) immediately.
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Richard Kenner]

    So is the objection really to its placement in libcpp, a library that
    is otherwise unused?  Maybe this is just a code organization issue.
    What if it were in libiberty instead?  Would that calm people down
    and make clear that it's not part of the GCC backend?

No, that isn't the issue.

The issue is that there's already *a lot* of code in the Ada front-end
(approximately 2,000 lines) dealing with source file issues.  That code deals
with numerous issues unique to Ada and GNAT (such as tracking casing of
identifiers and keywords, computing checksums and timestamps, and looking at
licenses).

Linking Ada code to C is not hard, but has a cost, both in clarity and in
losing the ability to inline the code being called.  It's worth doing that
linkage if (and only if!) there is some benefit in code sharing obtained from
that linkage.  The GCC backend is a perfect example of such a gain.  Hashing
code is a good example where it's a bad idea because the code is small enough
that the benefit of sharing is outweighed by losing the ability to inline.

In this case, if sinput.ad{s,b} used line-map, it's very doubtful we'd
actually save any code because of the need to map all the extra information
into the data structure used by line-map.  We'd also lose inlining.
Moreover, the complexity of the interfacing would make the code much less
clear.  And for what gain, to share 500 lines of code?

When the file/line structure isn't being used for significant amounts of
data, such as in the C and C++ front ends and the language that the code
using line-map is the same as line-map (C), the sharing does have a benefit.

I still haven't heard any reason why we simply can't use the existing Ada
Sloc as the source_location within GCC for Ada.

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Joe Buck]

On Wed, Oct 06, 2004 at 01:39:02PM -0700, Per Bothner wrote:
> As Zack and have I've patiently tried to explain: line-map.[ch] is not
> part of or dependent on GCC or its back-end.
> ...
> Thus it is not a call-back.  Using line-map.[ch] is like using say a
> regular expression package.  It's a utility package that happens to be
> written in C and reside in libcpp; neither of which should prevent it
> from being used from Ada code.

So is the objection really to its placement in libcpp, a library that
is otherwise unused?  Maybe this is just a code organization issue.
What if it were in libiberty instead?  Would that calm people down
and make clear that it's not part of the GCC backend?

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Richard Kenner]

    So?  gcc/ada is part of the GCC tree.  How is line-map.[ch] different?

The point is that when you aren't using GCC as a backend all you need is
that single directory and you don't have to go into outside of that
directory.  Adding the complexity of that reference to share a few
hundred lines of code does not seem worthwhile, especially since ...

    *Just* the implementation of the Source_Ptr type.

The point is that sinfo.ads and sinfo.adb is a huge and complex piece of
code, far larger than linemap.c.  It's going to be a tremendous amout
of work to do the type of surgery you suggest and I don't see *any*
gain in doing do.

In another thread I proposed a very simple solution.  Can you please
comment on whether it's workable or not?

    > The best way of handling such consistency is defining a specification for
    > the error messages and having each language be consistent with that
    > specification.

    Even better is for each language to use the same library.

Not at all.  The terminology and style issues are totally different
between languages.  Giving high-quality diagnostic error messages is a
very difficult task.  Sometimes you want to reference a language
standard, for example.  The way you refer to language items (such as
the case you use to mention keywords) can depend on subtle things in
the program in some languages.  Creating one common routine for
diagnostics will mean teaching that routine the details of *all*
languages and unnecessarily complicates that routine.

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Per Bothner]

Richard Kenner wrote:

>     As Zack and have I've patiently tried to explain: line-map.[ch] is not
>     part of or dependent on GCC or its back-end.
> 
> No, but it's part of the GCC tree.  Which means that it's inappropriate to
> include it in files that have nothing to do with the GCC tree (e.g., GNAT
> targeted for the AAMP).

So?  gcc/ada is part of the GCC tree.  How is line-map.[ch] different?

> To give you an idea of the complexity, this is the data structure that
> the Ada front end uses to store information about each file.
 > ...
 >   type Source_File_Record is record
 > ...

I'm not suggesting changing any of this.
*Just* the implementation of the Source_Ptr type.

Yes, there is a slight duplication in that (I assume) you need to map
from a Source_Ptr to a Source_File_Record.  I've thought about adding
an extra "client" hook into each line_map entry.  Ada could use this to
inexpensively map a source_location to a Source_Ptr.  It would also be
useful for the compile-server, which could use it to map source_location
to "fragments".

>     Your approach is to keep the Ada front-end as self-contained and "pure
>     Ada" as possible.  That may be in conflict with what to us are more
>     important goals: consistency between languages in things like option
>     handling, and diagnostic handling.  These things are important to
>     higher-level tools, such as IDEs.
> 
> The best way of handling such consistency is defining a specification for
> the error messages and having each language be consistent with that
> specification.

Even better is for each language to use the same library.

But that's not an issue we need to deal with now.
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Richard Kenner]

    As Zack and have I've patiently tried to explain: line-map.[ch] is not
    part of or dependent on GCC or its back-end.

No, but it's part of the GCC tree.  Which means that it's inappropriate to
include it in files that have nothing to do with the GCC tree (e.g., GNAT
targeted for the AAMP).

    Yet it was fairly straightforward to modify the Fortran front-end to
    use the *standard* line-map.[ch] data structure.  Java was more
    complicated, mainly because I used the opportunity to streamline the
    lexer.  Why is it such a problem for Ada?

Because there is a lot of code in the Ada front end dealing with
source files and line numbers and rewriting it all to deal with
different data structures is a tremendous amount of work.  All of
which is competely unnecessary.

To give you an idea of the complexity, this is the data structure that
the Ada front end uses to store information about each file.  I've deleted
the comments.  See sinput.ads for more details.

   type Source_File_Record is record

      File_Name         : File_Name_Type;
      File_Type         : Type_Of_File;
      Reference_Name    : File_Name_Type;
      Debug_Source_Name : File_Name_Type;
      Full_Debug_Name   : File_Name_Type;
      Full_File_Name    : File_Name_Type;
      Full_Ref_Name     : File_Name_Type;
      Inlined_Body      : Boolean;
      License           : License_Type;
      Num_SRef_Pragmas  : Nat;
      First_Mapped_Line : Logical_Line_Number;
      Source_Text       : Source_Buffer_Ptr;
      Source_First      : Source_Ptr;
      Source_Last       : Source_Ptr;
      Time_Stamp        : Time_Stamp_Type;
      Source_Checksum   : Word;
      Last_Source_Line  : Physical_Line_Number;
      Keyword_Casing    : Casing_Type;
      Identifier_Casing : Casing_Type;
      Instantiation     : Source_Ptr;
      Template          : Source_File_Index;
      Sloc_Adjust : Source_Ptr;
      Lines_Table : Lines_Table_Ptr;
      Logical_Lines_Table : Logical_Lines_Table_Ptr;
      Lines_Table_Max : Physical_Line_Number;
   end record;

There is no point in duplicating all of this code.

    I'm guessing Ada uses its own routines for diagnostics 

Yes.

    this may need to change at some point.  

Highly unlikely given the specialization of that code.

    Your approach is to keep the Ada front-end as self-contained and "pure
    Ada" as possible.  That may be in conflict with what to us are more
    important goals: consistency between languages in things like option
    handling, and diagnostic handling.  These things are important to
    higher-level tools, such as IDEs.

The best way of handling such consistency is defining a specification for
the error messages and having each language be consistent with that
specification.

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Laurent GUERBY]

On Wed, 2004-10-06 at 19:38, Zack Weinberg wrote:
> This is news to me; I had thought that hashtab.c was required.  I'm
> sure I don't have to explain to you why duplicated code is bad; and I
> don't see why Ada's packaging requirements prevent you from picking up
> C modules which have, after all, been carefully coded to be
> independent of the rest of GCC.  Such as hashtab.c and line-map.c.

A general point, C is an unsafe language (at least for most
implementations of C out there), carefully coded C
will have X % less buffer overflows or similar errors than "not
carefully coded" C but that still leaves some of them with all their
potential nastiness see the stream of security community news on the
topic, and unfortunately major, old, stable, ironed out C libraries are
not immune to this problem. Recoding in a safe language
C libraries is not without value in some contexts but of course
you can introduce new bugs as always :).

In the context of GNAT, most Ada checks are suppressed in the
default build configuration and we link with code using libiberty so
this does not directly applies, but since the Ada front-end has a
potential to be (re)used without GCC and in an environment where
libiberty won't be available like managed virtual machines 
(there's some work to do).

More to the point, in a multi language environment you loose inlining
which can be big performance-wise for class of code. If the source is C
you probably have to recode anyway all the C access macros which are
available only for the C family of languages. And of course on the other
side you gain some flexibility to tailor the data structure to your
specific usage pattern.

All in all, I think the situation is not really unconditionnally black
and white.

Laurent

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Per Bothner]

Richard Kenner wrote:

>     My suggestion that one option is to create thr Source_Ptr ->
>     source_location mapping (i.e. data structures to make the mapping
>     reasonably efficient) be done more-or-less at the same time as the
>     source file is read into a Source_Buffer and Source_Ptr mapping is
>     updated.
> 
> The problem is that that code is not dependent on the back-end being GCC.

As Zack and have I've patiently tried to explain: line-map.[ch] is not
part of or dependent on GCC or its back-end.

> So it's the wrong place to put callbacks into routines that are in the
> GCC hierarchy.

Thus it is not a call-back.  Using line-map.[ch] is like using say a
regular expression package.  It's a utility package that happens to be
written in C and reside in libcpp; neither of which should prevent it
from being used from Ada code.

> The right place for any such code is in ada/trans.c:Sloc_to_locus.  Indeed
> I believe that might be the *only* routine that needs to be modified.
> 
> Can you take a close look at that function and suggest needed changes to it?

No.  As I've explained, I don't think that is the right way to go about
it.  If you think it is the right way, then you should be able to do it.

>     Yes, but that doesn't preclude using the line_map structure, even with
>     non- GCC back-ends.  There is no reason why the Ada front-end can't use
>     line_map for a program that say just does analysis and never touches a
>     tree or an rtx.
> 
> Yes, there is because the Ada front end is also used with back-ends other
> than GCC.

Sigh.  I don't know how to explain again why that is irrelevant.

Geert Bosch wrote:

 > As I understand, Fortran also has a private line map already and may
 > not require the overhead of maintaining a duplicate data structure.

Yet it was fairly straightforward to modify the Fortran front-end to
use the *standard* line-map.[ch] data structure.  Java was more
complicated, mainly because I used the opportunity to streamline the
lexer.  Why is it such a problem for Ada?

 > As I see it, the interface between front end and back end now
 > only consists of the few simple accessor functions/macros,
 > instead of the much, much more complicated line-map.h interface.
 > If you look at all dependencies between front end and back end
 > as a graph, the goal is to be able to partition this graph with
 > cutting as few edges as possible.

Exactly.  If the Ada front-end uses line-map.[ch] it is a simple
case of the front-end calling a utility library, albeit one written
in C rather than Ada.  There are no call-backs.  On the other hand
if we make source_location depend on the front-end, then then back-end
will need call-backs into the front-end to so the latter can "take
apart" a source_location for the former.

Your own concern favors Zack's and my position.

 > During the entire parsing and analysis
 > of the Ada sources, we never call into any of the C part of GCC.
 > The only code executed is the Ada code in the gcc/ada subdirectory.

I assume you end up calling C libraries for I/O.

 > This is ideal from a maintenance point of view, since the separation
 > between front end and back end matters is localized in just a few
 > files comprising Gigi.

Error reporting is another example of a facility where the front-end
*should* be calling into shared code, for consistency of error handling
between front-end and back-end errors.  I'm guessing Ada uses its own
routines for diagnostics (as to some extent does Java); this may
need to change at some point.  At which point Ada using its own line
number representation may be a disadvantage.

Your approach is to keep the Ada front-end as self-contained and "pure
Ada" as possible.  That may be in conflict with what to us are more
important goals:  consistency between languages in things like option
handling, and diagnostic handling.  These things are important to
higher-level tools, such as IDEs.

 > As GCC will implement more languages over time, such a separation is
 > even more necessary. Just think of a GCC with 20 different front ends.
 > If every change to a data structure, such as the line map, would
 > require changes throughout these front ends, maintenance would be
 > a nightmare.

Well, the line map data structure is fairly opaque.  There are basically
5 functions and 1 macro.  (A few more functions and macros are only
used by cpp.)

 > How the front end generates its trees should be of no concern.
 > My objection against using the line-map.[ch] "library" is that
 > its "specification" makes so many assumptions on how its callers
 > are structured, that it ends up specifying how to implement
 > a front end instead of just providing a way of to build a mapping
 > from locus to file, line and column.

Well, the main asseumption is that somebody reads a source file in
forwards order to lex/parse it; while doing so that somebody once
in a while makes a request to "this is the current file/line/column -
give me a cookie so I can remember that."

The library has been designed for the case that cookie requests
are done in forwards order.  This is obviously straight-forward to
do if cookie requests are done using lexing.  Giving that lexing
has to be done in forwards order, this is not much of a constraint.

Note that Source_Ptr cookies are managed not too differently, except
they are "allocated" when a file is read into a Source_Buffer rather
then when it is lexed.  No big difference.  The advantage of the
Source_Ptr approach is a little less overhead dusing lexing; the
disadvantage is that you must keep the text of all the source files
in memory to decode the Source_Ptr, plus doing so appears to require
more work.  On balance, I think the source_location approach is
preferable.

I mentioned that it may be possible to assign allocate cookies
differently, before lexing, or after lexing.  You do have those
options, though I don't recommend them.

 > I see three solutions, listing in order of preference:
 >   -  Allow front ends to specify the mapping from "Locus" to
 >      file/line/col. The locus is just an abstract type encoded
 >      in a 32-bit value, and front ends provide hoods to get
 >      file/line/column information. This would fit well with
 >      Ada and Fortran front ends, and completely eliminates
 >      the need for any extra overhead of the line-map data
 >      structure.

Fortran is fine with source_location.

I'd have more sympathy with your suggestion if either:
(a) there was something special about Ada or its front-end
needing special hook, or
(b) the Source_Ptr encoding is more efficient or flexible (at
least for Ada) than the one in line-map.[ch].
Neither appears to be the case.

 >   -  Make the line-map interface a bit more flexible, so that
 >      mappings can be created in arbitrary order without large
 >      performance penalties.

We'd need a concrete design.  It would have to not measurably hurt
performance for other front-ends.

 >   -  Add an extra pass to Gigi, which sorts all loci by original
 >      source order and then uses the somewhat clumsy interface of
 >      line-map to generate the required data.

My recommended solution:

- Reimplement the Source_Ptr type to be the same as line-map.[ch]'s
source_location.  Have Ada's lexical scanner get source_location
cookies from line-map.[ch].  Whenever it currently remembers a
Source_Ptr, it would instead remember a source_location.

This should not be a lot of work.  It is changing the implementation
of an opaque data type to use an existing library.  This does *not*
prevent or even complicate use of non-GCC backends, except that they
need to link in with libcpp.

If you think I'm mistaken about the difficulty of this, I'll willing
to do the work for a modest fixed fee, with no cost to GNAT if I fail.
(Not that I'm hungry for this business - I have more intersting work
I'm behind on.  I'm just tired of arguing about this, and would
prefer to avoid what I think are inferior over-complicated solutions.)
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Zack Weinberg]

kenner@vlsi1.ultra.nyu.edu (Richard Kenner) writes:

> The point is that there's no reason to have to go outside the
> front-end until the transition point from Slocs to whatever the
> middle-end uses and Sloc_to_locus seems to me to be the right place
> to do that.

I don't have a problem with your doing just that, nor with modifying
line-map.[ch] if necessary to make that efficient.  And if that's the
quickest way to get this change implemented, then I'm definitely in
favor of it, because it would be nice to complete the switchover
before 4.0 branches.

Everything else I've said is aimed at larger and longer-term design
questions.

zw

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Zack Weinberg]

Geert Bosch <bosch@gnat.com> writes:

> On Oct 6, 2004, at 11:52, Zack Weinberg wrote:
>> Do you have the same complaint, then, about hashtab.c?  What about
>> xmalloc?
>
> No, because the front ends do not need to use these files.  Indeed,
> GNAT uses its own hash tables and allocation strategies. It would be
> different if you'd require front ends to store any identifiers it
> reads using hashtab.c.

This is news to me; I had thought that hashtab.c was required.  I'm
sure I don't have to explain to you why duplicated code is bad; and I
don't see why Ada's packaging requirements prevent you from picking up
C modules which have, after all, been carefully coded to be
independent of the rest of GCC.  Such as hashtab.c and line-map.c.

I'd also point out that, while proper interface boundaries are good,
sharing data structures is also good.  Any time you have two data
structures which are isomorphic representations of the same thing, and
logic to translate between them, that's wasting time and space which
could be saved by sharing the data structure.

zw

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Richard Kenner]

    line-map.[ch] should not be considered part of GCC in this context.
    The proper analogy is to code in libiberty, which is shared among many
    projects.  There is no reason the Ada front end cannot make use of it
    when linked to some other back-end (modulo source packaging issues).

But it doesn't use libiberty either!  Although Ada supports
interfacing with other languages relatively easily, it isn't free and
doing that does indeed complicate packaging issues for the non-GCC
backends (which are written purely in Ada).  The point is that there's
no reason to have to go outside the front-end until the transition
point from Slocs to whatever the middle-end uses and Sloc_to_locus
seems to me to be the right place to do that.

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Geert Bosch]

On Oct 6, 2004, at 11:52, Zack Weinberg wrote:
> Do you have the same complaint, then, about hashtab.c?  What about
> xmalloc?

No, because the front ends do not need to use these files.
Indeed, GNAT uses its own hash tables and allocation
strategies. It would be different if you'd require
front ends to store any identifiers it reads using
hashtab.c.

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Zack Weinberg]

Geert Bosch <bosch@gnat.com> writes:

> On Oct 6, 2004, at 11:21, Zack Weinberg wrote:
>> I don't care about the details of how Ada interfaces with the
>> language-independent compiler.  My comments were intended solely to
>> address the bogus argument that line-map.[ch] was part of GCC rather
>> than a generic utility library.
>
> With the change I discussed with Per Bothner, line-map will indeed
> be a generic utility library, which may be used to implement the
> locus type required by the back end. However, if any GCC front end
> is *required* to use the library, I think it is fair to say it is
> part of GCC.

Do you have the same complaint, then, about hashtab.c?  What about
xmalloc?

zw

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Geert Bosch]

On Oct 6, 2004, at 11:21, Zack Weinberg wrote:
> I don't care about the details of how Ada interfaces with the
> language-independent compiler.  My comments were intended solely to
> address the bogus argument that line-map.[ch] was part of GCC rather
> than a generic utility library.

With the change I discussed with Per Bothner, line-map will indeed
be a generic utility library, which may be used to implement the
locus type required by the back end. However, if any GCC front end
is *required* to use the library, I think it is fair to say it is
part of GCC.

   -Geert

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Zack Weinberg]

I don't care about the details of how Ada interfaces with the
language-independent compiler.  My comments were intended solely to
address the bogus argument that line-map.[ch] was part of GCC rather
than a generic utility library.

zw

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Geert Bosch]

On Oct 6, 2004, at 09:48, Zack Weinberg wrote:
> line-map.[ch] should not be considered part of GCC in this context.
> The proper analogy is to code in libiberty, which is shared among many
> projects.  There is no reason the Ada front end cannot make use of it
> when linked to some other back-end (modulo source packaging issues).

It is just wrong for GCC to require front ends to parse and analyze
sources in a particular way. During the entire parsing and analysis
of the Ada sources, we never call into any of the C part of GCC.
The only code executed is the Ada code in the gcc/ada subdirectory.

This is ideal from a maintenance point of view, since the separation
between front end and back end matters is localized in just a few
files comprising Gigi. So, even for large changes in the back end,
such as the recent move to tree-ssa, no front end changes are required.
Similarly, when GCC infrastructure and data structures change, no one
needs to look at or change the Ada sources. We'd like to keep it
this way.

As GCC will implement more languages over time, such a separation is
even more necessary. Just think of a GCC with 20 different front ends.
If every change to a data structure, such as the line map, would
require changes throughout these front ends, maintenance would be
a nightmare.

The reason that especially the C and C++ front ends are so intertwined
with the rest of GCC is partially historical and partially political.
However this has made GCC more of a mess than it should be. Fortunately,
we are moving (slowly) in the right direction by having the front ends
generate tree-representations of an entire compilation unit, before
handing this off to the middle- and back end. Similarly, semantics
of this tree are incrementally tightened.

This GENERIC tree representation is the data structure that needs
to be constructed. The specification that these trees must adhere to
should be the basis of the interface between front end and back end.
Ideally, for a compiler bug, one could just look at the GENERIC
tree dump and the specification of GENERIC and decide wether the
front end or the back end is wrong.

How the front end generates its trees should be of no concern.
My objection against using the line-map.[ch] "library" is that
its "specification" makes so many assumptions on how its callers
are structured, that it ends up specifying how to implement
a front end instead of just providing a way of to build a mapping
from locus to file, line and column.

I see three solutions, listing in order of preference:
   -  Allow front ends to specify the mapping from "Locus" to
      file/line/col. The locus is just an abstract type encoded
      in a 32-bit value, and front ends provide hoods to get
      file/line/column information. This would fit well with
      Ada and Fortran front ends, and completely eliminates
      the need for any extra overhead of the line-map data
      structure.

   -  Make the line-map interface a bit more flexible, so that
      mappings can be created in arbitrary order without large
      performance penalties.

   -  Add an extra pass to Gigi, which sorts all loci by original
      source order and then uses the somewhat clumsy interface of
      line-map to generate the required data.

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Geert Bosch]

On Sep 30, 2004, at 16:02, Per Bothner wrote:
>> ake any difference for the compiler. If it would,
>> that clearly would illustrate that this would create a tight coupling
>> between the front end and the back end. If 90% of the compiler is 
>> aware
>> of the details of the implementation of how locations are stored, that
>> means there is something very wrong in the design.
>
> No, most of the compiler does not need to know the details of how
> locations are stored.
>
> However, if we're to follow your suggestion (as I take it), we
> need two typedefs:
> (a) source_location int as managed by libcpp/line-map.c and used by
> libcpp.
> (b) a different abstract typedef that be used in most of the gcc,
> and whose "meaning" would be defined by language front-ends.  It
> would be defined as (a) in most front-ends, but not Ada.
>
> Accessor macros/functions like DECL_SOURCE_LINE would continue
> to be defined in terms of DECL_SOURCE_LOCATION and LOCATION_LINE.
> However, LOCATION_LINE would become a call-back to a language
> front-end function.

Essentially a) is a special case of b). The back end uses the simple
interface of accessor functions as you describe above. Front ends
can build a mapping using line-map.[ch] and then use its definitions
of these accessor macros, or use its own mapping.

As I understand, Fortran also has a private line map already and may
not require the overhead of maintaining a duplicate data structure.
>
> I think these changes would a add an small extra level of complication
> and confusion.  It's minor and managable, but it is an extra call-back
> interface, which are in principle to be avoided.

As I see it, the interface between front end and back end now
only consists of the few simple accessor functions/macros,
instead of the much, much more complicated line-map.h interface.
If you look at all dependencies between front end and back end
as a graph, the goal is to be able to partition this graph with
cutting as few edges as possible.

You'll see this effect also in Makefile dependencies: back end files
may call the LOCATION_LINE callback, but will not depend on 
line-map.[ch].

> It is also possible to make the line_maps type extensible by putting
> function (method) pointers in the structure, but that also seems
> unnecessary generalization at this point.
>
> Bottom line: you're going to have a hard time convincing people to
> make this change for sake of Ada, given that it's a minor extra
> conversion step for Ada.  Plus given the historical frustrations
> we've had with gnat means we're not going to bend over backwards to
> be optimal for Ada if it complicates the rest of the compiler.

As explained above (and I'll elaborate further in a subsequent email),
I think the above change makes for a much cleaner interface between
front ends and back ends. Having a clean and simple interface will
free you from having to consider Ada in the future when you make
changes to line-map.

   -Geert

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Zack Weinberg]

kenner@vlsi1.ultra.nyu.edu (Richard Kenner) writes:

>     Yes, but that doesn't preclude using the line_map structure, even with
>     non- GCC back-ends.  There is no reason why the Ada front-end can't use
>     line_map for a program that say just does analysis and never touches a
>     tree or an rtx.
>
> Yes, there is because the Ada front end is also used with back-ends other
> than GCC.

line-map.[ch] should not be considered part of GCC in this context.
The proper analogy is to code in libiberty, which is shared among many
projects.  There is no reason the Ada front end cannot make use of it
when linked to some other back-end (modulo source packaging issues).

zw

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Richard Kenner]

    My intention by the word "read" was when when the character of the
    source file are read into the compiler's Source_Buffer, which from
    what I gather happens *before* lexing.

    My suggestion that one option is to create thr Source_Ptr ->
    source_location mapping (i.e. data structures to make the mapping
    reasonably efficient) be done more-or-less at the same time as the
    source file is read into a Source_Buffer and Source_Ptr mapping is
    updated.

The problem is that that code is not dependent on the back-end being GCC.
So it's the wrong place to put callbacks into routines that are in the
GCC hierarchy.

The right place for any such code is in ada/trans.c:Sloc_to_locus.  Indeed
I believe that might be the *only* routine that needs to be modified.

Can you take a close look at that function and suggest needed changes to it?

    Yes, but that doesn't preclude using the line_map structure, even with
    non- GCC back-ends.  There is no reason why the Ada front-end can't use
    line_map for a program that say just does analysis and never touches a
    tree or an rtx.

Yes, there is because the Ada front end is also used with back-ends other
than GCC.

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Per Bothner]

Richard Kenner wrote:

>     To summarize:  you can create the Source_Ptr -> source_location
>     mapping in two basic ways:  interleaved with lexing, or as a
>     mapping when the Source_Buffer is read in.  Whichever you prefer.
> 
> I guess the question is what does "read" mean in the context of a
> compiler that doesn't call the lexer library.

I'm confused.  What is "the lexer library"?  It certainly seems that
Ada calls a lexer library - it appears to be the Scans package. If you 
mean libcpp (except for line-map.[hc]), that's not a lexer library,
that's a pre-processor library.  It is not used for Java or Fortran
either.

Don't get confused by line-map.[hc] being in libcpp.  That's mainly
because some people wanted to put libcpp in a separate top-level
library, and because libcpp depends on line-map[hc].  Logically,
line-map.[hc] is a separate library, but that seemed overkill.

My intention by the word "read" was when when the character of the
source file are read into the compiler's Source_Buffer, which from
what I gather happens *before* lexing.

My suggestion that one option is to create thr Source_Ptr ->
source_location mapping (i.e. data structures to make the mapping
reasonably efficient) be done more-or-less at the same time as the
source file is read into a Source_Buffer and Source_Ptr mapping is
updated.

Alternatively, you can so it using call-backs to line-map.[hc] while the
Ada front-end is scanning the Source_buffer.  That is my recommendation.

> It's tricky to make calls from the Ada part of the code since, as Geert
> pointed out, that code is also used wiht non-GCC back-ends.

Yes, but that doesn't preclude using the line_map structure, even
with non-GCC back-ends.  There is no reason why the Ada front-end
can't use line_map for a program that say just does analysis and
never touches a tree or an rtx.

The Source_Ptr type appears to be an abstract source position encoded
as a int.  I see no reason why you can't change its *implementation*
to be the source_location type defined in line-map.h.  You can do this
*regardless* of whether "back-end" you use.  True, it's a non-trivial
change, and since I obviously don't know all the ramifications and
features of your current implementation, it's not my place to suggest
you make such a pervasive change.  However, my instinct is that would
be the best long-term solution.

> Moreover, it's quite common for the Ada front-end to read a file and
> no item in the file is in the produced tree.

Like C, C++, and Java.

>> So I'd interpret "read" as meaning "being asked to convert the first
> GNAT tree node using this file into a GCC node".

You can certainly do that.  It's not what I would have suggested, but
it may be a reasonable solution.

> Is there a way to find
> out if a filename has been seen already or do we need a separate hash table
> just for that purpose?

Whether a filename has been seen is a front-end issue.

You clearly don't need a separate hash table, if you're willing to add a
boolean field to the Source_Buffer type.

Alternatively, the (relatively new) TRANSLATION_UNIT_DECL may be a
reasonably place to hang per-source-file data.
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Richard Kenner]

    To summarize:  you can create the Source_Ptr -> source_location
    mapping in two basic ways:  interleaved with lexing, or as a
    mapping when the Source_Buffer is read in.  Whichever you prefer.

I guess the question is what does "read" mean in the context of a 
compiler that doesn't call the lexer library.

It's tricky to make calls from the Ada part of the code since, as Geert
pointed out, that code is also used wiht non-GCC back-ends.

Moreover, it's quite common for the Ada front-end to read a file and
no item in the file is in the produced tree.

So I'd interpret "read" as meaning "being asked to convert the first
GNAT tree node using this file into a GCC node".  Is there a way to find
out if a filename has been seen already or do we need a separate hash table
just for that purpose?

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Per Bothner]

Richard Kenner wrote:

> Remember that Ada is not using the lexer, so these (and subsequent) calls
> make no sense to me.  Can you look at the existing code and tell me what
> has to be changed?

You basically need to map Ada's Source_Ptr to source_location.  It is
reasonable for the mapping to be monotonic, i.e. one increases with the
order.  I see 3 basic strategies:
(1) Incrementally, as each source file is added.
(2) After all lexing, using a separate pass after the Ada front-end
is done, to build tables to map Source_Ptr to source_location.
(3) Replace the Source_Ptr reprepresentation by source_location.

Long-term (3) may or may not be the best option, but it is probably
to extensive a change for now.  (2) has the advantage that the changes
may be more centralized and require fewer "call-backs" from Ada code
into the line-map code, but it is probably less efficient, and
complicates interleaving front-end and back-end code.

So that suggests (1).  That means that we call linemap_add
when a new source file is read - i.e. when a new Source_Buffer
in the Source_File.Table.  The mapping from Source_Ptr to
source_location can be constructed:
(a) when first processing a Source_Buffer;
(b) some time later but before the next Source_Buffer;
(c) as each token is lexed.

To summarize:  you can create the Source_Ptr -> source_location
mapping in two basic ways:  interleaved with lexing, or as a
mapping when the Source_Buffer is read in.  Whichever you prefer.
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Geert Bosch]

On Sep 30, 2004, at 13:05, Per Bothner wrote:
> I haven't seen any real reason why the line_table mapping would be a
> poor fit for Ada, or why there would be any problem (performance or
> otherwise) with using it for Ada.
>
> We use the line_table for C/C++/ObjC, Java, and Fortran without 
> problem.
> I don't see why Ada would have any problem.

Because we don't build GCC trees during parsing. Also, GNAT already has
a perfect mapping from locus to filename/line/column, and you're 
suggesting
adding another data structure on top of that. This at least duplicates
information.

> I don't know why you refer to the "back end".  line-map.[ch] is a self-
> contained "library" that was originally used for cpplib, which is part
> of the C/C++ *front-ends*.  There are no "back-end" calls.

The design of GNAT is such that the front end (all compiler files
written in Ada, except back_end.ad[bs]) should not need to know
about the GCC back end or its peculiarities. That this goal has
been achieved to a large part has been shown by implementing
non-GCC back ends for JVM and AAMP.

At a more practical level, the decoupling of front end and back end
ensures that most changes in the GCC back end do not affect the Ada
front end.

I'm not objecting to doing work to update Ada, I object to
having knowledge about GCC 4.0.0's specific line numbering
implementation spread out over the front end sources.

>> I think the ideal approach is to view the locus as private to the 
>> front ends.
>> Then we just use the Ada "Sloc" as locus and provide the necessary
>> functions to map this locus to file, line, column information.
>
> This slightly complicates 90% of the compiler (trees, rtl, debugging,
> error handling) adding a extra level of indirection because you don't
> want to add 10 lines to the Ada front-end.  That doesn't seem like a
> good tradeoff to me.

No, it should not make any difference for the compiler. If it would,
that clearly would illustrate that this would create a tight coupling
between the front end and the back end. If 90% of the compiler is aware
of the details of the implementation of how locations are stored, that
means there is something very wrong in the design.

    -Geert

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Per Bothner]

Geert Bosch wrote:

> There seems to be a severe abstraction inversion here. It's none
> of the back end's business how the front end parses source files.
> For all it cares, the front end might parse files back-to-front.

Even so it presumably would read tokens front-to-back.

> Also, the whole notion of "include" nesting is front end related.
> Ada, for example, has no notion of includes, but uses a module
> system. How this is represented in source files is a choice of
> the front end implementation, and the back end has nothing to
> do with it.

The include-file tracking is a relatively cheap bonus feature
of the line-map library" - just ignore it for Ada.

> It seems to me that the "locus" should be an opaque 32-bit type,
> with accessor functions to retrieve file, line and column information.
> Front ends implement these accessor functions, and the back end
> doesn't care how the information is stored.

It certainly is possible to add just an abstraction layer - I just
don't see the point.

> The C front ends, and maybe others, can use the new mapped location
> approach using the interface you described to create a new locus.
> Front ends for which this approach is not a good fit, can implement
> the much simpler accessor functions for loci.

I haven't seen any real reason why the line_table mapping would be a
poor fit for Ada, or why there would be any problem (performance or
otherwise) with using it for Ada.

We use the line_table for C/C++/ObjC, Java, and Fortran without problem.
I don't see why Ada would have any problem.

> For Ada, processing a single compilation unit might require parsing 
> hundreds
> of files.

Just like C, C++, and Java.

  This is much more common than with other languages, since Ada
> allows
> cross-unit inlining, which requires not only parsing all specifications, 
> but
> also bodies of units with potentially inlined subprograms.

Like C++ and Java.

> Apart from this, it is bad practice to litter the parser with calls into
> the back end to produce duplicate line number information.

I don't know why you refer to the "back end".  line-map.[ch] is a self-
contained "library" that was originally used for cpplib, which is part
of the C/C++ *front-ends*.  There are no "back-end" calls.

> I think the ideal approach is to view the locus as private to the front 
> ends.
> Then we just use the Ada "Sloc" as locus and provide the necessary
> functions to map this locus to file, line, column information.

This slightly complicates 90% of the compiler (trees, rtl, debugging,
error handling) adding a extra level of indirection because you don't
want to add 10 lines to the Ada front-end.  That doesn't seem like a
good tradeoff to me.
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Geert Bosch]

On Sep 29, 2004, at 22:09, Per Bothner wrote:
>> Excuse me for my ignorance, but this seems a very convoluted way to
>> gather some simple information.
>
> You're right.
>
>> Probably I am missing some higher
>> level description of what information the back end needs, and why.
>
> The issue is not *gathering* the information, but *representing* it
> (encoding it) compactly.
>
> Using a 32-bit source_location is more efficient than a 64/128-bit
> (filename, line[, column]) pair/triple.  (It also keeps track of
> "include" nesting, though we don't make full use of that.)

There seems to be a severe abstraction inversion here. It's none
of the back end's business how the front end parses source files.
For all it cares, the front end might parse files back-to-front.
Also, the whole notion of "include" nesting is front end related.
Ada, for example, has no notion of includes, but uses a module
system. How this is represented in source files is a choice of
the front end implementation, and the back end has nothing to
do with it.

> Currently, the "middle end" adds to each expression a "locus", which
> is a pointer to a ggc-allocated (filename, line)-pair, using this
> indirection to improve sharing.  Using the line_table is a cleaner more
> centralized more efficient way to do the sharing.

It seems to me that the "locus" should be an opaque 32-bit type,
with accessor functions to retrieve file, line and column information.
Front ends implement these accessor functions, and the back end
doesn't care how the information is stored.

The C front ends, and maybe others, can use the new mapped location
approach using the interface you described to create a new locus.
Front ends for which this approach is not a good fit, can implement
the much simpler accessor functions for loci.

>> Currently, parsing is fast. Why should we be calling an extra 
>> function for
>> every line parsed?
>
> Huh?  Except for comments and white space (which you can ignore for
> linemap_line_start) you're going to do *many* function calls for every
> line parsed by the time gcc is done with it ...
>
> Anyway, do function calls really matter on modern cpus?  Cache misses
> are what counts ...

For Ada, processing a single compilation unit might require parsing 
hundreds
of files. This is much more common than with other languages, since Ada 
allows
cross-unit inlining, which requires not only parsing all 
specifications, but
also bodies of units with potentially inlined subprograms.

Apart from this, it is bad practice to litter the parser with calls into
the back end to produce duplicate line number information. It's really
wrong for the back end to be so intermingled with the front end.
The way forward for GCC is to decouple the front ends more from the
back ends. Going to tree-ssa with GENERIC and GIMPLE are good steps
forward, but this location business seems a clear step back.

> That may be possible.  However, the line-map data structure is not
> optimized for handing out source_location integers in random order,
> so if successive calls to Sloc_to_locus jump around too much you'll
> get an excessive number of line_map entries.  In practice, it may be
> ok to jump around as long as you don't need to allocate a new line_map
> entry: same filename, column number no bigger than the max_column_hint,
> and the line number not less than or too much bigger than given by the
> most recent (explicit or implicit) linemap_add.  And if not, the code
> will allocate a new line_map entry, which is no big deal.  The point
> is if you're allocating line_map entries too frequently, you're losing
> the space-saving benefit of the line_table data structure, and just
> getting extra overhead.

I think the ideal approach is to view the locus as private to the front 
ends.
Then we just use the Ada "Sloc" as locus and provide the necessary
functions to map this locus to file, line, column information.

   -Geert

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Richard Kenner]

    * When starting to parse a new source file do:
       linemap_add (&line_table, LC_ENTER, false, filename, 1);
    but if it's the first/main source file it's probably better to
    use LC_RENAME instead of LC_ENTER, since otherwise the code will
    think that filename is "included" from <built-in>.  (The compile
    server branch tries to do things a little differently, btw.)

Remember that Ada is not using the lexer, so these (and subsequent) calls
make no sense to me.  Can you look at the existing code and tell me what
has to be changed?

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Per Bothner]

Geert Bosch wrote:

> Excuse me for my ignorance, but this seems a very convoluted way to
> gather some simple information.

You're right.

> Probably I am missing some higher
> level description of what information the back end needs, and why.

The issue is not *gathering* the information, but *representing* it
(encoding it) compactly.

Using a 32-bit source_location is more efficient than a 64/128-bit
(filename, line[, column]) pair/triple.  (It also keeps track of
"include" nesting, though we don't make full use of that.)

Currently, the "middle end" adds to each expression a "locus", which
is a pointer to a ggc-allocated (filename, line)-pair, using this
indirection to improve sharing.  Using the line_table is a cleaner more
centralized more efficient way to do the sharing.

> Currently, parsing is fast. Why should we be calling an extra function for
> every line parsed?

Huh?  Except for comments and white space (which you can ignore for
linemap_line_start) you're going to do *many* function calls for every
line parsed by the time gcc is done with it ...

Anyway, do function calls really matter on modern cpus?  Cache misses
are what counts ...

? Basically, the Ada model is to first build an entire
> Ada parse tree of the Ada source and expand/lower it to a level where
> Ada constructs like generic instantiations and tasks have disappeared.
> That tree gets then lowered to GENERIC. Currently we have about a
> dozen places in Gigi (the GNAT to GCC tree translator) where we call
> a routine Sloc_to_locus, which converts an Ada source location
> into a location_t. I would assume that there would be a way that
> we can change this routine to hand the back end the new information,
> including column number.

That may be possible.  However, the line-map data structure is not
optimized for handing out source_location integers in random order,
so if successive calls to Sloc_to_locus jump around too much you'll
get an excessive number of line_map entries.  In practice, it may be
ok to jump around as long as you don't need to allocate a new line_map
entry: same filename, column number no bigger than the max_column_hint,
and the line number not less than or too much bigger than given by the
most recent (explicit or implicit) linemap_add.  And if not, the code
will allocate a new line_map entry, which is no big deal.  The point
is if you're allocating line_map entries too frequently, you're losing
the space-saving benefit of the line_table data structure, and just
getting extra overhead.
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Geert Bosch]

\On Sep 29, 2004, at 19:31, Per Bothner wrote:
> Richard Kenner wrote:
>> So indeed I suspect the changes are tiny, but would appreciate a 
>> little more
>> details about what needs to be done.
> <long list snipped>

Excuse me for my ignorance, but this seems a very convoluted way to
gather some simple information. Probably I am missing some higher
level description of what information the back end needs, and why.

Currently, parsing is fast. Why should we be calling an extra function 
for
every line parsed? Basically, the Ada model is to first build an entire
Ada parse tree of the Ada source and expand/lower it to a level where
Ada constructs like generic instantiations and tasks have disappeared.
That tree gets then lowered to GENERIC. Currently we have about a
dozen places in Gigi (the GNAT to GCC tree translator) where we call
a routine Sloc_to_locus, which converts an Ada source location
into a location_t. I would assume that there would be a way that
we can change this routine to hand the back end the new information,
including column number.

   -Geert

Re: Ada, treelang needs to be converted to --enable-mapped-location

[James A. Morrison]

 Thanks for the explanation.  Given this outline and what I have already I
should be able to have --enable-mapped-location working this week.

-- 
Thanks,
Jim

http://www.student.cs.uwaterloo.ca/~ja2morri/
http://phython.blogspot.com
http://open.nit.ca/wiki/?page=jim

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Per Bothner]

Matt Austern wrote:

> On Sep 29, 2004, at 3:50 PM, Per Bothner wrote:
> 
>> I'm close to checking in patches to support --enable-mapped-location
>> for Java.  At that point all of the default-enabled languages work
>> more-or-less (i.e. with a small number of testsuite regressions).
>> If we can fix the regressions, and there are clear benefits (e.g.
>> measurable compilation-speed improvements or better errors reporting)
>> we might make --enable-mapped-location the default for 4.0; if not it
>> will at the very latest become the default in stage1 of 4.1.
> 
> I hope you're only talking about having it "become the default" as
> an intermediate stage toward getting rid of the older way of handling
> source locations.  gcc has way too many knobs, and there are far
> too many combinations of options to test.  If this works, and if it's
> an improvement, then we should just convert to it and not look back.

Of course.   Thus the next paragraph:

   However, we can't rip out the old --disable-mapped-location code
   until the remaining front-ends are converted, and in any case they
   should be fixed soon.
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Per Bothner]

Richard Kenner wrote:

> So indeed I suspect the changes are tiny, but would appreciate a little more
> details about what needs to be done.

* Before defining builtins do:
   linemap_add (&line_table, LC_ENTER, false, main_input_filename, 1);
   linemap_add (&line_table, LC_RENAME, false, "<built-in>", 0);

* When starting to parse a new source file do:
   linemap_add (&line_table, LC_ENTER, false, filename, 1);
but if it's the first/main source file it's probably better to
use LC_RENAME instead of LC_ENTER, since otherwise the code will
think that filename is "included" from <built-in>.  (The compile
server branch tries to do things a little differently, btw.)

* When done with a source file:
   linemap_add (&line_table, LC_LEAVE, 0, NULL, 0);
(Including a source file should not use an LC_LEAVE; just an
LC_ENTER of the included file, and then an LC_LEAVE when we
return to the including file.)

* At the start of each source line:
   source_location line_start_loc
     = linemap_line_start (&line_table, lineno, max_col_hint);
You can optionally:
   input_location = line_start_loc;
The lineno is the current 1-origin number of the new line.
The max_col_hint can be 1 if you're not keeping track of
column numbers, or some higher number if you are.  I used 120
in the Fortran front end, as that is less than 2^7.  I'd have
to look at the code to see what the limit is before we need an
extra bit, but 125 is presumably safe.

* Whenever you want a source_location with a column number, do
(after a linemap_line_start for the current line):
   LINEMAP_POSITION_FOR_COLUMN (loc, &line_table, col);
Here 'loc' is an lvalue (e.g. input_location) that gets a
source_location for column 'col' of the current line.

* In the current implementation, neither column numbers or line
numbers have to increase monotically, but it's probably a good
idea if they are.  It is also best to allocate source_location
numbers in a single pass:  There is no intelligence in terms of
trying to reuse previous line_map entries (except for the current
/most recent line_map).  Also, expand_location and other functions
that call linemap_lookup use a binary search, which is ok, but
there is obviously some overhead compared to the old mechanism
of just getting a filename or line field from a location_t pair.
So it is good to avoid needless round-trips between source_location
and expanded filename/line[/column] values.
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Matt Austern]

On Sep 29, 2004, at 3:50 PM, Per Bothner wrote:

> I'm close to checking in patches to support --enable-mapped-location
> for Java.  At that point all of the default-enabled languages work
> more-or-less (i.e. with a small number of testsuite regressions).
> If we can fix the regressions, and there are clear benefits (e.g.
> measurable compilation-speed improvements or better errors reporting)
> we might make --enable-mapped-location the default for 4.0; if not it
> will at the very latest become the default in stage1 of 4.1.

I hope you're only talking about having it "become the default" as
an intermediate stage toward getting rid of the older way of handling
source locations.  gcc has way too many knobs, and there are far
too many combinations of options to test.  If this works, and if it's
an improvement, then we should just convert to it and not look back.

			--Matt

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Zack Weinberg]

Per Bothner <per@bothner.com> writes:

> I'm close to checking in patches to support --enable-mapped-location
> for Java.  At that point all of the default-enabled languages work
> more-or-less (i.e. with a small number of testsuite regressions).
> If we can fix the regressions, and there are clear benefits (e.g.
> measurable compilation-speed improvements or better errors reporting)
> we might make --enable-mapped-location the default for 4.0; if not it
> will at the very latest become the default in stage1 of 4.1.

I have run some C++ benchmarks which found a 2-3% improvement in
system time and a .5% improvement in wall time.  However, I am
completely baffled by the PCH regressions.

zw

Re: Ada, treelang needs to be converted to --enable-mapped-location

[Richard Kenner]

    When I look at the Ada front-end it looks straight-forward too.  I can
    sketch out what is needed and/or provide advice, but I think it is
    best the respective front-end maintainers make and test the actual
    changes.

Indeed the sketch would help.

    The basic constraint is you cannot set filename or line independently
    in input_location, DECL_SOURCE_LOCATION, or EXPR_LOCATION, but must
    get them from the line_table.  And it recommended that you "allocate"
    the source_location integers during initial lexing.  Note you can
    optionally also remember column numbers of declarations and
    expressions. 

Note that Ada uses it own lexer and does the inverse operation:
setting "input_location" to the values it obtained in its scan.  With
the exception of that code, every other place does what you mention.

So indeed I suspect the changes are tiny, but would appreciate a little more
details about what needs to be done.

Ada, treelang needs to be converted to --enable-mapped-location

[Per Bothner]

I'm close to checking in patches to support --enable-mapped-location
for Java.  At that point all of the default-enabled languages work
more-or-less (i.e. with a small number of testsuite regressions).
If we can fix the regressions, and there are clear benefits (e.g.
measurable compilation-speed improvements or better errors reporting)
we might make --enable-mapped-location the default for 4.0; if not it
will at the very latest become the default in stage1 of 4.1.

However, we can't rip out the old --disable-mapped-location code
until the remaining front-ends are converted, and in any case they
should be fixed soon.  The remaining ones in CVS are treelang and
Ada.  It's on the treelang "to-do" list.  Please consider this a
request to make this high-priority on the Ada "to-do" list.

Converting the treelang or Ada front-ends doesn't look difficult.
It didn't take me long to convert (with Paul Brooks's help) the
Fortran front-end, even though I was unfamilar with it.  When I
look at the Ada front-end it looks straight-forward too.  I can
sketch out what is needed and/or provide advice, but I think it is
best the respective front-end maintainers make and test the actual
changes.

Here for illustration is the patch to convert the Fortran front-end:
http://gcc.gnu.org/ml/gcc-patches/2004-09/msg00635.html
The basic constraint is you cannot set filename or line independently
in input_location, DECL_SOURCE_LOCATION, or EXPR_LOCATION, but must
get them from the line_table.  And it recommended that you "allocate"
the source_location integers during initial lexing.  Note you can
optionally also remember column numbers of declarations and expressions.
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/