[Bug lto/65536] New: [5 regression] LTO line number information garbled

[Bug lto/65536] New: [5 regression] LTO line number information garbled

[hubicka at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

            Bug ID: 65536
           Summary: [5 regression] LTO line number information garbled
           Product: gcc
           Version: 5.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: lto
          Assignee: unassigned at gcc dot gnu.org
          Reporter: hubicka at gcc dot gnu.org

As shown in https://gcc.gnu.org/ml/gcc-patches/2015-03/msg01151.html
warnings seems to come out wrong with large programs.  I did not manage to
reproduce it with small testcase.  Columns tends to be 0 and line numbers
(somewhat) off usually pointing to the begging of type definition instead of
the field in question but sometimes they just point in further distance.

This reproduce both on Fireofx and Chromium

The problem goes away with:

Index: lto-streamer-in.c
===================================================================
--- lto-streamer-in.c   (revision 221582)
+++ lto-streamer-in.c   (working copy)
@@ -200,7 +200,7 @@ lto_input_location (struct bitpack_d *bp
   if (column_change)
     current_col = bp_unpack_var_len_unsigned (bp);

-  if (file_change)
+  if (file_change || 1)
     {
       if (prev_file)
        linemap_add (line_table, LC_LEAVE, false, NULL, 0);

(which also quite significantly increases memory use).  The warnings seems to
be right on beggining and gets worse at end, so I suspect it is some kind of
overflow in libcpp.

The problem stays with:

Index: lto-streamer-in.c
===================================================================
--- lto-streamer-in.c   (revision 221582)
+++ lto-streamer-in.c   (working copy)
@@ -207,7 +207,7 @@ lto_input_location (struct bitpack_d *bp

       linemap_add (line_table, LC_ENTER, false, current_file, current_line);
     }
-  else if (line_change)
+  else if (line_change || 1)
     linemap_line_start (line_table, current_line, current_col);

   return linemap_position_for_column (line_table, current_col);

One obvious thing is that linemap_line_start takes argument 3 to be max column
hint, but we pass current_col that is not cool.

I see that libcpp seems to drop the column info after some threshold (that is
clearly too small for LTO) but why the line numbers are off?

[Bug lto/65536] [5 regression] LTO line number information garbled

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #2 from Richard Biener <rguenth at gcc dot gnu.org> ---
The main issue with LTO is that it re-creates a combined linemap but in (most
of the time) quite awkward ordering (simply registering random-ordered
file:line:column entries by switching files/lines "appropriately").

I've argued that it would be better to stream those file:line:columns
separately so we can "sort" them before handing them over to libcpp for
linemap re-creation.

So currently we do, for each "file:line:column" location we stream in:

  if (file_change)
    {
      if (prev_file)
        linemap_add (line_table, LC_LEAVE, false, NULL, 0);

      linemap_add (line_table, LC_ENTER, false, current_file, current_line);
    }
  else if (line_change)
    linemap_line_start (line_table, current_line, current_col);

  return linemap_position_for_column (line_table, current_col);

which of course puts a heavy load on the linemap encoding...

But it should definitely work (not sure what it does on line or file
"overflow").

[Bug lto/65536] LTO line number information garbled

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|[5 regression] LTO line     |LTO line number information
                   |number information garbled  |garbled

--- Comment #3 from Richard Biener <rguenth at gcc dot gnu.org> ---
Btw, this can't be a regression in GCC 5 only (or even a regression at all?).
Maybe with macro expansion locations and better column tracking we now
more likely overflow tables, but well...

[Bug lto/65536] LTO line number information garbled

[manu at gcc dot gnu.org]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="UTF-8", Size: 3539 bytes --]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #6 from Manuel López-Ibáñez <manu at gcc dot gnu.org> ---
(In reply to Jan Hubicka from comment #0)
> One obvious thing is that linemap_line_start takes argument 3 to be max
> column hint, but we pass current_col that is not cool.

If I remember correctly, the column hint is only used to try to optimize
memory. That is, if you provide a bad hint, then memory will be wasted (a too
short hint may require additional maps for larger columns; a too large hint
requires additional maps to encode the same number of lines).

However, unless you pass a value of 100000 or more, it should not affect the
actual behavior.
>From gcc-bugs-return-481414-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Tue Mar 24 12:35:10 2015
Return-Path: <gcc-bugs-return-481414-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 48708 invoked by alias); 24 Mar 2015 12:35:09 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 48573 invoked by uid 48); 24 Mar 2015 12:35:06 -0000
From: "jakub at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug lto/65515] [5 Regression] FAIL: gcc.c-torture/compile/limits-fndefn.c   -O2 -flto -flto-partition=none  (ICE) -- SIGSEGV for stack growth failure
Date: Tue, 24 Mar 2015 12:56:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: lto
X-Bugzilla-Version: 5.0
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: jakub at gcc dot gnu.org
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: 5.0
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields: attachments.created
Message-ID: <bug-65515-4-AA9GCAadZL@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-65515-4@http.gcc.gnu.org/bugzilla/>
References: <bug-65515-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-03/txt/msg02558.txt.bz2
Content-length: 973

https://gcc.gnu.org/bugzilla/show_bug.cgi?ide515

--- Comment #7 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Created attachment 35124
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id5124&actioníit
gcc5-pr65515.patch

Lightly tested fix.  Basically, most of DFS::DFS_write_tree is now moved into a
loop in DFS::DFS, and for most worklist items we see them twice, once with NULL
w.cstate (that will result in pushing further worklist items to the stack) and
then again with non-NULL w.cstate (which is the part of DFS_write_tree at the
end of function.
The patch reorders the processing of trees embedded in a tree, we'll process
the last inserted once before the earlier inserted ones for the same tree, not
sure if that is a problem or not, but from the DFS POV they are all children of
the same tree.  If that would be a problem, it would be possible to rewrite
DFS_write_tree_body to call DFS_write_tree in reverse order.  But I hope it
isn't a problem.

[Bug lto/65536] LTO line number information garbled

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #7 from Richard Biener <rguenth at gcc dot gnu.org> ---
(In reply to Manuel López-Ibáñez from comment #5)
> (In reply to Richard Biener from comment #2)
> > The main issue with LTO is that it re-creates a combined linemap but in (most
> > of the time) quite awkward ordering (simply registering random-ordered
> > file:line:column entries by switching files/lines "appropriately").
> > 
> > I've argued that it would be better to stream those file:line:columns
> > separately so we can "sort" them before handing them over to libcpp for
> > linemap re-creation.
> 
> If the lines or columns are out of order, then all bets are off. Line-map
> locations have to be allocated incrementally. If you switch between files a
> lot, then you are wasting a lot of memory unnecessarily, since switching
> requires creating a new line-map.
> 
> How difficult would be to track on which file you are and simply have a
> line_table per file and switch the current line_table appropriately?
> 
> > So currently we do, for each "file:line:column" location we stream in:
> > 
> >   if (file_change)
> >     {
> >       if (prev_file)
> >         linemap_add (line_table, LC_LEAVE, false, NULL, 0);
> > 
> >       linemap_add (line_table, LC_ENTER, false, current_file, current_line);
> >     }
> >   else if (line_change)
> >     linemap_line_start (line_table, current_line, current_col);
> > 
> >   return linemap_position_for_column (line_table, current_col);
> > 
> > which of course puts a heavy load on the linemap encoding...
> 
> According to line-map.h:
> 
> /* Reason for creating a new line map with linemap_add.  LC_ENTER is
>    when including a new file, e.g. a #include directive in C.
>    LC_LEAVE is when reaching a file's end.  LC_RENAME is when a file
>    name or line number changes for neither of the above reasons
>    (e.g. a #line directive in C);
> 
> Thus I'm not sure the above is really correct. If a single line_table is
> desirable, it probably should use LC_RENAME for different main files (as if
> they were concatenated and using #line). LC_LEAVE/LC_ENTER only makes sense
> for included files, which I presume are slightly different than concatenated
> files.

I see.  An issue is that we don't keep track of whether we entered a file
already (and after re-computing line-maps we lose any "nesting" of includes,
that is, we always have enter/leave pairs).  But if LC_RENAME works even
when a file was never LC_ENTERed before then we can use it.

Not sure if it will make a difference though.

Index: gcc/lto-streamer-in.c
===================================================================
--- gcc/lto-streamer-in.c       (revision 221619)
+++ gcc/lto-streamer-in.c       (working copy)
@@ -182,7 +182,6 @@ lto_input_location (struct bitpack_d *bp
   static int current_line;
   static int current_col;
   bool file_change, line_change, column_change;
-  bool prev_file = current_file != NULL;

   if (bp_unpack_value (bp, 1))
     return UNKNOWN_LOCATION;
@@ -201,12 +200,7 @@ lto_input_location (struct bitpack_d *bp
     current_col = bp_unpack_var_len_unsigned (bp);

   if (file_change)
-    {
-      if (prev_file)
-       linemap_add (line_table, LC_LEAVE, false, NULL, 0);
-
-      linemap_add (line_table, LC_ENTER, false, current_file, current_line);
-    }
+    linemap_add (line_table, LC_RENAME, false, current_file, current_line);
   else if (line_change)
     linemap_line_start (line_table, current_line, current_col);

> line-map.c has code to make some erroneous cases "work": I think they should
> be replaced by linemap_assert_fails(), such that they trigger an assert when
> checking but they try to recover gracefully otherwise.
> 
> How are the original locations stored on disk?

As string for file and two ints for line and column.  Thus they are
stored "expanded".  But they are interleaved with the trees / stmts they
are associated to.
>From gcc-bugs-return-481423-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Tue Mar 24 13:08:54 2015
Return-Path: <gcc-bugs-return-481423-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 113376 invoked by alias); 24 Mar 2015 13:08:54 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 113318 invoked by uid 48); 24 Mar 2015 13:08:50 -0000
From: "paolo.carlini at oracle dot com" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug c++/58156] bogus ambigous overload with variadic template
Date: Tue, 24 Mar 2015 13:38:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: c++
X-Bugzilla-Version: unknown
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: paolo.carlini at oracle dot com
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields: bug_status cf_reconfirmed_on everconfirmed
Message-ID: <bug-58156-4-Qirbw7bG58@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-58156-4@http.gcc.gnu.org/bugzilla/>
References: <bug-58156-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-03/txt/msg02567.txt.bz2
Content-length: 407

https://gcc.gnu.org/bugzilla/show_bug.cgi?idX156

Paolo Carlini <paolo.carlini at oracle dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
   Last reconfirmed|                            |2015-03-24
     Ever confirmed|0                           |1

[Bug lto/65536] LTO line number information garbled

[manu at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #8 from Manuel López-Ibáñez <manu at gcc dot gnu.org> ---
(In reply to Richard Biener from comment #7)
> (In reply to Manuel López-Ibáñez from comment #5)
> > (In reply to Richard Biener from comment #2)
> > > The main issue with LTO is that it re-creates a combined linemap but in (most
> > > of the time) quite awkward ordering (simply registering random-ordered
> > > file:line:column entries by switching files/lines "appropriately").
> > > 
> > > I've argued that it would be better to stream those file:line:columns
> > > separately so we can "sort" them before handing them over to libcpp for
> > > linemap re-creation.

Do you mean build the line_table first in order, then when reading in the
trees/ statements, find the location_t corresponding to each given
file:line:column? 

Unfortunately, there is no fast way to find the maps that correspond to a given
file:line:column. 

You could collect all physical locations (file:line:columns), perhaps having
some sorted list of files that points to a sorted list of line:column pairs.
Then go through each file in order and build the line_table incrementally, once
done with a file, LC_RENAME for the next file and continue building the line
table and so on. Then, we streaming tree, find the location_t for a given
file:line:column by doing a (binary?) search in the list of maps. This last
step seems slow, not sure how it could be sped up.

> I see.  An issue is that we don't keep track of whether we entered a file
> already (and after re-computing line-maps we lose any "nesting" of includes,
> that is, we always have enter/leave pairs).  But if LC_RENAME works even
> when a file was never LC_ENTERed before then we can use it.
> 
> Not sure if it will make a difference though.

I think not much because line-map.c already has code to deal with this. Look
for:
          /* So this _should_ mean we are leaving the main file --
             effectively ending the compilation unit. But to_file not
             being NULL means the caller thinks we are leaving to
             another file. This is an erroneous behaviour but we'll
             try to recover from it. Let's pretend we are not leaving
             the main file.  */

> > How are the original locations stored on disk?
> 
> As string for file and two ints for line and column.  Thus they are
> stored "expanded".  But they are interleaved with the trees / stmts they
> are associated to.

I wonder if it would not be easier to store directly the line_table and
location_t per object file, then re-encode location_t when reading to be an
index into an array of line_tables, plus the actual location_t. It just needs
to be careful to undo the re-encoding and use the correct line_table before
passing the info to the line-map functions.
>From gcc-bugs-return-481438-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Tue Mar 24 14:12:13 2015
Return-Path: <gcc-bugs-return-481438-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 2524 invoked by alias); 24 Mar 2015 14:12:13 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 2233 invoked by uid 48); 24 Mar 2015 14:12:09 -0000
From: "redi at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug libstdc++/32261] Thread race segfault in std::string::append with -O and -s
Date: Tue, 24 Mar 2015 14:38:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: libstdc++
X-Bugzilla-Version: 3.3.6
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: redi at gcc dot gnu.org
X-Bugzilla-Status: VERIFIED
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: 3.4.6
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields:
Message-ID: <bug-32261-4-LWW3maewkT@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-32261-4@http.gcc.gnu.org/bugzilla/>
References: <bug-32261-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-03/txt/msg02582.txt.bz2
Content-length: 191

https://gcc.gnu.org/bugzilla/show_bug.cgi?id2261

--- Comment #13 from Jonathan Wakely <redi at gcc dot gnu.org> ---
For the record, I believe this was fixed by https://gcc.gnu.org/r67912

[Bug lto/65536] LTO line number information garbled

[manu at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #9 from Manuel López-Ibáñez <manu at gcc dot gnu.org> ---
(In reply to Richard Biener from comment #2)
>   if (file_change)
>     {
>       if (prev_file)
>         linemap_add (line_table, LC_LEAVE, false, NULL, 0);
> 
>       linemap_add (line_table, LC_ENTER, false, current_file, current_line);
>     }
>   else if (line_change)
>     linemap_line_start (line_table, current_line, current_col);
> 
>   return linemap_position_for_column (line_table, current_col);

By the way, just doing 

    else if (line_change)
-     linemap_line_start (line_table, current_line, current_col);
+     linemap_line_start (line_table, current_line, current_col + 1);

    return linemap_position_for_column (line_table, current_col);

should reduce memory consumption (and perhaps speed) of the line_table by a
lot. You are basically saying: I need a map not bigger than current_col. Then
saying, give me a position for current_col, which triggers another map
allocation.

And my guess is that:

    else if (line_change)
-     linemap_line_start (line_table, current_line, current_col);
+     linemap_line_start (line_table, current_line, 81);

    return linemap_position_for_column (line_table, current_col);

may reduce it even further if at least the file:line are mostly in order
(otherwise, the column number does not matter because you will create a new map
for every call to linemap_line_start).
>From gcc-bugs-return-481440-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Tue Mar 24 14:19:36 2015
Return-Path: <gcc-bugs-return-481440-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 24419 invoked by alias); 24 Mar 2015 14:19:35 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 24359 invoked by uid 48); 24 Mar 2015 14:19:32 -0000
From: "jakub at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug lto/65536] LTO line number information garbled
Date: Tue, 24 Mar 2015 14:48:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: lto
X-Bugzilla-Version: 5.0
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: jakub at gcc dot gnu.org
X-Bugzilla-Status: UNCONFIRMED
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields:
Message-ID: <bug-65536-4-zW15NCuG8V@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-65536-4@http.gcc.gnu.org/bugzilla/>
References: <bug-65536-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-03/txt/msg02584.txt.bz2
Content-length: 555

https://gcc.gnu.org/bugzilla/show_bug.cgi?ide536

--- Comment #10 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
You could just stream another table, containing the { file, line, column }
triplets, and stream location_t as indexes into this table (with 0/1 being
special for UNKNOWN_LOCATION and BUILTINS_LOCATION), or as pairs { location_t,
block }.  Then on LTO reading, you could perhaps read the location tables from
multiple source files, merge them and for each of the TUs build a hash_map
mapping from the table indexes to location_t values.

[Bug lto/65536] LTO line number information garbled

[manu at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #11 from Manuel López-Ibáñez <manu at gcc dot gnu.org> ---
(In reply to Manuel López-Ibáñez from comment #9)

> should reduce memory consumption (and perhaps speed) of the line_table by a

I meant "increase speed".
>From gcc-bugs-return-481444-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Tue Mar 24 14:38:10 2015
Return-Path: <gcc-bugs-return-481444-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 106958 invoked by alias); 24 Mar 2015 14:38:09 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 106865 invoked by uid 48); 24 Mar 2015 14:38:01 -0000
From: "manu at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug lto/65536] LTO line number information garbled
Date: Tue, 24 Mar 2015 14:56:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: lto
X-Bugzilla-Version: 5.0
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: manu at gcc dot gnu.org
X-Bugzilla-Status: UNCONFIRMED
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields:
Message-ID: <bug-65536-4-NULi9gE9bk@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-65536-4@http.gcc.gnu.org/bugzilla/>
References: <bug-65536-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-03/txt/msg02588.txt.bz2
Content-length: 1379

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #12 from Manuel López-Ibáñez <manu at gcc dot gnu.org> ---
(In reply to Jakub Jelinek from comment #10)
> You could just stream another table, containing the { file, line, column }
> triplets, and stream location_t as indexes into this table (with 0/1 being
> special for UNKNOWN_LOCATION and BUILTINS_LOCATION), or as pairs {
> location_t, block }.  Then on LTO reading, you could perhaps read the
> location tables from multiple source files, merge them and for each of the
> TUs build a hash_map mapping from the table indexes to location_t values.

That is a bit of waste of memory, no? The "{ file, line, column }" table might
be huge, presumably much larger than the actual line_tables. Thus, why not just
stream the line_tables directly? Then, you can stream the location_t as they
are.

When reading, it is possible to merge different TU line_tables into a single
one quite efficiently (in the order of the maps and not in the order of
triplets or maximum location_t). When adding each new TU line_table to the
unified one, one keeps track of the last location_t assigned, and uses this
later to convert out of order the old location_t from each TU line_table to the
new location_t of the new unified line_table.

Perhaps I'm missing some detail and Dodji can comment on this idea.
>From gcc-bugs-return-481445-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Tue Mar 24 14:43:00 2015
Return-Path: <gcc-bugs-return-481445-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 126465 invoked by alias); 24 Mar 2015 14:42:59 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 126304 invoked by uid 48); 24 Mar 2015 14:42:49 -0000
From: "manu at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug lto/65536] LTO line number information garbled
Date: Tue, 24 Mar 2015 14:56:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: lto
X-Bugzilla-Version: 5.0
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: manu at gcc dot gnu.org
X-Bugzilla-Status: UNCONFIRMED
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields:
Message-ID: <bug-65536-4-WEyMRwfdBF@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-65536-4@http.gcc.gnu.org/bugzilla/>
References: <bug-65536-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-03/txt/msg02589.txt.bz2
Content-length: 1045

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #13 from Manuel López-Ibáñez <manu at gcc dot gnu.org> ---
(In reply to Manuel López-Ibáñez from comment #12)
> When reading, it is possible to merge different TU line_tables into a single
> one quite efficiently (in the order of the maps and not in the order of
> triplets or maximum location_t). When adding each new TU line_table to the
> unified one, one keeps track of the last location_t assigned, and uses this
> later to convert out of order the old location_t from each TU line_table to
> the new location_t of the new unified line_table.

If the merge function is implemented within line-map.c, it could be extremely
efficient in terms of memory re-allocations. I think with the other approaches
we end up reallocating the whole table many times because we do not know how
many maps we need in advance (admittedly, anything else would be better than
the current approach, because it seems LTO is creating a lot of empty maps).
>From gcc-bugs-return-481443-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Tue Mar 24 14:34:13 2015
Return-Path: <gcc-bugs-return-481443-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 61077 invoked by alias); 24 Mar 2015 14:34:13 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 53777 invoked by uid 48); 24 Mar 2015 14:34:09 -0000
From: "rguenth at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug ipa/65076] [5 Regression] 16% tramp3d-v4.cpp compile time regression
Date: Tue, 24 Mar 2015 14:56:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: ipa
X-Bugzilla-Version: 5.0
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: rguenth at gcc dot gnu.org
X-Bugzilla-Status: UNCONFIRMED
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: 5.0
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields: cc
Message-ID: <bug-65076-4-A1QF7uaiK1@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-65076-4@http.gcc.gnu.org/bugzilla/>
References: <bug-65076-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-03/txt/msg02587.txt.bz2
Content-length: 3319

https://gcc.gnu.org/bugzilla/show_bug.cgi?ide076

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |rguenth at gcc dot gnu.org

--- Comment #14 from Richard Biener <rguenth at gcc dot gnu.org> ---
(In reply to Jan Hubicka from comment #11)
> Sorry, the number of clobbers drops at DSE1, not during ehcleanup2, I just
> messed up my grep.
>
> I tried the following patch:
>
> Index: passes.def
> ==================================================================> --- passes.def  (revision 221541)
> +++ passes.def  (working copy)
> @@ -87,6 +87,7 @@ along with GCC; see the file COPYING3.
>           NEXT_PASS (pass_build_ealias);
>           NEXT_PASS (pass_fre);
>           NEXT_PASS (pass_merge_phi);
> +         NEXT_PASS (pass_dse);
>           NEXT_PASS (pass_cd_dce);
>           NEXT_PASS (pass_early_ipa_sra);
>           NEXT_PASS (pass_tail_recursion);
>
> This brings number of CLOBBER statements at release_ssa time down to 7392
> (50% reduction).  A nice effect of this patch is that it tends to simplify
> destructors often to empty to make them more inlinable:
>
>  ObserverEvent::~ObserverEvent() (struct ObserverEvent * const this)
>  {
>    <bb 2>:
> -  this_2(D)->_vptr.ObserverEvent = &MEM[(void *)&_ZTV13ObserverEvent + 16B];
>    MEM[(struct  &)this_2(D)] ={v} {CLOBBER};
>    return;
>
> saves a lot of the clobbers:
>  Engine<3, double, ExpressionTag<UnaryNode<FnNorm, BinaryNode<OpSubtract,
> Reference<Field<NoMesh<3>, Vector<3, double, Full>, ViewEngine<3,
> IndexFunction<GenericURM<MeshTraits<3, double, UniformRectilinearTag,
> CartesianTag, 3> >::PositionsFunctor> > > >, Scalar<Vector<3, double, Full>
> > > > > >::~Engine() (struct Engine * const this)
>  {
>    <bb 2>:
> -  MEM[(struct  &)this_2(D) + 32] ={v} {CLOBBER};
> -  MEM[(struct  &)this_2(D) + 32] ={v} {CLOBBER};
> -  MEM[(struct  &)this_2(D) + 8] ={v} {CLOBBER};
> -  MEM[(struct  &)this_2(D) + 8] ={v} {CLOBBER};
> -  MEM[(struct  &)this_2(D) + 8] ={v} {CLOBBER};
> -  MEM[(struct  &)this_2(D)] ={v} {CLOBBER};
> -  MEM[(struct  &)this_2(D)] ={v} {CLOBBER};
> -  MEM[(struct  &)this_2(D)] ={v} {CLOBBER};
> +  MEM[(struct  &)this_1(D)] ={v} {CLOBBER};
>    return;
>
> which is especially nice for LTO streaming.
>
> and saves about 7% of code apparently after inlining:
>
> $ wc -l *copyprop2
> 200189 tramp3d-v4.ii.085t.copyprop2
> $ wc -l ../5/*copyprop2
> 215060 ../5/tramp3d-v4.ii.084t.copyprop2
>
> Even though the inline decisions does not seem to be changed considerably
> (at least on tramp3d).

Yeah, clobbers don't account for anything for size/inline estimates
(well, I hope so!).

And yes, doing DSE early is quite an old idea...  we should revisit it
next stage1.

> On unrelated note I noticed PR65502
>
> Still I guess this does not really explain the origin of regression in
> statement count relative to 4.9...

No idea.  I'll have to look myself - the &X + 4 vs. &MEM[&X, 4] is very
reecent so it can't be blamed for the regression.  But it might be blamed
for the number of stmt differences - but only from the very beginning.
That is, I can't see how the difference shows in .ssa but not in .cfg.

[Bug lto/65536] LTO line number information garbled

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #14 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
By streaming the line_table directly you'd stream lots of location_t that are
not actually used for anything that is streamed into the LTO.
I don't understand why the tables would be huge, the string would of course use
normal LTO section string sharing, and if we stream right now every location_t
as the triplet, it would mean in case a single location_t is never used by more
than one tree we'd stream an integer extra to what we stream now (just the
triplets would be streamed in a different place), but in the more common case
where one location_t is used by many trees, we'd stream less than what we
stream now, due to the additional indirection.

[Bug lto/65536] LTO line number information garbled

[manu at gcc dot gnu.org]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="UTF-8", Size: 5439 bytes --]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #15 from Manuel López-Ibáñez <manu at gcc dot gnu.org> ---
(In reply to Jakub Jelinek from comment #14)
> By streaming the line_table directly you'd stream lots of location_t that
> are not actually used for anything that is streamed into the LTO.
> I don't understand why the tables would be huge, the string would of course
> use normal LTO section string sharing, and if we stream right now every
> location_t as the triplet, it would mean in case a single location_t is
> never used by more than one tree we'd stream an integer extra to what we
> stream now (just the triplets would be streamed in a different place), but
> in the more common case where one location_t is used by many trees, we'd
> stream less than what we stream now, due to the additional indirection.

The line_table does not store every possible location_t, only the ones that
start new maps, it should not even store one location_t per line!

But it is true that a whole file:line:column table for all locations stored in
the object file might be smaller than the line_table for the whole TU. It
depends a lot how much of the input file ends up in the object file. But I
think it should be easy to estimate the size of the line_table (see
dump_line_table_statistics). I have no idea how large a file:line:column table
would be for a typical object file. If on-disk size is a concern, one can also
replace virtual (macro) locations with spelling/expansion point locations and
not stream out any macro maps.

Moreover, when streaming out, because of how the line_table is encoded as a
contiguous array, one can mark the maps that are not used by LTO and simply not
stream them out, which may reduce the size of the line_table significantly (and
speed up any linemap merging/lookup later). This cannot be much slower than
what you are doing already, since expanding "file:line:column" requires finding
the map that holds this info. 

Of course, if you are really unlucky, each location_t used by LTO may belong to
a different map, thus you cannot remove anything and then it would have been
more efficient to use a file:line:column table. In this worst case, the
overhead would be approximately sizeof(struct line_map) * number of rows in the
file:line:column table.

Even in this worst case, my intuition says that one could re-optimize the
line_table to make it more efficient (since one location_t per map is the most
inefficient case possible) before streaming out without actually changing the
location_t values, but I would need to think a bit more whether this
re-optimization is actually possible. Perhaps Dodji can figure out a problem
with it.
>From gcc-bugs-return-481465-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Tue Mar 24 16:22:23 2015
Return-Path: <gcc-bugs-return-481465-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 73073 invoked by alias); 24 Mar 2015 16:22:23 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 72774 invoked by uid 48); 24 Mar 2015 16:22:19 -0000
From: "spam.brian.taylor at gmail dot com" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug fortran/65542] New: SPREAD intrinsic incorrectly accepted in initialization expressions with -stdù5
Date: Tue, 24 Mar 2015 16:47:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: new
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: fortran
X-Bugzilla-Version: 4.9.1
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: spam.brian.taylor at gmail dot com
X-Bugzilla-Status: UNCONFIRMED
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields: bug_id short_desc product version bug_status bug_severity priority component assigned_to reporter
Message-ID: <bug-65542-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-03/txt/msg02609.txt.bz2
Content-length: 883

https://gcc.gnu.org/bugzilla/show_bug.cgi?ide542

            Bug ID: 65542
           Summary: SPREAD intrinsic incorrectly accepted in
                    initialization expressions with -stdù5
           Product: gcc
           Version: 4.9.1
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: fortran
          Assignee: unassigned at gcc dot gnu.org
          Reporter: spam.brian.taylor at gmail dot com

Prior to the Fortran 2003 standard, only a subset of transformational intrinsic
functions were allowed in initialization expressions (specifically: REPEAT,
RESHAPE, SELECTED_INT_KIND, SELECTED_REAL_KIND, TRANSFER, and TRIM).  See
Fortran 95 ISO draft standard, section 7.1.6.1, lines 9-11, page 94.

The following code should not compile with -stdù5:

module bug
integer :: ibug(42) = spread(42, 1, 42)
end module

[Bug lto/65536] LTO line number information garbled

[hubicka at ucw dot cz]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #16 from Jan Hubicka <hubicka at ucw dot cz> ---
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536
> 
> --- Comment #2 from Richard Biener <rguenth at gcc dot gnu.org> ---
> The main issue with LTO is that it re-creates a combined linemap but in (most
> of the time) quite awkward ordering (simply registering random-ordered
> file:line:column entries by switching files/lines "appropriately").
> 
> I've argued that it would be better to stream those file:line:columns
> separately so we can "sort" them before handing them over to libcpp for
> linemap re-creation.

Yep, we discussed this few times.  One issue is when we sort them. 
We can keep track of location as they are streamed out (and directly assign
them new IDs in the awkward ordering as they come) and then produce the
separate linemap section that would be read at once by WPA, sorted and
fed into linemaps.

Sorting at compile time is bit tricky as we probably do not want to patch
existing pickled tree stream with the new locator IDs in the sorted sequence
and we do not really know what locations we will need ahead of time.

Other easier to implement idea for GCC 5 may be to simply collect locaiton
info and pointers to trees per SCC component andonly if it survived merging
feed it into linemap and in memory patch the new trees. This still should
save quite some memory given that most of trees read are discarded by merging
and may be quite easy to implement.

Martin, the ICE by my patch is caused by libcpp getting out of memory?

[Bug lto/65536] LTO line number information garbled

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jakub at gcc dot gnu.org

--- Comment #19 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Guess that can't really pass make check or lto bootstrap, because the LTO
streamer needs to be in sync between out and in, so commenting out
streaming in of the column number, but not doing the same for streaming it out
will break things completely.

[Bug lto/65536] LTO line number information garbled

[hubicka at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #20 from Jan Hubicka <hubicka at gcc dot gnu.org> ---
It seems that manuel just commented out the actual insertion to linemap,
columns are still streamed and ignored.

Doing so will clearly make ODR waring more difficult to follow, so perhaps we
could try to go for it only after reading sufficiently large portion of
program?
Note that about 80-90% of code is removed by merging, so perhaps just applying
linemaps after merging fails is way to get enough buffer for chromium sized
programs?

[Bug lto/65536] LTO line number information garbled

[manu at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #21 from Manuel López-Ibáñez <manu at gcc dot gnu.org> ---
(In reply to Jakub Jelinek from comment #19)
> Guess that can't really pass make check or lto bootstrap, because the LTO
> streamer needs to be in sync between out and in, so commenting out
> streaming in of the column number, but not doing the same for streaming it
> out will break things completely.

I was trying to silence the unused-but-set warning. Maybe this works:

--- src/gcc/lto-streamer-in.c   (revision 221557)
+++ src/gcc/lto-streamer-in.c   (working copy)
@@ -198,19 +198,23 @@ lto_input_location (struct bitpack_d *bp
     current_line = bp_unpack_var_len_unsigned (bp);

   if (column_change)
     current_col = bp_unpack_var_len_unsigned (bp);

-  if (file_change)
+  /* FIXME: Disable columns for now until streaming of locations is
reimplemented.  */
+  /* Enough column_hint to disable columns */
+  current_col = LINE_MAP_MAX_COLUMN_NUMBER + 1;
+
+  if (file_change) 
     {
       if (prev_file)
-       linemap_add (line_table, LC_LEAVE, false, NULL, 0);
-
-      linemap_add (line_table, LC_ENTER, false, current_file, current_line);
+       linemap_add (line_table, LC_RENAME, false, current_file, current_line);
+      else
+       linemap_add (line_table, LC_ENTER, false, current_file, current_line);
     }
   else if (line_change)
-    linemap_line_start (line_table, current_line, current_col);
+    return linemap_line_start (line_table, current_line, current_col);

   return linemap_position_for_column (line_table, current_col);
 }


Note that using column_hint == 0, like in this patch:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=45375#c175 does not actually
disable columns as it should, in fact, it triggers a new map allocation. (I
think this is a bug in line-maps.c.)
>From gcc-bugs-return-481500-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Tue Mar 24 19:27:01 2015
Return-Path: <gcc-bugs-return-481500-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 16454 invoked by alias); 24 Mar 2015 19:27:00 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 16420 invoked by uid 48); 24 Mar 2015 19:26:56 -0000
From: "ktietz at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug libgomp/64972] [5 Regression] Build failure in libgomp for i686-w64-mingw32 target after latest merge from gomp-4_0-branch
Date: Tue, 24 Mar 2015 19:55:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: libgomp
X-Bugzilla-Version: 5.0
X-Bugzilla-Keywords: openacc
X-Bugzilla-Severity: critical
X-Bugzilla-Who: ktietz at gcc dot gnu.org
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: 5.0
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields:
Message-ID: <bug-64972-4-aH8O2IE7nq@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-64972-4@http.gcc.gnu.org/bugzilla/>
References: <bug-64972-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-03/txt/msg02644.txt.bz2
Content-length: 2731

https://gcc.gnu.org/bugzilla/show_bug.cgi?idd972

--- Comment #13 from Kai Tietz <ktietz at gcc dot gnu.org> ---
Rainer: does following patch works for you?

Index: oacc-parallel.c
==================================================================--- oacc-parallel.c    (Revision 221640)
+++ oacc-parallel.c    (Arbeitskopie)
@@ -31,6 +31,9 @@
 #include "libgomp_g.h"
 #include "gomp-constants.h"
 #include "oacc-int.h"
+#ifdef HAVE_INTTYPES_H
+# include <inttypes.h>  /* For PRIu64.  */
+#endif
 #include <string.h>
 #include <stdarg.h>
 #include <assert.h>
@@ -99,9 +102,14 @@ GOACC_parallel (int device, void (*fn) (void *),
     gomp_fatal ("num_workers (%d) different from one is not yet supported",
         num_workers);

+#ifdef HAVE_INTTYPES_H
+  gomp_debug (0, "%s: mapnum=%"PRIu64", hostaddrs=%p, size=%p, kinds=%p, "
+         "async = %d\n",
+          __FUNCTION__, (uint64_t) mapnum, hostaddrs, sizes, kinds, async);
+#else
   gomp_debug (0, "%s: mapnum=%zd, hostaddrs=%p, sizes=%p, kinds=%p,
async=%d\n",
           __FUNCTION__, mapnum, hostaddrs, sizes, kinds, async);
-
+#endif
   select_acc_device (device);

   thr = goacc_thread ();
@@ -178,8 +186,13 @@ GOACC_data_start (int device, size_t mapnum,
   bool host_fallback = device == GOMP_DEVICE_HOST_FALLBACK;
   struct target_mem_desc *tgt;

+#ifdef HAVE_INTTYPES_H
+  gomp_debug (0, "%s: mapnum=%"PRIu64", hostaddrs=%p, size=%p, kinds=%p\n",
+          __FUNCTION__, (uint64_t) mapnum, hostaddrs, sizes, kinds);
+#else
   gomp_debug (0, "%s: mapnum=%zd, hostaddrs=%p, sizes=%p, kinds=%p\n",
           __FUNCTION__, mapnum, hostaddrs, sizes, kinds);
+#endif

   select_acc_device (device);

Index: target.c
==================================================================--- target.c    (Revision 221640)
+++ target.c    (Arbeitskopie)
@@ -33,6 +33,9 @@
 #include <limits.h>
 #include <stdbool.h>
 #include <stdlib.h>
+#ifdef HAVE_INTTYPES_H
+# include <inttypes.h>  /* For PRIu64.  */
+#endif
 #include <string.h>
 #include <assert.h>

@@ -438,9 +441,16 @@ gomp_map_vars (struct gomp_device_descr *devicep,
               /* We already looked up the memory region above and it
              was missing.  */
               size_t size = k->host_end - k->host_start;
+#ifdef HAVE_INTTYPES_H
               gomp_fatal ("present clause: !acc_is_present (%p, "
+                  PRIu64" (0x"PRIx64"))",
+                  (void *) k->host_start,
+                  (uint64_t) size, (uint64_t) size);
+#else
+              gomp_fatal ("present clause: !acc_is_present (%p, "
                   "%zd (0x%zx))", (void *) k->host_start,
                   size, size);
+#endif
             }
             break;
           case GOMP_MAP_FORCE_DEVICEPTR:

[Bug lto/65536] LTO line number information garbled

[manu at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #22 from Manuel López-Ibáñez <manu at gcc dot gnu.org> ---
(In reply to Jan Hubicka from comment #20)
> It seems that manuel just commented out the actual insertion to linemap,
> columns are still streamed and ignored.
> 
> Doing so will clearly make ODR waring more difficult to follow, so perhaps
> we could try to go for it only after reading sufficiently large portion of
> program?

The problem is that by creating the line_table with physical locations
out-of-order, LTO is

1) creating a lot of maps, even for small column changes. Thus, consuming a lot
of extra memory.

2) increasing location_t values rapidly, thus probably triggering an overflow. 

The line-maps.c code already dynamically disables column numbers when
location_t values get very large (I think this is the reason that there are no
column numbers in the examples you show). If the location_t numbers get even
larger, it will start returning UNKNOWN_LOCATION. Now I think that if you see
wrong line numbers is most probably because of wrong location info in the
front-ends, not because linemaps overflowed.

There is also some mismatch between the limits used by linemap_line_start():

      if (max_column_hint > LINE_MAP_MAX_COLUMN_NUMBER
          || highest > 0x60000000)
        {
          /* If the column number is ridiculous or we've allocated a huge
             number of source_locations, give up on column numbers. */
          max_column_hint = 0;
          if (highest > 0x70000000)
            return 0; /* Give up completely  and return UKNOWN_LOCATION */
          column_bits = 0;
        }

and the limits used by linemap_position_for_column:

  if (to_column >= set->max_column_hint)
    {
      if (r >= 0xC000000 || to_column > LINE_MAP_MAX_COLUMN_NUMBER)
        {
          /* Running low on source_locations - disable column numbers.  */
          return r;
        }
      else
        {
          struct line_map *map = LINEMAPS_LAST_ORDINARY_MAP (set);
          r = linemap_line_start (set, SOURCE_LINE (map, r), to_column + 50);
        }
    }

I'm not sure if the gap may lead to some problem, but it seems unlikely. In the
worst case, it should return 0, thus UNKNOWN_LOCATION (I can see some instances
of this in your examples).
>From gcc-bugs-return-481502-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Tue Mar 24 19:39:03 2015
Return-Path: <gcc-bugs-return-481502-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 94035 invoked by alias); 24 Mar 2015 19:39:03 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 93970 invoked by uid 55); 24 Mar 2015 19:39:00 -0000
From: "jason at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug c++/65498] [5 Regression] ICE in cxx_eval_call_expression when using __func__ inside dependent context
Date: Tue, 24 Mar 2015 20:01:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: c++
X-Bugzilla-Version: 5.0
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: jason at gcc dot gnu.org
X-Bugzilla-Status: ASSIGNED
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: jason at gcc dot gnu.org
X-Bugzilla-Target-Milestone: 5.0
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields:
Message-ID: <bug-65498-4-uE2CSEk7z4@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-65498-4@http.gcc.gnu.org/bugzilla/>
References: <bug-65498-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-03/txt/msg02646.txt.bz2
Content-length: 593

https://gcc.gnu.org/bugzilla/show_bug.cgi?ide498

--- Comment #10 from Jason Merrill <jason at gcc dot gnu.org> ---
Author: jason
Date: Tue Mar 24 19:38:28 2015
New Revision: 221641

URL: https://gcc.gnu.org/viewcvs?rev"1641&root=gcc&view=rev
Log:
    PR c++/65498
    * pt.c (get_mostly_instantiated_function_type): Just return the
    type of the partially instantiated template in DECL_TI_TEMPLATE.

Added:
    trunk/gcc/testsuite/g++.dg/cpp0x/constexpr-targ2.C
Modified:
    trunk/gcc/cp/ChangeLog
    trunk/gcc/cp/pt.c
    trunk/gcc/testsuite/g++.dg/cpp0x/lambda/lambda-template13.C

[Bug lto/65536] LTO line number information garbled

[manu at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #23 from Manuel López-Ibáñez <manu at gcc dot gnu.org> ---
(In reply to Manuel López-Ibáñez from comment #15)
> (In reply to Jakub Jelinek from comment #14)
> > By streaming the line_table directly you'd stream lots of location_t that
> > are not actually used for anything that is streamed into the LTO.
> > I don't understand why the tables would be huge, the string would of course
> > use normal LTO section string sharing, and if we stream right now every
> > location_t as the triplet, it would mean in case a single location_t is
> > never used by more than one tree we'd stream an integer extra to what we
> > stream now (just the triplets would be streamed in a different place), but
> > in the more common case where one location_t is used by many trees, we'd
> > stream less than what we stream now, due to the additional indirection.
> 
[...]
> Of course, if you are really unlucky, each location_t used by LTO may belong
> to a different map, thus you cannot remove anything and then it would have
> been more efficient to use a file:line:column table. In this worst case, the
> overhead would be approximately sizeof(struct line_map) * number of rows in
> the file:line:column table.

In fact, looking at 

struct GTY(()) line_map_ordinary {
  const char *to_file;
  linenum_type to_line;

  /* An index into the set that gives the line mapping at whose end
     the current one was included.  File(s) at the bottom of the
     include stack have this set to -1.  */
  int included_from;

  /* SYSP is one for a system header, two for a C system header file
     that therefore needs to be extern "C" protected in C++, and zero
     otherwise.  This field isn't really needed now that it's in
     cpp_buffer.  */
  unsigned char sysp;

  /* Number of the low-order source_location bits used for a column number.  */
  unsigned int column_bits : 8;
};

If we do not care about sysp and included_from, then even in the worst case, it
will be more efficient to stream out the line_table than the file:line:column
table.

And it has the benefit that it is already sorted and the location_t values
within trees/gimple can be streamed out directly without calling linemap_lookup
to get the file:line:column for each of them. And since a location_t takes 1/3
of space that file:line:column, that would reduce the streamed out data even
further.
>From gcc-bugs-return-481507-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Tue Mar 24 19:58:26 2015
Return-Path: <gcc-bugs-return-481507-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 57995 invoked by alias); 24 Mar 2015 19:58:25 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 57233 invoked by uid 48); 24 Mar 2015 19:58:19 -0000
From: "paolo.carlini at oracle dot com" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug c++/61358] Bogus "duplicate label" error after label used within C++11 lambda
Date: Tue, 24 Mar 2015 20:25:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: c++
X-Bugzilla-Version: 4.8.1
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: paolo.carlini at oracle dot com
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields: bug_status cf_reconfirmed_on cc blocked everconfirmed
Message-ID: <bug-61358-4-zh4YhzVFUE@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-61358-4@http.gcc.gnu.org/bugzilla/>
References: <bug-61358-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-03/txt/msg02651.txt.bz2
Content-length: 543

https://gcc.gnu.org/bugzilla/show_bug.cgi?ida358

Paolo Carlini <paolo.carlini at oracle dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
   Last reconfirmed|                            |2015-03-24
                 CC|                            |paolo.carlini at oracle dot com
             Blocks|                            |54367
     Ever confirmed|0                           |1

[Bug lto/65536] LTO line number information garbled

[hubicka at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #24 from Jan Hubicka <hubicka at gcc dot gnu.org> ---
Manuel, you may be right person to implement the streaming of linemaps then :)
I suppose we do care about inline stacks in longer term to get proper warnings.
We even may want to preserve macro expansion and all other info we have at
compile time.

In addition to that we ought to be able to tell what compilation unit the line
infromation originate from
https://gcc.gnu.org/ml/gcc-patches/2014-07/msg01876.html because same location
in different units may have (and often has) different context. Not for GCC 5
though.

I will still experiment with the idea of adding locations to linemaps only once
tree merging found the tree location new.  On average we read every tree about
20 times for Firefox, so this ought to reduce the waste of linemap space quite
considerably perhaps pushing it to non-issue for GCC 5.

honza

[Bug lto/65536] LTO line number information garbled

[manu at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #25 from Manuel López-Ibáñez <manu at gcc dot gnu.org> ---
(In reply to Jan Hubicka from comment #24)
> Manuel, you may be right person to implement the streaming of linemaps then

I have in fact considered it already, since I think it should be fairly easy
and it can be done incrementally. However, as always in GCC, things are never
as trivial as they seem. I just tried following
https://gcc.gnu.org/wiki/DebuggingGCC#Debugging_LTO to see what
dump_line_table_statistics reports before and after my patch above, but it
doesn't print the lto1 invokation. The debugx scripts
(https://gcc.gnu.org/ml/gcc/2004-03/msg01195.html) do not seem to work with
lto1 (why those scripts are not in contrib/?). I feel like a new contributor
again! :)

Thus, sorry I don't really have the necessary free time to dedicate to this.
I'm too busy with my real job at the moment. If/When I have time to dedicate to
develop GCC, I will finish the transition of Fortran to the common diagnostics
machinery, and then I will focus on C/C++ diagnostics.

I still think that a patch similar to the one in comment #21 (+libcpp parts in
comment #17) may improve the situation for GCC 5 and fix PR54962.
>From gcc-bugs-return-481527-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Wed Mar 25 01:44:03 2015
Return-Path: <gcc-bugs-return-481527-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 99481 invoked by alias); 25 Mar 2015 01:44:03 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 99429 invoked by uid 48); 25 Mar 2015 01:43:59 -0000
From: "jvdelisle at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug fortran/65541] [5 Regression] namelist regression
Date: Wed, 25 Mar 2015 03:24:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: fortran
X-Bugzilla-Version: 5.0
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: jvdelisle at gcc dot gnu.org
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P4
X-Bugzilla-Assigned-To: jvdelisle at gcc dot gnu.org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields: assigned_to
Message-ID: <bug-65541-4-xv6k7YnLzq@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-65541-4@http.gcc.gnu.org/bugzilla/>
References: <bug-65541-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-03/txt/msg02671.txt.bz2
Content-length: 924

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65541

Jerry DeLisle <jvdelisle at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|unassigned at gcc dot gnu.org      |jvdelisle at gcc dot gnu.org

--- Comment #3 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
(In reply to Dominique d'Humieres from comment #2)
> This is due to revision r210934 (pr55117) from:
> 
> 	* trans-io.c (nml_full_name, transfer_namelist_element): Insert
> 	a '+' rather then '%' to differentiate namelist variable names
> 	that are based on extended derived types.
> 
> this seems to have been done on purpose (I did not follow the discussion in
> the PR).

Yes, it was done on purpose to differentiate two possible representation of
derived type namelists with extended types.  I have a patch in mind.
>From gcc-bugs-return-481528-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Wed Mar 25 02:18:40 2015
Return-Path: <gcc-bugs-return-481528-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 79419 invoked by alias); 25 Mar 2015 02:18:39 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 79384 invoked by uid 48); 25 Mar 2015 02:18:36 -0000
From: "msebor at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug target/61977] powerpc-eabi preprocessor breaks on lines that end with "vector"
Date: Wed, 25 Mar 2015 03:46:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: target
X-Bugzilla-Version: 4.9.1
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: msebor at gcc dot gnu.org
X-Bugzilla-Status: UNCONFIRMED
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields: cf_gcctarget cc
Message-ID: <bug-61977-4-OCz6JytfQh@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-61977-4@http.gcc.gnu.org/bugzilla/>
References: <bug-61977-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-03/txt/msg02672.txt.bz2
Content-length: 493

https://gcc.gnu.org/bugzilla/show_bug.cgi?ida977

Martin Sebor <msebor at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Target|powerpc-eabi                |powerpc*-*-*-*
                 CC|                            |msebor at gcc dot gnu.org

--- Comment #2 from Martin Sebor <msebor at gcc dot gnu.org> ---
All powerpc64*-*-*-* targets appear to be affected.

[Bug lto/65536] LTO line number information garbled

[hubicka at ucw dot cz]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #26 from Jan Hubicka <hubicka at ucw dot cz> ---
this is a proof of concept patch that makes streamer in to collect locations
into a "cache" and apply them in sorted order (looking up correct max_column
hints) at the end of handling of a given section.  It also has facility to
throw away locations of trees that are freed.
while sorting at stream in time is not cool, it does not show top in the
profiles
and memory use of the cache is actually dominated by other stuff we read,
so this seems to work quite well in practice. Main problem would be if someone
copied/used the locator before cache is applied.

This helps to get all lines and most of carrets right on firefox. I killed my
Chromium tree so can't test it there (Martin, perhaps you could try?)

I did not have much time today to test the patch.  It also saves quite a lot of
memory, about 400MB on firefox I guess.

Honza

[Bug lto/65536] LTO line number information garbled

[hubicka at ucw dot cz]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #27 from Jan Hubicka <hubicka at ucw dot cz> ---
> I have in fact considered it already, since I think it should be fairly easy
> and it can be done incrementally. However, as always in GCC, things are never
> as trivial as they seem. I just tried following
> https://gcc.gnu.org/wiki/DebuggingGCC#Debugging_LTO to see what
> dump_line_table_statistics reports before and after my patch above, but it
> doesn't print the lto1 invokation. The debugx scripts
> (https://gcc.gnu.org/ml/gcc/2004-03/msg01195.html) do not seem to work with
> lto1 (why those scripts are not in contrib/?). I feel like a new contributor
> again! :)

I simply use --save-temps --verbose and use lto1 invocation from there. It is
easy ;)
Having direct way to invoke lto1 in gdb from gcc wrapper or script may come
handy,
but it is not that hard to do by hand.

Honza

[Bug lto/65536] LTO line number information garbled

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #29 from Martin Liška <marxin at gcc dot gnu.org> ---
Created attachment 35131
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=35131&action=edit
Chrome: ODR warnings before
>From gcc-bugs-return-481560-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Wed Mar 25 09:02:16 2015
Return-Path: <gcc-bugs-return-481560-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 8553 invoked by alias); 25 Mar 2015 09:02:16 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 8479 invoked by uid 48); 25 Mar 2015 09:02:10 -0000
From: "marxin at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug lto/65536] LTO line number information garbled
Date: Wed, 25 Mar 2015 09:18:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: lto
X-Bugzilla-Version: 5.0
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: marxin at gcc dot gnu.org
X-Bugzilla-Status: UNCONFIRMED
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields:
Message-ID: <bug-65536-4-sIBKMULQed@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-65536-4@http.gcc.gnu.org/bugzilla/>
References: <bug-65536-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-03/txt/msg02704.txt.bz2
Content-length: 3010

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #28 from Martin Liška <marxin at gcc dot gnu.org> ---
(In reply to Jan Hubicka from comment #26)
> Created attachment 35130 [details]
> linemap
> 
> this is a proof of concept patch that makes streamer in to collect locations
> into a "cache" and apply them in sorted order (looking up correct max_column
> hints) at the end of handling of a given section.  It also has facility to
> throw away locations of trees that are freed.
> while sorting at stream in time is not cool, it does not show top in the
> profiles
> and memory use of the cache is actually dominated by other stuff we read,
> so this seems to work quite well in practice. Main problem would be if
> someone
> copied/used the locator before cache is applied.
> 
> This helps to get all lines and most of carrets right on firefox. I killed my
> Chromium tree so can't test it there (Martin, perhaps you could try?)

Sure.

Unfortunately, there's still ICE, which is caused by a function body load that
happens in IPA ICF:


In function ‘Create’:
lto1: internal compiler error: Segmentation fault
0x9037df crash_signal
    ../../gcc/toplev.c:383
0xed909c new_linemap
    ../../libcpp/line-map.c:265
0xed9467 linemap_add(line_maps*, lc_reason, unsigned int, char const*, unsigned
int)
    ../../libcpp/line-map.c:314
0xed98d0 linemap_line_start(line_maps*, unsigned int, unsigned int)
    ../../libcpp/line-map.c:566
0xa8b190 unpack_ts_block_value_fields
    ../../gcc/tree-streamer-in.c:414
0xa8b190 streamer_read_tree_bitfields(lto_input_block*, data_in*, tree_node*)
    ../../gcc/tree-streamer-in.c:538
0x80a791 lto_read_tree_1
    ../../gcc/lto-streamer-in.c:1197
0x80ab04 lto_read_tree
    ../../gcc/lto-streamer-in.c:1234
0x80ab04 lto_input_tree_1(lto_input_block*, data_in*, LTO_tags, unsigned int)
    ../../gcc/lto-streamer-in.c:1353
0x80ae01 lto_input_scc(lto_input_block*, data_in*, unsigned int*, unsigned
int*)
    ../../gcc/lto-streamer-in.c:1258
0x80ae64 lto_input_tree(lto_input_block*, data_in*)
    ../../gcc/lto-streamer-in.c:1368
0xe26ff1 input_gimple_stmt
    ../../gcc/gimple-streamer-in.c:186
0xe26ff1 input_bb(lto_input_block*, LTO_tags, data_in*, function*, int)
    ../../gcc/gimple-streamer-in.c:303
0x80c457 input_function
    ../../gcc/lto-streamer-in.c:992
0x80c457 lto_read_body_or_constructor
    ../../gcc/lto-streamer-in.c:1129
0x614d5d cgraph_node::get_untransformed_body()
    ../../gcc/cgraph.c:3225
0xe6e580 ipa_icf::sem_function::init()
    ../../gcc/ipa-icf.c:1167
0xe6b730 ipa_icf::sem_item_optimizer::parse_nonsingleton_classes()
    ../../gcc/ipa-icf.c:2617
0xe72b9a ipa_icf::sem_item_optimizer::execute()
    ../../gcc/ipa-icf.c:2419
0xe74386 ipa_icf_driver
    ../../gcc/ipa-icf.c:3304

I also attach ODR warnings before and after the patch.

Diff

> 
> I did not have much time today to test the patch.  It also saves quite a lot
> of
> memory, about 400MB on firefox I guess.
> 
> Honza
>From gcc-bugs-return-481562-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Wed Mar 25 09:03:24 2015
Return-Path: <gcc-bugs-return-481562-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 13313 invoked by alias); 25 Mar 2015 09:03:24 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 13255 invoked by uid 48); 25 Mar 2015 09:03:21 -0000
From: "marxin at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug lto/65536] LTO line number information garbled
Date: Wed, 25 Mar 2015 09:20:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: lto
X-Bugzilla-Version: 5.0
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: marxin at gcc dot gnu.org
X-Bugzilla-Status: UNCONFIRMED
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields: attachments.created
Message-ID: <bug-65536-4-0jfp0jzBba@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-65536-4@http.gcc.gnu.org/bugzilla/>
References: <bug-65536-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-03/txt/msg02706.txt.bz2
Content-length: 267

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #30 from Martin Liška <marxin at gcc dot gnu.org> ---
Created attachment 35132
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=35132&action=edit
Chrome: ODR warnings with patch from #c21
>From gcc-bugs-return-481563-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Wed Mar 25 09:04:00 2015
Return-Path: <gcc-bugs-return-481563-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 22609 invoked by alias); 25 Mar 2015 09:04:00 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 22549 invoked by uid 48); 25 Mar 2015 09:03:55 -0000
From: "rguenth at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug tree-optimization/64715] [5 Regression] __builtin_object_size (..., 1) fails to locate subobject
Date: Wed, 25 Mar 2015 09:37:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: tree-optimization
X-Bugzilla-Version: 5.0
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: rguenth at gcc dot gnu.org
X-Bugzilla-Status: ASSIGNED
X-Bugzilla-Priority: P1
X-Bugzilla-Assigned-To: rguenth at gcc dot gnu.org
X-Bugzilla-Target-Milestone: 5.0
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields:
Message-ID: <bug-64715-4-6crO7kCKS5@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-64715-4@http.gcc.gnu.org/bugzilla/>
References: <bug-64715-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-03/txt/msg02707.txt.bz2
Content-length: 2849

https://gcc.gnu.org/bugzilla/show_bug.cgi?idd715

--- Comment #19 from Richard Biener <rguenth at gcc dot gnu.org> ---
(In reply to Jakub Jelinek from comment #14)
> Or, as discussed on IRC we can consider MEM_REFs where first operand isn't
> just
> SSA_NAME or ADDR_EXPR of a decl, but allow also ADDR_EXPR of
> handled_component_p (with base being a decl or MEM_REF with SSA_NAME first
> operand).
>
> Or add a new tree code, like OFFSETTED_ADDR_EXPR which would be like
> ADDR_EXPR, but with integer offset address to the ADDR_EXPR.

Thinking about this some more we already have the ability to combine

 _2 = &a.b[i].c;
 _3 = _2 + 8;

by using handled-components:

  &ARRAY_REF <VIEW_CONVERT_EXPR <(__typeof__ (a.b[i].c)[])&a.b[i].c>, 8 /
__sizeof__(a.b[i].c)>

that is, we could attack this by implementing pointer arithmetic as represented
by the frontend in the way it is defined by the C standard (in terms of array
indexing).

  ptr + idx

would be lowered as

  &ARRAY_REF <VIEW_CONVERT_EXPR <(__typeof__ (*ptr) []) *ptr>, idx>

thereby also avoiding the need to apply promotion of idx to sizetype to
avoid the possible overflow of the multiplication by sizeof (*ptr).

Of course it would be nice to avoid the "awkward" VIEW_CONVERT_EXPR here,
but adding a new kind of handled-component isn't exactly non-intrusive either.

Btw, there is an alternative to the array-ref/view-convert-expr form that
is less restrictive on the actual 'idx'.  We could simply use

  COMPONENT_REF <*ptr, <NULL field-decl>, explicit-offset>

though the fact that COMPONENT_REFs have alias analysis effects and that
the lowered offset is measured in DECL_OFFSET_ALIGN units of the field-decl
makes that non-trivial to support as well.

That said, the alternative is to add a new handled-component, say,
OFFSET_VIEW_CONVERT_EXPR <type, base, offset>, that would have the
same effect as doing

  _ptr = &base + offset;
  MEM [_ptr, offset];

thus offset carries type-based alias information and 'type' carries
alignment info.  Type-based alias analysis would stop at
OFFSET_VIEW_CONVERT_EXPR as it stops now at VIEW_CONVERT_EXPRs.

Now we could also simply change VIEW_CONVERT_EXPR to take an (optional?)
offset parameter.

I like all of the above choices more than special-casing an address-only
variant for offsets.  All of the above enable us to avoid making
variable-indexed indirect references addressable like we have to do now
because of the restrictions of MEM_REF bases / offsets.  The ARRAY_REF
variant allows to handle non-constant pointer offsets as well while
the offsetted VIEW_CONVERT_EXPR would handle only constant offsets
(but allow non-"element"-size increments and arbitrary effective alias
sets on the memory reference).

We should already handle the ARRAY_REF <VIEW_CONVERT_EXPR <...> ...> form
correctly today, we just don't create it.

[Bug lto/65536] LTO line number information garbled

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #31 from Martin Liška <marxin at gcc dot gnu.org> ---
Beginning of the difference of ODR warnings:

210,211c210,211
< gen/blink/core/CSSPropertyNames.cpp:2330:0: warning: type ‘struct
stringpool_t’ violates one definition rule [-Wodr]
<  static const short lookup[] =
---
> gen/blink/core/CSSPropertyNames.cpp:1019:0: warning: type ‘struct stringpool_t’ violates one definition rule [-Wodr]
>  struct stringpool_t
213,214c213,214
< gen/blink/core/CSSValueKeywords.cpp:4309:0: note: a different type is defined
in another translation unit
<    static const short lookup[] =
---
> gen/blink/core/CSSValueKeywords.cpp:1850:0: note: a different type is defined in another translation unit
>  struct stringpool_t
216,217c216,217
< gen/blink/core/CSSPropertyNames.cpp:2330:0: note: the first difference of
corresponding definitions is field ‘stringpool_str0’
<  static const short lookup[] =
---
> gen/blink/core/CSSPropertyNames.cpp:1021:0: note: the first difference of corresponding definitions is field ‘stringpool_str0’
>      char stringpool_str0[sizeof("grid")];
219,220c219,220
< gen/blink/core/CSSValueKeywords.cpp:4309:0: note: a field of same name but
different type is defined in another translation unit
<    static const short lookup[] =
---
> gen/blink/core/CSSValueKeywords.cpp:1852:0: note: a field of same name but different type is defined in another translation unit
>      char stringpool_str0[sizeof("dot")];
230,232c230,232
< ../../../../out/Release/gen/blink/core/XPathGrammar.cpp:241:0: note: a
different type is defined in another translation unit
< ../../../../out/Release/gen/blink/core/CSSGrammar.cpp:557:0: note: the first
difference of corresponding definitions is field ‘yyvs_alloc’
< ../../../../out/Release/gen/blink/core/XPathGrammar.cpp:241:0: note: a field
of same name but different type is defined in another translation unit
---
> ../../../../out/Release/gen/blink/core/XPathGrammar.cpp:373:0: note: a different type is defined in another translation unit
> ../../../../out/Release/gen/blink/core/CSSGrammar.cpp:560:0: note: the first difference of corresponding definitions is field ‘yyvs_alloc’
> ../../../../out/Release/gen/blink/core/XPathGrammar.cpp:376:0: note: a field of same name but different type is defined in another translation unit


...
>From gcc-bugs-return-481565-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Wed Mar 25 09:15:53 2015
Return-Path: <gcc-bugs-return-481565-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 53541 invoked by alias); 25 Mar 2015 09:15:52 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 53431 invoked by uid 48); 25 Mar 2015 09:15:48 -0000
From: "vehre at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug fortran/65548] [5 Regression] gfc_conv_procedure_call
Date: Wed, 25 Mar 2015 09:57:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: fortran
X-Bugzilla-Version: unknown
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: vehre at gcc dot gnu.org
X-Bugzilla-Status: ASSIGNED
X-Bugzilla-Priority: P4
X-Bugzilla-Assigned-To: vehre at gcc dot gnu.org
X-Bugzilla-Target-Milestone: 5.0
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields: bug_status
Message-ID: <bug-65548-4-483sjaFY9l@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-65548-4@http.gcc.gnu.org/bugzilla/>
References: <bug-65548-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-03/txt/msg02709.txt.bz2
Content-length: 495

https://gcc.gnu.org/bugzilla/show_bug.cgi?ide548

vehre at gcc dot gnu.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED

--- Comment #5 from vehre at gcc dot gnu.org ---
The function call is processed w/o a scalarizer, which is quite errorprone. I
already have a fix. Now I just have get the other functionality of gfortran
back online.

[Bug lto/65536] LTO line number information garbled

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #32 from Richard Biener <rguenth at gcc dot gnu.org> ---
(In reply to Jan Hubicka from comment #16)
> > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536
> > 
> > --- Comment #2 from Richard Biener <rguenth at gcc dot gnu.org> ---
> > The main issue with LTO is that it re-creates a combined linemap but in (most
> > of the time) quite awkward ordering (simply registering random-ordered
> > file:line:column entries by switching files/lines "appropriately").
> > 
> > I've argued that it would be better to stream those file:line:columns
> > separately so we can "sort" them before handing them over to libcpp for
> > linemap re-creation.
> 
> Yep, we discussed this few times.  One issue is when we sort them. 
> We can keep track of location as they are streamed out (and directly assign
> them new IDs in the awkward ordering as they come) and then produce the
> separate linemap section that would be read at once by WPA, sorted and
> fed into linemaps.
> 
> Sorting at compile time is bit tricky as we probably do not want to patch
> existing pickled tree stream with the new locator IDs in the sorted sequence
> and we do not really know what locations we will need ahead of time.

Well, the idea was to stream another index representing the sorted order
so we can sort and merge at WPA in O(n) and re-map the index on stream-in
time.  I don't think we want to sort at WPA time.

> Other easier to implement idea for GCC 5 may be to simply collect locaiton
> info and pointers to trees per SCC component andonly if it survived merging
> feed it into linemap and in memory patch the new trees. This still should
> save quite some memory given that most of trees read are discarded by merging
> and may be quite easy to implement.
> 
> Martin, the ICE by my patch is caused by libcpp getting out of memory?

Btw, we don't have to fix this for GCC 5.  We can of course backport a
good solution later.

[Bug lto/65536] LTO line number information garbled

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #33 from Richard Biener <rguenth at gcc dot gnu.org> ---
(In reply to Jan Hubicka from comment #26)
> Created attachment 35130 [details]
> linemap
> 
> this is a proof of concept patch that makes streamer in to collect locations
> into a "cache" and apply them in sorted order (looking up correct max_column
> hints) at the end of handling of a given section.  It also has facility to
> throw away locations of trees that are freed.
> while sorting at stream in time is not cool, it does not show top in the
> profiles
> and memory use of the cache is actually dominated by other stuff we read,
> so this seems to work quite well in practice. Main problem would be if
> someone
> copied/used the locator before cache is applied.
> 
> This helps to get all lines and most of carrets right on firefox. I killed my
> Chromium tree so can't test it there (Martin, perhaps you could try?)
> 
> I did not have much time today to test the patch.  It also saves quite a lot
> of
> memory, about 400MB on firefox I guess.

Looks sensible to me (apart from sorting at WPA time and missing TLC like
adding function/code comments and removing no-op hunks).

Richard.

> Honza

[Bug lto/65536] LTO line number information garbled

[manu at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #34 from Manuel López-Ibáñez <manu at gcc dot gnu.org> ---
(In reply to Martin Liška from comment #30)
> Created attachment 35132 [details]
> Chrome: ODR warnings with patch from #c21

@Martin:

The previous comment where the ICE is mentioned seems to refer to Jan's latest
patch (with the cache), but this comment appears to refer to my patch in
comment 21. Is that right? 

I don't see how you can get that backtrace with Jan's patch: linemap_line_start
should only be called when applying the location cache, no?

@Jan:
I think:

+      linemap_line_start (line_table, loc.line,
+                  MAX (81, loc.col + 1))

should simply be:

+      linemap_line_start (line_table, loc.line,
+                  max + 1)


For what is worth, I think your patch should already help a lot in GCC 5, it
seems a move in the right direction. Perhaps the WPA sorting time is offset by
the memory savings and lower number of maps, which should reduce lookup times.

It would really be interesting to compute the sum of
dump_line_table_statistics() for each input file before stream out and for the
LTO line_table after stream in. It will give you some idea of how much you can
expect to save in streamed out data and LTO memory by streaming out the
line_tables directly.
>From gcc-bugs-return-481600-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Wed Mar 25 13:14:33 2015
Return-Path: <gcc-bugs-return-481600-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 111180 invoked by alias); 25 Mar 2015 13:14:33 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 105548 invoked by uid 48); 25 Mar 2015 13:14:24 -0000
From: "rguenth at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug ipa/65076] [5 Regression] 16% tramp3d-v4.cpp compile time regression
Date: Wed, 25 Mar 2015 13:33:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: ipa
X-Bugzilla-Version: 5.0
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: rguenth at gcc dot gnu.org
X-Bugzilla-Status: UNCONFIRMED
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: 5.0
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields:
Message-ID: <bug-65076-4-ISe2bPxlRG@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-65076-4@http.gcc.gnu.org/bugzilla/>
References: <bug-65076-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-03/txt/msg02744.txt.bz2
Content-length: 2012

https://gcc.gnu.org/bugzilla/show_bug.cgi?ide076

--- Comment #19 from Richard Biener <rguenth at gcc dot gnu.org> ---
I've meanwhile disabled pass_chkp_instrumentation_passes by default.

The fixup_cfg in pass_build_ssa_passes doesn't do anything on tramp3d
(and bootstrap/testing shows no issue removing it either).

Adding a local_pure_const to the end of build_ssa_passes unfortunately
makes us ICE in cgraph node verification during early inlining (we have
a dangling call-stmt without a BB on a cgraph edge).  Not sure how that
happens as we fixup the cfg and rebuild cgraph edges before early inlining.
Looks like re-building cgraph edges fixes that.

This brings down the number of clobbers and stmts to 4.9 level.

Index: gcc/passes.def
==================================================================--- gcc/passes.def      (revision 221660)
+++ gcc/passes.def      (working copy)
@@ -54,10 +54,12 @@ along with GCC; see the file COPYING3.
   NEXT_PASS (pass_build_ssa_passes);
   PUSH_INSERT_PASSES_WITHIN (pass_build_ssa_passes)
       NEXT_PASS (pass_fixup_cfg);
+      NEXT_PASS (pass_rebuild_cgraph_edges);
       NEXT_PASS (pass_init_datastructures);
       NEXT_PASS (pass_build_ssa);
       NEXT_PASS (pass_ubsan);
       NEXT_PASS (pass_early_warn_uninitialized);
+      NEXT_PASS (pass_local_pure_const);
   POP_INSERT_PASSES ()

   NEXT_PASS (pass_chkp_instrumentation_passes);

as opposed to the also possible patch (which doesn't change anything but
compile-time):

Index: gcc/passes.def
==================================================================--- gcc/passes.def      (revision 221657)
+++ gcc/passes.def      (working copy)
@@ -53,7 +53,6 @@ along with GCC; see the file COPYING3.
   NEXT_PASS (pass_ipa_chkp_early_produce_thunks);
   NEXT_PASS (pass_build_ssa_passes);
   PUSH_INSERT_PASSES_WITHIN (pass_build_ssa_passes)
-      NEXT_PASS (pass_fixup_cfg);
       NEXT_PASS (pass_init_datastructures);
       NEXT_PASS (pass_build_ssa);
       NEXT_PASS (pass_ubsan);

[Bug lto/65536] LTO line number information garbled

[hubicka at ucw dot cz]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #35 from Jan Hubicka <hubicka at ucw dot cz> ---
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536
> 
> --- Comment #31 from Martin Liška <marxin at gcc dot gnu.org> ---
> Beginning of the difference of ODR warnings:
> 
> 210,211c210,211
> < gen/blink/core/CSSPropertyNames.cpp:2330:0: warning: type ???struct
> stringpool_t??? violates one definition rule [-Wodr]
> <  static const short lookup[] =
> ---
> > gen/blink/core/CSSPropertyNames.cpp:1019:0: warning: type ???struct stringpool_t??? violates one definition rule [-Wodr]
> >  struct stringpool_t
> 213,214c213,214
> < gen/blink/core/CSSValueKeywords.cpp:4309:0: note: a different type is defined
> in another translation unit
> <    static const short lookup[] =
> ---
> > gen/blink/core/CSSValueKeywords.cpp:1850:0: note: a different type is defined in another translation unit
> >  struct stringpool_t
> 216,217c216,217
> < gen/blink/core/CSSPropertyNames.cpp:2330:0: note: the first difference of
> corresponding definitions is field ???stringpool_str0???
> <  static const short lookup[] =

This looks good, the messup in between stringppol_t and lookup was first one I
noticed.

Honza
>From gcc-bugs-return-481667-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Wed Mar 25 18:00:33 2015
Return-Path: <gcc-bugs-return-481667-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 129779 invoked by alias); 25 Mar 2015 18:00:33 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 129687 invoked by uid 48); 25 Mar 2015 18:00:23 -0000
From: "rainer@emrich-ebersheim.de" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug target/65560] New: pr24683.c:11:1: internal compiler error: in extract_insn, at recog.c:2343
Date: Wed, 25 Mar 2015 18:17:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: new
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: target
X-Bugzilla-Version: 5.0
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: rainer@emrich-ebersheim.de
X-Bugzilla-Status: UNCONFIRMED
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields: bug_id short_desc product version bug_status bug_severity priority component assigned_to reporter
Message-ID: <bug-65560-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-03/txt/msg02811.txt.bz2
Content-length: 3405

https://gcc.gnu.org/bugzilla/show_bug.cgi?ide560

            Bug ID: 65560
           Summary: pr24683.c:11:1: internal compiler error: in
                    extract_insn, at recog.c:2343
           Product: gcc
           Version: 5.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: target
          Assignee: unassigned at gcc dot gnu.org
          Reporter: rainer@emrich-ebersheim.de

Running the gcc testsuite I get this ICE:

Executing on host:
/opt/devel/SCRATCH/tmp.5jnZ8G4weh/gcc-5.0.0/gcc-5.0.0/gcc/xgcc
-B/opt/devel/SCRATCH/tmp.5jnZ8G4weh/gcc-5.0.0/gcc-5.0.0/gcc/
/opt/devel/gnu/src/gcc-mingw-w64/gcc-5.0.0/gcc/testsuite/gcc.dg/pr24683.c
-fno-diagnostics-show-caret -fdiagnostics-color=never   -O2 -fPIC -S   -o
pr24683.s    (timeout = 300)
spawn /opt/devel/SCRATCH/tmp.5jnZ8G4weh/gcc-5.0.0/gcc-5.0.0/gcc/xgcc
-B/opt/devel/SCRATCH/tmp.5jnZ8G4weh/gcc-5.0.0/gcc-5.0.0/gcc/
/opt/devel/gnu/src/gcc-mingw-w64/gcc-5.0.0/gcc/testsuite/gcc.dg/pr24683.c
-fno-diagnostics-show-caret -fdiagnostics-color=never -O2 -fPIC -S -o pr24683.s
D:/opt/devel/gnu/src/gcc-mingw-w64/gcc-5.0.0/gcc/testsuite/gcc.dg/pr24683.c:1:0:
warning: -fPIC ignored for target (all code is position independent)
D:/opt/devel/gnu/src/gcc-mingw-w64/gcc-5.0.0/gcc/testsuite/gcc.dg/pr24683.c: In
function 'final':
D:/opt/devel/gnu/src/gcc-mingw-w64/gcc-5.0.0/gcc/testsuite/gcc.dg/pr24683.c:11:1:
error: unrecognizable insn:
(insn 8 7 9 4 (set (reg/f:DI 89 [ block_lsm.4 ])
        (plus:DI (symbol_ref/f:DI ("*.LC0") [flags 0x2] <var_decl
000000000d8b7c60 *.LC0>)
            (reg:DI 93)))
D:/opt/devel/gnu/src/gcc-mingw-w64/gcc-5.0.0/gcc/testsuite/gcc.dg/pr24683.c:10
-1
     (expr_list:REG_EQUAL (const:DI (plus:DI (symbol_ref/f:DI ("*.LC0") [flags
0x2] <var_decl 000000000d8b7c60 *.LC0>)
                (const_int 4294967233 [0xffffffc1])))
        (nil)))
D:/opt/devel/gnu/src/gcc-mingw-w64/gcc-5.0.0/gcc/testsuite/gcc.dg/pr24683.c:11:1:
internal compiler error: in extract_insn, at recog.c:2343
libbacktrace could not find executable to open
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://gcc.gnu.org/bugs.html> for instructions.
compiler exited with status 1
output is:
D:/opt/devel/gnu/src/gcc-mingw-w64/gcc-5.0.0/gcc/testsuite/gcc.dg/pr24683.c:1:0:
warning: -fPIC ignored for target (all code is position independent)
D:/opt/devel/gnu/src/gcc-mingw-w64/gcc-5.0.0/gcc/testsuite/gcc.dg/pr24683.c: In
function 'final':
D:/opt/devel/gnu/src/gcc-mingw-w64/gcc-5.0.0/gcc/testsuite/gcc.dg/pr24683.c:11:1:
error: unrecognizable insn:
(insn 8 7 9 4 (set (reg/f:DI 89 [ block_lsm.4 ])
        (plus:DI (symbol_ref/f:DI ("*.LC0") [flags 0x2] <var_decl
000000000d8b7c60 *.LC0>)
            (reg:DI 93)))
D:/opt/devel/gnu/src/gcc-mingw-w64/gcc-5.0.0/gcc/testsuite/gcc.dg/pr24683.c:10
-1
     (expr_list:REG_EQUAL (const:DI (plus:DI (symbol_ref/f:DI ("*.LC0") [flags
0x2] <var_decl 000000000d8b7c60 *.LC0>)
                (const_int 4294967233 [0xffffffc1])))
        (nil)))
D:/opt/devel/gnu/src/gcc-mingw-w64/gcc-5.0.0/gcc/testsuite/gcc.dg/pr24683.c:11:1:
internal compiler error: in extract_insn, at recog.c:2343
libbacktrace could not find executable to open
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://gcc.gnu.org/bugs.html> for instructions.

FAIL: gcc.dg/pr24683.c (internal compiler error)

[Bug lto/65536] LTO line number information garbled

[hubicka at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

Jan Hubicka <hubicka at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
   Last reconfirmed|                            |2015-03-25
     Ever confirmed|0                           |1

--- Comment #36 from Jan Hubicka <hubicka at gcc dot gnu.org> ---
Manuel,
I returned back looking for reason lines are going out wrong when we get short
on locators.  I do not understand the following code:

  if (line_delta < 0
      || (line_delta > 10
          && line_delta * ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map) > 1000)
      || (max_column_hint >= (1U << ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map)))
      || (max_column_hint <= 80
          && ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map) >= 10)
      || (highest > 0x60000000
          && (set->max_column_hint || highest > 0x70000000)))
    add_map = true;
  else
    max_column_hint = set->max_column_hint;

here we seem to decide about adding new map and we always do that when we are
running tight on locators...

then
  if (add_map)
    {
      int column_bits;
      if (max_column_hint > 100000 || highest > 0x60000000)
        {
          /* If the column number is ridiculous or we've allocated a huge
             number of source_locations, give up on column numbers. */
          max_column_hint = 0;
          if (highest > 0x70000000)
            return 0;

I will add abort to highest and check if we reach it for chromium.
We probably can starting slowing down and allocating new linemap for every new
line, not try the heuristics with 1000 above.
However...

          column_bits = 0;
        }
      else
        {
          column_bits = 7;
          while (max_column_hint >= (1U << column_bits))
            column_bits++;
          max_column_hint = 1U << column_bits;
        }
      /* Allocate the new line_map.  However, if the current map only has a
         single line we can sometimes just increase its column_bits instead. */
      if (line_delta < 0
          || last_line != ORDINARY_MAP_STARTING_LINE_NUMBER (map)
          || SOURCE_COLUMN (map, highest) >= (1U << column_bits))
        map = (struct line_map *) linemap_add (set, LC_RENAME,
                                               ORDINARY_MAP_IN_SYSTEM_HEADER_P
                                               (map),
                                               ORDINARY_MAP_FILE_NAME (map),
                                               to_line);
      ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map) = column_bits;
      r = (MAP_START_LOCATION (map)
           + ((to_line - ORDINARY_MAP_STARTING_LINE_NUMBER (map))
              << column_bits));

Here we seem to sometimes affect ORDINARY_MAP_NUMBER_OF_COLUMN_BITS of an
existing line map.  How that can work?  We already have locators that do
depends on the previous COLUMN_BITS value.
I would expect the ORDINARY_MPA_NUMBER_OF_COLUMN_BITS set to be part of the
conditional and the computation bellow use map's column_bits.
Also I think the conditoinal guarding creation may want to check how far r will
be bumped and force creation when we are running short of locators.

Honza

[Bug lto/65536] LTO line number information garbled

[manu at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #37 from Manuel López-Ibáñez <manu at gcc dot gnu.org> ---
(In reply to Jan Hubicka from comment #36)
> Here we seem to sometimes affect ORDINARY_MAP_NUMBER_OF_COLUMN_BITS of an
> existing line map.  How that can work?  We already have locators that do
> depends on the previous COLUMN_BITS value.

The condition is checking that this map has more than one line

          || last_line != ORDINARY_MAP_STARTING_LINE_NUMBER (map)

or we have assigned a location_t for a column larger than what we can address
if we reduce the column bits of this map

          || SOURCE_COLUMN (map, highest) >= (1U << column_bits))

If none of those are true, we can reduce the column bits without affecting an
already assigned columns, because the location_t value for those columns is
simply the location_t value of the line + column.


> I would expect the ORDINARY_MPA_NUMBER_OF_COLUMN_BITS set to be part of the
> conditional and the computation bellow use map's column_bits.
> Also I think the conditoinal guarding creation may want to check how far r
> will be bumped and force creation when we are running short of locators.

I don't understand. Which conditional? Force creating of what?

In fact, when we are running short of locators, 

      || (highest > 0x60000000
          && (set->max_column_hint || highest > 0x70000000)))

you see that add_map=true; then later

      if (max_column_hint > 100000 || highest > 0x60000000)
        {
          /* If the column number is ridiculous or we've allocated a huge
             number of source_locations, give up on column numbers. */
          max_column_hint = 0;
          if (highest > 0x70000000)
            return 0;
          column_bits = 0; 

which will give up on column numbers and force creating a new map that does not
track columns (since SOURCE_COLUMN (map, highest) >= (1U << column_bits) will
be always true), or if we are really short on source_locations, then give up
completely and return UNKNOWN_LOCATION.

But now that I think about it. If linemap_star_line returns UNKNOWN_LOCATION
because set->highest_location > 0x70000000, but then you call
linemap_position_for_column and set->highest_location < 0xC0000000, then
linemap_line_start will return 0 again, thus linemap_position_for_column will
return 0 + to_column and that would be bad. Can you follow me? Do you also see
a potential bug there?
>From gcc-bugs-return-481724-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Wed Mar 25 21:40:37 2015
Return-Path: <gcc-bugs-return-481724-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 1717 invoked by alias); 25 Mar 2015 21:40:36 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 1694 invoked by uid 48); 25 Mar 2015 21:40:33 -0000
From: "rainer@emrich-ebersheim.de" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug target/65573] New: 13908.C:20:33: internal compiler error: in cp_build_addr_expr_1, at cp/typeck.c:5527
Date: Wed, 25 Mar 2015 22:50:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: new
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: target
X-Bugzilla-Version: 5.0
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: rainer@emrich-ebersheim.de
X-Bugzilla-Status: UNCONFIRMED
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields: bug_id short_desc product version bug_status bug_severity priority component assigned_to reporter
Message-ID: <bug-65573-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-03/txt/msg02868.txt.bz2
Content-length: 3280

https://gcc.gnu.org/bugzilla/show_bug.cgi?ide573

            Bug ID: 65573
           Summary: 13908.C:20:33: internal compiler error: in
                    cp_build_addr_expr_1, at cp/typeck.c:5527
           Product: gcc
           Version: 5.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: target
          Assignee: unassigned at gcc dot gnu.org
          Reporter: rainer@emrich-ebersheim.de

Running the g++ testsuite I get the following ICE:

Executing on host:
/opt/devel/SCRATCH/tmp.5jnZ8G4weh/gcc-5.0.0/gcc-5.0.0/gcc/testsuite/g++/../../xg++
-B/opt/devel/SCRATCH/tmp.5jnZ8G4weh/gcc-5.0.0/gcc-5.0.0/gcc/testsuite/g++/../../
 -fno-diagnostics-show-caret -fdiagnostics-color=never  -nostdinc++
-I/opt/devel/SCRATCH/tmp.5jnZ8G4weh/gcc-5.0.0/gcc-5.0.0/x86_64-w64-mingw32/libstdc++-v3/include/x86_64-w64-mingw32
-I/opt/devel/SCRATCH/tmp.5jnZ8G4weh/gcc-5.0.0/gcc-5.0.0/x86_64-w64-mingw32/libstdc++-v3/include
-I/opt/devel/gnu/src/gcc-mingw-w64/gcc-5.0.0/libstdc++-v3/libsupc++
-I/opt/devel/gnu/src/gcc-mingw-w64/gcc-5.0.0/libstdc++-v3/include/backward
-I/opt/devel/gnu/src/gcc-mingw-w64/gcc-5.0.0/libstdc++-v3/testsuite/util
-fmessage-length=0  -std=c++98  -pedantic-errors -Wno-long-long  -c   -o
13908.o
/opt/devel/gnu/src/gcc-mingw-w64/gcc-5.0.0/gcc/testsuite/g++.old-deja/g++.benjamin/13908.C
   (timeout = 300)
spawn
/opt/devel/SCRATCH/tmp.5jnZ8G4weh/gcc-5.0.0/gcc-5.0.0/gcc/testsuite/g++/../../xg++
-B/opt/devel/SCRATCH/tmp.5jnZ8G4weh/gcc-5.0.0/gcc-5.0.0/gcc/testsuite/g++/../../
-fno-diagnostics-show-caret -fdiagnostics-color=never -nostdinc++
-I/opt/devel/SCRATCH/tmp.5jnZ8G4weh/gcc-5.0.0/gcc-5.0.0/x86_64-w64-mingw32/libstdc++-v3/include/x86_64-w64-mingw32
-I/opt/devel/SCRATCH/tmp.5jnZ8G4weh/gcc-5.0.0/gcc-5.0.0/x86_64-w64-mingw32/libstdc++-v3/include
-I/opt/devel/gnu/src/gcc-mingw-w64/gcc-5.0.0/libstdc++-v3/libsupc++
-I/opt/devel/gnu/src/gcc-mingw-w64/gcc-5.0.0/libstdc++-v3/include/backward
-I/opt/devel/gnu/src/gcc-mingw-w64/gcc-5.0.0/libstdc++-v3/testsuite/util
-fmessage-length=0 -std=c++98 -pedantic-errors -Wno-long-long -c -o 13908.o
/opt/devel/gnu/src/gcc-mingw-w64/gcc-5.0.0/gcc/testsuite/g++.old-deja/g++.benjamin/13908.C
D:/opt/devel/gnu/src/gcc-mingw-w64/gcc-5.0.0/gcc/testsuite/g++.old-deja/g++.benjamin/13908.C:
In function 'void bar(chile*, pmf)':
D:/opt/devel/gnu/src/gcc-mingw-w64/gcc-5.0.0/gcc/testsuite/g++.old-deja/g++.benjamin/13908.C:20:33:
internal compiler error: in cp_build_addr_expr_1, at cp/typeck.c:5527
libbacktrace could not find executable to open
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://gcc.gnu.org/bugs.html> for instructions.
compiler exited with status 1
output is:
D:/opt/devel/gnu/src/gcc-mingw-w64/gcc-5.0.0/gcc/testsuite/g++.old-deja/g++.benjamin/13908.C:
In function 'void bar(chile*, pmf)':
D:/opt/devel/gnu/src/gcc-mingw-w64/gcc-5.0.0/gcc/testsuite/g++.old-deja/g++.benjamin/13908.C:20:33:
internal compiler error: in cp_build_addr_expr_1, at cp/typeck.c:5527
libbacktrace could not find executable to open
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://gcc.gnu.org/bugs.html> for instructions.

FAIL: g++.old-deja/g++.benjamin/13908.C  -std=c++98 (internal compiler error)

[Bug lto/65536] LTO line number information garbled

[manu at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #39 from Manuel López-Ibáñez <manu at gcc dot gnu.org> ---
(In reply to Jan Hubicka from comment #36)
> Manuel,
> I returned back looking for reason lines are going out wrong when we get
> short on locators.  I do not understand the following code:

I just noticed that linemap_add increases the highest_location by one everytime
it is called, but there is no overflow check there. Thus, the source_locations
may indeed overflow.
>From gcc-bugs-return-481732-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Wed Mar 25 22:54:41 2015
Return-Path: <gcc-bugs-return-481732-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 29502 invoked by alias); 25 Mar 2015 22:54:41 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 29451 invoked by uid 48); 25 Mar 2015 22:54:37 -0000
From: "redi at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug c++/65547] Gcc not giving preprocessor error for invalid invocation of variadic macro
Date: Wed, 25 Mar 2015 23:36:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: c++
X-Bugzilla-Version: 4.9.2
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: redi at gcc dot gnu.org
X-Bugzilla-Status: RESOLVED
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields:
Message-ID: <bug-65547-4-3FNBBX6gC7@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-65547-4@http.gcc.gnu.org/bugzilla/>
References: <bug-65547-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-03/txt/msg02876.txt.bz2
Content-length: 1604

https://gcc.gnu.org/bugzilla/show_bug.cgi?ide547

--- Comment #8 from Jonathan Wakely <redi at gcc dot gnu.org> ---
(In reply to Edward Diener from comment #7)
> So the GNU extension in this case is that the programmer does not have to
> pass an argument when a variadic parameter is used in a macro ?

Yes, as documented at
https://gcc.gnu.org/onlinedocs/gcc-4.9.2/gcc/Variadic-Macros.html

> That would
> not even bother me if there were a compiler flag which allowed it. But you
> appear to be telling me that this non-C++ is by default.

Yes, did you read the link I gave explaining the semantics of -pedantic?

> OK, how do I tell
> gcc to actually follow the C++ standard ? Do I pass '-pedantic' for that ?

Yes, as I already said in comment 5, in the text you quoted. And to make it an
error not a warning use -pedantic-errors.

> I realize gcc is used by many programmers and is loath to change the way it
> has done things so as not to upset long-time users of the compiler.
> Nonetheless I am going to suggest that the default should always be
> conformance to the C++ standard, and that GNU extensions should be turned on
> with some compiler option.

As it states in the manual (again, see the link in comment 5), if you write
valid ISO C++ programs using variadic macros then you will never notice this
extension. Your valid uses are accepted as required, and as an extension some
invalid ones are also silently accepted unless you use -pedantic.

This is consistent, well-documented, and what many users expect and prefer.

If you prefer something different, there's an option for that.

[Bug lto/65536] LTO line number information garbled

[hubicka at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #40 from Jan Hubicka <hubicka at gcc dot gnu.org> ---
Manuel,
the following way you get the lto1 invocation:

jan@linux-qos1:~> gcc t.c -flto -O2 -c
jan@linux-qos1:~> gcc t.o -flto -O2 --verbose -save-temps 2>&1 | grep lto1
 /usr/lib64/gcc/x86_64-suse-linux/4.8/lto1 -quiet -dumpbase t.o -mtune=generic
-march=x86-64 -mtune=generic -march=x86-64 -auxbase t -O2 -version
-fltrans-output-list=/tmp/ccbfSrzR.ltrans.out -fwpa -fresolution=t.res
@/tmp/ccSRKmvR
 /usr/lib64/gcc/x86_64-suse-linux/4.8/lto1 -quiet -dumpbase ccbfSrzR.ltrans0.o
-mtune=generic -march=x86-64 -mtune=generic -march=x86-64 -auxbase-strip
/tmp/ccbfSrzR.ltrans0.ltrans.o -O2 -version
-fltrans-output-list=/tmp/ccbfSrzR.ltrans.out -fltrans @/tmp/cccTxW0P -o
ccbfSrzR.ltrans0.s

For proper LTO you need a linker with plugin support check:

jan@linux-qos1:~> ld --help | grep plugin
  -plugin PLUGIN              Load named plugin
  -plugin-opt ARG             Send arg to last-loaded plugin

There is a path in linemap_add_line that does not increase locator, but it is
bit bogus. I am playing with patch to improve this.

[Bug lto/65536] LTO line number information garbled

[manu at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #41 from Manuel López-Ibáñez <manu at gcc dot gnu.org> ---
(In reply to Manuel López-Ibáñez from comment #37)
> But now that I think about it. If linemap_star_line returns UNKNOWN_LOCATION
> because set->highest_location > 0x70000000, but then you call
> linemap_position_for_column and set->highest_location < 0xC0000000, then
> linemap_line_start will return 0 again, thus linemap_position_for_column
> will return 0 + to_column and that would be bad. Can you follow me? Do you
> also see a potential bug there?

Ignore the above, I just noticed that linemap_position_for_column tests for
0xC000000 not for 0xC0000000, thus the above cannot happen.
>From gcc-bugs-return-481743-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Wed Mar 25 23:54:16 2015
Return-Path: <gcc-bugs-return-481743-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 114796 invoked by alias); 25 Mar 2015 23:54:16 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 114563 invoked by uid 48); 25 Mar 2015 23:54:13 -0000
From: "pinskia at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug target/65574] assembler options don't match predefined macros
Date: Thu, 26 Mar 2015 01:24:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: target
X-Bugzilla-Version: 5.0
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: pinskia at gcc dot gnu.org
X-Bugzilla-Status: RESOLVED
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields: bug_status resolution
Message-ID: <bug-65574-4-X5Xcjv8FJv@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-65574-4@http.gcc.gnu.org/bugzilla/>
References: <bug-65574-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-03/txt/msg02887.txt.bz2
Content-length: 503

https://gcc.gnu.org/bugzilla/show_bug.cgi?ide574

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|---                         |DUPLICATE

--- Comment #1 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Dup of bug 65341.

*** This bug has been marked as a duplicate of bug 65341 ***

[Bug lto/65536] LTO line number information garbled

[hubicka at ucw dot cz]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #42 from Jan Hubicka <hubicka at ucw dot cz> ---
Hi,
I read linemap_line_start and I think I noticed few issues with respect
to overflows and lines being added randomly.

1) line_delta is computed as to_line SOURCE_LINE (map, set->highest_line)
   I think the last inserted line is not very releavnt.  What we care about is
   the base of the last location and to keep thing dense how much we are
   stretching the value range from highest inserted element (inserting into
middle
   is cheap).

   For this reason I added base_line_delta and changed line_delta to be
   to_line - SOURCE_LINE (map, set->highest_location).

   Because things go in randomly, highest_line, which really is last inserted
   line, may be somewhere in between.
2) (line_delta > 10 && line_delta * ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map) >
1000)
   ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map) is in range 7... 15, so it never
   gets high enough to make this conditional trigger.  I changed it to:

      || line_delta > 1000
      || (line_delta << ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map)) > 1000

   I.e. we do not want to skip more than 1000 unused entries since highest
   inserted location.

3) (max_column_hint <= 80 && ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map) >= 10)
   seems to intend to reduce the column range when it is no longer needed.
   Again, this is not really good idea when line inserted is not last.

4) the code deciding whether to do reuse seems worng:
      if (line_delta < 0
      || last_line != ORDINARY_MAP_STARTING_LINE_NUMBER (map)
      || SOURCE_COLUMN (map, highest) >= (1U << column_bits))

   line_delta really should be base_line_delta, we do not need to give up
   when map's line is 1, SOURCE_LINE (map, set->highest_line) is 5
   and we are requested to switch to line 3.

   Second last_line != ORDINARY_MAP_STARTING_LINE_NUMBER (map) tests whether
   location has only one line that does not work (at least with my changes)
   because we may switch to next line and back.

   This conditoinal also seems to be completely missing hanlding of overflows.

The following patch makes all line info and all but one carret to to be right
on chromium warnings

    * line-map.c (linemap_line_start): Correct overflow tests.
Index: line-map.c
===================================================================
--- line-map.c    (revision 221568)
+++ line-map.c    (working copy)
@@ -519,25 +519,38 @@ linemap_line_start (struct line_maps *se
   struct line_map *map = LINEMAPS_LAST_ORDINARY_MAP (set);
   source_location highest = set->highest_location;
   source_location r;
-  linenum_type last_line =
-    SOURCE_LINE (map, set->highest_line);
-  int line_delta = to_line - last_line;
+  int base_line_delta = to_line - ORDINARY_MAP_STARTING_LINE_NUMBER (map);
+  int line_delta = to_line - SOURCE_LINE (map, set->highest_location);
   bool add_map = false;

-  if (line_delta < 0
-      || (line_delta > 10
-      && line_delta * ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map) > 1000)
-      || (max_column_hint >= (1U << ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map)))
+  /* Single MAP entry can be used to encode multiple source lines.
+     Look for situations when this is impossible or undesriable.  */
+  if (base_line_delta < 0
+      /* We want to keep maps resonably dense, so do not increase the range
+     of this linemap entry by more than 1000.  */
+      || line_delta > 1000
+      || (line_delta << ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map)) > 1000
+      /* If the max column is out of range and we are still not dropping line
+     info.  */
+      || (max_column_hint >= (1U << ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map))
+      && highest < 0x60000000)
+      /* If the prevoius line was long.  Ignore this problem is we already
+     re-used the map for lines with greater indexes.  */
       || (max_column_hint <= 80
-      && ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map) >= 10)
+      && ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map) >= 10 && line_delta > 0)
+      /* If we are just started running out of locations (which makes us to
drop
+     column info), but current line map still has column info, create fresh
+     one.  */
       || (highest > 0x60000000
-      && (set->max_column_hint || highest > 0x70000000)))
+      && (ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map)
+          || highest > 0x70000000)))
     add_map = true;
   else
     max_column_hint = set->max_column_hint;
   if (add_map)
     {
       int column_bits;
+      bool reuse_map = true;
       if (max_column_hint > 100000 || highest > 0x60000000)
     {
       /* If the column number is ridiculous or we've allocated a huge
@@ -554,11 +567,38 @@ linemap_line_start (struct line_maps *se
         column_bits++;
       max_column_hint = 1U << column_bits;
     }
+
       /* Allocate the new line_map.  However, if the current map only has a
      single line we can sometimes just increase its column_bits instead. */
-      if (line_delta < 0
-      || last_line != ORDINARY_MAP_STARTING_LINE_NUMBER (map)
-      || SOURCE_COLUMN (map, highest) >= (1U << column_bits))
+      if (base_line_delta < 0 || base_line_delta != line_delta
+      /* If we just started running out of locators, but current map still
+         has column info, do not reuse it.  */
+          || (highest > 0x60000000
+          && ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map)
+          && base_line_delta)
+      /* If the line delta is too large.  */
+      || line_delta > 1000)
+    reuse_map = false;
+
+      /* When reusing, we must be sure that column_bits is high enough
+     for already recorded locations.  */
+      if (reuse_map)
+    {
+      unsigned int max_column = SOURCE_COLUMN (map, highest) + 1;
+      int combined_column_bits = column_bits;
+
+      while (max_column >= (1U << combined_column_bits))
+        combined_column_bits++;
+
+      if ((line_delta << combined_column_bits) > 1000)
+        reuse_map = false;
+      else
+        {
+          column_bits = combined_column_bits;
+          max_column_hint = 1U << combined_column_bits;
+        }
+    }
+      if (!reuse_map)
     map = (struct line_map *) linemap_add (set, LC_RENAME,
                            ORDINARY_MAP_IN_SYSTEM_HEADER_P
                            (map),
@@ -609,6 +649,9 @@ linemap_position_for_column (struct line
     {
       struct line_map *map = LINEMAPS_LAST_ORDINARY_MAP (set);
       r = linemap_line_start (set, SOURCE_LINE (map, r), to_column + 50);
+      /* We just got to overflow; disable column numbers.  */
+      if (to_column >= set->max_column_hint)
+        return r;
     }
     }
   r = r + to_column;

[Bug lto/65536] LTO line number information garbled

[manu at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #43 from Manuel López-Ibáñez <manu at gcc dot gnu.org> ---
(In reply to Jan Hubicka from comment #40)
> Manuel,
> the following way you get the lto1 invocation:

Ah, ok, it seems to work now. It just takes ages to print the lto1 line and it
get printed way after the lto1 process is running already.

> For proper LTO you need a linker with plugin support check:

That seems correct. Thanks.

> There is a path in linemap_add_line that does not increase locator, but it
> is bit bogus. I am playing with patch to improve this.

Which path?

What for sure happens is that when linemap_start_line starts returning 0, then
linemap_postion_for_column will always return the same location until
linemap_add is called directly (because of a change of file in the case of
LTO).
>From gcc-bugs-return-481748-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Thu Mar 26 00:19:37 2015
Return-Path: <gcc-bugs-return-481748-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 40231 invoked by alias); 26 Mar 2015 00:19:37 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 25200 invoked by uid 55); 26 Mar 2015 00:19:29 -0000
From: "hubicka at ucw dot cz" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug lto/65536] LTO line number information garbled
Date: Thu, 26 Mar 2015 02:51:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: lto
X-Bugzilla-Version: 5.0
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: hubicka at ucw dot cz
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields:
Message-ID: <bug-65536-4-xADJWYGNIC@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-65536-4@http.gcc.gnu.org/bugzilla/>
References: <bug-65536-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-03/txt/msg02892.txt.bz2
Content-length: 1189

https://gcc.gnu.org/bugzilla/show_bug.cgi?ide536

--- Comment #44 from Jan Hubicka <hubicka at ucw dot cz> ---
>
> Ah, ok, it seems to work now. It just takes ages to print the lto1 line and it
> get printed way after the lto1 process is running already.

Yep, really anoying property that the stderr output is all buffered and output
only at once.  Something to do with libiberty.

> > There is a path in linemap_add_line that does not increase locator, but it
> > is bit bogus. I am playing with patch to improve this.
>
> Which path?
>
> What for sure happens is that when linemap_start_line starts returning 0, then
> linemap_postion_for_column will always return the same location until
> linemap_add is called directly (because of a change of file in the case of
> LTO).

Ah yes, you are right, it returns 0. I think this is bug too.
We probably want to return last map or have a dedicated location for out
of range?

GCC uses:
#define DECL_IS_BUILTIN(DECL)
  (LOCATION_LOCUS (DECL_SOURCE_LOCATION (DECL)) <= BUILTINS_LOCATION)
so I think returning 0 will make us to believe that declaration with out of
range location is builtin that will definitly lead to wrong code.

Honza

[Bug lto/65536] LTO line number information garbled

[manu at gcc dot gnu.org]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="UTF-8", Size: 4250 bytes --]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #45 from Manuel López-Ibáñez <manu at gcc dot gnu.org> ---
(In reply to Jan Hubicka from comment #42)
> Hi,
> I read linemap_line_start and I think I noticed few issues with respect
> to overflows and lines being added randomly.

I honestly think this is too sensitive for stage3. I'm also not very much in
favour for optimizing out-of-order creating of line-maps. That is not the
typical case outside LTO and I would argue it should not be the typical case in
LTO either. Of course, fixing overflow problems is orthogonal and desirable and
should be submitted as a separate patch.

> 1) line_delta is computed as to_line SOURCE_LINE (map, set->highest_line)
>    I think the last inserted line is not very releavnt.  What we care about
> is
>    the base of the last location and to keep thing dense how much we are
>    stretching the value range from highest inserted element (inserting into
> middle
>    is cheap).

My guess is that the motivation here is that, if there is a large gap, it means
that we didn't get asked any source_location since a lot of lines ago. thus it
would save a lot of source_locations to simply start a new map now. Of course,
this does not work for out-of-order, but why should we pessimize typical usage
for something that should be fixed in LTO?

> 2) (line_delta > 10 && line_delta * ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map)
> > 1000)
>    ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map) is in range 7... 15, so it never
>    gets high enough to make this conditional trigger.  I changed it to:

I don't understand this. A line_delta of 67 (1000/15) will already trigger it.

> 4) the code deciding whether to do reuse seems worng:
>       if (line_delta < 0
> 	  || last_line != ORDINARY_MAP_STARTING_LINE_NUMBER (map)
> 	  || SOURCE_COLUMN (map, highest) >= (1U << column_bits))
> 
>    line_delta really should be base_line_delta, we do not need to give up
>    when map's line is 1, SOURCE_LINE (map, set->highest_line) is 5
>    and we are requested to switch to line 3.

If you insert a line out of order without creating a new map, then
linemap_position_for_column will return the wrong value.
>From gcc-bugs-return-481858-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Thu Mar 26 17:17:07 2015
Return-Path: <gcc-bugs-return-481858-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 17334 invoked by alias); 26 Mar 2015 17:17:07 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 17254 invoked by uid 48); 26 Mar 2015 17:16:59 -0000
From: "doko at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug go/65587] C package incomplete/not working for powerpc-linux-gnu
Date: Thu, 26 Mar 2015 17:52:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: go
X-Bugzilla-Version: 5.0
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: doko at gcc dot gnu.org
X-Bugzilla-Status: UNCONFIRMED
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: ian at airs dot com
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields: attachments.created
Message-ID: <bug-65587-4-Mz4Ymgt7RY@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-65587-4@http.gcc.gnu.org/bugzilla/>
References: <bug-65587-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-03/txt/msg03002.txt.bz2
Content-length: 236

https://gcc.gnu.org/bugzilla/show_bug.cgi?ide587

--- Comment #2 from Matthias Klose <doko at gcc dot gnu.org> ---
Created attachment 35151
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id5151&actioníit
output of cgo command

[Bug lto/65536] LTO line number information garbled

[hubicka at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #46 from Jan Hubicka <hubicka at gcc dot gnu.org> ---
Manuel,
I will hopefully commit the cache patch today or tomorrow morning. It does not
solve full issue. What we have is
1) we still drop columns for firefox&chromium pretty early
2) there is a bug that we sometimes output wrong line number.  I think it is
caused by the logic reallocating ORDINARY_MAP_NUMBER_OF_COLUMN_BITS as
described in original email.

It would be nice to fix those.

> My guess is that the motivation here is that, if there is a large gap, it means that we didn't get asked any source_location since a lot of lines ago. thus it would save a lot of source_locations to simply start a new map now. Of course, this does not work for out-of-order, but why should we pessimize typical usage for something that should be fixed in LTO?

I do not think typical usage is pesimized here. Yes, code is trying to avoid
case where we skip too many location indexes because we entered lines 1...30
and now we want to insert line 1000. We do not want to fast forward by 970*80.

My code keeps this logic, only it allows ordering lines backward.

> > 2) (line_delta > 10 && line_delta * ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map)
> > > 1000)
> >    ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map) is in range 7... 15, so it never
> >    gets high enough to make this conditional trigger.  I changed it to:
> 
> I don't understand this. A line_delta of 67 (1000/15) will already trigger it.

You are right, I misread the condition.

> > 4) the code deciding whether to do reuse seems worng:
> >       if (line_delta < 0
> > 	  || last_line != ORDINARY_MAP_STARTING_LINE_NUMBER (map)
> > 	  || SOURCE_COLUMN (map, highest) >= (1U << column_bits))
> > 
> >    line_delta really should be base_line_delta, we do not need to give up
> >    when map's line is 1, SOURCE_LINE (map, set->highest_line) is 5
> >    and we are requested to switch to line 3.
>
> If you insert a line out of order without creating a new map, then
> linemap_position_for_column will return the wrong value.

I do not see why. If we insert first line 1 (80 columns), then we create a map
1.  Now we insert line 3, we do not create new map, but we offset highest_line
by 80*2.  Now if we start line 2, all we need is to offset highest_line by -80
and linemap_position_for_column will still work well.

Of course when we get request for column that is out of range, we need to
trigger creation of new map entry.  What I am missing here?
>From gcc-bugs-return-481942-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Fri Mar 27 06:21:00 2015
Return-Path: <gcc-bugs-return-481942-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 97551 invoked by alias); 27 Mar 2015 06:21:00 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 97502 invoked by uid 48); 27 Mar 2015 06:20:56 -0000
From: "hubicka at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug ipa/65076] [5 Regression] 16% tramp3d-v4.cpp compile time regression
Date: Fri, 27 Mar 2015 07:41:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: ipa
X-Bugzilla-Version: 5.0
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: hubicka at gcc dot gnu.org
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: 5.0
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields:
Message-ID: <bug-65076-4-cZRZDJXnFL@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-65076-4@http.gcc.gnu.org/bugzilla/>
References: <bug-65076-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-03/txt/msg03086.txt.bz2
Content-length: 3035

https://gcc.gnu.org/bugzilla/show_bug.cgi?ide076

--- Comment #25 from Jan Hubicka <hubicka at gcc dot gnu.org> ---

The memory use report:
rtl.c:317 (copy_rtx)                                9610680: 1.6%          0:
0.0%          0: 0.0%          0: 0.0%     401870
tree.c:8281 (build_method_type_directly)            2839536: 0.5%          0:
0.0%    6803496: 3.5%          0: 0.0%      57399
cfg.c:222 (connect_src)                             9958800: 1.6%          0:
0.0%          0: 0.0%          0: 0.0%     248970
cfg.c:232 (connect_dest)                           10182184: 1.7%      94040:
0.0%          0: 0.0%          0: 0.0%     254404
cp/pt.c:11057 (tsubst_decl)                         1826688: 0.3%          0:
0.0%    8401536: 4.4%          0: 0.0%      79908
emit-rtl.c:3852 (make_insn_raw)                    11540800: 1.9%          0:
0.0%         64: 0.0%          0: 0.0%     180326
cp/pt.c:3696 (reduce_template_parm_level)           3402072: 0.6%          0:
0.0%    8579552: 4.4%          0: 0.0%      78853
tree.c:6521 (build_distinct_type_copy)               498792: 0.1%          0:
0.0%   11808384: 6.1%          0: 0.0%      73257
gimple-expr.c:486 (create_tmp_var_raw)             12338640: 2.0%          0:
0.0%          0: 0.0%          0: 0.0%      85685
cp/lex.c:674 (copy_type)                            3760344: 0.6%          0:
0.0%    9471840: 4.9%          0: 0.0%      78763
fold-const.c:2229 (fold_convert_loc)               13500256: 2.2%          0:
0.0%      14400: 0.0%          0: 0.0%     422333
tree-inline.c:712 (remap_block)                    12999184: 2.1%          0:
0.0%    1007072: 0.5%          0: 0.0%     159162
cp/pt.c:7051 (coerce_template_parms)               14424944: 2.4%          0:
0.0%     612088: 0.3%          0: 0.0%     359017
cp/pt.c:10272 (tsubst_template_args)               13006088: 2.1%          0:
0.0%    3179968: 1.6%          0: 0.0%     377257
cp/lex.c:636 (copy_decl)                             317760: 0.1%          0:
0.0%   17295456: 9.0%          0: 0.0%      94604
gimple.c:1687 (gimple_copy)                        18912384: 3.1%          0:
0.0%          0: 0.0%     107608: 0.8%     217479
tree-ssanames.c:166 (make_ssa_name_fn)             24862536: 4.1%          0:
0.0%          0: 0.0%          0: 0.0%     345313
cfg.c:148 (alloc_block)                            24957400: 4.1%          0:
0.0%          0: 0.0%          0: 0.0%     239975
tree-inline.c:5000 (copy_tree_r)                   33060096: 5.4%          0:
0.0%        736: 0.0%          0: 0.0%     815260
hash-table.h:1336 (alloc_entries)                  31255400: 5.2%    5441592:
2.1%    4493776: 2.3%     416848: 3.0%      71795
Total                                             606753058        262201484
    193060957         13796011         15521861
source location                                     Garbage            Freed
         Leak         Overhead            Times

I suppose I could try to dig into who is producing so many of hash tables.

[Bug lto/65536] LTO line number information garbled

[hubicka at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #47 from Jan Hubicka <hubicka at gcc dot gnu.org> ---
Author: hubicka
Date: Fri Mar 27 06:58:59 2015
New Revision: 221720

URL: https://gcc.gnu.org/viewcvs?rev=221720&root=gcc&view=rev
Log:

    PR lto/65536
    * lto-streamer.h (class lto_location_cache): New.
    (struct data_in): Add location_cache.
    (lto_input_location): Update prototype.
    (stream_input_location_now): New.
    * streamer-hooks.h (struct streamer_hooks): Make input_location to take
    pointer to location.
    (stream_input_location): Update.
    * ipa-devirt.c: Include streamer-hooks.h and lto-streamer.h
    (warn_odr): Apply location cache before warning.
    (lto_input_location): Update prototype.
    * gimple-streamer-in.c (input_phi, input_gimple_stmt):
    Use stream_input_location_now.
    * lto/lto.c (unify_scc): Revert location cache when unification
    suceeded.
    (lto_read_decls): Accept location cache after sucess;
    apply location cache before calling debug hooks.
    * lto-streamer-in.c (lto_location_cache::current_cache): New static
    variable.
    (lto_location_cache::cmp_loc): New function.
    (lto_location_cache::apply_location_cache): New function.
    (lto_location_cache::accept_location_cache): New function.
    (lto_location_cache::revert_location_cache): New function.
    (lto_location_cache::input_location): New function.
    (lto_input_location): Do location caching.
    (stream_input_location_now): New function.
    (input_eh_region, input_struct_function_base): Use
    stream_input_location_now.
    (lto_data_in_create): use new.
    (lto_data_in_delete): Use delete.
    * tree-streamer-in.c (unpack_ts_block_value_fields,
    unpack_ts_omp_clause_value_fields, streamer_read_tree_bitfields,
    lto_input_ts_exp_tree_pointers): Update for cached location api.

Modified:
    trunk/gcc/gimple-streamer-in.c
    trunk/gcc/ipa-devirt.c
    trunk/gcc/lto-streamer-in.c
    trunk/gcc/lto-streamer.h
    trunk/gcc/lto/ChangeLog
    trunk/gcc/lto/lto.c
    trunk/gcc/streamer-hooks.h
    trunk/gcc/tree-streamer-in.c

[Bug lto/65536] LTO line number information garbled

[hubicka at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #48 from Jan Hubicka <hubicka at gcc dot gnu.org> ---
I run memory statistics with the cache and my patch. I will run stats with
cache w/o the libcpp patch tomorrow.  I would like to get this problem solved,
but perhaps we want to only track down the reason for wrong line numbers and do
more tuning next stage1 & backport.

I suppose I can implement checking that will lookup location after every insert
to double check lines are not garbled. Dropped columns are I guess acceptable
(it would be nice to drop the wrong carrets then)

So for memory use:
realloc_for_line_map is now at 16810000: 0.9% of GGC memory use pre-IPA. This
is definitly improvement. Gimple streaming however push it up to 75497496:4.2%
I guess this suggest that caching blocks and gimple stmt locators would make a
lot of difference.

ltrans report 4194328: 5.4% post main decl streaming, but before bodies are
read in.

[Bug lto/65536] LTO line number information garbled

[manu at gcc dot gnu.org]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="UTF-8", Size: 14914 bytes --]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #49 from Manuel López-Ibáñez <manu at gcc dot gnu.org> ---
(In reply to Jan Hubicka from comment #46)
> Manuel,
> I will hopefully commit the cache patch today or tomorrow morning. It does
> not solve full issue. What we have is
> 1) we still drop columns for firefox&chromium pretty early

I think the current limits are too conservative. In particular, the limit in
linemap_position_for_column does not make sense to me. I would suggest trying
something like:

Index: line-map.c
===================================================================
--- line-map.c  (revision 221728)
+++ line-map.c  (working copy)
@@ -24,10 +24,22 @@ along with this program; see the file CO
 #include "line-map.h"
 #include "cpplib.h"
 #include "internal.h"
 #include "hashtab.h"

+/* Do not track column numbers higher than this one.  As a result, the
+   range of column_bits is [7, 18] (or 0 if column numbers are
+   disabled).  */
+#define LINE_MAP_MAX_COLUMN_NUMBER (1U << 17)
+
+/* Do not track column numbers if locations get higher than this.  */
+#define LINE_MAP_MAX_LOCATION_WITH_COLS 0x70000000
+
+/* Highest possible source location encoded within an ordinary or
+   macro map.  */
+#define LINE_MAP_MAX_SOURCE_LOCATION 0x7FFFFFF0
+
 static void trace_include (const struct line_maps *, const struct line_map *);
 static const struct line_map * linemap_ordinary_map_lookup (struct line_maps
*,
                                                            source_location);
 static const struct line_map* linemap_macro_map_lookup (struct line_maps *,
                                                        source_location);
@@ -528,26 +543,28 @@ linemap_line_start (struct line_maps *se
       || (line_delta > 10
          && line_delta * ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map) > 1000)
       || (max_column_hint >= (1U << ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map)))
       || (max_column_hint <= 80
          && ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map) >= 10)
-      || (highest > 0x60000000
-         && (set->max_column_hint || highest > 0x70000000)))
+      || (highest > LINE_MAP_MAX_LOCATION_WITH_COLS
+         && (set->max_column_hint || highest >=
LINE_MAP_MAX_SOURCE_LOCATION)))
     add_map = true;
   else
     max_column_hint = set->max_column_hint;
   if (add_map)
     {
       int column_bits;
-      if (max_column_hint > 100000 || highest > 0x60000000)
+      if (max_column_hint > LINE_MAP_MAX_COLUMN_NUMBER
+         || highest > LINE_MAP_MAX_LOCATION_WITH_COLS)
        {
          /* If the column number is ridiculous or we've allocated a huge
             number of source_locations, give up on column numbers. */
          max_column_hint = 0;
-         if (highest > 0x70000000)
-           return 0;
          column_bits = 0;
+
+         if (set->highest_location >= LINEMAPS_MACRO_LOWEST_LOCATION (set) -
1)
+           return 0;
        }
       else
        {
          column_bits = 7;
          while (max_column_hint >= (1U << column_bits))
@@ -598,19 +628,25 @@ linemap_position_for_column (struct line
   linemap_assert
     (!linemap_macro_expansion_map_p (LINEMAPS_LAST_ORDINARY_MAP (set)));

   if (to_column >= set->max_column_hint)
     {
-      if (r >= 0xC000000 || to_column > 100000)
+      if (r > LINE_MAP_MAX_LOCATION_WITH_COLS
+         || to_column > LINE_MAP_MAX_COLUMN_NUMBER)
        {
          /* Running low on source_locations - disable column numbers.  */
+         if (r >= LINEMAPS_MACRO_LOWEST_LOCATION (set) - 1)
+           return 0;
          return r;
        }
       else
        {
          struct line_map *map = LINEMAPS_LAST_ORDINARY_MAP (set);
          r = linemap_line_start (set, SOURCE_LINE (map, r), to_column + 50);
+         /* We just got to overflow; disable column numbers.  */
+         if (to_column >= set->max_column_hint)
+           return r;
        }
     }
   r = r + to_column;
   if (r >= set->highest_location)
     set->highest_location = r;


Unfortunately, I cannot get a pristine lto-boostrap to succeed, it fails with:

/home/manuel/test1/221728/build/./prev-gcc/xg++
-B/home/manuel/test1/221728/build/./prev-gcc/
-B/home/manuel/test1/./221728/install/x86_64-unknown-linux-gnu/bin/ -nostdinc++
-B/home/manuel/test1/221728/build/prev-x86_64-unknown-linux-gnu/libstdc++-v3/src/.libs
-B/home/manuel/test1/221728/build/prev-x86_64-unknown-linux-gnu/libstdc++-v3/libsupc++/.libs

-I/home/manuel/test1/221728/build/prev-x86_64-unknown-linux-gnu/libstdc++-v3/include/x86_64-unknown-linux-gnu

-I/home/manuel/test1/221728/build/prev-x86_64-unknown-linux-gnu/libstdc++-v3/include
 -I/home/manuel/test1/pristine/libstdc++-v3/libsupc++
-L/home/manuel/test1/221728/build/prev-x86_64-unknown-linux-gnu/libstdc++-v3/src/.libs
-L/home/manuel/test1/221728/build/prev-x86_64-unknown-linux-gnu/libstdc++-v3/libsupc++/.libs
  -g -O2 -flto=jobserver -frandom-seed=1 -DIN_GCC    -fno-exceptions -fno-rtti
-fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-strings
-Wcast-qual -Wmissing-format-attribute -Woverloaded-virtual -pedantic
-Wno-long-long -Wno-variadic-macros -Wno-overlength-strings -Werror -fno-common
 -DHAVE_CONFIG_H -static-libstdc++ -static-libgcc  -o cc1 c/c-lang.o
c-family/stub-objc.o attribs.o c/c-errors.o c/c-decl.o c/c-typeck.o
c/c-convert.o c/c-aux-info.o c/c-objc-common.o c/c-parser.o
c/c-array-notation.o c-family/c-common.o c-family/c-cppbuiltin.o
c-family/c-dump.o c-family/c-format.o c-family/c-gimplify.o c-family/c-lex.o
c-family/c-omp.o c-family/c-opts.o c-family/c-pch.o c-family/c-ppoutput.o
c-family/c-pragma.o c-family/c-pretty-print.o c-family/c-semantics.o
c-family/c-ada-spec.o c-family/c-cilkplus.o c-family/array-notation-common.o
c-family/cilk.o c-family/c-ubsan.o i386-c.o glibc-c.o \
          cc1-checksum.o libbackend.a main.o tree-browser.o libcommon-target.a
libcommon.a ../libcpp/libcpp.a ../libdecnumber/libdecnumber.a libcommon.a
../libcpp/libcpp.a   ../libbacktrace/.libs/libbacktrace.a
../libiberty/libiberty.a ../libdecnumber/libdecnumber.a 
-L/opt/cfarm/isl-0.12.2/lib -lisl -L/opt/cfarm/gmp-4.3.2/lib
-L/opt/cfarm/mpfr-2.4.2/lib -L/opt/cfarm/mpc-0.8.1/lib -lmpc -lmpfr -lgmp
-rdynamic -ldl  -L../zlib -lz
/home/manuel/test1/pristine/gcc/c/c-lang.h:26:16: error: type ‘struct
lang_type’ violates one definition rule [-Werror=odr]
 struct GTY(()) lang_type {
                ^
/home/manuel/test1/pristine/gcc/cp/cp-tree.h:1566:16: note: a different type is
defined in another translation unit
 struct GTY(()) lang_type {
                ^
/home/manuel/test1/pristine/gcc/c/c-lang.h:28:72: note: the first difference of
corresponding definitions is field ‘s’
   struct sorted_fields_type * GTY ((reorder ("resort_sorted_fields"))) s;
                                                                        ^
/home/manuel/test1/pristine/gcc/cp/cp-tree.h:1572:45: note: a field with
different name is defined in another translation unit
   } GTY((desc ("%h.h.is_lang_type_class"))) u;
                                             ^

> 2) there is a bug that we sometimes output wrong line number.  I think it is
> caused by the logic reallocating ORDINARY_MAP_NUMBER_OF_COLUMN_BITS as
> described in original email.

Maybe but I do not see it yet. My guess is that it is caused either by
overflowing start_location in linemap_add, or when computing 'r' 
linemap_position_for_column after linemap_line_start starts returning 0 or
after column_hint has been truncated (your patch fixes this), or when the
computation of 'r' overflows in linemap_line_start. Any of those cases will
trigger a silent overflow and return a location for something else.

Contrary to what I said before, I think now that it really makes sense for
line-maps to return UNKNOWN_LOCATION rather than the location of something else
when overflow occurs, but then LTO has to detect this case and decide what to
do.

> > My guess is that the motivation here is that, if there is a large gap, it means that we didn't get asked any source_location since a lot of lines ago. thus it would save a lot of source_locations to simply start a new map now. Of course, this does not work for out-of-order, but why should we pessimize typical usage for something that should be fixed in LTO?
> 
> I do not think typical usage is pesimized here. Yes, code is trying to avoid
> case where we skip too many location indexes because we entered lines 1...30
> and now we want to insert line 1000. We do not want to fast forward by
> 970*80.

Yes, we want to. Because then set->highest_location goes up just by one and the
next map starts from that, thus we do not need to consume source_locations for
those 970*80 entries which will never be addressed (we jump directly from 30:80
to 1000:1).

> > > 4) the code deciding whether to do reuse seems worng:
> > >       if (line_delta < 0
> > > 	  || last_line != ORDINARY_MAP_STARTING_LINE_NUMBER (map)
> > > 	  || SOURCE_COLUMN (map, highest) >= (1U << column_bits))
> > > 
> > >    line_delta really should be base_line_delta, we do not need to give up
> > >    when map's line is 1, SOURCE_LINE (map, set->highest_line) is 5
> > >    and we are requested to switch to line 3.
> >
> > If you insert a line out of order without creating a new map, then
> > linemap_position_for_column will return the wrong value.
> 
> I do not see why. If we insert first line 1 (80 columns), then we create a
> map 1.  Now we insert line 3, we do not create new map, but we offset
> highest_line by 80*2.  Now if we start line 2, all we need is to offset
> highest_line by -80 and linemap_position_for_column will still work well.

But then you do not know which is the source_location of the highest line
encoded in the line_table, which is used in several places. I'm not saying it
is not worth it or possible, but it is not as trivial as just offsetting
highest_line (and then, it should not be called highest_line but perhaps
last_line_location).

Note that I'm not discussing all this to be negative. I'm very much in favour
of supporting some kind of out-of-order insertion (in particular for PR52952),
but we should be careful to not pessimize the in-order case and we should get
all the details right. The line-maps are tricky and getting the details wrong
could be disaster.
>From gcc-bugs-return-482040-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Fri Mar 27 15:26:12 2015
Return-Path: <gcc-bugs-return-482040-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 4439 invoked by alias); 27 Mar 2015 15:26:12 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 4228 invoked by uid 48); 27 Mar 2015 15:26:08 -0000
From: "manu at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug web/65601] add bugzilla keyword "easyhack"
Date: Fri, 27 Mar 2015 15:47:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: web
X-Bugzilla-Version: unknown
X-Bugzilla-Keywords:
X-Bugzilla-Severity: enhancement
X-Bugzilla-Who: manu at gcc dot gnu.org
X-Bugzilla-Status: RESOLVED
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields: bug_status resolution
Message-ID: <bug-65601-4-5DhJvcHITq@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-65601-4@http.gcc.gnu.org/bugzilla/>
References: <bug-65601-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-03/txt/msg03184.txt.bz2
Content-length: 604

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65601

Manuel López-Ibáñez <manu at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|---                         |FIXED

--- Comment #2 from Manuel López-Ibáñez <manu at gcc dot gnu.org> ---
(In reply to Andreas Schwab from comment #1)
> https://gcc.gnu.org/bugzilla/editkeywords.cgi?action=add

Ops, I missed that completely. Thanks!
>From gcc-bugs-return-482042-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Fri Mar 27 15:41:49 2015
Return-Path: <gcc-bugs-return-482042-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 74712 invoked by alias); 27 Mar 2015 15:41:49 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 74662 invoked by uid 48); 27 Mar 2015 15:41:42 -0000
From: "vries at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug tree-optimization/65511] transform_to_exit_first_loop looses edge probabilities
Date: Fri, 27 Mar 2015 15:49:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: tree-optimization
X-Bugzilla-Version: 5.0
X-Bugzilla-Keywords:
X-Bugzilla-Severity: trivial
X-Bugzilla-Who: vries at gcc dot gnu.org
X-Bugzilla-Status: UNCONFIRMED
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields: attachments.created
Message-ID: <bug-65511-4-YQXcIXxu76@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-65511-4@http.gcc.gnu.org/bugzilla/>
References: <bug-65511-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-03/txt/msg03186.txt.bz2
Content-length: 238

https://gcc.gnu.org/bugzilla/show_bug.cgi?ide511

--- Comment #4 from vries at gcc dot gnu.org ---
Created attachment 35164
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id5164&actioníit
patch with test-case, currently testing

[Bug lto/65536] LTO line number information garbled

[hubicka at ucw dot cz]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #50 from Jan Hubicka <hubicka at ucw dot cz> ---
> 
> +/* Do not track column numbers higher than this one.  As a result, the
> +   range of column_bits is [7, 18] (or 0 if column numbers are
> +   disabled).  */
> +#define LINE_MAP_MAX_COLUMN_NUMBER (1U << 17)
> +
> +/* Do not track column numbers if locations get higher than this.  */
> +#define LINE_MAP_MAX_LOCATION_WITH_COLS 0x70000000
> +
> +/* Highest possible source location encoded within an ordinary or
> +   macro map.  */
> +#define LINE_MAP_MAX_SOURCE_LOCATION 0x7FFFFFF0

I understood the limit of 0x70000000 to give some buffer so we can still record
new
files but also drop the line number information?
> -I/home/manuel/test1/221728/build/prev-x86_64-unknown-linux-gnu/libstdc++-v3/include
>  -I/home/manuel/test1/pristine/libstdc++-v3/libsupc++
> -L/home/manuel/test1/221728/build/prev-x86_64-unknown-linux-gnu/libstdc++-v3/src/.libs
> -L/home/manuel/test1/221728/build/prev-x86_64-unknown-linux-gnu/libstdc++-v3/libsupc++/.libs
>   -g -O2 -flto=jobserver -frandom-seed=1 -DIN_GCC    -fno-exceptions -fno-rtti
> -fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-strings
> -Wcast-qual -Wmissing-format-attribute -Woverloaded-virtual -pedantic
> -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings -Werror -fno-common
>  -DHAVE_CONFIG_H -static-libstdc++ -static-libgcc  -o cc1 c/c-lang.o
> c-family/stub-objc.o attribs.o c/c-errors.o c/c-decl.o c/c-typeck.o
> c/c-convert.o c/c-aux-info.o c/c-objc-common.o c/c-parser.o
> c/c-array-notation.o c-family/c-common.o c-family/c-cppbuiltin.o
> c-family/c-dump.o c-family/c-format.o c-family/c-gimplify.o c-family/c-lex.o
> c-family/c-omp.o c-family/c-opts.o c-family/c-pch.o c-family/c-ppoutput.o
> c-family/c-pragma.o c-family/c-pretty-print.o c-family/c-semantics.o
> c-family/c-ada-spec.o c-family/c-cilkplus.o c-family/array-notation-common.o
> c-family/cilk.o c-family/c-ubsan.o i386-c.o glibc-c.o \
>           cc1-checksum.o libbackend.a main.o tree-browser.o libcommon-target.a
> libcommon.a ../libcpp/libcpp.a ../libdecnumber/libdecnumber.a libcommon.a
> ../libcpp/libcpp.a   ../libbacktrace/.libs/libbacktrace.a
> ../libiberty/libiberty.a ../libdecnumber/libdecnumber.a 
> -L/opt/cfarm/isl-0.12.2/lib -lisl -L/opt/cfarm/gmp-4.3.2/lib
> -L/opt/cfarm/mpfr-2.4.2/lib -L/opt/cfarm/mpc-0.8.1/lib -lmpc -lmpfr -lgmp
> -rdynamic -ldl  -L../zlib -lz
> /home/manuel/test1/pristine/gcc/c/c-lang.h:26:16: error: type ???struct
> lang_type??? violates one definition rule [-Werror=odr]
>  struct GTY(()) lang_type {
>                 ^
> /home/manuel/test1/pristine/gcc/cp/cp-tree.h:1566:16: note: a different type is
> defined in another translation unit
>  struct GTY(()) lang_type {
>                 ^
> /home/manuel/test1/pristine/gcc/c/c-lang.h:28:72: note: the first difference of
> corresponding definitions is field ???s???
>    struct sorted_fields_type * GTY ((reorder ("resort_sorted_fields"))) s;
>                                                                         ^
> /home/manuel/test1/pristine/gcc/cp/cp-tree.h:1572:45: note: a field with
> different name is defined in another translation unit
>    } GTY((desc ("%h.h.is_lang_type_class"))) u;
>                                              ^

Well, this is a bug in C/C++ FEs clearly, lang_type should not be declared
twice with different meanings.  One needs to be renamed. You can however safely
ignore it and get build to complete with --disable-werror
> 
> > 2) there is a bug that we sometimes output wrong line number.  I think it is
> > caused by the logic reallocating ORDINARY_MAP_NUMBER_OF_COLUMN_BITS as
> > described in original email.
> 
> Maybe but I do not see it yet. My guess is that it is caused either by
> overflowing start_location in linemap_add, or when computing 'r' 
> linemap_position_for_column after linemap_line_start starts returning 0 or
> after column_hint has been truncated (your patch fixes this), or when the
> computation of 'r' overflows in linemap_line_start. Any of those cases will
> trigger a silent overflow and return a location for something else.

I will try to add assets and reproduce it (at least with cache disabled on gcc
bootstrap), lets see if I get any luck.  I only saw it on Chromium that is
order of mangnitude bigger than GCC.  Just to give you idea, GCC linktime
memory consumption is about 700MB, while Firefox takes 3GB and Chromium 9GB.
It has 11k files linked to one unit (which makes is so much bigger than firefox
that does include many .cpp units into one to get around the bloat).
Before my changes, the linemap occupied about 1GB.

> > I do not think typical usage is pesimized here. Yes, code is trying to avoid
> > case where we skip too many location indexes because we entered lines 1...30
> > and now we want to insert line 1000. We do not want to fast forward by
> > 970*80.
> 
> Yes, we want to. Because then set->highest_location goes up just by one and the
> next map starts from that, thus we do not need to consume source_locations for
> those 970*80 entries which will never be addressed (we jump directly from 30:80
> to 1000:1).

Yep, i think we have oposite meaning of fast forward.  I mean fast forward by
not
adding map entry, while you mean map entry. So I think we agree ;)
> > 
> > I do not see why. If we insert first line 1 (80 columns), then we create a
> > map 1.  Now we insert line 3, we do not create new map, but we offset
> > highest_line by 80*2.  Now if we start line 2, all we need is to offset
> > highest_line by -80 and linemap_position_for_column will still work well.
> 
> But then you do not know which is the source_location of the highest line
> encoded in the line_table, which is used in several places. I'm not saying it
> is not worth it or possible, but it is not as trivial as just offsetting
> highest_line (and then, it should not be called highest_line but perhaps
> last_line_location).

The code tracking hihest recorded location is intact (it is simple max
operation).
This needs to work, because no one guarantees the columns are inserted in
increasing
number, so linemap_position_for_column is not always returning highest location
at each invocation.
> 
> Note that I'm not discussing all this to be negative. I'm very much in favour
> of supporting some kind of out-of-order insertion (in particular for PR52952),
> but we should be careful to not pessimize the in-order case and we should get
> all the details right. The line-maps are tricky and getting the details wrong
> could be disaster.

Sure, I am trying to find solution that works well (better) for both.

Honza

[Bug lto/65536] LTO line number information garbled

[hubicka at ucw dot cz]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #51 from Jan Hubicka <hubicka at ucw dot cz> ---
> 
> Contrary to what I said before, I think now that it really makes sense for
> line-maps to return UNKNOWN_LOCATION rather than the location of something else
> when overflow occurs, but then LTO has to detect this case and decide what to
> do.

This would affect not only LTO code - all C++ FE's can face an overflow in
exterme cases
and becuase we special case everyting <=BUILTINS_LOCATION perhaps we really can
just
add extra special value WRONG_LOCATION that would be used to denote this?

Honza

[Bug lto/65536] LTO line number information garbled

[manu at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #52 from Manuel López-Ibáñez <manu at gcc dot gnu.org> ---
(In reply to Jan Hubicka from comment #50)
> > 
> > +/* Do not track column numbers higher than this one.  As a result, the
> > +   range of column_bits is [7, 18] (or 0 if column numbers are
> > +   disabled).  */
> > +#define LINE_MAP_MAX_COLUMN_NUMBER (1U << 17)
> > +
> > +/* Do not track column numbers if locations get higher than this.  */
> > +#define LINE_MAP_MAX_LOCATION_WITH_COLS 0x70000000
> > +
> > +/* Highest possible source location encoded within an ordinary or
> > +   macro map.  */
> > +#define LINE_MAP_MAX_SOURCE_LOCATION 0x7FFFFFF0
> 
> I understood the limit of 0x70000000 to give some buffer so we can still
> record new
> files but also drop the line number information?

Do we really need to reserve space for 0x7FFFFFF0 - 0x70000000 = 268435440
files?

> Well, this is a bug in C/C++ FEs clearly, lang_type should not be declared
> twice with different meanings.  One needs to be renamed. You can however
> safely
> ignore it and get build to complete with --disable-werror

I'll try that. Thanks!

> I will try to add assets and reproduce it (at least with cache disabled on
> gcc
> bootstrap), lets see if I get any luck.  I only saw it on Chromium that is
> order of mangnitude bigger than GCC.  Just to give you idea, GCC linktime
> memory consumption is about 700MB, while Firefox takes 3GB and Chromium 9GB.
> It has 11k files linked to one unit (which makes is so much bigger than
> firefox
> that does include many .cpp units into one to get around the bloat).
> Before my changes, the linemap occupied about 1GB.

You can get an estimate of how much memory would be required to stream in/out
directly the line_table by summing up the memory reported by
dump_line_table_statistics for each TU before streaming out (perhaps using
-ftrack-macro-expansion=0 to reduce it further). This would still be an
overestimate, because one can drop one char (sysp) and one int (included_from)
per map and one can drop all maps that are not used by LTO. Moreover, you will
not need a cache and everything will be in order already when you stream in.
>From gcc-bugs-return-482056-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Fri Mar 27 16:32:54 2015
Return-Path: <gcc-bugs-return-482056-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 94144 invoked by alias); 27 Mar 2015 16:32:54 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 94051 invoked by uid 48); 27 Mar 2015 16:32:50 -0000
From: "paolo.carlini at oracle dot com" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug c++/65579] [C++11] gcc requires definition of a static constexpr member even though it is not odr-used
Date: Fri, 27 Mar 2015 16:50:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: c++
X-Bugzilla-Version: 5.0
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: paolo.carlini at oracle dot com
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields: bug_status cf_reconfirmed_on blocked everconfirmed
Message-ID: <bug-65579-4-qgC8LRn0jH@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-65579-4@http.gcc.gnu.org/bugzilla/>
References: <bug-65579-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-03/txt/msg03200.txt.bz2
Content-length: 462

https://gcc.gnu.org/bugzilla/show_bug.cgi?ide579

Paolo Carlini <paolo.carlini at oracle dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
   Last reconfirmed|                            |2015-03-27
             Blocks|                            |55004
     Ever confirmed|0                           |1

[Bug lto/65536] LTO line number information garbled

[hubicka at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65536

--- Comment #53 from Jan Hubicka <hubicka at gcc dot gnu.org> ---
> You can get an estimate of how much memory would be required to stream in/out 
> directly the line_table by summing up the memory reported by 
> dump_line_table_statistics for each TU before streaming out (perhaps using 
> -ftrack-macro-expansion=0 to reduce it further). This would still be an 
> overestimate, because one can drop one char (sysp) and one int (included_from) > per map and one can drop all maps that are not used by LTO. Moreover, you will > not need a cache and everything will be in order already when you stream in.

I see, the stats actually are in -fmem-report, how convenient ;))

With Firefox I get:
Line Table allocations during the compilation process
Number of ordinary maps used:          407k
Ordinary map used size:                 15M
Number of ordinary maps allocated:     409k
Ordinary maps allocated size:           15M
Number of macro maps used:               0 
Macro maps used size:                    0 
Macro maps locations size:               0 
Macro maps size:                         0 
Duplicated maps locations size:          0 
Total allocated maps size:              15M
Total used maps size:                   15M

after streaming in all declarations&types and

Line Table allocations during the compilation process
Number of ordinary maps used:          769k
Ordinary map used size:                 30M
Number of ordinary maps allocated:    1638k
Ordinary maps allocated size:           63M
Number of macro maps used:               0 
Macro maps used size:                    0 
Macro maps locations size:               0 
Macro maps size:                         0 
Duplicated maps locations size:          0 
Total allocated maps size:              63M
Total used maps size:                   30M

by end of WPA stage. The extra growth is by variable initializers and function
bodies we bring in on demand (for merging, ICF, devirt machinery). Those by
large part bypass my cache.

Individual ltrans units consume after streaming in global decls:
Line Table allocations during the compilation process
Number of ordinary maps used:           29k
Ordinary map used size:               1189k
Number of ordinary maps allocated:     102k
Ordinary maps allocated size:         4095k
Number of macro maps used:               0 
Macro maps used size:                    0 
Macro maps locations size:               0 
Macro maps size:                         0 
Duplicated maps locations size:          0 
Total allocated maps size:            4095k
Total used maps size:                 1189k

this does not look terrible

I tried to sum the sizes for EON, and got to 92493kB and 8631 without macro
expansion tracking (note that this size is basically the same with or without
my proposed line-maps.c patch, so it does not seem to pesimize non-LTO builds).

At WPA time (with my proposed patch, will rebuild tree without after lunch) I
get:

Line Table allocations during the compilation process
Number of ordinary maps used:         6084 
Ordinary map used size:                237k
Number of ordinary maps allocated:    6553 
Ordinary maps allocated size:          255k
Number of macro maps used:               0 
Macro maps used size:                    0 
Macro maps locations size:               0 
Macro maps size:                         0 
Duplicated maps locations size:          0 
Total allocated maps size:             255k
Total used maps size:                  237k

So about 30x smaller, even though we do have information loss here (i.e. inline
stacks)

So it seems we can not really directly stream linemaps - there is way too much
of information that is discarded from parsing time to LTO time.
(about 95% of trees that are streamed out are discarded by tree merging and
thus with my cache never hits linemaps).

Linemap representation is interesting. I suppose for next release we want to do
kind of on-disk variant inspired by libcpp.  If we arrange cache to be
per-LTO-section (not randomly flushed as it is now), we can read the on-disk
format into LTO own format and apply relevant part to the line-maps after tree
merging.