[Bug preprocessor/99446] New: [11 Regression] ICE in linemap_position_for_loc_and_offset, at libcpp/line-map.c:1005

[Bug preprocessor/99446] New: [11 Regression] ICE in linemap_position_for_loc_and_offset, at libcpp/line-map.c:1005

[doko at debian dot org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99446

            Bug ID: 99446
           Summary: [11 Regression] ICE in
                    linemap_position_for_loc_and_offset, at
                    libcpp/line-map.c:1005
           Product: gcc
           Version: 11.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: preprocessor
          Assignee: unassigned at gcc dot gnu.org
          Reporter: doko at debian dot org
  Target Milestone: ---

Created attachment 50323
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=50323&action=edit
preprocessed source

seen building the firefox-esr 78.8.0 package, using trunk 20210306.

I have a compiler-generated ICE report, however even trying to re-run the
command doesn't always generates the ICE report.

make[3]: Entering directory
'/packages/tmp/firefox-esr-78.8.0esr/build-browser/config/external/rlbox_lucet_sandbox'
config/external/rlbox_lucet_sandbox/rlbox_lucet_thread_locals.o
/usr/bin/g++ -o rlbox_lucet_thread_locals.o -c 
-I/packages/tmp/firefox-esr-78.8.0esr/build-browser/dist/stl_wrappers
-I/packages/tmp/firefox-esr-78.8.0esr/build-browser/dist/system_wrappers
-include /packages/tmp/firefox-esr-78.8.0esr/config/gcc_hidden.h
-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -DNDEBUG=1
-DTRIMMED=1 -DMOZ_HAS_MOZGLUE -DMOZILLA_INTERNAL_API -DIMPL_LIBXUL
-DSTATIC_EXPORTABLE_JS_API
-I/packages/tmp/firefox-esr-78.8.0esr/config/external/rlbox_lucet_sandbox
-I/packages/tmp/firefox-esr-78.8.0esr/build-browser/config/external/rlbox_lucet_sandbox
-I/packages/tmp/firefox-esr-78.8.0esr/build-browser/dist/include
-I/usr/include/nspr -I/usr/include/nss -I/usr/include/nspr
-I/packages/tmp/firefox-esr-78.8.0esr/build-browser/dist/include/nss -fPIC
-DMOZILLA_CLIENT -include
/packages/tmp/firefox-esr-78.8.0esr/build-browser/mozilla-config.h -Wdate-time
-D_FORTIFY_SOURCE=2 -Wall -Wempty-body -Wignored-qualifiers
-Woverloaded-virtual -Wpointer-arith -Wsign-compare -Wtype-limits
-Wunreachable-code -Wwrite-strings -Wno-invalid-offsetof -Wc++2a-compat
-Wduplicated-cond -Wimplicit-fallthrough -Wunused-function -Wunused-variable
-Wno-error=maybe-uninitialized -Wno-error=deprecated-declarations
-Wno-error=array-bounds -Wno-error=coverage-mismatch
-Wno-error=free-nonheap-object -Wno-multistatement-macros
-Wno-error=class-memaccess -Wno-error=deprecated-copy -Wformat
-Wformat-overflow=2 -fno-sized-deallocation -fno-aligned-new -O2
-ffile-prefix-map=/packages/tmp/firefox-esr-78.8.0esr=.
-fstack-protector-strong -Wformat -Werror=format-security -fno-exceptions
-fno-strict-aliasing -fno-rtti -ffunction-sections -fdata-sections
-fno-exceptions -fno-math-errno -pthread -pipe -g -freorder-blocks -O2
-fomit-frame-pointer -funwind-tables  -MD -MP -MF
.deps/rlbox_lucet_thread_locals.o.pp  
/packages/tmp/firefox-esr-78.8.0esr/config/external/rlbox_lucet_sandbox/rlbox_lucet_thread_locals.cpp
/packages/tmp/firefox-esr-78.8.0esr/config/external/rlbox_lucet_sandbox/rlbox_lucet_thread_locals.cpp:1:
internal compiler error: in linemap_position_for_loc_and_offset, at
libcpp/line-map.c:1005
    1 | /* -*- Mode: C++; tab-width: 20; indent-tabs-mode: nil; c-basic-offset:
2 -*- */
      | 
0x656a93 linemap_position_for_loc_and_offset(line_maps*, unsigned int, unsigned
int)
        ../libcpp/../../src/libcpp/line-map.c:1005
0x18ebee1 cp_lexer_new_main
        ../../src/gcc/cp/parser.c:676
0x18ebee1 c_parse_file()
        ../../src/gcc/cp/parser.c:45227
0x18d4aba c_common_parse_file()
        ../../src/gcc/c-family/c-opts.c:1218
Please submit a full bug report,
with preprocessed source if appropriate.

[Bug preprocessor/99446] [11 Regression] ICE in linemap_position_for_loc_and_offset, at libcpp/line-map.c:1005

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99446

Martin Liška <marxin at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Last reconfirmed|                            |2021-03-08
             Status|UNCONFIRMED                 |WAITING
     Ever confirmed|0                           |1

--- Comment #1 from Martin Liška <marxin at gcc dot gnu.org> ---
Cannot reproduce that..

[Bug preprocessor/99446] [11 Regression] ICE in linemap_position_for_loc_and_offset, at libcpp/line-map.c:1005

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99446

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |11.0

[Bug preprocessor/99446] [11 Regression] ICE in linemap_position_for_loc_and_offset, at libcpp/line-map.c:1005

[sbergman at redhat dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99446

Stephan Bergmann <sbergman at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |sbergman at redhat dot com

--- Comment #2 from Stephan Bergmann <sbergman at redhat dot com> ---
At least with recent GCC master (bc2127767a0076afdbc9075fda29f97f82ef7ec6), I
can consistently reproduce the following:

> $ cat dir1/file2
> #pragma GCC diagnostic push
> #include_next <file2>
> #pragma GCC diagnostic pop

> $ cat dir2/file1
> #include <file2>

> $ cat dir2/file2

> $ g++ -Idir1 -Idir2 -fsyntax-only test.cc
> test.cc:2: internal compiler error: in linemap_position_for_loc_and_offset, at libcpp/line-map.c:1005
> 0x1ca855b linemap_position_for_loc_and_offset(line_maps*, unsigned int, unsigned int)
>         ../../src/libcpp/line-map.c:1005
> 0xa94ef3 cp_lexer_new_main
>         ../../src/gcc/cp/parser.c:676
> 0xa94ef3 c_parse_file()
>         ../../src/gcc/cp/parser.c:45237
> 0xbb8ead c_common_parse_file()
>         ../../src/gcc/c-family/c-opts.c:1218
> Please submit a full bug report,
> with preprocessed source if appropriate.
> Please include the complete backtrace with any bug report.
> See <https://gcc.gnu.org/bugs/> for instructions.

[Bug preprocessor/99446] [11 Regression] ICE in linemap_position_for_loc_and_offset, at libcpp/line-map.c:1005

[sbergman at redhat dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99446

--- Comment #3 from Stephan Bergmann <sbergman at redhat dot com> ---
(In reply to Stephan Bergmann from comment #2)
> At least with recent GCC master (bc2127767a0076afdbc9075fda29f97f82ef7ec6),
> I can consistently reproduce the following:

what I failed to include in comment 2 is

> $ cat test.cc
> #include <file1>

[Bug preprocessor/99446] [11 Regression] ICE in linemap_position_for_loc_and_offset, at libcpp/line-map.c:1005

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99446

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |NEW

--- Comment #4 from Richard Biener <rguenth at gcc dot gnu.org> ---
Confirmed.

[Bug preprocessor/99446] [11 Regression] ICE in linemap_position_for_loc_and_offset, at libcpp/line-map.c:1005

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99446

--- Comment #5 from Richard Biener <rguenth at gcc dot gnu.org> ---
Created attachment 50529
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=50529&action=edit
testcase

testcase for convenience.

> tar xf testcase.tar
> g++ -Idir1 -Idir2 -fsyntax-only test.cc
test.cc:2: internal compiler error: in linemap_position_for_loc_and_offset, at
libcpp/line-map.c:1005
0x1cd2ebb linemap_position_for_loc_and_offset(line_maps*, unsigned int,
unsigned int)
        /home/rguenther/src/trunk/libcpp/line-map.c:1005
0xaa4533 cp_lexer_new_main
        /home/rguenther/src/trunk/gcc/cp/parser.c:678
0xaa4533 c_parse_file()
        /home/rguenther/src/trunk/gcc/cp/parser.c:45264

[Bug preprocessor/99446] [11 Regression] ICE in linemap_position_for_loc_and_offset, at libcpp/line-map.c:1005

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99446

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Priority|P3                          |P1

[Bug preprocessor/99446] [11 Regression] ICE in linemap_position_for_loc_and_offset, at libcpp/line-map.c:1005 since r11-6325

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99446

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|[11 Regression] ICE in      |[11 Regression] ICE in
                   |linemap_position_for_loc_an |linemap_position_for_loc_an
                   |d_offset, at                |d_offset, at
                   |libcpp/line-map.c:1005      |libcpp/line-map.c:1005
                   |                            |since r11-6325
                 CC|                            |jakub at gcc dot gnu.org,
                   |                            |nathan at gcc dot gnu.org

--- Comment #6 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Started with r11-6325-g745f22096c3a2a5e63d5bab1fb079e2c437cf9bc

[Bug preprocessor/99446] [11 Regression] ICE in linemap_position_for_loc_and_offset, at libcpp/line-map.c:1005 since r11-6325

[bernd.edlinger at hotmail dot de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99446

Bernd Edlinger <bernd.edlinger at hotmail dot de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |bernd.edlinger at hotmail dot de

--- Comment #7 from Bernd Edlinger <bernd.edlinger at hotmail dot de> ---
Hmm,

I don't fully understand what's the reason why
the assertion fires here.

... but what's apparently wrong here, is the location
before the EOF is in a different file.

While it was intended to show the line above the end-of-file,
it was likely not intended to show the last token in
a different include file?

So something like that fixes the test case:

diff --git a/gcc/cp/parser.c b/gcc/cp/parser.c
index 59adac4..99026f4 100644
--- a/gcc/cp/parser.c
+++ b/gcc/cp/parser.c
@@ -668,7 +668,8 @@ cp_lexer_new_main (void)
                       + lexer->buffer->length ()
                      - 1;

-  if (lexer->buffer->length () != 1)
+  if (lexer->buffer->length () != 1
+      && LOCATION_FILE (tok[-1].location) == LOCATION_FILE(tok[0].location))
     {
       /* Set the EOF token's location to be the just after the previous
          token's range.  That way 'at-eof' diagnostics point at something

[Bug preprocessor/99446] [11 Regression] ICE in linemap_position_for_loc_and_offset, at libcpp/line-map.c:1005 since r11-6325

[nathan at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99446

--- Comment #8 from Nathan Sidwell <nathan at gcc dot gnu.org> ---
Well it would be nice if it could show the token in the included file, but if
that's difficult your solution is perfectly fine.  thanks

[Bug preprocessor/99446] [11 Regression] ICE in linemap_position_for_loc_and_offset, at libcpp/line-map.c:1005 since r11-6325

[bernd.edlinger at hotmail dot de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99446

--- Comment #9 from Bernd Edlinger <bernd.edlinger at hotmail dot de> ---
The last token is a CPP_PRAGMA_EOL, and has a line number 2,
while the include file has only one line, so it is similar to an EOL position.
I guess therefore this fails to add a column?

1002      location_t r = 
1003        linemap_position_for_line_and_column (set, map, line, column);
1004      if (linemap_assert_fails (r <= set->highest_location)
1005          || linemap_assert_fails (map == linemap_lookup (set, r)))
1006        return loc;


(gdb) p tok[0]
$30 = {type = CPP_EOF, keyword = RID_MAX, flags = 64 '@', implicit_extern_c =
false, error_reported = false, 
  purged_p = false, tree_check_p = false, main_source_p = true, location =
255712, u = {tree_check_value = 0x0, 
    value = 0x0}}
(gdb) p tok[-1]
$31 = {type = CPP_PRAGMA_EOL, keyword = RID_MAX, flags = 0 '\000',
implicit_extern_c = false, error_reported = false, 
  purged_p = false, tree_check_p = false, main_source_p = false, location =
251552, u = {tree_check_value = 0x0, 
    value = 0x0}}

location_t r = 
1003        linemap_position_for_line_and_column (set, map, line, column);
(gdb) p r
$68 = 252480
(gdb) p *map
$70 = {<line_map> = {start_location = 251584}, reason = LC_LEAVE, sysp = 0
'\000', m_column_and_range_bits = 12, 
  m_range_bits = 5, to_file = 0x2ac9800 "pr99446-1.h", to_line = 2,
included_from = 238400}
(gdb) p *result
$74 = {<line_map> = {start_location = 251616}, reason = LC_LEAVE, sysp = 0
'\000', m_column_and_range_bits = 12, 
  m_range_bits = 5, to_file = 0x2b0a2d0 "pr99446.c", to_line = 2, included_from
= 0}

but line 2 does not exist in both files.

[Bug preprocessor/99446] [11 Regression] ICE in linemap_position_for_loc_and_offset, at libcpp/line-map.c:1005 since r11-6325

[nathan at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99446

Nathan Sidwell <nathan at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
           Assignee|unassigned at gcc dot gnu.org      |nathan at gcc dot gnu.org

--- Comment #10 from Nathan Sidwell <nathan at gcc dot gnu.org> ---
I see the problem.  it's the logic in the earlier while loop.  testing a fix

[Bug preprocessor/99446] [11 Regression] ICE in linemap_position_for_loc_and_offset, at libcpp/line-map.c:1005 since r11-6325

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99446

--- Comment #11 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Nathan Sidwell <nathan@gcc.gnu.org>:

https://gcc.gnu.org/g:4acb3af3669db4ca79ffc97cd615fcea205bcccb

commit r11-8150-g4acb3af3669db4ca79ffc97cd615fcea205bcccb
Author: Nathan Sidwell <nathan@acm.org>
Date:   Tue Apr 13 05:03:19 2021 -0700

    preprocessor: Fix column adjustment [PR 99446]

    This ICE was because when adjusting a column offset we could advance
    into a linemap for a different file.  We only checked the next line
    map was not for a line further advanced in any file, forgetting that
    it could be for an earlier line in a different file.  The testcase
    needed adjusting as column 512 was unrepresentable, once that was
    taken into consideration.

            PR preprocessor/99446
            libcpp/
            * line-map.c (line-map.c): Do not advance to linemaps for
            different files.
            gcc/testsuite/
            * g++.dg/diagnostic/pr72803.C: Adjust expected column.

[Bug preprocessor/99446] [11 Regression] ICE in linemap_position_for_loc_and_offset, at libcpp/line-map.c:1005 since r11-6325

[nathan at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99446

Nathan Sidwell <nathan at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|ASSIGNED                    |RESOLVED

--- Comment #12 from Nathan Sidwell <nathan at gcc dot gnu.org> ---
* 8318ccc9e87 2021-04-13 | preprocessor: Fix column adjustment [PR 99446]

[Bug preprocessor/99446] [11 Regression] ICE in linemap_position_for_loc_and_offset, at libcpp/line-map.c:1005 since r11-6325

[bernd.edlinger at hotmail dot de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99446

--- Comment #13 from Bernd Edlinger <bernd.edlinger at hotmail dot de> ---
Hi Nathan,

I've been playing with a variant of c-c++-common/raw-string-6.c
with your patch:

$ cat raw-string-6.c
$ cat raw-string-6.c 
// { dg-do compile }
// { dg-options "-std=gnu99" { target c } }
// { dg-options "-std=c++0x" { target c++ } }

#include "xxxx.h"
$ cat xxxx.h 
const void *s0 = R"ouch()ouCh";
$ gcc -x c raw-string-6.c
In file included from raw-string-6.c:5:
xxxx.h:1:18: error: unterminated raw string
    1 | const void *s0 = R"ouch()ouCh";
      |                  ^
raw-string-6.c:6: error: expected expression at end of input
$ gcc -x c++ raw-string-6.c 
In file included from raw-string-6.c:5:
xxxx.h:1:18: error: unterminated raw string
    1 | const void *s0 = R"ouch()ouCh";
      |                  ^
xxxx.h:1:16: error: expected primary-expression at end of input
    1 | const void *s0 = R"ouch()ouCh";
      |                ^

Do you agree that the C++ FE places the error about the end of input
on a somewhat surprising place?


Bernd.