[Bug preprocessor/101968] New: Preprocessor line number statements become wrong after '#include'

["aaron.hill at wdc dot com" <gcc-bugzilla@gcc.gnu.org>]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101968

            Bug ID: 101968
           Summary: Preprocessor line number statements become wrong after
                    '#include'
           Product: gcc
           Version: 12.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: preprocessor
          Assignee: unassigned at gcc dot gnu.org
          Reporter: aaron.hill at wdc dot com
  Target Milestone: ---

Created attachment 51320
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=51320&action=edit
Add the 'test.cpp' file needed to reproduce the bug

When running the preprocessor with 'cpp -nostdinc test.cpp', the following
output is generated:

```
# 1 "test.cpp"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "test.cpp"
# 327681 "test.cpp"
# 1 "foo.h" 1
# 327681 "test.cpp" 2
MISSING_DEF
```

The file 'test.cpp' (attached to this issue) consists of 327,680 empty lines,
followed by:

```
#include "foo.h"
MISSING_DEF
```

The file 'foo.h' is empty.

Notice how the lines `# 327681 "test.cpp"` and `# 327681 "test.cpp" 2` both
refer to the same line number (the line number of the '#include' statement),
despite the fact that we have exited the file 'foo.h', and should now be on the
next line (line 327682).

The flag '-nostdinc' is needed to prevent the preprocessor from injecting an
include of '/usr/include/stdc-predef.h', which breaks the conditions needed to
reproduce the bug. Alternatively, running `arm-none-eabi-cpp test.cpp` will
also reproduce the bug, since that toolchain doesn't have 'stdc-predef.h'.

When run through the compiler, this produces an error message on the wrong line
(using 'g++-11 -nostdinc test.cpp'):

```
test.cpp:327681:1: error: ‘MISSING_DEF’ does not name a type
327681 | #include "foo.h"
       | ^~~~~~~~~~~
```

This issue occurs since GCC 9 (I haven't tested earlier versions), and still
occurs as of the latest commit (1b34248527472496ca3fe2a07183beac8cf69041 at the
time of writing).

This bug is caused by a decrement of `pfile->line_table->highest_location--` in
`libcpp/files.c`
(https://github.com/gcc-mirror/gcc/blob/f0fca213bc52644ba896da622b35842a6157bd98/libcpp/files.c#L988).
If reading the line following the '#include' caused us to create a new line
map, then 'pfile->line_table->highest_location' will be equal to
'LINEMAPS_LAST_ORDINARY_MAP(pfile->line_table)->start_location. Therefore,
decrementing 'pfile->line_table->highest_location' causes the location to move
outside of the correct line map.

Reproducing this bug in actual code is very difficult, but has happened on
several occasions. In this reproducer, the large number of blank lines causes
'linemap_line_start' to create a new line map, due to the condition 'line_delta
* map->m_column_and_range_bits > 1000'
(https://github.com/gcc-mirror/gcc/blob/f0fca213bc52644ba896da622b35842a6157bd98/libcpp/line-map.c#L775).
However, there are several other conditions that can cause 'linemap_line_start'
to create a new line map - triggering one of them on the line immediately
following an '#include' should also cause this bug to occur.

In my local tests, changing 'decrement' to also check that
`pfile->line_table->highest_location !=
LINEMAPS_LAST_ORDINARY_MAP(pfile->line_table)->start_location` is enough to
avoid the bug. However, I hadn't looked at the preprocessor code before
investigating this bug, so I'm not sure if this change will have any other
consequences.