From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: by sourceware.org (Postfix, from userid 48) id 9512D3848016; Wed, 18 Aug 2021 20:21:50 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 9512D3848016 From: "aaron.hill at wdc dot com" To: gcc-bugs@gcc.gnu.org Subject: [Bug preprocessor/101968] New: Preprocessor line number statements become wrong after '#include' Date: Wed, 18 Aug 2021 20:21:50 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: gcc X-Bugzilla-Component: preprocessor X-Bugzilla-Version: 12.0 X-Bugzilla-Keywords: X-Bugzilla-Severity: normal X-Bugzilla-Who: aaron.hill at wdc dot com X-Bugzilla-Status: UNCONFIRMED X-Bugzilla-Resolution: X-Bugzilla-Priority: P3 X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version bug_status bug_severity priority component assigned_to reporter target_milestone attachments.created Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: gcc-bugs@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-bugs mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Aug 2021 20:21:50 -0000 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=3D101968 Bug ID: 101968 Summary: Preprocessor line number statements become wrong after '#include' Product: gcc Version: 12.0 Status: UNCONFIRMED Severity: normal Priority: P3 Component: preprocessor Assignee: unassigned at gcc dot gnu.org Reporter: aaron.hill at wdc dot com Target Milestone: --- Created attachment 51320 --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=3D51320&action=3Dedit Add the 'test.cpp' file needed to reproduce the bug When running the preprocessor with 'cpp -nostdinc test.cpp', the following output is generated: ``` # 1 "test.cpp" # 1 "" # 1 "" # 1 "test.cpp" # 327681 "test.cpp" # 1 "foo.h" 1 # 327681 "test.cpp" 2 MISSING_DEF ``` The file 'test.cpp' (attached to this issue) consists of 327,680 empty line= s, followed by: ``` #include "foo.h" MISSING_DEF ``` The file 'foo.h' is empty. Notice how the lines `# 327681 "test.cpp"` and `# 327681 "test.cpp" 2` both refer to the same line number (the line number of the '#include' statement), despite the fact that we have exited the file 'foo.h', and should now be on= the next line (line 327682). The flag '-nostdinc' is needed to prevent the preprocessor from injecting an include of '/usr/include/stdc-predef.h', which breaks the conditions needed= to reproduce the bug. Alternatively, running `arm-none-eabi-cpp test.cpp` will also reproduce the bug, since that toolchain doesn't have 'stdc-predef.h'. When run through the compiler, this produces an error message on the wrong = line (using 'g++-11 -nostdinc test.cpp'): ``` test.cpp:327681:1: error: =E2=80=98MISSING_DEF=E2=80=99 does not name a type 327681 | #include "foo.h" | ^~~~~~~~~~~ ``` This issue occurs since GCC 9 (I haven't tested earlier versions), and still occurs as of the latest commit (1b34248527472496ca3fe2a07183beac8cf69041 at= the time of writing). This bug is caused by a decrement of `pfile->line_table->highest_location--= ` in `libcpp/files.c` (https://github.com/gcc-mirror/gcc/blob/f0fca213bc52644ba896da622b35842a615= 7bd98/libcpp/files.c#L988). If reading the line following the '#include' caused us to create a new line map, then 'pfile->line_table->highest_location' will be equal to 'LINEMAPS_LAST_ORDINARY_MAP(pfile->line_table)->start_location. Therefore, decrementing 'pfile->line_table->highest_location' causes the location to m= ove outside of the correct line map. Reproducing this bug in actual code is very difficult, but has happened on several occasions. In this reproducer, the large number of blank lines caus= es 'linemap_line_start' to create a new line map, due to the condition 'line_d= elta * map->m_column_and_range_bits > 1000' (https://github.com/gcc-mirror/gcc/blob/f0fca213bc52644ba896da622b35842a615= 7bd98/libcpp/line-map.c#L775). However, there are several other conditions that can cause 'linemap_line_st= art' to create a new line map - triggering one of them on the line immediately following an '#include' should also cause this bug to occur. In my local tests, changing 'decrement' to also check that `pfile->line_table->highest_location !=3D LINEMAPS_LAST_ORDINARY_MAP(pfile->line_table)->start_location` is enough to avoid the bug. However, I hadn't looked at the preprocessor code before investigating this bug, so I'm not sure if this change will have any other consequences.=