Re: [PATCH] c-family: Fix ICE with large column number after restoring a PCH [PR105608]

[Lewis Hyatt <lhyatt@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 3365 bytes --]

On Wed, Jan 31, 2024 at 03:18:01PM -0500, Jason Merrill wrote:
> On 1/30/24 21:49, Lewis Hyatt wrote:
> > On Fri, Jan 26, 2024 at 04:16:54PM -0500, Jason Merrill wrote:
> > > On 12/5/23 20:52, Lewis Hyatt wrote:
> > > > Hello-
> > > > 
> > > > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105608
> > > > 
> > > > There are two related issues here really, a regression since GCC 11 where we
> > > > can ICE after restoring a PCH, and a deeper issue with bogus locations
> > > > assigned to macros that were defined prior to restoring a PCH.  This patch
> > > > fixes the ICE regression with a simple change, and I think it's appropriate
> > > > for GCC 14 as well as backport to 11, 12, 13. The bad locations (wrong, but
> > > > not generally causing an ICE, and mostly affecting only the output of
> > > > -Wunused-macros) are not as problematic, and will be harder to fix. I could
> > > > take a stab at that for GCC 15. In the meantime the patch adds XFAILed
> > > > tests for the wrong locations (as well as passing tests for the regression
> > > > fix). Does it look OK please? Bootstrap + regtest all languages on x86-64
> > > > Linux. Thanks!
> > > 
> > > OK for trunk and branches, thanks!
> > > 
> > 
> > Thanks for the review! That is all taken care of. I have one more request if
> > you don't mind please... There have been some further comments on the PR
> > indicating that the new xfailed testcase I added is failing in an unexpected
> > way on at least one architecture. To recap, the idea here was that
> > 
> > 1) libcpp needs new logic to be able to output correct locations for this
> > case. That will be some new code that is suitable for stage 1, not now.
> > 
> > 2) In the meantime, we fixed things up enough to avoid an ICE that showed up
> > in GCC 11, and added an xfailed testcase to remind about #1.
> > 
> > The problem is that, the reason that libcpp outputs the wrong locations, is
> > that it has always used a location from the old line_map instance to index
> > into the new line_map instance, and so the exact details of the wrong
> > locations it outputs depend on the state of those two line maps, which may
> > differ depending on system includes and things like that. So I was hoping to
> > make one further one-line change to libcpp, not yet to output correct
> > locations, but at least to output one which is the same always and doesn't
> > depend on random things. This would assign all restored macros to a
> > consistent location, one line following the #include that triggered the PCH
> > process. I think this probably shouldn't be backported but it would be nice
> > to get into GCC 14, while nothing critical, at least it would avoid the new
> > test failure that's being reported. But more generally, I think using a
> > location from a totally different line map is dangerous and could have worse
> > consequences that haven't been seen yet. Does it look OK please? Thanks!
> 
> Can we use the line of the #include, as the test expects, rather than the
> following line?

Thanks, yes, that will work too, it just needs a few changes to
c-family/c-pch.cc to set the location there and then increment it
after. Patch which does that is attached. (This is a new one based on
master, not incremental to the prior patch.) The testcase does not require
any changes this way, and bootstrap + regtest looks good.

-Lewis

[-- Attachment #2: PR105608-part2-v2.txt --]
[-- Type: text/plain, Size: 3775 bytes --]

[PATCH] c-family: Stabilize the location for macros restored after PCH load [PR105608]

libcpp currently lacks the infrastructure to assign correct locations to
macros that were defined prior to loading a PCH and then restored
afterwards. While I plan to address that fully for GCC 15, this patch
improves things by using at least a valid location, even if it's not the
best one. Without this change, libcpp uses pfile->directive_line as the
location for the restored macros, but this location_t applies to the old
line map, not the one that was just restored from the PCH, so the resulting
location is unpredictable and depends on what was stored in the line maps
before. With this change, all restored macros get assigned locations at the
line of the #include that triggered the PCH restore. A future patch will
store the actual file name and line number of each definition and then
synthesize locations in the new line map pointing to the right place.

gcc/c-family/ChangeLog:

	PR preprocessor/105608
	* c-pch.cc (c_common_read_pch): Adjust line map so that libcpp
	assigns a location to restored macros which is the same location
	that triggered the PCH include.

libcpp/ChangeLog:

	PR preprocessor/105608
	* pch.cc (cpp_read_state): Set a valid location for restored
	macros.

diff --git a/gcc/c-family/c-pch.cc b/gcc/c-family/c-pch.cc
index 79b4f88fe4d..971af90be0d 100644
--- a/gcc/c-family/c-pch.cc
+++ b/gcc/c-family/c-pch.cc
@@ -318,6 +318,7 @@ c_common_read_pch (cpp_reader *pfile, const char *name,
   struct save_macro_data *smd;
   expanded_location saved_loc;
   bool saved_trace_includes;
+  int cpp_result;
 
   timevar_push (TV_PCH_RESTORE);
 
@@ -343,20 +344,26 @@ c_common_read_pch (cpp_reader *pfile, const char *name,
   cpp_set_line_map (pfile, line_table);
   rebuild_location_adhoc_htab (line_table);
   line_table->trace_includes = saved_trace_includes;
-  linemap_add (line_table, LC_ENTER, 0, saved_loc.file, saved_loc.line);
+
+  /* Set the current location to the line containing the #include (or the
+     #pragma GCC_preprocess) for the purpose of assigning locations to any
+     macros that are about to be restored.  */
+  linemap_add (line_table, LC_ENTER, 0, saved_loc.file,
+	       saved_loc.line > 1 ? saved_loc.line - 1 : saved_loc.line);
 
   timevar_push (TV_PCH_CPP_RESTORE);
-  if (cpp_read_state (pfile, name, f, smd) != 0)
-    {
-      fclose (f);
-      timevar_pop (TV_PCH_CPP_RESTORE);
-      goto end;
-    }
-  timevar_pop (TV_PCH_CPP_RESTORE);
+  cpp_result = cpp_read_state (pfile, name, f, smd);
 
+  /* Set the current location to the line following the #include, where we
+     were prior to processing the PCH.  */
+  linemap_line_start (line_table, saved_loc.line, 0);
 
+  timevar_pop (TV_PCH_CPP_RESTORE);
   fclose (f);
 
+  if (cpp_result != 0)
+    goto end;
+
   /* Give the front end a chance to take action after a PCH file has
      been loaded.  */
   if (lang_post_pch_load)
diff --git a/libcpp/pch.cc b/libcpp/pch.cc
index e156fe257b3..f2f74ed6ea9 100644
--- a/libcpp/pch.cc
+++ b/libcpp/pch.cc
@@ -838,7 +838,14 @@ cpp_read_state (cpp_reader *r, const char *name, FILE *f,
 	      != NULL)
 	    {
 	      _cpp_clean_line (r);
-	      if (!_cpp_create_definition (r, h, 0))
+
+	      /* ??? Using r->line_table->highest_line is not ideal here, but we
+		 do need to use some location that is relative to the new line
+		 map just loaded, not the old one that was in effect when these
+		 macros were lexed.  The proper fix is to remember the file name
+		 and line number where each macro was defined, and then add
+		 these locations into the new line map.  See PR105608.  */
+	      if (!_cpp_create_definition (r, h, r->line_table->highest_line))
 		abort ();
 	      _cpp_pop_buffer (r);
 	    }