preprocessor: Handle digit separators in #line [PR82359]

preprocessor: Handle digit separators in #line [PR82359]

[Joseph Myers]

As reported in bug 82359, the preprocessor does not allow C++ digit
separators in the line number in a #line directive, despite the
standard syntax for that directive using digit-sequence which allows
digit separators.

There is some confusion in that bug about whether C++ is meant to
allow digit separators there or not, but the last comment there
suggests they are meant to be allowed, and the version of digit
separators accepted for C2X at the March meeting explicitly mentions
digit separators in the #line specification to avoid any ambiguity
there.

This patch thus adds code to handle digit separators in the line
number in #line, as part of the preparation for enabling digit
separators in C2X mode.  The code changed does not contain any
conditionals for whether digit separators are supported in the chosen
language version, because that was handled earlier in pp-number lexing
and if they aren't supported they won't appear in the string passed to
that function.  It does however make sure not to allow adjacent digit
separators because those are only handled at a later stage of lexing
at present.  (Problems with how certain source character sequences
involving digit separators that don't actually match the pp-number
syntax get lexed as a pp-number and only diagnosed later, if at all,
are bugs 83873 and 97604, to be addressed separately.)

Making the change in this location will have the effect of allowing
digit separators in the "# <line-number> <file> <flags>" form of
directive as well as #line; I don't think that's a problem.

Bootstrapped with no regressions for x86_64-pc-linux-gnu.  Any C++
maintainer comments on this patch?

libcpp/
	PR preprocessor/82359
	* directives.c (strtolinenum): Handle digit separators.

gcc/testsuite/
	PR preprocessor/82359
	* g++.dg/cpp1y/digit-sep-line.C,
	g++.dg/cpp1y/digit-sep-line-neg.C: New tests.

diff --git a/gcc/testsuite/g++.dg/cpp1y/digit-sep-line-neg.C b/gcc/testsuite/g++.dg/cpp1y/digit-sep-line-neg.C
new file mode 100644
index 00000000000..fa3b1352109
--- /dev/null
+++ b/gcc/testsuite/g++.dg/cpp1y/digit-sep-line-neg.C
@@ -0,0 +1,4 @@
+// Test digit separators in #line (bug 82359).  Test invalid usage.
+// { dg-do preprocess { target c++14 } }
+
+#line 0''123 // { dg-error "is not a positive integer" }
diff --git a/gcc/testsuite/g++.dg/cpp1y/digit-sep-line.C b/gcc/testsuite/g++.dg/cpp1y/digit-sep-line.C
new file mode 100644
index 00000000000..48846e42221
--- /dev/null
+++ b/gcc/testsuite/g++.dg/cpp1y/digit-sep-line.C
@@ -0,0 +1,8 @@
+// Test digit separators in #line (bug 82359).
+// { dg-do compile { target c++14 } }
+
+#line 0'123
+static_assert (__LINE__ == 123, "#line with digit separator");
+
+#line 4'56'7'8'9
+static_assert (__LINE__ == 456789, "#line with digit separator");
diff --git a/libcpp/directives.c b/libcpp/directives.c
index f4aa17d1156..795f93e664b 100644
--- a/libcpp/directives.c
+++ b/libcpp/directives.c
@@ -922,12 +922,19 @@ strtolinenum (const uchar *str, size_t len, linenum_type *nump, bool *wrapped)
   linenum_type reg = 0;
 
   uchar c;
+  bool seen_digit_sep = false;
   *wrapped = false;
   while (len--)
     {
       c = *str++;
+      if (!seen_digit_sep && c == '\'' && len)
+	{
+	  seen_digit_sep = true;
+	  continue;
+	}
       if (!ISDIGIT (c))
 	return true;
+      seen_digit_sep = false;
       if (reg > ((linenum_type) -1) / 10)
 	*wrapped = true;
       reg *= 10;

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: preprocessor: Handle digit separators in #line [PR82359]

[Joseph Myers]

I've now committed this patch as preparation for further digit separator 
fixes and enabling digit separators for C2x, but would still welcome any 
C++ maintainer comments.

-- 
Joseph S. Myers
joseph@codesourcery.com