[Bug preprocessor/103165] New: [12 Regression] ICE unspellable token PRAGMA since r12-4797-g0078a058a5693871

[Bug preprocessor/103165] New: [12 Regression] ICE unspellable token PRAGMA since r12-4797-g0078a058a5693871

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103165

            Bug ID: 103165
           Summary: [12 Regression] ICE unspellable token PRAGMA since
                    r12-4797-g0078a058a5693871
           Product: gcc
           Version: 12.0
            Status: UNCONFIRMED
          Keywords: ice-on-invalid-code
          Severity: normal
          Priority: P3
         Component: preprocessor
          Assignee: unassigned at gcc dot gnu.org
          Reporter: marxin at gcc dot gnu.org
                CC: burnus at gcc dot gnu.org
  Target Milestone: ---

The following ICEs:

$ cat pragma.C
#define INTERNAL_CATCH_TEST(...) #__VA_ARGS__ _Pragma("GCC diagnostic pop")
#define REQUIRE_FALSE(...) INTERNAL_CATCH_TEST(__VA_ARGS__)
REQUIRE_FALSE(INTERNAL_CATCH_TEST())


$ g++ pragma.C -c 
pragma.C:3:36: internal compiler error: unspellable token PRAGMA
    3 | REQUIRE_FALSE(INTERNAL_CATCH_TEST())
      |                                    ^
0xc663d3 c_cpp_diagnostic(cpp_reader*, cpp_diagnostic_level,
cpp_warning_reason, rich_location*, char const*, __va_list_tag (*) [1])
        /home/marxin/Programming/gcc/gcc/c-family/c-common.c:6616
0x20f206e cpp_diagnostic_at
        /home/marxin/Programming/gcc/libcpp/errors.c:67
0x20f206e cpp_diagnostic
        /home/marxin/Programming/gcc/libcpp/errors.c:82
0x20f206e cpp_error(cpp_reader*, cpp_diagnostic_level, char const*, ...)
        /home/marxin/Programming/gcc/libcpp/errors.c:96
0x20fc483 cpp_spell_token(cpp_reader*, cpp_token const*, unsigned char*, bool)
        /home/marxin/Programming/gcc/libcpp/lex.c:3588
0x210576a stringify_arg
        /home/marxin/Programming/gcc/libcpp/macro.c:921
0x2106b85 replace_args
        /home/marxin/Programming/gcc/libcpp/macro.c:1946
0x2106b85 enter_macro_context
        /home/marxin/Programming/gcc/libcpp/macro.c:1496
0x2107d82 cpp_get_token_1
        /home/marxin/Programming/gcc/libcpp/macro.c:2986
0xca3c0a c_lex_with_flags(tree_node**, unsigned int*, unsigned char*, int)
        /home/marxin/Programming/gcc/gcc/c-family/c-lex.c:483
0xb240e6 cp_lexer_get_preprocessor_token
        /home/marxin/Programming/gcc/gcc/cp/parser.c:864
0xb783d4 cp_lexer_new_main
        /home/marxin/Programming/gcc/gcc/cp/parser.c:643
0xb783d4 c_parse_file()
        /home/marxin/Programming/gcc/gcc/cp/parser.c:47838
0xcb33c9 c_common_parse_file()
        /home/marxin/Programming/gcc/gcc/c-family/c-opts.c:1237
Please submit a full bug report,
with preprocessed source if appropriate.
Please include the complete backtrace with any bug report.
See <https://gcc.gnu.org/bugs/> for instructions.

The original code is a valid code, but it's hard to reduce it..

[Bug preprocessor/103165] [12 Regression] ICE unspellable token PRAGMA since r12-4797-g0078a058a5693871

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103165

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |12.0

[Bug preprocessor/103165] [12 Regression] ICE unspellable token PRAGMA since r12-4797-g0078a058a5693871

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103165

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
     Ever confirmed|0                           |1
   Last reconfirmed|                            |2021-11-10

--- Comment #1 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Confirmed.

[Bug preprocessor/103165] [12 Regression] ICE unspellable token PRAGMA since r12-4797-g0078a058a5693871

[burnus at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103165

--- Comment #2 from Tobias Burnus <burnus at gcc dot gnu.org> ---
Simpler to read example:

  #define inner(...) #__VA_ARGS__ _Pragma("GCC diagnostic pop")
  #define outer(...) inner(__VA_ARGS__)
  outer(inner(1,2))

the expected result is something like:
  "\"1,2\" _Pragma(\"foo bar\")"
  #pragma GCC diagnostic pop

I guess. That's what 'clang-11 -E' gives.

→ When running '-E' with GCC, one runs into the issue described in PR90400; see
also for issues with diagnostic pragmas PR91669.


Without -E, the 'GCC diagnostic' is known (registered) and when
  #__VA_ARGS__

is stringified for '"1,2" _Pragma("GCC diagnostic pop")' it fails because
cpp_spell_token does not handle _Pragma.

 * * *

Before r12-4797-g0078a058a5693871, _Pragma was not really handled but just
output before the macro was processed – leading to something like:
  #pragma GCC diagnostic pop
  "\"1,2\"
  #pragma GCC diagnostic pop

[Bug preprocessor/103165] [12 Regression] ICE unspellable token PRAGMA since r12-4797-g0078a058a5693871

[burnus at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103165

--- Comment #3 from Tobias Burnus <burnus at gcc dot gnu.org> ---
Patch: https://gcc.gnu.org/pipermail/gcc-patches/2021-November/584035.html

 * * *

Note the remark in
https://gcc.gnu.org/pipermail/gcc-patches/2021-November/584042.html regarding
preserving spacing with the stringified _Pragma.
This is not handled by the patch, nor are strings with prefixes like L"...".

Currently, _cpp_do__Pragma calls get__Pragma_string – and the latter strips off
all padding, but still keeps the prefix info via CPP_{,W,UTF8}STRING{,16,32}.
Of course after evaluating that string via destringize_and_run -> do_pragma,
the string and, hence, its type is gone.

The question is – for this issue and also for PR90400 – whether the string
should be kept for longer as string and only later destringize_and_run (or used
as is); at least when the pragma space/name is not registered, this seems to
make sense - such that it is properly handled with '-E' (→ PR90400). Now that
CPP_PRAGMA supports 'flags & PRAGMA_OP', it can be distinguished from
CPP_PRAGMA in macro.c and it could be workable? — Note: Requires to check
whether there are any issues with delaying it – or whether that's actually the
proper way of doing it.

[Bug preprocessor/103165] [12 Regression] ICE unspellable token PRAGMA since r12-4797-g0078a058a5693871

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103165

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jakub at gcc dot gnu.org

--- Comment #4 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Created attachment 51842
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=51842&action=edit
gcc12-pr103165.patch

So what about this instead?  Doesn't regress on make check-gcc check-g++, but
not tested beyond that so far.

[Bug preprocessor/103165] [12 Regression] ICE unspellable token PRAGMA since r12-4797-g0078a058a5693871

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103165

--- Comment #5 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Jakub Jelinek <jakub@gcc.gnu.org>:

https://gcc.gnu.org/g:a6e0d593707ae44dec0bdf2bcdc4f539050b46db

commit r12-5454-ga6e0d593707ae44dec0bdf2bcdc4f539050b46db
Author: Jakub Jelinek <jakub@redhat.com>
Date:   Mon Nov 22 22:29:20 2021 +0100

    libcpp: Fix _Pragma stringification [PR103165]

    As the testcase show, sometimes _Pragma is turned into CPP_PRAGMA
    .. CPP_PRAGMA_EOL tokens, even when it might still need to be
    stringized later on.  We are then ICEing because we don't handle
    stringification of CPP_PRAGMA or CPP_PRAGMA_EOL, but trying to
    reconstruct the exact tokens with exact spacing after it has been
    lowered is very hard.  So, instead this patch ensures we don't
    lower _Pragma during expand_arg calls, but only later when
    cpp_get_token_1 is called outside of expand_arg.

    2021-11-22  Jakub Jelinek  <jakub@redhat.com>
                Tobias Burnus  <tobias@codesourcery.com>

            PR preprocessor/103165
    libcpp/
            * internal.h (struct lexer_state): Add ignore__Pragma field.
            * macro.c (builtin_macro): Don't interpret _Pragma if
            pfile->state.ignore__Pragma.
            (expand_arg): Temporarily set pfile->state.ignore__Pragma to 1.
    gcc/testsuite/
            * c-c++-common/gomp/pragma-3.c: New test.
            * c-c++-common/gomp/pragma-4.c: New test.
            * c-c++-common/gomp/pragma-5.c: New test.

    Co-Authored-By: Tobias Burnus <tobias@codesourcery.com>

[Bug preprocessor/103165] [12 Regression] ICE unspellable token PRAGMA since r12-4797-g0078a058a5693871

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103165

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|unassigned at gcc dot gnu.org      |jakub at gcc dot gnu.org
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED

--- Comment #6 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Fixed.