[Bug c++/99072] New: [modules] Compiling header unit with partial preprocessing (-E -fdirectives-only) twice causes CRC mismatch

[Bug c++/99072] New: [modules] Compiling header unit with partial preprocessing (-E -fdirectives-only) twice causes CRC mismatch

[boris at kolpackov dot net]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99072

            Bug ID: 99072
           Summary: [modules] Compiling header unit with partial
                    preprocessing (-E -fdirectives-only) twice causes CRC
                    mismatch
           Product: gcc
           Version: 11.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c++
          Assignee: unassigned at gcc dot gnu.org
          Reporter: boris at kolpackov dot net
  Target Milestone: ---

With partial preprocessing (-E -fdirectives-only) compiling the same header
with the same options twice results in two CRC-incompatible .gcm files:

cat <<EOF >string.hxx 
#include <string>
EOF

cat <<EOF >hello.mxx
export module hello;

import "string.hxx";

export std::string format_hello (const char* n)
{
  return "Hello, " + std::string (n) + '!';
}
EOF

cat <<EOF >main.cxx
#include <cassert>

import "string.hxx";
import hello;

int main ()
{
  assert (format_hello ("World") == "Hello, World!");
}
EOF

cat <<EOF >hello.txt
./string.hxx string.so.gcm
hello hello.so.gcm
EOF

cat <<EOF >main.txt
./string.hxx string.gcm
hello hello.so.gcm
EOF

g++ -std=c++2a -fmodules-ts -fmodule-mapper=hello.txt -E -fdirectives-only
-fmodule-header -x c++-header -o string.so.ii string.hxx
g++ -std=c++2a -fmodules-ts -fmodule-mapper=hello.txt -fpreprocessed
-fdirectives-only -fmodule-header -x c++-header string.so.ii

g++ -std=c++2a -fmodules-ts -fmodule-mapper=hello.txt -x c++ -c hello.mxx

g++ -std=c++2a -fmodules-ts -fmodule-mapper=main.txt -E -fdirectives-only
-fmodule-header -x c++-header -o string.ii string.hxx
g++ -std=c++2a -fmodules-ts -fmodule-mapper=main.txt -fpreprocessed
-fdirectives-only -fmodule-header -x c++-header string.ii

g++ -std=c++2a -fmodules-ts -fmodule-mapper=main.txt -x c++ -c main.cxx
In module imported at main.cxx:5:1:
hello: error: import ‘./string.hxx’ has CRC mismatch

The really strange thing is that the .ii files are identical:

diff string.ii string.so.ii && echo same
same

Also, there is no error if to compile string.hxx without -E -fdirectives-only:

g++ -std=c++2a -fmodules-ts -fmodule-mapper=hello.txt -fmodule-header -x
c++-header string.hxx

g++ -std=c++2a -fmodules-ts -fmodule-mapper=main.txt -fmodule-header -x
c++-header string.hxx

[Bug c++/99072] [modules] Compiling header unit with partial preprocessing (-E -fdirectives-only) twice causes CRC mismatch

[nathan at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99072

Nathan Sidwell <nathan at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Last reconfirmed|                            |2021-02-16
             Status|UNCONFIRMED                 |WAITING
     Ever confirmed|0                           |1

--- Comment #1 from Nathan Sidwell <nathan at gcc dot gnu.org> ---
cannot reproduce:

3 first build of header-unit
devvm1702:41>./xg++ -B./ -nostdinc++
-I../x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu
-I../x86_64-pc-linux-gnu/libstdc++-v3/include
-I../../..//src/libstdc++-v3/libsupc++
-I../../../src/libstdc++-v3/include/backward
-I../../../src/libstdc++-v3/testsuite/util -std=c++2a -fmodule-header -E
-fdirectives-only -o 99072_a.ii 99072_a.H
devvm1702:42>./xg++ -B./ -nostdinc++
-I../x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu
-I../x86_64-pc-linux-gnu/libstdc++-v3/include
-I../../..//src/libstdc++-v3/libsupc++
-I../../../src/libstdc++-v3/include/backward
-I../../../src/libstdc++-v3/testsuite/util -std=c++2a -fmodule-header
-fpreprocessed -fdirectives-only -c 99072_a.ii

# build module 'hello'
devvm1702:43>./xg++ -B./ -nostdinc++
-I../x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu
-I../x86_64-pc-linux-gnu/libstdc++-v3/include
-I../../..//src/libstdc++-v3/libsupc++
-I../../../src/libstdc++-v3/include/backward
-I../../../src/libstdc++-v3/testsuite/util -std=c++2a -fmodules-ts -c 99072_b.C 

# save header-unit cmi
devvm1702:44>cp gcm.cache/,/99072_a.H.gcm .

# second build of string
devvm1702:45>./xg++ -B./ -nostdinc++
-I../x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu
-I../x86_64-pc-linux-gnu/libstdc++-v3/include
-I../../..//src/libstdc++-v3/libsupc++
-I../../../src/libstdc++-v3/include/backward
-I../../../src/libstdc++-v3/testsuite/util -std=c++2a -fmodule-header -E
-fdirectives-only -o 99072_a.ii 99072_a.H
devvm1702:46>./xg++ -B./ -nostdinc++
-I../x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu
-I../x86_64-pc-linux-gnu/libstdc++-v3/include
-I../../..//src/libstdc++-v3/libsupc++
-I../../../src/libstdc++-v3/include/backward
-I../../../src/libstdc++-v3/testsuite/util -std=c++2a -fmodule-header
-fpreprocessed -fdirectives-only -c 99072_a.ii

# build main
devvm1702:47>./xg++ -B./ -nostdinc++
-I../x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu
-I../x86_64-pc-linux-gnu/libstdc++-v3/include
-I../../..//src/libstdc++-v3/libsupc++
-I../../../src/libstdc++-v3/include/backward
-I../../../src/libstdc++-v3/testsuite/util -std=c++2a -fmodules-ts -c 99072_c.C

# no error

[Bug c++/99072] [modules] Compiling header unit with partial preprocessing (-E -fdirectives-only) twice causes CRC mismatch

[boris at kolpackov dot net]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99072

--- Comment #2 from Boris Kolpackov <boris at kolpackov dot net> ---
This has something to do with a different .ii file name the second time we
compile the header. Try to make this change to your command lines (notice .so 
before .ii):

# second build of string
devvm1702:45>./xg++ [...] -o 99072_a.so.ii 99072_a.H
devvm1702:46>./xg++ [...] -c 99072_a.so.ii

[Bug c++/99072] [modules] Compiling header unit with partial preprocessing (-E -fdirectives-only) twice causes CRC mismatch

[nathan at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99072

--- Comment #3 from Nathan Sidwell <nathan at gcc dot gnu.org> ---
Cannot reproduce:
devvm1702:150>./xg++ -B./ -nostdinc++
-I../x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu
-I../x86_64-pc-linux-gnu/libstdc++-v3/include
-I../../..//src/libstdc++-v3/libsupc++
-I../../../src/libstdc++-v3/include/backward
-I../../../src/libstdc++-v3/testsuite/util -std=c++2a -fmodule-header -E
-fdirectives-only -o 99072_a.so.ii 99072_a.H  
devvm1702:151>./xg++ -B./ -nostdinc++
-I../x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu
-I../x86_64-pc-linux-gnu/libstdc++-v3/include
-I../../..//src/libstdc++-v3/libsupc++
-I../../../src/libstdc++-v3/include/backward
-I../../../src/libstdc++-v3/testsuite/util -std=c++2a -fmodule-header
-fpreprocessed -fdirectives-only -c 99072_a.so.ii
devvm1702:152>./xg++ -B./ -nostdinc++
-I../x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu
-I../x86_64-pc-linux-gnu/libstdc++-v3/include
-I../../..//src/libstdc++-v3/libsupc++
-I../../../src/libstdc++-v3/include/backward
-I../../../src/libstdc++-v3/testsuite/util -std=c++2a -fmodules-ts -c 99072_b.C
devvm1702:153>./xg++ -B./ -nostdinc++
-I../x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu
-I../x86_64-pc-linux-gnu/libstdc++-v3/include
-I../../..//src/libstdc++-v3/libsupc++
-I../../../src/libstdc++-v3/include/backward
-I../../../src/libstdc++-v3/testsuite/util -std=c++2a -fmodule-header
-fpreprocessed -fdirectives-only -c 99072_a.so.ii
devvm1702:154>./xg++ -B./ -nostdinc++
-I../x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu
-I../x86_64-pc-linux-gnu/libstdc++-v3/include
-I../../..//src/libstdc++-v3/libsupc++
-I../../../src/libstdc++-v3/include/backward
-I../../../src/libstdc++-v3/testsuite/util -std=c++2a -fmodules-ts -c 99072_c.C

[Bug c++/99072] [modules] Compiling header unit with partial preprocessing (-E -fdirectives-only) twice causes CRC mismatch

[boris at kolpackov dot net]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99072

--- Comment #4 from Boris Kolpackov <boris at kolpackov dot net> ---
You need to use different .ii file names on the first and second header unit
builds. Using your original command lines as a reference:

# first build of header-unit
devvm1702:45>./xg++ [...] -o 99072_a.ii 99072_a.H
devvm1702:46>./xg++ [...] -c 99072_a.ii

# second build of string
devvm1702:45>./xg++ [...] -o 99072_a.so.ii 99072_a.H
devvm1702:46>./xg++ [...] -c 99072_a.so.ii

[Bug c++/99072] [modules] Compiling header unit with partial preprocessing (-E -fdirectives-only) twice causes CRC mismatch

[nathan at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99072

--- Comment #5 from Nathan Sidwell <nathan at gcc dot gnu.org> ---
99072_a.H:
// Empty file

99072_b.C:
export module Foo;
import "99072_a.H";

99072_c.C:
import Foo;

./cc1plus -quiet -E -fdirectives-only -fmodule-header -E 99072_a.H >
99072_a.one
./cc1plus -quiet -fpreprocessed -fdirectives-only -fmodule-header 99072_a.one
./cc1plus -quiet -fmodules-ts 99072_b.C
./cc1plus -quiet -E -fdirectives-only -fmodule-header -E 99072_a.H >
99072_a.two
./cc1plus -quiet -fpreprocessed -fdirectives-only -fmodule-header 99072_a.two
./cc1plus -quiet -fmodules-ts 99072_c.C                                      
In module imported at 99072_b.C:2:1,
of module Foo, imported at 99072_c.C:1:
./99072_a.H: error: module './99072_a.H' CRC mismatch
./99072_a.H: error: failed to read compiled module: Bad file data
./99072_a.H: note: compiled module file is 'gcm.cache/,/99072_a.H.gcm'
In module imported at 99072_c.C:1:1:
Foo: error: failed to read compiled module: Bad import dependency
Foo: note: compiled module file is 'gcm.cache/Foo.gcm'

[Bug c++/99072] [modules] Compiling header unit with partial preprocessing (-E -fdirectives-only) twice causes CRC mismatch

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99072

--- Comment #6 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Nathan Sidwell <nathan@gcc.gnu.org>:

https://gcc.gnu.org/g:a13be187cb2987db851b3f096f5319d5fe3a7301

commit r11-7366-ga13be187cb2987db851b3f096f5319d5fe3a7301
Author: Nathan Sidwell <nathan@acm.org>
Date:   Wed Feb 24 05:50:12 2021 -0800

    c++: modules & -fpreprocessed [PR 99072]

    When we read preprocessed source, we deal with a couple of special
    location lines at the start of the file.  These provide information
    about the original filename of the source and the current directory,
    so we can process the source in the same manner.  When updating that
    code, I had a somewhat philosophical question: Should the line table
    contain evidence of the filename the user provided to the compiler?  I
    figured to leave it there, as it did no harm.  But this defect shows
    an issue.  It's in the line table and our (non optimizing) line table
    serializer emits that filename.  Which means if one re-preprocesses
    the original source to a differently-named intermediate file, the
    resultant CMI is different.  Boo.  That's a difference that doesn't
    matter, except the CRC matching then fails.  We should elide the
    filename, so that one can preprocess to mktemp intermediate filenames
    for whatever reason.

    This patch takes the approach of expunging it from the line table --
    so the line table will end up with exactly the same form.  That seems
    a better bet than trying to fix up mismatching line tables in CMI
    emission.

            PR c++/99072
            libcpp/
            * init.c (read_original_filename): Expunge all evidence of the
            original filename.
            gcc/testsuite/
            * g++.dg/modules/pr99072.H: New.

[Bug c++/99072] [modules] Compiling header unit with partial preprocessing (-E -fdirectives-only) twice causes CRC mismatch

[nathan at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99072

Nathan Sidwell <nathan at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |RESOLVED
         Resolution|---                         |FIXED

--- Comment #7 from Nathan Sidwell <nathan at gcc dot gnu.org> ---
a13be187cb2 2021-02-24 | c++: modules & -fpreprocessed [PR 99072]

[Bug c++/99072] [modules] Compiling header unit with partial preprocessing (-E -fdirectives-only) twice causes CRC mismatch

[boris at kolpackov dot net]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99072

--- Comment #8 from Boris Kolpackov <boris at kolpackov dot net> ---
Can confirm works for me, thanks!