Re: [modules] Preprocessing requires compiled header unit modules

[Iain Sandoe <iain@sandoe.co.uk>]

> On 22 Apr 2022, at 15:08, Boris Kolpackov <boris@codesynthesis.com> wrote:
> 
> Ben Boeckel <ben.boeckel@kitware.com> writes:
> 
>> On Thu, Apr 21, 2022 at 06:05:52 +0200, Boris Kolpackov wrote:
>> 
>>> I don't think it is. A header unit (unlike a named module) may export
>>> macros which could affect further dependencies. Consider:
>>> 
>>> import "header-unit.hpp"; // May or may not export macro FOO.
>>> 
>>> #ifdef FOO
>>> import "header-unit2.hpp";
>>> #endif
>> 
>> I agree that the header needs to be *found*, but scanning cannot require
>> a pre-existing BMI for that header.
> 
> Well, if scanning cannot require a pre-existing BMI but a pre-existing
> BMI is required to get accurate dependency information, then something
> has to give.
> 
> You hint at a potential solution in your subsequent email:
> 
>> Can't it just read the header as if it wasn't imported? AFAIU, that's
>> what GCC did in Jan 2019. I understand that CPP state is probably not
>> easy, but something to consider.
> 
> The problem with this approach is that a header import and a header
> include have subtly different semantics around macros. In particular,
> the header import does not "see" macros defined by the importer while
> the header include does. Here is an example:
> 
> // file: header-unit.hpp
> //
> #ifdef BAR
> #define FOO
> #endif
> 
> // file: importer.cpp
> //
> #define BAR
> import "header-unit.hpp";    // Should not "see" BAR.
> //#include "header-unit.hpp" // Should "see" BAR.
> 
> #ifdef FOO
> import "header-unit2.hpp";
> #endif
> 
> In this example, if you treat import of header-unit.hpp as
> include, you will get incorrect dependency information.
> 
> So to make this work correctly we will need to re-create the
> macro isolation semantics of import for include.
> 
> Even if we manage to do this, there are some implications I
> am not sure we will like: the isolated macros will contain
> inclusion guards, which means we will keep re-scanning the
> same files potentially many many time. Here is an example,
> assume each header-unitN.hpp includes or imports <functional>:
> 
> // file: importer.cpp
> //
> import <functional>; // Defined _GLIBCXX_FUNCTIONAL include
> 
> import "header-unit1.hpp"; // Ignores _GLIBCXX_FUNCTIONAL
> import "header-unit2.hpp"; // Ditto.
> import "header-unit3.hpp"; // Ditto.
> import "header-unit4.hpp"; // Ditto.

The standard has the concept of an “importable header” which is implementation-defined.

We could choose that only headers that are self-contained (i.e. unaffected by external
defines) are “importable” (thus the remaining headers would not be eligible for include-
translation).  That would mean that we could rely on processing any import by processing
the header it is created from?  Perhaps that is too great a restriction and we need to be
 more clever….

@ben, in relation to an earlier question:
https://eel.is/c++draft/cpp.import#note-4

says that predefined macro names are not introduced by #define and that the implementation
is encouraged not to treat them as if they were

IIUC, that means that -D/U (and preamble ones) are not emitted into the macro stream - however
it might well be the case that they *are* part of the module identifing hash (and preserved
as part of the captured command line).

Iain