From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qt1-x834.google.com (mail-qt1-x834.google.com [IPv6:2607:f8b0:4864:20::834]) by sourceware.org (Postfix) with ESMTPS id 086AC3858413 for ; Wed, 19 Jul 2023 00:01:48 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 086AC3858413 Authentication-Results: sourceware.org; dmarc=pass (p=reject dis=none) header.from=kitware.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=kitware.com Received: by mail-qt1-x834.google.com with SMTP id d75a77b69052e-403b3213d8aso40725271cf.0 for ; Tue, 18 Jul 2023 17:01:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kitware.com; s=google; t=1689724906; x=1692316906; h=user-agent:in-reply-to:content-disposition:mime-version:references :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=533ZLUepcTz2gRbDRbQ8O+9TtQ0kvESY2tYUIvYi3JM=; b=OBB0gWyttANtGcQZpCZMcSBfni06HH6ToP+Kmyb2im3L7d9479OOogbzsMbjI/2Po/ Ma9ounvDtwME/iF9e1hpenouy9nV6vf/zmeFw3psc7gKG5ROY0ivPeZ9EOznmXxUHmu8 8q7KS9aNbgS2f0etrLVES+9WWmUjb3Up9+rrI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689724906; x=1692316906; h=user-agent:in-reply-to:content-disposition:mime-version:references :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=533ZLUepcTz2gRbDRbQ8O+9TtQ0kvESY2tYUIvYi3JM=; b=d+Bl6xhhPQoqKkXBo2UKz6Xw4Q7DHyR+M1Ov81LnBcSl2AivaTe2dEf6Na4BAojoZZ i4OEwnA54k+u68lfBAEXhTXMmN/CgyV2Fh7d+grbMluLD6ecNPPIeyzcjvbsmU7TTf2d 0ZBfXwlXSa+rVMm3l2/LZNUIO09B9J8nGe4vb3alu/MIeS4k4PfkfflEKRH32BRWB25k WLSibO8NXfklQGsT+/eUzIcikR5NKo+JwU5uc/cxdfYSK6WPSXHtdB39K7cFKf9o3Nzt 1eGiMHO7Vd6h2wvjelXmNBGgG7L9L4xJ9vbSZNQdayLH1Nna8V9AUJCmzRLGzzIx4Tg2 6Npg== X-Gm-Message-State: ABy/qLY0+Tnc/6kxc3rjx8JSlzPOWR0A91OUwUr8NxwSQFJ5I0gNb6EQ 1qMhbfCKyi2Rl5Vcvv+oAj0Yvg== X-Google-Smtp-Source: APBJJlHgBKqWsPtxHsXvYuA/uBOQiLeoaujbC93CjvnZ1Ajx33kMwN7QSx50I8UK8khLMtdZ7EjhYQ== X-Received: by 2002:a0c:df8f:0:b0:63a:3192:d379 with SMTP id w15-20020a0cdf8f000000b0063a3192d379mr3873490qvl.37.1689724906682; Tue, 18 Jul 2023 17:01:46 -0700 (PDT) Received: from localhost (cpe-142-105-146-128.nycap.res.rr.com. [142.105.146.128]) by smtp.gmail.com with ESMTPSA id i4-20020a0cf484000000b006322ce05eb4sm1064300qvm.100.2023.07.18.17.01.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Jul 2023 17:01:46 -0700 (PDT) Date: Tue, 18 Jul 2023 20:01:44 -0400 From: Ben Boeckel To: Jason Merrill Cc: Nathan Sidwell , gcc-patches@gcc.gnu.org, fortran@gcc.gnu.org, gcc@gcc.gnu.org, brad.king@kitware.com Subject: Re: [PATCH v5 4/5] c++modules: report imported CMI files as dependencies Message-ID: <20230719000144.GA1034956@farprobe> References: <20230125210636.2960049-1-ben.boeckel@kitware.com> <20230125210636.2960049-5-ben.boeckel@kitware.com> <734eef7c-f7c3-519c-e007-cb42f8dc8a82@redhat.com> <20230623024559.GA255658@farprobe> <541367ff-25da-9745-d768-c60f235cd5bc@acm.org> <20230625163610.GC270821@farprobe> <29600358-9421-cbe3-92b6-ab77a1a715d1@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <29600358-9421-cbe3-92b6-ab77a1a715d1@redhat.com> User-Agent: Mutt/2.2.9 (2022-11-12) X-Spam-Status: No, score=-5.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On Tue, Jul 18, 2023 at 16:52:44 -0400, Jason Merrill wrote: > On 6/25/23 12:36, Ben Boeckel wrote: > > On Fri, Jun 23, 2023 at 08:12:41 -0400, Nathan Sidwell wrote: > >> On 6/22/23 22:45, Ben Boeckel wrote: > >>> On Thu, Jun 22, 2023 at 17:21:42 -0400, Jason Merrill wrote: > >>>> On 1/25/23 16:06, Ben Boeckel wrote: > >>>>> They affect the build, so report them via `-MF` mechanisms. > >>>> > >>>> Why isn't this covered by the existing code in preprocessed_module? > >>> > >>> It appears as though it is neutered in patch 3 where > >>> `write_make_modules_deps` is used in `make_write` (or will use that name > >> > >> Why do you want to record the transitive modules? I would expect just noting the > >> ones with imports directly in the TU would suffice (i.e check the 'outermost' arg) > > > > FWIW, only GCC has "fat" modules. MSVC and Clang both require the > > transitive closure to be passed. The idea there is to minimize the size > > of individual module files. > > > > If GCC only reads the "fat" modules, then only those should be recorded. > > If it reads other modules, they should be recorded as well. For clarification, given: * a.cppm ``` export module a; ``` * b.cppm ``` export module b; import a; ``` * use.cppm ``` import b; ``` in a "fat" module setup, `use.cppm` only needs to be told about `b.cmi` because it contains everything that an importer needs to know about the `a` module (reachable types, re-exported bits, whatever). With the "thin" modules, `a.cmi` must be specified when compiling `use.cppm` to satisfy anything that may be required transitively (e.g., a return type of some `b` symbol is from `a`). MSVC and Clang (17+) use this model. I don't know MSVC's rationale, but Clang's is to make CMIs relocatable by not embedding paths to dependent modules in CMIs. This should help caching and network transfer sizes in distributed builds. LLVM/Clang discussion: https://discourse.llvm.org/t/c-20-modules-should-the-bmis-contain-paths-to-their-dependent-bmis/70422 https://github.com/llvm/llvm-project/issues/62707 Maybe I'm missing how this *actually* works in GCC as I've really only interacted with it through the command line, but I've not needed to mention `a.cmi` when compiling `use.cppm`. Is `a.cmi` referenced and read through some embedded information in `b.cmi` or does `b.cmi` include enough information to not need to read it at all? If the former, distributed builds are going to have a problem knowing what files to send just from the command line (I'll call this "implicit thin"). If the latter, that is the "fat" CMI that I'm thinking of. > But wouldn't the transitive modules be dependencies of the direct > imports, so (re)building the direct imports would first require building > the transitive modules anyway? Expressing the transitive closure of > dependencies for each importer seems redundant when it can be easily > derived from the direct dependencies of each module. I'm not concerned whether it is transitive or not, really. If a file is read, it should be reported here regardless of the reason. Note that caching mechanisms may skip actually *doing* the reading, but the dependencies should still be reported from the cached results as-if the real work had been performed. --Ben