From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qv1-xf2e.google.com (mail-qv1-xf2e.google.com [IPv6:2607:f8b0:4864:20::f2e]) by sourceware.org (Postfix) with ESMTPS id 34BA83830B69 for ; Fri, 1 Sep 2023 13:04:21 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 34BA83830B69 Authentication-Results: sourceware.org; dmarc=pass (p=reject dis=none) header.from=kitware.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=kitware.com Received: by mail-qv1-xf2e.google.com with SMTP id 6a1803df08f44-64f4ac604c2so9672266d6.1 for ; Fri, 01 Sep 2023 06:04:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kitware.com; s=google; t=1693573460; x=1694178260; darn=gcc.gnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4WOpSU81BFy+TUWfWtmlVIfS2uHQn9f9fi5pb17KBU4=; b=c9Ap4oso+odoa6haS60x0/47oyXm+X85hLkrMP7jarrcp64bvS56hEBxFw2MdMAj+s lLDidtgV0qADfxG13ZtnhPaCR6O1SzVntEzWccH1WMYMdkmdWVzB6ahnL3HkwDq662Yc o9OR9y6pKCtcVUE8asbIEoEz/IH8CbBjf+E7o= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693573460; x=1694178260; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4WOpSU81BFy+TUWfWtmlVIfS2uHQn9f9fi5pb17KBU4=; b=J6Lpo8vok3YIrMGbUQYExLpUHe8qDgilH01N2ci4Bmtv5WT9QEd1hrhlKAYMemIVJD /xb3vuo2thrYIJjtnIa1UbX28UNhZ1woLgNxD7vyKPJ1lcNimHyKKOSb+0QEMG+A+gJv gBJXXzo1oCecrbmAtbZPzVUGax3R09AIcavRB8pw+r2hkFKGhL3wso2bYDM7hGdlDzJz vdyVi9U3eIaRbbe6O3ZvcMw2gJ7Nyyg0tqGlPg55Xjfj0r7OZUtA7dkRIMv3kmzFq8Yt YMg/20ndusoa22RTYsUZRERHG9ialzbMN3anvz+M5yZ5UtsVWyqBu3bZtJEcE+kHPHBd 89LQ== X-Gm-Message-State: AOJu0YzuFuogmzuOY43bhSc6CKyyuCuoPaKJBGdI5lDY3E2kPIBAk6oC +6mEz4kg6KYxA9JEKpn7wq1ymg== X-Google-Smtp-Source: AGHT+IHE0itQVOZyCztFSRMgEqakMkTH4UDpa+hfXg0CJfJBydsWVulXLIePUlMe3RupM6r0q1IS1A== X-Received: by 2002:a0c:8e41:0:b0:641:e7b8:6a18 with SMTP id w1-20020a0c8e41000000b00641e7b86a18mr2240182qvb.56.1693573460113; Fri, 01 Sep 2023 06:04:20 -0700 (PDT) Received: from localhost (cpe-142-105-146-128.nycap.res.rr.com. [142.105.146.128]) by smtp.gmail.com with ESMTPSA id v5-20020a0ce1c5000000b0064f4ec872dfsm1434448qvl.0.2023.09.01.06.04.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Sep 2023 06:04:19 -0700 (PDT) From: Ben Boeckel To: gcc-patches@gcc.gnu.org Cc: Ben Boeckel , jason@redhat.com, nathan@acm.org, fortran@gcc.gnu.org, gcc@gcc.gnu.org, brad.king@kitware.com Subject: [PATCH v8 3/4] c++modules: report imported CMI files as dependencies Date: Fri, 1 Sep 2023 09:04:03 -0400 Message-ID: <20230901130407.259768-4-ben.boeckel@kitware.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230901130407.259768-1-ben.boeckel@kitware.com> References: <20230901130407.259768-1-ben.boeckel@kitware.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-13.0 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,GIT_PATCH_0,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: They affect the build, so report them via `-MF` mechanisms. gcc/cp/ * module.cc (do_import): Report imported CMI files as dependencies. gcc/testsuite/ * g++.dg/modules/depreport-1_a.C: New test. * g++.dg/modules/depreport-1_b.C: New test. * g++.dg/modules/test-depfile.py: New tool for validating depfile information. * lib/modules.exp: Support for validating depfile contents. Signed-off-by: Ben Boeckel --- gcc/cp/module.cc | 3 + gcc/testsuite/g++.dg/modules/depreport-1_a.C | 10 + gcc/testsuite/g++.dg/modules/depreport-1_b.C | 12 ++ gcc/testsuite/g++.dg/modules/test-depfile.py | 187 +++++++++++++++++++ gcc/testsuite/lib/modules.exp | 29 +++ 5 files changed, 241 insertions(+) create mode 100644 gcc/testsuite/g++.dg/modules/depreport-1_a.C create mode 100644 gcc/testsuite/g++.dg/modules/depreport-1_b.C create mode 100644 gcc/testsuite/g++.dg/modules/test-depfile.py diff --git a/gcc/cp/module.cc b/gcc/cp/module.cc index 9df60d695b1..f3acc4e02fe 100644 --- a/gcc/cp/module.cc +++ b/gcc/cp/module.cc @@ -18968,6 +18968,9 @@ module_state::do_import (cpp_reader *reader, bool outermost) dump () && dump ("CMI is %s", file); if (note_module_cmi_yes || inform_cmi_p) inform (loc, "reading CMI %qs", file); + /* Add the CMI file to the dependency tracking. */ + if (cpp_get_deps (reader)) + deps_add_dep (cpp_get_deps (reader), file); fd = open (file, O_RDONLY | O_CLOEXEC | O_BINARY); e = errno; } diff --git a/gcc/testsuite/g++.dg/modules/depreport-1_a.C b/gcc/testsuite/g++.dg/modules/depreport-1_a.C new file mode 100644 index 00000000000..241701728a2 --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/depreport-1_a.C @@ -0,0 +1,10 @@ +// { dg-additional-options -fmodules-ts } + +export module Foo; +// { dg-module-cmi Foo } + +export class Base +{ +public: + int m; +}; diff --git a/gcc/testsuite/g++.dg/modules/depreport-1_b.C b/gcc/testsuite/g++.dg/modules/depreport-1_b.C new file mode 100644 index 00000000000..b6e317c6703 --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/depreport-1_b.C @@ -0,0 +1,12 @@ +// { dg-additional-options -fmodules-ts } +// { dg-additional-options -MD } +// { dg-additional-options "-MF depreport-1.d" } + +import Foo; + +void foo () +{ + Base b; +} + +// { dg-final { run-check-module-dep-expect-input "depreport-1.d" "gcm.cache/Foo.gcm" } } diff --git a/gcc/testsuite/g++.dg/modules/test-depfile.py b/gcc/testsuite/g++.dg/modules/test-depfile.py new file mode 100644 index 00000000000..ea4edb61434 --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/test-depfile.py @@ -0,0 +1,187 @@ +import json + + +# Parameters. +ALL_ERRORS = False + + +def _report_error(msg): + '''Report an error.''' + full_msg = 'ERROR: ' + msg + if ALL_ERRORS: + print(full_msg) + else: + raise RuntimeError(full_msg) + + +class Token(object): + pass + + +class Output(Token): + def __init__(self, path): + self.path = path + + +class Input(Token): + def __init__(self, path): + self.path = path + + +class Colon(Token): + pass + + +class Append(Token): + pass + + +class Variable(Token): + def __init__(self, name): + self.name = name + + +class Word(Token): + def __init__(self, name): + self.name = name + + +def validate_depfile(depfile, expect_input=None): + '''Validate a depfile contains some information + + Returns `False` if the information is not found. + ''' + with open(depfile, 'r') as fin: + depfile_content = fin.read() + + real_lines = [] + join_line = False + for line in depfile_content.split('\n'): + # Join the line if needed. + if join_line: + line = real_lines.pop() + line + + # Detect line continuations. + join_line = line.endswith('\\') + # Strip line continuation characters. + if join_line: + line = line[:-1] + + # Add to the real line set. + real_lines.append(line) + + # Perform tokenization. + tokenized_lines = [] + for line in real_lines: + tokenized = [] + join_word = False + for word in line.split(' '): + if join_word: + word = tokenized.pop() + ' ' + word + + # Detect word joins. + join_word = word.endswith('\\') + # Strip escape character. + if join_word: + word = word[:-1] + + # Detect `:` at the end of a word. + if word.endswith(':'): + tokenized.append(word[:-1]) + word = word[-1] + + # Add word to the tokenized set. + tokenized.append(word) + + tokenized_lines.append(tokenized) + + # Parse. + ast = [] + for line in tokenized_lines: + kind = None + for token in line: + if token == ':': + kind = 'dependency' + elif token == '+=': + kind = 'append' + if line == ['']: + kind = 'empty' + + if kind is None: + _report_error('unknown line kind: %s' % line) + + line_parse = [] + if kind == 'dependency': + after_colon = False + for token in line: + if token == ':': + after_colon = True + elif after_colon: + line_parse.append(Input(token)) + else: + line_parse.append(Output(token)) + elif kind == 'append': + after_op = False + for token in line: + if token == '+=': + after_op = True + elif after_op: + line_parse.append(Word(token)) + else: + line_parse.append(Variable(token)) + + ast.append(line_parse) + + for node in ast: + for token in node: + if expect_input is not None: + # If the input is found, clear the expectation. + if isinstance(token, Input) and token.path == expect_input: + expect_input = None + + result = True + if expect_input: + _report_error('missing input: %s' % expect_input) + result = False + + return result + + +if __name__ == '__main__': + import sys + + depfile = None + have_expect = False + expect_input = None + + # Parse arguments. + args = sys.argv[1:] + while args: + # Take an argument. + arg = args.pop(0) + + # Flag to change how errors are reported. + if arg == '-A' or arg == '--all': + ALL_ERRORS = True + # Required arguments. + elif arg == '-d' or arg == '--depfile': + depfile = args.pop(0) + elif arg == '-i' or arg == '--expect-input': + expect_input = args.pop(0) + have_expect = True + + # Validate that we have the required arguments. + if depfile is None: + raise RuntimeError('missing "depfile" file') + if have_expect is None: + raise RuntimeError('missing an "expect" argument') + + # Do the actual work. + try: + is_ok = validate_depfile(depfile, expect_input=expect_input) + except BaseException as e: + _report_error('exception: %s' % e) + + # Fail if errors are found. + if not is_ok: + sys.exit(1) diff --git a/gcc/testsuite/lib/modules.exp b/gcc/testsuite/lib/modules.exp index d466a73cbe1..e24ee7618d9 100644 --- a/gcc/testsuite/lib/modules.exp +++ b/gcc/testsuite/lib/modules.exp @@ -69,3 +69,32 @@ proc run-check-p1689-valid { depfile template } { clean-p1689 $testcase } + +proc run-check-module-dep { depfile flag expected } { + global srcdir subdir + # Extract the test file name from the arguments. + set testcase [file rootname [file tail $depfile]] + + verbose "Verifying dependencies for $testcase in $srcdir/$subdir" 2 + set testcase [remote_download host $testcase] + + set pytest_script "test-depfile.py" + if { ![check_effective_target_recent_python3] } { + unsupported "$pytest_script python3 is missing" + return + } + + verbose "running script test-depfile.py" 1 + spawn -noecho python3 $srcdir/$subdir/$pytest_script --all --depfile $depfile $flag $expected + + expect { + -re "ERROR: (\[^\r\n\]*)" { + fail $expect_out(0,string) + exp_continue + } + } +} + +proc run-check-module-dep-expect-input { depfile expected } { + run-check-module-dep $depfile "--expect-input" $expected +} -- 2.41.0