From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from server28.superhosting.bg (server28.superhosting.bg [217.174.156.11]) by sourceware.org (Postfix) with ESMTPS id BF4E43830B54 for ; Sat, 22 Jun 2024 08:26:26 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org BF4E43830B54 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=dinux.eu Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=dinux.eu ARC-Filter: OpenARC Filter v1.0.0 sourceware.org BF4E43830B54 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=217.174.156.11 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1719044790; cv=none; b=Y9re3tDaW+Ogvw2qloTtxzrmFeKXNi7YveuOJiAAgDtI1jKknD9/Fcj+TRCJDfhXaEi+uo+Luf0wnQ1gxUDAm7DSDjdZPTgGljSgpWcr0kSmwS5bAGmELU23udPNdiRPz3UzyE2tDDblbLMQ/oy7+s+Emy4FcHnVpik467q8QQw= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1719044790; c=relaxed/simple; bh=ZqCF7yuV1jsiwd/O/bbN1HEoy04Qo5zzUy4uvMr3u8U=; h=DKIM-Signature:Date:From:To:Subject:Message-ID:MIME-Version; b=X9r66RLCnHsiwtbLKzxQIZpqpIs8gFzeJv3ZiuuHYt9WvdZpedR/W0ktFDLNmfxSn+BWpK7RizZdJndPhotDAKuwMcxh8lChzL2g0olMrebYZZRhE4JmAVsEKUVzBsmzCFCADbcrw2BoFDKCVuolz75s/BoFwrcYexhvD7377ww= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=dinux.eu; s=default; h=In-Reply-To:Content-Type:MIME-Version:References:Message-ID: Subject:Cc:To:From:Date:Sender:Reply-To:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=3N3qhoFSaBQmMoYXRLzAlSXhfTIJ+NGIo2X6wGWNpvw=; b=C/i3TLYe5FQo1YrK1UK7F9sce7 gjNu/TJTmkLftnHmlMD1GSumy03dKawLIfS8ehFrsA38aXgLCKTkOM5w/hftClloYD7ybSOImkSv5 NUFGX18BH9RwkfLjI7rhzjvsVFeV3aIoc7OimjUEDEV8SxJrZHeIa+RCU2UAmCXj/P7NUZii0cIj8 pl0DtP2eHn8YfPGcLo+iHWgo2kdOh3+SF2NZQFIVpV8QnfG6Ij6zCVwh56RmKc8peRuUjWd4cMxnu Pd9YdHdK/Khf8uoycVodoP+WdrC50gmWNxQ7pG0q9rtFOrz8utPMlYbtsil19cBkLDllVdtZ5h7gU +ETgiYWg==; Received: from [95.42.20.142] (port=60038 helo=localhost) by server28.superhosting.bg with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96.2) (envelope-from ) id 1sKw4k-0004gM-0g; Sat, 22 Jun 2024 11:26:24 +0300 Date: Sat, 22 Jun 2024 11:26:21 +0300 From: Dimitar Dimitrov To: David Malcolm Cc: gcc-patches@gcc.gnu.org Subject: Re: [pushed 2/2] testsuite: check that generated .sarif files validate against the SARIF schema [PR109360] Message-ID: References: <20240621125536.1134456-1-dmalcolm@redhat.com> <20240621125536.1134456-2-dmalcolm@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20240621125536.1134456-2-dmalcolm@redhat.com> X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server28.superhosting.bg X-AntiAbuse: Original Domain - gcc.gnu.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dinux.eu X-Get-Message-Sender-Via: server28.superhosting.bg: authenticated_id: dimitar@dinux.eu X-Authenticated-Sender: server28.superhosting.bg: dimitar@dinux.eu X-Source: X-Source-Args: X-Source-Dir: X-Spam-Status: No, score=-5.4 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,KAM_SHORT,SPF_HELO_PASS,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On Fri, Jun 21, 2024 at 08:55:36AM -0400, David Malcolm wrote: > This patch extends the dg directive verify-sarif-file so that if > the "jsonschema" tool is available, it will be used to validate the > generated .sarif file. > > Tested with jsonschema 3.2 with Python 3.8 Hi David, The new testcase fails on my Fedora 40 with jsonschema 4.19.1 and Python 3.12.3: ``` Executing on host: jsonschema --version (timeout = 300) spawn -ignore SIGHUP jsonschema --version /usr/bin/jsonschema:5: DeprecationWarning: The jsonschema CLI is deprecated and will be removed in a future version. Please use check-jsonschema instead, which can be installed from https://pypi.org/project/check-jsonschema/ from jsonschema.cli import main 4.19.1 FAIL: c-c++-common/analyzer/malloc-sarif-1.c (test .sarif output against SARIF schema) ``` The deprecation warning output seems to confuse DejaGnu and it fails the test. Regards, Dimitar > > With the previous patch, all files generated by the DejaGnu testsuite > validate. > > There were no validation failures in integration testing. > > Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu. > Successful run of analyzer integration tests on x86_64-pc-linux-gnu. > Pushed to trunk as r15-1541-ga84fe222029ff2. > > gcc/ChangeLog: > PR testsuite/109360 > * doc/install.texi: Mention optional usage of "jsonschema" tool. > > gcc/testsuite/ChangeLog: > PR testsuite/109360 > * lib/sarif-schema-2.1.0.json: New file, downloaded from > https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/schemas/sarif-schema-2.1.0.json > Licensing information can be seen at > https://github.com/oasis-tcs/sarif-spec/issues/583 > which states "They are free to incorporate it into their > implementation. No need for special permission or paperwork from > OASIS." > * lib/scansarif.exp (verify-sarif-file): If "jsonschema" is > available, use it to verify that the .sarif file complies with the > SARIF schema. > * lib/target-supports.exp (check_effective_target_jsonschema): > New. > > Signed-off-by: David Malcolm