From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <gcc-bugzilla@gcc.gnu.org>
Received: by sourceware.org (Postfix, from userid 48)
 id 8E5F53858C3B; Fri, 24 Sep 2021 06:15:56 +0000 (GMT)
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 8E5F53858C3B
From: "rguenther at suse dot de" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug analyzer/102471] RFE: add support to analyzer testsuite for
 running SAMATE/SARD tests (e.g. Juliet Test Suite)
Date: Fri, 24 Sep 2021 06:15:56 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: analyzer
X-Bugzilla-Version: 12.0
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: normal
X-Bugzilla-Who: rguenther at suse dot de
X-Bugzilla-Status: UNCONFIRMED
X-Bugzilla-Resolution: 
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: dmalcolm at gcc dot gnu.org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags: 
X-Bugzilla-Changed-Fields: 
Message-ID: <bug-102471-4-LrdVSiA91G@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-102471-4@http.gcc.gnu.org/bugzilla/>
References: <bug-102471-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-BeenThere: gcc-bugs@gcc.gnu.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Gcc-bugs mailing list <gcc-bugs.gcc.gnu.org>
List-Unsubscribe: <https://gcc.gnu.org/mailman/options/gcc-bugs>,
 <mailto:gcc-bugs-request@gcc.gnu.org?subject=unsubscribe>
List-Archive: <https://gcc.gnu.org/pipermail/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-request@gcc.gnu.org?subject=help>
List-Subscribe: <https://gcc.gnu.org/mailman/listinfo/gcc-bugs>,
 <mailto:gcc-bugs-request@gcc.gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 24 Sep 2021 06:15:56 -0000

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=3D102471

--- Comment #1 from rguenther at suse dot de <rguenther at suse dot de> ---
On Thu, 23 Sep 2021, dmalcolm at gcc dot gnu.org wrote:

> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=3D102471
>=20
>             Bug ID: 102471
>            Summary: RFE: add support to analyzer testsuite for running
>                     SAMATE/SARD tests (e.g. Juliet Test Suite)
>            Product: gcc
>            Version: 12.0
>             Status: UNCONFIRMED
>           Severity: normal
>           Priority: P3
>          Component: analyzer
>           Assignee: dmalcolm at gcc dot gnu.org
>           Reporter: dmalcolm at gcc dot gnu.org
>                 CC: rguenth at gcc dot gnu.org
>   Target Milestone: ---
>=20
> See:
>   https://www.nist.gov/itl/ssd/software-quality-group/samate
>   https://samate.nist.gov/SARD/testsuite.php
>=20
> The links above have various promising-looking testsuites e.g.
> - Juliet Test Suite
> - Klocwork test suite
> - ITC-Benchmarks
> etc
>=20
> It would be good to be able to (somehow) automatically run them as part of
> regression testing of the analyzer - either by turning them directly into
> DejaGnu tests, or by wrapping the suite's own harness in a way that we can
> invoke it during "make check".

It might be also feasible to think of on-the-side CI runs of
external testsuites on buildbots or similar and have results
reported on gcc.gnu.org.=