From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 13667 invoked by alias); 28 Nov 2012 11:25:05 -0000 Received: (qmail 13634 invoked by uid 22791); 28 Nov 2012 11:25:03 -0000 X-SWARE-Spam-Status: No, hits=-6.5 required=5.0 tests=AWL,BAYES_00,KHOP_RCVD_UNTRUST,RCVD_IN_DNSWL_HI,RCVD_IN_HOSTKARMA_W,RP_MATCHES_RCVD,SPF_HELO_PASS,TW_LV X-Spam-Check-By: sourceware.org Received: from mx1.redhat.com (HELO mx1.redhat.com) (209.132.183.28) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Wed, 28 Nov 2012 11:24:44 +0000 Received: from int-mx01.intmail.prod.int.phx2.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id qASBOiAJ025923 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Wed, 28 Nov 2012 06:24:44 -0500 Received: from zalov.redhat.com (vpn1-7-172.ams2.redhat.com [10.36.7.172]) by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id qASBOesA013024 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 28 Nov 2012 06:24:43 -0500 Received: from zalov.cz (localhost [127.0.0.1]) by zalov.redhat.com (8.14.5/8.14.5) with ESMTP id qASBOduE004172; Wed, 28 Nov 2012 12:24:39 +0100 Received: (from jakub@localhost) by zalov.cz (8.14.5/8.14.5/Submit) id qASBOdma004171; Wed, 28 Nov 2012 12:24:39 +0100 Date: Wed, 28 Nov 2012 11:25:00 -0000 From: Jakub Jelinek To: Konstantin Serebryany Cc: Wei Mi , GCC Patches , David Li , Diego Novillo , Kostya Serebryany , Dodji Seketeli Subject: Re: [PATCH] asan unit tests from llvm lit-test Message-ID: <20121128112433.GJ2315@tucnak.redhat.com> Reply-To: Jakub Jelinek References: <20121128102445.GH2315@tucnak.redhat.com> <20121128110327.GI2315@tucnak.redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20121128110327.GI2315@tucnak.redhat.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-IsSubscribed: yes Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org X-SW-Source: 2012-11/txt/msg02311.txt.bz2 On Wed, Nov 28, 2012 at 12:03:27PM +0100, Jakub Jelinek wrote: > Then the test would be run once without ASAN_DIE_IF in environment (or =0), > that would produce output full of > EXPECT_DEATH1 AddressSanitizer:.*heap-use-after-free EXPECT_DEATHEND1 > ... > which tcl could parse, and figure from it that it should run the test > again 156 or how many times, with ASAN_DIE_IF from 1 to 156, and at each > iteration try to match the output against the regexp for that iteration. > Then you'd essentially just have to tweak a few lines at the start of the > test, includes, first few lines in main and that would be it. That said, I find it very undesirable to put that many tests into one large one, especially if it triggers undefined behavior like: ATTRIBUTE_NO_ADDRESS_SAFETY_ANALYSIS static void NoAddressSafety() { char *foo = new char[10]; Ident(foo)[10] = 0; delete [] foo; } TEST(AddressSanitizer, AttributeNoAddressSafetyTest) { Ident(NoAddressSafety)(); } As soon as you corrupt malloc state, anything can happen. Things like this should be verified by a compile only test that no instrumentation calls have been added. Looking at the test, there aren't just EXPECT_DEATH kinds of tests, and running everything not guarded with EXPECT_DEATH macro many times might be too expensive. So perhaps it could run each TEST as a separate process, by first just printing all test names that sould be run, then running them one by one by asking for it in env, and for tests that would print EPXECT_DEATHX ... EXPECT_DEATHENDX run that particular test again with the requested EXPECT_DEATH counter. Or run everything except EXPECT_DEATH macros first, and in EXPECT_DEATH printouts print not just some counter, but also name of the current test, and then when rerunning for some particular EXPECT_DEATH just run the corresponding TEST and not all others. Still, it is a couple of dozens of lines in the test (defining the macros) and a little more than that in tcl. Jakub