From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 15247 invoked by alias); 23 Jun 2015 19:02:50 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Received: (qmail 14788 invoked by uid 89); 23 Jun 2015 19:02:49 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-2.0 required=5.0 tests=AWL,BAYES_00,KAM_LAZY_DOMAIN_SECURITY,RP_MATCHES_RCVD,SPF_HELO_PASS autolearn=ham version=3.3.2 X-HELO: mx1.redhat.com Received: from mx1.redhat.com (HELO mx1.redhat.com) (209.132.183.28) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (AES256-GCM-SHA384 encrypted) ESMTPS; Tue, 23 Jun 2015 19:02:48 +0000 Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by mx1.redhat.com (Postfix) with ESMTPS id 66A9E8E670 for ; Tue, 23 Jun 2015 19:02:47 +0000 (UTC) Received: from localhost.localdomain (ovpn-113-49.phx2.redhat.com [10.3.113.49]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t5NJ2kMG008815; Tue, 23 Jun 2015 15:02:47 -0400 Message-ID: <5589AD56.3010704@redhat.com> Date: Tue, 23 Jun 2015 19:06:00 -0000 From: Jeff Law User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: David Malcolm , Jakub Jelinek CC: gcc-patches@gcc.gnu.org Subject: Re: [PATCH 00/17] RFC: Addding a unit testing framework to gcc References: <1433949898-22033-1-git-send-email-dmalcolm@redhat.com> <20150610153447.GM10247@tucnak.redhat.com> <1433956580.12727.194.camel@surprise> In-Reply-To: <1433956580.12727.194.camel@surprise> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-IsSubscribed: yes X-SW-Source: 2015-06/txt/msg01608.txt.bz2 On 06/10/2015 11:16 AM, David Malcolm wrote: > On Wed, 2015-06-10 at 17:34 +0200, Jakub Jelinek wrote: >> On Wed, Jun 10, 2015 at 11:24:41AM -0400, David Malcolm wrote: >>> I picked the Google Test framework: >>> http://code.google.com/p/googletest/ >> >> I must say I'm not very excited about using this, it won't integrate >> very well with dejagnu > > Why is that a goal? I've been using DejaGnu's unittesting API for > testing the jit, and it is... suboptimal, to put it mildly. Everyone on the project is already using dejagnu. Adding another framework means everyone has to learn new ways of doing the same thing. I think Jakub touched on multilibs, flag passing, result parsing, etc. Note these things are embedded in the daily practices of many developers, build bots and even within our Makefiles for parallization of testing. I'd be less likely to object if the google test framework was wrapped by dejagnu and had those low level things wired up properly. I'd also be less likely to object if we converted *everything* to googletest. The former is most likely orders of magnitude easier :-) > >> whether talking about results (will it provide >> some *.log/*.sum file with FAIL/XFAIL/PASS/XPASS etc. lines?), > > It doesn't have an output formatter for the DejaGnu format, but I guess > I could write one. The gtest standard output format is IMHO superior to > DejaGnu's since it tells you start-of-test/end-of-test on separate > lines, so you can see which test killed things in the event of total > failure, and the per-test timings (which can be disabled if you want to > do diffs). You can still get all that output if you wrap googletest with dejagnu -- that low level information would go into the .log file and the .sum would just have the pass, fail, xfail, xpass, etc messages. > >> choosing >> what options to use, e.g. global >> RUNTESTFLAGS='--target_board=unix/\{-m32,-m64\}' >> to test everything twice for 32-bit and 64-bit, will that run just >> all unittests twice the same?, > > Fair enough; yes, as written, RUNTESTFLAGS is ignored; it will run > everything once. However we could express such a loop directly in the > "main" of the But using the syntax above is already well known by developers and build/test bots. I'm not keen to change it. And note, I *hate* dejagnu. Jeff