From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 21362 invoked by alias); 5 Jan 2015 13:32:39 -0000 Mailing-List: contact gdb-patches-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-patches-owner@sourceware.org Received: (qmail 21343 invoked by uid 89); 5 Jan 2015 13:32:37 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-1.7 required=5.0 tests=AWL,BAYES_00,RCVD_IN_DNSWL_NONE,SPF_PASS autolearn=ham version=3.3.2 X-HELO: relay1.mentorg.com Received: from relay1.mentorg.com (HELO relay1.mentorg.com) (192.94.38.131) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Mon, 05 Jan 2015 13:32:35 +0000 Received: from svr-orw-fem-03.mgc.mentorg.com ([147.34.97.39]) by relay1.mentorg.com with esmtp id 1Y87lo-0000qX-9d from Yao_Qi@mentor.com ; Mon, 05 Jan 2015 05:32:32 -0800 Received: from GreenOnly (147.34.91.1) by svr-orw-fem-03.mgc.mentorg.com (147.34.97.39) with Microsoft SMTP Server id 14.3.224.2; Mon, 5 Jan 2015 05:32:31 -0800 From: Yao Qi To: Doug Evans CC: Subject: Re: [RFC] Monster testcase generator for performance testsuite References: Date: Mon, 05 Jan 2015 13:32:00 -0000 In-Reply-To: (Doug Evans's message of "Fri, 2 Jan 2015 02:06:29 -0800") Message-ID: <87mw5xuzdc.fsf@codesourcery.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-IsSubscribed: yes X-SW-Source: 2015-01/txt/msg00045.txt.bz2 Doug Evans writes: Doug, First of all, it is great to have such generator for performance testing, but it doesn't have to be a monster and we don't need parallel build so far. The parallel build will get the generator over-complicated. See more below: > This patch adds preliminary support for generating large programs. > "Large" as in 10000 compunits or 5000 shared libraries or 3M ELF symbols. > Is there any reason we define the workload like this? Can they represent the typical and practical super large program? I feel that the workload you defined is too heavy to be practical, and the overweight causes the long compilation time you mentioned below. > There's still a bit more I want to add to this, but it's at a point > where I can use it, and thus now's a good time to get some feedback. > > One difference between these tests and current perf tests is that > one .exp is used to build the program and another .exp is used to > run the test. These programs take awhile to compile and link. > Generating the sources for these monster testcases takes hardly any time > at all relative to the amount of time to compile them. I measured 13.5 > minutes to compile the included gmonster1 benchmark (with -j6!), and about > an equivalent amount of time to run the benchmark. Therefore it makes > sense to be able to use one program in multiple performance tests, and > therefore it makes sense to separate the build from the test run. Compilation and run takes about 10 minutes respectively. However, I don't understand the importance that making tests running for 10 minutes, which is too long for a perf test case. IMO, a-two-minute-run program should be representative enough... > > These tests currently require separate build-perf and check-perf steps, > which is different from normal perf tests. However, due to the time > it takes to build the program I've added support for building the pieces > of the test in parallel, and hooking this parallel build support into > the existing framework required some pragmatic compromise. ... so the parallel build part may not be needed. > Running the gmonster1-ptype benchmark requires about 8G to link the progr= am, > and 11G to run it under gdb. I still need to add the ability to > have a small version enabled by default, and turn on the bigger version > from the command line. I don't expect everyone to have a big enough > machine to run the test configuration that I do. It looks like a monster rather than a perf test case :) It is good to have a small version enabled by default, which requires less than 1 G, for example, to run it under GDB. How much time it takes to compile (sequential build) and run the small version? --=20 Yao (=E9=BD=90=E5=B0=A7)