From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from delivery.mailspamprotection.com (delivery.mailspamprotection.com [185.56.84.16]) by sourceware.org (Postfix) with ESMTPS id 29F063858D39 for ; Wed, 23 Feb 2022 17:01:36 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 29F063858D39 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=tantosonline.com Authentication-Results: sourceware.org; spf=none smtp.mailfrom=tantosonline.com Received: from 107.112.208.35.bc.googleusercontent.com ([35.208.112.107] helo=siteground277.com) by se22.mailspamprotection.com with esmtps (TLSv1.2:AES128-GCM-SHA256:128) (Exim 4.92) (envelope-from ) id 1nMv18-000FQ1-6W for gcc@gcc.gnu.org; Wed, 23 Feb 2022 11:01:35 -0600 Received: from [127.0.0.1] (port=48436 helo=siteground277.com) by siteground277.com with esmtpa (Exim 4.90-.1) (envelope-from ) id 1nMv17-000JCL-UT; Wed, 23 Feb 2022 17:01:29 +0000 MIME-Version: 1.0 Date: Wed, 23 Feb 2022 09:01:29 -0800 From: andras@tantosonline.com To: Patrick McGehearty Cc: gcc@gcc.gnu.org Subject: Re: Benchmark recommendations needed In-Reply-To: <8a9755e8-4b6f-4407-8331-529fce20a118@oracle.com> References: <33a21357ddb10b187935cf4ea269bb5a7a800c1d.camel@tantosonline.com> <2216839E-05A8-4FBC-B86A-7C51E5DBCA90@comcast.net> <8a9755e8-4b6f-4407-8331-529fce20a118@oracle.com> Message-ID: <45ee07280cf3a63a00f92f1c202376f2@tantosonline.com> X-Sender: andras@tantosonline.com Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - siteground277.com X-AntiAbuse: Original Domain - gcc.gnu.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - tantosonline.com X-Source: X-Source-Args: X-Source-Dir: X-Originating-IP: 35.208.112.107 X-SpamExperts-Domain: siteground277.com X-SpamExperts-Username: 35.208.112.107 Authentication-Results: mailspamprotection.com; auth=pass smtp.auth=35.208.112.107@siteground277.com X-SpamExperts-Outgoing-Class: unsure X-SpamExperts-Outgoing-Evidence: Combined (0.44) X-Recommended-Action: accept X-Filter-ID: Pt3MvcO5N4iKaDQ5O6lkdGlMVN6RH8bjRMzItlySaT9Fi/sEiVj0ksmZQtkyaitEPUtbdvnXkggZ 3YnVId/Y5jcf0yeVQAvfjHznO7+bT5xBytxzJBgOlCIhBVn7zf5DAxQ+GhOWeJSm0X/4oD2rzaZb yM5jq1B5PwH1rWJH2ZUh55uqY3MhMgFAHq5BxPxPuiGzKfvALXq5hcZRyBAyL8N9X9K1DzVKciEj mqsTuLPQAKujagVaTNLZ5X+kLHvN6xDATBhDpuXgBAkFO113Os3sds9OdVAKVgmXR90ppFZj8HWw ctkLpirObDK6bqKBJDyGFXqXClYAZ2r7cPFe/+Iq/hdjQcwui3K93lnKmVeDwIE7VKe+bqpcdCns 72R1SuWb5s34tWzpPhSdjHPlkZYGzbEfYQIUH2PL6aoOtfIHqYdJv+2X6LPgMbH6cRFYzdIkiQcG kUQqfcDh+Efrnmlco+ujNcs/OWWYaVK4U0WwElWJwYXSlJxpgXRxuwVJ9nSulLUhHcJ98hbX5M39 oJYCEdt7JiwYwSs7TRWs7h7EBoS+z86ZMr7hNfmdoVNAlf3O4N9b4mmp9u/1bKFp0Dz9p2VbDMF6 E2S0WnUC7fRxpX13ZoCayowZU6VrO/MvwpHVKZ/ANojJCxy+4SkWr5x5dk0bQRRsQsDS6/1iF07o i6kLlGouP+Tr/CaH7O7rw8fXYGHztprhh9mzXTinzs6J9Wi9SrsI6B7XiiLfLS2tI4g+l6rCWbY0 MZcgnbHstjnA3jCWXS/LoF1jgFbktIJ3mJdgm5kNXP0WDog0Kp5cq9GIVR3A8ffJL0+HiR2fXPjD gCF41UPmepyMRu/VJVKf7C9ptlmr/Bu+1lMmxTWnTNpJlautN2lugYMi185FZdYlzgXGh6mv1ZId Hs3RCvNG1CthBaKxcQ2au7SxMCLY/9Dk/Zq+xfAgcIpGmfyafskAUG3oQpyhLwN3RleVGj+m7Uhj xMbcWSB7mAFgUJkiJvkP8KCN0iFrbX0yf13NwD/3EaCPu1onFa0dn3u5in2hNBmCDHGtmMHPzFon gpJic0YV1E3cK088GNIeQjFlO3w4DuQ2ZoRV8m8KPNnm0ez7DPZta/9edI1uavT0up7wem3oD2AD O5zOuAxVW1j3BP18vW6odrfO8cAzSdWUJ0ynZzGu0tSpqbApRcFFzucbasnvi/7JoEdihhf7K7ud FEADVr73UZ1ivWpwGNEFYA== X-Report-Abuse-To: spam@quarantine1.mailspamprotection.com X-Spam-Status: No, score=-7.8 required=5.0 tests=BAYES_00, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H5, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Feb 2022 17:01:38 -0000 On 2022-02-22 16:59, Patrick McGehearty wrote: > I studied Dhrystone about 30 years ago and found it > had a number of flaws back then. For example, most of > the loops in the code are only executed 1-3 times, which > minimizes the value of hoisting values out of inner loops. > Read the Dhrystone wikipedia article for more information. > > Going back to what benchmarks might be useful... > > you might consider the Livermore Loops > http://www.netlib.org/benchmark/livermorec > > These are 24 kernels (tight loops) originally in Fortran but > ported to C 30 years ago. They are reasonably representative > of floating point computational kernels. They are available > without a fee. > > Even if you have no interest in floating point computation for > your target architecture, examining the assembly output of these > kernels will be helpful in finding where your port of gcc > is doing well and where the machine architecture input to the > various optimizer phases need some tuning. > > You also might review that code and write some modest > test loops of your own for integer code patterns. > > Developing good benchmarks is a skill which requires the > developer to know the intended purpose of the benchmark. > I.e. is our goal to compare optimizer implementations? > or different architectures (i.e. arm vs x86)? > or different implementations of an architecture >    (i.e. intel vs amd or early x86 vs current x86) > or ... > well, you get the idea. > > Good luck, > > - Patrick McGehearty > > > > On 2/22/2022 3:49 PM, Paul Koning via Gcc wrote: >> >>> On Feb 22, 2022, at 4:26 PM, Gary Oblock via Gcc >>> wrote: >>> >>> Andras, >>> >>> The whole point of benchmarks is to judge a processor's performance. >>> That being said, just crippling GCC is not reasonable because >>> processors must be judged in the appropriate context and that >>> includes the current state of the art compiler technology. If you >>> have >>> a new processor I'd benchmark it using the applications you built it >>> for. >> Exactly. Part of what you want to see is that GCC optimizes well for >> the new machine, i.e., that there aren't artifacts of the machine >> description that get in the way of optimization. >> >> So you'd want to use applications that are good exercises not just of >> the code generator but also the optimizer. Dhrystone isn't really >> that, because it has evolved into mostly an optimizer test, not a >> machine or code generator test. >> >> paul >> Thank you for all the feedback. When I said 'crippling' GCC, I didn't mean to make it generally worse, just to make sure I don't trigger dhrystone-specific optimizations, if there are any. Your point about comparing processors is a fair one and that's where I'm heading, but I need to start small. Even just comparing object file (.text section) size between architectures have been hugely enlightening. There is another point though: my port of GCC is certainly not state of the art (well, it technically is, because that's the only port in existence, but you get what I mean). Part of the process is to hone in the port in terms of tuning the instruction selection, peepholes, etc. In some sense this work benchmarks compilers not processors. At any rate, I do get that dhrystone is a poor substitute for a real benchmark and I'll try to move away from it. I wasn't aware of Livermore loops being ported to C (and I don't yet have a Fortran port), so that's a great suggestion. I'll check it out. Thanks again for all the help, Andras