From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qt1-x82b.google.com (mail-qt1-x82b.google.com [IPv6:2607:f8b0:4864:20::82b]) by sourceware.org (Postfix) with ESMTPS id 6ADE53853C25; Tue, 17 Aug 2021 15:01:56 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 6ADE53853C25 Received: by mail-qt1-x82b.google.com with SMTP id e15so17385338qtx.1; Tue, 17 Aug 2021 08:01:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=7Bx+iZxuGHdYA+/5Fb41X7zKb/2L3UVsAjnQOvRjZ0o=; b=pRpqg4Z4AP4ZpyMik2eEboh0yqn6ADGu91Xm6YcT3YDLGcZQyTaQm13cFsk8jKsonH HV8bgUJYSFyv12UvJ+4jPZHs5y5N1dYgqJf5uwm7SL1f+x23KpGoB2OGFXDcaABumppi XgEMwxnKXUGmLACqo7+PVIr11i8hEg8aNKwhTjNZ34nwAki9PZtWa0X2XoOnLjYcZPpr 9AtXps3FMmYhbHv0uk918mYd/iRo3MGXSl4Wgmol5jpOcXPjTfOgV/b8Sz9UI4/H4Hn4 OPo3hXWebFAw6NlbwuTCzlhmDum/hGd3ZEmjfJ5Pk7B1yqJ+v5lI5hhGD8Ajmhli27EF dBCw== X-Gm-Message-State: AOAM532qSnb9ncMLE9vMSKHBvTjNvoMuoE9Hlem0FSxrBYvTpuWw7eLq Nhrzc6LfI2h0g0jdwrKr1cD0uRff4jo= X-Google-Smtp-Source: ABdhPJyLdhkWhDbdAoEguazca3nPPkD+GHgD/UGjswG6EsF0FWqOeKfZKKHJYHI4GxA4UlCvISek3g== X-Received: by 2002:a05:622a:14e:: with SMTP id v14mr3473967qtw.307.1629212515877; Tue, 17 Aug 2021 08:01:55 -0700 (PDT) Received: from [192.168.0.41] (97-118-104-61.hlrn.qwest.net. [97.118.104.61]) by smtp.gmail.com with ESMTPSA id p188sm1515521qka.114.2021.08.17.08.01.54 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Aug 2021 08:01:55 -0700 (PDT) Subject: Re: Expensive selftests To: Thomas Schwinge , David Malcolm Cc: Jonathan Wakely , Richard Biener , gcc@gcc.gnu.org, gcc-patches@gcc.gnu.org References: <87r1f6qzmx.fsf@euler.schwinge.homeip.net> <87czqd7e4k.fsf@euler.schwinge.homeip.net> <55126f14-dda1-2040-0976-70b89582a60c@gmail.com> <874kbopoah.fsf@euler.schwinge.homeip.net> From: Martin Sebor Message-ID: Date: Tue, 17 Aug 2021 09:01:53 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.2.2 MIME-Version: 1.0 In-Reply-To: <874kbopoah.fsf@euler.schwinge.homeip.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-5.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Aug 2021 15:01:58 -0000 On 8/17/21 12:40 AM, Thomas Schwinge wrote: > Hi! > > On 2021-08-16T14:10:00-0600, Martin Sebor wrote: >> On 8/16/21 6:44 AM, Thomas Schwinge wrote: >>> [...], to document the current behavior, I propose to >>> "Add more self-tests for 'hash_map' with Value type with non-trivial >>> constructor/destructor", see attached. OK to push to master branch? >>> (Also cherry-pick into release branches, eventually?) > > (Attached again, for easy reference.) > >> Adding more tests sounds like an excellent idea. I'm not sure about >> the idea of adding loopy selftests that iterate as many times as in >> the patch (looks like 1234 times two?) > > Correct, and I agree it's a sensible concern, generally. > > The current 1234 times two iterations is really arbitrary (should > document that in the test case), just so that we trigger a few hash table > expansions. > > For 'selftest-c', we've got originally: > > -fself-test: 74775 pass(es) in 0.309299 seconds > -fself-test: 74775 pass(es) in 0.366041 seconds > -fself-test: 74775 pass(es) in 0.356663 seconds > -fself-test: 74775 pass(es) in 0.355009 seconds > -fself-test: 74775 pass(es) in 0.367575 seconds > -fself-test: 74775 pass(es) in 0.320406 seconds > > ..., and with my changes we've got: > > -fself-test: 94519 pass(es) in 0.327755 seconds > -fself-test: 94519 pass(es) in 0.369522 seconds > -fself-test: 94519 pass(es) in 0.355531 seconds > -fself-test: 94519 pass(es) in 0.362179 seconds > -fself-test: 94519 pass(es) in 0.363176 seconds > -fself-test: 94519 pass(es) in 0.318930 seconds > > So it really seems to be all in the noise? > > Yet: > >> Selftests run each time GCC >> builds (i.e., even during day to day development). It seems to me >> that it might be better to run such selftests only as part of >> the bootstrap process. > > I'd rather have thought about a '--param self-test-expensive' (or > similar), and then invoke the selftests via a new > 'gcc/testsuite/selftests/expensive.exp' (or similar). > > Or, adapt 'gcc/testsuite/gcc.dg/plugin/expensive_selftests_plugin.c', > that is, invoke them via the GCC plugin mechanism, which also seems to be > easy enough? > > I don't have a strong opinion about where/when these tests get run, so > will happily take any suggestions. I think the right design is to move all these basic building blocks (at a minimum, all containers, but ultimately even higher level general-purpose APIs) into a standalone library with its own unit tests run independently of GCC. I'm fine with adding these tests if no one else is concerned about the overhead, especially with a lower number of iterations like Richard suggests (as long as it still exercises the expansion, of course). Thanks Martin > > > Grüße > Thomas > > > ----------------- > Siemens Electronic Design Automation GmbH; Anschrift: Arnulfstraße 201, 80634 München; Gesellschaft mit beschränkter Haftung; Geschäftsführer: Thomas Heurung, Frank Thürauf; Sitz der Gesellschaft: München; Registergericht München, HRB 106955 >