From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by sourceware.org (Postfix) with ESMTPS id CBAF83858D33 for ; Thu, 16 Feb 2023 10:20:48 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org CBAF83858D33 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1676542848; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references; bh=yz5UUcXH+4QqIPgB4Yr3P8TZxdZpsDzHl09IGh7MsY8=; b=RxBu6ry1k7DH0YwxqlSMI6XKs5/UMNoLfGPUO0ouZRH00r8s6fcWG2TN5WuUxy81vLhCHY iSwrJ92TyrZDOQx1aPGLAxNqk9t50FUhubK1c8Ti4YdcfMEWEb9k2NCOkP4fcLxWNnFsbH pM8TFO4HBnVyUXazlmsID3hlix2JOns= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-368-Sk56izx4PWubB2C0zD1fIQ-1; Thu, 16 Feb 2023 05:20:45 -0500 X-MC-Unique: Sk56izx4PWubB2C0zD1fIQ-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D6E661C06EE4; Thu, 16 Feb 2023 10:20:44 +0000 (UTC) Received: from tucnak.zalov.cz (unknown [10.39.193.203]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3B5CF2166B30; Thu, 16 Feb 2023 10:20:43 +0000 (UTC) Received: from tucnak.zalov.cz (localhost [127.0.0.1]) by tucnak.zalov.cz (8.17.1/8.17.1) with ESMTPS id 31GAKM9p2902361 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Thu, 16 Feb 2023 11:20:32 +0100 Received: (from jakub@localhost) by tucnak.zalov.cz (8.17.1/8.17.1/Submit) id 31GAKF432902360; Thu, 16 Feb 2023 11:20:15 +0100 Date: Thu, 16 Feb 2023 11:20:15 +0100 From: Jakub Jelinek To: "juzhe.zhong@rivai.ai" Cc: gcc-patches , "kito.cheng" , jeffreyalaw Subject: Re: Re: [PATCH] RISC-V: Add vm* mask C api tests Message-ID: Reply-To: Jakub Jelinek References: <20230216033619.16472-1-juzhe.zhong@rivai.ai> <447EFFD96B546C2E+2023021617534774493316@rivai.ai> MIME-Version: 1.0 In-Reply-To: <447EFFD96B546C2E+2023021617534774493316@rivai.ai> X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Spam-Status: No, score=-3.8 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,KAM_SHORT,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On Thu, Feb 16, 2023 at 05:53:48PM +0800, juzhe.zhong@rivai.ai wrote: > Thanks for reporting this. I think may be we can make reduce tests into 1/3. > For example: > We have: > * gcc.target/riscv/rvv/base/vmand_mm-1.c: New test. > * gcc.target/riscv/rvv/base/vmand_mm-2.c: New test. > * gcc.target/riscv/rvv/base/vmand_mm-3.c: New test. > > Maybe we can reduce it into one test: > vmand_mm.c only. > > I will improve and reduce all intrinsic tests like this soon (I almost done all intrinsic in this week, next week I will do this soon). > > RVV intrinsics are really huge, this is the document: > https://github.com/riscv-non-isa/rvv-intrinsic-doc/tree/master/auto-generated > > The testcases are directly come from LLVM (We just add assembler check into the test), they also have this amount of testcases and the just recently change them: > https://reviews.llvm.org/D142697 > https://reviews.llvm.org/D142644 > > Take a look at the changing LLVM patch, I am aggree with you ,the LLVM patch is quite huge and not easy to maintain. Yeah, LLVM does this all the time, their unit-tests where they embed e.g. matchers for IL in huge tests. I just think the way they are doing this is a very bad idea. If say one writes some C/C++ test, compile it, some helper program adds the IL into comments in the test then again any time you want to adjust something in the compiler that affects those tests, you need to regenerate them. Is the generator included somewhere, or does every user write his own tooling to do that? Anyway, if the solution is regenerate the IL, the test lost quite lot of its meaning, because when changing thousands of tests and regenerating the IL for all of them, one can hardly expect to carefully examine the changes to all those tests whether everything was intended. In GCC we have far fewer such unit-tests and big parts of the testsuite are testing everything from parsing through assembly through linking through runtime. In my experience over the years, many such tests can discover even bugs completely unrelated to the original reason why a test has been added. If they have some generator in LLVM for these riscv tests, even worse, there is another step for LLVM generator regenerates them on the LLVM side and somebody needs to reimport them into GCC and regenerate the scan-assembler regexps. riscv already uses what various other GCC backends use for builtins and intrinsics, various *.def files from which the actual support is created. So, can't we use the same files + something on top of that to have the testsuite coverage, or if it should be independent from it, at least have something similar which would describe intrinsic that should be tested, iterate over such and such types for which arguments and how to come up with the expected emitted code. So, rather than reducing the tests into 1/3, try to reduce them to one line per intrinsic or something of that scale. Jakub