From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-yw1-x1129.google.com (mail-yw1-x1129.google.com [IPv6:2607:f8b0:4864:20::1129]) by sourceware.org (Postfix) with ESMTPS id 66B783858C2F for ; Wed, 19 Oct 2022 01:43:32 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 66B783858C2F Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-yw1-x1129.google.com with SMTP id 00721157ae682-358bf076f1fso154180037b3.9 for ; Tue, 18 Oct 2022 18:43:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=TNbt6T9pWLyyqXBGHiWEEOLQQzSFjpZ/2xDVNu+aSlc=; b=G9kI6w2miTSJrHoNjcGUWUYolQD0nac8J1+8nmJqxRFhCH8WMge6HeW94jiTEIL6Q0 Xz9/4m70AD44lT/L1xjoCKFvsso4iuuIiDkFLBTVYhWrlLRrtzGNG6UzbYclzwmn0smM q/paljpXx2yD6wkkpSI6LLI6mZ10hP7S4GyF8wUae9owzANC4chzUeeXJIyhBHUuc1vW 7zDhtd6GgV/c3jRcV9fR8IkKTmeepdNUJF9ATz7szh+P/cy5/O82hFkE/6jHI5ok8+he flGCZO+xUOLoX2392YtupAN+81eaaK4D8eSpuQGhiJqurkvC5tYzdF99gC+7YH3eIuMR o9ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=TNbt6T9pWLyyqXBGHiWEEOLQQzSFjpZ/2xDVNu+aSlc=; b=t/NUqjAqs8FFF3wOczJe8zE8VwFkF/tplA4NBMJo9BK/tz1HwgyZ/dvfvCVRb65Uwt PjU2+sPum9qao6ZZeFSE3Yy5AUSMlfdN8gSsOWCiCaADYFp2IKiz3P0yqs7fnlYkbAkv HugvI1z92nunUiv3tvrAvrfFFdr7+JT/iciSJ+HltkrXYt+3eHZEzXwM+8OUg4EHcbJi tJdXBoAmfhqguoJe6s9DxL/k2KUOV/2TAesrBZCp5SRJBCbNI4gDcYhAVAKdDNVYtUsw pjWonpbmF8NnfipRlCbXAg6057dXrj6D4CXsPa1vwV+tXHqwwB/QuIn/YPwKJpXtTIMA Uzug== X-Gm-Message-State: ACrzQf0SaPyNw4zySKcxRrMRuXJ91/ulo0Vei933SRL0wr4OVcQqclb4 mmoEs6u9UQYP+yQmCtCgNdJYDHlj6VXlwhwbBuA= X-Google-Smtp-Source: AMsMyM6Qqyf/uoppTgUjBOGT+x1h5o2zXe0UNuOTRUVUF0CJCRdkoA5uWeKPc6JZPGANSNuFh48Ct4c4Sux7Vh+Ng+g= X-Received: by 2002:a0d:df0b:0:b0:358:83d4:95a2 with SMTP id i11-20020a0ddf0b000000b0035883d495a2mr4814964ywe.475.1666143811722; Tue, 18 Oct 2022 18:43:31 -0700 (PDT) MIME-Version: 1.0 References: <20221014075445.7938-1-haochen.jiang@intel.com> <20221018091727.82856-1-haochen.jiang@intel.com> In-Reply-To: <20221018091727.82856-1-haochen.jiang@intel.com> From: Hongtao Liu Date: Wed, 19 Oct 2022 09:43:20 +0800 Message-ID: Subject: Re: [PATCH] i386: Auto vectorize sdot_prod, udot_prod with VNNIINT8 instruction. To: Haochen Jiang Cc: gcc-patches@gcc.gnu.org, hongtao.liu@intel.com Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-7.9 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,GIT_PATCH_0,KAM_SHORT,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On Tue, Oct 18, 2022 at 5:18 PM Haochen Jiang via Gcc-patches wrote: > > Hi all, > > We would like to add one more patch to enhance the codegen with avxvnniint8. > Also renamed two awkward named mode_attr to make them more aligned with others. > > Regtested on x86_64-pc-linux-gnu. Ok for trunk? The patch LGTM, but please commit after [1] is checked in. [1] https://gcc.gnu.org/pipermail/gcc-patches/2022-October/603782.html > > BRs, > Haochen > > gcc/ChangeLog: > > * config/i386/sse.md (ssedvecmode): Rename from VI1SI. > (ssedvecmodelower): Rename from vi1si. > (sdot_prod): New define_expand. > (udot_prod): Ditto. > > gcc/testsuite/ChangeLog: > > * gcc.target/i386/vnniint8-auto-vectorize-1.c: New test. > * gcc.target/i386/vnniint8-auto-vectorize-2.c: Ditto. > --- > gcc/config/i386/sse.md | 61 ++++++++++++--- > .../i386/vnniint8-auto-vectorize-1.c | 28 +++++++ > .../i386/vnniint8-auto-vectorize-2.c | 75 +++++++++++++++++++ > 3 files changed, 153 insertions(+), 11 deletions(-) > create mode 100644 gcc/testsuite/gcc.target/i386/vnniint8-auto-vectorize-1.c > create mode 100644 gcc/testsuite/gcc.target/i386/vnniint8-auto-vectorize-2.c > > diff --git a/gcc/config/i386/sse.md b/gcc/config/i386/sse.md > index 29cf6fa090b..fc17b5193dc 100644 > --- a/gcc/config/i386/sse.md > +++ b/gcc/config/i386/sse.md > @@ -1043,6 +1043,13 @@ > (V16HI "v16hi") (V8HI "v8hi") > (V32QI "v32qi") (V16QI "v16qi")]) > > +;; Mapping of vector modes to an V*SImode of the same size > +(define_mode_attr ssedvecmode > + [(V64QI "V16SI") (V32QI "V8SI") (V16QI "V4SI")]) > + > +(define_mode_attr ssedvecmodelower > + [(V64QI "v16si") (V32QI "v8si") (V16QI "v4si")]) > + > ;; Mapping of vector modes to a vector mode of double size > (define_mode_attr ssedoublevecmode > [(V64QI "V128QI") (V32HI "V64HI") (V16SI "V32SI") (V8DI "V16DI") > @@ -28523,29 +28530,23 @@ > [(set_attr ("prefix") ("evex")) > (set_attr "mode" "")]) > > -(define_mode_attr VI1SI > - [(V64QI "V16SI") (V32QI "V8SI") (V16QI "V4SI")]) > - > -(define_mode_attr vi1si > - [(V64QI "v16si") (V32QI "v8si") (V16QI "v4si")]) > - > (define_expand "usdot_prod" > - [(match_operand: 0 "register_operand") > + [(match_operand: 0 "register_operand") > (match_operand:VI1_AVX512VNNI 1 "register_operand") > (match_operand:VI1_AVX512VNNI 2 "register_operand") > - (match_operand: 3 "register_operand")] > + (match_operand: 3 "register_operand")] > "( == 64 > ||((TARGET_AVX512VNNI && TARGET_AVX512VL) > || TARGET_AVXVNNI))" > { > - operands[1] = lowpart_subreg (mode, > + operands[1] = lowpart_subreg (mode, > force_reg (mode, operands[1]), > mode); > - operands[2] = lowpart_subreg (mode, > + operands[2] = lowpart_subreg (mode, > force_reg (mode, operands[2]), > mode); > emit_insn (gen_rtx_SET (operands[0], operands[3])); > - emit_insn (gen_vpdpbusd_ (operands[0], operands[3], > + emit_insn (gen_vpdpbusd_ (operands[0], operands[3], > operands[1], operands[2])); > DONE; > }) > @@ -29358,6 +29359,44 @@ > (UNSPEC_VPDPBSUD "bsud") (UNSPEC_VPDPBSUDS "bsuds") > (UNSPEC_VPDPBUUD "buud") (UNSPEC_VPDPBUUDS "buuds")]) > > +(define_expand "sdot_prod" > + [(match_operand: 0 "register_operand") > + (match_operand:VI1 1 "register_operand") > + (match_operand:VI1 2 "register_operand") > + (match_operand: 3 "register_operand")] > + "TARGET_AVXVNNIINT8" > +{ > + operands[1] = lowpart_subreg (mode, > + force_reg (mode, operands[1]), > + mode); > + operands[2] = lowpart_subreg (mode, > + force_reg (mode, operands[2]), > + mode); > + emit_insn (gen_rtx_SET (operands[0], operands[3])); > + emit_insn (gen_vpdpbssd_ (operands[0], operands[3], > + operands[1], operands[2])); > + DONE; > +}) > + > +(define_expand "udot_prod" > + [(match_operand: 0 "register_operand") > + (match_operand:VI1 1 "register_operand") > + (match_operand:VI1 2 "register_operand") > + (match_operand: 3 "register_operand")] > + "TARGET_AVXVNNIINT8" > +{ > + operands[1] = lowpart_subreg (mode, > + force_reg (mode, operands[1]), > + mode); > + operands[2] = lowpart_subreg (mode, > + force_reg (mode, operands[2]), > + mode); > + emit_insn (gen_rtx_SET (operands[0], operands[3])); > + emit_insn (gen_vpdpbuud_ (operands[0], operands[3], > + operands[1], operands[2])); > + DONE; > +}) > + > (define_insn "vpdp_" > [(set (match_operand:VI4_AVX 0 "register_operand" "=x") > (unspec:VI4_AVX > diff --git a/gcc/testsuite/gcc.target/i386/vnniint8-auto-vectorize-1.c b/gcc/testsuite/gcc.target/i386/vnniint8-auto-vectorize-1.c > new file mode 100644 > index 00000000000..9cadab6a845 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/i386/vnniint8-auto-vectorize-1.c > @@ -0,0 +1,28 @@ > +/* { dg-do compile } */ > +/* { dg-options "-mavxvnniint8 -O2" } */ > +/* { dg-final { scan-assembler "vpdpbssd\t" } } */ > +/* { dg-final { scan-assembler "vpdpbuud\t" } } */ > + > +int __attribute__((noinline, noclone, optimize("tree-vectorize"))) > +sdot_prod_qi (char * restrict a, char * restrict b, > + int c, int n) > +{ > + int i; > + for (i = 0; i < n; i++) > + { > + c += ((int) a[i] * (int) b[i]); > + } > + return c; > +} > + > +int __attribute__((noinline, noclone, optimize("tree-vectorize"))) > +udot_prod_qi (unsigned char * restrict a, unsigned char *restrict b, > + int c, int n) > +{ > + int i; > + for (i = 0; i < n; i++) > + { > + c += ((int) a[i] * (int) b[i]); > + } > + return c; > +} > diff --git a/gcc/testsuite/gcc.target/i386/vnniint8-auto-vectorize-2.c b/gcc/testsuite/gcc.target/i386/vnniint8-auto-vectorize-2.c > new file mode 100644 > index 00000000000..99853e6c3b7 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/i386/vnniint8-auto-vectorize-2.c > @@ -0,0 +1,75 @@ > +/* { dg-do run } */ > +/* { dg-options "-O2 -mavxvnniint8" } */ > +/* { dg-require-effective-target avxvnniint8 } */ > + > +#define AVXVNNIINT8 > +#ifndef CHECK > +#define CHECK "avx-check.h" > +#endif > + > +#ifndef TEST > +#define TEST avx_test > +#endif > + > +#include CHECK > +#include "vnniint8-auto-vectorize-1.c" > + > +#define N 256 > +char a_i8[N], b_i8[N]; > +unsigned char c_u8[N], d_u8[N]; > +int i8_exp, i8_ref; > + > +int __attribute__((noipa, optimize("no-tree-vectorize"))) > +sdot_prod_qi_scalar (char * restrict a, char * restrict b, > + int c, int n) > +{ > + int i; > + for (i = 0; i < n; i++) > + { > + c += ((int) a[i] * (int) b[i]); > + } > + return c; > +} > + > +int __attribute__((noipa, optimize("no-tree-vectorize"))) > +udot_prod_qi_scalar (unsigned char * restrict a, unsigned char *restrict b, > + int c, int n) > +{ > + int i; > + for (i = 0; i < n; i++) > + { > + c += ((int) a[i] * (int) b[i]); > + } > + return c; > +} > + > +void init () > +{ > + int i; > + > + i8_exp = i8_ref = 127; > + > + for (i = 0; i < N; i++) > + { > + a_i8[i] = (-i + 4) % 128; > + b_i8[i] = (i + 1) % 128; > + c_u8[i] = (i + 3) % 256; > + d_u8[i] = (i + 5) % 256; > + } > +} > + > +void > +TEST (void) > +{ > + init (); > + i8_exp = sdot_prod_qi (a_i8, b_i8, i8_exp, N); > + i8_ref = sdot_prod_qi_scalar (a_i8, b_i8, i8_ref, N); > + if (i8_exp != i8_ref) > + abort (); > + > + init (); > + i8_exp = udot_prod_qi (c_u8, d_u8, i8_exp, N); > + i8_ref = udot_prod_qi_scalar (c_u8, d_u8, i8_ref, N); > + if (i8_exp != i8_ref) > + abort (); > +} > -- > 2.18.1 > -- BR, Hongtao