From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com [IPv6:2a00:1450:4864:20::536]) by sourceware.org (Postfix) with ESMTPS id 32CFB385743E for ; Fri, 6 Aug 2021 06:44:29 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 32CFB385743E Received: by mail-ed1-x536.google.com with SMTP id cf5so11711870edb.2 for ; Thu, 05 Aug 2021 23:44:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=OW0RSfCJ5ouBcCYwkbQDFhhfcKCItZiTVzj8gssaWuE=; b=Mf+1ZfkWlF9oi0pjSgH47crNRPuYEc8NQeDkt121FGuSSRCnjgjqyJNtkau3MoR+/Y 2VqXe8pdaCOym2O1I0LCiVR6nZCh+B3aGVwb26zKpCaMA2LDAsOXi2CfnE5MRxUCSL47 qadxD3nwKt2IcTIdc3mux/UVDHbxd82ko20XiL8xEOA4Eofz6Kk3zGqiOJlPukfe+B+4 lZQHZLRFnx6/I3Kvd0A850stZXxEurMd61DMmEqwplrgcmrxsaezViL2lViuIP4ef8IK xlc8ftggh7MFi4MJqtUKhFJGUJK69Crw5clDoL+ULkOKVIoOjtnmo6Bcbh9Zv3ZrPSeZ jFuA== X-Gm-Message-State: AOAM533S/4iu2617D3A2sJSOp7EL5eOmTA7C0M+MSw6MlCPh+EanlWgN SE3LNXWo0kMBu4PuFCDs9y/bIRuB8WwuM9Xgb20= X-Google-Smtp-Source: ABdhPJwUBsSnC0uIxfEM+VDcB5DGsrRRNQbTCz7n82BWjkV8r4OgWq0e9Twtf8pX5l3gLLo1xfDtOrzVxKhW8Hoz2O8= X-Received: by 2002:a05:6402:1603:: with SMTP id f3mr11128032edv.274.1628232268298; Thu, 05 Aug 2021 23:44:28 -0700 (PDT) MIME-Version: 1.0 References: <98s2q47-607-130-21p-or6454oprsso@fhfr.qr> In-Reply-To: From: Richard Biener Date: Fri, 6 Aug 2021 08:44:17 +0200 Message-ID: Subject: Re: [PATCH 1/2] Add emulated gather capability to the vectorizer To: Christophe Lyon Cc: Richard Biener , Richard Sandiford , Hongtao Liu , GCC Patches Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-9.0 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Aug 2021 06:44:31 -0000 On Thu, Aug 5, 2021 at 3:26 PM Christophe Lyon via Gcc-patches wrote: > > On Thu, Aug 5, 2021 at 11:53 AM Richard Biener wrote: > > > On Thu, 5 Aug 2021, Christophe Lyon wrote: > > > > > On Wed, Aug 4, 2021 at 2:08 PM Richard Biener wrote: > > > > > > > On Wed, 4 Aug 2021, Richard Sandiford wrote: > > > > > > > > > Richard Biener writes: > > > > > > This adds a gather vectorization capability to the vectorizer > > > > > > without target support by decomposing the offset vector, doing > > > > > > sclar loads and then building a vector from the result. This > > > > > > is aimed mainly at cases where vectorizing the rest of the loop > > > > > > offsets the cost of vectorizing the gather. > > > > > > > > > > > > Note it's difficult to avoid vectorizing the offset load, but in > > > > > > some cases later passes can turn the vector load + extract into > > > > > > scalar loads, see the followup patch. > > > > > > > > > > > > On SPEC CPU 2017 510.parest_r this improves runtime from 250s > > > > > > to 219s on a Zen2 CPU which has its native gather instructions > > > > > > disabled (using those the runtime instead increases to 254s) > > > > > > using -Ofast -march=znver2 [-flto]. It turns out the critical > > > > > > loops in this benchmark all perform gather operations. > > > > > > > > > > > > Bootstrapped and tested on x86_64-unknown-linux-gnu. > > > > > > > > > > > > 2021-07-30 Richard Biener > > > > > > > > > > > > * tree-vect-data-refs.c (vect_check_gather_scatter): > > > > > > Include widening conversions only when the result is > > > > > > still handed by native gather or the current offset > > > > > > size not already matches the data size. > > > > > > Also succeed analysis in case there's no native support, > > > > > > noted by a IFN_LAST ifn and a NULL decl. > > > > > > (vect_analyze_data_refs): Always consider gathers. > > > > > > * tree-vect-patterns.c (vect_recog_gather_scatter_pattern): > > > > > > Test for no IFN gather rather than decl gather. > > > > > > * tree-vect-stmts.c (vect_model_load_cost): Pass in the > > > > > > gather-scatter info and cost emulated gathers accordingly. > > > > > > (vect_truncate_gather_scatter_offset): Properly test for > > > > > > no IFN gather. > > > > > > (vect_use_strided_gather_scatters_p): Likewise. > > > > > > (get_load_store_type): Handle emulated gathers and its > > > > > > restrictions. > > > > > > (vectorizable_load): Likewise. Emulate them by extracting > > > > > > scalar offsets, doing scalar loads and a vector construct. > > > > > > > > > > > > * gcc.target/i386/vect-gather-1.c: New testcase. > > > > > > * gfortran.dg/vect/vect-8.f90: Adjust. > > > > > > > > > > Hi, > > > > > > The adjusted testcase now fails on aarch64: > > > FAIL: gfortran.dg/vect/vect-8.f90 -O scan-tree-dump-times vect > > > "vectorized 23 loops" 1 > > > > That likely means it needs adjustment for the aarch64 case as well > > which I didn't touch. I suppose it's now vectorizing 24 loops? > > And 24 with SVE as well, so we might be able to merge the > > aarch64_sve and aarch64 && ! aarch64_sve cases? > > > > Like with > > > > diff --git a/gcc/testsuite/gfortran.dg/vect/vect-8.f90 > > b/gcc/testsuite/gfortran.dg/vect/vect-8.f90 > > index cc1aebfbd84..c8a7d896bac 100644 > > --- a/gcc/testsuite/gfortran.dg/vect/vect-8.f90 > > +++ b/gcc/testsuite/gfortran.dg/vect/vect-8.f90 > > @@ -704,7 +704,6 @@ CALL track('KERNEL ') > > RETURN > > END SUBROUTINE kernel > > > > -! { dg-final { scan-tree-dump-times "vectorized 24 loops" 1 "vect" { > > target aarch64_sve } } } > > -! { dg-final { scan-tree-dump-times "vectorized 23 loops" 1 "vect" { > > target { aarch64*-*-* && { ! aarch64_sve } } } } } > > +! { dg-final { scan-tree-dump-times "vectorized 24 loops" 1 "vect" { > > target aarch64*-*-* } } } > > ! { dg-final { scan-tree-dump-times "vectorized 2\[234\] loops" 1 "vect" > > { target { vect_intdouble_cvt && { ! aarch64*-*-* } } } } } > > ! { dg-final { scan-tree-dump-times "vectorized 17 loops" 1 "vect" { > > target { { ! vect_intdouble_cvt } && { ! aarch64*-*-* } } } } } > > > > f951 vect.exp testing with and without -march=armv8.3-a+sve shows > > this might work, but if you can double-check that would be nice. > > > > > Indeed LGTM, thanks Pushed. > > > Richard. > >