From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com [IPv6:2a00:1450:4864:20::52c]) by sourceware.org (Postfix) with ESMTPS id 6CC483858426 for ; Thu, 2 Mar 2023 15:27:27 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 6CC483858426 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-ed1-x52c.google.com with SMTP id g3so13619523eda.1 for ; Thu, 02 Mar 2023 07:27:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1677770845; h=to:in-reply-to:cc:references:message-id:date:subject:mime-version :from:content-transfer-encoding:from:to:cc:subject:date:message-id :reply-to; bh=ukLbdNw5jjmuArV0X3KcB+wD1nNhPHy7EO7tZdjONA0=; b=bLilKc5cGh2Dp3fMVq6s0k+1VrF/GR4gOCzDPXptqZNKgvD/uuSV9HMSay+wZS5nPG PNem2N0vvwjfaa5F0Kui6rdzyJ0cjFelJN3pPxeozJX0n4jsOQwNi6lnXkVjckfXz4x/ uPtMr9aozDkzDz9BErrCxQQzNK8Az+KlaN6Yew0e3qXXYM3WReIliT1ZEspAjkXnXPyY 1qh3QqyrjXE1Xk6jqchRtNz2YJj1TPhDBtTjYFCcOgtW2tYTNOSrWT/Q6anPvKelUj5U lM+09VrBkHIllt7IdKfoSBRq8CMV9wVbzy8waX415A5kOX6DEp1uTLllvlcBhB+5cwPB HRlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677770845; h=to:in-reply-to:cc:references:message-id:date:subject:mime-version :from:content-transfer-encoding:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ukLbdNw5jjmuArV0X3KcB+wD1nNhPHy7EO7tZdjONA0=; b=rAIbPID2v2bKSL5JkEPLFdfeASh9/NKtNE1ig/7OpFCodATxG12ytmH7Qp9sEyVh1E yGdc8cNd6t+mf5CfCijczxQ8pldXcbhoUlzRAbpBewNacZmpZ1Nlig+obw8RNO5fPWpm VvrVv6aDjOv7FO7c/oJ9MN/3wVkQFt3ajkoAmhw+sr7ypALhA+1kDuwp2p1srHaxTmWN ZezaB74WPdhCBZxLBdfqHrKOas0p6Iq3ao/6J2ZG2nQTCaIQoGuufmt+Rx8TXJotvRxK pfwaG5OIW7elBpe3kUowDrBjsCIA6j4xw+2edMSg54cstIBoG4ZGfKccYLo64e11l5Gr emqQ== X-Gm-Message-State: AO0yUKXH8gBqcN8wGAQx0UgK5AM2DnaB36zaibCg8XXawCTikG7lLaP9 v3nDIOhmx1ajSV97tStknheSHc89ME4= X-Google-Smtp-Source: AK7set8VNUlIP4nd2x+JSgm72FGw83MbecTvspubRUAfClP+iDMZeyuAqQXQB5zTfBKoBU/J4P846Q== X-Received: by 2002:a50:fa81:0:b0:4be:d2f9:35a7 with SMTP id w1-20020a50fa81000000b004bed2f935a7mr4396191edr.21.1677770845322; Thu, 02 Mar 2023 07:27:25 -0800 (PST) Received: from smtpclient.apple (dynamic-095-115-111-228.95.115.pool.telefonica.de. [95.115.111.228]) by smtp.gmail.com with ESMTPSA id cm18-20020a0564020c9200b004af5968cb3bsm6938220edb.17.2023.03.02.07.27.24 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 02 Mar 2023 07:27:24 -0800 (PST) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable From: Richard Biener Mime-Version: 1.0 (1.0) Subject: Re: [PATCH] vect: Don't apply masks to operations on invariants [PR108979] Date: Thu, 2 Mar 2023 16:27:13 +0100 Message-Id: <3EF550DD-A421-4AFF-A000-5076D73A0500@gmail.com> References: Cc: gcc-patches@gcc.gnu.org In-Reply-To: To: Richard Sandiford X-Mailer: iPhone Mail (20D67) X-Spam-Status: No, score=-10.5 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,GIT_PATCH_0,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: > Am 02.03.2023 um 15:28 schrieb Richard Sandiford via Gcc-patches : >=20 > =EF=BB=BFThe loop body in the testcase contains an operation on invariants= . > SLP detects this and can hoist/schedule the operation outside of > the loop. However, after the fix for PR96373, we would try to > apply a loop mask to this operation, even though the mask is > defined in the loop. >=20 > The patch does what Richi suggested in the PR: suppress the > masking for externs and constants. >=20 > Tested on aarch64-linux-gnu. OK to install? Ok. Richard=20 > Richard >=20 >=20 > gcc/ > PR tree-optimization/108979 > * tree-vect-stmts.cc (vectorizable_operation): Don't mask > operations on invariants. >=20 > gcc/testsuite/ > PR tree-optimization/108979 > * gfortran.dg/vect/pr108979.f90: New test. > --- > gcc/testsuite/gfortran.dg/vect/pr108979.f90 | 21 +++++++++++++++++ > gcc/tree-vect-stmts.cc | 25 ++++++++++++++++----- > 2 files changed, 40 insertions(+), 6 deletions(-) > create mode 100644 gcc/testsuite/gfortran.dg/vect/pr108979.f90 >=20 > diff --git a/gcc/testsuite/gfortran.dg/vect/pr108979.f90 b/gcc/testsuite/g= fortran.dg/vect/pr108979.f90 > new file mode 100644 > index 00000000000..623eb67826f > --- /dev/null > +++ b/gcc/testsuite/gfortran.dg/vect/pr108979.f90 > @@ -0,0 +1,21 @@ > +! { dg-do compile } > +! { dg-additional-options "-fnon-call-exceptions" } > +! { dg-additional-options "-march=3Darmv8.2-a+sve" { target aarch64*-*-* }= } > + > +MODULE hfx_contract_block > + INTEGER, PARAMETER :: dp=3D8 > +CONTAINS > + SUBROUTINE block_2_1_2_1(kbd,kbc,kad,kac,pbd,pbc,pad,pac,prim,scale) > + REAL(KIND=3Ddp) :: kbd(1*1), kbc(1*2), kad(2*1), kac(2*2), pbd(1*1), &= > + pbc(1*2), pad(2*1), pac(2*2), prim(2*1*2*1), scale > + DO md =3D 1,1 > + DO mc =3D 1,2 > + DO mb =3D 1,1 > + DO ma =3D 1,2 > + kac((mc-1)*2+ma) =3D kac((mc-1)*2+ma)-tmp*p_bd > + END DO > + END DO > + END DO > + END DO > + END SUBROUTINE block_2_1_2_1 > +END MODULE hfx_contract_block > diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc > index 77ad8b78506..b56457617c0 100644 > --- a/gcc/tree-vect-stmts.cc > +++ b/gcc/tree-vect-stmts.cc > @@ -6254,6 +6254,8 @@ vectorizable_operation (vec_info *vinfo, > "use not simple.\n"); > return false; > } > + bool is_invariant =3D (dt[0] =3D=3D vect_external_def > + || dt[0] =3D=3D vect_constant_def); > /* If op0 is an external or constant def, infer the vector type > from the scalar type. */ > if (!vectype) > @@ -6307,6 +6309,8 @@ vectorizable_operation (vec_info *vinfo, > "use not simple.\n"); > return false; > } > + is_invariant &=3D (dt[1] =3D=3D vect_external_def > + || dt[1] =3D=3D vect_constant_def); > if (vectype2 > && maybe_ne (nunits_out, TYPE_VECTOR_SUBPARTS (vectype2))) > return false; > @@ -6321,6 +6325,8 @@ vectorizable_operation (vec_info *vinfo, > "use not simple.\n"); > return false; > } > + is_invariant &=3D (dt[2] =3D=3D vect_external_def > + || dt[2] =3D=3D vect_constant_def); > if (vectype3 > && maybe_ne (nunits_out, TYPE_VECTOR_SUBPARTS (vectype3))) > return false; > @@ -6426,16 +6432,23 @@ vectorizable_operation (vec_info *vinfo, > int reduc_idx =3D STMT_VINFO_REDUC_IDX (stmt_info); > vec_loop_masks *masks =3D (loop_vinfo ? &LOOP_VINFO_MASKS (loop_vinfo) := NULL); > internal_fn cond_fn =3D get_conditional_internal_fn (code); > - bool could_trap =3D gimple_could_trap_p (stmt); > + > + /* If operating on inactive elements could generate spurious traps, > + we need to restrict the operation to active lanes. Note that this > + specifically doesn't apply to unhoisted invariants, since they > + operate on the same value for every lane. > + > + Similarly, if this operation is part of a reduction, a fully-masked > + loop should only change the active lanes of the reduction chain, > + keeping the inactive lanes as-is. */ > + bool mask_out_inactive =3D ((!is_invariant && gimple_could_trap_p (stmt= )) > + || reduc_idx >=3D 0); >=20 > if (!vec_stmt) /* transformation not required. */ > { > - /* If this operation is part of a reduction, a fully-masked loop > - should only change the active lanes of the reduction chain, > - keeping the inactive lanes as-is. */ > if (loop_vinfo > && LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) > - && (could_trap || reduc_idx >=3D 0)) > + && mask_out_inactive) > { > if (cond_fn =3D=3D IFN_LAST > || !direct_internal_fn_supported_p (cond_fn, vectype, > @@ -6578,7 +6591,7 @@ vectorizable_operation (vec_info *vinfo, > vop1 =3D ((op_type =3D=3D binary_op || op_type =3D=3D ternary_op) > ? vec_oprnds1[i] : NULL_TREE); > vop2 =3D ((op_type =3D=3D ternary_op) ? vec_oprnds2[i] : NULL_TREE);= > - if (masked_loop_p && (reduc_idx >=3D 0 || could_trap)) > + if (masked_loop_p && mask_out_inactive) > { > tree mask =3D vect_get_loop_mask (gsi, masks, vec_num * ncopies, > vectype, i); > --=20 > 2.25.1 >=20