From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-yw1-x1133.google.com (mail-yw1-x1133.google.com [IPv6:2607:f8b0:4864:20::1133]) by sourceware.org (Postfix) with ESMTPS id 624BF3858D32 for ; Mon, 29 May 2023 03:21:16 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 624BF3858D32 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-yw1-x1133.google.com with SMTP id 00721157ae682-565a77af71fso23308987b3.2 for ; Sun, 28 May 2023 20:21:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1685330475; x=1687922475; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=leq6+tQ6fGMIj69Kr+EWIvd/Cqx6fwDmOl/ZSOamuiw=; b=I/mvnKBUDthal67e1UGHxo9ZXhLNIfy8vJHg+KiS+PtzrJoLUV0ZL+VS54ksc/5PM5 eNI4CtxmM7GZBNB4A9b+cihdt4D1NT4fSOd1eirpc/cuid/Cbs02IXCOptnDDYBUzkML JjMIam1th8i2rblXBthzmjsfyzMd9vWoB2RIrhxqTcdTNPP93Lnyt+sx5a5Kj5KVzu/q F1yM7VNNtwNaRAhcBwQhkjiW5/lb+mKPQlNgyx6IeIQbCiTzVpaJq1Yb+2iEPz6WqoKN rHucIH+yaSlp/ruHWuVqI6ScSh5h9Vrh3kQX24wpnhR650c9LAN5Lw52ddJK5TkmNrj2 LXyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685330475; x=1687922475; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=leq6+tQ6fGMIj69Kr+EWIvd/Cqx6fwDmOl/ZSOamuiw=; b=aB+6RPsH6JTlFcjUclQkkwMDIHjVR5MhOhJpBNQeL+VlV82iZLqAMmnEP6JyGSj87P ga3DgwGuCO/VG10R6/JJWxHVBNlKxlaJpsZnJv3B4NWbh2B6eDB7EDuf3FU9n4pNymGm pQISCfygxnDtv/Ng/ffTZFgS7MwEmRXnOLlJm4zv04do6yfI3/OwixmilbqvdCzBjh/L 6L8Ia/dK/SIDaawZQ5xphyjy5Kl9wV4GaScRg3wtX9VAM1NsYC0Gbul9vIuXopfepgAB 3o1NN9gmGpnf8QcCYCpCHaGDxgQaJYNeWdMTbaJyhOTpGctRDohQNYZnzmE16stkOGqy e0eQ== X-Gm-Message-State: AC+VfDxXPYsLjDKGNiyJE8PFtCAi7Kax+96zCCOur3HoXNVdyvGHL96B pnEb+g5+h7TZ1NECNv43+LHG4lG/g+UAPm5rzlg= X-Google-Smtp-Source: ACHHUZ6/cZgWyoaUUG4BAkhmeF7gbR9p6qXbC9XEUdT1a8vPFiirCvHpJkonOoh42gZbyU89gGdsUQUMvYcbwHWV/pI= X-Received: by 2002:a81:7785:0:b0:561:c184:6568 with SMTP id s127-20020a817785000000b00561c1846568mr10059865ywc.18.1685330475543; Sun, 28 May 2023 20:21:15 -0700 (PDT) MIME-Version: 1.0 References: <20230508015900.3988239-1-hongtao.liu@intel.com> In-Reply-To: <20230508015900.3988239-1-hongtao.liu@intel.com> From: Hongtao Liu Date: Mon, 29 May 2023 11:21:04 +0800 Message-ID: Subject: Re: [PATCH V2] [vect]Enhance NARROW FLOAT_EXPR vectorization by truncating integer to lower precision. To: liuhongt Cc: gcc-patches@gcc.gnu.org, hjl.tools@gmail.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-8.2 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,GIT_PATCH_0,KAM_SHORT,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: ping. On Mon, May 8, 2023 at 9:59=E2=80=AFAM liuhongt wro= te: > > > > @@ -4799,7 +4800,8 @@ vect_create_vectorized_demotion_stmts (vec_info= *vinfo, vec *vec_oprnds, > > > stmt_vec_info stmt_info, > > > vec &vec_dsts, > > > gimple_stmt_iterator *gsi, > > > - slp_tree slp_node, enum tree_c= ode code) > > > + slp_tree slp_node, enum tree_c= ode code, > > > + bool last_stmt_p) > > > > Can you please document this new parameter? > > > Changed. > > > > > I understand what you are doing, but somehow it looks a bit awkward? > > Maybe we should split the NARROW case into NARROW_SRC and NARROW_DST? > > The case of narrowing the source because we know its range isn't a > > good fit for the > > flow. > Changed. > > Here's updated patch. > Bootstrapped and regtested on x86_64-pc-linux-gnu{-m32,}. > Ok for trunk? > > Similar like WIDEN FLOAT_EXPR, when direct_optab is not existed, try > intermediate integer type whenever gimple ranger can tell it's safe. > > .i.e. > When there's no direct optab for vector long long -> vector float, but > the value range of integer can be represented as int, try vector int > -> vector float if availble. > > gcc/ChangeLog: > > PR tree-optimization/108804 > * tree-vect-patterns.cc (vect_get_range_info): Remove static. > * tree-vect-stmts.cc (vect_create_vectorized_demotion_stmts): > Add new parameter narrow_src_p. > (vectorizable_conversion): Enhance NARROW FLOAT_EXPR > vectorization by truncating to lower precision. > * tree-vectorizer.h (vect_get_range_info): New declare. > > gcc/testsuite/ChangeLog: > > * gcc.target/i386/pr108804.c: New test. > --- > gcc/testsuite/gcc.target/i386/pr108804.c | 15 +++ > gcc/tree-vect-patterns.cc | 2 +- > gcc/tree-vect-stmts.cc | 135 +++++++++++++++++------ > gcc/tree-vectorizer.h | 1 + > 4 files changed, 121 insertions(+), 32 deletions(-) > create mode 100644 gcc/testsuite/gcc.target/i386/pr108804.c > > diff --git a/gcc/testsuite/gcc.target/i386/pr108804.c b/gcc/testsuite/gcc= .target/i386/pr108804.c > new file mode 100644 > index 00000000000..2a43c1e1848 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/i386/pr108804.c > @@ -0,0 +1,15 @@ > +/* { dg-do compile } */ > +/* { dg-options "-mavx2 -Ofast -fdump-tree-vect-details" } */ > +/* { dg-final { scan-tree-dump-times "vectorized \[1-3] loops" 1 "vect" = } } */ > + > +typedef unsigned long long uint64_t; > +uint64_t d[512]; > +float f[1024]; > + > +void foo() { > + for (int i=3D0; i<512; ++i) { > + uint64_t k =3D d[i]; > + f[i]=3D(k & 0x3F30FFFF); > + } > +} > + > diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc > index a49b0953977..dd546b488a4 100644 > --- a/gcc/tree-vect-patterns.cc > +++ b/gcc/tree-vect-patterns.cc > @@ -61,7 +61,7 @@ along with GCC; see the file COPYING3. If not see > /* Return true if we have a useful VR_RANGE range for VAR, storing it > in *MIN_VALUE and *MAX_VALUE if so. Note the range in the dump files= . */ > > -static bool > +bool > vect_get_range_info (tree var, wide_int *min_value, wide_int *max_value) > { > value_range vr; > diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc > index 6b7dbfd4a23..3da89a8402d 100644 > --- a/gcc/tree-vect-stmts.cc > +++ b/gcc/tree-vect-stmts.cc > @@ -51,6 +51,7 @@ along with GCC; see the file COPYING3. If not see > #include "internal-fn.h" > #include "tree-vector-builder.h" > #include "vec-perm-indices.h" > +#include "gimple-range.h" > #include "tree-ssa-loop-niter.h" > #include "gimple-fold.h" > #include "regs.h" > @@ -4791,7 +4792,9 @@ vect_gen_widened_results_half (vec_info *vinfo, enu= m tree_code code, > > /* Create vectorized demotion statements for vector operands from VEC_OP= RNDS. > For multi-step conversions store the resulting vectors and call the f= unction > - recursively. */ > + recursively. When NARROW_SRC_P is true, there's still a conversion af= ter > + narrowing, don't store the vectors in the SLP_NODE or in vector info = of > + the scalar statement(or in STMT_VINFO_RELATED_STMT chain). */ > > static void > vect_create_vectorized_demotion_stmts (vec_info *vinfo, vec *vec_o= prnds, > @@ -4799,7 +4802,8 @@ vect_create_vectorized_demotion_stmts (vec_info *vi= nfo, vec *vec_oprnds, > stmt_vec_info stmt_info, > vec &vec_dsts, > gimple_stmt_iterator *gsi, > - slp_tree slp_node, enum tree_code = code) > + slp_tree slp_node, enum tree_code = code, > + bool narrow_src_p) > { > unsigned int i; > tree vop0, vop1, new_tmp, vec_dest; > @@ -4815,9 +4819,9 @@ vect_create_vectorized_demotion_stmts (vec_info *vi= nfo, vec *vec_oprnds, > new_tmp =3D make_ssa_name (vec_dest, new_stmt); > gimple_assign_set_lhs (new_stmt, new_tmp); > vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi); > - > - if (multi_step_cvt) > - /* Store the resulting vector for next recursive call. */ > + if (multi_step_cvt || narrow_src_p) > + /* Store the resulting vector for next recursive call, > + or return the resulting vector_tmp for NARROW FLOAT_EXPR. */ > (*vec_oprnds)[i/2] =3D new_tmp; > else > { > @@ -4843,7 +4847,8 @@ vect_create_vectorized_demotion_stmts (vec_info *vi= nfo, vec *vec_oprnds, > vect_create_vectorized_demotion_stmts (vinfo, vec_oprnds, > multi_step_cvt - 1, > stmt_info, vec_dsts, gsi, > - slp_node, VEC_PACK_TRUNC_EXP= R); > + slp_node, VEC_PACK_TRUNC_EXP= R, > + narrow_src_p); > } > > vec_dsts.quick_push (vec_dest); > @@ -4988,7 +4993,15 @@ vectorizable_conversion (vec_info *vinfo, > tree vectype_out, vectype_in; > int ncopies, i; > tree lhs_type, rhs_type; > - enum { NARROW, NONE, WIDEN } modifier; > + /* For conversions between floating point and integer, there're 2 NARR= OW > + cases. NARROW_SRC is for FLOAT_EXPR, means > + integer --DEMOTION--> integer --FLOAT_EXPR--> floating point. > + This is safe when the range of the source integer can fit into the = lower > + precision. NARROW_DST is for FIX_TRUNC_EXPR, means > + floating point --FIX_TRUNC_EXPR--> integer --DEMOTION--> INTEGER. > + For other conversions, when there's narrowing, NARROW_DST is used a= s > + default. */ > + enum { NARROW_SRC, NARROW_DST, NONE, WIDEN } modifier; > vec vec_oprnds0 =3D vNULL; > vec vec_oprnds1 =3D vNULL; > tree vop0; > @@ -5123,7 +5136,7 @@ vectorizable_conversion (vec_info *vinfo, > else > modifier =3D NONE; > else if (multiple_p (nunits_out, nunits_in)) > - modifier =3D NARROW; > + modifier =3D NARROW_DST; > else > { > gcc_checking_assert (multiple_p (nunits_in, nunits_out)); > @@ -5135,7 +5148,7 @@ vectorizable_conversion (vec_info *vinfo, > case of SLP. */ > if (slp_node) > ncopies =3D 1; > - else if (modifier =3D=3D NARROW) > + else if (modifier =3D=3D NARROW_DST) > ncopies =3D vect_get_num_copies (loop_vinfo, vectype_out); > else > ncopies =3D vect_get_num_copies (loop_vinfo, vectype_in); > @@ -5241,29 +5254,63 @@ vectorizable_conversion (vec_info *vinfo, > } > break; > > - case NARROW: > + case NARROW_DST: > gcc_assert (op_type =3D=3D unary_op); > if (supportable_narrowing_operation (code, vectype_out, vectype_in= , > &code1, &multi_step_cvt, > &interm_types)) > break; > > - if (code !=3D FIX_TRUNC_EXPR > - || GET_MODE_SIZE (lhs_mode) >=3D GET_MODE_SIZE (rhs_mode)) > + if (GET_MODE_SIZE (lhs_mode) >=3D GET_MODE_SIZE (rhs_mode)) > goto unsupported; > > - cvt_type > - =3D build_nonstandard_integer_type (GET_MODE_BITSIZE (rhs_mode), = 0); > - cvt_type =3D get_same_sized_vectype (cvt_type, vectype_in); > - if (cvt_type =3D=3D NULL_TREE) > - goto unsupported; > - if (!supportable_convert_operation (code, cvt_type, vectype_in, > - &codecvt1)) > - goto unsupported; > - if (supportable_narrowing_operation (NOP_EXPR, vectype_out, cvt_ty= pe, > - &code1, &multi_step_cvt, > - &interm_types)) > - break; > + if (code =3D=3D FIX_TRUNC_EXPR) > + { > + cvt_type > + =3D build_nonstandard_integer_type (GET_MODE_BITSIZE (rhs_mod= e), 0); > + cvt_type =3D get_same_sized_vectype (cvt_type, vectype_in); > + if (cvt_type =3D=3D NULL_TREE) > + goto unsupported; > + if (!supportable_convert_operation (code, cvt_type, vectype_in, > + &codecvt1)) > + goto unsupported; > + if (supportable_narrowing_operation (NOP_EXPR, vectype_out, cvt= _type, > + &code1, &multi_step_cvt, > + &interm_types)) > + break; > + } > + /* If op0 can be represented with low precision integer, > + truncate it to cvt_type and the do FLOAT_EXPR. */ > + else if (code =3D=3D FLOAT_EXPR) > + { > + wide_int op_min_value, op_max_value; > + if (!vect_get_range_info (op0, &op_min_value, &op_max_value)) > + goto unsupported; > + > + cvt_type > + =3D build_nonstandard_integer_type (GET_MODE_BITSIZE (lhs_mod= e), 0); > + if (cvt_type =3D=3D NULL_TREE > + || (wi::min_precision (op_max_value, SIGNED) > + > TYPE_PRECISION (cvt_type)) > + || (wi::min_precision (op_min_value, SIGNED) > + > TYPE_PRECISION (cvt_type))) > + goto unsupported; > + > + cvt_type =3D get_same_sized_vectype (cvt_type, vectype_out); > + if (cvt_type =3D=3D NULL_TREE) > + goto unsupported; > + if (!supportable_narrowing_operation (NOP_EXPR, cvt_type, vecty= pe_in, > + &code1, &multi_step_cvt, > + &interm_types)) > + goto unsupported; > + if (supportable_convert_operation (code, vectype_out, > + cvt_type, &codecvt1)) > + { > + modifier =3D NARROW_SRC; > + break; > + } > + } > + > goto unsupported; > > default: > @@ -5288,7 +5335,7 @@ vectorizable_conversion (vec_info *vinfo, > vect_model_simple_cost (vinfo, stmt_info, ncopies, dt, ndts, sl= p_node, > cost_vec); > } > - else if (modifier =3D=3D NARROW) > + else if (modifier =3D=3D NARROW_SRC || modifier =3D=3D NARROW_DST) > { > STMT_VINFO_TYPE (stmt_info) =3D type_demotion_vec_info_type; > /* The final packing step produces one vector result per copy. = */ > @@ -5335,8 +5382,10 @@ vectorizable_conversion (vec_info *vinfo, > from supportable_*_operation, and store them in the correct order > for future use in vect_create_vectorized_*_stmts (). */ > auto_vec vec_dsts (multi_step_cvt + 1); > + bool widen_or_narrow_float_p > + =3D cvt_type && (modifier =3D=3D WIDEN || modifier =3D=3D NARROW_SRC= ); > vec_dest =3D vect_create_destination_var (scalar_dest, > - (cvt_type && modifier =3D=3D WI= DEN) > + widen_or_narrow_float_p > ? cvt_type : vectype_out); > vec_dsts.quick_push (vec_dest); > > @@ -5353,7 +5402,7 @@ vectorizable_conversion (vec_info *vinfo, > > if (cvt_type) > vec_dest =3D vect_create_destination_var (scalar_dest, > - modifier =3D=3D WIDEN > + widen_or_narrow_float_p > ? vectype_out : cvt_type); > > int ninputs =3D 1; > @@ -5361,7 +5410,7 @@ vectorizable_conversion (vec_info *vinfo, > { > if (modifier =3D=3D WIDEN) > ; > - else if (modifier =3D=3D NARROW) > + else if (modifier =3D=3D NARROW_SRC || modifier =3D=3D NARROW_DST) > { > if (multi_step_cvt) > ninputs =3D vect_pow2 (multi_step_cvt); > @@ -5448,7 +5497,8 @@ vectorizable_conversion (vec_info *vinfo, > } > break; > > - case NARROW: > + case NARROW_SRC: > + case NARROW_DST: > /* In case the vectorization factor (VF) is bigger than the number > of elements that we can fit in a vectype (nunits), we have to > generate more than one vector stmt - i.e - we need to "unroll" > @@ -5456,7 +5506,7 @@ vectorizable_conversion (vec_info *vinfo, > vect_get_vec_defs (vinfo, stmt_info, slp_node, ncopies * ninputs, > op0, &vec_oprnds0); > /* Arguments are ready. Create the new vector stmts. */ > - if (cvt_type) > + if (cvt_type && modifier =3D=3D NARROW_DST) > FOR_EACH_VEC_ELT (vec_oprnds0, i, vop0) > { > gcc_assert (TREE_CODE_LENGTH (codecvt1) =3D=3D unary_op); > @@ -5470,7 +5520,30 @@ vectorizable_conversion (vec_info *vinfo, > vect_create_vectorized_demotion_stmts (vinfo, &vec_oprnds0, > multi_step_cvt, > stmt_info, vec_dsts, gsi, > - slp_node, code1); > + slp_node, code1, > + modifier =3D=3D NARROW_SRC); > + /* After demoting op0 to cvt_type, convert it to dest. */ > + if (cvt_type && code =3D=3D FLOAT_EXPR) > + { > + for (unsigned int i =3D 0; i !=3D vec_oprnds0.length() / 2; i+= +) > + { > + /* Arguments are ready, create the new vector stmt. */ > + gcc_assert (TREE_CODE_LENGTH (codecvt1) =3D=3D unary_op); > + gassign *new_stmt > + =3D gimple_build_assign (vec_dest, codecvt1, vec_oprnds0[= i]); > + new_temp =3D make_ssa_name (vec_dest, new_stmt); > + gimple_assign_set_lhs (new_stmt, new_temp); > + vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gs= i); > + > + /* This is the last step of the conversion sequence. Store = the > + vectors in SLP_NODE or in vector info of the scalar stat= ement > + (or in STMT_VINFO_RELATED_STMT chain). */ > + if (slp_node) > + SLP_TREE_VEC_STMTS (slp_node).quick_push (new_stmt); > + else > + STMT_VINFO_VEC_STMTS (stmt_info).safe_push (new_stmt); > + } > + } > break; > } > if (!slp_node) > diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h > index 9cf2fb23fe3..27fa2306a5e 100644 > --- a/gcc/tree-vectorizer.h > +++ b/gcc/tree-vectorizer.h > @@ -2384,6 +2384,7 @@ extern bool compatible_calls_p (gcall *, gcall *); > /* In tree-vect-patterns.cc. */ > extern void > vect_mark_pattern_stmts (vec_info *, stmt_vec_info, gimple *, tree); > +extern bool vect_get_range_info (tree, wide_int*, wide_int*); > > /* Pattern recognition functions. > Additional pattern recognition functions can (and will) be added > -- > 2.39.1.388.g2fc9e9ca3c > --=20 BR, Hongtao