From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: by sourceware.org (Postfix, from userid 2119) id 92F293858C31; Mon, 26 Jun 2023 20:56:56 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 92F293858C31 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1687813016; bh=CsrFGy9v3ctYlaDOr4jvckD68ejuAdZE3Muo3vD1gas=; h=From:To:Subject:Date:From; b=BibKhexfxceze2gFFaNe1EY538kwnqcriDVCh4rA5EvSRrQiGxvm52OzymVoILH5o cykl0PFMmVj/uoSiX+HInhsXCnYCBuDc3H+J7R0n9owtUrbwvCSazjB/4YCi8tHw+/ oH5JgRVEC+fFkzvJQEW+CNIfGvKbP3/6DOOls0UM= Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: Jeff Law To: gcc-cvs@gcc.gnu.org Subject: [gcc(refs/vendors/riscv/heads/gcc-13-with-riscv-opts)] GIMPLE_FOLD: Apply LEN_MASK_{LOAD, STORE} into GIMPLE_FOLD X-Act-Checkin: gcc X-Git-Author: Ju-Zhe Zhong X-Git-Refname: refs/vendors/riscv/heads/gcc-13-with-riscv-opts X-Git-Oldrev: 3f0628fd46824ca08e6b735aa8a87451a06ee7f0 X-Git-Newrev: ae3a6823c27fa53bdf3dcecb166dd8719e0b6333 Message-Id: <20230626205656.92F293858C31@sourceware.org> Date: Mon, 26 Jun 2023 20:56:56 +0000 (GMT) List-Id: https://gcc.gnu.org/g:ae3a6823c27fa53bdf3dcecb166dd8719e0b6333 commit ae3a6823c27fa53bdf3dcecb166dd8719e0b6333 Author: Ju-Zhe Zhong Date: Fri Jun 23 21:48:27 2023 +0800 GIMPLE_FOLD: Apply LEN_MASK_{LOAD,STORE} into GIMPLE_FOLD Hi, since we are going to have LEN_MASK_{LOAD,STORE} into loopVectorizer. Currenly, 1. we can fold MASK_{LOAD,STORE} into MEM when mask is all ones. 2. we can fold LEN_{LOAD,STORE} into MEM when (len - bias) is VF. Now, I think it makes sense that we can support fold LEN_MASK_{LOAD,STORE} into MEM when both mask = all ones and (len - bias) is VF. gcc/ChangeLog: * gimple-fold.cc (arith_overflowed_p): Apply LEN_MASK_{LOAD,STORE}. (gimple_fold_partial_load_store_mem_ref): Ditto. (gimple_fold_partial_store): Ditto. (gimple_fold_call): Ditto. Diff: --- gcc/gimple-fold.cc | 23 ++++++++++++++++++----- 1 file changed, 18 insertions(+), 5 deletions(-) diff --git a/gcc/gimple-fold.cc b/gcc/gimple-fold.cc index 2b6855d1205..4a801feec65 100644 --- a/gcc/gimple-fold.cc +++ b/gcc/gimple-fold.cc @@ -5370,10 +5370,10 @@ arith_overflowed_p (enum tree_code code, const_tree type, return wi::min_precision (wres, sign) > TYPE_PRECISION (type); } -/* If IFN_{MASK,LEN}_LOAD/STORE call CALL is unconditional, return a MEM_REF - for the memory it references, otherwise return null. VECTYPE is the - type of the memory vector. MASK_P indicates it's for MASK if true, - otherwise it's for LEN. */ +/* If IFN_{MASK,LEN,LEN_MASK}_LOAD/STORE call CALL is unconditional, + return a MEM_REF for the memory it references, otherwise return null. + VECTYPE is the type of the memory vector. MASK_P indicates it's for + MASK if true, otherwise it's for LEN. */ static tree gimple_fold_partial_load_store_mem_ref (gcall *call, tree vectype, bool mask_p) @@ -5400,6 +5400,16 @@ gimple_fold_partial_load_store_mem_ref (gcall *call, tree vectype, bool mask_p) if (maybe_ne (wi::to_poly_widest (basic_len) - wi::to_widest (bias), GET_MODE_SIZE (TYPE_MODE (vectype)))) return NULL_TREE; + + /* For LEN_MASK_{LOAD,STORE}, we should also check whether + the mask is all ones mask. */ + internal_fn ifn = gimple_call_internal_fn (call); + if (ifn == IFN_LEN_MASK_LOAD || ifn == IFN_LEN_MASK_STORE) + { + tree mask = gimple_call_arg (call, internal_fn_mask_index (ifn)); + if (!integer_all_onesp (mask)) + return NULL_TREE; + } } unsigned HOST_WIDE_INT align = tree_to_uhwi (alias_align); @@ -5438,7 +5448,8 @@ static bool gimple_fold_partial_store (gimple_stmt_iterator *gsi, gcall *call, bool mask_p) { - tree rhs = gimple_call_arg (call, 3); + internal_fn ifn = gimple_call_internal_fn (call); + tree rhs = gimple_call_arg (call, internal_fn_stored_value_index (ifn)); if (tree lhs = gimple_fold_partial_load_store_mem_ref (call, TREE_TYPE (rhs), mask_p)) { @@ -5665,9 +5676,11 @@ gimple_fold_call (gimple_stmt_iterator *gsi, bool inplace) changed |= gimple_fold_partial_store (gsi, stmt, true); break; case IFN_LEN_LOAD: + case IFN_LEN_MASK_LOAD: changed |= gimple_fold_partial_load (gsi, stmt, false); break; case IFN_LEN_STORE: + case IFN_LEN_MASK_STORE: changed |= gimple_fold_partial_store (gsi, stmt, false); break; default: