From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id DB708386F0DE for ; Fri, 10 Jun 2022 09:13:13 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org DB708386F0DE Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5132E1063 for ; Fri, 10 Jun 2022 02:13:13 -0700 (PDT) Received: from localhost (e121540-lin.manchester.arm.com [10.32.98.37]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id CCBC23F73B for ; Fri, 10 Jun 2022 02:13:12 -0700 (PDT) From: Richard Sandiford To: gcc-patches@gcc.gnu.org Mail-Followup-To: gcc-patches@gcc.gnu.org, richard.sandiford@arm.com Subject: [PATCH 1/2] Factor out common internal-fn idiom Date: Fri, 10 Jun 2022 10:13:11 +0100 Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Status: No, score=-58.9 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Jun 2022 09:13:16 -0000 internal-fn.c has quite a few functions that simply map the result of the call to an instruction's output operand (if any) and map each argument to an instruction's input operand, in order. This patch adds a single function for doing that. It's really just a generalisation of expand_direct_optab_fn, but with the output operand being optional. Unfortunately, it isn't possible to do this for vcond_mask because the internal function has a different argument order from the optab. Tested on aarch64-linux-gnu and x86_64-linux-gnu. OK to install? Richard gcc/ * internal-fn.cc (expand_fn_using_insn): New function, split out and adapted from... (expand_direct_optab_fn): ...here. (expand_GOMP_SIMT_ENTER_ALLOC): Use it. (expand_GOMP_SIMT_EXIT): Likewise. (expand_GOMP_SIMT_LANE): Likewise. (expand_GOMP_SIMT_LAST_LANE): Likewise. (expand_GOMP_SIMT_ORDERED_PRED): Likewise. (expand_GOMP_SIMT_VOTE_ANY): Likewise. (expand_GOMP_SIMT_XCHG_BFLY): Likewise. (expand_GOMP_SIMT_XCHG_IDX): Likewise. --- gcc/internal-fn.cc | 243 +++++++++++++++++---------------------------- 1 file changed, 89 insertions(+), 154 deletions(-) diff --git a/gcc/internal-fn.cc b/gcc/internal-fn.cc index 8b1733e20c4..ab2b1baa893 100644 --- a/gcc/internal-fn.cc +++ b/gcc/internal-fn.cc @@ -140,6 +140,86 @@ const direct_internal_fn_info direct_internal_fn_array[IFN_LAST + 1] = { not_direct }; +/* Expand STMT using instruction ICODE. The instruction has NOUTPUTS + output operands and NINPUTS input operands, where NOUTPUTS is either + 0 or 1. The output operand (if any) comes first, followed by the + NINPUTS input operands. */ + +static void +expand_fn_using_insn (gcall *stmt, insn_code icode, unsigned int noutputs, + unsigned int ninputs) +{ + gcc_assert (icode != CODE_FOR_nothing); + + expand_operand *ops = XALLOCAVEC (expand_operand, noutputs + ninputs); + unsigned int opno = 0; + rtx lhs_rtx = NULL_RTX; + tree lhs = gimple_call_lhs (stmt); + + if (noutputs) + { + gcc_assert (noutputs == 1); + if (lhs) + lhs_rtx = expand_expr (lhs, NULL_RTX, VOIDmode, EXPAND_WRITE); + + /* Do not assign directly to a promoted subreg, since there is no + guarantee that the instruction will leave the upper bits of the + register in the state required by SUBREG_PROMOTED_SIGN. */ + rtx dest = lhs_rtx; + if (dest && GET_CODE (dest) == SUBREG && SUBREG_PROMOTED_VAR_P (dest)) + dest = NULL_RTX; + create_output_operand (&ops[opno], dest, + insn_data[icode].operand[opno].mode); + opno += 1; + } + else + gcc_assert (!lhs); + + for (unsigned int i = 0; i < ninputs; ++i) + { + tree rhs = gimple_call_arg (stmt, i); + tree rhs_type = TREE_TYPE (rhs); + rtx rhs_rtx = expand_normal (rhs); + if (INTEGRAL_TYPE_P (rhs_type)) + create_convert_operand_from (&ops[opno], rhs_rtx, + TYPE_MODE (rhs_type), + TYPE_UNSIGNED (rhs_type)); + else + create_input_operand (&ops[opno], rhs_rtx, TYPE_MODE (rhs_type)); + opno += 1; + } + + gcc_assert (opno == noutputs + ninputs); + expand_insn (icode, opno, ops); + if (lhs_rtx && !rtx_equal_p (lhs_rtx, ops[0].value)) + { + /* If the return value has an integral type, convert the instruction + result to that type. This is useful for things that return an + int regardless of the size of the input. If the instruction result + is smaller than required, assume that it is signed. + + If the return value has a nonintegral type, its mode must match + the instruction result. */ + if (GET_CODE (lhs_rtx) == SUBREG && SUBREG_PROMOTED_VAR_P (lhs_rtx)) + { + /* If this is a scalar in a register that is stored in a wider + mode than the declared mode, compute the result into its + declared mode and then convert to the wider mode. */ + gcc_checking_assert (INTEGRAL_TYPE_P (TREE_TYPE (lhs))); + rtx tmp = convert_to_mode (GET_MODE (lhs_rtx), ops[0].value, 0); + convert_move (SUBREG_REG (lhs_rtx), tmp, + SUBREG_PROMOTED_SIGN (lhs_rtx)); + } + else if (GET_MODE (lhs_rtx) == GET_MODE (ops[0].value)) + emit_move_insn (lhs_rtx, ops[0].value); + else + { + gcc_checking_assert (INTEGRAL_TYPE_P (TREE_TYPE (lhs))); + convert_move (lhs_rtx, ops[0].value, 0); + } + } +} + /* ARRAY_TYPE is an array of vector modes. Return the associated insn for load-lanes-style optab OPTAB, or CODE_FOR_nothing if none. */ @@ -233,22 +313,8 @@ expand_GOMP_SIMT_ENTER (internal_fn, gcall *) static void expand_GOMP_SIMT_ENTER_ALLOC (internal_fn, gcall *stmt) { - rtx target; - tree lhs = gimple_call_lhs (stmt); - if (lhs) - target = expand_expr (lhs, NULL_RTX, VOIDmode, EXPAND_WRITE); - else - target = gen_reg_rtx (Pmode); - rtx size = expand_normal (gimple_call_arg (stmt, 0)); - rtx align = expand_normal (gimple_call_arg (stmt, 1)); - class expand_operand ops[3]; - create_output_operand (&ops[0], target, Pmode); - create_input_operand (&ops[1], size, Pmode); - create_input_operand (&ops[2], align, Pmode); gcc_assert (targetm.have_omp_simt_enter ()); - expand_insn (targetm.code_for_omp_simt_enter, 3, ops); - if (!rtx_equal_p (target, ops[0].value)) - emit_move_insn (target, ops[0].value); + expand_fn_using_insn (stmt, targetm.code_for_omp_simt_enter, 1, 2); } /* Deallocate per-lane storage and leave non-uniform execution region. */ @@ -256,12 +322,8 @@ expand_GOMP_SIMT_ENTER_ALLOC (internal_fn, gcall *stmt) static void expand_GOMP_SIMT_EXIT (internal_fn, gcall *stmt) { - gcc_checking_assert (!gimple_call_lhs (stmt)); - rtx arg = expand_normal (gimple_call_arg (stmt, 0)); - class expand_operand ops[1]; - create_input_operand (&ops[0], arg, Pmode); gcc_assert (targetm.have_omp_simt_exit ()); - expand_insn (targetm.code_for_omp_simt_exit, 1, ops); + expand_fn_using_insn (stmt, targetm.code_for_omp_simt_exit, 0, 1); } /* Lane index on SIMT targets: thread index in the warp on NVPTX. On targets @@ -270,13 +332,8 @@ expand_GOMP_SIMT_EXIT (internal_fn, gcall *stmt) static void expand_GOMP_SIMT_LANE (internal_fn, gcall *stmt) { - tree lhs = gimple_call_lhs (stmt); - if (!lhs) - return; - - rtx target = expand_expr (lhs, NULL_RTX, VOIDmode, EXPAND_WRITE); gcc_assert (targetm.have_omp_simt_lane ()); - emit_insn (targetm.gen_omp_simt_lane (target)); + expand_fn_using_insn (stmt, targetm.code_for_omp_simt_lane, 1, 0); } /* This should get expanded in omp_device_lower pass. */ @@ -294,20 +351,8 @@ expand_GOMP_SIMT_VF (internal_fn, gcall *) static void expand_GOMP_SIMT_LAST_LANE (internal_fn, gcall *stmt) { - tree lhs = gimple_call_lhs (stmt); - if (!lhs) - return; - - rtx target = expand_expr (lhs, NULL_RTX, VOIDmode, EXPAND_WRITE); - rtx cond = expand_normal (gimple_call_arg (stmt, 0)); - machine_mode mode = TYPE_MODE (TREE_TYPE (lhs)); - class expand_operand ops[2]; - create_output_operand (&ops[0], target, mode); - create_input_operand (&ops[1], cond, mode); gcc_assert (targetm.have_omp_simt_last_lane ()); - expand_insn (targetm.code_for_omp_simt_last_lane, 2, ops); - if (!rtx_equal_p (target, ops[0].value)) - emit_move_insn (target, ops[0].value); + expand_fn_using_insn (stmt, targetm.code_for_omp_simt_last_lane, 1, 1); } /* Non-transparent predicate used in SIMT lowering of OpenMP "ordered". */ @@ -315,20 +360,8 @@ expand_GOMP_SIMT_LAST_LANE (internal_fn, gcall *stmt) static void expand_GOMP_SIMT_ORDERED_PRED (internal_fn, gcall *stmt) { - tree lhs = gimple_call_lhs (stmt); - if (!lhs) - return; - - rtx target = expand_expr (lhs, NULL_RTX, VOIDmode, EXPAND_WRITE); - rtx ctr = expand_normal (gimple_call_arg (stmt, 0)); - machine_mode mode = TYPE_MODE (TREE_TYPE (lhs)); - class expand_operand ops[2]; - create_output_operand (&ops[0], target, mode); - create_input_operand (&ops[1], ctr, mode); gcc_assert (targetm.have_omp_simt_ordered ()); - expand_insn (targetm.code_for_omp_simt_ordered, 2, ops); - if (!rtx_equal_p (target, ops[0].value)) - emit_move_insn (target, ops[0].value); + expand_fn_using_insn (stmt, targetm.code_for_omp_simt_ordered, 1, 1); } /* "Or" boolean reduction across SIMT lanes: return non-zero in all lanes if @@ -337,20 +370,8 @@ expand_GOMP_SIMT_ORDERED_PRED (internal_fn, gcall *stmt) static void expand_GOMP_SIMT_VOTE_ANY (internal_fn, gcall *stmt) { - tree lhs = gimple_call_lhs (stmt); - if (!lhs) - return; - - rtx target = expand_expr (lhs, NULL_RTX, VOIDmode, EXPAND_WRITE); - rtx cond = expand_normal (gimple_call_arg (stmt, 0)); - machine_mode mode = TYPE_MODE (TREE_TYPE (lhs)); - class expand_operand ops[2]; - create_output_operand (&ops[0], target, mode); - create_input_operand (&ops[1], cond, mode); gcc_assert (targetm.have_omp_simt_vote_any ()); - expand_insn (targetm.code_for_omp_simt_vote_any, 2, ops); - if (!rtx_equal_p (target, ops[0].value)) - emit_move_insn (target, ops[0].value); + expand_fn_using_insn (stmt, targetm.code_for_omp_simt_vote_any, 1, 1); } /* Exchange between SIMT lanes with a "butterfly" pattern: source lane index @@ -359,22 +380,8 @@ expand_GOMP_SIMT_VOTE_ANY (internal_fn, gcall *stmt) static void expand_GOMP_SIMT_XCHG_BFLY (internal_fn, gcall *stmt) { - tree lhs = gimple_call_lhs (stmt); - if (!lhs) - return; - - rtx target = expand_expr (lhs, NULL_RTX, VOIDmode, EXPAND_WRITE); - rtx src = expand_normal (gimple_call_arg (stmt, 0)); - rtx idx = expand_normal (gimple_call_arg (stmt, 1)); - machine_mode mode = TYPE_MODE (TREE_TYPE (lhs)); - class expand_operand ops[3]; - create_output_operand (&ops[0], target, mode); - create_input_operand (&ops[1], src, mode); - create_input_operand (&ops[2], idx, SImode); gcc_assert (targetm.have_omp_simt_xchg_bfly ()); - expand_insn (targetm.code_for_omp_simt_xchg_bfly, 3, ops); - if (!rtx_equal_p (target, ops[0].value)) - emit_move_insn (target, ops[0].value); + expand_fn_using_insn (stmt, targetm.code_for_omp_simt_xchg_bfly, 1, 2); } /* Exchange between SIMT lanes according to given source lane index. */ @@ -382,22 +389,8 @@ expand_GOMP_SIMT_XCHG_BFLY (internal_fn, gcall *stmt) static void expand_GOMP_SIMT_XCHG_IDX (internal_fn, gcall *stmt) { - tree lhs = gimple_call_lhs (stmt); - if (!lhs) - return; - - rtx target = expand_expr (lhs, NULL_RTX, VOIDmode, EXPAND_WRITE); - rtx src = expand_normal (gimple_call_arg (stmt, 0)); - rtx idx = expand_normal (gimple_call_arg (stmt, 1)); - machine_mode mode = TYPE_MODE (TREE_TYPE (lhs)); - class expand_operand ops[3]; - create_output_operand (&ops[0], target, mode); - create_input_operand (&ops[1], src, mode); - create_input_operand (&ops[2], idx, SImode); gcc_assert (targetm.have_omp_simt_xchg_idx ()); - expand_insn (targetm.code_for_omp_simt_xchg_idx, 3, ops); - if (!rtx_equal_p (target, ops[0].value)) - emit_move_insn (target, ops[0].value); + expand_fn_using_insn (stmt, targetm.code_for_omp_simt_xchg_idx, 1, 2); } /* This should get expanded in adjust_simduid_builtins. */ @@ -3565,67 +3558,9 @@ static void expand_direct_optab_fn (internal_fn fn, gcall *stmt, direct_optab optab, unsigned int nargs) { - expand_operand *ops = XALLOCAVEC (expand_operand, nargs + 1); - tree_pair types = direct_internal_fn_types (fn, stmt); insn_code icode = direct_optab_handler (optab, TYPE_MODE (types.first)); - gcc_assert (icode != CODE_FOR_nothing); - - tree lhs = gimple_call_lhs (stmt); - rtx lhs_rtx = NULL_RTX; - if (lhs) - lhs_rtx = expand_expr (lhs, NULL_RTX, VOIDmode, EXPAND_WRITE); - - /* Do not assign directly to a promoted subreg, since there is no - guarantee that the instruction will leave the upper bits of the - register in the state required by SUBREG_PROMOTED_SIGN. */ - rtx dest = lhs_rtx; - if (dest && GET_CODE (dest) == SUBREG && SUBREG_PROMOTED_VAR_P (dest)) - dest = NULL_RTX; - - create_output_operand (&ops[0], dest, insn_data[icode].operand[0].mode); - - for (unsigned int i = 0; i < nargs; ++i) - { - tree rhs = gimple_call_arg (stmt, i); - tree rhs_type = TREE_TYPE (rhs); - rtx rhs_rtx = expand_normal (rhs); - if (INTEGRAL_TYPE_P (rhs_type)) - create_convert_operand_from (&ops[i + 1], rhs_rtx, - TYPE_MODE (rhs_type), - TYPE_UNSIGNED (rhs_type)); - else - create_input_operand (&ops[i + 1], rhs_rtx, TYPE_MODE (rhs_type)); - } - - expand_insn (icode, nargs + 1, ops); - if (lhs_rtx && !rtx_equal_p (lhs_rtx, ops[0].value)) - { - /* If the return value has an integral type, convert the instruction - result to that type. This is useful for things that return an - int regardless of the size of the input. If the instruction result - is smaller than required, assume that it is signed. - - If the return value has a nonintegral type, its mode must match - the instruction result. */ - if (GET_CODE (lhs_rtx) == SUBREG && SUBREG_PROMOTED_VAR_P (lhs_rtx)) - { - /* If this is a scalar in a register that is stored in a wider - mode than the declared mode, compute the result into its - declared mode and then convert to the wider mode. */ - gcc_checking_assert (INTEGRAL_TYPE_P (TREE_TYPE (lhs))); - rtx tmp = convert_to_mode (GET_MODE (lhs_rtx), ops[0].value, 0); - convert_move (SUBREG_REG (lhs_rtx), tmp, - SUBREG_PROMOTED_SIGN (lhs_rtx)); - } - else if (GET_MODE (lhs_rtx) == GET_MODE (ops[0].value)) - emit_move_insn (lhs_rtx, ops[0].value); - else - { - gcc_checking_assert (INTEGRAL_TYPE_P (TREE_TYPE (lhs))); - convert_move (lhs_rtx, ops[0].value, 0); - } - } + expand_fn_using_insn (stmt, icode, 1, nargs); } /* Expand WHILE_ULT call STMT using optab OPTAB. */ -- 2.25.1