From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: by sourceware.org (Postfix, from userid 1005) id 4E615385840D; Mon, 18 Oct 2021 17:02:01 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 4E615385840D Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: Michael Meissner To: gcc-cvs@gcc.gnu.org Subject: [gcc(refs/users/meissner/heads/work071)] Generate XXSPLTIDP on power10. X-Act-Checkin: gcc X-Git-Author: Michael Meissner X-Git-Refname: refs/users/meissner/heads/work071 X-Git-Oldrev: 9a6ff130fbda39593fc2bf1db46a37207142cd0b X-Git-Newrev: 36dfb31512eaa866cc27741b300f6ef38671a640 Message-Id: <20211018170201.4E615385840D@sourceware.org> Date: Mon, 18 Oct 2021 17:02:01 +0000 (GMT) X-BeenThere: gcc-cvs@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-cvs mailing list List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Oct 2021 17:02:01 -0000 https://gcc.gnu.org/g:36dfb31512eaa866cc27741b300f6ef38671a640 commit 36dfb31512eaa866cc27741b300f6ef38671a640 Author: Michael Meissner Date: Mon Oct 18 13:01:36 2021 -0400 Generate XXSPLTIDP on power10. This patch implements XXSPLTIDP support for SF, and DF scalar constants and all vector constants. The XXSPLTIDP instruction is given a 32-bit immediate that is converted to a vector of two DFmode constants. The immediate is in SFmode format, so only constants that fit as SFmode values can be loaded with XXSPLTIDP. I added one new constraint (eP) which matches instructions that load a VSX register with one prefixed instruction. This patch adds the XXSPLTIDP support. The next patch will add the XXSPLTIW support. DImode scalar constants are not handled. This is due to the majority of DImode constants will be in the GPR registers. With vector registers, you have the problem that XXSPLTIDP splats the double word into both elements of the vector. However, if TImode is loaded with an integer constant, it wants a full 128-bit constant. I have added a temporary switch (-mxxspltidp) to control whether or not the XXSPLTIDP instruction is generated. I added 4 new tests to test loading up SF/DF scalar and vector constants. This patch updates the previous patch to take into account the comments from the patch review. The main change is that this patch does is map each vector and scalar to provide all of bits and then match those bits to see if the XXSPLTIDP instruction can generate the bits necessary, even if the values in the vector aren't DFmode constants. A framework is provided in this patch which will also be used in future patches adding LXVKQ and XXSPLTIW support (possibly XXSPLTI32DX). This makes it easy for easy_fp_constant and easy_vector_constant to have multiple checks (such as for XXSPLTIW, LXVKQ, and XXSPLTI32DX) that each want to build the bitmask of what the vector constant looks like. While the PowerPC is currently limited to 128-bit vectors, I have written the code so it can be changed in the future if we ever have larger vection sizes. 2021-10-18 Michael Meissner gcc/ * config/rs6000/constraints.md (eP): New constraint. * config/rs6000/predicates.md (easy_fp_constant): Add support for generating XXSPLTIDP. (vsx_prefixed_constant): New predicate. (easy_vector_constant): Add support for generating XXSPLTIDP. * config/rs6000/rs6000-protos.h (prefixed_xxsplti_p): New declaration. (VECTOR_CONST_*): New macros. (rs6000_vec_const): New structure to hold information about vector constants. (vec_const_to_bytes): New function. (vec_const_use_xxspltidp): New function. * config/rs6000/rs6000.c (output_vec_const_move): Add support for XXSPLTIDP. (prefixed_xxsplti_p): New function. (vec_const_integer): New helper function. (vec_const_floating_point): New helper function. (vec_const_use_xxspltidp): New function. (vec_const_to_bytes): New function. * config/rs6000/rs6000.md (UNSPEC_VSX_PREFIXED_CONST): New unspec. (prefixed attribute): Add support for prefixed instructions to load * constants into VSX registers. (movsf_hardfloat): Add support for XXSPLTIDP. (mov_hardfloat32, FMOVE64 iterator): Likewise. (mov_hardfloat64, FMOVE64 iterator): Likewise. (xxspltidp__internal): New insns. (splitter for VSX prefix constants): New splitters. * config/rs6000/rs6000.opt (-mxxspltidp): New debug option. * config/rs6000/vsx.md (vsx_mov_64bit): Add support for XXSPLTIDP. (vsx_mov_32bit): Likewise. * doc/md.texi (PowerPC and IBM RS6000 constraints): Document the eP constraint. gcc/testsuite/ * gcc.target/powerpc/pr86731-fwrapv-longlong.c: Update insn regex for power10. * gcc.target/powerpc/vec-splat-constant-df.c: New test. * gcc.target/powerpc/vec-splat-constant-sf.c: New test. * gcc.target/powerpc/vec-splat-constant-v2df.c: New test. * gcc.target/powerpc/vec-splat-constant-v2di.c: New test. * gcc.target/powerpc/vec-splati-runnable.c: Update insn counts. Diff: --- gcc/config/rs6000/constraints.md | 6 + gcc/config/rs6000/predicates.md | 51 +++ gcc/config/rs6000/rs6000-protos.h | 27 ++ gcc/config/rs6000/rs6000.c | 387 +++++++++++++++++++++ gcc/config/rs6000/rs6000.md | 82 ++++- gcc/config/rs6000/rs6000.opt | 4 + gcc/config/rs6000/vsx.md | 32 +- gcc/doc/md.texi | 4 + .../gcc.target/powerpc/pr86731-fwrapv-longlong.c | 9 +- .../gcc.target/powerpc/vec-splat-constant-df.c | 60 ++++ .../gcc.target/powerpc/vec-splat-constant-sf.c | 60 ++++ .../gcc.target/powerpc/vec-splat-constant-v2df.c | 64 ++++ .../gcc.target/powerpc/vec-splat-constant-v2di.c | 50 +++ .../gcc.target/powerpc/vec-splati-runnable.c | 2 +- 14 files changed, 809 insertions(+), 29 deletions(-) diff --git a/gcc/config/rs6000/constraints.md b/gcc/config/rs6000/constraints.md index c8cff1a3038..7d594872a78 100644 --- a/gcc/config/rs6000/constraints.md +++ b/gcc/config/rs6000/constraints.md @@ -213,6 +213,12 @@ "A signed 34-bit integer constant if prefixed instructions are supported." (match_operand 0 "cint34_operand")) +;; A SF/DF scalar constant or a vector constant that can be loaded into vector +;; registers with one prefixed instruction such as XXSPLTIDP. +(define_constraint "eP" + "A constant that can be loaded into a VSX register with one prefixed insn." + (match_operand 0 "vsx_prefixed_constant")) + ;; Floating-point constraints. These two are defined so that insn ;; length attributes can be calculated exactly. diff --git a/gcc/config/rs6000/predicates.md b/gcc/config/rs6000/predicates.md index 956e42bc514..4b2bbdf40e8 100644 --- a/gcc/config/rs6000/predicates.md +++ b/gcc/config/rs6000/predicates.md @@ -601,6 +601,15 @@ if (TARGET_VSX && op == CONST0_RTX (mode)) return 1; + /* Constants that can be generated with ISA 3.1 instructions are easy. */ + rs6000_vec_const vec_const; + + if (TARGET_POWER10 && vec_const_to_bytes (op, mode, &vec_const)) + { + if (vec_const_use_xxspltidp (&vec_const)) + return true; + } + /* Otherwise consider floating point constants hard, so that the constant gets pushed to memory during the early RTL phases. This has the advantage that double precision constants that can be @@ -609,6 +618,38 @@ return 0; }) +;; Return 1 if the operand is a 64-bit floating point scalar constant or a +;; vector constant that can be loaded to a VSX register with one prefixed +;; instruction, such as XXSPLTIDP. +;; +;; In addition regular constants, we also recognize constants formed with the +;; VEC_DUPLICATE insn from scalar constants. +;; +;; We don't handle scalar integer constants here because the assumption is the +;; normal integer constants will be loaded into GPR registers. For the +;; constants that need to be loaded into vector registers, the instructions +;; don't work well with TImode variables assigned a constant. This is because +;; the 64-bit scalar constants are splatted into both halves of the register. + +(define_predicate "vsx_prefixed_constant" + (match_code "const_double,const_vector,vec_duplicate") +{ + rs6000_vec_const vec_const; + + /* Do we have prefixed instructions and are VSX registers available? Is the + constant recognized? */ + if (!TARGET_PREFIXED || !TARGET_VSX) + return false; + + if (!vec_const_to_bytes (op, mode, &vec_const)) + return false; + + if (vec_const_use_xxspltidp (&vec_const)) + return true; + + return false; +}) + ;; Return 1 if the operand is a constant that can loaded with a XXSPLTIB ;; instruction and then a VUPKHSB, VECSB2W or VECSB2D instruction. @@ -657,6 +698,16 @@ && xxspltib_constant_p (op, mode, &num_insns, &value)) return true; + /* See if the constant can be generated with the ISA 3.1 + instructions. */ + rs6000_vec_const vec_const; + + if (TARGET_POWER10 && vec_const_to_bytes (op, mode, &vec_const)) + { + if (vec_const_use_xxspltidp (&vec_const)) + return true; + } + return easy_altivec_constant (op, mode); } diff --git a/gcc/config/rs6000/rs6000-protos.h b/gcc/config/rs6000/rs6000-protos.h index 14f6b313105..8eef955237a 100644 --- a/gcc/config/rs6000/rs6000-protos.h +++ b/gcc/config/rs6000/rs6000-protos.h @@ -198,6 +198,7 @@ enum non_prefixed_form reg_to_non_prefixed (rtx reg, machine_mode mode); extern bool prefixed_load_p (rtx_insn *); extern bool prefixed_store_p (rtx_insn *); extern bool prefixed_paddi_p (rtx_insn *); +extern bool prefixed_xxsplti_p (rtx_insn *); extern void rs6000_asm_output_opcode (FILE *); extern void output_pcrel_opt_reloc (rtx); extern void rs6000_final_prescan_insn (rtx_insn *, rtx [], int); @@ -222,6 +223,32 @@ address_is_prefixed (rtx addr, return (iform == INSN_FORM_PREFIXED_NUMERIC || iform == INSN_FORM_PCREL_LOCAL); } + +/* Functions and data structures relating to 128-bit vector constants. All + fields are kept in big endian order. */ +#define VECTOR_CONST_BITS 128 +#define VECTOR_CONST_BYTES (VECTOR_CONST_BITS / 8) +#define VECTOR_CONST_HALF_WORDS (VECTOR_CONST_BITS / 16) +#define VECTOR_CONST_WORDS (VECTOR_CONST_BITS / 32) +#define VECTOR_CONST_DOUBLE_WORDS (VECTOR_CONST_BITS / 64) + +typedef struct { + /* Vector constant as various sized items. */ + unsigned HOST_WIDE_INT double_words[VECTOR_CONST_DOUBLE_WORDS]; + unsigned int words[VECTOR_CONST_WORDS]; + unsigned short half_words[VECTOR_CONST_HALF_WORDS]; + unsigned char bytes[VECTOR_CONST_BYTES]; + + unsigned int xxspltidp_immediate; /* Immediate value for XXSPLTIDP. */ + bool fp_constant_p; /* Is the constant floating point? */ + bool all_double_words_same; /* Are the double words all equal? */ + bool all_words_same; /* Are the words all equal? */ + bool all_half_words_same; /* Are the halft words all equal? */ + bool all_bytes_same; /* Are the bytes all equal? */ +} rs6000_vec_const; + +extern bool vec_const_to_bytes (rtx, machine_mode, rs6000_vec_const *); +extern bool vec_const_use_xxspltidp (rs6000_vec_const *); #endif /* RTX_CODE */ #ifdef TREE_CODE diff --git a/gcc/config/rs6000/rs6000.c b/gcc/config/rs6000/rs6000.c index acba4d9f26c..353ec2b572d 100644 --- a/gcc/config/rs6000/rs6000.c +++ b/gcc/config/rs6000/rs6000.c @@ -6990,6 +6990,16 @@ output_vec_const_move (rtx *operands) gcc_unreachable (); } + rs6000_vec_const vec_const; + if (TARGET_POWER10 && vec_const_to_bytes (vec, mode, &vec_const)) + { + if (vec_const_use_xxspltidp (&vec_const)) + { + operands[2] = GEN_INT (vec_const.xxspltidp_immediate); + return "xxspltidp %x0,%2"; + } + } + if (TARGET_P9_VECTOR && xxspltib_constant_p (vec, mode, &num_insns, &xxspltib_value)) { @@ -26724,6 +26734,41 @@ prefixed_paddi_p (rtx_insn *insn) return (iform == INSN_FORM_PCREL_EXTERNAL || iform == INSN_FORM_PCREL_LOCAL); } +/* Whether a permute type instruction is a prefixed XXSPLTI* instruction. + This is called from the prefixed attribute processing. */ + +bool +prefixed_xxsplti_p (rtx_insn *insn) +{ + rtx set = single_set (insn); + if (!set) + return false; + + rtx dest = SET_DEST (set); + rtx src = SET_SRC (set); + machine_mode mode = GET_MODE (dest); + + if (!REG_P (dest) && !SUBREG_P (dest)) + return false; + + if (GET_CODE (src) == UNSPEC) + { + int unspec = XINT (src, 1); + return (unspec == UNSPEC_XXSPLTIW + || unspec == UNSPEC_XXSPLTIDP + || unspec == UNSPEC_XXSPLTI32DX); + } + + rs6000_vec_const vec_const; + if (vec_const_to_bytes (src, mode, &vec_const)) + { + if (vec_const_use_xxspltidp (&vec_const)) + return true; + } + + return false; +} + /* Whether the next instruction needs a 'p' prefix issued before the instruction is printed out. */ static bool prepend_p_to_next_insn; @@ -28587,6 +28632,348 @@ rs6000_output_addr_vec_elt (FILE *file, int value) fprintf (file, "\n"); } + +/* Copy an integer constant to the vector constant structure. */ + +static void +vec_const_integer (rtx op, + machine_mode mode, + size_t byte_num, + rs6000_vec_const *vec_const) +{ + unsigned HOST_WIDE_INT uvalue = UINTVAL (op); + unsigned bitsize = GET_MODE_BITSIZE (mode); + + for (int shift = bitsize - 8; shift >= 0; shift -= 8) + vec_const->bytes[byte_num++] = (uvalue >> shift) & 0xff; +} + +/* Copy an floating point constant to the vector constant structure. */ + +static void +vec_const_floating_point (rtx op, + machine_mode mode, + size_t byte_num, + rs6000_vec_const *vec_const) +{ + unsigned bitsize = GET_MODE_BITSIZE (mode); + unsigned num_words = bitsize / 32; + const REAL_VALUE_TYPE *rtype = CONST_DOUBLE_REAL_VALUE (op); + long real_words[VECTOR_CONST_WORDS]; + + /* Make sure we don't overflow the real_words array and that it is + filled completely. */ + gcc_assert (bitsize <= VECTOR_CONST_BITS && (bitsize % 32) == 0); + + real_to_target (real_words, rtype, mode); + + /* Iterate over each 32-bit word in the floating point constant. The + real_to_target function puts out words in endian fashion. We need + to arrange so the words are written in big endian order. */ + for (unsigned num = 0; num < num_words; num++) + { + unsigned endian_num = (BYTES_BIG_ENDIAN + ? num + : num_words - 1 - num); + + unsigned uvalue = real_words[endian_num]; + for (int shift = 32 - 8; shift >= 0; shift -= 8) + vec_const->bytes[byte_num++] = (uvalue >> shift) & 0xff; + } + + /* Mark that this constant involes floating point. */ + vec_const->fp_constant_p = true; +} + +/* Determine if a vector constant can be loaded with XXSPLTIDP. If so, + fill out the fields used to generate the instruction. */ + +bool +vec_const_use_xxspltidp (rs6000_vec_const *vec_const) +{ + if (!TARGET_XXSPLTIDP || !TARGET_PREFIXED || !TARGET_VSX) + return false; + + /* Make sure that the two 64-bit segments are the same. */ + if (!vec_const->all_double_words_same) + return false; + + /* If the bytes, half words, or words are all the same, don't use XXSPLTIDP. + Use a simpler instruction (XXSPLTIB, VSPLTISB, VSPLTISH, or VSPLTISW). */ + if (vec_const->all_bytes_same + || vec_const->all_half_words_same + || vec_const->all_words_same) + return false; + + unsigned HOST_WIDE_INT value = vec_const->double_words[0]; + + /* Avoid values that look like DFmode NaN's, except for the normal NaN bit + pattern and the signalling NaN bit pattern. Recognize infinity and + negative infinity. */ + + /* Bit representation of DFmode normal quiet NaN. */ +#define VECTOR_CONST_DF_NAN HOST_WIDE_INT_UC (0x7ff8000000000000) + + /* Bit representation of DFmode normal signaling NaN. */ +#define VECTOR_CONST_DF_NANS HOST_WIDE_INT_UC (0x7ff4000000000000) + + /* Bit representation of DFmode positive infinity. */ +#define VECTOR_CONST_DF_INF HOST_WIDE_INT_UC (0x7ff0000000000000) + + /* Bit representation of DFmode negative infinity. */ +#define VECTOR_CONST_DF_NEG_INF HOST_WIDE_INT_UC (0xfff0000000000000) + + if (value != VECTOR_CONST_DF_NAN + && value != VECTOR_CONST_DF_NANS + && value != VECTOR_CONST_DF_INF + && value != VECTOR_CONST_DF_NEG_INF) + { + /* The IEEE 754 64-bit floating format has 1 bit for sign, 11 bits for + the exponent, and 52 bits for the mantissa (not counting the hidden + bit used for normal numbers). NaN values have the exponent set to all + 1 bits, and the mantissa non-zero (mantissa == 0 is infinity). */ + + int df_exponent = (value >> 52) & 0x7ff; + unsigned HOST_WIDE_INT df_mantissa + = value & ((HOST_WIDE_INT_1U << 52) - HOST_WIDE_INT_1U); + + if (df_exponent == 0x7ff && df_mantissa != 0) /* other NaNs. */ + return false; + + /* Avoid values that are DFmode subnormal values. Subnormal numbers have + the exponent all 0 bits, and the mantissa non-zero. If the value is + subnormal, then the hidden bit in the mantissa is not set. */ + if (df_exponent == 0 && df_mantissa != 0) /* subnormal. */ + return false; + } + + /* Change the representation to DFmode constant. */ + long df_words[2] = { vec_const->words[0], vec_const->words[1] }; + + /* real_from_target takes the target words in target order. */ + if (!BYTES_BIG_ENDIAN) + std::swap (df_words[0], df_words[1]); + + REAL_VALUE_TYPE rv_type; + real_from_target (&rv_type, df_words, DFmode); + + const REAL_VALUE_TYPE *rv = &rv_type; + + /* Validate that the number can be stored as a SFmode value. */ + if (!exact_real_truncate (SFmode, rv)) + return false; + + /* Validate that the number is not a SFmode subnormal value (exponent is 0, + mantissa field is non-zero) which is undefined for the XXSPLTIDP + instruction. */ + long sf_value; + real_to_target (&sf_value, rv, SFmode); + + /* IEEE 754 32-bit values have 1 bit for the sign, 8 bits for the exponent, + and 23 bits for the mantissa. Subnormal numbers have the exponent all + 0 bits, and the mantissa non-zero. */ + long sf_exponent = (sf_value >> 23) & 0xFF; + long sf_mantissa = sf_value & 0x7FFFFF; + + if (sf_exponent == 0 && sf_mantissa != 0) + return false; + + /* Record the information in the vec_const structure for XXSPLTIDP. */ + vec_const->xxspltidp_immediate = sf_value; + + return true; +} + +/* Convert a vector constant to an internal structure, breaking it out to + bytes, half words, words, and double words. Return true if we have + successfully broken it out. */ + +bool +vec_const_to_bytes (rtx op, + machine_mode mode, + rs6000_vec_const *vec_const) +{ + /* Initialize vec const structure. */ + memset ((void *)vec_const, 0, sizeof (rs6000_vec_const)); + + /* Set up the vector bits. */ + switch (GET_CODE (op)) + { + /* Integer constants, default to double word. */ + case CONST_INT: + { + /* Scalars are treated as 64-bit integers. */ + if (mode == VOIDmode) + mode = DImode; + + vec_const_integer (op, mode, 0, vec_const); + + /* Splat the constant to the rest of the vector constant structure. */ + unsigned size = GET_MODE_SIZE (mode); + gcc_assert (size <= VECTOR_CONST_BYTES); + gcc_assert ((VECTOR_CONST_BYTES % size) == 0); + + for (size_t splat = size; splat < VECTOR_CONST_BYTES; splat += size) + memcpy ((void *) &vec_const->bytes[splat], + (void *) &vec_const->bytes[0], + size); + break; + } + + /* Floating point constants. */ + case CONST_DOUBLE: + { + /* Fail if the floating point constant is the wrong mode. */ + if (mode == VOIDmode) + mode = GET_MODE (op); + + else if (GET_MODE (op) != mode) + return false; + + /* SFmode stored as scalars are stored in DFmode format. */ + if (mode == SFmode) + mode = DFmode; + + vec_const_floating_point (op, mode, 0, vec_const); + + /* Splat the constant to the rest of the vector constant structure. */ + unsigned size = GET_MODE_SIZE (mode); + gcc_assert (size <= VECTOR_CONST_BYTES); + gcc_assert ((VECTOR_CONST_BYTES % size) == 0); + + for (size_t splat = size; splat < VECTOR_CONST_BYTES; splat += size) + memcpy ((void *) &vec_const->bytes[splat], + (void *) &vec_const->bytes[0], + size); + break; + } + + /* Vector constants, iterate each element. On little endian systems, we + have to reverse the element numbers. */ + case CONST_VECTOR: + { + /* Fail if the vector constant is the wrong mode. */ + if (mode == VOIDmode) + mode = GET_MODE (op); + + else if (GET_MODE (op) != mode) + return false; + + machine_mode ele_mode = GET_MODE_INNER (mode); + size_t nunits = GET_MODE_NUNITS (mode); + size_t size = GET_MODE_SIZE (ele_mode); + + for (size_t num = 0; num < nunits; num++) + { + rtx ele = (GET_CODE (op) == VEC_DUPLICATE + ? XEXP (op, 0) + : CONST_VECTOR_ELT (op, num)); + size_t byte_num = (BYTES_BIG_ENDIAN + ? num + : nunits - 1 - num) * size; + + if (CONST_INT_P (ele)) + vec_const_integer (ele, ele_mode, byte_num, vec_const); + else if (CONST_DOUBLE_P (ele)) + vec_const_floating_point (ele, ele_mode, byte_num, vec_const); + else + return false; + } + + break; + } + + /* Treat VEC_DUPLICATE of a constant just like a vector constant. */ + case VEC_DUPLICATE: + { + /* Fail if the vector duplicate is the wrong mode. */ + if (mode == VOIDmode) + mode = GET_MODE (op); + + else if (GET_MODE (op) != mode) + return false; + + machine_mode ele_mode = GET_MODE_INNER (mode); + size_t nunits = GET_MODE_NUNITS (mode); + size_t size = GET_MODE_SIZE (ele_mode); + rtx ele = XEXP (op, 0); + + if (!CONST_INT_P (ele) && !CONST_DOUBLE_P (ele)) + return false; + + for (size_t num = 0; num < nunits; num++) + { + size_t byte_num = num * size; + + if (CONST_INT_P (ele)) + vec_const_integer (ele, ele_mode, byte_num, vec_const); + else + vec_const_floating_point (ele, ele_mode, byte_num, vec_const); + } + + break; + } + + /* Any thing else, just return failure. */ + default: + return false; + } + + /* Pack half words together. */ + for (size_t i = 0; i < VECTOR_CONST_HALF_WORDS; i++) + vec_const->half_words[i] = ((vec_const->bytes[2*i] << 8) + | vec_const->bytes[(2 * i) + 1]); + + /* Pack words together. */ + for (size_t i = 0; i < VECTOR_CONST_WORDS; i++) + { + unsigned word = 0; + for (size_t j = 0; j < 4; j++) + word = (word << 8) | vec_const->bytes[(4 * i) + j]; + + vec_const->words[i] = word; + } + + /* Pack double words together. */ + for (size_t i = 0; i < VECTOR_CONST_DOUBLE_WORDS; i++) + { + unsigned HOST_WIDE_INT d_word = 0; + for (size_t j = 0; j < 8; j++) + d_word = (d_word << 8) | vec_const->bytes[(8 * i) + j]; + + vec_const->double_words[i] = d_word; + } + + /* Determine if the double words, words, half words, and bytes are all + equal. */ + unsigned HOST_WIDE_INT first_dword = vec_const->double_words[0]; + vec_const->all_double_words_same = true; + for (size_t i = 1; i < VECTOR_CONST_DOUBLE_WORDS; i++) + if (first_dword != vec_const->double_words[i]) + vec_const->all_double_words_same = false; + + unsigned int first_word = vec_const->words[0]; + vec_const->all_words_same = true; + for (size_t i = 1; i < VECTOR_CONST_WORDS; i++) + if (first_word != vec_const->words[i]) + vec_const->all_words_same = false; + + unsigned short first_hword = vec_const->half_words[0]; + vec_const->all_half_words_same = true; + for (size_t i = 1; i < VECTOR_CONST_HALF_WORDS; i++) + if (first_hword != vec_const->half_words[i]) + vec_const->all_half_words_same = false; + + unsigned char first_byte = vec_const->bytes[0]; + vec_const->all_bytes_same = true; + for (size_t i = 1; i < VECTOR_CONST_BYTES; i++) + if (first_byte != vec_const->bytes[i]) + vec_const->all_bytes_same = false; + + return true; +} + + struct gcc_target targetm = TARGET_INITIALIZER; #include "gt-rs6000.h" diff --git a/gcc/config/rs6000/rs6000.md b/gcc/config/rs6000/rs6000.md index 6bec2bddbde..b0ead908fe9 100644 --- a/gcc/config/rs6000/rs6000.md +++ b/gcc/config/rs6000/rs6000.md @@ -156,6 +156,7 @@ UNSPEC_PEXTD UNSPEC_HASHST UNSPEC_HASHCHK + UNSPEC_VSX_PREFIXED_CONST ]) ;; @@ -314,6 +315,11 @@ (eq_attr "type" "integer,add") (if_then_else (match_test "prefixed_paddi_p (insn)") + (const_string "yes") + (const_string "no")) + + (eq_attr "type" "vecperm") + (if_then_else (match_test "prefixed_xxsplti_p (insn)") (const_string "yes") (const_string "no"))] @@ -7759,17 +7765,17 @@ ;; ;; LWZ LFS LXSSP LXSSPX STFS STXSSP ;; STXSSPX STW XXLXOR LI FMR XSCPSGNDP -;; MR MT MF NOP +;; MR MT MF NOP XXSPLTIDP (define_insn "movsf_hardfloat" [(set (match_operand:SF 0 "nonimmediate_operand" "=!r, f, v, wa, m, wY, Z, m, wa, !r, f, wa, - !r, *c*l, !r, *h") + !r, *c*l, !r, *h, wa") (match_operand:SF 1 "input_operand" "m, m, wY, Z, f, v, wa, r, j, j, f, wa, - r, r, *h, 0"))] + r, r, *h, 0, eP"))] "(register_operand (operands[0], SFmode) || register_operand (operands[1], SFmode)) && TARGET_HARD_FLOAT @@ -7791,15 +7797,16 @@ mr %0,%1 mt%0 %1 mf%1 %0 - nop" + nop + #" [(set_attr "type" "load, fpload, fpload, fpload, fpstore, fpstore, fpstore, store, veclogical, integer, fpsimple, fpsimple, - *, mtjmpr, mfjmpr, *") + *, mtjmpr, mfjmpr, *, vecperm") (set_attr "isa" "*, *, p9v, p8v, *, p9v, p8v, *, *, *, *, *, - *, *, *, *")]) + *, *, *, *, p10")]) ;; LWZ LFIWZX STW STFIWX MTVSRWZ MFVSRWZ ;; FMR MR MT%0 MF%1 NOP @@ -8059,18 +8066,18 @@ ;; STFD LFD FMR LXSD STXSD ;; LXSD STXSD XXLOR XXLXOR GPR<-0 -;; LWZ STW MR +;; LWZ STW MR XXSPLTIDP (define_insn "*mov_hardfloat32" [(set (match_operand:FMOVE64 0 "nonimmediate_operand" "=m, d, d, , wY, , Z, , , !r, - Y, r, !r") + Y, r, !r, wa") (match_operand:FMOVE64 1 "input_operand" "d, m, d, wY, , Z, , , , , - r, Y, r"))] + r, Y, r, eP"))] "! TARGET_POWERPC64 && TARGET_HARD_FLOAT && (gpc_reg_operand (operands[0], mode) || gpc_reg_operand (operands[1], mode))" @@ -8087,20 +8094,21 @@ # # # + # #" [(set_attr "type" "fpstore, fpload, fpsimple, fpload, fpstore, fpload, fpstore, veclogical, veclogical, two, - store, load, two") + store, load, two, vecperm") (set_attr "size" "64") (set_attr "length" "*, *, *, *, *, *, *, *, *, 8, - 8, 8, 8") + 8, 8, 8, *") (set_attr "isa" "*, *, *, p9v, p9v, p7v, p7v, *, *, *, - *, *, *")]) + *, *, *, p10")]) ;; STW LWZ MR G-const H-const F-const @@ -8127,19 +8135,19 @@ ;; STFD LFD FMR LXSD STXSD ;; LXSDX STXSDX XXLOR XXLXOR LI 0 ;; STD LD MR MT{CTR,LR} MF{CTR,LR} -;; NOP MFVSRD MTVSRD +;; NOP MFVSRD MTVSRD XXSPLTIDP (define_insn "*mov_hardfloat64" [(set (match_operand:FMOVE64 0 "nonimmediate_operand" "=m, d, d, , wY, , Z, , , !r, YZ, r, !r, *c*l, !r, - *h, r, ") + *h, r, , wa") (match_operand:FMOVE64 1 "input_operand" "d, m, d, wY, , Z, , , , , r, YZ, r, r, *h, - 0, , r"))] + 0, , r, eP"))] "TARGET_POWERPC64 && TARGET_HARD_FLOAT && (gpc_reg_operand (operands[0], mode) || gpc_reg_operand (operands[1], mode))" @@ -8161,18 +8169,19 @@ mf%1 %0 nop mfvsrd %0,%x1 - mtvsrd %x0,%1" + mtvsrd %x0,%1 + #" [(set_attr "type" "fpstore, fpload, fpsimple, fpload, fpstore, fpload, fpstore, veclogical, veclogical, integer, store, load, *, mtjmpr, mfjmpr, - *, mfvsr, mtvsr") + *, mfvsr, mtvsr, vecperm") (set_attr "size" "64") (set_attr "isa" "*, *, *, p9v, p9v, p7v, p7v, *, *, *, *, *, *, *, *, - *, p8v, p8v")]) + *, p8v, p8v, p10")]) ;; STD LD MR MT MF G-const ;; H-const F-const Special @@ -8206,6 +8215,43 @@ (set_attr "length" "*, *, *, *, *, 8, 12, 16, *")]) + +;; Split the VSX prefixed instruction to support SFmode and DFmode scalar +;; constants that look like DFmode floating point values where both elements +;; are the same. The constant has to be expressible as a SFmode constant that +;; is not a SFmode denormal value. +;; +;; We don't need splitters for the 128-bit types, since the function +;; rs6000_output_move_128bit handles the generation of XXSPLTIDP. +(define_insn "*xxspltidp__internal" + [(set (match_operand:SFDF 0 "register_operand" "=wa") + (unspec:SFDF [(match_operand:SI 1 "c32bit_cint_operand" "n")] + UNSPEC_VSX_PREFIXED_CONST))] + "TARGET_POWER10" + "xxspltidp %x0,%1" + [(set_attr "type" "vecperm") + (set_attr "prefixed" "yes")]) + +(define_split + [(set (match_operand:SFDF 0 "vsx_register_operand") + (match_operand:SFDF 1 "vsx_prefixed_constant"))] + "TARGET_POWER10" + [(set (match_dup 0) + (unspec:SFDF [(match_dup 2)] UNSPEC_VSX_PREFIXED_CONST))] +{ + rtx src = operands[1]; + rs6000_vec_const vec_const; + + if (!vec_const_to_bytes (src, mode, &vec_const)) + gcc_unreachable (); + + if (vec_const_use_xxspltidp (&vec_const)) + operands[2] = GEN_INT (vec_const.xxspltidp_immediate); + + else + gcc_unreachable (); +}) + (define_expand "mov" [(set (match_operand:FMOVE128 0 "general_operand") diff --git a/gcc/config/rs6000/rs6000.opt b/gcc/config/rs6000/rs6000.opt index 9d7878f144a..1d7ce4cc94a 100644 --- a/gcc/config/rs6000/rs6000.opt +++ b/gcc/config/rs6000/rs6000.opt @@ -640,6 +640,10 @@ mprivileged Target Var(rs6000_privileged) Init(0) Generate code that will run in privileged state. +mxxspltidp +Target Undocumented Var(TARGET_XXSPLTIDP) Init(1) Save +Generate (do not generate) XXSPLTIDP instructions. + -param=rs6000-density-pct-threshold= Target Undocumented Joined UInteger Var(rs6000_density_pct_threshold) Init(85) IntegerRange(0, 100) Param When costing for loop vectorization, we probably need to penalize the loop body diff --git a/gcc/config/rs6000/vsx.md b/gcc/config/rs6000/vsx.md index bf033e31c1c..c8518496339 100644 --- a/gcc/config/rs6000/vsx.md +++ b/gcc/config/rs6000/vsx.md @@ -1192,16 +1192,19 @@ ;; VSX store VSX load VSX move VSX->GPR GPR->VSX LQ (GPR) ;; STQ (GPR) GPR load GPR store GPR move XXSPLTIB VSPLTISW +;; XXLSPLTIDP ;; VSX 0/-1 VMX const GPR const LVX (VMX) STVX (VMX) (define_insn "vsx_mov_64bit" [(set (match_operand:VSX_M 0 "nonimmediate_operand" "=ZwO, wa, wa, r, we, ?wQ, ?&r, ??r, ??Y, , wa, v, + wa, ?wa, v, , wZ, v") (match_operand:VSX_M 1 "input_operand" "wa, ZwO, wa, we, r, r, wQ, Y, r, r, wE, jwM, + eP, ?jwM, W, , v, wZ"))] "TARGET_POWERPC64 && VECTOR_MEM_VSX_P (mode) @@ -1213,36 +1216,47 @@ [(set_attr "type" "vecstore, vecload, vecsimple, mtvsr, mfvsr, load, store, load, store, *, vecsimple, vecsimple, + vecperm, vecsimple, *, *, vecstore, vecload") (set_attr "num_insns" "*, *, *, 2, *, 2, 2, 2, 2, 2, *, *, + *, *, 5, 2, *, *") (set_attr "max_prefixed_insns" "*, *, *, *, *, 2, 2, 2, 2, 2, *, *, + *, *, *, *, *, *") (set_attr "length" "*, *, *, 8, *, 8, 8, 8, 8, 8, *, *, + *, *, 20, 8, *, *") (set_attr "isa" ", , , *, *, *, *, *, *, *, p9v, *, + p10, , *, *, *, *")]) ;; VSX store VSX load VSX move GPR load GPR store GPR move -;; XXSPLTIB VSPLTISW VSX 0/-1 VMX const GPR const +;; XXSPLTIB VSPLTISW VSX 0/-1 +;; XXSPLTIDP +;; VMX const GPR const ;; LVX (VMX) STVX (VMX) (define_insn "*vsx_mov_32bit" [(set (match_operand:VSX_M 0 "nonimmediate_operand" "=ZwO, wa, wa, ??r, ??Y, , - wa, v, ?wa, v, , + wa, v, ?wa, + wa, + v, , wZ, v") (match_operand:VSX_M 1 "input_operand" "wa, ZwO, wa, Y, r, r, - wE, jwM, ?jwM, W, , + wE, jwM, ?jwM, + eP, + W, , v, wZ"))] "!TARGET_POWERPC64 && VECTOR_MEM_VSX_P (mode) @@ -1253,15 +1267,21 @@ } [(set_attr "type" "vecstore, vecload, vecsimple, load, store, *, - vecsimple, vecsimple, vecsimple, *, *, + vecsimple, vecsimple, vecsimple, + vecperm, + *, *, vecstore, vecload") (set_attr "length" "*, *, *, 16, 16, 16, - *, *, *, 20, 16, + *, *, *, + *, + 20, 16, *, *") (set_attr "isa" ", , , *, *, *, - p9v, *, , *, *, + p9v, *, , + p10, + *, *, *, *")]) ;; Explicit load/store expanders for the builtin functions diff --git a/gcc/doc/md.texi b/gcc/doc/md.texi index 41f1850bf6e..13b56279565 100644 --- a/gcc/doc/md.texi +++ b/gcc/doc/md.texi @@ -3336,6 +3336,10 @@ A constant whose negation is a signed 16-bit constant. @item eI A signed 34-bit integer constant if prefixed instructions are supported. +@item eP +A scalar floating point constant or a vector constant that can be +loaded with one prefixed instruction to a VSX register. + @ifset INTERNALS @item G A floating point constant that can be loaded into a register with one diff --git a/gcc/testsuite/gcc.target/powerpc/pr86731-fwrapv-longlong.c b/gcc/testsuite/gcc.target/powerpc/pr86731-fwrapv-longlong.c index bd1502bb30a..dcb30e1d886 100644 --- a/gcc/testsuite/gcc.target/powerpc/pr86731-fwrapv-longlong.c +++ b/gcc/testsuite/gcc.target/powerpc/pr86731-fwrapv-longlong.c @@ -24,11 +24,12 @@ vector signed long long splats4(void) return (vector signed long long) vec_sl(mzero, mzero); } -/* Codegen will consist of splat and shift instructions for most types. - If folding is enabled, the vec_sl tests using vector long long type will - generate a lvx instead of a vspltisw+vsld pair. */ +/* Codegen will consist of splat and shift instructions for most types. If + folding is enabled, the vec_sl tests using vector long long type will + generate a lvx instead of a vspltisw+vsld pair. On power10, it will + generate a xxspltidp instruction instead of the lvx. */ /* { dg-final { scan-assembler-times {\mvspltis[bhw]\M} 0 } } */ /* { dg-final { scan-assembler-times {\mvsl[bhwd]\M} 0 } } */ -/* { dg-final { scan-assembler-times {\mp?lxv\M|\mlxv\M|\mlxvd2x\M} 2 } } */ +/* { dg-final { scan-assembler-times {\mp?lxv\M|\mlxv\M|\mlxvd2x\M|\mxxspltidp\M} 2 } } */ diff --git a/gcc/testsuite/gcc.target/powerpc/vec-splat-constant-df.c b/gcc/testsuite/gcc.target/powerpc/vec-splat-constant-df.c new file mode 100644 index 00000000000..8f6e176f9af --- /dev/null +++ b/gcc/testsuite/gcc.target/powerpc/vec-splat-constant-df.c @@ -0,0 +1,60 @@ +/* { dg-do compile } */ +/* { dg-require-effective-target power10_ok } */ +/* { dg-options "-mdejagnu-cpu=power10 -O2" } */ + +#include + +/* Test generating DFmode constants with the ISA 3.1 (power10) XXSPLTIDP + instruction. */ + +double +scalar_double_0 (void) +{ + return 0.0; /* XXSPLTIB or XXLXOR. */ +} + +double +scalar_double_1 (void) +{ + return 1.0; /* XXSPLTIDP. */ +} + +#ifndef __FAST_MATH__ +double +scalar_double_m0 (void) +{ + return -0.0; /* XXSPLTIDP. */ +} + +double +scalar_double_nan (void) +{ + return __builtin_nan (""); /* XXSPLTIDP. */ +} + +double +scalar_double_inf (void) +{ + return __builtin_inf (); /* XXSPLTIDP. */ +} + +double +scalar_double_m_inf (void) /* XXSPLTIDP. */ +{ + return - __builtin_inf (); +} +#endif + +double +scalar_double_pi (void) +{ + return M_PI; /* PLFD. */ +} + +double +scalar_double_denorm (void) +{ + return 0x1p-149f; /* PLFD. */ +} + +/* { dg-final { scan-assembler-times {\mxxspltidp\M} 5 } } */ diff --git a/gcc/testsuite/gcc.target/powerpc/vec-splat-constant-sf.c b/gcc/testsuite/gcc.target/powerpc/vec-splat-constant-sf.c new file mode 100644 index 00000000000..72504bdfbbd --- /dev/null +++ b/gcc/testsuite/gcc.target/powerpc/vec-splat-constant-sf.c @@ -0,0 +1,60 @@ +/* { dg-do compile } */ +/* { dg-require-effective-target power10_ok } */ +/* { dg-options "-mdejagnu-cpu=power10 -O2" } */ + +#include + +/* Test generating SFmode constants with the ISA 3.1 (power10) XXSPLTIDP + instruction. */ + +float +scalar_float_0 (void) +{ + return 0.0f; /* XXSPLTIB or XXLXOR. */ +} + +float +scalar_float_1 (void) +{ + return 1.0f; /* XXSPLTIDP. */ +} + +#ifndef __FAST_MATH__ +float +scalar_float_m0 (void) +{ + return -0.0f; /* XXSPLTIDP. */ +} + +float +scalar_float_nan (void) +{ + return __builtin_nanf (""); /* XXSPLTIDP. */ +} + +float +scalar_float_inf (void) +{ + return __builtin_inff (); /* XXSPLTIDP. */ +} + +float +scalar_float_m_inf (void) /* XXSPLTIDP. */ +{ + return - __builtin_inff (); +} +#endif + +float +scalar_float_pi (void) +{ + return (float)M_PI; /* XXSPLTIDP. */ +} + +float +scalar_float_denorm (void) +{ + return 0x1p-149f; /* PLFS. */ +} + +/* { dg-final { scan-assembler-times {\mxxspltidp\M} 6 } } */ diff --git a/gcc/testsuite/gcc.target/powerpc/vec-splat-constant-v2df.c b/gcc/testsuite/gcc.target/powerpc/vec-splat-constant-v2df.c new file mode 100644 index 00000000000..82ffc86f8aa --- /dev/null +++ b/gcc/testsuite/gcc.target/powerpc/vec-splat-constant-v2df.c @@ -0,0 +1,64 @@ +/* { dg-do compile } */ +/* { dg-require-effective-target power10_ok } */ +/* { dg-options "-mdejagnu-cpu=power10 -O2" } */ + +#include + +/* Test generating V2DFmode constants with the ISA 3.1 (power10) XXSPLTIDP + instruction. */ + +vector double +v2df_double_0 (void) +{ + return (vector double) { 0.0, 0.0 }; /* XXSPLTIB or XXLXOR. */ +} + +vector double +v2df_double_1 (void) +{ + return (vector double) { 1.0, 1.0 }; /* XXSPLTIDP. */ +} + +#ifndef __FAST_MATH__ +vector double +v2df_double_m0 (void) +{ + return (vector double) { -0.0, -0.0 }; /* XXSPLTIDP. */ +} + +vector double +v2df_double_nan (void) +{ + return (vector double) { __builtin_nan (""), + __builtin_nan ("") }; /* XXSPLTIDP. */ +} + +vector double +v2df_double_inf (void) +{ + return (vector double) { __builtin_inf (), + __builtin_inf () }; /* XXSPLTIDP. */ +} + +vector double +v2df_double_m_inf (void) +{ + return (vector double) { - __builtin_inf (), + - __builtin_inf () }; /* XXSPLTIDP. */ +} +#endif + +vector double +v2df_double_pi (void) +{ + return (vector double) { M_PI, M_PI }; /* PLVX. */ +} + +vector double +v2df_double_denorm (void) +{ + return (vector double) { (double)0x1p-149f, + (double)0x1p-149f }; /* PLVX. */ +} + +/* { dg-final { scan-assembler-times {\mxxspltidp\M} 5 } } */ diff --git a/gcc/testsuite/gcc.target/powerpc/vec-splat-constant-v2di.c b/gcc/testsuite/gcc.target/powerpc/vec-splat-constant-v2di.c new file mode 100644 index 00000000000..4d44f943d26 --- /dev/null +++ b/gcc/testsuite/gcc.target/powerpc/vec-splat-constant-v2di.c @@ -0,0 +1,50 @@ +/* { dg-do compile } */ +/* { dg-require-effective-target power10_ok } */ +/* { dg-options "-mdejagnu-cpu=power10 -O2" } */ + +/* Test generating V2DImode constants that have the same bit pattern as + V2DFmode constants that can be loaded with the XXSPLTIDP instruction with + the ISA 3.1 (power10). */ + +vector long long +vector_0 (void) +{ + /* XXSPLTIB or XXLXOR. */ + return (vector long long) { 0LL, 0LL }; +} + +vector long long +vector_1 (void) +{ + /* XXSPLTIB and VEXTSB2D. */ + return (vector long long) { 1LL, 1LL }; +} + +/* 0x8000000000000000LL is the bit pattern for -0.0, which can be generated + with XXSPLTISDP. */ +vector long long +vector_float_neg_0 (void) +{ + /* XXSPLTIDP. */ + return (vector long long) { 0x8000000000000000LL, 0x8000000000000000LL }; +} + +/* 0x3ff0000000000000LL is the bit pattern for 1.0 which can be generated with + XXSPLTISDP. */ +vector long long +vector_float_1_0 (void) +{ + /* XXSPLTIDP. */ + return (vector long long) { 0x3ff0000000000000LL, 0x3ff0000000000000LL }; +} + +/* 0x400921fb54442d18LL is the bit pattern for PI, which cannot be generated + with XXSPLTIDP. */ +vector long long +scalar_pi (void) +{ + /* PLXV. */ + return (vector long long) { 0x400921fb54442d18LL, 0x400921fb54442d18LL }; +} + +/* { dg-final { scan-assembler-times {\mxxspltidp\M} 2 } } */ diff --git a/gcc/testsuite/gcc.target/powerpc/vec-splati-runnable.c b/gcc/testsuite/gcc.target/powerpc/vec-splati-runnable.c index a135279b1d7..5f84930e1a7 100644 --- a/gcc/testsuite/gcc.target/powerpc/vec-splati-runnable.c +++ b/gcc/testsuite/gcc.target/powerpc/vec-splati-runnable.c @@ -150,7 +150,7 @@ main (int argc, char *argv []) } /* { dg-final { scan-assembler-times {\mxxspltiw\M} 2 } } */ -/* { dg-final { scan-assembler-times {\mxxspltidp\M} 2 } } */ +/* { dg-final { scan-assembler-times {\mxxspltidp\M} 3 } } */ /* { dg-final { scan-assembler-times {\mxxsplti32dx\M} 3 } } */