* [PATCH 1/2] Add a new target hook: TARGET_HAVE_CONDITIONAL_MOVE_MEM_NOTRAP
[not found] <20240614005632.4088419-1-lingling.kong@intel.com>
@ 2024-06-14 1:38 ` Kong, Lingling
[not found] ` <20240614005632.4088419-2-lingling.kong@intel.com>
1 sibling, 0 replies; 4+ messages in thread
From: Kong, Lingling @ 2024-06-14 1:38 UTC (permalink / raw)
To: gcc-patches; +Cc: Liu, Hongtao, Kong, Lingling, Uros Bizjak
From: konglin1 <lingling.kong@intel.com>
gcc/ChangeLog:
* doc/tm.texi: Regenerated.
* doc/tm.texi.in: Add TARGET_HAVE_CONDITIONAL_MOVE_MEM_NOTRAP
* target.def (bool,): New hook.
* targhooks.cc (default_have_conditional_move_mem_notrap): New
function to hook TARGET_HAVE_CONDITIONAL_MOVE_MEM_NOTRAP.
* targhooks.h (default_have_conditional_move_mem_notrap): New
target hook declear.
---
gcc/doc/tm.texi | 6 ++++++
gcc/doc/tm.texi.in | 2 ++
gcc/target.def | 11 +++++++++++
gcc/targhooks.cc | 8 ++++++++
gcc/targhooks.h | 1 +
5 files changed, 28 insertions(+)
diff --git a/gcc/doc/tm.texi b/gcc/doc/tm.texi
index 8a7aa70d605..f8faf44ab73 100644
--- a/gcc/doc/tm.texi
+++ b/gcc/doc/tm.texi
@@ -7311,6 +7311,12 @@ candidate as a replacement for the if-convertible sequence described in
@code{if_info}.
@end deftypefn
+@deftypefn {Target Hook} bool TARGET_HAVE_CONDITIONAL_MOVE_MEM_NOTRAP (rtx @var{x})
+This hook returns true if the target supports condition move instructions
+ that enables fault suppression of memory operands when the condition code
+ evaluates to false.
+@end deftypefn
+
@deftypefn {Target Hook} bool TARGET_NEW_ADDRESS_PROFITABLE_P (rtx @var{memref}, rtx_insn * @var{insn}, rtx @var{new_addr})
Return @code{true} if it is profitable to replace the address in
@var{memref} with @var{new_addr}. This allows targets to prevent the
diff --git a/gcc/doc/tm.texi.in b/gcc/doc/tm.texi.in
index 9e0830758ae..17c122aea43 100644
--- a/gcc/doc/tm.texi.in
+++ b/gcc/doc/tm.texi.in
@@ -4748,6 +4748,8 @@ Define this macro if a non-short-circuit operation produced by
@hook TARGET_NOCE_CONVERSION_PROFITABLE_P
+@hook TARGET_HAVE_CONDITIONAL_MOVE_MEM_NOTRAP
+
@hook TARGET_NEW_ADDRESS_PROFITABLE_P
@hook TARGET_NO_SPECULATION_IN_DELAY_SLOTS_P
diff --git a/gcc/target.def b/gcc/target.def
index 70070caebc7..aa77737e006 100644
--- a/gcc/target.def
+++ b/gcc/target.def
@@ -3993,6 +3993,17 @@ candidate as a replacement for the if-convertible sequence described in\n\
bool, (rtx_insn *seq, struct noce_if_info *if_info),
default_noce_conversion_profitable_p)
+/* Return true if the target support condition move instructions that enables
+ fault suppression of memory operands when the condition code evaluates to
+ false. */
+DEFHOOK
+(have_conditional_move_mem_notrap,
+ "This hook returns true if the target supports condition move instructions\n\
+ that enables fault suppression of memory operands when the condition code\n\
+ evaluates to false.",
+bool, (rtx x),
+default_have_conditional_move_mem_notrap)
+
/* Return true if new_addr should be preferred over the existing address used by
memref in insn. */
DEFHOOK
diff --git a/gcc/targhooks.cc b/gcc/targhooks.cc
index fb339bf75dd..a616371b204 100644
--- a/gcc/targhooks.cc
+++ b/gcc/targhooks.cc
@@ -2816,4 +2816,12 @@ default_memtag_untagged_pointer (rtx tagged_pointer, rtx target)
return untagged_base;
}
+/* The default implementation of
+ TARGET_HAVE_CONDITIONAL_MOVE_MEM_NOTRAP. */
+bool
+default_have_conditional_move_mem_notrap (rtx x ATTRIBUTE_UNUSED)
+{
+ return false;
+}
+
#include "gt-targhooks.h"
diff --git a/gcc/targhooks.h b/gcc/targhooks.h
index 85f3817c176..f8ea2fde53d 100644
--- a/gcc/targhooks.h
+++ b/gcc/targhooks.h
@@ -305,5 +305,6 @@ extern rtx default_memtag_add_tag (rtx, poly_int64, uint8_t);
extern rtx default_memtag_set_tag (rtx, rtx, rtx);
extern rtx default_memtag_extract_tag (rtx, rtx);
extern rtx default_memtag_untagged_pointer (rtx, rtx);
+extern bool default_have_conditional_move_mem_notrap (rtx x);
#endif /* GCC_TARGHOOKS_H */
--
2.31.1
^ permalink raw reply [flat|nested] 4+ messages in thread
* [PATCH 2/2] [APX CFCMOV] Support APX CFCMOV
[not found] ` <20240614005632.4088419-2-lingling.kong@intel.com>
@ 2024-06-14 1:38 ` Kong, Lingling
2024-06-14 6:12 ` Richard Biener
0 siblings, 1 reply; 4+ messages in thread
From: Kong, Lingling @ 2024-06-14 1:38 UTC (permalink / raw)
To: gcc-patches; +Cc: Liu, Hongtao, Kong, Lingling, Uros Bizjak
[-- Attachment #1: Type: text/plain, Size: 29668 bytes --]
From: konglin1 <lingling.kong@intel.com<mailto:lingling.kong@intel.com>>
APX CFCMOV feature implements conditionally faulting which means that all
memory faults are suppressed when the condition code evaluates to false and
load or store a memory operand. Now we could load or store a memory operand
may trap or fault for conditional move.
To enable CFCMOV, we add a target HOOK TARGET_HAVE_CONDITIONAL_MOVE_MEM_NOTRAP
in if-conversion pass to allow convert to cmov.
Bootstrapped & regtested on x86-64-pc-linux-gnu with binutils 2.42 branch.
OK for trunk?
gcc/ChangeLog:
* config/i386/i386-expand.cc (ix86_can_cfcmov_p): New function that
test if the cfcmov can be generated.
(ix86_expand_int_movcc): Expand to cfcmov pattern if ix86_can_cfcmov_p
return ture.
* config/i386/i386-opts.h (enum apx_features): Add apx_cfcmov.
* config/i386/i386.cc (ix86_have_conditional_move_mem_notrap): New
function to hook TARGET_HAVE_CONDITIONAL_MOVE_MEM_NOTRAP
(TARGET_HAVE_CONDITIONAL_MOVE_MEM_NOTRAP): Target hook define.
(ix86_rtx_costs): Add UNSPEC_APX_CFCMOV cost;
* config/i386/i386.h (TARGET_APX_CFCMOV): Define.
* config/i386/i386.md (*cfcmov<mode>_1): New define_insn to support
cfcmov.
(*cfcmov<mode>_2): Ditto.
(UNSPEC_APX_CFCMOV): New unspec for cfcmov.
* config/i386/i386.opt: Add enum value for cfcmov.
* ifcvt.cc (noce_try_cmove_load_mem_notrap): Use target hook to allow
convert to cfcmov for conditional load.
(noce_try_cmove_store_mem_notrap): Convert to conditional store.
(noce_process_if_block): Ditto.
gcc/testsuite/ChangeLog:
* gcc.target/i386/apx-cfcmov-1.c: New test.
* gcc.target/i386/apx-cfcmov-2.c: Ditto.
---
gcc/config/i386/i386-expand.cc | 63 +++++
gcc/config/i386/i386-opts.h | 4 +-
gcc/config/i386/i386.cc | 33 ++-
gcc/config/i386/i386.h | 1 +
gcc/config/i386/i386.md | 53 +++-
gcc/config/i386/i386.opt | 3 +
gcc/config/i386/predicates.md | 7 +
gcc/ifcvt.cc | 247 ++++++++++++++++++-
gcc/testsuite/gcc.target/i386/apx-cfcmov-1.c | 73 ++++++
gcc/testsuite/gcc.target/i386/apx-cfcmov-2.c | 40 +++
10 files changed, 511 insertions(+), 13 deletions(-)
create mode 100644 gcc/testsuite/gcc.target/i386/apx-cfcmov-1.c
create mode 100644 gcc/testsuite/gcc.target/i386/apx-cfcmov-2.c
diff --git a/gcc/config/i386/i386-expand.cc b/gcc/config/i386/i386-expand.cc
index 312329e550b..c02a4bcbec3 100644
--- a/gcc/config/i386/i386-expand.cc
+++ b/gcc/config/i386/i386-expand.cc
@@ -3336,6 +3336,30 @@ ix86_expand_int_addcc (rtx operands[])
return true;
}
+/* Return TRUE if we could convert "if (test) x = a; else x = b;" to cfcmov,
+ especially when load a or b or x store may cause memmory faults. */
+bool
+ix86_can_cfcmov_p (rtx x, rtx a, rtx b)
+{
+ machine_mode mode = GET_MODE (x);
+ if (TARGET_APX_CFCMOV
+ && (mode == DImode || mode == SImode || mode == HImode))
+ {
+ /* C load (r m r), (r m C), (r r m). For r m m could use
+ two cfcmov. */
+ if (register_operand (x, mode)
+ && ((MEM_P (a) && register_operand (b, mode))
+ || (MEM_P (a) && b == const0_rtx)
+ || (register_operand (a, mode) && MEM_P (b))
+ || (MEM_P (a) && MEM_P (b))))
+ return true;
+ /* C store (m r 0). */
+ else if (MEM_P (x) && x == b && register_operand (a, mode))
+ return true;
+ }
+ return false;
+}
+
bool
ix86_expand_int_movcc (rtx operands[])
{
@@ -3366,6 +3390,45 @@ ix86_expand_int_movcc (rtx operands[])
compare_code = GET_CODE (compare_op);
+ if (MEM_P (operands[0])
+ && !ix86_can_cfcmov_p (operands[0], op2, op3))
+ return false;
+
+ if (may_trap_or_fault_p (op2) || may_trap_or_fault_p (op3))
+ {
+ if (ix86_can_cfcmov_p (operands[0], op2, op3))
+ {
+ if (may_trap_or_fault_p (op2))
+ op2 = gen_rtx_UNSPEC (mode, gen_rtvec (1, operands[2]),
+ UNSPEC_APX_CFCMOV);
+ if (may_trap_or_fault_p (op3))
+ op3 = gen_rtx_UNSPEC (mode, gen_rtvec (1, operands[3]),
+ UNSPEC_APX_CFCMOV);
+ emit_insn (compare_seq);
+
+ if (may_trap_or_fault_p (op2) && may_trap_or_fault_p (op3))
+ {
+ emit_insn (gen_rtx_SET (operands[0],
+ gen_rtx_IF_THEN_ELSE (mode,
+ compare_op,
+ op2,
+ operands[0])));
+ emit_insn (gen_rtx_SET (operands[0],
+ gen_rtx_IF_THEN_ELSE (mode,
+ compare_op,
+ operands[0],
+ op3)));
+ }
+ else
+ emit_insn (gen_rtx_SET (operands[0],
+ gen_rtx_IF_THEN_ELSE (mode,
+ compare_op,
+ op2, op3)));
+ return true;
+ }
+ return false;
+ }
+
if ((op1 == const0_rtx && (code == GE || code == LT))
|| (op1 == constm1_rtx && (code == GT || code == LE)))
sign_bit_compare_p = true;
diff --git a/gcc/config/i386/i386-opts.h b/gcc/config/i386/i386-opts.h
index c7ec0d9fd39..711519ffb53 100644
--- a/gcc/config/i386/i386-opts.h
+++ b/gcc/config/i386/i386-opts.h
@@ -143,8 +143,10 @@ enum apx_features {
apx_nf = 1 << 4,
apx_ccmp = 1 << 5,
apx_zu = 1 << 6,
+ apx_cfcmov = 1 << 7,
apx_all = apx_egpr | apx_push2pop2 | apx_ndd
- | apx_ppx | apx_nf | apx_ccmp | apx_zu,
+ | apx_ppx | apx_nf | apx_ccmp | apx_zu
+ | apx_cfcmov,
};
#endif
diff --git a/gcc/config/i386/i386.cc b/gcc/config/i386/i386.cc
index 173db213d14..b14c0a3d9f2 100644
--- a/gcc/config/i386/i386.cc
+++ b/gcc/config/i386/i386.cc
@@ -22349,10 +22349,18 @@ ix86_rtx_costs (rtx x, machine_mode mode, int outer_code_i, int opno,
*total = COSTS_N_INSNS (1);
if (!COMPARISON_P (XEXP (x, 0)) && !REG_P (XEXP (x, 0)))
*total += rtx_cost (XEXP (x, 0), mode, code, 0, speed);
- if (!REG_P (XEXP (x, 1)))
- *total += rtx_cost (XEXP (x, 1), mode, code, 1, speed);
- if (!REG_P (XEXP (x, 2)))
- *total += rtx_cost (XEXP (x, 2), mode, code, 2, speed);
+ rtx op1, op2;
+ op1 = XEXP (x, 1);
+ op2 = XEXP (x, 2);
+ /* Handle UNSPEC_APX_CFCMOV for cfcmov. */
+ if (GET_CODE (op1) == UNSPEC && XINT (op1, 1) == UNSPEC_APX_CFCMOV)
+ op1 = XVECEXP (op1, 0, 0);
+ if (GET_CODE (op2) == UNSPEC && XINT (op2, 1) == UNSPEC_APX_CFCMOV)
+ op2 = XVECEXP (op2, 0, 0);
+ if (!REG_P (op1))
+ *total += rtx_cost (op1, mode, code, 1, speed);
+ if (!REG_P (op2))
+ *total += rtx_cost (op2, mode, code, 2, speed);
return true;
}
return false;
@@ -24998,6 +25006,19 @@ ix86_noce_conversion_profitable_p (rtx_insn *seq, struct noce_if_info *if_info)
return default_noce_conversion_profitable_p (seq, if_info);
}
+
+/* Implement targetm.have_conditional_move_mem_notrap hook. */
+static bool
+ix86_have_conditional_move_mem_notrap (rtx x)
+{
+ machine_mode mode = GET_MODE (x);
+ if (TARGET_APX_CFCMOV
+ && (mode == DImode || mode == SImode || mode == HImode)
+ && MEM_P (x))
+ return true;
+ return false;
+}
+
/* x86-specific vector costs. */
class ix86_vector_costs : public vector_costs
{
@@ -26975,6 +26996,10 @@ ix86_libgcc_floating_mode_supported_p
#undef TARGET_NOCE_CONVERSION_PROFITABLE_P
#define TARGET_NOCE_CONVERSION_PROFITABLE_P ix86_noce_conversion_profitable_p
+#undef TARGET_HAVE_CONDITIONAL_MOVE_MEM_NOTRAP
+#define TARGET_HAVE_CONDITIONAL_MOVE_MEM_NOTRAP \
+ ix86_have_conditional_move_mem_notrap
+
#undef TARGET_HARD_REGNO_NREGS
#define TARGET_HARD_REGNO_NREGS ix86_hard_regno_nregs
#undef TARGET_HARD_REGNO_MODE_OK
diff --git a/gcc/config/i386/i386.h b/gcc/config/i386/i386.h
index dc1a1f44320..6a20fa678c8 100644
--- a/gcc/config/i386/i386.h
+++ b/gcc/config/i386/i386.h
@@ -58,6 +58,7 @@ see the files COPYING3 and COPYING.RUNTIME respectively. If not, see
#define TARGET_APX_NF (ix86_apx_features & apx_nf)
#define TARGET_APX_CCMP (ix86_apx_features & apx_ccmp)
#define TARGET_APX_ZU (ix86_apx_features & apx_zu)
+#define TARGET_APX_CFCMOV (ix86_apx_features & apx_cfcmov)
#include "config/vxworks-dummy.h"
diff --git a/gcc/config/i386/i386.md b/gcc/config/i386/i386.md
index fd48e764469..57448c07828 100644
--- a/gcc/config/i386/i386.md
+++ b/gcc/config/i386/i386.md
@@ -221,6 +221,9 @@
;; For APX CCMP support
;; DFV = default flag value
UNSPEC_APX_DFV
+
+ ;; For APX CFCMOV support
+ UNSPEC_APX_CFCMOV
])
(define_c_enum "unspecv" [
@@ -579,7 +582,7 @@
noavx512dq,fma_or_avx512vl,avx512vl,noavx512vl,avxvnni,
avx512vnnivl,avx512fp16,avxifma,avx512ifmavl,avxneconvert,
avx512bf16vl,vpclmulqdqvl,avx_noavx512f,avx_noavx512vl,
- vaes_avx512vl,noapx_nf"
+ vaes_avx512vl,noapx_nf,apx_cfcmov"
(const_string "base"))
;; The (bounding maximum) length of an instruction immediate.
@@ -986,6 +989,7 @@
(eq_attr "mmx_isa" "avx")
(symbol_ref "TARGET_MMX_WITH_SSE && TARGET_AVX")
(eq_attr "isa" "noapx_nf") (symbol_ref "!TARGET_APX_NF")
+ (eq_attr "isa" "apx_cfcmov") (symbol_ref "TARGET_APX_CFCMOV")
]
(const_int 1)))
@@ -24995,7 +24999,7 @@
;; Conditional move instructions.
(define_expand "mov<mode>cc"
- [(set (match_operand:SWIM 0 "register_operand")
+ [(set (match_operand:SWIM 0 "register_or_cfc_mem_operand")
(if_then_else:SWIM (match_operand 1 "comparison_operator")
(match_operand:SWIM 2 "<general_operand>")
(match_operand:SWIM 3 "<general_operand>")))]
@@ -25103,19 +25107,54 @@
(set (match_dup 0)
(neg:SWI (ltu:SWI (reg:CCC FLAGS_REG) (const_int 0))))])
+(define_insn "*cfcmov<mode>_1"
+ [(set (match_operand:SWI248 0 "register_operand" "=r,r")
+ (if_then_else:SWI248 (match_operator 1 "ix86_comparison_operator"
+ [(reg FLAGS_REG) (const_int 0)])
+ (unspec:SWI248
+ [(match_operand:SWI248 2 "memory_operand" "m,m")]
+ UNSPEC_APX_CFCMOV)
+ (match_operand:SWI248 3 "reg_or_0_operand" "C,r")))]
+ "TARGET_CMOVE && TARGET_APX_CFCMOV"
+ "@
+ cfcmov%O2%C1\t{%2, %0|%0, %2}
+ cfcmov%O2%C1\t{%2, %3, %0|%0, %3, %2}"
+ [(set_attr "isa" "*,apx_ndd")
+ (set_attr "type" "icmov")
+ (set_attr "prefix" "evex")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "*cfcmov<mode>_2"
+ [(set (match_operand:SWI248 0 "nonimmediate_operand" "=r,m")
+ (if_then_else:SWI248 (match_operator 1 "ix86_comparison_operator"
+ [(reg FLAGS_REG) (const_int 0)])
+ (match_operand:SWI248 2 "register_operand" "r,r")
+ (unspec:SWI248
+ [(match_operand:SWI248 3 "memory_operand" "m,0")]
+ UNSPEC_APX_CFCMOV)))]
+ "TARGET_CMOVE && TARGET_APX_CFCMOV"
+ "@
+ cfcmov%O2%c1\t{%3, %2, %0|%0, %2, %3}
+ cfcmov%O2%C1\t{%2, %0|%0, %2}"
+ [(set_attr "isa" "apx_ndd,*")
+ (set_attr "type" "icmov")
+ (set_attr "prefix" "evex")
+ (set_attr "mode" "<MODE>")])
+
(define_insn "*mov<mode>cc_noc"
- [(set (match_operand:SWI248 0 "register_operand" "=r,r,r,r")
+ [(set (match_operand:SWI248 0 "register_operand" "=r,r,r,r,r")
(if_then_else:SWI248 (match_operator 1 "ix86_comparison_operator"
[(reg FLAGS_REG) (const_int 0)])
- (match_operand:SWI248 2 "nonimmediate_operand" "rm,0,rm,r")
- (match_operand:SWI248 3 "nonimmediate_operand" "0,rm,r,rm")))]
+ (match_operand:SWI248 2 "nonimmediate_operand" "rm,0,rm,r,r")
+ (match_operand:SWI248 3 "nonimm_or_0_operand" "0,rm,r,rm,C")))]
"TARGET_CMOVE && !(MEM_P (operands[2]) && MEM_P (operands[3]))"
"@
cmov%O2%C1\t{%2, %0|%0, %2}
cmov%O2%c1\t{%3, %0|%0, %3}
cmov%O2%C1\t{%2, %3, %0|%0, %3, %2}
- cmov%O2%c1\t{%3, %2, %0|%0, %2, %3}"
- [(set_attr "isa" "*,*,apx_ndd,apx_ndd")
+ cmov%O2%c1\t{%3, %2, %0|%0, %2, %3}
+ cfcmov%O2%C1\t{%2, %0|%0, %2}"
+ [(set_attr "isa" "*,*,apx_ndd,apx_ndd,apx_cfcmov")
(set_attr "type" "icmov")
(set_attr "mode" "<MODE>")])
diff --git a/gcc/config/i386/i386.opt b/gcc/config/i386/i386.opt
index 353fffb2343..7d63d9abd95 100644
--- a/gcc/config/i386/i386.opt
+++ b/gcc/config/i386/i386.opt
@@ -1345,6 +1345,9 @@ Enum(apx_features) String(ccmp) Value(apx_ccmp) Set(7)
EnumValue
Enum(apx_features) String(zu) Value(apx_zu) Set(8)
+EnumValue
+Enum(apx_features) String(cfcmov) Value(apx_cfcmov) Set(9)
+
EnumValue
Enum(apx_features) String(all) Value(apx_all) Set(1)
diff --git a/gcc/config/i386/predicates.md b/gcc/config/i386/predicates.md
index 7afe3100cb7..d562e10ab41 100644
--- a/gcc/config/i386/predicates.md
+++ b/gcc/config/i386/predicates.md
@@ -2322,3 +2322,10 @@
return true;
})
+
+;; Return true if OP is a register operand or memory_operand is only
+;; supported under TARGET_APX_CFCMOV.
+(define_predicate "register_or_cfc_mem_operand"
+ (ior (match_operand 0 "register_operand")
+ (and (match_code "mem")
+ (match_test "TARGET_APX_CFCMOV"))))
diff --git a/gcc/ifcvt.cc b/gcc/ifcvt.cc
index 58ed42673e5..6e3e48af810 100644
--- a/gcc/ifcvt.cc
+++ b/gcc/ifcvt.cc
@@ -783,6 +783,8 @@ static rtx noce_emit_cmove (struct noce_if_info *, rtx, enum rtx_code, rtx,
rtx, rtx, rtx, rtx = NULL, rtx = NULL);
static bool noce_try_cmove (struct noce_if_info *);
static bool noce_try_cmove_arith (struct noce_if_info *);
+static bool noce_try_cmove_load_mem_notrap (struct noce_if_info *);
+static bool noce_try_cmove_store_mem_notrap (struct noce_if_info *, rtx *, rtx);
static rtx noce_get_alt_condition (struct noce_if_info *, rtx, rtx_insn **);
static bool noce_try_minmax (struct noce_if_info *);
static bool noce_try_abs (struct noce_if_info *);
@@ -2401,6 +2403,237 @@ noce_try_cmove_arith (struct noce_if_info *if_info)
return false;
}
+/* When target support suppress memory fault, try more complex cases involving
+ conditional_move's source or dest may trap or fault. */
+
+static bool
+noce_try_cmove_load_mem_notrap (struct noce_if_info *if_info)
+{
+ rtx a = if_info->a;
+ rtx b = if_info->b;
+ rtx x = if_info->x;
+
+ if (MEM_P (x))
+ return false;
+ /* Just handle a conditional move from one trap MEM + other non_trap,
+ non mem cases. */
+ if (!(MEM_P (a) ^ MEM_P (b)))
+ return false;
+ bool a_trap = may_trap_or_fault_p (a);
+ bool b_trap = may_trap_or_fault_p (b);
+
+ if (!(a_trap ^ b_trap))
+ return false;
+ if (a_trap && (!MEM_P (a) || !targetm.have_conditional_move_mem_notrap (a)))
+ return false;
+ if (b_trap && (!MEM_P (b) || !targetm.have_conditional_move_mem_notrap (b)))
+ return false;
+
+ rtx orig_b;
+ rtx_insn *insn_a, *insn_b;
+ bool a_simple = if_info->then_simple;
+ bool b_simple = if_info->else_simple;
+ basic_block then_bb = if_info->then_bb;
+ basic_block else_bb = if_info->else_bb;
+ rtx target;
+ enum rtx_code code;
+ rtx cond = if_info->cond;
+ rtx_insn *ifcvt_seq;
+
+ /* if (test) x = *a; else x = c - d;
+ => x = c - d;
+ if (test)
+ x = *a;
+ */
+
+ code = GET_CODE (cond);
+ insn_a = if_info->insn_a;
+ insn_b = if_info->insn_b;
+
+ machine_mode x_mode = GET_MODE (x);
+
+ if (!can_conditionally_move_p (x_mode))
+ return false;
+
+ /* Because we only handle one trap MEM + other non_trap, non mem cases,
+ just move one trap MEM always in then_bb. */
+ if (noce_reversed_cond_code (if_info) != UNKNOWN)
+ {
+ bool reversep = false;
+ if (b_trap)
+ reversep = true;
+
+ if (reversep)
+ {
+ if (if_info->rev_cond)
+ {
+ cond = if_info->rev_cond;
+ code = GET_CODE (cond);
+ }
+ else
+ code = reversed_comparison_code (cond, if_info->jump);
+ std::swap (a, b);
+ std::swap (insn_a, insn_b);
+ std::swap (a_simple, b_simple);
+ std::swap (then_bb, else_bb);
+ }
+ }
+
+ if (then_bb && else_bb
+ && (!bbs_ok_for_cmove_arith (then_bb, else_bb, if_info->orig_x)
+ || !bbs_ok_for_cmove_arith (else_bb, then_bb, if_info->orig_x)))
+ return false;
+
+ start_sequence ();
+
+ /* If one of the blocks is empty then the corresponding B or A value
+ came from the test block. The non-empty complex block that we will
+ emit might clobber the register used by B or A, so move it to a pseudo
+ first. */
+
+ rtx tmp_b = NULL_RTX;
+
+ /* Don't move trap mem to a pseudo. */
+ if (!may_trap_or_fault_p (b) && (b_simple || !else_bb))
+ tmp_b = gen_reg_rtx (x_mode);
+
+ orig_b = b;
+
+ rtx emit_a = NULL_RTX;
+ rtx emit_b = NULL_RTX;
+ rtx_insn *tmp_insn = NULL;
+ bool modified_in_a = false;
+ bool modified_in_b = false;
+ /* If either operand is complex, load it into a register first.
+ The best way to do this is to copy the original insn. In this
+ way we preserve any clobbers etc that the insn may have had.
+ This is of course not possible in the IS_MEM case. */
+
+ if (! general_operand (b, GET_MODE (b)) || tmp_b)
+ {
+ if (insn_b)
+ {
+ b = tmp_b ? tmp_b : gen_reg_rtx (GET_MODE (b));
+ rtx_insn *copy_of_b = as_a <rtx_insn *> (copy_rtx (insn_b));
+ rtx set = single_set (copy_of_b);
+
+ SET_DEST (set) = b;
+ emit_b = PATTERN (copy_of_b);
+ }
+ else
+ {
+ rtx tmp_reg = tmp_b ? tmp_b : gen_reg_rtx (GET_MODE (b));
+ emit_b = gen_rtx_SET (tmp_reg, b);
+ b = tmp_reg;
+ }
+ }
+
+ if (tmp_b && then_bb)
+ {
+ FOR_BB_INSNS (then_bb, tmp_insn)
+ /* Don't check inside insn_a. We will have changed it to emit_a
+ with a destination that doesn't conflict. */
+ if (!(insn_a && tmp_insn == insn_a)
+ && modified_in_p (orig_b, tmp_insn))
+ {
+ modified_in_a = true;
+ break;
+ }
+
+ }
+
+ modified_in_b = emit_b != NULL_RTX && modified_in_p (a, emit_b);
+ /* If insn to set up A clobbers any registers B depends on, try to
+ swap insn that sets up A with the one that sets up B. If even
+ that doesn't help, punt. */
+ if (modified_in_a && !modified_in_b)
+ {
+ if (!noce_emit_bb (emit_b, else_bb, b_simple))
+ goto end_seq_and_fail;
+
+ if (!noce_emit_bb (emit_a, then_bb, a_simple))
+ goto end_seq_and_fail;
+ }
+ else if (!modified_in_a)
+ {
+ if (!noce_emit_bb (emit_b, else_bb, b_simple))
+ goto end_seq_and_fail;
+
+ if (!noce_emit_bb (emit_a, then_bb, a_simple))
+ goto end_seq_and_fail;
+ }
+ else
+ goto end_seq_and_fail;
+
+ target = noce_emit_cmove (if_info, x, code, XEXP (cond, 0), XEXP (cond, 1),
+ a, b);
+
+ if (! target)
+ goto end_seq_and_fail;
+
+ if (target != x)
+ noce_emit_move_insn (x, target);
+
+ ifcvt_seq = end_ifcvt_sequence (if_info);
+ if (!ifcvt_seq || !targetm.noce_conversion_profitable_p (ifcvt_seq, if_info))
+ return false;
+
+ emit_insn_before_setloc (ifcvt_seq, if_info->jump,
+ INSN_LOCATION (if_info->insn_a));
+ if_info->transform_name = "noce_try_cmove_load_mem_notrap";
+ return true;
+
+ end_seq_and_fail:
+ end_sequence ();
+ return false;
+}
+
+static bool
+noce_try_cmove_store_mem_notrap (struct noce_if_info *if_info, rtx *x_ptr, rtx orig_x)
+{
+ rtx a = if_info->a;
+ rtx b = if_info->b;
+ rtx x = orig_x;
+ machine_mode x_mode = GET_MODE (x);
+
+ if (!MEM_P (x) || !rtx_equal_p (x, b))
+ return false;
+ if (!may_trap_or_fault_p (x) || !targetm.have_conditional_move_mem_notrap (x))
+ return false;
+ if (!if_info->then_simple || !register_operand (a, x_mode))
+ return false;
+
+ rtx cond = if_info->cond;
+ enum rtx_code code = GET_CODE (cond);
+ rtx_insn *ifcvt_seq;
+
+ start_sequence ();
+
+ rtx target = noce_emit_cmove (if_info, x, code, XEXP (cond, 0), XEXP (cond, 1),
+ a, b);
+
+ if (! target)
+ goto end_seq_and_fail;
+
+ if (target != x)
+ noce_emit_move_insn (x, target);
+
+ ifcvt_seq = end_ifcvt_sequence (if_info);
+ if (!ifcvt_seq || !targetm.noce_conversion_profitable_p (ifcvt_seq, if_info))
+ return false;
+
+ emit_insn_before_setloc (ifcvt_seq, if_info->jump,
+ INSN_LOCATION (if_info->insn_a));
+ if_info->transform_name = "noce_try_cmove_load_mem_notrap";
+ if_info->x = orig_x;
+ *x_ptr = orig_x;
+ return true;
+
+ end_seq_and_fail:
+ end_sequence ();
+ return false;
+}
+
/* For most cases, the simplified condition we found is the best
choice, but this is not the case for the min/max/abs transforms.
For these we wish to know that it is A or B in the condition. */
@@ -4121,12 +4354,21 @@ noce_process_if_block (struct noce_if_info *if_info)
}
if (!set_b && MEM_P (orig_x))
+ {
+ /* Conditional_move_suppress_fault for condition mem store would not
+ move any arithmetic calculations. */
+ if (targetm.have_conditional_move_mem_notrap (orig_x)
+ && HAVE_conditional_move
+ && noce_try_cmove_store_mem_notrap (if_info, &x, orig_x))
+ goto success;
+ else
/* We want to avoid store speculation to avoid cases like
if (pthread_mutex_trylock(mutex))
++global_variable;
Rather than go to much effort here, we rely on the SSA optimizers,
which do a good enough job these days. */
- return false;
+ return false;
+ }
if (noce_try_move (if_info))
goto success;
@@ -4160,6 +4402,9 @@ noce_process_if_block (struct noce_if_info *if_info)
if (HAVE_conditional_move
&& noce_try_cmove_arith (if_info))
goto success;
+ if (HAVE_conditional_move
+ && noce_try_cmove_load_mem_notrap (if_info))
+ goto success;
if (noce_try_sign_mask (if_info))
goto success;
}
diff --git a/gcc/testsuite/gcc.target/i386/apx-cfcmov-1.c b/gcc/testsuite/gcc.target/i386/apx-cfcmov-1.c
new file mode 100644
index 00000000000..4a1fb91b24c
--- /dev/null
+++ b/gcc/testsuite/gcc.target/i386/apx-cfcmov-1.c
@@ -0,0 +1,73 @@
+/* { dg-do compile { target { ! ia32 } } } */
+/* { dg-options "-O3 -mapxf" } */
+
+/* { dg-final { scan-assembler-times "cfcmovne" 1 } } */
+/* { dg-final { scan-assembler-times "cfcmovg" 2} } */
+/* { dg-final { scan-assembler-times "cfcmove" 1 } } */
+/* { dg-final { scan-assembler-times "cfcmovl" 2 } } */
+/* { dg-final { scan-assembler-times "cfcmovle" 1 } } */
+
+__attribute__((noinline, noclone, target("apxf")))
+int cfc_store (int a, int b, int c, int d, int *arr)
+{
+ if (a != b)
+ *arr = c;
+ return d;
+
+}
+
+__attribute__((noinline, noclone, target("apxf")))
+int cfc_load_ndd (int a, int b, int c, int *p)
+{
+ if (a > b)
+ return *p;
+ return c;
+}
+
+__attribute__((noinline, noclone, target("apxf")))
+int cfc_load_2_trap (int a, int b, int *c, int *p)
+{
+ if (a > b)
+ return *p;
+ return *c;
+}
+
+__attribute__((noinline, noclone, target("apxf")))
+int cfc_load_zero (int a, int b, int c)
+{
+ int sum = 0;
+ if (a == b)
+ return c;
+ return sum;
+}
+
+__attribute__((noinline, noclone, target("apxf")))
+int cfc_load_mem (int a, int b, int *p)
+{
+ int sum = 0;
+ if (a < b )
+ sum = *p;
+ return sum;
+}
+
+__attribute__((noinline, noclone, target("apxf")))
+int cfc_load_arith_1 (int a, int b, int c, int *p)
+{
+ int sum = 0;
+ if (a > b)
+ sum = *p;
+ else
+ sum = a + c;
+ return sum + 1;
+}
+
+__attribute__((noinline, noclone, target("apxf")))
+int cfc_load_arith_2 (int a, int b, int c, int *p)
+{
+ int sum = 0;
+ if (a > b)
+ sum = a + c;
+ else
+ sum = *p;
+ return sum + 1;
+}
diff --git a/gcc/testsuite/gcc.target/i386/apx-cfcmov-2.c b/gcc/testsuite/gcc.target/i386/apx-cfcmov-2.c
new file mode 100644
index 00000000000..2b1660f64fa
--- /dev/null
+++ b/gcc/testsuite/gcc.target/i386/apx-cfcmov-2.c
@@ -0,0 +1,40 @@
+/* { dg-do run { target { ! ia32 } } } */
+/* { dg-require-effective-target apxf } */
+/* { dg-options "-mapxf -march=x86-64 -O3" } */
+
+#include "apx-cfcmov-1.c"
+
+extern void abort (void);
+
+int main ()
+{
+ if (!__builtin_cpu_supports ("apxf"))
+ return 0;
+
+ int arr = 6;
+ int arr1 = 5;
+ int res = cfc_store (1, 2, 3, 4, &arr);
+ if (arr != 3 && res != 4)
+ abort ();
+ res = cfc_load_ndd (2, 1, 2, &arr);
+ if (res != 3)
+ abort ();
+ res = cfc_load_2_trap (1, 2, &arr1, &arr);
+ if (res != 5)
+ abort ();
+ res = cfc_load_zero (1, 2, 3);
+ res = cfc_load_zero (1, 2, 3);
+ if (res != 0)
+ abort ();
+ res = cfc_load_mem (2, 1, &arr);
+ if (res != 0)
+ abort ();
+ res = cfc_load_arith_1 (1, 2, 3, &arr);
+ if (res != 5)
+ abort();
+ res = cfc_load_arith_2 (2, 1, 3,&arr);
+ if (res != 6)
+ abort();
+ return 0;
+}
+
--
2.31.1
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH 2/2] [APX CFCMOV] Support APX CFCMOV
2024-06-14 1:38 ` [PATCH 2/2] [APX CFCMOV] Support APX CFCMOV Kong, Lingling
@ 2024-06-14 6:12 ` Richard Biener
2024-06-14 6:19 ` Liu, Hongtao
0 siblings, 1 reply; 4+ messages in thread
From: Richard Biener @ 2024-06-14 6:12 UTC (permalink / raw)
To: Kong, Lingling, Richard Sandiford; +Cc: gcc-patches, Liu, Hongtao, Uros Bizjak
On Fri, Jun 14, 2024 at 3:39 AM Kong, Lingling <lingling.kong@intel.com> wrote:
>
> From: konglin1 <lingling.kong@intel.com>
>
>
>
> APX CFCMOV feature implements conditionally faulting which means that all
>
> memory faults are suppressed when the condition code evaluates to false and
>
> load or store a memory operand. Now we could load or store a memory operand
>
> may trap or fault for conditional move.
>
>
>
> To enable CFCMOV, we add a target HOOK TARGET_HAVE_CONDITIONAL_MOVE_MEM_NOTRAP
>
> in if-conversion pass to allow convert to cmov.
>
>
>
> Bootstrapped & regtested on x86-64-pc-linux-gnu with binutils 2.42 branch.
>
> OK for trunk?
How does if-conversion end up modifying the IL?
I have the gut feeling that your hook changes semantics of RTL and you
should instead have an optab for a "masked" load/store?
Richard - do you already have plans how to represent the first-fault loads?
(are there first-fault stores?)
Richard.
>
>
> gcc/ChangeLog:
>
>
>
> * config/i386/i386-expand.cc (ix86_can_cfcmov_p): New function that
>
> test if the cfcmov can be generated.
>
> (ix86_expand_int_movcc): Expand to cfcmov pattern if ix86_can_cfcmov_p
>
> return ture.
>
> * config/i386/i386-opts.h (enum apx_features): Add apx_cfcmov.
>
> * config/i386/i386.cc (ix86_have_conditional_move_mem_notrap): New
>
> function to hook TARGET_HAVE_CONDITIONAL_MOVE_MEM_NOTRAP
>
> (TARGET_HAVE_CONDITIONAL_MOVE_MEM_NOTRAP): Target hook define.
>
> (ix86_rtx_costs): Add UNSPEC_APX_CFCMOV cost;
>
> * config/i386/i386.h (TARGET_APX_CFCMOV): Define.
>
> * config/i386/i386.md (*cfcmov<mode>_1): New define_insn to support
>
> cfcmov.
>
> (*cfcmov<mode>_2): Ditto.
>
> (UNSPEC_APX_CFCMOV): New unspec for cfcmov.
>
> * config/i386/i386.opt: Add enum value for cfcmov.
>
> * ifcvt.cc (noce_try_cmove_load_mem_notrap): Use target hook to allow
>
> convert to cfcmov for conditional load.
>
> (noce_try_cmove_store_mem_notrap): Convert to conditional store.
>
> (noce_process_if_block): Ditto.
>
>
>
> gcc/testsuite/ChangeLog:
>
>
>
> * gcc.target/i386/apx-cfcmov-1.c: New test.
>
> * gcc.target/i386/apx-cfcmov-2.c: Ditto.
>
> ---
>
> gcc/config/i386/i386-expand.cc | 63 +++++
>
> gcc/config/i386/i386-opts.h | 4 +-
>
> gcc/config/i386/i386.cc | 33 ++-
>
> gcc/config/i386/i386.h | 1 +
>
> gcc/config/i386/i386.md | 53 +++-
>
> gcc/config/i386/i386.opt | 3 +
>
> gcc/config/i386/predicates.md | 7 +
>
> gcc/ifcvt.cc | 247 ++++++++++++++++++-
>
> gcc/testsuite/gcc.target/i386/apx-cfcmov-1.c | 73 ++++++
>
> gcc/testsuite/gcc.target/i386/apx-cfcmov-2.c | 40 +++
>
> 10 files changed, 511 insertions(+), 13 deletions(-)
>
> create mode 100644 gcc/testsuite/gcc.target/i386/apx-cfcmov-1.c
>
> create mode 100644 gcc/testsuite/gcc.target/i386/apx-cfcmov-2.c
>
>
>
> diff --git a/gcc/config/i386/i386-expand.cc b/gcc/config/i386/i386-expand.cc
>
> index 312329e550b..c02a4bcbec3 100644
>
> --- a/gcc/config/i386/i386-expand.cc
>
> +++ b/gcc/config/i386/i386-expand.cc
>
> @@ -3336,6 +3336,30 @@ ix86_expand_int_addcc (rtx operands[])
>
> return true;
>
> }
>
>
>
> +/* Return TRUE if we could convert "if (test) x = a; else x = b;" to cfcmov,
>
> + especially when load a or b or x store may cause memmory faults. */
>
> +bool
>
> +ix86_can_cfcmov_p (rtx x, rtx a, rtx b)
>
> +{
>
> + machine_mode mode = GET_MODE (x);
>
> + if (TARGET_APX_CFCMOV
>
> + && (mode == DImode || mode == SImode || mode == HImode))
>
> + {
>
> + /* C load (r m r), (r m C), (r r m). For r m m could use
>
> + two cfcmov. */
>
> + if (register_operand (x, mode)
>
> + && ((MEM_P (a) && register_operand (b, mode))
>
> + || (MEM_P (a) && b == const0_rtx)
>
> + || (register_operand (a, mode) && MEM_P (b))
>
> + || (MEM_P (a) && MEM_P (b))))
>
> + return true;
>
> + /* C store (m r 0). */
>
> + else if (MEM_P (x) && x == b && register_operand (a, mode))
>
> + return true;
>
> + }
>
> + return false;
>
> +}
>
> +
>
> bool
>
> ix86_expand_int_movcc (rtx operands[])
>
> {
>
> @@ -3366,6 +3390,45 @@ ix86_expand_int_movcc (rtx operands[])
>
>
>
> compare_code = GET_CODE (compare_op);
>
>
>
> + if (MEM_P (operands[0])
>
> + && !ix86_can_cfcmov_p (operands[0], op2, op3))
>
> + return false;
>
> +
>
> + if (may_trap_or_fault_p (op2) || may_trap_or_fault_p (op3))
>
> + {
>
> + if (ix86_can_cfcmov_p (operands[0], op2, op3))
>
> + {
>
> + if (may_trap_or_fault_p (op2))
>
> + op2 = gen_rtx_UNSPEC (mode, gen_rtvec (1, operands[2]),
>
> + UNSPEC_APX_CFCMOV);
>
> + if (may_trap_or_fault_p (op3))
>
> + op3 = gen_rtx_UNSPEC (mode, gen_rtvec (1, operands[3]),
>
> + UNSPEC_APX_CFCMOV);
>
> + emit_insn (compare_seq);
>
> +
>
> + if (may_trap_or_fault_p (op2) && may_trap_or_fault_p (op3))
>
> + {
>
> + emit_insn (gen_rtx_SET (operands[0],
>
> + gen_rtx_IF_THEN_ELSE (mode,
>
> + compare_op,
>
> + op2,
>
> + operands[0])));
>
> + emit_insn (gen_rtx_SET (operands[0],
>
> + gen_rtx_IF_THEN_ELSE (mode,
>
> + compare_op,
>
> + operands[0],
>
> + op3)));
>
> + }
>
> + else
>
> + emit_insn (gen_rtx_SET (operands[0],
>
> + gen_rtx_IF_THEN_ELSE (mode,
>
> + compare_op,
>
> + op2, op3)));
>
> + return true;
>
> + }
>
> + return false;
>
> + }
>
> +
>
> if ((op1 == const0_rtx && (code == GE || code == LT))
>
> || (op1 == constm1_rtx && (code == GT || code == LE)))
>
> sign_bit_compare_p = true;
>
> diff --git a/gcc/config/i386/i386-opts.h b/gcc/config/i386/i386-opts.h
>
> index c7ec0d9fd39..711519ffb53 100644
>
> --- a/gcc/config/i386/i386-opts.h
>
> +++ b/gcc/config/i386/i386-opts.h
>
> @@ -143,8 +143,10 @@ enum apx_features {
>
> apx_nf = 1 << 4,
>
> apx_ccmp = 1 << 5,
>
> apx_zu = 1 << 6,
>
> + apx_cfcmov = 1 << 7,
>
> apx_all = apx_egpr | apx_push2pop2 | apx_ndd
>
> - | apx_ppx | apx_nf | apx_ccmp | apx_zu,
>
> + | apx_ppx | apx_nf | apx_ccmp | apx_zu
>
> + | apx_cfcmov,
>
> };
>
>
>
> #endif
>
> diff --git a/gcc/config/i386/i386.cc b/gcc/config/i386/i386.cc
>
> index 173db213d14..b14c0a3d9f2 100644
>
> --- a/gcc/config/i386/i386.cc
>
> +++ b/gcc/config/i386/i386.cc
>
> @@ -22349,10 +22349,18 @@ ix86_rtx_costs (rtx x, machine_mode mode, int outer_code_i, int opno,
>
> *total = COSTS_N_INSNS (1);
>
> if (!COMPARISON_P (XEXP (x, 0)) && !REG_P (XEXP (x, 0)))
>
> *total += rtx_cost (XEXP (x, 0), mode, code, 0, speed);
>
> - if (!REG_P (XEXP (x, 1)))
>
> - *total += rtx_cost (XEXP (x, 1), mode, code, 1, speed);
>
> - if (!REG_P (XEXP (x, 2)))
>
> - *total += rtx_cost (XEXP (x, 2), mode, code, 2, speed);
>
> + rtx op1, op2;
>
> + op1 = XEXP (x, 1);
>
> + op2 = XEXP (x, 2);
>
> + /* Handle UNSPEC_APX_CFCMOV for cfcmov. */
>
> + if (GET_CODE (op1) == UNSPEC && XINT (op1, 1) == UNSPEC_APX_CFCMOV)
>
> + op1 = XVECEXP (op1, 0, 0);
>
> + if (GET_CODE (op2) == UNSPEC && XINT (op2, 1) == UNSPEC_APX_CFCMOV)
>
> + op2 = XVECEXP (op2, 0, 0);
>
> + if (!REG_P (op1))
>
> + *total += rtx_cost (op1, mode, code, 1, speed);
>
> + if (!REG_P (op2))
>
> + *total += rtx_cost (op2, mode, code, 2, speed);
>
> return true;
>
> }
>
> return false;
>
> @@ -24998,6 +25006,19 @@ ix86_noce_conversion_profitable_p (rtx_insn *seq, struct noce_if_info *if_info)
>
> return default_noce_conversion_profitable_p (seq, if_info);
>
> }
>
>
>
> +
>
> +/* Implement targetm.have_conditional_move_mem_notrap hook. */
>
> +static bool
>
> +ix86_have_conditional_move_mem_notrap (rtx x)
>
> +{
>
> + machine_mode mode = GET_MODE (x);
>
> + if (TARGET_APX_CFCMOV
>
> + && (mode == DImode || mode == SImode || mode == HImode)
>
> + && MEM_P (x))
>
> + return true;
>
> + return false;
>
> +}
>
> +
>
> /* x86-specific vector costs. */
>
> class ix86_vector_costs : public vector_costs
>
> {
>
> @@ -26975,6 +26996,10 @@ ix86_libgcc_floating_mode_supported_p
>
> #undef TARGET_NOCE_CONVERSION_PROFITABLE_P
>
> #define TARGET_NOCE_CONVERSION_PROFITABLE_P ix86_noce_conversion_profitable_p
>
>
>
> +#undef TARGET_HAVE_CONDITIONAL_MOVE_MEM_NOTRAP
>
> +#define TARGET_HAVE_CONDITIONAL_MOVE_MEM_NOTRAP \
>
> + ix86_have_conditional_move_mem_notrap
>
> +
>
> #undef TARGET_HARD_REGNO_NREGS
>
> #define TARGET_HARD_REGNO_NREGS ix86_hard_regno_nregs
>
> #undef TARGET_HARD_REGNO_MODE_OK
>
> diff --git a/gcc/config/i386/i386.h b/gcc/config/i386/i386.h
>
> index dc1a1f44320..6a20fa678c8 100644
>
> --- a/gcc/config/i386/i386.h
>
> +++ b/gcc/config/i386/i386.h
>
> @@ -58,6 +58,7 @@ see the files COPYING3 and COPYING.RUNTIME respectively. If not, see
>
> #define TARGET_APX_NF (ix86_apx_features & apx_nf)
>
> #define TARGET_APX_CCMP (ix86_apx_features & apx_ccmp)
>
> #define TARGET_APX_ZU (ix86_apx_features & apx_zu)
>
> +#define TARGET_APX_CFCMOV (ix86_apx_features & apx_cfcmov)
>
>
>
> #include "config/vxworks-dummy.h"
>
>
>
> diff --git a/gcc/config/i386/i386.md b/gcc/config/i386/i386.md
>
> index fd48e764469..57448c07828 100644
>
> --- a/gcc/config/i386/i386.md
>
> +++ b/gcc/config/i386/i386.md
>
> @@ -221,6 +221,9 @@
>
> ;; For APX CCMP support
>
> ;; DFV = default flag value
>
> UNSPEC_APX_DFV
>
> +
>
> + ;; For APX CFCMOV support
>
> + UNSPEC_APX_CFCMOV
>
> ])
>
>
>
> (define_c_enum "unspecv" [
>
> @@ -579,7 +582,7 @@
>
> noavx512dq,fma_or_avx512vl,avx512vl,noavx512vl,avxvnni,
>
> avx512vnnivl,avx512fp16,avxifma,avx512ifmavl,avxneconvert,
>
> avx512bf16vl,vpclmulqdqvl,avx_noavx512f,avx_noavx512vl,
>
> - vaes_avx512vl,noapx_nf"
>
> + vaes_avx512vl,noapx_nf,apx_cfcmov"
>
> (const_string "base"))
>
>
>
> ;; The (bounding maximum) length of an instruction immediate.
>
> @@ -986,6 +989,7 @@
>
> (eq_attr "mmx_isa" "avx")
>
> (symbol_ref "TARGET_MMX_WITH_SSE && TARGET_AVX")
>
> (eq_attr "isa" "noapx_nf") (symbol_ref "!TARGET_APX_NF")
>
> + (eq_attr "isa" "apx_cfcmov") (symbol_ref "TARGET_APX_CFCMOV")
>
> ]
>
> (const_int 1)))
>
>
>
> @@ -24995,7 +24999,7 @@
>
> ;; Conditional move instructions.
>
>
>
> (define_expand "mov<mode>cc"
>
> - [(set (match_operand:SWIM 0 "register_operand")
>
> + [(set (match_operand:SWIM 0 "register_or_cfc_mem_operand")
>
> (if_then_else:SWIM (match_operand 1 "comparison_operator")
>
> (match_operand:SWIM 2 "<general_operand>")
>
> (match_operand:SWIM 3 "<general_operand>")))]
>
> @@ -25103,19 +25107,54 @@
>
> (set (match_dup 0)
>
> (neg:SWI (ltu:SWI (reg:CCC FLAGS_REG) (const_int 0))))])
>
>
>
> +(define_insn "*cfcmov<mode>_1"
>
> + [(set (match_operand:SWI248 0 "register_operand" "=r,r")
>
> + (if_then_else:SWI248 (match_operator 1 "ix86_comparison_operator"
>
> + [(reg FLAGS_REG) (const_int 0)])
>
> + (unspec:SWI248
>
> + [(match_operand:SWI248 2 "memory_operand" "m,m")]
>
> + UNSPEC_APX_CFCMOV)
>
> + (match_operand:SWI248 3 "reg_or_0_operand" "C,r")))]
>
> + "TARGET_CMOVE && TARGET_APX_CFCMOV"
>
> + "@
>
> + cfcmov%O2%C1\t{%2, %0|%0, %2}
>
> + cfcmov%O2%C1\t{%2, %3, %0|%0, %3, %2}"
>
> + [(set_attr "isa" "*,apx_ndd")
>
> + (set_attr "type" "icmov")
>
> + (set_attr "prefix" "evex")
>
> + (set_attr "mode" "<MODE>")])
>
> +
>
> +(define_insn "*cfcmov<mode>_2"
>
> + [(set (match_operand:SWI248 0 "nonimmediate_operand" "=r,m")
>
> + (if_then_else:SWI248 (match_operator 1 "ix86_comparison_operator"
>
> + [(reg FLAGS_REG) (const_int 0)])
>
> + (match_operand:SWI248 2 "register_operand" "r,r")
>
> + (unspec:SWI248
>
> + [(match_operand:SWI248 3 "memory_operand" "m,0")]
>
> + UNSPEC_APX_CFCMOV)))]
>
> + "TARGET_CMOVE && TARGET_APX_CFCMOV"
>
> + "@
>
> + cfcmov%O2%c1\t{%3, %2, %0|%0, %2, %3}
>
> + cfcmov%O2%C1\t{%2, %0|%0, %2}"
>
> + [(set_attr "isa" "apx_ndd,*")
>
> + (set_attr "type" "icmov")
>
> + (set_attr "prefix" "evex")
>
> + (set_attr "mode" "<MODE>")])
>
> +
>
> (define_insn "*mov<mode>cc_noc"
>
> - [(set (match_operand:SWI248 0 "register_operand" "=r,r,r,r")
>
> + [(set (match_operand:SWI248 0 "register_operand" "=r,r,r,r,r")
>
> (if_then_else:SWI248 (match_operator 1 "ix86_comparison_operator"
>
> [(reg FLAGS_REG) (const_int 0)])
>
> - (match_operand:SWI248 2 "nonimmediate_operand" "rm,0,rm,r")
>
> - (match_operand:SWI248 3 "nonimmediate_operand" "0,rm,r,rm")))]
>
> + (match_operand:SWI248 2 "nonimmediate_operand" "rm,0,rm,r,r")
>
> + (match_operand:SWI248 3 "nonimm_or_0_operand" "0,rm,r,rm,C")))]
>
> "TARGET_CMOVE && !(MEM_P (operands[2]) && MEM_P (operands[3]))"
>
> "@
>
> cmov%O2%C1\t{%2, %0|%0, %2}
>
> cmov%O2%c1\t{%3, %0|%0, %3}
>
> cmov%O2%C1\t{%2, %3, %0|%0, %3, %2}
>
> - cmov%O2%c1\t{%3, %2, %0|%0, %2, %3}"
>
> - [(set_attr "isa" "*,*,apx_ndd,apx_ndd")
>
> + cmov%O2%c1\t{%3, %2, %0|%0, %2, %3}
>
> + cfcmov%O2%C1\t{%2, %0|%0, %2}"
>
> + [(set_attr "isa" "*,*,apx_ndd,apx_ndd,apx_cfcmov")
>
> (set_attr "type" "icmov")
>
> (set_attr "mode" "<MODE>")])
>
>
>
> diff --git a/gcc/config/i386/i386.opt b/gcc/config/i386/i386.opt
>
> index 353fffb2343..7d63d9abd95 100644
>
> --- a/gcc/config/i386/i386.opt
>
> +++ b/gcc/config/i386/i386.opt
>
> @@ -1345,6 +1345,9 @@ Enum(apx_features) String(ccmp) Value(apx_ccmp) Set(7)
>
> EnumValue
>
> Enum(apx_features) String(zu) Value(apx_zu) Set(8)
>
>
>
> +EnumValue
>
> +Enum(apx_features) String(cfcmov) Value(apx_cfcmov) Set(9)
>
> +
>
> EnumValue
>
> Enum(apx_features) String(all) Value(apx_all) Set(1)
>
>
>
> diff --git a/gcc/config/i386/predicates.md b/gcc/config/i386/predicates.md
>
> index 7afe3100cb7..d562e10ab41 100644
>
> --- a/gcc/config/i386/predicates.md
>
> +++ b/gcc/config/i386/predicates.md
>
> @@ -2322,3 +2322,10 @@
>
>
>
> return true;
>
> })
>
> +
>
> +;; Return true if OP is a register operand or memory_operand is only
>
> +;; supported under TARGET_APX_CFCMOV.
>
> +(define_predicate "register_or_cfc_mem_operand"
>
> + (ior (match_operand 0 "register_operand")
>
> + (and (match_code "mem")
>
> + (match_test "TARGET_APX_CFCMOV"))))
>
> diff --git a/gcc/ifcvt.cc b/gcc/ifcvt.cc
>
> index 58ed42673e5..6e3e48af810 100644
>
> --- a/gcc/ifcvt.cc
>
> +++ b/gcc/ifcvt.cc
>
> @@ -783,6 +783,8 @@ static rtx noce_emit_cmove (struct noce_if_info *, rtx, enum rtx_code, rtx,
>
> rtx, rtx, rtx, rtx = NULL, rtx = NULL);
>
> static bool noce_try_cmove (struct noce_if_info *);
>
> static bool noce_try_cmove_arith (struct noce_if_info *);
>
> +static bool noce_try_cmove_load_mem_notrap (struct noce_if_info *);
>
> +static bool noce_try_cmove_store_mem_notrap (struct noce_if_info *, rtx *, rtx);
>
> static rtx noce_get_alt_condition (struct noce_if_info *, rtx, rtx_insn **);
>
> static bool noce_try_minmax (struct noce_if_info *);
>
> static bool noce_try_abs (struct noce_if_info *);
>
> @@ -2401,6 +2403,237 @@ noce_try_cmove_arith (struct noce_if_info *if_info)
>
> return false;
>
> }
>
>
>
> +/* When target support suppress memory fault, try more complex cases involving
>
> + conditional_move's source or dest may trap or fault. */
>
> +
>
> +static bool
>
> +noce_try_cmove_load_mem_notrap (struct noce_if_info *if_info)
>
> +{
>
> + rtx a = if_info->a;
>
> + rtx b = if_info->b;
>
> + rtx x = if_info->x;
>
> +
>
> + if (MEM_P (x))
>
> + return false;
>
> + /* Just handle a conditional move from one trap MEM + other non_trap,
>
> + non mem cases. */
>
> + if (!(MEM_P (a) ^ MEM_P (b)))
>
> + return false;
>
> + bool a_trap = may_trap_or_fault_p (a);
>
> + bool b_trap = may_trap_or_fault_p (b);
>
> +
>
> + if (!(a_trap ^ b_trap))
>
> + return false;
>
> + if (a_trap && (!MEM_P (a) || !targetm.have_conditional_move_mem_notrap (a)))
>
> + return false;
>
> + if (b_trap && (!MEM_P (b) || !targetm.have_conditional_move_mem_notrap (b)))
>
> + return false;
>
> +
>
> + rtx orig_b;
>
> + rtx_insn *insn_a, *insn_b;
>
> + bool a_simple = if_info->then_simple;
>
> + bool b_simple = if_info->else_simple;
>
> + basic_block then_bb = if_info->then_bb;
>
> + basic_block else_bb = if_info->else_bb;
>
> + rtx target;
>
> + enum rtx_code code;
>
> + rtx cond = if_info->cond;
>
> + rtx_insn *ifcvt_seq;
>
> +
>
> + /* if (test) x = *a; else x = c - d;
>
> + => x = c - d;
>
> + if (test)
>
> + x = *a;
>
> + */
>
> +
>
> + code = GET_CODE (cond);
>
> + insn_a = if_info->insn_a;
>
> + insn_b = if_info->insn_b;
>
> +
>
> + machine_mode x_mode = GET_MODE (x);
>
> +
>
> + if (!can_conditionally_move_p (x_mode))
>
> + return false;
>
> +
>
> + /* Because we only handle one trap MEM + other non_trap, non mem cases,
>
> + just move one trap MEM always in then_bb. */
>
> + if (noce_reversed_cond_code (if_info) != UNKNOWN)
>
> + {
>
> + bool reversep = false;
>
> + if (b_trap)
>
> + reversep = true;
>
> +
>
> + if (reversep)
>
> + {
>
> + if (if_info->rev_cond)
>
> + {
>
> + cond = if_info->rev_cond;
>
> + code = GET_CODE (cond);
>
> + }
>
> + else
>
> + code = reversed_comparison_code (cond, if_info->jump);
>
> + std::swap (a, b);
>
> + std::swap (insn_a, insn_b);
>
> + std::swap (a_simple, b_simple);
>
> + std::swap (then_bb, else_bb);
>
> + }
>
> + }
>
> +
>
> + if (then_bb && else_bb
>
> + && (!bbs_ok_for_cmove_arith (then_bb, else_bb, if_info->orig_x)
>
> + || !bbs_ok_for_cmove_arith (else_bb, then_bb, if_info->orig_x)))
>
> + return false;
>
> +
>
> + start_sequence ();
>
> +
>
> + /* If one of the blocks is empty then the corresponding B or A value
>
> + came from the test block. The non-empty complex block that we will
>
> + emit might clobber the register used by B or A, so move it to a pseudo
>
> + first. */
>
> +
>
> + rtx tmp_b = NULL_RTX;
>
> +
>
> + /* Don't move trap mem to a pseudo. */
>
> + if (!may_trap_or_fault_p (b) && (b_simple || !else_bb))
>
> + tmp_b = gen_reg_rtx (x_mode);
>
> +
>
> + orig_b = b;
>
> +
>
> + rtx emit_a = NULL_RTX;
>
> + rtx emit_b = NULL_RTX;
>
> + rtx_insn *tmp_insn = NULL;
>
> + bool modified_in_a = false;
>
> + bool modified_in_b = false;
>
> + /* If either operand is complex, load it into a register first.
>
> + The best way to do this is to copy the original insn. In this
>
> + way we preserve any clobbers etc that the insn may have had.
>
> + This is of course not possible in the IS_MEM case. */
>
> +
>
> + if (! general_operand (b, GET_MODE (b)) || tmp_b)
>
> + {
>
> + if (insn_b)
>
> + {
>
> + b = tmp_b ? tmp_b : gen_reg_rtx (GET_MODE (b));
>
> + rtx_insn *copy_of_b = as_a <rtx_insn *> (copy_rtx (insn_b));
>
> + rtx set = single_set (copy_of_b);
>
> +
>
> + SET_DEST (set) = b;
>
> + emit_b = PATTERN (copy_of_b);
>
> + }
>
> + else
>
> + {
>
> + rtx tmp_reg = tmp_b ? tmp_b : gen_reg_rtx (GET_MODE (b));
>
> + emit_b = gen_rtx_SET (tmp_reg, b);
>
> + b = tmp_reg;
>
> + }
>
> + }
>
> +
>
> + if (tmp_b && then_bb)
>
> + {
>
> + FOR_BB_INSNS (then_bb, tmp_insn)
>
> + /* Don't check inside insn_a. We will have changed it to emit_a
>
> + with a destination that doesn't conflict. */
>
> + if (!(insn_a && tmp_insn == insn_a)
>
> + && modified_in_p (orig_b, tmp_insn))
>
> + {
>
> + modified_in_a = true;
>
> + break;
>
> + }
>
> +
>
> + }
>
> +
>
> + modified_in_b = emit_b != NULL_RTX && modified_in_p (a, emit_b);
>
> + /* If insn to set up A clobbers any registers B depends on, try to
>
> + swap insn that sets up A with the one that sets up B. If even
>
> + that doesn't help, punt. */
>
> + if (modified_in_a && !modified_in_b)
>
> + {
>
> + if (!noce_emit_bb (emit_b, else_bb, b_simple))
>
> + goto end_seq_and_fail;
>
> +
>
> + if (!noce_emit_bb (emit_a, then_bb, a_simple))
>
> + goto end_seq_and_fail;
>
> + }
>
> + else if (!modified_in_a)
>
> + {
>
> + if (!noce_emit_bb (emit_b, else_bb, b_simple))
>
> + goto end_seq_and_fail;
>
> +
>
> + if (!noce_emit_bb (emit_a, then_bb, a_simple))
>
> + goto end_seq_and_fail;
>
> + }
>
> + else
>
> + goto end_seq_and_fail;
>
> +
>
> + target = noce_emit_cmove (if_info, x, code, XEXP (cond, 0), XEXP (cond, 1),
>
> + a, b);
>
> +
>
> + if (! target)
>
> + goto end_seq_and_fail;
>
> +
>
> + if (target != x)
>
> + noce_emit_move_insn (x, target);
>
> +
>
> + ifcvt_seq = end_ifcvt_sequence (if_info);
>
> + if (!ifcvt_seq || !targetm.noce_conversion_profitable_p (ifcvt_seq, if_info))
>
> + return false;
>
> +
>
> + emit_insn_before_setloc (ifcvt_seq, if_info->jump,
>
> + INSN_LOCATION (if_info->insn_a));
>
> + if_info->transform_name = "noce_try_cmove_load_mem_notrap";
>
> + return true;
>
> +
>
> + end_seq_and_fail:
>
> + end_sequence ();
>
> + return false;
>
> +}
>
> +
>
> +static bool
>
> +noce_try_cmove_store_mem_notrap (struct noce_if_info *if_info, rtx *x_ptr, rtx orig_x)
>
> +{
>
> + rtx a = if_info->a;
>
> + rtx b = if_info->b;
>
> + rtx x = orig_x;
>
> + machine_mode x_mode = GET_MODE (x);
>
> +
>
> + if (!MEM_P (x) || !rtx_equal_p (x, b))
>
> + return false;
>
> + if (!may_trap_or_fault_p (x) || !targetm.have_conditional_move_mem_notrap (x))
>
> + return false;
>
> + if (!if_info->then_simple || !register_operand (a, x_mode))
>
> + return false;
>
> +
>
> + rtx cond = if_info->cond;
>
> + enum rtx_code code = GET_CODE (cond);
>
> + rtx_insn *ifcvt_seq;
>
> +
>
> + start_sequence ();
>
> +
>
> + rtx target = noce_emit_cmove (if_info, x, code, XEXP (cond, 0), XEXP (cond, 1),
>
> + a, b);
>
> +
>
> + if (! target)
>
> + goto end_seq_and_fail;
>
> +
>
> + if (target != x)
>
> + noce_emit_move_insn (x, target);
>
> +
>
> + ifcvt_seq = end_ifcvt_sequence (if_info);
>
> + if (!ifcvt_seq || !targetm.noce_conversion_profitable_p (ifcvt_seq, if_info))
>
> + return false;
>
> +
>
> + emit_insn_before_setloc (ifcvt_seq, if_info->jump,
>
> + INSN_LOCATION (if_info->insn_a));
>
> + if_info->transform_name = "noce_try_cmove_load_mem_notrap";
>
> + if_info->x = orig_x;
>
> + *x_ptr = orig_x;
>
> + return true;
>
> +
>
> + end_seq_and_fail:
>
> + end_sequence ();
>
> + return false;
>
> +}
>
> +
>
> /* For most cases, the simplified condition we found is the best
>
> choice, but this is not the case for the min/max/abs transforms.
>
> For these we wish to know that it is A or B in the condition. */
>
> @@ -4121,12 +4354,21 @@ noce_process_if_block (struct noce_if_info *if_info)
>
> }
>
>
>
> if (!set_b && MEM_P (orig_x))
>
> + {
>
> + /* Conditional_move_suppress_fault for condition mem store would not
>
> + move any arithmetic calculations. */
>
> + if (targetm.have_conditional_move_mem_notrap (orig_x)
>
> + && HAVE_conditional_move
>
> + && noce_try_cmove_store_mem_notrap (if_info, &x, orig_x))
>
> + goto success;
>
> + else
>
> /* We want to avoid store speculation to avoid cases like
>
> if (pthread_mutex_trylock(mutex))
>
> ++global_variable;
>
> Rather than go to much effort here, we rely on the SSA optimizers,
>
> which do a good enough job these days. */
>
> - return false;
>
> + return false;
>
> + }
>
>
>
> if (noce_try_move (if_info))
>
> goto success;
>
> @@ -4160,6 +4402,9 @@ noce_process_if_block (struct noce_if_info *if_info)
>
> if (HAVE_conditional_move
>
> && noce_try_cmove_arith (if_info))
>
> goto success;
>
> + if (HAVE_conditional_move
>
> + && noce_try_cmove_load_mem_notrap (if_info))
>
> + goto success;
>
> if (noce_try_sign_mask (if_info))
>
> goto success;
>
> }
>
> diff --git a/gcc/testsuite/gcc.target/i386/apx-cfcmov-1.c b/gcc/testsuite/gcc.target/i386/apx-cfcmov-1.c
>
> new file mode 100644
>
> index 00000000000..4a1fb91b24c
>
> --- /dev/null
>
> +++ b/gcc/testsuite/gcc.target/i386/apx-cfcmov-1.c
>
> @@ -0,0 +1,73 @@
>
> +/* { dg-do compile { target { ! ia32 } } } */
>
> +/* { dg-options "-O3 -mapxf" } */
>
> +
>
> +/* { dg-final { scan-assembler-times "cfcmovne" 1 } } */
>
> +/* { dg-final { scan-assembler-times "cfcmovg" 2} } */
>
> +/* { dg-final { scan-assembler-times "cfcmove" 1 } } */
>
> +/* { dg-final { scan-assembler-times "cfcmovl" 2 } } */
>
> +/* { dg-final { scan-assembler-times "cfcmovle" 1 } } */
>
> +
>
> +__attribute__((noinline, noclone, target("apxf")))
>
> +int cfc_store (int a, int b, int c, int d, int *arr)
>
> +{
>
> + if (a != b)
>
> + *arr = c;
>
> + return d;
>
> +
>
> +}
>
> +
>
> +__attribute__((noinline, noclone, target("apxf")))
>
> +int cfc_load_ndd (int a, int b, int c, int *p)
>
> +{
>
> + if (a > b)
>
> + return *p;
>
> + return c;
>
> +}
>
> +
>
> +__attribute__((noinline, noclone, target("apxf")))
>
> +int cfc_load_2_trap (int a, int b, int *c, int *p)
>
> +{
>
> + if (a > b)
>
> + return *p;
>
> + return *c;
>
> +}
>
> +
>
> +__attribute__((noinline, noclone, target("apxf")))
>
> +int cfc_load_zero (int a, int b, int c)
>
> +{
>
> + int sum = 0;
>
> + if (a == b)
>
> + return c;
>
> + return sum;
>
> +}
>
> +
>
> +__attribute__((noinline, noclone, target("apxf")))
>
> +int cfc_load_mem (int a, int b, int *p)
>
> +{
>
> + int sum = 0;
>
> + if (a < b )
>
> + sum = *p;
>
> + return sum;
>
> +}
>
> +
>
> +__attribute__((noinline, noclone, target("apxf")))
>
> +int cfc_load_arith_1 (int a, int b, int c, int *p)
>
> +{
>
> + int sum = 0;
>
> + if (a > b)
>
> + sum = *p;
>
> + else
>
> + sum = a + c;
>
> + return sum + 1;
>
> +}
>
> +
>
> +__attribute__((noinline, noclone, target("apxf")))
>
> +int cfc_load_arith_2 (int a, int b, int c, int *p)
>
> +{
>
> + int sum = 0;
>
> + if (a > b)
>
> + sum = a + c;
>
> + else
>
> + sum = *p;
>
> + return sum + 1;
>
> +}
>
> diff --git a/gcc/testsuite/gcc.target/i386/apx-cfcmov-2.c b/gcc/testsuite/gcc.target/i386/apx-cfcmov-2.c
>
> new file mode 100644
>
> index 00000000000..2b1660f64fa
>
> --- /dev/null
>
> +++ b/gcc/testsuite/gcc.target/i386/apx-cfcmov-2.c
>
> @@ -0,0 +1,40 @@
>
> +/* { dg-do run { target { ! ia32 } } } */
>
> +/* { dg-require-effective-target apxf } */
>
> +/* { dg-options "-mapxf -march=x86-64 -O3" } */
>
> +
>
> +#include "apx-cfcmov-1.c"
>
> +
>
> +extern void abort (void);
>
> +
>
> +int main ()
>
> +{
>
> + if (!__builtin_cpu_supports ("apxf"))
>
> + return 0;
>
> +
>
> + int arr = 6;
>
> + int arr1 = 5;
>
> + int res = cfc_store (1, 2, 3, 4, &arr);
>
> + if (arr != 3 && res != 4)
>
> + abort ();
>
> + res = cfc_load_ndd (2, 1, 2, &arr);
>
> + if (res != 3)
>
> + abort ();
>
> + res = cfc_load_2_trap (1, 2, &arr1, &arr);
>
> + if (res != 5)
>
> + abort ();
>
> + res = cfc_load_zero (1, 2, 3);
>
> + res = cfc_load_zero (1, 2, 3);
>
> + if (res != 0)
>
> + abort ();
>
> + res = cfc_load_mem (2, 1, &arr);
>
> + if (res != 0)
>
> + abort ();
>
> + res = cfc_load_arith_1 (1, 2, 3, &arr);
>
> + if (res != 5)
>
> + abort();
>
> + res = cfc_load_arith_2 (2, 1, 3,&arr);
>
> + if (res != 6)
>
> + abort();
>
> + return 0;
>
> +}
>
> +
>
> --
>
> 2.31.1
>
>
^ permalink raw reply [flat|nested] 4+ messages in thread
* RE: [PATCH 2/2] [APX CFCMOV] Support APX CFCMOV
2024-06-14 6:12 ` Richard Biener
@ 2024-06-14 6:19 ` Liu, Hongtao
0 siblings, 0 replies; 4+ messages in thread
From: Liu, Hongtao @ 2024-06-14 6:19 UTC (permalink / raw)
To: Richard Biener, Kong, Lingling, Richard Sandiford
Cc: gcc-patches, Uros Bizjak
> -----Original Message-----
> From: Richard Biener <richard.guenther@gmail.com>
> Sent: Friday, June 14, 2024 2:13 PM
> To: Kong, Lingling <lingling.kong@intel.com>; Richard Sandiford
> <richard.sandiford@arm.com>
> Cc: gcc-patches@gcc.gnu.org; Liu, Hongtao <hongtao.liu@intel.com>; Uros
> Bizjak <ubizjak@gmail.com>
> Subject: Re: [PATCH 2/2] [APX CFCMOV] Support APX CFCMOV
>
> On Fri, Jun 14, 2024 at 3:39 AM Kong, Lingling <lingling.kong@intel.com>
> wrote:
> >
> > From: konglin1 <lingling.kong@intel.com>
> >
> >
> >
> > APX CFCMOV feature implements conditionally faulting which means that
> > all
> >
> > memory faults are suppressed when the condition code evaluates to
> > false and
> >
> > load or store a memory operand. Now we could load or store a memory
> > operand
> >
> > may trap or fault for conditional move.
> >
> >
> >
> > To enable CFCMOV, we add a target HOOK
> > TARGET_HAVE_CONDITIONAL_MOVE_MEM_NOTRAP
> >
> > in if-conversion pass to allow convert to cmov.
> >
> >
> >
> > Bootstrapped & regtested on x86-64-pc-linux-gnu with binutils 2.42 branch.
> >
> > OK for trunk?
>
> How does if-conversion end up modifying the IL?
>
> I have the gut feeling that your hook changes semantics of RTL and you should
> instead have an optab for a "masked" load/store?
>
> Richard - do you already have plans how to represent the first-fault loads?
> (are there first-fault stores?)
Yes.
>
> Richard.
>
> >
> >
> > gcc/ChangeLog:
> >
> >
> >
> > * config/i386/i386-expand.cc (ix86_can_cfcmov_p): New
> > function that
> >
> > test if the cfcmov can be generated.
> >
> > (ix86_expand_int_movcc): Expand to cfcmov pattern if
> > ix86_can_cfcmov_p
> >
> > return ture.
> >
> > * config/i386/i386-opts.h (enum apx_features): Add apx_cfcmov.
> >
> > * config/i386/i386.cc
> > (ix86_have_conditional_move_mem_notrap): New
> >
> > function to hook
> > TARGET_HAVE_CONDITIONAL_MOVE_MEM_NOTRAP
> >
> > (TARGET_HAVE_CONDITIONAL_MOVE_MEM_NOTRAP): Target hook
> define.
> >
> > (ix86_rtx_costs): Add UNSPEC_APX_CFCMOV cost;
> >
> > * config/i386/i386.h (TARGET_APX_CFCMOV): Define.
> >
> > * config/i386/i386.md (*cfcmov<mode>_1): New
> > define_insn to support
> >
> > cfcmov.
> >
> > (*cfcmov<mode>_2): Ditto.
> >
> > (UNSPEC_APX_CFCMOV): New unspec for cfcmov.
> >
> > * config/i386/i386.opt: Add enum value for cfcmov.
> >
> > * ifcvt.cc (noce_try_cmove_load_mem_notrap): Use target
> > hook to allow
> >
> > convert to cfcmov for conditional load.
> >
> > (noce_try_cmove_store_mem_notrap): Convert to conditional store.
> >
> > (noce_process_if_block): Ditto.
> >
> >
> >
> > gcc/testsuite/ChangeLog:
> >
> >
> >
> > * gcc.target/i386/apx-cfcmov-1.c: New test.
> >
> > * gcc.target/i386/apx-cfcmov-2.c: Ditto.
> >
> > ---
> >
> > gcc/config/i386/i386-expand.cc | 63 +++++
> >
> > gcc/config/i386/i386-opts.h | 4 +-
> >
> > gcc/config/i386/i386.cc | 33 ++-
> >
> > gcc/config/i386/i386.h | 1 +
> >
> > gcc/config/i386/i386.md | 53 +++-
> >
> > gcc/config/i386/i386.opt | 3 +
> >
> > gcc/config/i386/predicates.md | 7 +
> >
> > gcc/ifcvt.cc | 247 ++++++++++++++++++-
> >
> > gcc/testsuite/gcc.target/i386/apx-cfcmov-1.c | 73 ++++++
> >
> > gcc/testsuite/gcc.target/i386/apx-cfcmov-2.c | 40 +++
> >
> > 10 files changed, 511 insertions(+), 13 deletions(-)
> >
> > create mode 100644 gcc/testsuite/gcc.target/i386/apx-cfcmov-1.c
> >
> > create mode 100644 gcc/testsuite/gcc.target/i386/apx-cfcmov-2.c
> >
> >
> >
> > diff --git a/gcc/config/i386/i386-expand.cc
> > b/gcc/config/i386/i386-expand.cc
> >
> > index 312329e550b..c02a4bcbec3 100644
> >
> > --- a/gcc/config/i386/i386-expand.cc
> >
> > +++ b/gcc/config/i386/i386-expand.cc
> >
> > @@ -3336,6 +3336,30 @@ ix86_expand_int_addcc (rtx operands[])
> >
> > return true;
> >
> > }
> >
> >
> >
> > +/* Return TRUE if we could convert "if (test) x = a; else x = b;" to
> > +cfcmov,
> >
> > + especially when load a or b or x store may cause memmory faults.
> > + */
> >
> > +bool
> >
> > +ix86_can_cfcmov_p (rtx x, rtx a, rtx b)
> >
> > +{
> >
> > + machine_mode mode = GET_MODE (x);
> >
> > + if (TARGET_APX_CFCMOV
> >
> > + && (mode == DImode || mode == SImode || mode == HImode))
> >
> > + {
> >
> > + /* C load (r m r), (r m C), (r r m). For r m m could use
> >
> > + two cfcmov. */
> >
> > + if (register_operand (x, mode)
> >
> > + && ((MEM_P (a) && register_operand (b, mode))
> >
> > + || (MEM_P (a) && b == const0_rtx)
> >
> > + || (register_operand (a, mode) && MEM_P (b))
> >
> > + || (MEM_P (a) && MEM_P (b))))
> >
> > + return true;
> >
> > + /* C store (m r 0). */
> >
> > + else if (MEM_P (x) && x == b && register_operand (a, mode))
> >
> > + return true;
> >
> > + }
> >
> > + return false;
> >
> > +}
> >
> > +
> >
> > bool
> >
> > ix86_expand_int_movcc (rtx operands[])
> >
> > {
> >
> > @@ -3366,6 +3390,45 @@ ix86_expand_int_movcc (rtx operands[])
> >
> >
> >
> > compare_code = GET_CODE (compare_op);
> >
> >
> >
> > + if (MEM_P (operands[0])
> >
> > + && !ix86_can_cfcmov_p (operands[0], op2, op3))
> >
> > + return false;
> >
> > +
> >
> > + if (may_trap_or_fault_p (op2) || may_trap_or_fault_p (op3))
> >
> > + {
> >
> > + if (ix86_can_cfcmov_p (operands[0], op2, op3))
> >
> > + {
> >
> > + if (may_trap_or_fault_p (op2))
> >
> > + op2 = gen_rtx_UNSPEC (mode, gen_rtvec (1,
> > + operands[2]),
> >
> > +
> > + UNSPEC_APX_CFCMOV);
> >
> > + if (may_trap_or_fault_p (op3))
> >
> > + op3 = gen_rtx_UNSPEC (mode, gen_rtvec (1,
> > + operands[3]),
> >
> > +
> > + UNSPEC_APX_CFCMOV);
> >
> > + emit_insn (compare_seq);
> >
> > +
> >
> > + if (may_trap_or_fault_p (op2) && may_trap_or_fault_p
> > + (op3))
> >
> > + {
> >
> > + emit_insn (gen_rtx_SET (operands[0],
> >
> > +
> > + gen_rtx_IF_THEN_ELSE (mode,
> >
> > +
> > + compare_op,
> >
> > +
> > + op2,
> >
> > +
> > + operands[0])));
> >
> > + emit_insn (gen_rtx_SET (operands[0],
> >
> > +
> > + gen_rtx_IF_THEN_ELSE (mode,
> >
> > +
> > + compare_op,
> >
> > +
> > + operands[0],
> >
> > +
> > + op3)));
> >
> > + }
> >
> > + else
> >
> > + emit_insn (gen_rtx_SET (operands[0],
> >
> > +
> > + gen_rtx_IF_THEN_ELSE (mode,
> >
> > +
> > + compare_op,
> >
> > +
> > + op2, op3)));
> >
> > + return true;
> >
> > + }
> >
> > + return false;
> >
> > + }
> >
> > +
> >
> > if ((op1 == const0_rtx && (code == GE || code == LT))
> >
> > || (op1 == constm1_rtx && (code == GT || code == LE)))
> >
> > sign_bit_compare_p = true;
> >
> > diff --git a/gcc/config/i386/i386-opts.h b/gcc/config/i386/i386-opts.h
> >
> > index c7ec0d9fd39..711519ffb53 100644
> >
> > --- a/gcc/config/i386/i386-opts.h
> >
> > +++ b/gcc/config/i386/i386-opts.h
> >
> > @@ -143,8 +143,10 @@ enum apx_features {
> >
> > apx_nf = 1 << 4,
> >
> > apx_ccmp = 1 << 5,
> >
> > apx_zu = 1 << 6,
> >
> > + apx_cfcmov = 1 << 7,
> >
> > apx_all = apx_egpr | apx_push2pop2 | apx_ndd
> >
> > - | apx_ppx | apx_nf | apx_ccmp | apx_zu,
> >
> > + | apx_ppx | apx_nf | apx_ccmp | apx_zu
> >
> > + | apx_cfcmov,
> >
> > };
> >
> >
> >
> > #endif
> >
> > diff --git a/gcc/config/i386/i386.cc b/gcc/config/i386/i386.cc
> >
> > index 173db213d14..b14c0a3d9f2 100644
> >
> > --- a/gcc/config/i386/i386.cc
> >
> > +++ b/gcc/config/i386/i386.cc
> >
> > @@ -22349,10 +22349,18 @@ ix86_rtx_costs (rtx x, machine_mode mode,
> > int outer_code_i, int opno,
> >
> > *total = COSTS_N_INSNS (1);
> >
> > if (!COMPARISON_P (XEXP (x, 0)) && !REG_P (XEXP (x,
> > 0)))
> >
> > *total += rtx_cost (XEXP (x, 0), mode, code, 0,
> > speed);
> >
> > - if (!REG_P (XEXP (x, 1)))
> >
> > - *total += rtx_cost (XEXP (x, 1), mode, code, 1, speed);
> >
> > - if (!REG_P (XEXP (x, 2)))
> >
> > - *total += rtx_cost (XEXP (x, 2), mode, code, 2, speed);
> >
> > + rtx op1, op2;
> >
> > + op1 = XEXP (x, 1);
> >
> > + op2 = XEXP (x, 2);
> >
> > + /* Handle UNSPEC_APX_CFCMOV for cfcmov. */
> >
> > + if (GET_CODE (op1) == UNSPEC && XINT (op1, 1) ==
> > + UNSPEC_APX_CFCMOV)
> >
> > + op1 = XVECEXP (op1, 0, 0);
> >
> > + if (GET_CODE (op2) == UNSPEC && XINT (op2, 1) ==
> > + UNSPEC_APX_CFCMOV)
> >
> > + op2 = XVECEXP (op2, 0, 0);
> >
> > + if (!REG_P (op1))
> >
> > + *total += rtx_cost (op1, mode, code, 1, speed);
> >
> > + if (!REG_P (op2))
> >
> > + *total += rtx_cost (op2, mode, code, 2, speed);
> >
> > return true;
> >
> > }
> >
> > return false;
> >
> > @@ -24998,6 +25006,19 @@ ix86_noce_conversion_profitable_p (rtx_insn
> > *seq, struct noce_if_info *if_info)
> >
> > return default_noce_conversion_profitable_p (seq, if_info);
> >
> > }
> >
> >
> >
> > +
> >
> > +/* Implement targetm.have_conditional_move_mem_notrap hook. */
> >
> > +static bool
> >
> > +ix86_have_conditional_move_mem_notrap (rtx x)
> >
> > +{
> >
> > + machine_mode mode = GET_MODE (x);
> >
> > + if (TARGET_APX_CFCMOV
> >
> > + && (mode == DImode || mode == SImode || mode == HImode)
> >
> > + && MEM_P (x))
> >
> > + return true;
> >
> > + return false;
> >
> > +}
> >
> > +
> >
> > /* x86-specific vector costs. */
> >
> > class ix86_vector_costs : public vector_costs
> >
> > {
> >
> > @@ -26975,6 +26996,10 @@ ix86_libgcc_floating_mode_supported_p
> >
> > #undef TARGET_NOCE_CONVERSION_PROFITABLE_P
> >
> > #define TARGET_NOCE_CONVERSION_PROFITABLE_P
> > ix86_noce_conversion_profitable_p
> >
> >
> >
> > +#undef TARGET_HAVE_CONDITIONAL_MOVE_MEM_NOTRAP
> >
> > +#define TARGET_HAVE_CONDITIONAL_MOVE_MEM_NOTRAP \
> >
> > + ix86_have_conditional_move_mem_notrap
> >
> > +
> >
> > #undef TARGET_HARD_REGNO_NREGS
> >
> > #define TARGET_HARD_REGNO_NREGS ix86_hard_regno_nregs
> >
> > #undef TARGET_HARD_REGNO_MODE_OK
> >
> > diff --git a/gcc/config/i386/i386.h b/gcc/config/i386/i386.h
> >
> > index dc1a1f44320..6a20fa678c8 100644
> >
> > --- a/gcc/config/i386/i386.h
> >
> > +++ b/gcc/config/i386/i386.h
> >
> > @@ -58,6 +58,7 @@ see the files COPYING3 and COPYING.RUNTIME
> > respectively. If not, see
> >
> > #define TARGET_APX_NF (ix86_apx_features & apx_nf)
> >
> > #define TARGET_APX_CCMP (ix86_apx_features & apx_ccmp)
> >
> > #define TARGET_APX_ZU (ix86_apx_features & apx_zu)
> >
> > +#define TARGET_APX_CFCMOV (ix86_apx_features & apx_cfcmov)
> >
> >
> >
> > #include "config/vxworks-dummy.h"
> >
> >
> >
> > diff --git a/gcc/config/i386/i386.md b/gcc/config/i386/i386.md
> >
> > index fd48e764469..57448c07828 100644
> >
> > --- a/gcc/config/i386/i386.md
> >
> > +++ b/gcc/config/i386/i386.md
> >
> > @@ -221,6 +221,9 @@
> >
> > ;; For APX CCMP support
> >
> > ;; DFV = default flag value
> >
> > UNSPEC_APX_DFV
> >
> > +
> >
> > + ;; For APX CFCMOV support
> >
> > + UNSPEC_APX_CFCMOV
> >
> > ])
> >
> >
> >
> > (define_c_enum "unspecv" [
> >
> > @@ -579,7 +582,7 @@
> >
> >
> > noavx512dq,fma_or_avx512vl,avx512vl,noavx512vl,avxvnni,
> >
> >
> > avx512vnnivl,avx512fp16,avxifma,avx512ifmavl,avxneconvert,
> >
> >
> > avx512bf16vl,vpclmulqdqvl,avx_noavx512f,avx_noavx512vl,
> >
> > - vaes_avx512vl,noapx_nf"
> >
> > + vaes_avx512vl,noapx_nf,apx_cfcmov"
> >
> > (const_string "base"))
> >
> >
> >
> > ;; The (bounding maximum) length of an instruction immediate.
> >
> > @@ -986,6 +989,7 @@
> >
> > (eq_attr "mmx_isa" "avx")
> >
> > (symbol_ref "TARGET_MMX_WITH_SSE && TARGET_AVX")
> >
> > (eq_attr "isa" "noapx_nf") (symbol_ref
> > "!TARGET_APX_NF")
> >
> > + (eq_attr "isa" "apx_cfcmov") (symbol_ref
> > + "TARGET_APX_CFCMOV")
> >
> > ]
> >
> > (const_int 1)))
> >
> >
> >
> > @@ -24995,7 +24999,7 @@
> >
> > ;; Conditional move instructions.
> >
> >
> >
> > (define_expand "mov<mode>cc"
> >
> > - [(set (match_operand:SWIM 0 "register_operand")
> >
> > + [(set (match_operand:SWIM 0 "register_or_cfc_mem_operand")
> >
> > (if_then_else:SWIM (match_operand 1
> > "comparison_operator")
> >
> > (match_operand:SWIM 2
> > "<general_operand>")
> >
> > (match_operand:SWIM 3
> > "<general_operand>")))]
> >
> > @@ -25103,19 +25107,54 @@
> >
> > (set (match_dup 0)
> >
> > (neg:SWI (ltu:SWI (reg:CCC FLAGS_REG) (const_int
> > 0))))])
> >
> >
> >
> > +(define_insn "*cfcmov<mode>_1"
> >
> > + [(set (match_operand:SWI248 0 "register_operand" "=r,r")
> >
> > + (if_then_else:SWI248 (match_operator 1 "ix86_comparison_operator"
> >
> > + [(reg FLAGS_REG) (const_int 0)])
> >
> > + (unspec:SWI248
> >
> > + [(match_operand:SWI248 2 "memory_operand" "m,m")]
> >
> > + UNSPEC_APX_CFCMOV)
> >
> > + (match_operand:SWI248 3 "reg_or_0_operand" "C,r")))]
> >
> > + "TARGET_CMOVE && TARGET_APX_CFCMOV"
> >
> > + "@
> >
> > + cfcmov%O2%C1\t{%2, %0|%0, %2}
> >
> > + cfcmov%O2%C1\t{%2, %3, %0|%0, %3, %2}"
> >
> > + [(set_attr "isa" "*,apx_ndd")
> >
> > + (set_attr "type" "icmov")
> >
> > + (set_attr "prefix" "evex")
> >
> > + (set_attr "mode" "<MODE>")])
> >
> > +
> >
> > +(define_insn "*cfcmov<mode>_2"
> >
> > + [(set (match_operand:SWI248 0 "nonimmediate_operand" "=r,m")
> >
> > + (if_then_else:SWI248 (match_operator 1 "ix86_comparison_operator"
> >
> > + [(reg FLAGS_REG) (const_int 0)])
> >
> > + (match_operand:SWI248 2 "register_operand" "r,r")
> >
> > + (unspec:SWI248
> >
> > + [(match_operand:SWI248 3 "memory_operand" "m,0")]
> >
> > + UNSPEC_APX_CFCMOV)))]
> >
> > + "TARGET_CMOVE && TARGET_APX_CFCMOV"
> >
> > + "@
> >
> > + cfcmov%O2%c1\t{%3, %2, %0|%0, %2, %3}
> >
> > + cfcmov%O2%C1\t{%2, %0|%0, %2}"
> >
> > + [(set_attr "isa" "apx_ndd,*")
> >
> > + (set_attr "type" "icmov")
> >
> > + (set_attr "prefix" "evex")
> >
> > + (set_attr "mode" "<MODE>")])
> >
> > +
> >
> > (define_insn "*mov<mode>cc_noc"
> >
> > - [(set (match_operand:SWI248 0 "register_operand" "=r,r,r,r")
> >
> > + [(set (match_operand:SWI248 0 "register_operand" "=r,r,r,r,r")
> >
> > (if_then_else:SWI248 (match_operator 1
> "ix86_comparison_operator"
> >
> > [(reg FLAGS_REG)
> > (const_int 0)])
> >
> > - (match_operand:SWI248 2 "nonimmediate_operand" "rm,0,rm,r")
> >
> > - (match_operand:SWI248 3 "nonimmediate_operand"
> "0,rm,r,rm")))]
> >
> > + (match_operand:SWI248 2 "nonimmediate_operand"
> > + "rm,0,rm,r,r")
> >
> > + (match_operand:SWI248 3 "nonimm_or_0_operand"
> > + "0,rm,r,rm,C")))]
> >
> > "TARGET_CMOVE && !(MEM_P (operands[2]) && MEM_P (operands[3]))"
> >
> > "@
> >
> > cmov%O2%C1\t{%2, %0|%0, %2}
> >
> > cmov%O2%c1\t{%3, %0|%0, %3}
> >
> > cmov%O2%C1\t{%2, %3, %0|%0, %3, %2}
> >
> > - cmov%O2%c1\t{%3, %2, %0|%0, %2, %3}"
> >
> > - [(set_attr "isa" "*,*,apx_ndd,apx_ndd")
> >
> > + cmov%O2%c1\t{%3, %2, %0|%0, %2, %3}
> >
> > + cfcmov%O2%C1\t{%2, %0|%0, %2}"
> >
> > + [(set_attr "isa" "*,*,apx_ndd,apx_ndd,apx_cfcmov")
> >
> > (set_attr "type" "icmov")
> >
> > (set_attr "mode" "<MODE>")])
> >
> >
> >
> > diff --git a/gcc/config/i386/i386.opt b/gcc/config/i386/i386.opt
> >
> > index 353fffb2343..7d63d9abd95 100644
> >
> > --- a/gcc/config/i386/i386.opt
> >
> > +++ b/gcc/config/i386/i386.opt
> >
> > @@ -1345,6 +1345,9 @@ Enum(apx_features) String(ccmp)
> Value(apx_ccmp)
> > Set(7)
> >
> > EnumValue
> >
> > Enum(apx_features) String(zu) Value(apx_zu) Set(8)
> >
> >
> >
> > +EnumValue
> >
> > +Enum(apx_features) String(cfcmov) Value(apx_cfcmov) Set(9)
> >
> > +
> >
> > EnumValue
> >
> > Enum(apx_features) String(all) Value(apx_all) Set(1)
> >
> >
> >
> > diff --git a/gcc/config/i386/predicates.md
> > b/gcc/config/i386/predicates.md
> >
> > index 7afe3100cb7..d562e10ab41 100644
> >
> > --- a/gcc/config/i386/predicates.md
> >
> > +++ b/gcc/config/i386/predicates.md
> >
> > @@ -2322,3 +2322,10 @@
> >
> >
> >
> > return true;
> >
> > })
> >
> > +
> >
> > +;; Return true if OP is a register operand or memory_operand is only
> >
> > +;; supported under TARGET_APX_CFCMOV.
> >
> > +(define_predicate "register_or_cfc_mem_operand"
> >
> > + (ior (match_operand 0 "register_operand")
> >
> > + (and (match_code "mem")
> >
> > + (match_test "TARGET_APX_CFCMOV"))))
> >
> > diff --git a/gcc/ifcvt.cc b/gcc/ifcvt.cc
> >
> > index 58ed42673e5..6e3e48af810 100644
> >
> > --- a/gcc/ifcvt.cc
> >
> > +++ b/gcc/ifcvt.cc
> >
> > @@ -783,6 +783,8 @@ static rtx noce_emit_cmove (struct noce_if_info *,
> > rtx, enum rtx_code, rtx,
> >
> > rtx, rtx, rtx, rtx =
> > NULL, rtx = NULL);
> >
> > static bool noce_try_cmove (struct noce_if_info *);
> >
> > static bool noce_try_cmove_arith (struct noce_if_info *);
> >
> > +static bool noce_try_cmove_load_mem_notrap (struct noce_if_info *);
> >
> > +static bool noce_try_cmove_store_mem_notrap (struct noce_if_info *,
> > +rtx *, rtx);
> >
> > static rtx noce_get_alt_condition (struct noce_if_info *, rtx,
> > rtx_insn **);
> >
> > static bool noce_try_minmax (struct noce_if_info *);
> >
> > static bool noce_try_abs (struct noce_if_info *);
> >
> > @@ -2401,6 +2403,237 @@ noce_try_cmove_arith (struct noce_if_info
> > *if_info)
> >
> > return false;
> >
> > }
> >
> >
> >
> > +/* When target support suppress memory fault, try more complex cases
> > +involving
> >
> > + conditional_move's source or dest may trap or fault. */
> >
> > +
> >
> > +static bool
> >
> > +noce_try_cmove_load_mem_notrap (struct noce_if_info *if_info)
> >
> > +{
> >
> > + rtx a = if_info->a;
> >
> > + rtx b = if_info->b;
> >
> > + rtx x = if_info->x;
> >
> > +
> >
> > + if (MEM_P (x))
> >
> > + return false;
> >
> > + /* Just handle a conditional move from one trap MEM + other
> > + non_trap,
> >
> > + non mem cases. */
> >
> > + if (!(MEM_P (a) ^ MEM_P (b)))
> >
> > + return false;
> >
> > + bool a_trap = may_trap_or_fault_p (a);
> >
> > + bool b_trap = may_trap_or_fault_p (b);
> >
> > +
> >
> > + if (!(a_trap ^ b_trap))
> >
> > + return false;
> >
> > + if (a_trap && (!MEM_P (a) ||
> > + !targetm.have_conditional_move_mem_notrap (a)))
> >
> > + return false;
> >
> > + if (b_trap && (!MEM_P (b) ||
> > + !targetm.have_conditional_move_mem_notrap (b)))
> >
> > + return false;
> >
> > +
> >
> > + rtx orig_b;
> >
> > + rtx_insn *insn_a, *insn_b;
> >
> > + bool a_simple = if_info->then_simple;
> >
> > + bool b_simple = if_info->else_simple;
> >
> > + basic_block then_bb = if_info->then_bb;
> >
> > + basic_block else_bb = if_info->else_bb;
> >
> > + rtx target;
> >
> > + enum rtx_code code;
> >
> > + rtx cond = if_info->cond;
> >
> > + rtx_insn *ifcvt_seq;
> >
> > +
> >
> > + /* if (test) x = *a; else x = c - d;
> >
> > + => x = c - d;
> >
> > + if (test)
> >
> > + x = *a;
> >
> > + */
> >
> > +
> >
> > + code = GET_CODE (cond);
> >
> > + insn_a = if_info->insn_a;
> >
> > + insn_b = if_info->insn_b;
> >
> > +
> >
> > + machine_mode x_mode = GET_MODE (x);
> >
> > +
> >
> > + if (!can_conditionally_move_p (x_mode))
> >
> > + return false;
> >
> > +
> >
> > + /* Because we only handle one trap MEM + other non_trap, non mem
> > + cases,
> >
> > + just move one trap MEM always in then_bb. */
> >
> > + if (noce_reversed_cond_code (if_info) != UNKNOWN)
> >
> > + {
> >
> > + bool reversep = false;
> >
> > + if (b_trap)
> >
> > + reversep = true;
> >
> > +
> >
> > + if (reversep)
> >
> > + {
> >
> > + if (if_info->rev_cond)
> >
> > + {
> >
> > + cond = if_info->rev_cond;
> >
> > + code = GET_CODE (cond);
> >
> > + }
> >
> > + else
> >
> > + code = reversed_comparison_code (cond,
> > + if_info->jump);
> >
> > + std::swap (a, b);
> >
> > + std::swap (insn_a, insn_b);
> >
> > + std::swap (a_simple, b_simple);
> >
> > + std::swap (then_bb, else_bb);
> >
> > + }
> >
> > + }
> >
> > +
> >
> > + if (then_bb && else_bb
> >
> > + && (!bbs_ok_for_cmove_arith (then_bb, else_bb,
> > + if_info->orig_x)
> >
> > + || !bbs_ok_for_cmove_arith (else_bb, then_bb,
> > + if_info->orig_x)))
> >
> > + return false;
> >
> > +
> >
> > + start_sequence ();
> >
> > +
> >
> > + /* If one of the blocks is empty then the corresponding B or A
> > + value
> >
> > + came from the test block. The non-empty complex block that we
> > + will
> >
> > + emit might clobber the register used by B or A, so move it to a
> > + pseudo
> >
> > + first. */
> >
> > +
> >
> > + rtx tmp_b = NULL_RTX;
> >
> > +
> >
> > + /* Don't move trap mem to a pseudo. */
> >
> > + if (!may_trap_or_fault_p (b) && (b_simple || !else_bb))
> >
> > + tmp_b = gen_reg_rtx (x_mode);
> >
> > +
> >
> > + orig_b = b;
> >
> > +
> >
> > + rtx emit_a = NULL_RTX;
> >
> > + rtx emit_b = NULL_RTX;
> >
> > + rtx_insn *tmp_insn = NULL;
> >
> > + bool modified_in_a = false;
> >
> > + bool modified_in_b = false;
> >
> > + /* If either operand is complex, load it into a register first.
> >
> > + The best way to do this is to copy the original insn. In this
> >
> > + way we preserve any clobbers etc that the insn may have had.
> >
> > + This is of course not possible in the IS_MEM case. */
> >
> > +
> >
> > + if (! general_operand (b, GET_MODE (b)) || tmp_b)
> >
> > + {
> >
> > + if (insn_b)
> >
> > + {
> >
> > + b = tmp_b ? tmp_b : gen_reg_rtx (GET_MODE (b));
> >
> > + rtx_insn *copy_of_b = as_a <rtx_insn *> (copy_rtx
> > + (insn_b));
> >
> > + rtx set = single_set (copy_of_b);
> >
> > +
> >
> > + SET_DEST (set) = b;
> >
> > + emit_b = PATTERN (copy_of_b);
> >
> > + }
> >
> > + else
> >
> > + {
> >
> > + rtx tmp_reg = tmp_b ? tmp_b : gen_reg_rtx
> > + (GET_MODE (b));
> >
> > + emit_b = gen_rtx_SET (tmp_reg, b);
> >
> > + b = tmp_reg;
> >
> > + }
> >
> > + }
> >
> > +
> >
> > + if (tmp_b && then_bb)
> >
> > + {
> >
> > + FOR_BB_INSNS (then_bb, tmp_insn)
> >
> > + /* Don't check inside insn_a. We will have changed it
> > + to emit_a
> >
> > + with a destination that doesn't conflict. */
> >
> > + if (!(insn_a && tmp_insn == insn_a)
> >
> > + && modified_in_p (orig_b, tmp_insn))
> >
> > + {
> >
> > + modified_in_a = true;
> >
> > + break;
> >
> > + }
> >
> > +
> >
> > + }
> >
> > +
> >
> > + modified_in_b = emit_b != NULL_RTX && modified_in_p (a, emit_b);
> >
> > + /* If insn to set up A clobbers any registers B depends on, try to
> >
> > + swap insn that sets up A with the one that sets up B. If even
> >
> > + that doesn't help, punt. */
> >
> > + if (modified_in_a && !modified_in_b)
> >
> > + {
> >
> > + if (!noce_emit_bb (emit_b, else_bb, b_simple))
> >
> > + goto end_seq_and_fail;
> >
> > +
> >
> > + if (!noce_emit_bb (emit_a, then_bb, a_simple))
> >
> > + goto end_seq_and_fail;
> >
> > + }
> >
> > + else if (!modified_in_a)
> >
> > + {
> >
> > + if (!noce_emit_bb (emit_b, else_bb, b_simple))
> >
> > + goto end_seq_and_fail;
> >
> > +
> >
> > + if (!noce_emit_bb (emit_a, then_bb, a_simple))
> >
> > + goto end_seq_and_fail;
> >
> > + }
> >
> > + else
> >
> > + goto end_seq_and_fail;
> >
> > +
> >
> > + target = noce_emit_cmove (if_info, x, code, XEXP (cond, 0), XEXP
> > + (cond, 1),
> >
> > + a, b);
> >
> > +
> >
> > + if (! target)
> >
> > + goto end_seq_and_fail;
> >
> > +
> >
> > + if (target != x)
> >
> > + noce_emit_move_insn (x, target);
> >
> > +
> >
> > + ifcvt_seq = end_ifcvt_sequence (if_info);
> >
> > + if (!ifcvt_seq || !targetm.noce_conversion_profitable_p (ifcvt_seq,
> > + if_info))
> >
> > + return false;
> >
> > +
> >
> > + emit_insn_before_setloc (ifcvt_seq, if_info->jump,
> >
> > + INSN_LOCATION
> > + (if_info->insn_a));
> >
> > + if_info->transform_name = "noce_try_cmove_load_mem_notrap";
> >
> > + return true;
> >
> > +
> >
> > + end_seq_and_fail:
> >
> > + end_sequence ();
> >
> > + return false;
> >
> > +}
> >
> > +
> >
> > +static bool
> >
> > +noce_try_cmove_store_mem_notrap (struct noce_if_info *if_info, rtx
> > +*x_ptr, rtx orig_x)
> >
> > +{
> >
> > + rtx a = if_info->a;
> >
> > + rtx b = if_info->b;
> >
> > + rtx x = orig_x;
> >
> > + machine_mode x_mode = GET_MODE (x);
> >
> > +
> >
> > + if (!MEM_P (x) || !rtx_equal_p (x, b))
> >
> > + return false;
> >
> > + if (!may_trap_or_fault_p (x) ||
> > + !targetm.have_conditional_move_mem_notrap (x))
> >
> > + return false;
> >
> > + if (!if_info->then_simple || !register_operand (a, x_mode))
> >
> > + return false;
> >
> > +
> >
> > + rtx cond = if_info->cond;
> >
> > + enum rtx_code code = GET_CODE (cond);
> >
> > + rtx_insn *ifcvt_seq;
> >
> > +
> >
> > + start_sequence ();
> >
> > +
> >
> > + rtx target = noce_emit_cmove (if_info, x, code, XEXP (cond, 0),
> > + XEXP (cond, 1),
> >
> > + a, b);
> >
> > +
> >
> > + if (! target)
> >
> > + goto end_seq_and_fail;
> >
> > +
> >
> > + if (target != x)
> >
> > + noce_emit_move_insn (x, target);
> >
> > +
> >
> > + ifcvt_seq = end_ifcvt_sequence (if_info);
> >
> > + if (!ifcvt_seq || !targetm.noce_conversion_profitable_p (ifcvt_seq,
> > + if_info))
> >
> > + return false;
> >
> > +
> >
> > + emit_insn_before_setloc (ifcvt_seq, if_info->jump,
> >
> > + INSN_LOCATION
> > + (if_info->insn_a));
> >
> > + if_info->transform_name = "noce_try_cmove_load_mem_notrap";
> >
> > + if_info->x = orig_x;
> >
> > + *x_ptr = orig_x;
> >
> > + return true;
> >
> > +
> >
> > + end_seq_and_fail:
> >
> > + end_sequence ();
> >
> > + return false;
> >
> > +}
> >
> > +
> >
> > /* For most cases, the simplified condition we found is the best
> >
> > choice, but this is not the case for the min/max/abs transforms.
> >
> > For these we wish to know that it is A or B in the condition. */
> >
> > @@ -4121,12 +4354,21 @@ noce_process_if_block (struct noce_if_info
> > *if_info)
> >
> > }
> >
> >
> >
> > if (!set_b && MEM_P (orig_x))
> >
> > + {
> >
> > + /* Conditional_move_suppress_fault for condition mem store
> > + would not
> >
> > + move any arithmetic calculations. */
> >
> > + if (targetm.have_conditional_move_mem_notrap (orig_x)
> >
> > + && HAVE_conditional_move
> >
> > + && noce_try_cmove_store_mem_notrap (if_info, &x,
> > + orig_x))
> >
> > + goto success;
> >
> > + else
> >
> > /* We want to avoid store speculation to avoid cases like
> >
> > if (pthread_mutex_trylock(mutex))
> >
> > ++global_variable;
> >
> > Rather than go to much effort here, we rely on the SSA
> > optimizers,
> >
> > which do a good enough job these days. */
> >
> > - return false;
> >
> > + return false;
> >
> > + }
> >
> >
> >
> > if (noce_try_move (if_info))
> >
> > goto success;
> >
> > @@ -4160,6 +4402,9 @@ noce_process_if_block (struct noce_if_info
> > *if_info)
> >
> > if (HAVE_conditional_move
> >
> > && noce_try_cmove_arith (if_info))
> >
> > goto success;
> >
> > + if (HAVE_conditional_move
> >
> > + && noce_try_cmove_load_mem_notrap (if_info))
> >
> > + goto success;
> >
> > if (noce_try_sign_mask (if_info))
> >
> > goto success;
> >
> > }
> >
> > diff --git a/gcc/testsuite/gcc.target/i386/apx-cfcmov-1.c
> > b/gcc/testsuite/gcc.target/i386/apx-cfcmov-1.c
> >
> > new file mode 100644
> >
> > index 00000000000..4a1fb91b24c
> >
> > --- /dev/null
> >
> > +++ b/gcc/testsuite/gcc.target/i386/apx-cfcmov-1.c
> >
> > @@ -0,0 +1,73 @@
> >
> > +/* { dg-do compile { target { ! ia32 } } } */
> >
> > +/* { dg-options "-O3 -mapxf" } */
> >
> > +
> >
> > +/* { dg-final { scan-assembler-times "cfcmovne" 1 } } */
> >
> > +/* { dg-final { scan-assembler-times "cfcmovg" 2} } */
> >
> > +/* { dg-final { scan-assembler-times "cfcmove" 1 } } */
> >
> > +/* { dg-final { scan-assembler-times "cfcmovl" 2 } } */
> >
> > +/* { dg-final { scan-assembler-times "cfcmovle" 1 } } */
> >
> > +
> >
> > +__attribute__((noinline, noclone, target("apxf")))
> >
> > +int cfc_store (int a, int b, int c, int d, int *arr)
> >
> > +{
> >
> > + if (a != b)
> >
> > + *arr = c;
> >
> > + return d;
> >
> > +
> >
> > +}
> >
> > +
> >
> > +__attribute__((noinline, noclone, target("apxf")))
> >
> > +int cfc_load_ndd (int a, int b, int c, int *p)
> >
> > +{
> >
> > + if (a > b)
> >
> > + return *p;
> >
> > + return c;
> >
> > +}
> >
> > +
> >
> > +__attribute__((noinline, noclone, target("apxf")))
> >
> > +int cfc_load_2_trap (int a, int b, int *c, int *p)
> >
> > +{
> >
> > + if (a > b)
> >
> > + return *p;
> >
> > + return *c;
> >
> > +}
> >
> > +
> >
> > +__attribute__((noinline, noclone, target("apxf")))
> >
> > +int cfc_load_zero (int a, int b, int c)
> >
> > +{
> >
> > + int sum = 0;
> >
> > + if (a == b)
> >
> > + return c;
> >
> > + return sum;
> >
> > +}
> >
> > +
> >
> > +__attribute__((noinline, noclone, target("apxf")))
> >
> > +int cfc_load_mem (int a, int b, int *p)
> >
> > +{
> >
> > + int sum = 0;
> >
> > + if (a < b )
> >
> > + sum = *p;
> >
> > + return sum;
> >
> > +}
> >
> > +
> >
> > +__attribute__((noinline, noclone, target("apxf")))
> >
> > +int cfc_load_arith_1 (int a, int b, int c, int *p)
> >
> > +{
> >
> > + int sum = 0;
> >
> > + if (a > b)
> >
> > + sum = *p;
> >
> > + else
> >
> > + sum = a + c;
> >
> > + return sum + 1;
> >
> > +}
> >
> > +
> >
> > +__attribute__((noinline, noclone, target("apxf")))
> >
> > +int cfc_load_arith_2 (int a, int b, int c, int *p)
> >
> > +{
> >
> > + int sum = 0;
> >
> > + if (a > b)
> >
> > + sum = a + c;
> >
> > + else
> >
> > + sum = *p;
> >
> > + return sum + 1;
> >
> > +}
> >
> > diff --git a/gcc/testsuite/gcc.target/i386/apx-cfcmov-2.c
> > b/gcc/testsuite/gcc.target/i386/apx-cfcmov-2.c
> >
> > new file mode 100644
> >
> > index 00000000000..2b1660f64fa
> >
> > --- /dev/null
> >
> > +++ b/gcc/testsuite/gcc.target/i386/apx-cfcmov-2.c
> >
> > @@ -0,0 +1,40 @@
> >
> > +/* { dg-do run { target { ! ia32 } } } */
> >
> > +/* { dg-require-effective-target apxf } */
> >
> > +/* { dg-options "-mapxf -march=x86-64 -O3" } */
> >
> > +
> >
> > +#include "apx-cfcmov-1.c"
> >
> > +
> >
> > +extern void abort (void);
> >
> > +
> >
> > +int main ()
> >
> > +{
> >
> > + if (!__builtin_cpu_supports ("apxf"))
> >
> > + return 0;
> >
> > +
> >
> > + int arr = 6;
> >
> > + int arr1 = 5;
> >
> > + int res = cfc_store (1, 2, 3, 4, &arr);
> >
> > + if (arr != 3 && res != 4)
> >
> > + abort ();
> >
> > + res = cfc_load_ndd (2, 1, 2, &arr);
> >
> > + if (res != 3)
> >
> > + abort ();
> >
> > + res = cfc_load_2_trap (1, 2, &arr1, &arr);
> >
> > + if (res != 5)
> >
> > + abort ();
> >
> > + res = cfc_load_zero (1, 2, 3);
> >
> > + res = cfc_load_zero (1, 2, 3);
> >
> > + if (res != 0)
> >
> > + abort ();
> >
> > + res = cfc_load_mem (2, 1, &arr);
> >
> > + if (res != 0)
> >
> > + abort ();
> >
> > + res = cfc_load_arith_1 (1, 2, 3, &arr);
> >
> > + if (res != 5)
> >
> > + abort();
> >
> > + res = cfc_load_arith_2 (2, 1, 3,&arr);
> >
> > + if (res != 6)
> >
> > + abort();
> >
> > + return 0;
> >
> > +}
> >
> > +
> >
> > --
> >
> > 2.31.1
> >
> >
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2024-06-14 6:19 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <20240614005632.4088419-1-lingling.kong@intel.com>
2024-06-14 1:38 ` [PATCH 1/2] Add a new target hook: TARGET_HAVE_CONDITIONAL_MOVE_MEM_NOTRAP Kong, Lingling
[not found] ` <20240614005632.4088419-2-lingling.kong@intel.com>
2024-06-14 1:38 ` [PATCH 2/2] [APX CFCMOV] Support APX CFCMOV Kong, Lingling
2024-06-14 6:12 ` Richard Biener
2024-06-14 6:19 ` Liu, Hongtao
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).