public inbox for gcc-cvs@sourceware.org
help / color / mirror / Atom feed
* [gcc(refs/users/giulianob/heads/pfe_backport_dirty)] Backport Update x86 backend to enable Intel CET.
@ 2021-10-14 13:01 Giuliano Belinassi
0 siblings, 0 replies; only message in thread
From: Giuliano Belinassi @ 2021-10-14 13:01 UTC (permalink / raw)
To: gcc-cvs
https://gcc.gnu.org/g:629e134eded20123486bda9fdc7c7a5bad3e4030
commit 629e134eded20123486bda9fdc7c7a5bad3e4030
Author: Igor Tsimbalist <igor.v.tsimbalist@intel.com>
Date: Sat Oct 21 23:09:53 2017 +0200
Backport Update x86 backend to enable Intel CET.
All platforms except i386 will report the error and do no
instrumentation with -finstrument-control-flow option. i386
will provide the implementation based on a specification
published by Intel for a new technology called Control-flow
Enforcement Technology (CET). The spec is available at
https://software.intel.com/sites/default/files/managed/4d/2a/control-flow-enforcement-technology-preview.pdf
The implementation in this patch:
1) enables Control-flow Enforcement Technology (CET), published by
Intel. This part introduces i386 specific options -mcet, -mibt and
-mshstk, new instructions and intrinsics;
2) provides support for -fcf-protection option and 'nocf_check'
attribute by doing needed code instrumentation, which is based on
CET features.
gcc/
* common/config/i386/i386-common.c (OPTION_MASK_ISA_IBT_SET): New.
(OPTION_MASK_ISA_SHSTK_SET): Likewise.
(OPTION_MASK_ISA_IBT_UNSET): Likewise.
(OPTION_MASK_ISA_SHSTK_UNSET): Likewise.
(ix86_handle_option): Add -mibt, -mshstk, -mcet handling.
* config.gcc (extra_headers): Add cetintrin.h for x86 targets.
(extra_objs): Add cet.o for Linux/x86 targets.
(tmake_file): Add i386/t-cet for Linux/x86 targets.
* config/i386/cet.c: New file.
* config/i386/cetintrin.h: Likewise.
* config/i386/t-cet: Likewise.
* config/i386/cpuid.h (bit_SHSTK): New.
(bit_IBT): Likewise.
* config/i386/driver-i386.c (host_detect_local_cpu): Detect and
pass IBT and SHSTK bits.
* config/i386/i386-builtin-types.def
(VOID_FTYPE_UNSIGNED_PVOID): New.
(VOID_FTYPE_UINT64_PVOID): Likewise.
* config/i386/i386-builtin.def: Add CET intrinsics.
* config/i386/i386-c.c (ix86_target_macros_internal): Add
OPTION_MASK_ISA_IBT, OPTION_MASK_ISA_SHSTK handling.
* config/i386/i386-passes.def: Add pass_insert_endbranch pass.
* config/i386/i386-protos.h (make_pass_insert_endbranch): New
prototype.
* config/i386/i386.c (rest_of_insert_endbranch): New.
(pass_data_insert_endbranch): Likewise.
(pass_insert_endbranch): Likewise.
(make_pass_insert_endbranch): Likewise.
(ix86_notrack_prefixed_insn_p): Likewise.
(ix86_target_string): Add -mibt, -mshstk flags.
(ix86_option_override_internal): Add flag_cf_protection
processing.
(ix86_valid_target_attribute_inner_p): Set OPT_mibt, OPT_mshstk.
(ix86_print_operand): Add 'notrack' prefix output.
(ix86_init_mmx_sse_builtins): Add CET intrinsics.
(ix86_expand_builtin): Expand CET intrinsics.
(x86_output_mi_thunk): Add 'endbranch' instruction.
* config/i386/i386.h (TARGET_IBT): New.
(TARGET_IBT_P): Likewise.
(TARGET_SHSTK): Likewise.
(TARGET_SHSTK_P): Likewise.
* config/i386/i386.md (unspecv): Add UNSPECV_NOP_RDSSP,
UNSPECV_INCSSP, UNSPECV_SAVEPREVSSP, UNSPECV_RSTORSSP,
UNSPECV_WRSS, UNSPECV_WRUSS, UNSPECV_SETSSBSY, UNSPECV_CLRSSBSY.
(builtin_setjmp_setup): New pattern.
(builtin_longjmp): Likewise.
(rdssp<mode>): Likewise.
(incssp<mode>): Likewise.
(saveprevssp): Likewise.
(rstorssp): Likewise.
(wrss<mode>): Likewise.
(wruss<mode>): Likewise.
(setssbsy): Likewise.
(clrssbsy): Likewise.
(nop_endbr): Likewise.
* config/i386/i386.opt: Add -mcet, -mibt, -mshstk and -mcet-switch
options.
* config/i386/immintrin.h: Include <cetintrin.h>.
* config/i386/linux-common.h
(file_end_indicate_exec_stack_and_cet): New prototype.
(TARGET_ASM_FILE_END): New.
Diff:
---
gcc/ChangeLog | 67 +++++++
gcc/common/config/i386/i386-common.c | 33 ++++
gcc/config.gcc | 7 +-
gcc/config/i386/cet.c | 76 ++++++++
gcc/config/i386/cetintrin.h | 134 +++++++++++++
gcc/config/i386/cpuid.h | 2 +
gcc/config/i386/driver-i386.c | 8 +-
gcc/config/i386/i386-builtin-types.def | 2 +
gcc/config/i386/i386-builtin.def | 23 ++-
gcc/config/i386/i386-c.c | 12 ++
gcc/config/i386/i386-passes.def | 2 +
gcc/config/i386/i386-protos.h | 1 +
gcc/config/i386/i386.c | 340 ++++++++++++++++++++++++++++++++-
gcc/config/i386/i386.h | 4 +
gcc/config/i386/i386.md | 189 +++++++++++++++++-
gcc/config/i386/i386.opt | 21 ++
gcc/config/i386/immintrin.h | 2 +
gcc/config/i386/linux-common.h | 5 +
gcc/config/i386/t-cet | 21 ++
19 files changed, 933 insertions(+), 16 deletions(-)
diff --git a/gcc/ChangeLog b/gcc/ChangeLog
index 867b231c434..e661b1726af 100644
--- a/gcc/ChangeLog
+++ b/gcc/ChangeLog
@@ -1,3 +1,70 @@
+2021-10-13 Giuliano Belinassi <gbelinassi@suse.de>
+
+ Backport from newline
+ 2017-10-21 Igor Tsimbalist <igor.v.tsimbalist@intel.com>
+
+ * common/config/i386/i386-common.c (OPTION_MASK_ISA_IBT_SET): New.
+ (OPTION_MASK_ISA_SHSTK_SET): Likewise.
+ (OPTION_MASK_ISA_IBT_UNSET): Likewise.
+ (OPTION_MASK_ISA_SHSTK_UNSET): Likewise.
+ (ix86_handle_option): Add -mibt, -mshstk, -mcet handling.
+ * config.gcc (extra_headers): Add cetintrin.h for x86 targets.
+ (extra_objs): Add cet.o for Linux/x86 targets.
+ (tmake_file): Add i386/t-cet for Linux/x86 targets.
+ * config/i386/cet.c: New file.
+ * config/i386/cetintrin.h: Likewise.
+ * config/i386/t-cet: Likewise.
+ * config/i386/cpuid.h (bit_SHSTK): New.
+ (bit_IBT): Likewise.
+ * config/i386/driver-i386.c (host_detect_local_cpu): Detect and
+ pass IBT and SHSTK bits.
+ * config/i386/i386-builtin-types.def
+ (VOID_FTYPE_UNSIGNED_PVOID): New.
+ (VOID_FTYPE_UINT64_PVOID): Likewise.
+ * config/i386/i386-builtin.def: Add CET intrinsics.
+ * config/i386/i386-c.c (ix86_target_macros_internal): Add
+ OPTION_MASK_ISA_IBT, OPTION_MASK_ISA_SHSTK handling.
+ * config/i386/i386-passes.def: Add pass_insert_endbranch pass.
+ * config/i386/i386-protos.h (make_pass_insert_endbranch): New
+ prototype.
+ * config/i386/i386.c (rest_of_insert_endbranch): New.
+ (pass_data_insert_endbranch): Likewise.
+ (pass_insert_endbranch): Likewise.
+ (make_pass_insert_endbranch): Likewise.
+ (ix86_notrack_prefixed_insn_p): Likewise.
+ (ix86_target_string): Add -mibt, -mshstk flags.
+ (ix86_option_override_internal): Add flag_cf_protection
+ processing.
+ (ix86_valid_target_attribute_inner_p): Set OPT_mibt, OPT_mshstk.
+ (ix86_print_operand): Add 'notrack' prefix output.
+ (ix86_init_mmx_sse_builtins): Add CET intrinsics.
+ (ix86_expand_builtin): Expand CET intrinsics.
+ (x86_output_mi_thunk): Add 'endbranch' instruction.
+ * config/i386/i386.h (TARGET_IBT): New.
+ (TARGET_IBT_P): Likewise.
+ (TARGET_SHSTK): Likewise.
+ (TARGET_SHSTK_P): Likewise.
+ * config/i386/i386.md (unspecv): Add UNSPECV_NOP_RDSSP,
+ UNSPECV_INCSSP, UNSPECV_SAVEPREVSSP, UNSPECV_RSTORSSP,
+ UNSPECV_WRSS, UNSPECV_WRUSS, UNSPECV_SETSSBSY, UNSPECV_CLRSSBSY.
+ (builtin_setjmp_setup): New pattern.
+ (builtin_longjmp): Likewise.
+ (rdssp<mode>): Likewise.
+ (incssp<mode>): Likewise.
+ (saveprevssp): Likewise.
+ (rstorssp): Likewise.
+ (wrss<mode>): Likewise.
+ (wruss<mode>): Likewise.
+ (setssbsy): Likewise.
+ (clrssbsy): Likewise.
+ (nop_endbr): Likewise.
+ * config/i386/i386.opt: Add -mcet, -mibt, -mshstk and -mcet-switch
+ options.
+ * config/i386/immintrin.h: Include <cetintrin.h>.
+ * config/i386/linux-common.h
+ (file_end_indicate_exec_stack_and_cet): New prototype.
+ (TARGET_ASM_FILE_END): New.
+
2021-10-13 Giuliano Belinassi <gbelinassi@suse.de>
Backport from mainline
diff --git a/gcc/common/config/i386/i386-common.c b/gcc/common/config/i386/i386-common.c
index 34edcb895fe..ada918e6f2a 100644
--- a/gcc/common/config/i386/i386-common.c
+++ b/gcc/common/config/i386/i386-common.c
@@ -138,6 +138,8 @@ along with GCC; see the file COPYING3. If not see
#define OPTION_MASK_ISA_PKU_SET OPTION_MASK_ISA_PKU
#define OPTION_MASK_ISA_RDPID_SET OPTION_MASK_ISA_RDPID
#define OPTION_MASK_ISA_GFNI_SET OPTION_MASK_ISA_GFNI
+#define OPTION_MASK_ISA_IBT_SET OPTION_MASK_ISA_IBT
+#define OPTION_MASK_ISA_SHSTK_SET OPTION_MASK_ISA_SHSTK
/* Define a set of ISAs which aren't available when a given ISA is
disabled. MMX and SSE ISAs are handled separately. */
@@ -204,6 +206,8 @@ along with GCC; see the file COPYING3. If not see
#define OPTION_MASK_ISA_PKU_UNSET OPTION_MASK_ISA_PKU
#define OPTION_MASK_ISA_RDPID_UNSET OPTION_MASK_ISA_RDPID
#define OPTION_MASK_ISA_GFNI_UNSET OPTION_MASK_ISA_GFNI
+#define OPTION_MASK_ISA_IBT_UNSET OPTION_MASK_ISA_IBT
+#define OPTION_MASK_ISA_SHSTK_UNSET OPTION_MASK_ISA_SHSTK
/* SSE4 includes both SSE4.1 and SSE4.2. -mno-sse4 should the same
as -mno-sse4.1. */
@@ -499,6 +503,35 @@ ix86_handle_option (struct gcc_options *opts,
}
return true;
+ case OPT_mcet:
+ case OPT_mibt:
+ if (value)
+ {
+ opts->x_ix86_isa_flags2 |= OPTION_MASK_ISA_IBT_SET;
+ opts->x_ix86_isa_flags2_explicit |= OPTION_MASK_ISA_IBT_SET;
+ }
+ else
+ {
+ opts->x_ix86_isa_flags2 &= ~OPTION_MASK_ISA_IBT_UNSET;
+ opts->x_ix86_isa_flags2_explicit |= OPTION_MASK_ISA_IBT_UNSET;
+ }
+ if (code != OPT_mcet)
+ return true;
+ /* fall through. */
+
+ case OPT_mshstk:
+ if (value)
+ {
+ opts->x_ix86_isa_flags2 |= OPTION_MASK_ISA_SHSTK_SET;
+ opts->x_ix86_isa_flags2_explicit |= OPTION_MASK_ISA_SHSTK_SET;
+ }
+ else
+ {
+ opts->x_ix86_isa_flags2 &= ~OPTION_MASK_ISA_SHSTK_UNSET;
+ opts->x_ix86_isa_flags2_explicit |= OPTION_MASK_ISA_SHSTK_UNSET;
+ }
+ return true;
+
case OPT_mavx5124fmaps:
if (value)
{
diff --git a/gcc/config.gcc b/gcc/config.gcc
index 504b282db25..05cec9ef050 100644
--- a/gcc/config.gcc
+++ b/gcc/config.gcc
@@ -377,7 +377,7 @@ i[34567]86-*-*)
avx512ifmaintrin.h avx512ifmavlintrin.h avx512vbmiintrin.h
avx512vbmivlintrin.h avx5124fmapsintrin.h avx5124vnniwintrin.h
avx512vpopcntdqintrin.h clwbintrin.h mwaitxintrin.h
- clzerointrin.h pkuintrin.h sgxintrin.h"
+ clzerointrin.h pkuintrin.h sgxintrin.h cetintrin.h"
;;
x86_64-*-*)
cpu_type=i386
@@ -400,7 +400,7 @@ x86_64-*-*)
avx512ifmaintrin.h avx512ifmavlintrin.h avx512vbmiintrin.h
avx512vbmivlintrin.h avx5124fmapsintrin.h avx5124vnniwintrin.h
avx512vpopcntdqintrin.h clwbintrin.h mwaitxintrin.h
- clzerointrin.h pkuintrin.h sgxintrin.h"
+ clzerointrin.h pkuintrin.h sgxintrin.h cetintrin.h"
;;
ia64-*-*)
extra_headers=ia64intrin.h
@@ -4531,7 +4531,8 @@ case ${target} in
i[34567]86-*-darwin* | x86_64-*-darwin*)
;;
i[34567]86-*-linux* | x86_64-*-linux*)
- tmake_file="$tmake_file i386/t-linux"
+ extra_objs="${extra_objs} cet.o"
+ tmake_file="$tmake_file i386/t-linux i386/t-cet"
;;
i[34567]86-*-kfreebsd*-gnu | x86_64-*-kfreebsd*-gnu)
tmake_file="$tmake_file i386/t-kfreebsd"
diff --git a/gcc/config/i386/cet.c b/gcc/config/i386/cet.c
new file mode 100644
index 00000000000..a53c499fd92
--- /dev/null
+++ b/gcc/config/i386/cet.c
@@ -0,0 +1,76 @@
+/* Functions for CET/x86.
+ Copyright (C) 2017 Free Software Foundation, Inc.
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 3, or (at your option)
+any later version.
+
+GCC is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with GCC; see the file COPYING3. If not see
+<http://www.gnu.org/licenses/>. */
+
+#include "config.h"
+#include "system.h"
+#include "coretypes.h"
+#include "tm.h"
+#include "output.h"
+#include "linux-common.h"
+
+void
+file_end_indicate_exec_stack_and_cet (void)
+{
+ file_end_indicate_exec_stack ();
+
+ if (flag_cf_protection == CF_NONE)
+ return;
+
+ unsigned int feature_1 = 0;
+
+ if (TARGET_IBT)
+ /* GNU_PROPERTY_X86_FEATURE_1_IBT. */
+ feature_1 |= 0x1;
+
+ if (TARGET_SHSTK)
+ /* GNU_PROPERTY_X86_FEATURE_1_SHSTK. */
+ feature_1 |= 0x2;
+
+ if (feature_1)
+ {
+ int p2align = ptr_mode == SImode ? 2 : 3;
+
+ /* Generate GNU_PROPERTY_X86_FEATURE_1_XXX. */
+ switch_to_section (get_section (".note.gnu.property",
+ SECTION_NOTYPE, NULL));
+
+ ASM_OUTPUT_ALIGN (asm_out_file, p2align);
+ /* name length. */
+ fprintf (asm_out_file, ASM_LONG " 1f - 0f\n");
+ /* data length. */
+ fprintf (asm_out_file, ASM_LONG " 4f - 1f\n");
+ /* note type: NT_GNU_PROPERTY_TYPE_0. */
+ fprintf (asm_out_file, ASM_LONG " 5\n");
+ ASM_OUTPUT_LABEL (asm_out_file, "0");
+ /* vendor name: "GNU". */
+ fprintf (asm_out_file, STRING_ASM_OP " \"GNU\"\n");
+ ASM_OUTPUT_LABEL (asm_out_file, "1");
+ ASM_OUTPUT_ALIGN (asm_out_file, p2align);
+ /* pr_type: GNU_PROPERTY_X86_FEATURE_1_AND. */
+ fprintf (asm_out_file, ASM_LONG " 0xc0000002\n");
+ /* pr_datasz. */\
+ fprintf (asm_out_file, ASM_LONG " 3f - 2f\n");
+ ASM_OUTPUT_LABEL (asm_out_file, "2");
+ /* GNU_PROPERTY_X86_FEATURE_1_XXX. */
+ fprintf (asm_out_file, ASM_LONG " 0x%x\n", feature_1);
+ ASM_OUTPUT_LABEL (asm_out_file, "3");
+ ASM_OUTPUT_ALIGN (asm_out_file, p2align);
+ ASM_OUTPUT_LABEL (asm_out_file, "4");
+ }
+}
diff --git a/gcc/config/i386/cetintrin.h b/gcc/config/i386/cetintrin.h
new file mode 100644
index 00000000000..b15a776d7f8
--- /dev/null
+++ b/gcc/config/i386/cetintrin.h
@@ -0,0 +1,134 @@
+/* Copyright (C) 2015-2017 Free Software Foundation, Inc.
+
+ This file is part of GCC.
+
+ GCC is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 3, or (at your option)
+ any later version.
+
+ GCC is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ Under Section 7 of GPL version 3, you are granted additional
+ permissions described in the GCC Runtime Library Exception, version
+ 3.1, as published by the Free Software Foundation.
+
+ You should have received a copy of the GNU General Public License and
+ a copy of the GCC Runtime Library Exception along with this program;
+ see the files COPYING3 and COPYING.RUNTIME respectively. If not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if !defined _IMMINTRIN_H_INCLUDED
+# error "Never use <cetintrin.h> directly; include <x86intrin.h> instead."
+#endif
+
+#ifndef _CETINTRIN_H_INCLUDED
+#define _CETINTRIN_H_INCLUDED
+
+#ifndef __SHSTK__
+#pragma GCC push_options
+#pragma GCC target ("shstk")
+#define __DISABLE_SHSTK__
+#endif /* __SHSTK__ */
+
+extern __inline unsigned int
+__attribute__((__gnu_inline__, __always_inline__, __artificial__))
+_rdsspd (unsigned int __B)
+{
+ return __builtin_ia32_rdsspd (__B);
+}
+
+#ifdef __x86_64__
+extern __inline unsigned long long
+__attribute__((__gnu_inline__, __always_inline__, __artificial__))
+_rdsspq (unsigned long long __B)
+{
+ return __builtin_ia32_rdsspq (__B);
+}
+#endif
+
+extern __inline void
+__attribute__((__gnu_inline__, __always_inline__, __artificial__))
+_incsspd (unsigned int __B)
+{
+ __builtin_ia32_incsspd (__B);
+}
+
+#ifdef __x86_64__
+extern __inline void
+__attribute__((__gnu_inline__, __always_inline__, __artificial__))
+_incsspq (unsigned long long __B)
+{
+ __builtin_ia32_incsspq (__B);
+}
+#endif
+
+extern __inline void
+__attribute__((__gnu_inline__, __always_inline__, __artificial__))
+_saveprevssp (void)
+{
+ __builtin_ia32_saveprevssp ();
+}
+
+extern __inline void
+__attribute__((__gnu_inline__, __always_inline__, __artificial__))
+_rstorssp (void *__B)
+{
+ __builtin_ia32_rstorssp (__B);
+}
+
+extern __inline void
+__attribute__((__gnu_inline__, __always_inline__, __artificial__))
+_wrssd (unsigned int __B, void *__C)
+{
+ __builtin_ia32_wrssd (__B, __C);
+}
+
+#ifdef __x86_64__
+extern __inline void
+__attribute__((__gnu_inline__, __always_inline__, __artificial__))
+_wrssq (unsigned long long __B, void *__C)
+{
+ __builtin_ia32_wrssq (__B, __C);
+}
+#endif
+
+extern __inline void
+__attribute__((__gnu_inline__, __always_inline__, __artificial__))
+_wrussd (unsigned int __B, void *__C)
+{
+ __builtin_ia32_wrussd (__B, __C);
+}
+
+#ifdef __x86_64__
+extern __inline void
+__attribute__((__gnu_inline__, __always_inline__, __artificial__))
+_wrussq (unsigned long long __B, void *__C)
+{
+ __builtin_ia32_wrussq (__B, __C);
+}
+#endif
+
+extern __inline void
+__attribute__((__gnu_inline__, __always_inline__, __artificial__))
+_setssbsy (void)
+{
+ __builtin_ia32_setssbsy ();
+}
+
+extern __inline void
+__attribute__((__gnu_inline__, __always_inline__, __artificial__))
+_clrssbsy (void *__B)
+{
+ __builtin_ia32_clrssbsy (__B);
+}
+
+#ifdef __DISABLE_SHSTK__
+#undef __DISABLE_SHSTK__
+#pragma GCC pop_options
+#endif /* __DISABLE_SHSTK__ */
+
+#endif /* _CETINTRIN_H_INCLUDED. */
diff --git a/gcc/config/i386/cpuid.h b/gcc/config/i386/cpuid.h
index 61ee017003c..60d64ea571c 100644
--- a/gcc/config/i386/cpuid.h
+++ b/gcc/config/i386/cpuid.h
@@ -97,6 +97,7 @@
#define bit_AVX512VBMI (1 << 1)
#define bit_PKU (1 << 3)
#define bit_OSPKE (1 << 4)
+#define bit_SHSTK (1 << 7)
#define bit_GFNI (1 << 8)
#define bit_AVX512VPOPCNTDQ (1 << 14)
#define bit_RDPID (1 << 22)
@@ -104,6 +105,7 @@
/* %edx */
#define bit_AVX5124VNNIW (1 << 2)
#define bit_AVX5124FMAPS (1 << 3)
+#define bit_IBT (1 << 20)
/* XFEATURE_ENABLED_MASK register bits (%eax == 13, %ecx == 0) */
#define bit_BNDREGS (1 << 3)
diff --git a/gcc/config/i386/driver-i386.c b/gcc/config/i386/driver-i386.c
index 63ab12f3487..6d37492367c 100644
--- a/gcc/config/i386/driver-i386.c
+++ b/gcc/config/i386/driver-i386.c
@@ -416,6 +416,7 @@ const char *host_detect_local_cpu (int argc, const char **argv)
unsigned int has_mwaitx = 0, has_clzero = 0, has_pku = 0, has_rdpid = 0;
unsigned int has_avx5124fmaps = 0, has_avx5124vnniw = 0;
unsigned int has_gfni = 0;
+ unsigned int has_ibt = 0, has_shstk = 0;
bool arch;
@@ -509,6 +510,9 @@ const char *host_detect_local_cpu (int argc, const char **argv)
has_avx5124vnniw = edx & bit_AVX5124VNNIW;
has_avx5124fmaps = edx & bit_AVX5124FMAPS;
+
+ has_shstk = ecx & bit_SHSTK;
+ has_ibt = edx & bit_IBT;
}
if (max_level >= 13)
@@ -1038,6 +1042,8 @@ const char *host_detect_local_cpu (int argc, const char **argv)
const char *pku = has_pku ? " -mpku" : " -mno-pku";
const char *rdpid = has_rdpid ? " -mrdpid" : " -mno-rdpid";
const char *gfni = has_gfni ? " -mgfni" : " -mno-gfni";
+ const char *ibt = has_ibt ? " -mibt" : " -mno-ibt";
+ const char *shstk = has_shstk ? " -mshstk" : " -mno-shstk";
options = concat (options, mmx, mmx3dnow, sse, sse2, sse3, ssse3,
sse4a, cx16, sahf, movbe, aes, sha, pclmul,
popcnt, abm, lwp, fma, fma4, xop, bmi, sgx, bmi2,
@@ -1047,7 +1053,7 @@ const char *host_detect_local_cpu (int argc, const char **argv)
avx512cd, avx512pf, prefetchwt1, clflushopt,
xsavec, xsaves, avx512dq, avx512bw, avx512vl,
avx512ifma, avx512vbmi, avx5124fmaps, avx5124vnniw,
- clwb, mwaitx, clzero, pku, rdpid, gfni, NULL);
+ clwb, mwaitx, clzero, pku, rdpid, gfni, ibt, shstk, NULL);
}
done:
diff --git a/gcc/config/i386/i386-builtin-types.def b/gcc/config/i386/i386-builtin-types.def
index 1334633c65f..399e659c2cd 100644
--- a/gcc/config/i386/i386-builtin-types.def
+++ b/gcc/config/i386/i386-builtin-types.def
@@ -285,7 +285,9 @@ DEF_FUNCTION_TYPE (V8SI, V8SI)
DEF_FUNCTION_TYPE (VOID, PCVOID)
DEF_FUNCTION_TYPE (VOID, PVOID)
DEF_FUNCTION_TYPE (VOID, UINT64)
+DEF_FUNCTION_TYPE (VOID, UINT64, PVOID)
DEF_FUNCTION_TYPE (VOID, UNSIGNED)
+DEF_FUNCTION_TYPE (VOID, UNSIGNED, PVOID)
DEF_FUNCTION_TYPE (INT, PUSHORT)
DEF_FUNCTION_TYPE (INT, PUNSIGNED)
DEF_FUNCTION_TYPE (INT, PULONGLONG)
diff --git a/gcc/config/i386/i386-builtin.def b/gcc/config/i386/i386-builtin.def
index 35ab29204db..ed720e12275 100644
--- a/gcc/config/i386/i386-builtin.def
+++ b/gcc/config/i386/i386-builtin.def
@@ -2748,4 +2748,25 @@ BDESC (OPTION_MASK_ISA_XOP, CODE_FOR_xop_vpermil2v4sf3, "__builtin_ia32_vper
BDESC (OPTION_MASK_ISA_XOP, CODE_FOR_xop_vpermil2v4df3, "__builtin_ia32_vpermil2pd256", IX86_BUILTIN_VPERMIL2PD256, UNKNOWN, (int)MULTI_ARG_4_DF2_DI_I1)
BDESC (OPTION_MASK_ISA_XOP, CODE_FOR_xop_vpermil2v8sf3, "__builtin_ia32_vpermil2ps256", IX86_BUILTIN_VPERMIL2PS256, UNKNOWN, (int)MULTI_ARG_4_SF2_SI_I1)
-BDESC_END (MULTI_ARG, MAX)
+BDESC_END (MULTI_ARG, CET)
+
+/* CET. */
+BDESC_FIRST (cet, CET,
+ OPTION_MASK_ISA_SHSTK, CODE_FOR_incsspsi, "__builtin_ia32_incsspd", IX86_BUILTIN_INCSSPD, UNKNOWN, (int) VOID_FTYPE_UNSIGNED)
+BDESC (OPTION_MASK_ISA_SHSTK | OPTION_MASK_ISA_64BIT, CODE_FOR_incsspdi, "__builtin_ia32_incsspq", IX86_BUILTIN_INCSSPQ, UNKNOWN, (int) VOID_FTYPE_UINT64)
+BDESC (OPTION_MASK_ISA_SHSTK, CODE_FOR_saveprevssp, "__builtin_ia32_saveprevssp", IX86_BUILTIN_SAVEPREVSSP, UNKNOWN, (int) VOID_FTYPE_VOID)
+BDESC (OPTION_MASK_ISA_SHSTK, CODE_FOR_rstorssp, "__builtin_ia32_rstorssp", IX86_BUILTIN_RSTORSSP, UNKNOWN, (int) VOID_FTYPE_PVOID)
+BDESC (OPTION_MASK_ISA_SHSTK, CODE_FOR_wrsssi, "__builtin_ia32_wrssd", IX86_BUILTIN_WRSSD, UNKNOWN, (int) VOID_FTYPE_UNSIGNED_PVOID)
+BDESC (OPTION_MASK_ISA_SHSTK | OPTION_MASK_ISA_64BIT, CODE_FOR_wrssdi, "__builtin_ia32_wrssq", IX86_BUILTIN_WRSSQ, UNKNOWN, (int) VOID_FTYPE_UINT64_PVOID)
+BDESC (OPTION_MASK_ISA_SHSTK, CODE_FOR_wrusssi, "__builtin_ia32_wrussd", IX86_BUILTIN_WRUSSD, UNKNOWN, (int) VOID_FTYPE_UNSIGNED_PVOID)
+BDESC (OPTION_MASK_ISA_SHSTK | OPTION_MASK_ISA_64BIT, CODE_FOR_wrussdi, "__builtin_ia32_wrussq", IX86_BUILTIN_WRUSSQ, UNKNOWN, (int) VOID_FTYPE_UINT64_PVOID)
+BDESC (OPTION_MASK_ISA_SHSTK, CODE_FOR_setssbsy, "__builtin_ia32_setssbsy", IX86_BUILTIN_SETSSBSY, UNKNOWN, (int) VOID_FTYPE_VOID)
+BDESC (OPTION_MASK_ISA_SHSTK, CODE_FOR_clrssbsy, "__builtin_ia32_clrssbsy", IX86_BUILTIN_CLRSSBSY, UNKNOWN, (int) VOID_FTYPE_PVOID)
+
+BDESC_END (CET, CET_NORMAL)
+
+BDESC_FIRST (cet_rdssp, CET_NORMAL,
+ OPTION_MASK_ISA_SHSTK, CODE_FOR_rdsspsi, "__builtin_ia32_rdsspd", IX86_BUILTIN_RDSSPD, UNKNOWN, (int) UINT_FTYPE_UINT)
+BDESC (OPTION_MASK_ISA_SHSTK | OPTION_MASK_ISA_64BIT, CODE_FOR_rdsspdi, "__builtin_ia32_rdsspq", IX86_BUILTIN_RDSSPQ, UNKNOWN, (int) UINT64_FTYPE_UINT64)
+
+BDESC_END (CET_NORMAL, MAX)
diff --git a/gcc/config/i386/i386-c.c b/gcc/config/i386/i386-c.c
index 8118599b109..0d29b9d4784 100644
--- a/gcc/config/i386/i386-c.c
+++ b/gcc/config/i386/i386-c.c
@@ -452,6 +452,18 @@ ix86_target_macros_internal (HOST_WIDE_INT isa_flag,
def_or_undef (parse_in, "__RDPID__");
if (isa_flag2 & OPTION_MASK_ISA_GFNI)
def_or_undef (parse_in, "__GFNI__");
+ if (isa_flag2 & OPTION_MASK_ISA_IBT)
+ {
+ def_or_undef (parse_in, "__IBT__");
+ if (flag_cf_protection != CF_NONE)
+ def_or_undef (parse_in, "__CET__");
+ }
+ if (isa_flag2 & OPTION_MASK_ISA_SHSTK)
+ {
+ def_or_undef (parse_in, "__SHSTK__");
+ if (flag_cf_protection != CF_NONE)
+ def_or_undef (parse_in, "__CET__");
+ }
if (TARGET_IAMCU)
{
def_or_undef (parse_in, "__iamcu");
diff --git a/gcc/config/i386/i386-passes.def b/gcc/config/i386/i386-passes.def
index 49534619221..5c6e9c3494e 100644
--- a/gcc/config/i386/i386-passes.def
+++ b/gcc/config/i386/i386-passes.def
@@ -29,3 +29,5 @@ along with GCC; see the file COPYING3. If not see
/* Run the 64-bit STV pass before the CSE pass so that CONST0_RTX and
CONSTM1_RTX generated by the STV pass can be CSEed. */
INSERT_PASS_BEFORE (pass_cse2, 1, pass_stv, true /* timode_p */);
+
+ INSERT_PASS_BEFORE (pass_shorten_branches, 1, pass_insert_endbranch);
diff --git a/gcc/config/i386/i386-protos.h b/gcc/config/i386/i386-protos.h
index 9434e8463c1..98d815e436c 100644
--- a/gcc/config/i386/i386-protos.h
+++ b/gcc/config/i386/i386-protos.h
@@ -346,3 +346,4 @@ class rtl_opt_pass;
extern rtl_opt_pass *make_pass_insert_vzeroupper (gcc::context *);
extern rtl_opt_pass *make_pass_stv (gcc::context *);
+extern rtl_opt_pass *make_pass_insert_endbranch (gcc::context *);
diff --git a/gcc/config/i386/i386.c b/gcc/config/i386/i386.c
index d0308c51e14..26a29c34455 100644
--- a/gcc/config/i386/i386.c
+++ b/gcc/config/i386/i386.c
@@ -96,6 +96,9 @@ static rtx legitimize_dllimport_symbol (rtx, bool);
static rtx legitimize_pe_coff_extern_decl (rtx, bool);
static rtx legitimize_pe_coff_symbol (rtx, bool);
static void ix86_print_operand_address_as (FILE *, rtx, addr_space_t, bool);
+static bool ix86_save_reg (unsigned int, bool, bool);
+static bool ix86_function_naked (const_tree);
+static bool ix86_notrack_prefixed_insn_p (rtx);
#ifndef CHECK_STACK_LIMIT
#define CHECK_STACK_LIMIT (-1)
@@ -4212,15 +4215,151 @@ make_pass_stv (gcc::context *ctxt)
return new pass_stv (ctxt);
}
-/* Return true if a red-zone is in use. We can't use red-zone when
- there are local indirect jumps, like "indirect_jump" or "tablejump",
- which jumps to another place in the function, since "call" in the
- indirect thunk pushes the return address onto stack, destroying
- red-zone.
+/* Inserting ENDBRANCH instructions. */
- TODO: If we can reserve the first 2 WORDs, for PUSH and, another
- for CALL, in red-zone, we can allow local indirect jumps with
- indirect thunk. */
+static unsigned int
+rest_of_insert_endbranch (void)
+{
+ timevar_push (TV_MACH_DEP);
+
+ rtx cet_eb;
+ rtx_insn *insn;
+ basic_block bb;
+
+ /* Currently emit EB if it's a tracking function, i.e. 'nocf_check' is
+ absent among function attributes. Later an optimization will be
+ introduced to make analysis if an address of a static function is
+ taken. A static function whose address is not taken will get a
+ nocf_check attribute. This will allow to reduce the number of EB. */
+
+ if (!lookup_attribute ("nocf_check",
+ TYPE_ATTRIBUTES (TREE_TYPE (cfun->decl))))
+ {
+ cet_eb = gen_nop_endbr ();
+
+ bb = ENTRY_BLOCK_PTR_FOR_FN (cfun)->next_bb;
+ insn = BB_HEAD (bb);
+ emit_insn_before (cet_eb, insn);
+ }
+
+ bb = 0;
+ FOR_EACH_BB_FN (bb, cfun)
+ {
+ for (insn = BB_HEAD (bb); insn != NEXT_INSN (BB_END (bb));
+ insn = NEXT_INSN (insn))
+ {
+ if (INSN_P (insn) && GET_CODE (insn) == CALL_INSN)
+ {
+ rtx_insn *next_insn = insn;
+
+ while ((next_insn != BB_END (bb))
+ && (DEBUG_INSN_P (NEXT_INSN (next_insn))
+ || NOTE_P (NEXT_INSN (next_insn))
+ || BARRIER_P (NEXT_INSN (next_insn))))
+ next_insn = NEXT_INSN (next_insn);
+
+ /* Generate ENDBRANCH after CALL, which can return more than
+ twice, setjmp-like functions. */
+ if (find_reg_note (insn, REG_SETJMP, NULL) != NULL)
+ {
+ cet_eb = gen_nop_endbr ();
+ emit_insn_after (cet_eb, next_insn);
+ }
+ continue;
+ }
+
+ if (INSN_P (insn) && JUMP_P (insn) && flag_cet_switch)
+ {
+ rtx target = JUMP_LABEL (insn);
+ if (target == NULL_RTX || ANY_RETURN_P (target))
+ continue;
+
+ /* Check the jump is a switch table. */
+ rtx_insn *label = as_a<rtx_insn *> (target);
+ rtx_insn *table = next_insn (label);
+ if (table == NULL_RTX || !JUMP_TABLE_DATA_P (table))
+ continue;
+
+ /* For the indirect jump find out all places it jumps and insert
+ ENDBRANCH there. It should be done under a special flag to
+ control ENDBRANCH generation for switch stmts. */
+ edge_iterator ei;
+ edge e;
+ basic_block dest_blk;
+
+ FOR_EACH_EDGE (e, ei, bb->succs)
+ {
+ rtx_insn *insn;
+
+ dest_blk = e->dest;
+ insn = BB_HEAD (dest_blk);
+ gcc_assert (LABEL_P (insn));
+ cet_eb = gen_nop_endbr ();
+ emit_insn_after (cet_eb, insn);
+ }
+ continue;
+ }
+
+ if ((LABEL_P (insn) && LABEL_PRESERVE_P (insn))
+ || (NOTE_P (insn)
+ && NOTE_KIND (insn) == NOTE_INSN_DELETED_LABEL))
+/* TODO. Check /s bit also. */
+ {
+ cet_eb = gen_nop_endbr ();
+ emit_insn_after (cet_eb, insn);
+ continue;
+ }
+ }
+ }
+
+ timevar_pop (TV_MACH_DEP);
+ return 0;
+}
+
+namespace {
+
+const pass_data pass_data_insert_endbranch =
+{
+ RTL_PASS, /* type. */
+ "cet", /* name. */
+ OPTGROUP_NONE, /* optinfo_flags. */
+ TV_MACH_DEP, /* tv_id. */
+ 0, /* properties_required. */
+ 0, /* properties_provided. */
+ 0, /* properties_destroyed. */
+ 0, /* todo_flags_start. */
+ 0, /* todo_flags_finish. */
+};
+
+class pass_insert_endbranch : public rtl_opt_pass
+{
+public:
+ pass_insert_endbranch (gcc::context *ctxt)
+ : rtl_opt_pass (pass_data_insert_endbranch, ctxt)
+ {}
+
+ /* opt_pass methods: */
+ virtual bool gate (function *)
+ {
+ return ((flag_cf_protection & CF_BRANCH) && TARGET_IBT);
+ }
+
+ virtual unsigned int execute (function *)
+ {
+ return rest_of_insert_endbranch ();
+ }
+
+}; // class pass_insert_endbranch
+
+} // anon namespace
+
+rtl_opt_pass *
+make_pass_insert_endbranch (gcc::context *ctxt)
+{
+ return new pass_insert_endbranch (ctxt);
+}
+
+/* Return true if a red-zone is in use. */
bool
ix86_using_red_zone (void)
@@ -4255,7 +4394,9 @@ ix86_target_string (HOST_WIDE_INT isa, HOST_WIDE_INT isa2,
{ "-msgx", OPTION_MASK_ISA_SGX },
{ "-mavx5124vnniw", OPTION_MASK_ISA_AVX5124VNNIW },
{ "-mavx5124fmaps", OPTION_MASK_ISA_AVX5124FMAPS },
- { "-mavx512vpopcntdq", OPTION_MASK_ISA_AVX512VPOPCNTDQ }
+ { "-mavx512vpopcntdq", OPTION_MASK_ISA_AVX512VPOPCNTDQ },
+ { "-mibt", OPTION_MASK_ISA_IBT },
+ { "-mshstk", OPTION_MASK_ISA_SHSTK }
};
static struct ix86_target_opts isa_opts[] =
{
@@ -6289,6 +6430,37 @@ ix86_option_override_internal (bool main_args_p,
&& !opts_set->x_flag_jump_tables)
opts->x_flag_jump_tables = 0;
+ /* Do not support control flow instrumentation if CET is not enabled. */
+ if (opts->x_flag_cf_protection != CF_NONE)
+ {
+ if (!(TARGET_IBT_P (opts->x_ix86_isa_flags2)
+ || TARGET_SHSTK_P (opts->x_ix86_isa_flags2)))
+ {
+ if (flag_cf_protection == CF_FULL)
+ {
+ error ("%<-fcf-protection=full%> requires CET support "
+ "on this target. Use -mcet or one of -mibt, "
+ "-mshstk options to enable CET");
+ }
+ else if (flag_cf_protection == CF_BRANCH)
+ {
+ error ("%<-fcf-protection=branch%> requires CET support "
+ "on this target. Use -mcet or one of -mibt, "
+ "-mshstk options to enable CET");
+ }
+ else if (flag_cf_protection == CF_RETURN)
+ {
+ error ("%<-fcf-protection=return%> requires CET support "
+ "on this target. Use -mcet or one of -mibt, "
+ "-mshstk options to enable CET");
+ }
+ flag_cf_protection = CF_NONE;
+ return false;
+ }
+ opts->x_flag_cf_protection =
+ (cf_protection_level) (opts->x_flag_cf_protection | CF_SET);
+ }
+
return true;
}
@@ -6695,6 +6867,8 @@ ix86_valid_target_attribute_inner_p (tree args, char *p_strings[],
IX86_ATTR_ISA ("clwb", OPT_mclwb),
IX86_ATTR_ISA ("rdpid", OPT_mrdpid),
IX86_ATTR_ISA ("gfni", OPT_mgfni),
+ IX86_ATTR_ISA ("ibt", OPT_mibt),
+ IX86_ATTR_ISA ("shstk", OPT_mshstk),
/* enum options */
IX86_ATTR_ENUM ("fpmath=", OPT_mfpmath_),
@@ -18769,6 +18943,8 @@ ix86_print_operand (FILE *file, rtx x, int code)
case '!':
if (ix86_bnd_prefixed_insn_p (current_output_insn))
fputs ("bnd ", file);
+ if (ix86_notrack_prefixed_insn_p (current_output_insn))
+ fputs ("notrack ", file);
return;
default:
@@ -32324,8 +32500,12 @@ BDESC_VERIFYS (IX86_BUILTIN__BDESC_MPX_CONST_FIRST,
IX86_BUILTIN__BDESC_MPX_LAST, 1);
BDESC_VERIFYS (IX86_BUILTIN__BDESC_MULTI_ARG_FIRST,
IX86_BUILTIN__BDESC_MPX_CONST_LAST, 1);
-BDESC_VERIFYS (IX86_BUILTIN_MAX,
+BDESC_VERIFYS (IX86_BUILTIN__BDESC_CET_FIRST,
IX86_BUILTIN__BDESC_MULTI_ARG_LAST, 1);
+BDESC_VERIFYS (IX86_BUILTIN__BDESC_CET_NORMAL_FIRST,
+ IX86_BUILTIN__BDESC_CET_LAST, 1);
+BDESC_VERIFYS (IX86_BUILTIN_MAX,
+ IX86_BUILTIN__BDESC_CET_NORMAL_LAST, 1);
/* Set up all the MMX/SSE builtins, even builtins for instructions that are not
in the current target ISA to allow the user to compile particular modules
@@ -32986,6 +33166,35 @@ ix86_init_mmx_sse_builtins (void)
BDESC_VERIFYS (IX86_BUILTIN__BDESC_MULTI_ARG_LAST,
IX86_BUILTIN__BDESC_MULTI_ARG_FIRST,
ARRAY_SIZE (bdesc_multi_arg) - 1);
+
+ /* Add CET inrinsics. */
+ for (i = 0, d = bdesc_cet; i < ARRAY_SIZE (bdesc_cet); i++, d++)
+ {
+ BDESC_VERIFY (d->code, IX86_BUILTIN__BDESC_CET_FIRST, i);
+ if (d->name == 0)
+ continue;
+
+ ftype = (enum ix86_builtin_func_type) d->flag;
+ def_builtin2 (d->mask, d->name, ftype, d->code);
+ }
+ BDESC_VERIFYS (IX86_BUILTIN__BDESC_CET_LAST,
+ IX86_BUILTIN__BDESC_CET_FIRST,
+ ARRAY_SIZE (bdesc_cet) - 1);
+
+ for (i = 0, d = bdesc_cet_rdssp;
+ i < ARRAY_SIZE (bdesc_cet_rdssp);
+ i++, d++)
+ {
+ BDESC_VERIFY (d->code, IX86_BUILTIN__BDESC_CET_NORMAL_FIRST, i);
+ if (d->name == 0)
+ continue;
+
+ ftype = (enum ix86_builtin_func_type) d->flag;
+ def_builtin2 (d->mask, d->name, ftype, d->code);
+ }
+ BDESC_VERIFYS (IX86_BUILTIN__BDESC_CET_NORMAL_LAST,
+ IX86_BUILTIN__BDESC_CET_NORMAL_FIRST,
+ ARRAY_SIZE (bdesc_cet_rdssp) - 1);
}
static void
@@ -39393,6 +39602,57 @@ rdseed_step:
emit_insn (gen_xabort (op0));
return 0;
+ case IX86_BUILTIN_RSTORSSP:
+ case IX86_BUILTIN_CLRSSBSY:
+ arg0 = CALL_EXPR_ARG (exp, 0);
+ op0 = expand_normal (arg0);
+ icode = (fcode == IX86_BUILTIN_RSTORSSP
+ ? CODE_FOR_rstorssp
+ : CODE_FOR_clrssbsy);
+ if (!address_operand (op0, VOIDmode))
+ {
+ op1 = convert_memory_address (Pmode, op0);
+ op0 = copy_addr_to_reg (op1);
+ }
+ emit_insn (GEN_FCN (icode) (gen_rtx_MEM (Pmode, op0)));
+ return 0;
+
+ case IX86_BUILTIN_WRSSD:
+ case IX86_BUILTIN_WRSSQ:
+ case IX86_BUILTIN_WRUSSD:
+ case IX86_BUILTIN_WRUSSQ:
+ arg0 = CALL_EXPR_ARG (exp, 0);
+ op0 = expand_normal (arg0);
+ arg1 = CALL_EXPR_ARG (exp, 1);
+ op1 = expand_normal (arg1);
+ switch (fcode)
+ {
+ case IX86_BUILTIN_WRSSD:
+ icode = CODE_FOR_wrsssi;
+ mode = SImode;
+ break;
+ case IX86_BUILTIN_WRSSQ:
+ icode = CODE_FOR_wrssdi;
+ mode = DImode;
+ break;
+ case IX86_BUILTIN_WRUSSD:
+ icode = CODE_FOR_wrusssi;
+ mode = SImode;
+ break;
+ case IX86_BUILTIN_WRUSSQ:
+ icode = CODE_FOR_wrussdi;
+ mode = DImode;
+ break;
+ }
+ op0 = force_reg (mode, op0);
+ if (!address_operand (op1, VOIDmode))
+ {
+ op2 = convert_memory_address (Pmode, op1);
+ op1 = copy_addr_to_reg (op2);
+ }
+ emit_insn (GEN_FCN (icode) (op0, gen_rtx_MEM (mode, op1)));
+ return 0;
+
default:
break;
}
@@ -39695,6 +39955,22 @@ s4fma_expand:
d->flag, d->comparison);
}
+ if (fcode >= IX86_BUILTIN__BDESC_CET_FIRST
+ && fcode <= IX86_BUILTIN__BDESC_CET_LAST)
+ {
+ i = fcode - IX86_BUILTIN__BDESC_CET_FIRST;
+ return ix86_expand_special_args_builtin (bdesc_cet + i, exp,
+ target);
+ }
+
+ if (fcode >= IX86_BUILTIN__BDESC_CET_NORMAL_FIRST
+ && fcode <= IX86_BUILTIN__BDESC_CET_NORMAL_LAST)
+ {
+ i = fcode - IX86_BUILTIN__BDESC_CET_NORMAL_FIRST;
+ return ix86_expand_args_builtin (bdesc_cet_rdssp + i, exp,
+ target);
+ }
+
gcc_unreachable ();
}
@@ -42526,6 +42802,10 @@ x86_output_mi_thunk (FILE *file, tree, HOST_WIDE_INT delta,
emit_note (NOTE_INSN_PROLOGUE_END);
+ /* CET is enabled, insert EB instruction. */
+ if ((flag_cf_protection & CF_BRANCH) && TARGET_IBT)
+ emit_insn (gen_nop_endbr ());
+
/* If VCALL_OFFSET, we'll need THIS in a register. Might as well
pull it in now and let DELTA benefit. */
if (REG_P (this_param))
@@ -50269,6 +50549,46 @@ ix86_bnd_prefixed_insn_p (rtx insn)
return chkp_function_instrumented_p (current_function_decl);
}
+/* Return 1 if control tansfer instruction INSN
+ should be encoded with notrack prefix. */
+
+static bool
+ix86_notrack_prefixed_insn_p (rtx insn)
+{
+ if (!insn || !((flag_cf_protection & CF_BRANCH) && TARGET_IBT))
+ return false;
+
+ if (CALL_P (insn))
+ {
+ rtx call = get_call_rtx_from (insn);
+ gcc_assert (call != NULL_RTX);
+ rtx addr = XEXP (call, 0);
+
+ /* Do not emit 'notrack' if it's not an indirect call. */
+ if (MEM_P (addr)
+ && GET_CODE (XEXP (addr, 0)) == SYMBOL_REF)
+ return false;
+ else
+ return find_reg_note (insn, REG_CALL_NOCF_CHECK, 0);
+ }
+
+ if (JUMP_P (insn) && !flag_cet_switch)
+ {
+ rtx target = JUMP_LABEL (insn);
+ if (target == NULL_RTX || ANY_RETURN_P (target))
+ return false;
+
+ /* Check the jump is a switch table. */
+ rtx_insn *label = as_a<rtx_insn *> (target);
+ rtx_insn *table = next_insn (label);
+ if (table == NULL_RTX || !JUMP_TABLE_DATA_P (table))
+ return false;
+ else
+ return true;
+ }
+ return false;
+}
+
/* Calculate integer abs() using only SSE2 instructions. */
void
diff --git a/gcc/config/i386/i386.h b/gcc/config/i386/i386.h
index d4b14195d92..ffd554b2640 100644
--- a/gcc/config/i386/i386.h
+++ b/gcc/config/i386/i386.h
@@ -170,6 +170,10 @@ see the files COPYING3 and COPYING.RUNTIME respectively. If not, see
#define TARGET_MWAITX_P(x) TARGET_ISA_MWAITX_P(x)
#define TARGET_PKU TARGET_ISA_PKU
#define TARGET_PKU_P(x) TARGET_ISA_PKU_P(x)
+#define TARGET_IBT TARGET_ISA_IBT
+#define TARGET_IBT_P(x) TARGET_ISA_IBT_P(x)
+#define TARGET_SHSTK TARGET_ISA_SHSTK
+#define TARGET_SHSTK_P(x) TARGET_ISA_SHSTK_P(x)
#define TARGET_LP64 TARGET_ABI_64
#define TARGET_LP64_P(x) TARGET_ABI_64_P(x)
diff --git a/gcc/config/i386/i386.md b/gcc/config/i386/i386.md
index c36023f2c4d..49a37209f0b 100644
--- a/gcc/config/i386/i386.md
+++ b/gcc/config/i386/i386.md
@@ -63,7 +63,7 @@
;; ~ -- print "i" if TARGET_AVX2, "f" otherwise.
;; @ -- print a segment register of thread base pointer load
;; ^ -- print addr32 prefix if TARGET_64BIT and Pmode != word_mode
-;; ! -- print MPX prefix for jxx/call/ret instructions if required.
+;; ! -- print MPX or NOTRACK prefix for jxx/call/ret instructions if required.
(define_c_enum "unspec" [
;; Relocation specifiers
@@ -276,6 +276,17 @@
;; For RDPID support
UNSPECV_RDPID
+
+ ;; For CET support
+ UNSPECV_NOP_ENDBR
+ UNSPECV_NOP_RDSSP
+ UNSPECV_INCSSP
+ UNSPECV_SAVEPREVSSP
+ UNSPECV_RSTORSSP
+ UNSPECV_WRSS
+ UNSPECV_WRUSS
+ UNSPECV_SETSSBSY
+ UNSPECV_CLRSSBSY
])
;; Constants to represent rounding modes in the ROUND instruction
@@ -17690,6 +17701,28 @@
"* return output_probe_stack_range (operands[0], operands[2]);"
[(set_attr "type" "multi")])
+/* Additional processing for builtin_setjmp. Store the shadow stack pointer
+ as a forth element in jmpbuf. */
+(define_expand "builtin_setjmp_setup"
+ [(match_operand 0 "address_operand")]
+ "TARGET_SHSTK"
+{
+ if (flag_cf_protection & CF_RETURN)
+ {
+ rtx mem, reg_ssp;
+
+ mem = gen_rtx_MEM (Pmode, plus_constant (Pmode, operands[0],
+ 3 * GET_MODE_SIZE (Pmode)));
+ reg_ssp = gen_reg_rtx (Pmode);
+ emit_insn (gen_rtx_SET (reg_ssp, const0_rtx));
+ emit_insn ((Pmode == SImode)
+ ? gen_rdsspsi (reg_ssp, reg_ssp)
+ : gen_rdsspdi (reg_ssp, reg_ssp));
+ emit_move_insn (mem, reg_ssp);
+ }
+ DONE;
+})
+
(define_expand "builtin_setjmp_receiver"
[(label_ref (match_operand 0))]
"!TARGET_64BIT && flag_pic"
@@ -17710,6 +17743,83 @@
DONE;
})
+(define_expand "builtin_longjmp"
+ [(match_operand 0 "address_operand")]
+ "TARGET_SHSTK"
+{
+ rtx fp, lab, stack;
+ rtx jump, label, reg_adj, reg_ssp, reg_minus, mem_buf, tmp, clob;
+ machine_mode sa_mode = STACK_SAVEAREA_MODE (SAVE_NONLOCAL);
+
+ /* Adjust the shadow stack pointer (ssp) to the value saved in the
+ jmp_buf. The saving was done in the builtin_setjmp_setup. */
+ if (flag_cf_protection & CF_RETURN)
+ {
+ /* Get current shadow stack pointer. The code below will check if
+ SHSTK feature is enabled. If it's not enabled RDSSP instruction
+ is a NOP. */
+ reg_ssp = gen_reg_rtx (Pmode);
+ emit_insn (gen_rtx_SET (reg_ssp, const0_rtx));
+ emit_insn ((Pmode == SImode)
+ ? gen_rdsspsi (reg_ssp, reg_ssp)
+ : gen_rdsspdi (reg_ssp, reg_ssp));
+ mem_buf = gen_rtx_MEM (Pmode, plus_constant (Pmode, operands[0],
+ 3 * GET_MODE_SIZE (Pmode))),
+
+ /* Compare through substraction the saved and the current ssp to decide
+ if ssp has to be adjusted. */
+ reg_minus = gen_reg_rtx (Pmode);
+ tmp = gen_rtx_SET (reg_minus, gen_rtx_MINUS (Pmode, reg_ssp, mem_buf));
+ clob = gen_rtx_CLOBBER (VOIDmode, gen_rtx_REG (CCmode, FLAGS_REG));
+ tmp = gen_rtx_PARALLEL (VOIDmode, gen_rtvec (2, tmp, clob));
+ emit_insn (tmp);
+
+ /* Jump over adjustment code. */
+ label = gen_label_rtx ();
+ tmp = gen_rtx_REG (CCmode, FLAGS_REG);
+ tmp = gen_rtx_EQ (VOIDmode, tmp, const0_rtx);
+ tmp = gen_rtx_IF_THEN_ELSE (VOIDmode, tmp,
+ gen_rtx_LABEL_REF (VOIDmode, label),
+ pc_rtx);
+ jump = emit_jump_insn (gen_rtx_SET (pc_rtx, tmp));
+ JUMP_LABEL (jump) = label;
+
+ /* Adjust the ssp. */
+ reg_adj = gen_reg_rtx (Pmode);
+ tmp = gen_rtx_SET (reg_adj,
+ gen_rtx_LSHIFTRT (Pmode, negate_rtx (Pmode, reg_minus),
+ GEN_INT (3)));
+ clob = gen_rtx_CLOBBER (VOIDmode, gen_rtx_REG (CCmode, FLAGS_REG));
+ tmp = gen_rtx_PARALLEL (VOIDmode, gen_rtvec (2, tmp, clob));
+ emit_insn (tmp);
+ emit_insn ((Pmode == SImode)
+ ? gen_incsspsi (reg_adj)
+ : gen_incsspdi (reg_adj));
+
+ emit_label (label);
+ LABEL_NUSES (label) = 1;
+ }
+
+ /* This code is the same as in expand_buildin_longjmp. */
+ fp = gen_rtx_MEM (Pmode, operands[0]);
+ lab = gen_rtx_MEM (Pmode, plus_constant (Pmode, operands[0],
+ GET_MODE_SIZE (Pmode)));
+ stack = gen_rtx_MEM (sa_mode, plus_constant (Pmode, operands[0],
+ 2 * GET_MODE_SIZE (Pmode)));
+ lab = copy_to_reg (lab);
+
+ emit_clobber (gen_rtx_MEM (BLKmode, gen_rtx_SCRATCH (VOIDmode)));
+ emit_clobber (gen_rtx_MEM (BLKmode, hard_frame_pointer_rtx));
+
+ emit_move_insn (hard_frame_pointer_rtx, fp);
+ emit_stack_restore (SAVE_NONLOCAL, stack);
+
+ emit_use (hard_frame_pointer_rtx);
+ emit_use (stack_pointer_rtx);
+ emit_indirect_jump (lab);
+})
+
+
;; Avoid redundant prefixes by splitting HImode arithmetic to SImode.
;; Do not split instructions with mask registers.
(define_split
@@ -19315,6 +19425,83 @@
[(set_attr "length" "2")
(set_attr "memory" "unknown")])
+;; CET instructions
+(define_insn "rdssp<mode>"
+ [(set (match_operand:SWI48x 0 "register_operand" "=r")
+ (unspec_volatile:SWI48x
+ [(match_operand:SWI48x 1 "register_operand" "0")]
+ UNSPECV_NOP_RDSSP))]
+ "TARGET_SHSTK"
+ "rdssp<mskmodesuffix>\t%0"
+ [(set_attr "length" "4")
+ (set_attr "type" "other")])
+
+(define_insn "incssp<mode>"
+ [(unspec_volatile [(match_operand:SWI48x 0 "register_operand" "r")]
+ UNSPECV_INCSSP)]
+ "TARGET_SHSTK"
+ "incssp<mskmodesuffix>\t%0"
+ [(set_attr "length" "4")
+ (set_attr "type" "other")])
+
+(define_insn "saveprevssp"
+ [(unspec_volatile [(const_int 0)] UNSPECV_SAVEPREVSSP)]
+ "TARGET_SHSTK"
+ "saveprevssp"
+ [(set_attr "length" "5")
+ (set_attr "type" "other")])
+
+(define_insn "rstorssp"
+ [(unspec_volatile [(match_operand 0 "memory_operand" "m")]
+ UNSPECV_RSTORSSP)]
+ "TARGET_SHSTK"
+ "rstorssp\t%0"
+ [(set_attr "length" "5")
+ (set_attr "type" "other")])
+
+(define_insn "wrss<mode>"
+ [(unspec_volatile [(match_operand:SWI48x 0 "register_operand" "r")
+ (match_operand:SWI48x 1 "memory_operand" "m")]
+ UNSPECV_WRSS)]
+ "TARGET_SHSTK"
+ "wrss<mskmodesuffix>\t%0, %1"
+ [(set_attr "length" "3")
+ (set_attr "type" "other")])
+
+(define_insn "wruss<mode>"
+ [(unspec_volatile [(match_operand:SWI48x 0 "register_operand" "r")
+ (match_operand:SWI48x 1 "memory_operand" "m")]
+ UNSPECV_WRUSS)]
+ "TARGET_SHSTK"
+ "wruss<mskmodesuffix>\t%0, %1"
+ [(set_attr "length" "4")
+ (set_attr "type" "other")])
+
+(define_insn "setssbsy"
+ [(unspec_volatile [(const_int 0)] UNSPECV_SETSSBSY)]
+ "TARGET_SHSTK"
+ "setssbsy"
+ [(set_attr "length" "4")
+ (set_attr "type" "other")])
+
+(define_insn "clrssbsy"
+ [(unspec_volatile [(match_operand 0 "memory_operand" "m")]
+ UNSPECV_CLRSSBSY)]
+ "TARGET_SHSTK"
+ "clrssbsy\t%0"
+ [(set_attr "length" "4")
+ (set_attr "type" "other")])
+
+(define_insn "nop_endbr"
+ [(unspec_volatile [(const_int 0)] UNSPECV_NOP_ENDBR)]
+ "TARGET_IBT"
+ "*
+{ return (TARGET_64BIT)? \"endbr64\" : \"endbr32\"; }"
+ [(set_attr "length" "4")
+ (set_attr "length_immediate" "0")
+ (set_attr "modrm" "0")])
+
+;; For RTM support
(define_expand "xbegin"
[(set (match_operand:SI 0 "register_operand")
(unspec_volatile:SI [(const_int 0)] UNSPECV_XBEGIN))]
diff --git a/gcc/config/i386/i386.opt b/gcc/config/i386/i386.opt
index cdb156e4825..d9aca9bd4b5 100644
--- a/gcc/config/i386/i386.opt
+++ b/gcc/config/i386/i386.opt
@@ -959,3 +959,24 @@ Enum(indirect_branch) String(thunk-extern) Value(indirect_branch_thunk_extern)
mindirect-branch-register
Target Report Var(ix86_indirect_branch_register) Init(0)
Force indirect call and jump via register.
+
+mcet
+Target Report Var(flag_cet) Init(0)
+Support Control-flow Enforcment Technology (CET) built-in functions
+and code generation.
+
+mibt
+Target Report Mask(ISA_IBT) Var(ix86_isa_flags2) Save
+Specifically enables an indirect branch tracking feature from Control-flow
+Enforcment Technology (CET).
+
+mshstk
+Target Report Mask(ISA_SHSTK) Var(ix86_isa_flags2) Save
+Specifically enables an shadow stack support feature from Control-flow
+Enforcment Technology (CET).
+
+mcet-switch
+Target Report Undocumented Var(flag_cet_switch) Init(0)
+Turn on CET instrumentation for switch statements, which use jump table and
+indirect jump.
+
diff --git a/gcc/config/i386/immintrin.h b/gcc/config/i386/immintrin.h
index b52f58efa40..696cd20e538 100644
--- a/gcc/config/i386/immintrin.h
+++ b/gcc/config/i386/immintrin.h
@@ -90,6 +90,8 @@
#include <xtestintrin.h>
+#include <cetintrin.h>
+
#ifndef __RDRND__
#pragma GCC push_options
#pragma GCC target("rdrnd")
diff --git a/gcc/config/i386/linux-common.h b/gcc/config/i386/linux-common.h
index 6380639b204..6613807180e 100644
--- a/gcc/config/i386/linux-common.h
+++ b/gcc/config/i386/linux-common.h
@@ -121,3 +121,8 @@ along with GCC; see the file COPYING3. If not see
#define CHKP_SPEC "\
%{!nostdlib:%{!nodefaultlibs:" LIBMPX_SPEC LIBMPXWRAPPERS_SPEC "}}" MPX_SPEC
#endif
+
+extern void file_end_indicate_exec_stack_and_cet (void);
+
+#undef TARGET_ASM_FILE_END
+#define TARGET_ASM_FILE_END file_end_indicate_exec_stack_and_cet
diff --git a/gcc/config/i386/t-cet b/gcc/config/i386/t-cet
new file mode 100644
index 00000000000..317f30dbb98
--- /dev/null
+++ b/gcc/config/i386/t-cet
@@ -0,0 +1,21 @@
+# Copyright (C) 2017 Free Software Foundation, Inc.
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# GCC is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING3. If not see
+# <http://www.gnu.org/licenses/>.
+
+cet.o: $(srcdir)/config/i386/cet.c
+ $(COMPILE) $<
+ $(POSTCOMPILE)
^ permalink raw reply [flat|nested] only message in thread
only message in thread, other threads:[~2021-10-14 13:01 UTC | newest]
Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-10-14 13:01 [gcc(refs/users/giulianob/heads/pfe_backport_dirty)] Backport Update x86 backend to enable Intel CET Giuliano Belinassi
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).