[COMMITTED V2 0/7] BPF CO-RE Support

[COMMITTED V2 0/7] BPF CO-RE Support

[David Faust]

[ Changes from V1:

  All patches have been OK'd, but the prerequisite series "Allow means for late
  BTF generation for BPF CO-RE" had not been accepted. Now that that series has
  been applied, this can be pushed with some very minor tweaks:

  - Accomodate rename of option '-mco-re' (was -mcore) in option handling,
    tests and documentation.
  - Set BTF_WITH_CORE_DEBUG write symbols flag where needed. ]

Hello,

This patch series adds support for the BPF Compile Once - Run Everywhere
(BPF CO-RE) mechanism in GCC.

A BPF program is some user code which is injected (via a verifier and loader)
into a running kernel, and executed in kernel context. To do useful work, a BPF
program generally must interact with kernel data structures in some way.
Therefore, BPF programs written in C usually include kernel headers.

This introduces two major portability issues when compiling BPF programs:

   1. Kernel data structures regularly change, with fields added, moved or
      deleted between versions. An eBPF program cannot in general be expected
      to run on any systems which does not share an identical kernel version to
      the system on which it was compiled.

   2. Included kernel headers (and used data structures) may be internal, not
      exposed in an userspace API, and therefore target-specific. An eBPF
      program compiled on an x86_64 machine will include x86_64 kernel headers.
      The resulting program may not run well (or at all) in machines of
      another architecture.

BPF CO-RE is designed to solve the first issue by leveraging the BPF loader to
adjust references to kernel data structures made by the program as-needed
according to versions of structures actually present on the host kernel.

To achieve this, additional information is placed in a ".BTF.ext" section.  This
information tells the loader which references will require adjusting, and how to
perform each necessary adjustment.

For any access to a data structure which may require load-time adjustment,
the following information is recorded (making up a CO-RE relocation record):
- The BTF type ID of the outermost structure which is accessed.
- An access string encoding the accessed member via a series of member and
  array indexes. These indexes are used to look up detailed BTF information
  about the member.
- The offset of the appropriate instruction to patch in the BPF program.
- An integer specifying what kind of relocation to perform.

A CO-RE-capable BPF loader reads this information together with the BTF
information of the program, compares it against BTF information of the host
kernel, and determines the appropriate way to patch the specified instruction.

Once all CO-RE relocations are resolved, the program is loaded and verified as
usual. The process can be summarized with the following diagram:

              +------------+
              | C compiler |
              +-----+------+
                    | BPF + BTF + CO-RE relocations
                    v
              +------------+
         +--->| BPF loader |
         |    +-----+------+
         |          | BPF (adapted)
     BTF |          v
         |    +------------+
         +----+   Kernel   |
              +------------+

Note that a single ELF object may contain multiple eBPF programs. As a result, a
single .BTF.ext section can contain CO-RE relocations for multiple programs in
distinct sections.

Many data structure accesses (e.g., those described in the program itself) do
not need to be patched. So, GCC only generates CO-RE information for accesses
marked as being "of interest." To be compatible with LLVM a new BPF target
builtin, __builtin_preserve_access_index, is implemented. Any accesses to
aggregate data structures (structs, unions, arrays) in the argument will have
appropriate CO-RE information generated and output. This builtin is otherwise
transparent - it does not alter the program's functionality in any way.

In addition, a new BPF target attribute preserve_access_index is added.  This
attribute may annotate struct and union type definitions. Any access to a type
with this attribute is automatically "of interest," and will have CO-RE
information generated accordingly.

Finally, generation of BPF CO-RE information is gated behind a new BPF option,
-mcore (and its negative, -mno-core). Because CO-RE support is intimately tied
to BTF debug information, -gbtf for BPF target implies -mcore, and -mcore
requires BTF generation. For cases where BTF information is desired but CO-RE
is not important, it can be disabled with -mno-core.

David Faust (7):
  dwarf: externalize lookup_type_die
  ctfc: externalize ctf_dtd_lookup
  ctfc: add function to lookup CTF ID of a TREE type
  btf: expose get_btf_id
  bpf: BPF CO-RE support
  bpf testsuite: Add BPF CO-RE tests
  doc: BPF CO-RE documentation

 gcc/btfout.c                                  |   2 +-
 gcc/config.gcc                                |   3 +
 gcc/config/bpf/bpf-passes.def                 |  20 +
 gcc/config/bpf/bpf-protos.h                   |   2 +
 gcc/config/bpf/bpf.c                          | 591 ++++++++++++++++++
 gcc/config/bpf/coreout.c                      | 356 +++++++++++
 gcc/config/bpf/coreout.h                      | 114 ++++
 gcc/config/bpf/t-bpf                          |   8 +
 gcc/ctfc.c                                    |  18 +-
 gcc/ctfc.h                                    |   8 +-
 gcc/doc/extend.texi                           |  16 +
 gcc/doc/invoke.texi                           |  13 +-
 gcc/dwarf2out.c                               |   3 +-
 gcc/dwarf2out.h                               |   1 +
 gcc/testsuite/gcc.target/bpf/core-attr-1.c    |  23 +
 gcc/testsuite/gcc.target/bpf/core-attr-2.c    |  21 +
 gcc/testsuite/gcc.target/bpf/core-attr-3.c    |  41 ++
 gcc/testsuite/gcc.target/bpf/core-attr-4.c    |  35 ++
 gcc/testsuite/gcc.target/bpf/core-builtin-1.c |  64 ++
 gcc/testsuite/gcc.target/bpf/core-builtin-2.c |  26 +
 gcc/testsuite/gcc.target/bpf/core-builtin-3.c |  26 +
 gcc/testsuite/gcc.target/bpf/core-section-1.c |  38 ++
 22 files changed, 1423 insertions(+), 6 deletions(-)
 create mode 100644 gcc/config/bpf/bpf-passes.def
 create mode 100644 gcc/config/bpf/coreout.c
 create mode 100644 gcc/config/bpf/coreout.h
 create mode 100644 gcc/testsuite/gcc.target/bpf/core-attr-1.c
 create mode 100644 gcc/testsuite/gcc.target/bpf/core-attr-2.c
 create mode 100644 gcc/testsuite/gcc.target/bpf/core-attr-3.c
 create mode 100644 gcc/testsuite/gcc.target/bpf/core-attr-4.c
 create mode 100644 gcc/testsuite/gcc.target/bpf/core-builtin-1.c
 create mode 100644 gcc/testsuite/gcc.target/bpf/core-builtin-2.c
 create mode 100644 gcc/testsuite/gcc.target/bpf/core-builtin-3.c
 create mode 100644 gcc/testsuite/gcc.target/bpf/core-section-1.c

-- 
2.33.0

[COMMITTED V2 1/7] dwarf: externalize lookup_type_die

[David Faust]

Expose the function lookup_type_die in dwarf2out, so that it can be used
by CTF/BTF when adding BPF CO-RE information. The function is now
non-static, and an extern prototype is added in dwarf2out.h.

gcc/ChangeLog:

	* dwarf2out.c (lookup_type_die): Function is no longer static.
	* dwarf2out.h: Expose it here.
---
 gcc/dwarf2out.c | 3 +--
 gcc/dwarf2out.h | 1 +
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/gcc/dwarf2out.c b/gcc/dwarf2out.c
index 72cd1f51380..9876750e4f9 100644
--- a/gcc/dwarf2out.c
+++ b/gcc/dwarf2out.c
@@ -3740,7 +3740,6 @@ static bool remove_AT (dw_die_ref, enum dwarf_attribute);
 static void remove_child_TAG (dw_die_ref, enum dwarf_tag);
 static void add_child_die (dw_die_ref, dw_die_ref);
 static dw_die_ref new_die (enum dwarf_tag, dw_die_ref, tree);
-static dw_die_ref lookup_type_die (tree);
 static dw_die_ref strip_naming_typedef (tree, dw_die_ref);
 static dw_die_ref lookup_type_die_strip_naming_typedef (tree);
 static void equate_type_number_to_die (tree, dw_die_ref);
@@ -5838,7 +5837,7 @@ new_die (enum dwarf_tag tag_value, dw_die_ref parent_die, tree t)
 
 /* Return the DIE associated with the given type specifier.  */
 
-static inline dw_die_ref
+dw_die_ref
 lookup_type_die (tree type)
 {
   dw_die_ref die = TYPE_SYMTAB_DIE (type);
diff --git a/gcc/dwarf2out.h b/gcc/dwarf2out.h
index b2152a53bf9..312a9909784 100644
--- a/gcc/dwarf2out.h
+++ b/gcc/dwarf2out.h
@@ -417,6 +417,7 @@ extern dw_die_ref new_die_raw (enum dwarf_tag);
 extern dw_die_ref base_type_die (tree, bool);
 
 extern dw_die_ref lookup_decl_die (tree);
+extern dw_die_ref lookup_type_die (tree);
 
 extern dw_die_ref dw_get_die_child (dw_die_ref);
 extern dw_die_ref dw_get_die_sib (dw_die_ref);
-- 
2.33.0

[COMMITTED V2 2/7] ctfc: externalize ctf_dtd_lookup

[David Faust]

Expose the function ctf_dtd_lookup, so that it can be used by the BPF
CO-RE machinery. The function is no longer static, and an extern
prototype is added in ctfc.h.

gcc/ChangeLog:

	* ctfc.c (ctf_dtd_lookup): Function is no longer static.
	* ctfc.h: Analogous change.
---
 gcc/ctfc.c | 2 +-
 gcc/ctfc.h | 5 ++++-
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/gcc/ctfc.c b/gcc/ctfc.c
index 1a6ddb80829..db6ba030301 100644
--- a/gcc/ctfc.c
+++ b/gcc/ctfc.c
@@ -132,7 +132,7 @@ ctf_dtd_insert (ctf_container_ref ctfc, ctf_dtdef_ref dtd)
 
 /* Lookup CTF type given a DWARF die for the type.  */
 
-static ctf_dtdef_ref
+ctf_dtdef_ref
 ctf_dtd_lookup (const ctf_container_ref ctfc, const dw_die_ref type)
 {
   ctf_dtdef_t entry;
diff --git a/gcc/ctfc.h b/gcc/ctfc.h
index 39c527074b5..825570d807e 100644
--- a/gcc/ctfc.h
+++ b/gcc/ctfc.h
@@ -388,7 +388,10 @@ extern bool ctf_type_exists (ctf_container_ref, dw_die_ref, ctf_id_t *);
 
 extern void ctf_add_cuname (ctf_container_ref, const char *);
 
-extern ctf_dvdef_ref ctf_dvd_lookup (const ctf_container_ref, dw_die_ref);
+extern ctf_dtdef_ref ctf_dtd_lookup (const ctf_container_ref ctfc,
+				     dw_die_ref die);
+extern ctf_dvdef_ref ctf_dvd_lookup (const ctf_container_ref ctfc,
+				     dw_die_ref die);
 
 extern const char * ctf_add_string (ctf_container_ref, const char *,
 				    uint32_t *, int);
-- 
2.33.0

[COMMITTED V2 3/7] ctfc: add function to lookup CTF ID of a TREE type

[David Faust]

Add a new function, ctf_lookup_tree_type, to return the CTF type ID
associated with a type via its is TREE node. The function is exposed via
a prototype in ctfc.h.

gcc/ChangeLog:

	* ctfc.c (ctf_lookup_tree_type): New function.
	* ctfc.h: Likewise.
---
 gcc/ctfc.c | 16 ++++++++++++++++
 gcc/ctfc.h |  2 ++
 2 files changed, 18 insertions(+)

diff --git a/gcc/ctfc.c b/gcc/ctfc.c
index db6ba030301..73c118e3d49 100644
--- a/gcc/ctfc.c
+++ b/gcc/ctfc.c
@@ -791,6 +791,22 @@ ctf_add_sou (ctf_container_ref ctfc, uint32_t flag, const char * name,
   return type;
 }
 
+/* Given a TREE_TYPE node, return the CTF type ID for that type.  */
+
+ctf_id_t
+ctf_lookup_tree_type (ctf_container_ref ctfc, const tree type)
+{
+  dw_die_ref die = lookup_type_die (type);
+  if (die == NULL)
+    return CTF_NULL_TYPEID;
+
+  ctf_dtdef_ref dtd = ctf_dtd_lookup (ctfc, die);
+  if (dtd == NULL)
+    return CTF_NULL_TYPEID;
+
+  return dtd->dtd_type;
+}
+
 /* Check if CTF for TYPE has already been generated.  Mainstay for
    de-duplication.  If CTF type already exists, returns TRUE and updates
    the TYPE_ID for the caller.  */
diff --git a/gcc/ctfc.h b/gcc/ctfc.h
index 825570d807e..14180c1e5de 100644
--- a/gcc/ctfc.h
+++ b/gcc/ctfc.h
@@ -430,6 +430,8 @@ extern int ctf_add_function_arg (ctf_container_ref, dw_die_ref,
 extern int ctf_add_variable (ctf_container_ref, const char *, ctf_id_t,
 			     dw_die_ref, unsigned int);
 
+extern ctf_id_t ctf_lookup_tree_type (ctf_container_ref, const tree);
+
 /* CTF section does not emit location information; at this time, location
    information is needed for BTF CO-RE use-cases.  */
 
-- 
2.33.0

[COMMITTED V2 4/7] btf: expose get_btf_id

[David Faust]

Expose the function get_btf_id, so that it may be used by the BPF
backend. This enables the BPF CO-RE machinery in the BPF backend to
lookup BTF type IDs, in order to create CO-RE relocation records.

A prototype is added in ctfc.h

gcc/ChangeLog:

	* btfout.c (get_btf_id): Function is no longer static.
	* ctfc.h: Expose it here.
---
 gcc/btfout.c | 2 +-
 gcc/ctfc.h   | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/gcc/btfout.c b/gcc/btfout.c
index 8cdd9905fb6..cdc6c6378c0 100644
--- a/gcc/btfout.c
+++ b/gcc/btfout.c
@@ -156,7 +156,7 @@ init_btf_id_map (size_t len)
 /* Return the BTF type ID of CTF type ID KEY, or BTF_INVALID_TYPEID if the CTF
    type with ID KEY does not map to a BTF type.  */
 
-static inline ctf_id_t
+ctf_id_t
 get_btf_id (ctf_id_t key)
 {
   return btf_id_map[key];
diff --git a/gcc/ctfc.h b/gcc/ctfc.h
index 14180c1e5de..a0b7e4105a8 100644
--- a/gcc/ctfc.h
+++ b/gcc/ctfc.h
@@ -431,6 +431,7 @@ extern int ctf_add_variable (ctf_container_ref, const char *, ctf_id_t,
 			     dw_die_ref, unsigned int);
 
 extern ctf_id_t ctf_lookup_tree_type (ctf_container_ref, const tree);
+extern ctf_id_t get_btf_id (ctf_id_t);
 
 /* CTF section does not emit location information; at this time, location
    information is needed for BTF CO-RE use-cases.  */
-- 
2.33.0

[COMMITTED V2 5/7] bpf: BPF CO-RE support

[David Faust]

This commit introduces support for BPF Compile Once - Run
Everywhere (CO-RE) in GCC.

gcc/ChangeLog:

	* config/bpf/bpf.c: Adjust includes.
	(bpf_handle_preserve_access_index_attribute): New function.
	(bpf_attribute_table): Use it here.
	(bpf_builtins): Add BPF_BUILTIN_PRESERVE_ACCESS_INDEX.
	(bpf_option_override): Handle "-mco-re" option.
	(bpf_asm_init_sections): New.
	(TARGET_ASM_INIT_SECTIONS): Redefine.
	(bpf_file_end): New.
	(TARGET_ASM_FILE_END): Redefine.
	(bpf_init_builtins): Add "__builtin_preserve_access_index".
	(bpf_core_compute, bpf_core_get_index): New.
	(is_attr_preserve_access): New.
	(bpf_expand_builtin): Handle new builtins.
	(bpf_core_newdecl, bpf_core_is_maybe_aggregate_access): New.
	(bpf_core_walk): New.
	(bpf_resolve_overloaded_builtin): New.
	(TARGET_RESOLVE_OVERLOADED_BUILTIN): Redefine.
	(handle_attr): New.
	(pass_bpf_core_attr): New RTL pass.
	* config/bpf/bpf-passes.def: New file.
	* config/bpf/bpf-protos.h (make_pass_bpf_core_attr): New.
	* config/bpf/coreout.c: New file.
	* config/bpf/coreout.h: Likewise.
	* config/bpf/t-bpf (TM_H): Add $(srcdir)/config/bpf/coreout.h.
	(coreout.o): New rule.
	(PASSES_EXTRA): Add $(srcdir)/config/bpf/bpf-passes.def.
	* config.gcc (bpf): Add coreout.h to extra_headers.
	Add coreout.o to extra_objs.
	Add $(srcdir)/config/bpf/coreout.c to target_gtfiles.
---
 gcc/config.gcc                |   3 +
 gcc/config/bpf/bpf-passes.def |  20 ++
 gcc/config/bpf/bpf-protos.h   |   2 +
 gcc/config/bpf/bpf.c          | 591 ++++++++++++++++++++++++++++++++++
 gcc/config/bpf/coreout.c      | 356 ++++++++++++++++++++
 gcc/config/bpf/coreout.h      | 114 +++++++
 gcc/config/bpf/t-bpf          |   8 +
 7 files changed, 1094 insertions(+)
 create mode 100644 gcc/config/bpf/bpf-passes.def
 create mode 100644 gcc/config/bpf/coreout.c
 create mode 100644 gcc/config/bpf/coreout.h

diff --git a/gcc/config.gcc b/gcc/config.gcc
index e553ef34bc7..e3e9d8f676f 100644
--- a/gcc/config.gcc
+++ b/gcc/config.gcc
@@ -1525,6 +1525,9 @@ bpf-*-*)
         use_collect2=no
         extra_headers="bpf-helpers.h"
         use_gcc_stdint=provide
+        extra_headers="coreout.h"
+        extra_objs="coreout.o"
+        target_gtfiles="$target_gtfiles \$(srcdir)/config/bpf/coreout.c"
         ;;
 cr16-*-elf)
         tm_file="elfos.h ${tm_file} newlib-stdint.h"
diff --git a/gcc/config/bpf/bpf-passes.def b/gcc/config/bpf/bpf-passes.def
new file mode 100644
index 00000000000..3e961659411
--- /dev/null
+++ b/gcc/config/bpf/bpf-passes.def
@@ -0,0 +1,20 @@
+/* Declaration of target-specific passes for eBPF.
+   Copyright (C) 2021 Free Software Foundation, Inc.
+
+   This file is part of GCC.
+
+   GCC is free software; you can redistribute it and/or modify it
+   under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3, or (at your option)
+   any later version.
+
+   GCC is distributed in the hope that it will be useful, but
+   WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with GCC; see the file COPYING3.  If not see
+   <http://www.gnu.org/licenses/>.  */
+
+INSERT_PASS_AFTER (pass_df_initialize_opt, 1, pass_bpf_core_attr);
diff --git a/gcc/config/bpf/bpf-protos.h b/gcc/config/bpf/bpf-protos.h
index aeb512665ed..7ce3386ffda 100644
--- a/gcc/config/bpf/bpf-protos.h
+++ b/gcc/config/bpf/bpf-protos.h
@@ -30,4 +30,6 @@ extern void bpf_print_operand_address (FILE *, rtx);
 extern void bpf_expand_prologue (void);
 extern void bpf_expand_epilogue (void);
 
+rtl_opt_pass * make_pass_bpf_core_attr (gcc::context *);
+
 #endif /* ! GCC_BPF_PROTOS_H */
diff --git a/gcc/config/bpf/bpf.c b/gcc/config/bpf/bpf.c
index 7228978a3a9..01d9c03479e 100644
--- a/gcc/config/bpf/bpf.c
+++ b/gcc/config/bpf/bpf.c
@@ -56,6 +56,24 @@ along with GCC; see the file COPYING3.  If not see
 #include "langhooks.h"
 #include "flags.h"
 
+#include "cfg.h" /* needed for struct control_flow_graph used in BB macros */
+#include "gimple.h"
+#include "gimple-iterator.h"
+#include "gimple-walk.h"
+#include "tree-pass.h"
+#include "tree-iterator.h"
+
+#include "context.h"
+#include "pass_manager.h"
+
+#include "gimplify.h"
+#include "gimplify-me.h"
+
+#include "ctfc.h"
+#include "btf.h"
+
+#include "coreout.h"
+
 /* Per-function machine data.  */
 struct GTY(()) machine_function
 {
@@ -105,6 +123,27 @@ bpf_handle_fndecl_attribute (tree *node, tree name,
   return NULL_TREE;
 }
 
+/* Handle preserve_access_index attribute, which can be applied to structs,
+   unions and classes. Actually adding the attribute to the TYPE_DECL is
+   taken care of for us, so just warn for types that aren't supported.  */
+
+static tree
+bpf_handle_preserve_access_index_attribute (tree *node, tree name,
+					    tree args,
+					    int flags,
+					    bool *no_add_attrs)
+{
+  if (TREE_CODE (*node) != RECORD_TYPE && TREE_CODE (*node) != UNION_TYPE)
+    {
+      warning (OPT_Wattributes,
+	       "%qE attribute only applies to structure, union and class types",
+	       name);
+      *no_add_attrs = true;
+    }
+
+  return NULL_TREE;
+}
+
 /* Target-specific attributes.  */
 
 static const struct attribute_spec bpf_attribute_table[] =
@@ -117,6 +156,11 @@ static const struct attribute_spec bpf_attribute_table[] =
  { "kernel_helper", 1, 1, true, false, false, false,
    bpf_handle_fndecl_attribute, NULL },
 
+ /* CO-RE support: attribute to mark that all accesses to the declared
+    struct/union/array should be recorded.  */
+ { "preserve_access_index", 0, -1, false, true, false, true,
+   bpf_handle_preserve_access_index_attribute, NULL },
+
  /* The last attribute spec is set to be NULL.  */
  { NULL,	0,  0, false, false, false, false, NULL, NULL }
 };
@@ -137,11 +181,18 @@ enum bpf_builtins
   BPF_BUILTIN_LOAD_BYTE,
   BPF_BUILTIN_LOAD_HALF,
   BPF_BUILTIN_LOAD_WORD,
+
+  /* Compile Once - Run Everywhere (CO-RE) support.  */
+  BPF_BUILTIN_PRESERVE_ACCESS_INDEX,
+
   BPF_BUILTIN_MAX,
 };
 
 static GTY (()) tree bpf_builtins[(int) BPF_BUILTIN_MAX];
 
+
+void bpf_register_coreattr_pass (void);
+
 /* Initialize the per-function machine status.  */
 
 static struct machine_function *
@@ -183,11 +234,57 @@ bpf_option_override (void)
 
   if (flag_lto && TARGET_BPF_CORE)
     sorry ("BPF CO-RE does not support LTO");
+
+  /* -gbtf implies -mcore when using the BPF backend, unless -mno-co-re
+     is specified.  */
+  if (btf_debuginfo_p () && !(target_flags_explicit & MASK_BPF_CORE))
+    {
+      target_flags |= MASK_BPF_CORE;
+      write_symbols |= BTF_WITH_CORE_DEBUG;
+    }
 }
 
 #undef TARGET_OPTION_OVERRIDE
 #define TARGET_OPTION_OVERRIDE bpf_option_override
 
+/* Return FALSE iff -mcore has been specified.  */
+
+static bool
+ctfc_debuginfo_early_finish_p (void)
+{
+  if (TARGET_BPF_CORE)
+    return false;
+  else
+    return true;
+}
+
+#undef TARGET_CTFC_DEBUGINFO_EARLY_FINISH_P
+#define TARGET_CTFC_DEBUGINFO_EARLY_FINISH_P ctfc_debuginfo_early_finish_p
+
+/* Implement TARGET_ASM_INIT_SECTIONS.  */
+
+static void
+bpf_asm_init_sections (void)
+{
+  if (TARGET_BPF_CORE)
+    btf_ext_init ();
+}
+
+#undef TARGET_ASM_INIT_SECTIONS
+#define TARGET_ASM_INIT_SECTIONS bpf_asm_init_sections
+
+/* Implement TARGET_ASM_FILE_END.  */
+
+static void
+bpf_file_end (void)
+{
+  if (TARGET_BPF_CORE)
+    btf_ext_output ();
+}
+
+#undef TARGET_ASM_FILE_END
+#define TARGET_ASM_FILE_END bpf_file_end
+
 /* Define target-specific CPP macros.  This function in used in the
    definition of TARGET_CPU_CPP_BUILTINS in bpf.h */
 
@@ -837,11 +934,18 @@ bpf_init_builtins (void)
 	       build_function_type_list (ullt, ullt, 0));
   def_builtin ("__builtin_bpf_load_word", BPF_BUILTIN_LOAD_WORD,
 	       build_function_type_list (ullt, ullt, 0));
+  def_builtin ("__builtin_preserve_access_index",
+	       BPF_BUILTIN_PRESERVE_ACCESS_INDEX,
+	       build_function_type_list (ptr_type_node, ptr_type_node, 0));
 }
 
 #undef TARGET_INIT_BUILTINS
 #define TARGET_INIT_BUILTINS bpf_init_builtins
 
+static tree bpf_core_compute (tree, vec<unsigned int> *);
+static int bpf_core_get_index (const tree);
+static bool is_attr_preserve_access (tree);
+
 /* Expand a call to a BPF-specific built-in function that was set up
    with bpf_init_builtins.  */
 
@@ -892,7 +996,75 @@ bpf_expand_builtin (tree exp, rtx target ATTRIBUTE_UNUSED,
       /* The result of the load is in R0.  */
       return gen_rtx_REG (ops[0].mode, BPF_R0);
     }
+  else if (code == -1)
+    {
+      /* A resolved overloaded builtin, e.g. __bpf_preserve_access_index_si */
+      tree arg = CALL_EXPR_ARG (exp, 0);
+
+      if (arg == NULL_TREE)
+	return NULL_RTX;
+
+      auto_vec<unsigned int, 16> accessors;
+      tree container;
 
+      if (TREE_CODE (arg) == SSA_NAME)
+	{
+	  gimple *def_stmt = SSA_NAME_DEF_STMT (arg);
+
+	  if (is_gimple_assign (def_stmt))
+	    arg = gimple_assign_rhs1 (def_stmt);
+	  else
+	    return expand_normal (arg);
+	}
+
+      /* Avoid double-recording information if the argument is an access to
+	 a struct/union marked __attribute__((preserve_access_index)). This
+	 Will be handled by the attribute handling pass.  */
+      if (is_attr_preserve_access (arg))
+	return expand_normal (arg);
+
+      container = bpf_core_compute (arg, &accessors);
+
+      /* Any valid use of the builtin must have at least one access. Otherwise,
+	 there is nothing to record and nothing to do. This is primarily a
+	 guard against optimizations leading to unexpected expressions in the
+	 argument of the builtin. For example, if the builtin is used to read
+	 a field of a structure which can be statically determined to hold a
+	 constant value, the argument to the builtin will be optimized to that
+	 constant. This is OK, and means the builtin call is superfluous.
+	 e.g.
+	   struct S foo;
+	   foo.a = 5;
+	   int x = __preserve_access_index (foo.a);
+	   ... do stuff with x
+	 'foo.a' in the builtin argument will be optimized to '5' with -01+.
+	 This sequence does not warrant recording a CO-RE relocation.  */
+
+      if (accessors.length () < 1)
+	return expand_normal (arg);
+
+      accessors.reverse ();
+
+      container = TREE_TYPE (container);
+
+      rtx_code_label *label = gen_label_rtx ();
+      LABEL_PRESERVE_P (label) = 1;
+      emit_label (label);
+
+      /* Determine what output section this relocation will apply to.
+	 If this function is associated with a section, use that. Otherwise,
+	 fall back on '.text'.  */
+      const char * section_name;
+      if (current_function_decl && DECL_SECTION_NAME (current_function_decl))
+	section_name = DECL_SECTION_NAME (current_function_decl);
+      else
+	section_name = ".text";
+
+      /* Add the CO-RE relocation information to the BTF container.  */
+      bpf_core_reloc_add (container, section_name, &accessors, label);
+
+      return expand_normal (arg);
+    }
   gcc_unreachable ();
 }
 
@@ -946,6 +1118,425 @@ bpf_debug_unwind_info ()
 #undef TARGET_ASM_ALIGNED_DI_OP
 #define TARGET_ASM_ALIGNED_DI_OP "\t.dword\t"
 
+
+/* BPF Compile Once - Run Everywhere (CO-RE) support routines.
+
+   BPF CO-RE is supported in two forms:
+   - A target builtin, __builtin_preserve_access_index
+
+     This builtin accepts a single argument. Any access to an aggregate data
+     structure (struct, union or array) within the argument will be recorded by
+     the CO-RE machinery, resulting in a relocation record being placed in the
+     .BTF.ext section of the output.
+
+     It is implemented in bpf_resolve_overloaded_builtin () and
+     bpf_expand_builtin (), using the supporting routines below.
+
+   - An attribute, __attribute__((preserve_access_index))
+
+     This attribute can be applied to struct and union types. Any access to a
+     type with this attribute will be recorded by the CO-RE machinery.
+
+     The pass pass_bpf_core_attr, below, implements support for
+     this attribute.  */
+
+/* Traverse the subtree under NODE, which is expected to be some form of
+   aggregate access the CO-RE machinery cares about (like a read of a member of
+   a struct or union), collecting access indices for the components and storing
+   them in the vector referenced by ACCESSORS.
+
+   Return the ultimate (top-level) container of the aggregate access. In general,
+   this will be a VAR_DECL or some kind of REF.
+
+   Note that the accessors are computed *in reverse order* of how the BPF
+   CO-RE machinery defines them. The vector needs to be reversed (or simply
+   output in reverse order) for the .BTF.ext relocation information.  */
+
+static tree
+bpf_core_compute (tree node, vec<unsigned int> *accessors)
+{
+
+  if (TREE_CODE (node) == ADDR_EXPR)
+    node = TREE_OPERAND (node, 0);
+
+  else if (TREE_CODE (node) == INDIRECT_REF
+	   || TREE_CODE (node) == POINTER_PLUS_EXPR)
+    {
+      accessors->safe_push (0);
+      return TREE_OPERAND (node, 0);
+    }
+
+  while (1)
+    {
+      switch (TREE_CODE (node))
+	{
+	case COMPONENT_REF:
+	  accessors->safe_push (bpf_core_get_index (TREE_OPERAND (node, 1)));
+	  break;
+
+	case ARRAY_REF:
+	case ARRAY_RANGE_REF:
+	  accessors->safe_push (bpf_core_get_index (node));
+	  break;
+
+	case MEM_REF:
+	  accessors->safe_push (bpf_core_get_index (node));
+	  if (TREE_CODE (TREE_OPERAND (node, 0)) == ADDR_EXPR)
+	    node = TREE_OPERAND (TREE_OPERAND (node, 0), 0);
+	  goto done;
+
+	default:
+	  goto done;
+	}
+      node = TREE_OPERAND (node, 0);
+    }
+ done:
+  return node;
+
+}
+
+/* Compute the index of the NODE in its immediate container.
+   NODE should be a FIELD_DECL (i.e. of struct or union), or an ARRAY_REF. */
+static int
+bpf_core_get_index (const tree node)
+{
+  enum tree_code code = TREE_CODE (node);
+
+  if (code == FIELD_DECL)
+    {
+      /* Lookup the index from the BTF information.  Some struct/union members
+	 may not be emitted in BTF; only the BTF container has enough
+	 information to compute the correct index.  */
+      int idx = bpf_core_get_sou_member_index (ctf_get_tu_ctfc (), node);
+      if (idx >= 0)
+	return idx;
+    }
+
+  else if (code == ARRAY_REF || code == ARRAY_RANGE_REF || code == MEM_REF)
+    {
+      /* For array accesses, the index is operand 1.  */
+      tree index = TREE_OPERAND (node, 1);
+
+      /* If the indexing operand is a constant, extracting is trivial.  */
+      if (TREE_CODE (index) == INTEGER_CST && tree_fits_shwi_p (index))
+	return tree_to_shwi (index);
+    }
+
+  return -1;
+}
+
+/* Synthesize a new builtin function declaration at LOC with signature TYPE.
+   Used by bpf_resolve_overloaded_builtin to resolve calls to
+   __builtin_preserve_access_index.  */
+
+static tree
+bpf_core_newdecl (location_t loc, tree type)
+{
+  tree rettype = build_function_type_list (type, type, NULL);
+  tree newdecl = NULL_TREE;
+  char name[80];
+  int len = snprintf (name, sizeof (name), "%s", "__builtin_pai_");
+
+  static unsigned long cnt = 0;
+  len = snprintf (name + len, sizeof (name) - len, "%lu", cnt++);
+
+  return add_builtin_function_ext_scope (name, rettype, -1, BUILT_IN_MD, NULL,
+					 NULL_TREE);
+}
+
+/* Return whether EXPR could access some aggregate data structure that
+   BPF CO-RE support needs to know about.  */
+
+static int
+bpf_core_is_maybe_aggregate_access (tree expr)
+{
+  enum tree_code code = TREE_CODE (expr);
+  if (code == COMPONENT_REF || code == ARRAY_REF)
+    return 1;
+
+  if (code == ADDR_EXPR)
+      return bpf_core_is_maybe_aggregate_access (TREE_OPERAND (expr, 0));
+
+  return 0;
+}
+
+/* Callback function used with walk_tree from bpf_resolve_overloaded_builtin.  */
+
+static tree
+bpf_core_walk (tree *tp, int *walk_subtrees, void *data)
+{
+  location_t loc = *((location_t *) data);
+
+  /* If this is a type, don't do anything. */
+  if (TYPE_P (*tp))
+    {
+      *walk_subtrees = 0;
+      return NULL_TREE;
+    }
+
+  if (bpf_core_is_maybe_aggregate_access (*tp))
+    {
+      tree newdecl = bpf_core_newdecl (loc, TREE_TYPE (*tp));
+      tree newcall = build_call_expr_loc (loc, newdecl, 1, *tp);
+      *tp = newcall;
+      *walk_subtrees = 0;
+    }
+
+  return NULL_TREE;
+}
+
+
+/* Implement TARGET_RESOLVE_OVERLOADED_BUILTIN (see gccint manual section
+   Target Macros::Misc.).
+   We use this for the __builtin_preserve_access_index builtin for CO-RE
+   support.
+
+   FNDECL is the declaration of the builtin, and ARGLIST is the list of
+   arguments passed to it, and is really a vec<tree,_> *.
+
+   In this case, the 'operation' implemented by the builtin is a no-op;
+   the builtin is just a marker. So, the result is simply the argument.  */
+
+static tree
+bpf_resolve_overloaded_builtin (location_t loc, tree fndecl, void *arglist)
+{
+  if (DECL_MD_FUNCTION_CODE (fndecl) != BPF_BUILTIN_PRESERVE_ACCESS_INDEX)
+    return NULL_TREE;
+
+  /* We only expect one argument, but it may be an arbitrarily-complicated
+     statement-expression. */
+  vec<tree, va_gc> *params = static_cast<vec<tree, va_gc> *> (arglist);
+  unsigned n_params = params ? params->length() : 0;
+
+  if (n_params != 1)
+    {
+      error_at (loc, "expected exactly 1 argument");
+      return NULL_TREE;
+    }
+
+  tree param = (*params)[0];
+
+  /* If not generating BPF_CORE information, the builtin does nothing.  */
+  if (!TARGET_BPF_CORE)
+    return param;
+
+  /* Do remove_c_maybe_const_expr for the arg.
+     TODO: WHY do we have to do this here? Why doesn't c-typeck take care
+     of it before or after this hook? */
+  if (TREE_CODE (param) == C_MAYBE_CONST_EXPR)
+    param = C_MAYBE_CONST_EXPR_EXPR (param);
+
+  /* Construct a new function declaration with the correct type, and return
+     a call to it.
+
+     Calls with statement-expressions, for example:
+     _(({ foo->a = 1; foo->u[2].b = 2; }))
+     require special handling.
+
+     We rearrange this into a new block scope in which each statement
+     becomes a unique builtin call:
+     {
+       _ ({ foo->a = 1;});
+       _ ({ foo->u[2].b = 2;});
+     }
+
+     This ensures that all the relevant information remains within the
+     expression trees the builtin finally gets.  */
+
+  walk_tree (&param, bpf_core_walk, (void *) &loc, NULL);
+
+  return param;
+}
+
+#undef TARGET_RESOLVE_OVERLOADED_BUILTIN
+#define TARGET_RESOLVE_OVERLOADED_BUILTIN bpf_resolve_overloaded_builtin
+
+
+/* Handling for __attribute__((preserve_access_index)) for BPF CO-RE support.
+
+   This attribute marks a structure/union/array type as "preseve", so that
+   every access to that type should be recorded and replayed by the BPF loader;
+   this is just the same functionality as __builtin_preserve_access_index,
+   but in the form of an attribute for an entire aggregate type.
+
+   Note also that nested structs behave as though they all have the attribute.
+   For example:
+     struct X { int a; };
+     struct Y { struct X bar} __attribute__((preserve_access_index));
+     struct Y foo;
+     foo.bar.a;
+   will record access all the way to 'a', even though struct X does not have
+   the preserve_access_index attribute.
+
+   This is to follow LLVM behavior.
+
+   This pass finds all accesses to objects of types marked with the attribute,
+   and wraps them in the same "low-level" builtins used by the builtin version.
+   All logic afterwards is therefore identical to the builtin version of
+   preserve_access_index.  */
+
+/* True iff tree T accesses any member of a struct/union/class which is marked
+   with the PRESERVE_ACCESS_INDEX attribute.  */
+
+static bool
+is_attr_preserve_access (tree t)
+{
+  if (t == NULL_TREE)
+    return false;
+
+  poly_int64 bitsize, bitpos;
+  tree var_off;
+  machine_mode mode;
+  int sign, reverse, vol;
+
+  tree base = get_inner_reference (t, &bitsize, &bitpos, &var_off, &mode,
+				   &sign, &reverse, &vol);
+
+  if (TREE_CODE (base) == MEM_REF)
+    {
+      return lookup_attribute ("preserve_access_index",
+			       TYPE_ATTRIBUTES (TREE_TYPE (base)));
+    }
+
+  if (TREE_CODE (t) == COMPONENT_REF)
+    {
+      /* preserve_access_index propegates into nested structures,
+	 so check whether this is a component of another component
+	 which in turn is part of such a struct.  */
+
+      const tree op = TREE_OPERAND (t, 0);
+
+      if (TREE_CODE (op) == COMPONENT_REF)
+	return is_attr_preserve_access (op);
+
+      const tree container = DECL_CONTEXT (TREE_OPERAND (t, 1));
+
+      return lookup_attribute ("preserve_access_index",
+			       TYPE_ATTRIBUTES (container));
+    }
+
+  else if (TREE_CODE (t) == ADDR_EXPR)
+    return is_attr_preserve_access (TREE_OPERAND (t, 0));
+
+  return false;
+}
+
+/* The body of pass_bpf_core_attr. Scan RTL for accesses to structs/unions
+   marked with __attribute__((preserve_access_index)) and generate a CO-RE
+   relocation for any such access.  */
+
+static void
+handle_attr_preserve (function *fn)
+{
+  basic_block bb;
+  rtx_insn *insn;
+  rtx_code_label *label;
+  FOR_EACH_BB_FN (bb, fn)
+    {
+      FOR_BB_INSNS (bb, insn)
+	{
+	  if (!NONJUMP_INSN_P (insn))
+	    continue;
+	  rtx pat = PATTERN (insn);
+	  if (GET_CODE (pat) != SET)
+	    continue;
+
+	  start_sequence();
+
+	  for (int i = 0; i < 2; i++)
+	    {
+	      rtx mem = XEXP (pat, i);
+	      if (MEM_P (mem))
+		{
+		  tree expr = MEM_EXPR (mem);
+		  if (!expr)
+		    continue;
+
+		  if (TREE_CODE (expr) == MEM_REF
+		      && TREE_CODE (TREE_OPERAND (expr, 0)) == SSA_NAME)
+		    {
+		      gimple *def_stmt = SSA_NAME_DEF_STMT (TREE_OPERAND (expr, 0));
+		      if (is_gimple_assign (def_stmt))
+			expr = gimple_assign_rhs1 (def_stmt);
+		    }
+
+		  if (is_attr_preserve_access (expr))
+		    {
+		      auto_vec<unsigned int, 16> accessors;
+		      tree container = bpf_core_compute (expr, &accessors);
+		      if (accessors.length () < 1)
+			continue;
+		      accessors.reverse ();
+
+		      container = TREE_TYPE (container);
+		      const char * section_name;
+		      if (DECL_SECTION_NAME (fn->decl))
+			section_name = DECL_SECTION_NAME (fn->decl);
+		      else
+			section_name = ".text";
+
+		      label = gen_label_rtx ();
+		      LABEL_PRESERVE_P (label) = 1;
+		      emit_label (label);
+
+		      /* Add the CO-RE relocation information to the BTF container.  */
+		      bpf_core_reloc_add (container, section_name, &accessors, label);
+		    }
+		}
+	    }
+	  rtx_insn *seq = get_insns ();
+	  end_sequence ();
+	  emit_insn_before (seq, insn);
+	}
+    }
+}
+
+
+/* This pass finds accesses to structures marked with the BPF target attribute
+   __attribute__((preserve_access_index)). For every such access, a CO-RE
+   relocation record is generated, to be output in the .BTF.ext section.  */
+
+namespace {
+
+const pass_data pass_data_bpf_core_attr =
+{
+  RTL_PASS, /* type */
+  "bpf_core_attr", /* name */
+  OPTGROUP_NONE, /* optinfo_flags */
+  TV_NONE, /* tv_id */
+  0, /* properties_required */
+  0, /* properties_provided */
+  0, /* properties_destroyed */
+  0, /* todo_flags_start */
+  0, /* todo_flags_finish */
+};
+
+class pass_bpf_core_attr : public rtl_opt_pass
+{
+public:
+  pass_bpf_core_attr (gcc::context *ctxt)
+    : rtl_opt_pass (pass_data_bpf_core_attr, ctxt)
+  {}
+
+  virtual bool gate (function *) { return TARGET_BPF_CORE; }
+  virtual unsigned int execute (function *);
+};
+
+unsigned int
+pass_bpf_core_attr::execute (function *fn)
+{
+  handle_attr_preserve (fn);
+  return 0;
+}
+
+} /* Anonymous namespace.  */
+
+rtl_opt_pass *
+make_pass_bpf_core_attr (gcc::context *ctxt)
+{
+  return new pass_bpf_core_attr (ctxt);
+}
+
 /* Finally, build the GCC target.  */
 
 struct gcc_target targetm = TARGET_INITIALIZER;
diff --git a/gcc/config/bpf/coreout.c b/gcc/config/bpf/coreout.c
new file mode 100644
index 00000000000..d5486b463cf
--- /dev/null
+++ b/gcc/config/bpf/coreout.c
@@ -0,0 +1,356 @@
+/* BPF Compile Once - Run Everywhere (CO-RE) support.
+   Copyright (C) 2021 Free Software Foundation, Inc.
+
+   This file is part of GCC.
+
+   GCC is free software; you can redistribute it and/or modify it
+   under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3, or (at your option)
+   any later version.
+
+   GCC is distributed in the hope that it will be useful, but
+   WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with GCC; see the file COPYING3.  If not see
+   <http://www.gnu.org/licenses/>.  */
+
+#define IN_TARGET_CODE 1
+
+#include "config.h"
+#include "system.h"
+#include "coretypes.h"
+#include "target.h"
+#include "memmodel.h"
+#include "tm_p.h"
+#include "output.h"
+#include "dwarf2asm.h"
+#include "ctfc.h"
+#include "btf.h"
+#include "rtl.h"
+
+#include "coreout.h"
+
+/* This file contains data structures and routines for construction and output
+   of BPF Compile Once - Run Everywhere (BPF CO-RE) information.
+
+   eBPF programs written in C usually include Linux kernel headers, so that
+   they may interact with kernel data structures in a useful way. This
+   intrudces two major portability issues:
+
+   1. Kernel data structures regularly change, with fields added, moved or
+      deleted between versions. An eBPF program cannot in general be expected
+      to run on any systems which does not share an identical kernel version to
+      the system on which it was compiled.
+
+   2. Included kernel headers (and used data structures) may be internal, not
+      exposed in an userspace API, and therefore target-specific. An eBPF
+      program compiled on an x86_64 machine will include x86_64 kernel headers.
+      The resulting program may not run well (or at all) in machines of
+      another architecture.
+
+   BPF CO-RE is designed to solve the first issue by leveraging the BPF loader
+   to adjust references to kernel data structures made by the program as-needed
+   according to versions of structures actually present on the host kernel.
+
+   To achieve this, additional information is placed in a ".BTF.ext" section.
+   This information tells the loader which references will require adjusting,
+   and how to perform each necessary adjustment.
+
+   For any access to a data structure which may require load-time adjustment,
+   the following information is recorded (making up a CO-RE relocation record):
+   - The BTF type ID of the outermost structure which is accessed.
+   - An access string encoding the accessed member via a series of member and
+     array indexes. These indexes are used to look up detailed BTF information
+     about the member.
+   - The offset of the appropriate instruction to patch in the BPF program.
+   - An integer specifying what kind of relocation to perform.
+
+   A CO-RE-capable BPF loader reads this information together with the BTF
+   information of the program, compares it against BTF information of the host
+   kernel, and determines the appropriate way to patch the specified
+   instruction.
+
+   Once all CO-RE relocations are resolved, the program is loaded and verified
+   as usual. The process can be summarized with the following diagram:
+
+              +------------+
+              | C compiler |
+              +-----+------+
+                    | BPF + BTF + CO-RE relocations
+                    v
+              +------------+
+         +--->| BPF loader |
+         |    +-----+------+
+         |          | BPF (adapted)
+     BTF |          v
+         |    +------------+
+         +----+   Kernel   |
+              +------------+
+
+   Note that a single ELF object may contain multiple eBPF programs. As a
+   result, a single .BTF.ext section can contain CO-RE relocations for multiple
+   programs in distinct sections.  */
+
+/* Internal representation of a BPF CO-RE relocation record.  */
+
+typedef struct GTY (()) bpf_core_reloc {
+  unsigned int bpfcr_type;		/* BTF type ID of container.  */
+  unsigned int  bpfcr_astr_off;		/* Offset of access string in .BTF
+					   string table.  */
+  rtx_code_label * bpfcr_insn_label;	/* RTX label attached to instruction
+					   to patch.  */
+  enum btf_core_reloc_kind bpfcr_kind;	/* Kind of relocation to perform.  */
+} bpf_core_reloc_t;
+
+typedef bpf_core_reloc_t * bpf_core_reloc_ref;
+
+/* Internal representation of a CO-RE relocation (sub)section of the
+   .BTF.ext information. One such section is generated for each ELF section
+   in the output object having relocations that a BPF loader must resolve.  */
+
+typedef struct GTY (()) bpf_core_section {
+  /* Name of ELF section to which these CO-RE relocations apply.  */
+  const char * name;
+
+  /* Offset of section name in .BTF string table.  */
+  uint32_t name_offset;
+
+  /* Relocations in the section.  */
+  vec <bpf_core_reloc_ref, va_gc> * GTY (()) relocs;
+} bpf_core_section_t;
+
+typedef bpf_core_section_t * bpf_core_section_ref;
+
+/* BTF.ext debug info section.  */
+
+static GTY (()) section * btf_ext_info_section;
+
+static int btf_ext_label_num;
+
+#ifndef BTF_EXT_INFO_SECTION_NAME
+#define BTF_EXT_INFO_SECTION_NAME ".BTF.ext"
+#endif
+
+#define BTF_EXT_INFO_SECTION_FLAGS (SECTION_DEBUG)
+
+#define MAX_BTF_EXT_LABEL_BYTES 40
+
+static char btf_ext_info_section_label[MAX_BTF_EXT_LABEL_BYTES];
+
+#ifndef BTF_EXT_INFO_SECTION_LABEL
+#define BTF_EXT_INFO_SECTION_LABEL "Lbtfext"
+#endif
+
+static GTY (()) vec<bpf_core_section_ref, va_gc> *bpf_core_sections;
+
+
+/* Create a new BPF CO-RE relocation record, and add it to the appropriate
+   CO-RE section.  */
+
+void
+bpf_core_reloc_add (const tree type, const char * section_name,
+		    vec<unsigned int> *accessors, rtx_code_label *label)
+{
+  char buf[40];
+  unsigned int i, n = 0;
+
+  /* A valid CO-RE access must have at least one accessor.  */
+  if (accessors->length () < 1)
+    return;
+
+  for (i = 0; i < accessors->length () - 1; i++)
+    n += snprintf (buf + n, sizeof (buf) - n, "%u:", (*accessors)[i]);
+  snprintf (buf + n, sizeof (buf) - n, "%u", (*accessors)[i]);
+
+  bpf_core_reloc_ref bpfcr = ggc_cleared_alloc<bpf_core_reloc_t> ();
+  ctf_container_ref ctfc = ctf_get_tu_ctfc ();
+
+  /* Buffer the access string in the auxiliary strtab. Since the two string
+     tables are concatenated, add the length of the first to the offset.  */
+  size_t strtab_len = ctfc_get_strtab_len (ctfc, CTF_STRTAB);
+  ctf_add_string (ctfc, buf, &(bpfcr->bpfcr_astr_off), CTF_AUX_STRTAB);
+  bpfcr->bpfcr_astr_off += strtab_len;
+
+  bpfcr->bpfcr_type = get_btf_id (ctf_lookup_tree_type (ctfc, type));
+  bpfcr->bpfcr_insn_label = label;
+  bpfcr->bpfcr_kind = BPF_RELO_FIELD_BYTE_OFFSET;
+
+  /* Add the CO-RE reloc to the appropriate section.  */
+  bpf_core_section_ref sec;
+  FOR_EACH_VEC_ELT (*bpf_core_sections, i, sec)
+    if (strcmp (sec->name, section_name) == 0)
+      {
+	vec_safe_push (sec->relocs, bpfcr);
+	return;
+      }
+
+  /* If the CO-RE section does not yet exist, create it.  */
+  sec = ggc_cleared_alloc<bpf_core_section_t> ();
+
+  ctf_add_string (ctfc, section_name, &sec->name_offset, CTF_AUX_STRTAB);
+  sec->name_offset += strtab_len;
+  if (strcmp (section_name, ""))
+    ctfc->ctfc_aux_strlen += strlen (section_name) + 1;
+
+  sec->name = section_name;
+  vec_alloc (sec->relocs, 1);
+  vec_safe_push (sec->relocs, bpfcr);
+
+  vec_safe_push (bpf_core_sections, sec);
+}
+
+/* Return the 0-based index of the field NODE in its containing struct or union
+   type.  */
+
+int
+bpf_core_get_sou_member_index (ctf_container_ref ctfc, const tree node)
+{
+  if (TREE_CODE (node) == FIELD_DECL)
+    {
+      const tree container = DECL_CONTEXT (node);
+      const char * name = IDENTIFIER_POINTER (DECL_NAME (node));
+
+      /* Lookup the CTF type info for the containing type.  */
+      dw_die_ref die = lookup_type_die (container);
+      if (die == NULL)
+        return -1;
+
+      ctf_dtdef_ref dtd = ctf_dtd_lookup (ctfc, die);
+      if (dtd == NULL)
+        return -1;
+
+      unsigned int kind = CTF_V2_INFO_KIND (dtd->dtd_data.ctti_info);
+      if (kind != CTF_K_STRUCT && kind != CTF_K_UNION)
+        return -1;
+
+      int i = 0;
+      ctf_dmdef_t * dmd;
+      for (dmd = dtd->dtd_u.dtu_members;
+           dmd != NULL; dmd = (ctf_dmdef_t *) ctf_dmd_list_next (dmd))
+        {
+          if (get_btf_id (dmd->dmd_type) > BTF_MAX_TYPE)
+            continue;
+          if (strcmp (dmd->dmd_name, name) == 0)
+            return i;
+          i++;
+        }
+    }
+  return -1;
+}
+
+/* Compute and output the header of a .BTF.ext debug info section.  */
+
+static void
+output_btfext_header (void)
+{
+  switch_to_section (btf_ext_info_section);
+  ASM_OUTPUT_LABEL (asm_out_file, btf_ext_info_section_label);
+
+  dw2_asm_output_data (2, BTF_MAGIC, "btf_magic");
+  dw2_asm_output_data (1, BTF_VERSION, "btfext_version");
+  dw2_asm_output_data (1, 0, "btfext_flags");
+  dw2_asm_output_data (4, sizeof (struct btf_ext_header), "btfext_hdr_len");
+
+  uint32_t func_info_off = 0, func_info_len = 0;
+  uint32_t line_info_off = 0, line_info_len = 0;
+  uint32_t core_relo_off = 0, core_relo_len = 0;
+
+  /* Header core_relo_len is the sum total length in bytes of all CO-RE
+     relocation sections.  */
+  size_t i;
+  bpf_core_section_ref sec;
+  core_relo_len += vec_safe_length (bpf_core_sections)
+    * sizeof (struct btf_ext_section_header);
+
+  FOR_EACH_VEC_ELT (*bpf_core_sections, i, sec)
+    core_relo_len +=
+      vec_safe_length (sec->relocs) * sizeof (struct btf_ext_reloc);
+
+  dw2_asm_output_data (4, func_info_off, "func_info_offset");
+  dw2_asm_output_data (4, func_info_len, "func_info_len");
+
+  dw2_asm_output_data (4, line_info_off, "line_info_offset");
+  dw2_asm_output_data (4, line_info_len, "line_info_len");
+
+  dw2_asm_output_data (4, core_relo_off, "core_relo_offset");
+  dw2_asm_output_data (4, core_relo_len, "core_relo_len");
+}
+
+/* Output a single CO-RE relocation record.  */
+
+static void
+output_asm_btfext_core_reloc (bpf_core_reloc_ref bpfcr)
+{
+  dw2_assemble_integer (4, gen_rtx_LABEL_REF (Pmode, bpfcr->bpfcr_insn_label));
+  fprintf (asm_out_file, "\t%s bpfcr_insn\n", ASM_COMMENT_START);
+
+  dw2_asm_output_data (4, bpfcr->bpfcr_type, "bpfcr_type");
+  dw2_asm_output_data (4, bpfcr->bpfcr_astr_off, "bpfcr_astr_off");
+  dw2_asm_output_data (4, bpfcr->bpfcr_kind, "bpfcr_kind");
+}
+
+/* Output all CO-RE relocation records for a section.  */
+
+static void
+output_btfext_core_relocs (bpf_core_section_ref sec)
+{
+  size_t i;
+  bpf_core_reloc_ref bpfcr;
+  FOR_EACH_VEC_ELT (*(sec->relocs), i, bpfcr)
+    output_asm_btfext_core_reloc (bpfcr);
+}
+
+/* Output all CO-RE relocation sections.  */
+
+static void
+output_btfext_core_sections (void)
+{
+  size_t i;
+  bpf_core_section_ref sec;
+  FOR_EACH_VEC_ELT (*bpf_core_sections, i, sec)
+    {
+      /* BTF Ext section info. */
+      dw2_asm_output_data (4, sizeof (struct btf_ext_reloc),
+			   "btfext_secinfo_rec_size");
+
+      /* Section name offset, refers to the offset of a string with the name of
+	 the section to which these CORE relocations refer, e.g. '.text'.
+	 The string is buffered in the BTF strings table.  */
+      dw2_asm_output_data (4, sec->name_offset,  "btfext_secinfo_sec_name_off");
+      dw2_asm_output_data (4, vec_safe_length (sec->relocs),
+			   "btfext_secinfo_num_recs");
+
+      output_btfext_core_relocs (sec);
+    }
+}
+
+/* Initialize sections, labels, and data structures for BTF.ext output.  */
+
+void
+btf_ext_init (void)
+{
+  btf_ext_info_section = get_section (BTF_EXT_INFO_SECTION_NAME,
+				      BTF_EXT_INFO_SECTION_FLAGS, NULL);
+
+  ASM_GENERATE_INTERNAL_LABEL (btf_ext_info_section_label,
+			       BTF_EXT_INFO_SECTION_LABEL,
+			       btf_ext_label_num++);
+
+  vec_alloc (bpf_core_sections, 1);
+}
+
+/* Output the entire .BTF.ext section.  */
+
+void
+btf_ext_output (void)
+{
+  output_btfext_header ();
+  output_btfext_core_sections ();
+
+  bpf_core_sections = NULL;
+}
+
+#include "gt-coreout.h"
diff --git a/gcc/config/bpf/coreout.h b/gcc/config/bpf/coreout.h
new file mode 100644
index 00000000000..82c203df341
--- /dev/null
+++ b/gcc/config/bpf/coreout.h
@@ -0,0 +1,114 @@
+/* coreout.h - Declarations and definitions related to
+   BPF Compile Once - Run Everywhere (CO-RE) support.
+   Copyright (C) 2021 Free Software Foundation, Inc.
+
+   This file is part of GCC.
+
+   GCC is free software; you can redistribute it and/or modify it
+   under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3, or (at your option)
+   any later version.
+
+   GCC is distributed in the hope that it will be useful, but
+   WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with GCC; see the file COPYING3.  If not see
+   <http://www.gnu.org/licenses/>.  */
+
+
+#ifndef __COREOUT_H
+#define __COREOUT_H
+
+#include <stdint.h>
+
+#ifdef	__cplusplus
+extern "C"
+{
+#endif
+
+/* .BTF.ext information.  */
+
+struct btf_ext_section_header
+{
+  uint32_t kind;
+  uint32_t sec_name_off;
+  uint32_t num_records;
+};
+
+/* A funcinfo record, in the .BTF.ext funcinfo section.  */
+struct btf_ext_funcinfo
+{
+  uint32_t insn_off; /* Offset of the first instruction of the function.  */
+  uint32_t type;     /* Type ID of a BTF_KIND_FUNC type.  */
+};
+
+/* A lineinfo record, in the .BTF.ext lineinfo section.  */
+struct btf_ext_lineinfo
+{
+  uint32_t insn_off;      /* Offset of the instruction.  */
+  uint32_t file_name_off; /* Offset of file name in BTF string table.  */
+  uint32_t line_off;      /* Offset of source line in BTF string table.  */
+  uint32_t line_col;      /* Line number (bits 31-11) and column (11-0).  */
+};
+
+enum btf_core_reloc_kind
+{
+  BPF_RELO_FIELD_BYTE_OFFSET = 0,
+  BPF_RELO_FIELD_BYTE_SIZE = 1,
+  BPF_RELO_FIELD_EXISTS = 2,
+  BPF_RELO_FIELD_SIGNED = 3,
+  BPF_RELO_FIELD_LSHIFT_U64 = 4,
+  BPF_RELO_FIELD_RSHIFT_U64 = 5,
+  BPF_RELO_TYPE_ID_LOCAL = 6,
+  BPF_RELO_TYPE_ID_TARGET = 7,
+  BPF_RELO_TYPE_EXISTS = 8,
+  BPF_RELO_TYPE_SIZE = 9,
+  BPF_RELO_ENUMVAL_EXISTS = 10,
+  BPF_RELO_ENUMVAL_VALUE = 11
+};
+
+struct btf_ext_reloc
+{
+  uint32_t insn_off;       /* Offset of instruction to be patched. A
+			      section-relative label at compile time.  */
+  uint32_t type_id;        /* Type ID of the outermost containing entity, e.g.
+			      the containing structure.  */
+  uint32_t access_str_off; /* Offset of CO-RE accessor string in .BTF strings
+			      section.  */
+  uint32_t kind;           /* An enum btf_core_reloc_kind. Note that it always
+			      takes 32 bits.  */
+};
+
+struct btf_ext_header
+{
+  uint16_t magic;		/* Magic number (BTF_MAGIC).  */
+  uint8_t  version;		/* Data format version (BTF_VERSION).  */
+  uint8_t  flags;		/* Flags. Currently unused.  */
+  uint32_t hdr_len;		/* Length of this header in bytes.  */
+
+  /* Following offsets are relative to the end of this header, in bytes.
+     Following lengths are in bytes.  */
+  uint32_t func_info_off;	/* Offset of funcinfo section.  */
+  uint32_t func_info_len;	/* Length of funcinfo section.  */
+  uint32_t line_info_off;	/* Offset of lineinfo section.  */
+  uint32_t line_info_len;	/* Length of lineinfo section.  */
+
+  uint32_t core_relo_off;	/* Offset of CO-RE relocation section.  */
+  uint32_t core_relo_len;	/* Length of CO-RE relocation section.  */
+};
+
+extern void btf_ext_init (void);
+extern void btf_ext_output (void);
+
+extern void bpf_core_reloc_add (const tree, const char *, vec<unsigned int> *,
+				rtx_code_label *);
+extern int bpf_core_get_sou_member_index (ctf_container_ref, const tree);
+
+#ifdef	__cplusplus
+}
+#endif
+
+#endif /* __COREOUT_H */
diff --git a/gcc/config/bpf/t-bpf b/gcc/config/bpf/t-bpf
index e69de29bb2d..b37bf858d8f 100644
--- a/gcc/config/bpf/t-bpf
+++ b/gcc/config/bpf/t-bpf
@@ -0,0 +1,8 @@
+
+TM_H += $(srcdir)/config/bpf/coreout.h
+
+coreout.o: $(srcdir)/config/bpf/coreout.c
+	$(COMPILE) $<
+	$(POSTCOMPILE)
+
+PASSES_EXTRA += $(srcdir)/config/bpf/bpf-passes.def
-- 
2.33.0

[COMMITTED V2 6/7] bpf testsuite: Add BPF CO-RE tests

[David Faust]

This commit adds several tests for the new BPF CO-RE functionality to
the BPF target testsuite.

gcc/testsuite/ChangeLog:

	* gcc.target/bpf/core-attr-1.c: New test.
	* gcc.target/bpf/core-attr-2.c: Likewise.
	* gcc.target/bpf/core-attr-3.c: Likewise.
	* gcc.target/bpf/core-attr-4.c: Likewise
	* gcc.target/bpf/core-builtin-1.c: Likewise
	* gcc.target/bpf/core-builtin-2.c: Likewise.
	* gcc.target/bpf/core-builtin-3.c: Likewise.
	* gcc.target/bpf/core-section-1.c: Likewise.
---
 gcc/testsuite/gcc.target/bpf/core-attr-1.c    | 23 +++++++
 gcc/testsuite/gcc.target/bpf/core-attr-2.c    | 21 ++++++
 gcc/testsuite/gcc.target/bpf/core-attr-3.c    | 41 ++++++++++++
 gcc/testsuite/gcc.target/bpf/core-attr-4.c    | 35 ++++++++++
 gcc/testsuite/gcc.target/bpf/core-builtin-1.c | 64 +++++++++++++++++++
 gcc/testsuite/gcc.target/bpf/core-builtin-2.c | 26 ++++++++
 gcc/testsuite/gcc.target/bpf/core-builtin-3.c | 26 ++++++++
 gcc/testsuite/gcc.target/bpf/core-section-1.c | 38 +++++++++++
 8 files changed, 274 insertions(+)
 create mode 100644 gcc/testsuite/gcc.target/bpf/core-attr-1.c
 create mode 100644 gcc/testsuite/gcc.target/bpf/core-attr-2.c
 create mode 100644 gcc/testsuite/gcc.target/bpf/core-attr-3.c
 create mode 100644 gcc/testsuite/gcc.target/bpf/core-attr-4.c
 create mode 100644 gcc/testsuite/gcc.target/bpf/core-builtin-1.c
 create mode 100644 gcc/testsuite/gcc.target/bpf/core-builtin-2.c
 create mode 100644 gcc/testsuite/gcc.target/bpf/core-builtin-3.c
 create mode 100644 gcc/testsuite/gcc.target/bpf/core-section-1.c

diff --git a/gcc/testsuite/gcc.target/bpf/core-attr-1.c b/gcc/testsuite/gcc.target/bpf/core-attr-1.c
new file mode 100644
index 00000000000..1af9dc5ea6d
--- /dev/null
+++ b/gcc/testsuite/gcc.target/bpf/core-attr-1.c
@@ -0,0 +1,23 @@
+/* Basic test for struct __attribute__((preserve_access_index))
+   for BPF CO-RE support.  */
+
+/* { dg-do compile } */
+/* { dg-options "-O0 -dA -gbtf -mco-re" } */
+
+struct S {
+  int a;
+  int b;
+  int c;
+} __attribute__((preserve_access_index));
+
+void
+func (struct S * s)
+{
+  /* 0:2 */
+  int *x = &(s->c);
+
+  *x = 4;
+}
+
+/* { dg-final { scan-assembler-times "ascii \"0:2.0\"\[\t \]+\[^\n\]*btf_aux_string" 1 } } */
+/* { dg-final { scan-assembler-times "bpfcr_type" 1 } } */
diff --git a/gcc/testsuite/gcc.target/bpf/core-attr-2.c b/gcc/testsuite/gcc.target/bpf/core-attr-2.c
new file mode 100644
index 00000000000..25c819a0082
--- /dev/null
+++ b/gcc/testsuite/gcc.target/bpf/core-attr-2.c
@@ -0,0 +1,21 @@
+/* Basic test for union __attribute__((preserve_access_index))
+   for BPF CO-RE support.  */
+
+/* { dg-do compile } */
+/* { dg-options "-O0 -dA -gbtf -mco-re" } */
+
+union U {
+  int a;
+  char c;
+} __attribute__((preserve_access_index));
+
+void
+func (union U *u)
+{
+  /* 0:1 */
+  char *c = &(u->c);
+  *c = 'c';
+}
+
+/* { dg-final { scan-assembler-times "ascii \"0:1.0\"\[\t \]+\[^\n\]*btf_aux_string" 1 } } */
+/* { dg-final { scan-assembler-times "bpfcr_type" 1 } } */
diff --git a/gcc/testsuite/gcc.target/bpf/core-attr-3.c b/gcc/testsuite/gcc.target/bpf/core-attr-3.c
new file mode 100644
index 00000000000..b46549f788c
--- /dev/null
+++ b/gcc/testsuite/gcc.target/bpf/core-attr-3.c
@@ -0,0 +1,41 @@
+/* Test for __attribute__((preserve_access_index)) for BPF CO-RE support
+   for nested structure.
+
+   Note that even though struct O lacks the attribute, when accessed as a
+   member of another attributed type, CO-RE relocations should still be
+   generated.  */
+
+/* { dg-do compile } */
+/* { dg-options "-O0 -dA -gbtf -mco-re" } */
+
+struct O {
+  int e;
+  int f;
+};
+
+struct S {
+  int a;
+  struct {
+    int b;
+    int c;
+  } inner;
+  struct O other;
+} __attribute__((preserve_access_index));
+
+void
+func (struct S *foo)
+{
+  /* 0:1:1 */
+  int *x = &(foo->inner.c);
+
+  /* 0:2:0 */
+  int *y = &(foo->other.e);
+
+  *x = 4;
+  *y = 5;
+}
+
+/* { dg-final { scan-assembler-times "ascii \"0:1:1.0\"\[\t \]+\[^\n\]*btf_aux_string" 1 } } */
+/* { dg-final { scan-assembler-times "ascii \"0:2:0.0\"\[\t \]+\[^\n\]*btf_aux_string" 1 } } */
+
+/* { dg-final { scan-assembler-times "bpfcr_type" 2 } } */
diff --git a/gcc/testsuite/gcc.target/bpf/core-attr-4.c b/gcc/testsuite/gcc.target/bpf/core-attr-4.c
new file mode 100644
index 00000000000..9c0f966b556
--- /dev/null
+++ b/gcc/testsuite/gcc.target/bpf/core-attr-4.c
@@ -0,0 +1,35 @@
+/* Test for BPF CO-RE __attribute__((preserve_access_index)) with accesses on
+   LHS and both LHS and RHS of assignment.  */
+
+/* { dg-do compile } */
+/* { dg-options "-O0 -dA -gbtf -mco-re" } */
+
+struct T {
+  int a;
+  int b;
+  struct U {
+    int c;
+    struct V {
+      int d;
+      int e[4];
+      int f;
+    } v;
+  } u;
+} __attribute__((preserve_access_index));
+
+
+void
+func (struct T *t)
+{
+  /* 0:2:1:1:3 */
+  t->u.v.e[3] = 0xa1;
+
+  /* 0:2:0, 0:0, 0:1 */
+  t->u.c = t->a + t->b;
+}
+
+/* { dg-final { scan-assembler-times "ascii \"0:2:1:1:3.0\"\[\t \]+\[^\n\]*btf_aux_string" 1 } } */
+/* { dg-final { scan-assembler-times "ascii \"0:2:0.0\"\[\t \]+\[^\n\]*btf_aux_string" 1 } } */
+/* { dg-final { scan-assembler-times "ascii \"0:0.0\"\[\t \]+\[^\n\]*btf_aux_string" 1 } } */
+/* { dg-final { scan-assembler-times "ascii \"0:1.0\"\[\t \]+\[^\n\]*btf_aux_string" 1 } } */
+/* { dg-final { scan-assembler-times "bpfcr_type" 4 } } */
diff --git a/gcc/testsuite/gcc.target/bpf/core-builtin-1.c b/gcc/testsuite/gcc.target/bpf/core-builtin-1.c
new file mode 100644
index 00000000000..e994dc6add4
--- /dev/null
+++ b/gcc/testsuite/gcc.target/bpf/core-builtin-1.c
@@ -0,0 +1,64 @@
+/* { dg-do compile } */
+/* { dg-options "-O0 -dA -gbtf -mco-re" } */
+
+struct S {
+  int a;
+  int b;
+  char c;
+};
+
+union U {
+  unsigned int u;
+  int i;
+  unsigned char uc[4];
+  signed char ic[4];
+};
+
+struct S my_s;
+union U my_u;
+
+unsigned long ula[8];
+
+#define _(x) (__builtin_preserve_access_index (x))
+
+void
+func (void)
+{
+  /* 1 */
+  int b = _(my_s.b);
+
+  /* 2 */
+  char c = _(my_s.c);
+
+  /* 2:3 */
+  unsigned char uc = _(my_u.uc[3]);
+
+  /* 6 */
+  unsigned long ul = _(ula[6]);
+}
+
+char
+s_ptr (struct S *ps)
+{
+  /* 0:2 */
+  char x = _(ps->c);
+  return x;
+}
+
+unsigned char
+u_ptr (union U *pu)
+{
+  /* 0:2:3 */
+  unsigned char x = _(pu->uc[3]);
+  return x;
+}
+
+/* { dg-final { scan-assembler-times "ascii \"1.0\"\[\t \]+\[^\n\]*btf_aux_string" 1 } } */
+/* { dg-final { scan-assembler-times "ascii \"2.0\"\[\t \]+\[^\n\]*btf_aux_string" 1 } } */
+/* { dg-final { scan-assembler-times "ascii \"2:3.0\"\[\t \]+\[^\n\]*btf_aux_string" 1 } } */
+/* { dg-final { scan-assembler-times "ascii \"6.0\"\[\t \]+\[^\n\]*btf_aux_string" 1 } } */
+/* { dg-final { scan-assembler-times "ascii \"0:2.0\"\[\t \]+\[^\n\]*btf_aux_string" 1 } } */
+/* { dg-final { scan-assembler-times "ascii \"0:2:3.0\"\[\t \]+\[^\n\]*btf_aux_string" 1 } } */
+
+/* { dg-final { scan-assembler-times "bpfcr_type" 6 } } */
+/* { dg-final { scan-assembler-times "\[\t \]0x6c\[\t \]+\[^\n\]*core_relo_len" 1 } } */
diff --git a/gcc/testsuite/gcc.target/bpf/core-builtin-2.c b/gcc/testsuite/gcc.target/bpf/core-builtin-2.c
new file mode 100644
index 00000000000..c9ec8994ae3
--- /dev/null
+++ b/gcc/testsuite/gcc.target/bpf/core-builtin-2.c
@@ -0,0 +1,26 @@
+/* { dg-do compile } */
+/* { dg-options "-O0 -dA -gbtf -mco-re" } */
+
+struct S {
+  int a;
+  union {
+    int _unused;
+    int b;
+    char c;
+  } u[4];
+};
+
+struct S foo;
+
+#define _(x) (__builtin_preserve_access_index (x))
+
+void func (void)
+{
+  char *x = __builtin_preserve_access_index (&foo.u[3].c);
+
+  *x = 's';
+}
+
+/* { dg-final { scan-assembler-times "\[\t \]0x4000002\[\t \]+\[^\n\]*btt_info" 1 } } */
+/* { dg-final { scan-assembler-times "ascii \"1:3:2.0\"\[\t \]+\[^\n\]*btf_aux_string" 1 } } */
+/* { dg-final { scan-assembler-times "bpfcr_type" 1 } } */
diff --git a/gcc/testsuite/gcc.target/bpf/core-builtin-3.c b/gcc/testsuite/gcc.target/bpf/core-builtin-3.c
new file mode 100644
index 00000000000..190ec2657d3
--- /dev/null
+++ b/gcc/testsuite/gcc.target/bpf/core-builtin-3.c
@@ -0,0 +1,26 @@
+/* { dg-do compile } */
+/* { dg-options "-O0 -dA -gbtf -mco-re" } */
+
+struct T {
+  int a;
+  int b;
+  struct U {
+    int c;
+    struct V {
+      int d;
+      int e[4];
+      int f;
+    } v;
+  } u;
+};
+
+void func (struct T * foo)
+{
+  /* Access string: "0:2:1:1:3" */
+  int *x = __builtin_preserve_access_index (&(foo->u.v.e[3]));
+
+  *x = 17;
+}
+
+/* { dg-final { scan-assembler-times "ascii \"0:2:1:1:3.0\"\[\t \]+\[^\n\]*btf_aux_string" 1 } } */
+/* { dg-final { scan-assembler-times "bpfcr_type" 1 } } */
diff --git a/gcc/testsuite/gcc.target/bpf/core-section-1.c b/gcc/testsuite/gcc.target/bpf/core-section-1.c
new file mode 100644
index 00000000000..031acd5292e
--- /dev/null
+++ b/gcc/testsuite/gcc.target/bpf/core-section-1.c
@@ -0,0 +1,38 @@
+/* { dg-do compile } */
+/* { dg-options "-O0 -gbtf -dA -mco-re" } */
+
+struct T {
+  int a;
+  int b;
+  struct U {
+    int c;
+    struct V {
+      int d;
+      int e[4];
+      int f;
+    } v;
+  } u;
+};
+
+__attribute__((section("foo_sec"), used))
+int foo_func (struct T *t)
+{
+  t->u.c = 5;
+  return __builtin_preserve_access_index (t->u.v.e[3]);
+}
+
+__attribute__((section("bar_sec"), used))
+int bar_func (struct T *t)
+{
+  int *x = __builtin_preserve_access_index (&(t->u.v.f));
+  int old = *x;
+  *x = 4;
+  return old;
+}
+
+/* { dg-final { scan-assembler-times "ascii \"0:2:1:1:3.0\"\[\t \]+\[^\n\]*btf_aux_string" 1 } } */
+/* { dg-final { scan-assembler-times "ascii \"0:2:1:2.0\"\[\t \]+\[^\n\]*btf_aux_string" 1 } } */
+/* { dg-final { scan-assembler-times "ascii \"foo_sec.0\"\[\t \]+\[^\n\]*btf_aux_string" 1 } } */
+/* { dg-final { scan-assembler-times "ascii \"bar_sec.0\"\[\t \]+\[^\n\]*btf_aux_string" 1 } } */
+/* { dg-final { scan-assembler-times "bpfcr_type" 2 } } */
+/* { dg-final { scan-assembler-times "btfext_secinfo_rec_size" 2 } } */
-- 
2.33.0

[COMMITTED V2 7/7] doc: BPF CO-RE documentation

[David Faust]

Document the new command line options (-mco-re and -mno-co-re), the new
BPF target builtin (__builtin_preserve_access_index), and the new BPF
target attribute (preserve_access_index) introduced with BPF CO-RE.

gcc/ChangeLog:

	* doc/extend.texi (BPF Type Attributes) New node.
	Document new preserve_access_index attribute.
	Document new preserve_access_index builtin.
	* doc/invoke.texi: Document -mco-re and -mno-co-re options.
---
 gcc/doc/extend.texi | 16 ++++++++++++++++
 gcc/doc/invoke.texi | 13 ++++++++++++-
 2 files changed, 28 insertions(+), 1 deletion(-)

diff --git a/gcc/doc/extend.texi b/gcc/doc/extend.texi
index 7fb22ed8063..31319f7dd08 100644
--- a/gcc/doc/extend.texi
+++ b/gcc/doc/extend.texi
@@ -8256,6 +8256,7 @@ attributes.
 * Common Type Attributes::
 * ARC Type Attributes::
 * ARM Type Attributes::
+* BPF Type Attributes::
 * MeP Type Attributes::
 * PowerPC Type Attributes::
 * x86 Type Attributes::
@@ -8830,6 +8831,17 @@ virtual table for @code{C} is not exported.  (You can use
 @code{__attribute__} instead of @code{__declspec} if you prefer, but
 most Symbian OS code uses @code{__declspec}.)
 
+@node BPF Type Attributes
+@subsection BPF Type Attributes
+
+@cindex @code{preserve_access_index} type attribute, BPF
+BPF Compile Once - Run Everywhere (CO-RE) support. When attached to a
+@code{struct} or @code{union} type definition, indicates that CO-RE
+relocation information should be generated for any access to a variable
+of that type. The behavior is equivalent to the programmer manually
+wrapping every such access with @code{__builtin_preserve_access_index}.
+
+
 @node MeP Type Attributes
 @subsection MeP Type Attributes
 
@@ -15467,6 +15479,10 @@ Load 16-bits from the @code{struct sk_buff} packet data pointed by the register
 Load 32-bits from the @code{struct sk_buff} packet data pointed by the register @code{%r6} and return it.
 @end deftypefn
 
+@deftypefn {Built-in Function} void * __builtin_preserve_access_index (@var{expr})
+BPF Compile Once-Run Everywhere (CO-RE) support. Instruct GCC to generate CO-RE relocation records for any accesses to aggregate data structures (struct, union, array types) in @var{expr}. This builtin is otherwise transparent, the return value is whatever @var{expr} evaluates to. It is also overloaded: @var{expr} may be of any type (not necessarily a pointer), the return type is the same. Has no effect if @code{-mco-re} is not in effect (either specified or implied).
+@end deftypefn
+
 @node FR-V Built-in Functions
 @subsection FR-V Built-in Functions
 
diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index a9580a08a65..e39dde009ef 100644
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -904,7 +904,7 @@ Objective-C and Objective-C++ Dialects}.
 
 @emph{eBPF Options}
 @gccoptlist{-mbig-endian -mlittle-endian -mkernel=@var{version}
--mframe-limit=@var{bytes} -mxbpf}
+-mframe-limit=@var{bytes} -mxbpf -mco-re -mno-co-re}
 
 @emph{FR30 Options}
 @gccoptlist{-msmall-model  -mno-lsim}
@@ -22635,6 +22635,17 @@ Generate code for a big-endian target.
 @opindex mlittle-endian
 Generate code for a little-endian target.  This is the default.
 
+@item -mco-re
+@opindex mco-re
+Enable BPF Compile Once - Run Everywhere (CO-RE) support. Requires and
+is implied by @option{-gbtf}.
+
+@item -mno-co-re
+@opindex mno-co-re
+Disable BPF Compile Once - Run Everywhere (CO-RE) support. BPF CO-RE
+support is enabled by default when generating BTF debug information for
+the BPF target.
+
 @item -mxbpf
 Generate code for an expanded version of BPF, which relaxes some of
 the restrictions imposed by the BPF architecture:
-- 
2.33.0