From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by sourceware.org (Postfix) with ESMTPS id 4A5E63839428 for ; Wed, 4 Aug 2021 17:55:12 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 4A5E63839428 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 174Hpavs031167 for ; Wed, 4 Aug 2021 17:55:11 GMT Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by mx0b-00069f02.pphosted.com with ESMTP id 3a7aq0apxu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Wed, 04 Aug 2021 17:55:10 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 174HpUZq031193 for ; Wed, 4 Aug 2021 17:55:09 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2109.outbound.protection.outlook.com [104.47.58.109]) by userp3030.oracle.com with ESMTP id 3a4un24yd8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Wed, 04 Aug 2021 17:55:09 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=n2GQIFTaKymXHXuqgZkvhyKNHGyZRTcTjM5sDbPXA9nKCyT323TJr5hHzmvRxgqImZlfag6AKCOXx80OXwDdYXIyBJC0gNJsIJ9Fk0SpgQN156MtoPLrERjM5g4ArpQnSyDgGR0nJ+UY3akcgWnbu8wTaA2E4m2NawOGP/4YLDOdBGJ/L+yFRhR6stWRqnCgUtaEM8VqSMP8JWvHgDqXnWegbdwH+a6vkIlFdosh+fpQHlR7Vmmi1FtRktlf9XpNx4rMRa0SXVGK0WYQAOGvZgDn2e+Wu0D0iKRmYVUpJWjmL6Etn45GeOR7U6L8Nip4hGzcwIDIYUKxdL5XEipSbA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rb4rcAgu822KD5S+Ryi3WxvKagrsXFYtQmydjFsvTXw=; b=UNEXM2GhsyHQAoI3dgXTGi1VgCnlgg6GoMGKk4pI7UhRSMowESsUMouUNJX/a+byBIRTF/TmfcQDB9ZrF5+cTk3Sk5wJYu1jeUhKFc7Z61ITJKDlnRsKpaNezvarqQEumKH30nlx84aYQDfxOQZI5VqEo70jFFqdMvnnhy36wAuW/DhIi4W6h7woQfV+zvZZYic/2bB98JrVgRAf0bzDYdFwAcpsSYYyIU6PYocKjqC+L5y10YW92yLL0ezx+tnYHeB5b0Ow2AZNEPYPuczFFGePapqLzIQxf9Dvsso589pEI/hOJJkCDyw1GSyg/03sq2JKktL7q4Myh5317tqxBw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none Received: from SA2PR10MB4458.namprd10.prod.outlook.com (2603:10b6:806:f8::24) by SA2PR10MB4555.namprd10.prod.outlook.com (2603:10b6:806:115::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4394.16; Wed, 4 Aug 2021 17:55:07 +0000 Received: from SA2PR10MB4458.namprd10.prod.outlook.com ([fe80::9d08:a59a:abb8:3632]) by SA2PR10MB4458.namprd10.prod.outlook.com ([fe80::9d08:a59a:abb8:3632%5]) with mapi id 15.20.4394.016; Wed, 4 Aug 2021 17:55:07 +0000 From: David Faust To: gcc-patches@gcc.gnu.org Subject: [PATCH 0/7] BPF CO-RE Support Date: Wed, 4 Aug 2021 10:54:04 -0700 Message-Id: <20210804175411.6783-1-david.faust@oracle.com> X-Mailer: git-send-email 2.31.1 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: SA0PR11CA0203.namprd11.prod.outlook.com (2603:10b6:806:1bc::28) To SA2PR10MB4458.namprd10.prod.outlook.com (2603:10b6:806:f8::24) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from sequence.us.oracle.com (2606:b400:8004:44::1f) by SA0PR11CA0203.namprd11.prod.outlook.com (2603:10b6:806:1bc::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4394.17 via Frontend Transport; Wed, 4 Aug 2021 17:55:07 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9dfaa50a-741e-45ce-5f83-08d95770fede X-MS-TrafficTypeDiagnostic: SA2PR10MB4555: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: mQ7XYPMCCasd+svh7zDQWvEOnzW0ePIo3ybEoLfx22epXuZdxJlaQFhqHN/VUg8fyYAX6tUzOFeg2Qm5dvei4NiDrGEMuDzfAfQ9zdYlWwgzvATXm99W0fiuJ5fHwEs7/46FK72Fleu93UhKqRoefaYDiM05jThxo5q32pIRDn8RY20N8hxyIfVDENmgTS3x0fhQ+jpqZjeDYTn0lujgobWree+sexVg3TDbjGt/hDymXsX2uXWFds06SjeHiop551db9XxYygDqtMKzOaQTohc0usRi2rMw/puxxv1z6Dgj2BUwkH3VcGegk4Q4lTgY7XtpqUb+PMDL9LxDCAFy+l+cxVZbQAtxY8fucUwDPObdmyaLO57CJ2HmBfk2cVV6ouzsoyX3XHYtLsxP39rPxpLBbqCnH1EIobGshF0/iz6LUgB2Rxq3svcTiM5XRsl/fNNQulZahZqVR9rLrCALn+uphWfsSP8c9qUHetnDsCs4HtPy545tF0oCIJa6GstrkKU9okAD374DxEi84U7ga+pfTj8baO2lSvlff40THGVUFiafkCm8m1o+CrO2cunBxPdsB4kKELyzjZjuCRYzcaqadSU7v5CxE2SqtKdsDi7EWYtWNtxUSVG+guWUd2boGBR8lwcFNTU8QnBjOSaUZLzjOFQijaaGhtW/S8bW0WZbbD+xBM/VsOhHZjAhkE0tiSxAlWbtw32Owa/oj/JfQU12Ex99aWjLxPJbr4gUdSw= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SA2PR10MB4458.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(6486002)(44832011)(316002)(8936002)(66476007)(66946007)(6916009)(83380400001)(66556008)(86362001)(5660300002)(8676002)(7696005)(38100700002)(966005)(52116002)(508600001)(36756003)(186003)(2906002)(6666004)(1076003)(2616005); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?gm31FltgersfkpV7uqixThY9Pq99WNlp8LcOKlznehz1GNS7T7KKuGOrqqko?= =?us-ascii?Q?fSLHvz/rZfiqypbcacmAH84nCb5lEZN9cJr3QtBG5edA3p8gS1fSX7bEOzkB?= =?us-ascii?Q?ry/KKDH6wDw8OMbVPMpfuqWO6Nazvnf/8nWteeQIOsMz+nEqmnasaQEI+ZTq?= =?us-ascii?Q?0IiDeKxZ9vfRqpgVcE5d/ZYEWlfBss3gcwKJ0vwStJ8NaRBDptCb7KwoHc3x?= =?us-ascii?Q?Um0lDo4e6C7TluN1tEqc+xfaeHovJycWfXR/QGS6Ik1aLAzLFO+TX9UQFvxr?= =?us-ascii?Q?7aHAybMglSdE7KdyagRCzcDXCfRE43HeCuf/pTNOwdslVnp4rOGwZ/4vUq1h?= =?us-ascii?Q?v6s1FSC0byCtfNIc5+M6RvrX9g3g1+f6bkTx2JZw3g5Lgay/aVpuIuK7OWPw?= =?us-ascii?Q?siHmcCLM8knKr8+lfL/N8QVxFmM8RCNW1vRs8tcyzj+k6EbquVjYOBogV/N8?= =?us-ascii?Q?STAcafQGdID/QdoBc9db2gmE6wqnLbkMyDLCBIdA8o+MjwipiGSXsc7VV7m/?= =?us-ascii?Q?MXf9386UnVqH6mzwiXCfOaXDTAnhExdDOmVakdaR6xJOAnW5HSlsMXhknZLx?= =?us-ascii?Q?0X4ImS2tqCVY3Feq9G3IGopt1xgXdOaCEyX4DJ4tgNA9PoEVm96H8Icn/D8i?= =?us-ascii?Q?SC+KiQ+abIF4cF7YZ5AWHhkEoOhpeyH12mXS95c7sxEKYzsTO1NNTN+n0jgC?= =?us-ascii?Q?0/qGmIDf+UpjJvs3/rC6Q4+UMyF0NqOwbPLokwteLBkJ4mhcpst+6UcG3ExB?= =?us-ascii?Q?ebtZn/TmL1nRuL4hhBVJjwqt4i72i3W123iyov3PeEH5ZXWqQNBPN/tCQMAJ?= =?us-ascii?Q?5to/W2A6JKRODHibgn+tkJcVeUhcf3mOX8IoXj5mkUrXtLSOwhSHDGW/bjS9?= =?us-ascii?Q?03JbqKi5Uy375fDVfO+B8BZR3cVkDxv7MeyHUt7CdPhYxE7ZA3cTM7p7+AE+?= =?us-ascii?Q?hPsmjw3G7CqPVRFg4oJzjoXBT2zewwFwyup1CCCEKKLeI56gVj2P+g4UwjiL?= =?us-ascii?Q?vsqxw4Xo55cA0cZV6Pm76SzKXOtOPZTZVAq+0kPbTqcto8r9CPsMMjG6JYu2?= =?us-ascii?Q?VxqDHg7XvJeGHJ1TjYwGah7bhzW7uBWUKQh/bVUSoxUXsjmra8QQkTw3Q/6F?= =?us-ascii?Q?qH79J9zcx0N4DN2Y8Qzcf0lNLm3VIWh5pBX+mlSfwKmPcvXmz/YkRH0/VBso?= =?us-ascii?Q?BbOGQIvy+htMKDhjdRYWabFnMk+YNfOBq6OKIrVKLYTIK/Po6jcAfiMbXVve?= =?us-ascii?Q?VCUTs34F7w3DD7s6XA3snbuOCVPeLiZVeOvPgt4mWsrvEVGGJmRhr745f2E2?= =?us-ascii?Q?2QWwWUmCFlQpWM0hbAJcAqJh5mluURxrwBBm7Pvq/2oBhQ=3D=3D?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9dfaa50a-741e-45ce-5f83-08d95770fede X-MS-Exchange-CrossTenant-AuthSource: SA2PR10MB4458.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Aug 2021 17:55:07.5278 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: V1f4t1geBtVehclOHdu4SHvJtv6YaQoJk4Rw/6k/e+ONXyouNfWxceh7swc2yGT9JJ/SrehWLev7vOLyUowfCQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA2PR10MB4555 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=10066 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 phishscore=0 malwarescore=0 suspectscore=0 mlxscore=0 adultscore=0 mlxlogscore=999 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000 definitions=main-2108040104 X-Proofpoint-ORIG-GUID: Om_6aAKbAqOfIb-H54PO0Ubq4abgXfWR X-Proofpoint-GUID: Om_6aAKbAqOfIb-H54PO0Ubq4abgXfWR X-Spam-Status: No, score=-5.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, KAM_SHORT, MSGID_FROM_MTA_HEADER, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Aug 2021 17:55:14 -0000 [ These patches depend on the series "Allow means for late BTF generation for BPF CO-RE" by Indu Bhagat, here: https://gcc.gnu.org/pipermail/gcc-patches/2021-July/576446.html ] Hello, This patch series adds support for the BPF Compile Once - Run Everywhere (BPF CO-RE) mechanism in GCC. A BPF program is some user code which is injected (via a verifier and loader) into a running kernel, and executed in kernel context. To do useful work, a BPF program generally must interact with kernel data structures in some way. Therefore, BPF programs written in C usually include kernel headers. This introduces two major portability issues when compiling BPF programs: 1. Kernel data structures regularly change, with fields added, moved or deleted between versions. An eBPF program cannot in general be expected to run on any systems which does not share an identical kernel version to the system on which it was compiled. 2. Included kernel headers (and used data structures) may be internal, not exposed in an userspace API, and therefore target-specific. An eBPF program compiled on an x86_64 machine will include x86_64 kernel headers. The resulting program may not run well (or at all) in machines of another architecture. BPF CO-RE is designed to solve the first issue by leveraging the BPF loader to adjust references to kernel data structures made by the program as-needed according to versions of structures actually present on the host kernel. To achieve this, additional information is placed in a ".BTF.ext" section. This information tells the loader which references will require adjusting, and how to perform each necessary adjustment. For any access to a data structure which may require load-time adjustment, the following information is recorded (making up a CO-RE relocation record): - The BTF type ID of the outermost structure which is accessed. - An access string encoding the accessed member via a series of member and array indexes. These indexes are used to look up detailed BTF information about the member. - The offset of the appropriate instruction to patch in the BPF program. - An integer specifying what kind of relocation to perform. A CO-RE-capable BPF loader reads this information together with the BTF information of the program, compares it against BTF information of the host kernel, and determines the appropriate way to patch the specified instruction. Once all CO-RE relocations are resolved, the program is loaded and verified as usual. The process can be summarized with the following diagram: +------------+ | C compiler | +-----+------+ | BPF + BTF + CO-RE relocations v +------------+ +--->| BPF loader | | +-----+------+ | | BPF (adapted) BTF | v | +------------+ +----+ Kernel | +------------+ Note that a single ELF object may contain multiple eBPF programs. As a result, a single .BTF.ext section can contain CO-RE relocations for multiple programs in distinct sections. Many data structure accesses (e.g., those described in the program itself) do not need to be patched. So, GCC only generates CO-RE information for accesses marked as being "of interest." To be compatible with LLVM a new BPF target builtin, __builtin_preserve_access_index, is implemented. Any accesses to aggregate data structures (structs, unions, arrays) in the argument will have appropriate CO-RE information generated and output. This builtin is otherwise transparent - it does not alter the program's functionality in any way. In addition, a new BPF target attribute preserve_access_index is added. This attribute may annotate struct and union type definitions. Any access to a type with this attribute is automatically "of interest," and will have CO-RE information generated accordingly. Finally, generation of BPF CO-RE information is gated behind a new BPF option, -mcore (and its negative, -mno-core). Because CO-RE support is intimately tied to BTF debug information, -gbtf for BPF target implies -mcore, and -mcore requires BTF generation. For cases where BTF information is desired but CO-RE is not important, it can be disabled with -mno-core. David Faust (7): dwarf: externalize lookup_type_die ctfc: externalize ctf_dtd_lookup ctfc: add function to lookup CTF ID of a TREE type btf: expose get_btf_id bpf: BPF CO-RE support bpf testsuite: Add BPF CO-RE tests doc: BPF CO-RE documentation gcc/btfout.c | 2 +- gcc/config.gcc | 3 + gcc/config/bpf/bpf-passes.def | 20 + gcc/config/bpf/bpf-protos.h | 2 + gcc/config/bpf/bpf.c | 579 ++++++++++++++++++ gcc/config/bpf/coreout.c | 356 +++++++++++ gcc/config/bpf/coreout.h | 114 ++++ gcc/config/bpf/t-bpf | 8 + gcc/ctfc.c | 18 +- gcc/ctfc.h | 8 +- gcc/doc/extend.texi | 16 + gcc/doc/invoke.texi | 13 +- gcc/dwarf2out.c | 3 +- gcc/dwarf2out.h | 1 + gcc/testsuite/gcc.target/bpf/core-attr-1.c | 23 + gcc/testsuite/gcc.target/bpf/core-attr-2.c | 21 + gcc/testsuite/gcc.target/bpf/core-attr-3.c | 41 ++ gcc/testsuite/gcc.target/bpf/core-attr-4.c | 35 ++ gcc/testsuite/gcc.target/bpf/core-builtin-1.c | 64 ++ gcc/testsuite/gcc.target/bpf/core-builtin-2.c | 26 + gcc/testsuite/gcc.target/bpf/core-builtin-3.c | 26 + gcc/testsuite/gcc.target/bpf/core-section-1.c | 38 ++ 22 files changed, 1411 insertions(+), 6 deletions(-) create mode 100644 gcc/config/bpf/bpf-passes.def create mode 100644 gcc/config/bpf/coreout.c create mode 100644 gcc/config/bpf/coreout.h create mode 100644 gcc/testsuite/gcc.target/bpf/core-attr-1.c create mode 100644 gcc/testsuite/gcc.target/bpf/core-attr-2.c create mode 100644 gcc/testsuite/gcc.target/bpf/core-attr-3.c create mode 100644 gcc/testsuite/gcc.target/bpf/core-attr-4.c create mode 100644 gcc/testsuite/gcc.target/bpf/core-builtin-1.c create mode 100644 gcc/testsuite/gcc.target/bpf/core-builtin-2.c create mode 100644 gcc/testsuite/gcc.target/bpf/core-builtin-3.c create mode 100644 gcc/testsuite/gcc.target/bpf/core-section-1.c -- 2.32.0