From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by sourceware.org (Postfix) with ESMTPS id 21C893839DF0 for ; Wed, 31 Aug 2022 15:16:12 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 21C893839DF0 Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 27VEEVRG020425 for ; Wed, 31 Aug 2022 15:16:11 GMT Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3j7a229hyn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Wed, 31 Aug 2022 15:16:10 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 27VEN5Uu014536 for ; Wed, 31 Aug 2022 15:16:09 GMT Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11lp2169.outbound.protection.outlook.com [104.47.57.169]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3j79qbf8em-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Wed, 31 Aug 2022 15:16:09 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nvpGPsk8mhkqUau0JUSp1O/hCysICM+SiBEqF227bbPLfK2Hq7RqsmT6/sJHVg5o3BSsrS2TeZkqK8vFcx0l/qWIpS0cx8c+Lz30uiz8DBUqO3AkzSy5YXviByIBgNzsTinzIQZgpImmO5yfSwNsT00wRJyvNE5+chhfmJPAsbvb5xiETdYvZ2DKdgt6y34LHNm3WAQWycb0Mr1IXoSCAOhV4vBJphwsA2k3sKrqCDgZ2QqjusGMWSt1DkcidI6mTDRV6YjeZ//uY/mdA3n/8TnR2kxnrfk/P52FoxTwNjhmRyFMChJl4RXDtXn4xY8p/sZKmloA3jv4Th5eh+aD1Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XPl1k50WWSdOkTdm3wkCoLviGNj5DkGjOgjeIeaZwQg=; b=XQj5ieIdy7+R9XQcoG5f85fpcGazbsO6kQRHdoM2CqJ7evIziQswmec2EXPqCZxqNs92n4zbxa7GWJBUqqiZ5yTyPhLGPPFbghzLDM0QsheBDtDJeoyHG2CwJacfRXssNY++13TzK7NJjlYmyZZj9r0WSVksUx8q6OUncIFjThKaVPlERIMUYl48OCqpb+bZqcg5x7yOL4Vho+6fvC0BmYx7agJlbTJOYsvA59TNwnlBKBj3cTCf3ODF3RRyoXOSWx+tW7cPjtdVc4tteExfZ8Z5/sUZ3AKOV6pfve+dXIj5/fHsKo7TdB0en+j/GJL/Jehk+LJqcR/ev9ALocA4gA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none Received: from MWHPR10MB1407.namprd10.prod.outlook.com (2603:10b6:300:23::20) by IA1PR10MB6241.namprd10.prod.outlook.com (2603:10b6:208:3a3::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5566.21; Wed, 31 Aug 2022 15:16:07 +0000 Received: from MWHPR10MB1407.namprd10.prod.outlook.com ([fe80::18e8:e5f2:59a9:1ff5]) by MWHPR10MB1407.namprd10.prod.outlook.com ([fe80::18e8:e5f2:59a9:1ff5%2]) with mapi id 15.20.5588.010; Wed, 31 Aug 2022 15:16:07 +0000 From: "Guillermo E. Martinez" To: libabigail@sourceware.org Subject: [PATCH] ctf-reader: Lookup debug info for symbols in a non default archive member Date: Wed, 31 Aug 2022 10:16:03 -0500 Message-ID: <20220831151603.915945-1-guillermo.e.martinez@oracle.com> X-Mailer: git-send-email 2.35.1 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: SJ0PR13CA0199.namprd13.prod.outlook.com (2603:10b6:a03:2c3::24) To MWHPR10MB1407.namprd10.prod.outlook.com (2603:10b6:300:23::20) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 887a42d0-be54-4b25-75e3-08da8b63ba53 X-MS-TrafficTypeDiagnostic: IA1PR10MB6241:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: AGb0OIGHfIGySMu7wgkxRnQcFrmfh2V4gsVQZo0FRuhOtTwBuUgHESGxzuARdy7Sm0iO92pLz0r8X8mvEs3DvPb0x+I64qrbSzFTP5t1fBSi8j+fENcf8TN3QoDUhhsmisD+WWmoOwPaiUEA6jlRf5uOlxttZ7fhkGmdkpJ5QMR5//0nW+lO5Ow3PXtvQdM8gzKjm1G2biTiSQEQljK+rizh+xFdVkMc7lrYvz0XUsQwzlDh/bJUWMFZHSGONnmZbqsFrX3fGuduQ0ReOBSVNdU7rWFeUfHfv+Vz+Mv2/kWu4sltdtFEtvwtddgvsNgtGmbmGR/+zIhkXeMRoYsArTlXTpxHYeljJMdUxLsgZKWFbb6k8cW0smjRgpFasBNHHKHoMzmsCAg6YVWwMUQmPWGSlKTS6cADvRyrwuIkXVUmrv6Q1c+R6E6KsqlPLEkAO3iIILG16ZtBR9uDd8/Bl7ooh1xx3SSwTP6SLwoK9PSj13kM4erAWl6I6/KIAeu4AVFXKNrl+QmIU9VCS8Mg28zPc6NDGFdLoroPm9cEfjmSlBnA1uQ2EUDszHpZmZL7/iE0EbU19jhu1sFB2AhWyBVfigaZ91h5YMHGmTJOTjp6SvlI7TS9elUucSjKBDFIL6dlGk4iq466bSew+qOu6uvP5YQCZFUGRd6hTFBoijuJGAfS1cExJeoLTAvV+liE1xDHZdUwpH0JCVVzsOsmaQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MWHPR10MB1407.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(39860400002)(136003)(376002)(346002)(366004)(396003)(5660300002)(41300700001)(8936002)(1076003)(186003)(6666004)(2616005)(86362001)(107886003)(6506007)(6512007)(83380400001)(2906002)(103116003)(36756003)(316002)(6916009)(66946007)(478600001)(38100700002)(66476007)(4326008)(6486002)(8676002)(66556008); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?ZCyQy9ONo86y3RZyXCswXIdeZSV7/J21DeJ05nwjcDeEPY6Mb505ApzXlige?= =?us-ascii?Q?v5aRX0gYkipLYTYrYD7v3IMw4k4gj1osA6RRXGohPvfnsH5qHPHDPWTRJ0xD?= =?us-ascii?Q?UnIRmwIrMVB9Vwemuth1OvH3Nfoh9Cb/F+i4fTnB5CINi2G7CsmX8O74m98l?= =?us-ascii?Q?vWhcoMRpDKI4HrduX4zvmof85keCr20FRCwAcb668mYdTmkkfXG4VuESPdxH?= =?us-ascii?Q?+TmUQq6ustQ3MFruoUoFuSViPEmGXVnA0C/HKKyMi4kuiidrj2A9z7G5pq4G?= =?us-ascii?Q?YXjJYNTb5/d6sVpaurGCGbzWqbO6u/iGLAYiDt5BkbB+zZfrwrj7+6tnVoal?= =?us-ascii?Q?rnthtZcDe7Rygbu/8rdpb6X3S1vPDZ9wex+Maw4VpjbokFJqQNEAMkwu8Qvm?= =?us-ascii?Q?H6nwZV0cBUvem1SGNiPEdqs5u7rGmrc7Vr7xf1EGR8FK3W3P3ipFP5vnPHHw?= =?us-ascii?Q?UYCSkvmmBKqzusmRkWZIBg0wN1OOyeCruwjiMekna4ahUut0zIFoBIYmYEAv?= =?us-ascii?Q?ICi2u3F5eWJzjUpPwU+gVkPGM2WMQShDab3SpPprhAfsDWBnCckJ0snnalY0?= =?us-ascii?Q?Db06ab/SGmFnVU6Uwm/IFaS1qUobOM4TQicRrCG3xNNWO8383d0FJN639yA1?= =?us-ascii?Q?2UJG6LBDf5SykU2EuXnte1msOfNqMYcuD+fLsKzIq/sfksYEFcZY74p/1lvs?= =?us-ascii?Q?WHbatu9ErlQqUls1KPCTbks0tWpo8O2/1v/5j8osCf8TCkF4E8/o6dtSpBB4?= =?us-ascii?Q?foiC0JlrzZ/2AZksO3Cf9eqiar4C2SKb1rFllIGnMLAtxSc5S89eGVnsXQ2d?= =?us-ascii?Q?DAQgF+Cy3ZZ4PL49Ms01L1H0GRSaDTrep4OGV5IhObTA8HRE8DZEvwwxF5tb?= =?us-ascii?Q?Wg1hU1Qk2vP0A3tKjpCepD0jklgZQYALjlIVPGItV2pYC8x1x3lTeZB+RQul?= =?us-ascii?Q?syIMaIxQP2knLus+OiGPu6B/7lXpUr/Wsophseiuen4YaZcvZdjYGVBVWmal?= =?us-ascii?Q?coc73FkjWb0+/6H4Mtwq+brQI77ctoySuIwOYlyz6Z180jAR8L9y1ZEHzh7+?= =?us-ascii?Q?qznCzAn6Rzv41hxZWQlD7jDZc4aWRlMhNDkGOjmqX7XCjeJBqItLTP7DsrHz?= =?us-ascii?Q?gdOzRQAGJjZuNx4T5MwtOtnaBE1PbN5QFZ7JLyStcgiPHMwxq5NL1bFjdMt3?= =?us-ascii?Q?QLRqBQVf2sseMg1RluKLAhWGv1V38wJZlAGi+rV6bPAsB2ulUyog0UccReDW?= =?us-ascii?Q?i6nabbmhj5fhhPgiQ6jq5DffvGlmXyT3x1NvjRQzd33oazLzICFET6cFgbmE?= =?us-ascii?Q?kwQVYEd5oiMeaDZlwsdbbCEemmoiosB4f+d+VvC7+DMT1JXaJw6Cs1R7kWc8?= =?us-ascii?Q?wsbMpcMMiQf7SsIRI+MhYJ/Lk/0GujJ69C3j4ggoTTI2I/R561wHd+bjIobB?= =?us-ascii?Q?kE1alq9WCo3ywKiUKeV9JMBHH0oJS9ozrLM5yj4mGTdTd3IxPIdRGBmzyc+T?= =?us-ascii?Q?pxfatkKzDbvqavCw6Fd/uaY+42uZKA01+JjO+WlLPNQJOmClc6kGK1o7ip3K?= =?us-ascii?Q?1iXFPabqVtCWApBM5R/yErDtpYldwhrP+0CCnvszmrdnUJiQy8OjKI33B4P6?= =?us-ascii?Q?PuWg29GeRPdWfkl/akPu+tq/fBt53lw1bPJOZkEfvIOP?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 887a42d0-be54-4b25-75e3-08da8b63ba53 X-MS-Exchange-CrossTenant-AuthSource: MWHPR10MB1407.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Aug 2022 15:16:07.2432 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: e0//BoqkjU+vrGiWrWnuvagWzyvXNoq3p3DFs/Jvez24vXhcel8H22pJrgFcxHY0tLFwbCshX/2vjaIXRFtFXgHc7qblLI6omA7oX6yZDYg= X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR10MB6241 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-31_09,2022-08-31_03,2022-06-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 phishscore=0 mlxlogscore=999 adultscore=0 bulkscore=0 spamscore=0 malwarescore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2207270000 definitions=main-2208310076 X-Proofpoint-GUID: xKndxAFygT25E88dc69cxuGvC3qE-UKR X-Proofpoint-ORIG-GUID: xKndxAFygT25E88dc69cxuGvC3qE-UKR X-Spam-Status: No, score=-12.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libabigail@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list of the Libabigail project List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Aug 2022 15:16:14 -0000 Message-ID: <20220831151603.Zm9iiLn4FCEe6hnCE0fyuHiIhgJryREGO8PxcGNmzt8@z> Hello, This patch improves the ABI XML file generated by ctf reader, there are Linux symbols (EXPORT_SYMBOL*) that were missing. Comments will be grateful and appreciated!. Thanks in advanced, guillermo -- The current mechanism used by the ctf reader to looking for debug information given a specific Linux symbol, is open the dictionary (default) which the name match with the binary name being processing in the current corpus, e.g. `vmlinux' or `module-name.ko'. However there are symbol information is not located in a default dictionary, this is evident comparing the symbols in `Module.symvers' file with ABI XML file, so for example, the ctf reader is expecting to find the information for `LZ4_decompress_fast' symbol in the CTF `vmlinux' archive member, because this symbols is defined in `vmlinux' binary: 0x4c416eb9 LZ4_decompress_fast vmlinux EXPORT_SYMBOL But it figures out that it is missing. The correct location is `vmlinux#0' dictionary. CTF archive member: vmlinux: ... Function objects: ... CTF archive member: vmlinux#0: Function objects: ... LZ4_decompress_fast -> 0x80037400: (kind 5) int (*) (const char *, char *, int) (aligned at 0x8) ... Therefore, ctf reader is looking for debug information in the whole archive, fortunately `libctf' provides a fast lookup mechanism using cache, dictionary references, etc., so the penalty performance is ~10%. * src/abg-ctf-reader.cc (lookup_symbol_in_ctf_archive): New function. (process_ctf_archive): Use `lookup_symbol_in_ctf_archive'. Signed-off-by: Guillermo E. Martinez --- src/abg-ctf-reader.cc | 72 ++++++++++++++++++++++++++++++++++++++----- 1 file changed, 64 insertions(+), 8 deletions(-) diff --git a/src/abg-ctf-reader.cc b/src/abg-ctf-reader.cc index 71808f9a..8fa98a94 100644 --- a/src/abg-ctf-reader.cc +++ b/src/abg-ctf-reader.cc @@ -1204,6 +1204,62 @@ lookup_type(read_context *ctxt, corpus_sptr corp, return result; } +/// Given a symbol name, lookup the corresponding CTF information in +/// the default dictionary (CTF archive member provided by the caller) +/// If the search is not success, the looks for the symbol name +/// in _all_ archive members. +/// +/// @param ctfa the CTF archive. +/// @param dict the default dictionary to looks for. +/// @param sym_name the symbol name. +/// @param corp the IR corpus. +/// +/// Note that if @ref sym_name is found in other than default dictionary +/// @ref ctf_dict will be updated and it must be explicate closed by its +/// caller. +/// +/// @return a valid CTF type id, if @ref sym_name was found, -1 otherwise. + +static ctf_id_t +lookup_symbol_in_ctf_archive(ctf_archive_t *ctfa, ctf_dict_t **ctf_dict, + const char *sym_name, corpus_sptr corp) +{ + int ctf_err; + ctf_dict_t *dict = *ctf_dict; + ctf_id_t ctf_type = ctf_lookup_variable(dict, sym_name); + + /* lookup CTF type for a given symbol in its default + dictionary */ + if (ctf_type == (ctf_id_t) -1 + && !(corp->get_origin() & corpus::LINUX_KERNEL_BINARY_ORIGIN)) + ctf_type = ctf_lookup_by_symbol_name(dict, sym_name); + + /* Not lucky, then, search in whole archive */ + if (ctf_type == (ctf_id_t) -1) + { + ctf_dict_t *fp; + ctf_next_t *i = NULL; + const char *arcname; + + while ((fp = ctf_archive_next(ctfa, &i, &arcname, 1, &ctf_err)) != NULL) + { + ctf_type = ctf_lookup_variable (fp, sym_name); + if (ctf_type == (ctf_id_t) -1 + && !(corp->get_origin() & corpus::LINUX_KERNEL_BINARY_ORIGIN)) + ctf_type = ctf_lookup_by_symbol_name(fp, sym_name); + + if (ctf_type != (ctf_id_t) -1) + { + *ctf_dict = fp; + break; + } + ctf_dict_close(fp); + } + } + + return ctf_type; +} + /// Process a CTF archive and create libabigail IR for the types, /// variables and function declarations found in the archive, iterating /// over public symbols. The IR is added to the given corpus. @@ -1222,7 +1278,7 @@ process_ctf_archive(read_context *ctxt, corpus_sptr corp) corp->add(ir_translation_unit); int ctf_err; - ctf_dict_t *ctf_dict; + ctf_dict_t *ctf_dict, *dict_tmp; const auto symtab = ctxt->symtab; symtab_reader::symtab_filter filter = symtab->make_filter(); filter.set_public_symbols(); @@ -1248,19 +1304,17 @@ process_ctf_archive(read_context *ctxt, corpus_sptr corp) abort(); } + dict_tmp = ctf_dict; + for (const auto& symbol : symtab_reader::filtered_symtab(*symtab, filter)) { std::string sym_name = symbol->get_name(); ctf_id_t ctf_sym_type; - ctf_sym_type = ctf_lookup_variable(ctf_dict, sym_name.c_str()); - if (ctf_sym_type == (ctf_id_t) -1 - && !(corp->get_origin() & corpus::LINUX_KERNEL_BINARY_ORIGIN)) - // lookup in function objects - ctf_sym_type = ctf_lookup_by_symbol_name(ctf_dict, sym_name.c_str()); - + ctf_sym_type = lookup_symbol_in_ctf_archive(ctxt->ctfa, &ctf_dict, + sym_name.c_str(), corp); if (ctf_sym_type == (ctf_id_t) -1) - continue; + continue; if (ctf_type_kind(ctf_dict, ctf_sym_type) != CTF_K_FUNCTION) { @@ -1305,6 +1359,8 @@ process_ctf_archive(read_context *ctxt, corpus_sptr corp) func_declaration->set_is_in_public_symbol_table(true); ctxt->maybe_add_fn_to_exported_decls(func_declaration.get()); } + + ctf_dict = dict_tmp; } ctf_dict_close(ctf_dict); -- 2.35.1