From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by sourceware.org (Postfix) with ESMTPS id D7AC53898388 for ; Thu, 6 Oct 2022 19:53:43 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org D7AC53898388 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 296JSotZ016302; Thu, 6 Oct 2022 19:53:41 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=corp-2022-7-12; bh=SAuJapnm7QLVsQuWeDELvTgyvRCnNqLlWZtluzee8xI=; b=ezxzoLGq1BeeSbnKlZYYkG5PjW4Lgtdv5wHeHWwdvCBxvQaKptaClcMD1LGBSDvF6lqs 5il+bda+tgVj983CPDqGRcZQhssfmgffSg7ne0rZVGoJtj/+eSGQXuhx6fO5eq3brIK2 PJHE5WA6jIQVYtMFLvxRym6hwQBNz6IUVItHLu0pHFCAcLAtkxzkyJJwnsCLCTBJ+D0t duHD0sK8tqkC7zOCM2ctw3gwdbpa8f2r91gHaT+wmv9nfmKDTmgCtKRxU4qkoVVq4aKx 6XMypOYAiLb5olJTq9t3JTlM3z4NOljZqyb7VqqvK3PGDmd28rluP2hqIIEPfW/G9PE1 xA== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3jxcb2wcef-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 06 Oct 2022 19:53:41 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 296HKQkZ000614; Thu, 6 Oct 2022 19:53:40 GMT Received: from nam12-bn8-obe.outbound.protection.outlook.com (mail-bn8nam12lp2174.outbound.protection.outlook.com [104.47.55.174]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3jxc0657xw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 06 Oct 2022 19:53:40 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ofG7znO9lpDjQDOnlwwbfFESZu6BKYwIYBLxTMhjbhHg1/vi/hLwp7ANofqpCO8iPJ/s1FNQbr96getAIS/rgq/7Zi6yyM83fzl18/gNmar5mU2sp5NCecYr/fGtwBF4aktkr4RXaSjHXVXrIy+OcmnlUSEHOOtyGt0/zfVmPHdyxEbdHYJkcFaz3R1Hnse29yjnRX/fVQs+YgtamoWDk4L2bWX8AHazruVtPR4s1HUbyqPk2JV1m9QOH1dONnSrWPuqLdwMv+2ZTFgXvU8dvlIBbinsF+FOXIpdQnPqnEUEScwyjPyV0s/7g0up7+v9cQOyO5EjPXZ6aiM8MHTtdw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=SAuJapnm7QLVsQuWeDELvTgyvRCnNqLlWZtluzee8xI=; b=jiIsZvUdlmgKxNlumc3/uDKeNwzOpODdLr5YwisPjZbD2zRmK2OwZ4oQ6G3o4ncSa+nXEvVKM2VjjTrtswD1B0luJTnmRBKfF03WjLy/pGVhGZ3BXFjARiDYqoJCyJwezHxBe1TS52gbf0W2IE0sFVzbBZAOYhFJcQluJm14nMDjVuxCxshkHCYl4Z/q2p0Q58PAOywn80f6C4KRJKSlDR5Xa2Cj0YfEATN6oyBW/CAmqpVCg2GwBXH43pxkPSSbVMIaijvZ1xwRIRwwOBAa7AcDjYpxNOvI2DT7a8o3KGSz5MLQntBXL2f/vR1tmlF0g5WnqHQKuPy32lJlBoWXdw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SAuJapnm7QLVsQuWeDELvTgyvRCnNqLlWZtluzee8xI=; b=V+u2WceNgt3CKxdyBD4bZigATLF7A3veRZSg2YiSbmG3v9pvdlNK0au45GW5W675HDMXoVBjlYpjEWNU3No5yPb1eONsa3ehyV5j3iwywL26hQ8bRurF6uPB6b2QCWxjeqNM5ot4Vy1fnhn+4Brk4U5IXvg/TUfye6G+DEKuKaE= Received: from MWHPR10MB1407.namprd10.prod.outlook.com (2603:10b6:300:23::20) by MW4PR10MB5749.namprd10.prod.outlook.com (2603:10b6:303:184::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.32; Thu, 6 Oct 2022 19:53:37 +0000 Received: from MWHPR10MB1407.namprd10.prod.outlook.com ([fe80::d05e:773d:2a05:eaed]) by MWHPR10MB1407.namprd10.prod.outlook.com ([fe80::d05e:773d:2a05:eaed%2]) with mapi id 15.20.5676.036; Thu, 6 Oct 2022 19:53:37 +0000 From: Guillermo Martinez To: Dodji Seketeli CC: "Guillermo E. Martinez via Libabigail" Subject: Re: [PATCH] CTF as a fallback when no DWARF debug info is present Thread-Topic: [PATCH] CTF as a fallback when no DWARF debug info is present Thread-Index: AQHY1Sr29tC+Gl3jLUWsZ6m1mqyjvq399kOdgACZHQCAAnSnvoAAdzyA Date: Thu, 6 Oct 2022 19:53:37 +0000 Message-ID: <7dbdea77-dbc9-7bdd-94ee-8e4cf46ce886@oracle.com> References: <20221001001544.210234-1-guillermo.e.martinez@oracle.com> <877d1g2c52.fsf@seketeli.org> <568fe730-3bb9-0267-00bc-2873e94e502f@oracle.com> <86mta9bdpm.fsf@seketeli.org> In-Reply-To: <86mta9bdpm.fsf@seketeli.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: x-ms-exchange-imapappendstamp: MWHPR10MB1407.namprd10.prod.outlook.com (15.20.5676.036) user-agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.3.0 x-ms-publictraffictype: Email x-ms-traffictypediagnostic: MWHPR10MB1407:EE_|MW4PR10MB5749:EE_ x-ms-office365-filtering-correlation-id: a1dae300-b4af-4f06-b56b-08daa7d475cc x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 62Jg81rzxrgmqJRw7YsaDIEIKM5RybsZZFxDdPTqEpH/uutgL2po6j9CjObgbP0je3woqfSAcOb7wRkSwYCYuIHCeMaz9HAO2WNEGfK96XDiS6NqAO4CUAbmBS6Ocv7Pv91mNWEp/xOVK4kekaqktM62SaOB1QlOPRwNY82D03YoNDGQ3cJnop5/ZZ4DLHWjXFJ+804rKM8WB+/v3H+GX94IrC0bCAb6RRpfbbl7QEqRi2xNOYoD5mhXyE5YJUtXBh4Ly9lvlzOHBxuPYwVe4X/8wU6n/RcIZTmaEwphawSVhzTTJ2lSRZY1cO+UcHJpv+7r1mZ3AY1xF7FCHVIjwk++ZXhffkqkai95RqeW8F0DiXMiKFZF2fywAfDOWiWwAq02/62H8eb0lC3S9l7Ow3GeO3jWg8wACRhfHzsKiiJl84MbFiePa9ivFbifn5LuOcWt3Snvh3Hp5MA+W3cuQ3c5ATlF2rDc+iBOrwIfor8awNWU0r2JqmvsOUCxzxzksIG4CmyWeChBGcEYjqiHMCr9EQYsNIDyBYtzU1ogXyzdN6OlVYplGFGohH2ZpKih23pq6B9k00QZLEr7NngCuj5bVV3wYc55/s95wLIX3SHRm/+nXY3ZiJkeQ3dilYxdKRdjyCyyoYrFDHpp+p2G0QpXksP7NctimPp+6/nwh2y1qaa6C8jdqZtXImB1X0IloxHKdCNZMCvzq4oAo1GyurfGAWtiixccNueb9uRss1/sBB1IyG3Tdae1zO3ZERjde4ChyUTx69eArj3ULoyxxk2z8CUw7F6oaU0eh9k0H2fvWPIgHtwTnKuBF2aWDnUP x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR10MB1407.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230022)(39860400002)(396003)(366004)(136003)(376002)(346002)(451199015)(38100700002)(6506007)(53546011)(26005)(6512007)(71200400001)(6916009)(2906002)(316002)(8676002)(122000001)(478600001)(31696002)(4326008)(66446008)(64756008)(91956017)(66556008)(36756003)(66946007)(76116006)(38070700005)(66476007)(86362001)(83380400001)(8936002)(31686004)(186003)(41300700001)(5660300002)(66574015)(6486002)(2616005)(45980500001)(43740500002);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?bXEweWBpoZnGsn8+hh+3wtDaXTne08McRCSVWYnIKsgX7OkoY9aG8wi0IB?= =?iso-8859-1?Q?U2qIl28L8kQ+cTnYA4GAhUXt5gerxV58D+i/D1tNslE5t323MmIEM5Au6n?= =?iso-8859-1?Q?Cjr5A4Wlrkpi35YPtQbQlBJp5PxeWSLsE860SR48KQag+g6is008Dir+q/?= =?iso-8859-1?Q?7Obsu0F8fgsfR0UhQJHez7PvxYaI9BqFO/en9GACsqCo8WjSmjQpn1AB9k?= =?iso-8859-1?Q?McqJ3hatIjGDpwL96aa/T91vJPxn2cmYFVXSvHP8MQHUStEIL6S6gqH4Qt?= =?iso-8859-1?Q?5xPzriRkV9c3Xv5W3tf2UaNRtedhRw+SLuha3GikqMIW+Xqnn9OvMK2Ltb?= =?iso-8859-1?Q?21mUL5gTjXN9Gcg3ScZzmToV5eVFPVpmY44hJ805GnT+/NP4sXaw7RkrT7?= =?iso-8859-1?Q?wFTw+anUFaGDLu3+h56+pI7RXw3NfV4NjOwY28fFr3Ehou2p4/ugaSlCK0?= =?iso-8859-1?Q?l+4uGj7oAeYKKau+m28EqO7GdXqDGuBZWqWuQOBO5p3WDwAxVBMkRIBBar?= =?iso-8859-1?Q?mrXTmrpVU6iflwkpJqF0MzwU0YcmTe4IJck0il4dAOOIrtt76+KWqZQa/V?= =?iso-8859-1?Q?FdboxEhpY1vwpoWZc59drV8mZcnUWGVWUJspXAdql+9thG7FcWQ3q+NA0M?= =?iso-8859-1?Q?GYQODUzipsjwwheS+4IiqR4EspmrtkyIBBvSsratK2UIrAEmyVQf90E21n?= =?iso-8859-1?Q?R8tFufZxhe55K0q8k+K8Yj8UmZeI1IjqlyQkIKRLCIt4ehXj7/O9+Er+uu?= =?iso-8859-1?Q?XtsHb1W5R0ULU5/v+P/+NdmrVvPEas65fjGltfw1YTeyC3o0WsAuHQntnP?= =?iso-8859-1?Q?QXCQoJuQRE9qNqui1l1lauEyO+3RvK7UPENLw9JvlClqtgjQJry/DiWzrO?= =?iso-8859-1?Q?qh7RCmk4YtQkxMuMJCAgTENp2rqem9TglyUUiZ7My3WsPqnNjTfcy8zb+c?= =?iso-8859-1?Q?6oy7WS7Hs04LYxPVFUpFqQr9qKJJPtKL9Jo1XAUv+8MSc4UoZyS9sbazwI?= =?iso-8859-1?Q?ZbPigIFKbpRNjBJo64mrtyvq5zuZ7FFXnj1grENkSjJBvLZ393jf9KTKFJ?= =?iso-8859-1?Q?0BdQwOaS/jAjwFVTV+n7PShLX7cGUR+mUuvDut/GvXg0VCX6UZJAO3kVmQ?= =?iso-8859-1?Q?NaE4tJz5QZaeBqjUvIfsrsQC87HBNuWnTwVhtV3GwuzrAUQZcM6s0UFyDB?= =?iso-8859-1?Q?Df0TGs5W/PIa27yyN/b6wtDoBxDmTGoROdB6kf3/spCDgLTl549QWBKloZ?= =?iso-8859-1?Q?E6z4JR5Mv680WZrS9POXkKhnrZo/goNOLpLCeC+U7NI9JOhyDQSlFFElnt?= =?iso-8859-1?Q?sOtdmlk5iBtRfYDiuiUP3BVJ4zbf7beutFBTSpsRX3gae95VPqNx0YLkLj?= =?iso-8859-1?Q?UKI+ThKlNqgIBQyxdE3gih2e/z5ES1YiP4hhSupZt8VjiGdP+DnEL8HLl8?= =?iso-8859-1?Q?ffzar7v3gHbOsEI2kPBjHDbwgCRD3xB1b1IuZHZiYtK/NPQ9hBqB7IDUFd?= =?iso-8859-1?Q?7GoakAD+dZeJ+y0GYFg/3ZdLLT2sVlbJs7/QO/iSs0/UWpEPAezdBdUxXs?= =?iso-8859-1?Q?7pAT6IW0ailbWJIBAQPkRjJV2dy57wjH0ejl412dENvgnKUPUnlFSMtOC0?= =?iso-8859-1?Q?V9hKrnLzcSxPBgIXwSqv2NM95Mdmr+kH7DMibHIgDHrjAiHNb4ngq9IQ?= =?iso-8859-1?Q?=3D=3D?= Content-Type: text/plain; charset="iso-8859-1" Content-ID: <5C2377D5140EC9488AA54609D9100E58@oracle.onmicrosoft.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: T8WSB64ewrN9qJ666Ryl1TuI1hnFmvJ+WoD9wiQoaSKfZmwsrlsBqqe2nVXQHkmkEPz0M9xntMDaA5EbN9UTtTu8rX3+2f/Hg5X8Ob2Z8XqXDx/ew28/95tV7kD2ahRjLJPoRqWSqOxH3ETxUeq5FVCgUy94KPgH6TutqkBgRGkdTYy6+DccJqCgknl6cPXiZBLq8+eXV2rXygJRJzG1KOG/C6KUyff7xt/bOTqStQY24F3I+cVhE0UhxmQ7A6e1yaOwddTD5iRUzPezxMZuM3H7lhLrllinTGp9zgJXUL7JedWOqy72Cj0jFl6HeAWisR9HzyL893Q8yh1oilHeDVB7+TGw8z497qdMJJ67W3RevuTtDptv+nVSRnJDXnmIextb0FWvdl1p/mTmxw330/BTZq8ZB2wv0xRDH4OVRAu5GQrR5kc1A4iT7wewA2fO9mhAXVs0vL5ubCRQUzqrMfbLuhFjr/48UeCC8kUaWHxww3KgC/8A63Vskr2hcRP71YifYoecU+QNJGLWyEaIKNpd6GTR18S07phUfQdDu5GtsCt+aCGU3sE9yQYXhsisgW1W2GiHytaaWhKlfpUeIGZauXA0PWIU6sYTe2yFJWCnlCLxLJMnjPNRfqhu2TkhURIiLrRHGDSJYgOHfjcOPyjkqfhxIixu7VsNqk4GKmNFJMsUZHaSz03Dblp4AW2eDPuNeJo7tgi2NycceQvKRv5KnTPz7QABo4wyP74ko5NsJ9fI8qNDIadTFMue2BYrmaAP7qMF82ym/lOad7eaca5BPyVgrmfnjBLv9TcCXEU= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MWHPR10MB1407.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: a1dae300-b4af-4f06-b56b-08daa7d475cc X-MS-Exchange-CrossTenant-originalarrivaltime: 06 Oct 2022 19:53:37.7289 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: DBVUujJwOnzBiqS33CItF5zBzgd4J3w+wcQWYfUSFtlEpW9NJCXeoXBEy/bwkB6me85nq1OUKkZI7h03dlknZMkGMJcOporFnwVGuR/9/Qc= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR10MB5749 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.528,FMLib:17.11.122.1 definitions=2022-10-06_04,2022-10-06_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 adultscore=0 spamscore=0 suspectscore=0 bulkscore=0 mlxscore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2209130000 definitions=main-2210060117 X-Proofpoint-GUID: rUK5i6u1PXGHWOIgwV-Wn5WSnaQ-6R3m X-Proofpoint-ORIG-GUID: rUK5i6u1PXGHWOIgwV-Wn5WSnaQ-6R3m X-Spam-Status: No, score=-7.2 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: =0A= =0A= On 10/6/22 02:42, Dodji Seketeli wrote:=0A= > Hello Guillermo,=0A= > =0A= =0A= Hello Dodji,=0A= =0A= Thanks for your comments!=0A= =0A= > "Guillermo E. Martinez" a =E9crit:=0A= > =0A= > [...]=0A= > =0A= >>> I have also introduced a new function called=0A= >>> tools_utils::dir_contains_ctf_archive to look for a file that ends with= =0A= >>> ".ctfa". This abstracts away the search for "vmlinux.ctfa" as I wasn't= =0A= >>> sure if those archives could exist for normal (non-kernel) binaries as= =0A= >>> well:=0A= >>=0A= >> Ohh, perfect!, I'll use it in CTF reader to located the Linux archive fi= le.=0A= > =0A= > ACK.=0A= > =0A= > [...]=0A= > =0A= >>> @@ -2525,8 +2542,12 @@ get_binary_paths_from_kernel_dist(const st= ring& dist_root,=0A= >>> /// @param t time to trace time spent in each step.=0A= >>> ///=0A= >>> /// @param env the environment to create the corpus_group in.=0A= >>> -static void=0A= >>> -maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,=0A= >>> +///=0A= >>> +/// @return the status of the loading. If it's=0A= >>> +/// abigail::elf_reader::STATUS_UNKNOWN, then it means nothing w= as=0A= >>> +/// done, meaning the function got out early.=0A= >>> +static abigail::elf_reader::status=0A= >>> +maybe_load_vmlinux_dwarf_corpus(corpus::origin& origin,=0A= >>> corpus_group_sptr& group,=0A= >>> const string& vmlinux,=0A= >>> vector& modules,=0A= >>> @@ -2539,10 +2560,11 @@ maybe_load_vmlinux_dwarf_corpus(corpus::o= rigin origin,=0A= >>> timer& t,=0A= >>> environment_sptr& env)=0A= >>> {=0A= >>> + abigail::elf_reader::status status =3D abigail::elf_reader::ST= ATUS_UNKNOWN;=0A= >>> +=0A= >>> if (!(origin & corpus::DWARF_ORIGIN))=0A= >>> - return;=0A= >>> + return status;=0A= >>>=0A= >>> - abigail::elf_reader::status status =3D abigail::elf_reader::ST= ATUS_OK;=0A= >>> dwarf_reader::read_context_sptr ctxt;=0A= >>> ctxt =3D=0A= >>> dwarf_reader::create_read_context(vmlinux, di_roots, env.get(= ),=0A= >>> @@ -2569,6 +2591,7 @@ maybe_load_vmlinux_dwarf_corpus(corpus::ori= gin origin,=0A= >>> << vmlinux << "' ...\n" << std::flush;=0A= >>>=0A= >>> // Read the vmlinux corpus and add it to the group.=0A= >>> + status =3D abigail::elf_reader::STATUS_OK;=0A= >>> t.start();=0A= >>> read_and_add_corpus_to_group_from_elf(*ctxt, *group, status);= =0A= >>> t.stop();=0A= >>> @@ -2579,7 +2602,7 @@ maybe_load_vmlinux_dwarf_corpus(corpus::ori= gin origin,=0A= >>> << t << "\n";=0A= >>>=0A= >>=0A= >> At this point if `vmlinux' file doesn't have DWARF information, the `sta= tus'=0A= >> returned by `maybe_load_vmlinux_dwarf_corpus' will set the bit field=0A= >> `STATUS_DEBUG_INFO_NOT_FOUND', but it is not verified here, and since vm= linux=0A= >> corpus was already added into the group in `read_debug_info_into_corpus'= =0A= >> function, it continues processing modules without the main corpus inform= ation,=0A= > =0A= > I see. You are right. Yes, the debug info is not found in vmlinux and y= et the=0A= > whole thing continues, collecting just information from the ELF symbol=0A= > table, basically, and from the modules. Pretty useless, I guess.=0A= > =0A= >> Is this the expected behaviour?=0A= > =0A= > Hehe, no :-)=0A= > =0A= > I guess maybe the caller should look for the .debug_info section in the= =0A= > vmlinux section (or for split debug info), prior to even calling=0A= > maybe_load_vmlinux_dwarf_corpus. If there is no debug info, then the=0A= > function should proceed directly to calling=0A= > maybe_load_vmlinux_ctf_corpus? What do you think?=0A= > =0A= =0A= Yes, it sounds good!!, just I would like to know your opinion about of what= =0A= will happen when neither DWARF nor CTF debug information is found, the curr= ent=0A= behavior is to extract symbols information and compare them, so which symbo= l=0A= information should I use DWARF::symtab or CTF::symtab?=0A= =0A= And an additional use case is whether the tools `kmidiff', `abidiff' could= =0A= compare a DWARF IR with CTF IR? I exercised it with some libraries and bina= ries=0A= using `abidiff' (finding a couple of problems in CTF reader (already fixed)= =0A= and three possible issues for DWARF, I will submit information and the test= =0A= cases about those) but in general seems to be work!, but before to continu= e=0A= I would like to know your thoughts.=0A= =0A= > [...]=0A= > =0A= >>> I have also introduced a new function called=0A= >>> tools_utils::dir_contains_ctf_archive to look for a file that ends with= =0A= >>> ".ctfa". This abstracts away the search for "vmlinux.ctfa" as I wasn't= =0A= >>> sure if those archives could exist for normal (non-kernel) binaries as= =0A= >>> well:=0A= >>=0A= >> Ohh, perfect!, I'll use it in CTF reader to located the Linux archive fi= le.=0A= >> No. there is no `.ctfa' file for non-kernel binaries intead they have `.= ctf'=0A= >> section, I could implement a similary function to looks for `.ctf' secti= on=0A= >> using elf helpers=0A= > =0A= > Right, abg-elf-helpers.h does have find_section_by_name. That can be=0A= > used to look for the debug info, I guess. However, we also need to=0A= > support finding the debug info when it's split out into a different=0A= > place, like when it's packaged in a separate debug-info package. Today,= =0A= > abg-dwarf-reader.cc uses dwfl (dwarf front-end library, I believe) to do= =0A= > this, as dwfl knows how to find the DWARF debug info, wherever it is.=0A= > =0A= =0A= Ohh, your are right, I saw `find_alt_debug_info', and in case of CTF front-= end=0A= we don't use dwfl, it is done by `find_alt_debuginfo', reading directly the= content=0A= of `.gnu_debuglink' section, I'm not sure if CTF reader can use `find_alt_d= ebug_info'=0A= because it calls dwfl_* functions seems be DWARF specific. So I'll investig= ate.=0A= =0A= > You can see how this is done in read_context::load_debug_info(), in=0A= > abg-dwarf-reader.cc, around line 2654. Look for the comment "Look for=0A= > split debuginfo files". Basically, dwfl_module_getdwarf returns a=0A= > pointer to the debug info it's found, if it has found one. I think we=0A= > should split this logic out to make it re-usable somehow.=0A= > =0A= > If you think this is worthwhile, I can think of splitting it out and=0A= > stick it into elf-helpers, maybe?=0A= > =0A= =0A= It will be really useful!, but I'm not sure `dwfl_module_getdwarf'=0A= can operate in ELF without `.debug_*' sections.=0A= =0A= > =0A= >> and it can be used in `load_corpus_and_write_abixml'=0A= >> implementing a similar algorithm as with when we are processing the Kern= el,=0A= >> looking for DWARF information, and if it is not present then, test if=0A= >> `.ctf' section is in ELF file then extract it using CTF reader,=0A= >> to avoid duplication use of:=0A= >>=0A= >> abigail::ctf_reader::read_context_sptr ctxt=0A= >> =3D abigail::ctf_reader::create_read_context(opts.in_file_path,=0A= >> opts.prepared_di_root_paths,=0A= >> env.get());=0A= >>=0A= >> One for `opts.use_ctf' and other one when `STATUS_DEBUG_INFO_NOT_FOUND' = is returned.=0A= >> WDYT?=0A= > =0A= > Yes, along with the testing for the presence of DWARF debug info, that=0A= > might be useful, indeed.=0A= > =0A= =0A= Agree.=0A= =0A= > [...]=0A= > =0A= >>> But then, it's here that we are going to inspect c1_status to see if=0A= >>> loading DWARF failed. If it failed, then we'll try to load CTF. So,= =0A= >>> here is the change I am adding to the process of loading the corpus c1:= =0A= >>>=0A= >>>=0A= >>> @@ -1205,6 +1208,36 @@ main(int argc, char* argv[])=0A= >>> set_suppressions(*ctxt, opts);=0A= >>> abigail::dwarf_reader::set_do_log(*ctxt, opts.do_log= );=0A= >>> c1 =3D abigail::dwarf_reader::read_corpus_from_elf(*= ctxt, c1_status);=0A= >>> +=0A= >>> +#ifdef WITH_CTF=0A= >>> + if (// We were not instructed to use CTF ...=0A= >>> + !opts.use_ctf=0A= >>=0A= >> This is always true, because we are in the else block of `opts.use_ctf'.= =0A= > =0A= > Right.=0A= > =0A= >>=0A= >>> + // ... and yet, no debug info was found ...=0A= >>> + && (c1_status & STATUS_DEBUG_INFO_NOT_FOUND)=0A= >>> + // ... but we found ELF symbols ...=0A= >>> + && !(c1_status & STATUS_NO_SYMBOLS_FOUND))=0A= >>> + {=0A= >>> + string basenam, dirnam;=0A= >>> + base_name(opts.file1, basenam);=0A= >>> + dir_name(opts.file1, dirnam);=0A= >>> + // ... the input file seems to contain CTF=0A= >>> + // archive, so let's try to see if the file=0A= >>> + // contains a CTF archive, who knows ...=0A= >>> + if (dir_contains_ctf_archive(dirnam, basenam))=0A= >>=0A= >> Non-kernel binaries contains `.ctf' section instead of `ctfa' file,=0A= >> so I can implement a `file_contains_ctf_section' function to test if=0A= >> it is a valid input file for CTF reader.=0A= > =0A= > Great, thanks.=0A= > =0A= > OK, I'll look into trying to put together some facility to look for the= =0A= > presence of DWARF debug info, so tools can decide ahead of time what=0A= > front-end to use.=0A= > =0A= =0A= Really nice!.=0A= =0A= > [...]=0A= > =0A= > Cheers,=0A= > =0A= =0A= Regards,=0A= guillermo=0A= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by sourceware.org (Postfix) with ESMTPS id F41383858CDB for ; Fri, 7 Oct 2022 20:28:59 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org F41383858CDB Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 297IJGxd002291; Fri, 7 Oct 2022 20:28:56 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=message-id : date : from : subject : to : cc : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=corp-2022-7-12; bh=j4xB8F0EQ5cM+MI4DA50gyr2rdYUu1mEFRKKyIJCc4Q=; b=VSvQTCCrUWhgpho7nDzYmDCjouBbnOLxdr4G6ImKUb0oWnfRr4jQYEn7zFvF8QUumHjm rufspdf+Bnzg6DecVy7bp3q17I9RRvEf6dYiskMnU9XkpXUdtBksVbXeVZVL3qXUwBHS uvrM0LMkJX73zSJc+tYbbAMu+hV1NCmQVmH1cLsIK2RdHPzjJHP88oKABmxpurQAZ0x9 m/XP8S+sAHq2/vO1DDFg2HGplvJzN3PofYpvgdgYjXd+6o0sa4CRmYkuywJ8O1D7Hf/L pjcmVMo1uCVDs+8QrgChUorUR/cIrifeGUbLErxZoP/Ty8h4Mo1BVD7B8VMc7gz2bKaW qg== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3jxdeag5u9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 07 Oct 2022 20:28:56 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 297Jporu028246; Fri, 7 Oct 2022 20:28:55 GMT Received: from nam11-bn8-obe.outbound.protection.outlook.com (mail-bn8nam11lp2169.outbound.protection.outlook.com [104.47.58.169]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3jxc07mg7p-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 07 Oct 2022 20:28:55 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nO3EXr+qqulKTOOuFNz1uWKXq5i/GQbdTRxAQAkLm+Lvy1OrBocX5ukY8eQ0l6RBjMTrBzQK21ycCuhnlBeTueN45ZCSpQyDRVm1nufzqt4pZTCHaI0t240yu1tJky64svAccPMeFMSvIQKzRMtcUBkhuFptMyg7XmZnUybd3N8+6ksRnGzcCZndqm42i6QAknbZlrsWI3jj4haPM8XcK7+6iMlkPj+ixYcWTZNLuL8U0o6TArn/E9Ebx+hxOTS08PR3Djgmc56CH/wLjimZXd4YJ9vWBMJJnTBTLfzt2p/4yD+Gcp2KVZqPMTdZnhZ59xm6GVedzuJjX3tEgbYPQA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=j4xB8F0EQ5cM+MI4DA50gyr2rdYUu1mEFRKKyIJCc4Q=; b=FTVNweDyF+tpR8vdcX5BrpzuGafwDhei78751udAFwCbUgssCDqXqaG1nXkdEQ4LEJeK6EEQaF8uq1iOlMCD4e/1rzVttC/TY+keoQqYtI+JEpRWIlIDIhjbHYdwnrkgBIWxLRvkxl5su7ksa+va+rmwag1cUIQJ15jCIs3N07N5meQ+GpucLq/yqsPJ2IFV6cMsT1TbGKf/iVBgew75VP6kYo4i6uNg/iwtK5s+O63F1Q8umtkiwEJ8BvA4rqIa4RiW1FcvEDBOB02rdRobYzzNYRRVd7pxj0aVuBDTGcibWztHnEDGoXHKr/39I6i8WM+F5u3vbyYtYJWWDoR00g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=j4xB8F0EQ5cM+MI4DA50gyr2rdYUu1mEFRKKyIJCc4Q=; b=aLyaHSHc/Et0UWR/wf1S7xZsMIAitQsM+lWmM+Yq7yBO0KE4nlNNuzbn5TOoqbyEJ9nGa0Gq5Mrnav+qQ7QjwDFIJHZeFjHoccnjN78S/BNtnmKWbGUh3F+vdR0WF9fqLA/1V9ao0oZsUKLIolOEKAGx/A9c1QR79JvTSjbp6Gw= Received: from DM5PR10MB1401.namprd10.prod.outlook.com (2603:10b6:3:f::15) by CO1PR10MB4706.namprd10.prod.outlook.com (2603:10b6:303:9d::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.28; Fri, 7 Oct 2022 20:28:53 +0000 Received: from DM5PR10MB1401.namprd10.prod.outlook.com ([fe80::a8af:6a3c:fd82:5d30]) by DM5PR10MB1401.namprd10.prod.outlook.com ([fe80::a8af:6a3c:fd82:5d30%9]) with mapi id 15.20.5709.015; Fri, 7 Oct 2022 20:28:53 +0000 Message-ID: <7dbdea77-dbc9-7bdd-94ee-8e4cf46ce886@oracle.com> Date: Thu, 6 Oct 2022 14:50:22 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.3.0 From: "Guillermo E. Martinez" Subject: Re: [PATCH] CTF as a fallback when no DWARF debug info is present To: Dodji Seketeli Cc: "Guillermo E. Martinez via Libabigail" References: <20221001001544.210234-1-guillermo.e.martinez@oracle.com> <877d1g2c52.fsf@seketeli.org> <568fe730-3bb9-0267-00bc-2873e94e502f@oracle.com> <86mta9bdpm.fsf@seketeli.org> Content-Language: en-US Organization: Oracle Corporation In-Reply-To: <86mta9bdpm.fsf@seketeli.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-ClientProxiedBy: DM6PR05CA0059.namprd05.prod.outlook.com (2603:10b6:5:335::28) To DM5PR10MB1401.namprd10.prod.outlook.com (2603:10b6:3:f::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM5PR10MB1401:EE_|CO1PR10MB4706:EE_ X-MS-Office365-Filtering-Correlation-Id: c71c50fe-ba65-4997-f6ec-08daa8a28cf0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: j1a1KOfam54Ni++mBi10Hp7C35+LBLmOfQS+1o9TJPHyXOgwBnxdUDS6RCXk1JLy9MfT/Yy11dIVp3eSIeOPKxX7V+aZnUuFIcBI6MY9pFbqJPnm5rqysdbI+kNFBIXmda2IWpHz3Isgyg1nFzthI9GcCA6l/RmQa9WhTq3+ETk95td8mlFTaudvFDBgN2sNB+20VhRT1IfLBEEgoZ/PfZBP/hy+0zL+xmOM2HiAlt+B6ok54qHnSdoUNDKr+ym/jjcBjAnnu3hAM+d12E/EclfIhfpZ46DHYliTRGKjltrZv5KjNiiSiYF7ev+6ByW2wivf45gF6VRFLqS5H+HetNV8sIBSOsBSuSAXlECzDT9RO4MqOT3M1Yhc3XAprqFqcYZO0/6ssNjYjvMi3sje2MHmYwixAkPCwijXQLb1yjDOcLudXNsKemCXCOVRcjCa8hfh2d908GwWZgKjhikpFDFqjMCtspugFs+cl77PNPxCZIINCaXWO6CmYxlAdaQxd4idFLwE+8/4L7YAYTv3uxRDEjf1ujf/YXsCTo0mDO/SnNGk1HXsUr2ffezd1H0c4HgsSHoKqq2rzxXTa7xrE+FScQfa4obRm20KD8E4O/xJqiorI6Y35jolTGTDPupRRRR3EWyYweXwxPlLVcEoth06bsKAdNSTS5qH+TELeC5bH+2DMfOO9aFnVyTiM29bIvxGqaEWDaIRRm5ljPlPju7UE/tAELa/5pI06oH/x/NFqs9F+qtpf6qrNE6T41+qBzrxnotN0Q37ZErGDB/LVWdWerPy4P+ldfuO+3n3XKQ= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM5PR10MB1401.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230022)(136003)(39860400002)(396003)(346002)(366004)(376002)(451199015)(83380400001)(36916002)(186003)(66574015)(38100700002)(2616005)(8936002)(2906002)(5660300002)(41300700001)(66556008)(53546011)(6486002)(6506007)(4326008)(478600001)(8676002)(6666004)(66476007)(66946007)(316002)(31696002)(6512007)(36756003)(86362001)(31686004)(6916009)(43740500002)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?UTc0eTJKRHQ0U2NjeHloOE9EOGpwMmR5eEV0OU94WnJrYmp0V0tUNjFOcjNr?= =?utf-8?B?dGlyN1dYQ0NyMy9oUFltVGUvb25SNXc4LzRsVGsxZEpUWlBNT3Bwc1JwYS9u?= =?utf-8?B?UGpWTi9Ca21HakJMZlFRNXFROVA1cld0WXFBK1lYK0ZXWlNoVk8vVVJTWHNp?= =?utf-8?B?Mm1XTDdNL0FCZGFCZDBBMzVqTCtWQ0F5Mk8rU0Uwc1dOZmdYZGRFRmZmV3VY?= =?utf-8?B?T21YVE15SWZmSVhIUXJKZWhDZW1UY1p4c0ZROER1S2ZvVjBWbWxnVmI2ZmVa?= =?utf-8?B?MHUxSVpMbU1PUVp5bCtWblNpcDl2ajJ4dXVteVFLUm9mVm9uTG4zQjR1K2lY?= =?utf-8?B?a2FxVFlxbk1hWjNzQnhOdlZrVHV4WW53TG0rWEUxdWg1QmRPemw5U001MmZT?= =?utf-8?B?eGVYQ1FVQkEwbFFlSktqMjZJeGpwRVpsR3JXSTZNR0dwZkFrdHI1czUrR2Z3?= =?utf-8?B?bkxhVVo4YVNYL2d4QUYzZVhWV09BQ3BwVWVicHF5WW54eUszQ3dYM2owTDBh?= =?utf-8?B?NUkvekwrOStuemF2YmFQZ0VwSEpQais1eW9JaGFKbW9YRFA2VTZlcE0xcFJm?= =?utf-8?B?R0xGb0Y3MllVL0IrcTdmYm1rdFliTHJmbDZzTlVnN2J1RjBpYXFWMUNKTWFM?= =?utf-8?B?WVpBcnFKanE3UUNhb1M4TmxGZHkvNXJKd0QyZ3dTUWlsNFJzSGtsSXZFZnhX?= =?utf-8?B?YWFwUE9rNFVZQmRUQlBtdVFJUDN1bk5BdW4wdTNCU2U3RGdNRThKYmF1WWY5?= =?utf-8?B?MENscVdBVUNBbHFrSVd1bnVPRU4wM2h5T1FKK08reDVFc1E1WEJhYzFKdmtZ?= =?utf-8?B?VUI4VSsxOXpoUC9zRm1vVmt2RDU2ZWV0YTY0OTlKSVBMc25NeCtBNEJ5Yi9Y?= =?utf-8?B?czZNN0M5a0t4RUNBTDI3MmVnUWk0QUVIS2k5S3VNOTM5TWNCeVhWejVIYUNU?= =?utf-8?B?bnlnUkgrRkVGZHZMcC9PbjFCYVZYLzdwanZ3V2tZUys0a1BiVnFOYkJyUlpi?= =?utf-8?B?THV1bGRKZmU4QmptaGliVWxJWEZXUUEyMVdLZjZqbGo1VHBLcFE0cDg1bEdV?= =?utf-8?B?c3dCYjlqUjNTN3JLTHZnbkV6S09SaEhQV2c1d000alB5QUF1dmliOVJZdzZK?= =?utf-8?B?MU1MN1lQV0lHanpaTnhMcmU3NTk1R3o2R2l0TW01V2hZOXRVVXEwTXllUWJv?= =?utf-8?B?Z2p5anhqT0ZUaDdlYjc5UUwyZXJCSDRXZGQvYTQvSkhhOVhpc2JkWDJ3TWZp?= =?utf-8?B?bVdTQ1FNRFRBaGhvN21nZjNOM1NXd01id0o2TExDVnJrNkhoWDNTam45a2Jz?= =?utf-8?B?TE0rRjV6K0xuZDFVejNuc0ZFVDRBNGZOaTY1RDYwOXcvSjFBU1lidTFSME9S?= =?utf-8?B?NE9XenMvNkFSNTFlZ2xXL3crb0I2Myt0dVcvTit1cWp0REJRY2tzQzUrcjdH?= =?utf-8?B?MEE3bDk4Q3c0d0JwL3M5ZGF5YlpGbVZscjZoNVZGL0R0bFExSmtsTVc4bjdt?= =?utf-8?B?eHFTc0J0RW4zNUpHaUJWT0lQRFNYMndCMTdLTXhmb2NNSE1UM0x4SHF2b2J6?= =?utf-8?B?dXQzUWo1bk5USHhDNWhPY1liZWJLbDhMaDFMYXhseXZ6dXcxSUJEU0dhSnEw?= =?utf-8?B?SHo4MldYREROWTgyaDcxakluVmVRY0tCS0cyZFVaYitJOXlqQXQwRXVGZGZF?= =?utf-8?B?blBKYnhuUWl1dFFFbEdpTjE0NFVyaE04RFp3UkVxRXBoWHBMZXBUL3krTmF6?= =?utf-8?B?cTFMaWgvY0J3d0VBczBtYXJCQXNQLzAybWNGVjdtTmVYbXNva3R0VGVneGMy?= =?utf-8?B?d0JUVFJZNFVwYU4vZmthWWFtVmo3USttZ3BQNmJLZldKSW8zcTNNK0RQMGNT?= =?utf-8?B?TDBubC85UzF2QUpLd2JFSnlDVWlLTHNaclk2eUFLb3h1b3o4V1JLQkY2RGEw?= =?utf-8?B?TVo0TDh0azk1WGVvRUlJZmYyNmxsVXBJU1h2ZjhVOGlkWjlRVVFKQTBQY242?= =?utf-8?B?dmduTlNZa1FvOW1NdW5pRzRjNWhxRkUvZDBSd2RSUEZqYjh3K0EvYkI1d2lx?= =?utf-8?B?cXBtVlJCRDVGbEQ4cERLNnFWNCttWUJLcUpqUWJZaU9vYTFWWG55SDQ5Z2JC?= =?utf-8?B?bi9iWjZ1VE9VUXFjRWtTSFU1V3pTd2lWUmpxQ1ZZT3V0cTBCbmJBbGJ4QVo4?= =?utf-8?Q?X1FaIAOOFrmW+Acwt3vUQRo/8pwsy/XKTE2eCZRczvO6?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: c71c50fe-ba65-4997-f6ec-08daa8a28cf0 X-MS-Exchange-CrossTenant-AuthSource: DM5PR10MB1401.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Oct 2022 20:28:53.0054 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: YZl4Vobww3mu9M68tHN/odfkK5O5+4Jkq5OWlwkVBK5Yxn4BuvJvzRNf5LiRncnckkntR/rkWLvSV7enN/4AEJ0s4DoilEvCJD9paBzIi98= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR10MB4706 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.528,FMLib:17.11.122.1 definitions=2022-10-07_04,2022-10-07_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 mlxscore=0 mlxlogscore=999 bulkscore=0 malwarescore=0 adultscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2209130000 definitions=main-2210070121 X-Proofpoint-GUID: w7ZU4503LXJ7qAJVf2uuZWOHw4krl-eQ X-Proofpoint-ORIG-GUID: w7ZU4503LXJ7qAJVf2uuZWOHw4krl-eQ X-Spam-Status: No, score=-6.1 required=5.0 tests=BAYES_00,DATE_IN_PAST_24_48,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Message-ID: <20221006195022.hlqZ3EypA9aIhpJGBhW_MEAkfTRFZN9RTdiyTGDz2wk@z> On 10/6/22 02:42, Dodji Seketeli wrote: > Hello Guillermo, > Hello Dodji, Thanks for your comments! > "Guillermo E. Martinez" a écrit: > > [...] > >>> I have also introduced a new function called >>> tools_utils::dir_contains_ctf_archive to look for a file that ends with >>> ".ctfa". This abstracts away the search for "vmlinux.ctfa" as I wasn't >>> sure if those archives could exist for normal (non-kernel) binaries as >>> well: >> >> Ohh, perfect!, I'll use it in CTF reader to located the Linux archive file. > > ACK. > > [...] > >>> @@ -2525,8 +2542,12 @@ get_binary_paths_from_kernel_dist(const string& dist_root, >>> /// @param t time to trace time spent in each step. >>> /// >>> /// @param env the environment to create the corpus_group in. >>> -static void >>> -maybe_load_vmlinux_dwarf_corpus(corpus::origin origin, >>> +/// >>> +/// @return the status of the loading. If it's >>> +/// abigail::elf_reader::STATUS_UNKNOWN, then it means nothing was >>> +/// done, meaning the function got out early. >>> +static abigail::elf_reader::status >>> +maybe_load_vmlinux_dwarf_corpus(corpus::origin& origin, >>> corpus_group_sptr& group, >>> const string& vmlinux, >>> vector& modules, >>> @@ -2539,10 +2560,11 @@ maybe_load_vmlinux_dwarf_corpus(corpus::origin origin, >>> timer& t, >>> environment_sptr& env) >>> { >>> + abigail::elf_reader::status status = abigail::elf_reader::STATUS_UNKNOWN; >>> + >>> if (!(origin & corpus::DWARF_ORIGIN)) >>> - return; >>> + return status; >>> >>> - abigail::elf_reader::status status = abigail::elf_reader::STATUS_OK; >>> dwarf_reader::read_context_sptr ctxt; >>> ctxt = >>> dwarf_reader::create_read_context(vmlinux, di_roots, env.get(), >>> @@ -2569,6 +2591,7 @@ maybe_load_vmlinux_dwarf_corpus(corpus::origin origin, >>> << vmlinux << "' ...\n" << std::flush; >>> >>> // Read the vmlinux corpus and add it to the group. >>> + status = abigail::elf_reader::STATUS_OK; >>> t.start(); >>> read_and_add_corpus_to_group_from_elf(*ctxt, *group, status); >>> t.stop(); >>> @@ -2579,7 +2602,7 @@ maybe_load_vmlinux_dwarf_corpus(corpus::origin origin, >>> << t << "\n"; >>> >> >> At this point if `vmlinux' file doesn't have DWARF information, the `status' >> returned by `maybe_load_vmlinux_dwarf_corpus' will set the bit field >> `STATUS_DEBUG_INFO_NOT_FOUND', but it is not verified here, and since vmlinux >> corpus was already added into the group in `read_debug_info_into_corpus' >> function, it continues processing modules without the main corpus information, > > I see. You are right. Yes, the debug info is not found in vmlinux and yet the > whole thing continues, collecting just information from the ELF symbol > table, basically, and from the modules. Pretty useless, I guess. > >> Is this the expected behaviour? > > Hehe, no :-) > > I guess maybe the caller should look for the .debug_info section in the > vmlinux section (or for split debug info), prior to even calling > maybe_load_vmlinux_dwarf_corpus. If there is no debug info, then the > function should proceed directly to calling > maybe_load_vmlinux_ctf_corpus? What do you think? > Yes, it sounds good!!, just I would like to know your opinion about of what will happen when neither DWARF nor CTF debug information is found, the current behavior is to extract symbols information and compare them, so which symbol information should I use DWARF::symtab or CTF::symtab? And an additional use case is whether the tools `kmidiff', `abidiff' could compare a DWARF IR with CTF IR? I exercised it with some libraries and binaries using `abidiff' (finding a couple of problems in CTF reader (already fixed) and three possible issues for DWARF, I will submit information and the test cases about those) but in general seems to be work!, but before to continue I would like to know your thoughts. > [...] > >>> I have also introduced a new function called >>> tools_utils::dir_contains_ctf_archive to look for a file that ends with >>> ".ctfa". This abstracts away the search for "vmlinux.ctfa" as I wasn't >>> sure if those archives could exist for normal (non-kernel) binaries as >>> well: >> >> Ohh, perfect!, I'll use it in CTF reader to located the Linux archive file. >> No. there is no `.ctfa' file for non-kernel binaries intead they have `.ctf' >> section, I could implement a similary function to looks for `.ctf' section >> using elf helpers > > Right, abg-elf-helpers.h does have find_section_by_name. That can be > used to look for the debug info, I guess. However, we also need to > support finding the debug info when it's split out into a different > place, like when it's packaged in a separate debug-info package. Today, > abg-dwarf-reader.cc uses dwfl (dwarf front-end library, I believe) to do > this, as dwfl knows how to find the DWARF debug info, wherever it is. > Ohh, your are right, I saw `find_alt_debug_info', and in case of CTF front-end we don't use dwfl, it is done by `find_alt_debuginfo', reading directly the content of `.gnu_debuglink' section, I'm not sure if CTF reader can use `find_alt_debug_info' because it calls dwfl_* functions seems be DWARF specific. So I'll investigate. > You can see how this is done in read_context::load_debug_info(), in > abg-dwarf-reader.cc, around line 2654. Look for the comment "Look for > split debuginfo files". Basically, dwfl_module_getdwarf returns a > pointer to the debug info it's found, if it has found one. I think we > should split this logic out to make it re-usable somehow. > > If you think this is worthwhile, I can think of splitting it out and > stick it into elf-helpers, maybe? > It will be really useful!, but I'm not sure `dwfl_module_getdwarf' can operate in ELF without `.debug_*' sections. > >> and it can be used in `load_corpus_and_write_abixml' >> implementing a similar algorithm as with when we are processing the Kernel, >> looking for DWARF information, and if it is not present then, test if >> `.ctf' section is in ELF file then extract it using CTF reader, >> to avoid duplication use of: >> >> abigail::ctf_reader::read_context_sptr ctxt >> = abigail::ctf_reader::create_read_context(opts.in_file_path, >> opts.prepared_di_root_paths, >> env.get()); >> >> One for `opts.use_ctf' and other one when `STATUS_DEBUG_INFO_NOT_FOUND' is returned. >> WDYT? > > Yes, along with the testing for the presence of DWARF debug info, that > might be useful, indeed. > Agree. > [...] > >>> But then, it's here that we are going to inspect c1_status to see if >>> loading DWARF failed. If it failed, then we'll try to load CTF. So, >>> here is the change I am adding to the process of loading the corpus c1: >>> >>> >>> @@ -1205,6 +1208,36 @@ main(int argc, char* argv[]) >>> set_suppressions(*ctxt, opts); >>> abigail::dwarf_reader::set_do_log(*ctxt, opts.do_log); >>> c1 = abigail::dwarf_reader::read_corpus_from_elf(*ctxt, c1_status); >>> + >>> +#ifdef WITH_CTF >>> + if (// We were not instructed to use CTF ... >>> + !opts.use_ctf >> >> This is always true, because we are in the else block of `opts.use_ctf'. > > Right. > >> >>> + // ... and yet, no debug info was found ... >>> + && (c1_status & STATUS_DEBUG_INFO_NOT_FOUND) >>> + // ... but we found ELF symbols ... >>> + && !(c1_status & STATUS_NO_SYMBOLS_FOUND)) >>> + { >>> + string basenam, dirnam; >>> + base_name(opts.file1, basenam); >>> + dir_name(opts.file1, dirnam); >>> + // ... the input file seems to contain CTF >>> + // archive, so let's try to see if the file >>> + // contains a CTF archive, who knows ... >>> + if (dir_contains_ctf_archive(dirnam, basenam)) >> >> Non-kernel binaries contains `.ctf' section instead of `ctfa' file, >> so I can implement a `file_contains_ctf_section' function to test if >> it is a valid input file for CTF reader. > > Great, thanks. > > OK, I'll look into trying to put together some facility to look for the > presence of DWARF debug info, so tools can decide ahead of time what > front-end to use. > Really nice!. > [...] > > Cheers, > Regards, guillermo