From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by sourceware.org (Postfix) with ESMTPS id ECE363857C50 for ; Thu, 2 Sep 2021 19:25:15 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org ECE363857C50 Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 182IA49n029120; Thu, 2 Sep 2021 19:23:12 GMT Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by mx0b-00069f02.pphosted.com with ESMTP id 3atdw5btg8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 02 Sep 2021 19:23:11 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 182JAXLY000789; Thu, 2 Sep 2021 19:23:10 GMT Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2041.outbound.protection.outlook.com [104.47.66.41]) by userp3030.oracle.com with ESMTP id 3ate06asuf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 02 Sep 2021 19:23:10 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=C234LOryQcAKbTWRfybBLNSklZ7T84PHRktYIfhM1USf13rI6gGfc5GVcovnzrpVnt0SCbuHIYxHSmLrjuZ8klCCiWPT8q4ES/42lsgHBY+hgkG9gpc7QjjznZOmlgCv1QBDSsSELdXO62dsTFhjfgmPvXm1XxLUJ4m6mbDBzT/Aj043BTuZ2SyNhRWEYyV15+GJ0ri3VmD+/ebyMyGVSUCQpCLt7Tl2BX/42CNWcaCXeLM7XviDfEBPfqnpBup9+b5hHFKGBrd5g3qwUjwB9/wXg4Xpylya4Vw0tN+f8+f7jsJZCGWSaYkd6mt/q+M2EdP+htD1nOsr/mDBu0cOIQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4DB834rzD0BGv3I6WqY9BKkSBPtxCI5EOtNMf8Nh5cg=; b=gkzcmkm7o74gHVjZ/04mtYjmmu4p1VAutTfFxhNNPUfO8SO8rJTf1TOghx1y7SUnpVTPR58yiKDcEODTrFgMaE7Zs0BWj2TXYwAbMxlfGV2sqPp0DZKgBNzW4tADNEJgcpsRtlmgY1A9J7SsCtEICRtQhV/PGeZYy9b2ph49DTQAkRpZwrtg0grBCXoJQB4f+TdikynNgJzbwr0m7kjN5w+TN92fnWx+OdYDThNikO9XZrZBUVX/9nxZDXlQ+6JGt4WxCuVd3DOW9qRLipwLkmXy8+NYdfUuxSGZsn3IcsDI/5zJKbgW5ZCAE9b0xvECGHssIiYFa0IFsKsVm0/ytA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none Received: from BN6PR10MB1748.namprd10.prod.outlook.com (2603:10b6:405:9::16) by BN6PR1001MB2100.namprd10.prod.outlook.com (2603:10b6:405:35::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.24; Thu, 2 Sep 2021 19:23:07 +0000 Received: from BN6PR10MB1748.namprd10.prod.outlook.com ([fe80::93d:77e4:40df:9042]) by BN6PR10MB1748.namprd10.prod.outlook.com ([fe80::93d:77e4:40df:9042%9]) with mapi id 15.20.4478.021; Thu, 2 Sep 2021 19:23:07 +0000 Subject: Re: [PATCH V7 PING 5] CTF: multi-CU and archive support To: Nick Alcock Cc: gdb-patches@sourceware.org, tom@tromey.com, lsix@lancelotsix.com, "Jose E. Marchesi" References: <1626971472-1848-1-git-send-email-weimin.pan@oracle.com> <74af45fc-ed5e-fbb0-197e-881889440892@oracle.com> <87o89a2a8e.fsf@esperi.org.uk> From: Wei-min Pan Message-ID: Date: Thu, 2 Sep 2021 12:23:04 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 In-Reply-To: <87o89a2a8e.fsf@esperi.org.uk> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-ClientProxiedBy: SJ0PR03CA0071.namprd03.prod.outlook.com (2603:10b6:a03:331::16) To BN6PR10MB1748.namprd10.prod.outlook.com (2603:10b6:405:9::16) MIME-Version: 1.0 Received: from [IPv6:2606:b400:400:744d:8000::941] (2606:b400:8301:1010::16aa) by SJ0PR03CA0071.namprd03.prod.outlook.com (2603:10b6:a03:331::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17 via Frontend Transport; Thu, 2 Sep 2021 19:23:06 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4388dac3-ba3d-46dc-e6ec-08d96e471809 X-MS-TrafficTypeDiagnostic: BN6PR1001MB2100: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:151; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Udpq9tIBRhPFsdjwtmj++BPtJfStY/D4pX8ClGC50mCKcC10WFMV/igRoAOeu2z5SeuEu+hBvTuYkQv6BXmbqCGuOh5pxSlRFBzW0datZ4UGL+EJXwKOEZPGJGuR2WFQ2f/mj1sYUrRDDz1WVPtpiRvPF7tU3fyw9YgfQR/zj0HztkmkGd9OFnfUKy8R6KwZH2Q/ckgUu5CdKlL/ofk7JEVChUAMO7Grrp5XT+euwDW7kHmTwM9X4HtD8SgqqsQykygeSfKARdy4/qnmOiBJV5OokT0KdpioC/hyoEB2s5X62Lv8IaduM/Cwg/SVXtKJLHEO7IxasS1m1WDw4bqJT+CuG82sefWcBHu6MKqFoePsBCXsZ9Uf2LItZ2w7tPcp7NkX+HOAsR7LK5uszQDK8C/yPTpmT/bjFsyRj1NJTOELnzCvfbtA8IOIOdiqOnUZ8dNP6mCQw3dEvzAyIKUQsPD1eDXJkxj2qv4LgkFTTgIsZLGtExQj/HZbycc581jIeCTIMYnejGecERP8a7g7LD9CDK7gzjbAktkQ+S0UfODvE3Up3aLKjGl7AZ0RQ5AqgnpPP6hZPRi9ExDL+XuVqwm68hOZxuqZXHDt1P9bKt6L0/W6+Q1kAIkfnrlJouOqExKYzh2kYvmIXIz/jyZWxs5hr1fMzPGRHBDhkPzYRSgXow/jq6TT4q27LzsqHBMbE8mFlqgc5wc5rDg066eRkHafKsDRvsKTAwFKDOqqYCc= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN6PR10MB1748.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(136003)(366004)(396003)(346002)(376002)(39860400002)(66476007)(31696002)(478600001)(4326008)(66946007)(38100700002)(66556008)(5660300002)(86362001)(36756003)(53546011)(31686004)(186003)(52116002)(316002)(6636002)(2616005)(83380400001)(6486002)(107886003)(6862004)(8676002)(8936002)(2906002)(37006003)(45980500001)(43740500002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?QmJwSG0vdG5RSUdJN3htalFadXpuOFUrYURxZHdaRWhJNk9UUzZDSHBBblJQ?= =?utf-8?B?RVNHU2tVUG9QT1lLQ3NmT2Jzc2IzVzV6WGUyRXlUamVyT0ErdXhWZFZiR20y?= =?utf-8?B?cWRaYjZra2I5Vlp4MVgwbUdwRTFubVp4a1Z6ZldQV2pLRGFldE1Ed2lGMjZJ?= =?utf-8?B?bUNIZDI1UzJpZHQvUllCRnE0YWxqZVdyZ0tweDMwZ3pwcDIvSGZQeXN6djBZ?= =?utf-8?B?d0JpN1poZWtwQUJCZWFZVGFmdHUwSGEwZytMM2NVNW5RSjR6dHlZMnpDRjNj?= =?utf-8?B?clczVHlqMy9wcnE1V3ZPd2hqZU04SkFMZU4xa3lvbWRoZVdBQWthVXJEdjZM?= =?utf-8?B?MUk5R2xicjR2S3BKNHZhM1FXYXVub2JBQkdSeDFCbzErdjJSZmZaL3U1Q0J6?= =?utf-8?B?YmJjNEgvZFh2dDRGai9WcU81QW8yTHFRQkxkR05xVys4RXBKUTNvTVc1ejZX?= =?utf-8?B?ODVWeVdyQXM1NTdzblFWWXNnWmJQQUJYUktsanRjOFZEOGUzL2JZbXk3d0s1?= =?utf-8?B?OXlERlVnMU5GR0NQSFptQlJjeGZ5OXllK2pCcDdXWVNuMUl6eDQvRkI1NmNS?= =?utf-8?B?Nnhla2tEVUFiUjJHQWVSTkRPUGNDVEFNUlpIWWRGZjlWcTUwM1I5UWVhNzdh?= =?utf-8?B?MnhoQzd1bmNTaGs5ZWhZYkpQNE5HcktFajVGRE9zV2ZYVkNlazBMKytrUjJQ?= =?utf-8?B?cmQzcmlLblR0dDZZMmJwZGZlMytTSytsNmlZN2QwVEN3cU5CQ1UyQTM1QzBr?= =?utf-8?B?VWZ4RmlScHNrV3M0aVhIRTZMR044VDlBNUlrb2RMTjZLVUg2UERyeGZYRnI4?= =?utf-8?B?dTJxTk0rSzhMNDJpbjVCTzZwQW0vZHhPQjFMTFpBYk96TzZqVWxnWTVjeGpQ?= =?utf-8?B?dGdXYUJIY2tjVVh6enVZdXFXZ2o4a0JRT2d4Qko0UENGTjNBbzlLMnhYUjEz?= =?utf-8?B?MXVzYVRwOEs1NllFc3RtQnczYnpGb1BUVkhnWi9Gb21CQ2Q3MFlSRHRDVnRv?= =?utf-8?B?RTFUZVAxNmRjNHVscC83UnJrYllkTlpvcU4vTmZWS1hqNFkzenB1cDVjOWlo?= =?utf-8?B?RkRhaDRvR3VIcWxaZTV6WTdhV1pnOWllV3d5TGQwUGZ2a2tCMEtzaDE1SDVp?= =?utf-8?B?dzBKRkMva3VNRjE0bHJYSnl2aDYxaWF0KzltenpzVnZIOUxIYzF5SlZjK1ll?= =?utf-8?B?eHZqTFNYa21RV0ExTDdHWEdHT0prN0JSY01Ya2RtbldyK0RrbnF3TnZHSC9T?= =?utf-8?B?ejdEdkRwaTJXVUNwekF0TFhHeU1ZN2wraDhQUXo0Z001WFE0UnI2NFYxQzJU?= =?utf-8?B?Ujh3dTRaNURYMUpKUzFlRm13RjZKYm9FSzIyTGlvcXV0RnhaZnRMcHRDM3N3?= =?utf-8?B?NXFEVGtlU1VPSWFLU20zTXNBWW9DMEtXaTVobWt2eVhWdHpqa09UNGh1azQ2?= =?utf-8?B?RCs0SnNhSCtoTjFPTFNzQmJCRlRsS3IzeTY1VnhNS1RrL3F0NTl4MmxMSzA2?= =?utf-8?B?YXZJMWFEUjZGbTc2UkNzWk53STNsTTZLM3NqTWJSTnVBc2xjaHcvVHFZYzdW?= =?utf-8?B?VXRxUU9Ycm8ySE43VC9zcDN2RXdVVDZ1U25LcStCVHlIRFdSMGI0TW1JODZI?= =?utf-8?B?MHJ3UU83WHRlK1VQU1Nwb3ZUZnhrOGhRejJxRnpMa3lJMTNkQ3B1UVZnc1M2?= =?utf-8?B?MXZ3YkZEeEV2eTJiQWp4Sy9kNVdma2NEak5jRFIvd3JtQS9iQ051TmtwMm4v?= =?utf-8?B?YlRiamdtYlVCU3RxWXpHSkFLRzRoMm1uRFB2bENqMXp1bGg1R0UrdmtSN3k4?= =?utf-8?B?QmxPdXdXQXNTQkhwckp5Zz09?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4388dac3-ba3d-46dc-e6ec-08d96e471809 X-MS-Exchange-CrossTenant-AuthSource: BN6PR10MB1748.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2021 19:23:07.7380 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 8SbUL3HApm8XTZlkCM9iMRQsLHO7zWmGGMNFR9Xz/ZCYLwqZUXvqIGlZesTb/0/xhidnWTv9HpNPbX0QD7jbSg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR1001MB2100 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10095 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 adultscore=0 mlxscore=0 phishscore=0 malwarescore=0 suspectscore=0 bulkscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2108310000 definitions=main-2109020109 X-Proofpoint-GUID: HNTX3FD5210oMyzFOv7gPAQqP3xSbOOT X-Proofpoint-ORIG-GUID: HNTX3FD5210oMyzFOv7gPAQqP3xSbOOT X-Spam-Status: No, score=-6.4 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, KAM_STOCKGEN, MSGID_FROM_MTA_HEADER, NICE_REPLY_A, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gdb-patches@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gdb-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Sep 2021 19:25:17 -0000 On 9/2/2021 9:49 AM, Nick Alcock wrote: > On 5 Aug 2021, Wei-min Pan said: >> + while ((tid = ctf_symbol_next (ccp->fp, &i, &tname, functions)) != CTF_ERR) >> + { >> + type = get_tid_type (ccp->of, tid); >> + if (type == nullptr) >> + continue; >> + sym = new (&ccp->of->objfile_obstack) symbol; >> + OBJSTAT (ccp->of, n_syms++); >> + SYMBOL_TYPE (sym) = type; >> + SYMBOL_DOMAIN (sym) = VAR_DOMAIN; >> + SYMBOL_ACLASS_INDEX (sym) = LOC_STATIC; >> + sym->compute_and_set_names (tname, false, ccp->of->per_bfd); >> + add_symbol_to_list (sym, ccp->builder->get_global_symbols ()); >> + set_symbol_address (ccp->of, sym, tname); >> + } > This part is my biggest real concern, because it aggressively sucks in > every dict (costing performance, since almost no plausible use case is > going to need all those symbols) and then the code in > scan_partial_symbols introduces a dependency on the name of the shared > CTF section's dict! I'm not sure how you figure out which dict a given > symbol belongs in from this code, but this really does feel like > something GDB shouldn't need to do itself: we should be able to do type > lookups lazily, only when needed. (I wasn't sure this was possible given > the psymtab abstraction in GDB, but in light of Tom's recent patch > series that does something similar for DWARF, there is clearly nothing > fundamental stopping GDB doing this -- though it seems like it might be > painful to move away from psymtabs? I'm not sure.) > > I know next to nothing about the inside of GDB, but the libctf lookup > machinery in 2.37 and above lets you do this to look up the type of the > symbol named "foo" without traversing symbols at all (there is also > ctf_arc_lookup_symbol which takes a symbol index, if that is more > useful): > > ctf_archive_t *arc; > ctf_file_t *fp; > ctf_id_t id; > int err; > > /* ... */ > > if ((fp = ctf_arc_lookup_symbol_name (arc, "foo", &id, &err)) == NULL) > /* possible errors in 'err' include ECTF_NOTYPEDAT, ECTF_NOFUNCDAT, > ECTF_NOSYMTAB. If there is no error, you can do this: */ > > printf ("Type of foo is %s\n", foo = ctf_type_aname (fp, id)); > if (ctf_cuname (fp)) > printf ("Foo's type has conflicting definitions: this definition " > "is in %s\n", ctf_cuname (fp)); > ctf_dict_close (fp); > > ctf_arc_lookup_symbol_name() takes the CTF archive as a whole and a > symbol name and returns the dict in which that symbol's type is found, > and (optionally, but you'll almost always want it) the ctf_id_t of the > symbol's type as well. > > Almost all of this is cached, so subsequent lookups should be almost > instantaneous. libctf does need to traverse the symtab to do lookups by > name, but that is cached as well, so subsequent lookups won't need to > retraverse the portion of the symtab that has already been traversed by > prior calls. > > Doing this also means you don't need to worry about figuring out which > dict a symbol's type is found in (if the type has multiple conflicting > definitions it might well be in a child dict): > ctf_arc_lookup_symbol_name will always return the right one. If you > actually care whether the type is unambiguously defined, you can use > ctf_cuname and/or ctf_parent_dict and/or ctf_type_isparent to determine > if this type is one shared between dicts; but I suspect most uses won't > care, and almost certainly none within GDB will. > > With luck this means you can avoid having to store or pass around > ctf_dict_t's almost entirely, stop worrying about conflicting type > definitions and types defined in multiple child dicts, and just get > whichever one is most suitable at the time you need them via > ctf_arc_lookup_symbol_name. (You might still need to deal with them for > the form of ptype and/or whatis that takes a type name rather than a > symbol, for which you presumably want to find the child dict (if any) > that best matches the CU you're currently in from the perspective of the > rest of GDB, and then do the ctf_lookup_by_name on that dict.) > > (there are a good few uses of this in the libctf testsuite, if that's > any help.) "Partial symbol table" is the traditional implementation that was adopted by almost of all readers in gdb until Tom's indexer which offers a different approach. Note that a partial symtab is not the same as a full symtab and does not contain members of a struct or anything at the function scope, for example. My understanding is that the new indexer will scan and extract names from the .debug_names section and create a symbol name vector. With the help of libctf's symbol lookup functions, it seems we don't need to create such a vector if we decide to eliminate the partial symtab implementation.