From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from aserp2130.oracle.com (aserp2130.oracle.com [141.146.126.79]) by sourceware.org (Postfix) with ESMTPS id 609C43959C2F; Fri, 12 Mar 2021 16:54:25 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 609C43959C2F Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12CGnt5H010849; Fri, 12 Mar 2021 16:54:04 GMT Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by aserp2130.oracle.com with ESMTP id 373y8c2psc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 12 Mar 2021 16:54:03 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12CGrnma102344; Fri, 12 Mar 2021 16:54:03 GMT Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11lp2174.outbound.protection.outlook.com [104.47.57.174]) by aserp3020.oracle.com with ESMTP id 374kn4cpaa-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 12 Mar 2021 16:54:02 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kZqVuBEQTO/p8IT2r4n0Uw3N72Y1MvHHxIByRo5LdRv/Xf0XMl3thxDfVf6qQd8y+NekkMKT9Vem/PSZdE1HsyjRZTIf1NKLlw7zD9RVeH4zkmwrLil09YuK9mrZarscxWZSEiN92PFM/5GcCB7dM/FO4bO61rpDZL0fm2VDnxv8cYs9yUbfPgZiL+lwEcRTFGK0kWjMxyeAbJ46fcmwcS9cOsEbiv8Sg27sdnfbS8CF7dR0ZrNhxe4f0poNz/3HJWPUpvsnz0NKSHpkXPvByEJNb02dd0GjmgZ1v6JkydDOEk/WsxoEBBCxHTNkFnGaEqr11hlgFvh8VZP3Mp15HA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=m9lxMfRPd2/YCF87pC6kAt274JqposHYIuNU7AYRBaI=; b=KBrqNZmd96atnBkx09LWtBbHdIY//UQmP6bv8DtaNOQMeuBOdsUi0eMOiMPZiivHrDWdwu5rLGizv2CqJkzzp+m0d75Rfupm6z5MIzTNA1jDCrd8PXXN5hZr1BjwTjFZpgjEZZdRISKixzGVCFFJkPtQC1wDj5fUrORWrWD8wcyHy/y0qthQWAfWifVHTNtAx/mpM7NEMWIONE2IYrmGz03waLP8wxCtxxcHDTKx3uXltuN1Xg+oEgy8q6CZe0B5Vg6umS+l66TZTgDxzoTd646hvpBxRg46ndu1sKiDk8AQBRE8KfJ565dxqeSsedRr5ssMGOxhsHjFYpzylsZ9FQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none Received: from SA2PR10MB4715.namprd10.prod.outlook.com (2603:10b6:806:fb::10) by SA2PR10MB4699.namprd10.prod.outlook.com (2603:10b6:806:118::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3912.19; Fri, 12 Mar 2021 16:49:00 +0000 Received: from SA2PR10MB4715.namprd10.prod.outlook.com ([fe80::bdcf:f0:1350:5fa2]) by SA2PR10MB4715.namprd10.prod.outlook.com ([fe80::bdcf:f0:1350:5fa2%6]) with mapi id 15.20.3912.030; Fri, 12 Mar 2021 16:49:00 +0000 From: Nick Alcock To: Jakub Jelinek via Binutils Cc: Hannes Domani , Jakub Jelinek , "gcc\@gcc.gnu.org" , Mark Wielaard , "dwz\@sourceware.org" , Allan Sandfeld Jensen Subject: Re: DWZ 0.14 released References: <9f72f4e2-10b8-ac32-565e-b96314911d66@suse.de> <7438883.9MRIfJ1mQo@twilight> <20210309090918.GA28954@wildebeest.org> <463409719.1821183.1615289887767@mail.yahoo.com> <20210309114513.GM745611@tucnak> Emacs: resistance is futile; you will be assimilated and byte-compiled. Date: Fri, 12 Mar 2021 16:48:54 +0000 In-Reply-To: <20210309114513.GM745611@tucnak> (Jakub Jelinek via Binutils's message of "Tue, 9 Mar 2021 12:45:13 +0100") Message-ID: <87o8fo71xl.fsf@esperi.org.uk> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3.50 (gnu/linux) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Originating-IP: [2001:8b0:1101:10::2] X-ClientProxiedBy: LO2P265CA0160.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:9::28) To SA2PR10MB4715.namprd10.prod.outlook.com (2603:10b6:806:fb::10) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from loom (2001:8b0:1101:10::2) by LO2P265CA0160.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:9::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3912.26 via Frontend Transport; Fri, 12 Mar 2021 16:48:59 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b7b1de2b-68d5-4c9f-9e3c-08d8e576bc6f X-MS-TrafficTypeDiagnostic: SA2PR10MB4699: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8882; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Oxy8+sLgrj+u6x3xgWEgpO5hb3dElppmXjurT0p4/s781LY3OIEw9nAqybwImG1fZ8lc5e3Dz9aqXuyW0MeKMl42KyKLZRrc3eeyynT4lvnB9NlEI3KnHZ57s3EBufnWPBoHWXjy190SWNz7f+F0F1n4nmFbFygvXp+jZ9voxHnVq0msAxF/UZQmv4c2lsf8WOUtEKpf/HCau7A4UibKpIPXP8SBFCg8sBy5I49/dSYHzpVK4VeVRMkhuC1Jrz+yBbKkqDEA8q3WayF70pbvXjvVojtlkzoYazpQoSbGWS6U5LzhRFVb+M6xSgXPbDAbm+jvQnnlZoyqBj0YjxEEWyeYHcquCrpgSTCASvDD2c5W+QqwnwLDwu1naRWtEl3dLnI4euW98Y7ek8ggirvzZzQcogw5t8oNncG+znVDaNhtGyfPP9jcfPsCSxfckpYSA6FVioOwyQePUI/5sWRFw4jcsTafJWmlGBEgyqhwKCu0+1v8PoX4t5cDbef7C7WJrmlwEv+gV3Xp9qilD+TiWw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SA2PR10MB4715.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(396003)(376002)(39860400002)(346002)(136003)(366004)(8936002)(16526019)(7116003)(478600001)(6916009)(66556008)(2906002)(66574015)(66946007)(8676002)(54906003)(5660300002)(52116002)(316002)(186003)(66476007)(6496006)(9686003)(83380400001)(44832011)(86362001)(6666004)(36756003)(4326008)(6486002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: =?utf-8?B?THNjUzNBbndlOUJFV1JEUi9OSzhVZkR0SzRRUi93Z1drc2pGNnhKYXpXTzZo?= =?utf-8?B?ZVhkU1dtR2FUK0hXL0dibWlaemZEZjN1Wlg0aFhaYjN1QnNJWGd0UVFOcTdQ?= =?utf-8?B?T2t4NWc2WkUxL0lKeGlkYWgyaUhVa1AyN0poZjRJaGtrdlN3eE1xM0t6MWxM?= =?utf-8?B?TzllWE0zK2wrRmEvUGJkZVRxQmZIMCsvdTMwWHpMRHI2aThOQW1rSkVqWldB?= =?utf-8?B?R0ZXNEYxSm9kbGw0bnZad0c3SE83bEk2ZUdYbVRzT1g3ZnZrUjhwMEFFYSt4?= =?utf-8?B?enlNUEgwZ0F4SzE1S2V1QW5mSkJudXQvQVlNR2MyQjhPS0t5UVZISmxlMmdL?= =?utf-8?B?ODIzbHFGb1lZTXo2Y0tZWDlkd290dm1UbjZHa2s3V3lPT0FXTk5UNVV1cG1k?= =?utf-8?B?T3FNVG82U3lDRG91aUdEOGs4aEh5TUFucXoyenpCSUtkeXFyT1o3WlVNbngr?= =?utf-8?B?dnFqaHpDYXBHWDZaZGlQYnJVMHVrakRhLzltL3dRazNHOUJPS1BQMVZ4MmRr?= =?utf-8?B?SFlLUFRHdUZXTW9xQW83THpINElpTnI5NHozaW5QcWJTdGRyVFZSUWlwOFlS?= =?utf-8?B?VFZNOEF0VElHUW1ZT25OYmd0UjJjT04zeWpPOXVKQllpVnVrRlRQQlRNZy9r?= =?utf-8?B?MUt0OU5DK3Z5ZHlFbVE1RkVRbHdMaVpiKzAyYUtzYzZKTlhMdGZFVGpNdFdo?= =?utf-8?B?bVNQSUhYbHUveG1iZjJzK1J3RmgrZjhZK1hGNXFWb3E2MndhYWVxb1gyT3FL?= =?utf-8?B?T3dPWWxqdEVNRTZHbDl0dmlXVVloNnFYejJrM0VNMmg5d2FOOW9iV2hFWmtk?= =?utf-8?B?eUw5UFo2M25adHh4OVVUNnVpWmp2TkZMQTBBYzhSc3QvNzlZTTZEblgyNGRZ?= =?utf-8?B?bEVqZHVzU2RJWndsa1lXSU1kUTlkUVp0dVpSMDUwMDFURTFJd3g4T3Y1QzU3?= =?utf-8?B?Wm1RaU5vanJFdnZUOUFZOWlod2N2d0RWTmNMZkRkZkpsT3lsOVUwcjBjQnAx?= =?utf-8?B?b21oaEJlSGJsTUtnbU95WEZvbDFNN3BVYVppMmtsTHJ2OWgrZEhmcFNmTEFP?= =?utf-8?B?MnZwbVVhcGYrRUVFSHZUZFhnS3VMQnV6VUZZaFcvd21RUW0xMjVaZUwwblg2?= =?utf-8?B?SG5mZmdvR1ZnS2Rmd0lKT1NqNTNXeEtSbkxiU3J3M0NnS2RlOHJ4ZmRIaVdW?= =?utf-8?B?b0FuVWpGWkovVzBrb3Nxc0pmcjd4bGZXN2JubXN1akJ1T2pHN01JbHMvSThR?= =?utf-8?B?Sm9Fd3Fsdkx4blZzcUhSdlJkMzdTbndsQmdNQVZzN05yOWlTVDNieDYzNGNh?= =?utf-8?B?VGxrN3paa0ZHQVRnODlVZGR5V1VBTTN3TThGakpYV3l4aTYramlHL25TK25l?= =?utf-8?B?Qk5QQ1N1MUpWVTJwVmcvTFFld1lEdTZKd3k1bEg1MkxGWHZoVkpGREE0enhh?= =?utf-8?B?T0pWSnJlWW5oMzNkV21PREh4TWg3T29pUDEyWW5IV1ZHT1R3YXVsTytCZ1U3?= =?utf-8?B?S1BFb3ZTWklVOWFtTXNDeXJEVG1FQWZBaFRGeVBqczlKbmovMmd1TnJxY0Vq?= =?utf-8?B?d2owcFlTWnRCOXBNM1lwYkV4dkgvR3Q0Y3orSzc5ZXMvZzdZS3d0cGRqNzBF?= =?utf-8?B?ZUJYUGFBZ1B4cTJnNGxVWHYrZk0vMTRza3BGNE1IM0pEQXorNld5a1pTN21y?= =?utf-8?B?Qmp5aThCWUZGQ3pSMVk5RjFYNTN3OThjUmxOK3BiSlU1NHhCNnFyanY0b0hk?= =?utf-8?B?bnZXM1REZUs1cUVESHdmZTY5aUF2d1p6a1AxUURURnFrQUgrakkxaTZobzlr?= =?utf-8?B?dXFvTEVIekVYN3lNdFExQT09?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: b7b1de2b-68d5-4c9f-9e3c-08d8e576bc6f X-MS-Exchange-CrossTenant-AuthSource: SA2PR10MB4715.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Mar 2021 16:49:00.5677 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: aIkKl3ZEXAJCqcoBwIoZ8dDDHfirO6QcMQx2zuaZ2VQypk72UMPW2tKkCAFvlm5G3pQatOUuBO6qwecizNTVoQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA2PR10MB4699 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9921 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 bulkscore=0 malwarescore=0 spamscore=0 mlxlogscore=999 phishscore=0 adultscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2103120123 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9921 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 clxscore=1011 mlxscore=0 phishscore=0 lowpriorityscore=0 malwarescore=0 suspectscore=0 adultscore=0 mlxlogscore=999 spamscore=0 bulkscore=0 priorityscore=1501 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2103120122 X-Spam-Status: No, score=-5.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, MSGID_FROM_MTA_HEADER, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: dwz@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Dwz mailing list List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2021 16:54:27 -0000 On 9 Mar 2021, Jakub Jelinek via Binutils spake thusly: > On Tue, Mar 09, 2021 at 11:38:07AM +0000, Hannes Domani via Dwz wrote: >> Am Dienstag, 9. März 2021, 10:10:47 MEZ hat Mark Wielaard Folgendes geschrieben: >> >> > Hi Allan, >> > >> > On Tue, Mar 09, 2021 at 09:06:54AM +0100, Allan Sandfeld Jensen wrote: >> > > Btw, question for gcc/binutils >> > > >> > > Any reason the work done by tools like dwz couldn't be done in the compiler or >> > > linker? Seems a bit odd to have a post-linker that optimizes the generated >> > > code, when optimizations should already be enabled. >> > >> > >> > dwz does two kinds of optimization. First it attempts to optimize the >> > DWARF debugging information for a given object (executable or shared >> > library). Secondly it tries to put shared pieces of a list of given >> > objects into a supplemental file that gets referenced from all the >> > given object files. >> > >> > Technically the first optimization could be done by the linker. But >> > the second optimization is really a post-linker step. >> >> Related question: If it were part of binutils, maybe it could be adapted to >> optimize DWARF debugging information of PE files as well. > > dwz intentionally uses libelf, it often deals with very large amounts of > debug info that only barely fit into the address space limitations on > certain arches or physical memory for good performance, and any kind of > abstraction penalty (e.g. bfd) would make it slower and more memory hungry. Well, it's not *impossible*. You could in theory do what CTF has done: move the dedup machinery into a library which is then called both from the linker (providing its input via BFD and explicitly not supporting multifile unification of DWARF) and from a separate dedup tool still called 'dwz' (providing its input via libelf, and flipping a switch allowing dedup across files and producing an additional output which the tool then writes to the unified file). Howver, link speed would likely be affected if dedup is on by default (it's still a concern of mine with libctf even though none of my testcases take more than a couple of seconds to dedup: I know how to shave a *lot* of time off that, I just haven't done it yet). It's quite possible that you'll save as much time by not having to write as much DWARF out as you lose deduplicating -- but the write time is usually hidden from the user anyway since writeback is usually buffered on most operating systems. So this is probably not as helpful as it might appear :( Also, it might not be acceptable to have dwz depend on a shared library provided by binutils, nor to have binutils depend (even optionally?) on a shared library provided by dwz...