From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by sourceware.org (Postfix) with ESMTP id 53D27397142D for ; Wed, 10 Feb 2021 11:48:29 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 53D27397142D Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-29-pD0kggQeP6GTykwne_dWcg-1; Wed, 10 Feb 2021 06:48:25 -0500 X-MC-Unique: pD0kggQeP6GTykwne_dWcg-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 33DCF6D4E0; Wed, 10 Feb 2021 11:48:24 +0000 (UTC) Received: from tucnak.zalov.cz (ovpn-112-197.ams2.redhat.com [10.36.112.197]) by smtp.corp.redhat.com (Postfix) with ESMTPS id AD6F85C1BD; Wed, 10 Feb 2021 11:48:23 +0000 (UTC) Received: from tucnak.zalov.cz (localhost [127.0.0.1]) by tucnak.zalov.cz (8.16.1/8.16.1) with ESMTPS id 11ABmJ4o204606 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Wed, 10 Feb 2021 12:48:20 +0100 Received: (from jakub@localhost) by tucnak.zalov.cz (8.16.1/8.16.1/Submit) id 11ABmIkA204605; Wed, 10 Feb 2021 12:48:18 +0100 Date: Wed, 10 Feb 2021 12:48:18 +0100 From: Jakub Jelinek To: Tom de Vries Cc: dwz@sourceware.org, Mark Wielaard Subject: Re: dwz dwarf-5 optimization Message-ID: <20210210114818.GI4020736@tucnak> Reply-To: Jakub Jelinek References: <55754628-7080-992a-1a6c-48eda68e01b7@suse.de> MIME-Version: 1.0 In-Reply-To: <55754628-7080-992a-1a6c-48eda68e01b7@suse.de> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Spam-Status: No, score=-6.4 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: dwz@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Dwz mailing list List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Feb 2021 11:48:31 -0000 On Wed, Feb 10, 2021 at 11:57:45AM +0100, Tom de Vries wrote: > on IRC you mentioned you wanted to implement an optimization in dwz > supporting dwarf-5, but I don't recall what the optimization was. For > my understanding, could you shortly describe it? Basically, try to use DW_FORM_implicit_const whenever beneficial. The compiler doesn't create DW_FORM_implicit_const very much, optimize_implicit_const in GCC only sees a single TU at a time, while dwz can see the whole binary, shared library or the supplemental object. Also, GCC optimize_implicit_const works only on if all the attributes are the same use DW_FORM_implicit_const, rather than say if 50% of them have the same const, 25% another one and 25% another one, then it might be beneficial to create 3 abbrevs. So, somewhere in compute_abbrevs after build_abbrevs go through the abbrevs sorted by usage counts from highest usage count to lowest and for each abbrev that has candidate attributes (constant form other than DW_FORM_implicit_const with small enough sizes that it could be turned into DW_FORM_implicit_const) go through all DIEs referencing that abbrev and see how many have the same value for one (or more of those attributes), and in that case compare the costs - keeping it as is, or creating a new abbreviation (which implies the size of the new abbreviation added to .debug_abbrev, plus perhaps with >= abbrevs also using larger abbrev numbers vs. getting rid of the attribute in the DIEs) and when beneficial, add new abbreviation with DW_FORM_implicit_const for that (or multiple attributes) and repeat. Not sure if a greedy algorithm would be good enough for that, and what data structures to use for that, possibly gather just once pointers to all DIEs into array and qsort that so that we have DIEs for the same abbrev next to each other and when splitting reorder elts of that array for the particular DIE. compute_abbrevs right now works by doing everything intra-CU only and only afterwards perform some limited attempts to share abbrevs between multiple CUs. That perhaps might not play well with that optimization, because it increases number of abbreviations and so there could be less sharing. So, what we might also consider is doing test compute_abbrevs twice, once in the current mode, where each TU creates its own abbrevs and we then merge them when possible, and then in another mode, which would do build_abbrevs_for_die for all DIEs in all TUs at once (maybe better just all DIEs with the same dwarf version at once), and compare which of these brings bigger savings. Doing all DIEs in all TUs at once might result in larger abbrev numbers in .debug_info, but might decrease the overall size of .debug_abbrev a lot. Unrelated, if we decide to support -fdebug-types, I'd be for only supporting it for -gdwarf-5 when it is in .debug_info (or at least initially), and try to undo the size damage by turning all the DW_UT_type units into DW_UT_partial and replacing the long DW_FORM_ref_sig8 references with normal refs. Jakub