From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from omta34.uswest2.a.cloudfilter.net (omta34.uswest2.a.cloudfilter.net [35.89.44.33]) by sourceware.org (Postfix) with ESMTPS id 2BC20384DB45 for ; Thu, 23 Nov 2023 05:32:05 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 2BC20384DB45 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=tromey.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=tromey.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 2BC20384DB45 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=35.89.44.33 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1700717529; cv=none; b=uMsY3+dtAGpRUcBgkHvZu0hY/VPGeHORiqtj1hlfjIWWar1xcvAG5bjV7Sj2OGECxE91NA26QjULuSSwr63BNYbgPAOAomjATpolMYtjywK3VztGOO6eQ8NraqNHmBkpPef8pVKYskVD0eBxOG5rsSo3zCeOPi3tMgR1XjhaR4I= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1700717529; c=relaxed/simple; bh=5NwQElVnqJ/Uuzulcw+/cOaYJ/H2RObyPm+c6DQJB7Q=; h=DKIM-Signature:From:Subject:Date:Message-Id:MIME-Version:To; b=TPNieWEYBqTGdyEYuguDQ9RaitAGJscLjm8Tx5MhPiAWuz4SJRutlIbgQtSUA8kXycvw3pIUxY6D20sTQvWuI9GI/lfZSNwlzjhs4zbB8VAUsdriUmZqoC9AebSbaKhlxWs3HOz6JP7zvYT5ZJwRd2fQ7Ik8zz2JWTGAoIinhkw= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from eig-obgw-5007a.ext.cloudfilter.net ([10.0.29.141]) by cmsmtp with ESMTPS id 623MrfGCs8Hte62JnrdRyQ; Thu, 23 Nov 2023 05:32:03 +0000 Received: from box5379.bluehost.com ([162.241.216.53]) by cmsmtp with ESMTPS id 62Jmrzy7VqkaT62JmrQKB5; Thu, 23 Nov 2023 05:32:02 +0000 X-Authority-Analysis: v=2.4 cv=bY547cDB c=1 sm=1 tr=0 ts=655ee3d2 a=ApxJNpeYhEAb1aAlGBBbmA==:117 a=ApxJNpeYhEAb1aAlGBBbmA==:17 a=OWjo9vPv0XrRhIrVQ50Ab3nP57M=:19 a=dLZJa+xiwSxG16/P+YVxDGlgEgI=:19 a=IkcTkHD0fZMA:10 a=BNY50KLci1gA:10 a=Qbun_eYptAEA:10 a=CCpqsmhAAAAA:8 a=zstS-IiYAAAA:8 a=kCtKjGHnnMiuMzkq8BUA:9 a=QEXdDO2ut3YA:10 a=ul9cdbp4aOFLsgKbc677:22 a=4G6NA9xxw8l3yy4pmD5M:22 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=tromey.com; s=default; h=To:Content-Transfer-Encoding:Content-Type:MIME-Version: Message-Id:Date:Subject:From:Sender:Reply-To:Cc:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=C2kE7jdg3FM1mDI1kgTBzs1YUH6dotHpiVReSOmVLEA=; b=hHfXF2cotFfJ7ayGUypFl7syf8 MWw6/Lb0DxZE1ff+zbljxFKIYbdwuSZcC7rOb7YTdi2TAQ+GigRCDNIfzql19nCKTqYBkq425pMBn yH502pvNE+XvgpvJXlMmKOpSE; Received: from 97-122-77-73.hlrn.qwest.net ([97.122.77.73]:46354 helo=[192.168.0.21]) by box5379.bluehost.com with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96.2) (envelope-from ) id 1r62Jl-003TGR-3D for gdb-patches@sourceware.org; Wed, 22 Nov 2023 22:32:02 -0700 From: Tom Tromey Subject: [PATCH v3 00/18] Index DWARF in the background Date: Wed, 22 Nov 2023 22:32:09 -0700 Message-Id: <20231122-t-bg-dwarf-reading-v3-0-fc3180de63c4@tromey.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-B4-Tracking: v=1; b=H4sIANrjXmUC/3WOwQ6CMBBEf8X07JrdFkU9+R/GQwsL9AA12wYlh H+X6sF48PiSmTczq8jiOarzZlbCo48+DCuY7UZVnR1aBl+vrDRqQ0QaErgW6oeVBoRt7YcWEN1 h31gy6Ixai3fhxj/f0uttZWcjgxM7VF1W9TYmlhzsfExBpvf4SDn+2UF9otIczWlXlLSnAghS6 C9JQs/Trgq9ytpRfxt/no0aEEpsHJWIpJ39cSzL8gLegvPRAAEAAA== To: gdb-patches@sourceware.org X-Mailer: b4 0.12.4 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - box5379.bluehost.com X-AntiAbuse: Original Domain - sourceware.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - tromey.com X-BWhitelist: no X-Source-IP: 97.122.77.73 X-Source-L: No X-Exim-ID: 1r62Jl-003TGR-3D X-Source: X-Source-Args: X-Source-Dir: X-Source-Sender: 97-122-77-73.hlrn.qwest.net ([192.168.0.21]) [97.122.77.73]:46354 X-Source-Auth: tom+tromey.com X-Email-Count: 2 X-Org: HG=bhshared;ORG=bluehost; X-Source-Cap: ZWx5bnJvYmk7ZWx5bnJvYmk7Ym94NTM3OS5ibHVlaG9zdC5jb20= X-Local-Domain: yes X-CMAE-Envelope: MS4xfFO2vwwGJ7vbS0aVxbm4xeT9H3lthYQ1oDipF+kxx7odVptxCecLPW5/A6KvJpJa53xbXywz7YKuVy0g0FybcTpi3NUv0TX0cvyIjONpWMJ65cSPDr+p H+Vjj/dCIezYpG1URGmQYL605SQiS2hvxIn4onjBGhhFqnv22gmYBqOD4ks625nud0rASksOtjkiUWq2zqAW19BjTl1HSmpW49w= X-Spam-Status: No, score=-3018.0 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,JMQ_SPF_NEUTRAL,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: This series changes gdb to do its initial DWARF indexing in the background. This process is mostly asynchronous with respect to the rest of gdb. That is, rather than pre-emptively waiting for scanning to complete, now gdb's main thread will only wait when some result of the scan is required. This drastically improves gdb's apparent startup time in the "normal" case where a user does "gdb some-executable" -- e.g., for starting gdb on itself, the time until the prompt returns goes from ~1.2 seconds to ~0.06 seconds on my machine. This approach works by hiding most of the work from the user. Waiting can still be needed; for example if one starts gdb and immediately sets a breakpoint -- however, because the indexer is reasonably fast, and human reaction times are slow, this series still manages to be fairly successful. My current belief is that doing any better than this will probably require a new debug format that isn't quite so cursed to read. I regression tested this on x86-64 Fedora 38. I've also built it with TSAN and tested that, though TSAN seems to introduce random timeouts into the testsuite when I use "make -j8 check". --- Changes in v3: - Rebase on top of the C++17 changes - Apply Tom de Vries' bug fix - Fix a "save gdb-index" bug found by Alexandra - Link to v2: https://inbox.sourceware.org/gdb-patches/20231112-t-bg-dwarf-reading-v2-0-70fb170012ba@tromey.com Changes in v2: - Updated to final style of BFD locking - Added a thread safety fix for DWO - Added a patch to pre-read DWZ sections - Added calls to bfd_thread_cleanup and bfd_cache_close - Link to v1: https://inbox.sourceware.org/gdb-patches/20231029173839.471514-1-tom@tromey.com --- Tom Tromey (18): Don't use objfile::intern in DWO code Pre-read DWZ section data Add a couple of bfd_cache_close calls Add thread-safety to gdb's BFD wrappers Refactor complaint thread-safety approach Add quick_symbol_functions::compute_main_name Add gdb::task_group Move cooked_index_storage to cooked-index.h Add "maint set dwarf synchronous" Change how cooked index waits for threads Do more DWARF reading in the background Simplify the public DWARF API Remove two quick_symbol_functions methods Change current_language to be a macro Lazy language setting Optimize lookup_minimal_symbol_text Avoid language-based lookups in startup path Back out some parallel_for_each features gdb/NEWS | 4 + gdb/breakpoint.c | 4 +- gdb/coffread.c | 6 +- gdb/complaints.c | 24 +- gdb/complaints.h | 37 +- gdb/defs.h | 2 +- gdb/doc/gdb.texinfo | 18 + gdb/dwarf2/cooked-index.c | 201 ++++--- gdb/dwarf2/cooked-index.h | 316 ++++++++-- gdb/dwarf2/dwz.c | 90 ++- gdb/dwarf2/dwz.h | 13 +- gdb/dwarf2/macro.c | 2 - gdb/dwarf2/public.h | 18 +- gdb/dwarf2/read.c | 729 +++++++++++++---------- gdb/dwarf2/read.h | 2 +- gdb/elfread.c | 6 +- gdb/gdb_bfd.c | 58 ++ gdb/gdb_bfd.h | 5 + gdb/gdbthread.h | 3 +- gdb/jit.c | 4 +- gdb/language.c | 51 +- gdb/language.h | 34 +- gdb/machoread.c | 11 +- gdb/main.c | 3 +- gdb/minsyms.c | 69 ++- gdb/objfile-flags.h | 4 - gdb/objfiles.h | 17 +- gdb/psymtab.c | 1 - gdb/quick-symbol.h | 17 +- gdb/symfile-debug.c | 77 +-- gdb/symfile.c | 22 +- gdb/symtab.c | 2 + gdb/testsuite/gdb.dwarf2/dw2-error.exp | 1 + gdb/testsuite/gdb.dwarf2/dw2-missing-cu-tag.exp | 2 + gdb/testsuite/gdb.dwarf2/dw2-stack-boundary.exp | 2 + gdb/testsuite/gdb.dwarf2/dw2-using-debug-str.exp | 2 + gdb/testsuite/gdb.dwarf2/dw2-zero-range.exp | 4 +- gdb/testsuite/gdb.dwarf2/fission-reread.exp | 5 +- gdb/testsuite/gdb.dwarf2/no-gnu-debuglink.exp | 1 + gdb/testsuite/gdb.dwarf2/struct-with-sig-2.exp | 10 +- gdb/thread.c | 8 +- gdb/top.c | 2 +- gdb/unittests/parallel-for-selftests.c | 47 -- gdb/xcoffread.c | 3 +- gdbsupport/Makefile.am | 1 + gdbsupport/Makefile.in | 6 +- gdbsupport/parallel-for.h | 234 +------- gdbsupport/task-group.cc | 97 +++ gdbsupport/task-group.h | 64 ++ 49 files changed, 1409 insertions(+), 930 deletions(-) --- base-commit: a54a99a6e5159e4406e172f074b3729b2135c28c change-id: 20231112-t-bg-dwarf-reading-00b65fa130b3 Best regards, -- Tom Tromey