From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by sourceware.org (Postfix) with ESMTPS id 588BF3858C00 for ; Wed, 1 Nov 2023 10:46:54 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 588BF3858C00 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 588BF3858C00 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1698835617; cv=none; b=FI57gqGk/lafv0jU8bVTgq8JYA+8jTSC+7XddbCUaNTyiBDoFtNsNUB9qjBvXjqPiGqhK66T2cGzRl2TOCHw0qmwqHMbpcXKTT8zTMCN4+htFnhbg6b+U0t/65LlkziKw6tGmiFb5g8fFv3FpjArWhUpWcJZ2fY4/qhmyfEDOEw= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1698835617; c=relaxed/simple; bh=ZA6kCnEUC1UducLWtKSQfS6HwZgaAyw6v+dgaftC9eg=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=WP/db3nRzkcb5I2RaPzR/nFWIvQTwOaDfdv0wbHvD6JVQMFUwpUEVPcXZMG/HgfX9t1DfQiEoeOc3rACczSPWgV2hb8f4PWIiKYdfnuCsiSy6MDmMmmcTMx4nph2/rDvzJCJ7MhKygfY9bB9AW1zWu5pHEPTti/oC6e+uX90zHc= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698835614; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=B+TjzN8fBV/EFYXXems3FvuwvQA5+pHxVIjkkEifvTE=; b=fZWbHeqDTsKHDrc1wRQ0/O6xQDTClzCeEKHy72WqhBWy9IJwNpV9Un8MrjsKGY0cZC09HY O558onRLi3R1Rwgdzb+6sN5ZNBjVs+y1qCJOfSWR78EL1KFYASfVSDGKRSwFKPBT5tyfSc ZQ4KmyoxSHmHA5NKoKU1P569trqg/dI= Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-457-YkYyBlbmOiW-QrO7ZtUNHA-1; Wed, 01 Nov 2023 06:46:52 -0400 X-MC-Unique: YkYyBlbmOiW-QrO7ZtUNHA-1 Received: by mail-ed1-f71.google.com with SMTP id 4fb4d7f45d1cf-531373ea109so5187641a12.3 for ; Wed, 01 Nov 2023 03:46:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698835611; x=1699440411; h=mime-version:message-id:date:references:in-reply-to:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=B+TjzN8fBV/EFYXXems3FvuwvQA5+pHxVIjkkEifvTE=; b=NXiA0mgDSwIpEfK2uukvHyshMQp24jqaYRAgyXsnQ78A404pHeGTgfZOqinms4tAhy ZYy+gTvAQzkgksTuoTXp6aP7RbnomlBCH3uEbR0+63B+wKHZT+GlvTvoZdY/i2Q46W6Y 1kwPGHg2opEajKG4II04ck+5YBFF3rQKaAxq0Q+I5lRYUcCND/I86PzT1teUP4KmTHsq 339Nz2BqmKYkIWbVcf3qF+/mq7ZhshssuNnfBo6er1WZjbzGA6oopYWdETQPhAuBD/dt FtAhK/Hlb/KnbUCFtOgIq78GizoNOYY3RSrSJQhsu/yhgUHylGClHoI37m37ubfnveNf oYBw== X-Gm-Message-State: AOJu0YycLMLdVa1YIeqW6EpOu5uFug+YrfEw09HJYUKGhtLIis6IPzVO +OH3G2i+Nsu0hjXDlOmSfONfAJKdnSYvJp96ETkIYnsLjP2nP5ezl7VX3aBDqgINMpKPCbyzTGw 5eD/ZFwQxysUDAJ1c5/pKATEQ0PfNfQ== X-Received: by 2002:a50:d702:0:b0:542:d934:eb92 with SMTP id t2-20020a50d702000000b00542d934eb92mr8780575edi.17.1698835611215; Wed, 01 Nov 2023 03:46:51 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHkIhJfQxlMcjglod1rZOfQamqRC4GvS8QbdugaWUlNDU9W0h+NTQ4mVC/Kuulrg35MxmrjNA== X-Received: by 2002:a50:d702:0:b0:542:d934:eb92 with SMTP id t2-20020a50d702000000b00542d934eb92mr8780567edi.17.1698835610866; Wed, 01 Nov 2023 03:46:50 -0700 (PDT) Received: from localhost (105.226.159.143.dyn.plus.net. [143.159.226.105]) by smtp.gmail.com with ESMTPSA id cy24-20020a0564021c9800b005432f45bee9sm906092edb.19.2023.11.01.03.46.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Nov 2023 03:46:50 -0700 (PDT) From: Andrew Burgess To: Tom Tromey Cc: Tom Tromey , gdb-patches@sourceware.org Subject: Re: [PATCH 1/2] gdb: move all bfd_cache_close_all calls in gdb_bfd.c In-Reply-To: <87wmv2k5nb.fsf@tromey.com> References: <87o7gjlv6q.fsf@tromey.com> <87r0lc1kef.fsf@redhat.com> <87wmv2k5nb.fsf@tromey.com> Date: Wed, 01 Nov 2023 10:46:49 +0000 Message-ID: <87o7gdvjhi.fsf@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain X-Spam-Status: No, score=-4.9 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Tom Tromey writes: >>> However, there is one case where this will change: opening dwo files. >>> Now, the way the background reading is implemented, the worst case here >>> is some inefficiency: the main thread might close all the BFDs, and then >>> the background reader might immediately reopen one. > > Andrew> Right now I don't see bfd/cache.c as thread safe, so are you planning to > Andrew> make it thread safe? > > Yeah. I sent a series to the binutils list recently. Neat. It sounds like your background DWARF reader is further along than I'd thought. I'm trying to figure out what I should do to move this patch forward? I guess the existing bfd_cache_close_all() calls could cause problems for your background reader if the on-disk file changes between two reads from a BFD, and if GDB's main thread calls bfd_cache_close_all() in between those reads, e.g. one BFD section might indicate an offset into another section, and after the on-disk file changes the section offset is no longer valid... But you could just ignore the existing bfd_cache_close_all() calls, and rely on BFD to reopen the files as needed, and rely on GDB's error detection to handle any race conditions caused by the on-disk file changing... I'm happy to do whatever is needed to integrate this patch with your upcoming work, I'm just trying to figure out what work needs to be done: if you're currently ignoring bfd_cache_close_all() calls, and relying on BFD to reopen, and GDB's error detection, then I guess this patch is fine as is. But if you have some other solution for the bfd_cache_close_all() calls, then I guess this patch is stepping on your toes, in which case, I want to help combine these two work items if I can. > > Andrew> The other option I considered, is that you could add a mechanism to > Andrew> allow BFDs to be opened without going through the BFD cache. In this > Andrew> case, I think your thread safety issues go away, and also, you no longer > Andrew> care about my bfd_cache_close_all() calls. The BFD cache exists to > Andrew> handle systems that only allow a small number of open files (they quote > Andrew> 20 as being a limit on some systems), so avoiding the cache might run > Andrew> into an open FD limit issue ... but I'm not sure if this low limit > Andrew> problem is really a thing on systems that are going to be running > Andrew> multiple threads? > > We could definitely do this for the files the DWARF reader opens in the > worker threads. > > Not sure if I will do this or not though. Adding an explicit > bfd_cache_close call would also take care of the problem, and I think > the reader only needs the file open from the call to open_dwo_file (in > open_and_init_dwo_file) through mapping the sections -- and they could > be mapped more eagerly in open_and_init_dwo_file. It sounds like you're already heading towards making the bfd cache thread safe, so I don't think these ideas would be needed. Thanks, Andrew > > Andrew> I'd really prefer to avoid relying on folk to remember that they need to > Andrew> call bfd_cache_close_all() in certain places. I think its far too easy > Andrew> to forget, and these are bugs that are only going to crop up once in a > Andrew> while .... A user is running a particular Python extension, and performs > Andrew> a particular action, just at the same time as they recompile their test > Andrew> binary ... and suddenly GDB "misses" that the test executable changed. > Andrew> Bugs like this can be a nightmare to track down. > > Yeah. > > Tom