From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by sourceware.org (Postfix) with ESMTPS id 22554385840E for ; Thu, 11 Nov 2021 12:48:29 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 22554385840E Received: by mail-wr1-x433.google.com with SMTP id b12so9638365wrh.4 for ; Thu, 11 Nov 2021 04:48:29 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:in-reply-to:references:date :message-id:mime-version; bh=vHkiL/8Mod3UMguO/V3yc5uuenTC0GD2SREnqYMwopA=; b=TeV2DxiqEj2BkS0HWT8K+LTng7ZpN706N3GHzCgUNIRfvqMqvGAH/GL4lO6ViC5/5M jkdZvjQ5ZEy86bft6XHxd35cOSgD85q6n0MB1MIfTx7DfzgVDMvRLnxYRrhmOZs9jW7q LrrrKajpAnSOfUrqpQMBRCLDELa0wr7qkiiRoa/d/V2iP49pTpbcbqpg1dNzUbqRuEHh P0vGYXj86q30Z0lr3Y3OpD+17Cxx6I4mm1BMqTeniCNyb3AT5IpG/8qIcmRkq0HXZ7PX YuQDU60pGMC84kX8FiYlBZ1MgmTCVj6qSfOJl8ZjtnCuwx5X/ARbfrHg3GS2us00D+uF ZnbQ== X-Gm-Message-State: AOAM5339hXKX/ddpbrcaeiIZjvh2/bUdvPh9TiQEalhctEvh815WSaJU fqtiGsuQEPMrLRprM3gKPY7L0G5w4BMS/BfrF3cJJE2kKhI1D3815NvcWeHGiJULM9YtliFxSHD zaz0kz27YtAtwOyn4U4BDudRKOuxJb4ZiZeinLEtBZaIygbZlVkiV/KYdhVCUaCxu+npXe3AD X-Google-Smtp-Source: ABdhPJzafi3Gj7anJVPlfidExLHnt8Gyis2oIfNvDZ/SeMwkWjV2bEwS/BE8X27b6XLJoEgs4JoIhw== X-Received: by 2002:a5d:548d:: with SMTP id h13mr8408667wrv.30.1636634908211; Thu, 11 Nov 2021 04:48:28 -0800 (PST) Received: from bucheron-thinkpad (cpc120850-nrwh12-2-0-cust139.4-4.cable.virginm.net. [82.32.180.140]) by smtp.gmail.com with ESMTPSA id f7sm8330984wmg.6.2021.11.11.04.48.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Nov 2021 04:48:27 -0800 (PST) From: Magne Hov To: "Frank Ch. Eigler" , elfutils-devel@sourceware.org Subject: Re: [patch] PR28339: debuginfod groom/scan concurrency fix In-Reply-To: <20210914152058.GG19490@redhat.com> References: <20210914152058.GG19490@redhat.com> Date: Thu, 11 Nov 2021 12:48:26 +0000 Message-ID: <5sfss2g8o5.fsf@undo.io> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Status: No, score=-5.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: elfutils-devel@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Elfutils-devel mailing list List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Nov 2021 12:48:31 -0000 Hi Frank, On Tue, Sep 14 2021, Frank Ch. Eigler via Elfutils-devel wrote: > commit ce695bedf073127883bbbaab528dd1f2b0e955f1 (HEAD -> master) > Author: Frank Ch. Eigler > Date: Tue Sep 14 08:15:23 2021 -0400 > > PR28339: debuginfod: fix groom/scan race condition on just-emptied queue > > debuginfod's scan and groom operations (thread_main_scanner, > thread_main_fts_source_paths) are intended to be mutually exclusive, > as a substitute for more complicated sql transaction batching. (This > is because scanning / grooming involves inserting or deleting data > from multiple related tables.) > > The workq class that governs this in debuginfod.cxx has a problem: if > the workq just becomes empty, its sole entry pulled by a scanner > thread in response to a wait_front(), an 'idler' groomer thread is > ALSO permitted to run, because there is no indication as to when the > scanner thread operation finishes, only when it starts. > > Extending the workq with a counter ("fronters") to track any active > scanning activity (even if the workq is empty) lets us block idlers > groomers a little longer. > > Signed-off-by: Frank Ch. Eigler Thanks a lot for this fix. We've had an issue in automated testing where `nuke orphan buildids` is executed after `rpm-buildid-intern` has run but *before* `rpm-de-insert` has run. The result is missing BUILDIDS for the archive, and rescans of the archive fail to re-add them because the archive is registered as already having been scanned. I think this fix will resovle this.