From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gnu.wildebeest.org (wildebeest.demon.nl [212.238.236.112]) by sourceware.org (Postfix) with ESMTPS id A2A8A3858402 for ; Tue, 14 Sep 2021 15:25:58 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org A2A8A3858402 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=klomp.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=klomp.org Received: from tarox.wildebeest.org (83-87-18-245.cable.dynamic.v4.ziggo.nl [83.87.18.245]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by gnu.wildebeest.org (Postfix) with ESMTPSA id 578F830002C0; Tue, 14 Sep 2021 17:25:57 +0200 (CEST) Received: by tarox.wildebeest.org (Postfix, from userid 1000) id 24A5C413CE02; Tue, 14 Sep 2021 17:25:56 +0200 (CEST) Message-ID: <3f939699f54ce5f4cfca942655b1d854f2b7baca.camel@klomp.org> Subject: Re: [patch] PR28339: debuginfod groom/scan concurrency fix From: Mark Wielaard To: "Frank Ch. Eigler" , elfutils-devel@sourceware.org Date: Tue, 14 Sep 2021 17:25:56 +0200 In-Reply-To: <20210914152058.GG19490@redhat.com> References: <20210914152058.GG19490@redhat.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Evolution 3.28.5 (3.28.5-10.el7) Mime-Version: 1.0 X-Spam-Status: No, score=-4.1 required=5.0 tests=BAYES_00, JMQ_SPF_NEUTRAL, KAM_DMARC_STATUS, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: elfutils-devel@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Elfutils-devel mailing list List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Sep 2021 15:26:00 -0000 Hi Frank, On Tue, 2021-09-14 at 11:20 -0400, Frank Ch. Eigler via Elfutils-devel wrot= e: > commit ce695bedf073127883bbbaab528dd1f2b0e955f1 (HEAD -> master) > Author: Frank Ch. Eigler > Date: Tue Sep 14 08:15:23 2021 -0400 >=20 > PR28339: debuginfod: fix groom/scan race condition on just-emptied qu= eue > =20 > debuginfod's scan and groom operations (thread_main_scanner, > thread_main_fts_source_paths) are intended to be mutually exclusive, > as a substitute for more complicated sql transaction batching. (This > is because scanning / grooming involves inserting or deleting data > from multiple related tables.) > =20 > The workq class that governs this in debuginfod.cxx has a problem: if > the workq just becomes empty, its sole entry pulled by a scanner > thread in response to a wait_front(), an 'idler' groomer thread is > ALSO permitted to run, because there is no indication as to when the > scanner thread operation finishes, only when it starts. > =20 > Extending the workq with a counter ("fronters") to track any active > scanning activity (even if the workq is empty) lets us block idlers > groomers a little longer. > =20 > Signed-off-by: Frank Ch. Eigler Thanks so much for finding (and fixing!) this. It survived 3850+ cycles on my local machine. So I am hopeful this will also work on the buildbots. Please push. Cheers, Mark