From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp-out1.suse.de (smtp-out1.suse.de [IPv6:2001:67c:2178:6::1c]) by sourceware.org (Postfix) with ESMTPS id 021F63858C74 for ; Mon, 15 Aug 2022 08:34:11 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 021F63858C74 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: sourceware.org; spf=fail smtp.mailfrom=suse.cz Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id D33FB33726; Mon, 15 Aug 2022 08:34:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1660552449; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=cAQapxwbHAwdJnoC07DwvKeudQyCS/Rq3KtlofLBCBo=; b=xRpm3dLVJXyY+DlcG3iCYdfKX5tZnDmmLqNP0n07VMFi86D7jp6PP8/9O9WFOqCTmn3BHO +D4tkrsfNZ6Tjmwa1+yROfTEtHHI8/6DkjU0fgDxEJDADHuml36rz1xLzvZzkHPDmSlAuh yYhvYvbsSSDtTxNCgaXNPMgJiJfReqc= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1660552449; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=cAQapxwbHAwdJnoC07DwvKeudQyCS/Rq3KtlofLBCBo=; b=zUjY4Iu8rpGkMok0a3Wcrvm5XWE89NuedsfaYKDY0Aed9uTi1i7Bb9b3lmA8m9TTr3/OZs fkZ1wjYAPirupODQ== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id AD4CD13A93; Mon, 15 Aug 2022 08:34:09 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id IOJsKAEF+mIdQQAAMHmgww (envelope-from ); Mon, 15 Aug 2022 08:34:09 +0000 Message-ID: <5c4330bc-7af2-95a3-2e9e-def65d8692a0@suse.cz> Date: Mon, 15 Aug 2022 10:34:09 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.1.2 From: =?UTF-8?Q?Martin_Li=c5=a1ka?= Subject: [PATCH] debuginfod: Fix concurrent request leading to a same .rpm file [PR29474] To: elfutils-devel@sourceware.org Content-Language: en-US Cc: "Frank Ch. Eigler" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-11.9 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, SPF_HELO_NONE, SPF_SOFTFAIL, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: elfutils-devel@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Elfutils-devel mailing list List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Aug 2022 08:34:12 -0000 As explained in detail in the PR, the problem happens when multiple requests lead to a single RPM and one of the threads does prefetching. In that case, the requested file is added to fdcache and thus skip as interned. --- debuginfod/debuginfod.cxx | 69 ++++++++++++++++++++++++--------------- 1 file changed, 43 insertions(+), 26 deletions(-) diff --git a/debuginfod/debuginfod.cxx b/debuginfod/debuginfod.cxx index a089d0bd..e69a4bbc 100644 --- a/debuginfod/debuginfod.cxx +++ b/debuginfod/debuginfod.cxx @@ -1610,39 +1610,23 @@ string canonicalized_archive_entry_pathname(struct archive_entry *e) return string("/")+fn; } - - +// Try getting build_r match from fdcache. static struct MHD_Response* -handle_buildid_r_match (bool internal_req_p, - int64_t b_mtime, - const string& b_source0, - const string& b_source1, - int *result_fd) +try_buildid_r_match_from_cache (const string& b_source0, const string& b_source1, int *result_fd) { - struct stat fs; - int rc = stat (b_source0.c_str(), &fs); - if (rc != 0) - throw libc_exception (errno, string("stat ") + b_source0); - - if ((int64_t) fs.st_mtime != b_mtime) - { - if (verbose) - obatched(clog) << "mtime mismatch for " << b_source0 << endl; - return 0; - } - // check for a match in the fdcache first + struct stat fs; int fd = fdcache.lookup(b_source0, b_source1); - while (fd >= 0) // got one!; NB: this is really an if() with a possible branch out to the end + if (fd >= 0) // got one!; NB: this is really an if() with a possible branch out to the end { - rc = fstat(fd, &fs); + int rc = fstat(fd, &fs); if (rc < 0) // disappeared? { if (verbose) obatched(clog) << "cannot fstat fdcache " << b_source0 << endl; close(fd); fdcache.clear(b_source0, b_source1); - break; // branch out of if "loop", to try new libarchive fetch attempt + return 0; } struct MHD_Response* r = MHD_create_response_from_fd (fs.st_size, fd); @@ -1651,7 +1635,7 @@ handle_buildid_r_match (bool internal_req_p, if (verbose) obatched(clog) << "cannot create fd-response for " << b_source0 << endl; close(fd); - break; // branch out of if "loop", to try new libarchive fetch attempt + return 0; } inc_metric ("http_responses_total","result","archive fdcache"); @@ -1668,9 +1652,34 @@ handle_buildid_r_match (bool internal_req_p, if (result_fd) *result_fd = fd; return r; - // NB: see, we never go around the 'loop' more than once } + return NULL; +} + +static struct MHD_Response* +handle_buildid_r_match (bool internal_req_p, + int64_t b_mtime, + const string& b_source0, + const string& b_source1, + int *result_fd) +{ + struct stat fs; + int rc = stat (b_source0.c_str(), &fs); + if (rc != 0) + throw libc_exception (errno, string("stat ") + b_source0); + + if ((int64_t) fs.st_mtime != b_mtime) + { + if (verbose) + obatched(clog) << "mtime mismatch for " << b_source0 << endl; + return 0; + } + + struct MHD_Response *r = try_buildid_r_match_from_cache (b_source0, b_source1, result_fd); + if (r != 0) + return r; + // no match ... grumble, must process the archive string archive_decoder = "/dev/null"; string archive_extension = ""; @@ -1721,7 +1730,7 @@ handle_buildid_r_match (bool internal_req_p, // 2) extract the matching entry name (set r = result) // 3) extract some number of prefetched entries (just into fdcache) // 4) abort any further processing - struct MHD_Response* r = 0; // will set in stage 2 + r = 0; // will set in stage 2 unsigned prefetch_count = internal_req_p ? 0 : fdcache_prefetch; // will decrement in stage 3 @@ -1743,7 +1752,15 @@ handle_buildid_r_match (bool internal_req_p, continue; if (fdcache.probe (b_source0, fn)) // skip if already interned + { + if (r == 0) + { + r = try_buildid_r_match_from_cache (b_source0, b_source1, result_fd); + if (r != NULL) + return r; + } continue; + } // extract this file to a temporary file char* tmppath = NULL; @@ -1751,7 +1768,7 @@ handle_buildid_r_match (bool internal_req_p, if (rc < 0) throw libc_exception (ENOMEM, "cannot allocate tmppath"); defer_dtor tmmpath_freer (tmppath, free); - fd = mkstemp (tmppath); + int fd = mkstemp (tmppath); if (fd < 0) throw libc_exception (errno, "cannot create temporary file"); // NB: don't unlink (tmppath), as fdcache will take charge of it. -- 2.37.1