From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from elastic.org (elastic.org [96.126.110.187]) by sourceware.org (Postfix) with ESMTPS id F23A43858D20 for ; Thu, 13 Apr 2023 17:24:46 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org F23A43858D20 Authentication-Results: sourceware.org; dmarc=pass (p=quarantine dis=none) header.from=elastic.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=elastic.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=elastic.org ; s=default2; h=Content-Type:MIME-Version:Message-ID:Subject:To:From:Date: Sender:Reply-To:Cc:Content-Transfer-Encoding:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: In-Reply-To:References:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=pVxIIlpDPhITvnnTLM3tEYs+PMZjRNFzO+66vXuk/4o=; b=LFW0XNNy0eWs2A2sTZLI0zdM0o vLJj01KrItVzaHC1wTFz7zNjzoTyQIhJgsIe/zBupaUeBL02cmcF1jN9wcGWbw7tWusO4+Osa8tqq 6XOiK4gUq0JAJmga8mT0OSSeb7d9ETx1hfPvMN2lM79pL0X5NdYpHkoF/pmc/fAmNS0bAGVM9Hxpe 3i9MrMKcV2kxU2Xj5EdxXrrqNfiwplQN/l420+KXigeBEGwPYCqcX6yFssSJOwuMagqHxleaUxAFr M3HHlM/+Id6HajRr2iB4PsdcjE3swL8yVZIqkb2DYU5l3VZOVH9AJmqyv018jfswv1GuXmGGbkktI vRCL75gg==; Received: from vpn-home.elastic.org ([10.0.0.2] helo=elastic.org) by elastic.org with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1pn0gf-0004t2-39 for elfutils-devel@sourceware.org; Thu, 13 Apr 2023 17:24:46 +0000 Received: from very.elastic.org ([192.168.1.1]) by elastic.org with esmtp (Exim 4.96) (envelope-from ) id 1pn0gf-000Tv3-1v for elfutils-devel@sourceware.org; Thu, 13 Apr 2023 13:24:45 -0400 Received: from fche by very.elastic.org with local (Exim 4.96) (envelope-from ) id 1pn0gf-003YmF-1k for elfutils-devel@sourceware.org; Thu, 13 Apr 2023 13:24:45 -0400 Date: Thu, 13 Apr 2023 13:24:45 -0400 From: "Frank Ch. Eigler" To: elfutils-devel@sourceware.org Subject: [patch] [obv] PR30348: debuginfod: retry partial archive scans Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Sender-Verification: "" X-Spam-Status: No, score=-107.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,GIT_PATCH_0,SPF_HELO_PASS,SPF_PASS,TXREP,USER_IN_WELCOMELIST,USER_IN_WHITELIST autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Planning to push this shortly: Author: Frank Ch. Eigler Date: Thu Apr 13 13:11:56 2023 -0400 PR30348: debuginfod: retry partial archive scans On some public debuginfod servers, it was observed that errors may occur during individual archive scanning operations. That's fine, but previous code still went ahead and marked the archive "done" by inserting a record into the *_file_mtime_scanned table. New code ensures that exceptions propagate for these cases, and an archive that encountered an error while scanning will be retried later. Signed-off-by: Frank Ch. Eigler diff --git a/debuginfod/ChangeLog b/debuginfod/ChangeLog index f13c28d5c6f7..070dad03fba1 100644 --- a/debuginfod/ChangeLog +++ b/debuginfod/ChangeLog @@ -1,3 +1,8 @@ +2023-04-13 Frank Ch. Eigler + + * debuginfod.cxx (archive_classify, scan_archive_file): Catch and + propagate exceptions during archive scans. + 2023-03-30 Jan Alexander Steffens (heftig) * debuginfod-client.c (update_atime): New function. diff --git a/debuginfod/debuginfod.cxx b/debuginfod/debuginfod.cxx index 5ef6cc32189b..e981d1372233 100644 --- a/debuginfod/debuginfod.cxx +++ b/debuginfod/debuginfod.cxx @@ -3268,6 +3268,7 @@ archive_classify (const string& rps, string& archive_extension, if (verbose > 3) obatched(clog) << "libarchive scanning " << rps << endl; + bool any_exceptions = false; while(1) // parse archive entries { if (interrupted) @@ -3405,8 +3406,17 @@ archive_classify (const string& rps, string& archive_extension, catch (const reportable_exception& e) { e.report(clog); + any_exceptions = true; + // NB: but we allow the libarchive iteration to continue, in + // case we can still gather some useful information. That + // would allow some webapi queries to work, until later when + // this archive is rescanned. (Its vitals won't go into the + // _file_mtime_scanned table until after a successful scan.) } } + + if (any_exceptions) + throw reportable_exception("exceptions encountered during archive scan"); } @@ -3453,6 +3463,7 @@ scan_archive_file (const string& rps, const stat_t& st, // extract the archive contents unsigned my_fts_executable = 0, my_fts_debuginfo = 0, my_fts_sref = 0, my_fts_sdef = 0; bool my_fts_sref_complete_p = true; + bool any_exceptions = false; try { string archive_extension; @@ -3475,6 +3486,7 @@ scan_archive_file (const string& rps, const stat_t& st, catch (const reportable_exception& e) { e.report(clog); + any_exceptions = true; } if (verbose > 2) @@ -3484,6 +3496,7 @@ scan_archive_file (const string& rps, const stat_t& st, << " debuginfos=" << my_fts_debuginfo << " srefs=" << my_fts_sref << " sdefs=" << my_fts_sdef + << " exceptions=" << any_exceptions << endl; fts_executable += my_fts_executable; @@ -3491,6 +3504,9 @@ scan_archive_file (const string& rps, const stat_t& st, fts_sref += my_fts_sref; fts_sdef += my_fts_sdef; + if (any_exceptions) + throw reportable_exception("exceptions encountered during archive scan"); + if (my_fts_sref_complete_p) // leave incomplete? ps_scan_done .reset()