From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 98088 invoked by alias); 30 Nov 2019 15:58:58 -0000 Mailing-List: contact elfutils-devel-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Post: List-Help: List-Subscribe: Sender: elfutils-devel-owner@sourceware.org Received: (qmail 98073 invoked by uid 89); 30 Nov 2019 15:58:58 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Checked: by ClamAV 0.100.3 on sourceware.org X-Virus-Found: No X-Spam-SWARE-Status: No, score=-15.7 required=5.0 tests=AWL,BAYES_00,GIT_PATCH_0,GIT_PATCH_1,GIT_PATCH_2,GIT_PATCH_3 autolearn=ham version=3.3.1 spammy=political, serve, deb, DEB X-Spam-Status: No, score=-15.7 required=5.0 tests=AWL,BAYES_00,GIT_PATCH_0,GIT_PATCH_1,GIT_PATCH_2,GIT_PATCH_3 autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on sourceware.org X-Spam-Level: X-HELO: us-smtp-delivery-1.mimecast.com Received: from us-smtp-2.mimecast.com (HELO us-smtp-delivery-1.mimecast.com) (207.211.31.81) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Sat, 30 Nov 2019 15:58:56 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1575129534; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=IaLRslBWMt2DxQFH0hppKeyFABcPZSR6gjpVLVdA4GA=; b=dg+zBvURioqElAnR/6QG9SSP4D64pteUh6SyglJHtZ1NL+TgqxneNEHF0oRCbNuwpcDQCb pnYULra+dcmSS7oORmCNoCf18M4EpvUBcFz426ISdSFEpAT9wyv2vWlSeR4ad0OhXijozX 3YIUiw7AcF7znbOtsylu5DRguoi0KJw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-301-GILnt07qOjetd6HooXVNEQ-1; Sat, 30 Nov 2019 10:58:53 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 44739DB69 for ; Sat, 30 Nov 2019 15:58:52 +0000 (UTC) Received: from redhat.com (ovpn-116-64.phx2.redhat.com [10.3.116.64]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1190519C6A for ; Sat, 30 Nov 2019 15:58:52 +0000 (UTC) Received: from fche by redhat.com with local (Exim 4.92) (envelope-from ) id 1ib590-00052e-QK for elfutils-devel@sourceware.org; Sat, 30 Nov 2019 10:58:50 -0500 Date: Sat, 30 Nov 2019 15:58:00 -0000 From: "Frank Ch. Eigler" To: elfutils-devel@sourceware.org Subject: RFC: deb/ddeb support for debuginfod Message-ID: <20191130155850.GA19319@redhat.com> MIME-Version: 1.0 User-Agent: Mutt/1.12.0 (2019-05-25) X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-MC-Unique: GILnt07qOjetd6HooXVNEQ-1 X-Mimecast-Spam-Score: 0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-IsSubscribed: yes X-SW-Source: 2019-q4/txt/msg00224.txt.bz2 Hi - I present the following quick hack to add .deb/.ddeb support to debuginfod. It works ... but my reservation is political. :-) It works by letting the RPM-scanning code paths recognize .deb and .ddeb files by their suffix, and run them through a different stream-extractor (dpkg-deb --fsys-tarfile instead of rpm2cpio). Otherwise, nothing needs to change! And that's kind of the problem. The program diagnostics, options, all mention 'rpm'. I'd like your opinion about whether the proposed man page's explanation is good enough -- or whether debian fans would be so gravely offended by the preservation of the 'rpm' term that we must hide it. commit bb6a05faf5905e55f3289c70388b339113f843dd Author: Frank Ch. Eigler Date: Sat Nov 30 10:46:44 2019 -0500 deb/ddeb support prototype diff --git a/debuginfod/debuginfod.cxx b/debuginfod/debuginfod.cxx index aa7ffcf..851d0bd 100644 --- a/debuginfod/debuginfod.cxx +++ b/debuginfod/debuginfod.cxx @@ -106,6 +106,17 @@ using namespace std; #endif =20 =20 +inline bool +string_endswith(const string& haystack, const string& needle) +{ + return (haystack.size() >=3D needle.size() && + haystack.substr(haystack.size()-needle.size()) =3D=3D needle); +} + +=20=20=20=20 + + + // Roll this identifier for every sqlite schema incompatiblity. #define BUILDIDS "buildids9" =20 @@ -323,7 +334,7 @@ static const struct argp_option options[] =3D { { NULL, 0, NULL, 0, "Scanners:", 1 }, { "scan-file-dir", 'F', NULL, 0, "Enable ELF/DWARF file scanning thread= s.", 0 }, - { "scan-rpm-dir", 'R', NULL, 0, "Enable RPM scanning threads.", 0 }, + { "scan-rpm-dir", 'R', NULL, 0, "Enable RPM/DEB scanning threads.", 0 }, // "source-oci-imageregistry" ...=20 =20 { NULL, 0, NULL, 0, "Options:", 2 }, @@ -851,7 +862,11 @@ handle_buildid_r_match (int64_t b_mtime, return 0; } =20 - string popen_cmd =3D string("rpm2cpio " + shell_escape(b_source0)); + string popen_cmd; + if (string_endswith(b_source0, "rpm")) + popen_cmd =3D string("rpm2cpio " + shell_escape(b_source0)); + else // .deb or .ddeb + popen_cmd =3D string("dpkg-deb --fsys-tarfile " + shell_escape(b_sourc= e0)); FILE* fp =3D popen (popen_cmd.c_str(), "r"); // "e" O_CLOEXEC? if (fp =3D=3D NULL) throw libc_exception (errno, string("popen ") + popen_cmd); @@ -863,9 +878,9 @@ handle_buildid_r_match (int64_t b_mtime, throw archive_exception("cannot create archive reader"); defer_dtor archive_closer (a, archive_read_free); =20 - rc =3D archive_read_support_format_cpio(a); + rc =3D archive_read_support_format_all(a); if (rc !=3D ARCHIVE_OK) - throw archive_exception(a, "cannot select cpio format"); + throw archive_exception(a, "cannot select all format"); rc =3D archive_read_support_filter_all(a); if (rc !=3D ARCHIVE_OK) throw archive_exception(a, "cannot select all filters"); @@ -1867,7 +1882,11 @@ rpm_classify (const string& rps, sqlite_ps& ps_upser= t_buildids, sqlite_ps& ps_up unsigned& fts_executable, unsigned& fts_debuginfo, unsigned&= fts_sref, unsigned& fts_sdef, bool& fts_sref_complete_p) { - string popen_cmd =3D string("rpm2cpio " + shell_escape(rps)); + string popen_cmd; + if (string_endswith(rps, "rpm")) + popen_cmd =3D string("rpm2cpio " + shell_escape(rps)); + else // .deb or .ddeb + popen_cmd =3D string("dpkg-deb --fsys-tarfile " + shell_escape(rps)); FILE* fp =3D popen (popen_cmd.c_str(), "r"); // "e" O_CLOEXEC? if (fp =3D=3D NULL) throw libc_exception (errno, string("popen ") + popen_cmd); @@ -1879,9 +1898,9 @@ rpm_classify (const string& rps, sqlite_ps& ps_upsert= _buildids, sqlite_ps& ps_up throw archive_exception("cannot create archive reader"); defer_dtor archive_closer (a, archive_read_free); =20 - int rc =3D archive_read_support_format_cpio(a); + int rc =3D archive_read_support_format_all(a); if (rc !=3D ARCHIVE_OK) - throw archive_exception(a, "cannot select cpio format"); + throw archive_exception(a, "cannot select all formats"); rc =3D archive_read_support_filter_all(a); if (rc !=3D ARCHIVE_OK) throw archive_exception(a, "cannot select all filters"); @@ -2118,9 +2137,12 @@ scan_source_rpm_path (const string& dir) { // heuristic: reject if file name does not end with ".rpm" // (alternative: try opening with librpm etc., caching) - string suffix =3D ".rpm"; - if (rps.size() < suffix.size() || - rps.substr(rps.size()-suffix.size()) !=3D suffix) + auto suffixes =3D vector {".rpm", ".deb", ".ddeb"}; + bool any =3D false; + for (auto&& suffix : suffixes) + if (string_endswith (rps, suffix)) + any =3D true; + if (! any) continue; fts_rpm ++; =20 diff --git a/doc/debuginfod.8 b/doc/debuginfod.8 index 210550e..5ad3389 100644 --- a/doc/debuginfod.8 +++ b/doc/debuginfod.8 @@ -56,14 +56,19 @@ or even use debuginfod itself: .ESAMPLE =20 If the \fB\-R\fP option is given each listed PATH creates a thread to -scan for ELF/DWARF/source files contained in matching RPMs under the -given physical directory. Duplicate directories are ignored. You may -use a file name for a PATH, but source code indexing may be -incomplete; prefer using a directory that contains normal RPMs -alongside debuginfo/debugsource RPMs. Because of complications such -as DWZ-compressed debuginfo, may require \fItwo\fP scan passes to -identify all source code. Source files for RPMs are only served -from other RPMs, so the caution for \-F does not apply. +scan for ELF/DWARF/source files contained in matching RPMs or DEBs or +DDEBs under the given physical directory. (The terms RPM and DEB and +DDEB are used interchangeably here and in debuginfod diagnostic +messages. The different file types are simply recognied by their +extensions.) Duplicate directories are ignored. You may use a file +name for a PATH, but source code indexing may be incomplete; prefer +using a directory that contains normal RPMs alongside +debuginfo/debugsource RPMs. Because of complications such as +DWZ-compressed debuginfo, may require \fItwo\fP scan passes to +identify all source code. Source files for RPMs are only served from +other RPMs, so the caution for \-F does not apply. Note that due to +Debian/Ubuntu packaging policies & mechanisms, debuginfod cannot resolve +source files for DEB/DDEB at all. =20 If no PATH is listed, or neither \-F nor \-R option is given, then \fBdebuginfod\fP will simply serve content that it scanned into its