[Bug debuginfod/25583] New: Use libarchive to extract packages?

[Bug debuginfod/25583] New: Use libarchive to extract packages?

[ross at burtonini dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=25583

            Bug ID: 25583
           Summary: Use libarchive to extract packages?
           Product: elfutils
           Version: unspecified
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: debuginfod
          Assignee: unassigned at sourceware dot org
          Reporter: ross at burtonini dot com
                CC: elfutils-devel at sourceware dot org
  Target Milestone: ---

Using rpm2cpio to extract an RPM isn't ideal because the machine serving the
RPMs may not actually have that tool on.

However, libarchive which is already a dependency can extract RPMs directly as
demonstrated with the libarchive example tool bsdtar:

$ bsdtar -tvf
/home/ross/Yocto/openbmc/build/tmp/deploy/rpm/noarch/os-release-1.0-r0.noarch.rpm
drwxr-xr-x  1 0      0           0 Nov  8 16:41 ./etc
lrwxrwxrwx  1 0      0          21 Nov  8 16:41 ./etc/os-release ->
../usr/lib/os-release
drwxr-xr-x  1 0      0           0 Nov  8 16:41 ./usr
drwxr-xr-x  1 0      0           0 Nov  8 16:41 ./usr/lib
-rw-r--r--  1 0      0         287 Nov  8 16:41 ./usr/lib/os-release

Ditto for debs, dpkg-deb may not be present but libarchive can unpack the outer
and give you access to the internals:

$ bsdtar  -tvf /var/cache/apt/archives/gdb_8.2.1-2+b1_amd64.deb
-rw-r--r--  0 0      0           4 Aug 28 16:18 debian-binary
-rw-r--r--  0 0      0        3024 Aug 28 16:18 control.tar.xz
-rw-r--r--  0 0      0     3131068 Aug 28 16:18 data.tar.xz

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug debuginfod/25583] Use libarchive to extract packages?

[fche at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=25583

Frank Ch. Eigler <fche at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |WAITING
   Last reconfirmed|                            |2020-02-22
                 CC|                            |fche at redhat dot com
     Ever confirmed|0                           |1

--- Comment #1 from Frank Ch. Eigler <fche at redhat dot com> ---
Interesting idea.  OTOH, rpm2cpio and dpkg binaries are not too hard to come
by.

One can experiment with the former already with git-master debuginfod with the
"-Z .rpm" option instead of "-R".  Performance seems to be roughly the same.

The latter is less compelling in that it'd require hard-coding the inner
data.tar.xz name and its processing ... meh.

Maybe the status quo is good enough?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug debuginfod/25583] Use libarchive to extract packages?

[mark at klomp dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=25583

Mark Wielaard <mark at klomp dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |mark at klomp dot org

--- Comment #2 from Mark Wielaard <mark at klomp dot org> ---
(In reply to Frank Ch. Eigler from comment #1)
> Interesting idea.  OTOH, rpm2cpio and dpkg binaries are not too hard to come
> by.
> 
> One can experiment with the former already with git-master debuginfod with
> the
> "-Z .rpm" option instead of "-R".  Performance seems to be roughly the same.

If this works as well, then I would simply switch -R to:

diff --git a/debuginfod/debuginfod.cxx b/debuginfod/debuginfod.cxx
index 0acd70e4..c68aafa3 100644
--- a/debuginfod/debuginfod.cxx
+++ b/debuginfod/debuginfod.cxx
@@ -423,7 +423,7 @@ parse_opt (int key, char *arg,
       break;
     case 'F': scan_files = true; break;
     case 'R':
-      scan_archives[".rpm"]="rpm2cpio";
+      scan_archives[".rpm"]="cat";
       break;
     case 'U':
       scan_archives[".deb"]="dpkg-deb --fsys-tarfile";

It seems to just pass the run-debuginfod-find.sh testcase.


> The latter is less compelling in that it'd require hard-coding the inner
> data.tar.xz name and its processing ... meh.
> 
> Maybe the status quo is good enough?

If we can get rid of some extra dependencies with the same performance and no
real downsides, except a couple of lines of extra code, I would go with this.
Of course, still needs someone to write those couple of extra lines of code.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug debuginfod/25583] Use libarchive to extract packages?

[fche at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=25583

Frank Ch. Eigler <fche at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |ASSIGNED

--- Comment #3 from Frank Ch. Eigler <fche at redhat dot com> ---
ok, confirmed that relatively old bsdtar (libarchive 3.1.2 rhel7) supports rpm
natively too.  Will switch that over as Mark suggests.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug debuginfod/25583] Use libarchive to extract packages?

[fche at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=25583

Frank Ch. Eigler <fche at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #4 from Frank Ch. Eigler <fche at redhat dot com> ---
pushed as obvious

diff --git a/debuginfod/debuginfod.cxx b/debuginfod/debuginfod.cxx
index be3868bb1e42..7c7e85eb6d14 100644
--- a/debuginfod/debuginfod.cxx
+++ b/debuginfod/debuginfod.cxx
@@ -426,7 +426,7 @@ parse_opt (int key, char *arg,
       break;
     case 'F': scan_files = true; break;
     case 'R':
-      scan_archives[".rpm"]="rpm2cpio";
+      scan_archives[".rpm"]="cat"; // libarchive groks rpm natively
       break;
     case 'U':
       scan_archives[".deb"]="dpkg-deb --fsys-tarfile";

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug debuginfod/25583] Use libarchive to extract .deb packages?

[mark at klomp dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=25583

Mark Wielaard <mark at klomp dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
         Resolution|FIXED                       |---
            Summary|Use libarchive to extract   |Use libarchive to extract
                   |packages?                   |.deb packages?

--- Comment #5 from Mark Wielaard <mark at klomp dot org> ---
Great we have .rpm done now.

Lets keep this open in case someone wants to do the .deb variant too.

BTW. It seems we should tweak elfutils.spec too.
rpm isn't needed anymore, but we should recommend/require dpkg?

diff --git a/config/elfutils.spec.in b/config/elfutils.spec.in
index e992812a..ef7fe31f 100644
--- a/config/elfutils.spec.in
+++ b/config/elfutils.spec.in
@@ -157,8 +157,9 @@ Requires(post):   systemd
 Requires(preun):  systemd
 Requires(postun): systemd
 Requires(pre): shadow-utils
-# For /usr/bin/cpio2rpm
-Requires: rpm
+# To extract .deb files with dpkg-deb --fsys-tarfile
+# Can be Recommends if rpm supports that
+Requires: dpkg

 %description debuginfod-client
 The elfutils-debuginfod-client package contains shared libraries

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug debuginfod/25583] Use libarchive to extract .deb packages?

[fche at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=25583

--- Comment #6 from Frank Ch. Eigler <fche at redhat dot com> ---
> -# For /usr/bin/cpio2rpm
> -Requires: rpm

Heh, that was always moot on an rpm based system (.spec file!).

> +# To extract .deb files with dpkg-deb --fsys-tarfile
> +# Can be Recommends if rpm supports that
> +Requires: dpkg

Sure, as long as one remembers to remove it from rhel distros that have dpkg
only in epel.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug debuginfod/25583] Use libarchive to extract .deb packages?

[fche at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=25583

--- Comment #7 from Frank Ch. Eigler <fche at redhat dot com> ---
here's the recipe for .deb / .ddeb:

   -Z '.ddeb=(bsdtar -O -x -f - data.tar.xz)<'

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug debuginfod/25583] Use libarchive to extract .deb packages?

[fche at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=25583

--- Comment #8 from Frank Ch. Eigler <fche at redhat dot com> ---
https://sourceware.org/pipermail/elfutils-devel/2020q1/002529.html

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug debuginfod/25583] Use libarchive to extract .deb packages?

[fche at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=25583

Frank Ch. Eigler <fche at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #9 from Frank Ch. Eigler <fche at redhat dot com> ---
pushed as debuginfod-internal & uncontroversial

-- 
You are receiving this mail because:
You are on the CC list for the bug.