[Bug default/20299] New: abipkgdiff reports "cpio: Malformed number ..." when unpacking RPMs

[Bug default/20299] New: abipkgdiff reports "cpio: Malformed number ..." when unpacking RPMs

[joestringernz+sourceware at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=20299

            Bug ID: 20299
           Summary: abipkgdiff reports "cpio: Malformed number ..." when
                    unpacking RPMs
           Product: libabigail
           Version: unspecified
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: default
          Assignee: dodji at redhat dot com
          Reporter: joestringernz+sourceware at gmail dot com
                CC: libabigail at sourceware dot org
  Target Milestone: ---

Hi all, neat project!

I use ArchLinux for my main development environment, so I tried installing and
using libabigail from git (800ef86bcc20). With debian packages, abipkgdiff
worked pretty much out of the box and the summary looks straightforward to
understand.

Unfortunately when I tried to pass RPM packages to abipkgdiff, I see a large
number of complaints like the following from cpio 2.12-3 (used from rpmextract
1.0-7).

cpio: Malformed number <9c><90>$Í}ý^]Ò                                          
cpio: Malformed number <90>$Í}ý^]Òc                                             
cpio: Malformed number $Í}ý^]ÒcÚ

When I run "make check" in the source tree, I get failures for runtestdiffpkg
and runtestdefaultsupprs.py. Looking in the tests/output/test-diff-pkg
directory, none of the rpm reports have any content. I guess this is because
the test captures stdout to these files, but only stderr gets output because of
the cpio error I'm seeing above.

I've attached the error log. I know there are various environmental differences
on ArchLinux, eg python3 by default, but I'm not sure where to start to
identify what might be causing this.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

Re: [Bug default/20299] New: abipkgdiff reports "cpio: Malformed number ..." when unpacking RPMs

[Dodji Seketeli]

Hello Joe,

> Unfortunately when I tried to pass RPM packages to abipkgdiff, I see a large
> number of complaints like the following from cpio 2.12-3 (used from rpmextract
> 1.0-7).

On Fedora/CentOS, 'make check' works fine with cpio 2.12-3.  What
abipkgdiff does is rpm2cpio <package> | cpio -dium --quiet'.  So maybe
what rpm2cpio emits on your system has a problem?  There must be
something in your environment that is different from what we see on
Fedora, and that is impacting the unpacking.  We just need to figure
what, I guess.

So to see what the tool is doing, you can run abipkgdiff with the option
--verbose and attach the result here, so that we can learn more.

Sadly I don't have an Arch system at hand to test with right now.  So
I'd need your help with this.

> cpio: Malformed number <9c><90>$Í}ý^]Ò                                          
> cpio: Malformed number <90>$Í}ý^]Òc                                             
> cpio: Malformed number $Í}ý^]ÒcÚ

I'd need to have more context than this.

> When I run "make check" in the source tree, I get failures for runtestdiffpkg
> and runtestdefaultsupprs.py. Looking in the tests/output/test-diff-pkg
> directory, none of the rpm reports have any content. I guess this is because
> the test captures stdout to these files, but only stderr gets output because of
> the cpio error I'm seeing above.

I think you should look at the content of the file
<build-directory>/tests/test-suite.log.  That one should contain more
information, hopefully.  At least from there you'd see what invidual
abipkgdiff command you need to re-run with the --verbose option to learn
more about what went wrong.

> I've attached the error log. I know there are various environmental differences
> on ArchLinux, eg python3 by default, but I'm not sure where to start to
> identify what might be causing this.

Sorry, I don't see file attached to the bug.

Thank you for taking the time to file this problem report and I look
forward to us sorting this issue out together.

-- 
		Dodji

[Bug default/20299] abipkgdiff reports "cpio: Malformed number ..." when unpacking RPMs

[dodji at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=20299

--- Comment #5 from dodji at redhat dot com ---
"joe at wand dot net.nz" <sourceware-bugzilla@sourceware.org> writes:

> Arch's "rpmextract.sh" script does this:
> rpm2cpio "$1" | bsdtar -xf -

I see.

> On Archlinux, bsdtar is part of the standard platform. So, if I hardcode the
> abipkgdiff to use this instead of rp2mcpio | cpio, the test passes.

OK, good to know.

>> Also, why are you talking about RPM on ArchLinux at all?  I thought that
>> th package format on ArchLinux is tar.xz as see on this mirror at
>> http://delta.archlinux.fr/core/os/i686/ ?
>> 
>> Can you please enlighten me here?
>
> This is maybe more of a corner case, but I run ArchLinux as my development
> machine but commonly use a build system to compile both RPM and DEB packages.
> When I run the debian package diff it worked fine, so I was curious to see if I
> can also get away with doing RPM package diff even though my host platform
> isn't actually redhat.

Well, my initial goal was to enable uses cases like yours, precisely.  So
you do not have to be on a Red Hat based distribution to be able to compare
the ABIs of RPMs.  You shouldn't be obliged to be on a Debian based
distribution either to be able to compare .deb files.

I didn't know that there would be systems on which the command

  rpm2cpio <package> | cpio -idmu

would not work.

>> > I guess that if on redhat distributions the typical rpm2cpio does decompression
>> > by default then this is something I can take up with the rpmextract maintainer
>> > in ArchLinux. Worst case the archlinux abigail PKGBUILD could patch the
>> > abipkgdiff source to add a decompression step.
>> 
>> well, if rpm2cpio <package> | cpio -idmv could be made to work, I'd be
>> glad, yes.
>
> I think this is the right solution - it looks like the rpm2cpio utility on
> Archlinux is just a ~25 line script which does not cover many cases (for
> example, decompression).

Right.

But in the mean time, maybe libabigail's build system could detect the
presence of bsdtar.  So that when you are compiling libabigail, if
bsdtar is present, then abipkgdiff.cc would use it to extract the
content of the (possibly compressed) cpio archive emitted by rpm2cpio.

If bsdtar is not present at build time, then abipkgdiff.cc would use
cpio, just like it does today.

Would that work for you?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug default/20299] abipkgdiff reports "cpio: Malformed number ..." when unpacking RPMs

[dodji at seketeli dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=20299

--- Comment #1 from dodji at seketeli dot org ---
Hello Joe,

> Unfortunately when I tried to pass RPM packages to abipkgdiff, I see a large
> number of complaints like the following from cpio 2.12-3 (used from rpmextract
> 1.0-7).

On Fedora/CentOS, 'make check' works fine with cpio 2.12-3.  What
abipkgdiff does is rpm2cpio <package> | cpio -dium --quiet'.  So maybe
what rpm2cpio emits on your system has a problem?  There must be
something in your environment that is different from what we see on
Fedora, and that is impacting the unpacking.  We just need to figure
what, I guess.

So to see what the tool is doing, you can run abipkgdiff with the option
--verbose and attach the result here, so that we can learn more.

Sadly I don't have an Arch system at hand to test with right now.  So
I'd need your help with this.

> cpio: Malformed number <9c><90>$Í}ý^]Ò                                          
> cpio: Malformed number <90>$Í}ý^]Òc                                             
> cpio: Malformed number $Í}ý^]ÒcÚ

I'd need to have more context than this.

> When I run "make check" in the source tree, I get failures for runtestdiffpkg
> and runtestdefaultsupprs.py. Looking in the tests/output/test-diff-pkg
> directory, none of the rpm reports have any content. I guess this is because
> the test captures stdout to these files, but only stderr gets output because of
> the cpio error I'm seeing above.

I think you should look at the content of the file
<build-directory>/tests/test-suite.log.  That one should contain more
information, hopefully.  At least from there you'd see what invidual
abipkgdiff command you need to re-run with the --verbose option to learn
more about what went wrong.

> I've attached the error log. I know there are various environmental differences
> on ArchLinux, eg python3 by default, but I'm not sure where to start to
> identify what might be causing this.

Sorry, I don't see file attached to the bug.

Thank you for taking the time to file this problem report and I look
forward to us sorting this issue out together.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug default/20299] abipkgdiff reports "cpio: Malformed number ..." when unpacking RPMs

[joe at wand dot net.nz]

https://sourceware.org/bugzilla/show_bug.cgi?id=20299

--- Comment #4 from Joe Stringer <joe at wand dot net.nz> ---
(In reply to dodji from comment #3)
> Thank you Joe for following up on this.
> 
> "joe at wand dot net.nz" <sourceware-bugzilla@sourceware.org> a écrit:
> 
> > I think that the problem is to do with rpm compression; on ArchLinux, rpm2cpio
> > that comes with 'rpmextract' package doesn't decompress the rpm and cpio
> > doesn't autodetect compression and decompress, so cpio can't understand the
> > input it is given.
> 
> So, what is the common way on ArchLinux to extract the content of a
> package?
> 
> On Red Hat distributions, it's rpm2cpio <package> | cpio -idmv, or
> something similar.
>
> I'd be curious to know what the usual way is on ArchLinux, so that we
> can teach abipkgdiff to just do that.

Arch's "rpmextract.sh" script does this:
rpm2cpio "$1" | bsdtar -xf -

On Archlinux, bsdtar is part of the standard platform. So, if I hardcode the
abipkgdiff to use this instead of rp2mcpio | cpio, the test passes.

> Also, why are you talking about RPM on ArchLinux at all?  I thought that
> th package format on ArchLinux is tar.xz as see on this mirror at
> http://delta.archlinux.fr/core/os/i686/ ?
> 
> Can you please enlighten me here?

This is maybe more of a corner case, but I run ArchLinux as my development
machine but commonly use a build system to compile both RPM and DEB packages.
When I run the debian package diff it worked fine, so I was curious to see if I
can also get away with doing RPM package diff even though my host platform
isn't actually redhat.

> > I guess that if on redhat distributions the typical rpm2cpio does decompression
> > by default then this is something I can take up with the rpmextract maintainer
> > in ArchLinux. Worst case the archlinux abigail PKGBUILD could patch the
> > abipkgdiff source to add a decompression step.
> 
> well, if rpm2cpio <package> | cpio -idmv could be made to work, I'd be
> glad, yes.

I think this is the right solution - it looks like the rpm2cpio utility on
Archlinux is just a ~25 line script which does not cover many cases (for
example, decompression).

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug default/20299] abipkgdiff reports "cpio: Malformed number ..." when unpacking RPMs

[joe at wand dot net.nz]

https://sourceware.org/bugzilla/show_bug.cgi?id=20299

--- Comment #2 from Joe Stringer <joe at wand dot net.nz> ---
Created attachment 9372
  --> https://sourceware.org/bugzilla/attachment.cgi?id=9372&action=edit
test-suite.log

Sorry, I didn't realise the 10MB limit on attachments but the test-suite.log
was 33MB due to the "malformed number" lines. I've trimmed those pieces and
attached it. Unfortunately it doesn't provide much more context.

When I use --verbose, I get lines something like this at the start:

abipkgdiff: Extracting package <path> to <tmpdir> FAILED
abipkgdiff: command test -d
/home/joe/.cache/libabigail/abipkgdiff-tmp-dir-H6Dtfz/package1 && rm -rf
/home/joe/.cache/libabigail/abipkgdiff-tmp-dir-H6Dtfz/package1 FAILED 

Then the "malformed number" logs begin.

I think that the problem is to do with rpm compression; on ArchLinux, rpm2cpio
that comes with 'rpmextract' package doesn't decompress the rpm and cpio
doesn't autodetect compression and decompress, so cpio can't understand the
input it is given.

I guess that if on redhat distributions the typical rpm2cpio does decompression
by default then this is something I can take up with the rpmextract maintainer
in ArchLinux. Worst case the archlinux abigail PKGBUILD could patch the
abipkgdiff source to add a decompression step.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

Re: [Bug default/20299] abipkgdiff reports "cpio: Malformed number ..." when unpacking RPMs

[Dodji Seketeli]

Thank you Joe for following up on this.

"joe at wand dot net.nz" <sourceware-bugzilla@sourceware.org> a écrit:

> I think that the problem is to do with rpm compression; on ArchLinux, rpm2cpio
> that comes with 'rpmextract' package doesn't decompress the rpm and cpio
> doesn't autodetect compression and decompress, so cpio can't understand the
> input it is given.

So, what is the common way on ArchLinux to extract the content of a
package?

On Red Hat distributions, it's rpm2cpio <package> | cpio -idmv, or
something similar.

I'd be curious to know what the usual way is on ArchLinux, so that we
can teach abipkgdiff to just do that.

Also, why are you talking about RPM on ArchLinux at all?  I thought that
th package format on ArchLinux is tar.xz as see on this mirror at
http://delta.archlinux.fr/core/os/i686/ ?

Can you please enlighten me here?

> I guess that if on redhat distributions the typical rpm2cpio does decompression
> by default then this is something I can take up with the rpmextract maintainer
> in ArchLinux. Worst case the archlinux abigail PKGBUILD could patch the
> abipkgdiff source to add a decompression step.

well, if rpm2cpio <package> | cpio -idmv could be made to work, I'd be
glad, yes.

-- 
		Dodji

[Bug default/20299] abipkgdiff reports "cpio: Malformed number ..." when unpacking RPMs

[joe at wand dot net.nz]

https://sourceware.org/bugzilla/show_bug.cgi?id=20299

--- Comment #6 from Joe Stringer <joe at wand dot net.nz> ---
(In reply to dodji from comment #5)
> "joe at wand dot net.nz" <sourceware-bugzilla@sourceware.org> writes: 
> >> > I guess that if on redhat distributions the typical rpm2cpio does decompression
> >> > by default then this is something I can take up with the rpmextract maintainer
> >> > in ArchLinux. Worst case the archlinux abigail PKGBUILD could patch the
> >> > abipkgdiff source to add a decompression step.
> >> 
> >> well, if rpm2cpio <package> | cpio -idmv could be made to work, I'd be
> >> glad, yes.
> >
> > I think this is the right solution - it looks like the rpm2cpio utility on
> > Archlinux is just a ~25 line script which does not cover many cases (for
> > example, decompression).
> 
> Right.
> 
> But in the mean time, maybe libabigail's build system could detect the
> presence of bsdtar.  So that when you are compiling libabigail, if
> bsdtar is present, then abipkgdiff.cc would use it to extract the
> content of the (possibly compressed) cpio archive emitted by rpm2cpio.
> 
> If bsdtar is not present at build time, then abipkgdiff.cc would use
> cpio, just like it does today.
> 
> Would that work for you?

That sounds quite reasonable, and it would work for me.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug default/20299] abipkgdiff reports "cpio: Malformed number ..." when unpacking RPMs

[mariiakovaleva3592 at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=20299

Chernot <mariiakovaleva3592 at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |mariiakovaleva3592 at gmail dot co
                   |                            |m

--- Comment #7 from Chernot <mariiakovaleva3592 at gmail dot com> ---
Nice try! It worked for me, especially on this term -
https://apknature.com/pool-slide-story-apk/

Thank you!

-- 
You are receiving this mail because:
You are on the CC list for the bug.