debuginfod & Debian source packages

debuginfod & Debian source packages

[Sergio Durigan Junior]

[-- Attachment #1: Type: text/plain, Size: 2063 bytes --]

Hi,

I'm the maintainer of debuginfod.debian.net (currently offline due to a
hardware issue :-/).  The service provides only debuginfo for now, but I
would like to start indexing the source code of each package as well.

I know that Fedora has the "debugsource" RPM package which makes things
easier, but after talking to some Debian/Ubuntu Developers they were
(understandably) not happy with the idea of implementing a similar
solution for deb packages.

I'm now thinking about an alternative solution to this problem.  Debian
source packages already contain everything needed to be indexed and
served to debuginfo consumers; it has the full source code + all the
downstream patches.  It's represented by a .dsc file that can be fed to
dget(1) which will download the source tarball and apply all the patches
automatically.

Do you think we can teach debuginfod to consume these files and do the
right thing when indexing the source code of each package?  I'm not
entirely familiar with how debuginfod does things behind the scenes, but
initially I thought something along the following lines:

1) Obtain the .dsc file which corresponds to the -dbgsym package being
   analysed;

2) Use dget(1) to obtain its corresponding source + patches, then index
   everything.

3) Compress the source tree and store it somewhere, associating the
   compressed file with the build-id(s) of the -dbgsym file.  (Is this
   really necessary?  Not sure.)

4) When a build-id request comes in, we serve the debuginfo from the
   -dbgsym file and decompress the source tarball corresponding to it,
   serving whatever files are needed.  I'd assume that this is similar
   to what debuginfod already does for Fedora's -debugsource packages.

This is a "back-of-the-napkin" thought about how to adjust debuginfod
for Debian/Ubuntu, but of course I'd like to hear your opinion as well.

Thanks,

-- 
Sergio
GPG key ID: 237A 54B1 0287 28BF 00EF  31F4 D0EB 7628 65FC 5E36
Please send encrypted e-mail if possible
https://sergiodj.net/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: debuginfod & Debian source packages

[Frank Ch. Eigler]

Hi -

> I'm the maintainer of debuginfod.debian.net (currently offline due to a
> hardware issue :-/).

O no.  (And also, please consider upgrading your elfutils version, as
later ones have less pessimal behavior with long grooming ops.)

> The service provides only debuginfo for now, but I
> would like to start indexing the source code of each package as well.

Neat!

> [...]
> I'm now thinking about an alternative solution to this problem.  Debian
> source packages already contain everything needed to be indexed and
> served to debuginfo consumers; it has the full source code + all the
> downstream patches.  It's represented by a .dsc file that can be fed to
> dget(1) which will download the source tarball and apply all the patches
> automatically.
> 
> Do you think we can teach debuginfod to consume these files and do the
> right thing when indexing the source code of each package? [...]

Interesting idea, but I'd throw it back at you thusly: does debuginfod
need to this itself on the fly?  Other than perhaps a time/storage
tradeoff, is there some reason an auxiliary program couldn't do this
in the background?  Take designated .dsc's, download, apply patches,
and assemble the patched sources into, well, any old random tarball
format?  If someone(tm) were to write such a script, we could ship it
alongside debuginfod if desired.

As long as the archive file name was a close match to the binary deb
file names, and as long as the constituent files were named with the
same paths as mentioned in the binary dwarf, debuginfod would gladly
take the result and serve from it.


- FChE

Re: debuginfod & Debian source packages

[Sergio Durigan Junior]

On Thursday, June 02 2022, Frank Ch. Eigler wrote:

> Hi -

Hi Frank,

>> I'm the maintainer of debuginfod.debian.net (currently offline due to a
>> hardware issue :-/).
>
> O no.  (And also, please consider upgrading your elfutils version, as
> later ones have less pessimal behavior with long grooming ops.)

I have to backport the new version to Debian bullseye, but yeah, this is
on my TODO list.

>> [...]
>> I'm now thinking about an alternative solution to this problem.  Debian
>> source packages already contain everything needed to be indexed and
>> served to debuginfo consumers; it has the full source code + all the
>> downstream patches.  It's represented by a .dsc file that can be fed to
>> dget(1) which will download the source tarball and apply all the patches
>> automatically.
>> 
>> Do you think we can teach debuginfod to consume these files and do the
>> right thing when indexing the source code of each package? [...]
>
> Interesting idea, but I'd throw it back at you thusly: does debuginfod
> need to this itself on the fly?  Other than perhaps a time/storage
> tradeoff, is there some reason an auxiliary program couldn't do this
> in the background?  Take designated .dsc's, download, apply patches,
> and assemble the patched sources into, well, any old random tarball
> format?  If someone(tm) were to write such a script, we could ship it
> alongside debuginfod if desired.

Sure, I believe an external script/program could do things behind the
scenes without problem.

> As long as the archive file name was a close match to the binary deb
> file names, and as long as the constituent files were named with the
> same paths as mentioned in the binary dwarf, debuginfod would gladly
> take the result and serve from it.

OK, this was going to be my next question.  Out of curiosity, how would
debuginfod invoke this external program?

Thanks,

-- 
Sergio
GPG key ID: 237A 54B1 0287 28BF 00EF  31F4 D0EB 7628 65FC 5E36
Please send encrypted e-mail if possible
https://sergiodj.net/

Re: debuginfod & Debian source packages

[Frank Ch. Eigler]

Hi -

> [...]
> OK, this was going to be my next question.  Out of curiosity, how would
> debuginfod invoke this external program?

That's the best part.  It doesn't need to. :-) Whatever program you're
using to freshen the repository of .deb's that your debuginfod
scrapes, could also run this program for any new .dsc's.  i.e., keep
the problem from debuginfod entirely.

- FChE

Re: debuginfod & Debian source packages

[Sergio Durigan Junior]

On Thursday, June 02 2022, Frank Ch. Eigler wrote:

> Hi -
>
>> [...]
>> OK, this was going to be my next question.  Out of curiosity, how would
>> debuginfod invoke this external program?
>
> That's the best part.  It doesn't need to. :-) Whatever program you're
> using to freshen the repository of .deb's that your debuginfod
> scrapes, could also run this program for any new .dsc's.  i.e., keep
> the problem from debuginfod entirely.

Ah, OK.  Yeah, I guess it makes sense to do it this way.  I will do some
hacking this coming weekend and see what I come up with, and then I will
let you know.  Hopefully this will turn out to be easier than I thought.

Thanks,

-- 
Sergio
GPG key ID: 237A 54B1 0287 28BF 00EF  31F4 D0EB 7628 65FC 5E36
Please send encrypted e-mail if possible
https://sergiodj.net/