Re: [PATCH] find-debuginfo: Pass -j down to dwz

[Kalev Lember <klember@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 2226 bytes --]

On Fri, Jan 20, 2023 at 9:40 AM Panu Matilainen <pmatilai@redhat.com> wrote:

> On 1/19/23 18:03, Kalev Lember wrote:
> > Now that dwz 0.15 supports parallel jobs, add a way to control it from
> > here. find-debuginfo already has a -j parameter so we can just extend it
> > and pass the value down to dwz as well.
> >
> > This should fix building large packages on memory constrained builders,
> > such as webkitgtk on s390x in Fedora koji build system, where we can now
> > use the -j option to tune down parallelism to avoid running out of
> > memory during dwz run.
> >
> > Signed-off-by: Kalev Lember <klember@redhat.com>
> > ---
> >   scripts/find-debuginfo.in | 2 +-
> >   1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/scripts/find-debuginfo.in b/scripts/find-debuginfo.in
> > index b07a52f..8cc1bfd 100755
> > --- a/scripts/find-debuginfo.in
> > +++ b/scripts/find-debuginfo.in
> > @@ -585,7 +585,7 @@ if $run_dwz \
> >         dwz_multifile_suffix=".${dwz_multifile_idx}"
> >       done
> >       dwz_multifile_name="${dwz_multifile_name}${dwz_multifile_suffix}"
> > -    dwz_opts="-h -q -r"
> > +    dwz_opts="-h -q -r -j ${n_jobs}"
> >       [ ${#dwz_files[@]} -gt 1 ] && [ "$dwz_single_file_mode" = "false"
> ] \
> >         && dwz_opts="${dwz_opts} -m .dwz/${dwz_multifile_name}"
> >       mkdir -p "${RPM_BUILD_ROOT}/usr/lib/debug/.dwz"
>
> This is so new, even Fedora 37 doesn't have it yet. I tend to think
> find-debuginfo should be a bit conservative about the assumptions it
> makes about external tools.
>

Can we do some kind of configure check then and require the latest dwz? Or
add this as a downstream patch in rawhide?

The problem is that new dwz is already in rawhide and it broke webkitgtk
builds due to too much memory use. The ppc64le and s390x builders just
don't have enough memory now to do dwz on webkitgtk. Passing down the -j
option would help us control the memory use to get the builds going again.

Meanwhile we worked this around by completely disabling the dwz
optimization for webkitgtk, but it would be nice to get it back to working.

See
https://src.fedoraproject.org/rpms/webkitgtk/c/2c5dd84529070a8cfb0c6599dee587ca8fbcb6d2?branch=rawhide

-- 
Thanks,
Kalev