Re: [PATCH] find-debuginfo: Pass -j down to dwz

[Panu Matilainen <pmatilai@redhat.com>]

On 1/20/23 12:27, Kalev Lember wrote:
> On Fri, Jan 20, 2023 at 9:40 AM Panu Matilainen <pmatilai@redhat.com 
> <mailto:pmatilai@redhat.com>> wrote:
> 
>     On 1/19/23 18:03, Kalev Lember wrote:
>      > Now that dwz 0.15 supports parallel jobs, add a way to control it
>     from
>      > here. find-debuginfo already has a -j parameter so we can just
>     extend it
>      > and pass the value down to dwz as well.
>      >
>      > This should fix building large packages on memory constrained
>     builders,
>      > such as webkitgtk on s390x in Fedora koji build system, where we
>     can now
>      > use the -j option to tune down parallelism to avoid running out of
>      > memory during dwz run.
>      >
>      > Signed-off-by: Kalev Lember <klember@redhat.com
>     <mailto:klember@redhat.com>>
>      > ---
>      >   scripts/find-debuginfo.in <http://find-debuginfo.in> | 2 +-
>      >   1 file changed, 1 insertion(+), 1 deletion(-)
>      >
>      > diff --git a/scripts/find-debuginfo.in <http://find-debuginfo.in>
>     b/scripts/find-debuginfo.in <http://find-debuginfo.in>
>      > index b07a52f..8cc1bfd 100755
>      > --- a/scripts/find-debuginfo.in <http://find-debuginfo.in>
>      > +++ b/scripts/find-debuginfo.in <http://find-debuginfo.in>
>      > @@ -585,7 +585,7 @@ if $run_dwz \
>      >         dwz_multifile_suffix=".${dwz_multifile_idx}"
>      >       done
>      >     
>       dwz_multifile_name="${dwz_multifile_name}${dwz_multifile_suffix}"
>      > -    dwz_opts="-h -q -r"
>      > +    dwz_opts="-h -q -r -j ${n_jobs}"
>      >       [ ${#dwz_files[@]} -gt 1 ] && [ "$dwz_single_file_mode" =
>     "false" ] \
>      >         && dwz_opts="${dwz_opts} -m .dwz/${dwz_multifile_name}"
>      >       mkdir -p "${RPM_BUILD_ROOT}/usr/lib/debug/.dwz"
> 
>     This is so new, even Fedora 37 doesn't have it yet. I tend to think
>     find-debuginfo should be a bit conservative about the assumptions it
>     makes about external tools.
> 
> 
> Can we do some kind of configure check then and require the latest dwz? 
> Or add this as a downstream patch in rawhide?
> 
> The problem is that new dwz is already in rawhide and it broke webkitgtk 
> builds due to too much memory use. The ppc64le and s390x builders just 
> don't have enough memory now to do dwz on webkitgtk. Passing down the -j 
> option would help us control the memory use to get the builds going again.
> 

Oh, if rawhide needs this then rawhide needs this, I don't mind that at 
all. I just think that upstream find-debuginfo shouldn't assume "latest 
everything" any more than rpm can do so.

configure-time check seems reasonable to me.

	- Panu -