From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by sourceware.org (Postfix) with ESMTPS id 8569E3858C52 for ; Fri, 20 Jan 2023 10:27:50 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 8569E3858C52 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674210470; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=it3hHnLagyAvWDzBcQ+8IooCf0NdFnHHyyEn8oG1Tpw=; b=C5pctChkRYjGPc3anX+p8HiUJNPa6355983k9t6Ae5f+kzV/cEPvpUErkC0qPU4p7QKooi 7KJvIZCTKHwmW1S/70ZC0EKnaBVDi7ELn8FnIDR4peha4iTSYPJylM2EvKy1WRqJRbudgS YYg7hCpzeot/SNWCvVXFXxaS9qlBC4E= Received: from mail-il1-f199.google.com (mail-il1-f199.google.com [209.85.166.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-635-1jWM4kOMOf6-ydv8BQiK3Q-1; Fri, 20 Jan 2023 05:27:46 -0500 X-MC-Unique: 1jWM4kOMOf6-ydv8BQiK3Q-1 Received: by mail-il1-f199.google.com with SMTP id i14-20020a056e020d8e00b003034b93bd07so3466604ilj.14 for ; Fri, 20 Jan 2023 02:27:46 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=it3hHnLagyAvWDzBcQ+8IooCf0NdFnHHyyEn8oG1Tpw=; b=GI1yoNVurikZMAfXSr60E2ZWdeHWG4mG5Gmtl9Y2WcyKm1wFsDX1WkR/3NNqCXMWiH kfHufPuyZgZzlEZz6+7v3B+7HBlBsGmsJ7/QsyV17OdVJAe7kixIJq+2Sadr/zxxDSNB OXScyXAsvVHuNfX7jXpOdwj1Qmcug6LJ/mkLl3g7AMY0NbcP5cqwji1Os/CZP03x+UVc 92PNtYrVpXXFLFynR8XdvdRUHPIN9rF3dwqxI338nw9n3jM6c41+B5tn7CYd+ZOQnDvu 2ewgbpaqBZ3XTSV9Tmw/VQI+RCnRVy6wt4QaxjLqsPr6djZ4pU8Md7smGheDCTyupbL+ CvYg== X-Gm-Message-State: AFqh2kp38aejLNBDCtr3C7NL8iKOyfpq4whrvmCeN/jnU3JgpU0fkKag TG0SwnUwn69mPCpzEu61MXEFCp93LNqh+OlQK8EOuDttkuUh/7bdK0ik/LEJiraa8ejqrVnD6v9 D6TAxtVMaNkWnYjfQnK+fhnH6ciB+NstXqxo= X-Received: by 2002:a05:6638:3383:b0:38a:7bae:6a60 with SMTP id h3-20020a056638338300b0038a7bae6a60mr1708990jav.37.1674210465993; Fri, 20 Jan 2023 02:27:45 -0800 (PST) X-Google-Smtp-Source: AMrXdXsvLUNTrYq8ht5fYyv3DWrmxUjzrQkIeoC2J8llsi8OPjRlzK5K41ovkZ7t6In9edVMQbCrPluvSvoGSpxSoCo= X-Received: by 2002:a05:6638:3383:b0:38a:7bae:6a60 with SMTP id h3-20020a056638338300b0038a7bae6a60mr1708986jav.37.1674210465684; Fri, 20 Jan 2023 02:27:45 -0800 (PST) MIME-Version: 1.0 References: <20230119160318.3868439-1-klember@redhat.com> In-Reply-To: From: Kalev Lember Date: Fri, 20 Jan 2023 11:27:35 +0100 Message-ID: Subject: Re: [PATCH] find-debuginfo: Pass -j down to dwz To: Panu Matilainen Cc: debugedit@sourceware.org X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: multipart/alternative; boundary="00000000000016334905f2af8030" X-Spam-Status: No, score=-22.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,GIT_PATCH_0,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: --00000000000016334905f2af8030 Content-Type: text/plain; charset="UTF-8" On Fri, Jan 20, 2023 at 9:40 AM Panu Matilainen wrote: > On 1/19/23 18:03, Kalev Lember wrote: > > Now that dwz 0.15 supports parallel jobs, add a way to control it from > > here. find-debuginfo already has a -j parameter so we can just extend it > > and pass the value down to dwz as well. > > > > This should fix building large packages on memory constrained builders, > > such as webkitgtk on s390x in Fedora koji build system, where we can now > > use the -j option to tune down parallelism to avoid running out of > > memory during dwz run. > > > > Signed-off-by: Kalev Lember > > --- > > scripts/find-debuginfo.in | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/scripts/find-debuginfo.in b/scripts/find-debuginfo.in > > index b07a52f..8cc1bfd 100755 > > --- a/scripts/find-debuginfo.in > > +++ b/scripts/find-debuginfo.in > > @@ -585,7 +585,7 @@ if $run_dwz \ > > dwz_multifile_suffix=".${dwz_multifile_idx}" > > done > > dwz_multifile_name="${dwz_multifile_name}${dwz_multifile_suffix}" > > - dwz_opts="-h -q -r" > > + dwz_opts="-h -q -r -j ${n_jobs}" > > [ ${#dwz_files[@]} -gt 1 ] && [ "$dwz_single_file_mode" = "false" > ] \ > > && dwz_opts="${dwz_opts} -m .dwz/${dwz_multifile_name}" > > mkdir -p "${RPM_BUILD_ROOT}/usr/lib/debug/.dwz" > > This is so new, even Fedora 37 doesn't have it yet. I tend to think > find-debuginfo should be a bit conservative about the assumptions it > makes about external tools. > Can we do some kind of configure check then and require the latest dwz? Or add this as a downstream patch in rawhide? The problem is that new dwz is already in rawhide and it broke webkitgtk builds due to too much memory use. The ppc64le and s390x builders just don't have enough memory now to do dwz on webkitgtk. Passing down the -j option would help us control the memory use to get the builds going again. Meanwhile we worked this around by completely disabling the dwz optimization for webkitgtk, but it would be nice to get it back to working. See https://src.fedoraproject.org/rpms/webkitgtk/c/2c5dd84529070a8cfb0c6599dee587ca8fbcb6d2?branch=rawhide -- Thanks, Kalev --00000000000016334905f2af8030--