From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by sourceware.org (Postfix) with ESMTPS id 2C0BA3858D3C for ; Thu, 10 Nov 2022 14:16:07 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 2C0BA3858D3C Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=suse.cz Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 1AE461F8C2; Thu, 10 Nov 2022 14:16:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1668089766; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gupkuyOzTOmPLk1RfSkcqKu2y4Y415GDEuxAWMefCGM=; b=i7gZS3A9dyb6AiE4xmloYtxBVtU1YmxqpbjDpacVpHQ4Axis24n8BUlx8RkdHRnwOBFiTP K6zovG+/j2rwHdzeMmKOa93/k75jLOjv4XZgITx2GeqtOVldiel+j7SDfB2d0u/mjjlr3S l0P0Qw9YxwmIQAQLvlNtTYPkOpyOg7E= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1668089766; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gupkuyOzTOmPLk1RfSkcqKu2y4Y415GDEuxAWMefCGM=; b=AN4wC2x07f+CQ7SXSHg+zfbIoWCGyv585eUycGmDC9CCaX0tAUXmzNoFkhUdKXz+CZBleD 28UHmqsTBxpbqDBA== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id E7DD21332F; Thu, 10 Nov 2022 14:16:05 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id iC2qN6UHbWOYGwAAMHmgww (envelope-from ); Thu, 10 Nov 2022 14:16:05 +0000 Message-ID: Date: Thu, 10 Nov 2022 15:16:05 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.2 Subject: Re: [PATCH] maintainer-scripts/gcc_release: compress xz in parallel Content-Language: en-US To: Xi Ruoyao , Joseph Myers Cc: Sam James , gcc-patches@gcc.gnu.org, Gerald Pfeifer , Jakub Jelinek , Jeff Law References: <20221108071438.2523863-1-sam@gentoo.org> <6e8d39e1e3e9d6ce5f0be5781023127187bcb995.camel@xry111.site> <6e2f0c0dd7cfe1b4abe5ddcc08e45e984ccbe61f.camel@xry111.site> From: =?UTF-8?Q?Martin_Li=c5=a1ka?= In-Reply-To: <6e2f0c0dd7cfe1b4abe5ddcc08e45e984ccbe61f.camel@xry111.site> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-5.9 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On 11/9/22 03:06, Xi Ruoyao via Gcc-patches wrote: > On Wed, 2022-11-09 at 01:52 +0000, Joseph Myers wrote: >> On Tue, 8 Nov 2022, Xi Ruoyao via Gcc-patches wrote: >> >>> I'm wondering if running xz -T0 on different machines (with different >>> core numbers) may produce different compressed data.  The difference can >>> cause trouble distributing checksums. >> >> gcc_release definitely doesn't use any options to make the tar file >> reproducible (the timestamps, user and group names and ordering of the >> files in the tarball, and quite likely permissions other than whether a >> file has execute permission, may depend on when the script was run and on >> what system as what user - not just on the commit from which the tar file >> was built).  So I don't think possible variation of xz output matters here >> at present. > > OK then. I'm already using commands like > > git archive --format=tar --prefix=gcc-$(git gcc-descr HEAD)/ HEAD | xz -T0 > ../gcc-$(git gcc-descr HEAD).tar.xz > > when I generate a GCC snapshot tarball for my own use. > > Hi. We may consider using zstd compression that also support a multi-threaded compression (which is stable). Note the decompression of zstd is much faster than xz. Martin