From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 19324 invoked by alias); 23 Apr 2016 11:19:43 -0000 Mailing-List: contact cygwin-apps-help@cygwin.com; run by ezmlm Precedence: bulk Sender: cygwin-apps-owner@cygwin.com List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Mail-Followup-To: cygwin-apps@cygwin.com Received: (qmail 19311 invoked by uid 89); 23 Apr 2016 11:19:42 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-2.1 required=5.0 tests=AWL,BAYES_00,RCVD_IN_DNSWL_LOW,SPF_PASS autolearn=ham version=3.3.2 spammy=Turney, virtue, Hx-languages-length:3488, designation X-HELO: mail-in-15.arcor-online.net Received: from mail-in-15.arcor-online.net (HELO mail-in-15.arcor-online.net) (151.189.21.55) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (CAMELLIA256-SHA encrypted) ESMTPS; Sat, 23 Apr 2016 11:19:31 +0000 Received: from mail-in-20-z2.arcor-online.net (mail-in-20-z2.arcor-online.net [151.189.8.85]) by mx.arcor.de (Postfix) with ESMTP id 3qsVPW5P0vzB6xX for ; Sat, 23 Apr 2016 13:19:27 +0200 (CEST) Received: from mail-in-14.arcor-online.net (mail-in-14.arcor-online.net [151.189.21.54]) by mail-in-20-z2.arcor-online.net (Postfix) with ESMTP id B7480834560 for ; Sat, 23 Apr 2016 13:19:27 +0200 (CEST) X-DKIM: Sendmail DKIM Filter v2.8.2 mail-in-14.arcor-online.net 3qsVPW4Nncz6WBJ Received: from Gertrud (p54B46F42.dip0.t-ipconnect.de [84.180.111.66]) (Authenticated sender: stromeko@arcor.de) by mail-in-14.arcor-online.net (Postfix) with ESMTPSA id 3qsVPW4Nncz6WBJ for ; Sat, 23 Apr 2016 13:19:27 +0200 (CEST) From: Achim Gratz To: cygwin-apps@cygwin.com Subject: Re: Dedup x86/x86_64 --> noarch References: <87zistg99v.fsf@Rainer.invalid> <571B539D.4050304@dronecode.org.uk> Date: Sat, 23 Apr 2016 11:19:00 -0000 In-Reply-To: <571B539D.4050304@dronecode.org.uk> (Jon Turney's message of "Sat, 23 Apr 2016 11:51:09 +0100") Message-ID: <87fuucpo78.fsf@Rainer.invalid> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.0.92 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SW-Source: 2016-04/txt/msg00037.txt.bz2 Jon Turney writes: > I think I have implemented the changes to calm to support > all-or-nothing noarch (i.e. where all packages produced from a source > package must be noarch), so if you can nominate a suitable, > unimportant perl package, we can test it with that, initially. I'll have a look, there are quite a few packages that should have no arch-dependent components by virtue of them being pure Perl. In fact, some of these were noarch while they were still in my private repo since I kept the noarch alongside like Cygwin Ports has been doing. You just want to move them in the repo? > (This wasn't quite as straightforward as just looking in another > directory for packages, as the upload validation becomes more complex: > we must check that consistent package sets result for both x86 and > x86_64 before we can move noarch packages) Sure. For pure noarch this isn't much of a problem (or it never has been with genini anyway). Mixed packages might present a more interesting challenge. I think however, that if you simply treat everything in noarch// the same as / and coalesce these two directories logically, this problem should go away. > To make full use of this, cygport upload will need a feature to upload > noarch packages from dist/ to noarch/ rather than /. I haven't looked into that, but I'm not using cygport for the upload anyway. > I don't think this distinguishes between packages which are (or should > be) marked ARCH="noarch" in the cygport, and those where the build > products happen to be identical and can be deduped? No, I just took the archives and ran with it. In fact most of the big packages that I had memory problems with would clearly be "noarch" from the outset (fonts and some of texlive). > I would guess that this saving is dominated by some very large, > data-only noarch packages, but who knows? Thosa packages exists, but the commonalities in source and debuginfo packages are summarily larger. The scripting languages provide another set of common files that yould be split off into a noarch package that is then a dependency for the arch-dependent stuff. But that's not possible to set up right now I take it. > (Also, looking forward, perhaps cygport needs a separate command to > build the source package, rather than building it for each arch and > then deduping it?) I'm not sure what Yaakov thinks of that, but I really wanted to move the whole build under a single directory and indeed dedup on the packaging step (where I can just hardlink in the fs rather than needing to copy in memory). I haven't looked into that yet, because the changes necessary might be quite pervasive. But for the moment deduping the archives doesn't look too bad either, so for folks building on different machines and not sharing the build resources that would be another option (vs. unpacking the build results from one arch into the build directory of another). Another thing that cygport needs is a way to mark sub-packages as noarch so that the dedup can be skipped for files that are known to be noarch already (I think their correct designation should still be checked w.r.t. the actual build results). Again, I've not yet looked at what that will need in terms of implementation. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptation for Waldorf Blofeld V1.15B11: http://Synth.Stromeko.net/Downloads.html#WaldorfSDada