From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 16399 invoked by alias); 18 Apr 2016 19:45:05 -0000 Mailing-List: contact cygwin-apps-help@cygwin.com; run by ezmlm Precedence: bulk Sender: cygwin-apps-owner@cygwin.com List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Mail-Followup-To: cygwin-apps@cygwin.com Received: (qmail 16365 invoked by uid 89); 18 Apr 2016 19:45:04 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-2.1 required=5.0 tests=AWL,BAYES_00,RCVD_IN_DNSWL_LOW,SPF_PASS autolearn=ham version=3.3.2 spammy=laptop, Dedup, solid X-HELO: mail-in-09.arcor-online.net Received: from mail-in-09.arcor-online.net (HELO mail-in-09.arcor-online.net) (151.189.21.49) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (CAMELLIA256-SHA encrypted) ESMTPS; Mon, 18 Apr 2016 19:44:53 +0000 Received: from mail-in-16-z2.arcor-online.net (mail-in-16-z2.arcor-online.net [151.189.8.33]) by mx.arcor.de (Postfix) with ESMTP id 3qpdry0PsHzB4Bm for ; Mon, 18 Apr 2016 21:44:49 +0200 (CEST) Received: from mail-in-06.arcor-online.net (mail-in-06.arcor-online.net [151.189.21.46]) by mail-in-16-z2.arcor-online.net (Postfix) with ESMTP id BFD432177BC for ; Mon, 18 Apr 2016 21:44:49 +0200 (CEST) X-DKIM: Sendmail DKIM Filter v2.8.2 mail-in-06.arcor-online.net 3qpdrx3twWz7n1H Received: from Gertrud (p54B46F42.dip0.t-ipconnect.de [84.180.111.66]) (Authenticated sender: stromeko@arcor.de) by mail-in-06.arcor-online.net (Postfix) with ESMTPSA id 3qpdrx3twWz7n1H for ; Mon, 18 Apr 2016 21:44:49 +0200 (CEST) From: Achim Gratz To: cygwin-apps@cygwin.com Subject: Re: Dedup x86/x86_64 --> noarch References: <87zistg99v.fsf@Rainer.invalid> Date: Mon, 18 Apr 2016 19:45:00 -0000 In-Reply-To: <87zistg99v.fsf@Rainer.invalid> (Achim Gratz's message of "Sat, 16 Apr 2016 12:03:56 +0200") Message-ID: <87bn56puq7.fsf@Rainer.invalid> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.0.92 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SW-Source: 2016-04/txt/msg00028.txt.bz2 Achim Gratz writes: > Looking at the current repo content we'd save about 30GB from the dedup > of the src abd doc packages alone and probably about 20GB from dedup in > the remaining packages. I've implemented some POC code and deduped my Cygwin mirror (it is missing most of KDE and the cross-Cygwin compilation toolchains). This took a solid 12 hours of flat out 400% CPU load on my SandyBridge laptop and ballooned the page file to 21GiB. But it also removed almost exactly a third from the repo's size (going from 81.2GiB to 51.4GiB), so projected to the full repo it's slightly more than my original estimate. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf rackAttack: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds