From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by sourceware.org (Postfix) with ESMTPS id 7EE423857B98 for ; Tue, 31 May 2022 16:39:35 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 7EE423857B98 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-634-112_RWtrMTSfiY_lgxgZOA-1; Tue, 31 May 2022 12:39:34 -0400 X-MC-Unique: 112_RWtrMTSfiY_lgxgZOA-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D04DF3C0F721; Tue, 31 May 2022 16:39:33 +0000 (UTC) Received: from redhat.com (unknown [10.2.16.15]) by smtp.corp.redhat.com (Postfix) with ESMTPS id BC83C492C3B; Tue, 31 May 2022 16:39:33 +0000 (UTC) Received: from fche by redhat.com with local (Exim 4.94.2) (envelope-from ) id 1nw4u4-0006ef-SU; Tue, 31 May 2022 12:39:33 -0400 Date: Tue, 31 May 2022 12:39:32 -0400 From: "Frank Ch. Eigler" To: Overseers mailing list Cc: Mark Wielaard Subject: Re: GNU Toolchain Infrastructure at sourceware Message-ID: <20220531163932.GA25222@redhat.com> References: MIME-Version: 1.0 In-Reply-To: User-Agent: Mutt/1.12.0 (2019-05-25) X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: overseers@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Overseers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 May 2022 16:39:37 -0000 Hi - > builder.sourceware.org has been running for a couple of months > now. The buildbot process seems to have a low load on the machine. Low > single digit %CPU. The state database has grown to ~800MB, the > gitpoller-work dir is ~4GB (that is mainly gcc ~2G, libabigail ~750M > and binutils-gdb ~600M, others are < 200M). I suspect those git work trees could be made into shallow clones, or use git-alternates or somesuch to minimize unnecessary .git/objects duplication right there on the same machine. > I am slightly worried about the long time these uploads take. For > example the upload of results from the elfutils builders takes a > significant amount of time (almost half) of the build (4 to 5 out of > 10 to 12 minutes). Especially on the slower workers this can cause > them to get behind on the build queue. Yeah, this appears to be a buildbot infrastructure limitation. Something is very inefficient about the way it evaluates file upload directives. If it turns into a bigger problem, can try to revisit. > The bunsendb.git is currently ~400MB and contains ~2000 results. It > seems it compresses fairly well and could maybe use a periodic > repacking (making a local clone reduces the size to ~260MB). No big deal, auto repack will do most of the work fine. Added to that, there is the bunsenql.sqlite database, which is at the moment around 2GB. It will ebb and flow with the bunsendb.git contents. I think all of the above storage levels are sustainable for at least several months at this rate, after which point we could start aging out old data. (Plus we have an extra TB of space for /sourceware[12] that I plan to bring online shortly.) - FChE