From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gnu.wildebeest.org (gnu.wildebeest.org [45.83.234.184]) by sourceware.org (Postfix) with ESMTPS id 61CA73858D28; Tue, 11 Apr 2023 08:19:58 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 61CA73858D28 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=klomp.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=klomp.org Received: by gnu.wildebeest.org (Postfix, from userid 1000) id 8391B302BB02; Tue, 11 Apr 2023 10:19:57 +0200 (CEST) Date: Tue, 11 Apr 2023 10:19:57 +0200 From: Mark Wielaard To: Mark Wielaard via Overseers Cc: buildbot@sourceware.org, Arsen =?utf-8?Q?Arsenovi=C4=87?= , Gerald Pfeifer , Nick Clifton , Dodji Seketeli , "Jose E. Marchesi" Subject: Re: Using builder for (documentation) snapshots Message-ID: <20230411081957.GE6496@gnu.wildebeest.org> References: <20230409113050.GA6496@gnu.wildebeest.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230409113050.GA6496@gnu.wildebeest.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-3030.6 required=5.0 tests=BAYES_00,JMQ_SPF_NEUTRAL,KAM_DMARC_STATUS,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Hi, On Sun, Apr 09, 2023 at 01:30:50PM +0200, Mark Wielaard via Overseers wrote: > Thanks to OSUOSL we now have an extra buildbot worker with extra > storage so we can do container builds and publish (documentation) > snapshots. https://snapshots.sourceware.org/ > > It acts like a normal buildbot (x86_64) container worker, except that > whatever is left in the output directory will be available through the > snapshots server. > > Some examples of where this is useful: > > - elfutils test coverage reports. Currently they are only produced (by > hand) once an official release is made. Ideally we have coverage > reports of current git. > > - snapshots of binutils or gnupoke which are either currently done by > hand (and so need a script to be written to automate) or which have > a non-trivial boostrap script. > > - generating the gcc documentation using the lowest and highest > supported version of texinfo to make sure it works and looks as > intended. > > - Generating the documentation/website of libabigail which is > currently done by hand and needs a specific setup to generate from > the source code with sphynx. > > - valgrind the html and pdf manuals are currently only generated by > hand during a release. > > - dwarfstd.org is automatically generated through a git hook from the > main branch, but people have wanted to show how things look from a > specific branch. > > There is an example of how to do the elfutils coverage report in > builbot.git: > https://sourceware.org/cgit/builder/commit/?id=365a64a19b9a4b3cd677ec377dd1574e4dcdf022 > This also contains the setup instructions of the snapshots server plus > scripts used. The output can be seen here: > https://snapshots.sourceware.org/elfutils/coverage/ > > I'll help setup the other examples. I added two more: libabigail manual, api docs and website: https://snapshots.sourceware.org/libabigail/html-doc/ https://sourceware.org/cgit/builder/commit/?id=39641bd063c4bbeab4ce4f38625f1140a84f5b4d valgrind trunk snapshot, manual and doc (pdf) downloads: https://snapshots.sourceware.org/valgrind/trunk/ https://sourceware.org/cgit/builder/commit/?id=f338a430db0382d9bd26b447cfe762566c9e7935 The idea is the same: - Pick or create a new container file (add any build dependencies) - Add a Periodic for the project (trigger every 15 minutes, checking for any changes on a specific branch) - Create a BuildFactory with build steps (ideally existing make targets or a build script in the repo) which put any artifacts under the output dir and create a publish file containing the intended (branch/topic) name. - Add a BuildConfig using the container file and build factory. Cheers, Mark