Re: Using builder for (documentation) snapshots

[Mark Wielaard <mark@klomp.org>]

Hi,

On Sun, Apr 09, 2023 at 01:30:50PM +0200, Mark Wielaard via Overseers wrote:
> Thanks to OSUOSL we now have an extra buildbot worker with extra
> storage so we can do container builds and publish (documentation)
> snapshots. https://snapshots.sourceware.org/
> 
> It acts like a normal buildbot (x86_64) container worker, except that
> whatever is left in the output directory will be available through the
> snapshots server.
> 
> Some examples of where this is useful:
> 
> - elfutils test coverage reports. Currently they are only produced (by
>   hand) once an official release is made. Ideally we have coverage
>   reports of current git.
> 
> - snapshots of binutils or gnupoke which are either currently done by
>   hand (and so need a script to be written to automate) or which have
>   a non-trivial boostrap script.
> 
> - generating the gcc documentation using the lowest and highest
>   supported version of texinfo to make sure it works and looks as
>   intended.
> 
> - Generating the documentation/website of libabigail which is
>   currently done by hand and needs a specific setup to generate from
>   the source code with sphynx.
> 
> - valgrind the html and pdf manuals are currently only generated by
>   hand during a release.
> 
> - dwarfstd.org is automatically generated through a git hook from the
>   main branch, but people have wanted to show how things look from a
>   specific branch.
> 
> There is an example of how to do the elfutils coverage report in
> builbot.git:
> https://sourceware.org/cgit/builder/commit/?id=365a64a19b9a4b3cd677ec377dd1574e4dcdf022
> This also contains the setup instructions of the snapshots server plus
> scripts used. The output can be seen here:
> https://snapshots.sourceware.org/elfutils/coverage/
> 
> I'll help setup the other examples.

I added two more:

libabigail manual, api docs and website:
https://snapshots.sourceware.org/libabigail/html-doc/
https://sourceware.org/cgit/builder/commit/?id=39641bd063c4bbeab4ce4f38625f1140a84f5b4d

valgrind trunk snapshot, manual and doc (pdf) downloads:
https://snapshots.sourceware.org/valgrind/trunk/
https://sourceware.org/cgit/builder/commit/?id=f338a430db0382d9bd26b447cfe762566c9e7935

The idea is the same:
- Pick or create a new container file (add any build dependencies)
- Add a Periodic for the project (trigger every 15 minutes, checking
  for any changes on a specific branch)
- Create a BuildFactory with build steps (ideally existing make
  targets or a build script in the repo) which put any artifacts under
  the output dir and create a publish file containing the intended
  (branch/topic) name.
- Add a BuildConfig using the container file and build factory.

Cheers,

Mark