From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from eggs.gnu.org (eggs.gnu.org [IPv6:2001:470:142:3::10]) by sourceware.org (Postfix) with ESMTPS id F19F23858D1E for ; Sun, 16 Apr 2023 19:53:52 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org F19F23858D1E Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gnu.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gnu.org Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1po8RZ-0006AV-Hj; Sun, 16 Apr 2023 15:53:51 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=1nWrRB3078RJnPbj3OQP/bWUXjcrc62yX714jlr4hhQ=; b=p8McJAIeBc9UrJE/Zt/h k205Qq5iF654dz6M8iGUtVs9dBVTqguXkKiz69f6U+4YMIjXij/gt6YnFwF5kENf6594+UIG3MeVb GeWVRbhKKgBip4KQRLHrDYKbeDNv4jiNSrtcz1epu//h+KBryW7qB1Ck61m1nKYKlv0lON3n6pfIM vdoAxtOSEbnvEfYaAZWTnL8R1bXFBQ6LkDEZSiXtGoaTxXBDT1zTJ7WbgYYzo7yQmR6r9J4I3mRE+ ZQMKC5QSfZAAO/BPPlUVRWuipSMq4gNJJc2EJsfJZCMja1Uke2EbH4a6czbqFFTiex0vXtY9gip2S 07cCl8+ww5i0Uw==; Received: from dynamic-077-181-025-164.77.181.pool.telefonica.de ([77.181.25.164] helo=termi) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1po8RZ-0005GU-3X; Sun, 16 Apr 2023 15:53:49 -0400 From: "Jose E. Marchesi" To: Mark Wielaard Cc: buildbot@sourceware.org Subject: Re: [PATCH] Add gnupoke snapshot scheduler, factory and builder In-Reply-To: <20230416190515.248478-1-mark@klomp.org> (Mark Wielaard's message of "Sun, 16 Apr 2023 21:05:15 +0200") References: <20230416190515.248478-1-mark@klomp.org> Date: Sun, 16 Apr 2023 21:53:34 +0200 Message-ID: <87leirpps1.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Status: No, score=-10.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,GIT_PATCH_0,SPF_HELO_PASS,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Thanks Mark! :) This and the Fedora ARM support means I should add new little graphics to http://jemarch.net/poke? > --- > .../containers/Containerfile-fedora-latest | 1 + > builder/master.cfg | 73 ++++++++++++++++++- > 2 files changed, 70 insertions(+), 4 deletions(-) > > diff --git a/builder/containers/Containerfile-fedora-latest b/builder/containers/Containerfile-fedora-latest > index e4016af..4fa217b 100644 > --- a/builder/containers/Containerfile-fedora-latest > +++ b/builder/containers/Containerfile-fedora-latest > @@ -13,6 +13,7 @@ RUN dnf upgrade -y && \ > jq json-c-devel socat \ > texinfo elfutils-debuginfod elfutils-debuginfod-client-devel \ > gcc-plugin-devel binutils-devel \ > + libnbd-devel gc-devel readline-devel texinfo-tex \ > xmlto xmlto-tex fop poppler-utils libxslt docbook-dtds docbook-style-xsl \ > libxml2-devel dos2unix dpkg koji python3-devel python3-koji python3-mock \ > python3-rpm python3-pyxdg mailcap openmpi-devel gdb help2man wget xxhash-devel \ > diff --git a/builder/master.cfg b/builder/master.cfg > index 7564f58..fecdc76 100644 > --- a/builder/master.cfg > +++ b/builder/master.cfg > @@ -349,6 +349,15 @@ def master_and_users_try_branches(ref): > return True > return False > > +# For git maint branches (as gnu poke uses) > +def master_and_maint_branches(ref): > + if ref == "refs/heads/master": > + return True > + match = re.match ("refs/heads/maint/poke-[a-zA-Z0-9_-]+", ref) > + if match: > + return True > + return False > + > sourceware_repourl='https://sourceware.org/git/builder.git' > sourceware_gitpoller = changes.GitPoller(repourl=sourceware_repourl, > branches=['main'], > @@ -452,7 +461,7 @@ c['change_source'].append(glibc_gitpoller) > > gnupoke_repourl='https://git.savannah.gnu.org/git/poke.git' > gnupoke_gitpoller = changes.GitPoller(repourl=gnupoke_repourl, > - branches=['master'], > + branches=master_and_maint_branches, > pollInterval=3*60, > pollRandomDelayMax=2*60, > project='gnupoke') > @@ -982,6 +991,16 @@ gnupoke_scheduler = schedulers.SingleBranchScheduler( > "gnupoke-gentoo-sparc"]) > c['schedulers'].append(gnupoke_scheduler) > > +gnupoke_snapshot_scheduler = schedulers.Periodic( > + name="gnupoke-snapshots", > + change_filter=util.ChangeFilter(project="gnupoke", > + branch_re=re.compile("master$|poke-[a-zA-Z0-9_-]+$")), > + periodicBuildTimer=15*60, # 15 minutes in seconds > + onlyIfChanged=True, > + reason="gnupoke periodic project master/maint branch snapshot", > + builderNames=["poke-snapshots-html-pdf-dist"]) > +c['schedulers'].append(gnupoke_snapshot_scheduler) > + > annobin_scheduler = schedulers.SingleBranchScheduler( > name="annobin", > change_filter=util.ChangeFilter(project="annobin", > @@ -1082,6 +1101,12 @@ make_step = steps.Compile( > util.Interpolate('-j%(prop:ncpus)s')]), > name="make", > haltOnFailure=True) > +make_html_step = steps.Compile( > + command=["make", "html"], > + name="make html") > +make_pdf_step = steps.Compile( > + command=["make", "pdf"], > + name="make pdf") > make_check_step = steps.Test( > command=addOutputSync.withArgs(["make", "check", > util.Interpolate('-j%(prop:ncpus)s')]), > @@ -1749,6 +1774,11 @@ elfutils_create_publish_file_step = steps.ShellCommand( > name="create publish file", > command="echo elfutils/coverage > /home/builder/shared/publish") > > +# publish file based on branch name, cut away all but last / element > +poke_create_publish_file_step = steps.ShellCommand( > + name="create publish file", > + command=util.Interpolate('(echo -n gnupoke/; echo %(prop:branch)s | rev | cut -f1 -d/ | rev | sed -e s/master/trunk/) > /home/builder/shared/publish')) > + > # Upload automake test results, autoconf config.log, in both build and dist-build trees > elfutils_upload_to_bunsen = bunsen_logfile_upload_cpio_steps( > ["*.trs", "*.log"]) > @@ -4275,13 +4305,25 @@ systemtap_gentoo_sparc_builder = util.BuilderConfig( > c['builders'].append(systemtap_gentoo_sparc_builder) > > # GNU poke builders and factories > -gnupoke_factory = util.BuildFactory() > -gnupoke_factory.addStep(steps.Git( > + > +# publish file based on branch name, cut away all but last / element > +poke_create_publish_file_step = steps.ShellCommand( > + name="create publish file", > + command=util.Interpolate('(echo -n gnupoke/; echo %(prop:branch)s | rev | cut -f1 -d/ | rev | sed -e s/master/trunk/) > /home/builder/shared/publish')) > + > +poke_create_output_step = steps.ShellCommand( > + name="create output", > + command="mkdir /home/builder/shared/output && mv ./poke-*.tar.gz /home/builder/shared/output/ && mv ./doc/poke.html /home/builder/shared/output/html && mv ./doc/poke.pdf /home/builder/shared/output/") > + > +poke_git_step = steps.Git( > repourl=gnupoke_repourl, > mode='full', method='fresh', > name="git checkout", > getDescription={'tags':True, 'always':True}, > - haltOnFailure=True)) > + haltOnFailure=True) > + > +gnupoke_factory = util.BuildFactory() > +gnupoke_factory.addStep(poke_git_step) > gnupoke_factory.addStep(bootstrap_step) > gnupoke_factory.addStep(configure_step) > gnupoke_factory.addStep(make_step) > @@ -4291,6 +4333,19 @@ gnupoke_factory.addStep(steps.Test(command=["make", "syntax-check"], > haltOnFailure=False, flunkOnFailure=True)) > gnupoke_factory.addSteps(bunsen_logfile_upload_cpio_steps(["*.sum", "*.log"])) > > +# snapshots > +gnupoke_snapshots_factory = util.BuildFactory() > +gnupoke_snapshots_factory.addStep(poke_git_step) > +gnupoke_snapshots_factory.addStep(bootstrap_step) > +gnupoke_snapshots_factory.addStep(configure_step) > +gnupoke_snapshots_factory.addStep(make_step) > +gnupoke_snapshots_factory.addStep(make_html_step) > +gnupoke_snapshots_factory.addStep(make_pdf_step) > +gnupoke_snapshots_factory.addStep(make_dist_step) > +gnupoke_snapshots_factory.addStep(wait_snapshots_output_ready_step) > +gnupoke_snapshots_factory.addStep(poke_create_output_step) > +gnupoke_snapshots_factory.addStep(poke_create_publish_file_step) > + > gnupoke_debiantesting_x86_64_builder = util.BuilderConfig( > name="gnupoke-debian-testing-x86_64", > tags=["gnupoke", "debian-testing", "x86_64"], > @@ -4314,6 +4369,16 @@ gnupoke_gentoo_space_builder = util.BuilderConfig( > factory=gnupoke_factory) > c['builders'].append(gnupoke_gentoo_space_builder) > > +poke_snapshots_builder = util.BuilderConfig( > + name="poke-snapshots-html-pdf-dist", > + collapseRequests=True, > + properties={'container-file': > + readContainerFile('fedora-latest')}, > + workernames="snapshots", > + tags=["gnupoke-snapshots"], > + factory=gnupoke_snapshots_factory) > +c['builders'].append(poke_snapshots_builder) > + > # annobin steps, builders and factories > > # annobin checkouts have generated config files included, but might