From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gnu.wildebeest.org (gnu.wildebeest.org [45.83.234.184]) by sourceware.org (Postfix) with ESMTPS id 12E763858D32 for ; Wed, 15 Nov 2023 19:48:04 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 12E763858D32 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=klomp.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=klomp.org ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 12E763858D32 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=45.83.234.184 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1700077685; cv=none; b=ikjFnhiGd9iXfvAOkNJLcmmCrRTj9VFpLs+3eo7IyK7HfyI/g5BouJg9Vkj20/L4TrLzKLo25cf57fCGB/QrZJmq8ZwPT7LBJSHeFK1+FkW3MhmkBBZSR+MPnhg1NyKGN6fb0TZ8r8yPF2v+YsigUZhJQNfcMbkN6DjzYiEDyD4= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1700077685; c=relaxed/simple; bh=Eu7WGoEKxUt1SYlOdGpbTOQObhRjFwG8oo4Cd7wPAHQ=; h=Date:From:To:Subject:Message-ID:MIME-Version; b=eUV8/9BIWzekHQ/pXI75qNE+289pZNBuNhqVP4LS594PtFGdDOaqT63H0Ewg4DXNT169BQFqAitVeyJa4ZsHRX4uW4GrJW9VtOK6bZ3aFhxDwvQDYjaS5W1fZ+MSwz9yMgQCv0TB4BW6zt9BTm95D0snR1AYoZuO4AENbM8YWRU= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by gnu.wildebeest.org (Postfix, from userid 1000) id 20701302FDDB; Wed, 15 Nov 2023 20:48:03 +0100 (CET) Date: Wed, 15 Nov 2023 20:48:03 +0100 From: Mark Wielaard To: Martin Jambor Cc: GCC Mailing List , Maxim Kuvyrkov , binutils@sourceware.org, gdb@sourceware.org Subject: Re: Checks that autotools generated files were re-generated correctly Message-ID: <20231115194803.GW31613@gnu.wildebeest.org> References: <20231108233059.GA31613@gnu.wildebeest.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20231108233059.GA31613@gnu.wildebeest.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-3028.0 required=5.0 tests=BAYES_00,JMQ_SPF_NEUTRAL,KAM_DMARC_STATUS,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Hi! (adding gdb and binutils to the CC) On Thu, Nov 09, 2023 at 12:30:59AM +0100, Mark Wielaard wrote: > On Mon, Nov 06, 2023 at 06:04:48PM +0100, Martin Jambor wrote: > > I have inherited Martin Liška's buildbot script that checks that all > > sorts of autotools generated files, mainly configure scripts, were > > re-generated correctly when appropriate. While the checks are hopefully > > useful, they report issues surprisingly often and reporting them feels > > especially unproductive. > > Cool! A small python script cannot replace him of course. But it is > nice to get a small virtual mliska :) > > > Could such checks be added to our server side push hooks so that commits > > introducing these breakages would get refused automatically. While the > > check might be a bit expensive, it only needs to be run on files > > touching the generated files and/or the files these are generated from. > > So this doesn't just need that script, but also an execution > environment that contains the right versions of the autotools. We > could install those of course, but running them from a git hook on > checkin indeed seems a little expensive. > > Creating a container with the script and the right versions of all > tools might be the best thing. Then the script can be run from a cron > job or buildbot. Or even by someone hacking on the build/configure > scripts to make sure they are generating with the right tools. > > builder.sourceware.org already contains various of such containers: > https://sourceware.org/cgit/builder/tree/builder/containers > https://sourceware.org/cgit/builder/tree/README_containers > > Friday is Sourceware Open Office hour (#overseers on irc.libera.chat > at 18:00 UTC). We could hack something together then and see how to > hook it up. So we did indeed discuss and hack something together. The container file is here: https://sourceware.org/cgit/builder/tree/builder/containers/Containerfile-autotools The script is here: https://sourceware.org/cgit/builder/tree/builder/containers/autoregen.py A buildbot can then use that container to run that script and then check that git diff is empty on every commit to binutils-gdb.git and gcc.git. If it finds an issue it will sent email with the diff. It already found issues in both gcc and binutils-gdb. All fixed now. Thanks to Arsen and Sam for testing, fixing and updating the script (it now also runs aclocal). If you want to run this locally you can use the container file to create an image and then run autoregen.py on your local tree by bind mounting your git tree inside it. $ git clone https://sourceware.org/git/builder.git $ cd builder/ $ podman build -t autotools -f builder/containers/Containerfile-autotools \ builder/containers $ cd .. # assuming your binutils-gdb directory is here $ podman run --privileged -u root -v $(pwd)/binutils-gdb:/binutils-gdb \ -ti --entrypoint "/bin/bash" autotools # then inside the container cd /binutils-gdb autoregen.py Change binutils-gdb to gcc if you are working on gcc. You should also be able to do the above with docker instead of podman. Cheers, Mark