From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by sourceware.org (Postfix) with ESMTPS id 788273858C78; Thu, 16 Nov 2023 18:37:31 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 788273858C78 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=suse.cz ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 788273858C78 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=195.135.220.28 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1700159852; cv=none; b=U3DL0UEBEq62q6hXXTcfCzxcK50eOhF+VvZ2qMM8eZvEYNr2AxstFNuTDDu0L2nacCBQ/TePan8WlXevlenJLLlbhdEQluHdc+4fGgie/oUCBFv8ez/0/c/JHqFUZIrgxRKShoGwvsRdHj/+PXRcFLkhHGqWcQmXUH/IedHUcb4= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1700159852; c=relaxed/simple; bh=6Ui8SJZPquXNB77fY8tHKEXPZz320YUx+3pVS/JijJE=; h=DKIM-Signature:DKIM-Signature:From:To:Subject:Date:Message-ID: MIME-Version; b=Vtk/RycE4tLYrFjrnlXK8krrNMzvSYX/oY02F3RY/wUobNsGwiHBkwipY94HoMwgaE5xLBwPjf1za0ywGYDeyfsCkMNDJdj2Rd2TAL5ucRRfFTg/N8IQ+Va8UGAxNOQjHA044s8uPByMBbaE+MnWoYgGV3am1vfqLzYKlkpBgL0= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 52E922293D; Thu, 16 Nov 2023 18:37:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1700159850; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QsyTP5HIC9Z89nmzBTz7RqbcqdLuxV1yfosqMYBDVYU=; b=RDL0p/XRaEQvXHXytmkMaAau77ZSDNDW9sYlmU1tMNwC2c948dzh1iFH1Y1UzF6Fw7wCbY k8LdJh5LQNTQbWn9B4h36m+gX/SgP2zQdcds1frzozmGXzH62werCG74emYJLV99nP8lyW 9X4TeBdShp/IJmKr7nb1vQXyv0uk7x4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1700159850; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QsyTP5HIC9Z89nmzBTz7RqbcqdLuxV1yfosqMYBDVYU=; b=VfQsg3GXuWgUeTHVUmCZopTF8TGKBB26ht2ciWnJbshJSsPpVlglirMddkwTiMjVR/zErd sHGy35IvIkLTUMAg== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 439EC139C4; Thu, 16 Nov 2023 18:37:30 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id UxN8EGphVmWYPAAAMHmgww (envelope-from ); Thu, 16 Nov 2023 18:37:30 +0000 From: Martin Jambor To: Mark Wielaard Cc: GCC Mailing List , Maxim Kuvyrkov , binutils@sourceware.org, gdb@sourceware.org Subject: Re: Checks that autotools generated files were re-generated correctly In-Reply-To: <20231115194803.GW31613@gnu.wildebeest.org> References: <20231108233059.GA31613@gnu.wildebeest.org> <20231115194803.GW31613@gnu.wildebeest.org> User-Agent: Notmuch/0.37 (https://notmuchmail.org) Emacs/29.1 (x86_64-suse-linux-gnu) Date: Thu, 16 Nov 2023 19:37:29 +0100 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Authentication-Results: smtp-out1.suse.de; none X-Spam-Level: X-Spam-Score: -6.10 X-Spamd-Result: default: False [-6.10 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; REPLY(-4.00)[]; RCPT_COUNT_FIVE(0.00)[5]; DKIM_SIGNED(0.00)[suse.cz:s=susede2_rsa,suse.cz:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; INVALID_MSGID(1.70)[]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-3.00)[100.00%] X-Spam-Status: No, score=-5.5 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,INVALID_MSGID,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Hi, On Wed, Nov 15 2023, Mark Wielaard wrote: > Hi! (adding gdb and binutils to the CC) > > On Thu, Nov 09, 2023 at 12:30:59AM +0100, Mark Wielaard wrote: >> On Mon, Nov 06, 2023 at 06:04:48PM +0100, Martin Jambor wrote: >> > I have inherited Martin Li=C5=A1ka's buildbot script that checks that = all >> > sorts of autotools generated files, mainly configure scripts, were >> > re-generated correctly when appropriate. While the checks are hopeful= ly >> > useful, they report issues surprisingly often and reporting them feels >> > especially unproductive. >>=20 >> Cool! A small python script cannot replace him of course. But it is >> nice to get a small virtual mliska :) >>=20 >> > Could such checks be added to our server side push hooks so that commi= ts >> > introducing these breakages would get refused automatically. While the >> > check might be a bit expensive, it only needs to be run on files >> > touching the generated files and/or the files these are generated from. >>=20 >> So this doesn't just need that script, but also an execution >> environment that contains the right versions of the autotools. We >> could install those of course, but running them from a git hook on >> checkin indeed seems a little expensive. >>=20 >> Creating a container with the script and the right versions of all >> tools might be the best thing. Then the script can be run from a cron >> job or buildbot. Or even by someone hacking on the build/configure >> scripts to make sure they are generating with the right tools. >>=20 >> builder.sourceware.org already contains various of such containers: >> https://sourceware.org/cgit/builder/tree/builder/containers >> https://sourceware.org/cgit/builder/tree/README_containers >>=20 >> Friday is Sourceware Open Office hour (#overseers on irc.libera.chat >> at 18:00 UTC). We could hack something together then and see how to >> hook it up. > > So we did indeed discuss and hack something together. Wonderful, thanks to everyone working to improve the situation with these errors. Martin > > The container file is here: > https://sourceware.org/cgit/builder/tree/builder/containers/Containerfile= -autotools > > The script is here: > https://sourceware.org/cgit/builder/tree/builder/containers/autoregen.py > > A buildbot can then use that container to run that script and then > check that git diff is empty on every commit to binutils-gdb.git and > gcc.git. If it finds an issue it will sent email with the diff. > > It already found issues in both gcc and binutils-gdb. All fixed now. > > Thanks to Arsen and Sam for testing, fixing and updating the script > (it now also runs aclocal). > > If you want to run this locally you can use the container file to > create an image and then run autoregen.py on your local tree by bind > mounting your git tree inside it. > > $ git clone https://sourceware.org/git/builder.git > $ cd builder/ > $ podman build -t autotools -f builder/containers/Containerfile-autotools= \ > builder/containers > $ cd .. # assuming your binutils-gdb directory is here > $ podman run --privileged -u root -v $(pwd)/binutils-gdb:/binutils-gdb \ > -ti --entrypoint "/bin/bash" autotools > # then inside the container > cd /binutils-gdb > autoregen.py > > Change binutils-gdb to gcc if you are working on gcc. > You should also be able to do the above with docker instead of podman. > > Cheers, > > Mark