From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi1-x22c.google.com (mail-oi1-x22c.google.com [IPv6:2607:f8b0:4864:20::22c]) by sourceware.org (Postfix) with ESMTPS id A37CD3849AE7 for ; Fri, 19 Apr 2024 09:09:07 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org A37CD3849AE7 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=linaro.org ARC-Filter: OpenARC Filter v1.0.0 sourceware.org A37CD3849AE7 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::22c ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1713517750; cv=none; b=al70f3wFSelEM9U9rAZUJRsedZd1tc9mYtcoDmFIhq5g03px1zTEIHv72DNZG81t61DiZR2ljotmfxGC/gT3Cc1kvVrQ+Re9n4dRo5t7tY4iSrGhe98VnGu0nTpUeT3cOIqUEAe0A1AA7AO1Up4bG8Ol3iiyNyaDiOY1w0CT5+I= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1713517750; c=relaxed/simple; bh=QWKlvnBVsh1tq7O5lNWW46f/bGDblKSQKvwTxwn4SbM=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=FB/pTNIKmitbbMfNcFHXIARZxdGDWabTy1kFZTjYBk2FYHZsH7O+bp7dzzFJ0MuXXurjWIg0XvwITObOFkZkXTxU7jhIzcChQvY7C35WkYDJrfqhmRpTZLfEecLnMnJLdeStIE9iSowt+8tTFsIlSdM3uQxWnybtF4eNLCwERro= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-oi1-x22c.google.com with SMTP id 5614622812f47-3c709e5e4f9so1205034b6e.3 for ; Fri, 19 Apr 2024 02:09:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1713517746; x=1714122546; darn=sourceware.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=B2KgoRtTgOav/ibjlAPndwH6Tnv36M64CPIqoQPujSE=; b=luRn/qiCuJdzzRwTnnWcdJMFGVpyvrlMAJT3tLqC+IgjlfbxGM0sN5hxFYX8UOKyUq 3YRzTYF6m5NubGckydDPXaZivhWwiatkGSnNT0E1u6SSweqIkVvqnXjSzFl9F4rkkDCN ikI1LjSlKE3lhSo1pK/DHUFgYwCJmCRMmZWmQZRbTaf4olzVCUc1A30b6nomYiZ/5BTb zJALdI7xYLyE2nCquOEG1WtQbrPCLlT0V4fEoMuHyZu6c59uLwoulSQAkY/fgMJ6FKzr wJA06Mc6WflqPwP8rN1FSuIZdv9aG8yknya58scRJNH7dixhLvqkUZMRhoIfd4vfaFIm HmVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713517746; x=1714122546; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=B2KgoRtTgOav/ibjlAPndwH6Tnv36M64CPIqoQPujSE=; b=Jm8YXNQ9VZFXP7hbj1EnIGCqzF4zURSmg/qyYsqkFIy3TBnv+owNJdnVB9Bx+Cw4UC QaKlsDSrwp1tz7CzyU5JDxUl+BGNG5KdgEAs4t265VN/XOByM691hZODWHdZgtBdTJex c+2+XF7dzjnMy/kumNQBtZTIfioUvbSO2tewJstmpLsWzIATawZ/HHxHtnXD8hcMoszW ODT24euCG8HmM+ThGA8G8g8vlMs0ZAlUm2fESZXFXqT2eEr/myPM418gIrTY6Wz/D3GK W8E+I1Ux+LP/XGMEOjjS0Dg9+aegxpD+SW/WjSp/tvne0Xn2YrY9Gf1Z0aFm2yu3Nz0u H7iA== X-Gm-Message-State: AOJu0YzxHS9eCUU9s0h/LxkLP2e64kvbpoTxQc5PJGdckjOvWUzS7WhX 0IBJ1x50Mtog02bQJpSt9iHtoxYOykUCOINDT3v98m9UY45ZLm3DzUOSpA1RghAqt8l1zfkKPmr yqfczFQ== X-Google-Smtp-Source: AGHT+IHdDpHx8v5k55tb/qJ1kT52E8XkEY9w//q6Pv2RKrgoCTv1eOel5p4BIerQOOd7ystM2j/mwQ== X-Received: by 2002:a05:6808:1190:b0:3c7:4b9c:122 with SMTP id j16-20020a056808119000b003c74b9c0122mr442840oil.25.1713517746181; Fri, 19 Apr 2024 02:09:06 -0700 (PDT) Received: from localhost.localdomain ([139.178.84.207]) by smtp.gmail.com with ESMTPSA id ct3-20020a056808360300b003c5db0dabb7sm539573oib.40.2024.04.19.02.09.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 19 Apr 2024 02:09:05 -0700 (PDT) From: Christophe Lyon To: gdb@sourceware.org Cc: Christophe Lyon Subject: [PATCH] contrib: Add autoregen.py Date: Fri, 19 Apr 2024 09:09:03 +0000 Message-Id: <20240419090903.2080043-1-christophe.lyon@linaro.org> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-12.6 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,GIT_PATCH_0,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: This script is a copy of the current script used by Sourceware's autoregen buildbots. It is intended as a helper to regenerate files managed by autotools (autoconf, automake, aclocal, ....), as well as the toplevel Makefile.in which is created by autogen. Other files can be updated when using maintainer-mode, but this is not covered by this script. 2024-04-19 Christophe Lyon contrib/ * autoregen.py: New script. --- contrib/autoregen.py | 221 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 221 insertions(+) create mode 100755 contrib/autoregen.py diff --git a/contrib/autoregen.py b/contrib/autoregen.py new file mode 100755 index 00000000000..faffc88c5bd --- /dev/null +++ b/contrib/autoregen.py @@ -0,0 +1,221 @@ +#!/usr/bin/env python3 + +# This script helps to regenerate files managed by autotools and +# autogen in binutils-gdb and gcc repositories. + +# It can be used by buildbots to check that the current repository +# contents has been updated correctly, and by developers to update +# such files as expected. + +import os +import shutil +import subprocess +from pathlib import Path + + +# On Gentoo, vanilla unpatched autotools are packaged separately. +# We place the vanilla names first as we want to prefer those if both exist. +AUTOCONF_NAMES = ["autoconf-vanilla-2.69", "autoconf-2.69", "autoconf"] +AUTOMAKE_NAMES = ["automake-vanilla-1.15", "automake-1.15.1", "automake"] +ACLOCAL_NAMES = ["aclocal-vanilla-1.15", "aclocal-1.15.1", "aclocal"] +AUTOHEADER_NAMES = ["autoheader-vanilla-2.69", "autoheader-2.69", "autoheader"] +AUTORECONF_NAMES = ["autoreconf-vanilla-2.69", "autoreconf-2.69", "autoreconf"] + +# Pick the first for each list that exists on this system. +AUTOCONF_BIN = next(name for name in AUTOCONF_NAMES if shutil.which(name)) +AUTOMAKE_BIN = next(name for name in AUTOMAKE_NAMES if shutil.which(name)) +ACLOCAL_BIN = next(name for name in ACLOCAL_NAMES if shutil.which(name)) +AUTOHEADER_BIN = next(name for name in AUTOHEADER_NAMES if shutil.which(name)) +AUTORECONF_BIN = next(name for name in AUTORECONF_NAMES if shutil.which(name)) + +AUTOGEN_BIN = "autogen" + +# autoconf-wrapper and automake-wrapper from Gentoo look at this environment variable. +# It's harmless to set it on other systems though. +EXTRA_ENV = { + "WANT_AUTOCONF": AUTOCONF_BIN.split("-", 1)[1] if "-" in AUTOCONF_BIN else "", + "WANT_AUTOMAKE": AUTOMAKE_BIN.split("-", 1)[1] if "-" in AUTOMAKE_BIN else "", + "AUTOCONF": AUTOCONF_BIN, + "ACLOCAL": ACLOCAL_BIN, + "AUTOMAKE": AUTOMAKE_BIN, + "AUTOGEN": AUTOGEN_BIN, +} +ENV = os.environ.copy() +ENV.update(EXTRA_ENV) + + +# Directories we should skip entirely because they're vendored or have different +# autotools versions. +SKIP_DIRS = [ + # readline and minizip are maintained with different autotools versions + "readline", + "minizip", +] + +# these directories are known to be re-generatable with a simple autoreconf +# without special -I flags +# Entries commented out (and directories not listed) are handled by +# regenerate_manually(). +AUTORECONF_DIRS = [ + # subdirs common to binutils-gdb and gcc + "libbacktrace", + "libdecnumber", # No Makefile.am + "libiberty", # No Makefile.am + "zlib", + + # binutils-gdb subdirs + "bfd", + "binutils", + "etc", + "gas", + "gdb", + "gdbserver", + "gdbsupport", + "gnulib", + "gold", + "gprof", + "gprofng", + "gprofng/libcollector", + "ld", + "libctf", + "libsframe", + "opcodes", + "sim", + + # gcc subdirs + "c++tools", # No aclocal.m4 + "gcc", # No Makefile.am + #"fixincludes", # autoreconf complains about GCC_AC_FUNC_MMAP_BLACKLIST + "gnattools", # No aclocal.m4 + "gotools", + "libada", # No aclocal.m4 + "libatomic", + "libcc1", + "libcody", # No aclocal.m4 + "libcpp", # No Makefile.am + "libffi", + "libgcc", # No aclocal.m4 + "libgfortran", + # Hack: ACLOCAL_AMFLAGS = -I .. -I ../config in Makefile.in but we + # apply -I../config -I.. otherwise we do not match the current + # contents + #"libgm2", + "libgo", + "libgomp", + "libgrust", + "libitm", + "libobjc", # No Makefile.am + "libphobos", + "libquadmath", + "libsanitizer", + "libssp", + "libstdc++-v3", + # This does not cover libvtv/testsuite/other-tests/Makefile.in + "libvtv", + "lto-plugin", +] + + +# Run the shell command CMD. +# +# Print the command on stdout prior to running it. +def run_shell(cmd: str): + print(f"+ {cmd}", flush=True) + res = subprocess.run( + f"{cmd}", + shell=True, + encoding="utf8", + env=ENV, + ) + res.check_returncode() + + +def regenerate_with_autoreconf(): + run_shell(f"{AUTORECONF_BIN} -f") + +def regenerate_with_autogen(): + run_shell(f"{AUTOGEN_BIN} Makefile.def") + +def regenerate_manually(): + configure_lines = open("configure.ac").read().splitlines() + if folder.stem == "fixincludes" or folder.stem == "libgm2" or any( + True for line in configure_lines if line.startswith("AC_CONFIG_MACRO_DIR") + ): + include_arg = "" + include_arg2 = "" + if (folder / ".." / "config").is_dir(): + include_arg = "-I../config" + + # this is really a hack just for binutils-gdb/gprofng/libcollector + # make sure that the order of includes is done as --enable-maintainer-mode + if (folder / ".." / ".." / "config").is_dir(): + include_arg = "-I../.." + include_arg2 = "-I../../config" + + if folder.stem == "fixincludes": + include_arg = "-I.." + include_arg2 = "-I../config" + + if folder.stem == "libgm2": + include_arg = "-I../config" + include_arg2 = "-I.." + + # aclocal does not support the -f short option for force + run_shell(f"{ACLOCAL_BIN} --force {include_arg} {include_arg2}") + + if (folder / "config.in").is_file() or any( + True for line in configure_lines if line.startswith("AC_CONFIG_HEADERS") + ): + run_shell(f"{AUTOHEADER_BIN} -f") + + # The few lines below do not regenerate the exact same content as + # currently in the repo. Disable them for now. + # if (folder / "gm2-libs").is_dir(): + # run_shell(f"{AUTOCONF_BIN} -f gm2-libs/config-host.in > gm2-libs/config-host") + # run_shell(f"{AUTOHEADER_BIN} gm2-libs/config-host.in") + + + # apparently automake is somehow unstable -> skip it for gotools + if any( + True for line in configure_lines if line.startswith("AM_INIT_AUTOMAKE") + ) and not str(folder).endswith("gotools"): + run_shell(f"{AUTOMAKE_BIN} -f") + + run_shell(f"{AUTOCONF_BIN} -f") + + +run_shell(f"{AUTOCONF_BIN} --version") +run_shell(f"{AUTOMAKE_BIN} --version") +run_shell(f"{ACLOCAL_BIN} --version") +run_shell(f"{AUTOHEADER_BIN} --version") + +print(f"Extra environment: {EXTRA_ENV}", flush=True) + +config_folders: list[Path] = [] +autogen_folders: list[Path] = [] +repo_root = Path.cwd() + +for root, _, files in os.walk("."): + for file in files: + if file == "configure.ac": + config_folders.append(Path(root).resolve()) + if file == "Makefile.tpl": + autogen_folders.append(Path(root).resolve()) + +for folder in sorted(autogen_folders): + print(f"Entering directory {folder}", flush=True) + os.chdir(folder) + regenerate_with_autogen() + +for folder in sorted(config_folders): + if folder.stem in SKIP_DIRS: + print(f"Skipping directory {folder}", flush=True) + continue + + print(f"Entering directory {folder}", flush=True) + os.chdir(folder) + + if str(folder.relative_to(repo_root)) in AUTORECONF_DIRS: + regenerate_with_autoreconf() + else: + regenerate_manually() -- 2.34.1