From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oo1-xc32.google.com (mail-oo1-xc32.google.com [IPv6:2607:f8b0:4864:20::c32]) by sourceware.org (Postfix) with ESMTPS id 363AF384AB78 for ; Fri, 19 Apr 2024 09:08:17 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 363AF384AB78 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=linaro.org ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 363AF384AB78 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::c32 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1713517709; cv=none; b=FfGz2Wsj0+0o3QqoY78r2OOShp2z9dEdyyEVqaxIoCD5QjRmEKwYhg4izhsNlnC1057LCfozn/ifjyw3wm3wLOOh9ZG+Nmv9F/A50E9zmJyos/grQPXK/8O1AYuYYruQBGcmQkQqc8qvqhPBxa4SUKLbUYMnvYgKMCsmIGUp1PA= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1713517709; c=relaxed/simple; bh=QWKlvnBVsh1tq7O5lNWW46f/bGDblKSQKvwTxwn4SbM=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=rmYSrT0nr++PhXkaM+Ym6gAjtKbLc0H8bnX3RoJSyGx1Rher1EztJOuvOE9wpS6GJE805nvpI9y4Ggqec5Hkg3IBtXgU5FBB/lYtP4wyQEa/ZQ1ZbOTBxVCYnW7VrAupPL7XsQtBTv0ZaEhUYjL8WQwK7PUPN9dVMahAe7sCB7g= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-oo1-xc32.google.com with SMTP id 006d021491bc7-5ac90ad396dso943568eaf.3 for ; Fri, 19 Apr 2024 02:08:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1713517696; x=1714122496; darn=sourceware.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=B2KgoRtTgOav/ibjlAPndwH6Tnv36M64CPIqoQPujSE=; b=Zb+1n+CUC+PSLvEaYBz9TUSjSUwB2Yn5LBwoGA1yODILXrRXpRLbk/H+8UqKUKA19q rQylL1CP34GRcd+NAH6XXSfU/G7JdVifH14c/4zfCZZqJ5iqGsxe7XP/tq0+D5AtxO4x pVUxSgCK8SBBMXy8i8e97HYH7dX7UHnZFjosb9/CpOT39QuHrDE3SIBu5bSwG3b1KaBp x//9fYwrWd91V254y15soo2OaNBHrZmq6HCl2jvAB5hwL97XyXpHNT/U/21n+OgrByVj JJ22HE3Adr+JC5gwe44JDKcpHOH5sKnfm+mjpUAQ1rSeVPc7VZSNfCOAaRPluC8MVeUA 04/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713517696; x=1714122496; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=B2KgoRtTgOav/ibjlAPndwH6Tnv36M64CPIqoQPujSE=; b=TsDLiLtneZbItG18cgW4HLZtUZgFLCV28VOllVupYrcz0i4xZh3n/Xa6YcXAl/3GMM CIb6s7jsrrr5wTzTtxOFJog61liC9KcJ6gq+9mUBbo9D5kK5GPGDDEMFoTUFm6XOHbz5 OK0Uj2ASn8u6pvjkx0Xy6ezE3Ik5+HkqARv7tRlRXjFF6MIGRj4ZD7bbmbSyDKuH7Uhb sfzrmOBgiP1So/zowBj3YjqQZHbYCs2a/v4KRATsvVxiI3PcjtKmO1T1kUxVaHgHjcnz vkolMfs9vEg8lpp0K/Dp5LwuqtMaw/QXHgxPj++tSrUtSCfxHmvYynAZPaZeMYN6/lEw kH4g== X-Gm-Message-State: AOJu0YzodoebjWDYWLz6v4uCIiaqxSPcETKTfZcoYYI2e01JgFGX4PjH LoDKC8ORKQLm/nlcl4JdvQkTZ84xMz+0r/mLehVec8nVzP0P81NaNYbiWupnXvOZkC7A44clD8J y79WhFw== X-Google-Smtp-Source: AGHT+IF4uIfARv910mlpGfq9C0qHke9SQWUHEEP78Yec/nlrstAutr+sPTvACb0wjgGghMIoMCeThA== X-Received: by 2002:a4a:aa8c:0:b0:5ac:5c78:390e with SMTP id d12-20020a4aaa8c000000b005ac5c78390emr1621279oon.2.1713517695750; Fri, 19 Apr 2024 02:08:15 -0700 (PDT) Received: from localhost.localdomain ([139.178.84.207]) by smtp.gmail.com with ESMTPSA id v3-20020a4a8c43000000b005a21ec7b19fsm775498ooj.3.2024.04.19.02.08.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 19 Apr 2024 02:08:15 -0700 (PDT) From: Christophe Lyon To: binutils@sourceware.org, gdb@sourcewware.org Cc: Christophe Lyon Subject: [PATCH] contrib: Add autoregen.py Date: Fri, 19 Apr 2024 09:08:06 +0000 Message-Id: <20240419090806.2080022-1-christophe.lyon@linaro.org> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-12.6 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,GIT_PATCH_0,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: This script is a copy of the current script used by Sourceware's autoregen buildbots. It is intended as a helper to regenerate files managed by autotools (autoconf, automake, aclocal, ....), as well as the toplevel Makefile.in which is created by autogen. Other files can be updated when using maintainer-mode, but this is not covered by this script. 2024-04-19 Christophe Lyon contrib/ * autoregen.py: New script. --- contrib/autoregen.py | 221 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 221 insertions(+) create mode 100755 contrib/autoregen.py diff --git a/contrib/autoregen.py b/contrib/autoregen.py new file mode 100755 index 00000000000..faffc88c5bd --- /dev/null +++ b/contrib/autoregen.py @@ -0,0 +1,221 @@ +#!/usr/bin/env python3 + +# This script helps to regenerate files managed by autotools and +# autogen in binutils-gdb and gcc repositories. + +# It can be used by buildbots to check that the current repository +# contents has been updated correctly, and by developers to update +# such files as expected. + +import os +import shutil +import subprocess +from pathlib import Path + + +# On Gentoo, vanilla unpatched autotools are packaged separately. +# We place the vanilla names first as we want to prefer those if both exist. +AUTOCONF_NAMES = ["autoconf-vanilla-2.69", "autoconf-2.69", "autoconf"] +AUTOMAKE_NAMES = ["automake-vanilla-1.15", "automake-1.15.1", "automake"] +ACLOCAL_NAMES = ["aclocal-vanilla-1.15", "aclocal-1.15.1", "aclocal"] +AUTOHEADER_NAMES = ["autoheader-vanilla-2.69", "autoheader-2.69", "autoheader"] +AUTORECONF_NAMES = ["autoreconf-vanilla-2.69", "autoreconf-2.69", "autoreconf"] + +# Pick the first for each list that exists on this system. +AUTOCONF_BIN = next(name for name in AUTOCONF_NAMES if shutil.which(name)) +AUTOMAKE_BIN = next(name for name in AUTOMAKE_NAMES if shutil.which(name)) +ACLOCAL_BIN = next(name for name in ACLOCAL_NAMES if shutil.which(name)) +AUTOHEADER_BIN = next(name for name in AUTOHEADER_NAMES if shutil.which(name)) +AUTORECONF_BIN = next(name for name in AUTORECONF_NAMES if shutil.which(name)) + +AUTOGEN_BIN = "autogen" + +# autoconf-wrapper and automake-wrapper from Gentoo look at this environment variable. +# It's harmless to set it on other systems though. +EXTRA_ENV = { + "WANT_AUTOCONF": AUTOCONF_BIN.split("-", 1)[1] if "-" in AUTOCONF_BIN else "", + "WANT_AUTOMAKE": AUTOMAKE_BIN.split("-", 1)[1] if "-" in AUTOMAKE_BIN else "", + "AUTOCONF": AUTOCONF_BIN, + "ACLOCAL": ACLOCAL_BIN, + "AUTOMAKE": AUTOMAKE_BIN, + "AUTOGEN": AUTOGEN_BIN, +} +ENV = os.environ.copy() +ENV.update(EXTRA_ENV) + + +# Directories we should skip entirely because they're vendored or have different +# autotools versions. +SKIP_DIRS = [ + # readline and minizip are maintained with different autotools versions + "readline", + "minizip", +] + +# these directories are known to be re-generatable with a simple autoreconf +# without special -I flags +# Entries commented out (and directories not listed) are handled by +# regenerate_manually(). +AUTORECONF_DIRS = [ + # subdirs common to binutils-gdb and gcc + "libbacktrace", + "libdecnumber", # No Makefile.am + "libiberty", # No Makefile.am + "zlib", + + # binutils-gdb subdirs + "bfd", + "binutils", + "etc", + "gas", + "gdb", + "gdbserver", + "gdbsupport", + "gnulib", + "gold", + "gprof", + "gprofng", + "gprofng/libcollector", + "ld", + "libctf", + "libsframe", + "opcodes", + "sim", + + # gcc subdirs + "c++tools", # No aclocal.m4 + "gcc", # No Makefile.am + #"fixincludes", # autoreconf complains about GCC_AC_FUNC_MMAP_BLACKLIST + "gnattools", # No aclocal.m4 + "gotools", + "libada", # No aclocal.m4 + "libatomic", + "libcc1", + "libcody", # No aclocal.m4 + "libcpp", # No Makefile.am + "libffi", + "libgcc", # No aclocal.m4 + "libgfortran", + # Hack: ACLOCAL_AMFLAGS = -I .. -I ../config in Makefile.in but we + # apply -I../config -I.. otherwise we do not match the current + # contents + #"libgm2", + "libgo", + "libgomp", + "libgrust", + "libitm", + "libobjc", # No Makefile.am + "libphobos", + "libquadmath", + "libsanitizer", + "libssp", + "libstdc++-v3", + # This does not cover libvtv/testsuite/other-tests/Makefile.in + "libvtv", + "lto-plugin", +] + + +# Run the shell command CMD. +# +# Print the command on stdout prior to running it. +def run_shell(cmd: str): + print(f"+ {cmd}", flush=True) + res = subprocess.run( + f"{cmd}", + shell=True, + encoding="utf8", + env=ENV, + ) + res.check_returncode() + + +def regenerate_with_autoreconf(): + run_shell(f"{AUTORECONF_BIN} -f") + +def regenerate_with_autogen(): + run_shell(f"{AUTOGEN_BIN} Makefile.def") + +def regenerate_manually(): + configure_lines = open("configure.ac").read().splitlines() + if folder.stem == "fixincludes" or folder.stem == "libgm2" or any( + True for line in configure_lines if line.startswith("AC_CONFIG_MACRO_DIR") + ): + include_arg = "" + include_arg2 = "" + if (folder / ".." / "config").is_dir(): + include_arg = "-I../config" + + # this is really a hack just for binutils-gdb/gprofng/libcollector + # make sure that the order of includes is done as --enable-maintainer-mode + if (folder / ".." / ".." / "config").is_dir(): + include_arg = "-I../.." + include_arg2 = "-I../../config" + + if folder.stem == "fixincludes": + include_arg = "-I.." + include_arg2 = "-I../config" + + if folder.stem == "libgm2": + include_arg = "-I../config" + include_arg2 = "-I.." + + # aclocal does not support the -f short option for force + run_shell(f"{ACLOCAL_BIN} --force {include_arg} {include_arg2}") + + if (folder / "config.in").is_file() or any( + True for line in configure_lines if line.startswith("AC_CONFIG_HEADERS") + ): + run_shell(f"{AUTOHEADER_BIN} -f") + + # The few lines below do not regenerate the exact same content as + # currently in the repo. Disable them for now. + # if (folder / "gm2-libs").is_dir(): + # run_shell(f"{AUTOCONF_BIN} -f gm2-libs/config-host.in > gm2-libs/config-host") + # run_shell(f"{AUTOHEADER_BIN} gm2-libs/config-host.in") + + + # apparently automake is somehow unstable -> skip it for gotools + if any( + True for line in configure_lines if line.startswith("AM_INIT_AUTOMAKE") + ) and not str(folder).endswith("gotools"): + run_shell(f"{AUTOMAKE_BIN} -f") + + run_shell(f"{AUTOCONF_BIN} -f") + + +run_shell(f"{AUTOCONF_BIN} --version") +run_shell(f"{AUTOMAKE_BIN} --version") +run_shell(f"{ACLOCAL_BIN} --version") +run_shell(f"{AUTOHEADER_BIN} --version") + +print(f"Extra environment: {EXTRA_ENV}", flush=True) + +config_folders: list[Path] = [] +autogen_folders: list[Path] = [] +repo_root = Path.cwd() + +for root, _, files in os.walk("."): + for file in files: + if file == "configure.ac": + config_folders.append(Path(root).resolve()) + if file == "Makefile.tpl": + autogen_folders.append(Path(root).resolve()) + +for folder in sorted(autogen_folders): + print(f"Entering directory {folder}", flush=True) + os.chdir(folder) + regenerate_with_autogen() + +for folder in sorted(config_folders): + if folder.stem in SKIP_DIRS: + print(f"Skipping directory {folder}", flush=True) + continue + + print(f"Entering directory {folder}", flush=True) + os.chdir(folder) + + if str(folder.relative_to(repo_root)) in AUTORECONF_DIRS: + regenerate_with_autoreconf() + else: + regenerate_manually() -- 2.34.1