From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi1-x234.google.com (mail-oi1-x234.google.com [IPv6:2607:f8b0:4864:20::234]) by sourceware.org (Postfix) with ESMTPS id DAB20384AB48 for ; Fri, 19 Apr 2024 09:03:56 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org DAB20384AB48 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=linaro.org ARC-Filter: OpenARC Filter v1.0.0 sourceware.org DAB20384AB48 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::234 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1713517447; cv=none; b=h+oPr4vPPoncD4oVoUzvBSGcxYXIZ2QndvO12XCdC8MgZuQHo1ZKfynsVZpUyklLnWiUk+bNcAo8ruBdU8S6KTTRMbYUKBUMSNHhA0iZJx2jbw40lVXAIKHs+82/Ldvsr9rVifEW83SO7dHrF8vZXze5fKqA8xzkneRKJgqu9Tk= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1713517447; c=relaxed/simple; bh=q1nHxTun9N8sydRsaM+M4CIl8J8mozCE8/JIx3eJ0xI=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=toHIoekNeh86qOevRfGcw1FN+hQif/x0l9+JUCJh1752YklaSqosMEf9UwxywwB3eAenQnaShUoxn0zYJ2qqEJ7MogZKyUqdP+NIuJDMkgdOCvhW03HBTi3LhWeosZkRqrsYMied5T0U8B5Vkj6oBFNkNVen1MjFio2YUKu5c54= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-oi1-x234.google.com with SMTP id 5614622812f47-3c7220ec523so295068b6e.2 for ; Fri, 19 Apr 2024 02:03:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1713517435; x=1714122235; darn=gcc.gnu.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=I9zra3z/mW+dScgD+jPYQ+5z+bCP105GBKp01PmlJ/I=; b=L0CJWvXoJXTFk6J6/bd0mF8CzkxfT0VYzDvuyK+0xvCaPIUnRt/pey4vdqF+pafK6p v8bvhPcmnhJilZwqs3KX8jX5ZiZ5YNog/11jVI9G0/iD1xAcEWwE9mAwHNfqnx1HUEGe q8rKOFvjSr6I99BPtAYKES2FJOfh2TSpv2qN/4XWRhoZNqkOUyNPQGVzgb4uf87Y8q5s i2BDQC/TpAI5LES4D0ZwbPQlvR0+78CaUxRe4NDrDhUgmwhw2pedRQbta69XoReS5j90 5EdNeSf4GwDQ5GU23Za41ECiVwymEWdyVylzEEDziNvud0LeljzD+zhR6OPIzGrX+V/d sTfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713517435; x=1714122235; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=I9zra3z/mW+dScgD+jPYQ+5z+bCP105GBKp01PmlJ/I=; b=beSSkbSOqmwu7pFIiGLllrRIbi1WguY2Hj2fB7ayyP1GbF+xmKXn4nOYHT08MbvsT/ ttiOraH1EFN27d0cZTJj/e5d7TNB3OrR6KVmDSejLWqHCmzXw+t9cNYwoH4xrx/nu99i lfIa/o53vObh5pRKkzMQaKMvk2XI9P9+WCde4J0l2vX7kLYMCSf5V0dZRQ8gfcMbPxRt zd7zRFqtd4Ftb7XDz9d5pi8ozpnNJDe6F5ESn9bMOQ28vbxXeTBQhqjIsjzCGMomcrnS tE/eSU9WnqO1DrMFdt4oRBGJS6N7T13Qvtwhm60MKLhn23Ln4mJFtsFuBL5vQgD1y4d8 f0Rg== X-Gm-Message-State: AOJu0YwCtVwTkMG1HMT+wH5eHXDN4ybmOrN5Vo5WT7FguE5+yj3u8fiY VwRH1qrTZARwRriH74gcSEKxgGsQaNMncuYsztdMwyIg9sWDRnwsvkBgPKe5vRUnGMRDFem+gIh nUXE= X-Google-Smtp-Source: AGHT+IGRtzQUgG0VDZc1BqcwzjL4E6r5RZHeK1CuTPeewUGUvsj5d34yFj9fJp+LYB4M/oqFCOFluQ== X-Received: by 2002:aca:110b:0:b0:3c7:dfb:a295 with SMTP id 11-20020aca110b000000b003c70dfba295mr1346973oir.55.1713517433759; Fri, 19 Apr 2024 02:03:53 -0700 (PDT) Received: from localhost.localdomain ([139.178.84.207]) by smtp.gmail.com with ESMTPSA id j26-20020a54481a000000b003c73c5daef0sm449631oij.47.2024.04.19.02.03.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 19 Apr 2024 02:03:53 -0700 (PDT) From: Christophe Lyon To: gcc-patches@gcc.gnu.org Cc: Christophe Lyon Subject: [PATCH] contrib: Add autoregen.py Date: Fri, 19 Apr 2024 09:03:45 +0000 Message-Id: <20240419090345.2079649-1-christophe.lyon@linaro.org> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-12.6 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,GIT_PATCH_0,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: This script is a copy of the current script used by Sourceware's autoregen buildbots. It is intended as a helper to regenerate files managed by autotools (autoconf, automake, aclocal, ....), as well as the toplevel Makefile.in which is created by autogen. Other files can be updated when using maintainer-mode, but this is not covered by this script. 2024-04-19 Christophe Lyon contrib/ * autogen.py: New script. --- contrib/autoregen.py | 221 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 221 insertions(+) create mode 100755 contrib/autoregen.py diff --git a/contrib/autoregen.py b/contrib/autoregen.py new file mode 100755 index 00000000000..faffc88c5bd --- /dev/null +++ b/contrib/autoregen.py @@ -0,0 +1,221 @@ +#!/usr/bin/env python3 + +# This script helps to regenerate files managed by autotools and +# autogen in binutils-gdb and gcc repositories. + +# It can be used by buildbots to check that the current repository +# contents has been updated correctly, and by developers to update +# such files as expected. + +import os +import shutil +import subprocess +from pathlib import Path + + +# On Gentoo, vanilla unpatched autotools are packaged separately. +# We place the vanilla names first as we want to prefer those if both exist. +AUTOCONF_NAMES = ["autoconf-vanilla-2.69", "autoconf-2.69", "autoconf"] +AUTOMAKE_NAMES = ["automake-vanilla-1.15", "automake-1.15.1", "automake"] +ACLOCAL_NAMES = ["aclocal-vanilla-1.15", "aclocal-1.15.1", "aclocal"] +AUTOHEADER_NAMES = ["autoheader-vanilla-2.69", "autoheader-2.69", "autoheader"] +AUTORECONF_NAMES = ["autoreconf-vanilla-2.69", "autoreconf-2.69", "autoreconf"] + +# Pick the first for each list that exists on this system. +AUTOCONF_BIN = next(name for name in AUTOCONF_NAMES if shutil.which(name)) +AUTOMAKE_BIN = next(name for name in AUTOMAKE_NAMES if shutil.which(name)) +ACLOCAL_BIN = next(name for name in ACLOCAL_NAMES if shutil.which(name)) +AUTOHEADER_BIN = next(name for name in AUTOHEADER_NAMES if shutil.which(name)) +AUTORECONF_BIN = next(name for name in AUTORECONF_NAMES if shutil.which(name)) + +AUTOGEN_BIN = "autogen" + +# autoconf-wrapper and automake-wrapper from Gentoo look at this environment variable. +# It's harmless to set it on other systems though. +EXTRA_ENV = { + "WANT_AUTOCONF": AUTOCONF_BIN.split("-", 1)[1] if "-" in AUTOCONF_BIN else "", + "WANT_AUTOMAKE": AUTOMAKE_BIN.split("-", 1)[1] if "-" in AUTOMAKE_BIN else "", + "AUTOCONF": AUTOCONF_BIN, + "ACLOCAL": ACLOCAL_BIN, + "AUTOMAKE": AUTOMAKE_BIN, + "AUTOGEN": AUTOGEN_BIN, +} +ENV = os.environ.copy() +ENV.update(EXTRA_ENV) + + +# Directories we should skip entirely because they're vendored or have different +# autotools versions. +SKIP_DIRS = [ + # readline and minizip are maintained with different autotools versions + "readline", + "minizip", +] + +# these directories are known to be re-generatable with a simple autoreconf +# without special -I flags +# Entries commented out (and directories not listed) are handled by +# regenerate_manually(). +AUTORECONF_DIRS = [ + # subdirs common to binutils-gdb and gcc + "libbacktrace", + "libdecnumber", # No Makefile.am + "libiberty", # No Makefile.am + "zlib", + + # binutils-gdb subdirs + "bfd", + "binutils", + "etc", + "gas", + "gdb", + "gdbserver", + "gdbsupport", + "gnulib", + "gold", + "gprof", + "gprofng", + "gprofng/libcollector", + "ld", + "libctf", + "libsframe", + "opcodes", + "sim", + + # gcc subdirs + "c++tools", # No aclocal.m4 + "gcc", # No Makefile.am + #"fixincludes", # autoreconf complains about GCC_AC_FUNC_MMAP_BLACKLIST + "gnattools", # No aclocal.m4 + "gotools", + "libada", # No aclocal.m4 + "libatomic", + "libcc1", + "libcody", # No aclocal.m4 + "libcpp", # No Makefile.am + "libffi", + "libgcc", # No aclocal.m4 + "libgfortran", + # Hack: ACLOCAL_AMFLAGS = -I .. -I ../config in Makefile.in but we + # apply -I../config -I.. otherwise we do not match the current + # contents + #"libgm2", + "libgo", + "libgomp", + "libgrust", + "libitm", + "libobjc", # No Makefile.am + "libphobos", + "libquadmath", + "libsanitizer", + "libssp", + "libstdc++-v3", + # This does not cover libvtv/testsuite/other-tests/Makefile.in + "libvtv", + "lto-plugin", +] + + +# Run the shell command CMD. +# +# Print the command on stdout prior to running it. +def run_shell(cmd: str): + print(f"+ {cmd}", flush=True) + res = subprocess.run( + f"{cmd}", + shell=True, + encoding="utf8", + env=ENV, + ) + res.check_returncode() + + +def regenerate_with_autoreconf(): + run_shell(f"{AUTORECONF_BIN} -f") + +def regenerate_with_autogen(): + run_shell(f"{AUTOGEN_BIN} Makefile.def") + +def regenerate_manually(): + configure_lines = open("configure.ac").read().splitlines() + if folder.stem == "fixincludes" or folder.stem == "libgm2" or any( + True for line in configure_lines if line.startswith("AC_CONFIG_MACRO_DIR") + ): + include_arg = "" + include_arg2 = "" + if (folder / ".." / "config").is_dir(): + include_arg = "-I../config" + + # this is really a hack just for binutils-gdb/gprofng/libcollector + # make sure that the order of includes is done as --enable-maintainer-mode + if (folder / ".." / ".." / "config").is_dir(): + include_arg = "-I../.." + include_arg2 = "-I../../config" + + if folder.stem == "fixincludes": + include_arg = "-I.." + include_arg2 = "-I../config" + + if folder.stem == "libgm2": + include_arg = "-I../config" + include_arg2 = "-I.." + + # aclocal does not support the -f short option for force + run_shell(f"{ACLOCAL_BIN} --force {include_arg} {include_arg2}") + + if (folder / "config.in").is_file() or any( + True for line in configure_lines if line.startswith("AC_CONFIG_HEADERS") + ): + run_shell(f"{AUTOHEADER_BIN} -f") + + # The few lines below do not regenerate the exact same content as + # currently in the repo. Disable them for now. + # if (folder / "gm2-libs").is_dir(): + # run_shell(f"{AUTOCONF_BIN} -f gm2-libs/config-host.in > gm2-libs/config-host") + # run_shell(f"{AUTOHEADER_BIN} gm2-libs/config-host.in") + + + # apparently automake is somehow unstable -> skip it for gotools + if any( + True for line in configure_lines if line.startswith("AM_INIT_AUTOMAKE") + ) and not str(folder).endswith("gotools"): + run_shell(f"{AUTOMAKE_BIN} -f") + + run_shell(f"{AUTOCONF_BIN} -f") + + +run_shell(f"{AUTOCONF_BIN} --version") +run_shell(f"{AUTOMAKE_BIN} --version") +run_shell(f"{ACLOCAL_BIN} --version") +run_shell(f"{AUTOHEADER_BIN} --version") + +print(f"Extra environment: {EXTRA_ENV}", flush=True) + +config_folders: list[Path] = [] +autogen_folders: list[Path] = [] +repo_root = Path.cwd() + +for root, _, files in os.walk("."): + for file in files: + if file == "configure.ac": + config_folders.append(Path(root).resolve()) + if file == "Makefile.tpl": + autogen_folders.append(Path(root).resolve()) + +for folder in sorted(autogen_folders): + print(f"Entering directory {folder}", flush=True) + os.chdir(folder) + regenerate_with_autogen() + +for folder in sorted(config_folders): + if folder.stem in SKIP_DIRS: + print(f"Skipping directory {folder}", flush=True) + continue + + print(f"Entering directory {folder}", flush=True) + os.chdir(folder) + + if str(folder.relative_to(repo_root)) in AUTORECONF_DIRS: + regenerate_with_autoreconf() + else: + regenerate_manually() -- 2.34.1