Re: [PATCH 4/6] autoregen.py: Use autoreconf in most GCC directories

[Simon Marchi <simark@simark.ca>]

On 2024-04-12 16:05, Christophe Lyon wrote:
> Add most of GCC's subdirectories to AUTORECONF_DIRS, since 'autoreconf
> -f' works for them.
> 
> A few subdirs still need to be regenerated "manually", because they do
> not have Makefile.am which autoreconf uses to determine which aclocal
> flags to use.

gdb, for instance, doesn't use Makefile.am, and yet autoreconf works.  It
finds the aclocal include paths from the AC_CONFIG_MACRO_DIRS macro in
configure.ac.  If the remaining subdirs that still need to be manually
handled used AC_CONFIG_MACRO_DIRS, would autoreconf work in them?

Out of the directories you commented out from AUTORECONF_DIRS, the
following re-generated cleanly with autoreconf for me:

 - libdecnumber
 - gcc
 - libiberty
 - libobjc

So I would suggest adding them to AUTORECONF_DIRS.

> Most use our default aclocal -I../config, but a few need special
> flags, which I copied from the corresponding Makefile.in:
> * fixincludes: -I.. -I../config
> * gcc, libiberty, libobjc: -I../config -I..
> * libgm2: -I../config -I.. although Makefile.in says otherwise. Using
>   what Makefile.in defines results in generating aclocal.m4 with a
>   different contents than what it is in the repo. The repo is
>   presumably wrong, but use this until this is fixed.
> 
> Note that we do not regenerate
> libvtv/testsuite/other-tests/Makefile.in, because
> libvtv/testsuite/other-tests/ has no configure.ac.
> 
> Running this script over GCC's repository generates quite a few
> warnings, which are the same as before this patch. Namely, these
> subdirs seem to have incorrect autotools files (at least partially):
> * libgfortran
> * libgomp
> * libsanitizer
> 
> This patch does not support regenerating gcc/m2/gm2-libs/config-host
> and gcc/m2/gm2-libs/gm2-libs-host.h.in because I didn't manage to
> regenerate them with the exact same content. FTR I tried:
> autoconf -f config-host.in > config-host
> autoheader config-host.in
> as described in gcc/m2/Make-maintainer.in
> 
> Similarly, we skip libcpp because the files we regenerate have small
> differences with the current versions.

libcpp's files appear to be in an invalid state in the repo itself.
According to libcpp/Makefile.in line 123, aclocal.m4 should be
regenerated with `aclocal -I ../config`.  When I use that, I get the
same results as what autoreconf would give me.  For that one, I think
you could send a patch to gcc to fix it in the repo, after which we can
switch it to use autoreconf.

> Tested by removing all aclocal.m4 files, Makefile.in files derived
> from Makefile.am, configure files derived from configure.ac and
> checking they are correctly regenerated (that is, 'git status' does
> not report deleted nor modified files).
> ---
>  builder/containers/autoregen.py | 60 ++++++++++++++++++++++++++++++++-
>  1 file changed, 59 insertions(+), 1 deletion(-)
> 
> diff --git a/builder/containers/autoregen.py b/builder/containers/autoregen.py
> index c52a524..4731a87 100755
> --- a/builder/containers/autoregen.py
> +++ b/builder/containers/autoregen.py
> @@ -40,15 +40,56 @@ SKIP_DIRS = [
>      # readline and minizip are maintained with different autotools versions
>      "readline",
>      "minizip",
> +
> +    # aclocal.m4 gets an additional ../config/override.m4,
> +    # and config.in gets a few more empty lines.
> +    "libcpp",
>  ]
>  
>  # these directories are known to can be re-generatable with a simple autoreconf
>  # without special -I flags
>  AUTORECONF_DIRS = [
> +    # binutils-gdb subdirs
>      "gdb",
>      "gdbserver",
>      "gdbsupport",
>      "gnulib",
> +
> +    # gcc subdirs
> +    "c++tools", # No aclocal.m4
> +    #"gcc", # No Makefile.am
> +    #"fixincludes", # autoreconf complains about GCC_AC_FUNC_MMAP_BLACKLIST
> +    "gnattools", # No aclocal.m4
> +    "gotools",
> +    "libada", # No aclocal.m4
> +    "libatomic",
> +    "libbacktrace",
> +    "libcc1",
> +    "libcody", # No aclocal.m4
> +    #"libcpp", # No Makefile.am
> +    #"libdecnumber", # No Makefile.am
> +    "libffi",
> +    "libgcc", # No aclocal.m4
> +    "libgfortran",
> +    # Hack: ACLOCAL_AMFLAGS = -I .. -I ../config in Makefile.in but we
> +    # apply -I../config -I.. otherwise we do not match the current
> +    # contents
> +    #"libgm2",
> +    "libgo",
> +    "libgomp",
> +    "libgrust",
> +    #"libiberty", # No Makefile.am
> +    "libitm",
> +    #"libobjc", # No Makefile.am
> +    "libphobos",
> +    "libquadmath",
> +    "libsanitizer",
> +    "libssp",
> +    "libstdc++-v3",
> +    # This does not cover libvtv/testsuite/other-tests/Makefile.in
> +    "libvtv",
> +    "lto-plugin",
> +    "zlib",

If you really want to make the distinction between binutils-gdb and gcc
subdirs, perhaps add a "common" section for those directories that
appear in both repos?

>  ]
>  
>  
> @@ -71,7 +112,9 @@ def regenerate_with_autoreconf():
>  
>  def regenerate_manually():
>      configure_lines = open("configure.ac").read().splitlines()
> -    if any(True for line in configure_lines if line.startswith("AC_CONFIG_MACRO_DIR")):
> +    if folder.stem == "fixincludes" or folder.stem == "libgm2" or any(
> +            True for line in configure_lines if line.startswith("AC_CONFIG_MACRO_DIR")
> +    ):
>          include_arg = ""
>          include_arg2 = ""
>          if (folder / ".." / "config").is_dir():
> @@ -83,6 +126,14 @@ def regenerate_manually():
>              include_arg = "-I../.."
>              include_arg2 = "-I../../config"
>  
> +        if folder.stem == "fixincludes":
> +            include_arg = "-I.."
> +            include_arg2 = "-I../config"
> +
> +        if folder.stem == "libgm2" or folder.stem == "gcc" or folder.stem == "libiberty" or folder.stem == "libobjc":

This could be a bit shorter:

  if folder.stem in ["libgm2", "gcc", "libiberty", "libobjc"]:

Simon