From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com [IPv6:2a00:1450:4864:20::636]) by sourceware.org (Postfix) with ESMTPS id E73CE3858D20 for ; Wed, 17 Apr 2024 14:30:31 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org E73CE3858D20 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=linaro.org ARC-Filter: OpenARC Filter v1.0.0 sourceware.org E73CE3858D20 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2a00:1450:4864:20::636 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1713364235; cv=none; b=e+9+WHEEWeC8WA9nVu7VcozpPu1Xf2Ayk5uDl369TAl13m1GPl30Z6hrP1zWNk0ovWbWMYJgMcNUBCFD0LJeqWmIRQZjeb9Q5TUVEdzkU3bjWY/UIhHfV7Yp1xC9aDzbmW4XO1jMxoHTP8SSlgGzaZPH1XetaO5APenx69I6zNI= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1713364235; c=relaxed/simple; bh=VpXrtmH47uzMfZjMoCr0GTv2Yun/sZ8YMb3XirflIvU=; h=DKIM-Signature:MIME-Version:From:Date:Message-ID:Subject:To; b=F3iLauAgE6k7fJa2FoAYCMzZ/P5DGl7lIrTB8LGDFEpqNHwto3X20wOCLs9HLWFyLrd5+0G/piCchQIPxelq3aA380RNUTklbab9qWEP6RBnNNGzXrT5Q4Tw/T7fklmYaVH4vxabudA82NQxT/YkjN6ymwPQJXh1Q4HuDBcum+w= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-ej1-x636.google.com with SMTP id a640c23a62f3a-a53f131d9deso410182866b.3 for ; Wed, 17 Apr 2024 07:30:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1713364230; x=1713969030; darn=sourceware.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=TcsiWLD4OVrGe92BZhpdBUHgWWKHAPDWUrxRly9nbaU=; b=pbyFNEhemqoKYL0WuzzhcOUEufqCVGM+LFjdtAaYYzFE/ZFfcugoKEGhSNhX7lQOww o1SsuEv2GXtqpwQrtn7qa7Kum5jOIBdW9n6oYfuzeBBUPXlcTV8mrUkGjAXYcGhtAJOu 1iA92AJo9JKpb4zGoZzLD0Vxs7gI5byfehb6Ug0s+JvUqyBXEjtcpyVO4DjQZ8ox9pBb Ux7vrdXo0l/iq+cn4/mh750C73YlnbpZU72sVN7BsNqFh5zBBvAEVqRHMMoV7DlcCIjc XKV6Pkh7K/191dVOxUuhHRmBMqKNBpGYABmLuH57B1YYYQquaq7DuhXLbwVm2dy2RsMH evtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713364230; x=1713969030; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=TcsiWLD4OVrGe92BZhpdBUHgWWKHAPDWUrxRly9nbaU=; b=ERzymx3fnq8JfGRDd5ZjBTzPkhpJy1hjvkp7JZiJHe/yZ33ZCppJZh+twxVoco7tNh r0fJM2CBLveaInUfSL6AXMpEEHi7xyndt52a9nzbEc/KE+BfPJkWRVXMLSe8bFtvEUpi NblMl5NyPi0hkeSEGyEssRH1DeZvVJkVPgPBEWFPbyDVfPLZApzK2CRSFsyZV48TamnQ Ak1QRupZaW4lbaJ3Nv30ruseocLBIihi87xF744cjYqU7l7KMcYKa15jtE7vEvV9Z7Nz 04V1bUupRBnlc3kRDNr4TCwPoM2mJ0bWZUK+CCGce8lXJVaPsrBWRaWf5zaU/4+oUHeX CrIA== X-Gm-Message-State: AOJu0Yyjx5gXXZkEY5k80RI1xgwYOZDaqByJa9+yqVApathTdyvs2deA AwK2BtMlLe4+D7LzaHBEJhfgaQj+nRy7sPh+mQnMK9dxmWo/n36yxCQ1gw+GErzwcrvqJ34sMGk OY1/r4Mnn8GIluK0isqkrhnfPChu7Dcmt0hQBMBow/lrpbI9sNSoJBg== X-Google-Smtp-Source: AGHT+IHPl187g5FtLpRdqRH+P53SlNGmowZl6kVinlHVu8lTdjx7bCGwrQOncpKRTg9mrWIH4Sglhzpk9eaZAxNgwlo= X-Received: by 2002:a17:907:7208:b0:a51:a329:cd76 with SMTP id dr8-20020a170907720800b00a51a329cd76mr13354494ejc.13.1713364230348; Wed, 17 Apr 2024 07:30:30 -0700 (PDT) MIME-Version: 1.0 References: <20240412200559.1649050-1-christophe.lyon@linaro.org> <20240412200559.1649050-5-christophe.lyon@linaro.org> In-Reply-To: From: Christophe Lyon Date: Wed, 17 Apr 2024 16:30:24 +0200 Message-ID: Subject: Re: [PATCH 4/6] autoregen.py: Use autoreconf in most GCC directories To: Simon Marchi Cc: buildbot@sourceware.org Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,GIT_PATCH_0,KAM_SHORT,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On Mon, 15 Apr 2024 at 14:52, Christophe Lyon wrote: > > On Mon, 15 Apr 2024 at 05:07, Simon Marchi wrote: > > > > > > > > On 2024-04-12 16:05, Christophe Lyon wrote: > > > Add most of GCC's subdirectories to AUTORECONF_DIRS, since 'autoreconf > > > -f' works for them. > > > > > > A few subdirs still need to be regenerated "manually", because they do > > > not have Makefile.am which autoreconf uses to determine which aclocal > > > flags to use. > > > > gdb, for instance, doesn't use Makefile.am, and yet autoreconf works. It > > finds the aclocal include paths from the AC_CONFIG_MACRO_DIRS macro in > > configure.ac. If the remaining subdirs that still need to be manually > > handled used AC_CONFIG_MACRO_DIRS, would autoreconf work in them? > > that's because gdb's aclocal.m4 is generated with 'aclocal' without anyflag. > > If you look at gcc/gcc/Makefile.in, you'll see: > ACLOCAL_AMFLAGS = -I ../config -I .. > So to regenerate gcc/gcc/aclocal.m4, we need to call > aclocal -I ../config -I .. > but autoreconf looks for this info in Makefile.am which does not exist > in this case. > (see line 410 and below in autoreconf 2.69) > > I don't know how AC_CONFIG_MACRO_DIRS would help, maybe it would... > > > Out of the directories you commented out from AUTORECONF_DIRS, the > > following re-generated cleanly with autoreconf for me: > > > > - libdecnumber > So -I ../config is not needed by aclocal? > > > - gcc > as mentioned above, here we have ACLOCAL_AMFLAGS = -I ../config -I .. > so it's surprising it works without these flags? > > > - libiberty > > - libobjc > same as for gcc. > > > So I would suggest adding them to AUTORECONF_DIRS. > > Well, maybe :-) > I must confess the doc for autoconf is not clear enough for me :-) > https://www.gnu.org/software/autoconf/manual/autoconf-2.69/html_node/Input.html > does not mention AC_CONFIG_MACRO_DIRS, only AC_CONFIG_MACRO_DIR > > And the "Note that if you use aclocal from Automake to generate aclocal.m4[...]" > does not clearly say that AC_CONFIG_MACRO_DIR[S] is enough. > (and the code I saw in autoreconf only checks /^ACLOCAL_[A-Z_]*FLAGS\s*=\s*(.*) > in Makefile.am. > > https://www.gnu.org/software/autoconf/manual/autoconf-2.70/html_node/Input.html > is more verbose and does mention AC_CONFIG_MACRO_DIRS, but we still use 2.69 > so I prefered to stay on the "safe" side. > > Maybe worth asking on the autoconf/automake list? > > > > Most use our default aclocal -I../config, but a few need special > > > flags, which I copied from the corresponding Makefile.in: > > > * fixincludes: -I.. -I../config > > > * gcc, libiberty, libobjc: -I../config -I.. > > > * libgm2: -I../config -I.. although Makefile.in says otherwise. Using > > > what Makefile.in defines results in generating aclocal.m4 with a > > > different contents than what it is in the repo. The repo is > > > presumably wrong, but use this until this is fixed. > > > > > > Note that we do not regenerate > > > libvtv/testsuite/other-tests/Makefile.in, because > > > libvtv/testsuite/other-tests/ has no configure.ac. > > > > > > Running this script over GCC's repository generates quite a few > > > warnings, which are the same as before this patch. Namely, these > > > subdirs seem to have incorrect autotools files (at least partially): > > > * libgfortran > > > * libgomp > > > * libsanitizer > > > > > > This patch does not support regenerating gcc/m2/gm2-libs/config-host > > > and gcc/m2/gm2-libs/gm2-libs-host.h.in because I didn't manage to > > > regenerate them with the exact same content. FTR I tried: > > > autoconf -f config-host.in > config-host > > > autoheader config-host.in > > > as described in gcc/m2/Make-maintainer.in > > > > > > Similarly, we skip libcpp because the files we regenerate have small > > > differences with the current versions. > > > > libcpp's files appear to be in an invalid state in the repo itself. > > According to libcpp/Makefile.in line 123, aclocal.m4 should be > > regenerated with `aclocal -I ../config`. When I use that, I get the > > same results as what autoreconf would give me. For that one, I think > > you could send a patch to gcc to fix it in the repo, after which we can > > switch it to use autoreconf. > > Yes I noticed a discrepancy about override.m4, but I thought this is > too late in GCC stage4 to fix it. > Maybe I'm wrong on this assumption though :-) So I was wrong :-) I fixed the libcpp files, so we can now use autoreconf for it too. Thanks, Christophe > > > > > > Tested by removing all aclocal.m4 files, Makefile.in files derived > > > from Makefile.am, configure files derived from configure.ac and > > > checking they are correctly regenerated (that is, 'git status' does > > > not report deleted nor modified files). > > > --- > > > builder/containers/autoregen.py | 60 ++++++++++++++++++++++++++++++++- > > > 1 file changed, 59 insertions(+), 1 deletion(-) > > > > > > diff --git a/builder/containers/autoregen.py b/builder/containers/autoregen.py > > > index c52a524..4731a87 100755 > > > --- a/builder/containers/autoregen.py > > > +++ b/builder/containers/autoregen.py > > > @@ -40,15 +40,56 @@ SKIP_DIRS = [ > > > # readline and minizip are maintained with different autotools versions > > > "readline", > > > "minizip", > > > + > > > + # aclocal.m4 gets an additional ../config/override.m4, > > > + # and config.in gets a few more empty lines. > > > + "libcpp", > > > ] > > > > > > # these directories are known to can be re-generatable with a simple autoreconf > > > # without special -I flags > > > AUTORECONF_DIRS = [ > > > + # binutils-gdb subdirs > > > "gdb", > > > "gdbserver", > > > "gdbsupport", > > > "gnulib", > > > + > > > + # gcc subdirs > > > + "c++tools", # No aclocal.m4 > > > + #"gcc", # No Makefile.am > > > + #"fixincludes", # autoreconf complains about GCC_AC_FUNC_MMAP_BLACKLIST > > > + "gnattools", # No aclocal.m4 > > > + "gotools", > > > + "libada", # No aclocal.m4 > > > + "libatomic", > > > + "libbacktrace", > > > + "libcc1", > > > + "libcody", # No aclocal.m4 > > > + #"libcpp", # No Makefile.am > > > + #"libdecnumber", # No Makefile.am > > > + "libffi", > > > + "libgcc", # No aclocal.m4 > > > + "libgfortran", > > > + # Hack: ACLOCAL_AMFLAGS = -I .. -I ../config in Makefile.in but we > > > + # apply -I../config -I.. otherwise we do not match the current > > > + # contents > > > + #"libgm2", > > > + "libgo", > > > + "libgomp", > > > + "libgrust", > > > + #"libiberty", # No Makefile.am > > > + "libitm", > > > + #"libobjc", # No Makefile.am > > > + "libphobos", > > > + "libquadmath", > > > + "libsanitizer", > > > + "libssp", > > > + "libstdc++-v3", > > > + # This does not cover libvtv/testsuite/other-tests/Makefile.in > > > + "libvtv", > > > + "lto-plugin", > > > + "zlib", > > > > If you really want to make the distinction between binutils-gdb and gcc > > subdirs, perhaps add a "common" section for those directories that > > appear in both repos? > > OK > > > > > > ] > > > > > > > > > @@ -71,7 +112,9 @@ def regenerate_with_autoreconf(): > > > > > > def regenerate_manually(): > > > configure_lines = open("configure.ac").read().splitlines() > > > - if any(True for line in configure_lines if line.startswith("AC_CONFIG_MACRO_DIR")): > > > + if folder.stem == "fixincludes" or folder.stem == "libgm2" or any( > > > + True for line in configure_lines if line.startswith("AC_CONFIG_MACRO_DIR") > > > + ): > > > include_arg = "" > > > include_arg2 = "" > > > if (folder / ".." / "config").is_dir(): > > > @@ -83,6 +126,14 @@ def regenerate_manually(): > > > include_arg = "-I../.." > > > include_arg2 = "-I../../config" > > > > > > + if folder.stem == "fixincludes": > > > + include_arg = "-I.." > > > + include_arg2 = "-I../config" > > > + > > > + if folder.stem == "libgm2" or folder.stem == "gcc" or folder.stem == "libiberty" or folder.stem == "libobjc": > > > > This could be a bit shorter: > > > > if folder.stem in ["libgm2", "gcc", "libiberty", "libobjc"]: > Thanks, too late I think Mark has already pushed my patch as-is :-) > But this can be fixed later. > > Thanks, > > Christophe > > > > > > Simon