[PATCH v2 1/2] autoregen.py: adapt to Gentoo

[PATCH v2 1/2] autoregen.py: adapt to Gentoo

[Sam James]

In Gentoo, we have autoconf-vanilla and automake-vanilla packaged for toolchain
development with no patches applied. The binaries are named differently.

Adapt the script to check a list of binaries for each tool, and check the
vanilla names first as we want to prefer those if they exist.

This makes the script work out-of-the-box on Gentoo.

Signed-off-by: Sam James <sam@gentoo.org>
---
mjw, could you double check this does the right thing for you on your systems?

It should be OK as I've checked with a few combinations.

v2:
- Set WANT_{AUTOCONF,AUTOMAKE} environment variables for Gentoo's wrappers.
- Add some style tweaks in patch 2.

 builder/containers/autoregen.py | 27 ++++++++++++++++++++++-----
 1 file changed, 22 insertions(+), 5 deletions(-)

diff --git a/builder/containers/autoregen.py b/builder/containers/autoregen.py
index bfb8f3d..f33fb78 100755
--- a/builder/containers/autoregen.py
+++ b/builder/containers/autoregen.py
@@ -1,15 +1,32 @@
 #!/usr/bin/env python3
 
 import os
+import shutil
 import subprocess
 from pathlib import Path
 
-AUTOCONF_BIN = 'autoconf-2.69'
-AUTOMAKE_BIN = 'automake-1.15.1'
-ACLOCAL_BIN = 'aclocal-1.15.1'
-AUTOHEADER_BIN = 'autoheader-2.69'
+# On Gentoo, vanilla unpatched autotools are packaged separately.
+# We place the vanilla names first as we want to prefer those if both exist.
+autoconf_names = ['autoconf-vanilla-2.69', 'autoconf-2.69']
+automake_names = ['automake-vanilla-1.15', 'automake-1.15.1']
+aclocal_names = ['aclocal-vanilla-1.15', 'aclocal-1.15.1']
+autoheader_names = ['autoheader-vanilla-2.69', 'autoheader-2.69']
+
+# Pick the first for each list that exists on this system.
+AUTOCONF_BIN = next(name for name in autoconf_names if shutil.which(name))
+AUTOMAKE_BIN = next(name for name in automake_names if shutil.which(name))
+ACLOCAL_BIN = next(name for name in aclocal_names if shutil.which(name))
+AUTOHEADER_BIN = next(name for name in autoheader_names if shutil.which(name))
+
+# autoconf-wrapper and automake-wrapper from Gentoo look at this environment variable.
+# It's harmless to set it on other systems though.
+ENV = f'WANT_AUTOCONF={AUTOCONF_BIN.split("-", 1)[1]} '
+ENV += f'WANT_AUTOMAKE={AUTOMAKE_BIN.split("-", 1)[1]} '
+
+ENV += f' AUTOCONF={AUTOCONF_BIN} '
+ENV += f' ACLOCAL={ACLOCAL_BIN} '
+ENV += f' AUTOMAKE={AUTOMAKE_BIN}'
 
-ENV = f'AUTOCONF={AUTOCONF_BIN} ACLOCAL={ACLOCAL_BIN} AUTOMAKE={AUTOMAKE_BIN}'
 
 config_folders = []
 
-- 
2.44.0

[PATCH v2 2/2] autoregen.py: style tweaks

[Sam James]

Maybe a bit easier to read now.

Signed-off-by: Sam James <sam@gentoo.org>
---
 builder/containers/autoregen.py | 21 +++++++++++++++++----
 1 file changed, 17 insertions(+), 4 deletions(-)

diff --git a/builder/containers/autoregen.py b/builder/containers/autoregen.py
index f33fb78..3d307ad 100755
--- a/builder/containers/autoregen.py
+++ b/builder/containers/autoregen.py
@@ -27,6 +27,13 @@ ENV += f' AUTOCONF={AUTOCONF_BIN} '
 ENV += f' ACLOCAL={ACLOCAL_BIN} '
 ENV += f' AUTOMAKE={AUTOMAKE_BIN}'
 
+# Directories we should skip entirely because they're vendored or have different
+# autotools versions.
+skip_dirs = [
+    # readline and minizip are maintained with different autotools versions
+    'readline',
+    'minizip'
+]
 
 config_folders = []
 
@@ -37,11 +44,12 @@ for root, _, files in os.walk('.'):
 
 for folder in sorted(config_folders):
     print(folder, flush=True)
-    # readline and minizip are maintained with different autotools versions
-    if (str(folder).endswith('readline')
-        or str(folder).endswith('minizip')):
-        continue;
+
+    if folder.stem in skip_dirs:
+        continue
+
     os.chdir(folder)
+
     configure_lines = open('configure.ac').read().splitlines()
     if any(True for line in configure_lines if line.startswith('AC_CONFIG_MACRO_DIRS')):
         # aclocal does not support the -f short option for force
@@ -49,19 +57,24 @@ for folder in sorted(config_folders):
         include_arg2 = ''
         if (folder / '..' / 'config').is_dir():
             include_arg = '-I../config'
+
         # this is really a hack just for binutils-gdb/gprofng/libcollector
         # make sure that the order of includes is done as --enable-maintainer-mode
         if (folder / '..' / '..' / 'config').is_dir():
             include_arg = '-I../..'
             include_arg2 = '-I../../config'
+
         subprocess.check_output(f'{ENV} {ACLOCAL_BIN} --force {include_arg} {include_arg2}', shell=True, encoding='utf8')
+
     if ((folder / 'config.in').is_file()
         or any(True for line in configure_lines if line.startswith('AC_CONFIG_HEADERS'))):
         subprocess.check_output(f'{ENV} {AUTOHEADER_BIN} -f', shell=True, encoding='utf8')
+
     # apparently automake is somehow unstable -> skip it for gotools
     if (any(True for line in configure_lines if line.startswith('AM_INIT_AUTOMAKE'))
             and not str(folder).endswith('gotools')):
         subprocess.check_output(f'{ENV} {AUTOMAKE_BIN} -f',
                                 shell=True, encoding='utf8')
+
     subprocess.check_output(f'{ENV} {AUTOCONF_BIN} -f', shell=True, encoding='utf8')
 
-- 
2.44.0

Re: [PATCH v2 1/2] autoregen.py: adapt to Gentoo

[Mark Wielaard]

Hi Sam,

On Sat, Mar 02, 2024 at 06:02:19AM +0000, Sam James wrote:
> In Gentoo, we have autoconf-vanilla and automake-vanilla packaged for toolchain
> development with no patches applied. The binaries are named differently.
> 
> Adapt the script to check a list of binaries for each tool, and check the
> vanilla names first as we want to prefer those if they exist.
> 
> This makes the script work out-of-the-box on Gentoo.
> 
> Signed-off-by: Sam James <sam@gentoo.org>
> ---
> mjw, could you double check this does the right thing for you on your systems?
> 
> It should be OK as I've checked with a few combinations.
> 
> v2:
> - Set WANT_{AUTOCONF,AUTOMAKE} environment variables for Gentoo's wrappers.
> - Add some style tweaks in patch 2.

Work great for me too. Pushed both patches.

Thanks,

Mark