Re: [PATCH] ci: Check for necessary Debian packages when running build-many-glibcs.py

[Adhemerval Zanella Netto <adhemerval.zanella@linaro.org>]

On 26/05/23 19:12, DJ Delorie wrote:
> 
> [working my way through old patches ;-]
> 
> https://patchwork.sourceware.org/project/glibc/patch/20211108165255.15600-1-lukma@denx.de/
> https://patchwork.sourceware.org/project/glibc/patch/20211108201931.17159-1-lukma@denx.de/
> 
> Lukasz Majewski <lukma@denx.de> writes:
>> I thought that the above code is so different from mine, that you would
>> like to prepare patch for it?
> 
> How about something like this?  I filled out Adhemerval's sample code
> with some error checking, added some user messages, and ended up with
> the attached.  I split it up so that the *list* of required tools is at
> the beginning of the file (easily findable) but the *code* to handle it
> is near the end.
> 
> I also thought of adding two more columns to the required_tools table,
> one with common RPM package names, and one with common DEB package
> names.  This would purely be for convenience, although I assume most bmg
> users know how to find which package an executable is in.  Might be
> trickier for libraries, if we add "required libraries" to the list
> later.
> 
> I also considered moving the minimum-version list to the beginning, to
> make it easier to find, but decided that should be separate.

The patch organization sounds good to me, thanks for working on this.

> 
> 
> diff --git a/scripts/build-many-glibcs.py b/scripts/build-many-glibcs.py
> index 95726c4a29..0f35c62b7a 100755
> --- a/scripts/build-many-glibcs.py
> +++ b/scripts/build-many-glibcs.py
> @@ -56,6 +56,20 @@ import sys
>  import time
>  import urllib.request
>  
> +def get_list_of_required_tools():
> +    global REQUIRED_TOOLS
> +    REQUIRED_TOOLS = {
> +        'awk'      : (get_version_awk, (3,1,2)),
> +        'bison'    : (get_version,     (2,7)),
> +        'flex'     : (get_version,     (2,6,0)),
> +        'git'      : (get_version,     (2,32)),
> +        'make'     : (get_version,     (4,0)),
> +        'makeinfo' : (get_version,     (4,7)),
> +        'patch'    : (get_version,     (2,7,0)),
> +        'sed'      : (get_version,     (3,2)),
> +        'tar'      : (get_version,     (1,3,4)),

The script untar gz, bz2, and xz files, should it check for the associated
tools as well?

> +    }
> +
>  try:
>      subprocess.run
>  except:
> @@ -1865,8 +1879,66 @@ def get_parser():
>      return parser
>  
>  
> +def get_version_common(progname,line,word,delchars):
> +    try:
> +        out = subprocess.run([progname, '--version'],
> +                             stdout=subprocess.PIPE,
> +                             stderr=subprocess.DEVNULL,
> +                             check=True, universal_newlines=True).stdout
> +        v = out.splitlines()[line].split()[word]
> +        if delchars:
> +            v = v.replace(delchars,'')
> +        return [int(x) for x in v.split('.')]
> +    except:
> +        return 'missing';
> +
> +def get_version(progname):
> +    return get_version_common (progname, 0, -1, None);
> +
> +def get_version_awk(progname):
> +    return get_version_common (progname, 0, 2, ',');
> +
> +def check_version(ver, req):
> +    for v, r in zip(ver, req):
> +        if v > r:
> +            return True
> +        if v < r:
> +            return False
> +    return True
> +
> +def version_str(ver):
> +    return '.'.join([str (x) for x in ver])
> +
> +def check_for_required_tools():
> +    get_list_of_required_tools()
> +    count_old_tools = 0
> +    count_missing_tools = 0
> +    
> +    for k, v in REQUIRED_TOOLS.items():
> +        version = v[0](k)
> +        if version == 'missing':
> +            ok = 'missing'
> +        else:
> +            ok = 'ok' if check_version (version, v[1]) else 'old'
> +        if ok == 'old':
> +            if count_old_tools == 0:
> +                print("One or more required tools are too old:")
> +            count_old_tools = count_old_tools + 1
> +            print('{:9}: {:3} (obtained=\"{}\" required=\"{}\")'.format(k, ok,
> +                    version_str(version), version_str(v[1])))
> +        if ok == 'missing':
> +            if count_missing_tools == 0:
> +                print("One or more required tools are missing:")
> +            count_missing_tools = count_missing_tools + 1
> +            print('{:9}: {:3} (required=\"{}\")'.format(k, ok,
> +                    version_str(v[1])))
> +    
> +    if count_old_tools > 0 or count_missing_tools > 0:
> +        exit (1);
> +    
>  def main(argv):
>      """The main entry point."""
> +    check_for_required_tools();
>      parser = get_parser()
>      opts = parser.parse_args(argv)
>      topdir = os.path.abspath(opts.topdir)
>