From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by sourceware.org (Postfix) with ESMTPS id 31D403858C66 for ; Wed, 10 May 2023 10:35:45 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 31D403858C66 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-wr1-x435.google.com with SMTP id ffacd0b85a97d-306f2b42a86so4526361f8f.3 for ; Wed, 10 May 2023 03:35:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1683714944; x=1686306944; h=in-reply-to:from:content-language:references:to:subject:user-agent :mime-version:date:message-id:from:to:cc:subject:date:message-id :reply-to; bh=UktguHAZEkOZDa1LWpbdYucU8oSCySqg0RNq463bx/8=; b=b5hms57TqWamfoeSsWZ+sx1qaAs/jFxy98LGNKDL0b3g+nJb1yOomGJpWrw5haU5bb OcaOsU427hW75j264QFUBRXk+21YyFCQGbiSTyKCpeM04LsTQjpxXARUKfVioOQ96HLg iUCY2jyxcmrwBxkyCcN4AGPNhqO1j23rUcjxtk8N4LYiZBLuRrdmkAbAGBXU2wObXJZZ kjzpSw3xSqEnhtaPThErjbaji5bVAjUr9MvsQNKHbBxvpfajQfYCgq8tRZkg+By5xQmZ ks+LQyPXhmkAy0AFWBNy4SI8os1lwC5cW6jkowlT1I8yGRusys2kmxdAKkqxeOHDelA+ hOUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683714944; x=1686306944; h=in-reply-to:from:content-language:references:to:subject:user-agent :mime-version:date:message-id:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=UktguHAZEkOZDa1LWpbdYucU8oSCySqg0RNq463bx/8=; b=FvY3QwW4+PWjMPpCBxApi8QZukz4/nFm7yMuPbVsmCtZTTz8WdbeOOauBn9Kdl5ok1 UvV5pn0wp9RsffbUhQ4cugq9XgDKG+0/UMF5m7cGJXb4l30o3sfFrWk4hiCK6bCT4vPw lC24MrIW8N287byNCpoHxcH4CTDBeRzSui7x1+5YiysLEOZlkf2D0gcEHobEjyQi51fd YRe5yYThczMcuNktOBTqh70oN/i1h8/q+IfpLofODAtwWqc0CWw9vPi0lhUg4tsa3ZQi KviXIiee/fIfh0kOzMrmoH+z3dCMktl5okf1vSCS2KkOxwABWZJ7r1jJnXBX8Kk1tKLK 7K7Q== X-Gm-Message-State: AC+VfDxICue+ocn9xBDRivEF9f/phx+TYBNIfrj+9PxhngxwwNjjgwYk R3zcnCIDqc4xHkAeNhHBrTM= X-Google-Smtp-Source: ACHHUZ74nIRr1a/9UEyhzyjPDQdxbsM7TWD9r0TFxPGISfsUVRlSzx/1gZ9aBUFyCWQlI8v0o1rfgQ== X-Received: by 2002:adf:f00a:0:b0:306:3286:69a2 with SMTP id j10-20020adff00a000000b00306328669a2mr11940099wro.48.1683714943629; Wed, 10 May 2023 03:35:43 -0700 (PDT) Received: from [192.168.0.160] ([170.253.51.134]) by smtp.gmail.com with ESMTPSA id j18-20020adfff92000000b00307bc4e39e5sm895933wrr.117.2023.05.10.03.35.42 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 10 May 2023 03:35:42 -0700 (PDT) Message-ID: Date: Wed, 10 May 2023 12:35:34 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: [PATCH v2 1/2] scripts: Add sort-makefile-lines.py to sort Makefile variables. To: Carlos O'Donell , libc-alpha@sourceware.org, siddhesh@gotplt.org References: <20230509185152.2326078-1-carlos@redhat.com> <20230509185152.2326078-2-carlos@redhat.com> Content-Language: en-US From: Alejandro Colomar In-Reply-To: <20230509185152.2326078-2-carlos@redhat.com> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="------------dOVf4af3hfqX2GNbYMcq83dg" X-Spam-Status: No, score=-11.7 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,GIT_PATCH_0,KAM_SHORT,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --------------dOVf4af3hfqX2GNbYMcq83dg Content-Type: multipart/mixed; boundary="------------26R70BtnVS9OMmfFnJ70CN1h"; protected-headers="v1" From: Alejandro Colomar To: Carlos O'Donell , libc-alpha@sourceware.org, siddhesh@gotplt.org Message-ID: Subject: Re: [PATCH v2 1/2] scripts: Add sort-makefile-lines.py to sort Makefile variables. References: <20230509185152.2326078-1-carlos@redhat.com> <20230509185152.2326078-2-carlos@redhat.com> In-Reply-To: <20230509185152.2326078-2-carlos@redhat.com> --------------26R70BtnVS9OMmfFnJ70CN1h Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi Carlos, On 5/9/23 20:51, Carlos O'Donell via Libc-alpha wrote: > The scripts/sort-makefile-lines.py script sorts Makefile variables > according to project expected order. >=20 > The script can be used like this: >=20 > $ scripts/sort-makefile-lines.py -i elf/Makefile -o elf/Makefile I suggest a simpler interface: stdin and stdout $ scripts/sort-makefile-lines.py --- > scripts/sort-makefile-lines.py | 178 +++++++++++++++++++++++++++++++++= > 1 file changed, 178 insertions(+) > create mode 100755 scripts/sort-makefile-lines.py >=20 > diff --git a/scripts/sort-makefile-lines.py b/scripts/sort-makefile-lin= es.py > new file mode 100755 > index 0000000000..8e25dad5d7 > --- /dev/null > +++ b/scripts/sort-makefile-lines.py > @@ -0,0 +1,178 @@ > +#!/usr/bin/python3 > +# Sort Makefile lines as expected by project policy. > +# Copyright (C) 2023 Free Software Foundation, Inc. > +# This file is part of the GNU C Library. > +# > +# The GNU C Library is free software; you can redistribute it and/or > +# modify it under the terms of the GNU Lesser General Public > +# License as published by the Free Software Foundation; either > +# version 2.1 of the License, or (at your option) any later version. > +# > +# The GNU C Library is distributed in the hope that it will be useful,= > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > +# Lesser General Public License for more details. > +# > +# You should have received a copy of the GNU Lesser General Public > +# License along with the GNU C Library; if not, see > +# . > + > +# The project consensus is to split Makefile variable assignment > +# across multiple lines with one value per line. The values are > +# then sorted as described below, and terminated with a special > +# list termination marker. This splitting makes it much easier > +# to add new tests to the list since they become just a single > +# line insertion. It also makes backports and merges easier > +# since the new test may not conflict due to the ordering. > +# > +# Consensus discussion: > +# https://inbox.sourceware.org/libc-alpha/f6406204-84f5-adb1-d00e-979e= beebbbde@redhat.com/ > +# > +# To support cleaning up Makefiles we created this program to > +# help sort existing lists converted to the new format. > +# > +# The program takes as input the Makefile to sort correctly, > +# and the output file to write the correctly sorted output > +# (it can be the same file). > +# > +# Sorting is only carried out between two special markers: > +# (a) Marker start is ' +=3D \' (or '=3D \', or ':=3D \') > +# (b) Marker end is ' # ' (whitespace matters) > +# With everthing between (a) and (b) being sorted accordingly. > +# > +# You can use it like this: > +# $ scripts/sort-makefile-lines.py -i elf/Makefile -o elf/Makefile > +# > +# The Makefile lines in the project are sorted using the > +# following rules: > +# - All lines are sorted as-if `LC_COLLATE=3DC sort` > +# - Lines that have a numeric suffix and whose leading prefix > +# matches exactly are sorted according the numeric suffix > +# in increasing numerical order. > +# > +# For example: > +# ~~~ > +# tests +=3D \ > +# test-a \ > +# test-b \ > +# test-b1 \ > +# test-b2 \ > +# test-b10 \ > +# test-b20 \ > +# test-b100 \ > +# # tests > +# ~~~ > +# This example shows tests sorted alphabetically, followed > +# by a numeric suffix sort in increasing numeric order. > +# > +# Cleanups: > +# - Tests that end in "a" or "b" variants should be renamed to > +# end in just the numerical value. For example 'tst-mutex7robust' > +# should be renamed to 'tst-mutex12' (the highest numbered test) > +# or 'tst-robust11' (the highest numbered test) in order to get > +# reasonable ordering. > +# - Modules that end in "mod" or "mod1" should be renamed. For > +# example 'tst-atfork2mod' should be renamed to 'tst-mod-atfork2' > +# (test module for atfork2). If there are more than one module > +# then they should be named with a suffix that uses [0-9] first > +# then [A-Z] next for a total of 36 possible modules per test. > +# No manually listed test currently uses more than that (though > +# automatically generated tests may; they don't need sorting). > +# - Avoid including another test and instead refactor into common > +# code with all tests including hte common code, then give the > +# tests unique names. > +# > +# If you have a Makefile that needs converting, then you can > +# quickly split the values into one-per-line, ensure the start > +# and end markers are in place, and then run the script to > +# sort the values. > + > +import argparse > +import sys > +import locale > +import re > +import functools > + > +def glibc_makefile_numeric(string1, string2): > + # Check if string1 has a numeric suffix. > + var1 =3D re.search(r'([0-9]+) \\$', string1) > + var2 =3D re.search(r'([0-9]+) \\$', string2) > + if var1 and var2: > + if string1[0:var1.span()[0]] =3D=3D string2[0:var2.span()[0]]:= > + # string1 and string2 both share a prefix and > + # have a numeric suffix that can be compared. > + # Sort order is based on the numeric suffix. > + return int(var1.group(1)) > int(var2.group(1)) > + # Default to strcoll. > + return locale.strcoll(string1, string2) > + > +def sort_lines(lines): > + > + # Use the C locale for language independent collation. > + locale.setlocale (locale.LC_ALL, "C") > + > + # Sort using a glibc-specific sorting function. > + lines =3D sorted(lines, key=3Dfunctools.cmp_to_key(glibc_makefile_= numeric)) > + > + return lines > + > +def sort_makefile_lines(infile, outfile): > + > + # Read the whole Makefile. > + mfile =3D open(infile) > + lines =3D mfile.readlines() > + mfile.close() > + > + # We will output the Makefile here. Open it early to check > + # for any errors. > + ofile =3D open(outfile, "w") > + > + # Build a list of all start markers (tuple includes name). > + startmarks =3D [] > + for i in range(len(lines)): > + # Look for things like "var =3D \", "var :=3D \" or "var +=3D = \" > + # to start the sorted list. > + var =3D re.search(r'^([a-zA-Z0-9-]*) [\+:]?\=3D \\$', lines[i]= ) > + if var: > + # Remember the index and the name. > + startmarks.append((i, var.group(1))) > + > + # For each start marker try to find a matching end mark > + # and build a block that needs sorting. The end marker > + # must have the matching comment name for it to be valid. > + rangemarks =3D [] > + for sm in startmarks: > + # Look for things like " # var" to end the sorted list. > + reg =3D r'^ # ' + sm[1] + r'$' > + for j in range(sm[0] + 1, len(lines)): > + if re.search(reg, lines[j]): > + # Rembember the block to sort (inclusive). > + rangemarks.append((sm[0] + 1, j)) > + break > + > + # We now have a list of all ranges that need sorting. > + # Sort those ranges (inclusive). > + for r in rangemarks: > + lines[r[0]:r[1]] =3D sort_lines(lines[r[0]:r[1]]) > + > + # Output the whole list with sorted lines. > + for line in lines: > + ofile.write(line) > + > + ofile.close() > + > +def get_parser(): > + parser =3D argparse.ArgumentParser(description=3D__doc__) > + parser.add_argument('-i', dest=3D'infile', > + help=3D'Input Makefile to read lines from') > + parser.add_argument('-o', dest=3D'outfile', > + help=3D'Output Makefile to write sorted lines = to') > + return parser > + > +def main(argv): > + parser =3D get_parser() > + opts =3D parser.parse_args(argv) > + sort_makefile_lines (opts.infile, opts.outfile) > + > +if __name__ =3D=3D '__main__': > + main(sys.argv[1:]) --=20 GPG key fingerprint: A9348594CE31283A826FBDD8D57633D441E25BB5 --------------26R70BtnVS9OMmfFnJ70CN1h-- --------------dOVf4af3hfqX2GNbYMcq83dg Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEE6jqH8KTroDDkXfJAnowa+77/2zIFAmRbc3YACgkQnowa+77/ 2zKoTBAApyOGfiNNAc/uQ85uJErEFKNlU086R8/KoZHinOqVPZv2nke6urYFfdaT Liinn1JgiR9X/PYWyc8S2ynlh9r6DHfQ0ZUp+b1d/ClCw3mfEpvEbsVtwm0MNVGl pDxSqztc/mR390QtCe+hslrZttAwIHaAvpFqdHBNjShGRtyH3608vYEKr7rrH3ae JzlaGyeO1DfSL6W25DmBtzaY8m+X/oW79LHqGXxQUst2bSXe/Hv70ighoWRfYohS Ac3DGZKCq/sjc56V4q7YSYN8MiMjQCOmPBu4JvKFA5zUu2EMTQpgOMDe/MvXre5p 1BRyaSnn04GZ2zf3LmHkYd6qrUU9sGq93sl8MHMWoivm5c0YwHe3ehXX3t5AeXwA LCXjItFOFkp+ieRryCwlGs6s9dUgK67iKJqwQBTT4E2F3YT9Gw4SKmIV5KHWFxuI +rDFl0NJvN2nW3S3Pw3hozlIYfumHu2qSe254ZvrrerNVCMYLfQVKcdxTaK0eQ1+ ep+1QxLX6dZnp7L0n3RbdyYWnRnZi/nNJyBzdr6LUotDgJ5ZvZpTSnWhGJx1c+Nn CfO78sdV2Nj3LCENj1ReOI/djifzcRZgQx3WB7YkPxqFhP7g1arB4OSDPsZ9/6Bg RsBfyDA2UW/bPHT82umHC4iolZr22wTdDX5pg5O/qf6K1WOn1KU= =uqOt -----END PGP SIGNATURE----- --------------dOVf4af3hfqX2GNbYMcq83dg--