* [PATCH] RISC-V: Extend syntax for the multilib-generator
@ 2020-10-16 9:34 Kito Cheng
2020-10-22 2:36 ` Jim Wilson
0 siblings, 1 reply; 4+ messages in thread
From: Kito Cheng @ 2020-10-16 9:34 UTC (permalink / raw)
To: gcc-patches, kito.cheng, jimw, andrew; +Cc: Kito Cheng
- Support expansion operator (*) in the multilib config string.
- Motivation of this patch is reduce the complexity when we deal multilib with
sub-extension, expand the combinations by hand would be very painful and
error prone, no one deserve to experience this[1] again!
[1] https://github.com/sifive/freedom-tools/blob/f4d7facafb27d16125768c90ff1790c674e4be7a/Makefile#L348
gcc/ChangeLog:
* config/riscv/multilib-generator: Add TODO, import itertools
and functools.reduce.
Handle expantion operator.
(LONG_EXT_PREFIXES): New.
(arch_canonicalize): Update comment and improve python3
debuggability/compatibility.
(add_underline_prefix): New.
(_expand_combination): Ditto.
(unique): Ditto.
(expand_combination): Ditto.
---
gcc/config/riscv/multilib-generator | 109 ++++++++++++++++++++++++++--
1 file changed, 102 insertions(+), 7 deletions(-)
diff --git a/gcc/config/riscv/multilib-generator b/gcc/config/riscv/multilib-generator
index ade3e6e97725..d31e52bccd61 100755
--- a/gcc/config/riscv/multilib-generator
+++ b/gcc/config/riscv/multilib-generator
@@ -22,14 +22,26 @@
# Each argument to this script is of the form
# <primary arch>-<abi>-<additional arches>-<extensions>
-# For example,
+# Example 1:
# rv32imafd-ilp32d-rv32g-c,v
# means that, in addition to rv32imafd, these configurations can also use the
# rv32imafd-ilp32d libraries: rv32imafdc, rv32imafdv, rv32g, rv32gc, rv32gv
+#
+# Example 2:
+# rv32imafd-ilp32d--c*b
+# means that, in addition to rv32imafd, these configurations can also use the
+# rv32imafd-ilp32d libraries: rv32imafd-ilp32dc, rv32imafd-ilp32db,
+# rv32imafd-ilp32dcb
from __future__ import print_function
import sys
import collections
+import itertools
+from functools import reduce
+
+#
+# TODO: Add test for this script.
+#
arches = collections.OrderedDict()
abis = collections.OrderedDict()
@@ -37,6 +49,7 @@ required = []
reuse = []
canonical_order = "mafdgqlcbjtpvn"
+LONG_EXT_PREFIXES = ['z', 's', 'h', 'x']
#
# IMPLIED_EXT(ext) -> implied extension list.
@@ -59,14 +72,13 @@ def arch_canonicalize(arch):
# TODO: Support extension version.
new_arch = ""
if arch[:5] in ['rv32e', 'rv32i', 'rv32g', 'rv64i', 'rv64g']:
- # TODO: We should expand g to imadzifencei once we support newer spec.
+ # TODO: We should expand g to imad_zifencei once we support newer spec.
new_arch = arch[:5].replace("g", "imafd")
else:
raise Exception("Unexpected arch: `%s`" % arch[:5])
# Find any Z, S, H or X
- long_ext_prefixes = ['z', 's', 'h', 'x']
- long_ext_prefixes_idx = map(lambda x: arch.find(x), long_ext_prefixes)
+ long_ext_prefixes_idx = map(lambda x: arch.find(x), LONG_EXT_PREFIXES)
# Filter out any non-existent index.
long_ext_prefixes_idx = list(filter(lambda x: x != -1, long_ext_prefixes_idx))
@@ -93,7 +105,7 @@ def arch_canonicalize(arch):
std_exts += list(filter(lambda x:len(x) == 1, long_exts))
# Multi-letter extension must be in lexicographic order.
- long_exts = sorted(filter(lambda x:len(x) != 1, long_exts))
+ long_exts = list(sorted(filter(lambda x:len(x) != 1, long_exts)))
# Put extensions in canonical order.
for ext in canonical_order:
@@ -112,6 +124,85 @@ def arch_canonicalize(arch):
new_arch += "_" + "_".join(long_exts)
return new_arch
+#
+# add underline for each multi-char extensions.
+# e.g. ["a", "zfh"] -> ["a", "_zfh"]
+#
+def add_underline_prefix(ext):
+ for long_ext_prefix in LONG_EXT_PREFIXES:
+ if ext.startswith(long_ext_prefix):
+ return "_" + ext
+
+ return ext
+
+#
+# Handle expansion operation.
+#
+# e.g. "a*b" -> [("a",), ("b",), ("a", "b")]
+# "a" -> [("a",)]
+#
+def _expand_combination(ext):
+ exts = list(ext.split("*"))
+
+ # No need to expand if there is no `*`.
+ if len(exts) == 1:
+ return [(exts[0],)]
+
+ # Add underline to every extension.
+ # e.g.
+ # _b * zvamo => _b * _zvamo
+ exts = list(map(lambda x: '_' + x, exts))
+
+ # Generate combination!
+ ext_combs = []
+ for comb_len in range(1, len(exts)+1):
+ for ext_comb in itertools.combinations(exts, comb_len):
+ ext_combs.append(ext_comb)
+
+ return ext_combs
+
+#
+# Input a list and drop duplicated entry.
+# e.g.
+# ["a", "b", "ab", "a"] -> ["a", "b", "ab"]
+#
+def unique(x):
+ #
+ # Drop duplicated entry.
+ # Convert list to set and then convert back to list.
+ #
+ # Add sorted to prevent non-deterministic results in different env.
+ #
+ return list(sorted(list(set(x))))
+
+#
+# Expand EXT string if there is any expansion operator (*).
+# e.g.
+# "a*b,c" -> ["a", "b", "ab", "c"]
+#
+def expand_combination(ext):
+ ext = list(filter(None, ext.split(',')))
+
+ # Expand combination for EXT, got lots of list.
+ # e.g.
+ # a * b => [[("a",), ("b",)], [("a", "b")]]
+ ext_combs = list(map(_expand_combination, ext))
+
+ # Then fold to single list.
+ # e.g.
+ # [[("a",), ("b",)], [("a", "b")]] => [("a",), ("b",), ("a", "b")]
+ ext = list(reduce(lambda x, y: x + y, ext_combs, []))
+
+ # Fold the tuple to string.
+ # e.g.
+ # [("a",), ("b",), ("a", "b")] => ["a", "b", "ab"]
+ ext = map(lambda e : reduce(lambda x, y: x + y, e), ext)
+
+ # Drop duplicated entry.
+ ext = unique(ext)
+
+ return ext
+
for cfg in sys.argv[1:]:
try:
(arch, abi, extra, ext) = cfg.split('-')
@@ -125,9 +216,13 @@ for cfg in sys.argv[1:]:
arches[arch] = 1
abis[abi] = 1
extra = list(filter(None, extra.split(',')))
- ext = list(filter(None, ext.split(',')))
- alts = sum([[x] + [x + "_" + y for y in ext] for x in [arch] + extra], [])
+ ext_combs = expand_combination(ext)
+ alts = sum([[x] + [x + y for y in ext_combs] for x in [arch] + extra], [])
alts = list(map(arch_canonicalize, alts))
+
+ # Drop duplicated entry.
+ alts = unique(alts)
+
for alt in alts[1:]:
arches[alt] = 1
reuse.append('march.%s/mabi.%s=march.%s/mabi.%s' % (arch, abi, alt, abi))
--
2.28.0
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] RISC-V: Extend syntax for the multilib-generator
2020-10-16 9:34 [PATCH] RISC-V: Extend syntax for the multilib-generator Kito Cheng
@ 2020-10-22 2:36 ` Jim Wilson
2020-10-22 2:36 ` Jim Wilson
0 siblings, 1 reply; 4+ messages in thread
From: Jim Wilson @ 2020-10-22 2:36 UTC (permalink / raw)
To: Kito Cheng; +Cc: GCC Patches, Kito Cheng, Andrew Waterman
On Fri, Oct 16, 2020 at 2:34 AM Kito Cheng <kito.cheng@sifive.com> wrote:
> +# Example 2:
> +# rv32imafd-ilp32d--c*b
> +# means that, in addition to rv32imafd, these configurations can also use
> the
> +# rv32imafd-ilp32d libraries: rv32imafd-ilp32dc, rv32imafd-ilp32db,
> +# rv32imafd-ilp32dcb
>
You accidentally added the c and b to the abi not the arch.
>
> from __future__ import print_function
> import sys
> import collections
> +import itertools
> +from functools import reduce
> +
> +#
> +# TODO: Add test for this script.
> +#
>
> arches = collections.OrderedDict()
> abis = collections.OrderedDict()
> @@ -37,6 +49,7 @@ required = []
> reuse = []
>
> canonical_order = "mafdgqlcbjtpvn"
> +LONG_EXT_PREFIXES = ['z', 's', 'h', 'x']
>
> #
> # IMPLIED_EXT(ext) -> implied extension list.
> @@ -59,14 +72,13 @@ def arch_canonicalize(arch):
> # TODO: Support extension version.
> new_arch = ""
> if arch[:5] in ['rv32e', 'rv32i', 'rv32g', 'rv64i', 'rv64g']:
> - # TODO: We should expand g to imadzifencei once we support newer spec.
> + # TODO: We should expand g to imad_zifencei once we support newer
> spec.
> new_arch = arch[:5].replace("g", "imafd")
> else:
> raise Exception("Unexpected arch: `%s`" % arch[:5])
>
> # Find any Z, S, H or X
> - long_ext_prefixes = ['z', 's', 'h', 'x']
> - long_ext_prefixes_idx = map(lambda x: arch.find(x), long_ext_prefixes)
> + long_ext_prefixes_idx = map(lambda x: arch.find(x), LONG_EXT_PREFIXES)
>
> # Filter out any non-existent index.
> long_ext_prefixes_idx = list(filter(lambda x: x != -1,
> long_ext_prefixes_idx))
> @@ -93,7 +105,7 @@ def arch_canonicalize(arch):
> std_exts += list(filter(lambda x:len(x) == 1, long_exts))
>
> # Multi-letter extension must be in lexicographic order.
> - long_exts = sorted(filter(lambda x:len(x) != 1, long_exts))
> + long_exts = list(sorted(filter(lambda x:len(x) != 1, long_exts)))
>
> # Put extensions in canonical order.
> for ext in canonical_order:
> @@ -112,6 +124,85 @@ def arch_canonicalize(arch):
> new_arch += "_" + "_".join(long_exts)
> return new_arch
>
> +#
> +# add underline for each multi-char extensions.
> +# e.g. ["a", "zfh"] -> ["a", "_zfh"]
> +#
> +def add_underline_prefix(ext):
> + for long_ext_prefix in LONG_EXT_PREFIXES:
> + if ext.startswith(long_ext_prefix):
> + return "_" + ext
> +
> + return ext
> +
> +#
> +# Handle expansion operation.
> +#
> +# e.g. "a*b" -> [("a",), ("b",), ("a", "b")]
> +# "a" -> [("a",)]
> +#
> +def _expand_combination(ext):
> + exts = list(ext.split("*"))
> +
> + # No need to expand if there is no `*`.
> + if len(exts) == 1:
> + return [(exts[0],)]
> +
> + # Add underline to every extension.
> + # e.g.
> + # _b * zvamo => _b * _zvamo
> + exts = list(map(lambda x: '_' + x, exts))
> +
> + # Generate combination!
> + ext_combs = []
> + for comb_len in range(1, len(exts)+1):
> + for ext_comb in itertools.combinations(exts, comb_len):
> + ext_combs.append(ext_comb)
> +
> + return ext_combs
> +
> +#
> +# Input a list and drop duplicated entry.
> +# e.g.
> +# ["a", "b", "ab", "a"] -> ["a", "b", "ab"]
> +#
> +def unique(x):
> + #
> + # Drop duplicated entry.
> + # Convert list to set and then convert back to list.
> + #
> + # Add sorted to prevent non-deterministic results in different env.
> + #
> + return list(sorted(list(set(x))))
> +
> +#
> +# Expand EXT string if there is any expansion operator (*).
> +# e.g.
> +# "a*b,c" -> ["a", "b", "ab", "c"]
> +#
> +def expand_combination(ext):
> + ext = list(filter(None, ext.split(',')))
> +
> + # Expand combination for EXT, got lots of list.
> + # e.g.
> + # a * b => [[("a",), ("b",)], [("a", "b")]]
> + ext_combs = list(map(_expand_combination, ext))
> +
> + # Then fold to single list.
> + # e.g.
> + # [[("a",), ("b",)], [("a", "b")]] => [("a",), ("b",), ("a", "b")]
> + ext = list(reduce(lambda x, y: x + y, ext_combs, []))
> +
> + # Fold the tuple to string.
> + # e.g.
> + # [("a",), ("b",), ("a", "b")] => ["a", "b", "ab"]
> + ext = map(lambda e : reduce(lambda x, y: x + y, e), ext)
> +
> + # Drop duplicated entry.
> + ext = unique(ext)
> +
> + return ext
> +
> for cfg in sys.argv[1:]:
> try:
> (arch, abi, extra, ext) = cfg.split('-')
> @@ -125,9 +216,13 @@ for cfg in sys.argv[1:]:
> arches[arch] = 1
> abis[abi] = 1
> extra = list(filter(None, extra.split(',')))
> - ext = list(filter(None, ext.split(',')))
> - alts = sum([[x] + [x + "_" + y for y in ext] for x in [arch] + extra],
> [])
> + ext_combs = expand_combination(ext)
> + alts = sum([[x] + [x + y for y in ext_combs] for x in [arch] + extra],
> [])
> alts = list(map(arch_canonicalize, alts))
> +
> + # Drop duplicated entry.
> + alts = unique(alts)
> +
> for alt in alts[1:]:
> arches[alt] = 1
> reuse.append('march.%s/mabi.%s=march.%s/mabi.%s' % (arch, abi, alt,
> abi))
> --
> 2.28.0
>
>
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] RISC-V: Extend syntax for the multilib-generator
2020-10-22 2:36 ` Jim Wilson
@ 2020-10-22 2:36 ` Jim Wilson
2020-10-22 8:04 ` Kito Cheng
0 siblings, 1 reply; 4+ messages in thread
From: Jim Wilson @ 2020-10-22 2:36 UTC (permalink / raw)
To: Kito Cheng; +Cc: GCC Patches, Kito Cheng, Andrew Waterman
On Wed, Oct 21, 2020 at 7:36 PM Jim Wilson <jimw@sifive.com> wrote:
>
>
> On Fri, Oct 16, 2020 at 2:34 AM Kito Cheng <kito.cheng@sifive.com> wrote:
>
>> +# Example 2:
>> +# rv32imafd-ilp32d--c*b
>> +# means that, in addition to rv32imafd, these configurations can also
>> use the
>> +# rv32imafd-ilp32d libraries: rv32imafd-ilp32dc, rv32imafd-ilp32db,
>> +# rv32imafd-ilp32dcb
>>
>
> You accidentally added the c and b to the abi not the arch.
>
otherwise this looks good, and very useful.
Jim
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] RISC-V: Extend syntax for the multilib-generator
2020-10-22 2:36 ` Jim Wilson
@ 2020-10-22 8:04 ` Kito Cheng
0 siblings, 0 replies; 4+ messages in thread
From: Kito Cheng @ 2020-10-22 8:04 UTC (permalink / raw)
To: Jim Wilson; +Cc: Kito Cheng, GCC Patches, Andrew Waterman
Committed with comment fix, thanks :)
On Thu, Oct 22, 2020 at 10:37 AM Jim Wilson <jimw@sifive.com> wrote:
>
> On Wed, Oct 21, 2020 at 7:36 PM Jim Wilson <jimw@sifive.com> wrote:
>
> >
> >
> > On Fri, Oct 16, 2020 at 2:34 AM Kito Cheng <kito.cheng@sifive.com> wrote:
> >
> >> +# Example 2:
> >> +# rv32imafd-ilp32d--c*b
> >> +# means that, in addition to rv32imafd, these configurations can also
> >> use the
> >> +# rv32imafd-ilp32d libraries: rv32imafd-ilp32dc, rv32imafd-ilp32db,
> >> +# rv32imafd-ilp32dcb
> >>
> >
> > You accidentally added the c and b to the abi not the arch.
> >
>
> otherwise this looks good, and very useful.
>
> Jim
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2020-10-22 8:04 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-10-16 9:34 [PATCH] RISC-V: Extend syntax for the multilib-generator Kito Cheng
2020-10-22 2:36 ` Jim Wilson
2020-10-22 2:36 ` Jim Wilson
2020-10-22 8:04 ` Kito Cheng
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).