[PING] nvptx: Allow '--with-arch' to override the default '-misa' (was: nvptx multilib setup)

[Thomas Schwinge <thomas@codesourcery.com>]

[-- Attachment #1: Type: text/plain, Size: 5077 bytes --]

Hi Tom!

Ping.


Grüße
 Thomas


On 2022-06-15T23:18:10+0200, I wrote:
> Hi Tom!
>
> On 2022-05-13T16:20:14+0200, I wrote:
>> On 2022-02-04T13:09:29+0100, Tom de Vries via Gcc <gcc@gcc.gnu.org> wrote:
>>> On 2/4/22 08:21, Thomas Schwinge wrote:
>>>> On 2022-02-03T13:35:55+0000, "vries at gcc dot gnu.org via Gcc-bugs" <gcc-bugs@gcc.gnu.org> wrote:
>>>>> I've tested this using (recommended) driver 470.94 on boards:
>>
>>>>> while iterating over dimensions { -mptx=3.1 , -mptx=6.3 } x { GOMP_NVPTX_JIT=-O0, <default> }.
>>>>
>>>> Do you use separate (nvptx-none offload target only?) builds for
>>>> different '-mptx' variants (likewise: '-misa'), or have you hacked up the
>>>> multilib configuration?
>>>
>>> Neither, I'm using --target_board=unix/foffload= for that.
>>
>> ACK, I see.  So these flags then only affect GCC/nvptx code generation
>> for the actual user code (here: GCC libgomp test cases), but for the
>> GCC/nvptx target libraries (such as: libc, libm, libgfortran, libgomp --
>> the latter especially relevant for OpenMP), it uses PTX code from one of
>> the two "pre-compiled" GCC/nvptx multilibs: default or '-mptx=3.1'.
>>
>> Meaning, one can't just use such a flag for "completely building code"
>> for a specific configuration.  Random example,
>> '-foffload-options=nvptx-none=-march=sm_75': as GCC/nvptx target
>> libraries aren't being built for '-march=sm_75' multilib,
>> '-foffload-options=nvptx-none=-march=sm_75' uses the default multilib,
>> which isn't '-march=sm_75'.
>>
>>
>>>   ('gcc/config/nvptx/t-nvptx:MULTILIB_OPTIONS'
>>>> etc., I suppose?)  Should we add a few representative configurations to
>>>> be built by default?  And/or, should we have a way to 'configure' per
>>>> user needs (I suppose: '--with-multilib-list=[...]', as supported for a
>>>> few other targets?)?  (I see there's also a new
>>>> '--with-multilib-generator=[...]', haven't looked in detail.)  No matter
>>>> which way: again, combinatorial explosion is a problem, of course...
>>>
>>> As far as I know, the gcc build doesn't finish when switching default to
>>> higher than sm_35, so there's little point to go to a multilib setup at
>>> this point.  But once we fix that, we could reconsider, otherwise,
>>> things are likely to regress again.
>>
>> As far as I remember, several issues have been fixed.  Still waiting for
>> Roger's "middle-end: Support ABIs that pass FP values as wider integers"
>> or something similar, but that PR104489 issue is being worked around by
>> "Limit HFmode support to mexperimental", if I got that right.
>>
>> Now I'm not suggesting we should now enable all or any random GCC/nvptx
>> multilibs, to get all these variants of GCC/nvptx target libraries built;
>> especially also given that GCC/nvptx code generation currently doesn't
>> make too much use of the new capabilities.
>>
>> However, we do have a specific request that a customer would like to be
>> able to change at GCC 'configure' time the GCC/nvptx default multilib
>> (including that being used for building corresponding GCC/nvptx target
>> libraries).
>>
>> Per 'gcc/doc/install.texi', I do see that some GCC targets allow for
>> GCC 'configure'-time '--with-multilib-list=[...]', or
>> '--with-multilib-generator=[...]', and I suppose we could be doing
>> something similar?  But before starting implementing, I'd like your
>> input, as you'll be the one to approve in the end.  And/or, maybe you've
>> already made up your own ideas about that?
>
> So, instead of "random GCC/nvptx multilib configuration" (last
> paragraph), I've come up with a way to implement our customer's request
> (second last paragraph): 'configure' GCC/nvptx '--with-arch=sm_70'.
>
> I think I've implemented this in a way so that "random GCC/nvptx multilib
> configuration" may eventually be implemented on top of that.  For easy
> review/testing I've split my changes into three commits, see attached
> "nvptx: Make default '-misa=sm_30' explicit",
> "nvptx: Introduce dummy multilib option for default '-misa=sm_30'",
> "nvptx: Allow '--with-arch' to override the default '-misa'".
>
> To the best of my knowledge, the first two patches do not change any
> user-visible behavior (I generally 'diff'ed target libraries, and
> compared a good number of 'gcc -print-multi-directory [flags]'), and
> likewise with the third patch, given implicit (default) or explicit
> '--with-arch=sm_30', and that with '--with-arch=sm_70', for example, the
> '-misa=sm_70' multilib variants are used for implicit (default) or
> explicit '-misa=sm_70' or higher, and the '-misa=sm_30' multilib variants
> are used for explicit lower '-misa'.
>
> What do you think, OK to push to master branch?
>
>
> Grüße
>  Thomas


-----------------
Siemens Electronic Design Automation GmbH; Anschrift: Arnulfstraße 201, 80634 München; Gesellschaft mit beschränkter Haftung; Geschäftsführer: Thomas Heurung, Frank Thürauf; Sitz der Gesellschaft: München; Registergericht München, HRB 106955

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-nvptx-Make-default-misa-sm_30-explicit.patch --]
[-- Type: text/x-diff, Size: 2757 bytes --]

From ae782b3d1db7a9b2dea744d2a718bec94ac57677 Mon Sep 17 00:00:00 2001
From: Thomas Schwinge <thomas@codesourcery.com>
Date: Sat, 11 Jun 2022 12:28:36 +0200
Subject: [PATCH 1/3] nvptx: Make default '-misa=sm_30' explicit

... primarily in preparation for later changes.
---
 gcc/config.gcc             | 16 ++++++++++++++++
 gcc/config/nvptx/nvptx.cc  |  4 ++++
 gcc/config/nvptx/nvptx.h   |  4 ++++
 gcc/config/nvptx/nvptx.opt |  2 +-
 4 files changed, 25 insertions(+), 1 deletion(-)

diff --git a/gcc/config.gcc b/gcc/config.gcc
index 16d304286d7..3b1bde78391 100644
--- a/gcc/config.gcc
+++ b/gcc/config.gcc
@@ -4043,6 +4043,9 @@ if test x$with_arch = x ; then
     mips*-*-vxworks)
       with_arch=mips2
       ;;
+    nvptx-*)
+      with_arch=sm_30
+      ;;
   esac
 
   # Avoid overriding --with-arch-32 and --with-arch-64 values.
@@ -5436,6 +5439,19 @@ case "${target}" in
 			esac
 		;;
 
+	nvptx-*)
+		supported_defaults=arch
+		case $with_arch in
+			sm_30 )
+				# OK; default.
+				;;
+			* )
+				echo "Unknown arch used in --with-arch=$with_arch" 1>&2
+				exit 1
+				;;
+		esac
+		;;
+
 	powerpc*-*-* | rs6000-*-*)
 		supported_defaults="abi cpu cpu_32 cpu_64 float tune tune_32 tune_64 advance_toolchain"
 
diff --git a/gcc/config/nvptx/nvptx.cc b/gcc/config/nvptx/nvptx.cc
index e4297e2d6c3..071759b9582 100644
--- a/gcc/config/nvptx/nvptx.cc
+++ b/gcc/config/nvptx/nvptx.cc
@@ -335,6 +335,10 @@ nvptx_option_override (void)
 {
   init_machine_status = nvptx_init_machine_status;
 
+  /* Via nvptx 'OPTION_DEFAULT_SPECS', '-misa' always appears on the command
+     line.  */
+  gcc_checking_assert (OPTION_SET_P (ptx_isa_option));
+
   handle_ptx_version_option ();
 
   /* Set toplevel_reorder, unless explicitly disabled.  We need
diff --git a/gcc/config/nvptx/nvptx.h b/gcc/config/nvptx/nvptx.h
index b184f1d0150..629d4a5d8bc 100644
--- a/gcc/config/nvptx/nvptx.h
+++ b/gcc/config/nvptx/nvptx.h
@@ -27,6 +27,10 @@
 
 /* Run-time Target.  */
 
+/* Use '--with-arch' for default '-misa'.  */
+#define OPTION_DEFAULT_SPECS \
+  { "arch", "%{!misa=*:-misa=%(VALUE)}" }, \
+
 /* Assembler supports '-v' option; handle similar to
    '../../gcc.cc:asm_options', 'HAVE_GNU_AS'.  */
 #define ASM_SPEC "%{v}"
diff --git a/gcc/config/nvptx/nvptx.opt b/gcc/config/nvptx/nvptx.opt
index c5a5668fce5..71d3b68510b 100644
--- a/gcc/config/nvptx/nvptx.opt
+++ b/gcc/config/nvptx/nvptx.opt
@@ -53,7 +53,7 @@ Target Mask(GOMP)
 Generate code for OpenMP offloading: enables -msoft-stack and -muniform-simt.
 
 misa=
-Target RejectNegative ToLower Joined Enum(ptx_isa) Var(ptx_isa_option) Init(PTX_ISA_SM30)
+Target RejectNegative ToLower Joined Enum(ptx_isa) Var(ptx_isa_option)
 Specify the PTX ISA target architecture to use.
 
 march=
-- 
2.25.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0002-nvptx-Introduce-dummy-multilib-option-for-default-mi.patch --]
[-- Type: text/x-diff, Size: 1699 bytes --]

From 7889fb2d0ee1583a39ed1e2d4aadfd459edea9af Mon Sep 17 00:00:00 2001
From: Thomas Schwinge <thomas@codesourcery.com>
Date: Sat, 11 Jun 2022 12:28:36 +0200
Subject: [PATCH 2/3] nvptx: Introduce dummy multilib option for default
 '-misa=sm_30'

... primarily in preparation for later changes.
---
 gcc/config.gcc           |  1 +
 gcc/config/nvptx/t-nvptx | 20 +++++++++++++++++++-
 2 files changed, 20 insertions(+), 1 deletion(-)

diff --git a/gcc/config.gcc b/gcc/config.gcc
index 3b1bde78391..d492b7769fe 100644
--- a/gcc/config.gcc
+++ b/gcc/config.gcc
@@ -5441,6 +5441,7 @@ case "${target}" in
 
 	nvptx-*)
 		supported_defaults=arch
+		TM_MULTILIB_CONFIG=$with_arch
 		case $with_arch in
 			sm_30 )
 				# OK; default.
diff --git a/gcc/config/nvptx/t-nvptx b/gcc/config/nvptx/t-nvptx
index 2b68149ed7e..c797d57690f 100644
--- a/gcc/config/nvptx/t-nvptx
+++ b/gcc/config/nvptx/t-nvptx
@@ -31,4 +31,22 @@ s-nvptx-gen-opt: $(srcdir)/config/nvptx/nvptx-sm.def \
 	  tmp-nvptx-gen.opt $(srcdir)/config/nvptx/nvptx-gen.opt
 	$(STAMP) s-nvptx-gen-opt
 
-MULTILIB_OPTIONS = mgomp mptx=3.1
+
+# Multilib setup.
+
+MULTILIB_OPTIONS =
+MULTILIB_MATCHES =
+MULTILIB_EXCEPTIONS =
+
+MULTILIB_OPTIONS += mgomp
+
+multilib_options_isa_list := $(TM_MULTILIB_CONFIG)
+multilib_options_isa_default := $(word 1,$(multilib_options_isa_list))
+# Add the default '-misa' as a multilib option:
+MULTILIB_OPTIONS += misa=$(multilib_options_isa_default)
+# ..., but don't handle it specially (remap to default):
+MULTILIB_MATCHES += .=misa?$(multilib_options_isa_default)
+# ..., and don't actually build it:
+MULTILIB_EXCEPTIONS += *misa=$(multilib_options_isa_default)*
+
+MULTILIB_OPTIONS += mptx=3.1
-- 
2.25.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: 0003-nvptx-Allow-with-arch-to-override-the-default-misa.patch --]
[-- Type: text/x-diff, Size: 6427 bytes --]

From 8343b7d0ed96cf5d45c12b36ea34a156229b8745 Mon Sep 17 00:00:00 2001
From: Thomas Schwinge <thomas@codesourcery.com>
Date: Sat, 11 Jun 2022 19:37:10 +0200
Subject: [PATCH 3/3] nvptx: Allow '--with-arch' to override the default
 '-misa'

---
 gcc/config.gcc                           |  5 ++
 gcc/config/nvptx/gen-multilib-matches.sh | 60 ++++++++++++++++++++++++
 gcc/config/nvptx/t-nvptx                 | 21 +++++++--
 gcc/doc/install.texi                     |  9 ++++
 gcc/doc/invoke.texi                      |  4 +-
 5 files changed, 93 insertions(+), 6 deletions(-)
 create mode 100755 gcc/config/nvptx/gen-multilib-matches.sh

diff --git a/gcc/config.gcc b/gcc/config.gcc
index d492b7769fe..8f51d333668 100644
--- a/gcc/config.gcc
+++ b/gcc/config.gcc
@@ -5442,10 +5442,15 @@ case "${target}" in
 	nvptx-*)
 		supported_defaults=arch
 		TM_MULTILIB_CONFIG=$with_arch
+		#TODO 'sm_[...]' list per 'nvptx-sm.def'.
 		case $with_arch in
 			sm_30 )
 				# OK; default.
 				;;
+			sm_35 | sm_53 | sm_70 | sm_75 | sm_80 )
+				# OK, but we'd like 'sm_30', too.
+				TM_MULTILIB_CONFIG="$TM_MULTILIB_CONFIG sm_30"
+				;;
 			* )
 				echo "Unknown arch used in --with-arch=$with_arch" 1>&2
 				exit 1
diff --git a/gcc/config/nvptx/gen-multilib-matches.sh b/gcc/config/nvptx/gen-multilib-matches.sh
new file mode 100755
index 00000000000..9a5878e855b
--- /dev/null
+++ b/gcc/config/nvptx/gen-multilib-matches.sh
@@ -0,0 +1,60 @@
+#!/bin/sh
+
+# Print nvptx 'MULTILIB_MATCHES'
+
+# Copyright (C) 2022 Free Software Foundation, Inc.
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# GCC is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING3.  If not see
+# <http://www.gnu.org/licenses/>.
+
+set -e
+
+nvptx_sm_def="$1/nvptx-sm.def"
+multilib_options_isa_default=$2
+multilib_options_isa_list=$3
+
+sms=$(grep ^NVPTX_SM $nvptx_sm_def | sed 's/.*(//;s/,.*//')
+
+# Every variant in 'sms' has to either be remapped to the default variant
+# ('.', which is always built), or does get built as non-default variant
+# ('misa=sm_SM'; thus not remapped), or has to be remapped to the "next lower"
+# variant that does get built.
+
+# The "lowest" variant has to be built.
+sm_next_lower=INVALID
+
+for sm in $sms; do
+    if [ x"sm_$sm" = x"$multilib_options_isa_default" ]; then
+	sm_map=.
+    elif expr " $multilib_options_isa_list " : ".* sm_$sm " > /dev/null; then
+	sm_map=
+    else
+	sm_map=$sm_next_lower
+    fi
+
+    if [ x"$sm_map" = x ]; then
+	sm_next_lower=$sm
+    else
+	# Output format as required for 'MULTILIB_MATCHES'.
+	if [ x"$sm_map" = x. ]; then
+	    echo ".=misa?sm_$sm"
+	else
+	    echo "misa?sm_$sm_map=misa?sm_$sm"
+	fi
+
+	sm_next_lower=$sm_map
+    fi
+done
diff --git a/gcc/config/nvptx/t-nvptx b/gcc/config/nvptx/t-nvptx
index c797d57690f..9c5cbda0070 100644
--- a/gcc/config/nvptx/t-nvptx
+++ b/gcc/config/nvptx/t-nvptx
@@ -42,11 +42,22 @@ MULTILIB_OPTIONS += mgomp
 
 multilib_options_isa_list := $(TM_MULTILIB_CONFIG)
 multilib_options_isa_default := $(word 1,$(multilib_options_isa_list))
-# Add the default '-misa' as a multilib option:
-MULTILIB_OPTIONS += misa=$(multilib_options_isa_default)
-# ..., but don't handle it specially (remap to default):
-MULTILIB_MATCHES += .=misa?$(multilib_options_isa_default)
-# ..., and don't actually build it:
+multilib_options_misa_list := $(addprefix misa=,$(multilib_options_isa_list))
+# Add the requested '-misa' variants as a multilib option ('misa=VAR1/misa=VAR2/misa=VAR3' etc.):
+empty :=
+space := $(empty) $(empty)
+MULTILIB_OPTIONS += $(subst $(space),/,$(multilib_options_misa_list))
+# ..., and remap '-misa' variants as appropriate:
+multilib_matches := $(shell $(srcdir)/config/nvptx/gen-multilib-matches.sh $(srcdir)/config/nvptx $(multilib_options_isa_default) "$(multilib_options_isa_list)")
+MULTILIB_MATCHES += $(multilib_matches)
+# ..., and don't actually build what's the default '-misa':
 MULTILIB_EXCEPTIONS += *misa=$(multilib_options_isa_default)*
 
 MULTILIB_OPTIONS += mptx=3.1
+# Filter out invalid '-misa'/'-mptx=3.1' combinations; per 'nvptx-sm.def',
+# 'nvptx.opt:ptx_version', 'nvptx.cc:first_ptx_version_supporting_sm'
+# (that is, '-mptx=3.1' only for sm_30, sm_35 variants):
+MULTILIB_EXCEPTIONS += $(foreach misa,$(filter-out %=sm_30 %=sm_35,$(multilib_options_misa_list)),*$(misa)/mptx=3.1)
+# ..., and special care has to be taken if '-mptx=3.1' is invalid for the
+# default variant:
+MULTILIB_EXCEPTIONS += $(if $(filter-out sm_30 sm_35,$(multilib_options_isa_default)),mgomp/mptx=3.1 mptx=3.1)
diff --git a/gcc/doc/install.texi b/gcc/doc/install.texi
index 45ef39da04e..7b4073bdafc 100644
--- a/gcc/doc/install.texi
+++ b/gcc/doc/install.texi
@@ -4617,6 +4617,15 @@ the GCC sources.
 Use the @option{--disable-sjlj-exceptions} and
 @option{--enable-newlib-io-long-long} options when configuring.
 
+The @option{--with-arch} option may be specified to override the
+default value for the @option{-march} option, and to also build
+corresponding target libraries.
+The default is @option{--with-arch=sm_30}.
+
+For example, if @option{--with-arch=sm_70} is specified,
+@option{-march=sm_30} and @option{-march=sm_70} target libraries are
+built, and code generation defaults to @option{-march=sm_70}.
+
 @html
 <hr />
 @end html
diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index e11b5bdb347..d96152b32a8 100644
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -27549,7 +27549,9 @@ supported.
 Generate code for the specified PTX ISA target architecture
 (e.g.@: @samp{sm_35}).  Valid architecture strings are @samp{sm_30},
 @samp{sm_35}, @samp{sm_53}, @samp{sm_70}, @samp{sm_75} and
-@samp{sm_80}.  The default target architecture is sm_30.
+@samp{sm_80}.
+The default depends on how the compiler has been configured, see
+@option{--with-arch}.
 
 This option sets the value of the preprocessor macro
 @code{__PTX_SM__}; for instance, for @samp{sm_35}, it has the value
-- 
2.25.1