* Supporting RISC-V Vendor Extensions in the GNU Toolchain
@ 2022-05-11 0:01 Palmer Dabbelt
2022-05-13 10:00 ` Christoph Müllner
2022-07-20 21:24 ` Palmer Dabbelt
0 siblings, 2 replies; 10+ messages in thread
From: Palmer Dabbelt @ 2022-05-11 0:01 UTC (permalink / raw)
To: binutils, gcc-patches, libc-alpha, gdb-patches
[Sorry for cross-posting to a bunch of lists, I figured it'd be best to
have all the discussions in one thread.]
We currently only support what is defined by official RISC-V
specifications in the various GNU toolchain projects. There's certainly
some grey areas there, but in general that means not taking code that
relies on drafts or vendor defined extensions, even if that would result
in higher performance or more featured systems for users.
The original goal of these policies were to steer RISC-V implementers
towards a common set of specifications, but over the last year or so
it's become abundantly clear that this is causing more harm that good.
All extant RISC-V systems rely on behaviors defined outside the official
specifications, and while that's technically always been the case we've
gotten to the point where trying to ignore that fact is impacting real
users on real systems. There's been consistent feedback from users that
we're not meeting their needs, which can clearly be seen in the many out
of tree patch sets in common use.
There's been a handful of discussions about this, but we've yet to have
a proper discussion on the mailing lists. From the various discussions
I've had it seems that folks are broadly in favor of supporting vendor
extensions, but the devil's always in the details with this sort of
thing so I thought it'd be best to write something up so we can have a
concrete discussion.
The idea is to start taking code that depends on vendor-defined behavior
into the core GNU toolchain ports, as long as it meets the following
criteria:
* An ISA manual is available that can be redistributed/archived, defines
the behaviors in question as one or more vendor-specific extensions,
and is clearly versioned. The RISC-V foundation is setting various
guidelines around how vendor-defined extensions and instructions
should be named, we strongly suggest that vendors follow those
conventions whenever possible (this is all new, though, so exactly
what's necessary from vendor specifications will likely evolve as we
learn).
* There is a substantial user base that depends on the behavior in
question, which probably means there is hardware in the wild that
implements the extensions and users that require those extensions in
order for that hardware to be useful for common applications. This is
always going to be a grey area, but it's essentially the same spot
everyone else is in.
* There is a mechanism for testing the code in question without direct
access to hardware, which in practice means a QEMU port (or whatever
simulator is relevant in the space and that folks use for testing) or
some community commitment to long-term availability of the hardware
for testing (something like the GCC compile farm, for example).
* It is possible to produce binaries that are compatible with all
upstream vendors' implementations. That means we'll need mechanisms
to allow extensions from multiple vendors to be linked together and
then probed at runtime. That's not to say that all binaries will be
compatible, as users are always free to skip the compatibility code
and there will be conflicting definitions of instruction encodings,
but we can at least provide users with the option of compatibility.
These are pretty loosely written on purpose, both because this is all
new and because each project has its own set of contribution
requirements so it's going to be all but impossible to have a single
concrete set of rules that applies everywhere -- that's nothing specific
to the vendor extensions (or even RISC-V), it's just life. Specifically
a major goal here is to balance the needs of users, both in the short
term (ie, getting new hardware to work) and the long term (ie, the long
term stability of their software). We're not talking about taking code
that can't be tested, hasn't been reviewed, isn't going to be supported
long-term, or doesn't have a stable ABI; just dropping the specific
requirement that a specification must be furnished by the RISC-V
foundation in order to accept code.
Nothing is decided yet, so happy to hear any thought folks have. This
is certainly a very different development methodology than what we've
done in the past and isn't something that should be entreated into
lightly, so any comments are welcome.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Supporting RISC-V Vendor Extensions in the GNU Toolchain
2022-05-11 0:01 Supporting RISC-V Vendor Extensions in the GNU Toolchain Palmer Dabbelt
@ 2022-05-13 10:00 ` Christoph Müllner
2022-05-13 10:37 ` Philipp Tomsich
2022-05-13 10:58 ` Florian Weimer
2022-07-20 21:24 ` Palmer Dabbelt
1 sibling, 2 replies; 10+ messages in thread
From: Christoph Müllner @ 2022-05-13 10:00 UTC (permalink / raw)
To: Palmer Dabbelt
Cc: Binutils, GCC Patches, GNU C Library, gdb-patches, Philipp Tomsich
On Wed, May 11, 2022 at 2:02 AM Palmer Dabbelt <palmer@dabbelt.com> wrote:
>
> [Sorry for cross-posting to a bunch of lists, I figured it'd be best to
> have all the discussions in one thread.]
>
> We currently only support what is defined by official RISC-V
> specifications in the various GNU toolchain projects. There's certainly
> some grey areas there, but in general that means not taking code that
> relies on drafts or vendor defined extensions, even if that would result
> in higher performance or more featured systems for users.
>
> The original goal of these policies were to steer RISC-V implementers
> towards a common set of specifications, but over the last year or so
> it's become abundantly clear that this is causing more harm that good.
> All extant RISC-V systems rely on behaviors defined outside the official
> specifications, and while that's technically always been the case we've
> gotten to the point where trying to ignore that fact is impacting real
> users on real systems. There's been consistent feedback from users that
> we're not meeting their needs, which can clearly be seen in the many out
> of tree patch sets in common use.
>
> There's been a handful of discussions about this, but we've yet to have
> a proper discussion on the mailing lists. From the various discussions
> I've had it seems that folks are broadly in favor of supporting vendor
> extensions, but the devil's always in the details with this sort of
> thing so I thought it'd be best to write something up so we can have a
> concrete discussion.
>
> The idea is to start taking code that depends on vendor-defined behavior
> into the core GNU toolchain ports, as long as it meets the following
> criteria:
>
> * An ISA manual is available that can be redistributed/archived, defines
> the behaviors in question as one or more vendor-specific extensions,
> and is clearly versioned. The RISC-V foundation is setting various
> guidelines around how vendor-defined extensions and instructions
> should be named, we strongly suggest that vendors follow those
> conventions whenever possible (this is all new, though, so exactly
> what's necessary from vendor specifications will likely evolve as we
> learn).
> * There is a substantial user base that depends on the behavior in
> question, which probably means there is hardware in the wild that
> implements the extensions and users that require those extensions in
> order for that hardware to be useful for common applications. This is
> always going to be a grey area, but it's essentially the same spot
> everyone else is in.
> * There is a mechanism for testing the code in question without direct
> access to hardware, which in practice means a QEMU port (or whatever
> simulator is relevant in the space and that folks use for testing) or
> some community commitment to long-term availability of the hardware
> for testing (something like the GCC compile farm, for example).
> * It is possible to produce binaries that are compatible with all
> upstream vendors' implementations. That means we'll need mechanisms
> to allow extensions from multiple vendors to be linked together and
> then probed at runtime. That's not to say that all binaries will be
> compatible, as users are always free to skip the compatibility code
> and there will be conflicting definitions of instruction encodings,
> but we can at least provide users with the option of compatibility.
>
> These are pretty loosely written on purpose, both because this is all
> new and because each project has its own set of contribution
> requirements so it's going to be all but impossible to have a single
> concrete set of rules that applies everywhere -- that's nothing specific
> to the vendor extensions (or even RISC-V), it's just life. Specifically
> a major goal here is to balance the needs of users, both in the short
> term (ie, getting new hardware to work) and the long term (ie, the long
> term stability of their software). We're not talking about taking code
> that can't be tested, hasn't been reviewed, isn't going to be supported
> long-term, or doesn't have a stable ABI; just dropping the specific
> requirement that a specification must be furnished by the RISC-V
> foundation in order to accept code.
>
> Nothing is decided yet, so happy to hear any thought folks have. This
> is certainly a very different development methodology than what we've
> done in the past and isn't something that should be entreated into
> lightly, so any comments are welcome.
I'd like to add two points to this topic and raise two questions.
1) Accepting vendor extensions = avoidance of fragmentation
RISC-V implementors are actively encouraged to implement their
own ISA extensions. To avoid fragmentation in the SW ecosystem
(every vendor maintains a fork of tools, distros and binaries) there
needs to be a principle acceptance to get vendor extension support
upstream.
2) Leading upstream maintainers already agreed on supporting vendor-extensions
We have discussed the topic of vendor extensions in many forums last year.
This topic was also part of the discussions at the Linux Plumbers conference.
Further, there exists a place for documenting toolchain conventions of
the RISC-V
ecosystem ([1]), which everyone in the RISC-V ecosystem is aware about.
As a result of the discussions last year, a PR ([2]) has been crafted to clarify
the rules for upstreaming vendor extension support. These RISC-V
toolchain conventions recently added a section for vendor extensions
which covers important aspects like:
* naming schemes
* assembly mnemonic prefixes
* links to the documentation and version information
The PR even includes an explicit rule to clarify that maintainers decide on
upstream inclusion:
"""
Open source toolchain maintainer has final say on accepting vendor extension,
comply with this conventions isn't guarantee upstream will accept.
"""
A lot of people (maintainers and active developers) were notified
(including you)
and many also actively contributed to the PR. In the end there was an agreement
of how to document vendor extensions (as a requirement for upstreaming).
I believe that your set of rules is compatible with what is specified there.
Note however, that you could have mentioned them during the PRs review
process as you were notified when the PR was created.
My questions are now the following:
* Where to document the requirements?
Most RISC-V upstream maintainers are accepting the riscv-toolchains-convention
repository. Where if not there should we document requirements for the tool's
conventions? Why not accept what has already been agreed upon?
* Where to track the status?
You mentioned testing requirements (e.g. QEMU support).
I've created a wiki page to show the adoption status of all the
RISC-V extensions ([3])
last year. As the chair of the Toolchains SIG, I'm willing to create
and maintain one for
vendor extensions as well. This allows users to see which projects
support which
extensions upstream. However, a wiki is a joint effort. So
maintainers that merge
changes upstream need to update the page. Will you support this?
If not what is your proposal to track the status of the upstream
extension support?
BR
Christoph
[1] https://github.com/riscv-non-isa/riscv-toolchain-conventions#conventions-for-vendor-extension
[2] https://github.com/riscv-non-isa/riscv-toolchain-conventions/pull/17
[3] https://wiki.riscv.org/display/HOME/RISC-V+extension+and+feature+support+in+the+Open+Source+SW+Ecosystem
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Supporting RISC-V Vendor Extensions in the GNU Toolchain
2022-05-13 10:00 ` Christoph Müllner
@ 2022-05-13 10:37 ` Philipp Tomsich
2022-05-15 1:21 ` Palmer Dabbelt
2022-05-16 6:28 ` Andrew Waterman
2022-05-13 10:58 ` Florian Weimer
1 sibling, 2 replies; 10+ messages in thread
From: Philipp Tomsich @ 2022-05-13 10:37 UTC (permalink / raw)
To: Christoph Müllner
Cc: Palmer Dabbelt, Binutils, GCC Patches, GNU C Library,
gdb-patches, Greg Favor
On Fri, 13 May 2022 at 12:00, Christoph Müllner <cmuellner@gcc.gnu.org> wrote:
>
> On Wed, May 11, 2022 at 2:02 AM Palmer Dabbelt <palmer@dabbelt.com> wrote:
> >
> > [Sorry for cross-posting to a bunch of lists, I figured it'd be best to
> > have all the discussions in one thread.]
> >
> > We currently only support what is defined by official RISC-V
> > specifications in the various GNU toolchain projects. There's certainly
> > some grey areas there, but in general that means not taking code that
> > relies on drafts or vendor defined extensions, even if that would result
> > in higher performance or more featured systems for users.
> >
> > The original goal of these policies were to steer RISC-V implementers
> > towards a common set of specifications, but over the last year or so
> > it's become abundantly clear that this is causing more harm that good.
> > All extant RISC-V systems rely on behaviors defined outside the official
> > specifications, and while that's technically always been the case we've
> > gotten to the point where trying to ignore that fact is impacting real
> > users on real systems. There's been consistent feedback from users that
> > we're not meeting their needs, which can clearly be seen in the many out
> > of tree patch sets in common use.
> >
> > There's been a handful of discussions about this, but we've yet to have
> > a proper discussion on the mailing lists. From the various discussions
> > I've had it seems that folks are broadly in favor of supporting vendor
> > extensions, but the devil's always in the details with this sort of
> > thing so I thought it'd be best to write something up so we can have a
> > concrete discussion.
> >
> > The idea is to start taking code that depends on vendor-defined behavior
> > into the core GNU toolchain ports, as long as it meets the following
> > criteria:
> >
> > * An ISA manual is available that can be redistributed/archived, defines
> > the behaviors in question as one or more vendor-specific extensions,
> > and is clearly versioned. The RISC-V foundation is setting various
> > guidelines around how vendor-defined extensions and instructions
> > should be named, we strongly suggest that vendors follow those
> > conventions whenever possible (this is all new, though, so exactly
> > what's necessary from vendor specifications will likely evolve as we
> > learn).
> > * There is a substantial user base that depends on the behavior in
> > question, which probably means there is hardware in the wild that
> > implements the extensions and users that require those extensions in
> > order for that hardware to be useful for common applications. This is
> > always going to be a grey area, but it's essentially the same spot
> > everyone else is in.
I must take exception to the "There is a substantial user base" rule,
as this conflicts with the goal of avoiding fragmentation: the support
for vendor-defined extensions should ideally have landed in an
upstream release before the silicon is widely released. This would
see these extensions being sent upstream significantly before
widespread sampling (and sometimes around the time of the announcement
of a device on the roadmap). Simply put: I want everyone defining
vendor extensions to contribute to our mainline development efforts
instead of extending their own ancient forks.
I suspect that this rule is intended to ensure that experimental,
purely academic, or "closed" (as in: even if you have the silicon, it
will be so deeply embedded that no one can run their own software —
e.g. radio baseband controllers) extensions don't make the maintenance
work harder. If that is the case: could we use wording such as (a
native speaker might wordsmith something more accurate) "accessible to
run user-developed software" and "intended for a wider audience"?
> > * There is a mechanism for testing the code in question without direct
> > access to hardware, which in practice means a QEMU port (or whatever
> > simulator is relevant in the space and that folks use for testing) or
> > some community commitment to long-term availability of the hardware
> > for testing (something like the GCC compile farm, for example).
> > * It is possible to produce binaries that are compatible with all
> > upstream vendors' implementations. That means we'll need mechanisms
> > to allow extensions from multiple vendors to be linked together and
> > then probed at runtime. That's not to say that all binaries will be
> > compatible, as users are always free to skip the compatibility code
> > and there will be conflicting definitions of instruction encodings,
> > but we can at least provide users with the option of compatibility.
We today have:
- Tag_RISCV_arch (see
https://github.com/riscv-non-isa/riscv-elf-psabi-doc/blob/master/riscv-elf.adoc#tag_riscv_arch-5-ntbssubarch)
- ifunc support
Admittedly, there's some loose ends in the end-to-end story (e.g.
Unified Discovery -> DTB -> glibc ifunc initialisation): we know just
too well how this plays out as there are optimised string/memory
functions (Zbb, Zicboz, cache-block-length, …) in our pipeline as well
as OpenSSL support for Zbb and Zbc. However, this is a known gap and
will be fully addressed in the future.
Is there something specific beyond this that you'd be looking for?
Thanks,
Philipp.
> >
> > These are pretty loosely written on purpose, both because this is all
> > new and because each project has its own set of contribution
> > requirements so it's going to be all but impossible to have a single
> > concrete set of rules that applies everywhere -- that's nothing specific
> > to the vendor extensions (or even RISC-V), it's just life. Specifically
> > a major goal here is to balance the needs of users, both in the short
> > term (ie, getting new hardware to work) and the long term (ie, the long
> > term stability of their software). We're not talking about taking code
> > that can't be tested, hasn't been reviewed, isn't going to be supported
> > long-term, or doesn't have a stable ABI; just dropping the specific
> > requirement that a specification must be furnished by the RISC-V
> > foundation in order to accept code.
> >
> > Nothing is decided yet, so happy to hear any thought folks have. This
> > is certainly a very different development methodology than what we've
> > done in the past and isn't something that should be entreated into
> > lightly, so any comments are welcome.
>
> I'd like to add two points to this topic and raise two questions.
>
> 1) Accepting vendor extensions = avoidance of fragmentation
>
> RISC-V implementors are actively encouraged to implement their
> own ISA extensions. To avoid fragmentation in the SW ecosystem
> (every vendor maintains a fork of tools, distros and binaries) there
> needs to be a principle acceptance to get vendor extension support
> upstream.
>
> 2) Leading upstream maintainers already agreed on supporting vendor-extensions
>
> We have discussed the topic of vendor extensions in many forums last year.
> This topic was also part of the discussions at the Linux Plumbers conference.
> Further, there exists a place for documenting toolchain conventions of
> the RISC-V
> ecosystem ([1]), which everyone in the RISC-V ecosystem is aware about.
>
> As a result of the discussions last year, a PR ([2]) has been crafted to clarify
> the rules for upstreaming vendor extension support. These RISC-V
> toolchain conventions recently added a section for vendor extensions
> which covers important aspects like:
>
> * naming schemes
> * assembly mnemonic prefixes
> * links to the documentation and version information
>
> The PR even includes an explicit rule to clarify that maintainers decide on
> upstream inclusion:
> """
> Open source toolchain maintainer has final say on accepting vendor extension,
> comply with this conventions isn't guarantee upstream will accept.
> """
>
> A lot of people (maintainers and active developers) were notified
> (including you)
> and many also actively contributed to the PR. In the end there was an agreement
> of how to document vendor extensions (as a requirement for upstreaming).
>
> I believe that your set of rules is compatible with what is specified there.
> Note however, that you could have mentioned them during the PRs review
> process as you were notified when the PR was created.
>
> My questions are now the following:
>
> * Where to document the requirements?
>
> Most RISC-V upstream maintainers are accepting the riscv-toolchains-convention
> repository. Where if not there should we document requirements for the tool's
> conventions? Why not accept what has already been agreed upon?
>
> * Where to track the status?
>
> You mentioned testing requirements (e.g. QEMU support).
> I've created a wiki page to show the adoption status of all the
> RISC-V extensions ([3])
> last year. As the chair of the Toolchains SIG, I'm willing to create
> and maintain one for
> vendor extensions as well. This allows users to see which projects
> support which
> extensions upstream. However, a wiki is a joint effort. So
> maintainers that merge
> changes upstream need to update the page. Will you support this?
> If not what is your proposal to track the status of the upstream
> extension support?
>
> BR
> Christoph
>
> [1] https://github.com/riscv-non-isa/riscv-toolchain-conventions#conventions-for-vendor-extension
> [2] https://github.com/riscv-non-isa/riscv-toolchain-conventions/pull/17
> [3] https://wiki.riscv.org/display/HOME/RISC-V+extension+and+feature+support+in+the+Open+Source+SW+Ecosystem
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Supporting RISC-V Vendor Extensions in the GNU Toolchain
2022-05-13 10:00 ` Christoph Müllner
2022-05-13 10:37 ` Philipp Tomsich
@ 2022-05-13 10:58 ` Florian Weimer
2022-05-13 11:24 ` Philipp Tomsich
2022-05-13 12:26 ` Christoph Müllner
1 sibling, 2 replies; 10+ messages in thread
From: Florian Weimer @ 2022-05-13 10:58 UTC (permalink / raw)
To: Christoph Müllner via Binutils
Cc: Palmer Dabbelt, Christoph Müllner, Philipp Tomsich,
GNU C Library, GCC Patches, gdb-patches
* Christoph Müllner via Binutils:
> I'd like to add two points to this topic and raise two questions.
>
> 1) Accepting vendor extensions = avoidance of fragmentation
>
> RISC-V implementors are actively encouraged to implement their
> own ISA extensions. To avoid fragmentation in the SW ecosystem
> (every vendor maintains a fork of tools, distros and binaries) there
> needs to be a principle acceptance to get vendor extension support
> upstream.
If you eventually want portable binaries, it's necessary to converge on
a small set of widely implemented extensions. x86 didn't have this, and
adoption was poor outside specialized libraries (and JIT, of course).
Yet everything was as upstream as possible (ISA manuals, assemblers,
compiler intrinsics, even automated adoption by optimizers). So
upstreaming is only the first step.
Not every useful CPU feature can be adopted through run-time dispatching
(IFUNCs, function multi-versionining).
Thanks,
Florian
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Supporting RISC-V Vendor Extensions in the GNU Toolchain
2022-05-13 10:58 ` Florian Weimer
@ 2022-05-13 11:24 ` Philipp Tomsich
2022-05-13 12:26 ` Christoph Müllner
1 sibling, 0 replies; 10+ messages in thread
From: Philipp Tomsich @ 2022-05-13 11:24 UTC (permalink / raw)
To: Florian Weimer
Cc: Christoph Müllner via Binutils, Palmer Dabbelt,
Christoph Müllner, GNU C Library, GCC Patches, gdb-patches
On Fri, 13 May 2022 at 12:58, Florian Weimer <fweimer@redhat.com> wrote:
>
> * Christoph Müllner via Binutils:
>
> > I'd like to add two points to this topic and raise two questions.
> >
> > 1) Accepting vendor extensions = avoidance of fragmentation
> >
> > RISC-V implementors are actively encouraged to implement their
> > own ISA extensions. To avoid fragmentation in the SW ecosystem
> > (every vendor maintains a fork of tools, distros and binaries) there
> > needs to be a principle acceptance to get vendor extension support
> > upstream.
>
> If you eventually want portable binaries, it's necessary to converge on
> a small set of widely implemented extensions. x86 didn't have this, and
> adoption was poor outside specialized libraries (and JIT, of course).
> Yet everything was as upstream as possible (ISA manuals, assemblers,
> compiler intrinsics, even automated adoption by optimizers). So
> upstreaming is only the first step.
Some of the earlier discussion seems to have mixed two different goals:
1. making the vendor-defined features available to the developer and
ensuring that no unintended consequences (e.g., "accidental"
interlinking) happen, so developers can choose to adopt them (e.g.
through dynamic detection) where appropriate;
2. having widespread adoption for features across
libraries/applications that take advantage of all implemented features
As this is cross-posted to projects that provide the infrastructure,
tools, and plumbing, we should IMO focus on goal #1.
Coming from the RISC-V ISA philosophy, this also makes excellent
sense: after all, RISC-V is (in its purest form) an "ISA construction
kit": one can add extensions or leave extensions off.
For the essential development tools, this flexibility is reflected in
the myriad of combinations that "-march" can have (just consider that
there are 4 distinct Zb[abcs] extensions that add addressing, basic
bit-manipulation, carryless multiplication, and single-bit
operations…). If individual downstream users see benefits from any of
these (e.g., Zbb for strlen; Zbc for GHASH, …), they will contribute
optimized code-paths under ifunc (or whatever other mechanism a given
library/application uses); however: we first need to have our tools
support these extensions (both standard and vendor-defined) and ensure
that no accidental interlinking happens.
Finally, to enable binary distributions, a basic architecture level
that everyone agrees on (these are being defined at the RISC-V
Foundation under the "Profiles" and "Platforms" umbrellas) provides a
baseline to target that will provide some level of "runs everywhere"
based on such a "small set of widely implemented extensions".
Philipp.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Supporting RISC-V Vendor Extensions in the GNU Toolchain
2022-05-13 10:58 ` Florian Weimer
2022-05-13 11:24 ` Philipp Tomsich
@ 2022-05-13 12:26 ` Christoph Müllner
1 sibling, 0 replies; 10+ messages in thread
From: Christoph Müllner @ 2022-05-13 12:26 UTC (permalink / raw)
To: Florian Weimer
Cc: Christoph Müllner via Binutils, Palmer Dabbelt,
Philipp Tomsich, GNU C Library, GCC Patches, gdb-patches
On Fri, May 13, 2022 at 12:58 PM Florian Weimer <fweimer@redhat.com> wrote:
>
> * Christoph Müllner via Binutils:
>
> > I'd like to add two points to this topic and raise two questions.
> >
> > 1) Accepting vendor extensions = avoidance of fragmentation
> >
> > RISC-V implementors are actively encouraged to implement their
> > own ISA extensions. To avoid fragmentation in the SW ecosystem
> > (every vendor maintains a fork of tools, distros and binaries) there
> > needs to be a principle acceptance to get vendor extension support
> > upstream.
>
> If you eventually want portable binaries, it's necessary to converge on
> a small set of widely implemented extensions. x86 didn't have this, and
> adoption was poor outside specialized libraries (and JIT, of course).
> Yet everything was as upstream as possible (ISA manuals, assemblers,
> compiler intrinsics, even automated adoption by optimizers). So
> upstreaming is only the first step.
>
> Not every useful CPU feature can be adopted through run-time dispatching
> (IFUNCs, function multi-versionining).
I fully agree.
Therefore I can just recommend ISA extension designers to discuss the
SW enablement requirements and goals before CPUs are shipped.
In the end it is the responsibility of the CPU vendors to get their extensions
supported upstream and to provide appropriate patches to integrate well
and keep the upstream sources in a maintainable state.
However, I have full trust in the maintainers to make the right judgements
(like they have done in the past).
Thanks,
Christoph
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Supporting RISC-V Vendor Extensions in the GNU Toolchain
2022-05-13 10:37 ` Philipp Tomsich
@ 2022-05-15 1:21 ` Palmer Dabbelt
2022-05-16 12:32 ` Philipp Tomsich
2022-05-16 6:28 ` Andrew Waterman
1 sibling, 1 reply; 10+ messages in thread
From: Palmer Dabbelt @ 2022-05-15 1:21 UTC (permalink / raw)
To: philipp.tomsich
Cc: cmuellner, binutils, gcc-patches, libc-alpha, gdb-patches, gfavor
On Fri, 13 May 2022 03:37:22 PDT (-0700), philipp.tomsich@vrull.eu wrote:
> On Fri, 13 May 2022 at 12:00, Christoph Müllner <cmuellner@gcc.gnu.org> wrote:
>>
>> On Wed, May 11, 2022 at 2:02 AM Palmer Dabbelt <palmer@dabbelt.com> wrote:
>> >
>> > [Sorry for cross-posting to a bunch of lists, I figured it'd be best to
>> > have all the discussions in one thread.]
>> >
>> > We currently only support what is defined by official RISC-V
>> > specifications in the various GNU toolchain projects. There's certainly
>> > some grey areas there, but in general that means not taking code that
>> > relies on drafts or vendor defined extensions, even if that would result
>> > in higher performance or more featured systems for users.
>> >
>> > The original goal of these policies were to steer RISC-V implementers
>> > towards a common set of specifications, but over the last year or so
>> > it's become abundantly clear that this is causing more harm that good.
>> > All extant RISC-V systems rely on behaviors defined outside the official
>> > specifications, and while that's technically always been the case we've
>> > gotten to the point where trying to ignore that fact is impacting real
>> > users on real systems. There's been consistent feedback from users that
>> > we're not meeting their needs, which can clearly be seen in the many out
>> > of tree patch sets in common use.
>> >
>> > There's been a handful of discussions about this, but we've yet to have
>> > a proper discussion on the mailing lists. From the various discussions
>> > I've had it seems that folks are broadly in favor of supporting vendor
>> > extensions, but the devil's always in the details with this sort of
>> > thing so I thought it'd be best to write something up so we can have a
>> > concrete discussion.
>> >
>> > The idea is to start taking code that depends on vendor-defined behavior
>> > into the core GNU toolchain ports, as long as it meets the following
>> > criteria:
>> >
>> > * An ISA manual is available that can be redistributed/archived, defines
>> > the behaviors in question as one or more vendor-specific extensions,
>> > and is clearly versioned. The RISC-V foundation is setting various
>> > guidelines around how vendor-defined extensions and instructions
>> > should be named, we strongly suggest that vendors follow those
>> > conventions whenever possible (this is all new, though, so exactly
>> > what's necessary from vendor specifications will likely evolve as we
>> > learn).
>> > * There is a substantial user base that depends on the behavior in
>> > question, which probably means there is hardware in the wild that
>> > implements the extensions and users that require those extensions in
>> > order for that hardware to be useful for common applications. This is
>> > always going to be a grey area, but it's essentially the same spot
>> > everyone else is in.
>
> I must take exception to the "There is a substantial user base" rule,
> as this conflicts with the goal of avoiding fragmentation: the support
> for vendor-defined extensions should ideally have landed in an
> upstream release before the silicon is widely released. This would
> see these extensions being sent upstream significantly before
> widespread sampling (and sometimes around the time of the announcement
> of a device on the roadmap). Simply put: I want everyone defining
> vendor extensions to contribute to our mainline development efforts
> instead of extending their own ancient forks.
>
> I suspect that this rule is intended to ensure that experimental,
> purely academic, or "closed" (as in: even if you have the silicon, it
> will be so deeply embedded that no one can run their own software —
> e.g. radio baseband controllers) extensions don't make the maintenance
> work harder. If that is the case: could we use wording such as (a
> native speaker might wordsmith something more accurate) "accessible to
> run user-developed software" and "intended for a wider audience"?
I agree "substantial user base that depends on the behavior" is a bit
heavy-handed, as it essentially requires that the upstream software
shows up late enough that it's a headache for at least some users.
We've tried a bunch of times to come up with some sort of wording around
what it means to have hardware and everything's been full of holes,
which is why the language never gets written down.
I'm not personally all that worried about the experimental/academic
hardware as there's good precedent for pushing back on that. I'm also
not worried about embedded stuff: sure it's wacky, but a lot of vendors
end up doing the same things in subtly different ways so there'll be a
bunch of shared code. Plus, these vendors tend to err on the side of
secrecy anyway so I doubt it'll be a problem. I am worried about bad
actors leveraging any policy to make a bunch of noise, as that's a
pretty persistent problem in RISC-V land and it looks like things are
going to get worse before they get better. You can't really fake having
users, though, and in the end the whole point of the toolchain is to
make something useful for those users.
So I think I arrived at describing this in terms of a user base because
ultimately users is all I really care about. From talking to folks I
think that's a pretty common opinion, but someone else might have a more
nuanced way to state it so probably best to let others chime in here.
>> > * There is a mechanism for testing the code in question without direct
>> > access to hardware, which in practice means a QEMU port (or whatever
>> > simulator is relevant in the space and that folks use for testing) or
>> > some community commitment to long-term availability of the hardware
>> > for testing (something like the GCC compile farm, for example).
>> > * It is possible to produce binaries that are compatible with all
>> > upstream vendors' implementations. That means we'll need mechanisms
>> > to allow extensions from multiple vendors to be linked together and
>> > then probed at runtime. That's not to say that all binaries will be
>> > compatible, as users are always free to skip the compatibility code
>> > and there will be conflicting definitions of instruction encodings,
>> > but we can at least provide users with the option of compatibility.
>
> We today have:
> - Tag_RISCV_arch (see
> https://github.com/riscv-non-isa/riscv-elf-psabi-doc/blob/master/riscv-elf.adoc#tag_riscv_arch-5-ntbssubarch)
> - ifunc support
>
> Admittedly, there's some loose ends in the end-to-end story (e.g.
> Unified Discovery -> DTB -> glibc ifunc initialisation): we know just
> too well how this plays out as there are optimised string/memory
> functions (Zbb, Zicboz, cache-block-length, …) in our pipeline as well
> as OpenSSL support for Zbb and Zbc. However, this is a known gap and
> will be fully addressed in the future.
>
> Is there something specific beyond this that you'd be looking for?
I might be forgetting something, but at least:
* Tag_RISCV_arch attributes are really fundamentally based around
compatible extension sets and just don't work when faced with the
realities of what RISC-V is today -- that's true even for standard
extensions, but it's going to be way worse with vendor extensions.
* Some scheme that allows relocations from multiple vendors to be linked
together. There's been some proposals here, but nothing that the
psABI folks seem to like (and also might not play well with dynamic
relocations).
* There's a lot between device tree and ifunc (not to mention ACPI).
Kito had a proposal for how to get this up to userspace, there's an
earlier version from Plumbers last year but there's a lot of work that
needs to be done to turn that into reality.
* Some use cases won't be met by ifunc, there's a whole lot of
techniques available and we at least want to allow those to function.
In the long run binary compatibility is going to be a losing battle,
but we can at least try to keep things sane so the folks in charge at
the foundation have a chance to understand what a hole we're in with
enough time left to fix it.
I know it's a lot more work to give users the option of compatibility,
but once that's gone it'll never come back so I'm willing to at least
try -- though of course that'll put a burden on everyone, even those
outside the RISC-V ports, so everyone needs to be on board.
> Thanks,
> Philipp.
>
>> >
>> > These are pretty loosely written on purpose, both because this is all
>> > new and because each project has its own set of contribution
>> > requirements so it's going to be all but impossible to have a single
>> > concrete set of rules that applies everywhere -- that's nothing specific
>> > to the vendor extensions (or even RISC-V), it's just life. Specifically
>> > a major goal here is to balance the needs of users, both in the short
>> > term (ie, getting new hardware to work) and the long term (ie, the long
>> > term stability of their software). We're not talking about taking code
>> > that can't be tested, hasn't been reviewed, isn't going to be supported
>> > long-term, or doesn't have a stable ABI; just dropping the specific
>> > requirement that a specification must be furnished by the RISC-V
>> > foundation in order to accept code.
>> >
>> > Nothing is decided yet, so happy to hear any thought folks have. This
>> > is certainly a very different development methodology than what we've
>> > done in the past and isn't something that should be entreated into
>> > lightly, so any comments are welcome.
>>
>> I'd like to add two points to this topic and raise two questions.
>>
>> 1) Accepting vendor extensions = avoidance of fragmentation
>>
>> RISC-V implementors are actively encouraged to implement their
>> own ISA extensions. To avoid fragmentation in the SW ecosystem
>> (every vendor maintains a fork of tools, distros and binaries) there
>> needs to be a principle acceptance to get vendor extension support
>> upstream.
I'd actually argue that accepting vendor extensions upstream will
massively increase fragmentation. We've already got a huge problem with
fragmentation as it stands, but only allowing standard extensions in
upstream software provides an incentive to work with other vendors on
the standards. Accepting vendor extensions essentially changes the
incentive to just ship first. That's a well understood way to do things
in HW land, but it's bad for everyone in the long run.
I've personally been in favor of supporting vendor extensions for quite
a while now so the folks who have stuck to that for longer might have
other reasons, but pushing vendors towards common specs was the main
reason I was in favor of this policy when we first set it. Maybe other
folks can share what changed their minds, though.
>> 2) Leading upstream maintainers already agreed on supporting vendor-extensions
>>
>> We have discussed the topic of vendor extensions in many forums last year.
>> This topic was also part of the discussions at the Linux Plumbers conference.
>> Further, there exists a place for documenting toolchain conventions of
>> the RISC-V
>> ecosystem ([1]), which everyone in the RISC-V ecosystem is aware about.
>>
>> As a result of the discussions last year, a PR ([2]) has been crafted to clarify
>> the rules for upstreaming vendor extension support. These RISC-V
>> toolchain conventions recently added a section for vendor extensions
>> which covers important aspects like:
>>
>> * naming schemes
>> * assembly mnemonic prefixes
>> * links to the documentation and version information
>>
>> The PR even includes an explicit rule to clarify that maintainers decide on
>> upstream inclusion:
>> """
>> Open source toolchain maintainer has final say on accepting vendor extension,
>> comply with this conventions isn't guarantee upstream will accept.
>> """
I'm pretty sure that was added specifically as part of some discussions
around this topic, with the hope of making it clear that these policies
are decided upstream. That clearly didn't get the message across,
though, but we'd need to have this discussion either way so it doesn't
really matter that much.
And IMO it's way better to have a discussion regarding what we're going
to accept upstream ahead of time rather than just piecemeal as patches
go by, that way contributors to at least have an idea of what to expect.
>> A lot of people (maintainers and active developers) were notified
>> (including you)
>> and many also actively contributed to the PR. In the end there was an agreement
>> of how to document vendor extensions (as a requirement for upstreaming).
>>
>> I believe that your set of rules is compatible with what is specified there.
I didn't comment on the PR in question because it's talking about naming
requirements for vendor extensions/instructions and I don't care about
those at all. Luckily some of the other toolchain folks do and have put
together those guidelines, that's specifically called out here as what
should be used. That said, there's a lot more to accepting vendor
extensions than a naming scheme for them.
>> Note however, that you could have mentioned them during the PRs review
>> process as you were notified when the PR was created.
We've talked about this many times and my position has been consistent:
decisions happen in the relevant forum, which for most open source
projects means the project's mailing list -- we're even generally pretty
careful to make sure things like conferences/IRC get reflected to the
mailing lists, as not everyone can participate real-time. Certainly
these decisions are not made at the RISC-V foundation.
I get you guys don't listen to me, that's fine, but just pretending
that these discussions never happened is pretty silly.
>> My questions are now the following:
>>
>> * Where to document the requirements?
>>
>> Most RISC-V upstream maintainers are accepting the riscv-toolchains-convention
>> repository. Where if not there should we document requirements for the tool's
>> conventions? Why not accept what has already been agreed upon?
Each project generally has its own documentation, that varies from
project to project but is frequently somewhere like doc/. There are
generally guidelines/requirements for contributions, that seems like a
reasonable place to put this sort of thing to me.
>> * Where to track the status?
>>
>> You mentioned testing requirements (e.g. QEMU support).
>> I've created a wiki page to show the adoption status of all the
>> RISC-V extensions ([3])
>> last year. As the chair of the Toolchains SIG, I'm willing to create
>> and maintain one for
>> vendor extensions as well. This allows users to see which projects
>> support which
>> extensions upstream. However, a wiki is a joint effort. So
>> maintainers that merge
>> changes upstream need to update the page. Will you support this?
>> If not what is your proposal to track the status of the upstream
>> extension support?
Again that's really going to depend on the project, but there's
generally documentation of features somewhere like doc/ and high-level
new features are called out in the release notes: historically some sort
of NEWS file, but these days a lot of projects have another way of doing
things. We also have patch descriptions and such. To answer a question
like "does QEMU support Xvendorextension" I'd just read the code as
that's always both precise and accurate.
There are certainly lots of downstream collections of information about
various aspects of these projects and if you want to keep some on the
RISC-V wiki then that's fine with me. Happy to try and do a better job
on release notes and such, particularly if you've got specific issues
with them, but everyone else seems to get along tracking things that way
so I don't see any reason to do anything special for RISC-V.
>>
>> BR
>> Christoph
>>
>> [1] https://github.com/riscv-non-isa/riscv-toolchain-conventions#conventions-for-vendor-extension
>> [2] https://github.com/riscv-non-isa/riscv-toolchain-conventions/pull/17
>> [3] https://wiki.riscv.org/display/HOME/RISC-V+extension+and+feature+support+in+the+Open+Source+SW+Ecosystem
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Supporting RISC-V Vendor Extensions in the GNU Toolchain
2022-05-13 10:37 ` Philipp Tomsich
2022-05-15 1:21 ` Palmer Dabbelt
@ 2022-05-16 6:28 ` Andrew Waterman
1 sibling, 0 replies; 10+ messages in thread
From: Andrew Waterman @ 2022-05-16 6:28 UTC (permalink / raw)
To: Philipp Tomsich
Cc: Christoph Müllner, GNU C Library, Greg Favor, GCC Patches,
Binutils, gdb-patches
On Fri, May 13, 2022 at 3:38 AM Philipp Tomsich <philipp.tomsich@vrull.eu>
wrote:
> On Fri, 13 May 2022 at 12:00, Christoph Müllner <cmuellner@gcc.gnu.org>
> wrote:
> >
> > On Wed, May 11, 2022 at 2:02 AM Palmer Dabbelt <palmer@dabbelt.com>
> wrote:
> > >
> > > [Sorry for cross-posting to a bunch of lists, I figured it'd be best to
> > > have all the discussions in one thread.]
> > >
> > > We currently only support what is defined by official RISC-V
> > > specifications in the various GNU toolchain projects. There's
> certainly
> > > some grey areas there, but in general that means not taking code that
> > > relies on drafts or vendor defined extensions, even if that would
> result
> > > in higher performance or more featured systems for users.
> > >
> > > The original goal of these policies were to steer RISC-V implementers
> > > towards a common set of specifications, but over the last year or so
> > > it's become abundantly clear that this is causing more harm that good.
> > > All extant RISC-V systems rely on behaviors defined outside the
> official
> > > specifications, and while that's technically always been the case we've
> > > gotten to the point where trying to ignore that fact is impacting real
> > > users on real systems. There's been consistent feedback from users
> that
> > > we're not meeting their needs, which can clearly be seen in the many
> out
> > > of tree patch sets in common use.
> > >
> > > There's been a handful of discussions about this, but we've yet to have
> > > a proper discussion on the mailing lists. From the various discussions
> > > I've had it seems that folks are broadly in favor of supporting vendor
> > > extensions, but the devil's always in the details with this sort of
> > > thing so I thought it'd be best to write something up so we can have a
> > > concrete discussion.
> > >
> > > The idea is to start taking code that depends on vendor-defined
> behavior
> > > into the core GNU toolchain ports, as long as it meets the following
> > > criteria:
> > >
> > > * An ISA manual is available that can be redistributed/archived,
> defines
> > > the behaviors in question as one or more vendor-specific extensions,
> > > and is clearly versioned. The RISC-V foundation is setting various
> > > guidelines around how vendor-defined extensions and instructions
> > > should be named, we strongly suggest that vendors follow those
> > > conventions whenever possible (this is all new, though, so exactly
> > > what's necessary from vendor specifications will likely evolve as we
> > > learn).
> > > * There is a substantial user base that depends on the behavior in
> > > question, which probably means there is hardware in the wild that
> > > implements the extensions and users that require those extensions in
> > > order for that hardware to be useful for common applications. This
> is
> > > always going to be a grey area, but it's essentially the same spot
> > > everyone else is in.
>
> I must take exception to the "There is a substantial user base" rule,
> as this conflicts with the goal of avoiding fragmentation: the support
> for vendor-defined extensions should ideally have landed in an
> upstream release before the silicon is widely released.
The "substantial user base" standard is specious. Other recent emails have
suggested that phrase is a euphemism for "widely available
consumer-programmable product." It fails to account for RISC-V's most
important constituency: people using open-source toolchains to program
non-mass-market parts.
Our existing regime of "no custom extension support in standard software"
is draconian, but at least it's self-consistent. I'd support retaining
it. But, if we do change it, it's preposterous to reject extensions
like XVentanaCondOps on the basis that their silicon isn't available on
AliExpress.
This would
> see these extensions being sent upstream significantly before
> widespread sampling (and sometimes around the time of the announcement
> of a device on the roadmap). Simply put: I want everyone defining
> vendor extensions to contribute to our mainline development efforts
> instead of extending their own ancient forks.
>
> I suspect that this rule is intended to ensure that experimental,
> purely academic, or "closed" (as in: even if you have the silicon, it
> will be so deeply embedded that no one can run their own software —
> e.g. radio baseband controllers) extensions don't make the maintenance
> work harder. If that is the case: could we use wording such as (a
> native speaker might wordsmith something more accurate) "accessible to
> run user-developed software" and "intended for a wider audience"?
>
> > > * There is a mechanism for testing the code in question without direct
> > > access to hardware, which in practice means a QEMU port (or whatever
> > > simulator is relevant in the space and that folks use for testing) or
> > > some community commitment to long-term availability of the hardware
> > > for testing (something like the GCC compile farm, for example).
> > > * It is possible to produce binaries that are compatible with all
> > > upstream vendors' implementations. That means we'll need mechanisms
> > > to allow extensions from multiple vendors to be linked together and
> > > then probed at runtime. That's not to say that all binaries will be
> > > compatible, as users are always free to skip the compatibility code
> > > and there will be conflicting definitions of instruction encodings,
> > > but we can at least provide users with the option of compatibility.
>
> We today have:
> - Tag_RISCV_arch (see
>
> https://github.com/riscv-non-isa/riscv-elf-psabi-doc/blob/master/riscv-elf.adoc#tag_riscv_arch-5-ntbssubarch
> )
> - ifunc support
>
> Admittedly, there's some loose ends in the end-to-end story (e.g.
> Unified Discovery -> DTB -> glibc ifunc initialisation): we know just
> too well how this plays out as there are optimised string/memory
> functions (Zbb, Zicboz, cache-block-length, …) in our pipeline as well
> as OpenSSL support for Zbb and Zbc. However, this is a known gap and
> will be fully addressed in the future.
>
> Is there something specific beyond this that you'd be looking for?
>
> Thanks,
> Philipp.
>
> > >
> > > These are pretty loosely written on purpose, both because this is all
> > > new and because each project has its own set of contribution
> > > requirements so it's going to be all but impossible to have a single
> > > concrete set of rules that applies everywhere -- that's nothing
> specific
> > > to the vendor extensions (or even RISC-V), it's just life.
> Specifically
> > > a major goal here is to balance the needs of users, both in the short
> > > term (ie, getting new hardware to work) and the long term (ie, the long
> > > term stability of their software). We're not talking about taking code
> > > that can't be tested, hasn't been reviewed, isn't going to be supported
> > > long-term, or doesn't have a stable ABI; just dropping the specific
> > > requirement that a specification must be furnished by the RISC-V
> > > foundation in order to accept code.
> > >
> > > Nothing is decided yet, so happy to hear any thought folks have. This
> > > is certainly a very different development methodology than what we've
> > > done in the past and isn't something that should be entreated into
> > > lightly, so any comments are welcome.
> >
> > I'd like to add two points to this topic and raise two questions.
> >
> > 1) Accepting vendor extensions = avoidance of fragmentation
> >
> > RISC-V implementors are actively encouraged to implement their
> > own ISA extensions. To avoid fragmentation in the SW ecosystem
> > (every vendor maintains a fork of tools, distros and binaries) there
> > needs to be a principle acceptance to get vendor extension support
> > upstream.
> >
> > 2) Leading upstream maintainers already agreed on supporting
> vendor-extensions
> >
> > We have discussed the topic of vendor extensions in many forums last
> year.
> > This topic was also part of the discussions at the Linux Plumbers
> conference.
> > Further, there exists a place for documenting toolchain conventions of
> > the RISC-V
> > ecosystem ([1]), which everyone in the RISC-V ecosystem is aware about.
> >
> > As a result of the discussions last year, a PR ([2]) has been crafted to
> clarify
> > the rules for upstreaming vendor extension support. These RISC-V
> > toolchain conventions recently added a section for vendor extensions
> > which covers important aspects like:
> >
> > * naming schemes
> > * assembly mnemonic prefixes
> > * links to the documentation and version information
> >
> > The PR even includes an explicit rule to clarify that maintainers decide
> on
> > upstream inclusion:
> > """
> > Open source toolchain maintainer has final say on accepting vendor
> extension,
> > comply with this conventions isn't guarantee upstream will accept.
> > """
> >
> > A lot of people (maintainers and active developers) were notified
> > (including you)
> > and many also actively contributed to the PR. In the end there was an
> agreement
> > of how to document vendor extensions (as a requirement for upstreaming).
> >
> > I believe that your set of rules is compatible with what is specified
> there.
> > Note however, that you could have mentioned them during the PRs review
> > process as you were notified when the PR was created.
> >
> > My questions are now the following:
> >
> > * Where to document the requirements?
> >
> > Most RISC-V upstream maintainers are accepting the
> riscv-toolchains-convention
> > repository. Where if not there should we document requirements for the
> tool's
> > conventions? Why not accept what has already been agreed upon?
> >
> > * Where to track the status?
> >
> > You mentioned testing requirements (e.g. QEMU support).
> > I've created a wiki page to show the adoption status of all the
> > RISC-V extensions ([3])
> > last year. As the chair of the Toolchains SIG, I'm willing to create
> > and maintain one for
> > vendor extensions as well. This allows users to see which projects
> > support which
> > extensions upstream. However, a wiki is a joint effort. So
> > maintainers that merge
> > changes upstream need to update the page. Will you support this?
> > If not what is your proposal to track the status of the upstream
> > extension support?
> >
> > BR
> > Christoph
> >
> > [1]
> https://github.com/riscv-non-isa/riscv-toolchain-conventions#conventions-for-vendor-extension
> > [2] https://github.com/riscv-non-isa/riscv-toolchain-conventions/pull/17
> > [3]
> https://wiki.riscv.org/display/HOME/RISC-V+extension+and+feature+support+in+the+Open+Source+SW+Ecosystem
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Supporting RISC-V Vendor Extensions in the GNU Toolchain
2022-05-15 1:21 ` Palmer Dabbelt
@ 2022-05-16 12:32 ` Philipp Tomsich
0 siblings, 0 replies; 10+ messages in thread
From: Philipp Tomsich @ 2022-05-16 12:32 UTC (permalink / raw)
To: Palmer Dabbelt
Cc: cmuellner, binutils, gcc-patches, libc-alpha, gdb-patches, gfavor
A generous [snip], as this has been getting a bit long.
On Sun, 15 May 2022 at 03:21, Palmer Dabbelt <palmer@dabbelt.com> wrote:
> I am worried about bad
> actors leveraging any policy to make a bunch of noise, as that's a
> pretty persistent problem in RISC-V land and it looks like things are
> going to get worse before they get better.
>
I don't follow. Maybe you can walk me through the "bad actors" comment next
time we talk…
> > We today have:
> > - Tag_RISCV_arch (see
> >
> https://github.com/riscv-non-isa/riscv-elf-psabi-doc/blob/master/riscv-elf.adoc#tag_riscv_arch-5-ntbssubarch
> )
> > - ifunc support
> >
> > Admittedly, there's some loose ends in the end-to-end story (e.g.
> > Unified Discovery -> DTB -> glibc ifunc initialisation): we know just
> > too well how this plays out as there are optimised string/memory
> > functions (Zbb, Zicboz, cache-block-length, …) in our pipeline as well
> > as OpenSSL support for Zbb and Zbc. However, this is a known gap and
> > will be fully addressed in the future.
> >
> > Is there something specific beyond this that you'd be looking for?
>
> I might be forgetting something, but at least:
>
> * Tag_RISCV_arch attributes are really fundamentally based around
> compatible extension sets and just don't work when faced with the
> realities of what RISC-V is today -- that's true even for standard
> extensions, but it's going to be way worse with vendor extensions.
> * Some scheme that allows relocations from multiple vendors to be linked
> together. There's been some proposals here, but nothing that the
> psABI folks seem to like (and also might not play well with dynamic
> relocations).
>
I would recommend deferring solving the vendor-defined relocations to a
later time.
All vendor-defined extension proposals already on the table for upstream
inclusion (X-Ventana-CondOps, X-THead-CMO) don't require custom
relocation. I don't expect anything requiring these shortly — and whoever
submits it will have to provide a proposal for vendor-defined relocations
that finds some consensus.
> * There's a lot between device tree and ifunc (not to mention ACPI).
> Kito had a proposal for how to get this up to userspace, there's an
> earlier version from Plumbers last year but there's a lot of work that
> needs to be done to turn that into reality.
>
Agreed. Our team is looking into this already as Zbb and Zicboz are useful
in GLIBC.
> * Some use cases won't be met by ifunc, there's a whole lot of
> techniques available and we at least want to allow those to function.
> In the long run binary compatibility is going to be a losing battle,
> but we can at least try to keep things sane so the folks in charge at
> the foundation have a chance to understand what a hole we're in with
> enough time left to fix it.
>
> I know it's a lot more work to give users the option of compatibility,
> but once that's gone it'll never come back so I'm willing to at least
> try -- though of course that'll put a burden on everyone, even those
> outside the RISC-V ports, so everyone needs to be on board.
>
I have been discussing "fat binaries" on and off in the context of
reconciling the vector fragmentation.
This is a follow-on topic to getting things enabled and ensuring that no
accidental interworking occurs — once the basic support is mature enough, I
hope there will be takers for fat-binary support.
I hope this further clarifies my thinking: I would like to roll support for
vendor-defined extensions out in an incremental manner: starting with
rolling up some extensions into the development tools (assembler, linker,
and compiler); and only then improving runtime detection and library
usage. For vendor-defined relocations, I would build consensus once we
first encounter the need for them.
Philipp.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Supporting RISC-V Vendor Extensions in the GNU Toolchain
2022-05-11 0:01 Supporting RISC-V Vendor Extensions in the GNU Toolchain Palmer Dabbelt
2022-05-13 10:00 ` Christoph Müllner
@ 2022-07-20 21:24 ` Palmer Dabbelt
1 sibling, 0 replies; 10+ messages in thread
From: Palmer Dabbelt @ 2022-07-20 21:24 UTC (permalink / raw)
To: binutils, gcc-patches, libc-alpha, gdb-patches
On Tue, 10 May 2022 17:01:26 PDT (-0700), Palmer Dabbelt wrote:
> [Sorry for cross-posting to a bunch of lists, I figured it'd be best to
> have all the discussions in one thread.]
>
> We currently only support what is defined by official RISC-V
> specifications in the various GNU toolchain projects. There's certainly
> some grey areas there, but in general that means not taking code that
> relies on drafts or vendor defined extensions, even if that would result
> in higher performance or more featured systems for users.
>
> The original goal of these policies were to steer RISC-V implementers
> towards a common set of specifications, but over the last year or so
> it's become abundantly clear that this is causing more harm that good.
> All extant RISC-V systems rely on behaviors defined outside the official
> specifications, and while that's technically always been the case we've
> gotten to the point where trying to ignore that fact is impacting real
> users on real systems. There's been consistent feedback from users that
> we're not meeting their needs, which can clearly be seen in the many out
> of tree patch sets in common use.
>
> There's been a handful of discussions about this, but we've yet to have
> a proper discussion on the mailing lists. From the various discussions
> I've had it seems that folks are broadly in favor of supporting vendor
> extensions, but the devil's always in the details with this sort of
> thing so I thought it'd be best to write something up so we can have a
> concrete discussion.
>
> The idea is to start taking code that depends on vendor-defined behavior
> into the core GNU toolchain ports, as long as it meets the following
> criteria:
>
> * An ISA manual is available that can be redistributed/archived, defines
> the behaviors in question as one or more vendor-specific extensions,
> and is clearly versioned. The RISC-V foundation is setting various
> guidelines around how vendor-defined extensions and instructions
> should be named, we strongly suggest that vendors follow those
> conventions whenever possible (this is all new, though, so exactly
> what's necessary from vendor specifications will likely evolve as we
> learn).
> * There is a substantial user base that depends on the behavior in
> question, which probably means there is hardware in the wild that
> implements the extensions and users that require those extensions in
> order for that hardware to be useful for common applications. This is
> always going to be a grey area, but it's essentially the same spot
> everyone else is in.
> * There is a mechanism for testing the code in question without direct
> access to hardware, which in practice means a QEMU port (or whatever
> simulator is relevant in the space and that folks use for testing) or
> some community commitment to long-term availability of the hardware
> for testing (something like the GCC compile farm, for example).
> * It is possible to produce binaries that are compatible with all
> upstream vendors' implementations. That means we'll need mechanisms
> to allow extensions from multiple vendors to be linked together and
> then probed at runtime. That's not to say that all binaries will be
> compatible, as users are always free to skip the compatibility code
> and there will be conflicting definitions of instruction encodings,
> but we can at least provide users with the option of compatibility.
>
> These are pretty loosely written on purpose, both because this is all
> new and because each project has its own set of contribution
> requirements so it's going to be all but impossible to have a single
> concrete set of rules that applies everywhere -- that's nothing specific
> to the vendor extensions (or even RISC-V), it's just life. Specifically
> a major goal here is to balance the needs of users, both in the short
> term (ie, getting new hardware to work) and the long term (ie, the long
> term stability of their software). We're not talking about taking code
> that can't be tested, hasn't been reviewed, isn't going to be supported
> long-term, or doesn't have a stable ABI; just dropping the specific
> requirement that a specification must be furnished by the RISC-V
> foundation in order to accept code.
>
> Nothing is decided yet, so happy to hear any thought folks have. This
> is certainly a very different development methodology than what we've
> done in the past and isn't something that should be entreated into
> lightly, so any comments are welcome.
I'm going back to the start of the thread as this led to some heated
discussion, both here and in private. Clearly there's lots of opinions
here and everyone wants something different, but the nature of
compromise is that nobody gets exactly what they want and it looks like
this is as good as we're going to get any time soon. So I'm going to
propose that we go with this.
This was all purposefully a bit vague so we'll have to go sort out exactly
how to move forward as patches go by. Hopefully we'll be able to have
more constructive discussions on the specific patch sets, as at least
the issues will be a bit more focused. The C906 is a widely available
chip that needs vendor extensions to function in some very basic ways,
it's been blocked on a policy change for way too long and at least this
way we can get moving on that front.
Happy to continue the discussion if anyone has concrete concerns here,
either way let's give it at least a few days to get through everyone's
inbox before doing anything that depends on the policy change.
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2022-07-20 21:24 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-05-11 0:01 Supporting RISC-V Vendor Extensions in the GNU Toolchain Palmer Dabbelt
2022-05-13 10:00 ` Christoph Müllner
2022-05-13 10:37 ` Philipp Tomsich
2022-05-15 1:21 ` Palmer Dabbelt
2022-05-16 12:32 ` Philipp Tomsich
2022-05-16 6:28 ` Andrew Waterman
2022-05-13 10:58 ` Florian Weimer
2022-05-13 11:24 ` Philipp Tomsich
2022-05-13 12:26 ` Christoph Müllner
2022-07-20 21:24 ` Palmer Dabbelt
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).