Re: Supporting RISC-V Vendor Extensions in the GNU Toolchain

[Andrew Waterman <andrew@sifive.com>]

On Fri, May 13, 2022 at 3:38 AM Philipp Tomsich <philipp.tomsich@vrull.eu>
wrote:

> On Fri, 13 May 2022 at 12:00, Christoph Müllner <cmuellner@gcc.gnu.org>
> wrote:
> >
> > On Wed, May 11, 2022 at 2:02 AM Palmer Dabbelt <palmer@dabbelt.com>
> wrote:
> > >
> > > [Sorry for cross-posting to a bunch of lists, I figured it'd be best to
> > > have all the discussions in one thread.]
> > >
> > > We currently only support what is defined by official RISC-V
> > > specifications in the various GNU toolchain projects.  There's
> certainly
> > > some grey areas there, but in general that means not taking code that
> > > relies on drafts or vendor defined extensions, even if that would
> result
> > > in higher performance or more featured systems for users.
> > >
> > > The original goal of these policies were to steer RISC-V implementers
> > > towards a common set of specifications, but over the last year or so
> > > it's become abundantly clear that this is causing more harm that good.
> > > All extant RISC-V systems rely on behaviors defined outside the
> official
> > > specifications, and while that's technically always been the case we've
> > > gotten to the point where trying to ignore that fact is impacting real
> > > users on real systems.  There's been consistent feedback from users
> that
> > > we're not meeting their needs, which can clearly be seen in the many
> out
> > > of tree patch sets in common use.
> > >
> > > There's been a handful of discussions about this, but we've yet to have
> > > a proper discussion on the mailing lists.  From the various discussions
> > > I've had it seems that folks are broadly in favor of supporting vendor
> > > extensions, but the devil's always in the details with this sort of
> > > thing so I thought it'd be best to write something up so we can have a
> > > concrete discussion.
> > >
> > > The idea is to start taking code that depends on vendor-defined
> behavior
> > > into the core GNU toolchain ports, as long as it meets the following
> > > criteria:
> > >
> > > * An ISA manual is available that can be redistributed/archived,
> defines
> > >   the behaviors in question as one or more vendor-specific extensions,
> > >   and is clearly versioned.  The RISC-V foundation is setting various
> > >   guidelines around how vendor-defined extensions and instructions
> > >   should be named, we strongly suggest that vendors follow those
> > >   conventions whenever possible (this is all new, though, so exactly
> > >   what's necessary from vendor specifications will likely evolve as we
> > >   learn).
> > > * There is a substantial user base that depends on the behavior in
> > >   question, which probably means there is hardware in the wild that
> > >   implements the extensions and users that require those extensions in
> > >   order for that hardware to be useful for common applications.  This
> is
> > >   always going to be a grey area, but it's essentially the same spot
> > >   everyone else is in.
>
> I must take exception to the "There is a substantial user base" rule,
> as this conflicts with the goal of avoiding fragmentation: the support
> for vendor-defined extensions should ideally have landed in an
> upstream release before the silicon is widely released.


The "substantial user base" standard is specious.  Other recent emails have
suggested that phrase is a euphemism for "widely available
consumer-programmable product."  It fails to account for RISC-V's most
important constituency: people using open-source toolchains to program
non-mass-market parts.

Our existing regime of "no custom extension support in standard software"
is draconian, but at least it's self-consistent.  I'd support retaining
it.  But, if we do change it, it's preposterous to reject extensions
like XVentanaCondOps on the basis that their silicon isn't available on
AliExpress.

  This would
> see these extensions being sent upstream significantly before
> widespread sampling (and sometimes around the time of the announcement
> of a device on the roadmap).  Simply put: I want everyone defining
> vendor extensions to contribute to our mainline development efforts
> instead of extending their own ancient forks.
>
> I suspect that this rule is intended to ensure that experimental,
> purely academic, or "closed" (as in: even if you have the silicon, it
> will be so deeply embedded that no one can run their own software —
> e.g. radio baseband controllers) extensions don't make the maintenance
> work harder.  If that is the case: could we use wording such as (a
> native speaker might wordsmith something more accurate) "accessible to
> run user-developed software" and "intended for a wider audience"?
>
> > > * There is a mechanism for testing the code in question without direct
> > >   access to hardware, which in practice means a QEMU port (or whatever
> > >   simulator is relevant in the space and that folks use for testing) or
> > >   some community commitment to long-term availability of the hardware
> > >   for testing (something like the GCC compile farm, for example).
> > > * It is possible to produce binaries that are compatible with all
> > >   upstream vendors' implementations.  That means we'll need mechanisms
> > >   to allow extensions from multiple vendors to be linked together and
> > >   then probed at runtime.  That's not to say that all binaries will be
> > >   compatible, as users are always free to skip the compatibility code
> > >   and there will be conflicting definitions of instruction encodings,
> > >   but we can at least provide users with the option of compatibility.
>
> We today have:
> - Tag_RISCV_arch (see
>
> https://github.com/riscv-non-isa/riscv-elf-psabi-doc/blob/master/riscv-elf.adoc#tag_riscv_arch-5-ntbssubarch
> )
> - ifunc support
>
> Admittedly, there's some loose ends in the end-to-end story (e.g.
> Unified Discovery -> DTB -> glibc ifunc initialisation): we know just
> too well how this plays out as there are optimised string/memory
> functions (Zbb, Zicboz, cache-block-length, …) in our pipeline as well
> as OpenSSL support for Zbb and Zbc.  However, this is a known gap and
> will be fully addressed in the future.
>
> Is there something specific beyond this that you'd be looking for?
>
> Thanks,
> Philipp.
>
> > >
> > > These are pretty loosely written on purpose, both because this is all
> > > new and because each project has its own set of contribution
> > > requirements so it's going to be all but impossible to have a single
> > > concrete set of rules that applies everywhere -- that's nothing
> specific
> > > to the vendor extensions (or even RISC-V), it's just life.
> Specifically
> > > a major goal here is to balance the needs of users, both in the short
> > > term (ie, getting new hardware to work) and the long term (ie, the long
> > > term stability of their software).  We're not talking about taking code
> > > that can't be tested, hasn't been reviewed, isn't going to be supported
> > > long-term, or doesn't have a stable ABI; just dropping the specific
> > > requirement that a specification must be furnished by the RISC-V
> > > foundation in order to accept code.
> > >
> > > Nothing is decided yet, so happy to hear any thought folks have.  This
> > > is certainly a very different development methodology than what we've
> > > done in the past and isn't something that should be entreated into
> > > lightly, so any comments are welcome.
> >
> > I'd like to add two points to this topic and raise two questions.
> >
> > 1) Accepting vendor extensions = avoidance of fragmentation
> >
> > RISC-V implementors are actively encouraged to implement their
> > own ISA extensions. To avoid fragmentation in the SW ecosystem
> > (every vendor maintains a fork of tools, distros and binaries) there
> > needs to be a principle acceptance to get vendor extension support
> > upstream.
> >
> > 2) Leading upstream maintainers already agreed on supporting
> vendor-extensions
> >
> > We have discussed the topic of vendor extensions in many forums last
> year.
> > This topic was also part of the discussions at the Linux Plumbers
> conference.
> > Further, there exists a place for documenting toolchain conventions of
> > the RISC-V
> > ecosystem ([1]), which everyone in the RISC-V ecosystem is aware about.
> >
> > As a result of the discussions last year, a PR ([2]) has been crafted to
> clarify
> > the rules for upstreaming vendor extension support. These RISC-V
> > toolchain conventions recently added a section for vendor extensions
> > which covers important aspects like:
> >
> > * naming schemes
> > * assembly mnemonic prefixes
> > * links to the documentation and version information
> >
> > The PR even includes an explicit rule to clarify that maintainers decide
> on
> > upstream inclusion:
> > """
> > Open source toolchain maintainer has final say on accepting vendor
> extension,
> > comply with this conventions isn't guarantee upstream will accept.
> > """
> >
> > A lot of people (maintainers and active developers) were notified
> > (including you)
> > and many also actively contributed to the PR. In the end there was an
> agreement
> > of how to document vendor extensions (as a requirement for upstreaming).
> >
> > I believe that your set of rules is compatible with what is specified
> there.
> > Note however, that you could have mentioned them during the PRs review
> > process as you were notified when the PR was created.
> >
> > My questions are now the following:
> >
> > * Where to document the requirements?
> >
> >   Most RISC-V upstream maintainers are accepting the
> riscv-toolchains-convention
> >   repository. Where if not there should we document requirements for the
> tool's
> >   conventions? Why not accept what has already been agreed upon?
> >
> > * Where to track the status?
> >
> >   You mentioned testing requirements (e.g. QEMU support).
> >   I've created a wiki page to show the adoption status of all the
> > RISC-V extensions ([3])
> >   last year. As the chair of the Toolchains SIG, I'm willing to create
> > and maintain one for
> >   vendor extensions as well. This allows users to see which projects
> > support which
> >   extensions upstream. However, a wiki is a joint effort. So
> > maintainers that merge
> >   changes upstream need to update the page. Will you support this?
> >   If not what is your proposal to track the status of the upstream
> > extension support?
> >
> > BR
> > Christoph
> >
> > [1]
> https://github.com/riscv-non-isa/riscv-toolchain-conventions#conventions-for-vendor-extension
> > [2] https://github.com/riscv-non-isa/riscv-toolchain-conventions/pull/17
> > [3]
> https://wiki.riscv.org/display/HOME/RISC-V+extension+and+feature+support+in+the+Open+Source+SW+Ecosystem
>