[RFC] Rationale for passing vectors by value in SIMD registers

[RFC] Rationale for passing vectors by value in SIMD registers

[Matthew Fortune]

MIPS is currently evaluating the benefit of using SIMD registers to pass vector data by value. It is currently unclear how important it is for vector data to be passed in SIMD registers. I.e. the need for passing vector data by value in real world code is not immediately obvious. The performance advantage is therefore also unclear.

Can anyone offer insight in the rationale behind decision decisions made for other architectures ABIs? For example, the x86 and x86_64 calling convention for vector data types presumes that they will passed in SSE/AVX registers and raises warnings if passed when sse/avx support is not enabled. This is what MIPS is currently considering however there are two concerns:

1) What about the ability to create architecture/implementation independent APIs that may include vector types in the prototypes. Such APIs may be built for varying levels of hardware support to make the most of a specific architecture implementation but be called from otherwise implementation agnostic code. To support such a scenario we would need to use a common calling convention usable on all architecture variants.
2) Although vector types are not specifically covered by existing ABI definitions for MIPS we have unfortunately got a defacto standard for how to pass these by value. Vector types are simply considered to be small structures and passed as such following normal ABI rules. This is still a concern even though it is generally accepted that there is some room for change when it comes to vector data types in an existing ABI.

If anyone could offer a brief history the x86 ABI with respect to vector data types that may also be interesting. One question would be whether the use of vector registers in the calling convention was only enabled by default once there was a critical mass of implementations, and therefore the default ABI was changed to start making assumptions about the availability of features like SSE and AVX.

Comments from any other architecture that has had to make such changes over time would also be welcome.

Thanks in advance,
Matthew

Re: [RFC] Rationale for passing vectors by value in SIMD registers

[Andrew Pinski]

On Fri, Feb 14, 2014 at 2:17 AM, Matthew Fortune
<Matthew.Fortune@imgtec.com> wrote:
> MIPS is currently evaluating the benefit of using SIMD registers to pass vector data by value. It is currently unclear how important it is for vector data to be passed in SIMD registers. I.e. the need for passing vector data by value in real world code is not immediately obvious. The performance advantage is therefore also unclear.
>
> Can anyone offer insight in the rationale behind decision decisions made for other architectures ABIs? For example, the x86 and x86_64 calling convention for vector data types presumes that they will passed in SSE/AVX registers and raises warnings if passed when sse/avx support is not enabled. This is what MIPS is currently considering however there are two concerns:
>
> 1) What about the ability to create architecture/implementation independent APIs that may include vector types in the prototypes. Such APIs may be built for varying levels of hardware support to make the most of a specific architecture implementation but be called from otherwise implementation agnostic code. To support such a scenario we would need to use a common calling convention usable on all architecture variants.
> 2) Although vector types are not specifically covered by existing ABI definitions for MIPS we have unfortunately got a defacto standard for how to pass these by value. Vector types are simply considered to be small structures and passed as such following normal ABI rules. This is still a concern even though it is generally accepted that there is some room for change when it comes to vector data types in an existing ABI.
>
> If anyone could offer a brief history the x86 ABI with respect to vector data types that may also be interesting. One question would be whether the use of vector registers in the calling convention was only enabled by default once there was a critical mass of implementations, and therefore the default ABI was changed to start making assumptions about the availability of features like SSE and AVX.
>
> Comments from any other architecture that has had to make such changes over time would also be welcome.

PPC and arm and AARCH64 are common targets where vectors are
passed/return via value.  The idea is simple, sometimes you have
functions like vector float vsinf(vector float a) where you want to be
faster and avoid a round trip to L1 (or even L2).  These kind of
functions are common for vector programming.  That is extending the
scalar versions to the vector versions.

Thanks,
Andrew Pinski

>
> Thanks in advance,
> Matthew
>

RE: [RFC] Rationale for passing vectors by value in SIMD registers

[Matthew Fortune]

> On Fri, Feb 14, 2014 at 2:17 AM, Matthew Fortune
> <Matthew.Fortune@imgtec.com> wrote:
> > MIPS is currently evaluating the benefit of using SIMD registers to pass
> vector data by value. It is currently unclear how important it is for vector data
> to be passed in SIMD registers. I.e. the need for passing vector data by value
> in real world code is not immediately obvious. The performance advantage is
> therefore also unclear.
> >
> > Can anyone offer insight in the rationale behind decision decisions made
> for other architectures ABIs? For example, the x86 and x86_64 calling
> convention for vector data types presumes that they will passed in SSE/AVX
> registers and raises warnings if passed when sse/avx support is not enabled.
> This is what MIPS is currently considering however there are two concerns:
> >
> > 1) What about the ability to create architecture/implementation
> independent APIs that may include vector types in the prototypes. Such APIs
> may be built for varying levels of hardware support to make the most of a
> specific architecture implementation but be called from otherwise
> implementation agnostic code. To support such a scenario we would need to
> use a common calling convention usable on all architecture variants.
> > 2) Although vector types are not specifically covered by existing ABI
> definitions for MIPS we have unfortunately got a defacto standard for how
> to pass these by value. Vector types are simply considered to be small
> structures and passed as such following normal ABI rules. This is still a
> concern even though it is generally accepted that there is some room for
> change when it comes to vector data types in an existing ABI.
> >
> > If anyone could offer a brief history the x86 ABI with respect to vector data
> types that may also be interesting. One question would be whether the use
> of vector registers in the calling convention was only enabled by default once
> there was a critical mass of implementations, and therefore the default ABI
> was changed to start making assumptions about the availability of features
> like SSE and AVX.
> >
> > Comments from any other architecture that has had to make such changes
> over time would also be welcome.
> 
> PPC and arm and AARCH64 are common targets where vectors are
> passed/return via value.  The idea is simple, sometimes you have functions
> like vector float vsinf(vector float a) where you want to be faster and avoid a
> round trip to L1 (or even L2).  These kind of functions are common for vector
> programming.  That is extending the scalar versions to the vector versions.

I suppose this cost (L1/L2) is mitigated to some extent if the base ABI were to pass a vector in multiple GP/FP register rather than via the stack. There would of course still be a cost to marshall the data between GP/FP and SIMD registers. For such a support routine like vsinf I would expect it also needs a reduced clobber set to ensure that the caller's live SIMD registers don't need saving/restoring, such registers would normally be caller-saved. If the routine were to clobber all SIMD registers anyway then the improvement in argument passing seems negligible.

Do you/anyone know of any open source projects, which have started adopting generic vector types, and show the use of this kind of construct?

Thanks for your input.

Matthew

> 
> Thanks,
> Andrew Pinski
> 
> >
> > Thanks in advance,
> > Matthew
> >

Re: [RFC] Rationale for passing vectors by value in SIMD registers

[Andrew Pinski]

On Sat, Feb 15, 2014 at 12:16 AM, Matthew Fortune
<Matthew.Fortune@imgtec.com> wrote:
>> On Fri, Feb 14, 2014 at 2:17 AM, Matthew Fortune
>> <Matthew.Fortune@imgtec.com> wrote:
>> > MIPS is currently evaluating the benefit of using SIMD registers to pass
>> vector data by value. It is currently unclear how important it is for vector data
>> to be passed in SIMD registers. I.e. the need for passing vector data by value
>> in real world code is not immediately obvious. The performance advantage is
>> therefore also unclear.
>> >
>> > Can anyone offer insight in the rationale behind decision decisions made
>> for other architectures ABIs? For example, the x86 and x86_64 calling
>> convention for vector data types presumes that they will passed in SSE/AVX
>> registers and raises warnings if passed when sse/avx support is not enabled.
>> This is what MIPS is currently considering however there are two concerns:
>> >
>> > 1) What about the ability to create architecture/implementation
>> independent APIs that may include vector types in the prototypes. Such APIs
>> may be built for varying levels of hardware support to make the most of a
>> specific architecture implementation but be called from otherwise
>> implementation agnostic code. To support such a scenario we would need to
>> use a common calling convention usable on all architecture variants.
>> > 2) Although vector types are not specifically covered by existing ABI
>> definitions for MIPS we have unfortunately got a defacto standard for how
>> to pass these by value. Vector types are simply considered to be small
>> structures and passed as such following normal ABI rules. This is still a
>> concern even though it is generally accepted that there is some room for
>> change when it comes to vector data types in an existing ABI.
>> >
>> > If anyone could offer a brief history the x86 ABI with respect to vector data
>> types that may also be interesting. One question would be whether the use
>> of vector registers in the calling convention was only enabled by default once
>> there was a critical mass of implementations, and therefore the default ABI
>> was changed to start making assumptions about the availability of features
>> like SSE and AVX.
>> >
>> > Comments from any other architecture that has had to make such changes
>> over time would also be welcome.
>>
>> PPC and arm and AARCH64 are common targets where vectors are
>> passed/return via value.  The idea is simple, sometimes you have functions
>> like vector float vsinf(vector float a) where you want to be faster and avoid a
>> round trip to L1 (or even L2).  These kind of functions are common for vector
>> programming.  That is extending the scalar versions to the vector versions.
>
> I suppose this cost (L1/L2) is mitigated to some extent if the base ABI were to pass a vector in multiple GP/FP register rather than via the stack. There would of course still be a cost to marshall the data between GP/FP and SIMD registers. For such a support routine like vsinf I would expect it also needs a reduced clobber set to ensure that the caller's live SIMD registers don't need saving/restoring, such registers would normally be caller-saved. If the routine were to clobber all SIMD registers anyway then the improvement in argument passing seems negligible.
>
> Do you/anyone know of any open source projects, which have started adopting generic vector types, and show the use of this kind of construct?

Yes glibc provides these functions on x86 now.

Thanks,
Andrew

>
> Thanks for your input.
>
> Matthew
>
>>
>> Thanks,
>> Andrew Pinski
>>
>> >
>> > Thanks in advance,
>> > Matthew
>> >

RE: [RFC] Rationale for passing vectors by value in SIMD registers

[Matthew Fortune]

Andrew Pinski <pinskia@gmail.com> writes:
> On Sat, Feb 15, 2014 at 12:16 AM, Matthew Fortune
> <Matthew.Fortune@imgtec.com> wrote:
> >> On Fri, Feb 14, 2014 at 2:17 AM, Matthew Fortune
> >> <Matthew.Fortune@imgtec.com> wrote:
> >> > MIPS is currently evaluating the benefit of using SIMD registers to pass
> >> vector data by value. It is currently unclear how important it is for vector data
> >> to be passed in SIMD registers. I.e. the need for passing vector data by value
> >> in real world code is not immediately obvious. The performance advantage is
> >> therefore also unclear.
> >> >
> >> > Can anyone offer insight in the rationale behind decision decisions made
> >> for other architectures ABIs? For example, the x86 and x86_64 calling
> >> convention for vector data types presumes that they will passed in SSE/AVX
> >> registers and raises warnings if passed when sse/avx support is not enabled.
> >> This is what MIPS is currently considering however there are two concerns:
> >> >
> >> > 1) What about the ability to create architecture/implementation
> >> independent APIs that may include vector types in the prototypes. Such APIs
> >> may be built for varying levels of hardware support to make the most of a
> >> specific architecture implementation but be called from otherwise
> >> implementation agnostic code. To support such a scenario we would need to
> >> use a common calling convention usable on all architecture variants.
> >> > 2) Although vector types are not specifically covered by existing ABI
> >> definitions for MIPS we have unfortunately got a defacto standard for how
> >> to pass these by value. Vector types are simply considered to be small
> >> structures and passed as such following normal ABI rules. This is still a
> >> concern even though it is generally accepted that there is some room for
> >> change when it comes to vector data types in an existing ABI.
> >> >
> >> > If anyone could offer a brief history the x86 ABI with respect to vector data
> >> types that may also be interesting. One question would be whether the use
> >> of vector registers in the calling convention was only enabled by default once
> >> there was a critical mass of implementations, and therefore the default ABI
> >> was changed to start making assumptions about the availability of features
> >> like SSE and AVX.
> >> >
> >> > Comments from any other architecture that has had to make such changes
> >> over time would also be welcome.
> >>
> >> PPC and arm and AARCH64 are common targets where vectors are
> >> passed/return via value.  The idea is simple, sometimes you have functions
> >> like vector float vsinf(vector float a) where you want to be faster and avoid a
> >> round trip to L1 (or even L2).  These kind of functions are common for vector
> >> programming.  That is extending the scalar versions to the vector versions.
> >
> > I suppose this cost (L1/L2) is mitigated to some extent if the base ABI were to pass a
> vector in multiple GP/FP register rather than via the stack. There would of course still
> be a cost to marshall the data between GP/FP and SIMD registers. For such a support
> routine like vsinf I would expect it also needs a reduced clobber set to ensure that the
> caller's live SIMD registers don't need saving/restoring, such registers would normally be
> caller-saved. If the routine were to clobber all SIMD registers anyway then the
> improvement in argument passing seems negligible.
> >
> > Do you/anyone know of any open source projects, which have started adopting generic
> vector types, and show the use of this kind of construct?
> 
> Yes glibc provides these functions on x86 now.

Wow, old thread you must have a good memory! I saw libmvec go in some time ago, I
guess you are referring to that or is there something else now (I'm out of date
with glibc development)?

I am hoping to steer MIPS towards supporting passing vectors by value via a an ABI
extension that is opt-in rather than default. The main reason is the range of
competing vector extensions whether defined as official ASEs or core specific. I
think we can still get vectors passed by value with the only extra requirement
being that a prototype would need a calling convention attribute.

Thanks,
Matthew