IEEE Interchange floating point and extended floating point for C++

IEEE Interchange floating point and extended floating point for C++

[Kito Cheng]

Hi:

Would it be possible to support interchange floating point and/or
extended floating point for C++, which is introduced by ISO/IEC TS
18661-3?

I've read the note about C++ support from the initial commit log[1],
so I know there is some concern about C++ support for that, is it
possible to enable that for C++ like a language extension for C++?

The main demand comes from the data type for half-precision, ISO/IEC
TS 18661-3 is the only common spec which supports half-precision and
both GCC and clang are supported.

However it can't be used on C++ for GCC, so it's hard to use that as a
workable solution for half-precision data types.

Or maybe ISO/IEC JTC1 SC22 WG14 N2016[2] is a better way for
half-precision data types?


Thanks

[1] https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=c65699efcce48d68ef57ab3ce7fc5420fac5cbf9
[2] http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf

Re: IEEE Interchange floating point and extended floating point for C++

[Jonathan Wakely]

On Thu, 11 Mar 2021 at 09:43, Kito Cheng via Gcc <gcc@gcc.gnu.org> wrote:
>
> Hi:
>
> Would it be possible to support interchange floating point and/or
> extended floating point for C++, which is introduced by ISO/IEC TS
> 18661-3?
>
> I've read the note about C++ support from the initial commit log[1],
> so I know there is some concern about C++ support for that, is it
> possible to enable that for C++ like a language extension for C++?
>
> The main demand comes from the data type for half-precision, ISO/IEC
> TS 18661-3 is the only common spec which supports half-precision and
> both GCC and clang are supported.
>
> However it can't be used on C++ for GCC, so it's hard to use that as a
> workable solution for half-precision data types.
>
> Or maybe ISO/IEC JTC1 SC22 WG14 N2016[2] is a better way for
> half-precision data types?

See https://wg21.link/p1467 and https://wg21.link/p1467 for the
relevant C++ proposals.

Re: IEEE Interchange floating point and extended floating point for C++

[Gabriel Ravier]

On 3/11/21 1:56 PM, Jonathan Wakely via Gcc wrote:
> On Thu, 11 Mar 2021 at 09:43, Kito Cheng via Gcc <gcc@gcc.gnu.org> wrote:
>> Hi:
>>
>> Would it be possible to support interchange floating point and/or
>> extended floating point for C++, which is introduced by ISO/IEC TS
>> 18661-3?
>>
>> I've read the note about C++ support from the initial commit log[1],
>> so I know there is some concern about C++ support for that, is it
>> possible to enable that for C++ like a language extension for C++?
>>
>> The main demand comes from the data type for half-precision, ISO/IEC
>> TS 18661-3 is the only common spec which supports half-precision and
>> both GCC and clang are supported.
>>
>> However it can't be used on C++ for GCC, so it's hard to use that as a
>> workable solution for half-precision data types.
>>
>> Or maybe ISO/IEC JTC1 SC22 WG14 N2016[2] is a better way for
>> half-precision data types?
> See https://wg21.link/p1467 and https://wg21.link/p1467 for the
> relevant C++ proposals.
Isn't that the same link two times ?

Re: IEEE Interchange floating point and extended floating point for C++

[Jonathan Wakely]

On Thu, 11 Mar 2021 at 15:07, Gabriel Ravier via Gcc <gcc@gcc.gnu.org> wrote:
>
> On 3/11/21 1:56 PM, Jonathan Wakely via Gcc wrote:
> > On Thu, 11 Mar 2021 at 09:43, Kito Cheng via Gcc <gcc@gcc.gnu.org> wrote:
> >> Hi:
> >>
> >> Would it be possible to support interchange floating point and/or
> >> extended floating point for C++, which is introduced by ISO/IEC TS
> >> 18661-3?
> >>
> >> I've read the note about C++ support from the initial commit log[1],
> >> so I know there is some concern about C++ support for that, is it
> >> possible to enable that for C++ like a language extension for C++?
> >>
> >> The main demand comes from the data type for half-precision, ISO/IEC
> >> TS 18661-3 is the only common spec which supports half-precision and
> >> both GCC and clang are supported.
> >>
> >> However it can't be used on C++ for GCC, so it's hard to use that as a
> >> workable solution for half-precision data types.
> >>
> >> Or maybe ISO/IEC JTC1 SC22 WG14 N2016[2] is a better way for
> >> half-precision data types?
> > See https://wg21.link/p1467 and https://wg21.link/p1467 for the
> > relevant C++ proposals.
> Isn't that the same link two times ?

The difference is smaller than epsilon.

Sorry, the second one should be https://wg21.link/p14678

Re: IEEE Interchange floating point and extended floating point for C++

[Jonathan Wakely]

On Thu, 11 Mar 2021 at 15:14, Jonathan Wakely wrote:
>
> On Thu, 11 Mar 2021 at 15:07, Gabriel Ravier via Gcc <gcc@gcc.gnu.org> wrote:
> >
> > On 3/11/21 1:56 PM, Jonathan Wakely via Gcc wrote:
> > > On Thu, 11 Mar 2021 at 09:43, Kito Cheng via Gcc <gcc@gcc.gnu.org> wrote:
> > >> Hi:
> > >>
> > >> Would it be possible to support interchange floating point and/or
> > >> extended floating point for C++, which is introduced by ISO/IEC TS
> > >> 18661-3?
> > >>
> > >> I've read the note about C++ support from the initial commit log[1],
> > >> so I know there is some concern about C++ support for that, is it
> > >> possible to enable that for C++ like a language extension for C++?
> > >>
> > >> The main demand comes from the data type for half-precision, ISO/IEC
> > >> TS 18661-3 is the only common spec which supports half-precision and
> > >> both GCC and clang are supported.
> > >>
> > >> However it can't be used on C++ for GCC, so it's hard to use that as a
> > >> workable solution for half-precision data types.
> > >>
> > >> Or maybe ISO/IEC JTC1 SC22 WG14 N2016[2] is a better way for
> > >> half-precision data types?
> > > See https://wg21.link/p1467 and https://wg21.link/p1467 for the
> > > relevant C++ proposals.
> > Isn't that the same link two times ?
>
> The difference is smaller than epsilon.
>
> Sorry, the second one should be https://wg21.link/p14678

Argh! No, it should be https://wg21.link/p1468

Re: IEEE Interchange floating point and extended floating point for C++

[Kito Cheng]

Hi Jonathan:

Thanks for providing that information!


On Fri, Mar 12, 2021 at 12:17 AM Jonathan Wakely via Gcc
<gcc@gcc.gnu.org> wrote:
>
> On Thu, 11 Mar 2021 at 15:14, Jonathan Wakely wrote:
> >
> > On Thu, 11 Mar 2021 at 15:07, Gabriel Ravier via Gcc <gcc@gcc.gnu.org> wrote:
> > >
> > > On 3/11/21 1:56 PM, Jonathan Wakely via Gcc wrote:
> > > > On Thu, 11 Mar 2021 at 09:43, Kito Cheng via Gcc <gcc@gcc.gnu.org> wrote:
> > > >> Hi:
> > > >>
> > > >> Would it be possible to support interchange floating point and/or
> > > >> extended floating point for C++, which is introduced by ISO/IEC TS
> > > >> 18661-3?
> > > >>
> > > >> I've read the note about C++ support from the initial commit log[1],
> > > >> so I know there is some concern about C++ support for that, is it
> > > >> possible to enable that for C++ like a language extension for C++?
> > > >>
> > > >> The main demand comes from the data type for half-precision, ISO/IEC
> > > >> TS 18661-3 is the only common spec which supports half-precision and
> > > >> both GCC and clang are supported.
> > > >>
> > > >> However it can't be used on C++ for GCC, so it's hard to use that as a
> > > >> workable solution for half-precision data types.
> > > >>
> > > >> Or maybe ISO/IEC JTC1 SC22 WG14 N2016[2] is a better way for
> > > >> half-precision data types?
> > > > See https://wg21.link/p1467 and https://wg21.link/p1467 for the
> > > > relevant C++ proposals.
> > > Isn't that the same link two times ?
> >
> > The difference is smaller than epsilon.
> >
> > Sorry, the second one should be https://wg21.link/p14678
>
> Argh! No, it should be https://wg21.link/p1468

Re: IEEE Interchange floating point and extended floating point for C++

[Joseph Myers]

On Thu, 11 Mar 2021, Kito Cheng wrote:

> I've read the note about C++ support from the initial commit log[1],
> so I know there is some concern about C++ support for that, is it
> possible to enable that for C++ like a language extension for C++?

I don't know if C++ has reached any conclusions about what form C++ 
support for such types should take, but my expectation was that something 
library-based, similar to the support for decimal floating-point types, 
might be used.

> Or maybe ISO/IEC JTC1 SC22 WG14 N2016[2] is a better way for
> half-precision data types?

No.  short float has been specifically rejected in WG14, given the 
expectation that people using narrower-than-float types will want a 
specific format (whether binary16 or bfloat16) rather than something more 
generic where they don't know what format it has.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: IEEE Interchange floating point and extended floating point for C++

[Kito Cheng]

Hi Joseph:

> I don't know if C++ has reached any conclusions about what form C++
> support for such types should take, but my expectation was that something
> library-based, similar to the support for decimal floating-point types,
> might be used.

Thanks, my understanding is that we are not intending to implement those
types on C++ mode.

Sounds like it means if we need a standard half-precision type for C++
we might need to wait https://wg21.link/p1467 ?

> No.  short float has been specifically rejected in WG14, given the
> expectation that people using narrower-than-float types will want a
> specific format (whether binary16 or bfloat16) rather than something more
> generic where they don't know what format it has.

Thanks for the info, I didn't know too much about the

Re: IEEE Interchange floating point and extended floating point for C++

[Jonathan Wakely]

On Thu, 11 Mar 2021 at 15:32, Joseph Myers <joseph@codesourcery.com> wrote:
>
> On Thu, 11 Mar 2021, Kito Cheng wrote:
>
> > I've read the note about C++ support from the initial commit log[1],
> > so I know there is some concern about C++ support for that, is it
> > possible to enable that for C++ like a language extension for C++?
>
> I don't know if C++ has reached any conclusions about what form C++
> support for such types should take, but my expectation was that something
> library-based, similar to the support for decimal floating-point types,
> might be used.

I don't think a pure library solution is likely. It's definitely not
the direction taken by the recent proposals in the area.

Re: IEEE Interchange floating point and extended floating point for C++

[Sjoerd Meijer]

Hello GCC,

Greetings from Clang community! :-) I implemented _Float16 support in Clang.

On the Clang mailing list the request was made if we could rebrand __fp16 to a native half-precision type in Clang, but only for RISC-V. I objected to that for a few different reasons, and learned that the motivation was the unavailability of _Float16 in C++ mode in GCC. My request and proposal on the Clang list were to see about _Float16 support in C++ mode in GCC.

In Clang, _Float16 is available in both C and C++ mode. Thus, while _Float16 is indeed a C ISO/IEC TS 18661-3 extension, we have also enabled it for C++ ahead of the outcome of the C++ standardisation story. The obvious benefit is that we have, for quite some time now, a good C/C++ story to target Arm hardware with native FP16 support from Clang, which we do not have for GCC. This divergence between GCC and Clang is something we would like to address, or at lease explore what the possibilities are.

Very briefly about different half-precision source language types, in Arm we considering deprecating storage-only type __fp16, because with _Float16 and FP16 architecture extensions (RISC-V, ARM, etc.) there is no real good use-case for __fp16 anymore. Using it on modern hardware with FP16 support results in suboptimal results, and deprecating __fp16 addresses this and also the educational problem that we have of different behaving half-types.

This is all a bit of a side story, but illustrates that for FP16 architecture extensions we want a native half-precision source language type. And our current best guess is that this is _Float16, also in C++ mode. There's undeniably a standardisation gap here for C++ (and also in C), but if we speculate about possible outcomes for C++, then I think it would have to behave largely the same as _Float16 (in C mode). And in case of changes, then I hope they are small, and implementations can easily adapt. While the latter would obviously not be entirely ideal, (early) implementations might also steer and benefit the standarisation story.

So here's finally my concrete question: what do we think about making _Float16 available in C++ mode?

Thanks,
Sjoerd.



________________________________
From: Gcc <gcc-bounces@gcc.gnu.org> on behalf of Kito Cheng via Gcc <gcc@gcc.gnu.org>
Sent: 11 March 2021 09:07
To: GCC <gcc@gcc.gnu.org>
Cc: Joseph S. Myers <joseph@codesourcery.com>
Subject: IEEE Interchange floating point and extended floating point for C++

Hi:

Would it be possible to support interchange floating point and/or
extended floating point for C++, which is introduced by ISO/IEC TS
18661-3?

I've read the note about C++ support from the initial commit log[1],
so I know there is some concern about C++ support for that, is it
possible to enable that for C++ like a language extension for C++?

The main demand comes from the data type for half-precision, ISO/IEC
TS 18661-3 is the only common spec which supports half-precision and
both GCC and clang are supported.

However it can't be used on C++ for GCC, so it's hard to use that as a
workable solution for half-precision data types.

Or maybe ISO/IEC JTC1 SC22 WG14 N2016[2] is a better way for
half-precision data types?


Thanks

[1] https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=c65699efcce48d68ef57ab3ce7fc5420fac5cbf9
[2] http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf

Re: IEEE Interchange floating point and extended floating point for C++

[Jonathan Wakely]

On Fri, 12 Mar 2021 at 12:26, Sjoerd Meijer via Gcc <gcc@gcc.gnu.org> wrote:
> So here's finally my concrete question: what do we think about making _Float16 available in C++ mode?

I think GCC should do it.

Re: IEEE Interchange floating point and extended floating point for C++

[Joseph Myers]

On Fri, 12 Mar 2021, Jonathan Wakely wrote:

> On Fri, 12 Mar 2021 at 12:26, Sjoerd Meijer via Gcc <gcc@gcc.gnu.org> wrote:
> > So here's finally my concrete question: what do we think about making _Float16 available in C++ mode?
> 
> I think GCC should do it.

What about types with the same format as an existing type (but which are 
nevertheless distinct types in C), such as _Float64 (typically same format 
as double) or _Float32 (typically same format as float, but not promoted 
when passed in variable arguments)?

When those types escape into C++ code, they cause an ICE in name mangling; 
see bug 85518.

(Incidental note on _Float16: various architectures have some hardware 
support for that format that don't have support for the type in GCC, 
although they may well have GCC built-in functions corresponding to the 
hardware support; also library support in glibc hasn't yet been 
implemented.  In the case of x86_64 (F16C hardware conversion instructions 
from Ivy Bridge onwards), the psABI document defines an ABI for _Float16 
and the corresponding complex type; in general such an ABI should be 
agreed with interested parties when adding _Float16 support for an 
architecture.)

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: IEEE Interchange floating point and extended floating point for C++

[Joseph Myers]

On Fri, 12 Mar 2021, Joseph Myers wrote:

> On Fri, 12 Mar 2021, Jonathan Wakely wrote:
> 
> > On Fri, 12 Mar 2021 at 12:26, Sjoerd Meijer via Gcc <gcc@gcc.gnu.org> wrote:
> > > So here's finally my concrete question: what do we think about making _Float16 available in C++ mode?
> > 
> > I think GCC should do it.
> 
> What about types with the same format as an existing type (but which are 
> nevertheless distinct types in C), such as _Float64 (typically same format 
> as double) or _Float32 (typically same format as float, but not promoted 
> when passed in variable arguments)?

And also the _Float32x, _Float64x types (we don't have any targets 
supporting _Float128x).  (On all systems supporting those types in GCC, 
_Float32x has the same format as double and _Float64, and _Float64x has 
the same format as either long double or _Float128, but again they are 
different types.)

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: IEEE Interchange floating point and extended floating point for C++

[Jonathan Wakely]

On Fri, 12 Mar 2021, 18:14 Joseph Myers, <joseph@codesourcery.com> wrote:

> On Fri, 12 Mar 2021, Joseph Myers wrote:
>
> > On Fri, 12 Mar 2021, Jonathan Wakely wrote:
> >
> > > On Fri, 12 Mar 2021 at 12:26, Sjoerd Meijer via Gcc <gcc@gcc.gnu.org>
> wrote:
> > > > So here's finally my concrete question: what do we think about
> making _Float16 available in C++ mode?
> > >
> > > I think GCC should do it.
> >
> > What about types with the same format as an existing type (but which are
> > nevertheless distinct types in C), such as _Float64 (typically same
> format
> > as double) or _Float32 (typically same format as float, but not promoted
> > when passed in variable arguments)?
>
> And also the _Float32x, _Float64x types (we don't have any targets
> supporting _Float128x).  (On all systems supporting those types in GCC,
> _Float32x has the same format as double and _Float64, and _Float64x has
> the same format as either long double or _Float128, but again they are
> different types.)
>

I see less value in adding additional distinct types that don't add
anything you can't already do. _Float16 gives you access to an entirely new
data type. _Float32 just complicates overloading of it's a new type with
the same representation as an existing one. With the proposed std::float32
typedef for C++ users will still be able to write code that definitely uses
an IEEE binary32 type without needing a new type to be introduced. ISTM
that what users really want is "a type guaranteed to be IEEE binary32" and
whether that is float or _Float32 is mostly irrelevant.

Re: IEEE Interchange floating point and extended floating point for C++

[Joseph Myers]

On Fri, 12 Mar 2021, Jonathan Wakely via Gcc wrote:

> I see less value in adding additional distinct types that don't add
> anything you can't already do. _Float16 gives you access to an entirely new
> data type. _Float32 just complicates overloading of it's a new type with

So would you then support _Float128, and the corresponding f128/F128 
literal suffixes, in C++, as a "more standard" version of __float128, in 
the case where it has a different format from long double (e.g. x86_64) 
but not when it has the same format as long double?

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: IEEE Interchange floating point and extended floating point for C++

[Jonathan Wakely]

On Fri, 12 Mar 2021, 21:55 Joseph Myers, <joseph@codesourcery.com> wrote:

> On Fri, 12 Mar 2021, Jonathan Wakely via Gcc wrote:
>
> > I see less value in adding additional distinct types that don't add
> > anything you can't already do. _Float16 gives you access to an entirely
> new
> > data type. _Float32 just complicates overloading of it's a new type with
>
> So would you then support _Float128, and the corresponding f128/F128
> literal suffixes, in C++, as a "more standard" version of __float128, in
> the case where it has a different format from long double (e.g. x86_64)
> but not when it has the same format as long double?
>

Or support it in both cases, but for the latter case as an alias for long
double rather than as a distinct type.

I can ask the WG21 Numerics study group what they think makes sense for
C++, because I'm not certain my choices are the right ones.

Re: IEEE Interchange floating point and extended floating point for C++

[Sjoerd Meijer]

Many thanks for your replies and thoughts on this.

I wanted to add that I agree with this, and say that this is the
current state in Clang (exactly because of the reason that
_Float16 is entirely new):
I see less value in adding additional distinct types that don't add anything you can't already do. _Float16 gives you access to an entirely new data type. _Float32 just complicates overloading of it's a new type with the same representation as an existing one. With the proposed std::float32 typedef for C++ users will still be able to write code that definitely uses an IEEE binary32 type without needing a new type to be introduced. ISTM that what users really want is "a type guaranteed to be IEEE binary32" and whether that is float or _Float32 is mostly irrelevant.
And isn't the good news that this is all fairly orthogonal
and a start could be made with _Float16 and others could be
added later if required?

Cheers,
Sjoerd.

________________________________
From: Jonathan Wakely <jwakely.gcc@gmail.com>
Sent: 12 March 2021 22:03
To: Joseph Myers <joseph@codesourcery.com>
Cc: Sjoerd Meijer <Sjoerd.Meijer@arm.com>; Kito Cheng <kito.cheng@gmail.com>; GCC <gcc@gcc.gnu.org>
Subject: Re: IEEE Interchange floating point and extended floating point for C++



On Fri, 12 Mar 2021, 21:55 Joseph Myers, <joseph@codesourcery.com<mailto:joseph@codesourcery.com>> wrote:
On Fri, 12 Mar 2021, Jonathan Wakely via Gcc wrote:

> I see less value in adding additional distinct types that don't add
> anything you can't already do. _Float16 gives you access to an entirely new
> data type. _Float32 just complicates overloading of it's a new type with

So would you then support _Float128, and the corresponding f128/F128
literal suffixes, in C++, as a "more standard" version of __float128, in
the case where it has a different format from long double (e.g. x86_64)
but not when it has the same format as long double?

Or support it in both cases, but for the latter case as an alias for long double rather than as a distinct type.

I can ask the WG21 Numerics study group what they think makes sense for C++, because I'm not certain my choices are the right ones.

Re: IEEE Interchange floating point and extended floating point for C++

[Jonathan Wakely]

On Mon, 15 Mar 2021 at 10:17, Sjoerd Meijer wrote:
>
> Many thanks for your replies and thoughts on this.
>
> I wanted to add that I agree with this, and say that this is the
> current state in Clang (exactly because of the reason that
> _Float16 is entirely new):
>
> I see less value in adding additional distinct types that don't add anything you can't already do. _Float16 gives you access to an entirely new data type. _Float32 just complicates overloading of it's a new type with the same representation as an existing one. With the proposed std::float32 typedef for C++ users will still be able to write code that definitely uses an IEEE binary32 type without needing a new type to be introduced. ISTM that what users really want is "a type guaranteed to be IEEE binary32" and whether that is float or _Float32 is mostly irrelevant.
>
> And isn't the good news that this is all fairly orthogonal
> and a start could be made with _Float16 and others could be
> added later if required?

Yes, so I think that is the right incremental step forwards for C++.