unnormal Intel 80-bit long doubles and isnanl

unnormal Intel 80-bit long doubles and isnanl

[Siddhesh Poyarekar]

Hi,

The Intel 80-bit long double format has a concept of "unnormal" numbers 
that have a non-zero exponent and zero integer bit (i.e. bit 63) in the 
mantissa; all valid long double numbers have their integer bit set to 1. 
  Unnormal numbers are mentioned in "8.2.2 Unsupported Double 
Extended-Precision Floating-Point Encodings and Pseudo-Denormals" and 
listed in Table 8-3 in the Intel 64 and IA-32 Architectures Software 
Developer’s Manual Volume 1:Basic Architecture.

As per the manual, these numbers are considered unsupported and generate 
an invalid-operation exception if they are used as operands to any 
floating point instructions.  The question of this email is how the 
toolchain (including glibc) should treat these numbers since as things 
stand today, glibc and gcc disagree when it comes to isnanl.

glibc evaluates the bit pattern of the 80-bit long double and in the 
process, ignores the integer bit, i.e. bit 63.  As a result, it 
considers the unnormal number as a valid long double and isnanl returns 0.

gcc on the other hand, simply uses the number in a floating point 
comparison and uses the parity flag (which indicates an unordered 
compare, signalling a NaN) to decide if the number is a NaN.  The 
unnormal numbers behave like NaNs in this respect, in that they set the 
parity flag and with -fsignalling-nans, would result in an 
invalid-operation exception.  As a result, __builtin_isnanl returns 1 
for an unnormal number.

So the question is, which behaviour should be considered correct? 
Strictly speaking, unnormal numbers are listed separately from NaNs in 
the document and as such are distinct from NaNs.  So on the question of 
"is nan?" the answer ought to be "No".

On the flip side, the behaviour described (and experienced through code) 
is exactly the same as a NaN, i.e. a floating point operation sets the 
parity flag and generates an invalid-operation exception.  So if it 
looks like a NaN, behaves like a NaN, then even if the document hints 
(and it is just a hint right, since it doesn't specifically state it?) 
that it's different, it likely is a NaN.  What's more, one of the fixes 
to glibc[1] assumes that __builtin_isnanl will do the right thing.

The third alternative (which seems like a step back to me, but will 
concede that it is a valid resolution) is to state that unnormal input 
to isnanl would result in undefined behaviour and hence it is the 
responsibility of the application to ensure that inputs to isnanl are 
never unnormal.

Thoughts?

Siddhesh

[1] 
https://sourceware.org/git/?p=glibc.git;h=0474cd5de60448f31d7b872805257092faa626e4

Re: unnormal Intel 80-bit long doubles and isnanl

[Szabolcs Nagy]

The 11/24/2020 16:23, Siddhesh Poyarekar wrote:
> Hi,
> 
> The Intel 80-bit long double format has a concept of "unnormal" numbers that
> have a non-zero exponent and zero integer bit (i.e. bit 63) in the mantissa;
> all valid long double numbers have their integer bit set to 1.  Unnormal
> numbers are mentioned in "8.2.2 Unsupported Double Extended-Precision
> Floating-Point Encodings and Pseudo-Denormals" and listed in Table 8-3 in
> the Intel 64 and IA-32 Architectures Software Developer’s Manual Volume
> 1:Basic Architecture.
> 
> As per the manual, these numbers are considered unsupported and generate an
> invalid-operation exception if they are used as operands to any floating
> point instructions.  The question of this email is how the toolchain
> (including glibc) should treat these numbers since as things stand today,
> glibc and gcc disagree when it comes to isnanl.

ideally fpclassify (and other classification macros) would
handle all representations.

architecturally invalid or trap representations can be a
non-standard class but i think classifying them as FP_NAN
would break the least amount of code.

> glibc evaluates the bit pattern of the 80-bit long double and in the
> process, ignores the integer bit, i.e. bit 63.  As a result, it considers
> the unnormal number as a valid long double and isnanl returns 0.

i think m68k and x86 are different here.

> 
> gcc on the other hand, simply uses the number in a floating point comparison
> and uses the parity flag (which indicates an unordered compare, signalling a
> NaN) to decide if the number is a NaN.  The unnormal numbers behave like
> NaNs in this respect, in that they set the parity flag and with
> -fsignalling-nans, would result in an invalid-operation exception.  As a
> result, __builtin_isnanl returns 1 for an unnormal number.

compiling isnanl to a quiet fp compare is wrong with
-fsignalling-nans: classification is not supposed to
signal exceptions for snan.

> 
> So the question is, which behaviour should be considered correct? Strictly
> speaking, unnormal numbers are listed separately from NaNs in the document
> and as such are distinct from NaNs.  So on the question of "is nan?" the
> answer ought to be "No".
> 
> On the flip side, the behaviour described (and experienced through code) is
> exactly the same as a NaN, i.e. a floating point operation sets the parity
> flag and generates an invalid-operation exception.  So if it looks like a
> NaN, behaves like a NaN, then even if the document hints (and it is just a
> hint right, since it doesn't specifically state it?) that it's different, it
> likely is a NaN.  What's more, one of the fixes to glibc[1] assumes that
> __builtin_isnanl will do the right thing.
> 
> The third alternative (which seems like a step back to me, but will concede
> that it is a valid resolution) is to state that unnormal input to isnanl
> would result in undefined behaviour and hence it is the responsibility of
> the application to ensure that inputs to isnanl are never unnormal.
> 
> Thoughts?
> 
> Siddhesh
> 
> [1] https://sourceware.org/git/?p=glibc.git;h=0474cd5de60448f31d7b872805257092faa626e4

Re: unnormal Intel 80-bit long doubles and isnanl

[Florian Weimer]

* Szabolcs Nagy:

> ideally fpclassify (and other classification macros) would
> handle all representations.
>
> architecturally invalid or trap representations can be a
> non-standard class but i think classifying them as FP_NAN
> would break the least amount of code.

I think the fpclassify macro has the wrong signature for trap
representation checks.  It's macro, so in theory we could make it behave
in any way we want, but I think it's awkward to pass a dereferenced
pointer as a macro argument when it's as best unclear whether the
dereferencing operation itself triggers undefined behavior.

We can avoid that if we do not make these representations trap
representations, but give them a defined meaning.

>> glibc evaluates the bit pattern of the 80-bit long double and in the
>> process, ignores the integer bit, i.e. bit 63.  As a result, it considers
>> the unnormal number as a valid long double and isnanl returns 0.
>
> i think m68k and x86 are different here.

I think you are right.

Thanks,
Florian
-- 
Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill

Re: unnormal Intel 80-bit long doubles and isnanl

[Siddhesh Poyarekar]

On 11/24/20 7:11 PM, Szabolcs Nagy wrote:
> ideally fpclassify (and other classification macros) would
> handle all representations.
> 
> architecturally invalid or trap representations can be a
> non-standard class but i think classifying them as FP_NAN
> would break the least amount of code.

That's my impression too.

>> glibc evaluates the bit pattern of the 80-bit long double and in the
>> process, ignores the integer bit, i.e. bit 63.  As a result, it considers
>> the unnormal number as a valid long double and isnanl returns 0.
> 
> i think m68k and x86 are different here.
> 
>>
>> gcc on the other hand, simply uses the number in a floating point comparison
>> and uses the parity flag (which indicates an unordered compare, signalling a
>> NaN) to decide if the number is a NaN.  The unnormal numbers behave like
>> NaNs in this respect, in that they set the parity flag and with
>> -fsignalling-nans, would result in an invalid-operation exception.  As a
>> result, __builtin_isnanl returns 1 for an unnormal number.
> 
> compiling isnanl to a quiet fp compare is wrong with
> -fsignalling-nans: classification is not supposed to
> signal exceptions for snan.

I agree, but I think that issue with __builtin_isnanl is orthogonal to 
the question about unnormals.  Once that is fixed in gcc, we could 
actually use __builtin_isnanl all the time in glibc for isnanl.

Siddhesh

Re: unnormal Intel 80-bit long doubles and isnanl

[Adhemerval Zanella]

On 24/11/2020 10:59, Siddhesh Poyarekar wrote:
> On 11/24/20 7:11 PM, Szabolcs Nagy wrote:
>> ideally fpclassify (and other classification macros) would
>> handle all representations.
>>
>> architecturally invalid or trap representations can be a
>> non-standard class but i think classifying them as FP_NAN
>> would break the least amount of code.
> 
> That's my impression too.
> 
>>> glibc evaluates the bit pattern of the 80-bit long double and in the
>>> process, ignores the integer bit, i.e. bit 63.  As a result, it considers
>>> the unnormal number as a valid long double and isnanl returns 0.
>>
>> i think m68k and x86 are different here.
>>
>>>
>>> gcc on the other hand, simply uses the number in a floating point comparison
>>> and uses the parity flag (which indicates an unordered compare, signalling a
>>> NaN) to decide if the number is a NaN.  The unnormal numbers behave like
>>> NaNs in this respect, in that they set the parity flag and with
>>> -fsignalling-nans, would result in an invalid-operation exception.  As a
>>> result, __builtin_isnanl returns 1 for an unnormal number.
>>
>> compiling isnanl to a quiet fp compare is wrong with
>> -fsignalling-nans: classification is not supposed to
>> signal exceptions for snan.
> 
> I agree, but I think that issue with __builtin_isnanl is orthogonal to the question about unnormals.  Once that is fixed in gcc, we could actually use __builtin_isnanl all the time in glibc for isnanl.
> 
> Siddhesh

Which is the currently take from gcc developers on this semantic change of
__builtin_isnanl? Are they considering current behavior of non classifying
the 'unnormal' as NAN the expected behavior and waiting glibc to follow
it or are they willing to align with glibc behavior? 

Re: unnormal Intel 80-bit long doubles and isnanl

[Siddhesh Poyarekar]

On 11/24/20 7:46 PM, Adhemerval Zanella wrote:
> Which is the currently take from gcc developers on this semantic change of
> __builtin_isnanl? Are they considering current behavior of non classifying
> the 'unnormal' as NAN the expected behavior and waiting glibc to follow
> it or are they willing to align with glibc behavior?

The gcc ml is also in cc (apologies to those getting 2-3 copies of 
this!) so I'm hoping to get feedback from both communities to arrive and 
a consensus.

gcc currently considers unnormals as NaN.  I think that is the right 
behaviour and would like glibc to align with that but before making 
such a proposal for glibc, I wanted to make sure that this gcc behaviour 
is defined because currently there is nothing that makes that clear.

Siddhesh

Re: unnormal Intel 80-bit long doubles and isnanl

[Richard Biener]

On Tue, Nov 24, 2020 at 3:20 PM Adhemerval Zanella via Gcc
<gcc@gcc.gnu.org> wrote:
>
>
>
> On 24/11/2020 10:59, Siddhesh Poyarekar wrote:
> > On 11/24/20 7:11 PM, Szabolcs Nagy wrote:
> >> ideally fpclassify (and other classification macros) would
> >> handle all representations.
> >>
> >> architecturally invalid or trap representations can be a
> >> non-standard class but i think classifying them as FP_NAN
> >> would break the least amount of code.
> >
> > That's my impression too.
> >
> >>> glibc evaluates the bit pattern of the 80-bit long double and in the
> >>> process, ignores the integer bit, i.e. bit 63.  As a result, it considers
> >>> the unnormal number as a valid long double and isnanl returns 0.
> >>
> >> i think m68k and x86 are different here.
> >>
> >>>
> >>> gcc on the other hand, simply uses the number in a floating point comparison
> >>> and uses the parity flag (which indicates an unordered compare, signalling a
> >>> NaN) to decide if the number is a NaN.  The unnormal numbers behave like
> >>> NaNs in this respect, in that they set the parity flag and with
> >>> -fsignalling-nans, would result in an invalid-operation exception.  As a
> >>> result, __builtin_isnanl returns 1 for an unnormal number.
> >>
> >> compiling isnanl to a quiet fp compare is wrong with
> >> -fsignalling-nans: classification is not supposed to
> >> signal exceptions for snan.

Can you open a bugreport for this?  Note that the option is likely
to invoke isnanl from libm ...

> > I agree, but I think that issue with __builtin_isnanl is orthogonal to the question about unnormals.  Once that is fixed in gcc, we could actually use __builtin_isnanl all the time in glibc for isnanl.
> >
> > Siddhesh
>
> Which is the currently take from gcc developers on this semantic change of
> __builtin_isnanl? Are they considering current behavior of non classifying
> the 'unnormal' as NAN the expected behavior and waiting glibc to follow
> it or are they willing to align with glibc behavior?

I think GCC should follow standards and in case they do not apply do
sth reasonable - which I think classifying those as NaN is.

Richard.

Re: unnormal Intel 80-bit long doubles and isnanl

[Siddhesh Poyarekar]

On 11/24/20 8:13 PM, Richard Biener wrote:
>>>> compiling isnanl to a quiet fp compare is wrong with
>>>> -fsignalling-nans: classification is not supposed to
>>>> signal exceptions for snan.
> 
> Can you open a bugreport for this?  Note that the option is likely
> to invoke isnanl from libm ...

I believe Szabolcs is talking about this:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66462

Siddhesh

Re: unnormal Intel 80-bit long doubles and isnanl

[Andreas Schwab]

On Nov 24 2020, Szabolcs Nagy via Gcc wrote:

>> glibc evaluates the bit pattern of the 80-bit long double and in the
>> process, ignores the integer bit, i.e. bit 63.  As a result, it considers
>> the unnormal number as a valid long double and isnanl returns 0.
>
> i think m68k and x86 are different here.

The mc68881 doesn't trap on unnormal numbers, it implicitly converts
such input representations to normal, denormal or zero, as appropriate.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: unnormal Intel 80-bit long doubles and isnanl

[Joseph Myers]

On Tue, 24 Nov 2020, Siddhesh Poyarekar wrote:

> The third alternative (which seems like a step back to me, but will concede
> that it is a valid resolution) is to state that unnormal input to isnanl would
> result in undefined behaviour and hence it is the responsibility of the
> application to ensure that inputs to isnanl are never unnormal.

glibc effectively treats them as unspecified behavior - we don't expect 
them to produce any particular meaningful function return value (this 
includes the possibility that such an invalid encoding might be returned 
by a function given such an encoding as input), but if they result in 
buffer overflows, infinite loops or similar, that's fixed as a bug.

Note that there are lots of libm function implementations, not just 
isnanl, which work by looking at the encoding without considering the 
possibility it might be one of the kinds of encodings (other than sNaN) 
that cannot result from floating-point arithmetic.  This includes both 
ldbl-96 implementations in C and implementations in assembly for i386 and 
x86_64, and probably implementations for m68k and ia64 as well.  
iscanonical is the only operation in glibc that deliberately tries to 
detect and produce defined results for these encodings.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: unnormal Intel 80-bit long doubles and isnanl

[Siddhesh Poyarekar]

On 11/25/20 2:58 AM, Joseph Myers wrote:
> glibc effectively treats them as unspecified behavior - we don't expect
> them to produce any particular meaningful function return value (this
> includes the possibility that such an invalid encoding might be returned
> by a function given such an encoding as input), but if they result in
> buffer overflows, infinite loops or similar, that's fixed as a bug.

Knowing what to call these numbers would be the first step towards 
knowing what to do with them, which is why I considered classification 
the first thing to tackle.

> Note that there are lots of libm function implementations, not just
> isnanl, which work by looking at the encoding without considering the
> possibility it might be one of the kinds of encodings (other than sNaN)
> that cannot result from floating-point arithmetic.  This includes both
> ldbl-96 implementations in C and implementations in assembly for i386 and
> x86_64, and probably implementations for m68k and ia64 as well.
> iscanonical is the only operation in glibc that deliberately tries to
> detect and produce defined results for these encodings.

Once we have consensus on what unnormals are, it will be much easier for 
me to work through the toolchain and ensure that we consistently treat 
them as NaNs.

Would you agree to treating unnormals as NaNs and consequently have 
glibc provide that guarantee in the library instead of either declaring 
it undefined or maintaining the status quo, i.e. keeping it unspecified?

Siddhesh

Re: unnormal Intel 80-bit long doubles and isnanl

[Joseph Myers]

On Wed, 25 Nov 2020, Siddhesh Poyarekar wrote:

> Would you agree to treating unnormals as NaNs and consequently have glibc
> provide that guarantee in the library instead of either declaring it undefined
> or maintaining the status quo, i.e. keeping it unspecified?

I think it would be a pain to maintain test coverage for unnormals (and 
presumably all the other kinds of unsupported operands, and you'd need to 
work out what semantics you want for pseudo-denormals as well since those 
are the one kind of such representation the processor doesn't raise 
"invalid" for) for all the functions with floating-point arguments - and 
claiming to handle those consistently requires having such test coverage 
(there are only a few tests for such format-specific representations in 
sysdeps/ieee754/ldbl-96 at present).

But maybe you could set up some mechanism by which, when gen-libm-test.py 
processes a test using snan_value or snan_value_ld (but not 
snan_value_pl), and the relevant format is one of the format variants that 
has these representations, it automatically generates tests for all those 
variants (that the processor raises "invalid" for when handling as 
operands, i.e. treats much like sNaN).  I'm not sure if it's actually 
possible to generate a static initializer for a long double value with one 
of those representations, or only for a union containing a long double 
where another member is initialized; if a union type needs to be used in 
the tables of test inputs, that further complicates things.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: unnormal Intel 80-bit long doubles and isnanl

[Siddhesh Poyarekar]

On 11/26/20 12:57 AM, Joseph Myers wrote:
> I think it would be a pain to maintain test coverage for unnormals (and
> presumably all the other kinds of unsupported operands, and you'd need to
> work out what semantics you want for pseudo-denormals as well since those
> are the one kind of such representation the processor doesn't raise
> "invalid" for) for all the functions with floating-point arguments - and
> claiming to handle those consistently requires having such test coverage
> (there are only a few tests for such format-specific representations in
> sysdeps/ieee754/ldbl-96 at present).

pseudo-denormals are still considered valid, so I'm admittedly punting 
them for later since the processor manual still claims to handle them 
correctly.

> But maybe you could set up some mechanism by which, when gen-libm-test.py
> processes a test using snan_value or snan_value_ld (but not
> snan_value_pl), and the relevant format is one of the format variants that
> has these representations, it automatically generates tests for all those
> variants (that the processor raises "invalid" for when handling as
> operands, i.e. treats much like sNaN).  I'm not sure if it's actually
> possible to generate a static initializer for a long double value with one
> of those representations, or only for a union containing a long double
> where another member is initialized; if a union type needs to be used in
> the tables of test inputs, that further complicates things.
It would have to either be a union type with various bit patterns or a 
bit string copied into a long double; the CPU will never generate any of 
the pseudo numbers on its own.

Siddhesh

Re: unnormal Intel 80-bit long doubles and isnanl

[Florian Weimer]

* Joseph Myers:

> glibc effectively treats them as unspecified behavior - we don't expect 
> them to produce any particular meaningful function return value (this 
> includes the possibility that such an invalid encoding might be returned 
> by a function given such an encoding as input), but if they result in 
> buffer overflows, infinite loops or similar, that's fixed as a bug.

I think the last part (the “bug”) is new.  I welcome a consensus along
those lines.  I just want to highlight this aspect.

Thanks,
Florian
-- 
Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill

Re: unnormal Intel 80-bit long doubles and isnanl

[Siddhesh Poyarekar]

On 11/27/20 5:01 PM, Florian Weimer wrote:
> I think the last part (the “bug”) is new.  I welcome a consensus along
> those lines.  I just want to highlight this aspect.

Should we consider fixing behaviour if the bug manifests in a user 
application and not in glibc itself?  i.e. a crash because glibc either 
returned the unnormal or misclassified the unnormal number?

At the minimu ISTM that we should at least make the classification 
consistent with gcc.

Siddhesh

Re: unnormal Intel 80-bit long doubles and isnanl

[Florian Weimer]

* Siddhesh Poyarekar:

> On 11/27/20 5:01 PM, Florian Weimer wrote:
>> I think the last part (the “bug”) is new.  I welcome a consensus along
>> those lines.  I just want to highlight this aspect.
>
> Should we consider fixing behaviour if the bug manifests in a user
> application and not in glibc itself?  i.e. a crash because glibc
> either returned the unnormal or misclassified the unnormal number?

I think in general, that's a bit like fixing buffer overflows in
applications.  It's just not possible with the current compilation
model.  So I find it difficult to come up with a general rule.

The nature of these non-normal numbers is that the CPU does not produce
them.  I think we should make sure that glibc doesn't, either, with
obvious exceptions such as memcpy.  But beyond that, I don't know.

> At the minimu ISTM that we should at least make the classification
> consistent with gcc.

Yes, I agree.

Thanks,
Florian
-- 
Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill

Re: unnormal Intel 80-bit long doubles and isnanl

[Joseph Myers]

On Fri, 27 Nov 2020, Florian Weimer via Gcc wrote:

> The nature of these non-normal numbers is that the CPU does not produce
> them.  I think we should make sure that glibc doesn't, either, with
> obvious exceptions such as memcpy.  But beyond that, I don't know.

Exceptions probably also include the non-computational functions that just 
manipulate the sign bit (fabsl and copysignl).  And likewise the 
<complex.h> functions conjl, creall, cimagl.  (For all of these, built-in 
functions would normally be expanded inline rather than using the library 
version anyway.)

> > At the minimu ISTM that we should at least make the classification
> > consistent with gcc.
> 
> Yes, I agree.

If you want to be consistent with how the processor interprets these as 
invalid operands, that would also mean issignaling treats all these 
invalid operands as signaling NaNs.

We haven't so far tried to make <complex.h> functions do anything in 
particular with signaling NaNs (TS 18661-1 and C2x leave it 
implementation-defined how they are handled in such functions).  Arguably 
the functions other than those listed above should return (qNaN, qNaN) for 
any complex argument with at least one part sNaN (and thus for any complex 
argument with at least one part an invalid operand) rather than treating 
operands with one part sNaN and one Inf as an infinity.  (cabs should 
already do that, via its use of hypot which does handle sNaN.)

-- 
Joseph S. Myers
joseph@codesourcery.com