Avoid ldbl-96 stack corruption from range reduction of pseudo-zero (bug 25487) [committed]

Avoid ldbl-96 stack corruption from range reduction of pseudo-zero (bug 25487) [committed]

[Joseph Myers]

Bug 25487 reports stack corruption in ldbl-96 sinl on a pseudo-zero
argument (an representation where all the significand bits, including
the explicit high bit, are zero, but the exponent is not zero, which
is not a valid representation for the long double type).

Although this is not a valid long double representation, existing
practice in this area (see bug 4586, originally marked invalid but
subsequently fixed) is that we still seek to avoid invalid memory
accesses as a result, in case of programs that treat arbitrary binary
data as long double representations, although the invalid
representations of the ldbl-96 format do not need to be consistently
handled the same as any particular valid representation.

This patch makes the range reduction detect pseudo-zero and unnormal
representations that would otherwise go to __kernel_rem_pio2, and
returns a NaN for them instead of continuing with the range reduction
process.  (Pseudo-zero and unnormal representations whose unbiased
exponent is less than -1 have already been safely returned from the
function before this point without going through the rest of range
reduction.)  Pseudo-zero representations would previously result in
the value passed to __kernel_rem_pio2 being all-zero, which is
definitely unsafe; unnormal representations would previously result in
a value passed whose high bit is zero, which might well be unsafe
since that is not a form of input expected by __kernel_rem_pio2.

Tested for x86_64.

---

Committed.

diff --git a/sysdeps/ieee754/ldbl-96/Makefile b/sysdeps/ieee754/ldbl-96/Makefile
index 995e90d6da..318628aed6 100644
--- a/sysdeps/ieee754/ldbl-96/Makefile
+++ b/sysdeps/ieee754/ldbl-96/Makefile
@@ -17,5 +17,6 @@
 # <https://www.gnu.org/licenses/>.
 
 ifeq ($(subdir),math)
-tests += test-canonical-ldbl-96 test-totalorderl-ldbl-96
+tests += test-canonical-ldbl-96 test-totalorderl-ldbl-96 test-sinl-pseudo
+CFLAGS-test-sinl-pseudo.c += -fstack-protector-all
 endif
diff --git a/sysdeps/ieee754/ldbl-96/e_rem_pio2l.c b/sysdeps/ieee754/ldbl-96/e_rem_pio2l.c
index 5f742321ae..bcdf20179f 100644
--- a/sysdeps/ieee754/ldbl-96/e_rem_pio2l.c
+++ b/sysdeps/ieee754/ldbl-96/e_rem_pio2l.c
@@ -210,6 +210,18 @@ __ieee754_rem_pio2l (long double x, long double *y)
       return 0;
     }
 
+  if ((i0 & 0x80000000) == 0)
+    {
+      /* Pseudo-zero and unnormal representations are not valid
+	 representations of long double.  We need to avoid stack
+	 corruption in __kernel_rem_pio2, which expects input in a
+	 particular normal form, but those representations do not need
+	 to be consistently handled like any particular floating-point
+	 value.  */
+      y[1] = y[0] = __builtin_nanl ("");
+      return 0;
+    }
+
   /* Split the 64 bits of the mantissa into three 24-bit integers
      stored in a double array.  */
   exp = j0 - 23;
diff --git a/sysdeps/ieee754/ldbl-96/test-sinl-pseudo.c b/sysdeps/ieee754/ldbl-96/test-sinl-pseudo.c
new file mode 100644
index 0000000000..f59b97769d
--- /dev/null
+++ b/sysdeps/ieee754/ldbl-96/test-sinl-pseudo.c
@@ -0,0 +1,41 @@
+/* Test sinl for pseudo-zeros and unnormals for ldbl-96 (bug 25487).
+   Copyright (C) 2020 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <https://www.gnu.org/licenses/>.  */
+
+#include <math.h>
+#include <math_ldbl.h>
+#include <stdint.h>
+
+static int
+do_test (void)
+{
+  for (int i = 0; i < 64; i++)
+    {
+      uint64_t sig = i == 63 ? 0 : 1ULL << i;
+      long double ld;
+      SET_LDOUBLE_WORDS (ld, 0x4141,
+			 sig >> 32, sig & 0xffffffffULL);
+      /* The requirement is that no stack overflow occurs when the
+	 pseudo-zero or unnormal goes through range reduction.  */
+      volatile long double ldr;
+      ldr = sinl (ld);
+      (void) ldr;
+    }
+  return 0;
+}
+
+#include <support/test-driver.c>

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Avoid ldbl-96 stack corruption from range reduction of pseudo-zero (bug 25487) [committed]

[Florian Weimer]

* Joseph Myers:

> Bug 25487 reports stack corruption in ldbl-96 sinl on a pseudo-zero
> argument (an representation where all the significand bits, including
> diff --git a/sysdeps/ieee754/ldbl-96/Makefile b/sysdeps/ieee754/ldbl-96/Makefile
> index 995e90d6da..318628aed6 100644
> --- a/sysdeps/ieee754/ldbl-96/Makefile
> +++ b/sysdeps/ieee754/ldbl-96/Makefile
> @@ -17,5 +17,6 @@
>  # <https://www.gnu.org/licenses/>.
>  
>  ifeq ($(subdir),math)
> -tests += test-canonical-ldbl-96 test-totalorderl-ldbl-96
> +tests += test-canonical-ldbl-96 test-totalorderl-ldbl-96 test-sinl-pseudo
> +CFLAGS-test-sinl-pseudo.c += -fstack-protector-all
>  endif
> diff --git a/sysdeps/ieee754/ldbl-96/e_rem_pio2l.c b/sysdeps/ieee754/ldbl-96/e_rem_pio2l.c

Thanks for fixing this.  Unfortunately, the test doesn't build on ia64
because GCC does not support -fstack-protector-all there:

cc1: error: ‘-fstack-protector’ not supported for this target [-Werror]
cc1: all warnings being treated as errors
make[3]: *** [/mnt/scratch1/fw/bmg/default/build/glibcs/ia64-linux-gnu/glibc/sysd-rules:639: /mnt/scratch1/fw/bmg/default/build/glibcs/ia64-linux-gnu/glibc/math/test-sinl-pseudo.o] Error 1

Re: Avoid ldbl-96 stack corruption from range reduction of pseudo-zero (bug 25487) [committed]

[Joseph Myers]

[-- Attachment #1: Type: text/plain, Size: 924 bytes --]

On Thu, 13 Feb 2020, Florian Weimer wrote:

> Thanks for fixing this.  Unfortunately, the test doesn't build on ia64
> because GCC does not support -fstack-protector-all there:
> 
> cc1: error: ‘-fstack-protector’ not supported for this target [-Werror]
> cc1: all warnings being treated as errors
> make[3]: *** [/mnt/scratch1/fw/bmg/default/build/glibcs/ia64-linux-gnu/glibc/sysd-rules:639: /mnt/scratch1/fw/bmg/default/build/glibcs/ia64-linux-gnu/glibc/math/test-sinl-pseudo.o] Error 1

It looks like ifeq ($(have-ssp),yes) will be needed around that CFLAGS 
setting.  (I took the unconditional CFLAGS setting from where such a 
setting is used in debug/Makefile, but I see that in fact the addition of 
the test there is conditional.  In this case, the test is appropriate 
unconditionally, it's just the use of -fstack-protector-all that needs to 
be conditional.)

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Avoid ldbl-96 stack corruption from range reduction of pseudo-zero (bug 25487) [committed]

[Florian Weimer]

* Joseph Myers:

> On Thu, 13 Feb 2020, Florian Weimer wrote:
>
>> Thanks for fixing this.  Unfortunately, the test doesn't build on ia64
>> because GCC does not support -fstack-protector-all there:
>> 
>> cc1: error: ‘-fstack-protector’ not supported for this target [-Werror]
>> cc1: all warnings being treated as errors
>> make[3]: *** [/mnt/scratch1/fw/bmg/default/build/glibcs/ia64-linux-gnu/glibc/sysd-rules:639: /mnt/scratch1/fw/bmg/default/build/glibcs/ia64-linux-gnu/glibc/math/test-sinl-pseudo.o] Error 1
>
> It looks like ifeq ($(have-ssp),yes) will be needed around that CFLAGS 
> setting.  (I took the unconditional CFLAGS setting from where such a 
> setting is used in debug/Makefile, but I see that in fact the addition of 
> the test there is conditional.  In this case, the test is appropriate 
> unconditionally, it's just the use of -fstack-protector-all that needs to 
> be conditional.)

This patch seems to fix the build.  Okay to install?

Subject: math/test-sinl-pseudo: Use stack protector only if available

This fixes commit 9333498794cde1d5cca518badf79533a24114b6f ("Avoid
ldbl-96 stack corruption from range reduction of pseudo-zero (bug 25487).").

diff --git a/sysdeps/ieee754/ldbl-96/Makefile b/sysdeps/ieee754/ldbl-96/Makefile
index 318628aed6..9785b03801 100644
--- a/sysdeps/ieee754/ldbl-96/Makefile
+++ b/sysdeps/ieee754/ldbl-96/Makefile
@@ -18,5 +18,7 @@
 
 ifeq ($(subdir),math)
 tests += test-canonical-ldbl-96 test-totalorderl-ldbl-96 test-sinl-pseudo
+ifeq ($(have-ssp),yes)
 CFLAGS-test-sinl-pseudo.c += -fstack-protector-all
-endif
+endif 
+endif # $(subdir) == math

Re: Avoid ldbl-96 stack corruption from range reduction of pseudo-zero (bug 25487) [committed]

[Joseph Myers]

On Thu, 13 Feb 2020, Florian Weimer wrote:

> This patch seems to fix the build.  Okay to install?
> 
> Subject: math/test-sinl-pseudo: Use stack protector only if available
> 
> This fixes commit 9333498794cde1d5cca518badf79533a24114b6f ("Avoid
> ldbl-96 stack corruption from range reduction of pseudo-zero (bug 25487).").

OK.

-- 
Joseph S. Myers
joseph@codesourcery.com