[PATCH] Bail out HARD_REGISTER vars in asan (PR sanitizer/81224).

[PATCH] Bail out HARD_REGISTER vars in asan (PR sanitizer/81224).

[Martin Liška]

[-- Attachment #1: Type: text/plain, Size: 702 bytes --]

Similar to what we do for UBSAN and TSAN, DECL_HARD_REGISTER variables should not
be instrumented.

Patch can bootstrap on ppc64le-redhat-linux and survives regression tests.

Ready to be installed?
Martin

gcc/ChangeLog:

2017-06-27  Martin Liska  <mliska@suse.cz>

	PR sanitizer/81224
	* asan.c (instrument_derefs): Bail out inner references
	that are hard register variables.

gcc/testsuite/ChangeLog:

2017-06-27  Martin Liska  <mliska@suse.cz>

	PR sanitizer/81224
	* gcc.dg/asan/pr81224.c: New test.
---
 gcc/asan.c                          |  3 +++
 gcc/testsuite/gcc.dg/asan/pr81224.c | 10 ++++++++++
 2 files changed, 13 insertions(+)
 create mode 100644 gcc/testsuite/gcc.dg/asan/pr81224.c



[-- Attachment #2: 0001-Bail-out-HARD_REGISTER-vars-in-asan-PR-sanitizer-812.patch --]
[-- Type: text/x-patch, Size: 815 bytes --]

diff --git a/gcc/asan.c b/gcc/asan.c
index e730530930b..3f814819add 100644
--- a/gcc/asan.c
+++ b/gcc/asan.c
@@ -1875,6 +1875,9 @@ instrument_derefs (gimple_stmt_iterator *iter, tree t,
       || bitsize != size_in_bytes * BITS_PER_UNIT)
     return;
 
+  if (VAR_P (inner) && DECL_HARD_REGISTER (inner))
+    return;
+
   if (VAR_P (inner)
       && offset == NULL_TREE
       && bitpos >= 0
diff --git a/gcc/testsuite/gcc.dg/asan/pr81224.c b/gcc/testsuite/gcc.dg/asan/pr81224.c
new file mode 100644
index 00000000000..5586fe04391
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/asan/pr81224.c
@@ -0,0 +1,10 @@
+/* PR sanitizer/80659 */
+/* { dg-do compile { target { i?86-*-* x86_64-*-* } } } */
+
+int a;
+int
+b ()
+{
+  register __attribute__ ((__vector_size__ (sizeof (int)))) int c asm("xmm0");
+  return c[a];
+}

Re: [PATCH] Bail out HARD_REGISTER vars in asan (PR sanitizer/81224).

[Jakub Jelinek]

On Tue, Jun 27, 2017 at 04:42:18PM +0200, Martin Liška wrote:
> Similar to what we do for UBSAN and TSAN, DECL_HARD_REGISTER variables should not
> be instrumented.
> 
> Patch can bootstrap on ppc64le-redhat-linux and survives regression tests.
> 
> Ready to be installed?
> Martin
> 
> gcc/ChangeLog:
> 
> 2017-06-27  Martin Liska  <mliska@suse.cz>
> 
> 	PR sanitizer/81224
> 	* asan.c (instrument_derefs): Bail out inner references
> 	that are hard register variables.
> 
> gcc/testsuite/ChangeLog:
> 
> 2017-06-27  Martin Liska  <mliska@suse.cz>
> 
> 	PR sanitizer/81224
> 	* gcc.dg/asan/pr81224.c: New test.
> ---
>  gcc/asan.c                          |  3 +++
>  gcc/testsuite/gcc.dg/asan/pr81224.c | 10 ++++++++++
>  2 files changed, 13 insertions(+)
>  create mode 100644 gcc/testsuite/gcc.dg/asan/pr81224.c
> --- /dev/null
> +++ b/gcc/testsuite/gcc.dg/asan/pr81224.c
> @@ -0,0 +1,10 @@
> +/* PR sanitizer/80659 */
> +/* { dg-do compile { target { i?86-*-* x86_64-*-* } } } */
> +
> +int a;
> +int
> +b ()
> +{
> +  register __attribute__ ((__vector_size__ (sizeof (int)))) int c asm("xmm0");
> +  return c[a];
> +}

I'm sure this test will fail on i?86 if -msse isn't on by default.
So I think you want at least dg-additional-options "-msse2"
(no need for sse2 effective target, as it is dg-do compile only test).

And, I'd expect 4 * sizeof (int) instead of size (int) as vector_size.

So, please test the testcase with something like
RUNTESTFLAGS='--target_board=unix\{-m32,-m32/-mno-sse,-m64\} asan.exp=pr81224.c'

Ok with that fixed.

	Jakub