[PATCH, vectorizer]: Fix PR tree-optimization/33742, Segfault in vectorizable_operation

[PATCH, vectorizer]: Fix PR tree-optimization/33742, Segfault in vectorizable_operation

[Uros Bizjak]

[-- Attachment #1: Type: text/plain, Size: 823 bytes --]

Hello!

Attached patch introduces check for unsupported scalar types, where
get_vectype_for_scalar_type() returns NULL_TREE.

The same check was introduced in vectorizable_call(), to prevent
future failures due to the same cause.

(BTW: I belive that this patch also fixes PEAK 164.gzip failure at
http://vmakarov.fedorapeople.org/spec/index.html)

Patch was bootstrapped/regtested on i686-pc-linux-gnu.

OK for mainline?

2007-10-12  Uros Bizjak  <ubizjak@gmail.com>

	PR tree-optimization/33742
	* tree-vect-transform.c (vectorizable_operation): Return false
	if get_vectype_for_scalar_type for scalar_dest can't be determined.
	(vectorizable_call): Same for rhs_type and lhs_type.

testsuite/ChangeLog:

2007-10-12  Uros Bizjak  <ubizjak@gmail.com>

	PR tree-optimization/33742
	* gcc.dg/pr33742.c: New testcase.

Uros.

[-- Attachment #2: pr33742.diff.txt --]
[-- Type: text/plain, Size: 1473 bytes --]

Index: testsuite/gcc.dg/pr33742.c
===================================================================
--- testsuite/gcc.dg/pr33742.c	(revision 0)
+++ testsuite/gcc.dg/pr33742.c	(revision 0)
@@ -0,0 +1,16 @@
+/* { dg-do compile } */
+/* { dg-options "-O2 -ftree-vectorize" } */
+
+typedef unsigned short ush;
+extern ush prev[];
+
+void fill_window ()
+{
+  register unsigned n, m;
+
+  for (n = 0; n < 32768; n++)
+    {
+      m = prev[n];
+      prev[n] = (ush) (m >= 0x8000 ? m - 0x8000 : 0);
+    }
+}
Index: tree-vect-transform.c
===================================================================
--- tree-vect-transform.c	(revision 129251)
+++ tree-vect-transform.c	(working copy)
@@ -3016,10 +3016,14 @@ vectorizable_call (tree stmt, block_stmt
     return false;
 
   vectype_in = get_vectype_for_scalar_type (rhs_type);
+  if (!vectype_in)
+    return false;
   nunits_in = TYPE_VECTOR_SUBPARTS (vectype_in);
 
   lhs_type = TREE_TYPE (GIMPLE_STMT_OPERAND (stmt, 0));
   vectype_out = get_vectype_for_scalar_type (lhs_type);
+  if (!vectype_out)
+    return false;
   nunits_out = TYPE_VECTOR_SUBPARTS (vectype_out);
 
   /* FORNOW */
@@ -3804,6 +3808,8 @@ vectorizable_operation (tree stmt, block
 
   scalar_dest = GIMPLE_STMT_OPERAND (stmt, 0);
   vectype_out = get_vectype_for_scalar_type (TREE_TYPE (scalar_dest));
+  if (!vectype_out)
+    return false;
   nunits_out = TYPE_VECTOR_SUBPARTS (vectype_out);
   if (nunits_out != nunits_in)
     return false;

Re: [PATCH, vectorizer]: Fix PR tree-optimization/33742, Segfault in vectorizable_operation

[Richard Guenther]

On 10/12/07, Uros Bizjak <ubizjak@gmail.com> wrote:
> Hello!
>
> Attached patch introduces check for unsupported scalar types, where
> get_vectype_for_scalar_type() returns NULL_TREE.
>
> The same check was introduced in vectorizable_call(), to prevent
> future failures due to the same cause.
>
> (BTW: I belive that this patch also fixes PEAK 164.gzip failure at
> http://vmakarov.fedorapeople.org/spec/index.html)
>
> Patch was bootstrapped/regtested on i686-pc-linux-gnu.
>
> OK for mainline?

Ok.

Thanks,
Richard.

> 2007-10-12  Uros Bizjak  <ubizjak@gmail.com>
>
>         PR tree-optimization/33742
>         * tree-vect-transform.c (vectorizable_operation): Return false
>         if get_vectype_for_scalar_type for scalar_dest can't be determined.
>         (vectorizable_call): Same for rhs_type and lhs_type.
>
> testsuite/ChangeLog:
>
> 2007-10-12  Uros Bizjak  <ubizjak@gmail.com>
>
>         PR tree-optimization/33742
>         * gcc.dg/pr33742.c: New testcase.
>
> Uros.
>
>