From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 32159 invoked by alias); 9 Apr 2010 10:35:54 -0000 Received: (qmail 32124 invoked by uid 22791); 9 Apr 2010 10:35:49 -0000 X-SWARE-Spam-Status: No, hits=-1.8 required=5.0 tests=BAYES_00,TW_TM,T_RP_MATCHES_RCVD X-Spam-Check-By: sourceware.org Received: from cantor2.suse.de (HELO mx2.suse.de) (195.135.220.15) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Fri, 09 Apr 2010 10:35:39 +0000 Received: from relay1.suse.de (charybdis-ext.suse.de [195.135.221.2]) by mx2.suse.de (Postfix) with ESMTP id B024D5FC9F; Fri, 9 Apr 2010 12:35:35 +0200 (CEST) Date: Fri, 09 Apr 2010 10:35:00 -0000 From: Richard Guenther To: Ira Rosen Cc: gcc-patches@gcc.gnu.org Subject: Re: [PATCH][RFC] Clean up STMT_VINFO_VECTYPE In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (LNX 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org X-SW-Source: 2010-04/txt/msg00409.txt.bz2 On Mon, 1 Mar 2010, Ira Rosen wrote: > Richard Guenther wrote on 25/02/2010 07:05:10 PM: > > > This cleans up the semantic of STMT_VINFO_VECTYPE to be always the > > vector type of the LHS of the statement. This allows to get rid > > of a lot of vector type constructions in the various vectorization > > routines. Which makes it way easier for me to add support for > > multiple vector sizes to the vectorizer as I can restrict setting > > vector types to a central analysis place. > > > > The biggest part of the patch is re-ordering of tests in the > > analysis/transform functions where we have to call > > vect_is_simple_use_1 to get the input vector type(s). This new > > function can now simply look up the definition statement and > > return its STMT_VINFO_VECTYPE, something previously not possible > > (For non-loop related definitions the analysis/transform function > > is still responsible for choosing a proper vector type - but > > that's something most already do or should be easy to add. In > > the patch I simply fall back to the old method for now) > > > > Thus, this is part 1 of a series to teach the vectorizer about > > AVX 16/32 byte mixed vector sizes. It's a cleanup that looked > > obvious anyway, too. > > > > Bootstrap and regtest running on x86_64-unknown-linux-gnu. > > > > Does this look reasonable for 4.6? I'd appreciate testing > > on powerpc in case I missed some paths not excercised by x86_64/i?86. > > Looks good to me. > I tested with vectorization testsuite on powerpc. I am now running the > whole testsuite, will let you know if anything goes wrong. I have now merged some of the later changes into this patch (the introduction of get_same_sized_vectype) so I need to touch the same place only once. Re-bootstrapped and tested on x86_64-unknown-linux-gnu, committed to trunk as follows. Richard. 2010-04-09 Richard Guenther * tree-vectorizer.h (struct _stmt_vec_info): Document that vectype is the type of the LHS. (supportable_widening_operation, supportable_narrowing_operation): Get both input and output vector types as arguments. (vect_is_simple_use_1): Declare. (get_same_sized_vectype): Likewise. * tree-vect-loop.c (vect_determine_vectorization_factor): Set STMT_VINFO_VECTYPE to the vector type of the def. (vectorizable_reduction): Adjust. * tree-vect-patterns.c (vect_recog_widen_mult_pattern): Adjust. Specify the output vector type. (vect_pattern_recog_1): Adjust. * tree-vect-stmts.c (get_same_sized_vectype): New function. (vectorizable_call): Adjust. (vectorizable_conversion): Likewise. (vectorizable_operation): Likewise. (vectorizable_type_demotion): Likewise. (vectorizable_type_promotion): Likewise. (vect_analyze_stmt): Set STMT_VINFO_VECTYPE to the vector type of the def. (vect_is_simple_use_1): New function. (supportable_widening_operation): Get both input and output vector types. (supportable_narrowing_operation): Likewise. * tree-vect-slp.c (vect_schedule_slp_instance): Adjust. Index: trunk/gcc/tree-vect-loop.c =================================================================== *** trunk.orig/gcc/tree-vect-loop.c 2010-04-07 11:23:06.000000000 +0200 --- trunk/gcc/tree-vect-loop.c 2010-04-09 11:03:41.000000000 +0200 *************** vect_determine_vectorization_factor (loo *** 240,245 **** --- 240,246 ---- for (si = gsi_start_bb (bb); !gsi_end_p (si); gsi_next (&si)) { + tree vf_vectype; gimple stmt = gsi_stmt (si); stmt_info = vinfo_for_stmt (stmt); *************** vect_determine_vectorization_factor (loo *** 294,307 **** gcc_assert (!STMT_VINFO_DATA_REF (stmt_info) && !is_pattern_stmt_p (stmt_info)); ! scalar_type = vect_get_smallest_scalar_type (stmt, &dummy, ! &dummy); if (vect_print_dump_info (REPORT_DETAILS)) { fprintf (vect_dump, "get vectype for scalar type: "); print_generic_expr (vect_dump, scalar_type, TDF_SLIM); } - vectype = get_vectype_for_scalar_type (scalar_type); if (!vectype) { --- 295,306 ---- gcc_assert (!STMT_VINFO_DATA_REF (stmt_info) && !is_pattern_stmt_p (stmt_info)); ! scalar_type = TREE_TYPE (gimple_get_lhs (stmt)); if (vect_print_dump_info (REPORT_DETAILS)) { fprintf (vect_dump, "get vectype for scalar type: "); print_generic_expr (vect_dump, scalar_type, TDF_SLIM); } vectype = get_vectype_for_scalar_type (scalar_type); if (!vectype) { *************** vect_determine_vectorization_factor (loo *** 313,335 **** } return false; } STMT_VINFO_VECTYPE (stmt_info) = vectype; } if (vect_print_dump_info (REPORT_DETAILS)) { fprintf (vect_dump, "vectype: "); ! print_generic_expr (vect_dump, vectype, TDF_SLIM); } ! nunits = TYPE_VECTOR_SUBPARTS (vectype); if (vect_print_dump_info (REPORT_DETAILS)) fprintf (vect_dump, "nunits = %d", nunits); if (!vectorization_factor || (nunits > vectorization_factor)) vectorization_factor = nunits; - } } --- 312,371 ---- } return false; } + STMT_VINFO_VECTYPE (stmt_info) = vectype; } + /* The vectorization factor is according to the smallest + scalar type (or the largest vector size, but we only + support one vector size per loop). */ + scalar_type = vect_get_smallest_scalar_type (stmt, &dummy, + &dummy); + if (vect_print_dump_info (REPORT_DETAILS)) + { + fprintf (vect_dump, "get vectype for scalar type: "); + print_generic_expr (vect_dump, scalar_type, TDF_SLIM); + } + vf_vectype = get_vectype_for_scalar_type (scalar_type); + if (!vf_vectype) + { + if (vect_print_dump_info (REPORT_UNVECTORIZED_LOCATIONS)) + { + fprintf (vect_dump, + "not vectorized: unsupported data-type "); + print_generic_expr (vect_dump, scalar_type, TDF_SLIM); + } + return false; + } + + if ((GET_MODE_SIZE (TYPE_MODE (vectype)) + != GET_MODE_SIZE (TYPE_MODE (vf_vectype)))) + { + if (vect_print_dump_info (REPORT_UNVECTORIZED_LOCATIONS)) + { + fprintf (vect_dump, + "not vectorized: different sized vector " + "types in statement, "); + print_generic_expr (vect_dump, vectype, TDF_SLIM); + fprintf (vect_dump, " and "); + print_generic_expr (vect_dump, vf_vectype, TDF_SLIM); + } + return false; + } + if (vect_print_dump_info (REPORT_DETAILS)) { fprintf (vect_dump, "vectype: "); ! print_generic_expr (vect_dump, vf_vectype, TDF_SLIM); } ! nunits = TYPE_VECTOR_SUBPARTS (vf_vectype); if (vect_print_dump_info (REPORT_DETAILS)) fprintf (vect_dump, "nunits = %d", nunits); if (!vectorization_factor || (nunits > vectorization_factor)) vectorization_factor = nunits; } } *************** vectorizable_reduction (gimple stmt, gim *** 3446,3452 **** tree scalar_dest; tree loop_vec_def0 = NULL_TREE, loop_vec_def1 = NULL_TREE; stmt_vec_info stmt_info = vinfo_for_stmt (stmt); ! tree vectype = STMT_VINFO_VECTYPE (stmt_info); loop_vec_info loop_vinfo = STMT_VINFO_LOOP_VINFO (stmt_info); struct loop *loop = LOOP_VINFO_LOOP (loop_vinfo); enum tree_code code, orig_code, epilog_reduc_code; --- 3482,3489 ---- tree scalar_dest; tree loop_vec_def0 = NULL_TREE, loop_vec_def1 = NULL_TREE; stmt_vec_info stmt_info = vinfo_for_stmt (stmt); ! tree vectype_out = STMT_VINFO_VECTYPE (stmt_info); ! tree vectype_in = NULL_TREE; loop_vec_info loop_vinfo = STMT_VINFO_LOOP_VINFO (stmt_info); struct loop *loop = LOOP_VINFO_LOOP (loop_vinfo); enum tree_code code, orig_code, epilog_reduc_code; *************** vectorizable_reduction (gimple stmt, gim *** 3464,3471 **** stmt_vec_info orig_stmt_info; tree expr = NULL_TREE; int i; ! int nunits = TYPE_VECTOR_SUBPARTS (vectype); ! int ncopies = LOOP_VINFO_VECT_FACTOR (loop_vinfo) / nunits; int epilog_copies; stmt_vec_info prev_stmt_info, prev_phi_info; gimple first_phi = NULL; --- 3501,3507 ---- stmt_vec_info orig_stmt_info; tree expr = NULL_TREE; int i; ! int ncopies; int epilog_copies; stmt_vec_info prev_stmt_info, prev_phi_info; gimple first_phi = NULL; *************** vectorizable_reduction (gimple stmt, gim *** 3491,3498 **** nested_cycle = true; } - gcc_assert (ncopies >= 1); - /* FORNOW: SLP not supported. */ if (STMT_SLP_TYPE (stmt_info)) return false; --- 3527,3532 ---- *************** vectorizable_reduction (gimple stmt, gim *** 3579,3590 **** reduction variable. */ for (i = 0; i < op_type-1; i++) { /* The condition of COND_EXPR is checked in vectorizable_condition(). */ if (i == 0 && code == COND_EXPR) continue; ! is_simple_use = vect_is_simple_use (ops[i], loop_vinfo, NULL, &def_stmt, ! &def, &dt); gcc_assert (is_simple_use); if (dt != vect_internal_def && dt != vect_external_def --- 3613,3628 ---- reduction variable. */ for (i = 0; i < op_type-1; i++) { + tree tem; + /* The condition of COND_EXPR is checked in vectorizable_condition(). */ if (i == 0 && code == COND_EXPR) continue; ! is_simple_use = vect_is_simple_use_1 (ops[i], loop_vinfo, NULL, ! &def_stmt, &def, &dt, &tem); ! if (!vectype_in) ! vectype_in = tem; gcc_assert (is_simple_use); if (dt != vect_internal_def && dt != vect_external_def *************** vectorizable_reduction (gimple stmt, gim *** 3602,3608 **** } is_simple_use = vect_is_simple_use (ops[i], loop_vinfo, NULL, &def_stmt, ! &def, &dt); gcc_assert (is_simple_use); gcc_assert (dt == vect_reduction_def || dt == vect_nested_cycle --- 3640,3646 ---- } is_simple_use = vect_is_simple_use (ops[i], loop_vinfo, NULL, &def_stmt, ! &def, &dt); gcc_assert (is_simple_use); gcc_assert (dt == vect_reduction_def || dt == vect_nested_cycle *************** vectorizable_reduction (gimple stmt, gim *** 3625,3631 **** if (STMT_VINFO_LIVE_P (vinfo_for_stmt (reduc_def_stmt))) return false; ! vec_mode = TYPE_MODE (vectype); if (code == COND_EXPR) { --- 3663,3674 ---- if (STMT_VINFO_LIVE_P (vinfo_for_stmt (reduc_def_stmt))) return false; ! ! ncopies = (LOOP_VINFO_VECT_FACTOR (loop_vinfo) ! / TYPE_VECTOR_SUBPARTS (vectype_in)); ! gcc_assert (ncopies >= 1); ! ! vec_mode = TYPE_MODE (vectype_in); if (code == COND_EXPR) { *************** vectorizable_reduction (gimple stmt, gim *** 3642,3648 **** /* 4. Supportable by target? */ /* 4.1. check support for the operation in the loop */ ! optab = optab_for_tree_code (code, vectype, optab_default); if (!optab) { if (vect_print_dump_info (REPORT_DETAILS)) --- 3685,3691 ---- /* 4. Supportable by target? */ /* 4.1. check support for the operation in the loop */ ! optab = optab_for_tree_code (code, vectype_in, optab_default); if (!optab) { if (vect_print_dump_info (REPORT_DETAILS)) *************** vectorizable_reduction (gimple stmt, gim *** 3666,3672 **** } /* Worthwhile without SIMD support? */ ! if (!VECTOR_MODE_P (TYPE_MODE (vectype)) && LOOP_VINFO_VECT_FACTOR (loop_vinfo) < vect_min_worthwhile_factor (code)) { --- 3709,3715 ---- } /* Worthwhile without SIMD support? */ ! if (!VECTOR_MODE_P (TYPE_MODE (vectype_in)) && LOOP_VINFO_VECT_FACTOR (loop_vinfo) < vect_min_worthwhile_factor (code)) { *************** vectorizable_reduction (gimple stmt, gim *** 3716,3733 **** /* This is a reduction pattern: get the vectype from the type of the reduction variable, and get the tree-code from orig_stmt. */ orig_code = gimple_assign_rhs_code (orig_stmt); ! vectype = get_vectype_for_scalar_type (TREE_TYPE (def)); ! if (!vectype) ! { ! if (vect_print_dump_info (REPORT_DETAILS)) ! { ! fprintf (vect_dump, "unsupported data-type "); ! print_generic_expr (vect_dump, TREE_TYPE (def), TDF_SLIM); ! } ! return false; ! } ! ! vec_mode = TYPE_MODE (vectype); } else { --- 3759,3766 ---- /* This is a reduction pattern: get the vectype from the type of the reduction variable, and get the tree-code from orig_stmt. */ orig_code = gimple_assign_rhs_code (orig_stmt); ! gcc_assert (vectype_out); ! vec_mode = TYPE_MODE (vectype_out); } else { *************** vectorizable_reduction (gimple stmt, gim *** 3755,3761 **** epilog_reduc_code = ERROR_MARK; if (reduction_code_for_scalar_code (orig_code, &epilog_reduc_code)) { ! reduc_optab = optab_for_tree_code (epilog_reduc_code, vectype, optab_default); if (!reduc_optab) { --- 3788,3794 ---- epilog_reduc_code = ERROR_MARK; if (reduction_code_for_scalar_code (orig_code, &epilog_reduc_code)) { ! reduc_optab = optab_for_tree_code (epilog_reduc_code, vectype_out, optab_default); if (!reduc_optab) { *************** vectorizable_reduction (gimple stmt, gim *** 3812,3818 **** gcc_assert (ncopies == 1); /* Create the destination vector */ ! vec_dest = vect_create_destination_var (scalar_dest, vectype); /* In case the vectorization factor (VF) is bigger than the number of elements that we can fit in a vectype (nunits), we have to generate --- 3845,3851 ---- gcc_assert (ncopies == 1); /* Create the destination vector */ ! vec_dest = vect_create_destination_var (scalar_dest, vectype_out); /* In case the vectorization factor (VF) is bigger than the number of elements that we can fit in a vectype (nunits), we have to generate *************** vectorizable_reduction (gimple stmt, gim *** 3910,3931 **** if (op_type == binary_op) { if (reduc_index == 0) ! expr = build2 (code, vectype, reduc_def, loop_vec_def0); else ! expr = build2 (code, vectype, loop_vec_def0, reduc_def); } else { if (reduc_index == 0) ! expr = build3 (code, vectype, reduc_def, loop_vec_def0, loop_vec_def1); else { if (reduc_index == 1) ! expr = build3 (code, vectype, loop_vec_def0, reduc_def, loop_vec_def1); else ! expr = build3 (code, vectype, loop_vec_def0, loop_vec_def1, reduc_def); } } --- 3943,3964 ---- if (op_type == binary_op) { if (reduc_index == 0) ! expr = build2 (code, vectype_out, reduc_def, loop_vec_def0); else ! expr = build2 (code, vectype_out, loop_vec_def0, reduc_def); } else { if (reduc_index == 0) ! expr = build3 (code, vectype_out, reduc_def, loop_vec_def0, loop_vec_def1); else { if (reduc_index == 1) ! expr = build3 (code, vectype_out, loop_vec_def0, reduc_def, loop_vec_def1); else ! expr = build3 (code, vectype_out, loop_vec_def0, loop_vec_def1, reduc_def); } } Index: trunk/gcc/tree-vect-patterns.c =================================================================== *** trunk.orig/gcc/tree-vect-patterns.c 2010-04-07 11:23:06.000000000 +0200 --- trunk/gcc/tree-vect-patterns.c 2010-04-09 11:03:41.000000000 +0200 *************** vect_recog_widen_mult_pattern (gimple la *** 362,368 **** tree oprnd0, oprnd1; tree type, half_type0, half_type1; gimple pattern_stmt; ! tree vectype; tree dummy; tree var; enum tree_code dummy_code; --- 362,368 ---- tree oprnd0, oprnd1; tree type, half_type0, half_type1; gimple pattern_stmt; ! tree vectype, vectype_out; tree dummy; tree var; enum tree_code dummy_code; *************** vect_recog_widen_mult_pattern (gimple la *** 405,418 **** /* Check target support */ vectype = get_vectype_for_scalar_type (half_type0); if (!vectype ! || !supportable_widening_operation (WIDEN_MULT_EXPR, last_stmt, vectype, &dummy, &dummy, &dummy_code, &dummy_code, &dummy_int, &dummy_vec)) return NULL; *type_in = vectype; ! *type_out = NULL_TREE; /* Pattern supported. Create a stmt to be used to replace the pattern: */ var = vect_recog_temp_ssa_var (type, NULL); --- 405,420 ---- /* Check target support */ vectype = get_vectype_for_scalar_type (half_type0); + vectype_out = get_vectype_for_scalar_type (type); if (!vectype ! || !supportable_widening_operation (WIDEN_MULT_EXPR, last_stmt, ! vectype_out, vectype, &dummy, &dummy, &dummy_code, &dummy_code, &dummy_int, &dummy_vec)) return NULL; *type_in = vectype; ! *type_out = vectype_out; /* Pattern supported. Create a stmt to be used to replace the pattern: */ var = vect_recog_temp_ssa_var (type, NULL); *************** vect_pattern_recog_1 ( *** 677,683 **** { /* No need to check target support (already checked by the pattern recognition function). */ ! pattern_vectype = type_in; } else { --- 679,687 ---- { /* No need to check target support (already checked by the pattern recognition function). */ ! if (type_out) ! gcc_assert (VECTOR_MODE_P (TYPE_MODE (type_out))); ! pattern_vectype = type_out ? type_out : type_in; } else { *************** vect_pattern_recog_1 ( *** 686,694 **** optab optab; /* Check target support */ ! pattern_vectype = get_vectype_for_scalar_type (type_in); ! if (!pattern_vectype) ! return; if (is_gimple_assign (pattern_stmt)) code = gimple_assign_rhs_code (pattern_stmt); --- 690,703 ---- optab optab; /* Check target support */ ! type_in = get_vectype_for_scalar_type (type_in); ! if (!type_in) ! return; ! if (type_out) ! type_out = get_vectype_for_scalar_type (type_out); ! else ! type_out = type_in; ! pattern_vectype = type_out; if (is_gimple_assign (pattern_stmt)) code = gimple_assign_rhs_code (pattern_stmt); *************** vect_pattern_recog_1 ( *** 698,712 **** code = CALL_EXPR; } ! optab = optab_for_tree_code (code, pattern_vectype, optab_default); ! vec_mode = TYPE_MODE (pattern_vectype); if (!optab || (icode = optab_handler (optab, vec_mode)->insn_code) == CODE_FOR_nothing ! || (type_out ! && (!get_vectype_for_scalar_type (type_out) ! || (insn_data[icode].operand[0].mode != ! TYPE_MODE (get_vectype_for_scalar_type (type_out)))))) return; } --- 707,718 ---- code = CALL_EXPR; } ! optab = optab_for_tree_code (code, type_in, optab_default); ! vec_mode = TYPE_MODE (type_in); if (!optab || (icode = optab_handler (optab, vec_mode)->insn_code) == CODE_FOR_nothing ! || (insn_data[icode].operand[0].mode != TYPE_MODE (type_out))) return; } Index: trunk/gcc/tree-vectorizer.h =================================================================== *** trunk.orig/gcc/tree-vectorizer.h 2010-04-07 11:23:06.000000000 +0200 --- trunk/gcc/tree-vectorizer.h 2010-04-09 11:13:40.000000000 +0200 *************** typedef struct _stmt_vec_info { *** 409,415 **** used outside the loop. */ bool live; ! /* The vector type to be used. */ tree vectype; /* The vectorized version of the stmt. */ --- 409,415 ---- used outside the loop. */ bool live; ! /* The vector type to be used for the LHS of this statement. */ tree vectype; /* The vectorized version of the stmt. */ *************** extern bool vect_can_advance_ivs_p (loop *** 760,774 **** /* In tree-vect-stmts.c. */ extern tree get_vectype_for_scalar_type (tree); extern bool vect_is_simple_use (tree, loop_vec_info, bb_vec_info, gimple *, tree *, enum vect_def_type *); ! extern bool supportable_widening_operation (enum tree_code, gimple, tree, tree *, tree *, enum tree_code *, enum tree_code *, int *, VEC (tree, heap) **); ! extern bool supportable_narrowing_operation (enum tree_code, const_gimple, ! tree, enum tree_code *, int *, ! VEC (tree, heap) **); extern stmt_vec_info new_stmt_vec_info (gimple stmt, loop_vec_info, bb_vec_info); extern void free_stmt_vec_info (gimple stmt); --- 760,777 ---- /* In tree-vect-stmts.c. */ extern tree get_vectype_for_scalar_type (tree); + extern tree get_same_sized_vectype (tree, tree); extern bool vect_is_simple_use (tree, loop_vec_info, bb_vec_info, gimple *, tree *, enum vect_def_type *); ! extern bool vect_is_simple_use_1 (tree, loop_vec_info, bb_vec_info, gimple *, ! tree *, enum vect_def_type *, tree *); ! extern bool supportable_widening_operation (enum tree_code, gimple, tree, tree, tree *, tree *, enum tree_code *, enum tree_code *, int *, VEC (tree, heap) **); ! extern bool supportable_narrowing_operation (enum tree_code, tree, tree, ! enum tree_code *, ! int *, VEC (tree, heap) **); extern stmt_vec_info new_stmt_vec_info (gimple stmt, loop_vec_info, bb_vec_info); extern void free_stmt_vec_info (gimple stmt); Index: trunk/gcc/tree-vect-stmts.c =================================================================== *** trunk.orig/gcc/tree-vect-stmts.c 2010-04-07 11:23:06.000000000 +0200 --- trunk/gcc/tree-vect-stmts.c 2010-04-09 11:48:35.000000000 +0200 *************** vectorizable_call (gimple stmt, gimple_s *** 1223,1229 **** int nunits_in; int nunits_out; loop_vec_info loop_vinfo = STMT_VINFO_LOOP_VINFO (stmt_info); ! tree fndecl, new_temp, def, rhs_type, lhs_type; gimple def_stmt; enum vect_def_type dt[2] = {vect_unknown_def_type, vect_unknown_def_type}; gimple new_stmt = NULL; --- 1223,1229 ---- int nunits_in; int nunits_out; loop_vec_info loop_vinfo = STMT_VINFO_LOOP_VINFO (stmt_info); ! tree fndecl, new_temp, def, rhs_type; gimple def_stmt; enum vect_def_type dt[2] = {vect_unknown_def_type, vect_unknown_def_type}; gimple new_stmt = NULL; *************** vectorizable_call (gimple stmt, gimple_s *** 1252,1259 **** --- 1252,1262 ---- if (TREE_CODE (gimple_call_lhs (stmt)) != SSA_NAME) return false; + vectype_out = STMT_VINFO_VECTYPE (stmt_info); + /* Process function arguments. */ rhs_type = NULL_TREE; + vectype_in = NULL_TREE; nargs = gimple_call_num_args (stmt); /* Bail out if the function has more than two arguments, we *************** vectorizable_call (gimple stmt, gimple_s *** 1264,1269 **** --- 1267,1274 ---- for (i = 0; i < nargs; i++) { + tree opvectype; + op = gimple_call_arg (stmt, i); /* We can only handle calls with arguments of the same type. */ *************** vectorizable_call (gimple stmt, gimple_s *** 1274,1301 **** fprintf (vect_dump, "argument types differ."); return false; } ! rhs_type = TREE_TYPE (op); ! if (!vect_is_simple_use (op, loop_vinfo, NULL, &def_stmt, &def, &dt[i])) { if (vect_print_dump_info (REPORT_DETAILS)) fprintf (vect_dump, "use not simple."); return false; } - } ! vectype_in = get_vectype_for_scalar_type (rhs_type); if (!vectype_in) ! return false; ! nunits_in = TYPE_VECTOR_SUBPARTS (vectype_in); ! ! lhs_type = TREE_TYPE (gimple_call_lhs (stmt)); ! vectype_out = get_vectype_for_scalar_type (lhs_type); ! if (!vectype_out) ! return false; ! nunits_out = TYPE_VECTOR_SUBPARTS (vectype_out); /* FORNOW */ if (nunits_in == nunits_out / 2) modifier = NARROW; else if (nunits_out == nunits_in) --- 1279,1313 ---- fprintf (vect_dump, "argument types differ."); return false; } ! if (!rhs_type) ! rhs_type = TREE_TYPE (op); ! if (!vect_is_simple_use_1 (op, loop_vinfo, NULL, ! &def_stmt, &def, &dt[i], &opvectype)) { if (vect_print_dump_info (REPORT_DETAILS)) fprintf (vect_dump, "use not simple."); return false; } ! if (!vectype_in) ! vectype_in = opvectype; ! else if (opvectype ! && opvectype != vectype_in) ! { ! if (vect_print_dump_info (REPORT_DETAILS)) ! fprintf (vect_dump, "argument vector types differ."); ! return false; ! } ! } ! /* If all arguments are external or constant defs use a vector type with ! the same size as the output vector type. */ if (!vectype_in) ! vectype_in = get_same_sized_vectype (rhs_type, vectype_out); /* FORNOW */ + nunits_in = TYPE_VECTOR_SUBPARTS (vectype_in); + nunits_out = TYPE_VECTOR_SUBPARTS (vectype_out); if (nunits_in == nunits_out / 2) modifier = NARROW; else if (nunits_out == nunits_in) *************** vectorizable_conversion (gimple stmt, gi *** 1546,1552 **** int nunits_out; tree vectype_out, vectype_in; int ncopies, j; ! tree rhs_type, lhs_type; tree builtin_decl; enum { NARROW, NONE, WIDEN } modifier; int i; --- 1558,1564 ---- int nunits_out; tree vectype_out, vectype_in; int ncopies, j; ! tree rhs_type; tree builtin_decl; enum { NARROW, NONE, WIDEN } modifier; int i; *************** vectorizable_conversion (gimple stmt, gi *** 1578,1598 **** return false; /* Check types of lhs and rhs. */ op0 = gimple_assign_rhs1 (stmt); rhs_type = TREE_TYPE (op0); ! vectype_in = get_vectype_for_scalar_type (rhs_type); if (!vectype_in) ! return false; ! nunits_in = TYPE_VECTOR_SUBPARTS (vectype_in); ! ! scalar_dest = gimple_assign_lhs (stmt); ! lhs_type = TREE_TYPE (scalar_dest); ! vectype_out = get_vectype_for_scalar_type (lhs_type); ! if (!vectype_out) ! return false; ! nunits_out = TYPE_VECTOR_SUBPARTS (vectype_out); /* FORNOW */ if (nunits_in == nunits_out / 2) modifier = NARROW; else if (nunits_out == nunits_in) --- 1590,1616 ---- return false; /* Check types of lhs and rhs. */ + scalar_dest = gimple_assign_lhs (stmt); + vectype_out = STMT_VINFO_VECTYPE (stmt_info); + op0 = gimple_assign_rhs1 (stmt); rhs_type = TREE_TYPE (op0); ! /* Check the operands of the operation. */ ! if (!vect_is_simple_use_1 (op0, loop_vinfo, NULL, ! &def_stmt, &def, &dt[0], &vectype_in)) ! { ! if (vect_print_dump_info (REPORT_DETAILS)) ! fprintf (vect_dump, "use not simple."); ! return false; ! } ! /* If op0 is an external or constant defs use a vector type of ! the same size as the output vector type. */ if (!vectype_in) ! vectype_in = get_same_sized_vectype (rhs_type, vectype_out); /* FORNOW */ + nunits_in = TYPE_VECTOR_SUBPARTS (vectype_in); + nunits_out = TYPE_VECTOR_SUBPARTS (vectype_out); if (nunits_in == nunits_out / 2) modifier = NARROW; else if (nunits_out == nunits_in) *************** vectorizable_conversion (gimple stmt, gi *** 1602,1615 **** else return false; - if (modifier == NONE) - gcc_assert (STMT_VINFO_VECTYPE (stmt_info) == vectype_out); - - /* Bail out if the types are both integral or non-integral. */ - if ((INTEGRAL_TYPE_P (rhs_type) && INTEGRAL_TYPE_P (lhs_type)) - || (!INTEGRAL_TYPE_P (rhs_type) && !INTEGRAL_TYPE_P (lhs_type))) - return false; - integral_type = INTEGRAL_TYPE_P (rhs_type) ? vectype_in : vectype_out; if (modifier == NARROW) --- 1620,1625 ---- *************** vectorizable_conversion (gimple stmt, gi *** 1626,1649 **** needs to be generated. */ gcc_assert (ncopies >= 1); - /* Check the operands of the operation. */ - if (!vect_is_simple_use (op0, loop_vinfo, NULL, &def_stmt, &def, &dt[0])) - { - if (vect_print_dump_info (REPORT_DETAILS)) - fprintf (vect_dump, "use not simple."); - return false; - } - /* Supportable by target? */ if ((modifier == NONE && !targetm.vectorize.builtin_conversion (code, integral_type)) || (modifier == WIDEN ! && !supportable_widening_operation (code, stmt, vectype_in, &decl1, &decl2, &code1, &code2, &dummy_int, &dummy)) || (modifier == NARROW ! && !supportable_narrowing_operation (code, stmt, vectype_in, &code1, &dummy_int, &dummy))) { if (vect_print_dump_info (REPORT_DETAILS)) --- 1636,1652 ---- needs to be generated. */ gcc_assert (ncopies >= 1); /* Supportable by target? */ if ((modifier == NONE && !targetm.vectorize.builtin_conversion (code, integral_type)) || (modifier == WIDEN ! && !supportable_widening_operation (code, stmt, ! vectype_out, vectype_in, &decl1, &decl2, &code1, &code2, &dummy_int, &dummy)) || (modifier == NARROW ! && !supportable_narrowing_operation (code, vectype_out, vectype_in, &code1, &dummy_int, &dummy))) { if (vect_print_dump_info (REPORT_DETAILS)) *************** vectorizable_conversion (gimple stmt, gi *** 1653,1659 **** if (modifier != NONE) { - STMT_VINFO_VECTYPE (stmt_info) = vectype_in; /* FORNOW: SLP not supported. */ if (STMT_SLP_TYPE (stmt_info)) return false; --- 1656,1661 ---- *************** vectorizable_conversion (gimple stmt, gi *** 1719,1726 **** else vec_oprnd0 = vect_get_vec_def_for_stmt_copy (dt[0], vec_oprnd0); - STMT_VINFO_VECTYPE (stmt_info) = vectype_in; - /* Generate first half of the widened result: */ new_stmt = vect_gen_widened_results_half (code1, decl1, --- 1721,1726 ---- *************** vectorizable_operation (gimple stmt, gim *** 1918,1924 **** tree op0, op1 = NULL; tree vec_oprnd1 = NULL_TREE; stmt_vec_info stmt_info = vinfo_for_stmt (stmt); ! tree vectype = STMT_VINFO_VECTYPE (stmt_info); loop_vec_info loop_vinfo = STMT_VINFO_LOOP_VINFO (stmt_info); enum tree_code code; enum machine_mode vec_mode; --- 1918,1924 ---- tree op0, op1 = NULL; tree vec_oprnd1 = NULL_TREE; stmt_vec_info stmt_info = vinfo_for_stmt (stmt); ! tree vectype; loop_vec_info loop_vinfo = STMT_VINFO_LOOP_VINFO (stmt_info); enum tree_code code; enum machine_mode vec_mode; *************** vectorizable_operation (gimple stmt, gim *** 1932,1938 **** enum vect_def_type dt[2] = {vect_unknown_def_type, vect_unknown_def_type}; gimple new_stmt = NULL; stmt_vec_info prev_stmt_info; ! int nunits_in = TYPE_VECTOR_SUBPARTS (vectype); int nunits_out; tree vectype_out; int ncopies; --- 1932,1938 ---- enum vect_def_type dt[2] = {vect_unknown_def_type, vect_unknown_def_type}; gimple new_stmt = NULL; stmt_vec_info prev_stmt_info; ! int nunits_in; int nunits_out; tree vectype_out; int ncopies; *************** vectorizable_operation (gimple stmt, gim *** 1944,1964 **** bb_vec_info bb_vinfo = STMT_VINFO_BB_VINFO (stmt_info); int vf; - if (loop_vinfo) - vf = LOOP_VINFO_VECT_FACTOR (loop_vinfo); - else - vf = 1; - - /* Multiple types in SLP are handled by creating the appropriate number of - vectorized stmts for each SLP node. Hence, NCOPIES is always 1 in - case of SLP. */ - if (slp_node) - ncopies = 1; - else - ncopies = LOOP_VINFO_VECT_FACTOR (loop_vinfo) / nunits_in; - - gcc_assert (ncopies >= 1); - if (!STMT_VINFO_RELEVANT_P (stmt_info) && !bb_vinfo) return false; --- 1944,1949 ---- *************** vectorizable_operation (gimple stmt, gim *** 1972,1985 **** if (TREE_CODE (gimple_assign_lhs (stmt)) != SSA_NAME) return false; - scalar_dest = gimple_assign_lhs (stmt); - vectype_out = get_vectype_for_scalar_type (TREE_TYPE (scalar_dest)); - if (!vectype_out) - return false; - nunits_out = TYPE_VECTOR_SUBPARTS (vectype_out); - if (nunits_out != nunits_in) - return false; - code = gimple_assign_rhs_code (stmt); /* For pointer addition, we should use the normal plus for --- 1957,1962 ---- *************** vectorizable_operation (gimple stmt, gim *** 1996,2008 **** return false; } op0 = gimple_assign_rhs1 (stmt); ! if (!vect_is_simple_use (op0, loop_vinfo, bb_vinfo, &def_stmt, &def, &dt[0])) { if (vect_print_dump_info (REPORT_DETAILS)) fprintf (vect_dump, "use not simple."); return false; } if (op_type == binary_op) { --- 1973,1999 ---- return false; } + scalar_dest = gimple_assign_lhs (stmt); + vectype_out = STMT_VINFO_VECTYPE (stmt_info); + op0 = gimple_assign_rhs1 (stmt); ! if (!vect_is_simple_use_1 (op0, loop_vinfo, bb_vinfo, ! &def_stmt, &def, &dt[0], &vectype)) { if (vect_print_dump_info (REPORT_DETAILS)) fprintf (vect_dump, "use not simple."); return false; } + /* If op0 is an external or constant def use a vector type with + the same size as the output vector type. */ + if (!vectype) + vectype = get_same_sized_vectype (TREE_TYPE (op0), vectype_out); + gcc_assert (vectype); + + nunits_out = TYPE_VECTOR_SUBPARTS (vectype_out); + nunits_in = TYPE_VECTOR_SUBPARTS (vectype); + if (nunits_out != nunits_in) + return false; if (op_type == binary_op) { *************** vectorizable_operation (gimple stmt, gim *** 2016,2021 **** --- 2007,2027 ---- } } + if (loop_vinfo) + vf = LOOP_VINFO_VECT_FACTOR (loop_vinfo); + else + vf = 1; + + /* Multiple types in SLP are handled by creating the appropriate number of + vectorized stmts for each SLP node. Hence, NCOPIES is always 1 in + case of SLP. */ + if (slp_node) + ncopies = 1; + else + ncopies = LOOP_VINFO_VECT_FACTOR (loop_vinfo) / nunits_in; + + gcc_assert (ncopies >= 1); + /* If this is a shift/rotate, determine whether the shift amount is a vector, or scalar. If the shift/rotate amount is a vector, use the vector/vector shift optabs. */ *************** vectorizable_type_demotion (gimple stmt, *** 2423,2438 **** if (!CONVERT_EXPR_CODE_P (code)) return false; op0 = gimple_assign_rhs1 (stmt); ! vectype_in = get_vectype_for_scalar_type (TREE_TYPE (op0)); if (!vectype_in) return false; - nunits_in = TYPE_VECTOR_SUBPARTS (vectype_in); ! scalar_dest = gimple_assign_lhs (stmt); ! vectype_out = get_vectype_for_scalar_type (TREE_TYPE (scalar_dest)); ! if (!vectype_out) ! return false; nunits_out = TYPE_VECTOR_SUBPARTS (vectype_out); if (nunits_in >= nunits_out) return false; --- 2429,2460 ---- if (!CONVERT_EXPR_CODE_P (code)) return false; + scalar_dest = gimple_assign_lhs (stmt); + vectype_out = STMT_VINFO_VECTYPE (stmt_info); + + /* Check the operands of the operation. */ op0 = gimple_assign_rhs1 (stmt); ! if (! ((INTEGRAL_TYPE_P (TREE_TYPE (scalar_dest)) ! && INTEGRAL_TYPE_P (TREE_TYPE (op0))) ! || (SCALAR_FLOAT_TYPE_P (TREE_TYPE (scalar_dest)) ! && SCALAR_FLOAT_TYPE_P (TREE_TYPE (op0)) ! && CONVERT_EXPR_CODE_P (code)))) ! return false; ! if (!vect_is_simple_use_1 (op0, loop_vinfo, NULL, ! &def_stmt, &def, &dt[0], &vectype_in)) ! { ! if (vect_print_dump_info (REPORT_DETAILS)) ! fprintf (vect_dump, "use not simple."); ! return false; ! } ! /* If op0 is an external def use a vector type with the ! same size as the output vector type if possible. */ ! if (!vectype_in) ! vectype_in = get_same_sized_vectype (TREE_TYPE (op0), vectype_out); if (!vectype_in) return false; ! nunits_in = TYPE_VECTOR_SUBPARTS (vectype_in); nunits_out = TYPE_VECTOR_SUBPARTS (vectype_out); if (nunits_in >= nunits_out) return false; *************** vectorizable_type_demotion (gimple stmt, *** 2446,2473 **** ncopies = LOOP_VINFO_VECT_FACTOR (loop_vinfo) / nunits_out; gcc_assert (ncopies >= 1); - if (! ((INTEGRAL_TYPE_P (TREE_TYPE (scalar_dest)) - && INTEGRAL_TYPE_P (TREE_TYPE (op0))) - || (SCALAR_FLOAT_TYPE_P (TREE_TYPE (scalar_dest)) - && SCALAR_FLOAT_TYPE_P (TREE_TYPE (op0)) - && CONVERT_EXPR_CODE_P (code)))) - return false; - - /* Check the operands of the operation. */ - if (!vect_is_simple_use (op0, loop_vinfo, NULL, &def_stmt, &def, &dt[0])) - { - if (vect_print_dump_info (REPORT_DETAILS)) - fprintf (vect_dump, "use not simple."); - return false; - } - /* Supportable by target? */ ! if (!supportable_narrowing_operation (code, stmt, vectype_in, &code1, ! &multi_step_cvt, &interm_types)) return false; - STMT_VINFO_VECTYPE (stmt_info) = vectype_in; - if (!vec_stmt) /* transformation not required. */ { STMT_VINFO_TYPE (stmt_info) = type_demotion_vec_info_type; --- 2468,2478 ---- ncopies = LOOP_VINFO_VECT_FACTOR (loop_vinfo) / nunits_out; gcc_assert (ncopies >= 1); /* Supportable by target? */ ! if (!supportable_narrowing_operation (code, vectype_out, vectype_in, ! &code1, &multi_step_cvt, &interm_types)) return false; if (!vec_stmt) /* transformation not required. */ { STMT_VINFO_TYPE (stmt_info) = type_demotion_vec_info_type; *************** vectorizable_type_promotion (gimple stmt *** 2693,2708 **** && code != WIDEN_MULT_EXPR) return false; op0 = gimple_assign_rhs1 (stmt); ! vectype_in = get_vectype_for_scalar_type (TREE_TYPE (op0)); if (!vectype_in) return false; - nunits_in = TYPE_VECTOR_SUBPARTS (vectype_in); ! scalar_dest = gimple_assign_lhs (stmt); ! vectype_out = get_vectype_for_scalar_type (TREE_TYPE (scalar_dest)); ! if (!vectype_out) ! return false; nunits_out = TYPE_VECTOR_SUBPARTS (vectype_out); if (nunits_in <= nunits_out) return false; --- 2698,2729 ---- && code != WIDEN_MULT_EXPR) return false; + scalar_dest = gimple_assign_lhs (stmt); + vectype_out = STMT_VINFO_VECTYPE (stmt_info); + + /* Check the operands of the operation. */ op0 = gimple_assign_rhs1 (stmt); ! if (! ((INTEGRAL_TYPE_P (TREE_TYPE (scalar_dest)) ! && INTEGRAL_TYPE_P (TREE_TYPE (op0))) ! || (SCALAR_FLOAT_TYPE_P (TREE_TYPE (scalar_dest)) ! && SCALAR_FLOAT_TYPE_P (TREE_TYPE (op0)) ! && CONVERT_EXPR_CODE_P (code)))) ! return false; ! if (!vect_is_simple_use_1 (op0, loop_vinfo, NULL, ! &def_stmt, &def, &dt[0], &vectype_in)) ! { ! if (vect_print_dump_info (REPORT_DETAILS)) ! fprintf (vect_dump, "use not simple."); ! return false; ! } ! /* If op0 is an external or constant def use a vector type with ! the same size as the output vector type. */ ! if (!vectype_in) ! vectype_in = get_same_sized_vectype (TREE_TYPE (op0), vectype_out); if (!vectype_in) return false; ! nunits_in = TYPE_VECTOR_SUBPARTS (vectype_in); nunits_out = TYPE_VECTOR_SUBPARTS (vectype_out); if (nunits_in <= nunits_out) return false; *************** vectorizable_type_promotion (gimple stmt *** 2717,2737 **** gcc_assert (ncopies >= 1); - if (! ((INTEGRAL_TYPE_P (TREE_TYPE (scalar_dest)) - && INTEGRAL_TYPE_P (TREE_TYPE (op0))) - || (SCALAR_FLOAT_TYPE_P (TREE_TYPE (scalar_dest)) - && SCALAR_FLOAT_TYPE_P (TREE_TYPE (op0)) - && CONVERT_EXPR_CODE_P (code)))) - return false; - - /* Check the operands of the operation. */ - if (!vect_is_simple_use (op0, loop_vinfo, NULL, &def_stmt, &def, &dt[0])) - { - if (vect_print_dump_info (REPORT_DETAILS)) - fprintf (vect_dump, "use not simple."); - return false; - } - op_type = TREE_CODE_LENGTH (code); if (op_type == binary_op) { --- 2738,2743 ---- *************** vectorizable_type_promotion (gimple stmt *** 2745,2751 **** } /* Supportable by target? */ ! if (!supportable_widening_operation (code, stmt, vectype_in, &decl1, &decl2, &code1, &code2, &multi_step_cvt, &interm_types)) return false; --- 2751,2757 ---- } /* Supportable by target? */ ! if (!supportable_widening_operation (code, stmt, vectype_out, vectype_in, &decl1, &decl2, &code1, &code2, &multi_step_cvt, &interm_types)) return false; *************** vectorizable_type_promotion (gimple stmt *** 2754,2761 **** architecture. */ gcc_assert (!(multi_step_cvt && op_type == binary_op)); - STMT_VINFO_VECTYPE (stmt_info) = vectype_in; - if (!vec_stmt) /* transformation not required. */ { STMT_VINFO_TYPE (stmt_info) = type_promotion_vec_info_type; --- 2760,2765 ---- *************** vect_analyze_stmt (gimple stmt, bool *ne *** 3944,3950 **** bb_vec_info bb_vinfo = STMT_VINFO_BB_VINFO (stmt_info); enum vect_relevant relevance = STMT_VINFO_RELEVANT (stmt_info); bool ok; - HOST_WIDE_INT dummy; tree scalar_type, vectype; if (vect_print_dump_info (REPORT_DETAILS)) --- 3948,3953 ---- *************** vect_analyze_stmt (gimple stmt, bool *ne *** 4002,4008 **** { gcc_assert (PURE_SLP_STMT (stmt_info)); ! scalar_type = vect_get_smallest_scalar_type (stmt, &dummy, &dummy); if (vect_print_dump_info (REPORT_DETAILS)) { fprintf (vect_dump, "get vectype for scalar type: "); --- 4005,4011 ---- { gcc_assert (PURE_SLP_STMT (stmt_info)); ! scalar_type = TREE_TYPE (gimple_get_lhs (stmt)); if (vect_print_dump_info (REPORT_DETAILS)) { fprintf (vect_dump, "get vectype for scalar type: "); *************** get_vectype_for_scalar_type (tree scalar *** 4455,4460 **** --- 4458,4474 ---- return vectype; } + /* Function get_same_sized_vectype + + Returns a vector type corresponding to SCALAR_TYPE of size + VECTOR_TYPE if supported by the target. */ + + tree + get_same_sized_vectype (tree scalar_type, tree vector_type ATTRIBUTE_UNUSED) + { + return get_vectype_for_scalar_type (scalar_type); + } + /* Function vect_is_simple_use. Input: *************** vect_is_simple_use (tree operand, loop_v *** 4588,4599 **** return true; } /* Function supportable_widening_operation Check whether an operation represented by the code CODE is a widening operation that is supported by the target platform in ! vector form (i.e., when operating on arguments of type VECTYPE). Widening operations we currently support are NOP (CONVERT), FLOAT and WIDEN_MULT. This function checks if these operations are supported --- 4602,4656 ---- return true; } + /* Function vect_is_simple_use_1. + + Same as vect_is_simple_use_1 but also determines the vector operand + type of OPERAND and stores it to *VECTYPE. If the definition of + OPERAND is vect_uninitialized_def, vect_constant_def or + vect_external_def *VECTYPE will be set to NULL_TREE and the caller + is responsible to compute the best suited vector type for the + scalar operand. */ + + bool + vect_is_simple_use_1 (tree operand, loop_vec_info loop_vinfo, + bb_vec_info bb_vinfo, gimple *def_stmt, + tree *def, enum vect_def_type *dt, tree *vectype) + { + if (!vect_is_simple_use (operand, loop_vinfo, bb_vinfo, def_stmt, def, dt)) + return false; + + /* Now get a vector type if the def is internal, otherwise supply + NULL_TREE and leave it up to the caller to figure out a proper + type for the use stmt. */ + if (*dt == vect_internal_def + || *dt == vect_induction_def + || *dt == vect_reduction_def + || *dt == vect_double_reduction_def + || *dt == vect_nested_cycle) + { + stmt_vec_info stmt_info = vinfo_for_stmt (*def_stmt); + if (STMT_VINFO_IN_PATTERN_P (stmt_info)) + stmt_info = vinfo_for_stmt (STMT_VINFO_RELATED_STMT (stmt_info)); + *vectype = STMT_VINFO_VECTYPE (stmt_info); + gcc_assert (*vectype != NULL_TREE); + } + else if (*dt == vect_uninitialized_def + || *dt == vect_constant_def + || *dt == vect_external_def) + *vectype = NULL_TREE; + else + gcc_unreachable (); + + return true; + } + /* Function supportable_widening_operation Check whether an operation represented by the code CODE is a widening operation that is supported by the target platform in ! vector form (i.e., when operating on arguments of type VECTYPE_IN ! producing a result of type VECTYPE_OUT). Widening operations we currently support are NOP (CONVERT), FLOAT and WIDEN_MULT. This function checks if these operations are supported *************** vect_is_simple_use (tree operand, loop_v *** 4613,4619 **** widening operation (short in the above example). */ bool ! supportable_widening_operation (enum tree_code code, gimple stmt, tree vectype, tree *decl1, tree *decl2, enum tree_code *code1, enum tree_code *code2, int *multi_step_cvt, --- 4670,4677 ---- widening operation (short in the above example). */ bool ! supportable_widening_operation (enum tree_code code, gimple stmt, ! tree vectype_out, tree vectype_in, tree *decl1, tree *decl2, enum tree_code *code1, enum tree_code *code2, int *multi_step_cvt, *************** supportable_widening_operation (enum tre *** 4626,4633 **** enum machine_mode vec_mode; enum insn_code icode1, icode2; optab optab1, optab2; ! tree type = gimple_expr_type (stmt); ! tree wide_vectype = get_vectype_for_scalar_type (type); enum tree_code c1, c2; /* The result of a vectorized widening operation usually requires two vectors --- 4684,4691 ---- enum machine_mode vec_mode; enum insn_code icode1, icode2; optab optab1, optab2; ! tree vectype = vectype_in; ! tree wide_vectype = vectype_out; enum tree_code c1, c2; /* The result of a vectorized widening operation usually requires two vectors *************** supportable_widening_operation (enum tre *** 4726,4733 **** if (code == FIX_TRUNC_EXPR) { /* The signedness is determined from output operand. */ ! optab1 = optab_for_tree_code (c1, type, optab_default); ! optab2 = optab_for_tree_code (c2, type, optab_default); } else { --- 4784,4791 ---- if (code == FIX_TRUNC_EXPR) { /* The signedness is determined from output operand. */ ! optab1 = optab_for_tree_code (c1, vectype_out, optab_default); ! optab2 = optab_for_tree_code (c2, vectype_out, optab_default); } else { *************** supportable_widening_operation (enum tre *** 4809,4815 **** Check whether an operation represented by the code CODE is a narrowing operation that is supported by the target platform in ! vector form (i.e., when operating on arguments of type VECTYPE). Narrowing operations we currently support are NOP (CONVERT) and FIX_TRUNC. This function checks if these operations are supported by --- 4867,4874 ---- Check whether an operation represented by the code CODE is a narrowing operation that is supported by the target platform in ! vector form (i.e., when operating on arguments of type VECTYPE_IN ! and producing a result of type VECTYPE_OUT). Narrowing operations we currently support are NOP (CONVERT) and FIX_TRUNC. This function checks if these operations are supported by *************** supportable_widening_operation (enum tre *** 4826,4840 **** bool supportable_narrowing_operation (enum tree_code code, ! const_gimple stmt, tree vectype, enum tree_code *code1, int *multi_step_cvt, VEC (tree, heap) **interm_types) { enum machine_mode vec_mode; enum insn_code icode1; optab optab1, interm_optab; ! tree type = gimple_expr_type (stmt); ! tree narrow_vectype = get_vectype_for_scalar_type (type); enum tree_code c1; tree intermediate_type, prev_type; int i; --- 4885,4899 ---- bool supportable_narrowing_operation (enum tree_code code, ! tree vectype_out, tree vectype_in, enum tree_code *code1, int *multi_step_cvt, VEC (tree, heap) **interm_types) { enum machine_mode vec_mode; enum insn_code icode1; optab optab1, interm_optab; ! tree vectype = vectype_in; ! tree narrow_vectype = vectype_out; enum tree_code c1; tree intermediate_type, prev_type; int i; *************** supportable_narrowing_operation (enum tr *** 4860,4866 **** if (code == FIX_TRUNC_EXPR) /* The signedness is determined from output operand. */ ! optab1 = optab_for_tree_code (c1, type, optab_default); else optab1 = optab_for_tree_code (c1, vectype, optab_default); --- 4919,4925 ---- if (code == FIX_TRUNC_EXPR) /* The signedness is determined from output operand. */ ! optab1 = optab_for_tree_code (c1, vectype_out, optab_default); else optab1 = optab_for_tree_code (c1, vectype, optab_default); Index: trunk/gcc/tree-vect-slp.c =================================================================== *** trunk.orig/gcc/tree-vect-slp.c 2010-04-07 11:23:06.000000000 +0200 --- trunk/gcc/tree-vect-slp.c 2010-04-09 11:03:41.000000000 +0200 *************** vect_schedule_slp_instance (slp_tree nod *** 1971,1977 **** stmt_info = vinfo_for_stmt (stmt); /* VECTYPE is the type of the destination. */ ! vectype = get_vectype_for_scalar_type (TREE_TYPE (gimple_assign_lhs (stmt))); nunits = (unsigned int) TYPE_VECTOR_SUBPARTS (vectype); group_size = SLP_INSTANCE_GROUP_SIZE (instance); --- 1971,1977 ---- stmt_info = vinfo_for_stmt (stmt); /* VECTYPE is the type of the destination. */ ! vectype = STMT_VINFO_VECTYPE (stmt_info); nunits = (unsigned int) TYPE_VECTOR_SUBPARTS (vectype); group_size = SLP_INSTANCE_GROUP_SIZE (instance);