public inbox for gcc-cvs@sourceware.org help / color / mirror / Atom feed
From: Richard Biener <rguenth@gcc.gnu.org> To: gcc-cvs@gcc.gnu.org Subject: [gcc r12-8205] tree-optimization/104912 - ensure cost model is checked first Date: Wed, 20 Apr 2022 11:28:17 +0000 (GMT) [thread overview] Message-ID: <20220420112817.10AA83857358@sourceware.org> (raw) https://gcc.gnu.org/g:4d4eaa77a7d27bf7b220c3be79615410c873fcf1 commit r12-8205-g4d4eaa77a7d27bf7b220c3be79615410c873fcf1 Author: Richard Biener <rguenther@suse.de> Date: Mon Mar 21 14:08:25 2022 +0100 tree-optimization/104912 - ensure cost model is checked first The following makes sure that when we build the versioning condition for vectorization including the cost model check, we check for the cost model and branch over other versioning checks. That is what the cost modeling assumes, since the cost model check is the only one accounted for in the scalar outside cost. Currently we emit all checks as straight-line code combined with bitwise ops which can result in surprising ordering of checks in the final assembly. Since loop_version accepts only a single versioning condition the splitting is done after the fact. The result is a 1.5% speedup of 416.gamess on x86_64 when compiling with -Ofast and tuning for generic or skylake. That's not enough to recover from the slowdown when vectorizing but it now cuts off the expensive alias versioning test. 2022-03-21 Richard Biener <rguenther@suse.de> PR tree-optimization/104912 * tree-vect-loop-manip.cc (vect_loop_versioning): Split the cost model check to a separate BB to make sure it is checked first and not combined with other version checks. Diff: --- gcc/tree-vect-loop-manip.cc | 60 ++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 57 insertions(+), 3 deletions(-) diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc index 63fb6f669a0..e4381eb7079 100644 --- a/gcc/tree-vect-loop-manip.cc +++ b/gcc/tree-vect-loop-manip.cc @@ -3445,13 +3445,34 @@ vect_loop_versioning (loop_vec_info loop_vinfo, cond_expr = expr; } + tree cost_name = NULL_TREE; + profile_probability prob2 = profile_probability::uninitialized (); + if (cond_expr + && !integer_truep (cond_expr) + && (version_niter + || version_align + || version_alias + || version_simd_if_cond)) + { + cost_name = cond_expr = force_gimple_operand_1 (unshare_expr (cond_expr), + &cond_expr_stmt_list, + is_gimple_val, NULL_TREE); + /* Split prob () into two so that the overall probability of passing + both the cost-model and versioning checks is the orig prob. */ + prob2 = prob.split (prob); + } + if (version_niter) vect_create_cond_for_niters_checks (loop_vinfo, &cond_expr); if (cond_expr) - cond_expr = force_gimple_operand_1 (unshare_expr (cond_expr), - &cond_expr_stmt_list, - is_gimple_condexpr, NULL_TREE); + { + gimple_seq tem = NULL; + cond_expr = force_gimple_operand_1 (unshare_expr (cond_expr), + &tem, + is_gimple_condexpr, NULL_TREE); + gimple_seq_add_seq (&cond_expr_stmt_list, tem); + } if (version_align) vect_create_cond_for_align_checks (loop_vinfo, &cond_expr, @@ -3655,6 +3676,39 @@ vect_loop_versioning (loop_vec_info loop_vinfo, update_ssa (TODO_update_ssa); } + /* Split the cost model check off to a separate BB. Costing assumes + this is the only thing we perform when we enter the scalar loop + from a failed cost decision. */ + if (cost_name && TREE_CODE (cost_name) == SSA_NAME) + { + gimple *def = SSA_NAME_DEF_STMT (cost_name); + /* All uses of the cost check are 'true' after the check we + are going to insert. */ + replace_uses_by (cost_name, boolean_true_node); + /* And we're going to build the new single use of it. */ + gcond *cond = gimple_build_cond (NE_EXPR, cost_name, boolean_false_node, + NULL_TREE, NULL_TREE); + edge e = split_block (gimple_bb (def), def); + gimple_stmt_iterator gsi = gsi_for_stmt (def); + gsi_insert_after (&gsi, cond, GSI_NEW_STMT); + edge true_e, false_e; + extract_true_false_edges_from_block (e->dest, &true_e, &false_e); + e->flags &= ~EDGE_FALLTHRU; + e->flags |= EDGE_TRUE_VALUE; + edge e2 = make_edge (e->src, false_e->dest, EDGE_FALSE_VALUE); + e->probability = prob2; + e2->probability = prob2.invert (); + set_immediate_dominator (CDI_DOMINATORS, false_e->dest, e->src); + auto_vec<basic_block, 3> adj; + for (basic_block son = first_dom_son (CDI_DOMINATORS, e->dest); + son; + son = next_dom_son (CDI_DOMINATORS, son)) + if (EDGE_COUNT (son->preds) > 1) + adj.safe_push (son); + for (auto son : adj) + set_immediate_dominator (CDI_DOMINATORS, son, e->src); + } + if (version_niter) { /* The versioned loop could be infinite, we need to clear existing
reply other threads:[~2022-04-20 11:28 UTC|newest] Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20220420112817.10AA83857358@sourceware.org \ --to=rguenth@gcc.gnu.org \ --cc=gcc-cvs@gcc.gnu.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).