From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 61843 invoked by alias); 23 Oct 2017 17:25:08 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Received: (qmail 61724 invoked by uid 89); 23 Oct 2017 17:25:07 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-11.0 required=5.0 tests=AWL,BAYES_00,GIT_PATCH_2,GIT_PATCH_3,KAM_ASCII_DIVIDERS,RCVD_IN_DNSWL_NONE,SPF_PASS autolearn=ham version=3.3.2 spammy= X-HELO: mail-wr0-f179.google.com Received: from mail-wr0-f179.google.com (HELO mail-wr0-f179.google.com) (209.85.128.179) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Mon, 23 Oct 2017 17:25:05 +0000 Received: by mail-wr0-f179.google.com with SMTP id j15so6027075wre.8 for ; Mon, 23 Oct 2017 10:25:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:mail-followup-to:subject:references:date :in-reply-to:message-id:user-agent:mime-version; bh=xOFEZoYChArpkOGkzC0txOO+Yn4YR2VC27NIfrblWWw=; b=Xl16nOi5GfzlAUonDY6e6jUPTtVtQKgiS75pivuKfPUuE+YPXQtx8wg/UDOyRI5w0T WUYK+JP+owQgOwi2Q3KgKw5fmRxMsrF3PLAuKIesHXGkqrQiuwsO6fT/ey10DNHYzvXR 981b5eimyYmh+d1apE48iWzhOEfWd7U8iKF0hAMNxCpZj8SPfp9YAWhXR35qUSbAPoZA HPpT4Q4v1IJOZ73O4e7xePuqz1pB2SG5Yy9FTdfDs8XfP2yGefiFlYjChALYkBsJveVv T8COsplV4SPvvNJedKnFgzcE6w2jNA9bopNpHhzp/Pcfa68t25xqPFFGMPNqZ+TNXmDh UrWg== X-Gm-Message-State: AMCzsaVVcXXT9yadecBgUNy8JCTd8dNnMo+oRfKIAG3BPrXiwEC3jqSR rEVaF95lJpHxpWX0s9Rrby2JBQbhgmg= X-Google-Smtp-Source: ABhQp+QxwDGhRM+IFpZsrH0faTbvJoQNWy2k6G61bsyot2Gbzb1kKAPeaUlheNrobMCdr6c8poEAdg== X-Received: by 10.223.153.234 with SMTP id y97mr5578611wrb.165.1508779503605; Mon, 23 Oct 2017 10:25:03 -0700 (PDT) Received: from localhost ([2.26.27.199]) by smtp.gmail.com with ESMTPSA id m64sm4672116wmb.10.2017.10.23.10.25.02 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 23 Oct 2017 10:25:02 -0700 (PDT) From: Richard Sandiford To: gcc-patches@gcc.gnu.org Mail-Followup-To: gcc-patches@gcc.gnu.org, richard.sandiford@linaro.org Subject: [060/nnn] poly_int: loop versioning threshold References: <871sltvm7r.fsf@linaro.org> Date: Mon, 23 Oct 2017 17:25:00 -0000 In-Reply-To: <871sltvm7r.fsf@linaro.org> (Richard Sandiford's message of "Mon, 23 Oct 2017 17:54:32 +0100") Message-ID: <87o9oxkc9d.fsf@linaro.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SW-Source: 2017-10/txt/msg01561.txt.bz2 This patch splits the loop versioning threshold out from the cost model threshold so that the former can become a poly_uint64. We still use a single test to enforce both limits where possible. 2017-10-23 Richard Sandiford Alan Hayward David Sherwood gcc/ * tree-vectorizer.h (_loop_vec_info): Add a versioning_threshold field. (LOOP_VINFO_VERSIONING_THRESHOLD): New macro (vect_loop_versioning): Take the loop versioning threshold as a separate parameter. * tree-vect-loop-manip.c (vect_loop_versioning): Likewise. * tree-vect-loop.c (_loop_vec_info::_loop_vec_info): Initialize versioning_threshold. (vect_analyze_loop_2): Compute the loop versioning threshold whenever loop versioning is needed, and store it in the new field rather than combining it with the cost model threshold. (vect_transform_loop): Update call to vect_loop_versioning. Try to combine the loop versioning and cost thresholds here. Index: gcc/tree-vectorizer.h =================================================================== --- gcc/tree-vectorizer.h 2017-10-23 17:11:39.817127625 +0100 +++ gcc/tree-vectorizer.h 2017-10-23 17:22:23.377858186 +0100 @@ -238,6 +238,12 @@ typedef struct _loop_vec_info : public v PARAM_MIN_VECT_LOOP_BOUND. */ unsigned int th; + /* When applying loop versioning, the vector form should only be used + if the number of scalar iterations is >= this value, on top of all + the other requirements. Ignored when loop versioning is not being + used. */ + poly_uint64 versioning_threshold; + /* Unrolling factor */ int vectorization_factor; @@ -357,6 +363,7 @@ #define LOOP_VINFO_NITERS(L) #define LOOP_VINFO_NITERS_UNCHANGED(L) (L)->num_iters_unchanged #define LOOP_VINFO_NITERS_ASSUMPTIONS(L) (L)->num_iters_assumptions #define LOOP_VINFO_COST_MODEL_THRESHOLD(L) (L)->th +#define LOOP_VINFO_VERSIONING_THRESHOLD(L) (L)->versioning_threshold #define LOOP_VINFO_VECTORIZABLE_P(L) (L)->vectorizable #define LOOP_VINFO_VECT_FACTOR(L) (L)->vectorization_factor #define LOOP_VINFO_MAX_VECT_FACTOR(L) (L)->max_vectorization_factor @@ -1143,7 +1150,8 @@ extern void slpeel_make_loop_iterate_nti extern bool slpeel_can_duplicate_loop_p (const struct loop *, const_edge); struct loop *slpeel_tree_duplicate_loop_to_edge_cfg (struct loop *, struct loop *, edge); -extern void vect_loop_versioning (loop_vec_info, unsigned int, bool); +extern void vect_loop_versioning (loop_vec_info, unsigned int, bool, + poly_uint64); extern struct loop *vect_do_peeling (loop_vec_info, tree, tree, tree *, tree *, tree *, int, bool, bool); extern source_location find_loop_location (struct loop *); Index: gcc/tree-vect-loop-manip.c =================================================================== --- gcc/tree-vect-loop-manip.c 2017-10-23 17:11:39.816125711 +0100 +++ gcc/tree-vect-loop-manip.c 2017-10-23 17:22:23.376857985 +0100 @@ -2295,7 +2295,8 @@ vect_create_cond_for_alias_checks (loop_ void vect_loop_versioning (loop_vec_info loop_vinfo, - unsigned int th, bool check_profitability) + unsigned int th, bool check_profitability, + poly_uint64 versioning_threshold) { struct loop *loop = LOOP_VINFO_LOOP (loop_vinfo), *nloop; struct loop *scalar_loop = LOOP_VINFO_SCALAR_LOOP (loop_vinfo); @@ -2320,6 +2321,17 @@ vect_loop_versioning (loop_vec_info loop cond_expr = fold_build2 (GE_EXPR, boolean_type_node, scalar_loop_iters, build_int_cst (TREE_TYPE (scalar_loop_iters), th - 1)); + if (maybe_nonzero (versioning_threshold)) + { + tree expr = fold_build2 (GE_EXPR, boolean_type_node, scalar_loop_iters, + build_int_cst (TREE_TYPE (scalar_loop_iters), + versioning_threshold - 1)); + if (cond_expr) + cond_expr = fold_build2 (BIT_AND_EXPR, boolean_type_node, + expr, cond_expr); + else + cond_expr = expr; + } if (version_niter) vect_create_cond_for_niters_checks (loop_vinfo, &cond_expr); Index: gcc/tree-vect-loop.c =================================================================== --- gcc/tree-vect-loop.c 2017-10-23 17:11:39.816125711 +0100 +++ gcc/tree-vect-loop.c 2017-10-23 17:22:23.377858186 +0100 @@ -1110,6 +1110,7 @@ _loop_vec_info::_loop_vec_info (struct l num_iters_unchanged (NULL_TREE), num_iters_assumptions (NULL_TREE), th (0), + versioning_threshold (0), vectorization_factor (0), max_vectorization_factor (0), unaligned_dr (NULL), @@ -2174,11 +2175,9 @@ vect_analyze_loop_2 (loop_vec_info loop_ enough for both peeled prolog loop and vector loop. This check can be merged along with threshold check of loop versioning, so increase threshold for this case if necessary. */ - if (LOOP_REQUIRES_VERSIONING (loop_vinfo) - && (LOOP_VINFO_PEELING_FOR_GAPS (loop_vinfo) - || LOOP_VINFO_PEELING_FOR_NITER (loop_vinfo))) + if (LOOP_REQUIRES_VERSIONING (loop_vinfo)) { - unsigned niters_th; + poly_uint64 niters_th; /* Niters for peeled prolog loop. */ if (LOOP_VINFO_PEELING_FOR_ALIGNMENT (loop_vinfo) < 0) @@ -2195,9 +2194,8 @@ vect_analyze_loop_2 (loop_vec_info loop_ niters_th += LOOP_VINFO_VECT_FACTOR (loop_vinfo); /* One additional iteration because of peeling for gap. */ if (LOOP_VINFO_PEELING_FOR_GAPS (loop_vinfo)) - niters_th++; - if (LOOP_VINFO_COST_MODEL_THRESHOLD (loop_vinfo) < niters_th) - LOOP_VINFO_COST_MODEL_THRESHOLD (loop_vinfo) = niters_th; + niters_th += 1; + LOOP_VINFO_VERSIONING_THRESHOLD (loop_vinfo) = niters_th; } gcc_assert (vectorization_factor @@ -2300,6 +2298,7 @@ vect_analyze_loop_2 (loop_vec_info loop_ LOOP_VINFO_PEELING_FOR_NITER (loop_vinfo) = false; LOOP_VINFO_PEELING_FOR_GAPS (loop_vinfo) = false; LOOP_VINFO_COST_MODEL_THRESHOLD (loop_vinfo) = 0; + LOOP_VINFO_VERSIONING_THRESHOLD (loop_vinfo) = 0; goto start_over; } @@ -7320,7 +7319,17 @@ vect_transform_loop (loop_vec_info loop_ if (LOOP_REQUIRES_VERSIONING (loop_vinfo)) { - vect_loop_versioning (loop_vinfo, th, check_profitability); + poly_uint64 versioning_threshold + = LOOP_VINFO_VERSIONING_THRESHOLD (loop_vinfo); + if (check_profitability + && ordered_p (poly_uint64 (th), versioning_threshold)) + { + versioning_threshold = ordered_max (poly_uint64 (th), + versioning_threshold); + check_profitability = false; + } + vect_loop_versioning (loop_vinfo, th, check_profitability, + versioning_threshold); check_profitability = false; }