From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by sourceware.org (Postfix) with ESMTPS id 246163858D1E for ; Tue, 22 Aug 2023 13:18:18 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 246163858D1E Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=suse.de Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id 30ECB22C36; Tue, 22 Aug 2023 13:18:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1692710297; h=from:from:reply-to:date:date:to:to:cc:cc:mime-version:mime-version: content-type:content-type; bh=tpBNY8o6ZjDtIqbFIeE7lsDD/RM3gqEZO+yM0NA3uGA=; b=WKF4E36+XJ/Wt3kGfd+k/fx83aJoNEdM03+p7SMMDVVHjl2GaAPaJce533M1//d+fe6VBW 02b3I31waWNrtac+py94uoU6xat5jR3HjrjcZTacpKL9zqEa0Xa8rB+1pLB37HOx9v2Hp1 2VFGlwRmMM0cEJWWMO3PP5J4v3GBhj8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1692710297; h=from:from:reply-to:date:date:to:to:cc:cc:mime-version:mime-version: content-type:content-type; bh=tpBNY8o6ZjDtIqbFIeE7lsDD/RM3gqEZO+yM0NA3uGA=; b=unFNI4vyB9IhyFhhzeNAQSPvfKBqd8aj9r8G9hf1LwoeH7AxQpjx/K8oTBJQaQd4dW7v7W GzTXQaM/vqWpmDAA== Received: from wotan.suse.de (wotan.suse.de [10.160.0.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 2765E2C145; Tue, 22 Aug 2023 13:18:17 +0000 (UTC) Date: Tue, 22 Aug 2023 13:18:17 +0000 (UTC) From: Richard Biener To: gcc-patches@gcc.gnu.org cc: linkw@linux.ibm.com Subject: [PATCH] Simplify intereaved store vectorization processing User-Agent: Alpine 2.22 (LSU 394 2020-01-19) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII X-Spam-Status: No, score=-10.6 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,GIT_PATCH_0,MISSING_MID,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Message-ID: <20230822131817.Jy1bBHlKN9wplEuozgUofDg4yXX13V5t4hykoxx-_ac@z> When doing interleaving we perform code generation when visiting the last store of a chain. We keep track of this via DR_GROUP_STORE_COUNT, the following localizes this to the caller of vectorizable_store, also avoing redundant non-processing of the other stores. Bootstrapped and tested on x86_64-unknown-linux-gnu, pushed. * tree-vect-stmts.cc (vectorizable_store): Do not bump DR_GROUP_STORE_COUNT here. Remove early out. (vect_transform_stmt): Only call vectorizable_store on the last element of an interleaving chain. --- gcc/tree-vect-stmts.cc | 40 ++++++++++++++-------------------------- 1 file changed, 14 insertions(+), 26 deletions(-) diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc index 33f62b77710..43502dc169f 100644 --- a/gcc/tree-vect-stmts.cc +++ b/gcc/tree-vect-stmts.cc @@ -8429,24 +8429,11 @@ vectorizable_store (vec_info *vinfo, else if (STMT_VINFO_SIMD_LANE_ACCESS_P (stmt_info) >= 3) return vectorizable_scan_store (vinfo, stmt_info, gsi, vec_stmt, ncopies); - if (STMT_VINFO_GROUPED_ACCESS (stmt_info)) - DR_GROUP_STORE_COUNT (DR_GROUP_FIRST_ELEMENT (stmt_info))++; - if (grouped_store) { /* FORNOW */ gcc_assert (!loop || !nested_in_vect_loop_p (loop, stmt_info)); - /* We vectorize all the stmts of the interleaving group when we - reach the last stmt in the group. */ - if (DR_GROUP_STORE_COUNT (first_stmt_info) - < DR_GROUP_SIZE (first_stmt_info) - && !slp) - { - *vec_stmt = NULL; - return true; - } - if (slp) { grouped_store = false; @@ -12487,21 +12474,22 @@ vect_transform_stmt (vec_info *vinfo, break; case store_vec_info_type: - done = vectorizable_store (vinfo, stmt_info, - gsi, &vec_stmt, slp_node, NULL); - gcc_assert (done); - if (STMT_VINFO_GROUPED_ACCESS (stmt_info) && !slp_node) + if (STMT_VINFO_GROUPED_ACCESS (stmt_info) + && !slp_node + && (++DR_GROUP_STORE_COUNT (DR_GROUP_FIRST_ELEMENT (stmt_info)) + < DR_GROUP_SIZE (DR_GROUP_FIRST_ELEMENT (stmt_info)))) + /* In case of interleaving, the whole chain is vectorized when the + last store in the chain is reached. Store stmts before the last + one are skipped, and there vec_stmt_info shouldn't be freed + meanwhile. */ + ; + else { - /* In case of interleaving, the whole chain is vectorized when the - last store in the chain is reached. Store stmts before the last - one are skipped, and there vec_stmt_info shouldn't be freed - meanwhile. */ - stmt_vec_info group_info = DR_GROUP_FIRST_ELEMENT (stmt_info); - if (DR_GROUP_STORE_COUNT (group_info) == DR_GROUP_SIZE (group_info)) - is_store = true; + done = vectorizable_store (vinfo, stmt_info, + gsi, &vec_stmt, slp_node, NULL); + gcc_assert (done); + is_store = true; } - else - is_store = true; break; case condition_vec_info_type: -- 2.35.3