From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qt1-x82d.google.com (mail-qt1-x82d.google.com [IPv6:2607:f8b0:4864:20::82d]) by sourceware.org (Postfix) with ESMTPS id 5E8643850845 for ; Wed, 8 Mar 2023 03:27:47 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 5E8643850845 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com Received: by mail-qt1-x82d.google.com with SMTP id d7so16870072qtr.12 for ; Tue, 07 Mar 2023 19:27:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; t=1678246066; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=BezTAomqhWojoAENUJE/t4+vgnOPN/qikEYxJrjQMkA=; b=HrOzttUSGQAizEekV4TIYRVeZ5owvaEdMGVlWcaMtHJ5tiSZGT25duJTo60knXI4Hj 8MS/wCBYLiFEQ+drB/+QgUeG3c+y15IwgaZWyAalALcEAallhChNjzin/Oi7+SiMMWdV GihYtRwHHv2vlWi12j0Xt7HwiHAIg08GMqQtW8ivHsZznRtWpeAD5H+X2CSBSkDdnnPI CmUlPH9xc6AlcHW/XOo/xpPU3pLRxZtAEk+iLrBSKW4Dw8TcNzS9GaoomzL++x9ZDLV7 4+MjdqTbWX+nzdtVQ/auTuhenNaE/HUQaXMbzSldzvuKkfGPJIsVxqr6ZWcykjwK08HG +Wfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678246066; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BezTAomqhWojoAENUJE/t4+vgnOPN/qikEYxJrjQMkA=; b=482jbjyhvUiezXznjjydhev5vnOf0Yf8xZZyk4mKY8cSbU2dKFO7Obphva0+iliJFQ XmbvAXyqGDaiEq93jqgvAEcghljT1nOny7M9eMUeCcx9ubBitwZswdvSDQQ+8wnJWhPY K2eWrUX5t98yMQwZq6X9iMazHhRPciSnZSmWRwVsVq3i7SsKg4mXIqC98zZUaxLhn0oE 2goIdRi6B+Na/5cnLDd1wzxV+HEF9jg904GxDvLFfg56f95lmJyyPSgY5BDJblDO0qdK EZZIqiFEr23uJsv0X6XPaSEX9tBP2kBrBRqki898f+9z45XfohHoUv4YMoeLBNhYt4EU FAHw== X-Gm-Message-State: AO0yUKXIthiPcPg6V9X9gpGZxBdCWDAfaqXgNGlPwq0VjKscqSyC34hq VOed9+dk+ZVBaU6yORGT9FrG9++vqPkyX49/Ijw= X-Google-Smtp-Source: AK7set9uz+dbpYfDe09vCIJnyApMFY6bufwy8OIOc2n5IizrmiTwO3GsSsHIDJsQ/gJOX6Nfw6gjRA== X-Received: by 2002:a05:622a:108:b0:3b3:7707:9b92 with SMTP id u8-20020a05622a010800b003b377079b92mr27334583qtw.15.1678246066510; Tue, 07 Mar 2023 19:27:46 -0800 (PST) Received: from system76-pc.ba.rivosinc.com ([136.57.172.92]) by smtp.gmail.com with ESMTPSA id v21-20020ac87295000000b003b6382f66b1sm10759696qto.29.2023.03.07.19.27.45 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Mar 2023 19:27:45 -0800 (PST) From: Michael Collison To: gcc-patches@gcc.gnu.org Subject: [PATCH v3 4/6] RISC-V: autovec: Add target vectorization hooks Date: Tue, 7 Mar 2023 22:27:38 -0500 Message-Id: <20230308032740.989275-5-collison@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230308032740.989275-1-collison@rivosinc.com> References: <20230308032740.989275-1-collison@rivosinc.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-12.5 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,GIT_PATCH_0,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: 2023-03-02 Michael Collison Juzhe Zhong * config/riscv/riscv.cc (riscv_option_override): Set riscv_vectorization_factor. (riscv_estimated_poly_value): Implement TARGET_ESTIMATED_POLY_VALUE. (riscv_preferred_simd_mode): Implement TARGET_VECTORIZE_PREFERRED_SIMD_MODE. (riscv_autovectorize_vector_modes): Implement TARGET_AUTOVECTORIZE_VECTOR_MODES. (riscv_get_mask_mode): Implement TARGET_VECTORIZE_GET_MASK_MODE. (riscv_empty_mask_is_expensive): Implement TARGET_VECTORIZE_EMPTY_MASK_IS_EXPENSIVE. (riscv_vectorize_create_costs): Implement TARGET_VECTORIZE_CREATE_COSTS. (TARGET_ESTIMATED_POLY_VALUE): Register target macro. (TARGET_VECTORIZE_PREFERRED_SIMD_MODE): Ditto. (TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES): Ditto. (TARGET_VECTORIZE_GET_MASK_MODE): Ditto. (TARGET_VECTORIZE_EMPTY_MASK_IS_EXPENSIVE): Ditto. (TARGET_VECTORIZE_LOOP_LEN_OVERRIDE_MASK): Ditto. --- gcc/config/riscv/riscv.cc | 156 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 156 insertions(+) diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc index befb9b498b7..1ca9f3c7ae4 100644 --- a/gcc/config/riscv/riscv.cc +++ b/gcc/config/riscv/riscv.cc @@ -60,6 +60,15 @@ along with GCC; see the file COPYING3. If not see #include "opts.h" #include "tm-constrs.h" #include "rtl-iter.h" +#include "gimple.h" +#include "cfghooks.h" +#include "cfgloop.h" +#include "cfgrtl.h" +#include "sel-sched.h" +#include "fold-const.h" +#include "gimple-iterator.h" +#include "gimple-expr.h" +#include "tree-vectorizer.h" /* This file should be included last. */ #include "target-def.h" @@ -275,6 +284,9 @@ poly_uint16 riscv_vector_chunks; /* The number of bytes in a vector chunk. */ unsigned riscv_bytes_per_vector_chunk; +/* Prefer vf for auto-vectorizer. */ +unsigned riscv_vectorization_factor; + /* Index R is the smallest register class that contains register R. */ const enum reg_class riscv_regno_to_class[FIRST_PSEUDO_REGISTER] = { GR_REGS, GR_REGS, GR_REGS, GR_REGS, @@ -6199,6 +6211,10 @@ riscv_option_override (void) /* Convert -march to a chunks count. */ riscv_vector_chunks = riscv_convert_vector_bits (); + + if (TARGET_VECTOR) + riscv_vectorization_factor = riscv_vector_lmul; + } /* Implement TARGET_CONDITIONAL_REGISTER_USAGE. */ @@ -6893,6 +6909,128 @@ riscv_dwarf_poly_indeterminate_value (unsigned int i, unsigned int *factor, return RISCV_DWARF_VLENB; } +/* Implement TARGET_ESTIMATED_POLY_VALUE. + Look into the tuning structure for an estimate. + KIND specifies the type of requested estimate: min, max or likely. + For cores with a known RVV width all three estimates are the same. + For generic RVV tuning we want to distinguish the maximum estimate from + the minimum and likely ones. + The likely estimate is the same as the minimum in that case to give a + conservative behavior of auto-vectorizing with RVV when it is a win + even for 128-bit RVV. + When RVV width information is available VAL.coeffs[1] is multiplied by + the number of VQ chunks over the initial Advanced SIMD 128 bits. */ + +static HOST_WIDE_INT +riscv_estimated_poly_value (poly_int64 val, + poly_value_estimate_kind kind = POLY_VALUE_LIKELY) +{ + unsigned int width_source = BITS_PER_RISCV_VECTOR.is_constant () + ? (unsigned int) BITS_PER_RISCV_VECTOR.to_constant () + : (unsigned int) RVV_SCALABLE; + + /* If there is no core-specific information then the minimum and likely + values are based on 128-bit vectors and the maximum is based on + the architectural maximum of 2048 bits. */ + if (width_source == RVV_SCALABLE) + switch (kind) + { + case POLY_VALUE_MIN: + case POLY_VALUE_LIKELY: + return val.coeffs[0]; + + case POLY_VALUE_MAX: + return val.coeffs[0] + val.coeffs[1] * 15; + } + + /* Allow BITS_PER_RISCV_VECTOR to be a bitmask of different VL, treating the + lowest as likely. This could be made more general if future -mtune + options need it to be. */ + if (kind == POLY_VALUE_MAX) + width_source = 1 << floor_log2 (width_source); + else + width_source = least_bit_hwi (width_source); + + /* If the core provides width information, use that. */ + HOST_WIDE_INT over_128 = width_source - 128; + return val.coeffs[0] + val.coeffs[1] * over_128 / 128; +} + +/* Implement TARGET_VECTORIZE_PREFERRED_SIMD_MODE. */ + +static machine_mode +riscv_preferred_simd_mode (scalar_mode mode) +{ + machine_mode vmode = + riscv_vector::riscv_vector_preferred_simd_mode (mode, + riscv_vectorization_factor); + if (VECTOR_MODE_P (vmode)) + return vmode; + + return word_mode; +} + +/* Implement TARGET_AUTOVECTORIZE_VECTOR_MODES for RVV. */ +static unsigned int +riscv_autovectorize_vector_modes (vector_modes *modes, bool) +{ + if (!TARGET_VECTOR) + return 0; + + if (riscv_vectorization_factor == RVV_LMUL1) + { + modes->safe_push (VNx16QImode); + modes->safe_push (VNx8QImode); + modes->safe_push (VNx4QImode); + modes->safe_push (VNx2QImode); + } + else if (riscv_vectorization_factor == RVV_LMUL2) + { + modes->safe_push (VNx32QImode); + modes->safe_push (VNx16QImode); + modes->safe_push (VNx8QImode); + modes->safe_push (VNx4QImode); + } + else if (riscv_vectorization_factor == RVV_LMUL4) + { + modes->safe_push (VNx64QImode); + modes->safe_push (VNx32QImode); + modes->safe_push (VNx16QImode); + modes->safe_push (VNx8QImode); + } + else + { + modes->safe_push (VNx64QImode); + modes->safe_push (VNx32QImode); + modes->safe_push (VNx16QImode); + } + + return 0; +} + +/* Implement TARGET_VECTORIZE_GET_MASK_MODE. */ + +static opt_machine_mode +riscv_get_mask_mode (machine_mode mode) +{ + machine_mode mask_mode = VOIDmode; + if (TARGET_VECTOR + && riscv_vector::riscv_vector_get_mask_mode (mode).exists (&mask_mode)) + return mask_mode; + + return default_get_mask_mode (mode); +} + +/* Implement TARGET_VECTORIZE_EMPTY_MASK_IS_EXPENSIVE. Assume for now that + it isn't worth branching around empty masked ops (including masked + stores). */ + +static bool +riscv_empty_mask_is_expensive (unsigned) +{ + return false; +} + /* Return true if a shift-amount matches the trailing cleared bits on a bitmask. */ @@ -7144,6 +7282,24 @@ riscv_shamt_matches_mask_p (int shamt, HOST_WIDE_INT mask) #undef TARGET_VERIFY_TYPE_CONTEXT #define TARGET_VERIFY_TYPE_CONTEXT riscv_verify_type_context +#undef TARGET_ESTIMATED_POLY_VALUE +#define TARGET_ESTIMATED_POLY_VALUE riscv_estimated_poly_value + +#undef TARGET_VECTORIZE_PREFERRED_SIMD_MODE +#define TARGET_VECTORIZE_PREFERRED_SIMD_MODE riscv_preferred_simd_mode + +#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES +#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES riscv_autovectorize_vector_modes + +#undef TARGET_VECTORIZE_GET_MASK_MODE +#define TARGET_VECTORIZE_GET_MASK_MODE riscv_get_mask_mode + +#undef TARGET_VECTORIZE_EMPTY_MASK_IS_EXPENSIVE +#define TARGET_VECTORIZE_EMPTY_MASK_IS_EXPENSIVE riscv_empty_mask_is_expensive + +#undef TARGET_VECTORIZE_LOOP_LEN_OVERRIDE_MASK +#define TARGET_VECTORIZE_LOOP_LEN_OVERRIDE_MASK riscv_loop_len_override_mask + #undef TARGET_VECTOR_ALIGNMENT #define TARGET_VECTOR_ALIGNMENT riscv_vector_alignment -- 2.34.1