From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by sourceware.org (Postfix) with ESMTP id 0ADD1386486B for ; Tue, 27 Jul 2021 08:11:46 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 0ADD1386486B Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-542-GFKAgUrmNvGmQvZjPkWSXg-1; Tue, 27 Jul 2021 04:11:43 -0400 X-MC-Unique: GFKAgUrmNvGmQvZjPkWSXg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id AD955100E322; Tue, 27 Jul 2021 08:11:42 +0000 (UTC) Received: from tucnak.zalov.cz (ovpn-112-143.ams2.redhat.com [10.36.112.143]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 4C06E60BD9; Tue, 27 Jul 2021 08:11:42 +0000 (UTC) Received: from tucnak.zalov.cz (localhost [127.0.0.1]) by tucnak.zalov.cz (8.16.1/8.16.1) with ESMTPS id 16R8BdUm4161624 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Tue, 27 Jul 2021 10:11:39 +0200 Received: (from jakub@localhost) by tucnak.zalov.cz (8.16.1/8.16.1/Submit) id 16R8BcCW4161623; Tue, 27 Jul 2021 10:11:38 +0200 Date: Tue, 27 Jul 2021 10:11:38 +0200 From: Jakub Jelinek To: Uros Bizjak , Hongtao Liu Cc: gcc-patches@gcc.gnu.org Subject: [PATCH] i386: Improve AVX2 expansion of vector >> vector DImode arithm. shifts [PR101611] Message-ID: <20210727081138.GG2380545@tucnak> Reply-To: Jakub Jelinek MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Spam-Status: No, score=-6.0 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, KAM_SHORT, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jul 2021 08:11:48 -0000 Hi! AVX2 introduced vector >> vector shifts, but unfortunately for V{2,4}DImode it only supports logical and not arithmetic shifts, only AVX512F for V8DImode or AVX512VL for V{2,4}DImode fixed that omission. Earlier in GCC12 cycle I've committed vector >> scalar arithmetic shift emulation using various sequences, this patch handles the vector >> vector case. No need to adjust costs, the previous cost adjustment actually covers even the vector by vector shifts. The patch emits the right arithmetic V{2,4}DImode shifts using 2 logical right V{2,4}DImode shifts (once of the original operands, once of sign mask constant by the vector shift count), xor and subtraction, on each element (long long) x >> y is done as (((unsigned long long) x >> y) ^ (0x8000000000000000ULL >> y)) - (0x8000000000000000ULL >> y) i.e. if x doesn't have in some element the MSB set, it is just the logical shift, if it does, then the xor and subtraction cause also all higher bits to be set. Bootstrapped/regtested on x86_64-linux and i686-linux, ok for trunk? 2021-07-27 Jakub Jelinek PR target/101611 * config/i386/sse.md (vashr3): Split into vashrv8di3 expander and vashrv4di3 expander, where the latter requires just TARGET_AVX2 and has special !TARGET_AVX512VL expansion. (vashrv2di3): Rename to ... (vashrv2di3): ... this. Change condition to TARGET_XOP || TARGET_AVX2 and add special !TARGET_XOP && !TARGET_AVX512VL expansion. * gcc.target/i386/avx2-pr101611-1.c: New test. * gcc.target/i386/avx2-pr101611-2.c: New test. --- gcc/config/i386/sse.md.jj 2021-07-22 12:37:20.439532859 +0200 +++ gcc/config/i386/sse.md 2021-07-24 18:03:07.328126900 +0200 @@ -20499,13 +20499,34 @@ (define_expand "vlshr3" (match_operand:VI48_256 2 "nonimmediate_operand")))] "TARGET_AVX2") -(define_expand "vashr3" - [(set (match_operand:VI8_256_512 0 "register_operand") - (ashiftrt:VI8_256_512 - (match_operand:VI8_256_512 1 "register_operand") - (match_operand:VI8_256_512 2 "nonimmediate_operand")))] +(define_expand "vashrv8di3" + [(set (match_operand:V8DI 0 "register_operand") + (ashiftrt:V8DI + (match_operand:V8DI 1 "register_operand") + (match_operand:V8DI 2 "nonimmediate_operand")))] "TARGET_AVX512F") +(define_expand "vashrv4di3" + [(set (match_operand:V4DI 0 "register_operand") + (ashiftrt:V4DI + (match_operand:V4DI 1 "register_operand") + (match_operand:V4DI 2 "nonimmediate_operand")))] + "TARGET_AVX2" +{ + if (!TARGET_AVX512VL) + { + rtx mask = ix86_build_signbit_mask (V4DImode, 1, 0); + rtx t1 = gen_reg_rtx (V4DImode); + rtx t2 = gen_reg_rtx (V4DImode); + rtx t3 = gen_reg_rtx (V4DImode); + emit_insn (gen_vlshrv4di3 (t1, operands[1], operands[2])); + emit_insn (gen_vlshrv4di3 (t2, mask, operands[2])); + emit_insn (gen_xorv4di3 (t3, t1, t2)); + emit_insn (gen_subv4di3 (operands[0], t3, t2)); + DONE; + } +}) + (define_expand "vashr3" [(set (match_operand:VI12_128 0 "register_operand") (ashiftrt:VI12_128 @@ -20527,12 +20548,12 @@ (define_expand "vashr3" } }) -(define_expand "vashrv2di3" +(define_expand "vashrv2di3" [(set (match_operand:V2DI 0 "register_operand") (ashiftrt:V2DI (match_operand:V2DI 1 "register_operand") (match_operand:V2DI 2 "nonimmediate_operand")))] - "TARGET_XOP || TARGET_AVX512VL" + "TARGET_XOP || TARGET_AVX2" { if (TARGET_XOP) { @@ -20541,6 +20562,18 @@ (define_expand "vashrv2di3" emit_insn (gen_xop_shav2di3 (operands[0], operands[1], neg)); DONE; } + if (!TARGET_AVX512VL) + { + rtx mask = ix86_build_signbit_mask (V2DImode, 1, 0); + rtx t1 = gen_reg_rtx (V2DImode); + rtx t2 = gen_reg_rtx (V2DImode); + rtx t3 = gen_reg_rtx (V2DImode); + emit_insn (gen_vlshrv2di3 (t1, operands[1], operands[2])); + emit_insn (gen_vlshrv2di3 (t2, mask, operands[2])); + emit_insn (gen_xorv2di3 (t3, t1, t2)); + emit_insn (gen_subv2di3 (operands[0], t3, t2)); + DONE; + } }) (define_expand "vashrv4si3" --- gcc/testsuite/gcc.target/i386/avx2-pr101611-1.c.jj 2021-07-26 14:22:35.341226231 +0200 +++ gcc/testsuite/gcc.target/i386/avx2-pr101611-1.c 2021-07-26 14:21:29.806083664 +0200 @@ -0,0 +1,12 @@ +/* PR target/101611 */ +/* { dg-do compile } */ +/* { dg-options "-O2 -mavx2 -mno-avx512f" } */ +/* { dg-final { scan-assembler-times {\mvpsrlvq\M} 4 } } */ +/* { dg-final { scan-assembler-times {\mvpxor\M} 2 } } */ +/* { dg-final { scan-assembler-times {\mvpsubq\M} 2 } } */ + +typedef long long V __attribute__((vector_size(32))); +typedef long long W __attribute__((vector_size(16))); + +V foo (V a, V b) { return a >> b; } +W bar (W a, W b) { return a >> b; } --- gcc/testsuite/gcc.target/i386/avx2-pr101611-2.c.jj 2021-07-26 14:22:39.962165772 +0200 +++ gcc/testsuite/gcc.target/i386/avx2-pr101611-2.c 2021-07-26 14:38:38.904497928 +0200 @@ -0,0 +1,41 @@ +/* PR target/101611 */ +/* { dg-do run } */ +/* { dg-options "-O2 -mavx2 -mno-avx512f" } */ +/* { dg-require-effective-target avx2 } */ + +#include "avx2-check.h" + +typedef long long V __attribute__((vector_size(32))); +typedef long long W __attribute__((vector_size(16))); + +__attribute__((noipa)) V +foo (V a, V b) +{ + return a >> b; +} + +__attribute__((noipa)) W +bar (W a, W b) +{ + return a >> b; +} + +static void +avx2_test (void) +{ + V a = { 0x7f123456789abcdeLL, -0x30edcba987654322LL, + -0x30edcba987654322LL, 0x7f123456789abcdeLL }; + V b = { 17, 11, 23, 0 }; + V c = foo (a, b); + if (c[0] != 0x3f891a2b3c4dLL + || c[1] != -0x61db97530eca9LL + || c[2] != -0x61db97530fLL + || c[3] != 0x7f123456789abcdeLL) + abort (); + W d = { 0x7f123456789abcdeLL, -0x30edcba987654322LL }; + W e = { 45, 27 }; + W f = bar (d, e); + if (f[0] != 0x3f891LL + || f[1] != -0x61db97531LL) + abort (); +} Jakub