From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pf1-x42a.google.com (mail-pf1-x42a.google.com [IPv6:2607:f8b0:4864:20::42a]) by sourceware.org (Postfix) with ESMTPS id 50B57384E106 for ; Mon, 3 Jun 2024 21:54:21 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 50B57384E106 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 50B57384E106 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::42a ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1717451665; cv=none; b=EPFjWHDDYoQbAhoHqAfcvak/yXDIVTxjLrFWaPuwNMM/3e4HXdlez1XUpOhvu4RyOdmNjJi+HDYao0TH7SHqQPL0RpIgAHytZQ/vO4r7kuj7Bdt3CE3JLG6SWi7wyR51GSDadl3srimbj77doYT4IhwJ6y6I/zxOrKgJcbp0ZZ0= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1717451665; c=relaxed/simple; bh=MWLdZnCSMMJjW8APVXWPK5duUgGLMHj8PVAvqV6UdFM=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=TzDZ+DVXvO5R8ZhLz+I8N+R7Jm6/1/Z1zif51ChQPG30AH0op8LpA2tljiVgATwbsHLRK93nT4eXfwHAjtT6ReHL9y+FyKhi27IWXm7WwU0xY7uhC1zzqHSHxSfyRUYZsFviy7d75M0SVAZ3oM/QZ/1f1XNEw5392wmvK6DsTYI= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pf1-x42a.google.com with SMTP id d2e1a72fcca58-70276322ad8so444859b3a.1 for ; Mon, 03 Jun 2024 14:54:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1717451659; x=1718056459; darn=gcc.gnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vyfek9+ItpClYLke6wEejcuR0gsuO+kwPpwVObcgpjo=; b=zMFqNaeuUynNoa90kH7yXNkBy5KieKZBMxux1Z3CkfDTXHaDSVg8tZN2Ek2ZVscXJi g9CZG9UwCE1Awkor0/EqzAGgJl4uHUojkNfhAFX8myE6/Ou8H2NTpePnsVjAwpRpznbY lln80FeAPo+WdYrsZWcZTRM8C5ClLVsibvcw2m+g2GzvY9bfrZ8gpG5CyiACq6oSFeNw a+u16a69Vd76N6LHNwGDhKtzHgtzD/xrsb2iumyfXxMX1XHYz4oZuzvz8q6jIgrl278R VSF7OqyLerI1AX6odoZ3XUVms9HLmhUlQb1SiCq6hk/Iwn/RasK0aG9iBxHARrU/rLu4 zyrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717451660; x=1718056460; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vyfek9+ItpClYLke6wEejcuR0gsuO+kwPpwVObcgpjo=; b=oVAHb3PxQiapzLDQt6TaF5Ozy5qkijgpGMGqMWNZ6UQgcjGD4w8gJaKtRzsbMVvd1p hAesJL8A1w10AkNQEhpQmCizBL+Ooj2YZxXnQm97X/+ahrriylsk6Zx5lOvDPN8MYshR YZoJiM/rSs1sw452oK4oZuHdHAiyoiULZnwsYncDplWyuh4FZmO0Crv9e5KSXeRxd83+ r0kh4Q5GuQKe9UZ/M/sscI/5qXvARk6/AlFftFH453eKMA2lNyBdYXqk9eDT3fUIyH7i zrkuSafuI8jaBLqVhlPBn4qTQ+DpLHBFMVCuq0Q7Oc9tpRKfa0Xgr6TbZQrKhTsOSHM4 tQ9g== X-Gm-Message-State: AOJu0YzMGMNbMcorLftipsk1Cf/QwfZCCprkv+0RBt4EKyNH7ohUTmHO SDoxK3OeXyFp4GbA4fmWq2+ZDwZGFU6Rg7Z5K94NNeVak0+Wgi7cKp4Ogotv49zH7fl1e4acvq3 3 X-Google-Smtp-Source: AGHT+IHJgvJ4YxeRJviAU/gUWsSXG3BkewgoVQbOYgCzzK/jiK1o+Il4g5noWj4NuxFdJO7gpaW7oA== X-Received: by 2002:a05:6a00:18a8:b0:701:e698:7c12 with SMTP id d2e1a72fcca58-702477bb691mr11796464b3a.5.1717451659361; Mon, 03 Jun 2024 14:54:19 -0700 (PDT) Received: from patrick-ThinkPad-X1-Carbon-Gen-8.hq.rivosinc.com ([50.145.13.30]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70242ae89c6sm5958888b3a.100.2024.06.03.14.54.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Jun 2024 14:54:18 -0700 (PDT) From: Patrick O'Neill To: gcc-patches@gcc.gnu.org Cc: jeffreyalaw@gmail.com, palmer@dabbelt.com, gnu-toolchain@rivosinc.com, Patrick O'Neill Subject: [PATCH v2 3/3] RISC-V: Add Zalrsc amo-op patterns Date: Mon, 3 Jun 2024 14:53:29 -0700 Message-ID: <20240603215329.2664893-4-patrick@rivosinc.com> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240603215329.2664893-1-patrick@rivosinc.com> References: <20240603215329.2664893-1-patrick@rivosinc.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-11.5 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,GIT_PATCH_0,KAM_SHORT,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: All amo patterns can be represented with lrsc sequences. Add these patterns as a fallback when Zaamo is not enabled. gcc/ChangeLog: * config/riscv/sync.md (atomic_): New expand pattern. (amo_atomic_): Rename amo pattern. (atomic_fetch_): New lrsc sequence pattern. (lrsc_atomic_): New expand pattern. (amo_atomic_fetch_): Rename amo pattern. (lrsc_atomic_fetch_): New lrsc sequence pattern. (atomic_exchange): New expand pattern. (amo_atomic_exchange): Rename amo pattern. (lrsc_atomic_exchange): New lrsc sequence pattern. gcc/testsuite/ChangeLog: * gcc.target/riscv/amo-zaamo-preferred-over-zalrsc.c: New test. * gcc.target/riscv/amo-zalrsc-amo-add-1.c: New test. * gcc.target/riscv/amo-zalrsc-amo-add-2.c: New test. * gcc.target/riscv/amo-zalrsc-amo-add-3.c: New test. * gcc.target/riscv/amo-zalrsc-amo-add-4.c: New test. * gcc.target/riscv/amo-zalrsc-amo-add-5.c: New test. Signed-off-by: Patrick O'Neill ------ rv64imfdc_zalrsc has the same testsuite results as rv64imafdc after this patch is applied. --- AFAIK there isn't a way to subtract an extension similar to dg-add-options. As a result I needed to specify a -march string for amo-zaamo-preferred-over-zalrsc.c instead of using testsuite infra. --- gcc/config/riscv/sync.md | 124 +++++++++++++++++- .../riscv/amo-zaamo-preferred-over-zalrsc.c | 17 +++ .../gcc.target/riscv/amo-zalrsc-amo-add-1.c | 19 +++ .../gcc.target/riscv/amo-zalrsc-amo-add-2.c | 19 +++ .../gcc.target/riscv/amo-zalrsc-amo-add-3.c | 19 +++ .../gcc.target/riscv/amo-zalrsc-amo-add-4.c | 19 +++ .../gcc.target/riscv/amo-zalrsc-amo-add-5.c | 19 +++ 7 files changed, 231 insertions(+), 5 deletions(-) create mode 100644 gcc/testsuite/gcc.target/riscv/amo-zaamo-preferred-over-zalrsc.c create mode 100644 gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-1.c create mode 100644 gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-2.c create mode 100644 gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-3.c create mode 100644 gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-4.c create mode 100644 gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-5.c diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md index c9544176ead..4df9d0b5a5f 100644 --- a/gcc/config/riscv/sync.md +++ b/gcc/config/riscv/sync.md @@ -86,7 +86,24 @@ DONE; }) -(define_insn "atomic_" +;; AMO ops + +(define_expand "atomic_" + [(any_atomic:GPR (match_operand:GPR 0 "memory_operand") ;; mem location + (match_operand:GPR 1 "reg_or_0_operand")) ;; value for op + (match_operand:SI 2 "const_int_operand")] ;; model + "TARGET_ZAAMO || TARGET_ZALRSC" +{ + if (TARGET_ZAAMO) + emit_insn (gen_amo_atomic_ (operands[0], operands[1], + operands[2])); + else + emit_insn (gen_lrsc_atomic_ (operands[0], operands[1], + operands[2])); + DONE; +}) + +(define_insn "amo_atomic_" [(set (match_operand:GPR 0 "memory_operand" "+A") (unspec_volatile:GPR [(any_atomic:GPR (match_dup 0) @@ -98,7 +115,44 @@ [(set_attr "type" "atomic") (set (attr "length") (const_int 4))]) -(define_insn "atomic_fetch_" +(define_insn "lrsc_atomic_" + [(set (match_operand:GPR 0 "memory_operand" "+A") + (unspec_volatile:GPR + [(any_atomic:GPR (match_dup 0) + (match_operand:GPR 1 "reg_or_0_operand" "rJ")) + (match_operand:SI 2 "const_int_operand")] ;; model + UNSPEC_SYNC_OLD_OP)) + (clobber (match_scratch:GPR 3 "=&r"))] ;; tmp_1 + "!TARGET_ZAAMO && TARGET_ZALRSC" + { + return "1:\;" + "lr.%I2\t%3, %0\;" + "\t%3, %3, %1\;" + "sc.%J2\t%3, %3, %0\;" + "bnez\t%3, 1b"; + } + [(set_attr "type" "atomic") + (set (attr "length") (const_int 16))]) + +;; AMO fetch ops + +(define_expand "atomic_fetch_" + [(match_operand:GPR 0 "register_operand") ;; old value at mem + (any_atomic:GPR (match_operand:GPR 1 "memory_operand") ;; mem location + (match_operand:GPR 2 "reg_or_0_operand")) ;; value for op + (match_operand:SI 3 "const_int_operand")] ;; model + "TARGET_ZAAMO || TARGET_ZALRSC" + { + if (TARGET_ZAAMO) + emit_insn (gen_amo_atomic_fetch_ (operands[0], operands[1], + operands[2], operands[3])); + else + emit_insn (gen_lrsc_atomic_fetch_ (operands[0], operands[1], + operands[2], operands[3])); + DONE; + }) + +(define_insn "amo_atomic_fetch_" [(set (match_operand:GPR 0 "register_operand" "=&r") (match_operand:GPR 1 "memory_operand" "+A")) (set (match_dup 1) @@ -112,6 +166,27 @@ [(set_attr "type" "atomic") (set (attr "length") (const_int 4))]) +(define_insn "lrsc_atomic_fetch_" + [(set (match_operand:GPR 0 "register_operand" "=&r") + (match_operand:GPR 1 "memory_operand" "+A")) + (set (match_dup 1) + (unspec_volatile:GPR + [(any_atomic:GPR (match_dup 1) + (match_operand:GPR 2 "reg_or_0_operand" "rJ")) + (match_operand:SI 3 "const_int_operand")] ;; model + UNSPEC_SYNC_OLD_OP)) + (clobber (match_scratch:GPR 4 "=&r"))] ;; tmp_1 + "!TARGET_ZAAMO && TARGET_ZALRSC" + { + return "1:\;" + "lr.%I3\t%0, %1\;" + "\t%4, %0, %2\;" + "sc.%J3\t%4, %4, %1\;" + "bnez\t%4, 1b"; + } + [(set_attr "type" "atomic") + (set (attr "length") (const_int 20))]) + (define_insn "subword_atomic_fetch_strong_" [(set (match_operand:SI 0 "register_operand" "=&r") ;; old value at mem (match_operand:SI 1 "memory_operand" "+A")) ;; mem location @@ -248,7 +323,23 @@ DONE; }) -(define_insn "atomic_exchange" +(define_expand "atomic_exchange" + [(match_operand:GPR 0 "register_operand") ;; old value at mem + (match_operand:GPR 1 "memory_operand") ;; mem location + (match_operand:GPR 2 "register_operand") ;; value for op + (match_operand:SI 3 "const_int_operand")] ;; model + "TARGET_ZAAMO || TARGET_ZALRSC" + { + if (TARGET_ZAAMO) + emit_insn (gen_amo_atomic_exchange (operands[0], operands[1], + operands[2], operands[3])); + else + emit_insn (gen_lrsc_atomic_exchange (operands[0], operands[1], + operands[2], operands[3])); + DONE; + }) + +(define_insn "amo_atomic_exchange" [(set (match_operand:GPR 0 "register_operand" "=&r") (unspec_volatile:GPR [(match_operand:GPR 1 "memory_operand" "+A") @@ -261,6 +352,26 @@ [(set_attr "type" "atomic") (set (attr "length") (const_int 4))]) +(define_insn "lrsc_atomic_exchange" + [(set (match_operand:GPR 0 "register_operand" "=&r") + (unspec_volatile:GPR + [(match_operand:GPR 1 "memory_operand" "+A") + (match_operand:SI 3 "const_int_operand")] ;; model + UNSPEC_SYNC_EXCHANGE)) + (set (match_dup 1) + (match_operand:GPR 2 "register_operand" "0")) + (clobber (match_scratch:GPR 4 "=&r"))] ;; tmp_1 + "!TARGET_ZAAMO && TARGET_ZALRSC" + { + return "1:\;" + "lr.%I3\t%4, %1\;" + "sc.%J3\t%0, %0, %1\;" + "bnez\t%0, 1b\;" + "mv\t%0, %4"; + } + [(set_attr "type" "atomic") + (set (attr "length") (const_int 20))]) + (define_expand "atomic_exchange" [(match_operand:SHORT 0 "register_operand") ;; old value at mem (match_operand:SHORT 1 "memory_operand") ;; mem location @@ -516,7 +627,7 @@ [(match_operand:QI 0 "register_operand" "") ;; bool output (match_operand:QI 1 "memory_operand" "+A") ;; memory (match_operand:SI 2 "const_int_operand" "")] ;; model - "TARGET_ZALRSC" + "TARGET_ZAAMO || TARGET_ZALRSC" { /* We have no QImode atomics, so use the address LSBs to form a mask, then use an aligned SImode atomic. */ @@ -537,7 +648,10 @@ rtx shifted_set = gen_reg_rtx (SImode); riscv_lshift_subword (QImode, set, shift, &shifted_set); - emit_insn (gen_atomic_fetch_orsi (old, aligned_mem, shifted_set, model)); + if (TARGET_ZAAMO) + emit_insn (gen_amo_atomic_fetch_orsi (old, aligned_mem, shifted_set, model)); + else if (TARGET_ZALRSC) + emit_insn (gen_lrsc_atomic_fetch_orsi (old, aligned_mem, shifted_set, model)); emit_move_insn (old, gen_rtx_ASHIFTRT (SImode, old, gen_lowpart (QImode, shift))); diff --git a/gcc/testsuite/gcc.target/riscv/amo-zaamo-preferred-over-zalrsc.c b/gcc/testsuite/gcc.target/riscv/amo-zaamo-preferred-over-zalrsc.c new file mode 100644 index 00000000000..1c124c2b8b1 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/amo-zaamo-preferred-over-zalrsc.c @@ -0,0 +1,17 @@ +/* { dg-do compile } */ +/* Ensure that AMO ops are emitted when both zalrsc and zaamo are enabled. */ +/* { dg-options "-O3" } */ +/* { dg-add-options riscv_zalrsc } */ +/* { dg-add-options riscv_zaamo } */ +/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */ +/* { dg-final { check-function-bodies "**" "" } } */ + +/* +** foo: +** amoadd\.w\tzero,a1,0\(a0\) +** ret +*/ +void foo (int* bar, int* baz) +{ + __atomic_add_fetch(bar, baz, __ATOMIC_RELAXED); +} diff --git a/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-1.c b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-1.c new file mode 100644 index 00000000000..3cd6ce04830 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-1.c @@ -0,0 +1,19 @@ +/* { dg-do compile } */ +/* Verify that lrsc atomic op mappings match Table A.6's recommended mapping. */ +/* { dg-options "-O3 -march=rv64id_zalrsc" } */ +/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */ +/* { dg-final { check-function-bodies "**" "" } } */ + +/* +** foo: +** 1: +** lr.w\ta5, 0\(a0\) +** add\ta5, a5, a1 +** sc.w\ta5, a5, 0\(a0\) +** bnez\ta5, 1b +** ret +*/ +void foo (int* bar, int* baz) +{ + __atomic_add_fetch(bar, baz, __ATOMIC_RELAXED); +} diff --git a/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-2.c b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-2.c new file mode 100644 index 00000000000..d7371dac301 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-2.c @@ -0,0 +1,19 @@ +/* { dg-do compile } */ +/* Verify that lrsc atomic op mappings match Table A.6's recommended mapping. */ +/* { dg-options "-O3 -march=rv64id_zalrsc" } */ +/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */ +/* { dg-final { check-function-bodies "**" "" } } */ + +/* +** foo: +** 1: +** lr.w.aq\ta5, 0\(a0\) +** add\ta5, a5, a1 +** sc.w\ta5, a5, 0\(a0\) +** bnez\ta5, 1b +** ret +*/ +void foo (int* bar, int* baz) +{ + __atomic_add_fetch(bar, baz, __ATOMIC_ACQUIRE); +} diff --git a/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-3.c b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-3.c new file mode 100644 index 00000000000..25060b0894f --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-3.c @@ -0,0 +1,19 @@ +/* { dg-do compile } */ +/* Verify that lrsc atomic op mappings match Table A.6's recommended mapping. */ +/* { dg-options "-O3 -march=rv64id_zalrsc" } */ +/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */ +/* { dg-final { check-function-bodies "**" "" } } */ + +/* +** foo: +** 1: +** lr.w\ta5, 0\(a0\) +** add\ta5, a5, a1 +** sc.w.rl\ta5, a5, 0\(a0\) +** bnez\ta5, 1b +** ret +*/ +void foo (int* bar, int* baz) +{ + __atomic_add_fetch(bar, baz, __ATOMIC_RELEASE); +} diff --git a/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-4.c b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-4.c new file mode 100644 index 00000000000..7cf9357fdbc --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-4.c @@ -0,0 +1,19 @@ +/* { dg-do compile } */ +/* Verify that lrsc atomic op mappings match Table A.6's recommended mapping. */ +/* { dg-options "-O3 -march=rv64id_zalrsc" } */ +/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */ +/* { dg-final { check-function-bodies "**" "" } } */ + +/* +** foo: +** 1: +** lr.w.aq\ta5, 0\(a0\) +** add\ta5, a5, a1 +** sc.w.rl\ta5, a5, 0\(a0\) +** bnez\ta5, 1b +** ret +*/ +void foo (int* bar, int* baz) +{ + __atomic_add_fetch(bar, baz, __ATOMIC_ACQ_REL); +} diff --git a/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-5.c b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-5.c new file mode 100644 index 00000000000..df761892493 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-5.c @@ -0,0 +1,19 @@ +/* { dg-do compile } */ +/* Verify that lrsc atomic op mappings match Table A.6's recommended mapping. */ +/* { dg-options "-O3 -march=rv64id_zalrsc" } */ +/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */ +/* { dg-final { check-function-bodies "**" "" } } */ + +/* +** foo: +** 1: +** lr.w.aqrl\ta5, 0\(a0\) +** add\ta5, a5, a1 +** sc.w.rl\ta5, a5, 0\(a0\) +** bnez\ta5, 1b +** ret +*/ +void foo (int* bar, int* baz) +{ + __atomic_add_fetch(bar, baz, __ATOMIC_SEQ_CST); +} -- 2.34.1