From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pj1-x1041.google.com (mail-pj1-x1041.google.com [IPv6:2607:f8b0:4864:20::1041]) by sourceware.org (Postfix) with ESMTPS id 6F6743857716 for ; Wed, 5 Apr 2023 21:03:07 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 6F6743857716 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivosinc.com Received: by mail-pj1-x1041.google.com with SMTP id h12-20020a17090aea8c00b0023d1311fab3so38580506pjz.1 for ; Wed, 05 Apr 2023 14:03:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; t=1680728586; x=1683320586; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=I4Iv5OutDi8fQJpm4qflbaDO8Ma4+rSn9CJxchqX1Tk=; b=vTLxFRCJLmDAEQnq/7asQwzHn0K8RG2ouQWF9wUtfSsAlmQ4H8UGz7Srm6hWjEqxcZ BsFNfgdxo+fZrQxGLGX+wKPeSuSwDx37ebTPPGfsrA3crUt3oTsZHxqhGRFK3JRtUq/W uzeOG9P5l/o6d7AgP7T5CCeL/oEV9FdGNlAK2ffA6exCoq10fBl3wikkctovOu1KXJRz puwMECN5zLcz2YjJR1ANAWJE3IdELDtVC2GIwqBDn0vyBWwf7f7I5T2K8Asj6ADCUprK LcSF5PEU0LmmuhbPrHxFpTr+SH5VIt4pJw+LrZgaMRW4Q+YrLJTluID7M0mijo6xmjpp dSrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680728586; x=1683320586; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=I4Iv5OutDi8fQJpm4qflbaDO8Ma4+rSn9CJxchqX1Tk=; b=Te8euvy4t9LuFtn7dbOInWAmdMKo2CjPnoctQdq21HVH9sBWotVmv6YzZZGc5KBCOJ NMjV5sZwoskClILPfQEtKFSfCJT/9X1A9mXrKAUw9xHFeniNnIr3gRR4AqtegS0iObUC losP9tdyj+16umU4VMla6W/g2HqBaC8BA+rCD/DKp7lzq2u6R8fYhapnFCLawbxBmsyx JrkYvr+VF4F0R7h3WZD7+Cwyw8Hdh3OK2QK6ZfrwVzQRAZyoxSHWFOWelUXO0bzY37OQ BOOy0BsiocGvTR/gWgmfOb2h61GVdakY9L0RSAjSbSvhJ15Z8DE1uoQ6FKxu1zTuyN9g VTBQ== X-Gm-Message-State: AAQBX9d50n7F5JsTzLbVF7DDshvSPjZeBtNVWx1VO4wpmn2aitZIA9c8 XDnvQjodOoauafv9Rxzt0cuA1ZYxhTwU8go5H+k5leQcRHg= X-Google-Smtp-Source: AKy350aGZ5crscnzDkZkXX33VntqMAFaipeSWxInmqpnStYAu+ro7Yg7rDhDb0lYr8zlRYdR9KE7ZQ== X-Received: by 2002:a17:90b:1c01:b0:23d:4b01:b27 with SMTP id oc1-20020a17090b1c0100b0023d4b010b27mr8440461pjb.10.1680728586132; Wed, 05 Apr 2023 14:03:06 -0700 (PDT) Received: from patrick-ThinkPad-X1-Carbon-Gen-8.hq.rivosinc.com ([50.221.140.188]) by smtp.gmail.com with ESMTPSA id e6-20020a17090a77c600b002342ccc8280sm1809477pjs.6.2023.04.05.14.03.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Apr 2023 14:03:05 -0700 (PDT) From: Patrick O'Neill To: gcc-patches@gcc.gnu.org Cc: palmer@rivosinc.com, gnu-toolchain@rivosinc.com, vineetg@rivosinc.com, andrew@sifive.com, kito.cheng@sifive.com, dlustig@nvidia.com, jeffreyalaw@gmail.com, cmuellner@gcc.gnu.org, andrea@rivosinc.com, Patrick O'Neill Subject: [PATCH v2 6/8] RISCV: Weaken compare_exchange LR/SC pairs Date: Wed, 5 Apr 2023 14:01:16 -0700 Message-Id: <20230405210118.1969283-7-patrick@rivosinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230405210118.1969283-1-patrick@rivosinc.com> References: <20220407182918.294892-1-patrick@rivosinc.com> <20230405210118.1969283-1-patrick@rivosinc.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-12.2 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,GIT_PATCH_0,KAM_SHORT,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Introduce the %I and %J flags for setting the .aqrl bits on LR/SC pairs as needed. Atomic compare and exchange ops provide success and failure memory models. C++17 and later place no restrictions on the relative strength of each model, so ensure we cover both by using a model that enforces the ordering of both given models. This change brings compare_exchange LR/SC ops in line with table A.6 of the ISA manual. 2023-04-05 Patrick O'Neill * riscv.cc: Add function to get the union of two memmodels in sync.md. * riscv-protos.h: Likewise. * sync.md (atomic_cas_value_strong): Remove static .aqrl bits on SC op/.rl bits on LR op and replace with optimized %I, %J flags. * compare-exchange-atomics-model-1.c: New test. * compare-exchange-atomics-model-2.c: Likewise. * compare-exchange-atomics-model-3.c: Likewise. * compare-exchange-atomics-model-4.c: Likewise. * compare-exchange-atomics-model-5.c: Likewise. * compare-exchange-atomics-model-6.c: Likewise. Signed-off-by: Patrick O'Neill --- gcc/config/riscv/riscv-protos.h | 3 ++ gcc/config/riscv/riscv.cc | 44 +++++++++++++++++++ gcc/config/riscv/sync.md | 9 +++- .../riscv/compare-exchange-atomics-model-1.c | 12 +++++ .../riscv/compare-exchange-atomics-model-2.c | 12 +++++ .../riscv/compare-exchange-atomics-model-3.c | 12 +++++ .../riscv/compare-exchange-atomics-model-4.c | 12 +++++ .../riscv/compare-exchange-atomics-model-5.c | 12 +++++ 8 files changed, 114 insertions(+), 2 deletions(-) create mode 100644 gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-1.c create mode 100644 gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-2.c create mode 100644 gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-3.c create mode 100644 gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-4.c create mode 100644 gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-5.c diff --git a/gcc/config/riscv/riscv-protos.h b/gcc/config/riscv/riscv-protos.h index 4611447ddde..b03edc3e8a5 100644 --- a/gcc/config/riscv/riscv-protos.h +++ b/gcc/config/riscv/riscv-protos.h @@ -22,6 +22,8 @@ along with GCC; see the file COPYING3. If not see #ifndef GCC_RISCV_PROTOS_H #define GCC_RISCV_PROTOS_H +#include "memmodel.h" + /* Symbol types we understand. The order of this list must match that of the unspec enum in riscv.md, subsequent to UNSPEC_ADDRESS_FIRST. */ enum riscv_symbol_type { @@ -79,6 +81,7 @@ extern void riscv_reinit (void); extern poly_uint64 riscv_regmode_natural_size (machine_mode); extern bool riscv_v_ext_vector_mode_p (machine_mode); extern bool riscv_shamt_matches_mask_p (int, HOST_WIDE_INT); +extern enum memmodel riscv_union_memmodels (enum memmodel, enum memmodel); /* Routines implemented in riscv-c.cc. */ void riscv_cpu_cpp_builtins (cpp_reader *); diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc index 6576e9ae524..061d2cf42b4 100644 --- a/gcc/config/riscv/riscv.cc +++ b/gcc/config/riscv/riscv.cc @@ -4278,6 +4278,36 @@ riscv_print_operand_reloc (FILE *file, rtx op, bool hi_reloc) fputc (')', file); } +/* Return the memory model that encapuslates both given models. */ + +enum memmodel +riscv_union_memmodels (enum memmodel model1, enum memmodel model2) +{ + model1 = memmodel_base (model1); + model2 = memmodel_base (model2); + + enum memmodel weaker = model1 <= model2 ? model1: model2; + enum memmodel stronger = model1 > model2 ? model1: model2; + + switch (stronger) + { + case MEMMODEL_SEQ_CST: + case MEMMODEL_ACQ_REL: + return stronger; + case MEMMODEL_RELEASE: + if (weaker == MEMMODEL_ACQUIRE || weaker == MEMMODEL_CONSUME) + return MEMMODEL_ACQ_REL; + else + return stronger; + case MEMMODEL_ACQUIRE: + case MEMMODEL_CONSUME: + case MEMMODEL_RELAXED: + return stronger; + default: + gcc_unreachable (); + } +} + /* Return true if the .AQ suffix should be added to an AMO to implement the acquire portion of memory model MODEL. */ @@ -4331,6 +4361,8 @@ riscv_memmodel_needs_amo_release (enum memmodel model) 'R' Print the low-part relocation associated with OP. 'C' Print the integer branch condition for comparison OP. 'A' Print the atomic operation suffix for memory model OP. + 'I' Print the LR suffix for memory model OP. + 'J' Print the SC suffix for memory model OP. 'z' Print x0 if OP is zero, otherwise print OP normally. 'i' Print i if the operand is not a register. 'S' Print shift-index of single-bit mask OP. @@ -4500,6 +4532,18 @@ riscv_print_operand (FILE *file, rtx op, int letter) fputs (".rl", file); break; + case 'I': + if (model == MEMMODEL_SEQ_CST) + fputs (".aqrl", file); + else if (riscv_memmodel_needs_amo_acquire (model)) + fputs (".aq", file); + break; + + case 'J': + if (riscv_memmodel_needs_amo_release (model)) + fputs (".rl", file); + break; + case 'i': if (code != REG) fputs ("i", file); diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md index 1aa9ac81cee..b1a12545a19 100644 --- a/gcc/config/riscv/sync.md +++ b/gcc/config/riscv/sync.md @@ -116,10 +116,15 @@ (clobber (match_scratch:GPR 6 "=&r"))] "TARGET_ATOMIC" { + enum memmodel model_success = (enum memmodel) INTVAL(operands[4]); + enum memmodel model_failure = (enum memmodel) INTVAL(operands[5]); + /* Find the union of the two memory models so we can satisfy both success + and failure memory models. */ + operands[5] = GEN_INT(riscv_union_memmodels(model_success, model_failure)); return "1:\;" - "lr..aqrl\t%0,%1\;" + "lr.%I5\t%0,%1\;" "bne\t%0,%z2,1f\;" - "sc..rl\t%6,%z3,%1\;" + "sc.%J5\t%6,%z3,%1\;" "bnez\t%6,1b\;" "1:"; } diff --git a/gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-1.c b/gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-1.c new file mode 100644 index 00000000000..a2c3fc7a1b6 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-1.c @@ -0,0 +1,12 @@ +/* { dg-do compile } */ +/* Verify that appropriate bits are placed per memory model. */ +/* { dg-final { scan-assembler-not "lr.w.aq" } } */ +/* { dg-final { scan-assembler-not "lr.w.rl" } } */ +/* { dg-final { scan-assembler-not "sc.w.aq" } } */ +/* { dg-final { scan-assembler-not "sc.w.rl" } } */ + +void +foo (int bar, int baz, int qux) +{ + __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_RELAXED, __ATOMIC_RELAXED); +} diff --git a/gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-2.c b/gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-2.c new file mode 100644 index 00000000000..d23d4db945f --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-2.c @@ -0,0 +1,12 @@ +/* { dg-do compile } */ +/* Verify that appropriate bits are placed per memory model. */ +/* { dg-final { scan-assembler "lr.w.aq" } } */ +/* { dg-final { scan-assembler-not "lr.w.rl" } } */ +/* { dg-final { scan-assembler-not "sc.w.aq" } } */ +/* { dg-final { scan-assembler-not "sc.w.rl" } } */ + +void +foo (int bar, int baz, int qux) +{ + __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_CONSUME, __ATOMIC_CONSUME); +} diff --git a/gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-3.c b/gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-3.c new file mode 100644 index 00000000000..7379825c6f7 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-3.c @@ -0,0 +1,12 @@ +/* { dg-do compile } */ +/* Verify that appropriate bits are placed per memory model. */ +/* { dg-final { scan-assembler "lr.w.aq" } } */ +/* { dg-final { scan-assembler-not "lr.w.rl" } } */ +/* { dg-final { scan-assembler-not "sc.w.aq" } } */ +/* { dg-final { scan-assembler-not "sc.w.rl" } } */ + +void +foo (int bar, int baz, int qux) +{ + __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_ACQUIRE, __ATOMIC_ACQUIRE); +} diff --git a/gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-4.c b/gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-4.c new file mode 100644 index 00000000000..80ab9889288 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-4.c @@ -0,0 +1,12 @@ +/* { dg-do compile } */ +/* Verify that appropriate bits are placed per memory model. */ +/* { dg-final { scan-assembler "lr.w.aqrl" } } */ +/* { dg-final { scan-assembler "sc.w.rl" } } */ +/* { dg-final { scan-assembler-not "lr.w.rl" } } */ +/* { dg-final { scan-assembler-not "sc.w.aq" } } */ + +void +foo (int bar, int baz, int qux) +{ + __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST); +} diff --git a/gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-5.c b/gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-5.c new file mode 100644 index 00000000000..da905242317 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-5.c @@ -0,0 +1,12 @@ +/* { dg-do compile } */ +/* Verify that appropriate bits are placed per memory model. */ +/* { dg-final { scan-assembler "lr.w.aq" } } */ +/* { dg-final { scan-assembler "sc.w.rl" } } */ +/* { dg-final { scan-assembler-not "lr.w.rl" } } */ +/* { dg-final { scan-assembler-not "sc.w.aq" } } */ + +void +foo (int bar, int baz, int qux) +{ + __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_RELEASE, __ATOMIC_ACQUIRE); +} -- 2.25.1