From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2075.outbound.protection.outlook.com [40.107.20.75]) by sourceware.org (Postfix) with ESMTPS id BCABE3858D39 for ; Wed, 13 Sep 2023 14:54:45 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org BCABE3858D39 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lc1pISpndslPvEp4wc+lBNAOTqo8o3ZNpYeaXSsc5dw=; b=prNn1KBx0mCuHBUvJ1+8vtmPcNLlPgh72MipQ+6ybRUJeGCukmGuz5U9P97RXn3vZ7BnBSL1Txt8mtE6dOuD4Ub7L7uwkB2rtOyEeUKWrOMFXiK9LK/ZhkIAjOAjPcrKs8t+UalG7WBVrs8dXgpd1YoW7Bf50YrVscqya5H9nng= Received: from AM6P191CA0047.EURP191.PROD.OUTLOOK.COM (2603:10a6:209:7f::24) by AS2PR08MB9149.eurprd08.prod.outlook.com (2603:10a6:20b:57b::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6745.34; Wed, 13 Sep 2023 14:54:42 +0000 Received: from AM7EUR03FT005.eop-EUR03.prod.protection.outlook.com (2603:10a6:209:7f:cafe::5b) by AM6P191CA0047.outlook.office365.com (2603:10a6:209:7f::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6792.19 via Frontend Transport; Wed, 13 Sep 2023 14:54:42 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM7EUR03FT005.mail.protection.outlook.com (100.127.140.218) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6792.16 via Frontend Transport; Wed, 13 Sep 2023 14:54:42 +0000 Received: ("Tessian outbound 30c9f5e988c5:v175"); Wed, 13 Sep 2023 14:54:42 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 1e2be0ba8ff30cb2 X-CR-MTA-TID: 64aa7808 Received: from 5bf6b3768a99.2 by 64aa7808-outbound-1.mta.getcheckrecipient.com id D58948B4-910A-41ED-A9D5-A280850FF7CD.1; Wed, 13 Sep 2023 14:54:30 +0000 Received: from EUR04-HE1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 5bf6b3768a99.2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Wed, 13 Sep 2023 14:54:30 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HPxBXCMOi9zCva5wPN5CoUE15iPiR6dDzE3NhoTEElYAZ+YDiECp4KQGC2Wd1i1f+cLg/x6L3L3MqP8BbbE5oFz5+8lH6BYgLk1Kpi67LZwB0i17z8lwWa3hQGc6JPK8+bHOwa46pD+RJVpHRjrEWicJaJqygmS5P1TcmlvFMOxAph5DfET4k5eafy8iGZGEy2MjH00E+gMvw0Jhg7qf4765zWjWT9nXYa2g7qMUDHzbqSTXErqMCnZyea8Grt3anFap/bIRCDUDR8Jnh1f4agX0Fk6ha2UYEsLp403+bT4MLnYUgDSzM14cIZ9pZkqTzNRhHuuQIVB0+55E8xoARA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=lc1pISpndslPvEp4wc+lBNAOTqo8o3ZNpYeaXSsc5dw=; b=GCKbYQR4+IgMz/QeH0SGSYe7J5BwPct26rjq9RBlbUkcE2Xp+iioMJUSaw35oaSf0WInUw0WV60yMQmV52dzD3t1Yk6H/pACQNNuOA+uw03olmgR555dIGP2DD/QECeWn8olxVKDOkZF9/lsyB5cfBVo61I9OuVb7ibkeWifhga/oY3Uzby0KvD+A/FhGn1jnLlqNCdkF2SA1G06NFpDcIWAYDvlJG0UxwkkL37cMf+jUlEF2jZqwu4UKpQATY1l3C6wHFnnU+8T8deBNr7lHDiOmafBfD3uiAcsjiarLE6IZAGEAaWv/vBaRrjugLPMwjYpqncRqwHFQg6iBj217Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lc1pISpndslPvEp4wc+lBNAOTqo8o3ZNpYeaXSsc5dw=; b=prNn1KBx0mCuHBUvJ1+8vtmPcNLlPgh72MipQ+6ybRUJeGCukmGuz5U9P97RXn3vZ7BnBSL1Txt8mtE6dOuD4Ub7L7uwkB2rtOyEeUKWrOMFXiK9LK/ZhkIAjOAjPcrKs8t+UalG7WBVrs8dXgpd1YoW7Bf50YrVscqya5H9nng= Received: from PAWPR08MB8982.eurprd08.prod.outlook.com (2603:10a6:102:33f::20) by DU2PR08MB7325.eurprd08.prod.outlook.com (2603:10a6:10:2e4::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6792.19; Wed, 13 Sep 2023 14:54:28 +0000 Received: from PAWPR08MB8982.eurprd08.prod.outlook.com ([fe80::ff3d:6e95:9971:a7e]) by PAWPR08MB8982.eurprd08.prod.outlook.com ([fe80::ff3d:6e95:9971:a7e%5]) with mapi id 15.20.6745.034; Wed, 13 Sep 2023 14:54:28 +0000 From: Wilco Dijkstra To: GCC Patches CC: Richard Sandiford , Kyrylo Tkachov Subject: [PATCH] AArch64: Fix __sync_val_compare_and_swap [PR111404] Thread-Topic: [PATCH] AArch64: Fix __sync_val_compare_and_swap [PR111404] Thread-Index: AQHZ5lFymK1NCxAT/0uZ1wjY9RCv4w== Date: Wed, 13 Sep 2023 14:54:28 +0000 Message-ID: Accept-Language: en-GB, en-US Content-Language: en-GB X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; x-ms-traffictypediagnostic: PAWPR08MB8982:EE_|DU2PR08MB7325:EE_|AM7EUR03FT005:EE_|AS2PR08MB9149:EE_ X-MS-Office365-Filtering-Correlation-Id: 48873f7b-5a97-418b-2c96-08dbb4695cff x-checkrecipientrouted: true nodisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: QLLj1xXJ6rF2glQQ89p9XXvyHxD0Z8mwM05aoF99FZYNbxRphwJFpPx6pZspIATcQfdB37AEFmCfErfgDgS3LVMcGSJ5/QeOjfs3hygOIHWymMMR1AigOTnUlwWEiBoq+b1nyHT8mA+os0on1+WvZrzrqD1koSF7GUztozqurapBocOn2zH78hIayZMfZX6Ya/CJKwKUBLRcvcfAQdRMT4+1QjYbVTi12h1P7oDhySxCxWm8t7gpMv+AKvvj4zk/tv7jau5xkg7zWIbcyKyenBn2e48b0/1WDkl5eHNYiskKbBkNnoGDoYDGZ+xIx/ewsLgkelIJVyYIadislWSSe+M7QJHe2/IA/9D2tcy2Z6fVxNSdrc0Glr7puYHoZbwafIwBYGReHG1o5VKXviIEOOGU6x3YC756YMeXq/tO3b68n5wleZQUwDuk2XvimPrGXzBD8OADY/DHGLRL2+b/DDmje9n8LLdHHZRHWZpbrCRjO2HZHLGNYA5w3tq6ZH7MCBzG/+GZFC4PIkJJ+lZpQk5L3uwLGze7HyCD+scfinu7tYhkW27GKDtnUEEszSvQZMbE859ypni/lDDiFHiqWSNKcSE99sWdyUP15pL9Lllk0rP7JnlUU0KyPeDiKm5r X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAWPR08MB8982.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366004)(396003)(376002)(39860400002)(136003)(346002)(1800799009)(186009)(451199024)(2906002)(52536014)(86362001)(5660300002)(122000001)(33656002)(38070700005)(38100700002)(4326008)(8936002)(8676002)(55016003)(316002)(54906003)(41300700001)(6916009)(26005)(66446008)(66476007)(76116006)(64756008)(66556008)(66946007)(91956017)(9686003)(71200400001)(7696005)(6506007)(478600001);DIR:OUT;SFP:1101; Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2PR08MB7325 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM7EUR03FT005.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 924c72c7-5b64-4b98-911c-08dbb469543b X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: YX3ua7PF5X2TmSqE5LJMJQbvIHwNB5yYL9dw+653n4LONoZ/chC+GZQYMX9UEHcqQxM8OsPtBGae1fkF5Ux0Ga4NosJgYL3iQn4ZyVwI7kOoFxe4l73GPxapUCce6tVtPqW4goEQu1iHzrFDYCQqUtNpHMgK6rlhTqe1FDWom7Dp45JL0AgG2Vef0bwfH0mAxOtlH+NX23y5dT6we9PJv9Hp9C4Um4pZ0vbR1yfqLRWUQaEdv53FyTe467C69Eb740Hjsw3QOP01HKhhgjdA4j9UgXDXrjjJbCDkniEAwfUB49FlOQcxyKM5V9pQA42RmwGqrv45Yu9e0tq1+fCOWRxsJujFVFKh3IpPR6NNuD+V0+WBT1lPgqtvnG7HuCta3719CPfsLMfmCT46yLASuT4Crc7AgAjmwbrTZ6nLyIDyH+8+vHA/snV5qu5GvuBIstRaRoWHuAxk/nmz/uUdTaZy0LBLOulO4U/JX+Egbf+h8eXuOe8ChhM/QhD6YwPOUGKESWZ8DvmNCVn/VQYTmAzsjbmcMl/jJ6ijYAeXZhoOgVyTpHFo/sAt3w1w2yTgfDxHPSftTXE3opDe6OlVla56gg0tvPHh/pxdvdGz8kp9FXd8mUTJFN9mjh9/7ZGmpSIuDxk2TWIUTtfLmWvGculUHtcpVyK4ZgQKUib5bdUOcPV/uDNKYewXQZufkqCBNt9VPjolLXLZKo+kqz3JwTl5XLPCnU/f0eoiXTJIJTqFJt0C6GSlpCofnqnTq9AB X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(4636009)(39860400002)(346002)(376002)(396003)(136003)(82310400011)(1800799009)(186009)(451199024)(36840700001)(40470700004)(46966006)(6506007)(7696005)(356005)(40480700001)(40460700003)(36860700001)(82740400003)(86362001)(55016003)(47076005)(33656002)(81166007)(26005)(5660300002)(2906002)(336012)(41300700001)(478600001)(9686003)(4326008)(52536014)(316002)(8676002)(8936002)(70586007)(6916009)(54906003)(70206006);DIR:OUT;SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2023 14:54:42.7224 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 48873f7b-5a97-418b-2c96-08dbb4695cff X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM7EUR03FT005.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS2PR08MB9149 X-Spam-Status: No, score=-10.7 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,FORGED_SPF_HELO,GIT_PATCH_0,KAM_DMARC_NONE,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,TXREP,UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: =0A= __sync_val_compare_and_swap may be used on 128-bit types and either calls t= he=0A= outline atomic code or uses an inline loop. On AArch64 LDXP is only atomic= if=0A= the value is stored successfully using STXP, but the current implementation= s=0A= do not perform the store if the comparison fails. In this case the value r= eturned=0A= is not read atomically.=0A= =0A= Passes regress/bootstrap, OK for commit?=0A= =0A= gcc/ChangeLog/=0A= PR target/111404=0A= * config/aarch64/aarch64.cc (aarch64_split_compare_and_swap):=0A= For 128-bit store the loaded value and loop if needed.=0A= =0A= libgcc/ChangeLog/=0A= PR target/111404=0A= * config/aarch64/lse.S (__aarch64_cas16_acq_rel): Execute STLXP usi= ng=0A= either new value or loaded value.=0A= =0A= ---=0A= =0A= diff --git a/gcc/config/aarch64/aarch64.cc b/gcc/config/aarch64/aarch64.cc= =0A= index 5e8d0a0c91bc7719de2a8c5627b354cf905a4db0..c44c0b979d0cc3755c61dcf566c= fddedccebf1ea 100644=0A= --- a/gcc/config/aarch64/aarch64.cc=0A= +++ b/gcc/config/aarch64/aarch64.cc=0A= @@ -23413,11 +23413,11 @@ aarch64_split_compare_and_swap (rtx operands[])= =0A= mem =3D operands[1];=0A= oldval =3D operands[2];=0A= newval =3D operands[3];=0A= - is_weak =3D (operands[4] !=3D const0_rtx);=0A= model_rtx =3D operands[5];=0A= scratch =3D operands[7];=0A= mode =3D GET_MODE (mem);=0A= model =3D memmodel_from_int (INTVAL (model_rtx));=0A= + is_weak =3D operands[4] !=3D const0_rtx && mode !=3D TImode;=0A= =0A= /* When OLDVAL is zero and we want the strong version we can emit a tigh= ter=0A= loop:=0A= @@ -23478,6 +23478,33 @@ aarch64_split_compare_and_swap (rtx operands[])=0A= else=0A= aarch64_gen_compare_reg (NE, scratch, const0_rtx);=0A= =0A= + /* 128-bit LDAXP is not atomic unless STLXP succeeds. So for a mismatch= ,=0A= + store the returned value and loop if the STLXP fails. */=0A= + if (mode =3D=3D TImode)=0A= + {=0A= + rtx_code_label *label3 =3D gen_label_rtx ();=0A= + emit_jump_insn (gen_rtx_SET (pc_rtx, gen_rtx_LABEL_REF (Pmode, label= 3)));=0A= + emit_barrier ();=0A= +=0A= + emit_label (label2);=0A= + aarch64_emit_store_exclusive (mode, scratch, mem, rval, model_rtx);= =0A= +=0A= + if (aarch64_track_speculation)=0A= + {=0A= + /* Emit an explicit compare instruction, so that we can correctly=0A= + track the condition codes. */=0A= + rtx cc_reg =3D aarch64_gen_compare_reg (NE, scratch, const0_rtx);=0A= + x =3D gen_rtx_NE (GET_MODE (cc_reg), cc_reg, const0_rtx);=0A= + }=0A= + else=0A= + x =3D gen_rtx_NE (VOIDmode, scratch, const0_rtx);=0A= + x =3D gen_rtx_IF_THEN_ELSE (VOIDmode, x,=0A= + gen_rtx_LABEL_REF (Pmode, label1), pc_rtx);=0A= + aarch64_emit_unlikely_jump (gen_rtx_SET (pc_rtx, x));=0A= +=0A= + label2 =3D label3;=0A= + }=0A= +=0A= emit_label (label2);=0A= =0A= /* If we used a CBNZ in the exchange loop emit an explicit compare with = RVAL=0A= diff --git a/libgcc/config/aarch64/lse.S b/libgcc/config/aarch64/lse.S=0A= index dde3a28e07b13669533dfc5e8fac0a9a6ac33dbd..ba05047ff02b6fc5752235bffa9= 24fc4a2f48c04 100644=0A= --- a/libgcc/config/aarch64/lse.S=0A= +++ b/libgcc/config/aarch64/lse.S=0A= @@ -160,6 +160,8 @@ see the files COPYING3 and COPYING.RUNTIME respectively= . If not, see=0A= #define tmp0 16=0A= #define tmp1 17=0A= #define tmp2 15=0A= +#define tmp3 14=0A= +#define tmp4 13=0A= =0A= #define BTI_C hint 34=0A= =0A= @@ -233,10 +235,11 @@ STARTFN NAME(cas)=0A= 0: LDXP x0, x1, [x4]=0A= cmp x0, x(tmp0)=0A= ccmp x1, x(tmp1), #0, eq=0A= - bne 1f=0A= - STXP w(tmp2), x2, x3, [x4]=0A= - cbnz w(tmp2), 0b=0A= -1: BARRIER=0A= + csel x(tmp2), x2, x0, eq=0A= + csel x(tmp3), x3, x1, eq=0A= + STXP w(tmp4), x(tmp2), x(tmp3), [x4]=0A= + cbnz w(tmp4), 0b=0A= + BARRIER=0A= ret=0A= =0A= #endif=0A= =0A=