From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 83998 invoked by alias); 6 Sep 2019 16:53:34 -0000 Mailing-List: contact libc-stable-help@sourceware.org; run by ezmlm Precedence: bulk List-Post: List-Help: List-Subscribe: List-Archive: Sender: libc-stable-owner@sourceware.org Received: (qmail 83974 invoked by uid 89); 6 Sep 2019 16:53:34 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Checked: by ClamAV 0.100.3 on sourceware.org X-Virus-Found: No X-Spam-SWARE-Status: No, score=-19.8 required=5.0 tests=AWL,BAYES_00,GIT_PATCH_0,GIT_PATCH_1,GIT_PATCH_2,GIT_PATCH_3,RCVD_IN_DNSWL_NONE,SPF_HELO_PASS,SPF_PASS autolearn=ham version=3.3.1 spammy= X-Spam-Status: No, score=-19.8 required=5.0 tests=AWL,BAYES_00,GIT_PATCH_0,GIT_PATCH_1,GIT_PATCH_2,GIT_PATCH_3,RCVD_IN_DNSWL_NONE,SPF_HELO_PASS,SPF_PASS autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on sourceware.org X-Spam-Level: X-HELO: EUR03-DB5-obe.outbound.protection.outlook.com Received: from mail-eopbgr40088.outbound.protection.outlook.com (HELO EUR03-DB5-obe.outbound.protection.outlook.com) (40.107.4.88) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Fri, 06 Sep 2019 16:53:32 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LUvkXr5QT82KRYPhtj4wacq/W9bSV51wWxwiiQ3rZg0=; b=jFdWyVHXdkzTyhWgtUtpaNBjuAlRRjBHJQCR64OErzhaJdsippyghXiV4noh1Q2ZC/hHJPea8/j5P2y4khMZKAi/XPryzx7itg3qonzhnlhnVQHfWOmDkSNhJMNPDfUJi3pvhxNEVVGVijnpS8dJt5FG92cvJWUOEEeDEeujAZ0= Received: from VI1PR0802CA0006.eurprd08.prod.outlook.com (2603:10a6:800:aa::16) by AM6PR08MB5030.eurprd08.prod.outlook.com (2603:10a6:20b:ed::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2199.21; Fri, 6 Sep 2019 16:53:28 +0000 Received: from AM5EUR03FT030.eop-EUR03.prod.protection.outlook.com (2a01:111:f400:7e08::203) by VI1PR0802CA0006.outlook.office365.com (2603:10a6:800:aa::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2241.14 via Frontend Transport; Fri, 6 Sep 2019 16:53:27 +0000 Authentication-Results: spf=temperror (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; sourceware.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;sourceware.org; dmarc=temperror action=none header.from=arm.com; Received-SPF: TempError (protection.outlook.com: error in processing during lookup of arm.com: DNS Timeout) Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT030.mail.protection.outlook.com (10.152.16.117) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2241.14 via Frontend Transport; Fri, 6 Sep 2019 16:53:25 +0000 Received: ("Tessian outbound eec90fc31dfb:v27"); Fri, 06 Sep 2019 16:53:25 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: eddeaa06884d22cd X-CR-MTA-TID: 64aa7808 Received: from 86ae1b5d71d1.1 (cr-mta-lb-1.cr-mta-net [104.47.12.52]) by 64aa7808-outbound-1.mta.getcheckrecipient.com id 082A588D-CA93-43DE-BA8A-CB9AC3D467E5.1; Fri, 06 Sep 2019 16:53:20 +0000 Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-db3eur04lp2052.outbound.protection.outlook.com [104.47.12.52]) by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 86ae1b5d71d1.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 06 Sep 2019 16:53:20 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bzPBFuIDlFhbtgMabDiyRcQ+Qw3MDmtuANkedLqvzttMxkhJkAEKqUKls5EpdPLA2pXz4bqdOPi2MOxVszBUvcZt/yV6nDz3ce9rolqizD+ScTbuTHuwSD5lpHdENmK/fdbUvjo03JACEIsCRQQWl1C+N1a7dLLRK00bLOE2GYrnHDsactmrzN8Mpen0YQbncthR+uLqQKQsPAaNrtaKMY09z5nlv/AqFw4WdADrNRTO8AC6Wf4hwKFpsDPgklsTy6LcYNR49ti8x9C6ktmXQjxH8RWdUp7Gopqs7B2Ea7v/ulSBp1+NNysBvZey7HZSCF1xzzP7norUgBJchnWC6g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LUvkXr5QT82KRYPhtj4wacq/W9bSV51wWxwiiQ3rZg0=; b=OvKIGuaPFnxkdI64O4k+EGfV/UHg1QA8lwygWMKbomnYYyKTbkXsWuc3jRCOjKeCt4sNHEV5UOWuOUi0jUJoWxsZpm9CwYkE9n4PtlQRd7b4W2wgvubYn9Cmour0gCnIpqFuYrV9fweD6tnNI1O1hqNCnvFeO47u+NBsWUouBdBS9DmwBXPE2ZfCN1l/F+L2NsaIkRbVUg9wLooSvyrAJcUexawwLsKV4Be3+STw7bMIzqi9c+2hwja/mA0Xawrx+bEDYtGH53D3xSRfV3jU2qh/eVrgPAcM1fPJuogtMIPmAD9ONc/EtORs2ZiqjpJaqqIYjuHQQ4xa7bXSnYQcmQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LUvkXr5QT82KRYPhtj4wacq/W9bSV51wWxwiiQ3rZg0=; b=jFdWyVHXdkzTyhWgtUtpaNBjuAlRRjBHJQCR64OErzhaJdsippyghXiV4noh1Q2ZC/hHJPea8/j5P2y4khMZKAi/XPryzx7itg3qonzhnlhnVQHfWOmDkSNhJMNPDfUJi3pvhxNEVVGVijnpS8dJt5FG92cvJWUOEEeDEeujAZ0= Received: from VI1PR0801MB2127.eurprd08.prod.outlook.com (10.168.62.22) by VI1PR0801MB2109.eurprd08.prod.outlook.com (10.173.74.12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2220.19; Fri, 6 Sep 2019 16:53:19 +0000 Received: from VI1PR0801MB2127.eurprd08.prod.outlook.com ([fe80::7c75:98da:fbc1:da02]) by VI1PR0801MB2127.eurprd08.prod.outlook.com ([fe80::7c75:98da:fbc1:da02%11]) with mapi id 15.20.2241.018; Fri, 6 Sep 2019 16:53:19 +0000 From: Wilco Dijkstra To: "libc-stable@sourceware.org" CC: nd Subject: [2.26 COMMITTED][AArch64] Backport strcmp improvements Thread-Topic: [2.26 COMMITTED][AArch64] Backport strcmp improvements Thread-Index: AQHVZNNfLcsH4iFnVk22oz2VMYm94A== Date: Tue, 01 Jan 2019 00:00:00 -0000 Message-ID: Accept-Language: en-GB, en-US Content-Language: en-GB X-MS-Has-Attach: X-MS-TNEF-Correlator: Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Wilco.Dijkstra@arm.com; x-originating-ip: [217.140.106.54] x-ms-publictraffictype: Email X-MS-Office365-Filtering-Correlation-Id: e555a293-9f68-4520-6f78-08d732eabc30 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam-Untrusted: BCL:0;PCL:0;RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600166)(711020)(4605104)(1401327)(4618075)(2017052603328)(7193020);SRVR:VI1PR0801MB2109; X-MS-TrafficTypeDiagnostic: VI1PR0801MB2109:|AM6PR08MB5030: X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true x-ms-oob-tlc-oobclassifiers: OLM:4502;OLM:4502; x-forefront-prvs: 0152EBA40F X-Forefront-Antispam-Report-Untrusted: SFV:NSPM;SFS:(10009020)(4636009)(376002)(366004)(136003)(39860400002)(396003)(346002)(54534003)(189003)(199004)(186003)(26005)(316002)(7696005)(102836004)(6506007)(74316002)(305945005)(7736002)(2501003)(8676002)(4326008)(99286004)(81156014)(25786009)(81166006)(8936002)(5640700003)(256004)(14444005)(55016002)(66066001)(6436002)(71190400001)(71200400001)(486006)(476003)(9686003)(53936002)(14454004)(6916009)(33656002)(6116002)(3846002)(2906002)(86362001)(66556008)(66476007)(66946007)(2351001)(478600001)(66446008)(76116006)(52536014)(5660300002)(64756008)(40753002)(357404004);DIR:OUT;SFP:1101;SCL:1;SRVR:VI1PR0801MB2109;H:VI1PR0801MB2127.eurprd08.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Message-Info-Original: +jbsiTJr1T/5Y3I/KvUVSDPU7mcGxB6PSc4wVVwjoQLif5eApOrt/h90jRbWY9yha2W0ogvnEmTq5dXyyDtFPWxkPUX7ccaMzUbhYyej7FCW8tCH403oKgmBZ40qhesfZ2xsI+hdEPbizqmcuQK42uGEXaARAx23O7kwWDG1R8AlrKDNWSxdEUliz+lw5J519rRNPEwwqtHoWbQWUSH1hCXKybtFxAd8yIycU+CFA0eC8pcmqS1+VPtKcHPbkfXjxkbWeo7SoNrYs7Lhkoki0NRuwDKhYzwVJuNaZGfRiqn0LE51srIkVrd0bVcm0I86zSLqnYxxmIgf0zHOfdf/KMHxM8/SckyqIbUBQKHDZf/QhaoOthv7jYlG9FqO8FhSzyYcX/46PXZDgq+iKlH05tcVVPB8/EEuzMJxUWXJw6o= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0801MB2109 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Wilco.Dijkstra@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT030.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123;IPV:CAL;SCL:-1;CTRY:IE;EFV:NLI;SFV:NSPM;SFS:(10009020)(4636009)(376002)(346002)(39860400002)(396003)(136003)(2980300002)(54534003)(189003)(199004)(186003)(22756006)(2501003)(6116002)(86362001)(3846002)(70206006)(356004)(9686003)(6916009)(6506007)(55016002)(2351001)(23756003)(5660300002)(14444005)(5640700003)(63370400001)(63350400001)(52536014)(7696005)(316002)(4326008)(486006)(7736002)(50466002)(8676002)(478600001)(476003)(8936002)(25786009)(126002)(76130400001)(26826003)(99286004)(66066001)(26005)(102836004)(81166006)(81156014)(33656002)(14454004)(305945005)(2906002)(8746002)(47776003)(74316002)(336012)(70586007)(36906005)(40753002)(357404004);DIR:OUT;SFP:1101;SCL:1;SRVR:AM6PR08MB5030;H:64aa7808-outbound-1.mta.getcheckrecipient.com;FPR:;SPF:TempError;LANG:en;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;MX:1;A:1; X-MS-Office365-Filtering-Correlation-Id-Prvs: c5e06f26-6563-4a05-3d95-08d732eab843 X-Microsoft-Antispam: BCL:0;PCL:0;RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600166)(710020)(711020)(4605104)(1401327)(4618075)(2017052603328)(7193020);SRVR:AM6PR08MB5030; NoDisclaimer: True X-Forefront-PRVS: 0152EBA40F X-Microsoft-Antispam-Message-Info: fPtDPUIwD8DXffp0A78cv8p6PgfeN/D1IvEFJseNQalUze00wzPWTgQa5SjJG1kY552OPmrSxtpRGmd/OwnWB3ydbKP2ooMu1TSVfFWiJhuVua8AeWeYX967adpzjJLowtwDEKGb8wozkGIDG2vWsMBIKMJI3/oI+9rD0biwIoMEPAb9G8u4JbWLTvGwsDClanxbBcVxGEMEFNzbKmUbJUfImkgENI2utKnYZ0omxY8TyqO0jY5zal8nbXtT1E/au4NBmbf9VUjaLqImEsiUnqUIPVvHeI1pbTBpPG8iKL9G4YnBRvRXlXIU7KuJoC7eDn0qbsF+YQxxmz6HBCS4AO/jnrefdV1RxTb99JBLoOlOT0Hj9+U5iVi+nG+dj1WVpY9GGKPClkpnpqliAO1JLd3pieMij9Ixywqv/1K4nXk= X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2019 16:53:25.6150 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e555a293-9f68-4520-6f78-08d732eabc30 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB5030 X-SW-Source: 2019-09/txt/msg00008.txt.bz2 commit 01de24dbca4374665fb2a439be39c05427c0a24a Author: Siddhesh Poyarekar Date: Thu Feb 22 23:48:13 2018 +0530 aarch64/strcmp: fix misaligned loop jump target =20=20=20=20 I accidentally set the loop jump back label as misaligned8 instead of do_misaligned. The typo is harmless but it's always nice to not have to unnecessarily execute those two instructions. =20=20=20=20 * sysdeps/aarch64/strcmp.S (do_misaligned): Jump back to do_misaligned, not misaligned8. =20=20=20=20 (cherry picked from commit 6ca24c43481e2c93a6eec362b04c3e77a35b28e3) commit 4e75091d6ce3f7ac8b1750ca6135bc37d6707caf Author: Siddhesh Poyarekar Date: Wed Dec 13 18:50:27 2017 +0530 aarch64: Improve strcmp unaligned performance =20=20=20=20 Replace the simple byte-wise compare in the misaligned case with a dword compare with page boundary checks in place. For simplicity I've chosen a 4K page boundary so that we don't have to query the actual page size on the system. =20=20=20=20 This results in up to 3x improvement in performance in the unaligned case on falkor and about 2.5x improvement on mustang as measured using bench-strcmp. =20=20=20=20 * sysdeps/aarch64/strcmp.S (misaligned8): Compare dword at a time whenever possible. =20=20=20=20 (cherry picked from commit 2bce01ebbaf8db52ba4a5635eb5744f989cdbf69) diff --git a/ChangeLog b/ChangeLog index 18a01ed..29f9e1b 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,5 +1,15 @@ 2019-09-06 Siddhesh Poyarekar =20 + * sysdeps/aarch64/strcmp.S (do_misaligned): Jump back to + do_misaligned, not misaligned8. + +2019-09-06 Siddhesh Poyarekar + + * sysdeps/aarch64/strcmp.S (misaligned8): Compare dword at a + time whenever possible. + +2019-09-06 Siddhesh Poyarekar + * sysdeps/aarch64/memcmp.S (more16): Fix loop16 branch target. =20 * sysdeps/aarch64/memcmp.S: Widen comparison to 16 bytes at a diff --git a/sysdeps/aarch64/strcmp.S b/sysdeps/aarch64/strcmp.S index e99d662..7eed82c 100644 --- a/sysdeps/aarch64/strcmp.S +++ b/sysdeps/aarch64/strcmp.S @@ -72,6 +72,7 @@ L(start_realigned): cbz syndrome, L(loop_aligned) /* End of performance-critical section -- one 64B cache line. */ =20 +L(end): #ifndef __AARCH64EB__ rev syndrome, syndrome rev data1, data1 @@ -145,12 +146,38 @@ L(mutual_align): b L(start_realigned) =20 L(misaligned8): - /* We can do better than this. */ + /* Align SRC1 to 8 bytes and then compare 8 bytes at a time, always + checking to make sure that we don't access beyond page boundary = in + SRC2. */ + tst src1, #7 + b.eq L(loop_misaligned) +L(do_misaligned): ldrb data1w, [src1], #1 ldrb data2w, [src2], #1 cmp data1w, #1 ccmp data1w, data2w, #0, cs /* NZCV =3D 0b0000. */ - b.eq L(misaligned8) + b.ne L(done) + tst src1, #7 + b.ne L(do_misaligned) + +L(loop_misaligned): + /* Test if we are within the last dword of the end of a 4K page. If + yes then jump back to the misaligned loop to copy a byte at a ti= me. */ + and tmp1, src2, #0xff8 + eor tmp1, tmp1, #0xff8 + cbz tmp1, L(do_misaligned) + ldr data1, [src1], #8 + ldr data2, [src2], #8 + + sub tmp1, data1, zeroones + orr tmp2, data1, #REP8_7f + eor diff, data1, data2 /* Non-zero if differences found. = */ + bic has_nul, tmp1, tmp2 /* Non-zero if NUL terminator. */ + orr syndrome, diff, has_nul + cbz syndrome, L(loop_misaligned) + b L(end) + +L(done): sub result, data1, data2 RET END(strcmp)