From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR03-VI1-obe.outbound.protection.outlook.com (mail-vi1eur03on2063.outbound.protection.outlook.com [40.107.103.63]) by sourceware.org (Postfix) with ESMTPS id 65ECC3851885 for ; Mon, 21 Nov 2022 16:09:18 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 65ECC3851885 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=C3Wtx8VoQK/1sWbb+WaV2w/NkG/Qlk0JSmQxl8i15ws=; b=VL9tM7Egiv2pf+AxjGntilx2UsUxQ6hU2aWqAp7p7gP/uV6aKkJMpXGgcWxXC5doNucXWS4xSdCm+vzhyHhhi5YMntcdd2LDdfalTJ7lXh7VSZ7El/W4qTwuLawCT6jSb+GTL/5YGid/K4rBN4DY5EM1LM+XiJvHDk7GChX/9/0= Received: from DBBPR09CA0018.eurprd09.prod.outlook.com (2603:10a6:10:c0::30) by PA4PR08MB6015.eurprd08.prod.outlook.com (2603:10a6:102:e6::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5857.17; Mon, 21 Nov 2022 16:09:15 +0000 Received: from DBAEUR03FT007.eop-EUR03.prod.protection.outlook.com (2603:10a6:10:c0:cafe::f1) by DBBPR09CA0018.outlook.office365.com (2603:10a6:10:c0::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.15 via Frontend Transport; Mon, 21 Nov 2022 16:09:15 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DBAEUR03FT007.mail.protection.outlook.com (100.127.142.161) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.8 via Frontend Transport; Mon, 21 Nov 2022 16:09:15 +0000 Received: ("Tessian outbound aeae1c7b66fd:v130"); Mon, 21 Nov 2022 16:09:15 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: f747540ed2334902 X-CR-MTA-TID: 64aa7808 Received: from c52d8e13112c.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id E64E3B03-179F-4E23-801A-F230CD5C6F4E.1; Mon, 21 Nov 2022 16:09:08 +0000 Received: from EUR05-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id c52d8e13112c.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Mon, 21 Nov 2022 16:09:08 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=QJzmoPOMMlx+7FJ98lGRV+ssErdJidoxacu1AVCWuHhLubN1v2WZ63zd3XjkKDnSCasR13iootnC+thrCdwQR7q95xWfR/0XI3/YhOHuJb8wnHc4+qGQJPqs+t8XUjUOpGYTVEKF4jlFlwy4MkU3dmOwkuCVJpxmL9HlSOCm0YAqT1biclYyyaF0izo3su4uNxX4xahrhgVgBd4vGeWzpknIhvGlvqJXoTMdCK54leqaBAvYROAK89yD2JDHSGSZmCwNMHOZedoH5aSAG6dbHsqqiwDk7L3Z06mNoPpN59UuQaYREhG0RsiUDYCp5EVjmTnY/wGdOCBEtcbIKfn6aw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=C3Wtx8VoQK/1sWbb+WaV2w/NkG/Qlk0JSmQxl8i15ws=; b=a4hsF47w94pyaPWfoww/8HEjj5SyrX8cO/1WoNsvuOO7nrUsRCwmvCFXSQA0B9xMkfoPiMjOnBblhLtoEGsRD+R/CArFzR/GxZu3pt/MbMEGo3Qy0NLqJYnH/pt5fggQ+amKuvtiVBmQ4K/9TLCdlhzY80PY/gBY+DnUVkohU1XIdINI/ihVUc9Y0NiUXf0Dat7f8er8Z/ohBGtZ9PWHMSmNE5xKxy15UCtmQGy+Xdet/jmBbE+L5kSTVg7IjnWsTnt+u4xbkMKxJtWHQvdz0yOt/aqgJz+V1Nx+PV5sMoRiSIu+QVIuQCLdxIEhaTMuJESowOFaTcVS1SsSNkkUFw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=C3Wtx8VoQK/1sWbb+WaV2w/NkG/Qlk0JSmQxl8i15ws=; b=VL9tM7Egiv2pf+AxjGntilx2UsUxQ6hU2aWqAp7p7gP/uV6aKkJMpXGgcWxXC5doNucXWS4xSdCm+vzhyHhhi5YMntcdd2LDdfalTJ7lXh7VSZ7El/W4qTwuLawCT6jSb+GTL/5YGid/K4rBN4DY5EM1LM+XiJvHDk7GChX/9/0= Received: from PAWPR08MB8982.eurprd08.prod.outlook.com (2603:10a6:102:33f::20) by DB9PR08MB7469.eurprd08.prod.outlook.com (2603:10a6:10:36f::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5857.17; Mon, 21 Nov 2022 16:09:06 +0000 Received: from PAWPR08MB8982.eurprd08.prod.outlook.com ([fe80::4c73:7d14:fc39:a3cb]) by PAWPR08MB8982.eurprd08.prod.outlook.com ([fe80::4c73:7d14:fc39:a3cb%3]) with mapi id 15.20.5857.017; Mon, 21 Nov 2022 16:09:06 +0000 From: Wilco Dijkstra To: 'GNU C Library' CC: Florian Weimer Subject: [PATCH] malloc: Use correct C11 atomics for fastbin Thread-Topic: [PATCH] malloc: Use correct C11 atomics for fastbin Thread-Index: AQHY/cJS/cK/f0PorUWetLVbA8LI1g== Date: Mon, 21 Nov 2022 16:09:05 +0000 Message-ID: Accept-Language: en-GB, en-US Content-Language: en-GB X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; x-ms-traffictypediagnostic: PAWPR08MB8982:EE_|DB9PR08MB7469:EE_|DBAEUR03FT007:EE_|PA4PR08MB6015:EE_ X-MS-Office365-Filtering-Correlation-Id: a96dd25f-4072-4893-251e-08dacbdabc9b x-checkrecipientrouted: true nodisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: 1+Anjpv0FkPbOnPCsgHJH9t3Ir5x7xXFC9/mIjx6Bcf0WIleV2YVHSAkjMKMzheyMIAw5zn6G++hIhGnT3iQ7NUoFI4UqPfdjuEv+v2Hoib+3DJdTk7rqnKFS7APlqAqWIiI5ZUMJzk6LVjKlmAbURkZI2sQnEBj7EZb+AUi+WaY17NHML9V4tJVmvpA2yML5AzxKwqSMURH8dH6aTLwYI4AIJfwASRJXaIPLNdTien4H8QiVmIIi9jkMbc9u4MZX8nQRH2QDo5MTkBhLftzqxkTrKTjbM8vGITLEHUuHfEfU5vIxcZJ1j6hsA8HD8pTJxf78u6CdHNMpjmgkSrGf1Oq1QinDcA2bYXKB2gAHOhQZdkFkXOPfjt/KeikJ7+XzwwedZsA7PmSwARhk4Ww6JeQ+LNCfKktsbkzPQhiOpCErkEDmDF0zR8ChjbisbKc+RSrRAwjUrYqncnA3KRmzimaTZh5i+fqSIQG4Bmh7HgTl0EtiYvo+59HgiKT+HB55J6ttzVozKv0MJ6xezQjvu4RcPQTNJ0RZKDGK9N9txtdr1+BEb5/mYPUe5Mbh90DFJ19Moj07849HTyOBLdDDnrAUbJIjn1J1+Q6WmEw/dDbrDS12AKQIvY5lo62SVvjq/INTFfgJ+eVtvUEa5SmfvYM9z3xgM4++N37plenDr4aHxsatVVsf1vf3qD3K386s1HxtvmKHksGUA90JYJzOQ== X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAWPR08MB8982.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230022)(4636009)(366004)(376002)(396003)(39860400002)(136003)(346002)(451199015)(83380400001)(52536014)(5660300002)(8936002)(41300700001)(38070700005)(6506007)(7696005)(9686003)(26005)(478600001)(71200400001)(33656002)(186003)(91956017)(66946007)(66446008)(66476007)(76116006)(66556008)(64756008)(8676002)(316002)(4326008)(6916009)(86362001)(122000001)(38100700002)(2906002)(55016003);DIR:OUT;SFP:1101; Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB7469 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DBAEUR03FT007.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: a29a224d-3649-42f4-c175-08dacbdab705 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: TPXHRKCqQqRMHoFCNg+ayzuE0q/xBdDW1RtecD5O7inmd2VWjmjEzVmHNnWVDg7cfNzkHkSc82ID5YUIM14G+TA4V0bMu76bT6zovxlAGnNgdA8g2bDlGZX4LS4UA/jiokT67zm04D+81/4Cz+cleuLvyw8wduCCm7JPJJYK//laXrCee8cDm7px66pLkJJCYd9Z6wewP+oWeAFT6aC0/dPMhQNzULrvBwIbG1vLuQBr9ymN8tbOCGNku2WvY9wZ7gp/AYFUXjX7FIjOXdidxaNTJgGLvDwJADD7jlk7stCPU/f/6josPe5HOt7ldGjAPYvC1ccd4gQQ7QjJSXGihHnz0uyiY/Nb+9at6DEoRTRj6rr3hqG+ygb9xxKei1ofzBAOmOfZsjvs7i2iY5qYbp7tyTfPdUmGUPPpG35ByR1JI0fdo7IMvJfrWgjZ4Yy1tsejpBb9KnRVbv8jbXK9an90CBRx8KQrokd4o1BCI1kb9N+ctQnjZ+97Y6M+O4f0ZSZQyfXewxVwC1HVuyTqd60wljS3q2e6C/bFKRrOyW5nzOF2Mc0EDCkbn0yHecI8J0j5pgHplaewd2NWWf1n8N2FB0YJTiKNCW4k8kF2+1Trbx4Qb1BzNa6cBfShB5Ry0Lydw4AafqBllbMLLZy2shtKBluZrcP3WZmUKqiK44sZYeIfVwncevJT39y2ctMOP0H7u2CU5onxFk3Pu+fyVw== X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230022)(4636009)(39860400002)(376002)(346002)(396003)(136003)(451199015)(46966006)(40470700004)(36840700001)(82740400003)(86362001)(36860700001)(83380400001)(47076005)(41300700001)(81166007)(2906002)(52536014)(40460700003)(8936002)(82310400005)(40480700001)(5660300002)(478600001)(107886003)(55016003)(6506007)(7696005)(26005)(336012)(4326008)(8676002)(70206006)(316002)(9686003)(186003)(356005)(6916009)(70586007)(33656002);DIR:OUT;SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Nov 2022 16:09:15.3939 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a96dd25f-4072-4893-251e-08dacbdabc9b X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DBAEUR03FT007.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB6015 X-Spam-Status: No, score=-11.2 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,FORGED_SPF_HELO,GIT_PATCH_0,KAM_DMARC_NONE,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,TXREP,UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: =0A= Fix memory ordering issues in the fastbin implementation: in REMOVE_FB the= =0A= next pointer is read before any MO synchronization, however in the C11 atom= ic=0A= model this is only correct after a load acquire. Refactor the fastbin code= =0A= and add a dedicated fastbin_push/pop implementation. The number of acquire= =0A= or release atomics remains the same, and the new functions are inlined, so= =0A= performance is unaffected.=0A= =0A= Passes regress, OK for commit?=0A= =0A= ---=0A= =0A= diff --git a/malloc/malloc.c b/malloc/malloc.c=0A= index 2a61c8b5ee38f33c42c72f3c5ad441e26ed0e701..a3e7053d2db81cce211dd8c3cff= 43af0f59a1b71 100644=0A= --- a/malloc/malloc.c=0A= +++ b/malloc/malloc.c=0A= @@ -1786,6 +1786,73 @@ get_max_fast (void)=0A= return global_max_fast;=0A= }=0A= =0A= +=0A= +/* Atomically push onto the fastbin list. Return the previous head of the= =0A= + list, however it can only be referenced if the arena lock is acquired. = */=0A= +static __always_inline mchunkptr=0A= +fastbin_push_chunk (mchunkptr *fastbin, mchunkptr p)=0A= +{=0A= + mchunkptr head =3D atomic_load_relaxed (fastbin);=0A= +=0A= + /* Check that the top of the bin is not the record we are going to=0A= + add (i.e., double free). */=0A= + if (__builtin_expect (head =3D=3D p, 0))=0A= + malloc_printerr ("double free or corruption (fasttop)");=0A= +=0A= + if (SINGLE_THREAD_P)=0A= + {=0A= + p->fd =3D PROTECT_PTR (&p->fd, head);=0A= + *fastbin =3D p;=0A= + return head;=0A= + }=0A= +=0A= + /* Atomically update the fastbin head. Use release MO to synchronize=0A= + with acquire reads in fastbin_pop_chunk and malloc_consolidate (this= =0A= + ensures the next pointer update is valid). */=0A= + do=0A= + {=0A= + p->fd =3D PROTECT_PTR (&p->fd, head);=0A= + }=0A= + while (!atomic_compare_exchange_weak_release (fastbin, &head, p));=0A= +=0A= + return head;=0A= +}=0A= +=0A= +=0A= +/* Atomically pop from the fastbin list. The arena lock must be held to= =0A= + block other threads removing entries, avoiding the ABA issue. */=0A= +static __always_inline mchunkptr=0A= +fastbin_pop_chunk (mchunkptr *fastbin)=0A= +{=0A= + mchunkptr head, tail;=0A= +=0A= + if (SINGLE_THREAD_P)=0A= + {=0A= + head =3D *fastbin;=0A= + if (head =3D=3D NULL)=0A= + return head;=0A= + if (__glibc_unlikely (misaligned_chunk (head)))=0A= + malloc_printerr ("malloc(): unaligned fastbin chunk detected");=0A= + *fastbin =3D REVEAL_PTR (head->fd);=0A= + return head;=0A= + }=0A= +=0A= + do=0A= + {=0A= + /* Synchronize with release MO CAS in fastbin_pop_chunk - this ensur= es=0A= + the next pointer is valid. */=0A= + head =3D atomic_load_acquire (fastbin);=0A= + if (head =3D=3D NULL)=0A= + return head;=0A= + if (__glibc_unlikely (head !=3D NULL && misaligned_chunk (head)))=0A= + malloc_printerr ("malloc(): unaligned fastbin chunk detected");=0A= + tail =3D REVEAL_PTR (head->fd);=0A= + }=0A= + while (!atomic_compare_exchange_weak_relaxed (fastbin, &head, tail));=0A= +=0A= + return head;=0A= +}=0A= +=0A= /*=0A= ----------- Internal state representation and initialization ----------= -=0A= */=0A= @@ -3798,71 +3865,37 @@ _int_malloc (mstate av, size_t bytes)=0A= can try it without checking, which saves some time on this fast path.= =0A= */=0A= =0A= -#define REMOVE_FB(fb, victim, pp) \=0A= - do \=0A= - { \=0A= - victim =3D pp; \=0A= - if (victim =3D=3D NULL) \=0A= - break; \=0A= - pp =3D REVEAL_PTR (victim->fd); = \=0A= - if (__glibc_unlikely (pp !=3D NULL && misaligned_chunk (pp))) = \=0A= - malloc_printerr ("malloc(): unaligned fastbin chunk detected"); \=0A= - } \=0A= - while ((pp =3D catomic_compare_and_exchange_val_acq (fb, pp, victim)) \= =0A= - !=3D victim); \=0A= -=0A= if ((unsigned long) (nb) <=3D (unsigned long) (get_max_fast ()))=0A= {=0A= idx =3D fastbin_index (nb);=0A= mfastbinptr *fb =3D &fastbin (av, idx);=0A= - mchunkptr pp;=0A= - victim =3D *fb;=0A= + victim =3D fastbin_pop_chunk (fb);=0A= =0A= if (victim !=3D NULL)=0A= {=0A= - if (__glibc_unlikely (misaligned_chunk (victim)))=0A= - malloc_printerr ("malloc(): unaligned fastbin chunk detected 2");=0A= -=0A= - if (SINGLE_THREAD_P)=0A= - *fb =3D REVEAL_PTR (victim->fd);=0A= - else=0A= - REMOVE_FB (fb, pp, victim);=0A= - if (__glibc_likely (victim !=3D NULL))=0A= - {=0A= - size_t victim_idx =3D fastbin_index (chunksize (victim));=0A= - if (__builtin_expect (victim_idx !=3D idx, 0))=0A= - malloc_printerr ("malloc(): memory corruption (fast)");=0A= - check_remalloced_chunk (av, victim, nb);=0A= + size_t victim_idx =3D fastbin_index (chunksize (victim));=0A= + if (__builtin_expect (victim_idx !=3D idx, 0))=0A= + malloc_printerr ("malloc(): memory corruption (fast)");=0A= + check_remalloced_chunk (av, victim, nb);=0A= #if USE_TCACHE=0A= - /* While we're here, if we see other chunks of the same size,=0A= - stash them in the tcache. */=0A= - size_t tc_idx =3D csize2tidx (nb);=0A= - if (tcache && tc_idx < mp_.tcache_bins)=0A= + /* While we're here, if we see other chunks of the same size,=0A= + stash them in the tcache. */=0A= + size_t tc_idx =3D csize2tidx (nb);=0A= + if (tcache && tc_idx < mp_.tcache_bins)=0A= + {=0A= + /* While bin not empty and tcache not full, copy chunks. */=0A= + while (tcache->counts[tc_idx] < mp_.tcache_count)=0A= {=0A= - mchunkptr tc_victim;=0A= -=0A= - /* While bin not empty and tcache not full, copy chunks. */=0A= - while (tcache->counts[tc_idx] < mp_.tcache_count=0A= - && (tc_victim =3D *fb) !=3D NULL)=0A= - {=0A= - if (__glibc_unlikely (misaligned_chunk (tc_victim)))=0A= - malloc_printerr ("malloc(): unaligned fastbin chunk detected 3");=0A= - if (SINGLE_THREAD_P)=0A= - *fb =3D REVEAL_PTR (tc_victim->fd);=0A= - else=0A= - {=0A= - REMOVE_FB (fb, pp, tc_victim);=0A= - if (__glibc_unlikely (tc_victim =3D=3D NULL))=0A= - break;=0A= - }=0A= - tcache_put (tc_victim, tc_idx);=0A= - }=0A= + mchunkptr tc_victim =3D fastbin_pop_chunk (fb);=0A= + if (tc_victim =3D=3D NULL)=0A= + break;=0A= + tcache_put (tc_victim, tc_idx);=0A= }=0A= -#endif=0A= - void *p =3D chunk2mem (victim);=0A= - alloc_perturb (p, bytes);=0A= - return p;=0A= }=0A= +#endif=0A= + void *p =3D chunk2mem (victim);=0A= + alloc_perturb (p, bytes);=0A= + return p;=0A= }=0A= }=0A= =0A= @@ -4505,29 +4538,7 @@ _int_free (mstate av, mchunkptr p, int have_lock)=0A= fb =3D &fastbin (av, idx);=0A= =0A= /* Atomically link P to its fastbin: P->FD =3D *FB; *FB =3D P; */=0A= - mchunkptr old =3D *fb, old2;=0A= -=0A= - if (SINGLE_THREAD_P)=0A= - {=0A= - /* Check that the top of the bin is not the record we are going to=0A= - add (i.e., double free). */=0A= - if (__builtin_expect (old =3D=3D p, 0))=0A= - malloc_printerr ("double free or corruption (fasttop)");=0A= - p->fd =3D PROTECT_PTR (&p->fd, old);=0A= - *fb =3D p;=0A= - }=0A= - else=0A= - do=0A= - {=0A= - /* Check that the top of the bin is not the record we are going to=0A= - add (i.e., double free). */=0A= - if (__builtin_expect (old =3D=3D p, 0))=0A= - malloc_printerr ("double free or corruption (fasttop)");=0A= - old2 =3D old;=0A= - p->fd =3D PROTECT_PTR (&p->fd, old);=0A= - }=0A= - while ((old =3D catomic_compare_and_exchange_val_rel (fb, p, old2))= =0A= - !=3D old2);=0A= + mchunkptr old =3D fastbin_push_chunk (fb, p);=0A= =0A= /* Check that size of fastbin chunk at the top is the same as=0A= size of the chunk that we are adding. We can dereference OLD=0A= @@ -4718,6 +4729,7 @@ static void malloc_consolidate(mstate av)=0A= maxfb =3D &fastbin (av, NFASTBINS - 1);=0A= fb =3D &fastbin (av, 0);=0A= do {=0A= + /* Synchronize with relase MO CAS in fastbin_push_chunk. */=0A= p =3D atomic_exchange_acquire (fb, NULL);=0A= if (p !=3D 0) {=0A= do {=0A= =0A=