From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60081.outbound.protection.outlook.com [40.107.6.81]) by sourceware.org (Postfix) with ESMTPS id 75405393C871 for ; Tue, 13 Apr 2021 08:19:52 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 75405393C871 Received: from AS8PR04CA0013.eurprd04.prod.outlook.com (2603:10a6:20b:310::18) by AM0PR08MB4419.eurprd08.prod.outlook.com (2603:10a6:208:139::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.21; Tue, 13 Apr 2021 08:19:50 +0000 Received: from AM5EUR03FT038.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:310:cafe::6c) by AS8PR04CA0013.outlook.office365.com (2603:10a6:20b:310::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.16 via Frontend Transport; Tue, 13 Apr 2021 08:19:50 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; sourceware.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;sourceware.org; dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT038.mail.protection.outlook.com (10.152.17.118) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.17 via Frontend Transport; Tue, 13 Apr 2021 08:19:50 +0000 Received: ("Tessian outbound 34291f7cb530:v90"); Tue, 13 Apr 2021 08:19:49 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: b6c630544b3be085 X-CR-MTA-TID: 64aa7808 Received: from 046f3dc888f3.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 06A5E244-2A0D-4F68-8257-31BA7C376AF5.1; Tue, 13 Apr 2021 08:19:43 +0000 Received: from EUR05-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 046f3dc888f3.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 13 Apr 2021 08:19:43 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JarGtGggB/GSKypzU/CAkHuTJ0jodLLj2bkVEuvoHr1fgxrNqM8C0vX015EhL9J+ztQKjVeBF2HarK8NdtOfrPUDkdGc5Z7aF3InU8CrHkPck5iGZp8LshwvDXScQp+ok8NdA+A5cI4vGRi2EdfMC0Q6LVEYAdYYsIaTPyw2gS3daFS2n0CxoiJZizEzPizaPu/7XhmAGgBpChX4v/6nC++ip1rqOAjLKIWk3oqC2AcAtDyVd93y5jpdMcuU7LUFRoxhL9o3sRfBmyR88OGA7lyP7ZXE6Q//2Urx5pXOP4S13v3lWtEK6zXbwZ5LTpMOKGO7MDpaUgPLtpL0hno5HA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2TPN0KmXruJJ/WpJkQcnFnqZiHZIK9zDLTZRCkAbl8I=; b=eopT2ascSJ+54NSXAPZIrE1Uw4NLam+zz72uGqP/hVkzFmrXIMxLXzJXcpLBVgJzogWM84k3VKelLdd+GY0Bjyw/Mo7aqh1ls1dYhGbJq82C4WMXcAWwiFEnmtmLkLVamBHx0WvXzALNDPsyCmpC/wJnV9FAVLZpmH+vX57BiIBbgtzOu90LHjQqH81t16hQXBNgTtfSfNQqepqsOkoff6mbqR+5ORtUIg/49rSucTa5VP99FT5Ts7LSb9aMcVliIAC2CXogGOfcuAgTnL1AdW8fx2ek+PYjVXiOH9Qc8Y4aiP7yfBAK5bIZJn4hAfgCSt29K6JpddhTgUcqKjulJQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none Authentication-Results-Original: sourceware.org; dkim=none (message not signed) header.d=none;sourceware.org; dmarc=none action=none header.from=arm.com; Received: from PA4PR08MB6320.eurprd08.prod.outlook.com (2603:10a6:102:e5::9) by PAXPR08MB6397.eurprd08.prod.outlook.com (2603:10a6:102:12f::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.18; Tue, 13 Apr 2021 08:19:42 +0000 Received: from PA4PR08MB6320.eurprd08.prod.outlook.com ([fe80::c99f:671d:bb2c:f20b]) by PA4PR08MB6320.eurprd08.prod.outlook.com ([fe80::c99f:671d:bb2c:f20b%7]) with mapi id 15.20.4020.022; Tue, 13 Apr 2021 08:19:42 +0000 From: Szabolcs Nagy To: libc-alpha@sourceware.org Subject: [PATCH v2 06/14] elf: Use relaxed atomics for racy accesses [BZ #19329] Date: Tue, 13 Apr 2021 09:19:36 +0100 Message-Id: <10fb15a36b3f6bc3e5ca62cda081c86512f47d32.1618301209.git.szabolcs.nagy@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: Content-Type: text/plain X-Originating-IP: [217.140.106.55] X-ClientProxiedBy: LO2P265CA0429.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:a0::33) To PA4PR08MB6320.eurprd08.prod.outlook.com (2603:10a6:102:e5::9) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (217.140.106.55) by LO2P265CA0429.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:a0::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.22 via Frontend Transport; Tue, 13 Apr 2021 08:19:42 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2d74d91f-4241-4b26-b31e-08d8fe54e84f X-MS-TrafficTypeDiagnostic: PAXPR08MB6397:|AM0PR08MB4419: X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true NoDisclaimer: true X-MS-Oob-TLC-OOBClassifiers: OLM:9508;OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: zAw18BgXvK6t5D4Dw07vVVyrIY1Rj/mBJmpbnzErKVJC+1dIT6175TP+kyFICMK4zsZV2Kmmk1TibltKk/uWT4Iof5w91LCvw1XoG4syyWye0fbpVQ2QDIVk6GfwA87pB2IJsmYITZIU2KBzQHYPLZqp3XKJ8e+4DgDhawFLCS10JBmX9x31i+PAV0oS3arhrRRowxb6QZi8Fw8tmb1lSI4/YJQuWV8xQUSGT9U1p8hjggTfVWKQWI3IJq6+nvi+xUEZJEwY+GafE8sheOta+R9G8Hx/gpHkzIZpdB26EzH9KLIwuK9E5xG62sd1xtReLbAZDYCrPvgb3/KjDYFBfA24QOuk+9SaZ71O66OJAOl7HWlgnoC6yfFY5W9y7zw8pppdB5JGxnIBpwfMdf86sxWQYvXqwQ0yx7xQk9FAw9anjrco/I3jukNBOiAV03wKgySJT8oLonoNB2l3qOY0ZRbsqehi2xzO0czuGSzTUM+t0gVRgUZ1wsaSfBXt1SR9bFsG5Cqn/Q9Be6ybppyUBoTt9wibl6T2Yxcr4xtxTweKFyszm31aGh5V4MrDsQ2HyfiZSd/yhPKvQwuU7/RdXFI/loP6FxrEBY8gQhzZFAgbXmxmpWRXXf4i1e74wSHyrKIrMNrxgiRRWk4+fRF8+Vin7O6sKmlg2tMvVTbQlZcM9bphcovuvvdAAbq4WhKS X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PA4PR08MB6320.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(136003)(346002)(396003)(39860400002)(376002)(366004)(2906002)(6666004)(66556008)(6506007)(36756003)(26005)(66946007)(69590400012)(44832011)(38350700002)(83380400001)(956004)(86362001)(6486002)(16526019)(8676002)(316002)(8936002)(478600001)(6512007)(52116002)(38100700002)(5660300002)(2616005)(186003)(66476007)(6916009); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: =?us-ascii?Q?DiXtVoqBjgpEl2NBFGk3LE2t1yp2k9L/LFX4pzAvMXZZojnY+FU9JUheQH74?= =?us-ascii?Q?2YNUyAyHUHLSKfYNCog2xaH12UR04UBuAQ31a+SEWrYT5GW+PC7zkbnyArIO?= =?us-ascii?Q?jKLSKTKWcU1ZvgSPce6iSsAza3MaY0KXQtBm90tynlRvIkDsDggXczQwvNk7?= =?us-ascii?Q?PhBmD32ucjEXjOj+zr8giz+laCQpmAmuPDAISyfepbKFbkcvuBvg03hvh+Jh?= =?us-ascii?Q?SP7FC32prpJV8yjD7ujqoaZ8c+QhjVHkIGfV4ArRel+HL2+SpV3XbrzOrAMX?= =?us-ascii?Q?V4/9MTUtPKXbjA2gvNUq75mtlr3GsmQ8kW1VMkDZ1zQRzYfJbj5yevHiOaFl?= =?us-ascii?Q?Tpy94GTJGlSgBi69aAMPTktyZZr/0PPostut/r28ul7UwVSpjJFoi5QTUUK6?= =?us-ascii?Q?CkH5iZztoQZFKgflNBcOblfvM58Usu+6r0yCeDYHixjjXvaBhnmdTpbnbQz1?= =?us-ascii?Q?HI/A+kGsH5y6R8lUG3mngViG1J91HjQQ9UJEeTKa30nnMfQ8EIAujMZBtttk?= =?us-ascii?Q?+3XAoV+8kBUJT/nsDIcZY2uxQI7nQ4ic60hjx2+xHcuERawwqSbjrSXqczPp?= =?us-ascii?Q?TPQlLTlAQVt6NsecNkYz+0LzGRKbtf/No5ru0FttEGTk0ClgAPQA42D0iKWU?= =?us-ascii?Q?rU8vIvZJRe5ne/b79jqdtG8LRykL/HRmtukNs4Ys+NG7yOa0ajfyrdAx05Sx?= =?us-ascii?Q?hH2U/1aQpwPWx11vrU9rHB9vFuli6MaCkznDDDn7PHsWoRUVOGAWZbfbVtbi?= =?us-ascii?Q?y/gwx14g7G+Ew80vDT5ahU4vCPc6QNdhobQ5SiJDOCf/RUg/LSPGBzUzrTih?= =?us-ascii?Q?Vc+LEqJcI6DDVELJ/a181iyXoCU3G1rOCIvCorh4+DCVc6XK1YJ0sSp2Qqvm?= =?us-ascii?Q?Tw4j6VwWWxMmDw7ETFQfZtfGPQg9XxZnUO/+Qaes3Mu7LNXiQJp+EleNI2g0?= =?us-ascii?Q?GPlXQuEalOs/t9eoU2h6Q+3S5LE6EG/enkI1yHIEQ8u+frlzIfEnfX3ygbbK?= =?us-ascii?Q?GCDWy1hKJdS4F1lYNA4u4KPeq+ZG1koxQpUVarYIJkjSqxjpl0Frp3K5UZdM?= =?us-ascii?Q?HQoJ3sRwCSjUPZtD+Hkq6yD1RxWmqkkBbQ9cPY/KxEKEsRUBzOKkanD/thmi?= =?us-ascii?Q?kzEx8njy0UrwzbEwsrokF7xnPRG8Kgj/pMmsG5xwGtfYTT6wWxD0rk2q/qOt?= =?us-ascii?Q?/anHNKOIttUn+g14PaTAfUhOP7CAbJ3d6QV+Oj0mCmBLOsh2U8EIe4UxYse4?= =?us-ascii?Q?odfMblr9Nt3v06Qebc1E9T/oeXUpcF95zKFk5k79stLtVSBF/CuXdWS0yI84?= =?us-ascii?Q?EMZTPShtGFj5L6c1OtHJUmJI?= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6397 Original-Authentication-Results: sourceware.org; dkim=none (message not signed) header.d=none; sourceware.org; dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT038.eop-EUR03.prod.protection.outlook.com X-MS-Office365-Filtering-Correlation-Id-Prvs: c5619d48-3abc-481a-b883-08d8fe54e395 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: WDQszjidp+cjwzpr45+xrc/xIsZBhc8R6Z8z8FO8xWOxFJ8cbKdTmPwPw12ME3GUICbMZXBC89Lnorw4ZmWjTGLDphHgt7lwQupkYCs3dsB7R2kmYmxAGN+TVfDwUBLHDNnpTYp6ybZVfpjR6d1Pbcg0x3UD2WYuZdB/6woOTCJrCg58gVWN1V5obAqPRqrMzDJfUKU8NFr3U7BviTV2wllDg+ddCW0JD3rFv0O8wSniR9kEdSbKOwKX8qytpr492Mq5uomvy9QrBdddhbcu0L1OtaS+NhuKBaJiuavVoBdOWtsS5iLu6njj7MnzZ3d8xHkZ7MNPaqf5OCUWoX7JxRXcLP+hhrVNYxERR3juOzGs4Hkc/dvA3KGHYviitd1jGSejHlW2ex+qEFAEkRzfgxDuKT1z4xvRUIFoS1O5ergfA7wMlzkpehZHYSQ7O4sKWNQTetK7R73kJAEQ7G+BjGuQN1d6HRBcDoXge90gvdH72IWYMGVHcY4cPu94J1eyS6TTX4C+VjHC4f2Yfg+ZpFXY9RqwknlCvReih+IsVtvGFx3mawOlcBUhMpvfnZ3SpePyQGQdd2DzftPzInBrMc4m2+1ORClcKHsIjPQehkw0QOjW+0FsufH6Emr+x9pMJ7gz42Hl6SW3sfghwrU/2/xcuI1kycuYvmLHhIlAW4sG3qEutjzl+pUI1tKcoDkQ X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(4636009)(376002)(346002)(136003)(396003)(39860400002)(46966006)(36840700001)(36756003)(70586007)(82740400003)(47076005)(70206006)(956004)(2616005)(69590400012)(44832011)(8936002)(6486002)(336012)(6916009)(36860700001)(26005)(6666004)(5660300002)(82310400003)(2906002)(356005)(86362001)(316002)(83380400001)(8676002)(186003)(16526019)(6512007)(6506007)(478600001)(81166007); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Apr 2021 08:19:50.0336 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2d74d91f-4241-4b26-b31e-08d8fe54e84f X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM5EUR03FT038.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB4419 X-Spam-Status: No, score=-14.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, MSGID_FROM_MTA_HEADER, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Apr 2021 08:19:55 -0000 This is a follow up patch to the fix for bug 19329. This adds relaxed MO atomics to accesses that are racy, but relaxed MO is enough. -- v2: - handle x86_64 dl-tls.c too --- elf/dl-close.c | 20 +++++++++++++------- elf/dl-open.c | 5 ++++- elf/dl-tls.c | 31 +++++++++++++++++++++++-------- sysdeps/x86_64/dl-tls.c | 3 ++- 4 files changed, 42 insertions(+), 17 deletions(-) diff --git a/elf/dl-close.c b/elf/dl-close.c index c51becd06b..3720e47dd1 100644 --- a/elf/dl-close.c +++ b/elf/dl-close.c @@ -79,9 +79,10 @@ remove_slotinfo (size_t idx, struct dtv_slotinfo_list *listp, size_t disp, { assert (old_map->l_tls_modid == idx); - /* Mark the entry as unused. */ - listp->slotinfo[idx - disp].gen = GL(dl_tls_generation) + 1; - listp->slotinfo[idx - disp].map = NULL; + /* Mark the entry as unused. These can be read concurrently. */ + atomic_store_relaxed (&listp->slotinfo[idx - disp].gen, + GL(dl_tls_generation) + 1); + atomic_store_relaxed (&listp->slotinfo[idx - disp].map, NULL); } /* If this is not the last currently used entry no need to look @@ -96,8 +97,8 @@ remove_slotinfo (size_t idx, struct dtv_slotinfo_list *listp, size_t disp, if (listp->slotinfo[idx - disp].map != NULL) { - /* Found a new last used index. */ - GL(dl_tls_max_dtv_idx) = idx; + /* Found a new last used index. This can be read concurrently. */ + atomic_store_relaxed (&GL(dl_tls_max_dtv_idx), idx); return true; } } @@ -571,7 +572,9 @@ _dl_close_worker (struct link_map *map, bool force) GL(dl_tls_dtv_slotinfo_list), 0, imap->l_init_called)) /* All dynamically loaded modules with TLS are unloaded. */ - GL(dl_tls_max_dtv_idx) = GL(dl_tls_static_nelem); + /* Can be read concurrently. */ + atomic_store_relaxed (&GL(dl_tls_max_dtv_idx), + GL(dl_tls_static_nelem)); if (imap->l_tls_offset != NO_TLS_OFFSET && imap->l_tls_offset != FORCED_DYNAMIC_TLS_OFFSET) @@ -769,8 +772,11 @@ _dl_close_worker (struct link_map *map, bool force) /* If we removed any object which uses TLS bump the generation counter. */ if (any_tls) { - if (__glibc_unlikely (++GL(dl_tls_generation) == 0)) + size_t newgen = GL(dl_tls_generation) + 1; + if (__glibc_unlikely (newgen == 0)) _dl_fatal_printf ("TLS generation counter wrapped! Please report as described in "REPORT_BUGS_TO".\n"); + /* Can be read concurrently. */ + atomic_store_relaxed (&GL(dl_tls_generation), newgen); if (tls_free_end == GL(dl_tls_static_used)) GL(dl_tls_static_used) = tls_free_start; diff --git a/elf/dl-open.c b/elf/dl-open.c index ab7aaa345e..83b8e96a5c 100644 --- a/elf/dl-open.c +++ b/elf/dl-open.c @@ -395,9 +395,12 @@ update_tls_slotinfo (struct link_map *new) } } - if (__builtin_expect (++GL(dl_tls_generation) == 0, 0)) + size_t newgen = GL(dl_tls_generation) + 1; + if (__builtin_expect (newgen == 0, 0)) _dl_fatal_printf (N_("\ TLS generation counter wrapped! Please report this.")); + /* Can be read concurrently. */ + atomic_store_relaxed (&GL(dl_tls_generation), newgen); /* We need a second pass for static tls data, because _dl_update_slotinfo must not be run while calls to diff --git a/elf/dl-tls.c b/elf/dl-tls.c index 33c06782b1..c4466bd9fc 100644 --- a/elf/dl-tls.c +++ b/elf/dl-tls.c @@ -175,7 +175,9 @@ _dl_next_tls_modid (void) /* No gaps, allocate a new entry. */ nogaps: - result = ++GL(dl_tls_max_dtv_idx); + result = GL(dl_tls_max_dtv_idx) + 1; + /* Can be read concurrently. */ + atomic_store_relaxed (&GL(dl_tls_max_dtv_idx), result); } return result; @@ -359,10 +361,12 @@ allocate_dtv (void *result) dtv_t *dtv; size_t dtv_length; + /* Relaxed MO, because the dtv size is later rechecked, not relied on. */ + size_t max_modid = atomic_load_relaxed (&GL(dl_tls_max_dtv_idx)); /* We allocate a few more elements in the dtv than are needed for the initial set of modules. This should avoid in most cases expansions of the dtv. */ - dtv_length = GL(dl_tls_max_dtv_idx) + DTV_SURPLUS; + dtv_length = max_modid + DTV_SURPLUS; dtv = calloc (dtv_length + 2, sizeof (dtv_t)); if (dtv != NULL) { @@ -767,7 +771,7 @@ _dl_update_slotinfo (unsigned long int req_modid) if (modid > max_modid) break; - size_t gen = listp->slotinfo[cnt].gen; + size_t gen = atomic_load_relaxed (&listp->slotinfo[cnt].gen); if (gen > new_gen) /* Not relevant. */ @@ -779,7 +783,8 @@ _dl_update_slotinfo (unsigned long int req_modid) continue; /* If there is no map this means the entry is empty. */ - struct link_map *map = listp->slotinfo[cnt].map; + struct link_map *map + = atomic_load_relaxed (&listp->slotinfo[cnt].map); /* Check whether the current dtv array is large enough. */ if (dtv[-1].counter < modid) { @@ -923,7 +928,12 @@ __tls_get_addr (GET_ADDR_ARGS) { dtv_t *dtv = THREAD_DTV (); - if (__glibc_unlikely (dtv[0].counter != GL(dl_tls_generation))) + /* Update is needed if dtv[0].counter < the generation of the accessed + module. The global generation counter is used here as it is easier + to check. Synchronization for the relaxed MO access is guaranteed + by user code, see CONCURRENCY NOTES in _dl_update_slotinfo. */ + size_t gen = atomic_load_relaxed (&GL(dl_tls_generation)); + if (__glibc_unlikely (dtv[0].counter != gen)) return update_get_addr (GET_ADDR_PARAM); void *p = dtv[GET_ADDR_MODULE].pointer.val; @@ -946,7 +956,10 @@ _dl_tls_get_addr_soft (struct link_map *l) return NULL; dtv_t *dtv = THREAD_DTV (); - if (__glibc_unlikely (dtv[0].counter != GL(dl_tls_generation))) + /* This may be called without holding the GL(dl_load_lock). Reading + arbitrary gen value is fine since this is best effort code. */ + size_t gen = atomic_load_relaxed (&GL(dl_tls_generation)); + if (__glibc_unlikely (dtv[0].counter != gen)) { /* This thread's DTV is not completely current, but it might already cover this module. */ @@ -1032,7 +1045,9 @@ cannot create TLS data structures")); /* Add the information into the slotinfo data structure. */ if (do_add) { - listp->slotinfo[idx].map = l; - listp->slotinfo[idx].gen = GL(dl_tls_generation) + 1; + /* Can be read concurrently. See _dl_update_slotinfo. */ + atomic_store_relaxed (&listp->slotinfo[idx].map, l); + atomic_store_relaxed (&listp->slotinfo[idx].gen, + GL(dl_tls_generation) + 1); } } diff --git a/sysdeps/x86_64/dl-tls.c b/sysdeps/x86_64/dl-tls.c index 6595f6615b..24ef560b71 100644 --- a/sysdeps/x86_64/dl-tls.c +++ b/sysdeps/x86_64/dl-tls.c @@ -40,7 +40,8 @@ __tls_get_addr_slow (GET_ADDR_ARGS) { dtv_t *dtv = THREAD_DTV (); - if (__glibc_unlikely (dtv[0].counter != GL(dl_tls_generation))) + size_t gen = atomic_load_relaxed (&GL(dl_tls_generation)); + if (__glibc_unlikely (dtv[0].counter != gen)) return update_get_addr (GET_ADDR_PARAM); return tls_get_addr_tail (GET_ADDR_PARAM, dtv, NULL); -- 2.17.1