From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from sonic310-25.consmr.mail.ne1.yahoo.com (sonic310-25.consmr.mail.ne1.yahoo.com [66.163.186.206]) by sourceware.org (Postfix) with ESMTPS id 0E54C385840D for ; Wed, 24 Apr 2024 00:57:55 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 0E54C385840D Authentication-Results: sourceware.org; dmarc=pass (p=reject dis=none) header.from=yahoo.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=yahoo.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 0E54C385840D Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=66.163.186.206 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1713920276; cv=none; b=tto65uFBD9buZyAYL0Zk8LwBIshPFOQbNz9r+bS9rlbc9LIjPf46yApyIOypOOviZptM4HRTg++ZQobcffDBQd2xqMvmf6MVIBPfJ9/pYSnsVGQamYOTmqJBL/3e3GZG7cGrsX/1RuNVhYtRBaUD33MHT7U2nmopvd1ZGnv1nWo= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1713920276; c=relaxed/simple; bh=CFLhDFyom3CoMoINL4Iad3hxIRClgt1NhyRnjp89m30=; h=DKIM-Signature:From:Mime-Version:Subject:Message-Id:Date:To; b=GkmVn/f84pzDpvVbaatECnOuZ5IPCZc0Y5gBi6zXMc2lyxr7j2E2ByLYRM/0MIuxZaE8zPeE3826rbegtXrgFeqdND3xu8/qRYzd8Zf0U77h0O+5szbc0E1qStx+fmNwO07gJEPNtO8kYU+dlKgI3pp/GleG81DC3vyjhBUswLo= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1713920274; bh=SUDcAqP9q6p3kJ7bfnMqGibabZELWbEDrm8PlioKCs0=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=EELrDDEa1HPD7WEfMXI0hFSVQNYfVKZymB7SXtrEl8O8oDfDoTlFCMpLUBmIkqCqPjMzGDueApPsoBgZIYgfzqy7EcJ23TQAION2hv+Yr1cd9SgYK/P8ny5pJhuKCBFq4hPTnUwk7Pq65UW1Pxa903J/cMdTi3WBVBhDG1wkvu0dP8AQ9b12n+8sr8OkDLUisuhcoZ5gzbIYMHm7lOGEGwmAAUGWygI8A6UrOJyixU9qIEyYuYToh/ZX1icOKzDYEWzpzkyKxui3T/1JF/7sS+du0ozfmijmHn2rZunwupGmJU1UW3lkDHuFtEEzADlT4wOwSEnY26GdxlIc5eUrKw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1713920274; bh=oZFMwCGTpWa3kc61CxO7l5aXxjQN4sMzyc0P/RfMmto=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=ofnlhEMsTp0J7FWf2IROneqIYtAXM7G977hTxOzsTvG1gedSBDXH8CqhGm9/uL+6lJvGJXGJiXygXaLrBRPdg6OOOqW0cxYokcM9ByqhSZF/kH0qNMgqqczQI9/CYlZNIhtUVZIR2bDqy0DoCnrenzNuhElKVZaNmZLU06kPedt+ErPTw9C/ZfHSBlgyx1uPGaaue3RY4EiCuMDHgxfKtLdxN4SXR5u0o5zggeU39BpYd2gfpzMtow0+Ry9jSUvB9SFwkSkys8Nz3tMIkKcoGqDVG3NafpPHjKATkCb9tG1wHUNkPGC2h6GN2/I4lzLfqhISQiHn/sX/sXQBS8rVYw== X-YMail-OSG: 4v4_COcVM1ns1YE02.9ksSc0K_eqOMUobh7FtmmiW5G6yRWn57ttsmRSth3YVvN 8SOPkuUSTmdeLq4i35gkVEoF_9jpxY0EDcCa305DMMmYJeCofvaDgwf7ZInFWPogQMHcquduDouU HAWaxEyeXCVNZ.T7_Ipx8.7w8UPFsNVEGRpBNRPayp8mCmTyADEV3F7qFV2kDCfq9h7.iueQMvdm 8swbminSYjfj.fJjZmk3SL53Mg0.CE4Z9f2oca1kMHApoGodnqHrAtpMpSquztEswIjh_91L2KIA PrntraLVqo9ChqlyO697dPsJ_xhkw82FeQUiBk4PYAS.9qVBNnT7O9hXteAs7M2XpwYB116qhOgM EiEeIbw0XPGsp0o6p01hL8TZaVgYcIjEW1wv1G7cjGy6sQLXHI6fmdYCA22LZoh7NRvSNdeLcPT7 iG0rHF_qnKvH5wmp7DN4U45W1n8sORdP6o35WCgB2UTeI5uW45i9IY8ivD0.GpUKZ161aLQ2pvrc oMf4AvD3ROLirTlraqFnps3MgXyi7OmXQGEwqPyH2fROXWKCHxjL8gWO6OvxcOhigBRFdnKvEjne BEmkRbWkLRjdY87_or2lKBomp0CqkNn_nFiXF8w8gEaDOpTDKE5xQhBuZ9_6dUUCCcgdZ9cl6y4G vy4KNPlxC3OzsUrwK1wC..KkLqu284n3FXOQyNR5u.ew_Gx_gHX.axqrlnQv9DLUEOvLXXqd1rRC arpgR1NgkAHaiQD2xewfMAwnLyxyT6NUrF1AFicr5d6fAKk3K9qWDPBH3CQXviEA6THJW.pFTWEN y_yebn2BX46BZnajzRdJZ91_tHOt8TqPDb747pcv0UfmxdM9a8F5kZXmLnnU1Lod2q8TV8HfH3PG kFVIlpzD2AMqkr9TKVRWUfrIHZFjXpv0jw3IX_lgq6MVG8g7e.KIESjm9naXt_VdAU5NGYOv0Vro MSuNAr0VBWKLJFatlKcAREGEmwFETRY9r8FBtzzF9TW0._mVFd1j9qfmcsje97ohiDqf6322uqBp UhQ5BddnbQ_961_40XA5t1Y.k76pt7MvDhfEgmo8MNNSrp.iJsl_ZCfkZXnNTA939XZ2R4CZzhFd Ut.X0xaPhrCDkg7jSJ8Vf7bvrdsHFnhKBWj2BU2caECb9ox2E13vND3BSRPTPjUzOQi4JPiAkbqT sDueBd6bVG4BAI_5bPGL3j2lci9FBE86UfwiYMxfyRWs77Eq0Rt2VqsyzfgFLLFWFwOsFSWBs9ce u9GlJSYyFZql9GdnpP5UzjTFM0oasxpJngxEwzpKzgTUDVe5yQMMa8RHiO1mzWPcWUuC1RG0nx7T SespTtNtzb1JRghI94LD1orRTYWpAmUw6W1oCGOsVDDXq9.NKS6Uj_4RBMCPU2z5JjUrmNsrSfwN 1FCxkVJk_BSwXRPH5Ov4ZZ3GiA6R_9.Rko5DDKVQ598zYy.VFuMzRvnbqt23Dj5ogLC4hehDpoaX XMqHjsclLiWpxOZZ9HdapoEX3eW6QezW8MOxaJ.6YRnMe09Wn1IpQVLc9bmFLrd0KMpUc0kCUMoe 9n8vG4g7U2AFdjyY_GjZXWpme1y96w_H8sFwqQty8JPP2xbW9a964Nh6mlvIavQgstHTOa7IUUZm VFstXS2JPKkXGWE4rsoy4K4nSNQmPNaVpeQdy1gda0xW2ivhknnvjxnH18Pevwu0Hl2SwadfRrJl Zo7JTRoP6UlIUJwy9_BMKpagUwK6vmHdSBisbT06.t_4Q.._wyD5cZiiZ8ub6PvEV6.5bpGeyCYA rvJ5Q0RerBmDQNAuwxSSRIZK9gvFxhn7LXniyORPe9jG10eBGS3UlOgda3efWO4UejJLeX2JBCDV oPFudnke89aOdLxe5J5CMjn5mbka04La0.a1F5oLfFtmgE1P3B3iFarKz_GrPXbvM5iJYjbSV2yv 45IHwjq4d.GDUscswb2lRKDZbBHC_Tc.rDCfZhuBXjOtnwxYoY4JAFzoDsCxXCzlDYipMEPDS5AU pHICB_wKz36c.4eF.Lo377CkvT1k6ZBYY8sVLGEoD35s2FBxfNc8nztuT9wsZabv1TEJ2sj5nV8m nwuJj2M.rZLU8GhASjS7TPyDJ84iNbs9x690HG7ghhDhUL_AcuMVM0VsqkGlWVw5M1pyd74jztON nWW0woefv6sLPOwBzMkqvBkyGSd5PbtozOvhxa06NtJYChAfdXHDiDBI92Mm1YG3VZF.8bf1GZ04 .tdjZtxy898YLWE9eicskxRJSaBvQn4S5gLHsIeKIiO8P0wjpUmE- X-Sonic-MF: X-Sonic-ID: c77e5a32-bdb2-4899-9b8f-64a470fb5ef8 Received: from sonic.gate.mail.ne1.yahoo.com by sonic310.consmr.mail.ne1.yahoo.com with HTTP; Wed, 24 Apr 2024 00:57:54 +0000 Received: by hermes--production-gq1-59c575df44-wmc6c (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 4d53d4a1bef17c81fa806fe77fddcb21; Wed, 24 Apr 2024 00:57:49 +0000 (UTC) From: Sivaprakasam Suresh Content-Type: multipart/alternative; boundary="Apple-Mail=_8CDDAD8E-1025-4D76-8962-C6D8B51E82C0" Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\)) Subject: pthread_rwlock_wrlock performance Message-Id: <081D9537-0E1A-4447-AAEC-CCCFB2016C0E@yahoo.com> Date: Tue, 23 Apr 2024 17:57:37 -0700 To: libc-help@sourceware.org X-Mailer: Apple Mail (2.3774.400.31) References: <081D9537-0E1A-4447-AAEC-CCCFB2016C0E.ref@yahoo.com> X-Spam-Status: No, score=-0.4 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: --Apple-Mail=_8CDDAD8E-1025-4D76-8962-C6D8B51E82C0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hi, We recently moved from el7 to el8, the glibc version changed from 2.17 to 2= .28. We have a storage application which scales in pthreads, on 8cpu system= it can scale dynamically from 8 to about 128 pthreads. We have noticed pth= read_rwlock_wrlock performance is worse in 2.28 than from 2.17 as we goto h= igher thread counts. For lower thread counts 2.28 pthread_rwlock_wrlock per= forms better than 2.17 pthread_rwlock_wrlock when the ratio of vcpu to thre= ads is lower but when its goes higher is when the performance is worse. I investigated this further it appears around glibc 2.25 the implementation= of pthread_rwlock_wrlock changed, we removed the internal locks and replac= ed with atomic increment, we spin more on cpu than we sleep. I think for hi= gher thread counts we are likely seeing writer starvation because we are co= mpeting with spinning threads. We did bpftrace saw lock contention times in= crease with 2.28 compared to 2.17 all else remaining the same. I have standalone test program where we have tried different thread counts,= as we goto higher thread counts the performance worsens. Has anyone encountered this problem ? Any suggestions what to do about this= ? What is the next step ? Regards, -Suresh --Apple-Mail=_8CDDAD8E-1025-4D76-8962-C6D8B51E82C0--