pthread_rwlock_wrlock performance

[Sivaprakasam Suresh <suresh420@yahoo.com>]

[-- Attachment #1: Type: text/plain, Size: 1237 bytes --]

Hi,

We recently moved from el7 to el8, the glibc version changed from 2.17 to 2.28. We have a storage application which scales in pthreads, on 8cpu system it can scale dynamically from 8 to about 128 pthreads. We have noticed pthread_rwlock_wrlock performance is worse in 2.28 than from 2.17 as we goto higher thread counts. For lower thread counts 2.28 pthread_rwlock_wrlock performs better than 2.17 pthread_rwlock_wrlock when the ratio of vcpu to threads is lower but when its goes higher is when the performance is worse.

I investigated this further it appears around glibc 2.25 the implementation of pthread_rwlock_wrlock changed, we removed the internal locks and replaced with atomic increment, we spin more on cpu than we sleep. I think for higher thread counts we are likely seeing writer starvation because we are competing with spinning threads. We did bpftrace saw lock contention times increase with 2.28 compared to 2.17 all else remaining the same.

I have standalone test program where we have tried different thread counts, as we goto higher thread counts the performance worsens.

Has anyone encountered this problem ? Any suggestions what to do about this ? What is the next step ?

Regards,

-Suresh