From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <sourceware-bugzilla@sourceware.org>
Received: by sourceware.org (Postfix, from userid 48)
	id 63C413858281; Tue, 12 Mar 2024 11:34:00 +0000 (GMT)
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 63C413858281
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org;
	s=default; t=1710243240;
	bh=UICHZGrtmpLYzJ8xJKI+mu7H2eTw19rom6Ib7xxSaTQ=;
	h=From:To:Subject:Date:From;
	b=f0EpInpv8DA8YKPiDFGpqljdm1RiVaScoRXWq6pOlVu5NR2FdKIZe1kEGpZvN7+fE
	 VvT82XDdP5PP3Gkn5C40KEoeFHdGLI1uH0WDtKabX75ochtfPrwNHjkkmMoD6qcfLL
	 etrr1Ux3LLM5KgdXlB47jeaNHSPtxvnpzma7Xzvk=
From: "pengzheng at apache dot org" <sourceware-bugzilla@sourceware.org>
To: glibc-bugs@sourceware.org
Subject: [Bug nptl/31477] New: Priority Inversion and Unlimited Spin of
 RWlock
Date: Tue, 12 Mar 2024 11:33:59 +0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: new
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: glibc
X-Bugzilla-Component: nptl
X-Bugzilla-Version: 2.29
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: normal
X-Bugzilla-Who: pengzheng at apache dot org
X-Bugzilla-Status: UNCONFIRMED
X-Bugzilla-Resolution: 
X-Bugzilla-Priority: P2
X-Bugzilla-Assigned-To: unassigned at sourceware dot org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags: 
X-Bugzilla-Changed-Fields: bug_id short_desc product version bug_status
 bug_severity priority component assigned_to reporter cc target_milestone
Message-ID: <bug-31477-131@http.sourceware.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Bugzilla-URL: http://sourceware.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
List-Id: <glibc-bugs.sourceware.org>

https://sourceware.org/bugzilla/show_bug.cgi?id=3D31477

            Bug ID: 31477
           Summary: Priority Inversion and Unlimited Spin of RWlock
           Product: glibc
           Version: 2.29
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: nptl
          Assignee: unassigned at sourceware dot org
          Reporter: pengzheng at apache dot org
                CC: drepper.fsp at gmail dot com
  Target Milestone: ---

I found that there are several unlimited spins in the current pthread_rwloc=
k's
implementation.
Therefore, it suffers from the same issue of user-space spinlocks as mentio=
ned
in this LWN article ([0]):

> One thread might be spinning on a lock while the holder has been preempte=
d=20
> and isn't running at all. In such cases, the lock will not be released so=
on,
> and the spinning just wastes CPU time. In the worst case, the thread that=
 is=20
> spinning may be the one that is keeping the lock holder from running, mea=
ning=20
> that the spinning thread is actively preventing the lock it needs from be=
ing=20
> released. In such situations, the code should simply stop spinning and go=
 to=20
> sleep until the lock is released.

I just encountered one such issue in an embedded Linux system: there were
several readers of priority SCHED_RR, and one writer of priority SCHED_OTHE=
R.

It was found that two high priority readers are spinning (consuming 100% CP=
U)
within the loop near the end of `__pthread_rwlock_rdlock_full`:

  for (;;)
    {
      while (((wpf =3D atomic_load_relaxed (&rwlock->__data.__wrphase_futex=
))
          | PTHREAD_RWLOCK_FUTEX_USED) =3D=3D (1 | PTHREAD_RWLOCK_FUTEX_USE=
D))
      {/*omitted*/}
      if (ready)
    /* See below.  */
             break;
      if ((atomic_load_acquire (&rwlock->__data.__readers)
       & PTHREAD_RWLOCK_WRPHASE) =3D=3D 0)
            ready =3D true;
    }
  return 0;

And the SCHED_OTHER writer was just about to enable the `__wrphase_futex` in
`__pthread_rwlock_wrlock_full` (just one ARM instruction away)
but never able to do that (the two readers ate nearly all available CPUs):

  while ((r & PTHREAD_RWLOCK_WRPHASE) =3D=3D 0
     && (r >> PTHREAD_RWLOCK_READER_SHIFT) =3D=3D 0)
    {
      if (atomic_compare_exchange_weak_acquire (&rwlock->__data.__readers,
                        &r, r | PTHREAD_RWLOCK_WRPHASE))
    {
      atomic_store_relaxed (&rwlock->__data.__wrphase_futex, 1);  /* writer=
 was
stuck HERE! */

      goto done;
    }
      /* TODO Back-off.  */
    }

In ARM assembly:

move r3, #1 ; the writer is stuck HERE!
str r3,[r12,#8] ; r12 holds the address of rwlock->__data, and 8 is the off=
set
of __readers in __data


Unlimited user space spin is too dangerous to be used, how about limiting t=
he
total number of spins before suspending using futex? Or using rseq as menti=
oned
in the LWN artible?

Any ideas?

[0] https://lwn.net/Articles/944895/

--=20
You are receiving this mail because:
You are on the CC list for the bug.=