From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lj1-x22f.google.com (mail-lj1-x22f.google.com [IPv6:2a00:1450:4864:20::22f]) by sourceware.org (Postfix) with ESMTPS id 0A68C3858D39 for ; Mon, 13 Dec 2021 19:28:09 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 0A68C3858D39 Received: by mail-lj1-x22f.google.com with SMTP id i63so25213195lji.3 for ; Mon, 13 Dec 2021 11:28:08 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=wuhs1uu3xd+DAtVvYo8LfoJq6FMmwCoLXFe+b3mQ9lk=; b=s3QL1lRmAP1lfEc+YslKVZ6OpMk8qyK6QZ+QhgmH4fmHMu5gis08FPAglmU0lviWFg NEhIUyD8HvLeQGCDsFtdxHRyY6RHihkpHz8OZjgYdCeb65oEz+iFgZCxOoMq8Gg6HkM5 UqjTnZzC4sJM31WUjfUW6d+fPW2/NuNSM2a9GSUXTbgGzNDpEdqnTjEkSlxuR627lH0M D5oLZ6A/Wq6BnaW9TDtTJ3uFlL8HOtjGJNbSIzeML0hmW45x56lBYjwF/mFZEEb5Ztx2 3l60ZZpiNuoJlnRYtbRF3D5aArAhYLof2B0+kTsqLlkW3SNzOWWtLF6mU0tcPimzkgOL 1JFg== X-Gm-Message-State: AOAM533xRttXWwX2tpOue/83QoTC+5MwofSBWdO28Oe5hlJF/I8ucNv3 4thp6jiafKTNELPhciXDUxjP1ttdQa2EcmVBmkEsEA== X-Google-Smtp-Source: ABdhPJxqWHONSnBCMMMMigrcQTyIdrM1feHJHhhHJ+AuRFOFD/oPpUM/4g0Ux0zeJp2mf9xcOHyDILt1fheukMheXoQ= X-Received: by 2002:a05:651c:324:: with SMTP id b4mr493697ljp.188.1639423687564; Mon, 13 Dec 2021 11:28:07 -0800 (PST) MIME-Version: 1.0 References: <87tufctk82.fsf@oldenburg.str.redhat.com> In-Reply-To: <87tufctk82.fsf@oldenburg.str.redhat.com> From: Jann Horn Date: Mon, 13 Dec 2021 20:27:41 +0100 Message-ID: Subject: Re: rseq + membarrier programming model To: Florian Weimer Cc: linux-api@vger.kernel.org, Mathieu Desnoyers , libc-alpha@sourceware.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-18.6 required=5.0 tests=BAYES_00, DKIMWL_WL_MED, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, ENV_AND_HDR_SPF_MATCH, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, USER_IN_DEF_DKIM_WL, USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Dec 2021 19:28:11 -0000 On Mon, Dec 13, 2021 at 7:48 PM Florian Weimer wrote: > I've been studying Jann Horn's biased locking example: > > Re: [PATCH 0/4 POC] Allow executing code and syscalls in another address space > > > It uses MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ as part of the biased lock > revocation. > > How does the this code know that the process has called > MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ? Could it fall back to > MEMBARRIER_CMD_GLOBAL instead? AFAIK no - MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ specifically forces targeted processes to go through an RSEQ preemption. That only happens when this special membarrier command is used and when an actual task switch happens; other membarrier flavors don't guarantee that. Also, MEMBARRIER_CMD_GLOBAL can take really long in terms of wall clock time - it's basically just synchronize_rcu(), and as the documentation at https://www.kernel.org/doc/html/latest/RCU/Design/Requirements/Requirements.html says: "The synchronize_rcu() grace-period-wait primitive is optimized for throughput. It may therefore incur several milliseconds of latency in addition to the duration of the longest RCU read-side critical section." You can see that synchronize_rcu() indeed takes quite long in terms of wall clock time (but not in terms of CPU time - as the documentation says, it's optimized for throughput in a parallel context) with a simple test program: jannh@laptop:~/test/rcu$ cat rcu_membarrier.c #define _GNU_SOURCE #include #include #include #include #include #include int main(void) { for (int i=0; i<20; i++) { struct timespec ts1; if (clock_gettime(CLOCK_MONOTONIC, &ts1)) err(1, "time"); if (syscall(__NR_membarrier, MEMBARRIER_CMD_GLOBAL, 0, 0)) err(1, "membarrier"); struct timespec ts2; if (clock_gettime(CLOCK_MONOTONIC, &ts2)) err(1, "time"); unsigned long delta_ns = (ts2.tv_nsec - ts1.tv_nsec) + (1000UL*1000*1000) * (ts2.tv_sec - ts1.tv_sec); printf("MEMBARRIER_CMD_GLOBAL took %lu nanoseconds\n", delta_ns); } } jannh@laptop:~/test/rcu$ gcc -o rcu_membarrier rcu_membarrier.c -Wall jannh@laptop:~/test/rcu$ time ./rcu_membarrier MEMBARRIER_CMD_GLOBAL took 17155142 nanoseconds MEMBARRIER_CMD_GLOBAL took 19207001 nanoseconds MEMBARRIER_CMD_GLOBAL took 16087350 nanoseconds MEMBARRIER_CMD_GLOBAL took 15963711 nanoseconds MEMBARRIER_CMD_GLOBAL took 16336149 nanoseconds MEMBARRIER_CMD_GLOBAL took 15931331 nanoseconds MEMBARRIER_CMD_GLOBAL took 16020315 nanoseconds MEMBARRIER_CMD_GLOBAL took 15873814 nanoseconds MEMBARRIER_CMD_GLOBAL took 15945667 nanoseconds MEMBARRIER_CMD_GLOBAL took 23815452 nanoseconds MEMBARRIER_CMD_GLOBAL took 23626444 nanoseconds MEMBARRIER_CMD_GLOBAL took 19911435 nanoseconds MEMBARRIER_CMD_GLOBAL took 23967343 nanoseconds MEMBARRIER_CMD_GLOBAL took 15943147 nanoseconds MEMBARRIER_CMD_GLOBAL took 23914809 nanoseconds MEMBARRIER_CMD_GLOBAL took 32498986 nanoseconds MEMBARRIER_CMD_GLOBAL took 19450932 nanoseconds MEMBARRIER_CMD_GLOBAL took 16281308 nanoseconds MEMBARRIER_CMD_GLOBAL took 24045168 nanoseconds MEMBARRIER_CMD_GLOBAL took 15406698 nanoseconds real 0m0.458s user 0m0.058s sys 0m0.031s jannh@laptop:~/test/rcu$ Every invocation of MEMBARRIER_CMD_GLOBAL on my laptop took >10 ms.