From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr1-x44a.google.com (mail-wr1-x44a.google.com [IPv6:2a00:1450:4864:20::44a]) by sourceware.org (Postfix) with ESMTPS id 145D83858C27 for ; Tue, 26 Sep 2023 20:52:28 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 145D83858C27 Authentication-Results: sourceware.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=flex--dvyukov.bounces.google.com Received: by mail-wr1-x44a.google.com with SMTP id ffacd0b85a97d-321726b0238so7094847f8f.2 for ; Tue, 26 Sep 2023 13:52:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1695761547; x=1696366347; darn=sourceware.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=xrypDOHQ7EnWLT1fAC4mj+J/Ny7QwGyNqoMoBzWEChA=; b=I3K+ckO4bxiP/5rQ+V0Oyhr9UTDScveiQALFstWiLedpmcFxdoaSy2EUPCv9fKRXDJ CAoNMs0N07V+bMnCZzZ17XGRGZ49o2eRuhL8JkI0f723+c7ZuGLSEXV8ZAQZmcB7kLW/ H3wiKp3yUV0LWsTyF+WhMHzATnXtt+MP+AO1g1GyZsdcLdzzkV5o0i/8wHg4iqtvR6FD JIqbPqRzQxmOAPeMg+vjgjbEjMEVPqv9gS2HYbyvmQQSEnXrmMuUDnjDZLvS5oZ7YWqm oFwD/sRop11fhVHQRW6WyrZCfqna8V0WxWMeHSQXwafAsEPSljCweM2kIyJPD1qDzNJd cpcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695761547; x=1696366347; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=xrypDOHQ7EnWLT1fAC4mj+J/Ny7QwGyNqoMoBzWEChA=; b=VNMgd/Y0pIXFVHcy4DR7TFHkgiZNM1M8u8IozAKttQ5pNjGUYKeLTEHkRFfClVwCsi Rsz6cS/OV7zNWV13RGCKCHm80xFn1nzlhcS5vywzkdIqN92uz1z9vnoInDiDeuq4Hx2P ztxOGdWcIageh7irf9EzkMaobwGB9LQddmprV4lrOJf14uj9BGqYRKHOKZ7+WWQCPMQ4 0UkQWsoEbF5wEy7YghGnqT0vv14G50IFmalBmhhSox7wkxPX2+BxOxIroFAE72mAhF1j ukDmrKw+VxNImkA92UUvBxf3/PGQaLJxpxDoJlG0rNf4FY5HjrNK8pjZ6T2uszKPgxZU obWQ== X-Gm-Message-State: AOJu0Yxa77FRErI1sFjD87Xv2riLp2AQjHmJYdDrG+DAQJfazeeCty+Z XpWqGWdm+XBZ3dFfeTqbgId7RFL9Nhhj X-Google-Smtp-Source: AGHT+IFbugUBPJsObmukOcm7PN9PUqCtpIpnfXBZrDwvp1SMbjd8ej9aDBvvhWOhXNltTOfXtK7SKfPc3zlc X-Received: from dvyukov-desk.muc.corp.google.com ([2a00:79e0:9c:201:2a5f:6690:fe14:d69a]) (user=dvyukov job=sendgmr) by 2002:a5d:4b90:0:b0:321:a6b5:b50e with SMTP id b16-20020a5d4b90000000b00321a6b5b50emr57461wrt.11.1695761546817; Tue, 26 Sep 2023 13:52:26 -0700 (PDT) Date: Tue, 26 Sep 2023 22:52:15 +0200 In-Reply-To: <2c421e36-a749-7dc3-3562-7a8cf256df3c@efficios.com> Mime-Version: 1.0 References: <2c421e36-a749-7dc3-3562-7a8cf256df3c@efficios.com> X-Mailer: git-send-email 2.42.0.515.g380fc7ccd1-goog Message-ID: <20230926205215.472650-1-dvyukov@google.com> Subject: Re: [RFC PATCH v2 1/4] rseq: Add sched_state field to struct rseq From: Dmitry Vyukov To: mathieu.desnoyers@efficios.com Cc: David.Laight@ACULAB.COM, alexander@mihalicyn.com, andrealmeid@igalia.com, boqun.feng@gmail.com, brauner@kernel.org, carlos@redhat.com, ckennelly@google.com, corbet@lwn.net, dancol@google.com, dave@stgolabs.net, dvhart@infradead.org, fweimer@redhat.com, goldstein.w.n@gmail.com, hpa@zytor.com, libc-alpha@sourceware.org, linux-api@vger.kernel.org, linux-kernel@vger.kernel.org, longman@redhat.com, mingo@redhat.com, paulmck@kernel.org, peterz@infradead.org, pjt@google.com, posk@posk.io, rostedt@goodmis.org, tglx@linutronix.de Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-12.2 required=5.0 tests=BAYES_00,DKIMWL_WL_MED,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: >> I don't see why we can't stick this directly into struct rseq because >> it's all public anyway. > > The motivation for moving this to a different cache line is to handle > the prior comment from Boqun, who is concerned that busy-waiting > repeatedly loading a field from struct rseq will cause false-sharing and > make other stores to that cache line slower, especially stores to > rseq_cs to begin rseq critical sections, thus slightly increasing the > overhead of rseq critical sections taken while mutexes are held. > > If we want to embed this field into struct rseq with its own cache line, > then we need to add a lot of padding, which is inconvenient. > > That being said, perhaps this is premature optimization, what do you think ? Hi Mathieu, Florian, This is exciting! I thought the motivation for moving rseq_sched_state out of struct rseq is lifetime management problem. I assume when a thread locks a mutex, it stores pointer to rseq_sched_state in the mutex state for other threads to poll. So the waiting thread would do something along the following lines: rseq_sched_state* state = __atomic_load_n(mutex->sched_state, __ATOMIC_RELAXED); if (state && !(state->state & RSEQ_SCHED_STATE_FLAG_ON_CPU)) futex_wait(); Now if the state is struct rseq, which is stored in TLS, then the owning thread can unlock the mutex, exit and unmap TLS in between. Consequently, load of state->state will cause a paging fault. And we do want rseq in TLS to save 1 indirection. If rseq_sched_state is separated from struct rseq, then it can be allocated in type stable memory that is never unmapped. What am I missing here? However, if we can store this state in struct rseq, then an alternative interface would for the kernel to do: rseq->cpu_id = -1; to denote that the thread is not running on any CPU. I think it kinda makes sense, rseq->cpu_id is the thread's current CPU, and -1 naturally means "not running at all". And we already store -1 right after init, so it shouldn't be a surprising value.