public inbox for glibc-bugs@sourceware.org
help / color / mirror / Atom feed
* [Bug libc/28991] New: sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible
@ 2022-03-23 0:29 robert at ocallahan dot org
2022-03-23 11:57 ` [Bug libc/28991] " adhemerval.zanella at linaro dot org
` (15 more replies)
0 siblings, 16 replies; 17+ messages in thread
From: robert at ocallahan dot org @ 2022-03-23 0:29 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=28991
Bug ID: 28991
Summary: sysconf(_SC_NPROCESSORS_CONF) should read
/sys/devices/system/cpu/possible
Product: glibc
Version: unspecified
Status: UNCONFIRMED
Severity: normal
Priority: P2
Component: libc
Assignee: unassigned at sourceware dot org
Reporter: robert at ocallahan dot org
CC: drepper.fsp at gmail dot com
Target Milestone: ---
Currently on Linux `__get_nprocs_conf` first tries to enumerate
`/sys/devices/system/cpu`. This enumerates the CPUs that are present in the
system (but possibly offline). However, new CPUs can be added via hotplugging
that were not previously enumerated. This is a problem for applications that
want to allocate an array and index it by CPU ID (e.g. the `cpu_id` field of
`struct rseq`). Currently on Linux the only safe way to allocate such an array
is to read `/sys/devices/system/cpu/possible` to determine the array size. Not
many applications are doing that.
Rather than require applications to do that (which is non-portable as well as
being painful), I think it would make more sense to modify `__get_nprocs_conf`
to try reading `/sys/devices/system/cpu/possible` instead of enumerating the
CPUs.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 17+ messages in thread
* [Bug libc/28991] sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible
2022-03-23 0:29 [Bug libc/28991] New: sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible robert at ocallahan dot org
@ 2022-03-23 11:57 ` adhemerval.zanella at linaro dot org
2022-03-23 12:00 ` adhemerval.zanella at linaro dot org
` (14 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: adhemerval.zanella at linaro dot org @ 2022-03-23 11:57 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=28991
Adhemerval Zanella <adhemerval.zanella at linaro dot org> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |adhemerval.zanella at linaro dot o
| |rg
--- Comment #1 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> ---
I am not sure if we can actually /sys/devices/system/cpu/possible, since for
some architecture it reports possible hotplug cpus that might not actually be
present in the system.
For instance, on my system with a Ryzen 5900x (12c, 24t) I see:
$ sudo dmesg
[...]
[ 0.065499] smpboot: Allowing 32 CPUs, 8 hotplug CPUs
[...]
$ cat /sys/devices/system/cpu/possible
0-31
$ cat /sys/devices/system/cpu/online
0-23
$ cat /sys/devices/system/cpu/offline
24-31
And thus using 'possible' will wrongly report 32 cpus on a system with the
maximum of 24.
The Linux source give us some hints why it is doing it:
arch/x86/kernel/smpboot.c
1513 * Three ways to find out the number of additional hotplug CPUs:
1514 * - If the BIOS specified disabled CPUs in ACPI/mptables use that.
1515 * - The user can overwrite it with possible_cpus=NUM
1516 * - Otherwise don't reserve additional CPUs.
1517 * We do this because additional CPUs waste a lot of memory.
So I am guessing that BIOS in my case it reporting 32 because it supports the
Ryzen 5950x, even thought he system does not actually support a large number of
cpus.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 17+ messages in thread
* [Bug libc/28991] sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible
2022-03-23 0:29 [Bug libc/28991] New: sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible robert at ocallahan dot org
2022-03-23 11:57 ` [Bug libc/28991] " adhemerval.zanella at linaro dot org
@ 2022-03-23 12:00 ` adhemerval.zanella at linaro dot org
2022-03-23 12:52 ` schwab@linux-m68k.org
` (13 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: adhemerval.zanella at linaro dot org @ 2022-03-23 12:00 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=28991
--- Comment #2 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> ---
I see a similar behaviour on a sparc64 machine:
$ cat /sys/devices/system/cpu/possible
0-255
$ cat /sys/devices/system/cpu/online
0-63
$ cat /sys/devices/system/cpu/offline
64-255
So I think reporting the actual presented CPU for __get_nprocs_conf is the
correct semantic here.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 17+ messages in thread
* [Bug libc/28991] sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible
2022-03-23 0:29 [Bug libc/28991] New: sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible robert at ocallahan dot org
2022-03-23 11:57 ` [Bug libc/28991] " adhemerval.zanella at linaro dot org
2022-03-23 12:00 ` adhemerval.zanella at linaro dot org
@ 2022-03-23 12:52 ` schwab@linux-m68k.org
2022-03-23 13:34 ` adhemerval.zanella at linaro dot org
` (12 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: schwab@linux-m68k.org @ 2022-03-23 12:52 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=28991
--- Comment #3 from Andreas Schwab <schwab@linux-m68k.org> ---
But the systems are configured for 32 and 256 cpus, resp.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 17+ messages in thread
* [Bug libc/28991] sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible
2022-03-23 0:29 [Bug libc/28991] New: sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible robert at ocallahan dot org
` (2 preceding siblings ...)
2022-03-23 12:52 ` schwab@linux-m68k.org
@ 2022-03-23 13:34 ` adhemerval.zanella at linaro dot org
2022-03-23 13:43 ` schwab@linux-m68k.org
` (11 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: adhemerval.zanella at linaro dot org @ 2022-03-23 13:34 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=28991
--- Comment #4 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> ---
Does it really make sense to report a large number of cpus even if caller won't
be able to actually activate them? And can you assure changing this interface
won't regress again?
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 17+ messages in thread
* [Bug libc/28991] sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible
2022-03-23 0:29 [Bug libc/28991] New: sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible robert at ocallahan dot org
` (3 preceding siblings ...)
2022-03-23 13:34 ` adhemerval.zanella at linaro dot org
@ 2022-03-23 13:43 ` schwab@linux-m68k.org
2022-03-23 13:48 ` adhemerval.zanella at linaro dot org
` (10 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: schwab@linux-m68k.org @ 2022-03-23 13:43 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=28991
--- Comment #5 from Andreas Schwab <schwab@linux-m68k.org> ---
How do you know that they cannot be activated?
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 17+ messages in thread
* [Bug libc/28991] sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible
2022-03-23 0:29 [Bug libc/28991] New: sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible robert at ocallahan dot org
` (4 preceding siblings ...)
2022-03-23 13:43 ` schwab@linux-m68k.org
@ 2022-03-23 13:48 ` adhemerval.zanella at linaro dot org
2022-03-23 13:49 ` adhemerval.zanella at linaro dot org
` (9 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: adhemerval.zanella at linaro dot org @ 2022-03-23 13:48 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=28991
--- Comment #6 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> ---
Because it is not listed on /sys/devices/system/cpu/cpu*.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 17+ messages in thread
* [Bug libc/28991] sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible
2022-03-23 0:29 [Bug libc/28991] New: sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible robert at ocallahan dot org
` (5 preceding siblings ...)
2022-03-23 13:48 ` adhemerval.zanella at linaro dot org
@ 2022-03-23 13:49 ` adhemerval.zanella at linaro dot org
2022-03-23 14:11 ` schwab@linux-m68k.org
` (8 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: adhemerval.zanella at linaro dot org @ 2022-03-23 13:49 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=28991
--- Comment #7 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> ---
But I am more interested whether it would trigger any regressions, as we
recently hit some in recent changes.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 17+ messages in thread
* [Bug libc/28991] sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible
2022-03-23 0:29 [Bug libc/28991] New: sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible robert at ocallahan dot org
` (6 preceding siblings ...)
2022-03-23 13:49 ` adhemerval.zanella at linaro dot org
@ 2022-03-23 14:11 ` schwab@linux-m68k.org
2022-03-23 14:33 ` adhemerval.zanella at linaro dot org
` (7 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: schwab@linux-m68k.org @ 2022-03-23 14:11 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=28991
--- Comment #8 from Andreas Schwab <schwab@linux-m68k.org> ---
The are not listed because they aren't currently present. But they can be
hotplugged any time (in theory).
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 17+ messages in thread
* [Bug libc/28991] sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible
2022-03-23 0:29 [Bug libc/28991] New: sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible robert at ocallahan dot org
` (7 preceding siblings ...)
2022-03-23 14:11 ` schwab@linux-m68k.org
@ 2022-03-23 14:33 ` adhemerval.zanella at linaro dot org
2022-03-23 14:42 ` schwab@linux-m68k.org
` (6 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: adhemerval.zanella at linaro dot org @ 2022-03-23 14:33 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=28991
--- Comment #9 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> ---
And that is my question: does it make to report the configured cpu number on a
system where hotplug might not be possible? And are the programs that use this
interface aware of it?
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 17+ messages in thread
* [Bug libc/28991] sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible
2022-03-23 0:29 [Bug libc/28991] New: sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible robert at ocallahan dot org
` (8 preceding siblings ...)
2022-03-23 14:33 ` adhemerval.zanella at linaro dot org
@ 2022-03-23 14:42 ` schwab@linux-m68k.org
2022-03-23 14:48 ` adhemerval.zanella at linaro dot org
` (5 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: schwab@linux-m68k.org @ 2022-03-23 14:42 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=28991
--- Comment #10 from Andreas Schwab <schwab@linux-m68k.org> ---
How do you know if hotplug is not possible?
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 17+ messages in thread
* [Bug libc/28991] sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible
2022-03-23 0:29 [Bug libc/28991] New: sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible robert at ocallahan dot org
` (9 preceding siblings ...)
2022-03-23 14:42 ` schwab@linux-m68k.org
@ 2022-03-23 14:48 ` adhemerval.zanella at linaro dot org
2022-03-23 15:15 ` schwab@linux-m68k.org
` (4 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: adhemerval.zanella at linaro dot org @ 2022-03-23 14:48 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=28991
--- Comment #11 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> ---
That's why I put 'might not be possible'. Do you think we should make this
change?
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 17+ messages in thread
* [Bug libc/28991] sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible
2022-03-23 0:29 [Bug libc/28991] New: sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible robert at ocallahan dot org
` (10 preceding siblings ...)
2022-03-23 14:48 ` adhemerval.zanella at linaro dot org
@ 2022-03-23 15:15 ` schwab@linux-m68k.org
2022-03-23 18:43 ` robert at ocallahan dot org
` (3 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: schwab@linux-m68k.org @ 2022-03-23 15:15 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=28991
--- Comment #12 from Andreas Schwab <schwab@linux-m68k.org> ---
The same question: how do you know that? We can only use the information that
the system provides.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 17+ messages in thread
* [Bug libc/28991] sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible
2022-03-23 0:29 [Bug libc/28991] New: sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible robert at ocallahan dot org
` (11 preceding siblings ...)
2022-03-23 15:15 ` schwab@linux-m68k.org
@ 2022-03-23 18:43 ` robert at ocallahan dot org
2022-03-23 18:52 ` robert at ocallahan dot org
` (2 subsequent siblings)
15 siblings, 0 replies; 17+ messages in thread
From: robert at ocallahan dot org @ 2022-03-23 18:43 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=28991
--- Comment #13 from Robert O'Callahan <robert at ocallahan dot org> ---
Applications that allocate an array indexed by rseq cpu_id or getcpu() or
similar need to size it to accommodate all possible CPUs. On Linux they have to
use /sys/devices/system/cpu/possible, regardless of how accurate that is. (I
don't think there's any better source for this information, and if there is,
the kernel should use it to make /sys/devices/system/cpu/possible more
accurate.)
It's not a big problem if the number of CPUs from
/sys/devices/system/cpu/possible is larger than necessary. That will only waste
a small amount of memory. The kernel internally is already sizing its per-cpu
structures based on cpu_possible_mask.
The only real question is what sysconf(_SC_NPROCESSORS_CONF) should actually
mean. If it's intended to be an upper bound for CPU IDs, then it needs to use
/sys/devices/system/cpu/possible. If it isn't, then I don't know what it's
useful for, but at least it should have a big warning that it is not an upper
bound for CPU IDs.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 17+ messages in thread
* [Bug libc/28991] sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible
2022-03-23 0:29 [Bug libc/28991] New: sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible robert at ocallahan dot org
` (12 preceding siblings ...)
2022-03-23 18:43 ` robert at ocallahan dot org
@ 2022-03-23 18:52 ` robert at ocallahan dot org
2022-05-16 17:30 ` adhemerval.zanella at linaro dot org
2022-05-16 17:36 ` adhemerval.zanella at linaro dot org
15 siblings, 0 replies; 17+ messages in thread
From: robert at ocallahan dot org @ 2022-03-23 18:52 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=28991
--- Comment #14 from Robert O'Callahan <robert at ocallahan dot org> ---
And if you choose to not use /sys/devices/system/cpu/possible for
_SC_NPROCESSORS_CONF, then someone what _SC_NPROCESSORS_CONF is useful for,
because I have no idea.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 17+ messages in thread
* [Bug libc/28991] sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible
2022-03-23 0:29 [Bug libc/28991] New: sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible robert at ocallahan dot org
` (13 preceding siblings ...)
2022-03-23 18:52 ` robert at ocallahan dot org
@ 2022-05-16 17:30 ` adhemerval.zanella at linaro dot org
2022-05-16 17:36 ` adhemerval.zanella at linaro dot org
15 siblings, 0 replies; 17+ messages in thread
From: adhemerval.zanella at linaro dot org @ 2022-05-16 17:30 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=28991
Adhemerval Zanella <adhemerval.zanella at linaro dot org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Last reconfirmed| |2022-05-16
Assignee|unassigned at sourceware dot org |adhemerval.zanella at linaro dot o
| |rg
Ever confirmed|0 |1
Status|UNCONFIRMED |NEW
Target Milestone|--- |2.36
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 17+ messages in thread
* [Bug libc/28991] sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible
2022-03-23 0:29 [Bug libc/28991] New: sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible robert at ocallahan dot org
` (14 preceding siblings ...)
2022-05-16 17:30 ` adhemerval.zanella at linaro dot org
@ 2022-05-16 17:36 ` adhemerval.zanella at linaro dot org
15 siblings, 0 replies; 17+ messages in thread
From: adhemerval.zanella at linaro dot org @ 2022-05-16 17:36 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=28991
Adhemerval Zanella <adhemerval.zanella at linaro dot org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution|--- |FIXED
--- Comment #15 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> ---
Fixed on 2.36.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2022-05-16 17:36 UTC | newest]
Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-03-23 0:29 [Bug libc/28991] New: sysconf(_SC_NPROCESSORS_CONF) should read /sys/devices/system/cpu/possible robert at ocallahan dot org
2022-03-23 11:57 ` [Bug libc/28991] " adhemerval.zanella at linaro dot org
2022-03-23 12:00 ` adhemerval.zanella at linaro dot org
2022-03-23 12:52 ` schwab@linux-m68k.org
2022-03-23 13:34 ` adhemerval.zanella at linaro dot org
2022-03-23 13:43 ` schwab@linux-m68k.org
2022-03-23 13:48 ` adhemerval.zanella at linaro dot org
2022-03-23 13:49 ` adhemerval.zanella at linaro dot org
2022-03-23 14:11 ` schwab@linux-m68k.org
2022-03-23 14:33 ` adhemerval.zanella at linaro dot org
2022-03-23 14:42 ` schwab@linux-m68k.org
2022-03-23 14:48 ` adhemerval.zanella at linaro dot org
2022-03-23 15:15 ` schwab@linux-m68k.org
2022-03-23 18:43 ` robert at ocallahan dot org
2022-03-23 18:52 ` robert at ocallahan dot org
2022-05-16 17:30 ` adhemerval.zanella at linaro dot org
2022-05-16 17:36 ` adhemerval.zanella at linaro dot org
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).