public inbox for glibc-bugs@sourceware.org
help / color / mirror / Atom feed
* [Bug malloc/30723] New: posix_memalign performance regression
@ 2023-08-05 2:37 toolybird at tuta dot io
2023-08-05 3:02 ` [Bug malloc/30723] posix_memalign performance regression in glibc-2.38 (since 24cdd6c71debfd10a9f7cb217fe2a2c4c486ed6f?) sam at gentoo dot org
` (12 more replies)
0 siblings, 13 replies; 14+ messages in thread
From: toolybird at tuta dot io @ 2023-08-05 2:37 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=30723
Bug ID: 30723
Summary: posix_memalign performance regression
Product: glibc
Version: 2.38
Status: UNCONFIRMED
Severity: normal
Priority: P2
Component: malloc
Assignee: unassigned at sourceware dot org
Reporter: toolybird at tuta dot io
Target Milestone: ---
Already reported on the glibc list [1] but creating a ticket here for tracking
purposes. This is currently breaking ffmpeg based apps [2]
[1] https://sourceware.org/pipermail/libc-alpha/2023-August/150653.html
[2] https://bugs.archlinux.org/task/79300
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Bug malloc/30723] posix_memalign performance regression in glibc-2.38 (since 24cdd6c71debfd10a9f7cb217fe2a2c4c486ed6f?)
2023-08-05 2:37 [Bug malloc/30723] New: posix_memalign performance regression toolybird at tuta dot io
@ 2023-08-05 3:02 ` sam at gentoo dot org
2023-08-05 3:04 ` sam at gentoo dot org
` (11 subsequent siblings)
12 siblings, 0 replies; 14+ messages in thread
From: sam at gentoo dot org @ 2023-08-05 3:02 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=30723
Sam James <sam at gentoo dot org> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |dj at redhat dot com,
| |sam at gentoo dot org,
| |xry111 at xry111 dot site
Summary|posix_memalign performance |posix_memalign performance
|regression |regression in glibc-2.38
| |(since
| |24cdd6c71debfd10a9f7cb217fe
| |2a2c4c486ed6f?)
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Bug malloc/30723] posix_memalign performance regression in glibc-2.38 (since 24cdd6c71debfd10a9f7cb217fe2a2c4c486ed6f?)
2023-08-05 2:37 [Bug malloc/30723] New: posix_memalign performance regression toolybird at tuta dot io
2023-08-05 3:02 ` [Bug malloc/30723] posix_memalign performance regression in glibc-2.38 (since 24cdd6c71debfd10a9f7cb217fe2a2c4c486ed6f?) sam at gentoo dot org
@ 2023-08-05 3:04 ` sam at gentoo dot org
2023-08-05 13:13 ` freswa at archlinux dot org
` (10 subsequent siblings)
12 siblings, 0 replies; 14+ messages in thread
From: sam at gentoo dot org @ 2023-08-05 3:04 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=30723
Sam James <sam at gentoo dot org> changed:
What |Removed |Added
----------------------------------------------------------------------------
See Also| |https://github.com/mpv-play
| |er/mpv/issues/12076
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Bug malloc/30723] posix_memalign performance regression in glibc-2.38 (since 24cdd6c71debfd10a9f7cb217fe2a2c4c486ed6f?)
2023-08-05 2:37 [Bug malloc/30723] New: posix_memalign performance regression toolybird at tuta dot io
2023-08-05 3:02 ` [Bug malloc/30723] posix_memalign performance regression in glibc-2.38 (since 24cdd6c71debfd10a9f7cb217fe2a2c4c486ed6f?) sam at gentoo dot org
2023-08-05 3:04 ` sam at gentoo dot org
@ 2023-08-05 13:13 ` freswa at archlinux dot org
2023-08-05 21:54 ` dilfridge at gentoo dot org
` (9 subsequent siblings)
12 siblings, 0 replies; 14+ messages in thread
From: freswa at archlinux dot org @ 2023-08-05 13:13 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=30723
Frederik Schwan <freswa at archlinux dot org> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |freswa at archlinux dot org
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Bug malloc/30723] posix_memalign performance regression in glibc-2.38 (since 24cdd6c71debfd10a9f7cb217fe2a2c4c486ed6f?)
2023-08-05 2:37 [Bug malloc/30723] New: posix_memalign performance regression toolybird at tuta dot io
` (2 preceding siblings ...)
2023-08-05 13:13 ` freswa at archlinux dot org
@ 2023-08-05 21:54 ` dilfridge at gentoo dot org
2023-08-06 18:23 ` kocelfc at tutanota dot com
` (8 subsequent siblings)
12 siblings, 0 replies; 14+ messages in thread
From: dilfridge at gentoo dot org @ 2023-08-05 21:54 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=30723
Andreas K. Huettel <dilfridge at gentoo dot org> changed:
What |Removed |Added
----------------------------------------------------------------------------
URL| |https://bugs.archlinux.org/
| |task/79300
CC| |dilfridge at gentoo dot org
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Bug malloc/30723] posix_memalign performance regression in glibc-2.38 (since 24cdd6c71debfd10a9f7cb217fe2a2c4c486ed6f?)
2023-08-05 2:37 [Bug malloc/30723] New: posix_memalign performance regression toolybird at tuta dot io
` (3 preceding siblings ...)
2023-08-05 21:54 ` dilfridge at gentoo dot org
@ 2023-08-06 18:23 ` kocelfc at tutanota dot com
2023-08-09 18:54 ` [Bug malloc/30723] Repeated calls posix_memalign calls produce long free lists, high fragmentation fweimer at redhat dot com
` (7 subsequent siblings)
12 siblings, 0 replies; 14+ messages in thread
From: kocelfc at tutanota dot com @ 2023-08-06 18:23 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=30723
Kostadin Shishmanov <kocelfc at tutanota dot com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |kocelfc at tutanota dot com
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Bug malloc/30723] Repeated calls posix_memalign calls produce long free lists, high fragmentation
2023-08-05 2:37 [Bug malloc/30723] New: posix_memalign performance regression toolybird at tuta dot io
` (4 preceding siblings ...)
2023-08-06 18:23 ` kocelfc at tutanota dot com
@ 2023-08-09 18:54 ` fweimer at redhat dot com
2023-08-09 19:37 ` [Bug malloc/30723] Repeated " fweimer at redhat dot com
` (6 subsequent siblings)
12 siblings, 0 replies; 14+ messages in thread
From: fweimer at redhat dot com @ 2023-08-09 18:54 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=30723
Florian Weimer <fweimer at redhat dot com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|UNCONFIRMED |ASSIGNED
Summary|posix_memalign performance |Repeated calls
|regression in glibc-2.38 |posix_memalign calls
|(since |produce long free lists,
|24cdd6c71debfd10a9f7cb217fe |high fragmentation
|2a2c4c486ed6f?) |
Ever confirmed|0 |1
Last reconfirmed| |2023-08-09
CC| |fweimer at redhat dot com
Assignee|unassigned at sourceware dot org |fweimer at redhat dot com
--- Comment #1 from Florian Weimer <fweimer at redhat dot com> ---
Patch posted:
[PATCH] malloc: Enable merging of remainders in memalign (bug 30723)
<https://inbox.sourceware.org/libc-alpha/877cq4yrpg.fsf@oldenburg.str.redhat.com/T/>
We may need additional heuristics to bound the free list scanning.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Bug malloc/30723] Repeated posix_memalign calls produce long free lists, high fragmentation
2023-08-05 2:37 [Bug malloc/30723] New: posix_memalign performance regression toolybird at tuta dot io
` (5 preceding siblings ...)
2023-08-09 18:54 ` [Bug malloc/30723] Repeated calls posix_memalign calls produce long free lists, high fragmentation fweimer at redhat dot com
@ 2023-08-09 19:37 ` fweimer at redhat dot com
2023-08-10 5:45 ` toolybird at tuta dot io
` (5 subsequent siblings)
12 siblings, 0 replies; 14+ messages in thread
From: fweimer at redhat dot com @ 2023-08-09 19:37 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=30723
Florian Weimer <fweimer at redhat dot com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Summary|Repeated calls |Repeated posix_memalign
|posix_memalign calls |calls produce long free
|produce long free lists, |lists, high fragmentation
|high fragmentation |
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Bug malloc/30723] Repeated posix_memalign calls produce long free lists, high fragmentation
2023-08-05 2:37 [Bug malloc/30723] New: posix_memalign performance regression toolybird at tuta dot io
` (6 preceding siblings ...)
2023-08-09 19:37 ` [Bug malloc/30723] Repeated " fweimer at redhat dot com
@ 2023-08-10 5:45 ` toolybird at tuta dot io
2023-08-11 6:11 ` toolybird at tuta dot io
` (4 subsequent siblings)
12 siblings, 0 replies; 14+ messages in thread
From: toolybird at tuta dot io @ 2023-08-10 5:45 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=30723
--- Comment #2 from Toolybird <toolybird at tuta dot io> ---
Hey thanks for the patch! Unfortunately multiple testers in the downstream Arch
ticket are still getting 100% CPU when using the "mpv" app. Please see this
comment [1] for a supposed reproducer (I haven't tested it yet...my role is
downstream bug wrangler, so I'm just the message relay here currently..)
[1] https://bugs.archlinux.org/task/79300#comment220881
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Bug malloc/30723] Repeated posix_memalign calls produce long free lists, high fragmentation
2023-08-05 2:37 [Bug malloc/30723] New: posix_memalign performance regression toolybird at tuta dot io
` (7 preceding siblings ...)
2023-08-10 5:45 ` toolybird at tuta dot io
@ 2023-08-11 6:11 ` toolybird at tuta dot io
2023-08-11 9:24 ` fweimer at redhat dot com
` (3 subsequent siblings)
12 siblings, 0 replies; 14+ messages in thread
From: toolybird at tuta dot io @ 2023-08-11 6:11 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=30723
--- Comment #3 from Toolybird <toolybird at tuta dot io> ---
Latest patch [1] on top of the previous one appears to have done the trick.
Thanks so much for all your efforts. We will get this out to users once it hits
the tree (and hopefully the 2.38 branch).
[1]
https://inbox.sourceware.org/libc-alpha/87pm3uajev.fsf@oldenburg.str.redhat.com/
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Bug malloc/30723] Repeated posix_memalign calls produce long free lists, high fragmentation
2023-08-05 2:37 [Bug malloc/30723] New: posix_memalign performance regression toolybird at tuta dot io
` (8 preceding siblings ...)
2023-08-11 6:11 ` toolybird at tuta dot io
@ 2023-08-11 9:24 ` fweimer at redhat dot com
2023-08-15 6:25 ` fweimer at redhat dot com
` (2 subsequent siblings)
12 siblings, 0 replies; 14+ messages in thread
From: fweimer at redhat dot com @ 2023-08-11 9:24 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=30723
--- Comment #4 from Florian Weimer <fweimer at redhat dot com> ---
First part committed:
commit 542b1105852568c3ebc712225ae78b8c8ba31a78
Author: Florian Weimer <fweimer@redhat.com>
Date: Fri Aug 11 11:18:17 2023 +0200
malloc: Enable merging of remainders in memalign (bug 30723)
Previously, calling _int_free from _int_memalign could put remainders
into the tcache or into fastbins, where they are invisible to the
low-level allocator. This results in missed merge opportunities
because once these freed chunks become available to the low-level
allocator, further memalign allocations (even of the same size are)
likely obstructing merges.
Furthermore, during forwards merging in _int_memalign, do not
completely give up when the remainder is too small to serve as a
chunk on its own. We can still give it back if it can be merged
with the following unused chunk. This makes it more likely that
memalign calls in a loop achieve a compact memory layout,
independently of initial heap layout.
Drop some useless (unsigned long) casts along the way, and tweak
the style to more closely match GNU on changed lines.
Reviewed-by: DJ Delorie <dj@redhat.com>
Second part is still under review, I need to send a v2.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Bug malloc/30723] Repeated posix_memalign calls produce long free lists, high fragmentation
2023-08-05 2:37 [Bug malloc/30723] New: posix_memalign performance regression toolybird at tuta dot io
` (9 preceding siblings ...)
2023-08-11 9:24 ` fweimer at redhat dot com
@ 2023-08-15 6:25 ` fweimer at redhat dot com
2023-09-11 8:02 ` [Bug malloc/30723] Poor posix_memalign performance with long free lists fweimer at redhat dot com
2023-09-11 8:03 ` fweimer at redhat dot com
12 siblings, 0 replies; 14+ messages in thread
From: fweimer at redhat dot com @ 2023-08-15 6:25 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=30723
--- Comment #5 from Florian Weimer <fweimer at redhat dot com> ---
Second part committed:
commit 0dc7fc1cf094406a138e4d1bcf9553e59edcf89d
Author: Florian Weimer <fweimer@redhat.com>
Date: Thu Aug 10 19:36:56 2023 +0200
malloc: Remove bin scanning from memalign (bug 30723)
On the test workload (mpv --cache=yes with VP9 video decoding), the
bin scanning has a very poor success rate (less than 2%). The tcache
scanning has about 50% success rate, so keep that.
Update comments in malloc/tst-memalign-2 to indicate the purpose
of the tests. Even with the scanning removed, the additional
merging opportunities since commit 542b1105852568c3ebc712225ae78b
("malloc: Enable merging of remainders in memalign (bug 30723)")
are sufficient to pass the existing large bins test.
Remove leftover variables from _int_free from refactoring in the
same commit.
Reviewed-by: DJ Delorie <dj@redhat.com>
I will handle the 2.38 backport after a brief delay.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Bug malloc/30723] Poor posix_memalign performance with long free lists
2023-08-05 2:37 [Bug malloc/30723] New: posix_memalign performance regression toolybird at tuta dot io
` (10 preceding siblings ...)
2023-08-15 6:25 ` fweimer at redhat dot com
@ 2023-09-11 8:02 ` fweimer at redhat dot com
2023-09-11 8:03 ` fweimer at redhat dot com
12 siblings, 0 replies; 14+ messages in thread
From: fweimer at redhat dot com @ 2023-09-11 8:02 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=30723
Florian Weimer <fweimer at redhat dot com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Summary|Repeated posix_memalign |Poor posix_memalign
|calls produce long free |performance with long free
|lists, high fragmentation |lists
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Bug malloc/30723] Poor posix_memalign performance with long free lists
2023-08-05 2:37 [Bug malloc/30723] New: posix_memalign performance regression toolybird at tuta dot io
` (11 preceding siblings ...)
2023-09-11 8:02 ` [Bug malloc/30723] Poor posix_memalign performance with long free lists fweimer at redhat dot com
@ 2023-09-11 8:03 ` fweimer at redhat dot com
12 siblings, 0 replies; 14+ messages in thread
From: fweimer at redhat dot com @ 2023-09-11 8:03 UTC (permalink / raw)
To: glibc-bugs
https://sourceware.org/bugzilla/show_bug.cgi?id=30723
Florian Weimer <fweimer at redhat dot com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Resolution|--- |FIXED
Status|ASSIGNED |RESOLVED
--- Comment #6 from Florian Weimer <fweimer at redhat dot com> ---
The glibc 2.38 release branch has been fixed.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2023-09-11 8:03 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-08-05 2:37 [Bug malloc/30723] New: posix_memalign performance regression toolybird at tuta dot io
2023-08-05 3:02 ` [Bug malloc/30723] posix_memalign performance regression in glibc-2.38 (since 24cdd6c71debfd10a9f7cb217fe2a2c4c486ed6f?) sam at gentoo dot org
2023-08-05 3:04 ` sam at gentoo dot org
2023-08-05 13:13 ` freswa at archlinux dot org
2023-08-05 21:54 ` dilfridge at gentoo dot org
2023-08-06 18:23 ` kocelfc at tutanota dot com
2023-08-09 18:54 ` [Bug malloc/30723] Repeated calls posix_memalign calls produce long free lists, high fragmentation fweimer at redhat dot com
2023-08-09 19:37 ` [Bug malloc/30723] Repeated " fweimer at redhat dot com
2023-08-10 5:45 ` toolybird at tuta dot io
2023-08-11 6:11 ` toolybird at tuta dot io
2023-08-11 9:24 ` fweimer at redhat dot com
2023-08-15 6:25 ` fweimer at redhat dot com
2023-09-11 8:02 ` [Bug malloc/30723] Poor posix_memalign performance with long free lists fweimer at redhat dot com
2023-09-11 8:03 ` fweimer at redhat dot com
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).