* [PATCH] Reduce number of mmap calls from __libc_memalign in ld.so
@ 2016-04-02 15:34 H.J. Lu
2016-04-02 17:33 ` Mike Frysinger
0 siblings, 1 reply; 7+ messages in thread
From: H.J. Lu @ 2016-04-02 15:34 UTC (permalink / raw)
To: GNU C Library
__libc_memalign in ld.so allocates one page at a time and tries to
optimize consecutive __libc_memalign calls by hoping that the next
mmap is after the current memory allocation.
However, the kernel hands out mmap addresses in top-down order, so
this optimization in practice never happens, with the result that we
have more mmap calls and waste a bunch of space for each __libc_memalign.
This change makes __libc_memalign to mmap one page extra. Worst case,
the kernel never puts a backing page behind it, but best case it allows
__libc_memalign to operate much much better. For elf/tst-align --direct,
it reduces number of mmap calls from 12 to 9.
Tested on x86-64. OK for master?
H.J.
---
* elf/dl-minimal.c (__libc_memalign): Mmap one extra page.
---
elf/dl-minimal.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/elf/dl-minimal.c b/elf/dl-minimal.c
index 762e65b..d6f87f1 100644
--- a/elf/dl-minimal.c
+++ b/elf/dl-minimal.c
@@ -75,6 +75,7 @@ __libc_memalign (size_t align, size_t n)
return NULL;
nup = GLRO(dl_pagesize);
}
+ nup += GLRO(dl_pagesize);
page = __mmap (0, nup, PROT_READ|PROT_WRITE,
MAP_ANON|MAP_PRIVATE, -1, 0);
if (page == MAP_FAILED)
--
2.5.5
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] Reduce number of mmap calls from __libc_memalign in ld.so
2016-04-02 15:34 [PATCH] Reduce number of mmap calls from __libc_memalign in ld.so H.J. Lu
@ 2016-04-02 17:33 ` Mike Frysinger
2016-04-02 17:43 ` H.J. Lu
0 siblings, 1 reply; 7+ messages in thread
From: Mike Frysinger @ 2016-04-02 17:33 UTC (permalink / raw)
To: H.J. Lu; +Cc: GNU C Library
[-- Attachment #1: Type: text/plain, Size: 1038 bytes --]
On 02 Apr 2016 08:34, H.J. Lu wrote:
> __libc_memalign in ld.so allocates one page at a time and tries to
> optimize consecutive __libc_memalign calls by hoping that the next
> mmap is after the current memory allocation.
>
> However, the kernel hands out mmap addresses in top-down order, so
> this optimization in practice never happens, with the result that we
> have more mmap calls and waste a bunch of space for each __libc_memalign.
>
> This change makes __libc_memalign to mmap one page extra. Worst case,
> the kernel never puts a backing page behind it, but best case it allows
> __libc_memalign to operate much much better. For elf/tst-align --direct,
> it reduces number of mmap calls from 12 to 9.
>
> --- a/elf/dl-minimal.c
> +++ b/elf/dl-minimal.c
> @@ -75,6 +75,7 @@ __libc_memalign (size_t align, size_t n)
> return NULL;
> nup = GLRO(dl_pagesize);
> }
> + nup += GLRO(dl_pagesize);
should this be in the else case ?
also the comment above this code needs updating
-mike
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] Reduce number of mmap calls from __libc_memalign in ld.so
2016-04-02 17:33 ` Mike Frysinger
@ 2016-04-02 17:43 ` H.J. Lu
2016-04-02 22:41 ` H.J. Lu
0 siblings, 1 reply; 7+ messages in thread
From: H.J. Lu @ 2016-04-02 17:43 UTC (permalink / raw)
To: GNU C Library
[-- Attachment #1: Type: text/plain, Size: 1180 bytes --]
On Sat, Apr 2, 2016 at 10:33 AM, Mike Frysinger <vapier@gentoo.org> wrote:
> On 02 Apr 2016 08:34, H.J. Lu wrote:
>> __libc_memalign in ld.so allocates one page at a time and tries to
>> optimize consecutive __libc_memalign calls by hoping that the next
>> mmap is after the current memory allocation.
>>
>> However, the kernel hands out mmap addresses in top-down order, so
>> this optimization in practice never happens, with the result that we
>> have more mmap calls and waste a bunch of space for each __libc_memalign.
>>
>> This change makes __libc_memalign to mmap one page extra. Worst case,
>> the kernel never puts a backing page behind it, but best case it allows
>> __libc_memalign to operate much much better. For elf/tst-align --direct,
>> it reduces number of mmap calls from 12 to 9.
>>
>> --- a/elf/dl-minimal.c
>> +++ b/elf/dl-minimal.c
>> @@ -75,6 +75,7 @@ __libc_memalign (size_t align, size_t n)
>> return NULL;
>> nup = GLRO(dl_pagesize);
>> }
>> + nup += GLRO(dl_pagesize);
>
> should this be in the else case ?
>
> also the comment above this code needs updating
> -mike
You are right. Here is the updated patch.
--
H.J.
[-- Attachment #2: 0001-Reduce-number-of-mmap-calls-from-__libc_memalign-in-.patch --]
[-- Type: text/x-patch, Size: 1875 bytes --]
From d56ca4f3269e47cba3e8d22ba8e48cd20d470757 Mon Sep 17 00:00:00 2001
From: "H.J. Lu" <hjl.tools@gmail.com>
Date: Sat, 2 Apr 2016 08:25:31 -0700
Subject: [PATCH] Reduce number of mmap calls from __libc_memalign in ld.so
__libc_memalign in ld.so allocates one page at a time and tries to
optimize consecutive __libc_memalign calls by hoping that the next
mmap is after the current memory allocation.
However, the kernel hands out mmap addresses in top-down order, so
this optimization in practice never happens, with the result that we
have more mmap calls and waste a bunch of space for each __libc_memalign.
This change makes __libc_memalign to mmap one page extra. Worst case,
the kernel never puts a backing page behind it, but best case it allows
__libc_memalign to operate much much better. For elf/tst-align --direct,
it reduces number of mmap calls from 12 to 9.
* elf/dl-minimal.c (__libc_memalign): Mmap one extra page.
---
elf/dl-minimal.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/elf/dl-minimal.c b/elf/dl-minimal.c
index 762e65b..8bffdc7 100644
--- a/elf/dl-minimal.c
+++ b/elf/dl-minimal.c
@@ -66,7 +66,8 @@ __libc_memalign (size_t align, size_t n)
if (alloc_ptr + n >= alloc_end || n >= -(uintptr_t) alloc_ptr)
{
- /* Insufficient space left; allocate another page. */
+ /* Insufficient space left; allocate another page plus one extra
+ page to reduce number of mmap calls. */
caddr_t page;
size_t nup = (n + GLRO(dl_pagesize) - 1) & ~(GLRO(dl_pagesize) - 1);
if (__glibc_unlikely (nup == 0))
@@ -75,6 +76,8 @@ __libc_memalign (size_t align, size_t n)
return NULL;
nup = GLRO(dl_pagesize);
}
+ else
+ nup += GLRO(dl_pagesize);
page = __mmap (0, nup, PROT_READ|PROT_WRITE,
MAP_ANON|MAP_PRIVATE, -1, 0);
if (page == MAP_FAILED)
--
2.5.5
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] Reduce number of mmap calls from __libc_memalign in ld.so
2016-04-02 17:43 ` H.J. Lu
@ 2016-04-02 22:41 ` H.J. Lu
2016-04-03 6:56 ` Andreas Schwab
0 siblings, 1 reply; 7+ messages in thread
From: H.J. Lu @ 2016-04-02 22:41 UTC (permalink / raw)
To: GNU C Library
[-- Attachment #1: Type: text/plain, Size: 1337 bytes --]
On Sat, Apr 2, 2016 at 10:43 AM, H.J. Lu <hjl.tools@gmail.com> wrote:
> On Sat, Apr 2, 2016 at 10:33 AM, Mike Frysinger <vapier@gentoo.org> wrote:
>> On 02 Apr 2016 08:34, H.J. Lu wrote:
>>> __libc_memalign in ld.so allocates one page at a time and tries to
>>> optimize consecutive __libc_memalign calls by hoping that the next
>>> mmap is after the current memory allocation.
>>>
>>> However, the kernel hands out mmap addresses in top-down order, so
>>> this optimization in practice never happens, with the result that we
>>> have more mmap calls and waste a bunch of space for each __libc_memalign.
>>>
>>> This change makes __libc_memalign to mmap one page extra. Worst case,
>>> the kernel never puts a backing page behind it, but best case it allows
>>> __libc_memalign to operate much much better. For elf/tst-align --direct,
>>> it reduces number of mmap calls from 12 to 9.
>>>
>>> --- a/elf/dl-minimal.c
>>> +++ b/elf/dl-minimal.c
>>> @@ -75,6 +75,7 @@ __libc_memalign (size_t align, size_t n)
>>> return NULL;
>>> nup = GLRO(dl_pagesize);
>>> }
>>> + nup += GLRO(dl_pagesize);
>>
>> should this be in the else case ?
>>
>> also the comment above this code needs updating
>> -mike
>
> You are right. Here is the updated patch.
>
We can just always increment number of pages by one.
--
H.J.
[-- Attachment #2: 0001-Reduce-number-of-mmap-calls-from-__libc_memalign-in-.patch --]
[-- Type: text/x-patch, Size: 1895 bytes --]
From 4aad224c5dc8c8e8496868cc1bb00d587aa4f1ed Mon Sep 17 00:00:00 2001
From: "H.J. Lu" <hjl.tools@gmail.com>
Date: Sat, 2 Apr 2016 08:25:31 -0700
Subject: [PATCH] Reduce number of mmap calls from __libc_memalign in ld.so
__libc_memalign in ld.so allocates one page at a time and tries to
optimize consecutive __libc_memalign calls by hoping that the next
mmap is after the current memory allocation.
However, the kernel hands out mmap addresses in top-down order, so
this optimization in practice never happens, with the result that we
have more mmap calls and waste a bunch of space for each __libc_memalign.
This change makes __libc_memalign to mmap one page extra. Worst case,
the kernel never puts a backing page behind it, but best case it allows
__libc_memalign to operate much much better. For elf/tst-align --direct,
it reduces number of mmap calls from 12 to 9.
* elf/dl-minimal.c (__libc_memalign): Mmap one extra page.
---
elf/dl-minimal.c | 12 +++++-------
1 file changed, 5 insertions(+), 7 deletions(-)
diff --git a/elf/dl-minimal.c b/elf/dl-minimal.c
index 762e65b..0660fcf 100644
--- a/elf/dl-minimal.c
+++ b/elf/dl-minimal.c
@@ -66,15 +66,13 @@ __libc_memalign (size_t align, size_t n)
if (alloc_ptr + n >= alloc_end || n >= -(uintptr_t) alloc_ptr)
{
- /* Insufficient space left; allocate another page. */
+ /* Insufficient space left; allocate another page plus one extra
+ page to reduce number of mmap calls. */
caddr_t page;
size_t nup = (n + GLRO(dl_pagesize) - 1) & ~(GLRO(dl_pagesize) - 1);
- if (__glibc_unlikely (nup == 0))
- {
- if (n)
- return NULL;
- nup = GLRO(dl_pagesize);
- }
+ if (__glibc_unlikely (nup == 0 && n))
+ return NULL;
+ nup += GLRO(dl_pagesize);
page = __mmap (0, nup, PROT_READ|PROT_WRITE,
MAP_ANON|MAP_PRIVATE, -1, 0);
if (page == MAP_FAILED)
--
2.5.5
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] Reduce number of mmap calls from __libc_memalign in ld.so
2016-04-02 22:41 ` H.J. Lu
@ 2016-04-03 6:56 ` Andreas Schwab
2016-04-03 13:42 ` H.J. Lu
0 siblings, 1 reply; 7+ messages in thread
From: Andreas Schwab @ 2016-04-03 6:56 UTC (permalink / raw)
To: H.J. Lu; +Cc: GNU C Library
"H.J. Lu" <hjl.tools@gmail.com> writes:
> + if (__glibc_unlikely (nup == 0 && n))
Please also fix the implicit boolean coercion.
Andreas.
--
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5
"And now for something completely different."
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] Reduce number of mmap calls from __libc_memalign in ld.so
2016-04-03 6:56 ` Andreas Schwab
@ 2016-04-03 13:42 ` H.J. Lu
2016-04-23 12:57 ` H.J. Lu
0 siblings, 1 reply; 7+ messages in thread
From: H.J. Lu @ 2016-04-03 13:42 UTC (permalink / raw)
To: Andreas Schwab; +Cc: GNU C Library
[-- Attachment #1: Type: text/plain, Size: 275 bytes --]
On Sat, Apr 2, 2016 at 11:55 PM, Andreas Schwab <schwab@linux-m68k.org> wrote:
> "H.J. Lu" <hjl.tools@gmail.com> writes:
>
>> + if (__glibc_unlikely (nup == 0 && n))
>
> Please also fix the implicit boolean coercion.
>
> Andreas.
>
Like this? OK for master?
--
H.J.
[-- Attachment #2: 0001-Reduce-number-of-mmap-calls-from-__libc_memalign-in-.patch --]
[-- Type: application/octet-stream, Size: 1900 bytes --]
From 5d673379a907611be18ebd48a16a2df7b3ab3f9c Mon Sep 17 00:00:00 2001
From: "H.J. Lu" <hjl.tools@gmail.com>
Date: Sat, 2 Apr 2016 08:25:31 -0700
Subject: [PATCH] Reduce number of mmap calls from __libc_memalign in ld.so
__libc_memalign in ld.so allocates one page at a time and tries to
optimize consecutive __libc_memalign calls by hoping that the next
mmap is after the current memory allocation.
However, the kernel hands out mmap addresses in top-down order, so
this optimization in practice never happens, with the result that we
have more mmap calls and waste a bunch of space for each __libc_memalign.
This change makes __libc_memalign to mmap one page extra. Worst case,
the kernel never puts a backing page behind it, but best case it allows
__libc_memalign to operate much much better. For elf/tst-align --direct,
it reduces number of mmap calls from 12 to 9.
* elf/dl-minimal.c (__libc_memalign): Mmap one extra page.
---
elf/dl-minimal.c | 12 +++++-------
1 file changed, 5 insertions(+), 7 deletions(-)
diff --git a/elf/dl-minimal.c b/elf/dl-minimal.c
index 762e65b..c8a8f8d 100644
--- a/elf/dl-minimal.c
+++ b/elf/dl-minimal.c
@@ -66,15 +66,13 @@ __libc_memalign (size_t align, size_t n)
if (alloc_ptr + n >= alloc_end || n >= -(uintptr_t) alloc_ptr)
{
- /* Insufficient space left; allocate another page. */
+ /* Insufficient space left; allocate another page plus one extra
+ page to reduce number of mmap calls. */
caddr_t page;
size_t nup = (n + GLRO(dl_pagesize) - 1) & ~(GLRO(dl_pagesize) - 1);
- if (__glibc_unlikely (nup == 0))
- {
- if (n)
- return NULL;
- nup = GLRO(dl_pagesize);
- }
+ if (__glibc_unlikely (nup == 0 && n != 0))
+ return NULL;
+ nup += GLRO(dl_pagesize);
page = __mmap (0, nup, PROT_READ|PROT_WRITE,
MAP_ANON|MAP_PRIVATE, -1, 0);
if (page == MAP_FAILED)
--
2.5.5
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] Reduce number of mmap calls from __libc_memalign in ld.so
2016-04-03 13:42 ` H.J. Lu
@ 2016-04-23 12:57 ` H.J. Lu
0 siblings, 0 replies; 7+ messages in thread
From: H.J. Lu @ 2016-04-23 12:57 UTC (permalink / raw)
To: Andreas Schwab; +Cc: GNU C Library
On Sun, Apr 3, 2016 at 6:42 AM, H.J. Lu <hjl.tools@gmail.com> wrote:
> On Sat, Apr 2, 2016 at 11:55 PM, Andreas Schwab <schwab@linux-m68k.org> wrote:
>> "H.J. Lu" <hjl.tools@gmail.com> writes:
>>
>>> + if (__glibc_unlikely (nup == 0 && n))
>>
>> Please also fix the implicit boolean coercion.
>>
>> Andreas.
>>
>
> Like this? OK for master?
>
I am checking it now.
--
H.J.
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2016-04-23 12:57 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-04-02 15:34 [PATCH] Reduce number of mmap calls from __libc_memalign in ld.so H.J. Lu
2016-04-02 17:33 ` Mike Frysinger
2016-04-02 17:43 ` H.J. Lu
2016-04-02 22:41 ` H.J. Lu
2016-04-03 6:56 ` Andreas Schwab
2016-04-03 13:42 ` H.J. Lu
2016-04-23 12:57 ` H.J. Lu
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).