ld.so - text and .data application segments to huge pages

ld.so - text and .data application segments to huge pages

[Daniel Black]

Hi,

I received this exceptionally large (slightly convoluted) patch to
MariaDB to move the .text and .data sections to huge pages in the
MariaDB issue https://jira.mariadb.org/browse/MDEV-24051.

The author Dmitriy Philimonov (Huawei) claimed great gains of "9% in
TPS speedup. iTLB && dTLB misses reduced in 38 times and 5 times
respectively. Latency becomes lower, the jitter in the TPS reduced
significantly", so it's definitely beneficial.

Obviously maintaining such a large loader work around in a userspace
application is going to be a burden on the skill set of developers we
have available.

How complex would it be for the glibc elf loader to use huge pages for
data segments on load? (extending _dl_map_segments with an environment
variable control like LD_HUGEPAGES I assume).

Is there any required kernel work to implement this feature?

Would such a concept be supported by the developer community here?

If so I'm happy to write up a feature request (https://sourceware.org/bugzilla).

Anyone off hand interested in writing it?

Cheers,

Daniel
MariaDB Foundation

Re: ld.so - text and .data application segments to huge pages

[Florian Weimer]

* Daniel Black:

> Hi,
>
> I received this exceptionally large (slightly convoluted) patch to
> MariaDB to move the .text and .data sections to huge pages in the
> MariaDB issue https://jira.mariadb.org/browse/MDEV-24051.
>
> The author Dmitriy Philimonov (Huawei) claimed great gains of "9% in
> TPS speedup. iTLB && dTLB misses reduced in 38 times and 5 times
> respectively. Latency becomes lower, the jitter in the TPS reduced
> significantly", so it's definitely beneficial.
>
> Obviously maintaining such a large loader work around in a userspace
> application is going to be a burden on the skill set of developers we
> have available.
>
> How complex would it be for the glibc elf loader to use huge pages for
> data segments on load? (extending _dl_map_segments with an environment
> variable control like LD_HUGEPAGES I assume).
>
> Is there any required kernel work to implement this feature?
>
> Would such a concept be supported by the developer community here?
>
> If so I'm happy to write up a feature request (https://sourceware.org/bugzilla).
>
> Anyone off hand interested in writing it?

I think the kernel already supports this via transparent hugepages, if
you increase the segment alignment via those link editor flags (as in
the proposed patch).

Cc:ing Andrea for additional comments.

Thanks,
Florian
-- 
Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill

Re: ld.so - text and .data application segments to huge pages

[Daniel Black]

On Thu, Feb 4, 2021 at 9:28 PM Florian Weimer <fweimer@redhat.com> wrote:
>
> * Daniel Black:
>
> > Hi,
> >
> > I received this exceptionally large (slightly convoluted) patch to
> > MariaDB to move the .text and .data sections to huge pages in the
> > MariaDB issue https://jira.mariadb.org/browse/MDEV-24051.
> >
> > The author Dmitriy Philimonov (Huawei) claimed great gains of "9% in
> > TPS speedup. iTLB && dTLB misses reduced in 38 times and 5 times
> > respectively. Latency becomes lower, the jitter in the TPS reduced
> > significantly", so it's definitely beneficial.
> >
> > Obviously maintaining such a large loader work around in a userspace
> > application is going to be a burden on the skill set of developers we
> > have available.
> >
> > How complex would it be for the glibc elf loader to use huge pages for
> > data segments on load? (extending _dl_map_segments with an environment
> > variable control like LD_HUGEPAGES I assume).
> >
> > Is there any required kernel work to implement this feature?
> >
> > Would such a concept be supported by the developer community here?
> >
> > If so I'm happy to write up a feature request (https://sourceware.org/bugzilla).
> >
> > Anyone off hand interested in writing it?
>
> I think the kernel already supports this via transparent hugepages, if
> you increase the segment alignment via those link editor flags (as in
> the proposed patch).

Thanks Florian, sounds like a good move to me.

> Cc:ing Andrea for additional comments.

Still welcome if you have time/insights.