[PATCH 00/16] memory tagging improvements

[Wilco Dijkstra <Wilco.Dijkstra@arm.com>]

Hi,

> - request2size and checked_request2size are not consistent which looks
>   wrong (likely works in practice other than some sizes don't end up in
>   fastbin or tcache that one might expected to, but i think this can
>   break easily). Fixing it is not obvious because of assumptions about
>   the chunk layout that is no longer true with mtag enabled.

It's better to change request2size to round up differently if tagging is
required. That avoids dynamically changing how chunks are rounded up.

> Other possible optimizations:
> - ifunc malloc apis
>   good: near zero overhead for runtime dispatch.
>   bad: malloc has to be built twice or larger refactoring is needed.

Using ifuncs is a good idea for the future - it allows removing the old
hooks and other terrible hacks like DUMPED_MAIN_ARENA_CHUNK.
Also it allows more modern malloc implementations to be added while
keeping backwards compatibility if needed (for eg. emacs).

> - unconditional top byte zero
>   internal data is tagged 0, only user allocation is tagged non-zero

That's a great idea! Many chunk2mem can be changed into chunk2rawmem
approximately halving the overhead. And mem2chunk can unconditionally mask
out the tag (note all this could be made target-dependent if needed).

> - unconditional layout
>   padding up to tag granule can be done unconditionally
>   good: no runtime check, same layout with and without heap tagging.
>   bad: wasting space with small allocations, design change.

This is unconditionally good for performance and avoids the request2size oddities.
If we're worried about memory usage, we need to talk about modern malloc
implementations that don't add a 16-byte header to every block.

> Benchmarked using bench-malloc-thread glibc ubenchmark on Neoverse N1.
> For size=16 the runtime check overhead is
>   ~ 32% before the patch set
>   ~ 16% after the patch set.
> On a practical workload or for large sizes the overhead should be tiny.

This is a huge improvement, but still far from being practical given malloc is already
slow, so making it this much slower is a non-starter. For this to be usable for distros,
we really need to get the overhead below 5% (and it looks like that is feasible - with
the optimizations mentioned I get ~3.2% overhead).

Cheers,
Wilco