* [PATCH v4 0/6] Memory tagging support
@ 2020-12-18 19:29 Richard Earnshaw
2020-12-18 19:29 ` [PATCH v4 1/6] config: Allow memory tagging to be enabled when configuring glibc Richard Earnshaw
` (7 more replies)
0 siblings, 8 replies; 25+ messages in thread
From: Richard Earnshaw @ 2020-12-18 19:29 UTC (permalink / raw)
To: libc-alpha; +Cc: Richard Earnshaw, dj, szabolcs.nagy
Main changes in this version:
- collapse the changes to the malloc code to a single patch.
- change _LIBC_MTAG to USE_MTAG.
- comments around definition of PROT_MTE.
- tunable renamed to glibc.mem.tagging.
- cleanups to assembler files for aarch64 support.
I'll push a copy of this patch set to rearnsha/mte-v4.0 shortly.
Richard Earnshaw (6):
config: Allow memory tagging to be enabled when configuring glibc
elf: Add a tunable to control use of tagged memory
malloc: Basic support for memory tagging in the malloc() family
linux: Add compatibility definitions to sys/prctl.h for MTE
aarch64: Add sysv specific enabling code for memory tagging
aarch64: Add aarch64-specific files for memory tagging support
INSTALL | 14 +
config.h.in | 3 +
config.make.in | 2 +
configure | 22 ++
configure.ac | 15 +
elf/dl-tunables.list | 9 +
malloc/arena.c | 59 ++-
malloc/hooks.c | 79 ++--
malloc/malloc.c | 336 ++++++++++++++----
malloc/malloc.h | 7 +
manual/install.texi | 13 +
manual/tunables.texi | 35 ++
sysdeps/aarch64/Makefile | 5 +
sysdeps/aarch64/__mtag_address_get_tag.S | 32 ++
sysdeps/aarch64/__mtag_memset_tag.S | 53 +++
sysdeps/aarch64/__mtag_new_tag.S | 37 ++
sysdeps/aarch64/__mtag_tag_region.S | 51 +++
sysdeps/aarch64/libc-mtag.h | 57 +++
sysdeps/generic/libc-mtag.h | 52 +++
sysdeps/unix/sysv/linux/aarch64/bits/hwcap.h | 1 +
sysdeps/unix/sysv/linux/aarch64/bits/mman.h | 1 +
.../unix/sysv/linux/aarch64/cpu-features.c | 30 ++
.../unix/sysv/linux/aarch64/cpu-features.h | 2 +
sysdeps/unix/sysv/linux/sys/prctl.h | 18 +
24 files changed, 836 insertions(+), 97 deletions(-)
create mode 100644 sysdeps/aarch64/__mtag_address_get_tag.S
create mode 100644 sysdeps/aarch64/__mtag_memset_tag.S
create mode 100644 sysdeps/aarch64/__mtag_new_tag.S
create mode 100644 sysdeps/aarch64/__mtag_tag_region.S
create mode 100644 sysdeps/aarch64/libc-mtag.h
create mode 100644 sysdeps/generic/libc-mtag.h
--
2.29.2
^ permalink raw reply [flat|nested] 25+ messages in thread
* [PATCH v4 1/6] config: Allow memory tagging to be enabled when configuring glibc
2020-12-18 19:29 [PATCH v4 0/6] Memory tagging support Richard Earnshaw
@ 2020-12-18 19:29 ` Richard Earnshaw
2020-12-21 12:40 ` Siddhesh Poyarekar
2020-12-18 19:29 ` [PATCH v4 2/6] elf: Add a tunable to control use of tagged memory Richard Earnshaw
` (6 subsequent siblings)
7 siblings, 1 reply; 25+ messages in thread
From: Richard Earnshaw @ 2020-12-18 19:29 UTC (permalink / raw)
To: libc-alpha; +Cc: Richard Earnshaw, dj, szabolcs.nagy
[-- Attachment #1: Type: text/plain, Size: 44 bytes --]
This is a multi-part message in MIME format.
[-- Attachment #2: Type: text/plain, Size: 518 bytes --]
This patch adds the configuration machinery to allow memory tagging to be
enabled from the command line via the configure option --enable-memory-tagging.
The current default is off, though in time we may change that once the API
is more stable.
---
INSTALL | 14 ++++++++++++++
config.h.in | 3 +++
config.make.in | 2 ++
configure | 22 ++++++++++++++++++++++
configure.ac | 15 +++++++++++++++
manual/install.texi | 13 +++++++++++++
6 files changed, 69 insertions(+)
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: v4-0001-config-Allow-memory-tagging-to-be-enabled-when-co.patch --]
[-- Type: text/x-patch; name="v4-0001-config-Allow-memory-tagging-to-be-enabled-when-co.patch", Size: 5298 bytes --]
diff --git a/INSTALL b/INSTALL
index accdc39821..de75e72760 100644
--- a/INSTALL
+++ b/INSTALL
@@ -142,6 +142,20 @@ if 'CFLAGS' is specified it must enable optimization. For example:
non-CET processors. '--enable-cet' has been tested for i686,
x86_64 and x32 on CET processors.
+'--enable-memory-tagging'
+ Enable memory tagging support if the architecture supports it.
+ When the GNU C Library is built with this option then the resulting
+ library will be able to control the use of tagged memory when
+ hardware support is present by use of the tunable
+ 'glibc.mem.tagging'. This includes the generation of tagged memory
+ when using the 'malloc' APIs.
+
+ At present only AArch64 platforms with MTE provide this
+ functionality, although the library will still operate (without
+ memory tagging) on older versions of the architecture.
+
+ The default is to disable support for memory tagging.
+
'--disable-profile'
Don't build libraries with profiling information. You may want to
use this option if you don't plan to do profiling.
diff --git a/config.h.in b/config.h.in
index b823c8e080..eca2d66a2c 100644
--- a/config.h.in
+++ b/config.h.in
@@ -160,6 +160,9 @@
/* Define if __stack_chk_guard canary should be randomized at program startup. */
#undef ENABLE_STACKGUARD_RANDOMIZE
+/* Define if memory tagging support should be enabled. */
+#undef USE_MTAG
+
/* Package description. */
#undef PKGVERSION
diff --git a/config.make.in b/config.make.in
index 1ac9417245..7ae27564fd 100644
--- a/config.make.in
+++ b/config.make.in
@@ -84,6 +84,8 @@ mach-interface-list = @mach_interface_list@
experimental-malloc = @experimental_malloc@
+memory-tagging = @memory_tagging@
+
nss-crypt = @libc_cv_nss_crypt@
static-nss-crypt = @libc_cv_static_nss_crypt@
diff --git a/configure b/configure
index 4795e721e5..6a35553805 100755
--- a/configure
+++ b/configure
@@ -676,6 +676,7 @@ build_nscd
libc_cv_static_nss_crypt
libc_cv_nss_crypt
build_crypt
+memory_tagging
experimental_malloc
enable_werror
all_warnings
@@ -781,6 +782,7 @@ enable_all_warnings
enable_werror
enable_multi_arch
enable_experimental_malloc
+enable_memory_tagging
enable_crypt
enable_nss_crypt
enable_systemtap
@@ -1450,6 +1452,8 @@ Optional Features:
architectures
--disable-experimental-malloc
disable experimental malloc features
+ --enable-memory-tagging enable memory tagging if supported by the
+ architecture [default=no]
--disable-crypt do not build nor install the passphrase hashing
library, libcrypt
--enable-nss-crypt enable libcrypt to use nss
@@ -3519,6 +3523,24 @@ fi
+# Check whether --enable-memory-tagging was given.
+if test "${enable_memory_tagging+set}" = set; then :
+ enableval=$enable_memory_tagging; memory_tagging=$enableval
+else
+ memory_tagging=no
+fi
+
+if test "$memory_tagging" = yes; then
+ # Only enable this on architectures that support it.
+ case $host_cpu in
+ aarch64)
+ $as_echo "#define USE_MTAG 1" >>confdefs.h
+
+ ;;
+ esac
+fi
+
+
# Check whether --enable-crypt was given.
if test "${enable_crypt+set}" = set; then :
enableval=$enable_crypt; build_crypt=$enableval
diff --git a/configure.ac b/configure.ac
index 93e68fb696..43cfac9d48 100644
--- a/configure.ac
+++ b/configure.ac
@@ -311,6 +311,21 @@ AC_ARG_ENABLE([experimental-malloc],
[experimental_malloc=yes])
AC_SUBST(experimental_malloc)
+AC_ARG_ENABLE([memory-tagging],
+ AC_HELP_STRING([--enable-memory-tagging],
+ [enable memory tagging if supported by the architecture @<:@default=no@:>@]),
+ [memory_tagging=$enableval],
+ [memory_tagging=no])
+if test "$memory_tagging" = yes; then
+ # Only enable this on architectures that support it.
+ case $host_cpu in
+ aarch64)
+ AC_DEFINE(USE_MTAG)
+ ;;
+ esac
+fi
+AC_SUBST(memory_tagging)
+
AC_ARG_ENABLE([crypt],
AC_HELP_STRING([--disable-crypt],
[do not build nor install the passphrase hashing library, libcrypt]),
diff --git a/manual/install.texi b/manual/install.texi
index 648f366371..8f26bb2a81 100644
--- a/manual/install.texi
+++ b/manual/install.texi
@@ -171,6 +171,19 @@ NOTE: @option{--enable-cet} has been tested for i686, x86_64 and x32
on non-CET processors. @option{--enable-cet} has been tested for
i686, x86_64 and x32 on CET processors.
+@item --enable-memory-tagging
+Enable memory tagging support if the architecture supports it. When
+@theglibc{} is built with this option then the resulting library will
+be able to control the use of tagged memory when hardware support is
+present by use of the tunable @samp{glibc.mem.tagging}. This includes
+the generation of tagged memory when using the @code{malloc} APIs.
+
+At present only AArch64 platforms with MTE provide this functionality,
+although the library will still operate (without memory tagging) on
+older versions of the architecture.
+
+The default is to disable support for memory tagging.
+
@item --disable-profile
Don't build libraries with profiling information. You may want to use
this option if you don't plan to do profiling.
^ permalink raw reply [flat|nested] 25+ messages in thread
* [PATCH v4 2/6] elf: Add a tunable to control use of tagged memory
2020-12-18 19:29 [PATCH v4 0/6] Memory tagging support Richard Earnshaw
2020-12-18 19:29 ` [PATCH v4 1/6] config: Allow memory tagging to be enabled when configuring glibc Richard Earnshaw
@ 2020-12-18 19:29 ` Richard Earnshaw
2020-12-21 12:42 ` Siddhesh Poyarekar
2020-12-18 19:29 ` [PATCH v4 3/6] malloc: Basic support for memory tagging in the malloc() family Richard Earnshaw
` (5 subsequent siblings)
7 siblings, 1 reply; 25+ messages in thread
From: Richard Earnshaw @ 2020-12-18 19:29 UTC (permalink / raw)
To: libc-alpha; +Cc: Richard Earnshaw, dj, szabolcs.nagy
[-- Attachment #1: Type: text/plain, Size: 539 bytes --]
Add a new glibc tunable: mem.tagging. This is a decimal constant in
the range 0-255 but used as a bit-field.
Bit 0 enables use of tagged memory in the malloc family of functions.
Bit 1 enables precise faulting of tag failure on platforms where this
can be controlled.
Other bits are currently unused, but if set will cause memory tag
checking for the current process to be enabled in the kernel.
---
elf/dl-tunables.list | 9 +++++++++
manual/tunables.texi | 35 +++++++++++++++++++++++++++++++++++
2 files changed, 44 insertions(+)
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: v4-0002-elf-Add-a-tunable-to-control-use-of-tagged-memory.patch --]
[-- Type: text/x-patch; name="v4-0002-elf-Add-a-tunable-to-control-use-of-tagged-memory.patch", Size: 2114 bytes --]
diff --git a/elf/dl-tunables.list b/elf/dl-tunables.list
index e1d8225128..4c44ead715 100644
--- a/elf/dl-tunables.list
+++ b/elf/dl-tunables.list
@@ -141,4 +141,13 @@ glibc {
default: 512
}
}
+
+ mem {
+ tagging {
+ type: INT_32
+ minval: 0
+ maxval: 255
+ security_level: SXID_IGNORE
+ }
+ }
}
diff --git a/manual/tunables.texi b/manual/tunables.texi
index d72d7a5ec0..1bbdc88281 100644
--- a/manual/tunables.texi
+++ b/manual/tunables.texi
@@ -36,6 +36,8 @@ their own namespace.
* POSIX Thread Tunables:: Tunables in the POSIX thread subsystem
* Hardware Capability Tunables:: Tunables that modify the hardware
capabilities seen by @theglibc{}
+* Memory Related Tunables:: Tunables that control the use of memory by
+ @theglibc{}.
@end menu
@node Tunable names
@@ -484,3 +486,36 @@ instead.
This tunable is specific to i386 and x86-64.
@end deftp
+
+@node Memory Related Tunables
+@section Memory Related Tunables
+@cindex memory related tunables
+
+@deftp {Tunable namespace} glibc.mem
+This tunable namespace supports operations that affect the way @theglibc{}
+and the process manage memory.
+@end deftp
+
+@deftp Tunable glibc.mem.tagging
+If the hardware supports memory tagging, this tunable can be used to
+control the way @theglibc{} uses this feature. At present this is only
+supported on AArch64 systems with the MTE extention; it is ignored for
+all other systems.
+
+This tunable takes a value between 0 and 255 and acts as a bitmask
+that enables various capabilities.
+
+Bit 0 (the least significant bit) causes the malloc subsystem to allocate
+tagged memory, with each allocation being assigned a random tag.
+
+Bit 1 enables precise faulting mode for tag violations on systems that
+support deferred tag violation reporting. This may cause programs
+to run more slowly.
+
+Other bits are currently reserved.
+
+@Theglibc{} startup code will automatically enable memory tagging
+support in the kernel if this tunable has any non-zero value.
+
+The default value is @samp{0}, which disables all memory tagging.
+@end deftp
^ permalink raw reply [flat|nested] 25+ messages in thread
* [PATCH v4 3/6] malloc: Basic support for memory tagging in the malloc() family
2020-12-18 19:29 [PATCH v4 0/6] Memory tagging support Richard Earnshaw
2020-12-18 19:29 ` [PATCH v4 1/6] config: Allow memory tagging to be enabled when configuring glibc Richard Earnshaw
2020-12-18 19:29 ` [PATCH v4 2/6] elf: Add a tunable to control use of tagged memory Richard Earnshaw
@ 2020-12-18 19:29 ` Richard Earnshaw
2020-12-21 13:27 ` Siddhesh Poyarekar
2020-12-21 13:46 ` Florian Weimer
2020-12-18 19:29 ` [PATCH v4 4/6] linux: Add compatibility definitions to sys/prctl.h for MTE Richard Earnshaw
` (4 subsequent siblings)
7 siblings, 2 replies; 25+ messages in thread
From: Richard Earnshaw @ 2020-12-18 19:29 UTC (permalink / raw)
To: libc-alpha; +Cc: Richard Earnshaw, dj, szabolcs.nagy
[-- Attachment #1: Type: text/plain, Size: 900 bytes --]
This patch adds the basic support for memory tagging.
Various flavours are supported, particularly being able to turn on
tagged memory at run-time: this allows the same code to be used on
systems where memory tagging support is not present without neededing
a separate build of glibc. Also, depending on whether the kernel
supports it, the code will use mmap for the default arena if morecore
does not, or cannot support tagged memory (on AArch64 it is not
available).
All the hooks use function pointers to allow this to work without
needing ifuncs.
---
malloc/arena.c | 59 ++++++-
malloc/hooks.c | 79 ++++++---
malloc/malloc.c | 336 +++++++++++++++++++++++++++++-------
malloc/malloc.h | 7 +
sysdeps/generic/libc-mtag.h | 52 ++++++
5 files changed, 436 insertions(+), 97 deletions(-)
create mode 100644 sysdeps/generic/libc-mtag.h
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: v4-0003-malloc-Basic-support-for-memory-tagging-in-the-ma.patch --]
[-- Type: text/x-patch; name="v4-0003-malloc-Basic-support-for-memory-tagging-in-the-ma.patch", Size: 43139 bytes --]
diff --git a/malloc/arena.c b/malloc/arena.c
index 3c9c0ecd86..ba833a2d98 100644
--- a/malloc/arena.c
+++ b/malloc/arena.c
@@ -274,17 +274,52 @@ next_env_entry (char ***position)
#endif
-#ifdef SHARED
+#if defined(SHARED) || defined(USE_MTAG)
static void *
__failing_morecore (ptrdiff_t d)
{
return (void *) MORECORE_FAILURE;
}
+#endif
+#ifdef SHARED
extern struct dl_open_hook *_dl_open_hook;
libc_hidden_proto (_dl_open_hook);
#endif
+#ifdef USE_MTAG
+
+/* Generate a new (random) tag value for PTR and tag the memory it
+ points to upto the end of the usable size for the chunk containing
+ it. Return the newly tagged pointer. */
+static void *
+__mtag_tag_new_usable (void *ptr)
+{
+ if (ptr)
+ {
+ mchunkptr cp = mem2chunk(ptr);
+ /* This likely will never happen, but we can't handle retagging
+ chunks from the dumped main arena. So just return the
+ existing pointer. */
+ if (DUMPED_MAIN_ARENA_CHUNK (cp))
+ return ptr;
+ ptr = __libc_mtag_tag_region (__libc_mtag_new_tag (ptr),
+ CHUNK_AVAILABLE_SIZE (cp) - CHUNK_HDR_SZ);
+ }
+ return ptr;
+}
+
+/* Generate a new (random) tag value for PTR, set the tags for the
+ memory to the new tag and initialize the memory contents to VAL.
+ In practice this function will only be called with VAL=0, but we
+ keep this parameter to maintain the same prototype as memset. */
+static void *
+__mtag_tag_new_memset (void *ptr, int val, size_t size)
+{
+ return __libc_mtag_memset_with_tag (__libc_mtag_new_tag (ptr), val, size);
+}
+#endif
+
static void
ptmalloc_init (void)
{
@@ -293,6 +328,24 @@ ptmalloc_init (void)
__malloc_initialized = 0;
+#ifdef USE_MTAG
+ if ((TUNABLE_GET_FULL (glibc, mem, tagging, int32_t, NULL) & 1) != 0)
+ {
+ /* If the environment says that we should be using tagged memory
+ and that morecore does not support tagged regions, then
+ disable it. */
+ if (__MTAG_SBRK_UNTAGGED)
+ __morecore = __failing_morecore;
+
+ __mtag_mmap_flags = __MTAG_MMAP_FLAGS;
+ __tag_new_memset = __mtag_tag_new_memset;
+ __tag_region = __libc_mtag_tag_region;
+ __tag_new_usable = __mtag_tag_new_usable;
+ __tag_at = __libc_mtag_address_get_tag;
+ __mtag_granule_mask = ~(size_t)(__MTAG_GRANULE_SIZE - 1);
+ }
+#endif
+
#ifdef SHARED
/* In case this libc copy is in a non-default namespace, never use
brk. Likewise if dlopened from statically linked program. The
@@ -509,7 +562,7 @@ new_heap (size_t size, size_t top_pad)
}
}
}
- if (__mprotect (p2, size, PROT_READ | PROT_WRITE) != 0)
+ if (__mprotect (p2, size, MTAG_MMAP_FLAGS | PROT_READ | PROT_WRITE) != 0)
{
__munmap (p2, HEAP_MAX_SIZE);
return 0;
@@ -539,7 +592,7 @@ grow_heap (heap_info *h, long diff)
{
if (__mprotect ((char *) h + h->mprotect_size,
(unsigned long) new_size - h->mprotect_size,
- PROT_READ | PROT_WRITE) != 0)
+ MTAG_MMAP_FLAGS | PROT_READ | PROT_WRITE) != 0)
return -2;
h->mprotect_size = new_size;
diff --git a/malloc/hooks.c b/malloc/hooks.c
index a2b93e5446..8a1c16dfa4 100644
--- a/malloc/hooks.c
+++ b/malloc/hooks.c
@@ -63,6 +63,13 @@ __malloc_check_init (void)
__memalign_hook = memalign_check;
}
+/* When memory is tagged, the checking data is stored in the user part
+ of the chunk. We can't rely on the user not having modified the
+ tags, so fetch the tag at each location before dereferencing
+ it. */
+#define SAFE_CHAR_OFFSET(p,offset) \
+ ((unsigned char *) TAG_AT (((unsigned char *) p) + offset))
+
/* A simple, standard set of debugging hooks. Overhead is `only' one
byte per chunk; still this will catch most cases of double frees or
overruns. The goal here is to avoid obscure crashes due to invalid
@@ -80,7 +87,6 @@ magicbyte (const void *p)
return magic;
}
-
/* Visualize the chunk as being partitioned into blocks of 255 bytes from the
highest address of the chunk, downwards. The end of each block tells
us the size of that block, up to the actual size of the requested
@@ -96,16 +102,16 @@ malloc_check_get_size (mchunkptr p)
assert (using_malloc_checking == 1);
- for (size = chunksize (p) - 1 + (chunk_is_mmapped (p) ? 0 : SIZE_SZ);
- (c = ((unsigned char *) p)[size]) != magic;
+ for (size = CHUNK_AVAILABLE_SIZE (p) - 1;
+ (c = *SAFE_CHAR_OFFSET (p, size)) != magic;
size -= c)
{
- if (c <= 0 || size < (c + 2 * SIZE_SZ))
+ if (c <= 0 || size < (c + CHUNK_HDR_SZ))
malloc_printerr ("malloc_check_get_size: memory corruption");
}
/* chunk2mem size. */
- return size - 2 * SIZE_SZ;
+ return size - CHUNK_HDR_SZ;
}
/* Instrument a chunk with overrun detector byte(s) and convert it
@@ -124,9 +130,8 @@ mem2mem_check (void *ptr, size_t req_sz)
p = mem2chunk (ptr);
magic = magicbyte (p);
- max_sz = chunksize (p) - 2 * SIZE_SZ;
- if (!chunk_is_mmapped (p))
- max_sz += SIZE_SZ;
+ max_sz = CHUNK_AVAILABLE_SIZE (p) - CHUNK_HDR_SZ;
+
for (i = max_sz - 1; i > req_sz; i -= block_sz)
{
block_sz = MIN (i - req_sz, 0xff);
@@ -135,9 +140,9 @@ mem2mem_check (void *ptr, size_t req_sz)
if (block_sz == magic)
--block_sz;
- m_ptr[i] = block_sz;
+ *SAFE_CHAR_OFFSET (m_ptr, i) = block_sz;
}
- m_ptr[req_sz] = magic;
+ *SAFE_CHAR_OFFSET (m_ptr, req_sz) = magic;
return (void *) m_ptr;
}
@@ -170,9 +175,11 @@ mem2chunk_check (void *mem, unsigned char **magic_p)
next_chunk (prev_chunk (p)) != p)))
return NULL;
- for (sz += SIZE_SZ - 1; (c = ((unsigned char *) p)[sz]) != magic; sz -= c)
+ for (sz = CHUNK_AVAILABLE_SIZE (p) - 1;
+ (c = *SAFE_CHAR_OFFSET (p, sz)) != magic;
+ sz -= c)
{
- if (c == 0 || sz < (c + 2 * SIZE_SZ))
+ if (c == 0 || sz < (c + CHUNK_HDR_SZ))
return NULL;
}
}
@@ -193,15 +200,19 @@ mem2chunk_check (void *mem, unsigned char **magic_p)
((prev_size (p) + sz) & page_mask) != 0)
return NULL;
- for (sz -= 1; (c = ((unsigned char *) p)[sz]) != magic; sz -= c)
+ for (sz = CHUNK_AVAILABLE_SIZE (p) - 1;
+ (c = *SAFE_CHAR_OFFSET (p, sz)) != magic;
+ sz -= c)
{
- if (c == 0 || sz < (c + 2 * SIZE_SZ))
+ if (c == 0 || sz < (c + CHUNK_HDR_SZ))
return NULL;
}
}
- ((unsigned char *) p)[sz] ^= 0xFF;
+
+ unsigned char* safe_p = SAFE_CHAR_OFFSET (p, sz);
+ *safe_p ^= 0xFF;
if (magic_p)
- *magic_p = (unsigned char *) p + sz;
+ *magic_p = safe_p;
return p;
}
@@ -238,7 +249,7 @@ malloc_check (size_t sz, const void *caller)
top_check ();
victim = _int_malloc (&main_arena, nb);
__libc_lock_unlock (main_arena.mutex);
- return mem2mem_check (victim, sz);
+ return mem2mem_check (TAG_NEW_USABLE (victim), sz);
}
static void
@@ -249,6 +260,12 @@ free_check (void *mem, const void *caller)
if (!mem)
return;
+#ifdef USE_MTAG
+ /* Quickly check that the freed pointer matches the tag for the memory.
+ This gives a useful double-free detection. */
+ *(volatile char *)mem;
+#endif
+
__libc_lock_lock (main_arena.mutex);
p = mem2chunk_check (mem, NULL);
if (!p)
@@ -259,6 +276,8 @@ free_check (void *mem, const void *caller)
munmap_chunk (p);
return;
}
+ /* Mark the chunk as belonging to the library again. */
+ (void)TAG_REGION (chunk2rawmem (p), CHUNK_AVAILABLE_SIZE (p) - CHUNK_HDR_SZ);
_int_free (&main_arena, p, 1);
__libc_lock_unlock (main_arena.mutex);
}
@@ -266,7 +285,7 @@ free_check (void *mem, const void *caller)
static void *
realloc_check (void *oldmem, size_t bytes, const void *caller)
{
- INTERNAL_SIZE_T nb;
+ INTERNAL_SIZE_T chnb;
void *newmem = 0;
unsigned char *magic_p;
size_t rb;
@@ -284,14 +303,21 @@ realloc_check (void *oldmem, size_t bytes, const void *caller)
free_check (oldmem, NULL);
return NULL;
}
+
+#ifdef USE_MTAG
+ /* Quickly check that the freed pointer matches the tag for the memory.
+ This gives a useful double-free detection. */
+ *(volatile char *)oldmem;
+#endif
+
__libc_lock_lock (main_arena.mutex);
const mchunkptr oldp = mem2chunk_check (oldmem, &magic_p);
__libc_lock_unlock (main_arena.mutex);
if (!oldp)
malloc_printerr ("realloc(): invalid pointer");
- const INTERNAL_SIZE_T oldsize = chunksize (oldp);
+ const INTERNAL_SIZE_T oldchsize = CHUNK_AVAILABLE_SIZE (oldp);
- if (!checked_request2size (rb, &nb))
+ if (!checked_request2size (rb, &chnb))
goto invert;
__libc_lock_lock (main_arena.mutex);
@@ -299,14 +325,13 @@ realloc_check (void *oldmem, size_t bytes, const void *caller)
if (chunk_is_mmapped (oldp))
{
#if HAVE_MREMAP
- mchunkptr newp = mremap_chunk (oldp, nb);
+ mchunkptr newp = mremap_chunk (oldp, chnb);
if (newp)
newmem = chunk2mem (newp);
else
#endif
{
- /* Note the extra SIZE_SZ overhead. */
- if (oldsize - SIZE_SZ >= nb)
+ if (oldchsize >= chnb)
newmem = oldmem; /* do nothing */
else
{
@@ -315,7 +340,7 @@ realloc_check (void *oldmem, size_t bytes, const void *caller)
newmem = _int_malloc (&main_arena, rb);
if (newmem)
{
- memcpy (newmem, oldmem, oldsize - 2 * SIZE_SZ);
+ memcpy (newmem, oldmem, oldchsize - CHUNK_HDR_SZ);
munmap_chunk (oldp);
}
}
@@ -324,7 +349,7 @@ realloc_check (void *oldmem, size_t bytes, const void *caller)
else
{
top_check ();
- newmem = _int_realloc (&main_arena, oldp, oldsize, nb);
+ newmem = _int_realloc (&main_arena, oldp, oldchsize, chnb);
}
DIAG_PUSH_NEEDS_COMMENT;
@@ -343,7 +368,7 @@ invert:
__libc_lock_unlock (main_arena.mutex);
- return mem2mem_check (newmem, bytes);
+ return mem2mem_check (TAG_NEW_USABLE (newmem), bytes);
}
static void *
@@ -385,7 +410,7 @@ memalign_check (size_t alignment, size_t bytes, const void *caller)
top_check ();
mem = _int_memalign (&main_arena, alignment, bytes + 1);
__libc_lock_unlock (main_arena.mutex);
- return mem2mem_check (mem, bytes);
+ return mem2mem_check (TAG_NEW_USABLE (mem), bytes);
}
#if SHLIB_COMPAT (libc, GLIBC_2_0, GLIBC_2_25)
diff --git a/malloc/malloc.c b/malloc/malloc.c
index 326075e704..a3e914fa8a 100644
--- a/malloc/malloc.c
+++ b/malloc/malloc.c
@@ -242,6 +242,9 @@
/* For DIAG_PUSH/POP_NEEDS_COMMENT et al. */
#include <libc-diag.h>
+/* For memory tagging. */
+#include <libc-mtag.h>
+
#include <malloc/malloc-internal.h>
/* For SINGLE_THREAD_P. */
@@ -380,6 +383,96 @@ __malloc_assert (const char *assertion, const char *file, unsigned int line,
void * __default_morecore (ptrdiff_t);
void *(*__morecore)(ptrdiff_t) = __default_morecore;
+/* Memory tagging. */
+
+/* Some systems support the concept of tagging (sometimes known as
+ coloring) memory locations on a fine grained basis. Each memory
+ location is given a color (normally allocated randomly) and
+ pointers are also colored. When the pointer is dereferenced, the
+ pointer's color is checked against the memory's color and if they
+ differ the access is faulted (sometimes lazily).
+
+ We use this in glibc by maintaining a single color for the malloc
+ data structures that are interleaved with the user data and then
+ assigning separate colors for each block allocation handed out. In
+ this way simple buffer overruns will be rapidly detected. When
+ memory is freed, the memory is recolored back to the glibc default
+ so that simple use-after-free errors can also be detected.
+
+ If memory is reallocated the buffer is recolored even if the
+ address remains the same. This has a performance impact, but
+ guarantees that the old pointer cannot mistakenly be reused (code
+ that compares old against new will see a mismatch and will then
+ need to behave as though realloc moved the data to a new location).
+
+ Internal API for memory tagging support.
+
+ The aim is to keep the code for memory tagging support as close to
+ the normal APIs in glibc as possible, so that if tagging is not
+ enabled in the library, or is disabled at runtime then standard
+ operations can continue to be used. Support macros are used to do
+ this:
+
+ void *TAG_NEW_MEMSET (void *ptr, int, val, size_t size)
+
+ Has the same interface as memset(), but additionally allocates a
+ new tag, colors the memory with that tag and returns a pointer that
+ is correctly colored for that location. The non-tagging version
+ will simply call memset.
+
+ void *TAG_REGION (void *ptr, size_t size)
+
+ Color the region of memory pointed to by PTR and size SIZE with
+ the color of PTR. Returns the original pointer.
+
+ void *TAG_NEW_USABLE (void *ptr)
+
+ Allocate a new random color and use it to color the user region of
+ a chunk; this may include data from the subsequent chunk's header
+ if tagging is sufficiently fine grained. Returns PTR suitably
+ recolored for accessing the memory there.
+
+ void *TAG_AT (void *ptr)
+
+ Read the current color of the memory at the address pointed to by
+ PTR (ignoring it's current color) and return PTR recolored to that
+ color. PTR must be valid address in all other respects. When
+ tagging is not enabled, it simply returns the original pointer.
+*/
+
+#ifdef USE_MTAG
+
+/* Default implementaions when memory tagging is supported, but disabled. */
+static void *
+__default_tag_region (void *ptr, size_t size)
+{
+ return ptr;
+}
+
+static void *
+__default_tag_nop (void *ptr)
+{
+ return ptr;
+}
+
+static int __mtag_mmap_flags = 0;
+static size_t __mtag_granule_mask = ~(size_t)0;
+
+static void *(*__tag_new_memset)(void *, int, size_t) = memset;
+static void *(*__tag_region)(void *, size_t) = __default_tag_region;
+static void *(*__tag_new_usable)(void *) = __default_tag_nop;
+static void *(*__tag_at)(void *) = __default_tag_nop;
+
+# define TAG_NEW_MEMSET(ptr, val, size) __tag_new_memset (ptr, val, size)
+# define TAG_REGION(ptr, size) __tag_region (ptr, size)
+# define TAG_NEW_USABLE(ptr) __tag_new_usable (ptr)
+# define TAG_AT(ptr) __tag_at (ptr)
+#else
+# define TAG_NEW_MEMSET(ptr, val, size) memset (ptr, val, size)
+# define TAG_REGION(ptr, size) (ptr)
+# define TAG_NEW_USABLE(ptr) (ptr)
+# define TAG_AT(ptr) (ptr)
+#endif
#include <string.h>
@@ -1187,10 +1280,31 @@ nextchunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
---------- Size and alignment checks and conversions ----------
*/
-/* conversion from malloc headers to user pointers, and back */
+/* Conversion from malloc headers to user pointers, and back. When
+ using memory tagging the user data and the malloc data structure
+ headers have distinct tags. Converting fully from one to the other
+ involves extracting the tag at the other address and creating a
+ suitable pointer using it. That can be quite expensive. There are
+ many occasions, though when the pointer will not be dereferenced
+ (for example, because we only want to assert that the pointer is
+ correctly aligned). In these cases it is more efficient not
+ to extract the tag, since the answer will be the same either way.
+ chunk2rawmem() can be used in these cases.
+ */
+
+/* The chunk header is two SIZE_SZ elements, but this is used widely, so
+ we define it here for clarity later. */
+#define CHUNK_HDR_SZ (2 * SIZE_SZ)
+
+/* Convert a user mem pointer to a chunk address without correcting
+ the tag. */
+#define chunk2rawmem(p) ((void*)((char*)(p) + CHUNK_HDR_SZ))
-#define chunk2mem(p) ((void*)((char*)(p) + 2*SIZE_SZ))
-#define mem2chunk(mem) ((mchunkptr)((char*)(mem) - 2*SIZE_SZ))
+/* Convert between user mem pointers and chunk pointers, updating any
+ memory tags on the pointer to respect the tag value at that
+ location. */
+#define chunk2mem(p) ((void*)TAG_AT (((char*)(p) + CHUNK_HDR_SZ)))
+#define mem2chunk(mem) ((mchunkptr)TAG_AT (((char*)(mem) - CHUNK_HDR_SZ)))
/* The smallest possible chunk */
#define MIN_CHUNK_SIZE (offsetof(struct malloc_chunk, fd_nextsize))
@@ -1205,16 +1319,28 @@ nextchunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
#define aligned_OK(m) (((unsigned long)(m) & MALLOC_ALIGN_MASK) == 0)
#define misaligned_chunk(p) \
- ((uintptr_t)(MALLOC_ALIGNMENT == 2 * SIZE_SZ ? (p) : chunk2mem (p)) \
+ ((uintptr_t)(MALLOC_ALIGNMENT == CHUNK_HDR_SZ ? (p) : chunk2mem (p)) \
& MALLOC_ALIGN_MASK)
/* pad request bytes into a usable size -- internal version */
-
+/* Note: This must be a macro that evaluates to a compile time constant
+ if passed a literal constant. */
#define request2size(req) \
(((req) + SIZE_SZ + MALLOC_ALIGN_MASK < MINSIZE) ? \
MINSIZE : \
((req) + SIZE_SZ + MALLOC_ALIGN_MASK) & ~MALLOC_ALIGN_MASK)
+/* Available size of chunk. This is the size of the real usable data
+ in the chunk, plus the chunk header. */
+#ifdef USE_MTAG
+#define CHUNK_AVAILABLE_SIZE(p) \
+ ((chunksize (p) + (chunk_is_mmapped (p) ? 0 : SIZE_SZ)) \
+ & __mtag_granule_mask)
+#else
+#define CHUNK_AVAILABLE_SIZE(p) \
+ (chunksize (p) + (chunk_is_mmapped (p) ? 0 : SIZE_SZ))
+#endif
+
/* Check if REQ overflows when padded and aligned and if the resulting value
is less than PTRDIFF_T. Returns TRUE and the requested size or MINSIZE in
case the value is less than MINSIZE on SZ or false if any of the previous
@@ -1224,6 +1350,18 @@ checked_request2size (size_t req, size_t *sz) __nonnull (1)
{
if (__glibc_unlikely (req > PTRDIFF_MAX))
return false;
+
+#ifdef USE_MTAG
+ /* When using tagged memory, we cannot share the end of the user
+ block with the header for the next chunk, so ensure that we
+ allocate blocks that are rounded up to the granule size. Take
+ care not to overflow from close to MAX_SIZE_T to a small
+ number. Ideally, this would be part of request2size(), but that
+ must be a macro that produces a compile time constant if passed
+ a constant literal. */
+ req = (req + ~__mtag_granule_mask) & __mtag_granule_mask;
+#endif
+
*sz = request2size (req);
return true;
}
@@ -1322,7 +1460,6 @@ checked_request2size (size_t req, size_t *sz) __nonnull (1)
/* Set size at footer (only when chunk is not in use) */
#define set_foot(p, s) (((mchunkptr) ((char *) (p) + (s)))->mchunk_prev_size = (s))
-
#pragma GCC poison mchunk_size
#pragma GCC poison mchunk_prev_size
@@ -1418,7 +1555,7 @@ typedef struct malloc_chunk *mbinptr;
#define NBINS 128
#define NSMALLBINS 64
#define SMALLBIN_WIDTH MALLOC_ALIGNMENT
-#define SMALLBIN_CORRECTION (MALLOC_ALIGNMENT > 2 * SIZE_SZ)
+#define SMALLBIN_CORRECTION (MALLOC_ALIGNMENT > CHUNK_HDR_SZ)
#define MIN_LARGE_SIZE ((NSMALLBINS - SMALLBIN_CORRECTION) * SMALLBIN_WIDTH)
#define in_smallbin_range(sz) \
@@ -1969,7 +2106,7 @@ do_check_chunk (mstate av, mchunkptr p)
/* chunk is page-aligned */
assert (((prev_size (p) + sz) & (GLRO (dl_pagesize) - 1)) == 0);
/* mem is aligned */
- assert (aligned_OK (chunk2mem (p)));
+ assert (aligned_OK (chunk2rawmem (p)));
}
}
@@ -1993,7 +2130,7 @@ do_check_free_chunk (mstate av, mchunkptr p)
if ((unsigned long) (sz) >= MINSIZE)
{
assert ((sz & MALLOC_ALIGN_MASK) == 0);
- assert (aligned_OK (chunk2mem (p)));
+ assert (aligned_OK (chunk2rawmem (p)));
/* ... matching footer field */
assert (prev_size (next_chunk (p)) == sz);
/* ... and is fully consolidated */
@@ -2072,7 +2209,7 @@ do_check_remalloced_chunk (mstate av, mchunkptr p, INTERNAL_SIZE_T s)
assert ((sz & MALLOC_ALIGN_MASK) == 0);
assert ((unsigned long) (sz) >= MINSIZE);
/* ... and alignment */
- assert (aligned_OK (chunk2mem (p)));
+ assert (aligned_OK (chunk2rawmem (p)));
/* chunk is less than MINSIZE more than request */
assert ((long) (sz) - (long) (s) >= 0);
assert ((long) (sz) - (long) (s + MINSIZE) < 0);
@@ -2318,7 +2455,7 @@ sysmalloc (INTERNAL_SIZE_T nb, mstate av)
See the front_misalign handling below, for glibc there is no
need for further alignments unless we have have high alignment.
*/
- if (MALLOC_ALIGNMENT == 2 * SIZE_SZ)
+ if (MALLOC_ALIGNMENT == CHUNK_HDR_SZ)
size = ALIGN_UP (nb + SIZE_SZ, pagesize);
else
size = ALIGN_UP (nb + SIZE_SZ + MALLOC_ALIGN_MASK, pagesize);
@@ -2327,7 +2464,8 @@ sysmalloc (INTERNAL_SIZE_T nb, mstate av)
/* Don't try if size wraps around 0 */
if ((unsigned long) (size) > (unsigned long) (nb))
{
- mm = (char *) (MMAP (0, size, PROT_READ | PROT_WRITE, 0));
+ mm = (char *) (MMAP (0, size,
+ MTAG_MMAP_FLAGS | PROT_READ | PROT_WRITE, 0));
if (mm != MAP_FAILED)
{
@@ -2339,16 +2477,18 @@ sysmalloc (INTERNAL_SIZE_T nb, mstate av)
address argument for later munmap in free() and realloc().
*/
- if (MALLOC_ALIGNMENT == 2 * SIZE_SZ)
+ if (MALLOC_ALIGNMENT == CHUNK_HDR_SZ)
{
- /* For glibc, chunk2mem increases the address by 2*SIZE_SZ and
- MALLOC_ALIGN_MASK is 2*SIZE_SZ-1. Each mmap'ed area is page
- aligned and therefore definitely MALLOC_ALIGN_MASK-aligned. */
- assert (((INTERNAL_SIZE_T) chunk2mem (mm) & MALLOC_ALIGN_MASK) == 0);
+ /* For glibc, chunk2rawmem increases the address by
+ CHUNK_HDR_SZ and MALLOC_ALIGN_MASK is
+ CHUNK_HDR_SZ-1. Each mmap'ed area is page
+ aligned and therefore definitely
+ MALLOC_ALIGN_MASK-aligned. */
+ assert (((INTERNAL_SIZE_T) chunk2rawmem (mm) & MALLOC_ALIGN_MASK) == 0);
front_misalign = 0;
}
else
- front_misalign = (INTERNAL_SIZE_T) chunk2mem (mm) & MALLOC_ALIGN_MASK;
+ front_misalign = (INTERNAL_SIZE_T) chunk2rawmem (mm) & MALLOC_ALIGN_MASK;
if (front_misalign > 0)
{
correction = MALLOC_ALIGNMENT - front_misalign;
@@ -2436,18 +2576,20 @@ sysmalloc (INTERNAL_SIZE_T nb, mstate av)
become the top chunk again later. Note that a footer is set
up, too, although the chunk is marked in use. */
old_size = (old_size - MINSIZE) & ~MALLOC_ALIGN_MASK;
- set_head (chunk_at_offset (old_top, old_size + 2 * SIZE_SZ), 0 | PREV_INUSE);
+ set_head (chunk_at_offset (old_top, old_size + CHUNK_HDR_SZ),
+ 0 | PREV_INUSE);
if (old_size >= MINSIZE)
{
- set_head (chunk_at_offset (old_top, old_size), (2 * SIZE_SZ) | PREV_INUSE);
- set_foot (chunk_at_offset (old_top, old_size), (2 * SIZE_SZ));
+ set_head (chunk_at_offset (old_top, old_size),
+ CHUNK_HDR_SZ | PREV_INUSE);
+ set_foot (chunk_at_offset (old_top, old_size), CHUNK_HDR_SZ);
set_head (old_top, old_size | PREV_INUSE | NON_MAIN_ARENA);
_int_free (av, old_top, 1);
}
else
{
- set_head (old_top, (old_size + 2 * SIZE_SZ) | PREV_INUSE);
- set_foot (old_top, (old_size + 2 * SIZE_SZ));
+ set_head (old_top, (old_size + CHUNK_HDR_SZ) | PREV_INUSE);
+ set_foot (old_top, (old_size + CHUNK_HDR_SZ));
}
}
else if (!tried_mmap)
@@ -2520,7 +2662,9 @@ sysmalloc (INTERNAL_SIZE_T nb, mstate av)
/* Don't try if size wraps around 0 */
if ((unsigned long) (size) > (unsigned long) (nb))
{
- char *mbrk = (char *) (MMAP (0, size, PROT_READ | PROT_WRITE, 0));
+ char *mbrk = (char *) (MMAP (0, size,
+ MTAG_MMAP_FLAGS | PROT_READ | PROT_WRITE,
+ 0));
if (mbrk != MAP_FAILED)
{
@@ -2591,7 +2735,7 @@ sysmalloc (INTERNAL_SIZE_T nb, mstate av)
/* Guarantee alignment of first new chunk made from this space */
- front_misalign = (INTERNAL_SIZE_T) chunk2mem (brk) & MALLOC_ALIGN_MASK;
+ front_misalign = (INTERNAL_SIZE_T) chunk2rawmem (brk) & MALLOC_ALIGN_MASK;
if (front_misalign > 0)
{
/*
@@ -2647,12 +2791,12 @@ sysmalloc (INTERNAL_SIZE_T nb, mstate av)
/* handle non-contiguous cases */
else
{
- if (MALLOC_ALIGNMENT == 2 * SIZE_SZ)
+ if (MALLOC_ALIGNMENT == CHUNK_HDR_SZ)
/* MORECORE/mmap must correctly align */
- assert (((unsigned long) chunk2mem (brk) & MALLOC_ALIGN_MASK) == 0);
+ assert (((unsigned long) chunk2rawmem (brk) & MALLOC_ALIGN_MASK) == 0);
else
{
- front_misalign = (INTERNAL_SIZE_T) chunk2mem (brk) & MALLOC_ALIGN_MASK;
+ front_misalign = (INTERNAL_SIZE_T) chunk2rawmem (brk) & MALLOC_ALIGN_MASK;
if (front_misalign > 0)
{
/*
@@ -2697,7 +2841,7 @@ sysmalloc (INTERNAL_SIZE_T nb, mstate av)
multiple of MALLOC_ALIGNMENT. We know there is at least
enough space in old_top to do this.
*/
- old_size = (old_size - 4 * SIZE_SZ) & ~MALLOC_ALIGN_MASK;
+ old_size = (old_size - 2 * CHUNK_HDR_SZ) & ~MALLOC_ALIGN_MASK;
set_head (old_top, old_size | PREV_INUSE);
/*
@@ -2707,9 +2851,10 @@ sysmalloc (INTERNAL_SIZE_T nb, mstate av)
lost.
*/
set_head (chunk_at_offset (old_top, old_size),
- (2 * SIZE_SZ) | PREV_INUSE);
- set_head (chunk_at_offset (old_top, old_size + 2 * SIZE_SZ),
- (2 * SIZE_SZ) | PREV_INUSE);
+ CHUNK_HDR_SZ | PREV_INUSE);
+ set_head (chunk_at_offset (old_top,
+ old_size + CHUNK_HDR_SZ),
+ CHUNK_HDR_SZ | PREV_INUSE);
/* If possible, release the rest. */
if (old_size >= MINSIZE)
@@ -2837,7 +2982,7 @@ munmap_chunk (mchunkptr p)
if (DUMPED_MAIN_ARENA_CHUNK (p))
return;
- uintptr_t mem = (uintptr_t) chunk2mem (p);
+ uintptr_t mem = (uintptr_t) chunk2rawmem (p);
uintptr_t block = (uintptr_t) p - prev_size (p);
size_t total_size = prev_size (p) + size;
/* Unfortunately we have to do the compilers job by hand here. Normally
@@ -2892,7 +3037,7 @@ mremap_chunk (mchunkptr p, size_t new_size)
p = (mchunkptr) (cp + offset);
- assert (aligned_OK (chunk2mem (p)));
+ assert (aligned_OK (chunk2rawmem (p)));
assert (prev_size (p) == offset);
set_head (p, (new_size - offset) | IS_MMAPPED);
@@ -3073,14 +3218,15 @@ __libc_malloc (size_t bytes)
&& tcache
&& tcache->counts[tc_idx] > 0)
{
- return tcache_get (tc_idx);
+ victim = tcache_get (tc_idx);
+ return TAG_NEW_USABLE (victim);
}
DIAG_POP_NEEDS_COMMENT;
#endif
if (SINGLE_THREAD_P)
{
- victim = _int_malloc (&main_arena, bytes);
+ victim = TAG_NEW_USABLE (_int_malloc (&main_arena, bytes));
assert (!victim || chunk_is_mmapped (mem2chunk (victim)) ||
&main_arena == arena_for_chunk (mem2chunk (victim)));
return victim;
@@ -3101,6 +3247,8 @@ __libc_malloc (size_t bytes)
if (ar_ptr != NULL)
__libc_lock_unlock (ar_ptr->mutex);
+ victim = TAG_NEW_USABLE (victim);
+
assert (!victim || chunk_is_mmapped (mem2chunk (victim)) ||
ar_ptr == arena_for_chunk (mem2chunk (victim)));
return victim;
@@ -3124,8 +3272,17 @@ __libc_free (void *mem)
if (mem == 0) /* free(0) has no effect */
return;
+#ifdef USE_MTAG
+ /* Quickly check that the freed pointer matches the tag for the memory.
+ This gives a useful double-free detection. */
+ *(volatile char *)mem;
+#endif
+
p = mem2chunk (mem);
+ /* Mark the chunk as belonging to the library again. */
+ (void)TAG_REGION (chunk2rawmem (p), CHUNK_AVAILABLE_SIZE (p) - CHUNK_HDR_SZ);
+
if (chunk_is_mmapped (p)) /* release mmapped memory. */
{
/* See if the dynamic brk/mmap threshold needs adjusting.
@@ -3175,6 +3332,12 @@ __libc_realloc (void *oldmem, size_t bytes)
if (oldmem == 0)
return __libc_malloc (bytes);
+#ifdef USE_MTAG
+ /* Perform a quick check to ensure that the pointer's tag matches the
+ memory's tag. */
+ *(volatile char*) oldmem;
+#endif
+
/* chunk corresponding to oldmem */
const mchunkptr oldp = mem2chunk (oldmem);
/* its size */
@@ -3217,7 +3380,7 @@ __libc_realloc (void *oldmem, size_t bytes)
return NULL;
/* Copy as many bytes as are available from the old chunk
and fit into the new size. NB: The overhead for faked
- mmapped chunks is only SIZE_SZ, not 2 * SIZE_SZ as for
+ mmapped chunks is only SIZE_SZ, not CHUNK_HDR_SZ as for
regular mmapped chunks. */
if (bytes > oldsize - SIZE_SZ)
bytes = oldsize - SIZE_SZ;
@@ -3230,7 +3393,15 @@ __libc_realloc (void *oldmem, size_t bytes)
#if HAVE_MREMAP
newp = mremap_chunk (oldp, nb);
if (newp)
- return chunk2mem (newp);
+ {
+ void *newmem = chunk2rawmem (newp);
+ /* Give the new block a different tag. This helps to ensure
+ that stale handles to the previous mapping are not
+ reused. There's a performance hit for both us and the
+ caller for doing this, so we might want to
+ reconsider. */
+ return TAG_NEW_USABLE (newmem);
+ }
#endif
/* Note the extra SIZE_SZ overhead. */
if (oldsize - SIZE_SZ >= nb)
@@ -3241,7 +3412,7 @@ __libc_realloc (void *oldmem, size_t bytes)
if (newmem == 0)
return 0; /* propagate failure */
- memcpy (newmem, oldmem, oldsize - 2 * SIZE_SZ);
+ memcpy (newmem, oldmem, oldsize - CHUNK_HDR_SZ);
munmap_chunk (oldp);
return newmem;
}
@@ -3328,8 +3499,7 @@ _mid_memalign (size_t alignment, size_t bytes, void *address)
p = _int_memalign (&main_arena, alignment, bytes);
assert (!p || chunk_is_mmapped (mem2chunk (p)) ||
&main_arena == arena_for_chunk (mem2chunk (p)));
-
- return p;
+ return TAG_NEW_USABLE (p);
}
arena_get (ar_ptr, bytes + alignment + MINSIZE);
@@ -3347,7 +3517,7 @@ _mid_memalign (size_t alignment, size_t bytes, void *address)
assert (!p || chunk_is_mmapped (mem2chunk (p)) ||
ar_ptr == arena_for_chunk (mem2chunk (p)));
- return p;
+ return TAG_NEW_USABLE (p);
}
/* For ISO C11. */
weak_alias (__libc_memalign, aligned_alloc)
@@ -3356,17 +3526,22 @@ libc_hidden_def (__libc_memalign)
void *
__libc_valloc (size_t bytes)
{
+ void *p;
+
if (__malloc_initialized < 0)
ptmalloc_init ();
void *address = RETURN_ADDRESS (0);
size_t pagesize = GLRO (dl_pagesize);
- return _mid_memalign (pagesize, bytes, address);
+ p = _mid_memalign (pagesize, bytes, address);
+ return TAG_NEW_USABLE (p);
}
void *
__libc_pvalloc (size_t bytes)
{
+ void *p;
+
if (__malloc_initialized < 0)
ptmalloc_init ();
@@ -3383,19 +3558,22 @@ __libc_pvalloc (size_t bytes)
}
rounded_bytes = rounded_bytes & -(pagesize - 1);
- return _mid_memalign (pagesize, rounded_bytes, address);
+ p = _mid_memalign (pagesize, rounded_bytes, address);
+ return TAG_NEW_USABLE (p);
}
void *
__libc_calloc (size_t n, size_t elem_size)
{
mstate av;
- mchunkptr oldtop, p;
- INTERNAL_SIZE_T sz, csz, oldtopsize;
+ mchunkptr oldtop;
+ INTERNAL_SIZE_T sz, oldtopsize;
void *mem;
+#ifndef USE_MTAG
unsigned long clearsize;
unsigned long nclears;
INTERNAL_SIZE_T *d;
+#endif
ptrdiff_t bytes;
if (__glibc_unlikely (__builtin_mul_overflow (n, elem_size, &bytes)))
@@ -3403,6 +3581,7 @@ __libc_calloc (size_t n, size_t elem_size)
__set_errno (ENOMEM);
return NULL;
}
+
sz = bytes;
void *(*hook) (size_t, const void *) =
@@ -3472,7 +3651,14 @@ __libc_calloc (size_t n, size_t elem_size)
if (mem == 0)
return 0;
- p = mem2chunk (mem);
+ mchunkptr p = mem2chunk (mem);
+ /* If we are using memory tagging, then we need to set the tags
+ regardless of MORECORE_CLEARS, so we zero the whole block while
+ doing so. */
+#ifdef USE_MTAG
+ return TAG_NEW_MEMSET (mem, 0, CHUNK_AVAILABLE_SIZE (p) - CHUNK_HDR_SZ);
+#else
+ INTERNAL_SIZE_T csz = chunksize (p);
/* Two optional cases in which clearing not necessary */
if (chunk_is_mmapped (p))
@@ -3483,8 +3669,6 @@ __libc_calloc (size_t n, size_t elem_size)
return mem;
}
- csz = chunksize (p);
-
#if MORECORE_CLEARS
if (perturb_byte == 0 && (p == oldtop && csz > oldtopsize))
{
@@ -3527,6 +3711,7 @@ __libc_calloc (size_t n, size_t elem_size)
}
return mem;
+#endif
}
/*
@@ -3764,10 +3949,10 @@ _int_malloc (mstate av, size_t bytes)
size = chunksize (victim);
mchunkptr next = chunk_at_offset (victim, size);
- if (__glibc_unlikely (size <= 2 * SIZE_SZ)
+ if (__glibc_unlikely (size <= CHUNK_HDR_SZ)
|| __glibc_unlikely (size > av->system_mem))
malloc_printerr ("malloc(): invalid size (unsorted)");
- if (__glibc_unlikely (chunksize_nomask (next) < 2 * SIZE_SZ)
+ if (__glibc_unlikely (chunksize_nomask (next) < CHUNK_HDR_SZ)
|| __glibc_unlikely (chunksize_nomask (next) > av->system_mem))
malloc_printerr ("malloc(): invalid next size (unsorted)");
if (__glibc_unlikely ((prev_size (next) & ~(SIZE_BITS)) != size))
@@ -4269,7 +4454,7 @@ _int_free (mstate av, mchunkptr p, int have_lock)
) {
if (__builtin_expect (chunksize_nomask (chunk_at_offset (p, size))
- <= 2 * SIZE_SZ, 0)
+ <= CHUNK_HDR_SZ, 0)
|| __builtin_expect (chunksize (chunk_at_offset (p, size))
>= av->system_mem, 0))
{
@@ -4280,7 +4465,7 @@ _int_free (mstate av, mchunkptr p, int have_lock)
if (!have_lock)
{
__libc_lock_lock (av->mutex);
- fail = (chunksize_nomask (chunk_at_offset (p, size)) <= 2 * SIZE_SZ
+ fail = (chunksize_nomask (chunk_at_offset (p, size)) <= CHUNK_HDR_SZ
|| chunksize (chunk_at_offset (p, size)) >= av->system_mem);
__libc_lock_unlock (av->mutex);
}
@@ -4289,7 +4474,7 @@ _int_free (mstate av, mchunkptr p, int have_lock)
malloc_printerr ("free(): invalid next size (fast)");
}
- free_perturb (chunk2mem(p), size - 2 * SIZE_SZ);
+ free_perturb (chunk2mem(p), size - CHUNK_HDR_SZ);
atomic_store_relaxed (&av->have_fastchunks, true);
unsigned int idx = fastbin_index(size);
@@ -4358,11 +4543,11 @@ _int_free (mstate av, mchunkptr p, int have_lock)
malloc_printerr ("double free or corruption (!prev)");
nextsize = chunksize(nextchunk);
- if (__builtin_expect (chunksize_nomask (nextchunk) <= 2 * SIZE_SZ, 0)
+ if (__builtin_expect (chunksize_nomask (nextchunk) <= CHUNK_HDR_SZ, 0)
|| __builtin_expect (nextsize >= av->system_mem, 0))
malloc_printerr ("free(): invalid next size (normal)");
- free_perturb (chunk2mem(p), size - 2 * SIZE_SZ);
+ free_perturb (chunk2mem(p), size - CHUNK_HDR_SZ);
/* consolidate backward */
if (!prev_inuse(p)) {
@@ -4593,7 +4778,7 @@ _int_realloc(mstate av, mchunkptr oldp, INTERNAL_SIZE_T oldsize,
unsigned long remainder_size; /* its size */
/* oldmem size */
- if (__builtin_expect (chunksize_nomask (oldp) <= 2 * SIZE_SZ, 0)
+ if (__builtin_expect (chunksize_nomask (oldp) <= CHUNK_HDR_SZ, 0)
|| __builtin_expect (oldsize >= av->system_mem, 0))
malloc_printerr ("realloc(): invalid old size");
@@ -4604,7 +4789,7 @@ _int_realloc(mstate av, mchunkptr oldp, INTERNAL_SIZE_T oldsize,
next = chunk_at_offset (oldp, oldsize);
INTERNAL_SIZE_T nextsize = chunksize (next);
- if (__builtin_expect (chunksize_nomask (next) <= 2 * SIZE_SZ, 0)
+ if (__builtin_expect (chunksize_nomask (next) <= CHUNK_HDR_SZ, 0)
|| __builtin_expect (nextsize >= av->system_mem, 0))
malloc_printerr ("realloc(): invalid next size");
@@ -4626,7 +4811,7 @@ _int_realloc(mstate av, mchunkptr oldp, INTERNAL_SIZE_T oldsize,
av->top = chunk_at_offset (oldp, nb);
set_head (av->top, (newsize - nb) | PREV_INUSE);
check_inuse_chunk (av, oldp);
- return chunk2mem (oldp);
+ return TAG_NEW_USABLE (chunk2rawmem (oldp));
}
/* Try to expand forward into next chunk; split off remainder below */
@@ -4659,7 +4844,11 @@ _int_realloc(mstate av, mchunkptr oldp, INTERNAL_SIZE_T oldsize,
}
else
{
- memcpy (newmem, chunk2mem (oldp), oldsize - SIZE_SZ);
+ void *oldmem = chunk2mem (oldp);
+ newmem = TAG_NEW_USABLE (newmem);
+ memcpy (newmem, oldmem,
+ CHUNK_AVAILABLE_SIZE (oldp) - CHUNK_HDR_SZ);
+ (void) TAG_REGION (chunk2rawmem (oldp), oldsize);
_int_free (av, oldp, 1);
check_inuse_chunk (av, newp);
return chunk2mem (newp);
@@ -4681,6 +4870,8 @@ _int_realloc(mstate av, mchunkptr oldp, INTERNAL_SIZE_T oldsize,
else /* split remainder */
{
remainder = chunk_at_offset (newp, nb);
+ /* Clear any user-space tags before writing the header. */
+ remainder = TAG_REGION (remainder, remainder_size);
set_head_size (newp, nb | (av != &main_arena ? NON_MAIN_ARENA : 0));
set_head (remainder, remainder_size | PREV_INUSE |
(av != &main_arena ? NON_MAIN_ARENA : 0));
@@ -4690,7 +4881,7 @@ _int_realloc(mstate av, mchunkptr oldp, INTERNAL_SIZE_T oldsize,
}
check_inuse_chunk (av, newp);
- return chunk2mem (newp);
+ return TAG_NEW_USABLE (chunk2rawmem (newp));
}
/*
@@ -4768,7 +4959,7 @@ _int_memalign (mstate av, size_t alignment, size_t bytes)
p = newp;
assert (newsize >= nb &&
- (((unsigned long) (chunk2mem (p))) % alignment) == 0);
+ (((unsigned long) (chunk2rawmem (p))) % alignment) == 0);
}
/* Also give back spare room at the end */
@@ -4822,7 +5013,8 @@ mtrim (mstate av, size_t pad)
+ sizeof (struct malloc_chunk)
+ psm1) & ~psm1);
- assert ((char *) chunk2mem (p) + 4 * SIZE_SZ <= paligned_mem);
+ assert ((char *) chunk2rawmem (p) + 2 * CHUNK_HDR_SZ
+ <= paligned_mem);
assert ((char *) p + size > paligned_mem);
/* This is the size we could potentially free. */
@@ -4885,20 +5077,30 @@ musable (void *mem)
mchunkptr p;
if (mem != 0)
{
+ size_t result = 0;
+
p = mem2chunk (mem);
if (__builtin_expect (using_malloc_checking == 1, 0))
- return malloc_check_get_size (p);
+ return malloc_check_get_size (p);
if (chunk_is_mmapped (p))
{
if (DUMPED_MAIN_ARENA_CHUNK (p))
- return chunksize (p) - SIZE_SZ;
+ result = chunksize (p) - SIZE_SZ;
else
- return chunksize (p) - 2 * SIZE_SZ;
+ result = chunksize (p) - CHUNK_HDR_SZ;
}
else if (inuse (p))
- return chunksize (p) - SIZE_SZ;
+ result = chunksize (p) - SIZE_SZ;
+
+#ifdef USE_MTAG
+ /* The usable space may be reduced if memory tagging is needed,
+ since we cannot share the user-space data with malloc's internal
+ data structure. */
+ result &= __mtag_granule_mask;
+#endif
+ return result;
}
return 0;
}
diff --git a/malloc/malloc.h b/malloc/malloc.h
index b2371f7704..0b20786b58 100644
--- a/malloc/malloc.h
+++ b/malloc/malloc.h
@@ -77,6 +77,13 @@ extern void *pvalloc (size_t __size) __THROW __attribute_malloc__ __wur;
contiguous pieces of memory. */
extern void *(*__morecore) (ptrdiff_t __size) __MALLOC_DEPRECATED;
+#ifdef USE_MTAG
+extern int __mtag_mmap_flags;
+#define MTAG_MMAP_FLAGS __mtag_mmap_flags
+#else
+#define MTAG_MMAP_FLAGS 0
+#endif
+
/* Default value of `__morecore'. */
extern void *__default_morecore (ptrdiff_t __size)
__THROW __attribute_malloc__ __MALLOC_DEPRECATED;
diff --git a/sysdeps/generic/libc-mtag.h b/sysdeps/generic/libc-mtag.h
new file mode 100644
index 0000000000..07f0203253
--- /dev/null
+++ b/sysdeps/generic/libc-mtag.h
@@ -0,0 +1,52 @@
+/* libc-internal interface for tagged (colored) memory support.
+ Copyright (C) 2020 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#ifndef _GENERIC_LIBC_MTAG_H
+#define _GENERIC_LIBC_MTAG_H 1
+
+/* Generic bindings for systems that do not support memory tagging. */
+
+/* Used to ensure additional alignment when objects need to have distinct
+ tags. */
+#define __MTAG_GRANULE_SIZE 1
+
+/* Non-zero if memory obtained via morecore (sbrk) is not tagged. */
+#define __MTAG_SBRK_UNTAGGED 0
+
+/* Extra flags to pass to mmap() to request a tagged region of memory. */
+#define __MTAG_MMAP_FLAGS 0
+
+/* Set the tags for a region of memory, which must have size and alignment
+ that are multiples of __MTAG_GRANULE_SIZE. Size cannot be zero.
+ void *__libc_mtag_tag_region (const void *, size_t) */
+#define __libc_mtag_tag_region(p, s) (p)
+
+/* Optimized equivalent to __libc_mtag_tag_region followed by memset. */
+#define __libc_mtag_memset_with_tag memset
+
+/* Convert address P to a pointer that is tagged correctly for that
+ location.
+ void *__libc_mtag_address_get_tag (void*) */
+#define __libc_mtag_address_get_tag(p) (p)
+
+/* Assign a new (random) tag to a pointer P (does not adjust the tag on
+ the memory addressed).
+ void *__libc_mtag_new_tag (void*) */
+#define __libc_mtag_new_tag(p) (p)
+
+#endif /* _GENERIC_LIBC_MTAG_H */
^ permalink raw reply [flat|nested] 25+ messages in thread
* [PATCH v4 4/6] linux: Add compatibility definitions to sys/prctl.h for MTE
2020-12-18 19:29 [PATCH v4 0/6] Memory tagging support Richard Earnshaw
` (2 preceding siblings ...)
2020-12-18 19:29 ` [PATCH v4 3/6] malloc: Basic support for memory tagging in the malloc() family Richard Earnshaw
@ 2020-12-18 19:29 ` Richard Earnshaw
2020-12-21 13:32 ` Siddhesh Poyarekar
2020-12-18 19:29 ` [PATCH v4 5/6] aarch64: Add sysv specific enabling code for memory tagging Richard Earnshaw
` (3 subsequent siblings)
7 siblings, 1 reply; 25+ messages in thread
From: Richard Earnshaw @ 2020-12-18 19:29 UTC (permalink / raw)
To: libc-alpha; +Cc: Richard Earnshaw, dj, szabolcs.nagy
[-- Attachment #1: Type: text/plain, Size: 432 bytes --]
Older versions of the Linux kernel headers obviously lack support for
memory tagging, but we still want to be able to build in support when
using those (obviously it can't be enabled on such systems).
The linux kernel extensions are made to the platform-independent
header (linux/prctl.h), so this patch takes a similar approach.
---
sysdeps/unix/sysv/linux/sys/prctl.h | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: v4-0004-linux-Add-compatibility-definitions-to-sys-prctl..patch --]
[-- Type: text/x-patch; name="v4-0004-linux-Add-compatibility-definitions-to-sys-prctl..patch", Size: 971 bytes --]
diff --git a/sysdeps/unix/sysv/linux/sys/prctl.h b/sysdeps/unix/sysv/linux/sys/prctl.h
index 7f748ebeeb..4d01379c23 100644
--- a/sysdeps/unix/sysv/linux/sys/prctl.h
+++ b/sysdeps/unix/sysv/linux/sys/prctl.h
@@ -21,6 +21,24 @@
#include <features.h>
#include <linux/prctl.h> /* The magic values come from here */
+/* Recent extensions to linux which may post-date the kernel headers
+ we're picking up... */
+
+/* Memory tagging control operations (for AArch64). */
+#ifndef PR_TAGGED_ADDR_ENABLE
+# define PR_TAGGED_ADDR_ENABLE (1UL << 8)
+#endif
+
+#ifndef PR_MTE_TCF_SHIFT
+# define PR_MTE_TCF_SHIFT 1
+# define PR_MTE_TCF_NONE (0UL << PR_MTE_TCF_SHIFT)
+# define PR_MTE_TCF_SYNC (1UL << PR_MTE_TCF_SHIFT)
+# define PR_MTE_TCF_ASYNC (2UL << PR_MTE_TCF_SHIFT)
+# define PR_MTE_TCF_MASK (3UL << PR_MTE_TCF_SHIFT)
+# define PR_MTE_TAG_SHIFT 3
+# define PR_MTE_TAG_MASK (0xffffUL << PR_MTE_TAG_SHIFT)
+#endif
+
__BEGIN_DECLS
/* Control process execution. */
^ permalink raw reply [flat|nested] 25+ messages in thread
* [PATCH v4 5/6] aarch64: Add sysv specific enabling code for memory tagging
2020-12-18 19:29 [PATCH v4 0/6] Memory tagging support Richard Earnshaw
` (3 preceding siblings ...)
2020-12-18 19:29 ` [PATCH v4 4/6] linux: Add compatibility definitions to sys/prctl.h for MTE Richard Earnshaw
@ 2020-12-18 19:29 ` Richard Earnshaw
2020-12-21 12:27 ` Szabolcs Nagy
2020-12-21 13:36 ` Siddhesh Poyarekar
2020-12-18 19:29 ` [PATCH v4 6/6] aarch64: Add aarch64-specific files for memory tagging support Richard Earnshaw
` (2 subsequent siblings)
7 siblings, 2 replies; 25+ messages in thread
From: Richard Earnshaw @ 2020-12-18 19:29 UTC (permalink / raw)
To: libc-alpha; +Cc: Richard Earnshaw, dj, szabolcs.nagy
[-- Attachment #1: Type: text/plain, Size: 853 bytes --]
Add various defines and stubs for enabling MTE on AArch64 sysv-like
systems such as Linux. The HWCAP feature bit is copied over in the
same way as other feature bits. Similarly we add a new wrapper header
for mman.h to define the PROT_MTE flag that can be used with mmap and
related functions.
We add a new field to struct cpu_features that can be used, for
example, to check whether or not certain ifunc'd routines should be
bound to MTE-safe versions.
Finally, if we detect that MTE should be enabled (ie via the glibc
tunable); we enable MTE during startup as required.
---
sysdeps/unix/sysv/linux/aarch64/bits/hwcap.h | 1 +
sysdeps/unix/sysv/linux/aarch64/bits/mman.h | 1 +
.../unix/sysv/linux/aarch64/cpu-features.c | 30 +++++++++++++++++++
.../unix/sysv/linux/aarch64/cpu-features.h | 2 ++
4 files changed, 34 insertions(+)
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: v4-0005-aarch64-Add-sysv-specific-enabling-code-for-memor.patch --]
[-- Type: text/x-patch; name="v4-0005-aarch64-Add-sysv-specific-enabling-code-for-memor.patch", Size: 3139 bytes --]
diff --git a/sysdeps/unix/sysv/linux/aarch64/bits/hwcap.h b/sysdeps/unix/sysv/linux/aarch64/bits/hwcap.h
index af90d8a626..389852f1d9 100644
--- a/sysdeps/unix/sysv/linux/aarch64/bits/hwcap.h
+++ b/sysdeps/unix/sysv/linux/aarch64/bits/hwcap.h
@@ -73,3 +73,4 @@
#define HWCAP2_DGH (1 << 15)
#define HWCAP2_RNG (1 << 16)
#define HWCAP2_BTI (1 << 17)
+#define HWCAP2_MTE (1 << 18)
diff --git a/sysdeps/unix/sysv/linux/aarch64/bits/mman.h b/sysdeps/unix/sysv/linux/aarch64/bits/mman.h
index ecae046344..c5ec0aa7d0 100644
--- a/sysdeps/unix/sysv/linux/aarch64/bits/mman.h
+++ b/sysdeps/unix/sysv/linux/aarch64/bits/mman.h
@@ -24,6 +24,7 @@
arch/arm64/include/uapi/asm/mman.h. */
#define PROT_BTI 0x10
+#define PROT_MTE 0x20
#include <bits/mman-map-flags-generic.h>
diff --git a/sysdeps/unix/sysv/linux/aarch64/cpu-features.c b/sysdeps/unix/sysv/linux/aarch64/cpu-features.c
index b9ab827aca..bd899c4b09 100644
--- a/sysdeps/unix/sysv/linux/aarch64/cpu-features.c
+++ b/sysdeps/unix/sysv/linux/aarch64/cpu-features.c
@@ -19,10 +19,17 @@
#include <cpu-features.h>
#include <sys/auxv.h>
#include <elf/dl-hwcaps.h>
+#include <sys/prctl.h>
#define DCZID_DZP_MASK (1 << 4)
#define DCZID_BS_MASK (0xf)
+/* The maximal set of permitted tags that the MTE random tag generation
+ instruction may use. We exclude tag 0 because a) we want to reserve
+ that for the libc heap structures and b) because it makes it easier
+ to see when pointer have been correctly tagged. */
+#define MTE_ALLOWED_TAGS (0xfffe << PR_MTE_TAG_SHIFT)
+
#if HAVE_TUNABLES
struct cpu_list
{
@@ -86,4 +93,27 @@ init_cpu_features (struct cpu_features *cpu_features)
/* Check if BTI is supported. */
cpu_features->bti = GLRO (dl_hwcap2) & HWCAP2_BTI;
+
+ /* Setup memory tagging support if the HW and kernel support it, and if
+ the user has requested it. */
+ cpu_features->mte_state = 0;
+
+#ifdef USE_MTAG
+# if HAVE_TUNABLES
+ int mte_state = TUNABLE_GET (glibc, mem, tagging, unsigned, 0);
+ cpu_features->mte_state = (GLRO (dl_hwcap2) & HWCAP2_MTE) ? mte_state : 0;
+ /* If we lack the MTE feature, disable the tunable, since it will
+ otherwise cause instructions that won't run on this CPU to be used. */
+ TUNABLE_SET (glibc, mem, tagging, unsigned, cpu_features->mte_state);
+# endif
+
+ if (cpu_features->mte_state & 2)
+ __prctl (PR_SET_TAGGED_ADDR_CTRL,
+ (PR_TAGGED_ADDR_ENABLE | PR_MTE_TCF_SYNC | MTE_ALLOWED_TAGS),
+ 0, 0, 0);
+ else if (cpu_features->mte_state)
+ __prctl (PR_SET_TAGGED_ADDR_CTRL,
+ (PR_TAGGED_ADDR_ENABLE | PR_MTE_TCF_ASYNC | MTE_ALLOWED_TAGS),
+ 0, 0, 0);
+#endif
}
diff --git a/sysdeps/unix/sysv/linux/aarch64/cpu-features.h b/sysdeps/unix/sysv/linux/aarch64/cpu-features.h
index 00a4d0c8e7..bebf321a21 100644
--- a/sysdeps/unix/sysv/linux/aarch64/cpu-features.h
+++ b/sysdeps/unix/sysv/linux/aarch64/cpu-features.h
@@ -70,6 +70,8 @@ struct cpu_features
uint64_t midr_el1;
unsigned zva_size;
bool bti;
+ /* Currently, the GLIBC memory tagging tunable only defines 8 bits. */
+ uint8_t mte_state;
};
#endif /* _CPU_FEATURES_AARCH64_H */
^ permalink raw reply [flat|nested] 25+ messages in thread
* [PATCH v4 6/6] aarch64: Add aarch64-specific files for memory tagging support
2020-12-18 19:29 [PATCH v4 0/6] Memory tagging support Richard Earnshaw
` (4 preceding siblings ...)
2020-12-18 19:29 ` [PATCH v4 5/6] aarch64: Add sysv specific enabling code for memory tagging Richard Earnshaw
@ 2020-12-18 19:29 ` Richard Earnshaw
2020-12-21 12:44 ` Szabolcs Nagy
2020-12-18 20:18 ` [PATCH v4 0/6] Memory " H.J. Lu
2020-12-21 12:28 ` Siddhesh Poyarekar
7 siblings, 1 reply; 25+ messages in thread
From: Richard Earnshaw @ 2020-12-18 19:29 UTC (permalink / raw)
To: libc-alpha; +Cc: Richard Earnshaw, dj, szabolcs.nagy
[-- Attachment #1: Type: text/plain, Size: 818 bytes --]
This final patch provides the architecture-specific implementation of
the memory-tagging support hooks for aarch64.
---
sysdeps/aarch64/Makefile | 5 +++
sysdeps/aarch64/__mtag_address_get_tag.S | 32 +++++++++++++
sysdeps/aarch64/__mtag_memset_tag.S | 53 ++++++++++++++++++++++
sysdeps/aarch64/__mtag_new_tag.S | 37 +++++++++++++++
sysdeps/aarch64/__mtag_tag_region.S | 51 +++++++++++++++++++++
sysdeps/aarch64/libc-mtag.h | 57 ++++++++++++++++++++++++
6 files changed, 235 insertions(+)
create mode 100644 sysdeps/aarch64/__mtag_address_get_tag.S
create mode 100644 sysdeps/aarch64/__mtag_memset_tag.S
create mode 100644 sysdeps/aarch64/__mtag_new_tag.S
create mode 100644 sysdeps/aarch64/__mtag_tag_region.S
create mode 100644 sysdeps/aarch64/libc-mtag.h
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: v4-0006-aarch64-Add-aarch64-specific-files-for-memory-tag.patch --]
[-- Type: text/x-patch; name="v4-0006-aarch64-Add-aarch64-specific-files-for-memory-tag.patch", Size: 9021 bytes --]
diff --git a/sysdeps/aarch64/Makefile b/sysdeps/aarch64/Makefile
index d8e06d27b2..d3ab37a40a 100644
--- a/sysdeps/aarch64/Makefile
+++ b/sysdeps/aarch64/Makefile
@@ -40,4 +40,9 @@ endif
ifeq ($(subdir),misc)
sysdep_headers += sys/ifunc.h
+sysdep_routines += __mtag_address_get_tag \
+ __mtag_memset_tag \
+ __mtag_new_tag \
+ __mtag_tag_region
+
endif
diff --git a/sysdeps/aarch64/__mtag_address_get_tag.S b/sysdeps/aarch64/__mtag_address_get_tag.S
new file mode 100644
index 0000000000..4ffe884c95
--- /dev/null
+++ b/sysdeps/aarch64/__mtag_address_get_tag.S
@@ -0,0 +1,32 @@
+/* Copyright (C) 2020 Free Software Foundation, Inc.
+
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library. If not, see
+ <http://www.gnu.org/licenses/>. */
+
+#include <sysdep.h>
+
+#ifdef USE_MTAG
+#define ptr x0
+
+ .arch armv8.5-a
+ .arch_extension memtag
+
+ENTRY (__libc_mtag_address_get_tag)
+
+ ldg ptr, [ptr]
+ ret
+END (__libc_mtag_address_get_tag)
+#endif /* USE_MTAG */
diff --git a/sysdeps/aarch64/__mtag_memset_tag.S b/sysdeps/aarch64/__mtag_memset_tag.S
new file mode 100644
index 0000000000..6f78d91b79
--- /dev/null
+++ b/sysdeps/aarch64/__mtag_memset_tag.S
@@ -0,0 +1,53 @@
+/* Copyright (C) 2020 Free Software Foundation, Inc.
+
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library. If not, see
+ <http://www.gnu.org/licenses/>. */
+
+#include <sysdep.h>
+
+#ifdef USE_MTAG
+
+/* Use the same register names and assignments as memset. */
+#include "memset-reg.h"
+
+ .arch armv8.5-a
+ .arch_extension memtag
+
+/* NB, only supported on variants with 64-bit pointers. */
+
+/* FIXME: This is a minimal implementation. We could do much better than
+ this for large values of COUNT. */
+
+ENTRY(__libc_mtag_memset_with_tag)
+
+ and valw, valw, 255
+ orr valw, valw, valw, lsl 8
+ orr valw, valw, valw, lsl 16
+ orr val, val, val, lsl 32
+ mov dst, dstin
+
+L(loop):
+ stgp val, val, [dst], #16
+ subs count, count, 16
+ bne L(loop)
+#if 0
+ /* This is not currently needed, since for now we are only called
+ to tag memory that is taggable. */
+ ldg dstin, [dstin] // Recover the tag created (might be untagged).
+#endif
+ ret
+END (__libc_mtag_memset_with_tag)
+#endif /* USE_MTAG */
diff --git a/sysdeps/aarch64/__mtag_new_tag.S b/sysdeps/aarch64/__mtag_new_tag.S
new file mode 100644
index 0000000000..663e3a484f
--- /dev/null
+++ b/sysdeps/aarch64/__mtag_new_tag.S
@@ -0,0 +1,37 @@
+/* Copyright (C) 2020 Free Software Foundation, Inc.
+
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library. If not, see
+ <http://www.gnu.org/licenses/>. */
+
+#include <sysdep.h>
+
+#ifdef USE_MTAG
+
+ .arch armv8.5-a
+ .arch_extension memtag
+
+/* NB, only supported on variants with 64-bit pointers. */
+
+#define ptr x0
+#define xset x1
+
+ENTRY(__libc_mtag_new_tag)
+ // Guarantee that the new tag is not the same as now.
+ gmi xset, ptr, xzr
+ irg ptr, ptr, xset
+ ret
+END (__libc_mtag_new_tag)
+#endif /* USE_MTAG */
diff --git a/sysdeps/aarch64/__mtag_tag_region.S b/sysdeps/aarch64/__mtag_tag_region.S
new file mode 100644
index 0000000000..7af4f0fa0a
--- /dev/null
+++ b/sysdeps/aarch64/__mtag_tag_region.S
@@ -0,0 +1,51 @@
+/* Copyright (C) 2020 Free Software Foundation, Inc.
+
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library. If not, see
+ <http://www.gnu.org/licenses/>. */
+
+#include <sysdep.h>
+
+#ifdef USE_MTAG
+
+/* Use the same register names and assignments as memset. */
+
+ .arch armv8.5-a
+ .arch_extension memtag
+
+/* NB, only supported on variants with 64-bit pointers. */
+
+/* FIXME: This is a minimal implementation. We could do better than
+ this for larger values of COUNT. */
+
+#define dstin x0
+#define count x1
+#define dst x2
+
+ENTRY_ALIGN(__libc_mtag_tag_region, 6)
+
+ mov dst, dstin
+L(loop):
+ stg dst, [dst], #16
+ subs count, count, 16
+ bne L(loop)
+#if 0
+ /* This is not currently needed, since for now we are only called
+ to tag memory that is taggable. */
+ ldg dstin, [dstin] // Recover the tag created (might be untagged).
+#endif
+ ret
+END (__libc_mtag_tag_region)
+#endif /* USE_MTAG */
diff --git a/sysdeps/aarch64/libc-mtag.h b/sysdeps/aarch64/libc-mtag.h
new file mode 100644
index 0000000000..0e9b63e95b
--- /dev/null
+++ b/sysdeps/aarch64/libc-mtag.h
@@ -0,0 +1,57 @@
+/* libc-internal interface for tagged (colored) memory support.
+ Copyright (C) 2020 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#ifndef _AARCH64_LIBC_MTAG_H
+#define _AARCH64_LIBC_MTAG_H 1
+
+#ifndef USE_MTAG
+/* Generic bindings for systems that do not support memory tagging. */
+#include_next "libc-mtag.h"
+#else
+
+/* Used to ensure additional alignment when objects need to have distinct
+ tags. */
+#define __MTAG_GRANULE_SIZE 16
+
+/* Non-zero if memory obtained via morecore (sbrk) is not tagged. */
+#define __MTAG_SBRK_UNTAGGED 1
+
+/* Extra flags to pass to mmap to get tagged pages. */
+#define __MTAG_MMAP_FLAGS PROT_MTE
+
+/* Set the tags for a region of memory, which must have size and alignment
+ that are multiples of __MTAG_GRANULE_SIZE. Size cannot be zero.
+ void *__libc_mtag_tag_region (const void *, size_t) */
+void *__libc_mtag_tag_region (void *, size_t);
+
+/* Optimized equivalent to __libc_mtag_tag_region followed by memset. */
+void *__libc_mtag_memset_with_tag(void *, int, size_t);
+
+/* Convert address P to a pointer that is tagged correctly for that
+ location.
+ void *__libc_mtag_address_get_tag (void*) */
+void *__libc_mtag_address_get_tag(void *);
+
+/* Assign a new (random) tag to a pointer P (does not adjust the tag on
+ the memory addressed).
+ void *__libc_mtag_new_tag (void*) */
+void *__libc_mtag_new_tag(void *);
+
+#endif /* USE_MTAG */
+
+#endif /* _AARCH64_LIBC_MTAG_H */
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v4 0/6] Memory tagging support
2020-12-18 19:29 [PATCH v4 0/6] Memory tagging support Richard Earnshaw
` (5 preceding siblings ...)
2020-12-18 19:29 ` [PATCH v4 6/6] aarch64: Add aarch64-specific files for memory tagging support Richard Earnshaw
@ 2020-12-18 20:18 ` H.J. Lu
2020-12-21 12:28 ` Siddhesh Poyarekar
7 siblings, 0 replies; 25+ messages in thread
From: H.J. Lu @ 2020-12-18 20:18 UTC (permalink / raw)
To: Richard Earnshaw; +Cc: GNU C Library
On Fri, Dec 18, 2020 at 11:31 AM Richard Earnshaw via Libc-alpha
<libc-alpha@sourceware.org> wrote:
>
> Main changes in this version:
>
> - collapse the changes to the malloc code to a single patch.
> - change _LIBC_MTAG to USE_MTAG.
Should we use ENABLE_MTAG instead of USE_MTAG? The same
glibc binary should work with and without memory tag
> - comments around definition of PROT_MTE.
> - tunable renamed to glibc.mem.tagging.
> - cleanups to assembler files for aarch64 support.
>
> I'll push a copy of this patch set to rearnsha/mte-v4.0 shortly.
>
> Richard Earnshaw (6):
> config: Allow memory tagging to be enabled when configuring glibc
> elf: Add a tunable to control use of tagged memory
> malloc: Basic support for memory tagging in the malloc() family
> linux: Add compatibility definitions to sys/prctl.h for MTE
> aarch64: Add sysv specific enabling code for memory tagging
> aarch64: Add aarch64-specific files for memory tagging support
>
> INSTALL | 14 +
> config.h.in | 3 +
> config.make.in | 2 +
> configure | 22 ++
> configure.ac | 15 +
> elf/dl-tunables.list | 9 +
> malloc/arena.c | 59 ++-
> malloc/hooks.c | 79 ++--
> malloc/malloc.c | 336 ++++++++++++++----
> malloc/malloc.h | 7 +
> manual/install.texi | 13 +
> manual/tunables.texi | 35 ++
> sysdeps/aarch64/Makefile | 5 +
> sysdeps/aarch64/__mtag_address_get_tag.S | 32 ++
> sysdeps/aarch64/__mtag_memset_tag.S | 53 +++
> sysdeps/aarch64/__mtag_new_tag.S | 37 ++
> sysdeps/aarch64/__mtag_tag_region.S | 51 +++
> sysdeps/aarch64/libc-mtag.h | 57 +++
> sysdeps/generic/libc-mtag.h | 52 +++
> sysdeps/unix/sysv/linux/aarch64/bits/hwcap.h | 1 +
> sysdeps/unix/sysv/linux/aarch64/bits/mman.h | 1 +
> .../unix/sysv/linux/aarch64/cpu-features.c | 30 ++
> .../unix/sysv/linux/aarch64/cpu-features.h | 2 +
> sysdeps/unix/sysv/linux/sys/prctl.h | 18 +
> 24 files changed, 836 insertions(+), 97 deletions(-)
> create mode 100644 sysdeps/aarch64/__mtag_address_get_tag.S
> create mode 100644 sysdeps/aarch64/__mtag_memset_tag.S
> create mode 100644 sysdeps/aarch64/__mtag_new_tag.S
> create mode 100644 sysdeps/aarch64/__mtag_tag_region.S
> create mode 100644 sysdeps/aarch64/libc-mtag.h
> create mode 100644 sysdeps/generic/libc-mtag.h
>
> --
> 2.29.2
>
--
H.J.
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v4 5/6] aarch64: Add sysv specific enabling code for memory tagging
2020-12-18 19:29 ` [PATCH v4 5/6] aarch64: Add sysv specific enabling code for memory tagging Richard Earnshaw
@ 2020-12-21 12:27 ` Szabolcs Nagy
2020-12-21 13:36 ` Siddhesh Poyarekar
1 sibling, 0 replies; 25+ messages in thread
From: Szabolcs Nagy @ 2020-12-21 12:27 UTC (permalink / raw)
To: Richard Earnshaw; +Cc: libc-alpha, dj
The 12/18/2020 19:29, Richard Earnshaw wrote:
> Add various defines and stubs for enabling MTE on AArch64 sysv-like
> systems such as Linux. The HWCAP feature bit is copied over in the
> same way as other feature bits. Similarly we add a new wrapper header
> for mman.h to define the PROT_MTE flag that can be used with mmap and
> related functions.
>
> We add a new field to struct cpu_features that can be used, for
> example, to check whether or not certain ifunc'd routines should be
> bound to MTE-safe versions.
>
> Finally, if we detect that MTE should be enabled (ie via the glibc
> tunable); we enable MTE during startup as required.
i'd mention in the commit message that the linux api is
committed in linux version 5.10.
Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com>
> ---
> sysdeps/unix/sysv/linux/aarch64/bits/hwcap.h | 1 +
> sysdeps/unix/sysv/linux/aarch64/bits/mman.h | 1 +
> .../unix/sysv/linux/aarch64/cpu-features.c | 30 +++++++++++++++++++
> .../unix/sysv/linux/aarch64/cpu-features.h | 2 ++
> 4 files changed, 34 insertions(+)
>
> diff --git a/sysdeps/unix/sysv/linux/aarch64/bits/hwcap.h b/sysdeps/unix/sysv/linux/aarch64/bits/hwcap.h
> index af90d8a626..389852f1d9 100644
> --- a/sysdeps/unix/sysv/linux/aarch64/bits/hwcap.h
> +++ b/sysdeps/unix/sysv/linux/aarch64/bits/hwcap.h
> @@ -73,3 +73,4 @@
> #define HWCAP2_DGH (1 << 15)
> #define HWCAP2_RNG (1 << 16)
> #define HWCAP2_BTI (1 << 17)
> +#define HWCAP2_MTE (1 << 18)
> diff --git a/sysdeps/unix/sysv/linux/aarch64/bits/mman.h b/sysdeps/unix/sysv/linux/aarch64/bits/mman.h
> index ecae046344..c5ec0aa7d0 100644
> --- a/sysdeps/unix/sysv/linux/aarch64/bits/mman.h
> +++ b/sysdeps/unix/sysv/linux/aarch64/bits/mman.h
> @@ -24,6 +24,7 @@
> arch/arm64/include/uapi/asm/mman.h. */
>
> #define PROT_BTI 0x10
> +#define PROT_MTE 0x20
>
> #include <bits/mman-map-flags-generic.h>
>
> diff --git a/sysdeps/unix/sysv/linux/aarch64/cpu-features.c b/sysdeps/unix/sysv/linux/aarch64/cpu-features.c
> index b9ab827aca..bd899c4b09 100644
> --- a/sysdeps/unix/sysv/linux/aarch64/cpu-features.c
> +++ b/sysdeps/unix/sysv/linux/aarch64/cpu-features.c
> @@ -19,10 +19,17 @@
> #include <cpu-features.h>
> #include <sys/auxv.h>
> #include <elf/dl-hwcaps.h>
> +#include <sys/prctl.h>
>
> #define DCZID_DZP_MASK (1 << 4)
> #define DCZID_BS_MASK (0xf)
>
> +/* The maximal set of permitted tags that the MTE random tag generation
> + instruction may use. We exclude tag 0 because a) we want to reserve
> + that for the libc heap structures and b) because it makes it easier
> + to see when pointer have been correctly tagged. */
> +#define MTE_ALLOWED_TAGS (0xfffe << PR_MTE_TAG_SHIFT)
> +
> #if HAVE_TUNABLES
> struct cpu_list
> {
> @@ -86,4 +93,27 @@ init_cpu_features (struct cpu_features *cpu_features)
>
> /* Check if BTI is supported. */
> cpu_features->bti = GLRO (dl_hwcap2) & HWCAP2_BTI;
> +
> + /* Setup memory tagging support if the HW and kernel support it, and if
> + the user has requested it. */
> + cpu_features->mte_state = 0;
> +
> +#ifdef USE_MTAG
> +# if HAVE_TUNABLES
> + int mte_state = TUNABLE_GET (glibc, mem, tagging, unsigned, 0);
> + cpu_features->mte_state = (GLRO (dl_hwcap2) & HWCAP2_MTE) ? mte_state : 0;
> + /* If we lack the MTE feature, disable the tunable, since it will
> + otherwise cause instructions that won't run on this CPU to be used. */
> + TUNABLE_SET (glibc, mem, tagging, unsigned, cpu_features->mte_state);
> +# endif
> +
> + if (cpu_features->mte_state & 2)
> + __prctl (PR_SET_TAGGED_ADDR_CTRL,
> + (PR_TAGGED_ADDR_ENABLE | PR_MTE_TCF_SYNC | MTE_ALLOWED_TAGS),
> + 0, 0, 0);
> + else if (cpu_features->mte_state)
> + __prctl (PR_SET_TAGGED_ADDR_CTRL,
> + (PR_TAGGED_ADDR_ENABLE | PR_MTE_TCF_ASYNC | MTE_ALLOWED_TAGS),
> + 0, 0, 0);
> +#endif
> }
> diff --git a/sysdeps/unix/sysv/linux/aarch64/cpu-features.h b/sysdeps/unix/sysv/linux/aarch64/cpu-features.h
> index 00a4d0c8e7..bebf321a21 100644
> --- a/sysdeps/unix/sysv/linux/aarch64/cpu-features.h
> +++ b/sysdeps/unix/sysv/linux/aarch64/cpu-features.h
> @@ -70,6 +70,8 @@ struct cpu_features
> uint64_t midr_el1;
> unsigned zva_size;
> bool bti;
> + /* Currently, the GLIBC memory tagging tunable only defines 8 bits. */
> + uint8_t mte_state;
> };
>
> #endif /* _CPU_FEATURES_AARCH64_H */
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v4 0/6] Memory tagging support
2020-12-18 19:29 [PATCH v4 0/6] Memory tagging support Richard Earnshaw
` (6 preceding siblings ...)
2020-12-18 20:18 ` [PATCH v4 0/6] Memory " H.J. Lu
@ 2020-12-21 12:28 ` Siddhesh Poyarekar
2020-12-21 13:44 ` Siddhesh Poyarekar
7 siblings, 1 reply; 25+ messages in thread
From: Siddhesh Poyarekar @ 2020-12-21 12:28 UTC (permalink / raw)
To: Richard Earnshaw, libc-alpha
On 12/19/20 12:59 AM, Richard Earnshaw via Libc-alpha wrote:
> Main changes in this version:
>
> - collapse the changes to the malloc code to a single patch.
> - change _LIBC_MTAG to USE_MTAG.
> - comments around definition of PROT_MTE.
> - tunable renamed to glibc.mem.tagging.
> - cleanups to assembler files for aarch64 support.
>
I've tested the build to verify that --enable-memory-tagging works
correctly on x86_64 and the resulting bins don't regress the testsuite.
I'm working through the individual patches to review; will post
comments shortly.
Siddhesh
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v4 1/6] config: Allow memory tagging to be enabled when configuring glibc
2020-12-18 19:29 ` [PATCH v4 1/6] config: Allow memory tagging to be enabled when configuring glibc Richard Earnshaw
@ 2020-12-21 12:40 ` Siddhesh Poyarekar
0 siblings, 0 replies; 25+ messages in thread
From: Siddhesh Poyarekar @ 2020-12-21 12:40 UTC (permalink / raw)
To: Richard Earnshaw, libc-alpha; +Cc: Richard Earnshaw
On 12/19/20 12:59 AM, Richard Earnshaw via Libc-alpha wrote:
>
> This patch adds the configuration machinery to allow memory tagging to be
> enabled from the command line via the configure option --enable-memory-tagging.
>
> The current default is off, though in time we may change that once the API
> is more stable.
> ---
> INSTALL | 14 ++++++++++++++
> config.h.in | 3 +++
> config.make.in | 2 ++
> configure | 22 ++++++++++++++++++++++
> configure.ac | 15 +++++++++++++++
> manual/install.texi | 13 +++++++++++++
> 6 files changed, 69 insertions(+)
>
This looks fine now; the messaging in INSTALL matches what we do, i.e.
enable memory tagging only if the architecture supports it.
Siddhesh
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v4 2/6] elf: Add a tunable to control use of tagged memory
2020-12-18 19:29 ` [PATCH v4 2/6] elf: Add a tunable to control use of tagged memory Richard Earnshaw
@ 2020-12-21 12:42 ` Siddhesh Poyarekar
0 siblings, 0 replies; 25+ messages in thread
From: Siddhesh Poyarekar @ 2020-12-21 12:42 UTC (permalink / raw)
To: Richard Earnshaw, libc-alpha
On 12/19/20 12:59 AM, Richard Earnshaw via Libc-alpha wrote:
>
> Add a new glibc tunable: mem.tagging. This is a decimal constant in
> the range 0-255 but used as a bit-field.
>
> Bit 0 enables use of tagged memory in the malloc family of functions.
> Bit 1 enables precise faulting of tag failure on platforms where this
> can be controlled.
> Other bits are currently unused, but if set will cause memory tag
> checking for the current process to be enabled in the kernel.
> ---
> elf/dl-tunables.list | 9 +++++++++
> manual/tunables.texi | 35 +++++++++++++++++++++++++++++++++++
> 2 files changed, 44 insertions(+)
>
> diff --git a/elf/dl-tunables.list b/elf/dl-tunables.list
> index e1d8225128..4c44ead715 100644
> --- a/elf/dl-tunables.list
> +++ b/elf/dl-tunables.list
> @@ -141,4 +141,13 @@ glibc {
> default: 512
> }
> }
> +
> + mem {
> + tagging {
> + type: INT_32
> + minval: 0
> + maxval: 255
> + security_level: SXID_IGNORE
> + }
> + }
> }
OK.
> diff --git a/manual/tunables.texi b/manual/tunables.texi
> index d72d7a5ec0..1bbdc88281 100644
> --- a/manual/tunables.texi
> +++ b/manual/tunables.texi
> @@ -36,6 +36,8 @@ their own namespace.
> * POSIX Thread Tunables:: Tunables in the POSIX thread subsystem
> * Hardware Capability Tunables:: Tunables that modify the hardware
> capabilities seen by @theglibc{}
> +* Memory Related Tunables:: Tunables that control the use of memory by
> + @theglibc{}.
> @end menu
>
> @node Tunable names
> @@ -484,3 +486,36 @@ instead.
>
> This tunable is specific to i386 and x86-64.
> @end deftp
> +
> +@node Memory Related Tunables
> +@section Memory Related Tunables
> +@cindex memory related tunables
> +
> +@deftp {Tunable namespace} glibc.mem
> +This tunable namespace supports operations that affect the way @theglibc{}
> +and the process manage memory.
> +@end deftp
> +
That's a good description.
> +@deftp Tunable glibc.mem.tagging
> +If the hardware supports memory tagging, this tunable can be used to
> +control the way @theglibc{} uses this feature. At present this is only
> +supported on AArch64 systems with the MTE extention; it is ignored for
> +all other systems.
> +
> +This tunable takes a value between 0 and 255 and acts as a bitmask
> +that enables various capabilities.
> +
> +Bit 0 (the least significant bit) causes the malloc subsystem to allocate
> +tagged memory, with each allocation being assigned a random tag.
> +
> +Bit 1 enables precise faulting mode for tag violations on systems that
> +support deferred tag violation reporting. This may cause programs
> +to run more slowly.
> +
> +Other bits are currently reserved.
> +
> +@Theglibc{} startup code will automatically enable memory tagging
> +support in the kernel if this tunable has any non-zero value.
> +
> +The default value is @samp{0}, which disables all memory tagging.
> +@end deftp
Looks good to me.
Siddhesh
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v4 6/6] aarch64: Add aarch64-specific files for memory tagging support
2020-12-18 19:29 ` [PATCH v4 6/6] aarch64: Add aarch64-specific files for memory tagging support Richard Earnshaw
@ 2020-12-21 12:44 ` Szabolcs Nagy
2020-12-21 12:50 ` Richard Earnshaw
0 siblings, 1 reply; 25+ messages in thread
From: Szabolcs Nagy @ 2020-12-21 12:44 UTC (permalink / raw)
To: Richard Earnshaw; +Cc: libc-alpha, dj
The 12/18/2020 19:29, Richard Earnshaw wrote:
> This final patch provides the architecture-specific implementation of
> the memory-tagging support hooks for aarch64.
i see some gnu style issues in libc-mtag.h otherwise it looks ok.
Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com>
> ---
> sysdeps/aarch64/Makefile | 5 +++
> sysdeps/aarch64/__mtag_address_get_tag.S | 32 +++++++++++++
> sysdeps/aarch64/__mtag_memset_tag.S | 53 ++++++++++++++++++++++
> sysdeps/aarch64/__mtag_new_tag.S | 37 +++++++++++++++
> sysdeps/aarch64/__mtag_tag_region.S | 51 +++++++++++++++++++++
> sysdeps/aarch64/libc-mtag.h | 57 ++++++++++++++++++++++++
> 6 files changed, 235 insertions(+)
> create mode 100644 sysdeps/aarch64/__mtag_address_get_tag.S
> create mode 100644 sysdeps/aarch64/__mtag_memset_tag.S
> create mode 100644 sysdeps/aarch64/__mtag_new_tag.S
> create mode 100644 sysdeps/aarch64/__mtag_tag_region.S
> create mode 100644 sysdeps/aarch64/libc-mtag.h
>
> diff --git a/sysdeps/aarch64/Makefile b/sysdeps/aarch64/Makefile
> index d8e06d27b2..d3ab37a40a 100644
> --- a/sysdeps/aarch64/Makefile
> +++ b/sysdeps/aarch64/Makefile
> @@ -40,4 +40,9 @@ endif
>
> ifeq ($(subdir),misc)
> sysdep_headers += sys/ifunc.h
> +sysdep_routines += __mtag_address_get_tag \
> + __mtag_memset_tag \
> + __mtag_new_tag \
> + __mtag_tag_region
> +
> endif
> diff --git a/sysdeps/aarch64/__mtag_address_get_tag.S b/sysdeps/aarch64/__mtag_address_get_tag.S
> new file mode 100644
> index 0000000000..4ffe884c95
> --- /dev/null
> +++ b/sysdeps/aarch64/__mtag_address_get_tag.S
> @@ -0,0 +1,32 @@
> +/* Copyright (C) 2020 Free Software Foundation, Inc.
> +
> + This file is part of the GNU C Library.
> +
> + The GNU C Library is free software; you can redistribute it and/or
> + modify it under the terms of the GNU Lesser General Public
> + License as published by the Free Software Foundation; either
> + version 2.1 of the License, or (at your option) any later version.
> +
> + The GNU C Library is distributed in the hope that it will be useful,
> + but WITHOUT ANY WARRANTY; without even the implied warranty of
> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + Lesser General Public License for more details.
> +
> + You should have received a copy of the GNU Lesser General Public
> + License along with the GNU C Library. If not, see
> + <http://www.gnu.org/licenses/>. */
> +
> +#include <sysdep.h>
> +
> +#ifdef USE_MTAG
> +#define ptr x0
> +
> + .arch armv8.5-a
> + .arch_extension memtag
> +
> +ENTRY (__libc_mtag_address_get_tag)
> +
> + ldg ptr, [ptr]
> + ret
> +END (__libc_mtag_address_get_tag)
> +#endif /* USE_MTAG */
> diff --git a/sysdeps/aarch64/__mtag_memset_tag.S b/sysdeps/aarch64/__mtag_memset_tag.S
> new file mode 100644
> index 0000000000..6f78d91b79
> --- /dev/null
> +++ b/sysdeps/aarch64/__mtag_memset_tag.S
> @@ -0,0 +1,53 @@
> +/* Copyright (C) 2020 Free Software Foundation, Inc.
> +
> + This file is part of the GNU C Library.
> +
> + The GNU C Library is free software; you can redistribute it and/or
> + modify it under the terms of the GNU Lesser General Public
> + License as published by the Free Software Foundation; either
> + version 2.1 of the License, or (at your option) any later version.
> +
> + The GNU C Library is distributed in the hope that it will be useful,
> + but WITHOUT ANY WARRANTY; without even the implied warranty of
> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + Lesser General Public License for more details.
> +
> + You should have received a copy of the GNU Lesser General Public
> + License along with the GNU C Library. If not, see
> + <http://www.gnu.org/licenses/>. */
> +
> +#include <sysdep.h>
> +
> +#ifdef USE_MTAG
> +
> +/* Use the same register names and assignments as memset. */
> +#include "memset-reg.h"
> +
> + .arch armv8.5-a
> + .arch_extension memtag
> +
> +/* NB, only supported on variants with 64-bit pointers. */
> +
> +/* FIXME: This is a minimal implementation. We could do much better than
> + this for large values of COUNT. */
> +
> +ENTRY(__libc_mtag_memset_with_tag)
> +
> + and valw, valw, 255
> + orr valw, valw, valw, lsl 8
> + orr valw, valw, valw, lsl 16
> + orr val, val, val, lsl 32
> + mov dst, dstin
> +
> +L(loop):
> + stgp val, val, [dst], #16
> + subs count, count, 16
> + bne L(loop)
> +#if 0
> + /* This is not currently needed, since for now we are only called
> + to tag memory that is taggable. */
> + ldg dstin, [dstin] // Recover the tag created (might be untagged).
> +#endif
> + ret
> +END (__libc_mtag_memset_with_tag)
> +#endif /* USE_MTAG */
> diff --git a/sysdeps/aarch64/__mtag_new_tag.S b/sysdeps/aarch64/__mtag_new_tag.S
> new file mode 100644
> index 0000000000..663e3a484f
> --- /dev/null
> +++ b/sysdeps/aarch64/__mtag_new_tag.S
> @@ -0,0 +1,37 @@
> +/* Copyright (C) 2020 Free Software Foundation, Inc.
> +
> + This file is part of the GNU C Library.
> +
> + The GNU C Library is free software; you can redistribute it and/or
> + modify it under the terms of the GNU Lesser General Public
> + License as published by the Free Software Foundation; either
> + version 2.1 of the License, or (at your option) any later version.
> +
> + The GNU C Library is distributed in the hope that it will be useful,
> + but WITHOUT ANY WARRANTY; without even the implied warranty of
> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + Lesser General Public License for more details.
> +
> + You should have received a copy of the GNU Lesser General Public
> + License along with the GNU C Library. If not, see
> + <http://www.gnu.org/licenses/>. */
> +
> +#include <sysdep.h>
> +
> +#ifdef USE_MTAG
> +
> + .arch armv8.5-a
> + .arch_extension memtag
> +
> +/* NB, only supported on variants with 64-bit pointers. */
> +
> +#define ptr x0
> +#define xset x1
> +
> +ENTRY(__libc_mtag_new_tag)
> + // Guarantee that the new tag is not the same as now.
> + gmi xset, ptr, xzr
> + irg ptr, ptr, xset
> + ret
> +END (__libc_mtag_new_tag)
> +#endif /* USE_MTAG */
> diff --git a/sysdeps/aarch64/__mtag_tag_region.S b/sysdeps/aarch64/__mtag_tag_region.S
> new file mode 100644
> index 0000000000..7af4f0fa0a
> --- /dev/null
> +++ b/sysdeps/aarch64/__mtag_tag_region.S
> @@ -0,0 +1,51 @@
> +/* Copyright (C) 2020 Free Software Foundation, Inc.
> +
> + This file is part of the GNU C Library.
> +
> + The GNU C Library is free software; you can redistribute it and/or
> + modify it under the terms of the GNU Lesser General Public
> + License as published by the Free Software Foundation; either
> + version 2.1 of the License, or (at your option) any later version.
> +
> + The GNU C Library is distributed in the hope that it will be useful,
> + but WITHOUT ANY WARRANTY; without even the implied warranty of
> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + Lesser General Public License for more details.
> +
> + You should have received a copy of the GNU Lesser General Public
> + License along with the GNU C Library. If not, see
> + <http://www.gnu.org/licenses/>. */
> +
> +#include <sysdep.h>
> +
> +#ifdef USE_MTAG
> +
> +/* Use the same register names and assignments as memset. */
> +
> + .arch armv8.5-a
> + .arch_extension memtag
> +
> +/* NB, only supported on variants with 64-bit pointers. */
> +
> +/* FIXME: This is a minimal implementation. We could do better than
> + this for larger values of COUNT. */
> +
> +#define dstin x0
> +#define count x1
> +#define dst x2
> +
> +ENTRY_ALIGN(__libc_mtag_tag_region, 6)
> +
> + mov dst, dstin
> +L(loop):
> + stg dst, [dst], #16
> + subs count, count, 16
> + bne L(loop)
> +#if 0
> + /* This is not currently needed, since for now we are only called
> + to tag memory that is taggable. */
> + ldg dstin, [dstin] // Recover the tag created (might be untagged).
> +#endif
> + ret
> +END (__libc_mtag_tag_region)
> +#endif /* USE_MTAG */
> diff --git a/sysdeps/aarch64/libc-mtag.h b/sysdeps/aarch64/libc-mtag.h
> new file mode 100644
> index 0000000000..0e9b63e95b
> --- /dev/null
> +++ b/sysdeps/aarch64/libc-mtag.h
> @@ -0,0 +1,57 @@
> +/* libc-internal interface for tagged (colored) memory support.
> + Copyright (C) 2020 Free Software Foundation, Inc.
> + This file is part of the GNU C Library.
> +
> + The GNU C Library is free software; you can redistribute it and/or
> + modify it under the terms of the GNU Lesser General Public
> + License as published by the Free Software Foundation; either
> + version 2.1 of the License, or (at your option) any later version.
> +
> + The GNU C Library is distributed in the hope that it will be useful,
> + but WITHOUT ANY WARRANTY; without even the implied warranty of
> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + Lesser General Public License for more details.
> +
> + You should have received a copy of the GNU Lesser General Public
> + License along with the GNU C Library; if not, see
> + <http://www.gnu.org/licenses/>. */
> +
> +#ifndef _AARCH64_LIBC_MTAG_H
> +#define _AARCH64_LIBC_MTAG_H 1
> +
> +#ifndef USE_MTAG
> +/* Generic bindings for systems that do not support memory tagging. */
> +#include_next "libc-mtag.h"
> +#else
> +
> +/* Used to ensure additional alignment when objects need to have distinct
> + tags. */
> +#define __MTAG_GRANULE_SIZE 16
> +
> +/* Non-zero if memory obtained via morecore (sbrk) is not tagged. */
> +#define __MTAG_SBRK_UNTAGGED 1
> +
> +/* Extra flags to pass to mmap to get tagged pages. */
> +#define __MTAG_MMAP_FLAGS PROT_MTE
> +
> +/* Set the tags for a region of memory, which must have size and alignment
> + that are multiples of __MTAG_GRANULE_SIZE. Size cannot be zero.
> + void *__libc_mtag_tag_region (const void *, size_t) */
> +void *__libc_mtag_tag_region (void *, size_t);
> +
> +/* Optimized equivalent to __libc_mtag_tag_region followed by memset. */
> +void *__libc_mtag_memset_with_tag(void *, int, size_t);
> +
> +/* Convert address P to a pointer that is tagged correctly for that
> + location.
> + void *__libc_mtag_address_get_tag (void*) */
> +void *__libc_mtag_address_get_tag(void *);
> +
> +/* Assign a new (random) tag to a pointer P (does not adjust the tag on
> + the memory addressed).
> + void *__libc_mtag_new_tag (void*) */
> +void *__libc_mtag_new_tag(void *);
missing spaces before (
> +
> +#endif /* USE_MTAG */
> +
> +#endif /* _AARCH64_LIBC_MTAG_H */
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v4 6/6] aarch64: Add aarch64-specific files for memory tagging support
2020-12-21 12:44 ` Szabolcs Nagy
@ 2020-12-21 12:50 ` Richard Earnshaw
0 siblings, 0 replies; 25+ messages in thread
From: Richard Earnshaw @ 2020-12-21 12:50 UTC (permalink / raw)
To: Szabolcs Nagy, Richard Earnshaw; +Cc: libc-alpha
On 21/12/2020 12:44, Szabolcs Nagy via Libc-alpha wrote:
> The 12/18/2020 19:29, Richard Earnshaw wrote:
>> This final patch provides the architecture-specific implementation of
>> the memory-tagging support hooks for aarch64.
>
> i see some gnu style issues in libc-mtag.h otherwise it looks ok.
>
> Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com>
>
>> ---
>> sysdeps/aarch64/Makefile | 5 +++
>> sysdeps/aarch64/__mtag_address_get_tag.S | 32 +++++++++++++
>> sysdeps/aarch64/__mtag_memset_tag.S | 53 ++++++++++++++++++++++
>> sysdeps/aarch64/__mtag_new_tag.S | 37 +++++++++++++++
>> sysdeps/aarch64/__mtag_tag_region.S | 51 +++++++++++++++++++++
>> sysdeps/aarch64/libc-mtag.h | 57 ++++++++++++++++++++++++
>> 6 files changed, 235 insertions(+)
>> create mode 100644 sysdeps/aarch64/__mtag_address_get_tag.S
>> create mode 100644 sysdeps/aarch64/__mtag_memset_tag.S
>> create mode 100644 sysdeps/aarch64/__mtag_new_tag.S
>> create mode 100644 sysdeps/aarch64/__mtag_tag_region.S
>> create mode 100644 sysdeps/aarch64/libc-mtag.h
>>
>
>> diff --git a/sysdeps/aarch64/Makefile b/sysdeps/aarch64/Makefile
>> index d8e06d27b2..d3ab37a40a 100644
>> --- a/sysdeps/aarch64/Makefile
>> +++ b/sysdeps/aarch64/Makefile
>> @@ -40,4 +40,9 @@ endif
>>
>> ifeq ($(subdir),misc)
>> sysdep_headers += sys/ifunc.h
>> +sysdep_routines += __mtag_address_get_tag \
>> + __mtag_memset_tag \
>> + __mtag_new_tag \
>> + __mtag_tag_region
>> +
>> endif
>> diff --git a/sysdeps/aarch64/__mtag_address_get_tag.S b/sysdeps/aarch64/__mtag_address_get_tag.S
>> new file mode 100644
>> index 0000000000..4ffe884c95
>> --- /dev/null
>> +++ b/sysdeps/aarch64/__mtag_address_get_tag.S
>> @@ -0,0 +1,32 @@
>> +/* Copyright (C) 2020 Free Software Foundation, Inc.
>> +
>> + This file is part of the GNU C Library.
>> +
>> + The GNU C Library is free software; you can redistribute it and/or
>> + modify it under the terms of the GNU Lesser General Public
>> + License as published by the Free Software Foundation; either
>> + version 2.1 of the License, or (at your option) any later version.
>> +
>> + The GNU C Library is distributed in the hope that it will be useful,
>> + but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
>> + Lesser General Public License for more details.
>> +
>> + You should have received a copy of the GNU Lesser General Public
>> + License along with the GNU C Library. If not, see
>> + <http://www.gnu.org/licenses/>. */
>> +
>> +#include <sysdep.h>
>> +
>> +#ifdef USE_MTAG
>> +#define ptr x0
>> +
>> + .arch armv8.5-a
>> + .arch_extension memtag
>> +
>> +ENTRY (__libc_mtag_address_get_tag)
>> +
>> + ldg ptr, [ptr]
>> + ret
>> +END (__libc_mtag_address_get_tag)
>> +#endif /* USE_MTAG */
>> diff --git a/sysdeps/aarch64/__mtag_memset_tag.S b/sysdeps/aarch64/__mtag_memset_tag.S
>> new file mode 100644
>> index 0000000000..6f78d91b79
>> --- /dev/null
>> +++ b/sysdeps/aarch64/__mtag_memset_tag.S
>> @@ -0,0 +1,53 @@
>> +/* Copyright (C) 2020 Free Software Foundation, Inc.
>> +
>> + This file is part of the GNU C Library.
>> +
>> + The GNU C Library is free software; you can redistribute it and/or
>> + modify it under the terms of the GNU Lesser General Public
>> + License as published by the Free Software Foundation; either
>> + version 2.1 of the License, or (at your option) any later version.
>> +
>> + The GNU C Library is distributed in the hope that it will be useful,
>> + but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
>> + Lesser General Public License for more details.
>> +
>> + You should have received a copy of the GNU Lesser General Public
>> + License along with the GNU C Library. If not, see
>> + <http://www.gnu.org/licenses/>. */
>> +
>> +#include <sysdep.h>
>> +
>> +#ifdef USE_MTAG
>> +
>> +/* Use the same register names and assignments as memset. */
>> +#include "memset-reg.h"
>> +
>> + .arch armv8.5-a
>> + .arch_extension memtag
>> +
>> +/* NB, only supported on variants with 64-bit pointers. */
>> +
>> +/* FIXME: This is a minimal implementation. We could do much better than
>> + this for large values of COUNT. */
>> +
>> +ENTRY(__libc_mtag_memset_with_tag)
>> +
>> + and valw, valw, 255
>> + orr valw, valw, valw, lsl 8
>> + orr valw, valw, valw, lsl 16
>> + orr val, val, val, lsl 32
>> + mov dst, dstin
>> +
>> +L(loop):
>> + stgp val, val, [dst], #16
>> + subs count, count, 16
>> + bne L(loop)
>> +#if 0
>> + /* This is not currently needed, since for now we are only called
>> + to tag memory that is taggable. */
>> + ldg dstin, [dstin] // Recover the tag created (might be untagged).
>> +#endif
>> + ret
>> +END (__libc_mtag_memset_with_tag)
>> +#endif /* USE_MTAG */
>> diff --git a/sysdeps/aarch64/__mtag_new_tag.S b/sysdeps/aarch64/__mtag_new_tag.S
>> new file mode 100644
>> index 0000000000..663e3a484f
>> --- /dev/null
>> +++ b/sysdeps/aarch64/__mtag_new_tag.S
>> @@ -0,0 +1,37 @@
>> +/* Copyright (C) 2020 Free Software Foundation, Inc.
>> +
>> + This file is part of the GNU C Library.
>> +
>> + The GNU C Library is free software; you can redistribute it and/or
>> + modify it under the terms of the GNU Lesser General Public
>> + License as published by the Free Software Foundation; either
>> + version 2.1 of the License, or (at your option) any later version.
>> +
>> + The GNU C Library is distributed in the hope that it will be useful,
>> + but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
>> + Lesser General Public License for more details.
>> +
>> + You should have received a copy of the GNU Lesser General Public
>> + License along with the GNU C Library. If not, see
>> + <http://www.gnu.org/licenses/>. */
>> +
>> +#include <sysdep.h>
>> +
>> +#ifdef USE_MTAG
>> +
>> + .arch armv8.5-a
>> + .arch_extension memtag
>> +
>> +/* NB, only supported on variants with 64-bit pointers. */
>> +
>> +#define ptr x0
>> +#define xset x1
>> +
>> +ENTRY(__libc_mtag_new_tag)
>> + // Guarantee that the new tag is not the same as now.
>> + gmi xset, ptr, xzr
>> + irg ptr, ptr, xset
>> + ret
>> +END (__libc_mtag_new_tag)
>> +#endif /* USE_MTAG */
>> diff --git a/sysdeps/aarch64/__mtag_tag_region.S b/sysdeps/aarch64/__mtag_tag_region.S
>> new file mode 100644
>> index 0000000000..7af4f0fa0a
>> --- /dev/null
>> +++ b/sysdeps/aarch64/__mtag_tag_region.S
>> @@ -0,0 +1,51 @@
>> +/* Copyright (C) 2020 Free Software Foundation, Inc.
>> +
>> + This file is part of the GNU C Library.
>> +
>> + The GNU C Library is free software; you can redistribute it and/or
>> + modify it under the terms of the GNU Lesser General Public
>> + License as published by the Free Software Foundation; either
>> + version 2.1 of the License, or (at your option) any later version.
>> +
>> + The GNU C Library is distributed in the hope that it will be useful,
>> + but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
>> + Lesser General Public License for more details.
>> +
>> + You should have received a copy of the GNU Lesser General Public
>> + License along with the GNU C Library. If not, see
>> + <http://www.gnu.org/licenses/>. */
>> +
>> +#include <sysdep.h>
>> +
>> +#ifdef USE_MTAG
>> +
>> +/* Use the same register names and assignments as memset. */
>> +
>> + .arch armv8.5-a
>> + .arch_extension memtag
>> +
>> +/* NB, only supported on variants with 64-bit pointers. */
>> +
>> +/* FIXME: This is a minimal implementation. We could do better than
>> + this for larger values of COUNT. */
>> +
>> +#define dstin x0
>> +#define count x1
>> +#define dst x2
>> +
>> +ENTRY_ALIGN(__libc_mtag_tag_region, 6)
>> +
>> + mov dst, dstin
>> +L(loop):
>> + stg dst, [dst], #16
>> + subs count, count, 16
>> + bne L(loop)
>> +#if 0
>> + /* This is not currently needed, since for now we are only called
>> + to tag memory that is taggable. */
>> + ldg dstin, [dstin] // Recover the tag created (might be untagged).
>> +#endif
>> + ret
>> +END (__libc_mtag_tag_region)
>> +#endif /* USE_MTAG */
>> diff --git a/sysdeps/aarch64/libc-mtag.h b/sysdeps/aarch64/libc-mtag.h
>> new file mode 100644
>> index 0000000000..0e9b63e95b
>> --- /dev/null
>> +++ b/sysdeps/aarch64/libc-mtag.h
>> @@ -0,0 +1,57 @@
>> +/* libc-internal interface for tagged (colored) memory support.
>> + Copyright (C) 2020 Free Software Foundation, Inc.
>> + This file is part of the GNU C Library.
>> +
>> + The GNU C Library is free software; you can redistribute it and/or
>> + modify it under the terms of the GNU Lesser General Public
>> + License as published by the Free Software Foundation; either
>> + version 2.1 of the License, or (at your option) any later version.
>> +
>> + The GNU C Library is distributed in the hope that it will be useful,
>> + but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
>> + Lesser General Public License for more details.
>> +
>> + You should have received a copy of the GNU Lesser General Public
>> + License along with the GNU C Library; if not, see
>> + <http://www.gnu.org/licenses/>. */
>> +
>> +#ifndef _AARCH64_LIBC_MTAG_H
>> +#define _AARCH64_LIBC_MTAG_H 1
>> +
>> +#ifndef USE_MTAG
>> +/* Generic bindings for systems that do not support memory tagging. */
>> +#include_next "libc-mtag.h"
>> +#else
>> +
>> +/* Used to ensure additional alignment when objects need to have distinct
>> + tags. */
>> +#define __MTAG_GRANULE_SIZE 16
>> +
>> +/* Non-zero if memory obtained via morecore (sbrk) is not tagged. */
>> +#define __MTAG_SBRK_UNTAGGED 1
>> +
>> +/* Extra flags to pass to mmap to get tagged pages. */
>> +#define __MTAG_MMAP_FLAGS PROT_MTE
>> +
>> +/* Set the tags for a region of memory, which must have size and alignment
>> + that are multiples of __MTAG_GRANULE_SIZE. Size cannot be zero.
>> + void *__libc_mtag_tag_region (const void *, size_t) */
>> +void *__libc_mtag_tag_region (void *, size_t);
>> +
>> +/* Optimized equivalent to __libc_mtag_tag_region followed by memset. */
>> +void *__libc_mtag_memset_with_tag(void *, int, size_t);
>> +
>> +/* Convert address P to a pointer that is tagged correctly for that
>> + location.
>> + void *__libc_mtag_address_get_tag (void*) */
>> +void *__libc_mtag_address_get_tag(void *);
>> +
>> +/* Assign a new (random) tag to a pointer P (does not adjust the tag on
>> + the memory addressed).
>> + void *__libc_mtag_new_tag (void*) */
>> +void *__libc_mtag_new_tag(void *);
>
> missing spaces before (
Oops, fixed.
>
>> +
>> +#endif /* USE_MTAG */
>> +
>> +#endif /* _AARCH64_LIBC_MTAG_H */
R.
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v4 3/6] malloc: Basic support for memory tagging in the malloc() family
2020-12-18 19:29 ` [PATCH v4 3/6] malloc: Basic support for memory tagging in the malloc() family Richard Earnshaw
@ 2020-12-21 13:27 ` Siddhesh Poyarekar
2020-12-21 13:46 ` Florian Weimer
1 sibling, 0 replies; 25+ messages in thread
From: Siddhesh Poyarekar @ 2020-12-21 13:27 UTC (permalink / raw)
To: Richard Earnshaw, libc-alpha
On 12/19/20 12:59 AM, Richard Earnshaw via Libc-alpha wrote:
>
> This patch adds the basic support for memory tagging.
>
> Various flavours are supported, particularly being able to turn on
> tagged memory at run-time: this allows the same code to be used on
> systems where memory tagging support is not present without neededing
> a separate build of glibc. Also, depending on whether the kernel
> supports it, the code will use mmap for the default arena if morecore
> does not, or cannot support tagged memory (on AArch64 it is not
> available).
>
> All the hooks use function pointers to allow this to work without
> needing ifuncs.
> ---
> malloc/arena.c | 59 ++++++-
> malloc/hooks.c | 79 ++++++---
> malloc/malloc.c | 336 +++++++++++++++++++++++++++++-------
> malloc/malloc.h | 7 +
> sysdeps/generic/libc-mtag.h | 52 ++++++
> 5 files changed, 436 insertions(+), 97 deletions(-)
> create mode 100644 sysdeps/generic/libc-mtag.h
In summary, a nit or two below, the primary one being that the header
changes in malloc/malloc.h should probably go into include/malloc.h
since I couldn't see a reason for those changes to go into an installed
header.
The change looks fine otherwise.
> diff --git a/malloc/arena.c b/malloc/arena.c
> index 3c9c0ecd86..ba833a2d98 100644
> --- a/malloc/arena.c
> +++ b/malloc/arena.c
> @@ -274,17 +274,52 @@ next_env_entry (char ***position)
> #endif
>
>
> -#ifdef SHARED
> +#if defined(SHARED) || defined(USE_MTAG)
> static void *
> __failing_morecore (ptrdiff_t d)
> {
> return (void *) MORECORE_FAILURE;
> }
> +#endif
>
> +#ifdef SHARED
> extern struct dl_open_hook *_dl_open_hook;
> libc_hidden_proto (_dl_open_hook);
> #endif
>
> +#ifdef USE_MTAG
> +
> +/* Generate a new (random) tag value for PTR and tag the memory it
> + points to upto the end of the usable size for the chunk containing
> + it. Return the newly tagged pointer. */
> +static void *
> +__mtag_tag_new_usable (void *ptr)
> +{
> + if (ptr)
> + {
> + mchunkptr cp = mem2chunk(ptr);
> + /* This likely will never happen, but we can't handle retagging
> + chunks from the dumped main arena. So just return the
> + existing pointer. */
> + if (DUMPED_MAIN_ARENA_CHUNK (cp))
> + return ptr;
> + ptr = __libc_mtag_tag_region (__libc_mtag_new_tag (ptr),
> + CHUNK_AVAILABLE_SIZE (cp) - CHUNK_HDR_SZ);
> + }
> + return ptr;
> +}
> +
> +/* Generate a new (random) tag value for PTR, set the tags for the
> + memory to the new tag and initialize the memory contents to VAL.
> + In practice this function will only be called with VAL=0, but we
> + keep this parameter to maintain the same prototype as memset. */
> +static void *
> +__mtag_tag_new_memset (void *ptr, int val, size_t size)
> +{
> + return __libc_mtag_memset_with_tag (__libc_mtag_new_tag (ptr), val, size);
> +}
> +#endif
Ok.
> +
> static void
> ptmalloc_init (void)
> {
> @@ -293,6 +328,24 @@ ptmalloc_init (void)
>
> __malloc_initialized = 0;
>
> +#ifdef USE_MTAG
> + if ((TUNABLE_GET_FULL (glibc, mem, tagging, int32_t, NULL) & 1) != 0)
> + {
> + /* If the environment says that we should be using tagged memory
If the *tunable* says...
> + and that morecore does not support tagged regions, then
> + disable it. */
> + if (__MTAG_SBRK_UNTAGGED)
> + __morecore = __failing_morecore;
> +
> + __mtag_mmap_flags = __MTAG_MMAP_FLAGS;
> + __tag_new_memset = __mtag_tag_new_memset;
> + __tag_region = __libc_mtag_tag_region;
> + __tag_new_usable = __mtag_tag_new_usable;
> + __tag_at = __libc_mtag_address_get_tag;
> + __mtag_granule_mask = ~(size_t)(__MTAG_GRANULE_SIZE - 1);
> + }
> +#endif
> +
> #ifdef SHARED
> /* In case this libc copy is in a non-default namespace, never use
> brk. Likewise if dlopened from statically linked program. The
> @@ -509,7 +562,7 @@ new_heap (size_t size, size_t top_pad)
> }
> }
> }
> - if (__mprotect (p2, size, PROT_READ | PROT_WRITE) != 0)
> + if (__mprotect (p2, size, MTAG_MMAP_FLAGS | PROT_READ | PROT_WRITE) != 0)
> {
> __munmap (p2, HEAP_MAX_SIZE);
> return 0;
> @@ -539,7 +592,7 @@ grow_heap (heap_info *h, long diff)
> {
> if (__mprotect ((char *) h + h->mprotect_size,
> (unsigned long) new_size - h->mprotect_size,
> - PROT_READ | PROT_WRITE) != 0)
> + MTAG_MMAP_FLAGS | PROT_READ | PROT_WRITE) != 0)
> return -2;
>
> h->mprotect_size = new_size;
> diff --git a/malloc/hooks.c b/malloc/hooks.c
> index a2b93e5446..8a1c16dfa4 100644
> --- a/malloc/hooks.c
> +++ b/malloc/hooks.c
> @@ -63,6 +63,13 @@ __malloc_check_init (void)
> __memalign_hook = memalign_check;
> }
>
> +/* When memory is tagged, the checking data is stored in the user part
> + of the chunk. We can't rely on the user not having modified the
> + tags, so fetch the tag at each location before dereferencing
> + it. */
> +#define SAFE_CHAR_OFFSET(p,offset) \
> + ((unsigned char *) TAG_AT (((unsigned char *) p) + offset))
> +
OK.
> /* A simple, standard set of debugging hooks. Overhead is `only' one
> byte per chunk; still this will catch most cases of double frees or
> overruns. The goal here is to avoid obscure crashes due to invalid
> @@ -80,7 +87,6 @@ magicbyte (const void *p)
> return magic;
> }
>
> -
> /* Visualize the chunk as being partitioned into blocks of 255 bytes from the
> highest address of the chunk, downwards. The end of each block tells
> us the size of that block, up to the actual size of the requested
> @@ -96,16 +102,16 @@ malloc_check_get_size (mchunkptr p)
>
> assert (using_malloc_checking == 1);
>
> - for (size = chunksize (p) - 1 + (chunk_is_mmapped (p) ? 0 : SIZE_SZ);
> - (c = ((unsigned char *) p)[size]) != magic;
> + for (size = CHUNK_AVAILABLE_SIZE (p) - 1;
> + (c = *SAFE_CHAR_OFFSET (p, size)) != magic;
> size -= c)
> {
> - if (c <= 0 || size < (c + 2 * SIZE_SZ))
> + if (c <= 0 || size < (c + CHUNK_HDR_SZ))
> malloc_printerr ("malloc_check_get_size: memory corruption");
> }
>
> /* chunk2mem size. */
> - return size - 2 * SIZE_SZ;
> + return size - CHUNK_HDR_SZ;
> }
>
> /* Instrument a chunk with overrun detector byte(s) and convert it
> @@ -124,9 +130,8 @@ mem2mem_check (void *ptr, size_t req_sz)
>
> p = mem2chunk (ptr);
> magic = magicbyte (p);
> - max_sz = chunksize (p) - 2 * SIZE_SZ;
> - if (!chunk_is_mmapped (p))
> - max_sz += SIZE_SZ;
> + max_sz = CHUNK_AVAILABLE_SIZE (p) - CHUNK_HDR_SZ;
> +
> for (i = max_sz - 1; i > req_sz; i -= block_sz)
> {
> block_sz = MIN (i - req_sz, 0xff);
> @@ -135,9 +140,9 @@ mem2mem_check (void *ptr, size_t req_sz)
> if (block_sz == magic)
> --block_sz;
>
> - m_ptr[i] = block_sz;
> + *SAFE_CHAR_OFFSET (m_ptr, i) = block_sz;
> }
> - m_ptr[req_sz] = magic;
> + *SAFE_CHAR_OFFSET (m_ptr, req_sz) = magic;
> return (void *) m_ptr;
> }
>
> @@ -170,9 +175,11 @@ mem2chunk_check (void *mem, unsigned char **magic_p)
> next_chunk (prev_chunk (p)) != p)))
> return NULL;
>
> - for (sz += SIZE_SZ - 1; (c = ((unsigned char *) p)[sz]) != magic; sz -= c)
> + for (sz = CHUNK_AVAILABLE_SIZE (p) - 1;
> + (c = *SAFE_CHAR_OFFSET (p, sz)) != magic;
> + sz -= c)
> {
> - if (c == 0 || sz < (c + 2 * SIZE_SZ))
> + if (c == 0 || sz < (c + CHUNK_HDR_SZ))
> return NULL;
> }
> }
> @@ -193,15 +200,19 @@ mem2chunk_check (void *mem, unsigned char **magic_p)
> ((prev_size (p) + sz) & page_mask) != 0)
> return NULL;
>
> - for (sz -= 1; (c = ((unsigned char *) p)[sz]) != magic; sz -= c)
> + for (sz = CHUNK_AVAILABLE_SIZE (p) - 1;
> + (c = *SAFE_CHAR_OFFSET (p, sz)) != magic;
> + sz -= c)
> {
> - if (c == 0 || sz < (c + 2 * SIZE_SZ))
> + if (c == 0 || sz < (c + CHUNK_HDR_SZ))
> return NULL;
> }
> }
> - ((unsigned char *) p)[sz] ^= 0xFF;
> +
> + unsigned char* safe_p = SAFE_CHAR_OFFSET (p, sz);
> + *safe_p ^= 0xFF;
> if (magic_p)
> - *magic_p = (unsigned char *) p + sz;
> + *magic_p = safe_p;
> return p;
> }
>
> @@ -238,7 +249,7 @@ malloc_check (size_t sz, const void *caller)
> top_check ();
> victim = _int_malloc (&main_arena, nb);
> __libc_lock_unlock (main_arena.mutex);
> - return mem2mem_check (victim, sz);
> + return mem2mem_check (TAG_NEW_USABLE (victim), sz);
> }
>
> static void
> @@ -249,6 +260,12 @@ free_check (void *mem, const void *caller)
> if (!mem)
> return;
>
> +#ifdef USE_MTAG
> + /* Quickly check that the freed pointer matches the tag for the memory.
> + This gives a useful double-free detection. */
> + *(volatile char *)mem;
> +#endif
> +
OK.
> __libc_lock_lock (main_arena.mutex);
> p = mem2chunk_check (mem, NULL);
> if (!p)
> @@ -259,6 +276,8 @@ free_check (void *mem, const void *caller)
> munmap_chunk (p);
> return;
> }
> + /* Mark the chunk as belonging to the library again. */
> + (void)TAG_REGION (chunk2rawmem (p), CHUNK_AVAILABLE_SIZE (p) - CHUNK_HDR_SZ);
> _int_free (&main_arena, p, 1);
> __libc_lock_unlock (main_arena.mutex);
> }
> @@ -266,7 +285,7 @@ free_check (void *mem, const void *caller)
> static void *
> realloc_check (void *oldmem, size_t bytes, const void *caller)
> {
> - INTERNAL_SIZE_T nb;
> + INTERNAL_SIZE_T chnb;
> void *newmem = 0;
> unsigned char *magic_p;
> size_t rb;
> @@ -284,14 +303,21 @@ realloc_check (void *oldmem, size_t bytes, const void *caller)
> free_check (oldmem, NULL);
> return NULL;
> }
> +
> +#ifdef USE_MTAG
> + /* Quickly check that the freed pointer matches the tag for the memory.
> + This gives a useful double-free detection. */
> + *(volatile char *)oldmem;
> +#endif
> +
> __libc_lock_lock (main_arena.mutex);
> const mchunkptr oldp = mem2chunk_check (oldmem, &magic_p);
> __libc_lock_unlock (main_arena.mutex);
> if (!oldp)
> malloc_printerr ("realloc(): invalid pointer");
> - const INTERNAL_SIZE_T oldsize = chunksize (oldp);
> + const INTERNAL_SIZE_T oldchsize = CHUNK_AVAILABLE_SIZE (oldp);
>
> - if (!checked_request2size (rb, &nb))
> + if (!checked_request2size (rb, &chnb))
> goto invert;
>
> __libc_lock_lock (main_arena.mutex);
> @@ -299,14 +325,13 @@ realloc_check (void *oldmem, size_t bytes, const void *caller)
> if (chunk_is_mmapped (oldp))
> {
> #if HAVE_MREMAP
> - mchunkptr newp = mremap_chunk (oldp, nb);
> + mchunkptr newp = mremap_chunk (oldp, chnb);
> if (newp)
> newmem = chunk2mem (newp);
> else
> #endif
> {
> - /* Note the extra SIZE_SZ overhead. */
> - if (oldsize - SIZE_SZ >= nb)
> + if (oldchsize >= chnb)
> newmem = oldmem; /* do nothing */
> else
> {
> @@ -315,7 +340,7 @@ realloc_check (void *oldmem, size_t bytes, const void *caller)
> newmem = _int_malloc (&main_arena, rb);
> if (newmem)
> {
> - memcpy (newmem, oldmem, oldsize - 2 * SIZE_SZ);
> + memcpy (newmem, oldmem, oldchsize - CHUNK_HDR_SZ);
> munmap_chunk (oldp);
> }
> }
> @@ -324,7 +349,7 @@ realloc_check (void *oldmem, size_t bytes, const void *caller)
> else
> {
> top_check ();
> - newmem = _int_realloc (&main_arena, oldp, oldsize, nb);
> + newmem = _int_realloc (&main_arena, oldp, oldchsize, chnb);
> }
>
> DIAG_PUSH_NEEDS_COMMENT;
> @@ -343,7 +368,7 @@ invert:
>
> __libc_lock_unlock (main_arena.mutex);
>
> - return mem2mem_check (newmem, bytes);
> + return mem2mem_check (TAG_NEW_USABLE (newmem), bytes);
> }
>
> static void *
> @@ -385,7 +410,7 @@ memalign_check (size_t alignment, size_t bytes, const void *caller)
> top_check ();
> mem = _int_memalign (&main_arena, alignment, bytes + 1);
> __libc_lock_unlock (main_arena.mutex);
> - return mem2mem_check (mem, bytes);
> + return mem2mem_check (TAG_NEW_USABLE (mem), bytes);
> }
>
> #if SHLIB_COMPAT (libc, GLIBC_2_0, GLIBC_2_25)
> diff --git a/malloc/malloc.c b/malloc/malloc.c
> index 326075e704..a3e914fa8a 100644
> --- a/malloc/malloc.c
> +++ b/malloc/malloc.c
> @@ -242,6 +242,9 @@
> /* For DIAG_PUSH/POP_NEEDS_COMMENT et al. */
> #include <libc-diag.h>
>
> +/* For memory tagging. */
> +#include <libc-mtag.h>
> +
> #include <malloc/malloc-internal.h>
>
> /* For SINGLE_THREAD_P. */
> @@ -380,6 +383,96 @@ __malloc_assert (const char *assertion, const char *file, unsigned int line,
> void * __default_morecore (ptrdiff_t);
> void *(*__morecore)(ptrdiff_t) = __default_morecore;
>
> +/* Memory tagging. */
> +
> +/* Some systems support the concept of tagging (sometimes known as
> + coloring) memory locations on a fine grained basis. Each memory
> + location is given a color (normally allocated randomly) and
> + pointers are also colored. When the pointer is dereferenced, the
> + pointer's color is checked against the memory's color and if they
> + differ the access is faulted (sometimes lazily).
> +
> + We use this in glibc by maintaining a single color for the malloc
> + data structures that are interleaved with the user data and then
> + assigning separate colors for each block allocation handed out. In
> + this way simple buffer overruns will be rapidly detected. When
> + memory is freed, the memory is recolored back to the glibc default
> + so that simple use-after-free errors can also be detected.
> +
> + If memory is reallocated the buffer is recolored even if the
> + address remains the same. This has a performance impact, but
> + guarantees that the old pointer cannot mistakenly be reused (code
> + that compares old against new will see a mismatch and will then
> + need to behave as though realloc moved the data to a new location).
> +
> + Internal API for memory tagging support.
> +
> + The aim is to keep the code for memory tagging support as close to
> + the normal APIs in glibc as possible, so that if tagging is not
> + enabled in the library, or is disabled at runtime then standard
> + operations can continue to be used. Support macros are used to do
> + this:
> +
> + void *TAG_NEW_MEMSET (void *ptr, int, val, size_t size)
> +
> + Has the same interface as memset(), but additionally allocates a
> + new tag, colors the memory with that tag and returns a pointer that
> + is correctly colored for that location. The non-tagging version
> + will simply call memset.
> +
> + void *TAG_REGION (void *ptr, size_t size)
> +
> + Color the region of memory pointed to by PTR and size SIZE with
> + the color of PTR. Returns the original pointer.
> +
> + void *TAG_NEW_USABLE (void *ptr)
> +
> + Allocate a new random color and use it to color the user region of
> + a chunk; this may include data from the subsequent chunk's header
> + if tagging is sufficiently fine grained. Returns PTR suitably
> + recolored for accessing the memory there.
> +
> + void *TAG_AT (void *ptr)
> +
> + Read the current color of the memory at the address pointed to by
> + PTR (ignoring it's current color) and return PTR recolored to that
> + color. PTR must be valid address in all other respects. When
> + tagging is not enabled, it simply returns the original pointer.
> +*/
OK.
> +
> +#ifdef USE_MTAG
> +
> +/* Default implementaions when memory tagging is supported, but disabled. */
> +static void *
> +__default_tag_region (void *ptr, size_t size)
> +{
> + return ptr;
> +}
> +
> +static void *
> +__default_tag_nop (void *ptr)
> +{
> + return ptr;
> +}
> +
> +static int __mtag_mmap_flags = 0;
> +static size_t __mtag_granule_mask = ~(size_t)0;
> +
> +static void *(*__tag_new_memset)(void *, int, size_t) = memset;
> +static void *(*__tag_region)(void *, size_t) = __default_tag_region;
> +static void *(*__tag_new_usable)(void *) = __default_tag_nop;
> +static void *(*__tag_at)(void *) = __default_tag_nop;
> +
> +# define TAG_NEW_MEMSET(ptr, val, size) __tag_new_memset (ptr, val, size)
> +# define TAG_REGION(ptr, size) __tag_region (ptr, size)
> +# define TAG_NEW_USABLE(ptr) __tag_new_usable (ptr)
> +# define TAG_AT(ptr) __tag_at (ptr)
> +#else
> +# define TAG_NEW_MEMSET(ptr, val, size) memset (ptr, val, size)
> +# define TAG_REGION(ptr, size) (ptr)
> +# define TAG_NEW_USABLE(ptr) (ptr)
> +# define TAG_AT(ptr) (ptr)
> +#endif
Default implementations, OK.
>
> #include <string.h>
>
> @@ -1187,10 +1280,31 @@ nextchunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> ---------- Size and alignment checks and conversions ----------
> */
>
> -/* conversion from malloc headers to user pointers, and back */
> +/* Conversion from malloc headers to user pointers, and back. When
> + using memory tagging the user data and the malloc data structure
> + headers have distinct tags. Converting fully from one to the other
> + involves extracting the tag at the other address and creating a
> + suitable pointer using it. That can be quite expensive. There are
> + many occasions, though when the pointer will not be dereferenced
> + (for example, because we only want to assert that the pointer is
> + correctly aligned). In these cases it is more efficient not
> + to extract the tag, since the answer will be the same either way.
> + chunk2rawmem() can be used in these cases.
> + */
> +
> +/* The chunk header is two SIZE_SZ elements, but this is used widely, so
> + we define it here for clarity later. */
> +#define CHUNK_HDR_SZ (2 * SIZE_SZ)
> +
> +/* Convert a user mem pointer to a chunk address without correcting
> + the tag. */
> +#define chunk2rawmem(p) ((void*)((char*)(p) + CHUNK_HDR_SZ))
>
> -#define chunk2mem(p) ((void*)((char*)(p) + 2*SIZE_SZ))
> -#define mem2chunk(mem) ((mchunkptr)((char*)(mem) - 2*SIZE_SZ))
> +/* Convert between user mem pointers and chunk pointers, updating any
> + memory tags on the pointer to respect the tag value at that
> + location. */
> +#define chunk2mem(p) ((void*)TAG_AT (((char*)(p) + CHUNK_HDR_SZ)))
> +#define mem2chunk(mem) ((mchunkptr)TAG_AT (((char*)(mem) - CHUNK_HDR_SZ)))
>
> /* The smallest possible chunk */
> #define MIN_CHUNK_SIZE (offsetof(struct malloc_chunk, fd_nextsize))
> @@ -1205,16 +1319,28 @@ nextchunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> #define aligned_OK(m) (((unsigned long)(m) & MALLOC_ALIGN_MASK) == 0)
>
> #define misaligned_chunk(p) \
> - ((uintptr_t)(MALLOC_ALIGNMENT == 2 * SIZE_SZ ? (p) : chunk2mem (p)) \
> + ((uintptr_t)(MALLOC_ALIGNMENT == CHUNK_HDR_SZ ? (p) : chunk2mem (p)) \
> & MALLOC_ALIGN_MASK)
>
> /* pad request bytes into a usable size -- internal version */
> -
> +/* Note: This must be a macro that evaluates to a compile time constant
> + if passed a literal constant. */
> #define request2size(req) \
> (((req) + SIZE_SZ + MALLOC_ALIGN_MASK < MINSIZE) ? \
> MINSIZE : \
> ((req) + SIZE_SZ + MALLOC_ALIGN_MASK) & ~MALLOC_ALIGN_MASK)
>
> +/* Available size of chunk. This is the size of the real usable data
> + in the chunk, plus the chunk header. */
> +#ifdef USE_MTAG
> +#define CHUNK_AVAILABLE_SIZE(p) \
> + ((chunksize (p) + (chunk_is_mmapped (p) ? 0 : SIZE_SZ)) \
> + & __mtag_granule_mask)
> +#else
> +#define CHUNK_AVAILABLE_SIZE(p) \
> + (chunksize (p) + (chunk_is_mmapped (p) ? 0 : SIZE_SZ))
> +#endif
> +
> /* Check if REQ overflows when padded and aligned and if the resulting value
> is less than PTRDIFF_T. Returns TRUE and the requested size or MINSIZE in
> case the value is less than MINSIZE on SZ or false if any of the previous
> @@ -1224,6 +1350,18 @@ checked_request2size (size_t req, size_t *sz) __nonnull (1)
> {
> if (__glibc_unlikely (req > PTRDIFF_MAX))
> return false;
> +
> +#ifdef USE_MTAG
> + /* When using tagged memory, we cannot share the end of the user
> + block with the header for the next chunk, so ensure that we
> + allocate blocks that are rounded up to the granule size. Take
> + care not to overflow from close to MAX_SIZE_T to a small
> + number. Ideally, this would be part of request2size(), but that
> + must be a macro that produces a compile time constant if passed
> + a constant literal. */
> + req = (req + ~__mtag_granule_mask) & __mtag_granule_mask;
> +#endif
> +
> *sz = request2size (req);
> return true;
> }
> @@ -1322,7 +1460,6 @@ checked_request2size (size_t req, size_t *sz) __nonnull (1)
> /* Set size at footer (only when chunk is not in use) */
> #define set_foot(p, s) (((mchunkptr) ((char *) (p) + (s)))->mchunk_prev_size = (s))
>
> -
> #pragma GCC poison mchunk_size
> #pragma GCC poison mchunk_prev_size
>
> @@ -1418,7 +1555,7 @@ typedef struct malloc_chunk *mbinptr;
> #define NBINS 128
> #define NSMALLBINS 64
> #define SMALLBIN_WIDTH MALLOC_ALIGNMENT
> -#define SMALLBIN_CORRECTION (MALLOC_ALIGNMENT > 2 * SIZE_SZ)
> +#define SMALLBIN_CORRECTION (MALLOC_ALIGNMENT > CHUNK_HDR_SZ)
> #define MIN_LARGE_SIZE ((NSMALLBINS - SMALLBIN_CORRECTION) * SMALLBIN_WIDTH)
>
> #define in_smallbin_range(sz) \
> @@ -1969,7 +2106,7 @@ do_check_chunk (mstate av, mchunkptr p)
> /* chunk is page-aligned */
> assert (((prev_size (p) + sz) & (GLRO (dl_pagesize) - 1)) == 0);
> /* mem is aligned */
> - assert (aligned_OK (chunk2mem (p)));
> + assert (aligned_OK (chunk2rawmem (p)));
> }
> }
>
> @@ -1993,7 +2130,7 @@ do_check_free_chunk (mstate av, mchunkptr p)
> if ((unsigned long) (sz) >= MINSIZE)
> {
> assert ((sz & MALLOC_ALIGN_MASK) == 0);
> - assert (aligned_OK (chunk2mem (p)));
> + assert (aligned_OK (chunk2rawmem (p)));
> /* ... matching footer field */
> assert (prev_size (next_chunk (p)) == sz);
> /* ... and is fully consolidated */
> @@ -2072,7 +2209,7 @@ do_check_remalloced_chunk (mstate av, mchunkptr p, INTERNAL_SIZE_T s)
> assert ((sz & MALLOC_ALIGN_MASK) == 0);
> assert ((unsigned long) (sz) >= MINSIZE);
> /* ... and alignment */
> - assert (aligned_OK (chunk2mem (p)));
> + assert (aligned_OK (chunk2rawmem (p)));
> /* chunk is less than MINSIZE more than request */
> assert ((long) (sz) - (long) (s) >= 0);
> assert ((long) (sz) - (long) (s + MINSIZE) < 0);
> @@ -2318,7 +2455,7 @@ sysmalloc (INTERNAL_SIZE_T nb, mstate av)
> See the front_misalign handling below, for glibc there is no
> need for further alignments unless we have have high alignment.
> */
> - if (MALLOC_ALIGNMENT == 2 * SIZE_SZ)
> + if (MALLOC_ALIGNMENT == CHUNK_HDR_SZ)
> size = ALIGN_UP (nb + SIZE_SZ, pagesize);
> else
> size = ALIGN_UP (nb + SIZE_SZ + MALLOC_ALIGN_MASK, pagesize);
> @@ -2327,7 +2464,8 @@ sysmalloc (INTERNAL_SIZE_T nb, mstate av)
> /* Don't try if size wraps around 0 */
> if ((unsigned long) (size) > (unsigned long) (nb))
> {
> - mm = (char *) (MMAP (0, size, PROT_READ | PROT_WRITE, 0));
> + mm = (char *) (MMAP (0, size,
> + MTAG_MMAP_FLAGS | PROT_READ | PROT_WRITE, 0));
>
> if (mm != MAP_FAILED)
> {
> @@ -2339,16 +2477,18 @@ sysmalloc (INTERNAL_SIZE_T nb, mstate av)
> address argument for later munmap in free() and realloc().
> */
>
> - if (MALLOC_ALIGNMENT == 2 * SIZE_SZ)
> + if (MALLOC_ALIGNMENT == CHUNK_HDR_SZ)
> {
> - /* For glibc, chunk2mem increases the address by 2*SIZE_SZ and
> - MALLOC_ALIGN_MASK is 2*SIZE_SZ-1. Each mmap'ed area is page
> - aligned and therefore definitely MALLOC_ALIGN_MASK-aligned. */
> - assert (((INTERNAL_SIZE_T) chunk2mem (mm) & MALLOC_ALIGN_MASK) == 0);
> + /* For glibc, chunk2rawmem increases the address by
> + CHUNK_HDR_SZ and MALLOC_ALIGN_MASK is
> + CHUNK_HDR_SZ-1. Each mmap'ed area is page
> + aligned and therefore definitely
> + MALLOC_ALIGN_MASK-aligned. */
> + assert (((INTERNAL_SIZE_T) chunk2rawmem (mm) & MALLOC_ALIGN_MASK) == 0);
> front_misalign = 0;
> }
> else
> - front_misalign = (INTERNAL_SIZE_T) chunk2mem (mm) & MALLOC_ALIGN_MASK;
> + front_misalign = (INTERNAL_SIZE_T) chunk2rawmem (mm) & MALLOC_ALIGN_MASK;
> if (front_misalign > 0)
> {
> correction = MALLOC_ALIGNMENT - front_misalign;
> @@ -2436,18 +2576,20 @@ sysmalloc (INTERNAL_SIZE_T nb, mstate av)
> become the top chunk again later. Note that a footer is set
> up, too, although the chunk is marked in use. */
> old_size = (old_size - MINSIZE) & ~MALLOC_ALIGN_MASK;
> - set_head (chunk_at_offset (old_top, old_size + 2 * SIZE_SZ), 0 | PREV_INUSE);
> + set_head (chunk_at_offset (old_top, old_size + CHUNK_HDR_SZ),
> + 0 | PREV_INUSE);
> if (old_size >= MINSIZE)
> {
> - set_head (chunk_at_offset (old_top, old_size), (2 * SIZE_SZ) | PREV_INUSE);
> - set_foot (chunk_at_offset (old_top, old_size), (2 * SIZE_SZ));
> + set_head (chunk_at_offset (old_top, old_size),
> + CHUNK_HDR_SZ | PREV_INUSE);
> + set_foot (chunk_at_offset (old_top, old_size), CHUNK_HDR_SZ);
> set_head (old_top, old_size | PREV_INUSE | NON_MAIN_ARENA);
> _int_free (av, old_top, 1);
> }
> else
> {
> - set_head (old_top, (old_size + 2 * SIZE_SZ) | PREV_INUSE);
> - set_foot (old_top, (old_size + 2 * SIZE_SZ));
> + set_head (old_top, (old_size + CHUNK_HDR_SZ) | PREV_INUSE);
> + set_foot (old_top, (old_size + CHUNK_HDR_SZ));
> }
> }
> else if (!tried_mmap)
> @@ -2520,7 +2662,9 @@ sysmalloc (INTERNAL_SIZE_T nb, mstate av)
> /* Don't try if size wraps around 0 */
> if ((unsigned long) (size) > (unsigned long) (nb))
> {
> - char *mbrk = (char *) (MMAP (0, size, PROT_READ | PROT_WRITE, 0));
> + char *mbrk = (char *) (MMAP (0, size,
> + MTAG_MMAP_FLAGS | PROT_READ | PROT_WRITE,
> + 0));
>
> if (mbrk != MAP_FAILED)
> {
> @@ -2591,7 +2735,7 @@ sysmalloc (INTERNAL_SIZE_T nb, mstate av)
>
> /* Guarantee alignment of first new chunk made from this space */
>
> - front_misalign = (INTERNAL_SIZE_T) chunk2mem (brk) & MALLOC_ALIGN_MASK;
> + front_misalign = (INTERNAL_SIZE_T) chunk2rawmem (brk) & MALLOC_ALIGN_MASK;
> if (front_misalign > 0)
> {
> /*
> @@ -2647,12 +2791,12 @@ sysmalloc (INTERNAL_SIZE_T nb, mstate av)
> /* handle non-contiguous cases */
> else
> {
> - if (MALLOC_ALIGNMENT == 2 * SIZE_SZ)
> + if (MALLOC_ALIGNMENT == CHUNK_HDR_SZ)
> /* MORECORE/mmap must correctly align */
> - assert (((unsigned long) chunk2mem (brk) & MALLOC_ALIGN_MASK) == 0);
> + assert (((unsigned long) chunk2rawmem (brk) & MALLOC_ALIGN_MASK) == 0);
> else
> {
> - front_misalign = (INTERNAL_SIZE_T) chunk2mem (brk) & MALLOC_ALIGN_MASK;
> + front_misalign = (INTERNAL_SIZE_T) chunk2rawmem (brk) & MALLOC_ALIGN_MASK;
> if (front_misalign > 0)
> {
> /*
> @@ -2697,7 +2841,7 @@ sysmalloc (INTERNAL_SIZE_T nb, mstate av)
> multiple of MALLOC_ALIGNMENT. We know there is at least
> enough space in old_top to do this.
> */
> - old_size = (old_size - 4 * SIZE_SZ) & ~MALLOC_ALIGN_MASK;
> + old_size = (old_size - 2 * CHUNK_HDR_SZ) & ~MALLOC_ALIGN_MASK;
> set_head (old_top, old_size | PREV_INUSE);
>
> /*
> @@ -2707,9 +2851,10 @@ sysmalloc (INTERNAL_SIZE_T nb, mstate av)
> lost.
> */
> set_head (chunk_at_offset (old_top, old_size),
> - (2 * SIZE_SZ) | PREV_INUSE);
> - set_head (chunk_at_offset (old_top, old_size + 2 * SIZE_SZ),
> - (2 * SIZE_SZ) | PREV_INUSE);
> + CHUNK_HDR_SZ | PREV_INUSE);
> + set_head (chunk_at_offset (old_top,
> + old_size + CHUNK_HDR_SZ),
> + CHUNK_HDR_SZ | PREV_INUSE);
>
> /* If possible, release the rest. */
> if (old_size >= MINSIZE)
> @@ -2837,7 +2982,7 @@ munmap_chunk (mchunkptr p)
> if (DUMPED_MAIN_ARENA_CHUNK (p))
> return;
>
> - uintptr_t mem = (uintptr_t) chunk2mem (p);
> + uintptr_t mem = (uintptr_t) chunk2rawmem (p);
> uintptr_t block = (uintptr_t) p - prev_size (p);
> size_t total_size = prev_size (p) + size;
> /* Unfortunately we have to do the compilers job by hand here. Normally
> @@ -2892,7 +3037,7 @@ mremap_chunk (mchunkptr p, size_t new_size)
>
> p = (mchunkptr) (cp + offset);
>
> - assert (aligned_OK (chunk2mem (p)));
> + assert (aligned_OK (chunk2rawmem (p)));
>
> assert (prev_size (p) == offset);
> set_head (p, (new_size - offset) | IS_MMAPPED);
> @@ -3073,14 +3218,15 @@ __libc_malloc (size_t bytes)
> && tcache
> && tcache->counts[tc_idx] > 0)
> {
> - return tcache_get (tc_idx);
> + victim = tcache_get (tc_idx);
> + return TAG_NEW_USABLE (victim);
> }
> DIAG_POP_NEEDS_COMMENT;
> #endif
>
> if (SINGLE_THREAD_P)
> {
> - victim = _int_malloc (&main_arena, bytes);
> + victim = TAG_NEW_USABLE (_int_malloc (&main_arena, bytes));
> assert (!victim || chunk_is_mmapped (mem2chunk (victim)) ||
> &main_arena == arena_for_chunk (mem2chunk (victim)));
> return victim;
> @@ -3101,6 +3247,8 @@ __libc_malloc (size_t bytes)
> if (ar_ptr != NULL)
> __libc_lock_unlock (ar_ptr->mutex);
>
> + victim = TAG_NEW_USABLE (victim);
> +
> assert (!victim || chunk_is_mmapped (mem2chunk (victim)) ||
> ar_ptr == arena_for_chunk (mem2chunk (victim)));
> return victim;
> @@ -3124,8 +3272,17 @@ __libc_free (void *mem)
> if (mem == 0) /* free(0) has no effect */
> return;
>
> +#ifdef USE_MTAG
> + /* Quickly check that the freed pointer matches the tag for the memory.
> + This gives a useful double-free detection. */
> + *(volatile char *)mem;
> +#endif
> +
> p = mem2chunk (mem);
>
> + /* Mark the chunk as belonging to the library again. */
> + (void)TAG_REGION (chunk2rawmem (p), CHUNK_AVAILABLE_SIZE (p) - CHUNK_HDR_SZ);
> +
Tagging mem with chunk colour. OK.
> if (chunk_is_mmapped (p)) /* release mmapped memory. */
> {
> /* See if the dynamic brk/mmap threshold needs adjusting.
> @@ -3175,6 +3332,12 @@ __libc_realloc (void *oldmem, size_t bytes)
> if (oldmem == 0)
> return __libc_malloc (bytes);
>
> +#ifdef USE_MTAG
> + /* Perform a quick check to ensure that the pointer's tag matches the
> + memory's tag. */
> + *(volatile char*) oldmem;
> +#endif
> +
> /* chunk corresponding to oldmem */
> const mchunkptr oldp = mem2chunk (oldmem);
> /* its size */
> @@ -3217,7 +3380,7 @@ __libc_realloc (void *oldmem, size_t bytes)
> return NULL;
> /* Copy as many bytes as are available from the old chunk
> and fit into the new size. NB: The overhead for faked
> - mmapped chunks is only SIZE_SZ, not 2 * SIZE_SZ as for
> + mmapped chunks is only SIZE_SZ, not CHUNK_HDR_SZ as for
> regular mmapped chunks. */
> if (bytes > oldsize - SIZE_SZ)
> bytes = oldsize - SIZE_SZ;
> @@ -3230,7 +3393,15 @@ __libc_realloc (void *oldmem, size_t bytes)
> #if HAVE_MREMAP
> newp = mremap_chunk (oldp, nb);
> if (newp)
> - return chunk2mem (newp);
> + {
> + void *newmem = chunk2rawmem (newp);
> + /* Give the new block a different tag. This helps to ensure
> + that stale handles to the previous mapping are not
> + reused. There's a performance hit for both us and the
> + caller for doing this, so we might want to
> + reconsider. */
> + return TAG_NEW_USABLE (newmem);
> + }
> #endif
> /* Note the extra SIZE_SZ overhead. */
> if (oldsize - SIZE_SZ >= nb)
> @@ -3241,7 +3412,7 @@ __libc_realloc (void *oldmem, size_t bytes)
> if (newmem == 0)
> return 0; /* propagate failure */
>
> - memcpy (newmem, oldmem, oldsize - 2 * SIZE_SZ);
> + memcpy (newmem, oldmem, oldsize - CHUNK_HDR_SZ);
> munmap_chunk (oldp);
> return newmem;
> }
> @@ -3328,8 +3499,7 @@ _mid_memalign (size_t alignment, size_t bytes, void *address)
> p = _int_memalign (&main_arena, alignment, bytes);
> assert (!p || chunk_is_mmapped (mem2chunk (p)) ||
> &main_arena == arena_for_chunk (mem2chunk (p)));
> -
> - return p;
> + return TAG_NEW_USABLE (p);
> }
>
> arena_get (ar_ptr, bytes + alignment + MINSIZE);
> @@ -3347,7 +3517,7 @@ _mid_memalign (size_t alignment, size_t bytes, void *address)
>
> assert (!p || chunk_is_mmapped (mem2chunk (p)) ||
> ar_ptr == arena_for_chunk (mem2chunk (p)));
> - return p;
> + return TAG_NEW_USABLE (p);
> }
> /* For ISO C11. */
> weak_alias (__libc_memalign, aligned_alloc)
> @@ -3356,17 +3526,22 @@ libc_hidden_def (__libc_memalign)
> void *
> __libc_valloc (size_t bytes)
> {
> + void *p;
> +
> if (__malloc_initialized < 0)
> ptmalloc_init ();
>
> void *address = RETURN_ADDRESS (0);
> size_t pagesize = GLRO (dl_pagesize);
> - return _mid_memalign (pagesize, bytes, address);
> + p = _mid_memalign (pagesize, bytes, address);
> + return TAG_NEW_USABLE (p);
> }
>
> void *
> __libc_pvalloc (size_t bytes)
> {
> + void *p;
> +
> if (__malloc_initialized < 0)
> ptmalloc_init ();
>
> @@ -3383,19 +3558,22 @@ __libc_pvalloc (size_t bytes)
> }
> rounded_bytes = rounded_bytes & -(pagesize - 1);
>
> - return _mid_memalign (pagesize, rounded_bytes, address);
> + p = _mid_memalign (pagesize, rounded_bytes, address);
> + return TAG_NEW_USABLE (p);
> }
>
> void *
> __libc_calloc (size_t n, size_t elem_size)
> {
> mstate av;
> - mchunkptr oldtop, p;
> - INTERNAL_SIZE_T sz, csz, oldtopsize;
> + mchunkptr oldtop;
> + INTERNAL_SIZE_T sz, oldtopsize;
> void *mem;
> +#ifndef USE_MTAG
> unsigned long clearsize;
> unsigned long nclears;
> INTERNAL_SIZE_T *d;
> +#endif
> ptrdiff_t bytes;
>
> if (__glibc_unlikely (__builtin_mul_overflow (n, elem_size, &bytes)))
> @@ -3403,6 +3581,7 @@ __libc_calloc (size_t n, size_t elem_size)
> __set_errno (ENOMEM);
> return NULL;
> }
> +
> sz = bytes;
>
> void *(*hook) (size_t, const void *) =
> @@ -3472,7 +3651,14 @@ __libc_calloc (size_t n, size_t elem_size)
> if (mem == 0)
> return 0;
>
> - p = mem2chunk (mem);
> + mchunkptr p = mem2chunk (mem);
> + /* If we are using memory tagging, then we need to set the tags
> + regardless of MORECORE_CLEARS, so we zero the whole block while
> + doing so. */
> +#ifdef USE_MTAG
> + return TAG_NEW_MEMSET (mem, 0, CHUNK_AVAILABLE_SIZE (p) - CHUNK_HDR_SZ);
> +#else
> + INTERNAL_SIZE_T csz = chunksize (p);
>
> /* Two optional cases in which clearing not necessary */
> if (chunk_is_mmapped (p))
> @@ -3483,8 +3669,6 @@ __libc_calloc (size_t n, size_t elem_size)
> return mem;
> }
>
> - csz = chunksize (p);
> -
> #if MORECORE_CLEARS
> if (perturb_byte == 0 && (p == oldtop && csz > oldtopsize))
> {
> @@ -3527,6 +3711,7 @@ __libc_calloc (size_t n, size_t elem_size)
> }
>
> return mem;
> +#endif
> }
Clearing using TAG_MEMSET. OK.
>
> /*
> @@ -3764,10 +3949,10 @@ _int_malloc (mstate av, size_t bytes)
> size = chunksize (victim);
> mchunkptr next = chunk_at_offset (victim, size);
>
> - if (__glibc_unlikely (size <= 2 * SIZE_SZ)
> + if (__glibc_unlikely (size <= CHUNK_HDR_SZ)
> || __glibc_unlikely (size > av->system_mem))
> malloc_printerr ("malloc(): invalid size (unsorted)");
> - if (__glibc_unlikely (chunksize_nomask (next) < 2 * SIZE_SZ)
> + if (__glibc_unlikely (chunksize_nomask (next) < CHUNK_HDR_SZ)
> || __glibc_unlikely (chunksize_nomask (next) > av->system_mem))
> malloc_printerr ("malloc(): invalid next size (unsorted)");
> if (__glibc_unlikely ((prev_size (next) & ~(SIZE_BITS)) != size))
> @@ -4269,7 +4454,7 @@ _int_free (mstate av, mchunkptr p, int have_lock)
> ) {
>
> if (__builtin_expect (chunksize_nomask (chunk_at_offset (p, size))
> - <= 2 * SIZE_SZ, 0)
> + <= CHUNK_HDR_SZ, 0)
> || __builtin_expect (chunksize (chunk_at_offset (p, size))
> >= av->system_mem, 0))
> {
> @@ -4280,7 +4465,7 @@ _int_free (mstate av, mchunkptr p, int have_lock)
> if (!have_lock)
> {
> __libc_lock_lock (av->mutex);
> - fail = (chunksize_nomask (chunk_at_offset (p, size)) <= 2 * SIZE_SZ
> + fail = (chunksize_nomask (chunk_at_offset (p, size)) <= CHUNK_HDR_SZ
> || chunksize (chunk_at_offset (p, size)) >= av->system_mem);
> __libc_lock_unlock (av->mutex);
> }
> @@ -4289,7 +4474,7 @@ _int_free (mstate av, mchunkptr p, int have_lock)
> malloc_printerr ("free(): invalid next size (fast)");
> }
>
> - free_perturb (chunk2mem(p), size - 2 * SIZE_SZ);
> + free_perturb (chunk2mem(p), size - CHUNK_HDR_SZ);
>
> atomic_store_relaxed (&av->have_fastchunks, true);
> unsigned int idx = fastbin_index(size);
> @@ -4358,11 +4543,11 @@ _int_free (mstate av, mchunkptr p, int have_lock)
> malloc_printerr ("double free or corruption (!prev)");
>
> nextsize = chunksize(nextchunk);
> - if (__builtin_expect (chunksize_nomask (nextchunk) <= 2 * SIZE_SZ, 0)
> + if (__builtin_expect (chunksize_nomask (nextchunk) <= CHUNK_HDR_SZ, 0)
> || __builtin_expect (nextsize >= av->system_mem, 0))
> malloc_printerr ("free(): invalid next size (normal)");
>
> - free_perturb (chunk2mem(p), size - 2 * SIZE_SZ);
> + free_perturb (chunk2mem(p), size - CHUNK_HDR_SZ);
>
> /* consolidate backward */
> if (!prev_inuse(p)) {
> @@ -4593,7 +4778,7 @@ _int_realloc(mstate av, mchunkptr oldp, INTERNAL_SIZE_T oldsize,
> unsigned long remainder_size; /* its size */
>
> /* oldmem size */
> - if (__builtin_expect (chunksize_nomask (oldp) <= 2 * SIZE_SZ, 0)
> + if (__builtin_expect (chunksize_nomask (oldp) <= CHUNK_HDR_SZ, 0)
> || __builtin_expect (oldsize >= av->system_mem, 0))
> malloc_printerr ("realloc(): invalid old size");
>
> @@ -4604,7 +4789,7 @@ _int_realloc(mstate av, mchunkptr oldp, INTERNAL_SIZE_T oldsize,
>
> next = chunk_at_offset (oldp, oldsize);
> INTERNAL_SIZE_T nextsize = chunksize (next);
> - if (__builtin_expect (chunksize_nomask (next) <= 2 * SIZE_SZ, 0)
> + if (__builtin_expect (chunksize_nomask (next) <= CHUNK_HDR_SZ, 0)
> || __builtin_expect (nextsize >= av->system_mem, 0))
> malloc_printerr ("realloc(): invalid next size");
>
> @@ -4626,7 +4811,7 @@ _int_realloc(mstate av, mchunkptr oldp, INTERNAL_SIZE_T oldsize,
> av->top = chunk_at_offset (oldp, nb);
> set_head (av->top, (newsize - nb) | PREV_INUSE);
> check_inuse_chunk (av, oldp);
> - return chunk2mem (oldp);
> + return TAG_NEW_USABLE (chunk2rawmem (oldp));
> }
>
> /* Try to expand forward into next chunk; split off remainder below */
> @@ -4659,7 +4844,11 @@ _int_realloc(mstate av, mchunkptr oldp, INTERNAL_SIZE_T oldsize,
> }
> else
> {
> - memcpy (newmem, chunk2mem (oldp), oldsize - SIZE_SZ);
> + void *oldmem = chunk2mem (oldp);
> + newmem = TAG_NEW_USABLE (newmem);
> + memcpy (newmem, oldmem,
> + CHUNK_AVAILABLE_SIZE (oldp) - CHUNK_HDR_SZ);
> + (void) TAG_REGION (chunk2rawmem (oldp), oldsize);
> _int_free (av, oldp, 1);
> check_inuse_chunk (av, newp);
> return chunk2mem (newp);
> @@ -4681,6 +4870,8 @@ _int_realloc(mstate av, mchunkptr oldp, INTERNAL_SIZE_T oldsize,
> else /* split remainder */
> {
> remainder = chunk_at_offset (newp, nb);
> + /* Clear any user-space tags before writing the header. */
> + remainder = TAG_REGION (remainder, remainder_size);
> set_head_size (newp, nb | (av != &main_arena ? NON_MAIN_ARENA : 0));
> set_head (remainder, remainder_size | PREV_INUSE |
> (av != &main_arena ? NON_MAIN_ARENA : 0));
> @@ -4690,7 +4881,7 @@ _int_realloc(mstate av, mchunkptr oldp, INTERNAL_SIZE_T oldsize,
> }
>
> check_inuse_chunk (av, newp);
> - return chunk2mem (newp);
> + return TAG_NEW_USABLE (chunk2rawmem (newp));
> }
OK.
>
> /*
> @@ -4768,7 +4959,7 @@ _int_memalign (mstate av, size_t alignment, size_t bytes)
> p = newp;
>
> assert (newsize >= nb &&
> - (((unsigned long) (chunk2mem (p))) % alignment) == 0);
> + (((unsigned long) (chunk2rawmem (p))) % alignment) == 0);
> }
>
> /* Also give back spare room at the end */
> @@ -4822,7 +5013,8 @@ mtrim (mstate av, size_t pad)
> + sizeof (struct malloc_chunk)
> + psm1) & ~psm1);
>
> - assert ((char *) chunk2mem (p) + 4 * SIZE_SZ <= paligned_mem);
> + assert ((char *) chunk2rawmem (p) + 2 * CHUNK_HDR_SZ
> + <= paligned_mem);
> assert ((char *) p + size > paligned_mem);
>
> /* This is the size we could potentially free. */
> @@ -4885,20 +5077,30 @@ musable (void *mem)
> mchunkptr p;
> if (mem != 0)
> {
> + size_t result = 0;
> +
> p = mem2chunk (mem);
>
> if (__builtin_expect (using_malloc_checking == 1, 0))
> - return malloc_check_get_size (p);
> + return malloc_check_get_size (p);
>
> if (chunk_is_mmapped (p))
> {
> if (DUMPED_MAIN_ARENA_CHUNK (p))
> - return chunksize (p) - SIZE_SZ;
> + result = chunksize (p) - SIZE_SZ;
> else
> - return chunksize (p) - 2 * SIZE_SZ;
> + result = chunksize (p) - CHUNK_HDR_SZ;
> }
> else if (inuse (p))
> - return chunksize (p) - SIZE_SZ;
> + result = chunksize (p) - SIZE_SZ;
> +
> +#ifdef USE_MTAG
> + /* The usable space may be reduced if memory tagging is needed,
> + since we cannot share the user-space data with malloc's internal
> + data structure. */
> + result &= __mtag_granule_mask;
> +#endif
> + return result;
> }
> return 0;
> }
OK.
> diff --git a/malloc/malloc.h b/malloc/malloc.h
> index b2371f7704..0b20786b58 100644
> --- a/malloc/malloc.h
> +++ b/malloc/malloc.h
> @@ -77,6 +77,13 @@ extern void *pvalloc (size_t __size) __THROW __attribute_malloc__ __wur;
> contiguous pieces of memory. */
> extern void *(*__morecore) (ptrdiff_t __size) __MALLOC_DEPRECATED;
>
> +#ifdef USE_MTAG
> +extern int __mtag_mmap_flags;
> +#define MTAG_MMAP_FLAGS __mtag_mmap_flags
> +#else
> +#define MTAG_MMAP_FLAGS 0
> +#endif
> +
Do you really need this in an installed header? If not, maybe just put
it into include/malloc.h instead.
> /* Default value of `__morecore'. */
> extern void *__default_morecore (ptrdiff_t __size)
> __THROW __attribute_malloc__ __MALLOC_DEPRECATED;
> diff --git a/sysdeps/generic/libc-mtag.h b/sysdeps/generic/libc-mtag.h
> new file mode 100644
> index 0000000000..07f0203253
> --- /dev/null
> +++ b/sysdeps/generic/libc-mtag.h
> @@ -0,0 +1,52 @@
> +/* libc-internal interface for tagged (colored) memory support.
> + Copyright (C) 2020 Free Software Foundation, Inc.
> + This file is part of the GNU C Library.
> +
> + The GNU C Library is free software; you can redistribute it and/or
> + modify it under the terms of the GNU Lesser General Public
> + License as published by the Free Software Foundation; either
> + version 2.1 of the License, or (at your option) any later version.
> +
> + The GNU C Library is distributed in the hope that it will be useful,
> + but WITHOUT ANY WARRANTY; without even the implied warranty of
> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + Lesser General Public License for more details.
> +
> + You should have received a copy of the GNU Lesser General Public
> + License along with the GNU C Library; if not, see
> + <http://www.gnu.org/licenses/>. */
> +
> +#ifndef _GENERIC_LIBC_MTAG_H
> +#define _GENERIC_LIBC_MTAG_H 1
> +
> +/* Generic bindings for systems that do not support memory tagging. */
> +
> +/* Used to ensure additional alignment when objects need to have distinct
> + tags. */
> +#define __MTAG_GRANULE_SIZE 1
> +
> +/* Non-zero if memory obtained via morecore (sbrk) is not tagged. */
> +#define __MTAG_SBRK_UNTAGGED 0
> +
> +/* Extra flags to pass to mmap() to request a tagged region of memory. */
> +#define __MTAG_MMAP_FLAGS 0
> +
> +/* Set the tags for a region of memory, which must have size and alignment
> + that are multiples of __MTAG_GRANULE_SIZE. Size cannot be zero.
> + void *__libc_mtag_tag_region (const void *, size_t) */
> +#define __libc_mtag_tag_region(p, s) (p)
> +
> +/* Optimized equivalent to __libc_mtag_tag_region followed by memset. */
> +#define __libc_mtag_memset_with_tag memset
> +
> +/* Convert address P to a pointer that is tagged correctly for that
> + location.
> + void *__libc_mtag_address_get_tag (void*) */
> +#define __libc_mtag_address_get_tag(p) (p)
> +
> +/* Assign a new (random) tag to a pointer P (does not adjust the tag on
> + the memory addressed).
> + void *__libc_mtag_new_tag (void*) */
> +#define __libc_mtag_new_tag(p) (p)
> +
> +#endif /* _GENERIC_LIBC_MTAG_H */
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v4 4/6] linux: Add compatibility definitions to sys/prctl.h for MTE
2020-12-18 19:29 ` [PATCH v4 4/6] linux: Add compatibility definitions to sys/prctl.h for MTE Richard Earnshaw
@ 2020-12-21 13:32 ` Siddhesh Poyarekar
2020-12-21 13:34 ` Richard Earnshaw
2020-12-21 13:39 ` Florian Weimer
0 siblings, 2 replies; 25+ messages in thread
From: Siddhesh Poyarekar @ 2020-12-21 13:32 UTC (permalink / raw)
To: Richard Earnshaw, libc-alpha
On 12/19/20 12:59 AM, Richard Earnshaw via Libc-alpha wrote:
>
> Older versions of the Linux kernel headers obviously lack support for
> memory tagging, but we still want to be able to build in support when
> using those (obviously it can't be enabled on such systems).
>
> The linux kernel extensions are made to the platform-independent
> header (linux/prctl.h), so this patch takes a similar approach.
> ---
> sysdeps/unix/sysv/linux/sys/prctl.h | 18 ++++++++++++++++++
> 1 file changed, 18 insertions(+)
Please put the additions into an architecture-specific
sysdeps/unix/sysv/linux/aarch64/sys/prctl.h and include_next this file
from there.
>
> diff --git a/sysdeps/unix/sysv/linux/sys/prctl.h b/sysdeps/unix/sysv/linux/sys/prctl.h
> index 7f748ebeeb..4d01379c23 100644
> --- a/sysdeps/unix/sysv/linux/sys/prctl.h
> +++ b/sysdeps/unix/sysv/linux/sys/prctl.h
> @@ -21,6 +21,24 @@
> #include <features.h>
> #include <linux/prctl.h> /* The magic values come from here */
>
> +/* Recent extensions to linux which may post-date the kernel headers
> + we're picking up... */
> +
> +/* Memory tagging control operations (for AArch64). */
> +#ifndef PR_TAGGED_ADDR_ENABLE
> +# define PR_TAGGED_ADDR_ENABLE (1UL << 8)
> +#endif
> +
> +#ifndef PR_MTE_TCF_SHIFT
> +# define PR_MTE_TCF_SHIFT 1
> +# define PR_MTE_TCF_NONE (0UL << PR_MTE_TCF_SHIFT)
> +# define PR_MTE_TCF_SYNC (1UL << PR_MTE_TCF_SHIFT)
> +# define PR_MTE_TCF_ASYNC (2UL << PR_MTE_TCF_SHIFT)
> +# define PR_MTE_TCF_MASK (3UL << PR_MTE_TCF_SHIFT)
> +# define PR_MTE_TAG_SHIFT 3
> +# define PR_MTE_TAG_MASK (0xffffUL << PR_MTE_TAG_SHIFT)
> +#endif
> +
> __BEGIN_DECLS
>
> /* Control process execution. */
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v4 4/6] linux: Add compatibility definitions to sys/prctl.h for MTE
2020-12-21 13:32 ` Siddhesh Poyarekar
@ 2020-12-21 13:34 ` Richard Earnshaw
2020-12-21 13:38 ` Siddhesh Poyarekar
2020-12-21 13:39 ` Florian Weimer
1 sibling, 1 reply; 25+ messages in thread
From: Richard Earnshaw @ 2020-12-21 13:34 UTC (permalink / raw)
To: Siddhesh Poyarekar, Richard Earnshaw, libc-alpha
On 21/12/2020 13:32, Siddhesh Poyarekar wrote:
> On 12/19/20 12:59 AM, Richard Earnshaw via Libc-alpha wrote:
>>
>> Older versions of the Linux kernel headers obviously lack support for
>> memory tagging, but we still want to be able to build in support when
>> using those (obviously it can't be enabled on such systems).
>>
>> The linux kernel extensions are made to the platform-independent
>> header (linux/prctl.h), so this patch takes a similar approach.
>> ---
>> sysdeps/unix/sysv/linux/sys/prctl.h | 18 ++++++++++++++++++
>> 1 file changed, 18 insertions(+)
>
> Please put the additions into an architecture-specific
> sysdeps/unix/sysv/linux/aarch64/sys/prctl.h and include_next this file
> from there.
>
>>
>> diff --git a/sysdeps/unix/sysv/linux/sys/prctl.h
>> b/sysdeps/unix/sysv/linux/sys/prctl.h
>> index 7f748ebeeb..4d01379c23 100644
>> --- a/sysdeps/unix/sysv/linux/sys/prctl.h
>> +++ b/sysdeps/unix/sysv/linux/sys/prctl.h
>> @@ -21,6 +21,24 @@
>> #include <features.h>
>> #include <linux/prctl.h> /* The magic values come from here */
>>
>> +/* Recent extensions to linux which may post-date the kernel headers
>> + we're picking up... */
>> +
>> +/* Memory tagging control operations (for AArch64). */
>> +#ifndef PR_TAGGED_ADDR_ENABLE
>> +# define PR_TAGGED_ADDR_ENABLE (1UL << 8)
>> +#endif
>> +
>> +#ifndef PR_MTE_TCF_SHIFT
>> +# define PR_MTE_TCF_SHIFT 1
>> +# define PR_MTE_TCF_NONE (0UL << PR_MTE_TCF_SHIFT)
>> +# define PR_MTE_TCF_SYNC (1UL << PR_MTE_TCF_SHIFT)
>> +# define PR_MTE_TCF_ASYNC (2UL << PR_MTE_TCF_SHIFT)
>> +# define PR_MTE_TCF_MASK (3UL << PR_MTE_TCF_SHIFT)
>> +# define PR_MTE_TAG_SHIFT 3
>> +# define PR_MTE_TAG_MASK (0xffffUL << PR_MTE_TAG_SHIFT)
>> +#endif
>> +
>> __BEGIN_DECLS
>>
>> /* Control process execution. */
>
>
The standard linux kernel headers have this in an architecture
independent file, so why should glibc need to do this differently?
R.
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v4 5/6] aarch64: Add sysv specific enabling code for memory tagging
2020-12-18 19:29 ` [PATCH v4 5/6] aarch64: Add sysv specific enabling code for memory tagging Richard Earnshaw
2020-12-21 12:27 ` Szabolcs Nagy
@ 2020-12-21 13:36 ` Siddhesh Poyarekar
1 sibling, 0 replies; 25+ messages in thread
From: Siddhesh Poyarekar @ 2020-12-21 13:36 UTC (permalink / raw)
To: Richard Earnshaw, libc-alpha
On 12/19/20 12:59 AM, Richard Earnshaw via Libc-alpha wrote:
>
> Add various defines and stubs for enabling MTE on AArch64 sysv-like
> systems such as Linux. The HWCAP feature bit is copied over in the
> same way as other feature bits. Similarly we add a new wrapper header
> for mman.h to define the PROT_MTE flag that can be used with mmap and
> related functions.
>
> We add a new field to struct cpu_features that can be used, for
> example, to check whether or not certain ifunc'd routines should be
> bound to MTE-safe versions.
>
> Finally, if we detect that MTE should be enabled (ie via the glibc
> tunable); we enable MTE during startup as required.
> ---
> sysdeps/unix/sysv/linux/aarch64/bits/hwcap.h | 1 +
> sysdeps/unix/sysv/linux/aarch64/bits/mman.h | 1 +
> .../unix/sysv/linux/aarch64/cpu-features.c | 30 +++++++++++++++++++
> .../unix/sysv/linux/aarch64/cpu-features.h | 2 ++
> 4 files changed, 34 insertions(+)
Looks OK to me.
> diff --git a/sysdeps/unix/sysv/linux/aarch64/bits/hwcap.h b/sysdeps/unix/sysv/linux/aarch64/bits/hwcap.h
> index af90d8a626..389852f1d9 100644
> --- a/sysdeps/unix/sysv/linux/aarch64/bits/hwcap.h
> +++ b/sysdeps/unix/sysv/linux/aarch64/bits/hwcap.h
> @@ -73,3 +73,4 @@
> #define HWCAP2_DGH (1 << 15)
> #define HWCAP2_RNG (1 << 16)
> #define HWCAP2_BTI (1 << 17)
> +#define HWCAP2_MTE (1 << 18)
> diff --git a/sysdeps/unix/sysv/linux/aarch64/bits/mman.h b/sysdeps/unix/sysv/linux/aarch64/bits/mman.h
> index ecae046344..c5ec0aa7d0 100644
> --- a/sysdeps/unix/sysv/linux/aarch64/bits/mman.h
> +++ b/sysdeps/unix/sysv/linux/aarch64/bits/mman.h
> @@ -24,6 +24,7 @@
> arch/arm64/include/uapi/asm/mman.h. */
>
> #define PROT_BTI 0x10
> +#define PROT_MTE 0x20
>
> #include <bits/mman-map-flags-generic.h>
>
> diff --git a/sysdeps/unix/sysv/linux/aarch64/cpu-features.c b/sysdeps/unix/sysv/linux/aarch64/cpu-features.c
> index b9ab827aca..bd899c4b09 100644
> --- a/sysdeps/unix/sysv/linux/aarch64/cpu-features.c
> +++ b/sysdeps/unix/sysv/linux/aarch64/cpu-features.c
> @@ -19,10 +19,17 @@
> #include <cpu-features.h>
> #include <sys/auxv.h>
> #include <elf/dl-hwcaps.h>
> +#include <sys/prctl.h>
>
> #define DCZID_DZP_MASK (1 << 4)
> #define DCZID_BS_MASK (0xf)
>
> +/* The maximal set of permitted tags that the MTE random tag generation
> + instruction may use. We exclude tag 0 because a) we want to reserve
> + that for the libc heap structures and b) because it makes it easier
> + to see when pointer have been correctly tagged. */
> +#define MTE_ALLOWED_TAGS (0xfffe << PR_MTE_TAG_SHIFT)
A Nice(TM) looking variable to mask the beautiful hex that should stay
hidden ;)
> +
> #if HAVE_TUNABLES
> struct cpu_list
> {
> @@ -86,4 +93,27 @@ init_cpu_features (struct cpu_features *cpu_features)
>
> /* Check if BTI is supported. */
> cpu_features->bti = GLRO (dl_hwcap2) & HWCAP2_BTI;
> +
> + /* Setup memory tagging support if the HW and kernel support it, and if
> + the user has requested it. */
> + cpu_features->mte_state = 0;
> +
> +#ifdef USE_MTAG
> +# if HAVE_TUNABLES
> + int mte_state = TUNABLE_GET (glibc, mem, tagging, unsigned, 0);
> + cpu_features->mte_state = (GLRO (dl_hwcap2) & HWCAP2_MTE) ? mte_state : 0;
> + /* If we lack the MTE feature, disable the tunable, since it will
> + otherwise cause instructions that won't run on this CPU to be used. */
> + TUNABLE_SET (glibc, mem, tagging, unsigned, cpu_features->mte_state);
> +# endif
> +
> + if (cpu_features->mte_state & 2)
> + __prctl (PR_SET_TAGGED_ADDR_CTRL,
> + (PR_TAGGED_ADDR_ENABLE | PR_MTE_TCF_SYNC | MTE_ALLOWED_TAGS),
> + 0, 0, 0);
> + else if (cpu_features->mte_state)
> + __prctl (PR_SET_TAGGED_ADDR_CTRL,
> + (PR_TAGGED_ADDR_ENABLE | PR_MTE_TCF_ASYNC | MTE_ALLOWED_TAGS),
> + 0, 0, 0);
> +#endif
> }
> diff --git a/sysdeps/unix/sysv/linux/aarch64/cpu-features.h b/sysdeps/unix/sysv/linux/aarch64/cpu-features.h
> index 00a4d0c8e7..bebf321a21 100644
> --- a/sysdeps/unix/sysv/linux/aarch64/cpu-features.h
> +++ b/sysdeps/unix/sysv/linux/aarch64/cpu-features.h
> @@ -70,6 +70,8 @@ struct cpu_features
> uint64_t midr_el1;
> unsigned zva_size;
> bool bti;
> + /* Currently, the GLIBC memory tagging tunable only defines 8 bits. */
> + uint8_t mte_state;
> };
>
> #endif /* _CPU_FEATURES_AARCH64_H */
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v4 4/6] linux: Add compatibility definitions to sys/prctl.h for MTE
2020-12-21 13:34 ` Richard Earnshaw
@ 2020-12-21 13:38 ` Siddhesh Poyarekar
0 siblings, 0 replies; 25+ messages in thread
From: Siddhesh Poyarekar @ 2020-12-21 13:38 UTC (permalink / raw)
To: Richard Earnshaw, Richard Earnshaw, libc-alpha
On 12/21/20 7:04 PM, Richard Earnshaw wrote:
> The standard linux kernel headers have this in an architecture
> independent file, so why should glibc need to do this differently?
Only because that's the precedent we set with x86, where we avoided
writing into the architecture-independent header. It seems cleaner that
way too.
Siddhesh
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v4 4/6] linux: Add compatibility definitions to sys/prctl.h for MTE
2020-12-21 13:32 ` Siddhesh Poyarekar
2020-12-21 13:34 ` Richard Earnshaw
@ 2020-12-21 13:39 ` Florian Weimer
2020-12-21 13:41 ` Siddhesh Poyarekar
1 sibling, 1 reply; 25+ messages in thread
From: Florian Weimer @ 2020-12-21 13:39 UTC (permalink / raw)
To: Siddhesh Poyarekar; +Cc: Richard Earnshaw, libc-alpha
* Siddhesh Poyarekar:
> On 12/19/20 12:59 AM, Richard Earnshaw via Libc-alpha wrote:
>> Older versions of the Linux kernel headers obviously lack support
>> for
>> memory tagging, but we still want to be able to build in support when
>> using those (obviously it can't be enabled on such systems).
>> The linux kernel extensions are made to the platform-independent
>> header (linux/prctl.h), so this patch takes a similar approach.
>> ---
>> sysdeps/unix/sysv/linux/sys/prctl.h | 18 ++++++++++++++++++
>> 1 file changed, 18 insertions(+)
>
> Please put the additions into an architecture-specific
> sysdeps/unix/sysv/linux/aarch64/sys/prctl.h and include_next this file
> from there.
This does not work for an installed header. A separate bits/ header is
needed for such cases (if it is indeed required not to make the
definitions generic, which is probably not the case here—that is,
this part of Richard's patch can be used as-is).
Thanks,
Florian
--
Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v4 4/6] linux: Add compatibility definitions to sys/prctl.h for MTE
2020-12-21 13:39 ` Florian Weimer
@ 2020-12-21 13:41 ` Siddhesh Poyarekar
0 siblings, 0 replies; 25+ messages in thread
From: Siddhesh Poyarekar @ 2020-12-21 13:41 UTC (permalink / raw)
To: Florian Weimer; +Cc: Richard Earnshaw, libc-alpha
On 12/21/20 7:09 PM, Florian Weimer wrote:
> * Siddhesh Poyarekar:
>
>> On 12/19/20 12:59 AM, Richard Earnshaw via Libc-alpha wrote:
>>> Older versions of the Linux kernel headers obviously lack support
>>> for
>>> memory tagging, but we still want to be able to build in support when
>>> using those (obviously it can't be enabled on such systems).
>>> The linux kernel extensions are made to the platform-independent
>>> header (linux/prctl.h), so this patch takes a similar approach.
>>> ---
>>> sysdeps/unix/sysv/linux/sys/prctl.h | 18 ++++++++++++++++++
>>> 1 file changed, 18 insertions(+)
>>
>> Please put the additions into an architecture-specific
>> sysdeps/unix/sysv/linux/aarch64/sys/prctl.h and include_next this file
>> from there.
>
> This does not work for an installed header. A separate bits/ header is
> needed for such cases (if it is indeed required not to make the
> definitions generic, which is probably not the case here—that is,
> this part of Richard's patch can be used as-is).
Uff, indeed I didn't notice that it was an installed header. This is OK
as is then.
Thanks,
Siddhesh
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v4 0/6] Memory tagging support
2020-12-21 12:28 ` Siddhesh Poyarekar
@ 2020-12-21 13:44 ` Siddhesh Poyarekar
0 siblings, 0 replies; 25+ messages in thread
From: Siddhesh Poyarekar @ 2020-12-21 13:44 UTC (permalink / raw)
To: Richard Earnshaw, libc-alpha
On 12/21/20 5:58 PM, Siddhesh Poyarekar wrote:
> On 12/19/20 12:59 AM, Richard Earnshaw via Libc-alpha wrote:
>> Main changes in this version:
>>
>> - collapse the changes to the malloc code to a single patch.
>> - change _LIBC_MTAG to USE_MTAG.
>> - comments around definition of PROT_MTE.
>> - tunable renamed to glibc.mem.tagging.
>> - cleanups to assembler files for aarch64 support.
>>
>
> I've tested the build to verify that --enable-memory-tagging works
> correctly on x86_64 and the resulting bins don't regress the testsuite.
> I'm working through the individual patches to review; will post
> comments shortly.
I've gone through the patch series and I think it is good to go with a
couple of minor changes I've suggested in the reviews. I've mostly
deferred to Szabolcs' review for 5/6 and 6/6 since they're aarch64-specific.
The changes are small enough that you could commit with the changes.
However, please post the series that you commit so that we know what
went in finally.
Thanks,
Siddhesh
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v4 3/6] malloc: Basic support for memory tagging in the malloc() family
2020-12-18 19:29 ` [PATCH v4 3/6] malloc: Basic support for memory tagging in the malloc() family Richard Earnshaw
2020-12-21 13:27 ` Siddhesh Poyarekar
@ 2020-12-21 13:46 ` Florian Weimer
2020-12-21 14:31 ` Richard Earnshaw
2020-12-21 14:31 ` Szabolcs Nagy
1 sibling, 2 replies; 25+ messages in thread
From: Florian Weimer @ 2020-12-21 13:46 UTC (permalink / raw)
To: Richard Earnshaw via Libc-alpha; +Cc: Richard Earnshaw
* Richard Earnshaw via Libc-alpha:
> +/* Generate a new (random) tag value for PTR, set the tags for the
> + memory to the new tag and initialize the memory contents to VAL.
> + In practice this function will only be called with VAL=0, but we
> + keep this parameter to maintain the same prototype as memset. */
> +static void *
> +__mtag_tag_new_memset (void *ptr, int val, size_t size)
> +{
> + return __libc_mtag_memset_with_tag (__libc_mtag_new_tag (ptr), val, size);
> +}
I would like to point out that random choice from all possible tag bits
precludes some memory tagging applications. Some applications might
want to unconditionally force certain tag bits on a load, to assert that
the pointer refers to a specific kind of memory. If glibc malloc
randomly assigns tag bits from entire range, this kind of memory type
assertion is no longer eliable.
Thanks,
Florian
--
Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v4 3/6] malloc: Basic support for memory tagging in the malloc() family
2020-12-21 13:46 ` Florian Weimer
@ 2020-12-21 14:31 ` Richard Earnshaw
2020-12-21 14:31 ` Szabolcs Nagy
1 sibling, 0 replies; 25+ messages in thread
From: Richard Earnshaw @ 2020-12-21 14:31 UTC (permalink / raw)
To: Florian Weimer, Richard Earnshaw via Libc-alpha; +Cc: Richard Earnshaw
On 21/12/2020 13:46, Florian Weimer via Libc-alpha wrote:
> * Richard Earnshaw via Libc-alpha:
>
>> +/* Generate a new (random) tag value for PTR, set the tags for the
>> + memory to the new tag and initialize the memory contents to VAL.
>> + In practice this function will only be called with VAL=0, but we
>> + keep this parameter to maintain the same prototype as memset. */
>> +static void *
>> +__mtag_tag_new_memset (void *ptr, int val, size_t size)
>> +{
>> + return __libc_mtag_memset_with_tag (__libc_mtag_new_tag (ptr), val, size);
>> +}
>
> I would like to point out that random choice from all possible tag bits
> precludes some memory tagging applications. Some applications might
> want to unconditionally force certain tag bits on a load, to assert that
> the pointer refers to a specific kind of memory. If glibc malloc
> randomly assigns tag bits from entire range, this kind of memory type
> assertion is no longer eliable.
>
I'm sure that there are several enhancements to these patches that are
possible and the above is just one of them. I think they can be worked
on as follow ups. The more hooks, however, the more performance impact
there is likely to be, even if it's just an extra indirect function call.
A limitation of the architecture is that, in effect, the tags used are
completely random (with no seed that can be used to get repeatability);
that's good from a security standpoint (avoids common-mode failures),
but makes things more difficult if you're trying to reproduce an issue
under a debugger. I'm not sure how to address that either.
R.
> Thanks,
> Florian
>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v4 3/6] malloc: Basic support for memory tagging in the malloc() family
2020-12-21 13:46 ` Florian Weimer
2020-12-21 14:31 ` Richard Earnshaw
@ 2020-12-21 14:31 ` Szabolcs Nagy
1 sibling, 0 replies; 25+ messages in thread
From: Szabolcs Nagy @ 2020-12-21 14:31 UTC (permalink / raw)
To: Florian Weimer; +Cc: Richard Earnshaw via Libc-alpha, Richard Earnshaw
The 12/21/2020 14:46, Florian Weimer via Libc-alpha wrote:
> * Richard Earnshaw via Libc-alpha:
> > +/* Generate a new (random) tag value for PTR, set the tags for the
> > + memory to the new tag and initialize the memory contents to VAL.
> > + In practice this function will only be called with VAL=0, but we
> > + keep this parameter to maintain the same prototype as memset. */
> > +static void *
> > +__mtag_tag_new_memset (void *ptr, int val, size_t size)
> > +{
> > + return __libc_mtag_memset_with_tag (__libc_mtag_new_tag (ptr), val, size);
> > +}
>
> I would like to point out that random choice from all possible tag bits
> precludes some memory tagging applications. Some applications might
> want to unconditionally force certain tag bits on a load, to assert that
> the pointer refers to a specific kind of memory. If glibc malloc
> randomly assigns tag bits from entire range, this kind of memory type
> assertion is no longer eliable.
in the mte architecture we can control the set of tags
__libc_mtag_new_tag (irg instruction) may select from.
currently we set the MTE_ALLOWED_TAGS in aarch64 via
prctl such that all tags are allowed except 0.
i imagine if we have a usecase for using specific tags
somewhere then we would exclude those from the allowed
random tags.
(e.g. malloc metadata in heap memory is 0 tagged now
which is guaranteed to be different from the user
allocation tags, but we could reserve a special tag
for metadata and exclude that from the allowed tags.)
note that currently user code cannot easily use tagging:
the prctl settings are owned by the libc and cannot be
changed easily in a multithreaded process. suballocators
cannot retag heap memory unless they revert the tags
before calling free. and the PROT_MTE setting for most
variables are libc controlled (globals in elf objects
heap and stack). so only manually mmaped memory can use
tags in user code. we could reserve some tags for such
usage that are distinct from heap tags, but havent so
far.
since we haven't committed to a stable abi yet with the
tunables i think we have opportunity to change this if
necessary.
^ permalink raw reply [flat|nested] 25+ messages in thread
end of thread, other threads:[~2020-12-21 14:32 UTC | newest]
Thread overview: 25+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-12-18 19:29 [PATCH v4 0/6] Memory tagging support Richard Earnshaw
2020-12-18 19:29 ` [PATCH v4 1/6] config: Allow memory tagging to be enabled when configuring glibc Richard Earnshaw
2020-12-21 12:40 ` Siddhesh Poyarekar
2020-12-18 19:29 ` [PATCH v4 2/6] elf: Add a tunable to control use of tagged memory Richard Earnshaw
2020-12-21 12:42 ` Siddhesh Poyarekar
2020-12-18 19:29 ` [PATCH v4 3/6] malloc: Basic support for memory tagging in the malloc() family Richard Earnshaw
2020-12-21 13:27 ` Siddhesh Poyarekar
2020-12-21 13:46 ` Florian Weimer
2020-12-21 14:31 ` Richard Earnshaw
2020-12-21 14:31 ` Szabolcs Nagy
2020-12-18 19:29 ` [PATCH v4 4/6] linux: Add compatibility definitions to sys/prctl.h for MTE Richard Earnshaw
2020-12-21 13:32 ` Siddhesh Poyarekar
2020-12-21 13:34 ` Richard Earnshaw
2020-12-21 13:38 ` Siddhesh Poyarekar
2020-12-21 13:39 ` Florian Weimer
2020-12-21 13:41 ` Siddhesh Poyarekar
2020-12-18 19:29 ` [PATCH v4 5/6] aarch64: Add sysv specific enabling code for memory tagging Richard Earnshaw
2020-12-21 12:27 ` Szabolcs Nagy
2020-12-21 13:36 ` Siddhesh Poyarekar
2020-12-18 19:29 ` [PATCH v4 6/6] aarch64: Add aarch64-specific files for memory tagging support Richard Earnshaw
2020-12-21 12:44 ` Szabolcs Nagy
2020-12-21 12:50 ` Richard Earnshaw
2020-12-18 20:18 ` [PATCH v4 0/6] Memory " H.J. Lu
2020-12-21 12:28 ` Siddhesh Poyarekar
2020-12-21 13:44 ` Siddhesh Poyarekar
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).