* [PATCH] Fix thread-safety for elfutils
@ 2023-08-08 17:07 Heather McIntyre
2023-08-21 22:08 ` John Mellor-Crummey
2023-10-10 13:40 ` Mark Wielaard
0 siblings, 2 replies; 51+ messages in thread
From: Heather McIntyre @ 2023-08-08 17:07 UTC (permalink / raw)
To: elfutils-devel; +Cc: John Mellor-Crummey
[-- Attachment #1.1: Type: text/plain, Size: 699 bytes --]
Hello all,
This patch was created to address thread-safety issues reported in bug 26921
<https://sourceware.org/bugzilla/show_bug.cgi?id=26921> and bug 26930
<https://sourceware.org/bugzilla/show_bug.cgi?id=26930>.
Additionally, other thread-safety fixes were applied during the process.
Brief Description:
Locking was implemented for tsearch and tfind.
Changes to multiple files within the libelf library were made such that
USE_LOCKS no longer causes tests to fail when enabled.
New tests were added to confirm thread-safety.
Please review the attached patch file and feel free to provide feedback. I
am available for any clarifications or modifications needed.
Best regards,
Heather McIntyre
[-- Attachment #2: 0001-Fix-thread-safety-for-elfutils.patch --]
[-- Type: application/octet-stream, Size: 83636 bytes --]
From ee7bf3f07a3fe99cf6e6152803271a1b77445350 Mon Sep 17 00:00:00 2001
From: "Heather S. McIntyre" <hsm2@rice.edu>
Date: Tue, 8 Aug 2023 11:24:01 -0500
Subject: [PATCH] Fix thread-safety for elfutils
Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
---
ChangeLog | 5 +
configure.ac | 6 +-
lib/ChangeLog | 10 ++
lib/Makefile.am | 2 +-
lib/eu-config.h | 33 +++--
lib/eu-search.c | 58 ++++++++
lib/eu-search.h | 37 +++++
libcpu/ChangeLog | 6 +
libcpu/Makefile.am | 3 +
libcpu/i386_parse.y | 48 +++----
libdw/ChangeLog | 19 +++
libdw/cie.c | 8 +-
libdw/dwarf_getalt.c | 44 ++++--
libdw/dwarf_getlocation.c | 16 +--
libdw/dwarf_getmacros.c | 6 +-
libdw/dwarf_getsrclines.c | 6 +-
libdw/dwarf_hasattr.c | 9 ++
libdw/fde.c | 6 +-
libdw/libdw_find_split_unit.c | 66 ++++++---
libdw/libdw_findcu.c | 52 ++++---
libdwfl/ChangeLog | 5 +
libdwfl/cu.c | 4 +-
libelf/ChangeLog | 15 +-
libelf/elf32_getchdr.c | 46 +-----
libelf/elf32_getchdr.h | 61 ++++++++
libelf/elf32_updatenull.c | 2 +-
libelf/elf_cntl.c | 11 +-
libelf/elf_end.c | 6 +-
libelf/elf_getdata.c | 14 ++
libelf/elf_readall.c | 5 +-
libelf/elf_version.c | 11 +-
libelf/libelfP.h | 4 +
src/ChangeLog | 7 +
src/Makefile.am | 3 +
src/findtextrel.c | 10 +-
src/nm.c | 10 +-
tests/ChangeLog | 21 +++
tests/Makefile.am | 16 ++-
tests/eu_search_cfi.c | 234 ++++++++++++++++++++++++++++++
tests/eu_search_die.c | 262 ++++++++++++++++++++++++++++++++++
tests/eu_search_lines.c | 228 +++++++++++++++++++++++++++++
tests/eu_search_macros.c | 216 ++++++++++++++++++++++++++++
tests/run-eu-search-tests.sh | 168 ++++++++++++++++++++++
43 files changed, 1618 insertions(+), 181 deletions(-)
create mode 100644 lib/eu-search.c
create mode 100644 lib/eu-search.h
create mode 100644 libelf/elf32_getchdr.h
create mode 100644 tests/eu_search_cfi.c
create mode 100644 tests/eu_search_die.c
create mode 100644 tests/eu_search_lines.c
create mode 100644 tests/eu_search_macros.c
create mode 100644 tests/run-eu-search-tests.sh
diff --git a/ChangeLog b/ChangeLog
index 6aed95b6..c555c986 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,8 @@
+2023-08-08 Heather McIntyre <hsm2@rice.edu>
+
+ * NEWS: Changes implemented to support thread-safety.
+ * configure.ac (--enable-thread-safety): Remove experimental warning.
+
2023-03-27 Di Chen <dichen@redhat.com>
* NEWS: Support readelf -Ds for using dynamic segment to
diff --git a/configure.ac b/configure.ac
index 4b67c844..f86f6492 100644
--- a/configure.ac
+++ b/configure.ac
@@ -79,12 +79,10 @@ AC_DEFINE_UNQUOTED(DEFAULT_AR_DETERMINISTIC, $default_ar_deterministic,
AC_ARG_ENABLE([thread-safety],
AS_HELP_STRING([--enable-thread-safety],
- [enable thread safety of libraries EXPERIMENTAL]),
+ [enable thread safety of libraries]),
use_locks=$enableval, use_locks=no)
AM_CONDITIONAL(USE_LOCKS, test "$use_locks" = yes)
AS_IF([test "$use_locks" = yes], [AC_DEFINE(USE_LOCKS)])
-AS_IF([test "$use_locks" = yes],
- [AC_MSG_WARN([thread-safety is EXPERIMENTAL tests might fail.])])
AH_TEMPLATE([USE_LOCKS], [Defined if libraries should be thread-safe.])
@@ -906,10 +904,10 @@ AC_MSG_NOTICE([
Symbol versioning : ${enable_symbol_versioning}
NOT RECOMMENDED FEATURES (should all be no)
- Experimental thread safety : ${use_locks}
install elf.h : ${install_elfh}
OTHER FEATURES
+ Enable thread safety : ${use_locks}
Deterministic archives by default : ${default_ar_deterministic}
Native language support : ${USE_NLS}
Extra Valgrind annotations : ${use_vg_annotations}
diff --git a/lib/ChangeLog b/lib/ChangeLog
index 5ab9477e..7f0533cc 100644
--- a/lib/ChangeLog
+++ b/lib/ChangeLog
@@ -1,3 +1,13 @@
+2023-08-08 Heather McIntyre <hsm2@rice.edu>
+
+ * eu-search.h: New file.
+ Declarations for read/write locked eu_tsearch/eu_tfind.
+ * eu-search.c: New file.
+ Definitions for read/write locked eu_tsearch/eu_tfind.
+ * eu-config.h New macros.
+ [USE_LOCKS] (ONCE_CALL): (once_define, once)
+ * Makefile.am (libeu_a_SOURCES): Add eu-search.c.
+
2022-12-20 Mark Wielaard <mark@klomp.org>
* Makefile.am (xmalloc_CFLAGS): Remove.
diff --git a/lib/Makefile.am b/lib/Makefile.am
index b3bb929f..ce8f3e1b 100644
--- a/lib/Makefile.am
+++ b/lib/Makefile.am
@@ -34,7 +34,7 @@ AM_CPPFLAGS += -I$(srcdir)/../libelf
noinst_LIBRARIES = libeu.a
libeu_a_SOURCES = xasprintf.c xstrdup.c xstrndup.c xmalloc.c next_prime.c \
- crc32.c crc32_file.c \
+ crc32.c crc32_file.c eu-search.c \
color.c error.c printversion.c
noinst_HEADERS = fixedsizehash.h libeu.h system.h dynamicsizehash.h list.h \
diff --git a/lib/eu-config.h b/lib/eu-config.h
index 78a5c4fe..d43a8f78 100644
--- a/lib/eu-config.h
+++ b/lib/eu-config.h
@@ -32,24 +32,27 @@
#ifdef USE_LOCKS
# include <pthread.h>
# include <assert.h>
-# define rwlock_define(class,name) class pthread_rwlock_t name
-# define RWLOCK_CALL(call) \
+# define rwlock_define(class,name) class pthread_rwlock_t name
+# define once_define(class,name) class pthread_once_t name = PTHREAD_ONCE_INIT
+# define RWLOCK_CALL(call) \
({ int _err = pthread_rwlock_ ## call; assert_perror (_err); })
-# define rwlock_init(lock) RWLOCK_CALL (init (&lock, NULL))
-# define rwlock_fini(lock) RWLOCK_CALL (destroy (&lock))
-# define rwlock_rdlock(lock) RWLOCK_CALL (rdlock (&lock))
-# define rwlock_wrlock(lock) RWLOCK_CALL (wrlock (&lock))
-# define rwlock_unlock(lock) RWLOCK_CALL (unlock (&lock))
+# define ONCE_CALL(call) \
+ ({ int _err = pthread_ ## call; assert_perror (_err); })
+# define rwlock_init(lock) RWLOCK_CALL (init (&lock, NULL))
+# define rwlock_fini(lock) RWLOCK_CALL (destroy (&lock))
+# define rwlock_rdlock(lock) RWLOCK_CALL (rdlock (&lock))
+# define rwlock_wrlock(lock) RWLOCK_CALL (wrlock (&lock))
+# define rwlock_unlock(lock) RWLOCK_CALL (unlock (&lock))
+# define once(once_control, init_routine) ONCE_CALL (once (&once_control, init_routine))
#else
-/* Eventually we will allow multi-threaded applications to use the
- libraries. Therefore we will add the necessary locking although
- the macros used expand to nothing for now. */
# define rwlock_define(class,name) class int name
-# define rwlock_init(lock) ((void) (lock))
-# define rwlock_fini(lock) ((void) (lock))
-# define rwlock_rdlock(lock) ((void) (lock))
-# define rwlock_wrlock(lock) ((void) (lock))
-# define rwlock_unlock(lock) ((void) (lock))
+# define rwlock_init(lock) ((void) (lock))
+# define rwlock_fini(lock) ((void) (lock))
+# define rwlock_rdlock(lock) ((void) (lock))
+# define rwlock_wrlock(lock) ((void) (lock))
+# define rwlock_unlock(lock) ((void) (lock))
+# define once_define(class,name)
+# define once(once_control, init_routine) init_routine()
#endif /* USE_LOCKS */
#include <libintl.h>
diff --git a/lib/eu-search.c b/lib/eu-search.c
new file mode 100644
index 00000000..b15d61b9
--- /dev/null
+++ b/lib/eu-search.c
@@ -0,0 +1,58 @@
+/* Definitions for thread-safe tsearch/tfind
+ Copyright (C) 2023 Red Hat, Inc.
+ This file is part of elfutils.
+
+ This file is free software; you can redistribute it and/or modify
+ it under the terms of either
+
+ * the GNU Lesser General Public License as published by the Free
+ Software Foundation; either version 3 of the License, or (at
+ your option) any later version
+
+ or
+
+ * the GNU General Public License as published by the Free
+ Software Foundation; either version 2 of the License, or (at
+ your option) any later version
+
+ or both in parallel, as here.
+
+ elfutils is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ General Public License for more details.
+
+ You should have received copies of the GNU General Public License and
+ the GNU Lesser General Public License along with this program. If
+ not, see <http://www.gnu.org/licenses/>. */
+
+#ifdef HAVE_CONFIG_H
+#include <config.h>
+#endif
+
+#include <stdlib.h>
+#include "eu-search.h"
+
+rwlock_define(static, search_find_lock);
+
+void *eu_tsearch(const void *key, void **rootp, int (*compar)(const void *, const void *))
+{
+ void *ret = NULL;
+ rwlock_wrlock(search_find_lock);
+
+ ret = tsearch(key, rootp, compar);
+
+ rwlock_unlock(search_find_lock);
+ return ret;
+}
+
+void *eu_tfind(const void *key, void *const *rootp, int (*compar)(const void *, const void *))
+{
+ void *ret = NULL;
+ rwlock_rdlock(search_find_lock);
+
+ ret = tfind(key, rootp, compar);
+
+ rwlock_unlock(search_find_lock);
+ return ret;
+}
\ No newline at end of file
diff --git a/lib/eu-search.h b/lib/eu-search.h
new file mode 100644
index 00000000..5fb335f2
--- /dev/null
+++ b/lib/eu-search.h
@@ -0,0 +1,37 @@
+/* Calls for thread-safe tsearch/tfind
+ Copyright (C) 2023 Red Hat, Inc.
+ This file is part of elfutils.
+
+ This file is free software; you can redistribute it and/or modify
+ it under the terms of either
+
+ * the GNU Lesser General Public License as published by the Free
+ Software Foundation; either version 3 of the License, or (at
+ your option) any later version
+
+ or
+
+ * the GNU General Public License as published by the Free
+ Software Foundation; either version 2 of the License, or (at
+ your option) any later version
+
+ or both in parallel, as here.
+
+ elfutils is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ General Public License for more details.
+
+ You should have received copies of the GNU General Public License and
+ the GNU Lesser General Public License along with this program. If
+ not, see <http://www.gnu.org/licenses/>. */
+
+#ifndef EU_SEARCH_H
+#define EU_SEARCH_H 1
+
+#include <search.h>
+
+extern void *eu_tsearch(const void *key, void **rootp, int (*compar)(const void *, const void *));
+extern void *eu_tfind(const void *key, void *const *rootp, int (*compar)(const void *, const void *));
+
+#endif
\ No newline at end of file
diff --git a/libcpu/ChangeLog b/libcpu/ChangeLog
index d14cd237..8c6cfd68 100644
--- a/libcpu/ChangeLog
+++ b/libcpu/ChangeLog
@@ -1,3 +1,9 @@
+2023-08-08 Heather McIntyre <hsm2@rice.edu>
+
+ * Makefile.am: Add USE_LOCKS condition for -pthread.
+ * i386_parse.y: Add eu-search.h and remove search.h.
+ Change calls for tsearch/tfind to eu_tsearch/eu_tfind.
+
2022-12-18 Yonggang Luo <luoyonggang@gmail.com>
* i386_mne.h: New file, extracted from i386_disasm.c.
diff --git a/libcpu/Makefile.am b/libcpu/Makefile.am
index 4ba1be56..a51334b5 100644
--- a/libcpu/Makefile.am
+++ b/libcpu/Makefile.am
@@ -33,6 +33,9 @@ AM_CPPFLAGS += -I$(srcdir)/../libelf -I$(srcdir)/../libebl \
if BUILD_STATIC
AM_CFLAGS += $(fpic_CFLAGS)
endif
+if USE_LOCKS
+ AM_CFLAGS += -pthread
+endif
AM_CFLAGS += -fdollars-in-identifiers
LEXCOMPILE = $(LEX) $(LFLAGS) $(AM_LFLAGS) -P$(<F:lex.l=)
LEX_OUTPUT_ROOT = lex.$(<F:lex.l=)
diff --git a/libcpu/i386_parse.y b/libcpu/i386_parse.y
index 459684c6..3d7cb89e 100644
--- a/libcpu/i386_parse.y
+++ b/libcpu/i386_parse.y
@@ -37,7 +37,7 @@
#include <inttypes.h>
#include <math.h>
#include <obstack.h>
-#include <search.h>
+#include <eu-search.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
@@ -269,12 +269,12 @@ mask: kMASK kBITFIELD kNUMBER
struct synonym *newp = xmalloc (sizeof (*newp));
newp->from = $2;
newp->to = $3;
- if (tfind (newp, &synonyms, compare_syn) != NULL)
+ if (eu_tfind (newp, &synonyms, compare_syn) != NULL)
error (0, 0,
"%d: duplicate definition for synonym '%s'",
i386_lineno, $2);
- else if (tsearch ( newp, &synonyms, compare_syn) == NULL)
- error (EXIT_FAILURE, 0, "tsearch");
+ else if (eu_tsearch ( newp, &synonyms, compare_syn) == NULL)
+ error (EXIT_FAILURE, 0, "eu_tsearch");
}
|
;
@@ -308,12 +308,12 @@ instr: bytes ':' bitfieldopt kID bitfieldopt optargs
newp->bytes = $1;
newp->mnemonic = $4;
if (newp->mnemonic != (void *) -1l
- && tfind ($4, &mnemonics,
+ && eu_tfind ($4, &mnemonics,
(int (*)(const void *, const void *)) strcmp) == NULL)
{
- if (tsearch ($4, &mnemonics,
+ if (eu_tsearch ($4, &mnemonics,
(int (*)(const void *, const void *)) strcmp) == NULL)
- error (EXIT_FAILURE, errno, "tsearch");
+ error (EXIT_FAILURE, errno, "eu_tsearch");
++nmnemonics;
}
@@ -339,15 +339,15 @@ instr: bytes ':' bitfieldopt kID bitfieldopt optargs
infname, i386_lineno - 1, $5->name);
struct suffix search = { .name = $5->name };
- if (tfind (&search, &suffixes, compare_suf)
+ if (eu_tfind (&search, &suffixes, compare_suf)
== NULL)
{
struct suffix *ns = xmalloc (sizeof (*ns));
ns->name = $5->name;
ns->idx = ++nsuffixes;
- if (tsearch (ns, &suffixes, compare_suf)
+ if (eu_tsearch (ns, &suffixes, compare_suf)
== NULL)
- error (EXIT_FAILURE, errno, "tsearch");
+ error (EXIT_FAILURE, errno, "eu_tsearch");
}
}
@@ -374,7 +374,7 @@ bitfieldopt: kBITFIELD
struct known_bitfield search;
search.name = $1;
struct known_bitfield **res;
- res = tfind (&search, &bitfields, bitfield_compare);
+ res = eu_tfind (&search, &bitfields, bitfield_compare);
if (res == NULL)
{
error (0, 0, "%d: unknown bitfield '%s'",
@@ -437,7 +437,7 @@ bit: '0'
struct known_bitfield search;
search.name = $1;
struct known_bitfield **res;
- res = tfind (&search, &bitfields, bitfield_compare);
+ res = eu_tfind (&search, &bitfields, bitfield_compare);
if (res == NULL)
{
error (0, 0, "%d: unknown bitfield '%s'",
@@ -497,7 +497,7 @@ argcomp: kBITFIELD
struct known_bitfield search;
search.name = $1;
struct known_bitfield **res;
- res = tfind (&search, &bitfields, bitfield_compare);
+ res = eu_tfind (&search, &bitfields, bitfield_compare);
if (res == NULL)
{
if (strcmp ($1, "ax") == 0)
@@ -575,7 +575,7 @@ new_bitfield (char *name, unsigned long int num)
newp->bits = num;
newp->tmp = 0;
- if (tfind (newp, &bitfields, bitfield_compare) != NULL)
+ if (eu_tfind (newp, &bitfields, bitfield_compare) != NULL)
{
error (0, 0, "%d: duplicated definition of bitfield '%s'",
i386_lineno, name);
@@ -584,7 +584,7 @@ new_bitfield (char *name, unsigned long int num)
return;
}
- if (tsearch (newp, &bitfields, bitfield_compare) == NULL)
+ if (eu_tsearch (newp, &bitfields, bitfield_compare) == NULL)
error (EXIT_FAILURE, errno, "%d: cannot insert new bitfield '%s'",
i386_lineno, name);
}
@@ -813,7 +813,7 @@ fillin_arg (struct bitvalue *bytes, struct argname *name,
struct synonym search = { .from = fieldname };
- struct synonym **res = tfind (&search, &synonyms, compare_syn);
+ struct synonym **res = eu_tfind (&search, &synonyms, compare_syn);
if (res != NULL)
fieldname = (*res)->to;
@@ -914,26 +914,26 @@ find_numbers (void)
if (runp->operands[i].fct != NULL)
{
struct argstring search = { .str = runp->operands[i].fct };
- if (tfind (&search, &fct_names[i], compare_argstring) == NULL)
+ if (eu_tfind (&search, &fct_names[i], compare_argstring) == NULL)
{
struct argstring *newp = xmalloc (sizeof (*newp));
newp->str = runp->operands[i].fct;
newp->idx = 0;
- if (tsearch (newp, &fct_names[i], compare_argstring) == NULL)
- error (EXIT_FAILURE, errno, "tsearch");
+ if (eu_tsearch (newp, &fct_names[i], compare_argstring) == NULL)
+ error (EXIT_FAILURE, errno, "eu_tsearch");
++nfct_names[i];
}
if (runp->operands[i].str != NULL)
{
search.str = runp->operands[i].str;
- if (tfind (&search, &strs[i], compare_argstring) == NULL)
+ if (eu_tfind (&search, &strs[i], compare_argstring) == NULL)
{
struct argstring *newp = xmalloc (sizeof (*newp));
newp->str = runp->operands[i].str;
newp->idx = 0;
- if (tsearch (newp, &strs[i], compare_argstring) == NULL)
- error (EXIT_FAILURE, errno, "tsearch");
+ if (eu_tsearch (newp, &strs[i], compare_argstring) == NULL)
+ error (EXIT_FAILURE, errno, "eu_tsearch");
++nstrs[i];
}
}
@@ -1206,7 +1206,7 @@ instrtable_out (void)
if (instr->operands[i].fct != NULL)
{
struct argstring search = { .str = instr->operands[i].fct };
- struct argstring **res = tfind (&search, &fct_names[i],
+ struct argstring **res = eu_tfind (&search, &fct_names[i],
compare_argstring);
assert (res != NULL);
idx = (*res)->idx;
@@ -1217,7 +1217,7 @@ instrtable_out (void)
if (instr->operands[i].str != NULL)
{
struct argstring search = { .str = instr->operands[i].str };
- struct argstring **res = tfind (&search, &strs[i],
+ struct argstring **res = eu_tfind (&search, &strs[i],
compare_argstring);
assert (res != NULL);
idx = (*res)->idx;
diff --git a/libdw/ChangeLog b/libdw/ChangeLog
index 406310ef..3efbbfee 100644
--- a/libdw/ChangeLog
+++ b/libdw/ChangeLog
@@ -1,3 +1,22 @@
+2023-08-08 Heather McIntyre <hsm2@rice.edu>
+
+ * cie.c: Add eu-search.h and remove search.h.
+ Change calls of tsearch/tfind to eu_tsearch/eu_tfind.
+ * dwarf_getlocation.c: Likewise.
+ * dwarf_getmacros.c: Likewise.
+ * dwarf_getsrclines.c: :Likewise.
+ * fde.c: Likewise.
+ * libdw_find_split_unit.c: Likewise.
+ * libdw_findcu.c: Likewise.
+ Add lock for next_tu_offset/next_cu_offset.
+ Reduce return statements in __libdw_findcu().
+ * dwarf_getalt.c: Likewise.
+ Add lock for dbg->alt_dwarf/main->alt_dwarf.
+ * dwarf_hasattr.c: Likewise.
+ Add lock for __libdw_dieabbrev().
+ * libdw_find_split_unit.c: Likewise.
+ Add lock for cu->split.
+
2023-02-22 Mark Wielaard <mark@klomp.org>
* dwarf_getscopes.c (origin_match): Don't free a->scopes.
diff --git a/libdw/cie.c b/libdw/cie.c
index 1b0aae7c..758daef5 100644
--- a/libdw/cie.c
+++ b/libdw/cie.c
@@ -33,7 +33,7 @@
#include "cfi.h"
#include "encoded-value.h"
#include <assert.h>
-#include <search.h>
+#include <eu-search.h>
#include <stdlib.h>
@@ -144,7 +144,7 @@ intern_new_cie (Dwarf_CFI *cache, Dwarf_Off offset, const Dwarf_CIE *info)
cie->initial_state = NULL;
/* Add the new entry to the search tree. */
- if (tsearch (cie, &cache->cie_tree, &compare_cie) == NULL)
+ if (eu_tsearch (cie, &cache->cie_tree, &compare_cie) == NULL)
{
free (cie);
__libdw_seterrno (DWARF_E_NOMEM);
@@ -160,7 +160,7 @@ internal_function
__libdw_find_cie (Dwarf_CFI *cache, Dwarf_Off offset)
{
const struct dwarf_cie cie_key = { .offset = offset };
- struct dwarf_cie **found = tfind (&cie_key, &cache->cie_tree, &compare_cie);
+ struct dwarf_cie **found = eu_tfind (&cie_key, &cache->cie_tree, &compare_cie);
if (found != NULL)
return *found;
@@ -189,7 +189,7 @@ internal_function
__libdw_intern_cie (Dwarf_CFI *cache, Dwarf_Off offset, const Dwarf_CIE *info)
{
const struct dwarf_cie cie_key = { .offset = offset };
- struct dwarf_cie **found = tfind (&cie_key, &cache->cie_tree, &compare_cie);
+ struct dwarf_cie **found = eu_tfind (&cie_key, &cache->cie_tree, &compare_cie);
if (found == NULL)
/* We have not read this CIE yet. Enter it. */
(void) intern_new_cie (cache, offset, info);
diff --git a/libdw/dwarf_getalt.c b/libdw/dwarf_getalt.c
index 0a12dfae..f0bc7b56 100644
--- a/libdw/dwarf_getalt.c
+++ b/libdw/dwarf_getalt.c
@@ -44,6 +44,10 @@
#include <sys/types.h>
#include <sys/stat.h>
+/* find_debug_altlink() modifies "dbg->alt_dwarf".
+ dwarf_getalt() reads "main->alt_dwarf".
+ Mutual exclusion is enforced to prevent a race. */
+rwlock_define(static, alt_dwarf_lock);
char *
internal_function
@@ -151,34 +155,52 @@ find_debug_altlink (Dwarf *dbg)
{
Dwarf *alt = dwarf_begin (fd, O_RDONLY);
if (alt != NULL)
- {
- dbg->alt_dwarf = alt;
- dbg->alt_fd = fd;
- }
+ {
+ rwlock_wrlock(alt_dwarf_lock);
+ dbg->alt_dwarf = alt;
+ rwlock_unlock(alt_dwarf_lock);
+
+ dbg->alt_fd = fd;
+ }
else
- close (fd);
- }
+ close (fd);
+ }
}
Dwarf *
dwarf_getalt (Dwarf *main)
{
+ rwlock_rdlock(alt_dwarf_lock);
+ Dwarf *alt_dwarf_local = main->alt_dwarf;
+ rwlock_unlock(alt_dwarf_lock);
+
/* Only try once. */
- if (main == NULL || main->alt_dwarf == (void *) -1)
+ if (main == NULL || alt_dwarf_local == (void *) -1)
+ {
return NULL;
+ }
- if (main->alt_dwarf != NULL)
- return main->alt_dwarf;
+ if (alt_dwarf_local != NULL)
+ {
+ return alt_dwarf_local;
+ }
find_debug_altlink (main);
+ rwlock_rdlock(alt_dwarf_lock);
+ alt_dwarf_local = main->alt_dwarf;
+ rwlock_unlock(alt_dwarf_lock);
+
/* If we found nothing, make sure we don't try again. */
- if (main->alt_dwarf == NULL)
+ if (alt_dwarf_local == NULL)
{
+ rwlock_wrlock(alt_dwarf_lock);
main->alt_dwarf = (void *) -1;
+ rwlock_unlock(alt_dwarf_lock);
+
return NULL;
}
- return main->alt_dwarf;
+ return alt_dwarf_local;
}
INTDEF (dwarf_getalt)
diff --git a/libdw/dwarf_getlocation.c b/libdw/dwarf_getlocation.c
index 553fdc98..e26afe37 100644
--- a/libdw/dwarf_getlocation.c
+++ b/libdw/dwarf_getlocation.c
@@ -31,7 +31,7 @@
#endif
#include <dwarf.h>
-#include <search.h>
+#include <eu-search.h>
#include <stdlib.h>
#include <assert.h>
@@ -137,7 +137,7 @@ loc_compare (const void *p1, const void *p2)
/* For each DW_OP_implicit_value, we store a special entry in the cache.
This points us directly to the block data for later fetching.
- Returns zero on success, -1 on bad DWARF or 1 if tsearch failed. */
+ Returns zero on success, -1 on bad DWARF or 1 if eu_tsearch failed. */
static int
store_implicit_value (Dwarf *dbg, void **cache, Dwarf_Op *op)
{
@@ -154,7 +154,7 @@ store_implicit_value (Dwarf *dbg, void **cache, Dwarf_Op *op)
block->addr = op;
block->data = (unsigned char *) data;
block->length = op->number;
- if (unlikely (tsearch (block, cache, loc_compare) == NULL))
+ if (unlikely (eu_tsearch (block, cache, loc_compare) == NULL))
return 1;
return 0;
}
@@ -167,7 +167,7 @@ dwarf_getlocation_implicit_value (Dwarf_Attribute *attr, const Dwarf_Op *op,
return -1;
struct loc_block_s fake = { .addr = (void *) op };
- struct loc_block_s **found = tfind (&fake, &attr->cu->locs, loc_compare);
+ struct loc_block_s **found = eu_tfind (&fake, &attr->cu->locs, loc_compare);
if (unlikely (found == NULL))
{
__libdw_seterrno (DWARF_E_NO_BLOCK);
@@ -211,7 +211,7 @@ is_constant_offset (Dwarf_Attribute *attr,
/* Check whether we already cached this location. */
struct loc_s fake = { .addr = attr->valp };
- struct loc_s **found = tfind (&fake, &attr->cu->locs, loc_compare);
+ struct loc_s **found = eu_tfind (&fake, &attr->cu->locs, loc_compare);
if (found == NULL)
{
@@ -235,7 +235,7 @@ is_constant_offset (Dwarf_Attribute *attr,
newp->loc = result;
newp->nloc = 1;
- found = tsearch (newp, &attr->cu->locs, loc_compare);
+ found = eu_tsearch (newp, &attr->cu->locs, loc_compare);
}
assert ((*found)->nloc == 1);
@@ -266,7 +266,7 @@ __libdw_intern_expression (Dwarf *dbg, bool other_byte_order,
/* Check whether we already looked at this list. */
struct loc_s fake = { .addr = block->data };
- struct loc_s **found = tfind (&fake, cache, loc_compare);
+ struct loc_s **found = eu_tfind (&fake, cache, loc_compare);
if (found != NULL)
{
/* We already saw it. */
@@ -655,7 +655,7 @@ __libdw_intern_expression (Dwarf *dbg, bool other_byte_order,
newp->addr = block->data;
newp->loc = result;
newp->nloc = *listlen;
- (void) tsearch (newp, cache, loc_compare);
+ (void) eu_tsearch (newp, cache, loc_compare);
/* We did it. */
return 0;
diff --git a/libdw/dwarf_getmacros.c b/libdw/dwarf_getmacros.c
index fd929669..1ac5e402 100644
--- a/libdw/dwarf_getmacros.c
+++ b/libdw/dwarf_getmacros.c
@@ -32,7 +32,7 @@
#include <assert.h>
#include <dwarf.h>
-#include <search.h>
+#include <eu-search.h>
#include <stdlib.h>
#include <string.h>
@@ -275,7 +275,7 @@ cache_op_table (Dwarf *dbg, int sec_index, Dwarf_Off macoff,
Dwarf_Die *cudie)
{
Dwarf_Macro_Op_Table fake = { .offset = macoff, .sec_index = sec_index };
- Dwarf_Macro_Op_Table **found = tfind (&fake, &dbg->macro_ops,
+ Dwarf_Macro_Op_Table **found = eu_tfind (&fake, &dbg->macro_ops,
macro_op_compare);
if (found != NULL)
return *found;
@@ -287,7 +287,7 @@ cache_op_table (Dwarf *dbg, int sec_index, Dwarf_Off macoff,
if (table == NULL)
return NULL;
- Dwarf_Macro_Op_Table **ret = tsearch (table, &dbg->macro_ops,
+ Dwarf_Macro_Op_Table **ret = eu_tsearch (table, &dbg->macro_ops,
macro_op_compare);
if (unlikely (ret == NULL))
{
diff --git a/libdw/dwarf_getsrclines.c b/libdw/dwarf_getsrclines.c
index df003c5f..356284a0 100644
--- a/libdw/dwarf_getsrclines.c
+++ b/libdw/dwarf_getsrclines.c
@@ -33,7 +33,7 @@
#include <assert.h>
#include <stdlib.h>
#include <string.h>
-#include <search.h>
+#include <eu-search.h>
#include "dwarf.h"
#include "libdwP.h"
@@ -1138,7 +1138,7 @@ __libdw_getsrclines (Dwarf *dbg, Dwarf_Off debug_line_offset,
Dwarf_Lines **linesp, Dwarf_Files **filesp)
{
struct files_lines_s fake = { .debug_line_offset = debug_line_offset };
- struct files_lines_s **found = tfind (&fake, &dbg->files_lines,
+ struct files_lines_s **found = eu_tfind (&fake, &dbg->files_lines,
files_lines_compare);
if (found == NULL)
{
@@ -1160,7 +1160,7 @@ __libdw_getsrclines (Dwarf *dbg, Dwarf_Off debug_line_offset,
node->debug_line_offset = debug_line_offset;
- found = tsearch (node, &dbg->files_lines, files_lines_compare);
+ found = eu_tsearch (node, &dbg->files_lines, files_lines_compare);
if (found == NULL)
{
__libdw_seterrno (DWARF_E_NOMEM);
diff --git a/libdw/dwarf_hasattr.c b/libdw/dwarf_hasattr.c
index eca08394..92f8de68 100644
--- a/libdw/dwarf_hasattr.c
+++ b/libdw/dwarf_hasattr.c
@@ -34,6 +34,10 @@
#include <dwarf.h>
#include "libdwP.h"
+/* dwarf_hasattr() calls __libdw_dieabbrev() in libdwP.h.
+ __libdw_dieabbrev() reads/writes "die->abbrev".
+ Mutual exclusion is enforced around the call to __libdw_dieabbrev to prevent a race. */
+rwlock_define(static, die_abbrev_lock);
int
dwarf_hasattr (Dwarf_Die *die, unsigned int search_name)
@@ -41,8 +45,13 @@ dwarf_hasattr (Dwarf_Die *die, unsigned int search_name)
if (die == NULL)
return 0;
+ rwlock_wrlock(die_abbrev_lock);
+
/* Find the abbreviation entry. */
Dwarf_Abbrev *abbrevp = __libdw_dieabbrev (die, NULL);
+
+ rwlock_unlock(die_abbrev_lock);
+
if (unlikely (abbrevp == DWARF_END_ABBREV))
{
__libdw_seterrno (DWARF_E_INVALID_DWARF);
diff --git a/libdw/fde.c b/libdw/fde.c
index 73d551b6..55065528 100644
--- a/libdw/fde.c
+++ b/libdw/fde.c
@@ -31,7 +31,7 @@
#endif
#include "cfi.h"
-#include <search.h>
+#include <eu-search.h>
#include <stdlib.h>
#include "encoded-value.h"
@@ -122,7 +122,7 @@ intern_fde (Dwarf_CFI *cache, const Dwarf_FDE *entry)
fde->instructions += cie->fde_augmentation_data_size;
/* Add the new entry to the search tree. */
- struct dwarf_fde **tres = tsearch (fde, &cache->fde_tree, &compare_fde);
+ struct dwarf_fde **tres = eu_tsearch (fde, &cache->fde_tree, &compare_fde);
if (tres == NULL)
{
free (fde);
@@ -252,7 +252,7 @@ __libdw_find_fde (Dwarf_CFI *cache, Dwarf_Addr address)
/* Look for a cached FDE covering this address. */
const struct dwarf_fde fde_key = { .start = address, .end = 0 };
- struct dwarf_fde **found = tfind (&fde_key, &cache->fde_tree, &compare_fde);
+ struct dwarf_fde **found = eu_tfind (&fde_key, &cache->fde_tree, &compare_fde);
if (found != NULL)
return *found;
diff --git a/libdw/libdw_find_split_unit.c b/libdw/libdw_find_split_unit.c
index a22e7bc9..4da78be4 100644
--- a/libdw/libdw_find_split_unit.c
+++ b/libdw/libdw_find_split_unit.c
@@ -34,13 +34,18 @@
#include "libelfP.h"
#include <limits.h>
-#include <search.h>
+#include <eu-search.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
+/* __libdw_link_skel_split() modifies "skel->split = split" and "split->split = skel".
+ "cu->split" is read at multiple locations and conditionally updated.
+ Mutual exclusion is enforced to prevent a race. */
+rwlock_define(static, cu_split_lock);
+
void
try_split_file (Dwarf_CU *cu, const char *dwo_path)
{
@@ -57,7 +62,7 @@ try_split_file (Dwarf_CU *cu, const char *dwo_path)
if (split->unit_type == DW_UT_split_compile
&& cu->unit_id8 == split->unit_id8)
{
- if (tsearch (split->dbg, &cu->dbg->split_tree,
+ if (eu_tsearch (split->dbg, &cu->dbg->split_tree,
__libdw_finddbg_cb) == NULL)
{
/* Something went wrong. Don't link. */
@@ -65,8 +70,12 @@ try_split_file (Dwarf_CU *cu, const char *dwo_path)
break;
}
- /* Link skeleton and split compile units. */
- __libdw_link_skel_split (cu, split);
+ rwlock_wrlock(cu_split_lock);
+
+ /* Link skeleton and split compile units. */
+ __libdw_link_skel_split (cu, split);
+
+ rwlock_unlock(cu_split_lock);
/* We have everything we need from this ELF
file. And we are going to close the fd to
@@ -75,8 +84,13 @@ try_split_file (Dwarf_CU *cu, const char *dwo_path)
break;
}
}
- if (cu->split == (Dwarf_CU *) -1)
- dwarf_end (split_dwarf);
+
+ rwlock_rdlock(cu_split_lock);
+
+ if (cu->split == (Dwarf_CU *) -1)
+ dwarf_end (split_dwarf);
+
+ rwlock_unlock(cu_split_lock);
}
/* Always close, because we don't want to run out of file
descriptors. See also the elf_fcntl ELF_C_FDDONE call
@@ -89,20 +103,26 @@ Dwarf_CU *
internal_function
__libdw_find_split_unit (Dwarf_CU *cu)
{
- /* Only try once. */
- if (cu->split != (Dwarf_CU *) -1)
- return cu->split;
+ rwlock_rdlock(cu_split_lock);
+ Dwarf_CU *cu_split_local = cu->split;
+ rwlock_unlock(cu_split_lock);
- /* We need a skeleton unit with a comp_dir and [GNU_]dwo_name attributes.
+ /* Only try once. */
+ if (cu_split_local != (Dwarf_CU *) -1)
+ {
+ return cu_split_local;
+ }
+
+ /* We need a skeleton unit with a comp_dir and [GNU_]dwo_name attributes.
The split unit will be the first in the dwo file and should have the
same id as the skeleton. */
- if (cu->unit_type == DW_UT_skeleton)
+ if (cu->unit_type == DW_UT_skeleton)
{
Dwarf_Die cudie = CUDIE (cu);
Dwarf_Attribute dwo_name;
/* It is fine if dwo_dir doesn't exists, but then dwo_name needs
to be an absolute path. */
- if (dwarf_attr (&cudie, DW_AT_dwo_name, &dwo_name) != NULL
+ if (dwarf_attr (&cudie, DW_AT_dwo_name, &dwo_name) != NULL
|| dwarf_attr (&cudie, DW_AT_GNU_dwo_name, &dwo_name) != NULL)
{
/* First try the dwo file name in the same directory
@@ -110,13 +130,17 @@ __libdw_find_split_unit (Dwarf_CU *cu)
const char *dwo_file = dwarf_formstring (&dwo_name);
const char *debugdir = cu->dbg->debugdir;
char *dwo_path = __libdw_filepath (debugdir, NULL, dwo_file);
- if (dwo_path != NULL)
+ if (dwo_path != NULL)
{
try_split_file (cu, dwo_path);
free (dwo_path);
}
- if (cu->split == (Dwarf_CU *) -1)
+ rwlock_rdlock(cu_split_lock);
+ cu_split_local = cu->split;
+ rwlock_unlock(cu_split_lock);
+
+ if (cu_split_local == (Dwarf_CU *) -1)
{
/* Try compdir plus dwo_name. */
Dwarf_Attribute compdir;
@@ -138,9 +162,15 @@ __libdw_find_split_unit (Dwarf_CU *cu)
}
}
- /* If we found nothing, make sure we don't try again. */
- if (cu->split == (Dwarf_CU *) -1)
- cu->split = NULL;
+ rwlock_wrlock(cu_split_lock);
+
+ /* If we found nothing, make sure we don't try again. */
+ if (cu->split == (Dwarf_CU *) -1)
+ {
+ cu->split = NULL;
+ cu_split_local = cu->split;
+ }
- return cu->split;
+ rwlock_unlock(cu_split_lock);
+ return cu_split_local;
}
diff --git a/libdw/libdw_findcu.c b/libdw/libdw_findcu.c
index ed744231..97fcf812 100644
--- a/libdw/libdw_findcu.c
+++ b/libdw/libdw_findcu.c
@@ -32,9 +32,13 @@
#endif
#include <assert.h>
-#include <search.h>
+#include <eu-search.h>
#include "libdwP.h"
+/* __libdw_findcu modifies "&dbg->next_tu_offset : &dbg->next_cu_offset".
+ May read or write, so mutual exclusion is enforced to prevent a race. */
+rwlock_define(static, next_tucu_offset_lock);
+
static int
findcu_cb (const void *arg1, const void *arg2)
{
@@ -213,7 +217,7 @@ __libdw_intern_next_unit (Dwarf *dbg, bool debug_types)
Dwarf_Sig8_Hash_insert (&dbg->sig8_hash, unit_id8, newp);
/* Add the new entry to the search tree. */
- if (tsearch (newp, tree, findcu_cb) == NULL)
+ if (eu_tsearch (newp, tree, findcu_cb) == NULL)
{
/* Something went wrong. Undo the operation. */
*offsetp = oldoff;
@@ -234,28 +238,40 @@ __libdw_findcu (Dwarf *dbg, Dwarf_Off start, bool v4_debug_types)
/* Maybe we already know that CU. */
struct Dwarf_CU fake = { .start = start, .end = 0 };
- struct Dwarf_CU **found = tfind (&fake, tree, findcu_cb);
- if (found != NULL)
+ struct Dwarf_CU **found = eu_tfind (&fake, tree, findcu_cb);
+ struct Dwarf_CU *result = NULL;
+
+ if (found != NULL){
return *found;
+ }
- if (start < *next_offset)
- {
- __libdw_seterrno (DWARF_E_INVALID_DWARF);
- return NULL;
- }
+ rwlock_wrlock(next_tucu_offset_lock);
- /* No. Then read more CUs. */
- while (1)
+ if (start < *next_offset)
+ {
+ __libdw_seterrno (DWARF_E_INVALID_DWARF);
+ }
+ else
+ {
+ /* No. Then read more CUs. */
+ while (1)
{
struct Dwarf_CU *newp = __libdw_intern_next_unit (dbg, v4_debug_types);
- if (newp == NULL)
- return NULL;
+ if (newp == NULL){
+ result = NULL;
+ break;
+ }
/* Is this the one we are looking for? */
- if (start < *next_offset || start == newp->start)
- return newp;
+ if (start < *next_offset || start == newp->start){
+ result = newp;
+ break;
+ }
}
- /* NOTREACHED */
+ }
+
+ rwlock_unlock(next_tucu_offset_lock);
+ return result;
}
struct Dwarf_CU *
@@ -283,7 +299,7 @@ __libdw_findcu_addr (Dwarf *dbg, void *addr)
return NULL;
struct Dwarf_CU fake = { .start = start, .end = 0 };
- struct Dwarf_CU **found = tfind (&fake, tree, findcu_cb);
+ struct Dwarf_CU **found = eu_tfind (&fake, tree, findcu_cb);
if (found != NULL)
return *found;
@@ -298,7 +314,7 @@ __libdw_find_split_dbg_addr (Dwarf *dbg, void *addr)
/* XXX Assumes split DWARF only has CUs in main IDX_debug_info. */
Elf_Data fake_data = { .d_buf = addr, .d_size = 0 };
Dwarf fake = { .sectiondata[IDX_debug_info] = &fake_data };
- Dwarf **found = tfind (&fake, &dbg->split_tree, __libdw_finddbg_cb);
+ Dwarf **found = eu_tfind (&fake, &dbg->split_tree, __libdw_finddbg_cb);
if (found != NULL)
return *found;
diff --git a/libdwfl/ChangeLog b/libdwfl/ChangeLog
index 54d85921..b90920ac 100644
--- a/libdwfl/ChangeLog
+++ b/libdwfl/ChangeLog
@@ -1,3 +1,8 @@
+2023-08-08 Heather McIntyre <hsm2@rice.edu>
+
+ * cu.c: Add eu-search.h and remove search.h.
+ Change call of tsearch to eu_tsearch.
+
2023-04-24 John Gallagher <john@gllghr.com>
* gzip.c: Fix memory leak in unzip()
diff --git a/libdwfl/cu.c b/libdwfl/cu.c
index b1afb19a..71838011 100644
--- a/libdwfl/cu.c
+++ b/libdwfl/cu.c
@@ -33,7 +33,7 @@
#include "libdwflP.h"
#include "libdwP.h"
#include "memory-access.h"
-#include <search.h>
+#include <eu-search.h>
static inline Dwarf_Arange *
@@ -198,7 +198,7 @@ intern_cu (Dwfl_Module *mod, Dwarf_Off cuoff, struct dwfl_cu **result)
struct dwfl_cu key;
key.die.cu = die->cu;
- struct dwfl_cu **found = tsearch (&key, &mod->lazy_cu_root, &compare_cukey);
+ struct dwfl_cu **found = eu_tsearch (&key, &mod->lazy_cu_root, &compare_cukey);
if (unlikely (found == NULL))
return DWFL_E_NOMEM;
diff --git a/libelf/ChangeLog b/libelf/ChangeLog
index 1d5178ca..4a1ece30 100644
--- a/libelf/ChangeLog
+++ b/libelf/ChangeLog
@@ -1,6 +1,15 @@
-2023-04-01 Youling Tang <tangyouling@loongson.cn>
-
- * elf.h: Update from glibc.
+2023-08-08 Heather McIntyre <hsm2@rice.edu>
+
+ * elf32_getchdr.c: Move getchdr function to elf32_getchdr.h
+ * elf32_getchdr.h: New file.
+ Add macro to create getchdr_wrlock.
+ * elf32_updatenull.c: Change call from getchdr to getchdr_wrlock.
+ * elf_getdata.c: Add elf_getdata_wrlock.
+ * libelfP.h: Add internal function declarations.
+ * elf_cntl.c: Adjust locks to prevent deadlock.
+ * elf_end.c: Likewise.
+ * elf_readall.c: Likewise.
+ * elf_version.c: Likewise.
2023-03-03 Mark Wielaard <mark@klomp.org>
diff --git a/libelf/elf32_getchdr.c b/libelf/elf32_getchdr.c
index 982a614c..41591300 100644
--- a/libelf/elf32_getchdr.c
+++ b/libelf/elf32_getchdr.c
@@ -38,46 +38,8 @@
# define LIBELFBITS 32
#endif
+#define ELF_WRLOCK_HELD 1
+#include "elf32_getchdr.h"
-ElfW2(LIBELFBITS,Chdr) *
-elfw2(LIBELFBITS,getchdr) (Elf_Scn *scn)
-{
- ElfW2(LIBELFBITS,Shdr) *shdr = elfw2(LIBELFBITS,getshdr) (scn);
- if (shdr == NULL)
- return NULL;
-
- /* Must have SHF_COMPRESSED flag set. Allocated or no bits sections
- can never be compressed. */
- if ((shdr->sh_flags & SHF_ALLOC) != 0)
- {
- __libelf_seterrno (ELF_E_INVALID_SECTION_FLAGS);
- return NULL;
- }
-
- if (shdr->sh_type == SHT_NULL
- || shdr->sh_type == SHT_NOBITS)
- {
- __libelf_seterrno (ELF_E_INVALID_SECTION_TYPE);
- return NULL;
- }
-
- if ((shdr->sh_flags & SHF_COMPRESSED) == 0)
- {
- __libelf_seterrno (ELF_E_NOT_COMPRESSED);
- return NULL;
- }
-
- /* This makes sure the data is in the correct format, so we don't
- need to swap fields. */
- Elf_Data *d = elf_getdata (scn, NULL);
- if (d == NULL)
- return NULL;
-
- if (d->d_size < sizeof (ElfW2(LIBELFBITS,Chdr)) || d->d_buf == NULL)
- {
- __libelf_seterrno (ELF_E_INVALID_DATA);
- return NULL;
- }
-
- return (ElfW2(LIBELFBITS,Chdr) *) d->d_buf;
-}
+#define ELF_WRLOCK_HELD 0
+#include "elf32_getchdr.h"
\ No newline at end of file
diff --git a/libelf/elf32_getchdr.h b/libelf/elf32_getchdr.h
new file mode 100644
index 00000000..04d47e7a
--- /dev/null
+++ b/libelf/elf32_getchdr.h
@@ -0,0 +1,61 @@
+#undef ADD_ROUTINE_PREFIX
+#undef ADD_ROUTINE_SUFFIX
+
+#if ELF_WRLOCK_HELD
+#define CONCAT(x,y) x##y
+#define ADD_ROUTINE_PREFIX(y) CONCAT(__,y)
+#define ADD_ROUTINE_SUFFIX(x) x ## _wrlock
+#define INTERNAL internal_function
+#else
+#define ADD_ROUTINE_PREFIX(y) y
+#define ADD_ROUTINE_SUFFIX(x) x
+#define INTERNAL
+#endif
+
+ElfW2(LIBELFBITS,Chdr) *
+INTERNAL
+ADD_ROUTINE_PREFIX(elfw2(LIBELFBITS, ADD_ROUTINE_SUFFIX(getchdr))) (Elf_Scn *scn)
+{
+
+ ElfW2(LIBELFBITS,Shdr) *shdr = ADD_ROUTINE_PREFIX(elfw2(LIBELFBITS, ADD_ROUTINE_SUFFIX(getshdr)))(scn);
+
+ if (shdr == NULL)
+ return NULL;
+
+ /* Must have SHF_COMPRESSED flag set. Allocated or no bits sections
+ can never be compressed. */
+ if ((shdr->sh_flags & SHF_ALLOC) != 0)
+ {
+ __libelf_seterrno (ELF_E_INVALID_SECTION_FLAGS);
+ return NULL;
+ }
+
+ if (shdr->sh_type == SHT_NULL
+ || shdr->sh_type == SHT_NOBITS)
+ {
+ __libelf_seterrno (ELF_E_INVALID_SECTION_TYPE);
+ return NULL;
+ }
+
+ if ((shdr->sh_flags & SHF_COMPRESSED) == 0)
+ {
+ __libelf_seterrno (ELF_E_NOT_COMPRESSED);
+ return NULL;
+ }
+
+ /* This makes sure the data is in the correct format, so we don't
+ need to swap fields. */
+ Elf_Data *d = ADD_ROUTINE_PREFIX(ADD_ROUTINE_SUFFIX(elf_getdata)) (scn, NULL);
+ if (d == NULL)
+ return NULL;
+
+ if (d->d_size < sizeof (ElfW2(LIBELFBITS,Chdr)) || d->d_buf == NULL)
+ {
+ __libelf_seterrno (ELF_E_INVALID_DATA);
+ return NULL;
+ }
+
+ return (ElfW2(LIBELFBITS,Chdr) *) d->d_buf;
+}
+#undef INTERNAL
+#undef ELF_WRLOCK_HELD
\ No newline at end of file
diff --git a/libelf/elf32_updatenull.c b/libelf/elf32_updatenull.c
index 6c06e5e4..04bec8d4 100644
--- a/libelf/elf32_updatenull.c
+++ b/libelf/elf32_updatenull.c
@@ -404,7 +404,7 @@ __elfw2(LIBELFBITS,updatenull_wrlock) (Elf *elf, int *change_bop, size_t shnum)
else
{
ElfW2(LIBELFBITS,Chdr) *chdr;
- chdr = elfw2(LIBELFBITS,getchdr) (scn);
+ chdr = __elfw2(LIBELFBITS,getchdr_wrlock) (scn);
if (unlikely (chdr == NULL))
return -1;
sh_size = chdr->ch_size;
diff --git a/libelf/elf_cntl.c b/libelf/elf_cntl.c
index 04aa9132..64087c7d 100644
--- a/libelf/elf_cntl.c
+++ b/libelf/elf_cntl.c
@@ -48,13 +48,16 @@ elf_cntl (Elf *elf, Elf_Cmd cmd)
return -1;
}
- rwlock_wrlock (elf->lock);
+
switch (cmd)
{
case ELF_C_FDREAD:
+ rwlock_rdlock (elf->lock);
+ int addr_isnull = elf->map_address == NULL;
+ rwlock_unlock(elf->lock);
/* If not all of the file is in the memory read it now. */
- if (elf->map_address == NULL && __libelf_readall (elf) == NULL)
+ if (addr_isnull && __libelf_readall (elf) == NULL)
{
/* We were not able to read everything. */
result = -1;
@@ -64,7 +67,9 @@ elf_cntl (Elf *elf, Elf_Cmd cmd)
case ELF_C_FDDONE:
/* Mark the file descriptor as not usable. */
+ rwlock_wrlock (elf->lock);
elf->fildes = -1;
+ rwlock_unlock (elf->lock);
break;
default:
@@ -73,7 +78,5 @@ elf_cntl (Elf *elf, Elf_Cmd cmd)
break;
}
- rwlock_unlock (elf->lock);
-
return result;
}
diff --git a/libelf/elf_end.c b/libelf/elf_end.c
index 3e5d4c86..afe52c19 100644
--- a/libelf/elf_end.c
+++ b/libelf/elf_end.c
@@ -81,8 +81,10 @@ elf_end (Elf *elf)
free (elf->state.ar.ar_sym);
elf->state.ar.ar_sym = NULL;
- if (elf->state.ar.children != NULL)
- return 0;
+ if (elf->state.ar.children != NULL){
+ rwlock_unlock(elf->lock);
+ return 0;
+ }
}
/* Remove this structure from the children list. */
diff --git a/libelf/elf_getdata.c b/libelf/elf_getdata.c
index 5ebd270f..7c3ac043 100644
--- a/libelf/elf_getdata.c
+++ b/libelf/elf_getdata.c
@@ -582,4 +582,18 @@ elf_getdata (Elf_Scn *scn, Elf_Data *data)
return result;
}
+
+Elf_Data *
+internal_function
+__elf_getdata_wrlock (Elf_Scn *scn, Elf_Data *data)
+{
+ Elf_Data *result;
+
+ if (scn == NULL)
+ return NULL;
+
+ result = __elf_getdata_rdlock (scn, data);
+
+ return result;
+}
INTDEF(elf_getdata)
diff --git a/libelf/elf_readall.c b/libelf/elf_readall.c
index d0f9a28c..1af98417 100644
--- a/libelf/elf_readall.c
+++ b/libelf/elf_readall.c
@@ -84,6 +84,7 @@ __libelf_readall (Elf *elf)
/* If this is an archive and we have derived descriptors get the
locks for all of them. */
+ rwlock_unlock(elf->lock); // lock will be reacquired next line
libelf_acquire_all (elf);
if (elf->maximum_size == ~((size_t) 0))
@@ -141,10 +142,8 @@ __libelf_readall (Elf *elf)
__libelf_seterrno (ELF_E_NOMEM);
/* Free the locks on the children. */
- libelf_release_all (elf);
+ libelf_release_all (elf); // lock is released
}
- rwlock_unlock (elf->lock);
-
return (char *) elf->map_address;
}
diff --git a/libelf/elf_version.c b/libelf/elf_version.c
index 6ec534ab..8296bb65 100644
--- a/libelf/elf_version.c
+++ b/libelf/elf_version.c
@@ -32,12 +32,21 @@
#endif
#include <libelfP.h>
+#include <pthread.h>
+/* Multiple threads may initialize __libelf_version.
+ pthread_once() ensures that __libelf_version is initialized only once. */
+once_define(static, version_once);
/* Currently selected version. Should be EV_CURRENT.
Will be EV_NONE if elf_version () has not been called yet. */
unsigned int __libelf_version = EV_NONE;
+static void initialize_version(void)
+{
+ __libelf_version = EV_CURRENT;
+}
+
unsigned int
elf_version (unsigned int version)
{
@@ -49,7 +58,7 @@ elf_version (unsigned int version)
/* Phew, we know this version. */
/* Signal that the version is now initialized. */
- __libelf_version = EV_CURRENT;
+ once(version_once, initialize_version);
/* And return the last (or initial) version. */
return EV_CURRENT;
diff --git a/libelf/libelfP.h b/libelf/libelfP.h
index d3c241e5..5e5df589 100644
--- a/libelf/libelfP.h
+++ b/libelf/libelfP.h
@@ -512,6 +512,8 @@ extern Elf32_Shdr *__elf32_getshdr_rdlock (Elf_Scn *__scn) internal_function;
extern Elf64_Shdr *__elf64_getshdr_rdlock (Elf_Scn *__scn) internal_function;
extern Elf32_Shdr *__elf32_getshdr_wrlock (Elf_Scn *__scn) internal_function;
extern Elf64_Shdr *__elf64_getshdr_wrlock (Elf_Scn *__scn) internal_function;
+extern Elf32_Chdr *__elf32_getchdr_wrlock (Elf_Scn *__scn) internal_function;
+extern Elf64_Chdr *__elf64_getchdr_wrlock (Elf_Scn *__scn) internal_function;
extern Elf_Scn *__elf_getscn_internal (Elf *__elf, size_t __index)
attribute_hidden;
extern Elf_Scn *__elf_nextscn_internal (Elf *__elf, Elf_Scn *__scn)
@@ -521,6 +523,8 @@ extern Elf_Data *__elf_getdata_internal (Elf_Scn *__scn, Elf_Data *__data)
attribute_hidden;
extern Elf_Data *__elf_getdata_rdlock (Elf_Scn *__scn, Elf_Data *__data)
internal_function;
+extern Elf_Data *__elf_getdata_wrlock (Elf_Scn *__scn, Elf_Data *__data)
+ internal_function;
extern Elf_Data *__elf_rawdata_internal (Elf_Scn *__scn, Elf_Data *__data)
attribute_hidden;
/* Should be called to setup first section data element if
diff --git a/src/ChangeLog b/src/ChangeLog
index ae62f4ed..55a2887a 100644
--- a/src/ChangeLog
+++ b/src/ChangeLog
@@ -1,3 +1,10 @@
+2023-08-08 Heather McIntyre <hsm2@rice.edu>
+
+ * Makefile.am: Add USE_LOCKS condition for -pthread.
+ * findtextrel.c: Add eu-search.h and remove search.h.
+ Change calls of tsearch/tfind to eu_tsearch/eu_tfind.
+ * nm.c: Likewise.
+
2023-03-27 Di Chen <dichen@redhat.com>
* readelf.c (options): Support dynamic symtab print with '-Ds'.
diff --git a/src/Makefile.am b/src/Makefile.am
index 10d59a48..fea5d43e 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -22,6 +22,9 @@ DEFS += $(YYDEBUG) -DDEBUGPRED=@DEBUGPRED@ \
AM_CPPFLAGS += -I$(srcdir)/../libelf -I$(srcdir)/../libebl \
-I$(srcdir)/../libdw -I$(srcdir)/../libdwelf \
-I$(srcdir)/../libdwfl -I$(srcdir)/../libasm
+if USE_LOCKS
+ AM_CFLAGS += -pthread
+endif
AM_LDFLAGS = -Wl,-rpath-link,../libelf:../libdw $(STACK_USAGE_NO_ERROR)
diff --git a/src/findtextrel.c b/src/findtextrel.c
index d3021a3a..5ac4c29a 100644
--- a/src/findtextrel.c
+++ b/src/findtextrel.c
@@ -27,7 +27,7 @@
#include <gelf.h>
#include <libdw.h>
#include <locale.h>
-#include <search.h>
+#include <eu-search.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
@@ -501,10 +501,10 @@ check_rel (size_t nsegments, struct segments segments[nsegments],
/* There can be more than one relocation against one file.
Try to avoid multiple messages. And yes, the code uses
pointer comparison. */
- if (tfind (src, knownsrcs, ptrcompare) == NULL)
+ if (eu_tfind (src, knownsrcs, ptrcompare) == NULL)
{
printf (_("%s not compiled with -fpic/-fPIC\n"), src);
- tsearch (src, knownsrcs, ptrcompare);
+ eu_tsearch (src, knownsrcs, ptrcompare);
}
return;
}
@@ -555,12 +555,12 @@ check_rel (size_t nsegments, struct segments segments[nsegments],
if (sym->st_value + sym->st_size > addr)
{
/* It is this function. */
- if (tfind (lowstr, knownsrcs, ptrcompare) == NULL)
+ if (eu_tfind (lowstr, knownsrcs, ptrcompare) == NULL)
{
printf (_("\
the file containing the function '%s' is not compiled with -fpic/-fPIC\n"),
lowstr);
- tsearch (lowstr, knownsrcs, ptrcompare);
+ eu_tsearch (lowstr, knownsrcs, ptrcompare);
}
}
else if (highidx == -1)
diff --git a/src/nm.c b/src/nm.c
index fbdee8e1..44c20fb2 100644
--- a/src/nm.c
+++ b/src/nm.c
@@ -32,7 +32,7 @@
#include <libdw.h>
#include <locale.h>
#include <obstack.h>
-#include <search.h>
+#include <eu-search.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdio_ext.h>
@@ -537,7 +537,7 @@ static int
get_global (Dwarf *dbg __attribute__ ((unused)), Dwarf_Global *global,
void *arg __attribute__ ((unused)))
{
- tsearch (memcpy (xmalloc (sizeof (Dwarf_Global)), global,
+ eu_tsearch (memcpy (xmalloc (sizeof (Dwarf_Global)), global,
sizeof (Dwarf_Global)),
&global_root, global_compare);
@@ -696,7 +696,7 @@ get_local_names (Dwarf *dbg)
/* Check whether a similar local_name is already in the
cache. That should not happen. But if it does, we
don't want to leak memory. */
- struct local_name **tres = tsearch (newp, &local_root,
+ struct local_name **tres = eu_tsearch (newp, &local_root,
local_compare);
if (tres == NULL)
error_exit (errno, _("cannot create search tree"));
@@ -1387,7 +1387,7 @@ show_symbols (int fd, Ebl *ebl, GElf_Ehdr *ehdr,
&& global_root != NULL)
{
Dwarf_Global fake = { .name = symstr };
- Dwarf_Global **found = tfind (&fake, &global_root,
+ Dwarf_Global **found = eu_tfind (&fake, &global_root,
global_compare);
if (found != NULL)
{
@@ -1442,7 +1442,7 @@ show_symbols (int fd, Ebl *ebl, GElf_Ehdr *ehdr,
.lowpc = sym->st_value,
.highpc = sym->st_value,
};
- struct local_name **found = tfind (&fake, &local_root,
+ struct local_name **found = eu_tfind (&fake, &local_root,
local_compare);
if (found != NULL)
{
diff --git a/tests/ChangeLog b/tests/ChangeLog
index d816873c..116876f9 100644
--- a/tests/ChangeLog
+++ b/tests/ChangeLog
@@ -1,3 +1,24 @@
+2023-08-08 Heather McIntyre <hsm2@rice.edu>
+
+ * eu_search_cfi.c: New file.
+ * eu_search_die.c: New file.
+ * eu_search_lines.c: New file.
+ * eu_search_macros.c: New file.
+ * run-eu-search-tests.sh: New test.
+ * Makefile.am: Add USE_LOCKS condition for -pthread.
+ * Makefile.am (check_PROGRAMS): Add eu_search_cfi, eu_search_die,
+ eu_search_lines, and eu_search_macros.
+ (TESTS): Add run-eu-search-tests.sh
+ (eu_search_cfi_LDADD): New variable.
+ (eu_search_die_LDADD): New variable.
+ (eu_search_lines_LDADD): New variable.
+ (eu_search_macros_LDADD): New variable.
+ (eu_search_cfi_LDFLAGS): New variable.
+ Add -pthread if USE_LOCKS is not defined.
+ (eu_search_die_LDFLAGS): Likewise.
+ (eu_search_lines_LDFLAGS): Likewise.
+ (eu_search_macros_LDFLAGS): Likewise.
+
2023-04-21 Frank Ch. Eigler <fche@redhat.com>
* run-debuginfod-IXr.sh: New test.
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 0c77f658..961e282d 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -32,6 +32,10 @@ else
tests_rpath = no
endif
+if USE_LOCKS
+ AM_CFLAGS += -pthread
+endif
+
check_PROGRAMS = arextract arsymtest newfile saridx scnnames sectiondump \
showptable update1 update2 update3 update4 test-nlist \
show-die-info get-files next-files get-lines next-lines \
@@ -63,6 +67,7 @@ check_PROGRAMS = arextract arsymtest newfile saridx scnnames sectiondump \
getphdrnum leb128 read_unaligned \
msg_tst system-elf-libelf-test \
nvidia_extended_linemap_libdw \
+ eu_search_cfi eu_search_die eu_search_lines eu_search_macros \
$(asm_TESTS)
asm_TESTS = asm-tst1 asm-tst2 asm-tst3 asm-tst4 asm-tst5 \
@@ -211,7 +216,8 @@ TESTS = run-arextract.sh run-arsymtest.sh run-ar.sh newfile test-nlist \
$(asm_TESTS) run-disasm-bpf.sh run-low_high_pc-dw-form-indirect.sh \
run-nvidia-extended-linemap-libdw.sh run-nvidia-extended-linemap-readelf.sh \
run-readelf-dw-form-indirect.sh run-strip-largealign.sh \
- run-readelf-Dd.sh
+ run-readelf-Dd.sh \
+ run-eu-search-tests.sh
if !BIARCH
export ELFUTILS_DISABLE_BIARCH = 1
@@ -804,6 +810,14 @@ getphdrnum_LDADD = $(libelf) $(libdw)
leb128_LDADD = $(libelf) $(libdw)
read_unaligned_LDADD = $(libelf) $(libdw)
nvidia_extended_linemap_libdw_LDADD = $(libelf) $(libdw)
+eu_search_cfi_LDFLAGS = $(if $(filter undefined,$(origin USE_LOCKS)),-pthread) $(AM_LDFLAGS)
+eu_search_die_LDFLAGS = $(if $(filter undefined,$(origin USE_LOCKS)),-pthread) $(AM_LDFLAGS)
+eu_search_lines_LDFLAGS = $(if $(filter undefined,$(origin USE_LOCKS)),-pthread) $(AM_LDFLAGS)
+eu_search_macros_LDFLAGS = $(if $(filter undefined,$(origin USE_LOCKS)),-pthread) $(AM_LDFLAGS)
+eu_search_cfi_LDADD = $(libeu) $(libelf) $(libdw)
+eu_search_die_LDADD = $(libdw)
+eu_search_lines_LDADD = $(libdw) $(libelf)
+eu_search_macros_LDADD = $(libdw)
# We want to test the libelf header against the system elf.h header.
# Don't include any -I CPPFLAGS. Except when we install our own elf.h.
diff --git a/tests/eu_search_cfi.c b/tests/eu_search_cfi.c
new file mode 100644
index 00000000..0b63b213
--- /dev/null
+++ b/tests/eu_search_cfi.c
@@ -0,0 +1,234 @@
+/*Test program for eu_search_cfi
+ Copyright (C) 2023 Red Hat, Inc.
+ This file is part of elfutils.
+
+ This file is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 3 of the License, or
+ (at your option) any later version.
+
+ elfutils is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see<http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+#include <assert.h>
+#include <inttypes.h>
+#include ELFUTILS_HEADER(dw)
+#include <dwarf.h>
+#include <argp.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <locale.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+#include "system.h"
+#include <pthread.h>
+
+static void handle_section(char *name, const unsigned char e_ident[], Elf_Scn *scn, const bool is_eh);
+static void *thread_work(void *arg);
+
+typedef struct
+{
+ char *name;
+ const unsigned char *e_ident;
+ Elf_Scn * scn;
+ bool is_eh;
+}
+
+ThreadData;
+
+static void *thread_work(void *arg)
+{
+ ThreadData *data = (ThreadData*) arg;
+ handle_section(data->name, data->e_ident, data->scn, data->is_eh);
+ free(data);
+ return NULL;
+}
+
+static void handle_section(char *name, const unsigned char e_ident[],
+ Elf_Scn *scn, const bool is_eh)
+{
+ if (is_eh)
+ printf(".eh_frame\n");
+ else
+ printf(".debug_frame\n");
+
+ GElf_Shdr mem;
+ GElf_Shdr *shdr = gelf_getshdr(scn, &mem);
+ if (shdr == NULL)
+ error(EXIT_FAILURE, 0, "Couldn't get section header: %s",
+ elf_errmsg(-1));
+ if ((shdr->sh_flags &SHF_COMPRESSED) != 0)
+ {
+ if (elf_compress(scn, 0, 0) < 0)
+ error(EXIT_FAILURE, 0, "Couldn't decompress section: %s",
+ elf_errmsg(-1));
+ }
+ else if (name[0] == '.' && name[1] == 'z')
+ {
+ if (elf_compress_gnu(scn, 0, 0) < 0)
+ error(EXIT_FAILURE, 0, "Couldn't decompress section: %s",
+ elf_errmsg(-1));
+ }
+
+ Elf_Data *data = elf_getdata(scn, NULL);
+ if (data == NULL || data->d_buf == NULL)
+ error(EXIT_FAILURE, 0, "no section data");
+
+ int res;
+ Dwarf_Off off;
+ Dwarf_Off next_off = 0;
+ Dwarf_CFI_Entry entry;
+ while ((res = dwarf_next_cfi(e_ident, data, is_eh, off = next_off, &next_off, &entry)) == 0)
+ {
+ printf("[%" PRId64 "] ", off);
+ if (dwarf_cfi_cie_p(&entry))
+ printf("CIE augmentation=\"%s\"\n", entry.cie.augmentation);
+ else
+ {
+ printf("FDE cie=[%" PRId64 "]\n", entry.fde.CIE_pointer);
+
+ Dwarf_Off cie_off = entry.fde.CIE_pointer;
+ Dwarf_Off cie_off_next;
+ Dwarf_CFI_Entry cie_entry;
+ if (dwarf_next_cfi(e_ident, data, is_eh, cie_off, &cie_off_next, &cie_entry) != 0 ||
+ !dwarf_cfi_cie_p(&cie_entry))
+ error(EXIT_FAILURE, 0, "FDE doesn't point to CIE");
+ }
+ }
+
+ if (res < 0)
+ error(EXIT_FAILURE, 0, "dwarf_next_cfi failed: %s\n",
+ dwarf_errmsg(-1));
+}
+
+int main(int argc, char *argv[])
+{
+ if (argc != 2)
+ error(EXIT_FAILURE, 0, "need file name argument");
+
+ const char *file = argv[1];
+ printf("%s\n", file);
+
+ int fd = open(file, O_RDONLY);
+ if (fd == -1)
+ error(EXIT_FAILURE, errno, "cannot open input file `%s'", file);
+
+ elf_version(EV_CURRENT);
+
+ Elf *elf = elf_begin(fd, ELF_C_READ, NULL);
+ if (elf == NULL)
+ error(EXIT_FAILURE, 0, "cannot create ELF descriptor: %s", elf_errmsg(-1));
+
+ size_t esize;
+ const unsigned char *ident = (const unsigned char *) elf_getident(elf, &esize);
+ if (ident == NULL || esize < EI_NIDENT)
+ error(EXIT_FAILURE, 0, "no, or too small, ELF ident");
+
+ GElf_Ehdr ehdr;
+ if (gelf_getehdr(elf, &ehdr) == NULL)
+ error(EXIT_FAILURE, 0, "cannot get the ELF header: %s\n", elf_errmsg(-1));
+
+ size_t strndx = ehdr.e_shstrndx;
+
+ int num_threads = 4;
+ pthread_t *threads = (pthread_t*) malloc(num_threads * sizeof(pthread_t));
+ ThreadData **thread_data = (ThreadData**) malloc(num_threads * sizeof(ThreadData*));
+ int thread_count = 0;
+
+ if (!threads || !thread_data)
+ {
+ fprintf(stderr, "Failed to allocate memory for threads.\n");
+ free(threads);
+ free(thread_data);
+ return 1;
+ }
+
+ Elf_Scn *scn = NULL;
+ while ((scn = elf_nextscn(elf, scn)) != NULL)
+ {
+ GElf_Shdr shdr;
+ if (gelf_getshdr(scn, &shdr) != NULL)
+ {
+ char *name = elf_strptr(elf, strndx, (size_t) shdr.sh_name);
+ if (name != NULL && shdr.sh_type == SHT_PROGBITS)
+ {
+ bool is_eh = false;
+ if (strcmp(name, ".eh_frame") == 0)
+ {
+ is_eh = true;
+ }
+ else if (strcmp(name, ".debug_frame") == 0 || strcmp(name, ".zdebug_frame") == 0)
+ {
+ is_eh = false;
+ }
+ else
+ {
+ continue;
+ }
+
+ if (thread_count >= num_threads)
+ {
+ num_threads *= 2;
+ threads = realloc(threads, num_threads * sizeof(pthread_t));
+ thread_data = realloc(thread_data, num_threads * sizeof(ThreadData*));
+ }
+
+ thread_data[thread_count] = malloc(sizeof(ThreadData));
+ thread_data[thread_count]->name = name;
+ thread_data[thread_count]->e_ident = ident;
+ thread_data[thread_count]->scn = scn;
+ thread_data[thread_count]->is_eh = is_eh;
+
+ if (pthread_create(&threads[thread_count], NULL, thread_work, thread_data[thread_count]) != 0)
+ {
+ perror("Failed to create thread");
+ for (int j = 0; j < thread_count; j++)
+ {
+ pthread_cancel(threads[j]);
+ }
+ free(threads);
+ free(thread_data);
+ return 1;
+ }
+ else
+ {
+ thread_count++;
+ }
+ }
+ }
+ }
+
+ for (int i = 0; i < thread_count; i++)
+ {
+ if (pthread_join(threads[i], NULL) != 0)
+ {
+ perror("Failed to join thread");
+ free(threads);
+ free(thread_data);
+ return 1;
+ }
+ }
+
+ for (int i = 0; i < thread_count; i++)
+ {
+ free(thread_data[i]);
+ }
+
+ free(threads);
+ free(thread_data);
+
+ elf_end(elf);
+ close(fd);
+
+ return 0;
+}
\ No newline at end of file
diff --git a/tests/eu_search_die.c b/tests/eu_search_die.c
new file mode 100644
index 00000000..a7f75521
--- /dev/null
+++ b/tests/eu_search_die.c
@@ -0,0 +1,262 @@
+/*Test program for eu_search_die.
+ Copyright (C) 2023 Red Hat, Inc.
+ This file is part of elfutils.
+
+ This file is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 3 of the License, or
+ (at your option) any later version.
+
+ elfutils is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see<http://www.gnu.org/licenses/>. */
+#include <config.h>
+#include ELFUTILS_HEADER(dw)
+#include <dwarf.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <stdio.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <assert.h>
+#include <inttypes.h>
+#include <string.h>
+#include <errno.h>
+#include <unistd.h>
+#include <pthread.h>
+
+static void *thread_work(void *arg);
+static int check_die(Dwarf_Die *die);
+static int check_dbg(Dwarf *dbg);
+
+/*The main Dwarf file. */
+static Dwarf * dwarf;
+
+typedef struct
+{
+ Dwarf * dbg;
+ Dwarf_Off start_offset;
+ Dwarf_Off end_offset;
+ int result;
+}
+
+ThreadData;
+
+static void *thread_work(void *arg)
+{
+ ThreadData *data = (ThreadData*) arg;
+ data->result = check_dbg(data->dbg);
+ return NULL;
+}
+
+static int check_die(Dwarf_Die *die)
+{
+ if (dwarf_tag(die) == DW_TAG_invalid)
+ {
+ printf("Invalid die\n");
+ return -1;
+ }
+
+ int res = 0;
+ void *addr = die->addr;
+ Dwarf_Die die2;
+ if (dwarf_die_addr_die(dwarf, addr, &die2) == NULL)
+ {
+ printf("Bad die addr die at offset %" PRIx64 "\n", dwarf_dieoffset(die));
+ res = -1;
+ }
+
+ if (dwarf_tag(die) != dwarf_tag(&die2))
+ {
+ printf("Tags differ for die at offset %" PRIx64 "\n", dwarf_dieoffset(die));
+ res = -1;
+ }
+
+ if (dwarf_cuoffset(die) != dwarf_cuoffset(&die2))
+ {
+ printf("CU offsets differ for die at offset %" PRIx64 "\n", dwarf_dieoffset(die));
+ res = -1;
+ }
+
+ Dwarf_Die child;
+ if (dwarf_child(die, &child) == 0)
+ res |= check_die(&child);
+
+ Dwarf_Die sibling;
+ if (dwarf_siblingof(die, &sibling) == 0)
+ res |= check_die(&sibling);
+
+ return res;
+}
+
+static int check_dbg(Dwarf *dbg)
+{
+ int res = 0;
+ Dwarf_Off off = 0;
+ Dwarf_Off old_off = 0;
+ size_t hsize;
+ Dwarf_Off abbrev;
+ uint8_t addresssize;
+ uint8_t offsetsize;
+
+ while (dwarf_nextcu(dbg, off, &off, &hsize, &abbrev, &addresssize, &offsetsize) == 0)
+ {
+ Dwarf_Die die;
+ if (dwarf_offdie(dbg, old_off + hsize, &die) != NULL)
+ {
+ printf("checking CU at %" PRIx64 "\n", old_off);
+ res |= check_die(&die);
+ }
+
+ old_off = off;
+ }
+
+ // Same for type...
+ Dwarf_Half version;
+ uint64_t typesig;
+ Dwarf_Off typeoff;
+ off = 0;
+ old_off = 0;
+
+ while (dwarf_next_unit(dbg, off, &off, &hsize, &version, &abbrev, &addresssize, &offsetsize, &typesig, &typeoff) == 0)
+ {
+ Dwarf_Die die;
+ if (dwarf_offdie_types(dbg, old_off + hsize, &die) != NULL)
+ {
+ printf("checking TU at %" PRIx64 "\n", old_off);
+ res |= check_die(&die);
+ }
+
+ // We should have seen this already, but double check...
+ if (dwarf_offdie_types(dbg, old_off + typeoff, &die) != NULL)
+ {
+ printf("checking Type DIE at %" PRIx64 "\n", old_off + hsize + typeoff);
+ res |= check_die(&die);
+ }
+
+ old_off = off;
+ }
+
+ Dwarf *alt = dwarf_getalt(dbg);
+
+ if (alt != NULL)
+ {
+ printf("checking alt debug\n");
+ res |= check_dbg(alt);
+ }
+
+ // Split or Type Dwarf_Dies gotten through dwarf_get_units.
+ Dwarf_CU *cu = NULL;
+ Dwarf_Die subdie;
+ uint8_t unit_type;
+ while (dwarf_get_units(dbg, cu, &cu, NULL, &unit_type, NULL, &subdie) == 0)
+ {
+ if (dwarf_tag(&subdie) != DW_TAG_invalid)
+ {
+ printf("checking %"
+ PRIx8 " subdie\n", unit_type);
+ res |= check_die(&subdie);
+ }
+ }
+
+ return res;
+}
+
+int main(int argc, char *argv[])
+{
+ if (argc < 2)
+ {
+ printf("No file given.\n");
+ return -1;
+ }
+
+ const char *name = argv[1];
+ int fd = open(name, O_RDONLY);
+ if (fd < 0)
+ {
+ printf("Cannot open '%s': %s\n", name, strerror(errno));
+ return -1;
+ }
+
+ dwarf = dwarf_begin(fd, DWARF_C_READ);
+ if (dwarf == NULL)
+ {
+ printf("Not a Dwarf file '%s': %s\n", name, dwarf_errmsg(-1));
+ close(fd);
+ return -1;
+ }
+
+ printf("checking %s\n", name);
+
+ int num_threads = 4;
+ pthread_t *threads = (pthread_t*) malloc(num_threads* sizeof(pthread_t));
+ ThreadData *thread_data = (ThreadData*) malloc(num_threads* sizeof(ThreadData));
+
+ if (!threads || !thread_data)
+ {
+ fprintf(stderr, "Failed to allocate memory for threads.\n");
+ free(threads);
+ free(thread_data);
+ return 1;
+ }
+
+ Dwarf_Off total_off = 0;
+ Dwarf_Off unit_off = 0;
+ size_t hsize;
+ Dwarf_Off abbrev;
+ uint8_t addresssize;
+ uint8_t offsetsize;
+
+ while (dwarf_nextcu(dwarf, unit_off, &unit_off, &hsize, &abbrev, &addresssize, &offsetsize) == 0)
+ {
+ thread_data[total_off % num_threads].start_offset = unit_off;
+ thread_data[total_off % num_threads].end_offset = unit_off + hsize;
+ total_off++;
+ }
+
+ for (int i = 0; i < num_threads; i++)
+ {
+ thread_data[i].dbg = dwarf;
+ if (pthread_create(&threads[i], NULL, thread_work, (void*) &thread_data[i]) != 0)
+ {
+ perror("Failed to create thread");
+ for (int j = 0; j < i; j++)
+ {
+ pthread_cancel(threads[j]);
+ }
+ free(threads);
+ free(thread_data);
+ return 1;
+ }
+ }
+
+ for (int i = 0; i < num_threads; i++)
+ {
+ if (pthread_join(threads[i], NULL) != 0)
+ {
+ perror("Failed to join thread");
+ free(threads);
+ free(thread_data);
+ return 1;
+ }
+ }
+
+ int res = 0;
+ for (int i = 0; i < num_threads; i++)
+ {
+ res |= thread_data[i].result;
+ }
+
+ free(threads);
+ free(thread_data);
+
+ dwarf_end(dwarf);
+ close(fd);
+
+ return res;
+}
\ No newline at end of file
diff --git a/tests/eu_search_lines.c b/tests/eu_search_lines.c
new file mode 100644
index 00000000..b7a875d8
--- /dev/null
+++ b/tests/eu_search_lines.c
@@ -0,0 +1,228 @@
+/*Test program for eu_search_lines.
+ Copyright (C) 2023 Red Hat, Inc.
+ This file is part of elfutils.
+
+ This file is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 3 of the License, or
+ (at your option) any later version.
+
+ elfutils is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see<http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+#include <fcntl.h>
+#include <inttypes.h>
+#include <libelf.h>
+#include ELFUTILS_HEADER(dw)
+#include <stdio.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdlib.h>
+#include <pthread.h>
+
+typedef struct
+{
+ const char *filename;
+ int result;
+}
+
+ThreadData;
+
+static void *thread_work(void *arg)
+{
+ ThreadData *data = (ThreadData*) arg;
+
+ int fd = open(data->filename, O_RDONLY);
+
+ Dwarf *dbg = dwarf_begin(fd, DWARF_C_READ);
+ if (dbg == NULL)
+ {
+ printf("%s not usable: %s\n", data->filename, dwarf_errmsg(-1));
+ close(fd);
+ free(data);
+ pthread_exit(NULL);
+ }
+
+ Dwarf_Off cuoff = 0;
+ Dwarf_Off old_cuoff = 0;
+ size_t hsize;
+ Dwarf_Off ao;
+ uint8_t asz;
+ uint8_t osz;
+ while (dwarf_nextcu(dbg, cuoff, &cuoff, &hsize, &ao, &asz, &osz) == 0)
+ {
+ printf("cuhl = %zu, o = %llu, asz = %hhu, osz = %hhu, ncu = %llu\n",
+ hsize, (unsigned long long int) ao,
+ asz, osz, (unsigned long long int) cuoff);
+
+ // Get the DIE for the CU.
+ Dwarf_Die die;
+ if (dwarf_offdie(dbg, old_cuoff + hsize, &die) == NULL)
+ {
+ printf("%s: cannot get CU die\n", data->filename);
+ data->result = 1;
+ break;
+ }
+
+ old_cuoff = cuoff;
+
+ Dwarf_Lines * lb;
+ size_t nlb;
+ if (dwarf_getsrclines(&die, &lb, &nlb) != 0)
+ {
+ printf("%s: cannot get lines\n", data->filename);
+ data->result = 1;
+ break;
+ }
+
+ printf(" %zu lines\n", nlb);
+
+ for (size_t i = 0; i < nlb; ++i)
+ {
+ Dwarf_Line *l = dwarf_onesrcline(lb, i);
+ if (l == NULL)
+ {
+ printf("%s: cannot get individual line\n", data->filename);
+ data->result = 1;
+ break;
+ }
+
+ Dwarf_Addr addr;
+ if (dwarf_lineaddr(l, &addr) != 0)
+ addr = 0;
+ const char *file = dwarf_linesrc(l, NULL, NULL);
+ int line;
+ if (dwarf_lineno(l, &line) != 0)
+ line = 0;
+
+ printf("%" PRIx64 ": %s:%d:", (uint64_t) addr, file ? : "???", line);
+
+ // Getting the file path through the Dwarf_Files should
+ // result in the same path.
+ Dwarf_Files * files;
+ size_t idx;
+ if (dwarf_line_file(l, &files, &idx) != 0)
+ {
+ printf("%s: cannot get file from line (%zd): %s\n",
+ data->filename, i, dwarf_errmsg(-1));
+ data->result = 1;
+ break;
+ }
+
+ const char *path = dwarf_filesrc(files, idx, NULL, NULL);
+ if ((path == NULL && file != NULL) ||
+ (path != NULL && file == NULL) ||
+ (strcmp(file, path) != 0))
+ {
+ printf("%s: line %zd srcline (%s) != file srcline (%s)\n",
+ data->filename, i, file ? : "???", path ? : "???");
+ data->result = 1;
+ break;
+ }
+
+ int column;
+ if (dwarf_linecol(l, &column) != 0)
+ column = 0;
+ if (column >= 0)
+ printf("%d:", column);
+
+ bool is_stmt;
+ if (dwarf_linebeginstatement(l, &is_stmt) != 0)
+ is_stmt = false;
+ bool end_sequence;
+ if (dwarf_lineendsequence(l, &end_sequence) != 0)
+ end_sequence = false;
+ bool basic_block;
+ if (dwarf_lineblock(l, &basic_block) != 0)
+ basic_block = false;
+ bool prologue_end;
+ if (dwarf_lineprologueend(l, &prologue_end) != 0)
+ prologue_end = false;
+ bool epilogue_begin;
+ if (dwarf_lineepiloguebegin(l, &epilogue_begin) != 0)
+ epilogue_begin = false;
+
+ printf(" is_stmt:%s, end_seq:%s, bb:%s, prologue:%s, epilogue:%s\n",
+ is_stmt ? "yes" : "no", end_sequence ? "yes" : "no",
+ basic_block ? "yes" : "no", prologue_end ? "yes" : "no",
+ epilogue_begin ? "yes" : "no");
+ }
+ }
+
+ dwarf_end(dbg);
+ close(fd);
+ free(data);
+
+ pthread_exit(NULL);
+}
+
+int main(int argc, char *argv[])
+{
+ int result = 0;
+ int cnt;
+
+ if (argc < 2)
+ {
+ printf("Usage: %s<filename1>[<filename2> ...]\n", argv[0]);
+ return 1;
+ }
+
+ pthread_t *threads = (pthread_t*) malloc((argc - 1) *sizeof(pthread_t));
+ ThreadData **thread_data = (ThreadData **) malloc((argc - 1) *sizeof(ThreadData*));
+
+ if (!threads || !thread_data)
+ {
+ fprintf(stderr, "Failed to allocate memory for threads.\n");
+ free(threads);
+ free(thread_data);
+ return 1;
+ }
+
+ for (cnt = 1; cnt < argc; ++cnt)
+ {
+ thread_data[cnt - 1] = (ThreadData*) malloc(sizeof(ThreadData));
+ thread_data[cnt - 1]->filename = argv[cnt];
+ thread_data[cnt - 1]->result = 0;
+
+ if (pthread_create(&threads[cnt - 1], NULL, thread_work, thread_data[cnt - 1]) != 0)
+ {
+ perror("Failed to create thread");
+ for (int j = 0; j < cnt; j++)
+ {
+ pthread_cancel(threads[j]);
+ }
+ free(threads);
+ free(thread_data);
+ return 1;
+ }
+ }
+
+ for (cnt = 0; cnt < argc - 1; ++cnt)
+ {
+ if (pthread_join(threads[cnt], NULL) != 0)
+ {
+ perror("Failed to join thread");
+ free(threads);
+ free(thread_data);
+ return 1;
+ }
+
+ if (thread_data[cnt]->result != 0)
+ {
+ result = 1;
+ }
+
+ free(thread_data[cnt]);
+ }
+
+ free(threads);
+ free(thread_data);
+
+ return result;
+}
\ No newline at end of file
diff --git a/tests/eu_search_macros.c b/tests/eu_search_macros.c
new file mode 100644
index 00000000..3dca828e
--- /dev/null
+++ b/tests/eu_search_macros.c
@@ -0,0 +1,216 @@
+/*Test program for eu_search_macros
+ Copyright (C) 2023 Red Hat, Inc.
+ This file is part of elfutils.
+
+ This file is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 3 of the License, or
+ (at your option) any later version.
+
+ elfutils is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see<http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+#include ELFUTILS_HEADER(dw)
+#include <dwarf.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <stdio.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <assert.h>
+#include <inttypes.h>
+#include <stdatomic.h>
+#include <pthread.h>
+
+static void *thread_work(void *arg);
+static int mac(Dwarf_Macro *macro, void *dbg);
+static void include(Dwarf *dbg, Dwarf_Off macoff, ptrdiff_t token);
+
+typedef struct
+{
+ Dwarf * dbg;
+ Dwarf_Die * cudie;
+ bool new_style;
+}
+
+ThreadData;
+
+static void *thread_work(void *arg)
+{
+ ThreadData *data = (ThreadData*) arg;
+ Dwarf *dbg = data->dbg;
+ Dwarf_Die *cudie = data->cudie;
+ bool new_style = data->new_style;
+
+ for (ptrdiff_t off = new_style ? DWARF_GETMACROS_START : 0;
+ (off = dwarf_getmacros(cudie, mac, dbg, off));)
+ {
+ if (off == -1)
+ {
+ puts(dwarf_errmsg(dwarf_errno()));
+ break;
+ }
+ }
+
+ return NULL;
+}
+
+static void include(Dwarf *dbg, Dwarf_Off macoff, ptrdiff_t token)
+{
+ while ((token = dwarf_getmacros_off(dbg, macoff, mac, dbg, token)) != 0)
+ {
+ if (token == -1)
+ {
+ puts(dwarf_errmsg(dwarf_errno()));
+ break;
+ }
+ }
+}
+
+static int
+mac(Dwarf_Macro *macro, void *dbg)
+{
+ static atomic_int level = 0;
+
+ unsigned int opcode;
+ dwarf_macro_opcode(macro, &opcode);
+ switch (opcode)
+ {
+ case DW_MACRO_import:
+ {
+ Dwarf_Attribute at;
+ int r = dwarf_macro_param(macro, 0, &at);
+ assert(r == 0);
+
+ Dwarf_Word w;
+ r = dwarf_formudata(&at, &w);
+ assert(r == 0);
+
+ printf ("%*sinclude %#" PRIx64 "\n", level, "", w);
+
+ atomic_fetch_add(&level, 1);
+
+ include(dbg, w, DWARF_GETMACROS_START);
+
+ atomic_fetch_sub(&level, 1);
+
+ printf ("%*s/include\n", level, "");
+ break;
+ }
+
+ case DW_MACRO_start_file:
+ {
+ Dwarf_Files * files;
+ size_t nfiles;
+ if (dwarf_macro_getsrcfiles(dbg, macro, &files, &nfiles) < 0)
+ printf("dwarf_macro_getsrcfiles: %s\n", dwarf_errmsg(dwarf_errno()));
+
+ Dwarf_Word w = 0;
+ dwarf_macro_param2(macro, &w, NULL);
+
+ const char *name = dwarf_filesrc (files, (size_t) w, NULL, NULL);
+ printf ("%*sfile %s\n", level, "", name);
+ atomic_fetch_add(&level, 1);
+ break;
+ }
+
+ case DW_MACRO_end_file:
+ {
+ atomic_fetch_sub(&level, 1);
+ printf ("%*s/file\n", level, "");
+ break;
+ }
+
+ case DW_MACINFO_define:
+ case DW_MACRO_define_strp:
+ {
+ const char *value;
+ dwarf_macro_param2(macro, NULL, &value);
+ printf ("%*s%s\n", level, "", value);
+ break;
+ }
+
+ case DW_MACINFO_undef:
+ case DW_MACRO_undef_strp:
+ break;
+
+ default:
+ {
+ size_t paramcnt;
+ dwarf_macro_getparamcnt(macro, ¶mcnt);
+ printf ("%*sopcode %u with %zd arguments\n", level, "", opcode, paramcnt);
+ break;
+ }
+ }
+
+ return DWARF_CB_ABORT;
+}
+
+int main(int argc, char *argv[])
+{
+ assert(argc >= 3);
+ const char *name = argv[1];
+ ptrdiff_t cuoff = strtol(argv[2], NULL, 0);
+ bool new_style = argc > 3;
+
+ int fd = open(name, O_RDONLY);
+ Dwarf *dbg = dwarf_begin(fd, DWARF_C_READ);
+
+ Dwarf_Die cudie_mem, *cudie = dwarf_offdie(dbg, cuoff, &cudie_mem);
+
+ int num_threads = 4;
+ pthread_t *threads = malloc(num_threads* sizeof(pthread_t));
+ ThreadData *thread_data = malloc(num_threads* sizeof(ThreadData));
+
+ if (!threads || !thread_data)
+ {
+ fprintf(stderr, "Failed to allocate memory for threads.\n");
+ free(threads);
+ free(thread_data);
+ return 1;
+ }
+
+ for (int i = 0; i < num_threads; i++)
+ {
+ thread_data[i].dbg = dbg;
+ thread_data[i].cudie = cudie;
+ thread_data[i].new_style = new_style;
+
+ if (pthread_create(&threads[i], NULL, thread_work, (void*) &thread_data[i]) != 0)
+ {
+ perror("Failed to create thread");
+ for (int j = 0; j < i; j++)
+ {
+ pthread_cancel(threads[j]);
+ }
+ free(threads);
+ free(thread_data);
+ return 1;
+ }
+ }
+
+ for (int i = 0; i < num_threads; i++)
+ {
+ if (pthread_join(threads[i], NULL) != 0)
+ {
+ perror("Failed to join thread");
+ free(threads);
+ free(thread_data);
+ return 1;
+ }
+ }
+
+ free(threads);
+ free(thread_data);
+
+ dwarf_end(dbg);
+
+ return 0;
+}
\ No newline at end of file
diff --git a/tests/run-eu-search-tests.sh b/tests/run-eu-search-tests.sh
new file mode 100644
index 00000000..84edc234
--- /dev/null
+++ b/tests/run-eu-search-tests.sh
@@ -0,0 +1,168 @@
+#! /bin/sh
+# Copyright (C) 2015, 2018 Red Hat, Inc.
+# This file is part of elfutils.
+#
+# This file is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# elfutils is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+. $srcdir/test-subr.sh
+
+# Extract the value of USE_ADDRESS_SANITIZER_TRUE from config.status
+# Cannot use Helgrind and Address Sanitizer together.
+# Test will be skipped if Address Sanitizer is enabled.
+USE_ADDRESS_SANITIZER=$(grep 'USE_ADDRESS_SANITIZER_TRUE' ${abs_builddir}/../config.status | awk -F'=' '{print $2}')
+
+if [[ "$USE_ADDRESS_SANITIZER" == "\"#\"" ]]; then
+ echo "Address Sanitizer is disabled."
+else
+ echo "Address Sanitizer is enabled. Skipping test."
+ exit 77
+fi
+
+# Extract the value of USE_MEMORY_SANITIZER_TRUE from config.status
+# Cannot use Helgrind and Memory Sanitizer together.
+# Test will be skipped if Memory Sanitizer is enabled.
+USE_MEMORY_SANITIZER=$(grep 'USE_MEMORY_SANITIZER_TRUE' ${abs_builddir}/../config.status | awk -F'=' '{print $2}')
+
+if [[ "$USE_MEMORY_SANITIZER" == "\"#\"" ]]; then
+ echo "Memory Sanitizer is disabled."
+else
+ echo "Memory Sanitizer is enabled. Skipping test."
+ exit 77
+fi
+
+# Extract the value of USE_LOCKS from config.h
+# Test will only be run if USE_LOCKS is defined. Otherwise, skip.
+USE_LOCKS=$(grep '^#define USE_LOCKS' ${abs_builddir}/../config.h | awk '{print $3}')
+
+if [[ "$USE_LOCKS" -eq 1 ]]; then
+ echo "USE_LOCKS is defined."
+else
+ echo "USE_LOCKS is not defined. Skipping test."
+ exit 77
+fi
+
+# Disable valgrind if configured, since we are already using it here.
+SAVED_VALGRIND_CMD="$VALGRIND_CMD"
+unset VALGRIND_CMD
+
+echo "Begin tests..."
+
+# Begin data race test for parallelized dwarf-die-addr-die
+# Tests thread safety for updated libdw_findcu.c and libdw_find_split_unit.c
+testfiles testfile-debug-types
+testfiles testfile_multi_main testfile_multi.dwz
+testfiles testfilebazdbgppc64.debug
+testfiles testfile-dwarf-4 testfile-dwarf-5
+testfiles testfile-splitdwarf-4 testfile-hello4.dwo testfile-world4.dwo
+testfiles testfile-splitdwarf-5 testfile-hello5.dwo testfile-world5.dwo
+
+die_test_files=("testfile-debug-types"
+ "testfile_multi_main" "testfile_multi.dwz"
+ "testfilebazdbgppc64.debug"
+ "testfile-dwarf-4" "testfile-dwarf-5"
+ "testfile-splitdwarf-4" "testfile-hello4.dwo" "testfile-world4.dwo"
+ "testfile-splitdwarf-5" "testfile-hello5.dwo" "testfile-world5.dwo")
+
+echo -e "\nStarting data race test for dwarf-die-addr-die"
+
+for file in "${die_test_files[@]}"; do
+ helgrind_output=$(valgrind --tool=helgrind "${abs_builddir}/eu_search_die" "$file" 2>&1)
+
+ if grep -q "ERROR SUMMARY: 0 errors" <<< "$helgrind_output"; then
+ echo "No data races found for $file. Test passed."
+ else
+ echo "Data races found for $file. Test failed."
+ echo "$helgrind_output"
+ exit 1
+ fi
+done
+
+# Begin data race test for parallelized next_cfi
+# Tests thread safety for updated cie.c and fde.c
+testfiles testfile11 testfile12
+testfiles testfilearm testfileaarch64
+testfiles testfileppc32 testfileppc64
+
+cfi_test_files=("testfile11" "testfile12"
+ "testfilearm" "testfileaarch64"
+ "testfileppc32" "testfileppc64")
+
+echo -e "\nStarting data race test for next_cfi"
+
+for file in "${cfi_test_files[@]}"; do
+
+ helgrind_output=$(valgrind --tool=helgrind "${abs_builddir}/eu_search_cfi" $file 2>&1)
+
+ if grep -q "ERROR SUMMARY: 0 errors" <<< "$helgrind_output"; then
+ echo "No data races found for $file. Test passed."
+ else
+ echo "Data races found for $file. Test failed."
+ echo "$helgrind_output"
+ exit 1
+ fi
+
+done
+
+# Begin data race test for parallelizd dwarf-getmacros
+# Tests thread safety for updated dwarf_getmacros.c
+testfiles testfile51
+testfiles testfile-macros
+testfiles testfile-macros-0xff
+
+macro_test_files=("testfile51 0xb"
+ "testfile51 0x84"
+ "testfile-macrosm 0xb"
+ "testfile-macros-0xff 0xb")
+
+echo -e "\nStarting data race test for dwarf-getmacros"
+
+for file in "${macro_test_files[@]}"; do
+
+ helgrind_output=$(valgrind --tool=helgrind "${abs_builddir}/eu_search_macros" $file 2>&1)
+
+ if grep -q "ERROR SUMMARY: 0 errors" <<< "$helgrind_output"; then
+ echo "No data races found for $file. Test passed."
+ else
+ echo "Data races found for $file. Test failed."
+ echo "$helgrind_output"
+ exit 1
+ fi
+
+done
+
+# Begin data race test for parallelized get-lines
+# Tests thread safety for updated dwarf_getsrclines.c
+testfiles testfile testfile2 testfilenolines
+
+lines_test_files=("testfile" "testfile2" "testfilenolines")
+
+echo -e "\nStarting data race test for get-lines"
+
+for file in "${lines_test_files[@]}"; do
+
+ helgrind_output=$(valgrind --tool=helgrind "${abs_builddir}/eu_search_lines" $file 2>&1)
+
+ if grep -q "ERROR SUMMARY: 0 errors" <<< "$helgrind_output"; then
+ echo "No data races found for $file. Test passed."
+ else
+ echo -e "$helgrind_output"
+ echo "Data races found for $file. Test failed."
+ exit 1
+ fi
+
+done
+
+# This line is reached only if no data races were found in any test
+# Exit with success status.
+exit 0
\ No newline at end of file
--
2.39.3
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH] Fix thread-safety for elfutils
2023-08-08 17:07 [PATCH] Fix thread-safety for elfutils Heather McIntyre
@ 2023-08-21 22:08 ` John Mellor-Crummey
2023-08-25 14:10 ` Mark Wielaard
2023-10-10 13:40 ` Mark Wielaard
1 sibling, 1 reply; 51+ messages in thread
From: John Mellor-Crummey @ 2023-08-21 22:08 UTC (permalink / raw)
To: elfutils-devel; +Cc: John Mellor-Crummey, Heather McIntyre
[-- Attachment #1: Type: text/plain, Size: 1060 bytes --]
Any thoughts about the patch from my student, Heather McIntyre?
--
John Mellor-Crummey Professor
Dept of Computer Science Rice University
email: johnmc@rice.edu phone: 713-348-5179
> On Aug 8, 2023, at 12:07 PM, Heather McIntyre <hsm2@rice.edu> wrote:
>
> Hello all,
>
> This patch was created to address thread-safety issues reported in bug 26921 <https://sourceware.org/bugzilla/show_bug.cgi?id=26921> and bug 26930 <https://sourceware.org/bugzilla/show_bug.cgi?id=26930>.
> Additionally, other thread-safety fixes were applied during the process.
>
> Brief Description:
> Locking was implemented for tsearch and tfind.
> Changes to multiple files within the libelf library were made such that USE_LOCKS no longer causes tests to fail when enabled.
> New tests were added to confirm thread-safety.
>
> Please review the attached patch file and feel free to provide feedback. I am available for any clarifications or modifications needed.
>
> Best regards,
> Heather McIntyre
> <0001-Fix-thread-safety-for-elfutils.patch>
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH] Fix thread-safety for elfutils
2023-08-21 22:08 ` John Mellor-Crummey
@ 2023-08-25 14:10 ` Mark Wielaard
0 siblings, 0 replies; 51+ messages in thread
From: Mark Wielaard @ 2023-08-25 14:10 UTC (permalink / raw)
To: John Mellor-Crummey, elfutils-devel; +Cc: Heather McIntyre
Hi John, Hi Heather,
On Mon, 2023-08-21 at 17:08 -0500, John Mellor-Crummey via Elfutils-
devel wrote:
> Any thoughts about the patch from my student, Heather McIntyre?
Apologies for not responding sooner. The patch was posted when I was on
vacation and I still haven't fully caught up with all work. But I
certainly haven't forgotten. I will try to look at it soon since it
would be exciting to finally be able to enable thread safety.
Cheers,
Mark
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH] Fix thread-safety for elfutils
2023-08-08 17:07 [PATCH] Fix thread-safety for elfutils Heather McIntyre
2023-08-21 22:08 ` John Mellor-Crummey
@ 2023-10-10 13:40 ` Mark Wielaard
2023-10-10 13:42 ` [PATCH 01/16] lib: Add new once_define and once macros to eu-config.h Mark Wielaard
2023-10-14 15:39 ` [PATCH] Fix thread-safety for elfutils Mark Wielaard
1 sibling, 2 replies; 51+ messages in thread
From: Mark Wielaard @ 2023-10-10 13:40 UTC (permalink / raw)
To: Heather McIntyre, elfutils-devel; +Cc: John Mellor-Crummey
Hi Heather,
On Tue, 2023-08-08 at 12:07 -0500, Heather McIntyre via Elfutils-devel
wrote:
> This patch was created to address thread-safety issues reported in bug 26921
> <https://sourceware.org/bugzilla/show_bug.cgi?id=26921> and bug 26930
> <https://sourceware.org/bugzilla/show_bug.cgi?id=26930>.
> Additionally, other thread-safety fixes were applied during the process.
>
> Brief Description:
> Locking was implemented for tsearch and tfind.
> Changes to multiple files within the libelf library were made such that
> USE_LOCKS no longer causes tests to fail when enabled.
> New tests were added to confirm thread-safety.
Very nice. That is a lot of work. And I must admit that I cannot hold
that much work in my little head at the same time. So I have split up
your commit into (what I hope are) logical independent parts. That will
make it easier to review. I might have split it into too many parts,
but that at least makes it easier to just add those parts that are
trivially correct.
The only changes I made were:
1. Move the ChangeLog entries into the commit message
(This is something we do now and makes cherry picking small
changes easier, but I see it isn't actually documented
in CONTRIBUTING, sorry. I'll try to update that.)
2. Fixed up some Copyright notices as discussed off-list.
3. Made some whitespace/indentation changes which made the
diffs slightly smaller (in most cases).
I'll comment/review the individual commits. Which I'll post to the
list.
Heather McIntyre (16):
lib: Add new once_define and once macros to eu-config.h
libelf: Make elf_version thread-safe
libelf: Fix deadlock in __libelf_readall
libelf: Fix deadlock in elf_cntl
libelf: Fix elf_end deadlock
libelf: Make elf32_getchdr and elf64_getchdr thread-safe
lib: Add eu_tsearch and eu_tfind
libcpu: Change calls for tsearch/tfind to eu_tsearch/eu_tfind.
src: Use eu-search in nm and findtextrel.
libdw: make dwarf_getalt thread-safe
libdw: Add locking around __libdw_dieabbrev for dwarf_hasattr
libdw: Make libdw_find_split_unit thread-safe
libdw: Make libdw_findcu thread-safe
libdw,libdwfl: Use eu-search for thread-safety
tests: Add eu-search tests
configure: No longer mark --enable-thread-safety as EXPERIMENTAL
Which can also be found here:
https://code.wildebeest.org/git/user/mjw/elfutils/log/?h=thread-safety
Cheers,
Mark
^ permalink raw reply [flat|nested] 51+ messages in thread
* [PATCH 01/16] lib: Add new once_define and once macros to eu-config.h
2023-10-10 13:40 ` Mark Wielaard
@ 2023-10-10 13:42 ` Mark Wielaard
2023-10-10 13:42 ` [PATCH 02/16] libelf: Make elf_version thread-safe Mark Wielaard
` (15 more replies)
2023-10-14 15:39 ` [PATCH] Fix thread-safety for elfutils Mark Wielaard
1 sibling, 16 replies; 51+ messages in thread
From: Mark Wielaard @ 2023-10-10 13:42 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2, Mark Wielaard
From: Heather McIntyre <hsm2@rice.edu>
* lib/eu-config.h New macros.
[USE_LOCKS] (ONCE_CALL): (once_define, once)
Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
Signed-off-by: Mark Wielaard <mark@klomp.org>
---
lib/eu-config.h | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/lib/eu-config.h b/lib/eu-config.h
index 78a5c4fe..feb079db 100644
--- a/lib/eu-config.h
+++ b/lib/eu-config.h
@@ -33,13 +33,18 @@
# include <pthread.h>
# include <assert.h>
# define rwlock_define(class,name) class pthread_rwlock_t name
+# define once_define(class,name) class pthread_once_t name = PTHREAD_ONCE_INIT
# define RWLOCK_CALL(call) \
({ int _err = pthread_rwlock_ ## call; assert_perror (_err); })
+# define ONCE_CALL(call) \
+ ({ int _err = pthread_ ## call; assert_perror (_err); })
# define rwlock_init(lock) RWLOCK_CALL (init (&lock, NULL))
# define rwlock_fini(lock) RWLOCK_CALL (destroy (&lock))
# define rwlock_rdlock(lock) RWLOCK_CALL (rdlock (&lock))
# define rwlock_wrlock(lock) RWLOCK_CALL (wrlock (&lock))
# define rwlock_unlock(lock) RWLOCK_CALL (unlock (&lock))
+# define once(once_control, init_routine) \
+ ONCE_CALL (once (&once_control, init_routine))
#else
/* Eventually we will allow multi-threaded applications to use the
libraries. Therefore we will add the necessary locking although
@@ -50,6 +55,8 @@
# define rwlock_rdlock(lock) ((void) (lock))
# define rwlock_wrlock(lock) ((void) (lock))
# define rwlock_unlock(lock) ((void) (lock))
+# define once_define(class,name)
+# define once(once_control, init_routine) init_routine()
#endif /* USE_LOCKS */
#include <libintl.h>
--
2.41.0
^ permalink raw reply [flat|nested] 51+ messages in thread
* [PATCH 02/16] libelf: Make elf_version thread-safe
2023-10-10 13:42 ` [PATCH 01/16] lib: Add new once_define and once macros to eu-config.h Mark Wielaard
@ 2023-10-10 13:42 ` Mark Wielaard
2023-10-10 14:00 ` Mark Wielaard
2023-10-10 13:42 ` [PATCH 03/16] libelf: Fix deadlock in __libelf_readall Mark Wielaard
` (14 subsequent siblings)
15 siblings, 1 reply; 51+ messages in thread
From: Mark Wielaard @ 2023-10-10 13:42 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2, Mark Wielaard
From: Heather McIntyre <hsm2@rice.edu>
* elf_version.c (version_once): Define once.
(initialize_version): New static function.
(elf_version): Use initialize_version version_once.
Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
Signed-off-by: Mark Wielaard <mark@klomp.org>
---
libelf/elf_version.c | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/libelf/elf_version.c b/libelf/elf_version.c
index 6ec534ab..8296bb65 100644
--- a/libelf/elf_version.c
+++ b/libelf/elf_version.c
@@ -32,12 +32,21 @@
#endif
#include <libelfP.h>
+#include <pthread.h>
+/* Multiple threads may initialize __libelf_version.
+ pthread_once() ensures that __libelf_version is initialized only once. */
+once_define(static, version_once);
/* Currently selected version. Should be EV_CURRENT.
Will be EV_NONE if elf_version () has not been called yet. */
unsigned int __libelf_version = EV_NONE;
+static void initialize_version(void)
+{
+ __libelf_version = EV_CURRENT;
+}
+
unsigned int
elf_version (unsigned int version)
{
@@ -49,7 +58,7 @@ elf_version (unsigned int version)
/* Phew, we know this version. */
/* Signal that the version is now initialized. */
- __libelf_version = EV_CURRENT;
+ once(version_once, initialize_version);
/* And return the last (or initial) version. */
return EV_CURRENT;
--
2.41.0
^ permalink raw reply [flat|nested] 51+ messages in thread
* [PATCH 03/16] libelf: Fix deadlock in __libelf_readall
2023-10-10 13:42 ` [PATCH 01/16] lib: Add new once_define and once macros to eu-config.h Mark Wielaard
2023-10-10 13:42 ` [PATCH 02/16] libelf: Make elf_version thread-safe Mark Wielaard
@ 2023-10-10 13:42 ` Mark Wielaard
2023-10-10 15:06 ` Mark Wielaard
2023-10-10 13:42 ` [PATCH 04/16] libelf: Fix deadlock in elf_cntl Mark Wielaard
` (13 subsequent siblings)
15 siblings, 1 reply; 51+ messages in thread
From: Mark Wielaard @ 2023-10-10 13:42 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2, Mark Wielaard
From: Heather McIntyre <hsm2@rice.edu>
* libelf/elf_readall.c (__libelf_readall): Move rwlock_unlock
before libelf_acquire_all.
Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
Signed-off-by: Mark Wielaard <mark@klomp.org>
---
libelf/elf_readall.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/libelf/elf_readall.c b/libelf/elf_readall.c
index d0f9a28c..2d62d447 100644
--- a/libelf/elf_readall.c
+++ b/libelf/elf_readall.c
@@ -84,6 +84,7 @@ __libelf_readall (Elf *elf)
/* If this is an archive and we have derived descriptors get the
locks for all of them. */
+ rwlock_unlock(elf->lock); // lock will be reacquired next line
libelf_acquire_all (elf);
if (elf->maximum_size == ~((size_t) 0))
@@ -141,10 +142,8 @@ __libelf_readall (Elf *elf)
__libelf_seterrno (ELF_E_NOMEM);
/* Free the locks on the children. */
- libelf_release_all (elf);
+ libelf_release_all (elf); // lock is released
}
- rwlock_unlock (elf->lock);
-
return (char *) elf->map_address;
}
--
2.41.0
^ permalink raw reply [flat|nested] 51+ messages in thread
* [PATCH 04/16] libelf: Fix deadlock in elf_cntl
2023-10-10 13:42 ` [PATCH 01/16] lib: Add new once_define and once macros to eu-config.h Mark Wielaard
2023-10-10 13:42 ` [PATCH 02/16] libelf: Make elf_version thread-safe Mark Wielaard
2023-10-10 13:42 ` [PATCH 03/16] libelf: Fix deadlock in __libelf_readall Mark Wielaard
@ 2023-10-10 13:42 ` Mark Wielaard
2023-10-10 15:23 ` Mark Wielaard
2023-10-10 13:42 ` [PATCH 05/16] libelf: Fix elf_end deadlock Mark Wielaard
` (12 subsequent siblings)
15 siblings, 1 reply; 51+ messages in thread
From: Mark Wielaard @ 2023-10-10 13:42 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2, Mark Wielaard
From: Heather McIntyre <hsm2@rice.edu>
* libelf/elf_cntl.c (elf_cntl): Move rwlock_wrlock, rwlock_unlock,
inside case switch statements.
Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
Signed-off-by: Mark Wielaard <mark@klomp.org>
---
libelf/elf_cntl.c | 11 +++++++----
1 file changed, 7 insertions(+), 4 deletions(-)
diff --git a/libelf/elf_cntl.c b/libelf/elf_cntl.c
index 04aa9132..64087c7d 100644
--- a/libelf/elf_cntl.c
+++ b/libelf/elf_cntl.c
@@ -48,13 +48,16 @@ elf_cntl (Elf *elf, Elf_Cmd cmd)
return -1;
}
- rwlock_wrlock (elf->lock);
+
switch (cmd)
{
case ELF_C_FDREAD:
+ rwlock_rdlock (elf->lock);
+ int addr_isnull = elf->map_address == NULL;
+ rwlock_unlock(elf->lock);
/* If not all of the file is in the memory read it now. */
- if (elf->map_address == NULL && __libelf_readall (elf) == NULL)
+ if (addr_isnull && __libelf_readall (elf) == NULL)
{
/* We were not able to read everything. */
result = -1;
@@ -64,7 +67,9 @@ elf_cntl (Elf *elf, Elf_Cmd cmd)
case ELF_C_FDDONE:
/* Mark the file descriptor as not usable. */
+ rwlock_wrlock (elf->lock);
elf->fildes = -1;
+ rwlock_unlock (elf->lock);
break;
default:
@@ -73,7 +78,5 @@ elf_cntl (Elf *elf, Elf_Cmd cmd)
break;
}
- rwlock_unlock (elf->lock);
-
return result;
}
--
2.41.0
^ permalink raw reply [flat|nested] 51+ messages in thread
* [PATCH 05/16] libelf: Fix elf_end deadlock
2023-10-10 13:42 ` [PATCH 01/16] lib: Add new once_define and once macros to eu-config.h Mark Wielaard
` (2 preceding siblings ...)
2023-10-10 13:42 ` [PATCH 04/16] libelf: Fix deadlock in elf_cntl Mark Wielaard
@ 2023-10-10 13:42 ` Mark Wielaard
2023-10-10 15:28 ` Mark Wielaard
2023-10-10 13:42 ` [PATCH 06/16] libelf: Make elf32_getchdr and elf64_getchdr thread-safe Mark Wielaard
` (11 subsequent siblings)
15 siblings, 1 reply; 51+ messages in thread
From: Mark Wielaard @ 2023-10-10 13:42 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2, Mark Wielaard
From: Heather McIntyre <hsm2@rice.edu>
* libelf/elf_end.c (elf_end): Add rwlock_unlock before
early return.
Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
Signed-off-by: Mark Wielaard <mark@klomp.org>
---
libelf/elf_end.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/libelf/elf_end.c b/libelf/elf_end.c
index 89727cb3..80f4d13f 100644
--- a/libelf/elf_end.c
+++ b/libelf/elf_end.c
@@ -82,7 +82,10 @@ elf_end (Elf *elf)
elf->state.ar.ar_sym = NULL;
if (elf->state.ar.children != NULL)
- return 0;
+ {
+ rwlock_unlock(elf->lock);
+ return 0;
+ }
}
/* Remove this structure from the children list. */
--
2.41.0
^ permalink raw reply [flat|nested] 51+ messages in thread
* [PATCH 06/16] libelf: Make elf32_getchdr and elf64_getchdr thread-safe
2023-10-10 13:42 ` [PATCH 01/16] lib: Add new once_define and once macros to eu-config.h Mark Wielaard
` (3 preceding siblings ...)
2023-10-10 13:42 ` [PATCH 05/16] libelf: Fix elf_end deadlock Mark Wielaard
@ 2023-10-10 13:42 ` Mark Wielaard
2023-10-10 16:28 ` Mark Wielaard
2023-10-10 13:42 ` [PATCH 07/16] lib: Add eu_tsearch and eu_tfind Mark Wielaard
` (10 subsequent siblings)
15 siblings, 1 reply; 51+ messages in thread
From: Mark Wielaard @ 2023-10-10 13:42 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2, Mark Wielaard
From: Heather McIntyre <hsm2@rice.edu>
* libelf/elf32_getchdr.c: Move getchdr function to
elf32_getchdr.h.
* libelf/elf32_getchdr.h: New file.
Add macro to create getchdr_wrlock.
* libelf/elf32_updatenull.c: Change call from getchdr to
getchdr_wrlock.
* libelf/elf_getdata.c: Add elf_getdata_wrlock.
* libelf/libelfP.h: Add internal function declarations.
Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
Signed-off-by: Mark Wielaard <mark@klomp.org>
---
libelf/elf32_getchdr.c | 46 +++--------------------------
libelf/elf32_getchdr.h | 61 +++++++++++++++++++++++++++++++++++++++
libelf/elf32_updatenull.c | 2 +-
libelf/elf_getdata.c | 14 +++++++++
libelf/libelfP.h | 4 +++
5 files changed, 84 insertions(+), 43 deletions(-)
create mode 100644 libelf/elf32_getchdr.h
diff --git a/libelf/elf32_getchdr.c b/libelf/elf32_getchdr.c
index 982a614c..41591300 100644
--- a/libelf/elf32_getchdr.c
+++ b/libelf/elf32_getchdr.c
@@ -38,46 +38,8 @@
# define LIBELFBITS 32
#endif
+#define ELF_WRLOCK_HELD 1
+#include "elf32_getchdr.h"
-ElfW2(LIBELFBITS,Chdr) *
-elfw2(LIBELFBITS,getchdr) (Elf_Scn *scn)
-{
- ElfW2(LIBELFBITS,Shdr) *shdr = elfw2(LIBELFBITS,getshdr) (scn);
- if (shdr == NULL)
- return NULL;
-
- /* Must have SHF_COMPRESSED flag set. Allocated or no bits sections
- can never be compressed. */
- if ((shdr->sh_flags & SHF_ALLOC) != 0)
- {
- __libelf_seterrno (ELF_E_INVALID_SECTION_FLAGS);
- return NULL;
- }
-
- if (shdr->sh_type == SHT_NULL
- || shdr->sh_type == SHT_NOBITS)
- {
- __libelf_seterrno (ELF_E_INVALID_SECTION_TYPE);
- return NULL;
- }
-
- if ((shdr->sh_flags & SHF_COMPRESSED) == 0)
- {
- __libelf_seterrno (ELF_E_NOT_COMPRESSED);
- return NULL;
- }
-
- /* This makes sure the data is in the correct format, so we don't
- need to swap fields. */
- Elf_Data *d = elf_getdata (scn, NULL);
- if (d == NULL)
- return NULL;
-
- if (d->d_size < sizeof (ElfW2(LIBELFBITS,Chdr)) || d->d_buf == NULL)
- {
- __libelf_seterrno (ELF_E_INVALID_DATA);
- return NULL;
- }
-
- return (ElfW2(LIBELFBITS,Chdr) *) d->d_buf;
-}
+#define ELF_WRLOCK_HELD 0
+#include "elf32_getchdr.h"
\ No newline at end of file
diff --git a/libelf/elf32_getchdr.h b/libelf/elf32_getchdr.h
new file mode 100644
index 00000000..04d47e7a
--- /dev/null
+++ b/libelf/elf32_getchdr.h
@@ -0,0 +1,61 @@
+#undef ADD_ROUTINE_PREFIX
+#undef ADD_ROUTINE_SUFFIX
+
+#if ELF_WRLOCK_HELD
+#define CONCAT(x,y) x##y
+#define ADD_ROUTINE_PREFIX(y) CONCAT(__,y)
+#define ADD_ROUTINE_SUFFIX(x) x ## _wrlock
+#define INTERNAL internal_function
+#else
+#define ADD_ROUTINE_PREFIX(y) y
+#define ADD_ROUTINE_SUFFIX(x) x
+#define INTERNAL
+#endif
+
+ElfW2(LIBELFBITS,Chdr) *
+INTERNAL
+ADD_ROUTINE_PREFIX(elfw2(LIBELFBITS, ADD_ROUTINE_SUFFIX(getchdr))) (Elf_Scn *scn)
+{
+
+ ElfW2(LIBELFBITS,Shdr) *shdr = ADD_ROUTINE_PREFIX(elfw2(LIBELFBITS, ADD_ROUTINE_SUFFIX(getshdr)))(scn);
+
+ if (shdr == NULL)
+ return NULL;
+
+ /* Must have SHF_COMPRESSED flag set. Allocated or no bits sections
+ can never be compressed. */
+ if ((shdr->sh_flags & SHF_ALLOC) != 0)
+ {
+ __libelf_seterrno (ELF_E_INVALID_SECTION_FLAGS);
+ return NULL;
+ }
+
+ if (shdr->sh_type == SHT_NULL
+ || shdr->sh_type == SHT_NOBITS)
+ {
+ __libelf_seterrno (ELF_E_INVALID_SECTION_TYPE);
+ return NULL;
+ }
+
+ if ((shdr->sh_flags & SHF_COMPRESSED) == 0)
+ {
+ __libelf_seterrno (ELF_E_NOT_COMPRESSED);
+ return NULL;
+ }
+
+ /* This makes sure the data is in the correct format, so we don't
+ need to swap fields. */
+ Elf_Data *d = ADD_ROUTINE_PREFIX(ADD_ROUTINE_SUFFIX(elf_getdata)) (scn, NULL);
+ if (d == NULL)
+ return NULL;
+
+ if (d->d_size < sizeof (ElfW2(LIBELFBITS,Chdr)) || d->d_buf == NULL)
+ {
+ __libelf_seterrno (ELF_E_INVALID_DATA);
+ return NULL;
+ }
+
+ return (ElfW2(LIBELFBITS,Chdr) *) d->d_buf;
+}
+#undef INTERNAL
+#undef ELF_WRLOCK_HELD
\ No newline at end of file
diff --git a/libelf/elf32_updatenull.c b/libelf/elf32_updatenull.c
index c5d26b00..3594e8ba 100644
--- a/libelf/elf32_updatenull.c
+++ b/libelf/elf32_updatenull.c
@@ -407,7 +407,7 @@ __elfw2(LIBELFBITS,updatenull_wrlock) (Elf *elf, int *change_bop, size_t shnum)
else
{
ElfW2(LIBELFBITS,Chdr) *chdr;
- chdr = elfw2(LIBELFBITS,getchdr) (scn);
+ chdr = __elfw2(LIBELFBITS,getchdr_wrlock) (scn);
if (unlikely (chdr == NULL))
return -1;
sh_size = chdr->ch_size;
diff --git a/libelf/elf_getdata.c b/libelf/elf_getdata.c
index 5ebd270f..7c3ac043 100644
--- a/libelf/elf_getdata.c
+++ b/libelf/elf_getdata.c
@@ -582,4 +582,18 @@ elf_getdata (Elf_Scn *scn, Elf_Data *data)
return result;
}
+
+Elf_Data *
+internal_function
+__elf_getdata_wrlock (Elf_Scn *scn, Elf_Data *data)
+{
+ Elf_Data *result;
+
+ if (scn == NULL)
+ return NULL;
+
+ result = __elf_getdata_rdlock (scn, data);
+
+ return result;
+}
INTDEF(elf_getdata)
diff --git a/libelf/libelfP.h b/libelf/libelfP.h
index 96476064..ed061abb 100644
--- a/libelf/libelfP.h
+++ b/libelf/libelfP.h
@@ -514,6 +514,8 @@ extern Elf32_Shdr *__elf32_getshdr_rdlock (Elf_Scn *__scn) internal_function;
extern Elf64_Shdr *__elf64_getshdr_rdlock (Elf_Scn *__scn) internal_function;
extern Elf32_Shdr *__elf32_getshdr_wrlock (Elf_Scn *__scn) internal_function;
extern Elf64_Shdr *__elf64_getshdr_wrlock (Elf_Scn *__scn) internal_function;
+extern Elf32_Chdr *__elf32_getchdr_wrlock (Elf_Scn *__scn) internal_function;
+extern Elf64_Chdr *__elf64_getchdr_wrlock (Elf_Scn *__scn) internal_function;
extern Elf_Scn *__elf_getscn_internal (Elf *__elf, size_t __index)
attribute_hidden;
extern Elf_Scn *__elf_nextscn_internal (Elf *__elf, Elf_Scn *__scn)
@@ -523,6 +525,8 @@ extern Elf_Data *__elf_getdata_internal (Elf_Scn *__scn, Elf_Data *__data)
attribute_hidden;
extern Elf_Data *__elf_getdata_rdlock (Elf_Scn *__scn, Elf_Data *__data)
internal_function;
+extern Elf_Data *__elf_getdata_wrlock (Elf_Scn *__scn, Elf_Data *__data)
+ internal_function;
extern Elf_Data *__elf_rawdata_internal (Elf_Scn *__scn, Elf_Data *__data)
attribute_hidden;
/* Should be called to setup first section data element if
--
2.41.0
^ permalink raw reply [flat|nested] 51+ messages in thread
* [PATCH 07/16] lib: Add eu_tsearch and eu_tfind
2023-10-10 13:42 ` [PATCH 01/16] lib: Add new once_define and once macros to eu-config.h Mark Wielaard
` (4 preceding siblings ...)
2023-10-10 13:42 ` [PATCH 06/16] libelf: Make elf32_getchdr and elf64_getchdr thread-safe Mark Wielaard
@ 2023-10-10 13:42 ` Mark Wielaard
2023-10-10 16:51 ` Mark Wielaard
2023-10-10 13:42 ` [PATCH 08/16] libcpu: Change calls for tsearch/tfind to eu_tsearch/eu_tfind Mark Wielaard
` (9 subsequent siblings)
15 siblings, 1 reply; 51+ messages in thread
From: Mark Wielaard @ 2023-10-10 13:42 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2, Mark Wielaard
From: Heather McIntyre <hsm2@rice.edu>
* lib/eu-search.h: New file.
Declarations for read/write locked eu_tsearch/eu_tfind.
* lib/eu-search.c: New file.
Definitions for read/write locked eu_tsearch/eu_tfind.
* Makefile.am (libeu_a_SOURCES): Add eu-search.c.
Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
Signed-off-by: Mark Wielaard <mark@klomp.org>
---
lib/Makefile.am | 2 +-
lib/eu-search.c | 60 +++++++++++++++++++++++++++++++++++++++++++++++++
lib/eu-search.h | 39 ++++++++++++++++++++++++++++++++
3 files changed, 100 insertions(+), 1 deletion(-)
create mode 100644 lib/eu-search.c
create mode 100644 lib/eu-search.h
diff --git a/lib/Makefile.am b/lib/Makefile.am
index b3bb929f..ce8f3e1b 100644
--- a/lib/Makefile.am
+++ b/lib/Makefile.am
@@ -34,7 +34,7 @@ AM_CPPFLAGS += -I$(srcdir)/../libelf
noinst_LIBRARIES = libeu.a
libeu_a_SOURCES = xasprintf.c xstrdup.c xstrndup.c xmalloc.c next_prime.c \
- crc32.c crc32_file.c \
+ crc32.c crc32_file.c eu-search.c \
color.c error.c printversion.c
noinst_HEADERS = fixedsizehash.h libeu.h system.h dynamicsizehash.h list.h \
diff --git a/lib/eu-search.c b/lib/eu-search.c
new file mode 100644
index 00000000..a6b04f4f
--- /dev/null
+++ b/lib/eu-search.c
@@ -0,0 +1,60 @@
+/* Definitions for thread-safe tsearch/tfind
+ Copyright (C) 2023 Rice University
+ This file is part of elfutils.
+
+ This file is free software; you can redistribute it and/or modify
+ it under the terms of either
+
+ * the GNU Lesser General Public License as published by the Free
+ Software Foundation; either version 3 of the License, or (at
+ your option) any later version
+
+ or
+
+ * the GNU General Public License as published by the Free
+ Software Foundation; either version 2 of the License, or (at
+ your option) any later version
+
+ or both in parallel, as here.
+
+ elfutils is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ General Public License for more details.
+
+ You should have received copies of the GNU General Public License and
+ the GNU Lesser General Public License along with this program. If
+ not, see <http://www.gnu.org/licenses/>. */
+
+#ifdef HAVE_CONFIG_H
+#include <config.h>
+#endif
+
+#include <stdlib.h>
+#include "eu-search.h"
+
+rwlock_define(static, search_find_lock);
+
+void *eu_tsearch(const void *key, void **rootp,
+ int (*compar)(const void *, const void *))
+{
+ void *ret = NULL;
+ rwlock_wrlock(search_find_lock);
+
+ ret = tsearch(key, rootp, compar);
+
+ rwlock_unlock(search_find_lock);
+ return ret;
+}
+
+void *eu_tfind(const void *key, void *const *rootp,
+ int (*compar)(const void *, const void *))
+{
+ void *ret = NULL;
+ rwlock_rdlock(search_find_lock);
+
+ ret = tfind(key, rootp, compar);
+
+ rwlock_unlock(search_find_lock);
+ return ret;
+}
diff --git a/lib/eu-search.h b/lib/eu-search.h
new file mode 100644
index 00000000..4ce0139a
--- /dev/null
+++ b/lib/eu-search.h
@@ -0,0 +1,39 @@
+/* Calls for thread-safe tsearch/tfind
+ Copyright (C) 2023 Rice University
+ This file is part of elfutils.
+
+ This file is free software; you can redistribute it and/or modify
+ it under the terms of either
+
+ * the GNU Lesser General Public License as published by the Free
+ Software Foundation; either version 3 of the License, or (at
+ your option) any later version
+
+ or
+
+ * the GNU General Public License as published by the Free
+ Software Foundation; either version 2 of the License, or (at
+ your option) any later version
+
+ or both in parallel, as here.
+
+ elfutils is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ General Public License for more details.
+
+ You should have received copies of the GNU General Public License and
+ the GNU Lesser General Public License along with this program. If
+ not, see <http://www.gnu.org/licenses/>. */
+
+#ifndef EU_SEARCH_H
+#define EU_SEARCH_H 1
+
+#include <search.h>
+
+extern void *eu_tsearch(const void *key, void **rootp,
+ int (*compar)(const void *, const void *));
+extern void *eu_tfind(const void *key, void *const *rootp,
+ int (*compar)(const void *, const void *));
+
+#endif
--
2.41.0
^ permalink raw reply [flat|nested] 51+ messages in thread
* [PATCH 08/16] libcpu: Change calls for tsearch/tfind to eu_tsearch/eu_tfind.
2023-10-10 13:42 ` [PATCH 01/16] lib: Add new once_define and once macros to eu-config.h Mark Wielaard
` (5 preceding siblings ...)
2023-10-10 13:42 ` [PATCH 07/16] lib: Add eu_tsearch and eu_tfind Mark Wielaard
@ 2023-10-10 13:42 ` Mark Wielaard
2023-10-10 21:10 ` Mark Wielaard
2023-10-10 13:42 ` [PATCH 09/16] src: Use eu-search in nm and findtextrel Mark Wielaard
` (8 subsequent siblings)
15 siblings, 1 reply; 51+ messages in thread
From: Mark Wielaard @ 2023-10-10 13:42 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2, Mark Wielaard
From: Heather McIntyre <hsm2@rice.edu>
* libcpu/Makefile.am: Add USE_LOCKS condition for -pthread.
* libcpu/i386_parse.y: Add eu-search.h and remove search.h.
Change calls for tsearch/tfind to eu_tsearch/eu_tfind.
Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
Signed-off-by: Mark Wielaard <mark@klomp.org>
---
libcpu/Makefile.am | 3 +++
libcpu/i386_parse.y | 48 ++++++++++++++++++++++-----------------------
2 files changed, 27 insertions(+), 24 deletions(-)
diff --git a/libcpu/Makefile.am b/libcpu/Makefile.am
index 4ba1be56..a51334b5 100644
--- a/libcpu/Makefile.am
+++ b/libcpu/Makefile.am
@@ -33,6 +33,9 @@ AM_CPPFLAGS += -I$(srcdir)/../libelf -I$(srcdir)/../libebl \
if BUILD_STATIC
AM_CFLAGS += $(fpic_CFLAGS)
endif
+if USE_LOCKS
+ AM_CFLAGS += -pthread
+endif
AM_CFLAGS += -fdollars-in-identifiers
LEXCOMPILE = $(LEX) $(LFLAGS) $(AM_LFLAGS) -P$(<F:lex.l=)
LEX_OUTPUT_ROOT = lex.$(<F:lex.l=)
diff --git a/libcpu/i386_parse.y b/libcpu/i386_parse.y
index 459684c6..3d7cb89e 100644
--- a/libcpu/i386_parse.y
+++ b/libcpu/i386_parse.y
@@ -37,7 +37,7 @@
#include <inttypes.h>
#include <math.h>
#include <obstack.h>
-#include <search.h>
+#include <eu-search.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
@@ -269,12 +269,12 @@ mask: kMASK kBITFIELD kNUMBER
struct synonym *newp = xmalloc (sizeof (*newp));
newp->from = $2;
newp->to = $3;
- if (tfind (newp, &synonyms, compare_syn) != NULL)
+ if (eu_tfind (newp, &synonyms, compare_syn) != NULL)
error (0, 0,
"%d: duplicate definition for synonym '%s'",
i386_lineno, $2);
- else if (tsearch ( newp, &synonyms, compare_syn) == NULL)
- error (EXIT_FAILURE, 0, "tsearch");
+ else if (eu_tsearch ( newp, &synonyms, compare_syn) == NULL)
+ error (EXIT_FAILURE, 0, "eu_tsearch");
}
|
;
@@ -308,12 +308,12 @@ instr: bytes ':' bitfieldopt kID bitfieldopt optargs
newp->bytes = $1;
newp->mnemonic = $4;
if (newp->mnemonic != (void *) -1l
- && tfind ($4, &mnemonics,
+ && eu_tfind ($4, &mnemonics,
(int (*)(const void *, const void *)) strcmp) == NULL)
{
- if (tsearch ($4, &mnemonics,
+ if (eu_tsearch ($4, &mnemonics,
(int (*)(const void *, const void *)) strcmp) == NULL)
- error (EXIT_FAILURE, errno, "tsearch");
+ error (EXIT_FAILURE, errno, "eu_tsearch");
++nmnemonics;
}
@@ -339,15 +339,15 @@ instr: bytes ':' bitfieldopt kID bitfieldopt optargs
infname, i386_lineno - 1, $5->name);
struct suffix search = { .name = $5->name };
- if (tfind (&search, &suffixes, compare_suf)
+ if (eu_tfind (&search, &suffixes, compare_suf)
== NULL)
{
struct suffix *ns = xmalloc (sizeof (*ns));
ns->name = $5->name;
ns->idx = ++nsuffixes;
- if (tsearch (ns, &suffixes, compare_suf)
+ if (eu_tsearch (ns, &suffixes, compare_suf)
== NULL)
- error (EXIT_FAILURE, errno, "tsearch");
+ error (EXIT_FAILURE, errno, "eu_tsearch");
}
}
@@ -374,7 +374,7 @@ bitfieldopt: kBITFIELD
struct known_bitfield search;
search.name = $1;
struct known_bitfield **res;
- res = tfind (&search, &bitfields, bitfield_compare);
+ res = eu_tfind (&search, &bitfields, bitfield_compare);
if (res == NULL)
{
error (0, 0, "%d: unknown bitfield '%s'",
@@ -437,7 +437,7 @@ bit: '0'
struct known_bitfield search;
search.name = $1;
struct known_bitfield **res;
- res = tfind (&search, &bitfields, bitfield_compare);
+ res = eu_tfind (&search, &bitfields, bitfield_compare);
if (res == NULL)
{
error (0, 0, "%d: unknown bitfield '%s'",
@@ -497,7 +497,7 @@ argcomp: kBITFIELD
struct known_bitfield search;
search.name = $1;
struct known_bitfield **res;
- res = tfind (&search, &bitfields, bitfield_compare);
+ res = eu_tfind (&search, &bitfields, bitfield_compare);
if (res == NULL)
{
if (strcmp ($1, "ax") == 0)
@@ -575,7 +575,7 @@ new_bitfield (char *name, unsigned long int num)
newp->bits = num;
newp->tmp = 0;
- if (tfind (newp, &bitfields, bitfield_compare) != NULL)
+ if (eu_tfind (newp, &bitfields, bitfield_compare) != NULL)
{
error (0, 0, "%d: duplicated definition of bitfield '%s'",
i386_lineno, name);
@@ -584,7 +584,7 @@ new_bitfield (char *name, unsigned long int num)
return;
}
- if (tsearch (newp, &bitfields, bitfield_compare) == NULL)
+ if (eu_tsearch (newp, &bitfields, bitfield_compare) == NULL)
error (EXIT_FAILURE, errno, "%d: cannot insert new bitfield '%s'",
i386_lineno, name);
}
@@ -813,7 +813,7 @@ fillin_arg (struct bitvalue *bytes, struct argname *name,
struct synonym search = { .from = fieldname };
- struct synonym **res = tfind (&search, &synonyms, compare_syn);
+ struct synonym **res = eu_tfind (&search, &synonyms, compare_syn);
if (res != NULL)
fieldname = (*res)->to;
@@ -914,26 +914,26 @@ find_numbers (void)
if (runp->operands[i].fct != NULL)
{
struct argstring search = { .str = runp->operands[i].fct };
- if (tfind (&search, &fct_names[i], compare_argstring) == NULL)
+ if (eu_tfind (&search, &fct_names[i], compare_argstring) == NULL)
{
struct argstring *newp = xmalloc (sizeof (*newp));
newp->str = runp->operands[i].fct;
newp->idx = 0;
- if (tsearch (newp, &fct_names[i], compare_argstring) == NULL)
- error (EXIT_FAILURE, errno, "tsearch");
+ if (eu_tsearch (newp, &fct_names[i], compare_argstring) == NULL)
+ error (EXIT_FAILURE, errno, "eu_tsearch");
++nfct_names[i];
}
if (runp->operands[i].str != NULL)
{
search.str = runp->operands[i].str;
- if (tfind (&search, &strs[i], compare_argstring) == NULL)
+ if (eu_tfind (&search, &strs[i], compare_argstring) == NULL)
{
struct argstring *newp = xmalloc (sizeof (*newp));
newp->str = runp->operands[i].str;
newp->idx = 0;
- if (tsearch (newp, &strs[i], compare_argstring) == NULL)
- error (EXIT_FAILURE, errno, "tsearch");
+ if (eu_tsearch (newp, &strs[i], compare_argstring) == NULL)
+ error (EXIT_FAILURE, errno, "eu_tsearch");
++nstrs[i];
}
}
@@ -1206,7 +1206,7 @@ instrtable_out (void)
if (instr->operands[i].fct != NULL)
{
struct argstring search = { .str = instr->operands[i].fct };
- struct argstring **res = tfind (&search, &fct_names[i],
+ struct argstring **res = eu_tfind (&search, &fct_names[i],
compare_argstring);
assert (res != NULL);
idx = (*res)->idx;
@@ -1217,7 +1217,7 @@ instrtable_out (void)
if (instr->operands[i].str != NULL)
{
struct argstring search = { .str = instr->operands[i].str };
- struct argstring **res = tfind (&search, &strs[i],
+ struct argstring **res = eu_tfind (&search, &strs[i],
compare_argstring);
assert (res != NULL);
idx = (*res)->idx;
--
2.41.0
^ permalink raw reply [flat|nested] 51+ messages in thread
* [PATCH 09/16] src: Use eu-search in nm and findtextrel.
2023-10-10 13:42 ` [PATCH 01/16] lib: Add new once_define and once macros to eu-config.h Mark Wielaard
` (6 preceding siblings ...)
2023-10-10 13:42 ` [PATCH 08/16] libcpu: Change calls for tsearch/tfind to eu_tsearch/eu_tfind Mark Wielaard
@ 2023-10-10 13:42 ` Mark Wielaard
2023-10-10 21:25 ` Mark Wielaard
2023-10-10 13:42 ` [PATCH 10/16] libdw: make dwarf_getalt thread-safe Mark Wielaard
` (7 subsequent siblings)
15 siblings, 1 reply; 51+ messages in thread
From: Mark Wielaard @ 2023-10-10 13:42 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2, Mark Wielaard
From: Heather McIntyre <hsm2@rice.edu>
* src/Makefile.am: Add USE_LOCKS condition for -pthread.
* src/findtextrel.c: Add eu-search.h and remove search.h.
Change calls of tsearch/tfind to eu_tsearch/eu_tfind.
* src/nm.c: Likewise.
Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
Signed-off-by: Mark Wielaard <mark@klomp.org>
---
src/Makefile.am | 3 +++
src/findtextrel.c | 10 +++++-----
src/nm.c | 10 +++++-----
3 files changed, 13 insertions(+), 10 deletions(-)
diff --git a/src/Makefile.am b/src/Makefile.am
index 10d59a48..fea5d43e 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -22,6 +22,9 @@ DEFS += $(YYDEBUG) -DDEBUGPRED=@DEBUGPRED@ \
AM_CPPFLAGS += -I$(srcdir)/../libelf -I$(srcdir)/../libebl \
-I$(srcdir)/../libdw -I$(srcdir)/../libdwelf \
-I$(srcdir)/../libdwfl -I$(srcdir)/../libasm
+if USE_LOCKS
+ AM_CFLAGS += -pthread
+endif
AM_LDFLAGS = -Wl,-rpath-link,../libelf:../libdw $(STACK_USAGE_NO_ERROR)
diff --git a/src/findtextrel.c b/src/findtextrel.c
index d3021a3a..5ac4c29a 100644
--- a/src/findtextrel.c
+++ b/src/findtextrel.c
@@ -27,7 +27,7 @@
#include <gelf.h>
#include <libdw.h>
#include <locale.h>
-#include <search.h>
+#include <eu-search.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
@@ -501,10 +501,10 @@ check_rel (size_t nsegments, struct segments segments[nsegments],
/* There can be more than one relocation against one file.
Try to avoid multiple messages. And yes, the code uses
pointer comparison. */
- if (tfind (src, knownsrcs, ptrcompare) == NULL)
+ if (eu_tfind (src, knownsrcs, ptrcompare) == NULL)
{
printf (_("%s not compiled with -fpic/-fPIC\n"), src);
- tsearch (src, knownsrcs, ptrcompare);
+ eu_tsearch (src, knownsrcs, ptrcompare);
}
return;
}
@@ -555,12 +555,12 @@ check_rel (size_t nsegments, struct segments segments[nsegments],
if (sym->st_value + sym->st_size > addr)
{
/* It is this function. */
- if (tfind (lowstr, knownsrcs, ptrcompare) == NULL)
+ if (eu_tfind (lowstr, knownsrcs, ptrcompare) == NULL)
{
printf (_("\
the file containing the function '%s' is not compiled with -fpic/-fPIC\n"),
lowstr);
- tsearch (lowstr, knownsrcs, ptrcompare);
+ eu_tsearch (lowstr, knownsrcs, ptrcompare);
}
}
else if (highidx == -1)
diff --git a/src/nm.c b/src/nm.c
index fbdee8e1..44c20fb2 100644
--- a/src/nm.c
+++ b/src/nm.c
@@ -32,7 +32,7 @@
#include <libdw.h>
#include <locale.h>
#include <obstack.h>
-#include <search.h>
+#include <eu-search.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdio_ext.h>
@@ -537,7 +537,7 @@ static int
get_global (Dwarf *dbg __attribute__ ((unused)), Dwarf_Global *global,
void *arg __attribute__ ((unused)))
{
- tsearch (memcpy (xmalloc (sizeof (Dwarf_Global)), global,
+ eu_tsearch (memcpy (xmalloc (sizeof (Dwarf_Global)), global,
sizeof (Dwarf_Global)),
&global_root, global_compare);
@@ -696,7 +696,7 @@ get_local_names (Dwarf *dbg)
/* Check whether a similar local_name is already in the
cache. That should not happen. But if it does, we
don't want to leak memory. */
- struct local_name **tres = tsearch (newp, &local_root,
+ struct local_name **tres = eu_tsearch (newp, &local_root,
local_compare);
if (tres == NULL)
error_exit (errno, _("cannot create search tree"));
@@ -1387,7 +1387,7 @@ show_symbols (int fd, Ebl *ebl, GElf_Ehdr *ehdr,
&& global_root != NULL)
{
Dwarf_Global fake = { .name = symstr };
- Dwarf_Global **found = tfind (&fake, &global_root,
+ Dwarf_Global **found = eu_tfind (&fake, &global_root,
global_compare);
if (found != NULL)
{
@@ -1442,7 +1442,7 @@ show_symbols (int fd, Ebl *ebl, GElf_Ehdr *ehdr,
.lowpc = sym->st_value,
.highpc = sym->st_value,
};
- struct local_name **found = tfind (&fake, &local_root,
+ struct local_name **found = eu_tfind (&fake, &local_root,
local_compare);
if (found != NULL)
{
--
2.41.0
^ permalink raw reply [flat|nested] 51+ messages in thread
* [PATCH 10/16] libdw: make dwarf_getalt thread-safe
2023-10-10 13:42 ` [PATCH 01/16] lib: Add new once_define and once macros to eu-config.h Mark Wielaard
` (7 preceding siblings ...)
2023-10-10 13:42 ` [PATCH 09/16] src: Use eu-search in nm and findtextrel Mark Wielaard
@ 2023-10-10 13:42 ` Mark Wielaard
2023-10-10 22:02 ` Mark Wielaard
2023-10-10 13:42 ` [PATCH 11/16] libdw: Add locking around __libdw_dieabbrev for dwarf_hasattr Mark Wielaard
` (6 subsequent siblings)
15 siblings, 1 reply; 51+ messages in thread
From: Mark Wielaard @ 2023-10-10 13:42 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2, Mark Wielaard
From: Heather McIntyre <hsm2@rice.edu>
* libdw/dwarf_getalt.c: Add lock for
dbg->alt_dwarf/main->alt_dwarf.
Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
Signed-off-by: Mark Wielaard <mark@klomp.org>
---
libdw/dwarf_getalt.c | 27 ++++++++++++++++++++++-----
1 file changed, 22 insertions(+), 5 deletions(-)
diff --git a/libdw/dwarf_getalt.c b/libdw/dwarf_getalt.c
index 0a12dfae..e3894c8c 100644
--- a/libdw/dwarf_getalt.c
+++ b/libdw/dwarf_getalt.c
@@ -44,6 +44,10 @@
#include <sys/types.h>
#include <sys/stat.h>
+/* find_debug_altlink() modifies "dbg->alt_dwarf".
+ dwarf_getalt() reads "main->alt_dwarf".
+ Mutual exclusion is enforced to prevent a race. */
+rwlock_define(static, alt_dwarf_lock);
char *
internal_function
@@ -152,7 +156,9 @@ find_debug_altlink (Dwarf *dbg)
Dwarf *alt = dwarf_begin (fd, O_RDONLY);
if (alt != NULL)
{
+ rwlock_wrlock(alt_dwarf_lock);
dbg->alt_dwarf = alt;
+ rwlock_unlock(alt_dwarf_lock);
dbg->alt_fd = fd;
}
else
@@ -163,22 +169,33 @@ find_debug_altlink (Dwarf *dbg)
Dwarf *
dwarf_getalt (Dwarf *main)
{
+ rwlock_rdlock(alt_dwarf_lock);
+ Dwarf *alt_dwarf_local = main->alt_dwarf;
+ rwlock_unlock(alt_dwarf_lock);
+
/* Only try once. */
- if (main == NULL || main->alt_dwarf == (void *) -1)
+ if (main == NULL || alt_dwarf_local == (void *) -1)
return NULL;
- if (main->alt_dwarf != NULL)
- return main->alt_dwarf;
+ if (alt_dwarf_local != NULL)
+ return alt_dwarf_local;
find_debug_altlink (main);
+ rwlock_rdlock(alt_dwarf_lock);
+ alt_dwarf_local = main->alt_dwarf;
+ rwlock_unlock(alt_dwarf_lock);
+
/* If we found nothing, make sure we don't try again. */
- if (main->alt_dwarf == NULL)
+ if (alt_dwarf_local == NULL)
{
+ rwlock_wrlock(alt_dwarf_lock);
main->alt_dwarf = (void *) -1;
+ rwlock_unlock(alt_dwarf_lock);
+
return NULL;
}
- return main->alt_dwarf;
+ return alt_dwarf_local;
}
INTDEF (dwarf_getalt)
--
2.41.0
^ permalink raw reply [flat|nested] 51+ messages in thread
* [PATCH 11/16] libdw: Add locking around __libdw_dieabbrev for dwarf_hasattr
2023-10-10 13:42 ` [PATCH 01/16] lib: Add new once_define and once macros to eu-config.h Mark Wielaard
` (8 preceding siblings ...)
2023-10-10 13:42 ` [PATCH 10/16] libdw: make dwarf_getalt thread-safe Mark Wielaard
@ 2023-10-10 13:42 ` Mark Wielaard
2023-10-11 15:10 ` Mark Wielaard
2023-10-17 19:57 ` Heather McIntyre
2023-10-10 13:42 ` [PATCH 12/16] libdw: Make libdw_find_split_unit thread-safe Mark Wielaard
` (5 subsequent siblings)
15 siblings, 2 replies; 51+ messages in thread
From: Mark Wielaard @ 2023-10-10 13:42 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2, Mark Wielaard
From: Heather McIntyre <hsm2@rice.edu>
* libdw/dwarf_hasattr.c (dwarf_hasattr): Use die_abbrev_lock
around __libdw_dieabbrev call.
Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
Signed-off-by: Mark Wielaard <mark@klomp.org>
---
libdw/dwarf_hasattr.c | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/libdw/dwarf_hasattr.c b/libdw/dwarf_hasattr.c
index eca08394..92f8de68 100644
--- a/libdw/dwarf_hasattr.c
+++ b/libdw/dwarf_hasattr.c
@@ -34,6 +34,10 @@
#include <dwarf.h>
#include "libdwP.h"
+/* dwarf_hasattr() calls __libdw_dieabbrev() in libdwP.h.
+ __libdw_dieabbrev() reads/writes "die->abbrev".
+ Mutual exclusion is enforced around the call to __libdw_dieabbrev to prevent a race. */
+rwlock_define(static, die_abbrev_lock);
int
dwarf_hasattr (Dwarf_Die *die, unsigned int search_name)
@@ -41,8 +45,13 @@ dwarf_hasattr (Dwarf_Die *die, unsigned int search_name)
if (die == NULL)
return 0;
+ rwlock_wrlock(die_abbrev_lock);
+
/* Find the abbreviation entry. */
Dwarf_Abbrev *abbrevp = __libdw_dieabbrev (die, NULL);
+
+ rwlock_unlock(die_abbrev_lock);
+
if (unlikely (abbrevp == DWARF_END_ABBREV))
{
__libdw_seterrno (DWARF_E_INVALID_DWARF);
--
2.41.0
^ permalink raw reply [flat|nested] 51+ messages in thread
* [PATCH 12/16] libdw: Make libdw_find_split_unit thread-safe
2023-10-10 13:42 ` [PATCH 01/16] lib: Add new once_define and once macros to eu-config.h Mark Wielaard
` (9 preceding siblings ...)
2023-10-10 13:42 ` [PATCH 11/16] libdw: Add locking around __libdw_dieabbrev for dwarf_hasattr Mark Wielaard
@ 2023-10-10 13:42 ` Mark Wielaard
2023-10-11 17:17 ` Mark Wielaard
2023-10-10 13:42 ` [PATCH 13/16] libdw: Make libdw_findcu thread-safe Mark Wielaard
` (4 subsequent siblings)
15 siblings, 1 reply; 51+ messages in thread
From: Mark Wielaard @ 2023-10-10 13:42 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2, Mark Wielaard
From: Heather McIntyre <hsm2@rice.edu>
* (try_split_file): Use eu_tsearch.
Add lock for cu->split.
Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
Signed-off-by: Mark Wielaard <mark@klomp.org>
---
libdw/libdw_find_split_unit.c | 43 +++++++++++++++++++++++++++++------
1 file changed, 36 insertions(+), 7 deletions(-)
diff --git a/libdw/libdw_find_split_unit.c b/libdw/libdw_find_split_unit.c
index 533f8072..a288e9bd 100644
--- a/libdw/libdw_find_split_unit.c
+++ b/libdw/libdw_find_split_unit.c
@@ -34,13 +34,19 @@
#include "libelfP.h"
#include <limits.h>
-#include <search.h>
+#include <eu-search.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
+/* __libdw_link_skel_split() modifies "skel->split = split"
+ and "split->split = skel".
+ "cu->split" is read at multiple locations and conditionally updated.
+ Mutual exclusion is enforced to prevent a race. */
+rwlock_define(static, cu_split_lock);
+
static void
try_split_file (Dwarf_CU *cu, const char *dwo_path)
{
@@ -57,7 +63,7 @@ try_split_file (Dwarf_CU *cu, const char *dwo_path)
if (split->unit_type == DW_UT_split_compile
&& cu->unit_id8 == split->unit_id8)
{
- if (tsearch (split->dbg, &cu->dbg->split_tree,
+ if (eu_tsearch (split->dbg, &cu->dbg->split_tree,
__libdw_finddbg_cb) == NULL)
{
/* Something went wrong. Don't link. */
@@ -65,9 +71,13 @@ try_split_file (Dwarf_CU *cu, const char *dwo_path)
break;
}
+ rwlock_wrlock(cu_split_lock);
+
/* Link skeleton and split compile units. */
__libdw_link_skel_split (cu, split);
+ rwlock_unlock(cu_split_lock);
+
/* We have everything we need from this ELF
file. And we are going to close the fd to
not run out of file descriptors. */
@@ -75,8 +85,13 @@ try_split_file (Dwarf_CU *cu, const char *dwo_path)
break;
}
}
+
+ rwlock_rdlock(cu_split_lock);
+
if (cu->split == (Dwarf_CU *) -1)
dwarf_end (split_dwarf);
+
+ rwlock_unlock(cu_split_lock);
}
/* Always close, because we don't want to run out of file
descriptors. See also the elf_fcntl ELF_C_FDDONE call
@@ -89,9 +104,13 @@ Dwarf_CU *
internal_function
__libdw_find_split_unit (Dwarf_CU *cu)
{
+ rwlock_rdlock(cu_split_lock);
+ Dwarf_CU *cu_split_local = cu->split;
+ rwlock_unlock(cu_split_lock);
+
/* Only try once. */
- if (cu->split != (Dwarf_CU *) -1)
- return cu->split;
+ if (cu_split_local != (Dwarf_CU *) -1)
+ return cu_split_local;
/* We need a skeleton unit with a comp_dir and [GNU_]dwo_name attributes.
The split unit will be the first in the dwo file and should have the
@@ -116,7 +135,11 @@ __libdw_find_split_unit (Dwarf_CU *cu)
free (dwo_path);
}
- if (cu->split == (Dwarf_CU *) -1)
+ rwlock_rdlock(cu_split_lock);
+ cu_split_local = cu->split;
+ rwlock_unlock(cu_split_lock);
+
+ if (cu_split_local == (Dwarf_CU *) -1)
{
/* Try compdir plus dwo_name. */
Dwarf_Attribute compdir;
@@ -138,9 +161,15 @@ __libdw_find_split_unit (Dwarf_CU *cu)
}
}
+ rwlock_wrlock(cu_split_lock);
+
/* If we found nothing, make sure we don't try again. */
if (cu->split == (Dwarf_CU *) -1)
- cu->split = NULL;
+ {
+ cu->split = NULL;
+ cu_split_local = cu->split;
+ }
- return cu->split;
+ rwlock_unlock(cu_split_lock);
+ return cu_split_local;
}
--
2.41.0
^ permalink raw reply [flat|nested] 51+ messages in thread
* [PATCH 13/16] libdw: Make libdw_findcu thread-safe
2023-10-10 13:42 ` [PATCH 01/16] lib: Add new once_define and once macros to eu-config.h Mark Wielaard
` (10 preceding siblings ...)
2023-10-10 13:42 ` [PATCH 12/16] libdw: Make libdw_find_split_unit thread-safe Mark Wielaard
@ 2023-10-10 13:42 ` Mark Wielaard
2023-10-12 22:02 ` Mark Wielaard
2023-10-10 13:42 ` [PATCH 14/16] libdw,libdwfl: Use eu-search for thread-safety Mark Wielaard
` (3 subsequent siblings)
15 siblings, 1 reply; 51+ messages in thread
From: Mark Wielaard @ 2023-10-10 13:42 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2, Mark Wielaard
From: Heather McIntyre <hsm2@rice.edu>
* libdw/libdw_findcu.c (findcu_cb): Use eu_tsearch.
(__libdw_findcu): Use eu_tfind and next_tucu_offset_lock.
(__libdw_findcu_addr): Use eu_tfind.
(__libdw_find_split_dbg_addr): Likewise.
Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
Signed-off-by: Mark Wielaard <mark@klomp.org>
---
libdw/libdw_findcu.c | 54 ++++++++++++++++++++++++++++----------------
1 file changed, 35 insertions(+), 19 deletions(-)
diff --git a/libdw/libdw_findcu.c b/libdw/libdw_findcu.c
index ed744231..e546fb0f 100644
--- a/libdw/libdw_findcu.c
+++ b/libdw/libdw_findcu.c
@@ -32,9 +32,13 @@
#endif
#include <assert.h>
-#include <search.h>
+#include <eu-search.h>
#include "libdwP.h"
+/* __libdw_findcu modifies "&dbg->next_tu_offset : &dbg->next_cu_offset".
+ May read or write, so mutual exclusion is enforced to prevent a race. */
+rwlock_define(static, next_tucu_offset_lock);
+
static int
findcu_cb (const void *arg1, const void *arg2)
{
@@ -213,7 +217,7 @@ __libdw_intern_next_unit (Dwarf *dbg, bool debug_types)
Dwarf_Sig8_Hash_insert (&dbg->sig8_hash, unit_id8, newp);
/* Add the new entry to the search tree. */
- if (tsearch (newp, tree, findcu_cb) == NULL)
+ if (eu_tsearch (newp, tree, findcu_cb) == NULL)
{
/* Something went wrong. Undo the operation. */
*offsetp = oldoff;
@@ -234,28 +238,40 @@ __libdw_findcu (Dwarf *dbg, Dwarf_Off start, bool v4_debug_types)
/* Maybe we already know that CU. */
struct Dwarf_CU fake = { .start = start, .end = 0 };
- struct Dwarf_CU **found = tfind (&fake, tree, findcu_cb);
+ struct Dwarf_CU **found = eu_tfind (&fake, tree, findcu_cb);
+ struct Dwarf_CU *result = NULL;
+
if (found != NULL)
return *found;
- if (start < *next_offset)
- {
- __libdw_seterrno (DWARF_E_INVALID_DWARF);
- return NULL;
- }
+ rwlock_wrlock(next_tucu_offset_lock);
- /* No. Then read more CUs. */
- while (1)
+ if (start < *next_offset)
+ __libdw_seterrno (DWARF_E_INVALID_DWARF);
+ else
{
- struct Dwarf_CU *newp = __libdw_intern_next_unit (dbg, v4_debug_types);
- if (newp == NULL)
- return NULL;
+ /* No. Then read more CUs. */
+ while (1)
+ {
+ struct Dwarf_CU *newp = __libdw_intern_next_unit (dbg,
+ v4_debug_types);
+ if (newp == NULL)
+ {
+ result = NULL;
+ break;
+ }
- /* Is this the one we are looking for? */
- if (start < *next_offset || start == newp->start)
- return newp;
+ /* Is this the one we are looking for? */
+ if (start < *next_offset || start == newp->start)
+ {
+ result = newp;
+ break;
+ }
+ }
}
- /* NOTREACHED */
+
+ rwlock_unlock(next_tucu_offset_lock);
+ return result;
}
struct Dwarf_CU *
@@ -283,7 +299,7 @@ __libdw_findcu_addr (Dwarf *dbg, void *addr)
return NULL;
struct Dwarf_CU fake = { .start = start, .end = 0 };
- struct Dwarf_CU **found = tfind (&fake, tree, findcu_cb);
+ struct Dwarf_CU **found = eu_tfind (&fake, tree, findcu_cb);
if (found != NULL)
return *found;
@@ -298,7 +314,7 @@ __libdw_find_split_dbg_addr (Dwarf *dbg, void *addr)
/* XXX Assumes split DWARF only has CUs in main IDX_debug_info. */
Elf_Data fake_data = { .d_buf = addr, .d_size = 0 };
Dwarf fake = { .sectiondata[IDX_debug_info] = &fake_data };
- Dwarf **found = tfind (&fake, &dbg->split_tree, __libdw_finddbg_cb);
+ Dwarf **found = eu_tfind (&fake, &dbg->split_tree, __libdw_finddbg_cb);
if (found != NULL)
return *found;
--
2.41.0
^ permalink raw reply [flat|nested] 51+ messages in thread
* [PATCH 14/16] libdw,libdwfl: Use eu-search for thread-safety
2023-10-10 13:42 ` [PATCH 01/16] lib: Add new once_define and once macros to eu-config.h Mark Wielaard
` (11 preceding siblings ...)
2023-10-10 13:42 ` [PATCH 13/16] libdw: Make libdw_findcu thread-safe Mark Wielaard
@ 2023-10-10 13:42 ` Mark Wielaard
2023-10-12 22:05 ` Mark Wielaard
2023-10-10 13:42 ` [PATCH 15/16] tests: Add eu-search tests Mark Wielaard
` (2 subsequent siblings)
15 siblings, 1 reply; 51+ messages in thread
From: Mark Wielaard @ 2023-10-10 13:42 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2, Mark Wielaard
From: Heather McIntyre <hsm2@rice.edu>
* libdw/cie.c: Add eu-search.h and remove search.h.
Change calls of tsearch/tfind to eu_tsearch/eu_tfind.
* libdw/dwarf_getlocation.c: Likewise.
* libdw/dwarf_getmacros.c: Likewise.
* libdw/dwarf_getsrclines.c: Likewise.
* libdw/fde.c: Likewise.
* libdwfl/cu.c: Likewise.
Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
Signed-off-by: Mark Wielaard <mark@klomp.org>
---
libdw/cie.c | 8 ++++----
libdw/dwarf_getlocation.c | 16 ++++++++--------
libdw/dwarf_getmacros.c | 6 +++---
libdw/dwarf_getsrclines.c | 6 +++---
libdw/fde.c | 6 +++---
libdwfl/cu.c | 4 ++--
6 files changed, 23 insertions(+), 23 deletions(-)
diff --git a/libdw/cie.c b/libdw/cie.c
index 1b0aae7c..758daef5 100644
--- a/libdw/cie.c
+++ b/libdw/cie.c
@@ -33,7 +33,7 @@
#include "cfi.h"
#include "encoded-value.h"
#include <assert.h>
-#include <search.h>
+#include <eu-search.h>
#include <stdlib.h>
@@ -144,7 +144,7 @@ intern_new_cie (Dwarf_CFI *cache, Dwarf_Off offset, const Dwarf_CIE *info)
cie->initial_state = NULL;
/* Add the new entry to the search tree. */
- if (tsearch (cie, &cache->cie_tree, &compare_cie) == NULL)
+ if (eu_tsearch (cie, &cache->cie_tree, &compare_cie) == NULL)
{
free (cie);
__libdw_seterrno (DWARF_E_NOMEM);
@@ -160,7 +160,7 @@ internal_function
__libdw_find_cie (Dwarf_CFI *cache, Dwarf_Off offset)
{
const struct dwarf_cie cie_key = { .offset = offset };
- struct dwarf_cie **found = tfind (&cie_key, &cache->cie_tree, &compare_cie);
+ struct dwarf_cie **found = eu_tfind (&cie_key, &cache->cie_tree, &compare_cie);
if (found != NULL)
return *found;
@@ -189,7 +189,7 @@ internal_function
__libdw_intern_cie (Dwarf_CFI *cache, Dwarf_Off offset, const Dwarf_CIE *info)
{
const struct dwarf_cie cie_key = { .offset = offset };
- struct dwarf_cie **found = tfind (&cie_key, &cache->cie_tree, &compare_cie);
+ struct dwarf_cie **found = eu_tfind (&cie_key, &cache->cie_tree, &compare_cie);
if (found == NULL)
/* We have not read this CIE yet. Enter it. */
(void) intern_new_cie (cache, offset, info);
diff --git a/libdw/dwarf_getlocation.c b/libdw/dwarf_getlocation.c
index 553fdc98..e26afe37 100644
--- a/libdw/dwarf_getlocation.c
+++ b/libdw/dwarf_getlocation.c
@@ -31,7 +31,7 @@
#endif
#include <dwarf.h>
-#include <search.h>
+#include <eu-search.h>
#include <stdlib.h>
#include <assert.h>
@@ -137,7 +137,7 @@ loc_compare (const void *p1, const void *p2)
/* For each DW_OP_implicit_value, we store a special entry in the cache.
This points us directly to the block data for later fetching.
- Returns zero on success, -1 on bad DWARF or 1 if tsearch failed. */
+ Returns zero on success, -1 on bad DWARF or 1 if eu_tsearch failed. */
static int
store_implicit_value (Dwarf *dbg, void **cache, Dwarf_Op *op)
{
@@ -154,7 +154,7 @@ store_implicit_value (Dwarf *dbg, void **cache, Dwarf_Op *op)
block->addr = op;
block->data = (unsigned char *) data;
block->length = op->number;
- if (unlikely (tsearch (block, cache, loc_compare) == NULL))
+ if (unlikely (eu_tsearch (block, cache, loc_compare) == NULL))
return 1;
return 0;
}
@@ -167,7 +167,7 @@ dwarf_getlocation_implicit_value (Dwarf_Attribute *attr, const Dwarf_Op *op,
return -1;
struct loc_block_s fake = { .addr = (void *) op };
- struct loc_block_s **found = tfind (&fake, &attr->cu->locs, loc_compare);
+ struct loc_block_s **found = eu_tfind (&fake, &attr->cu->locs, loc_compare);
if (unlikely (found == NULL))
{
__libdw_seterrno (DWARF_E_NO_BLOCK);
@@ -211,7 +211,7 @@ is_constant_offset (Dwarf_Attribute *attr,
/* Check whether we already cached this location. */
struct loc_s fake = { .addr = attr->valp };
- struct loc_s **found = tfind (&fake, &attr->cu->locs, loc_compare);
+ struct loc_s **found = eu_tfind (&fake, &attr->cu->locs, loc_compare);
if (found == NULL)
{
@@ -235,7 +235,7 @@ is_constant_offset (Dwarf_Attribute *attr,
newp->loc = result;
newp->nloc = 1;
- found = tsearch (newp, &attr->cu->locs, loc_compare);
+ found = eu_tsearch (newp, &attr->cu->locs, loc_compare);
}
assert ((*found)->nloc == 1);
@@ -266,7 +266,7 @@ __libdw_intern_expression (Dwarf *dbg, bool other_byte_order,
/* Check whether we already looked at this list. */
struct loc_s fake = { .addr = block->data };
- struct loc_s **found = tfind (&fake, cache, loc_compare);
+ struct loc_s **found = eu_tfind (&fake, cache, loc_compare);
if (found != NULL)
{
/* We already saw it. */
@@ -655,7 +655,7 @@ __libdw_intern_expression (Dwarf *dbg, bool other_byte_order,
newp->addr = block->data;
newp->loc = result;
newp->nloc = *listlen;
- (void) tsearch (newp, cache, loc_compare);
+ (void) eu_tsearch (newp, cache, loc_compare);
/* We did it. */
return 0;
diff --git a/libdw/dwarf_getmacros.c b/libdw/dwarf_getmacros.c
index a3a78884..d150f05e 100644
--- a/libdw/dwarf_getmacros.c
+++ b/libdw/dwarf_getmacros.c
@@ -32,7 +32,7 @@
#include <assert.h>
#include <dwarf.h>
-#include <search.h>
+#include <eu-search.h>
#include <stdlib.h>
#include <string.h>
@@ -293,7 +293,7 @@ cache_op_table (Dwarf *dbg, int sec_index, Dwarf_Off macoff,
Dwarf_Die *cudie)
{
Dwarf_Macro_Op_Table fake = { .offset = macoff, .sec_index = sec_index };
- Dwarf_Macro_Op_Table **found = tfind (&fake, &dbg->macro_ops,
+ Dwarf_Macro_Op_Table **found = eu_tfind (&fake, &dbg->macro_ops,
macro_op_compare);
if (found != NULL)
return *found;
@@ -305,7 +305,7 @@ cache_op_table (Dwarf *dbg, int sec_index, Dwarf_Off macoff,
if (table == NULL)
return NULL;
- Dwarf_Macro_Op_Table **ret = tsearch (table, &dbg->macro_ops,
+ Dwarf_Macro_Op_Table **ret = eu_tsearch (table, &dbg->macro_ops,
macro_op_compare);
if (unlikely (ret == NULL))
{
diff --git a/libdw/dwarf_getsrclines.c b/libdw/dwarf_getsrclines.c
index df003c5f..356284a0 100644
--- a/libdw/dwarf_getsrclines.c
+++ b/libdw/dwarf_getsrclines.c
@@ -33,7 +33,7 @@
#include <assert.h>
#include <stdlib.h>
#include <string.h>
-#include <search.h>
+#include <eu-search.h>
#include "dwarf.h"
#include "libdwP.h"
@@ -1138,7 +1138,7 @@ __libdw_getsrclines (Dwarf *dbg, Dwarf_Off debug_line_offset,
Dwarf_Lines **linesp, Dwarf_Files **filesp)
{
struct files_lines_s fake = { .debug_line_offset = debug_line_offset };
- struct files_lines_s **found = tfind (&fake, &dbg->files_lines,
+ struct files_lines_s **found = eu_tfind (&fake, &dbg->files_lines,
files_lines_compare);
if (found == NULL)
{
@@ -1160,7 +1160,7 @@ __libdw_getsrclines (Dwarf *dbg, Dwarf_Off debug_line_offset,
node->debug_line_offset = debug_line_offset;
- found = tsearch (node, &dbg->files_lines, files_lines_compare);
+ found = eu_tsearch (node, &dbg->files_lines, files_lines_compare);
if (found == NULL)
{
__libdw_seterrno (DWARF_E_NOMEM);
diff --git a/libdw/fde.c b/libdw/fde.c
index 73d551b6..55065528 100644
--- a/libdw/fde.c
+++ b/libdw/fde.c
@@ -31,7 +31,7 @@
#endif
#include "cfi.h"
-#include <search.h>
+#include <eu-search.h>
#include <stdlib.h>
#include "encoded-value.h"
@@ -122,7 +122,7 @@ intern_fde (Dwarf_CFI *cache, const Dwarf_FDE *entry)
fde->instructions += cie->fde_augmentation_data_size;
/* Add the new entry to the search tree. */
- struct dwarf_fde **tres = tsearch (fde, &cache->fde_tree, &compare_fde);
+ struct dwarf_fde **tres = eu_tsearch (fde, &cache->fde_tree, &compare_fde);
if (tres == NULL)
{
free (fde);
@@ -252,7 +252,7 @@ __libdw_find_fde (Dwarf_CFI *cache, Dwarf_Addr address)
/* Look for a cached FDE covering this address. */
const struct dwarf_fde fde_key = { .start = address, .end = 0 };
- struct dwarf_fde **found = tfind (&fde_key, &cache->fde_tree, &compare_fde);
+ struct dwarf_fde **found = eu_tfind (&fde_key, &cache->fde_tree, &compare_fde);
if (found != NULL)
return *found;
diff --git a/libdwfl/cu.c b/libdwfl/cu.c
index b1afb19a..71838011 100644
--- a/libdwfl/cu.c
+++ b/libdwfl/cu.c
@@ -33,7 +33,7 @@
#include "libdwflP.h"
#include "libdwP.h"
#include "memory-access.h"
-#include <search.h>
+#include <eu-search.h>
static inline Dwarf_Arange *
@@ -198,7 +198,7 @@ intern_cu (Dwfl_Module *mod, Dwarf_Off cuoff, struct dwfl_cu **result)
struct dwfl_cu key;
key.die.cu = die->cu;
- struct dwfl_cu **found = tsearch (&key, &mod->lazy_cu_root, &compare_cukey);
+ struct dwfl_cu **found = eu_tsearch (&key, &mod->lazy_cu_root, &compare_cukey);
if (unlikely (found == NULL))
return DWFL_E_NOMEM;
--
2.41.0
^ permalink raw reply [flat|nested] 51+ messages in thread
* [PATCH 15/16] tests: Add eu-search tests
2023-10-10 13:42 ` [PATCH 01/16] lib: Add new once_define and once macros to eu-config.h Mark Wielaard
` (12 preceding siblings ...)
2023-10-10 13:42 ` [PATCH 14/16] libdw,libdwfl: Use eu-search for thread-safety Mark Wielaard
@ 2023-10-10 13:42 ` Mark Wielaard
2023-10-13 14:38 ` Mark Wielaard
2023-10-10 13:43 ` [PATCH 16/16] configure: No longer mark --enable-thread-safety as EXPERIMENTAL Mark Wielaard
2023-10-10 13:54 ` [PATCH 01/16] lib: Add new once_define and once macros to eu-config.h Mark Wielaard
15 siblings, 1 reply; 51+ messages in thread
From: Mark Wielaard @ 2023-10-10 13:42 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2, Mark Wielaard
From: Heather McIntyre <hsm2@rice.edu>
* tests/eu_search_cfi.c: New file.
* tests/eu_search_die.c: New file.
* tests/eu_search_lines.c: New file.
* tests/eu_search_macros.c: New file.
* tests/run-eu-search-tests.sh: New test.
* tests/Makefile.am: Add USE_LOCKS condition for -pthread.
(check_PROGRAMS): Add eu_search_cfi, eu_search_die,
eu_search_lines, and eu_search_macros.
(TESTS): Add run-eu-search-tests.sh
(eu_search_cfi_LDADD): New variable.
(eu_search_die_LDADD): New variable.
(eu_search_lines_LDADD): New variable.
(eu_search_macros_LDADD): New variable.
(eu_search_cfi_LDFLAGS): New variable.
Add -pthread if USE_LOCKS is not defined.
(eu_search_die_LDFLAGS): Likewise.
(eu_search_lines_LDFLAGS): Likewise.
(eu_search_macros_LDFLAGS): Likewise.
Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
Signed-off-by: Mark Wielaard <mark@klomp.org>
---
tests/Makefile.am | 16 ++-
tests/eu_search_cfi.c | 234 +++++++++++++++++++++++++++++++
tests/eu_search_die.c | 262 +++++++++++++++++++++++++++++++++++
tests/eu_search_lines.c | 228 ++++++++++++++++++++++++++++++
tests/eu_search_macros.c | 216 +++++++++++++++++++++++++++++
tests/run-eu-search-tests.sh | 168 ++++++++++++++++++++++
6 files changed, 1123 insertions(+), 1 deletion(-)
create mode 100644 tests/eu_search_cfi.c
create mode 100644 tests/eu_search_die.c
create mode 100644 tests/eu_search_lines.c
create mode 100644 tests/eu_search_macros.c
create mode 100755 tests/run-eu-search-tests.sh
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 32b18e6e..a66b1033 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -32,6 +32,10 @@ else
tests_rpath = no
endif
+if USE_LOCKS
+ AM_CFLAGS += -pthread
+endif
+
check_PROGRAMS = arextract arsymtest newfile saridx scnnames sectiondump \
showptable update1 update2 update3 update4 test-nlist \
show-die-info get-files next-files get-lines next-lines \
@@ -63,6 +67,7 @@ check_PROGRAMS = arextract arsymtest newfile saridx scnnames sectiondump \
getphdrnum leb128 read_unaligned \
msg_tst system-elf-libelf-test system-elf-gelf-test \
nvidia_extended_linemap_libdw \
+ eu_search_cfi eu_search_die eu_search_lines eu_search_macros \
$(asm_TESTS)
asm_TESTS = asm-tst1 asm-tst2 asm-tst3 asm-tst4 asm-tst5 \
@@ -211,7 +216,8 @@ TESTS = run-arextract.sh run-arsymtest.sh run-ar.sh newfile test-nlist \
$(asm_TESTS) run-disasm-bpf.sh run-low_high_pc-dw-form-indirect.sh \
run-nvidia-extended-linemap-libdw.sh run-nvidia-extended-linemap-readelf.sh \
run-readelf-dw-form-indirect.sh run-strip-largealign.sh \
- run-readelf-Dd.sh
+ run-readelf-Dd.sh \
+ run-eu-search-tests.sh
if !BIARCH
export ELFUTILS_DISABLE_BIARCH = 1
@@ -804,6 +810,14 @@ getphdrnum_LDADD = $(libelf) $(libdw)
leb128_LDADD = $(libelf) $(libdw)
read_unaligned_LDADD = $(libelf) $(libdw)
nvidia_extended_linemap_libdw_LDADD = $(libelf) $(libdw)
+eu_search_cfi_LDFLAGS = $(if $(filter undefined,$(origin USE_LOCKS)),-pthread) $(AM_LDFLAGS)
+eu_search_die_LDFLAGS = $(if $(filter undefined,$(origin USE_LOCKS)),-pthread) $(AM_LDFLAGS)
+eu_search_lines_LDFLAGS = $(if $(filter undefined,$(origin USE_LOCKS)),-pthread) $(AM_LDFLAGS)
+eu_search_macros_LDFLAGS = $(if $(filter undefined,$(origin USE_LOCKS)),-pthread) $(AM_LDFLAGS)
+eu_search_cfi_LDADD = $(libeu) $(libelf) $(libdw)
+eu_search_die_LDADD = $(libdw)
+eu_search_lines_LDADD = $(libdw) $(libelf)
+eu_search_macros_LDADD = $(libdw)
# We want to test the libelf headers against the system elf.h header.
# Don't include any -I CPPFLAGS. Except when we install our own elf.h.
diff --git a/tests/eu_search_cfi.c b/tests/eu_search_cfi.c
new file mode 100644
index 00000000..0b63b213
--- /dev/null
+++ b/tests/eu_search_cfi.c
@@ -0,0 +1,234 @@
+/*Test program for eu_search_cfi
+ Copyright (C) 2023 Red Hat, Inc.
+ This file is part of elfutils.
+
+ This file is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 3 of the License, or
+ (at your option) any later version.
+
+ elfutils is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see<http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+#include <assert.h>
+#include <inttypes.h>
+#include ELFUTILS_HEADER(dw)
+#include <dwarf.h>
+#include <argp.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <locale.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+#include "system.h"
+#include <pthread.h>
+
+static void handle_section(char *name, const unsigned char e_ident[], Elf_Scn *scn, const bool is_eh);
+static void *thread_work(void *arg);
+
+typedef struct
+{
+ char *name;
+ const unsigned char *e_ident;
+ Elf_Scn * scn;
+ bool is_eh;
+}
+
+ThreadData;
+
+static void *thread_work(void *arg)
+{
+ ThreadData *data = (ThreadData*) arg;
+ handle_section(data->name, data->e_ident, data->scn, data->is_eh);
+ free(data);
+ return NULL;
+}
+
+static void handle_section(char *name, const unsigned char e_ident[],
+ Elf_Scn *scn, const bool is_eh)
+{
+ if (is_eh)
+ printf(".eh_frame\n");
+ else
+ printf(".debug_frame\n");
+
+ GElf_Shdr mem;
+ GElf_Shdr *shdr = gelf_getshdr(scn, &mem);
+ if (shdr == NULL)
+ error(EXIT_FAILURE, 0, "Couldn't get section header: %s",
+ elf_errmsg(-1));
+ if ((shdr->sh_flags &SHF_COMPRESSED) != 0)
+ {
+ if (elf_compress(scn, 0, 0) < 0)
+ error(EXIT_FAILURE, 0, "Couldn't decompress section: %s",
+ elf_errmsg(-1));
+ }
+ else if (name[0] == '.' && name[1] == 'z')
+ {
+ if (elf_compress_gnu(scn, 0, 0) < 0)
+ error(EXIT_FAILURE, 0, "Couldn't decompress section: %s",
+ elf_errmsg(-1));
+ }
+
+ Elf_Data *data = elf_getdata(scn, NULL);
+ if (data == NULL || data->d_buf == NULL)
+ error(EXIT_FAILURE, 0, "no section data");
+
+ int res;
+ Dwarf_Off off;
+ Dwarf_Off next_off = 0;
+ Dwarf_CFI_Entry entry;
+ while ((res = dwarf_next_cfi(e_ident, data, is_eh, off = next_off, &next_off, &entry)) == 0)
+ {
+ printf("[%" PRId64 "] ", off);
+ if (dwarf_cfi_cie_p(&entry))
+ printf("CIE augmentation=\"%s\"\n", entry.cie.augmentation);
+ else
+ {
+ printf("FDE cie=[%" PRId64 "]\n", entry.fde.CIE_pointer);
+
+ Dwarf_Off cie_off = entry.fde.CIE_pointer;
+ Dwarf_Off cie_off_next;
+ Dwarf_CFI_Entry cie_entry;
+ if (dwarf_next_cfi(e_ident, data, is_eh, cie_off, &cie_off_next, &cie_entry) != 0 ||
+ !dwarf_cfi_cie_p(&cie_entry))
+ error(EXIT_FAILURE, 0, "FDE doesn't point to CIE");
+ }
+ }
+
+ if (res < 0)
+ error(EXIT_FAILURE, 0, "dwarf_next_cfi failed: %s\n",
+ dwarf_errmsg(-1));
+}
+
+int main(int argc, char *argv[])
+{
+ if (argc != 2)
+ error(EXIT_FAILURE, 0, "need file name argument");
+
+ const char *file = argv[1];
+ printf("%s\n", file);
+
+ int fd = open(file, O_RDONLY);
+ if (fd == -1)
+ error(EXIT_FAILURE, errno, "cannot open input file `%s'", file);
+
+ elf_version(EV_CURRENT);
+
+ Elf *elf = elf_begin(fd, ELF_C_READ, NULL);
+ if (elf == NULL)
+ error(EXIT_FAILURE, 0, "cannot create ELF descriptor: %s", elf_errmsg(-1));
+
+ size_t esize;
+ const unsigned char *ident = (const unsigned char *) elf_getident(elf, &esize);
+ if (ident == NULL || esize < EI_NIDENT)
+ error(EXIT_FAILURE, 0, "no, or too small, ELF ident");
+
+ GElf_Ehdr ehdr;
+ if (gelf_getehdr(elf, &ehdr) == NULL)
+ error(EXIT_FAILURE, 0, "cannot get the ELF header: %s\n", elf_errmsg(-1));
+
+ size_t strndx = ehdr.e_shstrndx;
+
+ int num_threads = 4;
+ pthread_t *threads = (pthread_t*) malloc(num_threads * sizeof(pthread_t));
+ ThreadData **thread_data = (ThreadData**) malloc(num_threads * sizeof(ThreadData*));
+ int thread_count = 0;
+
+ if (!threads || !thread_data)
+ {
+ fprintf(stderr, "Failed to allocate memory for threads.\n");
+ free(threads);
+ free(thread_data);
+ return 1;
+ }
+
+ Elf_Scn *scn = NULL;
+ while ((scn = elf_nextscn(elf, scn)) != NULL)
+ {
+ GElf_Shdr shdr;
+ if (gelf_getshdr(scn, &shdr) != NULL)
+ {
+ char *name = elf_strptr(elf, strndx, (size_t) shdr.sh_name);
+ if (name != NULL && shdr.sh_type == SHT_PROGBITS)
+ {
+ bool is_eh = false;
+ if (strcmp(name, ".eh_frame") == 0)
+ {
+ is_eh = true;
+ }
+ else if (strcmp(name, ".debug_frame") == 0 || strcmp(name, ".zdebug_frame") == 0)
+ {
+ is_eh = false;
+ }
+ else
+ {
+ continue;
+ }
+
+ if (thread_count >= num_threads)
+ {
+ num_threads *= 2;
+ threads = realloc(threads, num_threads * sizeof(pthread_t));
+ thread_data = realloc(thread_data, num_threads * sizeof(ThreadData*));
+ }
+
+ thread_data[thread_count] = malloc(sizeof(ThreadData));
+ thread_data[thread_count]->name = name;
+ thread_data[thread_count]->e_ident = ident;
+ thread_data[thread_count]->scn = scn;
+ thread_data[thread_count]->is_eh = is_eh;
+
+ if (pthread_create(&threads[thread_count], NULL, thread_work, thread_data[thread_count]) != 0)
+ {
+ perror("Failed to create thread");
+ for (int j = 0; j < thread_count; j++)
+ {
+ pthread_cancel(threads[j]);
+ }
+ free(threads);
+ free(thread_data);
+ return 1;
+ }
+ else
+ {
+ thread_count++;
+ }
+ }
+ }
+ }
+
+ for (int i = 0; i < thread_count; i++)
+ {
+ if (pthread_join(threads[i], NULL) != 0)
+ {
+ perror("Failed to join thread");
+ free(threads);
+ free(thread_data);
+ return 1;
+ }
+ }
+
+ for (int i = 0; i < thread_count; i++)
+ {
+ free(thread_data[i]);
+ }
+
+ free(threads);
+ free(thread_data);
+
+ elf_end(elf);
+ close(fd);
+
+ return 0;
+}
\ No newline at end of file
diff --git a/tests/eu_search_die.c b/tests/eu_search_die.c
new file mode 100644
index 00000000..a7f75521
--- /dev/null
+++ b/tests/eu_search_die.c
@@ -0,0 +1,262 @@
+/*Test program for eu_search_die.
+ Copyright (C) 2023 Red Hat, Inc.
+ This file is part of elfutils.
+
+ This file is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 3 of the License, or
+ (at your option) any later version.
+
+ elfutils is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see<http://www.gnu.org/licenses/>. */
+#include <config.h>
+#include ELFUTILS_HEADER(dw)
+#include <dwarf.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <stdio.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <assert.h>
+#include <inttypes.h>
+#include <string.h>
+#include <errno.h>
+#include <unistd.h>
+#include <pthread.h>
+
+static void *thread_work(void *arg);
+static int check_die(Dwarf_Die *die);
+static int check_dbg(Dwarf *dbg);
+
+/*The main Dwarf file. */
+static Dwarf * dwarf;
+
+typedef struct
+{
+ Dwarf * dbg;
+ Dwarf_Off start_offset;
+ Dwarf_Off end_offset;
+ int result;
+}
+
+ThreadData;
+
+static void *thread_work(void *arg)
+{
+ ThreadData *data = (ThreadData*) arg;
+ data->result = check_dbg(data->dbg);
+ return NULL;
+}
+
+static int check_die(Dwarf_Die *die)
+{
+ if (dwarf_tag(die) == DW_TAG_invalid)
+ {
+ printf("Invalid die\n");
+ return -1;
+ }
+
+ int res = 0;
+ void *addr = die->addr;
+ Dwarf_Die die2;
+ if (dwarf_die_addr_die(dwarf, addr, &die2) == NULL)
+ {
+ printf("Bad die addr die at offset %" PRIx64 "\n", dwarf_dieoffset(die));
+ res = -1;
+ }
+
+ if (dwarf_tag(die) != dwarf_tag(&die2))
+ {
+ printf("Tags differ for die at offset %" PRIx64 "\n", dwarf_dieoffset(die));
+ res = -1;
+ }
+
+ if (dwarf_cuoffset(die) != dwarf_cuoffset(&die2))
+ {
+ printf("CU offsets differ for die at offset %" PRIx64 "\n", dwarf_dieoffset(die));
+ res = -1;
+ }
+
+ Dwarf_Die child;
+ if (dwarf_child(die, &child) == 0)
+ res |= check_die(&child);
+
+ Dwarf_Die sibling;
+ if (dwarf_siblingof(die, &sibling) == 0)
+ res |= check_die(&sibling);
+
+ return res;
+}
+
+static int check_dbg(Dwarf *dbg)
+{
+ int res = 0;
+ Dwarf_Off off = 0;
+ Dwarf_Off old_off = 0;
+ size_t hsize;
+ Dwarf_Off abbrev;
+ uint8_t addresssize;
+ uint8_t offsetsize;
+
+ while (dwarf_nextcu(dbg, off, &off, &hsize, &abbrev, &addresssize, &offsetsize) == 0)
+ {
+ Dwarf_Die die;
+ if (dwarf_offdie(dbg, old_off + hsize, &die) != NULL)
+ {
+ printf("checking CU at %" PRIx64 "\n", old_off);
+ res |= check_die(&die);
+ }
+
+ old_off = off;
+ }
+
+ // Same for type...
+ Dwarf_Half version;
+ uint64_t typesig;
+ Dwarf_Off typeoff;
+ off = 0;
+ old_off = 0;
+
+ while (dwarf_next_unit(dbg, off, &off, &hsize, &version, &abbrev, &addresssize, &offsetsize, &typesig, &typeoff) == 0)
+ {
+ Dwarf_Die die;
+ if (dwarf_offdie_types(dbg, old_off + hsize, &die) != NULL)
+ {
+ printf("checking TU at %" PRIx64 "\n", old_off);
+ res |= check_die(&die);
+ }
+
+ // We should have seen this already, but double check...
+ if (dwarf_offdie_types(dbg, old_off + typeoff, &die) != NULL)
+ {
+ printf("checking Type DIE at %" PRIx64 "\n", old_off + hsize + typeoff);
+ res |= check_die(&die);
+ }
+
+ old_off = off;
+ }
+
+ Dwarf *alt = dwarf_getalt(dbg);
+
+ if (alt != NULL)
+ {
+ printf("checking alt debug\n");
+ res |= check_dbg(alt);
+ }
+
+ // Split or Type Dwarf_Dies gotten through dwarf_get_units.
+ Dwarf_CU *cu = NULL;
+ Dwarf_Die subdie;
+ uint8_t unit_type;
+ while (dwarf_get_units(dbg, cu, &cu, NULL, &unit_type, NULL, &subdie) == 0)
+ {
+ if (dwarf_tag(&subdie) != DW_TAG_invalid)
+ {
+ printf("checking %"
+ PRIx8 " subdie\n", unit_type);
+ res |= check_die(&subdie);
+ }
+ }
+
+ return res;
+}
+
+int main(int argc, char *argv[])
+{
+ if (argc < 2)
+ {
+ printf("No file given.\n");
+ return -1;
+ }
+
+ const char *name = argv[1];
+ int fd = open(name, O_RDONLY);
+ if (fd < 0)
+ {
+ printf("Cannot open '%s': %s\n", name, strerror(errno));
+ return -1;
+ }
+
+ dwarf = dwarf_begin(fd, DWARF_C_READ);
+ if (dwarf == NULL)
+ {
+ printf("Not a Dwarf file '%s': %s\n", name, dwarf_errmsg(-1));
+ close(fd);
+ return -1;
+ }
+
+ printf("checking %s\n", name);
+
+ int num_threads = 4;
+ pthread_t *threads = (pthread_t*) malloc(num_threads* sizeof(pthread_t));
+ ThreadData *thread_data = (ThreadData*) malloc(num_threads* sizeof(ThreadData));
+
+ if (!threads || !thread_data)
+ {
+ fprintf(stderr, "Failed to allocate memory for threads.\n");
+ free(threads);
+ free(thread_data);
+ return 1;
+ }
+
+ Dwarf_Off total_off = 0;
+ Dwarf_Off unit_off = 0;
+ size_t hsize;
+ Dwarf_Off abbrev;
+ uint8_t addresssize;
+ uint8_t offsetsize;
+
+ while (dwarf_nextcu(dwarf, unit_off, &unit_off, &hsize, &abbrev, &addresssize, &offsetsize) == 0)
+ {
+ thread_data[total_off % num_threads].start_offset = unit_off;
+ thread_data[total_off % num_threads].end_offset = unit_off + hsize;
+ total_off++;
+ }
+
+ for (int i = 0; i < num_threads; i++)
+ {
+ thread_data[i].dbg = dwarf;
+ if (pthread_create(&threads[i], NULL, thread_work, (void*) &thread_data[i]) != 0)
+ {
+ perror("Failed to create thread");
+ for (int j = 0; j < i; j++)
+ {
+ pthread_cancel(threads[j]);
+ }
+ free(threads);
+ free(thread_data);
+ return 1;
+ }
+ }
+
+ for (int i = 0; i < num_threads; i++)
+ {
+ if (pthread_join(threads[i], NULL) != 0)
+ {
+ perror("Failed to join thread");
+ free(threads);
+ free(thread_data);
+ return 1;
+ }
+ }
+
+ int res = 0;
+ for (int i = 0; i < num_threads; i++)
+ {
+ res |= thread_data[i].result;
+ }
+
+ free(threads);
+ free(thread_data);
+
+ dwarf_end(dwarf);
+ close(fd);
+
+ return res;
+}
\ No newline at end of file
diff --git a/tests/eu_search_lines.c b/tests/eu_search_lines.c
new file mode 100644
index 00000000..b7a875d8
--- /dev/null
+++ b/tests/eu_search_lines.c
@@ -0,0 +1,228 @@
+/*Test program for eu_search_lines.
+ Copyright (C) 2023 Red Hat, Inc.
+ This file is part of elfutils.
+
+ This file is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 3 of the License, or
+ (at your option) any later version.
+
+ elfutils is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see<http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+#include <fcntl.h>
+#include <inttypes.h>
+#include <libelf.h>
+#include ELFUTILS_HEADER(dw)
+#include <stdio.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdlib.h>
+#include <pthread.h>
+
+typedef struct
+{
+ const char *filename;
+ int result;
+}
+
+ThreadData;
+
+static void *thread_work(void *arg)
+{
+ ThreadData *data = (ThreadData*) arg;
+
+ int fd = open(data->filename, O_RDONLY);
+
+ Dwarf *dbg = dwarf_begin(fd, DWARF_C_READ);
+ if (dbg == NULL)
+ {
+ printf("%s not usable: %s\n", data->filename, dwarf_errmsg(-1));
+ close(fd);
+ free(data);
+ pthread_exit(NULL);
+ }
+
+ Dwarf_Off cuoff = 0;
+ Dwarf_Off old_cuoff = 0;
+ size_t hsize;
+ Dwarf_Off ao;
+ uint8_t asz;
+ uint8_t osz;
+ while (dwarf_nextcu(dbg, cuoff, &cuoff, &hsize, &ao, &asz, &osz) == 0)
+ {
+ printf("cuhl = %zu, o = %llu, asz = %hhu, osz = %hhu, ncu = %llu\n",
+ hsize, (unsigned long long int) ao,
+ asz, osz, (unsigned long long int) cuoff);
+
+ // Get the DIE for the CU.
+ Dwarf_Die die;
+ if (dwarf_offdie(dbg, old_cuoff + hsize, &die) == NULL)
+ {
+ printf("%s: cannot get CU die\n", data->filename);
+ data->result = 1;
+ break;
+ }
+
+ old_cuoff = cuoff;
+
+ Dwarf_Lines * lb;
+ size_t nlb;
+ if (dwarf_getsrclines(&die, &lb, &nlb) != 0)
+ {
+ printf("%s: cannot get lines\n", data->filename);
+ data->result = 1;
+ break;
+ }
+
+ printf(" %zu lines\n", nlb);
+
+ for (size_t i = 0; i < nlb; ++i)
+ {
+ Dwarf_Line *l = dwarf_onesrcline(lb, i);
+ if (l == NULL)
+ {
+ printf("%s: cannot get individual line\n", data->filename);
+ data->result = 1;
+ break;
+ }
+
+ Dwarf_Addr addr;
+ if (dwarf_lineaddr(l, &addr) != 0)
+ addr = 0;
+ const char *file = dwarf_linesrc(l, NULL, NULL);
+ int line;
+ if (dwarf_lineno(l, &line) != 0)
+ line = 0;
+
+ printf("%" PRIx64 ": %s:%d:", (uint64_t) addr, file ? : "???", line);
+
+ // Getting the file path through the Dwarf_Files should
+ // result in the same path.
+ Dwarf_Files * files;
+ size_t idx;
+ if (dwarf_line_file(l, &files, &idx) != 0)
+ {
+ printf("%s: cannot get file from line (%zd): %s\n",
+ data->filename, i, dwarf_errmsg(-1));
+ data->result = 1;
+ break;
+ }
+
+ const char *path = dwarf_filesrc(files, idx, NULL, NULL);
+ if ((path == NULL && file != NULL) ||
+ (path != NULL && file == NULL) ||
+ (strcmp(file, path) != 0))
+ {
+ printf("%s: line %zd srcline (%s) != file srcline (%s)\n",
+ data->filename, i, file ? : "???", path ? : "???");
+ data->result = 1;
+ break;
+ }
+
+ int column;
+ if (dwarf_linecol(l, &column) != 0)
+ column = 0;
+ if (column >= 0)
+ printf("%d:", column);
+
+ bool is_stmt;
+ if (dwarf_linebeginstatement(l, &is_stmt) != 0)
+ is_stmt = false;
+ bool end_sequence;
+ if (dwarf_lineendsequence(l, &end_sequence) != 0)
+ end_sequence = false;
+ bool basic_block;
+ if (dwarf_lineblock(l, &basic_block) != 0)
+ basic_block = false;
+ bool prologue_end;
+ if (dwarf_lineprologueend(l, &prologue_end) != 0)
+ prologue_end = false;
+ bool epilogue_begin;
+ if (dwarf_lineepiloguebegin(l, &epilogue_begin) != 0)
+ epilogue_begin = false;
+
+ printf(" is_stmt:%s, end_seq:%s, bb:%s, prologue:%s, epilogue:%s\n",
+ is_stmt ? "yes" : "no", end_sequence ? "yes" : "no",
+ basic_block ? "yes" : "no", prologue_end ? "yes" : "no",
+ epilogue_begin ? "yes" : "no");
+ }
+ }
+
+ dwarf_end(dbg);
+ close(fd);
+ free(data);
+
+ pthread_exit(NULL);
+}
+
+int main(int argc, char *argv[])
+{
+ int result = 0;
+ int cnt;
+
+ if (argc < 2)
+ {
+ printf("Usage: %s<filename1>[<filename2> ...]\n", argv[0]);
+ return 1;
+ }
+
+ pthread_t *threads = (pthread_t*) malloc((argc - 1) *sizeof(pthread_t));
+ ThreadData **thread_data = (ThreadData **) malloc((argc - 1) *sizeof(ThreadData*));
+
+ if (!threads || !thread_data)
+ {
+ fprintf(stderr, "Failed to allocate memory for threads.\n");
+ free(threads);
+ free(thread_data);
+ return 1;
+ }
+
+ for (cnt = 1; cnt < argc; ++cnt)
+ {
+ thread_data[cnt - 1] = (ThreadData*) malloc(sizeof(ThreadData));
+ thread_data[cnt - 1]->filename = argv[cnt];
+ thread_data[cnt - 1]->result = 0;
+
+ if (pthread_create(&threads[cnt - 1], NULL, thread_work, thread_data[cnt - 1]) != 0)
+ {
+ perror("Failed to create thread");
+ for (int j = 0; j < cnt; j++)
+ {
+ pthread_cancel(threads[j]);
+ }
+ free(threads);
+ free(thread_data);
+ return 1;
+ }
+ }
+
+ for (cnt = 0; cnt < argc - 1; ++cnt)
+ {
+ if (pthread_join(threads[cnt], NULL) != 0)
+ {
+ perror("Failed to join thread");
+ free(threads);
+ free(thread_data);
+ return 1;
+ }
+
+ if (thread_data[cnt]->result != 0)
+ {
+ result = 1;
+ }
+
+ free(thread_data[cnt]);
+ }
+
+ free(threads);
+ free(thread_data);
+
+ return result;
+}
\ No newline at end of file
diff --git a/tests/eu_search_macros.c b/tests/eu_search_macros.c
new file mode 100644
index 00000000..3dca828e
--- /dev/null
+++ b/tests/eu_search_macros.c
@@ -0,0 +1,216 @@
+/*Test program for eu_search_macros
+ Copyright (C) 2023 Red Hat, Inc.
+ This file is part of elfutils.
+
+ This file is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 3 of the License, or
+ (at your option) any later version.
+
+ elfutils is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see<http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+#include ELFUTILS_HEADER(dw)
+#include <dwarf.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <stdio.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <assert.h>
+#include <inttypes.h>
+#include <stdatomic.h>
+#include <pthread.h>
+
+static void *thread_work(void *arg);
+static int mac(Dwarf_Macro *macro, void *dbg);
+static void include(Dwarf *dbg, Dwarf_Off macoff, ptrdiff_t token);
+
+typedef struct
+{
+ Dwarf * dbg;
+ Dwarf_Die * cudie;
+ bool new_style;
+}
+
+ThreadData;
+
+static void *thread_work(void *arg)
+{
+ ThreadData *data = (ThreadData*) arg;
+ Dwarf *dbg = data->dbg;
+ Dwarf_Die *cudie = data->cudie;
+ bool new_style = data->new_style;
+
+ for (ptrdiff_t off = new_style ? DWARF_GETMACROS_START : 0;
+ (off = dwarf_getmacros(cudie, mac, dbg, off));)
+ {
+ if (off == -1)
+ {
+ puts(dwarf_errmsg(dwarf_errno()));
+ break;
+ }
+ }
+
+ return NULL;
+}
+
+static void include(Dwarf *dbg, Dwarf_Off macoff, ptrdiff_t token)
+{
+ while ((token = dwarf_getmacros_off(dbg, macoff, mac, dbg, token)) != 0)
+ {
+ if (token == -1)
+ {
+ puts(dwarf_errmsg(dwarf_errno()));
+ break;
+ }
+ }
+}
+
+static int
+mac(Dwarf_Macro *macro, void *dbg)
+{
+ static atomic_int level = 0;
+
+ unsigned int opcode;
+ dwarf_macro_opcode(macro, &opcode);
+ switch (opcode)
+ {
+ case DW_MACRO_import:
+ {
+ Dwarf_Attribute at;
+ int r = dwarf_macro_param(macro, 0, &at);
+ assert(r == 0);
+
+ Dwarf_Word w;
+ r = dwarf_formudata(&at, &w);
+ assert(r == 0);
+
+ printf ("%*sinclude %#" PRIx64 "\n", level, "", w);
+
+ atomic_fetch_add(&level, 1);
+
+ include(dbg, w, DWARF_GETMACROS_START);
+
+ atomic_fetch_sub(&level, 1);
+
+ printf ("%*s/include\n", level, "");
+ break;
+ }
+
+ case DW_MACRO_start_file:
+ {
+ Dwarf_Files * files;
+ size_t nfiles;
+ if (dwarf_macro_getsrcfiles(dbg, macro, &files, &nfiles) < 0)
+ printf("dwarf_macro_getsrcfiles: %s\n", dwarf_errmsg(dwarf_errno()));
+
+ Dwarf_Word w = 0;
+ dwarf_macro_param2(macro, &w, NULL);
+
+ const char *name = dwarf_filesrc (files, (size_t) w, NULL, NULL);
+ printf ("%*sfile %s\n", level, "", name);
+ atomic_fetch_add(&level, 1);
+ break;
+ }
+
+ case DW_MACRO_end_file:
+ {
+ atomic_fetch_sub(&level, 1);
+ printf ("%*s/file\n", level, "");
+ break;
+ }
+
+ case DW_MACINFO_define:
+ case DW_MACRO_define_strp:
+ {
+ const char *value;
+ dwarf_macro_param2(macro, NULL, &value);
+ printf ("%*s%s\n", level, "", value);
+ break;
+ }
+
+ case DW_MACINFO_undef:
+ case DW_MACRO_undef_strp:
+ break;
+
+ default:
+ {
+ size_t paramcnt;
+ dwarf_macro_getparamcnt(macro, ¶mcnt);
+ printf ("%*sopcode %u with %zd arguments\n", level, "", opcode, paramcnt);
+ break;
+ }
+ }
+
+ return DWARF_CB_ABORT;
+}
+
+int main(int argc, char *argv[])
+{
+ assert(argc >= 3);
+ const char *name = argv[1];
+ ptrdiff_t cuoff = strtol(argv[2], NULL, 0);
+ bool new_style = argc > 3;
+
+ int fd = open(name, O_RDONLY);
+ Dwarf *dbg = dwarf_begin(fd, DWARF_C_READ);
+
+ Dwarf_Die cudie_mem, *cudie = dwarf_offdie(dbg, cuoff, &cudie_mem);
+
+ int num_threads = 4;
+ pthread_t *threads = malloc(num_threads* sizeof(pthread_t));
+ ThreadData *thread_data = malloc(num_threads* sizeof(ThreadData));
+
+ if (!threads || !thread_data)
+ {
+ fprintf(stderr, "Failed to allocate memory for threads.\n");
+ free(threads);
+ free(thread_data);
+ return 1;
+ }
+
+ for (int i = 0; i < num_threads; i++)
+ {
+ thread_data[i].dbg = dbg;
+ thread_data[i].cudie = cudie;
+ thread_data[i].new_style = new_style;
+
+ if (pthread_create(&threads[i], NULL, thread_work, (void*) &thread_data[i]) != 0)
+ {
+ perror("Failed to create thread");
+ for (int j = 0; j < i; j++)
+ {
+ pthread_cancel(threads[j]);
+ }
+ free(threads);
+ free(thread_data);
+ return 1;
+ }
+ }
+
+ for (int i = 0; i < num_threads; i++)
+ {
+ if (pthread_join(threads[i], NULL) != 0)
+ {
+ perror("Failed to join thread");
+ free(threads);
+ free(thread_data);
+ return 1;
+ }
+ }
+
+ free(threads);
+ free(thread_data);
+
+ dwarf_end(dbg);
+
+ return 0;
+}
\ No newline at end of file
diff --git a/tests/run-eu-search-tests.sh b/tests/run-eu-search-tests.sh
new file mode 100755
index 00000000..84edc234
--- /dev/null
+++ b/tests/run-eu-search-tests.sh
@@ -0,0 +1,168 @@
+#! /bin/sh
+# Copyright (C) 2015, 2018 Red Hat, Inc.
+# This file is part of elfutils.
+#
+# This file is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# elfutils is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+. $srcdir/test-subr.sh
+
+# Extract the value of USE_ADDRESS_SANITIZER_TRUE from config.status
+# Cannot use Helgrind and Address Sanitizer together.
+# Test will be skipped if Address Sanitizer is enabled.
+USE_ADDRESS_SANITIZER=$(grep 'USE_ADDRESS_SANITIZER_TRUE' ${abs_builddir}/../config.status | awk -F'=' '{print $2}')
+
+if [[ "$USE_ADDRESS_SANITIZER" == "\"#\"" ]]; then
+ echo "Address Sanitizer is disabled."
+else
+ echo "Address Sanitizer is enabled. Skipping test."
+ exit 77
+fi
+
+# Extract the value of USE_MEMORY_SANITIZER_TRUE from config.status
+# Cannot use Helgrind and Memory Sanitizer together.
+# Test will be skipped if Memory Sanitizer is enabled.
+USE_MEMORY_SANITIZER=$(grep 'USE_MEMORY_SANITIZER_TRUE' ${abs_builddir}/../config.status | awk -F'=' '{print $2}')
+
+if [[ "$USE_MEMORY_SANITIZER" == "\"#\"" ]]; then
+ echo "Memory Sanitizer is disabled."
+else
+ echo "Memory Sanitizer is enabled. Skipping test."
+ exit 77
+fi
+
+# Extract the value of USE_LOCKS from config.h
+# Test will only be run if USE_LOCKS is defined. Otherwise, skip.
+USE_LOCKS=$(grep '^#define USE_LOCKS' ${abs_builddir}/../config.h | awk '{print $3}')
+
+if [[ "$USE_LOCKS" -eq 1 ]]; then
+ echo "USE_LOCKS is defined."
+else
+ echo "USE_LOCKS is not defined. Skipping test."
+ exit 77
+fi
+
+# Disable valgrind if configured, since we are already using it here.
+SAVED_VALGRIND_CMD="$VALGRIND_CMD"
+unset VALGRIND_CMD
+
+echo "Begin tests..."
+
+# Begin data race test for parallelized dwarf-die-addr-die
+# Tests thread safety for updated libdw_findcu.c and libdw_find_split_unit.c
+testfiles testfile-debug-types
+testfiles testfile_multi_main testfile_multi.dwz
+testfiles testfilebazdbgppc64.debug
+testfiles testfile-dwarf-4 testfile-dwarf-5
+testfiles testfile-splitdwarf-4 testfile-hello4.dwo testfile-world4.dwo
+testfiles testfile-splitdwarf-5 testfile-hello5.dwo testfile-world5.dwo
+
+die_test_files=("testfile-debug-types"
+ "testfile_multi_main" "testfile_multi.dwz"
+ "testfilebazdbgppc64.debug"
+ "testfile-dwarf-4" "testfile-dwarf-5"
+ "testfile-splitdwarf-4" "testfile-hello4.dwo" "testfile-world4.dwo"
+ "testfile-splitdwarf-5" "testfile-hello5.dwo" "testfile-world5.dwo")
+
+echo -e "\nStarting data race test for dwarf-die-addr-die"
+
+for file in "${die_test_files[@]}"; do
+ helgrind_output=$(valgrind --tool=helgrind "${abs_builddir}/eu_search_die" "$file" 2>&1)
+
+ if grep -q "ERROR SUMMARY: 0 errors" <<< "$helgrind_output"; then
+ echo "No data races found for $file. Test passed."
+ else
+ echo "Data races found for $file. Test failed."
+ echo "$helgrind_output"
+ exit 1
+ fi
+done
+
+# Begin data race test for parallelized next_cfi
+# Tests thread safety for updated cie.c and fde.c
+testfiles testfile11 testfile12
+testfiles testfilearm testfileaarch64
+testfiles testfileppc32 testfileppc64
+
+cfi_test_files=("testfile11" "testfile12"
+ "testfilearm" "testfileaarch64"
+ "testfileppc32" "testfileppc64")
+
+echo -e "\nStarting data race test for next_cfi"
+
+for file in "${cfi_test_files[@]}"; do
+
+ helgrind_output=$(valgrind --tool=helgrind "${abs_builddir}/eu_search_cfi" $file 2>&1)
+
+ if grep -q "ERROR SUMMARY: 0 errors" <<< "$helgrind_output"; then
+ echo "No data races found for $file. Test passed."
+ else
+ echo "Data races found for $file. Test failed."
+ echo "$helgrind_output"
+ exit 1
+ fi
+
+done
+
+# Begin data race test for parallelizd dwarf-getmacros
+# Tests thread safety for updated dwarf_getmacros.c
+testfiles testfile51
+testfiles testfile-macros
+testfiles testfile-macros-0xff
+
+macro_test_files=("testfile51 0xb"
+ "testfile51 0x84"
+ "testfile-macrosm 0xb"
+ "testfile-macros-0xff 0xb")
+
+echo -e "\nStarting data race test for dwarf-getmacros"
+
+for file in "${macro_test_files[@]}"; do
+
+ helgrind_output=$(valgrind --tool=helgrind "${abs_builddir}/eu_search_macros" $file 2>&1)
+
+ if grep -q "ERROR SUMMARY: 0 errors" <<< "$helgrind_output"; then
+ echo "No data races found for $file. Test passed."
+ else
+ echo "Data races found for $file. Test failed."
+ echo "$helgrind_output"
+ exit 1
+ fi
+
+done
+
+# Begin data race test for parallelized get-lines
+# Tests thread safety for updated dwarf_getsrclines.c
+testfiles testfile testfile2 testfilenolines
+
+lines_test_files=("testfile" "testfile2" "testfilenolines")
+
+echo -e "\nStarting data race test for get-lines"
+
+for file in "${lines_test_files[@]}"; do
+
+ helgrind_output=$(valgrind --tool=helgrind "${abs_builddir}/eu_search_lines" $file 2>&1)
+
+ if grep -q "ERROR SUMMARY: 0 errors" <<< "$helgrind_output"; then
+ echo "No data races found for $file. Test passed."
+ else
+ echo -e "$helgrind_output"
+ echo "Data races found for $file. Test failed."
+ exit 1
+ fi
+
+done
+
+# This line is reached only if no data races were found in any test
+# Exit with success status.
+exit 0
\ No newline at end of file
--
2.41.0
^ permalink raw reply [flat|nested] 51+ messages in thread
* [PATCH 16/16] configure: No longer mark --enable-thread-safety as EXPERIMENTAL
2023-10-10 13:42 ` [PATCH 01/16] lib: Add new once_define and once macros to eu-config.h Mark Wielaard
` (13 preceding siblings ...)
2023-10-10 13:42 ` [PATCH 15/16] tests: Add eu-search tests Mark Wielaard
@ 2023-10-10 13:43 ` Mark Wielaard
2023-10-12 22:09 ` Mark Wielaard
2023-10-10 13:54 ` [PATCH 01/16] lib: Add new once_define and once macros to eu-config.h Mark Wielaard
15 siblings, 1 reply; 51+ messages in thread
From: Mark Wielaard @ 2023-10-10 13:43 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2, Mark Wielaard
From: Heather McIntyre <hsm2@rice.edu>
* configure.ac (--enable-thread-safety): Remove experimental
warning.
Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
Signed-off-by: Mark Wielaard <mark@klomp.org>
---
configure.ac | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/configure.ac b/configure.ac
index 29ed32fe..be2103e2 100644
--- a/configure.ac
+++ b/configure.ac
@@ -79,12 +79,10 @@ AC_DEFINE_UNQUOTED(DEFAULT_AR_DETERMINISTIC, $default_ar_deterministic,
AC_ARG_ENABLE([thread-safety],
AS_HELP_STRING([--enable-thread-safety],
- [enable thread safety of libraries EXPERIMENTAL]),
+ [enable thread safety of libraries]),
use_locks=$enableval, use_locks=no)
AM_CONDITIONAL(USE_LOCKS, test "$use_locks" = yes)
AS_IF([test "$use_locks" = yes], [AC_DEFINE(USE_LOCKS)])
-AS_IF([test "$use_locks" = yes],
- [AC_MSG_WARN([thread-safety is EXPERIMENTAL tests might fail.])])
AH_TEMPLATE([USE_LOCKS], [Defined if libraries should be thread-safe.])
@@ -907,10 +905,10 @@ AC_MSG_NOTICE([
Symbol versioning : ${enable_symbol_versioning}
NOT RECOMMENDED FEATURES (should all be no)
- Experimental thread safety : ${use_locks}
install elf.h : ${install_elfh}
OTHER FEATURES
+ Enable thread safety : ${use_locks}
Deterministic archives by default : ${default_ar_deterministic}
Native language support : ${USE_NLS}
Extra Valgrind annotations : ${use_vg_annotations}
--
2.41.0
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH 01/16] lib: Add new once_define and once macros to eu-config.h
2023-10-10 13:42 ` [PATCH 01/16] lib: Add new once_define and once macros to eu-config.h Mark Wielaard
` (14 preceding siblings ...)
2023-10-10 13:43 ` [PATCH 16/16] configure: No longer mark --enable-thread-safety as EXPERIMENTAL Mark Wielaard
@ 2023-10-10 13:54 ` Mark Wielaard
15 siblings, 0 replies; 51+ messages in thread
From: Mark Wielaard @ 2023-10-10 13:54 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2
Hi Heather,
On Tue, 2023-10-10 at 15:42 +0200, Mark Wielaard wrote:
> From: Heather McIntyre <hsm2@rice.edu>
>
> * lib/eu-config.h New macros.
> [USE_LOCKS] (ONCE_CALL): (once_define, once)
>
> Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
> Signed-off-by: Mark Wielaard <mark@klomp.org>
> ---
> lib/eu-config.h | 7 +++++++
> 1 file changed, 7 insertions(+)
>
> diff --git a/lib/eu-config.h b/lib/eu-config.h
> index 78a5c4fe..feb079db 100644
> --- a/lib/eu-config.h
> +++ b/lib/eu-config.h
> @@ -33,13 +33,18 @@
> # include <pthread.h>
> # include <assert.h>
> # define rwlock_define(class,name) class pthread_rwlock_t name
> +# define once_define(class,name) class pthread_once_t name = PTHREAD_ONCE_INIT
> # define RWLOCK_CALL(call) \
> ({ int _err = pthread_rwlock_ ## call; assert_perror (_err); })
> +# define ONCE_CALL(call) \
> + ({ int _err = pthread_ ## call; assert_perror (_err); })
> # define rwlock_init(lock) RWLOCK_CALL (init (&lock, NULL))
> # define rwlock_fini(lock) RWLOCK_CALL (destroy (&lock))
> # define rwlock_rdlock(lock) RWLOCK_CALL (rdlock (&lock))
> # define rwlock_wrlock(lock) RWLOCK_CALL (wrlock (&lock))
> # define rwlock_unlock(lock) RWLOCK_CALL (unlock (&lock))
> +# define once(once_control, init_routine) \
> + ONCE_CALL (once (&once_control, init_routine))
> #else
> /* Eventually we will allow multi-threaded applications to use the
> libraries. Therefore we will add the necessary locking although
> @@ -50,6 +55,8 @@
> # define rwlock_rdlock(lock) ((void) (lock))
> # define rwlock_wrlock(lock) ((void) (lock))
> # define rwlock_unlock(lock) ((void) (lock))
> +# define once_define(class,name)
> +# define once(once_control, init_routine) init_routine()
> #endif /* USE_LOCKS */
>
> #include <libintl.h>
I made sure that only the newly added lines are changed. That made the
diff a bit easier to review.
This looks good. In the case without locks, the init_routine will of
course get called multiple times, but that should in theory be fine
(these init routines are in generally really simple).
Hopefully we'll eventually end up with just the locked version.
Cheers,
Mark
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH 02/16] libelf: Make elf_version thread-safe
2023-10-10 13:42 ` [PATCH 02/16] libelf: Make elf_version thread-safe Mark Wielaard
@ 2023-10-10 14:00 ` Mark Wielaard
2023-10-17 19:05 ` Heather McIntyre
0 siblings, 1 reply; 51+ messages in thread
From: Mark Wielaard @ 2023-10-10 14:00 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2
Hi Heather,
On Tue, 2023-10-10 at 15:42 +0200, Mark Wielaard wrote:
> From: Heather McIntyre <hsm2@rice.edu>
>
> * elf_version.c (version_once): Define once.
> (initialize_version): New static function.
> (elf_version): Use initialize_version version_once.
>
> Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
> Signed-off-by: Mark Wielaard <mark@klomp.org>
> ---
> libelf/elf_version.c | 11 ++++++++++-
> 1 file changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/libelf/elf_version.c b/libelf/elf_version.c
> index 6ec534ab..8296bb65 100644
> --- a/libelf/elf_version.c
> +++ b/libelf/elf_version.c
> @@ -32,12 +32,21 @@
> #endif
>
> #include <libelfP.h>
> +#include <pthread.h>
>
> +/* Multiple threads may initialize __libelf_version.
> + pthread_once() ensures that __libelf_version is initialized only once. */
> +once_define(static, version_once);
>
> /* Currently selected version. Should be EV_CURRENT.
> Will be EV_NONE if elf_version () has not been called yet. */
> unsigned int __libelf_version = EV_NONE;
>
> +static void initialize_version(void)
> +{
> + __libelf_version = EV_CURRENT;
> +}
> +
> unsigned int
> elf_version (unsigned int version)
> {
> @@ -49,7 +58,7 @@ elf_version (unsigned int version)
> /* Phew, we know this version. */
>
> /* Signal that the version is now initialized. */
> - __libelf_version = EV_CURRENT;
> + once(version_once, initialize_version);
>
> /* And return the last (or initial) version. */
> return EV_CURRENT;
This is an odd function. The intention clearly was to support more "ELF
versions" at some point. But (luckily) that never happened and I doubt
there will ever be a different (incompatible) ELF version that we'll
have to support. So in the end this will always be EV_CURRENT == 1. But
the function has to be called to make the rest of the library work.
I think this works and is fine. There will most likely never be real
contention calling elf_version because normally a program just calls it
once at the start.
But have you thought about using some atomic operation here instead?
Cheers,
Mark
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH 03/16] libelf: Fix deadlock in __libelf_readall
2023-10-10 13:42 ` [PATCH 03/16] libelf: Fix deadlock in __libelf_readall Mark Wielaard
@ 2023-10-10 15:06 ` Mark Wielaard
2023-10-17 19:11 ` Heather McIntyre
0 siblings, 1 reply; 51+ messages in thread
From: Mark Wielaard @ 2023-10-10 15:06 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2
Hi Heather,
On Tue, 2023-10-10 at 15:42 +0200, Mark Wielaard wrote:
> From: Heather McIntyre <hsm2@rice.edu>
>
> * libelf/elf_readall.c (__libelf_readall): Move rwlock_unlock
> before libelf_acquire_all.
>
> Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
> Signed-off-by: Mark Wielaard <mark@klomp.org>
> ---
> libelf/elf_readall.c | 5 ++---
> 1 file changed, 2 insertions(+), 3 deletions(-)
>
> diff --git a/libelf/elf_readall.c b/libelf/elf_readall.c
> index d0f9a28c..2d62d447 100644
> --- a/libelf/elf_readall.c
> +++ b/libelf/elf_readall.c
> @@ -84,6 +84,7 @@ __libelf_readall (Elf *elf)
>
> /* If this is an archive and we have derived descriptors get the
> locks for all of them. */
> + rwlock_unlock(elf->lock); // lock will be reacquired next line
> libelf_acquire_all (elf);
>
> if (elf->maximum_size == ~((size_t) 0))
> @@ -141,10 +142,8 @@ __libelf_readall (Elf *elf)
> __libelf_seterrno (ELF_E_NOMEM);
>
> /* Free the locks on the children. */
> - libelf_release_all (elf);
> + libelf_release_all (elf); // lock is released
> }
>
> - rwlock_unlock (elf->lock);
> -
> return (char *) elf->map_address;
> }
I think this is wrong when this if statement, at the start of the
block, fails:
/* If the file is not mmap'ed and not previously loaded, do it now. */
if (elf->map_address == NULL)
So if elf->map_address != NULL we now never call
rwlock_unlock (elf->lock).
One way to simplify this locking might be to rewrite libelf_acquire_all
and libelf_release_all to libelf_acquire_all_children and
libelf_release_all_children (which would only be called with the elf-
>lock already acquired).
__libelf_readall is the only caller of these functions.
Cheers,
Mark
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH 04/16] libelf: Fix deadlock in elf_cntl
2023-10-10 13:42 ` [PATCH 04/16] libelf: Fix deadlock in elf_cntl Mark Wielaard
@ 2023-10-10 15:23 ` Mark Wielaard
2023-10-17 19:14 ` Heather McIntyre
0 siblings, 1 reply; 51+ messages in thread
From: Mark Wielaard @ 2023-10-10 15:23 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2
Hi Heather,
On Tue, 2023-10-10 at 15:42 +0200, Mark Wielaard wrote:
> From: Heather McIntyre <hsm2@rice.edu>
>
> * libelf/elf_cntl.c (elf_cntl): Move rwlock_wrlock, rwlock_unlock,
> inside case switch statements.
>
> Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
> Signed-off-by: Mark Wielaard <mark@klomp.org>
> ---
> libelf/elf_cntl.c | 11 +++++++----
> 1 file changed, 7 insertions(+), 4 deletions(-)
>
> diff --git a/libelf/elf_cntl.c b/libelf/elf_cntl.c
> index 04aa9132..64087c7d 100644
> --- a/libelf/elf_cntl.c
> +++ b/libelf/elf_cntl.c
> @@ -48,13 +48,16 @@ elf_cntl (Elf *elf, Elf_Cmd cmd)
> return -1;
> }
>
> - rwlock_wrlock (elf->lock);
> +
>
> switch (cmd)
> {
> case ELF_C_FDREAD:
> + rwlock_rdlock (elf->lock);
> + int addr_isnull = elf->map_address == NULL;
> + rwlock_unlock(elf->lock);
> /* If not all of the file is in the memory read it now. */
> - if (elf->map_address == NULL && __libelf_readall (elf) == NULL)
> + if (addr_isnull && __libelf_readall (elf) == NULL)
> {
> /* We were not able to read everything. */
> result = -1;
Can't we just rely on if (__libelf_readall (elf) == NULL)?
__libelf_readall already does locking and will return non-NULL if elf-
>map_address is already set. So it looks like the extra check (and
locking) to check addr_isnull is redundant and just make the code more
complex.
> @@ -64,7 +67,9 @@ elf_cntl (Elf *elf, Elf_Cmd cmd)
>
> case ELF_C_FDDONE:
> /* Mark the file descriptor as not usable. */
> + rwlock_wrlock (elf->lock);
> elf->fildes = -1;
> + rwlock_unlock (elf->lock);
> break;
>
> default:
This looks correct. All other accesses to elf->fildes seem to be done
under the elf->lock too.
> @@ -73,7 +78,5 @@ elf_cntl (Elf *elf, Elf_Cmd cmd)
> break;
> }
>
> - rwlock_unlock (elf->lock);
> -
> return result;
> }
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH 05/16] libelf: Fix elf_end deadlock
2023-10-10 13:42 ` [PATCH 05/16] libelf: Fix elf_end deadlock Mark Wielaard
@ 2023-10-10 15:28 ` Mark Wielaard
0 siblings, 0 replies; 51+ messages in thread
From: Mark Wielaard @ 2023-10-10 15:28 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2
Hi Heather,
On Tue, 2023-10-10 at 15:42 +0200, Mark Wielaard wrote:
> From: Heather McIntyre <hsm2@rice.edu>
>
> * libelf/elf_end.c (elf_end): Add rwlock_unlock before
> early return.
>
> Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
> Signed-off-by: Mark Wielaard <mark@klomp.org>
> ---
> libelf/elf_end.c | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/libelf/elf_end.c b/libelf/elf_end.c
> index 89727cb3..80f4d13f 100644
> --- a/libelf/elf_end.c
> +++ b/libelf/elf_end.c
> @@ -82,7 +82,10 @@ elf_end (Elf *elf)
> elf->state.ar.ar_sym = NULL;
>
> if (elf->state.ar.children != NULL)
> - return 0;
> + {
> + rwlock_unlock(elf->lock);
> + return 0;
> + }
> }
>
> /* Remove this structure from the children list. */
This looks obviously correct. All other early returns also release the
lock.
Thanks,
Mark
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH 06/16] libelf: Make elf32_getchdr and elf64_getchdr thread-safe
2023-10-10 13:42 ` [PATCH 06/16] libelf: Make elf32_getchdr and elf64_getchdr thread-safe Mark Wielaard
@ 2023-10-10 16:28 ` Mark Wielaard
0 siblings, 0 replies; 51+ messages in thread
From: Mark Wielaard @ 2023-10-10 16:28 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2
Hi Heather,
On Tue, 2023-10-10 at 15:42 +0200, Mark Wielaard wrote:
> From: Heather McIntyre <hsm2@rice.edu>
>
> * libelf/elf32_getchdr.c: Move getchdr function to
> elf32_getchdr.h.
> * libelf/elf32_getchdr.h: New file.
> Add macro to create getchdr_wrlock.
That is clever. I do wonder if the new file should be named
elf_getchdr.h (since it isn't really directly 32 or 64 bit, but
included (indirectly) from one of them. This is a nitpick though.
You do have to include it in the noinst_HEADERS:
diff --git a/libelf/Makefile.am b/libelf/Makefile.am
index aabce43e..3402863e 100644
--- a/libelf/Makefile.am
+++ b/libelf/Makefile.am
@@ -41,7 +41,7 @@ include_HEADERS = libelf.h gelf.h nlist.h
noinst_HEADERS = abstract.h common.h exttypes.h gelf_xlate.h libelfP.h \
version_xlate.h gnuhash_xlate.h note_xlate.h dl-hash.h \
- chdr_xlate.h
+ chdr_xlate.h elf32_getchdr.h
if INSTALL_ELFH
include_HEADERS += elf.h
Otherwise it won't be included in a dist (make distcheck would show
that as an issue).
> * libelf/elf32_updatenull.c: Change call from getchdr to
> getchdr_wrlock.
> * libelf/elf_getdata.c: Add elf_getdata_wrlock.
> * libelf/libelfP.h: Add internal function declarations.
>
> Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
> Signed-off-by: Mark Wielaard <mark@klomp.org>
> ---
> libelf/elf32_getchdr.c | 46 +++--------------------------
> libelf/elf32_getchdr.h | 61 +++++++++++++++++++++++++++++++++++++++
> libelf/elf32_updatenull.c | 2 +-
> libelf/elf_getdata.c | 14 +++++++++
> libelf/libelfP.h | 4 +++
> 5 files changed, 84 insertions(+), 43 deletions(-)
> create mode 100644 libelf/elf32_getchdr.h
>
> diff --git a/libelf/elf32_getchdr.c b/libelf/elf32_getchdr.c
> index 982a614c..41591300 100644
> --- a/libelf/elf32_getchdr.c
> +++ b/libelf/elf32_getchdr.c
> @@ -38,46 +38,8 @@
> # define LIBELFBITS 32
> #endif
>
> +#define ELF_WRLOCK_HELD 1
> +#include "elf32_getchdr.h"
>
> -ElfW2(LIBELFBITS,Chdr) *
> -elfw2(LIBELFBITS,getchdr) (Elf_Scn *scn)
> -{
> - ElfW2(LIBELFBITS,Shdr) *shdr = elfw2(LIBELFBITS,getshdr) (scn);
> - if (shdr == NULL)
> - return NULL;
> -
> - /* Must have SHF_COMPRESSED flag set. Allocated or no bits sections
> - can never be compressed. */
> - if ((shdr->sh_flags & SHF_ALLOC) != 0)
> - {
> - __libelf_seterrno (ELF_E_INVALID_SECTION_FLAGS);
> - return NULL;
> - }
> -
> - if (shdr->sh_type == SHT_NULL
> - || shdr->sh_type == SHT_NOBITS)
> - {
> - __libelf_seterrno (ELF_E_INVALID_SECTION_TYPE);
> - return NULL;
> - }
> -
> - if ((shdr->sh_flags & SHF_COMPRESSED) == 0)
> - {
> - __libelf_seterrno (ELF_E_NOT_COMPRESSED);
> - return NULL;
> - }
> -
> - /* This makes sure the data is in the correct format, so we don't
> - need to swap fields. */
> - Elf_Data *d = elf_getdata (scn, NULL);
> - if (d == NULL)
> - return NULL;
> -
> - if (d->d_size < sizeof (ElfW2(LIBELFBITS,Chdr)) || d->d_buf == NULL)
> - {
> - __libelf_seterrno (ELF_E_INVALID_DATA);
> - return NULL;
> - }
> -
> - return (ElfW2(LIBELFBITS,Chdr) *) d->d_buf;
> -}
> +#define ELF_WRLOCK_HELD 0
> +#include "elf32_getchdr.h"
> \ No newline at end of file
> diff --git a/libelf/elf32_getchdr.h b/libelf/elf32_getchdr.h
> new file mode 100644
> index 00000000..04d47e7a
> --- /dev/null
> +++ b/libelf/elf32_getchdr.h
> @@ -0,0 +1,61 @@
> +#undef ADD_ROUTINE_PREFIX
> +#undef ADD_ROUTINE_SUFFIX
> +
> +#if ELF_WRLOCK_HELD
> +#define CONCAT(x,y) x##y
> +#define ADD_ROUTINE_PREFIX(y) CONCAT(__,y)
> +#define ADD_ROUTINE_SUFFIX(x) x ## _wrlock
> +#define INTERNAL internal_function
> +#else
> +#define ADD_ROUTINE_PREFIX(y) y
> +#define ADD_ROUTINE_SUFFIX(x) x
> +#define INTERNAL
> +#endif
> +
> +ElfW2(LIBELFBITS,Chdr) *
> +INTERNAL
> +ADD_ROUTINE_PREFIX(elfw2(LIBELFBITS, ADD_ROUTINE_SUFFIX(getchdr))) (Elf_Scn *scn)
> +{
> +
> + ElfW2(LIBELFBITS,Shdr) *shdr = ADD_ROUTINE_PREFIX(elfw2(LIBELFBITS, ADD_ROUTINE_SUFFIX(getshdr)))(scn);
> +
> + if (shdr == NULL)
> + return NULL;
> +
> + /* Must have SHF_COMPRESSED flag set. Allocated or no bits sections
> + can never be compressed. */
> + if ((shdr->sh_flags & SHF_ALLOC) != 0)
> + {
> + __libelf_seterrno (ELF_E_INVALID_SECTION_FLAGS);
> + return NULL;
> + }
> +
> + if (shdr->sh_type == SHT_NULL
> + || shdr->sh_type == SHT_NOBITS)
> + {
> + __libelf_seterrno (ELF_E_INVALID_SECTION_TYPE);
> + return NULL;
> + }
> +
> + if ((shdr->sh_flags & SHF_COMPRESSED) == 0)
> + {
> + __libelf_seterrno (ELF_E_NOT_COMPRESSED);
> + return NULL;
> + }
> +
> + /* This makes sure the data is in the correct format, so we don't
> + need to swap fields. */
> + Elf_Data *d = ADD_ROUTINE_PREFIX(ADD_ROUTINE_SUFFIX(elf_getdata)) (scn, NULL);
> + if (d == NULL)
> + return NULL;
> +
> + if (d->d_size < sizeof (ElfW2(LIBELFBITS,Chdr)) || d->d_buf == NULL)
> + {
> + __libelf_seterrno (ELF_E_INVALID_DATA);
> + return NULL;
> + }
> +
> + return (ElfW2(LIBELFBITS,Chdr) *) d->d_buf;
> +}
It is interesting the general version and the version that is called
when the lock are held are totally separate functions. As far as I can
see that works out fine here.
Just surprising because in most other functions that have a wrlock
variant the general version takes a lock and then calls the wrlock
variant.
> +#undef INTERNAL
> +#undef ELF_WRLOCK_HELD
> \ No newline at end of file
> diff --git a/libelf/elf32_updatenull.c b/libelf/elf32_updatenull.c
> index c5d26b00..3594e8ba 100644
> --- a/libelf/elf32_updatenull.c
> +++ b/libelf/elf32_updatenull.c
> @@ -407,7 +407,7 @@ __elfw2(LIBELFBITS,updatenull_wrlock) (Elf *elf, int *change_bop, size_t shnum)
> else
> {
> ElfW2(LIBELFBITS,Chdr) *chdr;
> - chdr = elfw2(LIBELFBITS,getchdr) (scn);
> + chdr = __elfw2(LIBELFBITS,getchdr_wrlock) (scn);
> if (unlikely (chdr == NULL))
> return -1;
> sh_size = chdr->ch_size;
OK, so this is the only caller of the new getchdr_wrlock variants.
> diff --git a/libelf/elf_getdata.c b/libelf/elf_getdata.c
> index 5ebd270f..7c3ac043 100644
> --- a/libelf/elf_getdata.c
> +++ b/libelf/elf_getdata.c
> @@ -582,4 +582,18 @@ elf_getdata (Elf_Scn *scn, Elf_Data *data)
>
> return result;
> }
> +
> +Elf_Data *
> +internal_function
> +__elf_getdata_wrlock (Elf_Scn *scn, Elf_Data *data)
> +{
> + Elf_Data *result;
> +
> + if (scn == NULL)
> + return NULL;
> +
> + result = __elf_getdata_rdlock (scn, data);
> +
> + return result;
> +}
> INTDEF(elf_getdata)
Right. So now the getchdr_wrlock variants can call elf_getdata.
This is good. But I have been tinkering with trying to get rid of the
elf_getdata call from elf[32|64]_getchdr. It can result in a lot of
work (copying all of the compressed data) just to get the header. But
that is a work in progress (and would also change elf_compress a bit).
> diff --git a/libelf/libelfP.h b/libelf/libelfP.h
> index 96476064..ed061abb 100644
> --- a/libelf/libelfP.h
> +++ b/libelf/libelfP.h
> @@ -514,6 +514,8 @@ extern Elf32_Shdr *__elf32_getshdr_rdlock (Elf_Scn *__scn) internal_function;
> extern Elf64_Shdr *__elf64_getshdr_rdlock (Elf_Scn *__scn) internal_function;
> extern Elf32_Shdr *__elf32_getshdr_wrlock (Elf_Scn *__scn) internal_function;
> extern Elf64_Shdr *__elf64_getshdr_wrlock (Elf_Scn *__scn) internal_function;
> +extern Elf32_Chdr *__elf32_getchdr_wrlock (Elf_Scn *__scn) internal_function;
> +extern Elf64_Chdr *__elf64_getchdr_wrlock (Elf_Scn *__scn) internal_function;
> extern Elf_Scn *__elf_getscn_internal (Elf *__elf, size_t __index)
> attribute_hidden;
> extern Elf_Scn *__elf_nextscn_internal (Elf *__elf, Elf_Scn *__scn)
> @@ -523,6 +525,8 @@ extern Elf_Data *__elf_getdata_internal (Elf_Scn *__scn, Elf_Data *__data)
> attribute_hidden;
> extern Elf_Data *__elf_getdata_rdlock (Elf_Scn *__scn, Elf_Data *__data)
> internal_function;
> +extern Elf_Data *__elf_getdata_wrlock (Elf_Scn *__scn, Elf_Data *__data)
> + internal_function;
> extern Elf_Data *__elf_rawdata_internal (Elf_Scn *__scn, Elf_Data *__data)
> attribute_hidden;
> /* Should be called to setup first section data element if
Looks good with that noinstall_HEADERS fix.
Cheers,
Mark
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH 07/16] lib: Add eu_tsearch and eu_tfind
2023-10-10 13:42 ` [PATCH 07/16] lib: Add eu_tsearch and eu_tfind Mark Wielaard
@ 2023-10-10 16:51 ` Mark Wielaard
2023-10-17 20:52 ` Heather McIntyre
0 siblings, 1 reply; 51+ messages in thread
From: Mark Wielaard @ 2023-10-10 16:51 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2
Hi Heather,
On Tue, 2023-10-10 at 15:42 +0200, Mark Wielaard wrote:
> From: Heather McIntyre <hsm2@rice.edu>
>
> * lib/eu-search.h: New file.
> Declarations for read/write locked eu_tsearch/eu_tfind.
Like in the previous case, don't forget to add such a new header to
noinst_HEADERS so make dist adds them.
diff --git a/lib/Makefile.am b/lib/Makefile.am
index ce8f3e1b..9df0a8c6 100644
--- a/lib/Makefile.am
+++ b/lib/Makefile.am
@@ -39,5 +39,6 @@ libeu_a_SOURCES = xasprintf.c xstrdup.c xstrndup.c xmalloc.c next_prime.c \
noinst_HEADERS = fixedsizehash.h libeu.h system.h dynamicsizehash.h list.h \
eu-config.h color.h printversion.h bpf.h \
- atomics.h stdatomic-fbsd.h dynamicsizehash_concurrent.h
+ atomics.h stdatomic-fbsd.h dynamicsizehash_concurrent.h \
+ eu-search.h
EXTRA_DIST = dynamicsizehash.c dynamicsizehash_concurrent.c
> * lib/eu-search.c: New file.
> Definitions for read/write locked eu_tsearch/eu_tfind.
> * Makefile.am (libeu_a_SOURCES): Add eu-search.c.
>
> Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
> Signed-off-by: Mark Wielaard <mark@klomp.org>
> ---
> lib/Makefile.am | 2 +-
> lib/eu-search.c | 60 +++++++++++++++++++++++++++++++++++++++++++++++++
> lib/eu-search.h | 39 ++++++++++++++++++++++++++++++++
> 3 files changed, 100 insertions(+), 1 deletion(-)
> create mode 100644 lib/eu-search.c
> create mode 100644 lib/eu-search.h
>
> diff --git a/lib/Makefile.am b/lib/Makefile.am
> index b3bb929f..ce8f3e1b 100644
> --- a/lib/Makefile.am
> +++ b/lib/Makefile.am
> @@ -34,7 +34,7 @@ AM_CPPFLAGS += -I$(srcdir)/../libelf
> noinst_LIBRARIES = libeu.a
>
> libeu_a_SOURCES = xasprintf.c xstrdup.c xstrndup.c xmalloc.c next_prime.c \
> - crc32.c crc32_file.c \
> + crc32.c crc32_file.c eu-search.c \
> color.c error.c printversion.c
OK.
> noinst_HEADERS = fixedsizehash.h libeu.h system.h dynamicsizehash.h list.h \
> diff --git a/lib/eu-search.c b/lib/eu-search.c
> new file mode 100644
> index 00000000..a6b04f4f
> --- /dev/null
> +++ b/lib/eu-search.c
> @@ -0,0 +1,60 @@
> +/* Definitions for thread-safe tsearch/tfind
> + Copyright (C) 2023 Rice University
> + This file is part of elfutils.
> +
> + This file is free software; you can redistribute it and/or modify
> + it under the terms of either
> +
> + * the GNU Lesser General Public License as published by the Free
> + Software Foundation; either version 3 of the License, or (at
> + your option) any later version
> +
> + or
> +
> + * the GNU General Public License as published by the Free
> + Software Foundation; either version 2 of the License, or (at
> + your option) any later version
> +
> + or both in parallel, as here.
> +
> + elfutils is distributed in the hope that it will be useful, but
> + WITHOUT ANY WARRANTY; without even the implied warranty of
> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + General Public License for more details.
> +
> + You should have received copies of the GNU General Public License and
> + the GNU Lesser General Public License along with this program. If
> + not, see <http://www.gnu.org/licenses/>. */
> +
> +#ifdef HAVE_CONFIG_H
> +#include <config.h>
> +#endif
> +
> +#include <stdlib.h>
> +#include "eu-search.h"
> +
> +rwlock_define(static, search_find_lock);
> +
> +void *eu_tsearch(const void *key, void **rootp,
> + int (*compar)(const void *, const void *))
> +{
> + void *ret = NULL;
> + rwlock_wrlock(search_find_lock);
> +
> + ret = tsearch(key, rootp, compar);
> +
> + rwlock_unlock(search_find_lock);
> + return ret;
> +}
> +
> +void *eu_tfind(const void *key, void *const *rootp,
> + int (*compar)(const void *, const void *))
> +{
> + void *ret = NULL;
> + rwlock_rdlock(search_find_lock);
> +
> + ret = tfind(key, rootp, compar);
> +
> + rwlock_unlock(search_find_lock);
> + return ret;
> +}
So this uses on static lock for all eu-tsearch/eu-tfind calls. But
searches with different roots should be independent. Would it make
sense to add different locks for different roots?
That would make the code a little more complicated, but we could hide
the root and lock in a new structure that will be passed the the
eu_tsearch/eu_tfind calls.
Maybe something like:
struct eu_search_root
{
void *root;
rwlock_define (,lock);
};
With helper functions to create/init and destroy/delete them?
void eu_tinit (struct eu_search_root *search_root);
(Or is there no real initing to do?)
void eu_tdestroy (struct eu_search_root *search_root,
void (*free_node)(void *nodep));
Or is all this way too complicated and a global lock just fine?
Cheers,
Mark
> diff --git a/lib/eu-search.h b/lib/eu-search.h
> new file mode 100644
> index 00000000..4ce0139a
> --- /dev/null
> +++ b/lib/eu-search.h
> @@ -0,0 +1,39 @@
> +/* Calls for thread-safe tsearch/tfind
> + Copyright (C) 2023 Rice University
> + This file is part of elfutils.
> +
> + This file is free software; you can redistribute it and/or modify
> + it under the terms of either
> +
> + * the GNU Lesser General Public License as published by the Free
> + Software Foundation; either version 3 of the License, or (at
> + your option) any later version
> +
> + or
> +
> + * the GNU General Public License as published by the Free
> + Software Foundation; either version 2 of the License, or (at
> + your option) any later version
> +
> + or both in parallel, as here.
> +
> + elfutils is distributed in the hope that it will be useful, but
> + WITHOUT ANY WARRANTY; without even the implied warranty of
> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + General Public License for more details.
> +
> + You should have received copies of the GNU General Public License and
> + the GNU Lesser General Public License along with this program. If
> + not, see <http://www.gnu.org/licenses/>. */
> +
> +#ifndef EU_SEARCH_H
> +#define EU_SEARCH_H 1
> +
> +#include <search.h>
> +
> +extern void *eu_tsearch(const void *key, void **rootp,
> + int (*compar)(const void *, const void *));
> +extern void *eu_tfind(const void *key, void *const *rootp,
> + int (*compar)(const void *, const void *));
> +
> +#endif
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH 08/16] libcpu: Change calls for tsearch/tfind to eu_tsearch/eu_tfind.
2023-10-10 13:42 ` [PATCH 08/16] libcpu: Change calls for tsearch/tfind to eu_tsearch/eu_tfind Mark Wielaard
@ 2023-10-10 21:10 ` Mark Wielaard
0 siblings, 0 replies; 51+ messages in thread
From: Mark Wielaard @ 2023-10-10 21:10 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2
Hi Heather,
On Tue, Oct 10, 2023 at 03:42:52PM +0200, Mark Wielaard wrote:
> From: Heather McIntyre <hsm2@rice.edu>
>
> * libcpu/Makefile.am: Add USE_LOCKS condition for -pthread.
> * libcpu/i386_parse.y: Add eu-search.h and remove search.h.
> Change calls for tsearch/tfind to eu_tsearch/eu_tfind.
So in theory this looks like a simple obvious change to use eu-search
instead of search. But I am not sure ths bison parser can even be used
concurrently. It looks like it is using a lot of statics. Also
i386_parse.y uses twalk, which also would have to be wrapped (no other
part of elfutils uses twalk).
I think I would advise to skip the i386 assembler. It isn't really
complete at the moment and it is probably a lot of work to get a
concurrent bison parser.
Cheers,
Mark
> Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
> Signed-off-by: Mark Wielaard <mark@klomp.org>
> ---
> libcpu/Makefile.am | 3 +++
> libcpu/i386_parse.y | 48 ++++++++++++++++++++++-----------------------
> 2 files changed, 27 insertions(+), 24 deletions(-)
>
> diff --git a/libcpu/Makefile.am b/libcpu/Makefile.am
> index 4ba1be56..a51334b5 100644
> --- a/libcpu/Makefile.am
> +++ b/libcpu/Makefile.am
> @@ -33,6 +33,9 @@ AM_CPPFLAGS += -I$(srcdir)/../libelf -I$(srcdir)/../libebl \
> if BUILD_STATIC
> AM_CFLAGS += $(fpic_CFLAGS)
> endif
> +if USE_LOCKS
> + AM_CFLAGS += -pthread
> +endif
> AM_CFLAGS += -fdollars-in-identifiers
> LEXCOMPILE = $(LEX) $(LFLAGS) $(AM_LFLAGS) -P$(<F:lex.l=)
> LEX_OUTPUT_ROOT = lex.$(<F:lex.l=)
> diff --git a/libcpu/i386_parse.y b/libcpu/i386_parse.y
> index 459684c6..3d7cb89e 100644
> --- a/libcpu/i386_parse.y
> +++ b/libcpu/i386_parse.y
> @@ -37,7 +37,7 @@
> #include <inttypes.h>
> #include <math.h>
> #include <obstack.h>
> -#include <search.h>
> +#include <eu-search.h>
> #include <stdbool.h>
> #include <stdio.h>
> #include <stdlib.h>
> @@ -269,12 +269,12 @@ mask: kMASK kBITFIELD kNUMBER
> struct synonym *newp = xmalloc (sizeof (*newp));
> newp->from = $2;
> newp->to = $3;
> - if (tfind (newp, &synonyms, compare_syn) != NULL)
> + if (eu_tfind (newp, &synonyms, compare_syn) != NULL)
> error (0, 0,
> "%d: duplicate definition for synonym '%s'",
> i386_lineno, $2);
> - else if (tsearch ( newp, &synonyms, compare_syn) == NULL)
> - error (EXIT_FAILURE, 0, "tsearch");
> + else if (eu_tsearch ( newp, &synonyms, compare_syn) == NULL)
> + error (EXIT_FAILURE, 0, "eu_tsearch");
> }
> |
> ;
> @@ -308,12 +308,12 @@ instr: bytes ':' bitfieldopt kID bitfieldopt optargs
> newp->bytes = $1;
> newp->mnemonic = $4;
> if (newp->mnemonic != (void *) -1l
> - && tfind ($4, &mnemonics,
> + && eu_tfind ($4, &mnemonics,
> (int (*)(const void *, const void *)) strcmp) == NULL)
> {
> - if (tsearch ($4, &mnemonics,
> + if (eu_tsearch ($4, &mnemonics,
> (int (*)(const void *, const void *)) strcmp) == NULL)
> - error (EXIT_FAILURE, errno, "tsearch");
> + error (EXIT_FAILURE, errno, "eu_tsearch");
> ++nmnemonics;
> }
>
> @@ -339,15 +339,15 @@ instr: bytes ':' bitfieldopt kID bitfieldopt optargs
> infname, i386_lineno - 1, $5->name);
>
> struct suffix search = { .name = $5->name };
> - if (tfind (&search, &suffixes, compare_suf)
> + if (eu_tfind (&search, &suffixes, compare_suf)
> == NULL)
> {
> struct suffix *ns = xmalloc (sizeof (*ns));
> ns->name = $5->name;
> ns->idx = ++nsuffixes;
> - if (tsearch (ns, &suffixes, compare_suf)
> + if (eu_tsearch (ns, &suffixes, compare_suf)
> == NULL)
> - error (EXIT_FAILURE, errno, "tsearch");
> + error (EXIT_FAILURE, errno, "eu_tsearch");
> }
> }
>
> @@ -374,7 +374,7 @@ bitfieldopt: kBITFIELD
> struct known_bitfield search;
> search.name = $1;
> struct known_bitfield **res;
> - res = tfind (&search, &bitfields, bitfield_compare);
> + res = eu_tfind (&search, &bitfields, bitfield_compare);
> if (res == NULL)
> {
> error (0, 0, "%d: unknown bitfield '%s'",
> @@ -437,7 +437,7 @@ bit: '0'
> struct known_bitfield search;
> search.name = $1;
> struct known_bitfield **res;
> - res = tfind (&search, &bitfields, bitfield_compare);
> + res = eu_tfind (&search, &bitfields, bitfield_compare);
> if (res == NULL)
> {
> error (0, 0, "%d: unknown bitfield '%s'",
> @@ -497,7 +497,7 @@ argcomp: kBITFIELD
> struct known_bitfield search;
> search.name = $1;
> struct known_bitfield **res;
> - res = tfind (&search, &bitfields, bitfield_compare);
> + res = eu_tfind (&search, &bitfields, bitfield_compare);
> if (res == NULL)
> {
> if (strcmp ($1, "ax") == 0)
> @@ -575,7 +575,7 @@ new_bitfield (char *name, unsigned long int num)
> newp->bits = num;
> newp->tmp = 0;
>
> - if (tfind (newp, &bitfields, bitfield_compare) != NULL)
> + if (eu_tfind (newp, &bitfields, bitfield_compare) != NULL)
> {
> error (0, 0, "%d: duplicated definition of bitfield '%s'",
> i386_lineno, name);
> @@ -584,7 +584,7 @@ new_bitfield (char *name, unsigned long int num)
> return;
> }
>
> - if (tsearch (newp, &bitfields, bitfield_compare) == NULL)
> + if (eu_tsearch (newp, &bitfields, bitfield_compare) == NULL)
> error (EXIT_FAILURE, errno, "%d: cannot insert new bitfield '%s'",
> i386_lineno, name);
> }
> @@ -813,7 +813,7 @@ fillin_arg (struct bitvalue *bytes, struct argname *name,
>
> struct synonym search = { .from = fieldname };
>
> - struct synonym **res = tfind (&search, &synonyms, compare_syn);
> + struct synonym **res = eu_tfind (&search, &synonyms, compare_syn);
> if (res != NULL)
> fieldname = (*res)->to;
>
> @@ -914,26 +914,26 @@ find_numbers (void)
> if (runp->operands[i].fct != NULL)
> {
> struct argstring search = { .str = runp->operands[i].fct };
> - if (tfind (&search, &fct_names[i], compare_argstring) == NULL)
> + if (eu_tfind (&search, &fct_names[i], compare_argstring) == NULL)
> {
> struct argstring *newp = xmalloc (sizeof (*newp));
> newp->str = runp->operands[i].fct;
> newp->idx = 0;
> - if (tsearch (newp, &fct_names[i], compare_argstring) == NULL)
> - error (EXIT_FAILURE, errno, "tsearch");
> + if (eu_tsearch (newp, &fct_names[i], compare_argstring) == NULL)
> + error (EXIT_FAILURE, errno, "eu_tsearch");
> ++nfct_names[i];
> }
>
> if (runp->operands[i].str != NULL)
> {
> search.str = runp->operands[i].str;
> - if (tfind (&search, &strs[i], compare_argstring) == NULL)
> + if (eu_tfind (&search, &strs[i], compare_argstring) == NULL)
> {
> struct argstring *newp = xmalloc (sizeof (*newp));
> newp->str = runp->operands[i].str;
> newp->idx = 0;
> - if (tsearch (newp, &strs[i], compare_argstring) == NULL)
> - error (EXIT_FAILURE, errno, "tsearch");
> + if (eu_tsearch (newp, &strs[i], compare_argstring) == NULL)
> + error (EXIT_FAILURE, errno, "eu_tsearch");
> ++nstrs[i];
> }
> }
> @@ -1206,7 +1206,7 @@ instrtable_out (void)
> if (instr->operands[i].fct != NULL)
> {
> struct argstring search = { .str = instr->operands[i].fct };
> - struct argstring **res = tfind (&search, &fct_names[i],
> + struct argstring **res = eu_tfind (&search, &fct_names[i],
> compare_argstring);
> assert (res != NULL);
> idx = (*res)->idx;
> @@ -1217,7 +1217,7 @@ instrtable_out (void)
> if (instr->operands[i].str != NULL)
> {
> struct argstring search = { .str = instr->operands[i].str };
> - struct argstring **res = tfind (&search, &strs[i],
> + struct argstring **res = eu_tfind (&search, &strs[i],
> compare_argstring);
> assert (res != NULL);
> idx = (*res)->idx;
> --
> 2.41.0
>
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH 09/16] src: Use eu-search in nm and findtextrel.
2023-10-10 13:42 ` [PATCH 09/16] src: Use eu-search in nm and findtextrel Mark Wielaard
@ 2023-10-10 21:25 ` Mark Wielaard
2023-10-17 19:20 ` Heather McIntyre
0 siblings, 1 reply; 51+ messages in thread
From: Mark Wielaard @ 2023-10-10 21:25 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2
Hi Heather,
On Tue, Oct 10, 2023 at 03:42:53PM +0200, Mark Wielaard wrote:
> From: Heather McIntyre <hsm2@rice.edu>
>
> * src/Makefile.am: Add USE_LOCKS condition for -pthread.
> * src/findtextrel.c: Add eu-search.h and remove search.h.
> Change calls of tsearch/tfind to eu_tsearch/eu_tfind.
> * src/nm.c: Likewise.
This does look technically correct. But both nm and findtextrel are
single threaded programs. It might be interesting to try to make them
parallel. But currently I think this would just introduce unnecessary
locking.
Cheers,
Mark
> Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
> Signed-off-by: Mark Wielaard <mark@klomp.org>
> ---
> src/Makefile.am | 3 +++
> src/findtextrel.c | 10 +++++-----
> src/nm.c | 10 +++++-----
> 3 files changed, 13 insertions(+), 10 deletions(-)
>
> diff --git a/src/Makefile.am b/src/Makefile.am
> index 10d59a48..fea5d43e 100644
> --- a/src/Makefile.am
> +++ b/src/Makefile.am
> @@ -22,6 +22,9 @@ DEFS += $(YYDEBUG) -DDEBUGPRED=@DEBUGPRED@ \
> AM_CPPFLAGS += -I$(srcdir)/../libelf -I$(srcdir)/../libebl \
> -I$(srcdir)/../libdw -I$(srcdir)/../libdwelf \
> -I$(srcdir)/../libdwfl -I$(srcdir)/../libasm
> +if USE_LOCKS
> + AM_CFLAGS += -pthread
> +endif
>
> AM_LDFLAGS = -Wl,-rpath-link,../libelf:../libdw $(STACK_USAGE_NO_ERROR)
>
> diff --git a/src/findtextrel.c b/src/findtextrel.c
> index d3021a3a..5ac4c29a 100644
> --- a/src/findtextrel.c
> +++ b/src/findtextrel.c
> @@ -27,7 +27,7 @@
> #include <gelf.h>
> #include <libdw.h>
> #include <locale.h>
> -#include <search.h>
> +#include <eu-search.h>
> #include <stdbool.h>
> #include <stdio.h>
> #include <stdlib.h>
> @@ -501,10 +501,10 @@ check_rel (size_t nsegments, struct segments segments[nsegments],
> /* There can be more than one relocation against one file.
> Try to avoid multiple messages. And yes, the code uses
> pointer comparison. */
> - if (tfind (src, knownsrcs, ptrcompare) == NULL)
> + if (eu_tfind (src, knownsrcs, ptrcompare) == NULL)
> {
> printf (_("%s not compiled with -fpic/-fPIC\n"), src);
> - tsearch (src, knownsrcs, ptrcompare);
> + eu_tsearch (src, knownsrcs, ptrcompare);
> }
> return;
> }
> @@ -555,12 +555,12 @@ check_rel (size_t nsegments, struct segments segments[nsegments],
> if (sym->st_value + sym->st_size > addr)
> {
> /* It is this function. */
> - if (tfind (lowstr, knownsrcs, ptrcompare) == NULL)
> + if (eu_tfind (lowstr, knownsrcs, ptrcompare) == NULL)
> {
> printf (_("\
> the file containing the function '%s' is not compiled with -fpic/-fPIC\n"),
> lowstr);
> - tsearch (lowstr, knownsrcs, ptrcompare);
> + eu_tsearch (lowstr, knownsrcs, ptrcompare);
> }
> }
> else if (highidx == -1)
> diff --git a/src/nm.c b/src/nm.c
> index fbdee8e1..44c20fb2 100644
> --- a/src/nm.c
> +++ b/src/nm.c
> @@ -32,7 +32,7 @@
> #include <libdw.h>
> #include <locale.h>
> #include <obstack.h>
> -#include <search.h>
> +#include <eu-search.h>
> #include <stdbool.h>
> #include <stdio.h>
> #include <stdio_ext.h>
> @@ -537,7 +537,7 @@ static int
> get_global (Dwarf *dbg __attribute__ ((unused)), Dwarf_Global *global,
> void *arg __attribute__ ((unused)))
> {
> - tsearch (memcpy (xmalloc (sizeof (Dwarf_Global)), global,
> + eu_tsearch (memcpy (xmalloc (sizeof (Dwarf_Global)), global,
> sizeof (Dwarf_Global)),
> &global_root, global_compare);
>
> @@ -696,7 +696,7 @@ get_local_names (Dwarf *dbg)
> /* Check whether a similar local_name is already in the
> cache. That should not happen. But if it does, we
> don't want to leak memory. */
> - struct local_name **tres = tsearch (newp, &local_root,
> + struct local_name **tres = eu_tsearch (newp, &local_root,
> local_compare);
> if (tres == NULL)
> error_exit (errno, _("cannot create search tree"));
> @@ -1387,7 +1387,7 @@ show_symbols (int fd, Ebl *ebl, GElf_Ehdr *ehdr,
> && global_root != NULL)
> {
> Dwarf_Global fake = { .name = symstr };
> - Dwarf_Global **found = tfind (&fake, &global_root,
> + Dwarf_Global **found = eu_tfind (&fake, &global_root,
> global_compare);
> if (found != NULL)
> {
> @@ -1442,7 +1442,7 @@ show_symbols (int fd, Ebl *ebl, GElf_Ehdr *ehdr,
> .lowpc = sym->st_value,
> .highpc = sym->st_value,
> };
> - struct local_name **found = tfind (&fake, &local_root,
> + struct local_name **found = eu_tfind (&fake, &local_root,
> local_compare);
> if (found != NULL)
> {
> --
> 2.41.0
>
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH 10/16] libdw: make dwarf_getalt thread-safe
2023-10-10 13:42 ` [PATCH 10/16] libdw: make dwarf_getalt thread-safe Mark Wielaard
@ 2023-10-10 22:02 ` Mark Wielaard
2023-10-17 19:25 ` Heather McIntyre
0 siblings, 1 reply; 51+ messages in thread
From: Mark Wielaard @ 2023-10-10 22:02 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2
Hi Heather,
On Tue, Oct 10, 2023 at 03:42:54PM +0200, Mark Wielaard wrote:
> * libdw/dwarf_getalt.c: Add lock for
> dbg->alt_dwarf/main->alt_dwarf.
This takes care of dwarf_getalt. Shouldn't dwarf_setalt also use
locking?
> Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
> Signed-off-by: Mark Wielaard <mark@klomp.org>
> ---
> libdw/dwarf_getalt.c | 27 ++++++++++++++++++++++-----
> 1 file changed, 22 insertions(+), 5 deletions(-)
>
> diff --git a/libdw/dwarf_getalt.c b/libdw/dwarf_getalt.c
> index 0a12dfae..e3894c8c 100644
> --- a/libdw/dwarf_getalt.c
> +++ b/libdw/dwarf_getalt.c
> @@ -44,6 +44,10 @@
> #include <sys/types.h>
> #include <sys/stat.h>
>
> +/* find_debug_altlink() modifies "dbg->alt_dwarf".
> + dwarf_getalt() reads "main->alt_dwarf".
> + Mutual exclusion is enforced to prevent a race. */
> +rwlock_define(static, alt_dwarf_lock);
Probably overkill, but should we consider a Dwarf lock object instead
of having a static global lock?
> char *
> internal_function
> @@ -152,7 +156,9 @@ find_debug_altlink (Dwarf *dbg)
> Dwarf *alt = dwarf_begin (fd, O_RDONLY);
> if (alt != NULL)
> {
> + rwlock_wrlock(alt_dwarf_lock);
> dbg->alt_dwarf = alt;
> + rwlock_unlock(alt_dwarf_lock);
> dbg->alt_fd = fd;
> }
> else
Is this lock wide enough? See also below. It looks like multiple
threads could arrive at this point at the same time. so alt_dwarf and
alt_fd can be (re)set multiple times, causing leaks of the Dwarf and
fd.
> @@ -163,22 +169,33 @@ find_debug_altlink (Dwarf *dbg)
> Dwarf *
> dwarf_getalt (Dwarf *main)
> {
> + rwlock_rdlock(alt_dwarf_lock);
> + Dwarf *alt_dwarf_local = main->alt_dwarf;
> + rwlock_unlock(alt_dwarf_lock);
> +
> /* Only try once. */
> - if (main == NULL || main->alt_dwarf == (void *) -1)
> + if (main == NULL || alt_dwarf_local == (void *) -1)
> return NULL;
>
> - if (main->alt_dwarf != NULL)
> - return main->alt_dwarf;
> + if (alt_dwarf_local != NULL)
> + return alt_dwarf_local;
>
So at this point it looks like we can have multiple threads running
(because the lock has been dropped above) all with alt_dwarf_local
(and main->alt_dwarf) being NULL.
> find_debug_altlink (main);
>
> + rwlock_rdlock(alt_dwarf_lock);
> + alt_dwarf_local = main->alt_dwarf;
> + rwlock_unlock(alt_dwarf_lock);
> +
> /* If we found nothing, make sure we don't try again. */
> - if (main->alt_dwarf == NULL)
> + if (alt_dwarf_local == NULL)
> {
> + rwlock_wrlock(alt_dwarf_lock);
> main->alt_dwarf = (void *) -1;
> + rwlock_unlock(alt_dwarf_lock);
> +
> return NULL;
> }
>
> - return main->alt_dwarf;
> + return alt_dwarf_local;
> }
> INTDEF (dwarf_getalt)
> --
> 2.41.0
>
It might be better to take the lock over the whole function (and only
call find_debug_altlink with the lock held).
Cheers,
Mark
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH 11/16] libdw: Add locking around __libdw_dieabbrev for dwarf_hasattr
2023-10-10 13:42 ` [PATCH 11/16] libdw: Add locking around __libdw_dieabbrev for dwarf_hasattr Mark Wielaard
@ 2023-10-11 15:10 ` Mark Wielaard
2023-10-17 19:57 ` Heather McIntyre
1 sibling, 0 replies; 51+ messages in thread
From: Mark Wielaard @ 2023-10-11 15:10 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2
Hi Heather,
On Tue, 2023-10-10 at 15:42 +0200, Mark Wielaard wrote:
> From: Heather McIntyre <hsm2@rice.edu>
>
> * libdw/dwarf_hasattr.c (dwarf_hasattr): Use die_abbrev_lock
> around __libdw_dieabbrev call.
>
> Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
> Signed-off-by: Mark Wielaard <mark@klomp.org>
> ---
> libdw/dwarf_hasattr.c | 9 +++++++++
> 1 file changed, 9 insertions(+)
>
> diff --git a/libdw/dwarf_hasattr.c b/libdw/dwarf_hasattr.c
> index eca08394..92f8de68 100644
> --- a/libdw/dwarf_hasattr.c
> +++ b/libdw/dwarf_hasattr.c
> @@ -34,6 +34,10 @@
> #include <dwarf.h>
> #include "libdwP.h"
>
> +/* dwarf_hasattr() calls __libdw_dieabbrev() in libdwP.h.
> + __libdw_dieabbrev() reads/writes "die->abbrev".
> + Mutual exclusion is enforced around the call to __libdw_dieabbrev to prevent a race. */
> +rwlock_define(static, die_abbrev_lock);
dwarf_child, dwarf_getattrs, dwarf_haschildren and dwarf_tag also use
__libdw_dieabbrev to get the Dwarf_Abbrev pointer for the given
Dwarf_DIE. Shouldn't they also use such locking? Or have the locking
inside __libdw_dieabbrev itself?
Also there are many Dwarf_Dies which all start out "lazy" without
abbrev set. So taking a global static lock, or even taking any pthread
lock at all might be a big overhead. Is there some way we can do this
with atomics instead?
>
> int
> dwarf_hasattr (Dwarf_Die *die, unsigned int search_name)
> @@ -41,8 +45,13 @@ dwarf_hasattr (Dwarf_Die *die, unsigned int search_name)
> if (die == NULL)
> return 0;
>
> + rwlock_wrlock(die_abbrev_lock);
> +
> /* Find the abbreviation entry. */
> Dwarf_Abbrev *abbrevp = __libdw_dieabbrev (die, NULL);
> +
> + rwlock_unlock(die_abbrev_lock);
> +
> if (unlikely (abbrevp == DWARF_END_ABBREV))
> {
> __libdw_seterrno (DWARF_E_INVALID_DWARF);
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH 12/16] libdw: Make libdw_find_split_unit thread-safe
2023-10-10 13:42 ` [PATCH 12/16] libdw: Make libdw_find_split_unit thread-safe Mark Wielaard
@ 2023-10-11 17:17 ` Mark Wielaard
2023-10-17 20:01 ` Heather McIntyre
0 siblings, 1 reply; 51+ messages in thread
From: Mark Wielaard @ 2023-10-11 17:17 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2
Hi Heather,
On Tue, Oct 10, 2023 at 03:42:56PM +0200, Mark Wielaard wrote:
> From: Heather McIntyre <hsm2@rice.edu>
>
> * (try_split_file): Use eu_tsearch.
> Add lock for cu->split.
>
> Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
> Signed-off-by: Mark Wielaard <mark@klomp.org>
> ---
> libdw/libdw_find_split_unit.c | 43 +++++++++++++++++++++++++++++------
> 1 file changed, 36 insertions(+), 7 deletions(-)
>
> diff --git a/libdw/libdw_find_split_unit.c b/libdw/libdw_find_split_unit.c
> index 533f8072..a288e9bd 100644
> --- a/libdw/libdw_find_split_unit.c
> +++ b/libdw/libdw_find_split_unit.c
> @@ -34,13 +34,19 @@
> #include "libelfP.h"
>
> #include <limits.h>
> -#include <search.h>
> +#include <eu-search.h>
> #include <stdlib.h>
> #include <string.h>
> #include <sys/types.h>
> #include <sys/stat.h>
> #include <fcntl.h>
>
> +/* __libdw_link_skel_split() modifies "skel->split = split"
> + and "split->split = skel".
> + "cu->split" is read at multiple locations and conditionally updated.
> + Mutual exclusion is enforced to prevent a race. */
> +rwlock_define(static, cu_split_lock);
__libdw_link_skel_split is defined in libdw/libdwP.h and is called in
try_split_file. (It is also called in src/readelf.c but that is a
terrible hack, so ignore that for now.)
A Dwarf_CU is created (see libdw/libdw_findcu.c) with split set to
(Dwarf_CU *) -1 to indicate the split dwarf is not yet set. If
cu->split is NULL it means the split dwarf was searched for, but not
found.
> static void
> try_split_file (Dwarf_CU *cu, const char *dwo_path)
> {
> @@ -57,7 +63,7 @@ try_split_file (Dwarf_CU *cu, const char *dwo_path)
> if (split->unit_type == DW_UT_split_compile
> && cu->unit_id8 == split->unit_id8)
> {
> - if (tsearch (split->dbg, &cu->dbg->split_tree,
> + if (eu_tsearch (split->dbg, &cu->dbg->split_tree,
> __libdw_finddbg_cb) == NULL)
> {
> /* Something went wrong. Don't link. */
> @@ -65,9 +71,13 @@ try_split_file (Dwarf_CU *cu, const char *dwo_path)
> break;
> }
>
> + rwlock_wrlock(cu_split_lock);
> +
> /* Link skeleton and split compile units. */
> __libdw_link_skel_split (cu, split);
>
> + rwlock_unlock(cu_split_lock);
> +
Is this locking wide enough? It seems like multiple threads can race
to this code and set different Dwarfs (which means they'll leak). See
below.
> /* We have everything we need from this ELF
> file. And we are going to close the fd to
> not run out of file descriptors. */
> @@ -75,8 +85,13 @@ try_split_file (Dwarf_CU *cu, const char *dwo_path)
> break;
> }
> }
> +
> + rwlock_rdlock(cu_split_lock);
> +
> if (cu->split == (Dwarf_CU *) -1)
> dwarf_end (split_dwarf);
> +
> + rwlock_unlock(cu_split_lock);
This means __libdw_link_skel_split wasn't called above (so the
split_dwarf created by dwarf_begin didn't contain the expected CU).
> }
> /* Always close, because we don't want to run out of file
> descriptors. See also the elf_fcntl ELF_C_FDDONE call
> @@ -89,9 +104,13 @@ Dwarf_CU *
> internal_function
> __libdw_find_split_unit (Dwarf_CU *cu)
> {
> + rwlock_rdlock(cu_split_lock);
> + Dwarf_CU *cu_split_local = cu->split;
> + rwlock_unlock(cu_split_lock);
> +
> /* Only try once. */
> - if (cu->split != (Dwarf_CU *) -1)
> - return cu->split;
> + if (cu_split_local != (Dwarf_CU *) -1)
> + return cu_split_local;
>
> /* We need a skeleton unit with a comp_dir and [GNU_]dwo_name attributes.
> The split unit will be the first in the dwo file and should have the
> @@ -116,7 +135,11 @@ __libdw_find_split_unit (Dwarf_CU *cu)
> free (dwo_path);
> }
>
> - if (cu->split == (Dwarf_CU *) -1)
> + rwlock_rdlock(cu_split_lock);
> + cu_split_local = cu->split;
> + rwlock_unlock(cu_split_lock);
> +
> + if (cu_split_local == (Dwarf_CU *) -1)
> {
> /* Try compdir plus dwo_name. */
> Dwarf_Attribute compdir;
It looks like multiple threads can end up here after having seen
cu->split/cu_split_local == (Dwarf_CU *) -1) Which means multiple
threads will enter try_split_file and might all call
__libdw_link_skel_split as mentioned above.
> @@ -138,9 +161,15 @@ __libdw_find_split_unit (Dwarf_CU *cu)
> }
> }
>
> + rwlock_wrlock(cu_split_lock);
> +
> /* If we found nothing, make sure we don't try again. */
> if (cu->split == (Dwarf_CU *) -1)
> - cu->split = NULL;
> + {
> + cu->split = NULL;
> + cu_split_local = cu->split;
> + }
>
> - return cu->split;
> + rwlock_unlock(cu_split_lock);
> + return cu_split_local;
> }
I think it would be better to keep the lock during the whole
__libdw_find_split_unit call.
Cheers,
Mark
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH 13/16] libdw: Make libdw_findcu thread-safe
2023-10-10 13:42 ` [PATCH 13/16] libdw: Make libdw_findcu thread-safe Mark Wielaard
@ 2023-10-12 22:02 ` Mark Wielaard
2023-10-17 20:10 ` Heather McIntyre
0 siblings, 1 reply; 51+ messages in thread
From: Mark Wielaard @ 2023-10-12 22:02 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2
Hi Heather,
On Tue, Oct 10, 2023 at 03:42:57PM +0200, Mark Wielaard wrote:
> From: Heather McIntyre <hsm2@rice.edu>
>
> * libdw/libdw_findcu.c (findcu_cb): Use eu_tsearch.
> (__libdw_findcu): Use eu_tfind and next_tucu_offset_lock.
> (__libdw_findcu_addr): Use eu_tfind.
> (__libdw_find_split_dbg_addr): Likewise.
>
> Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
> Signed-off-by: Mark Wielaard <mark@klomp.org>
> ---
> libdw/libdw_findcu.c | 54 ++++++++++++++++++++++++++++----------------
> 1 file changed, 35 insertions(+), 19 deletions(-)
>
> diff --git a/libdw/libdw_findcu.c b/libdw/libdw_findcu.c
> index ed744231..e546fb0f 100644
> --- a/libdw/libdw_findcu.c
> +++ b/libdw/libdw_findcu.c
> @@ -32,9 +32,13 @@
> #endif
>
> #include <assert.h>
> -#include <search.h>
> +#include <eu-search.h>
> #include "libdwP.h"
>
> +/* __libdw_findcu modifies "&dbg->next_tu_offset : &dbg->next_cu_offset".
> + May read or write, so mutual exclusion is enforced to prevent a race. */
> +rwlock_define(static, next_tucu_offset_lock);
> +
Could this be a per struct Dwarf lock?
> static int
> findcu_cb (const void *arg1, const void *arg2)
> {
> @@ -213,7 +217,7 @@ __libdw_intern_next_unit (Dwarf *dbg, bool debug_types)
>
> Dwarf_Sig8_Hash_insert (&dbg->sig8_hash, unit_id8, newp);
>
> /* Add the new entry to the search tree. */
> - if (tsearch (newp, tree, findcu_cb) == NULL)
> + if (eu_tsearch (newp, tree, findcu_cb) == NULL)
> {
> /* Something went wrong. Undo the operation. */
> *offsetp = oldoff;
This is OK since when __libdw_intern_next_unit is called from
__libdw_findcu the next_tucu_offset_lock is held.
But there is also a call to __libdw_intern_next_unit from
dwarf_formref_die. So there should also be a lock there?
> @@ -234,28 +238,40 @@ __libdw_findcu (Dwarf *dbg, Dwarf_Off start, bool v4_debug_types)
>
> /* Maybe we already know that CU. */
> struct Dwarf_CU fake = { .start = start, .end = 0 };
> - struct Dwarf_CU **found = tfind (&fake, tree, findcu_cb);
> + struct Dwarf_CU **found = eu_tfind (&fake, tree, findcu_cb);
> + struct Dwarf_CU *result = NULL;
> +
> if (found != NULL)
> return *found;
>
> - if (start < *next_offset)
> - {
> - __libdw_seterrno (DWARF_E_INVALID_DWARF);
> - return NULL;
> - }
> + rwlock_wrlock(next_tucu_offset_lock);
>
> - /* No. Then read more CUs. */
> - while (1)
> + if (start < *next_offset)
> + __libdw_seterrno (DWARF_E_INVALID_DWARF);
> + else
> {
> - struct Dwarf_CU *newp = __libdw_intern_next_unit (dbg, v4_debug_types);
> - if (newp == NULL)
> - return NULL;
> + /* No. Then read more CUs. */
> + while (1)
> + {
> + struct Dwarf_CU *newp = __libdw_intern_next_unit (dbg,
> + v4_debug_types);
> + if (newp == NULL)
> + {
> + result = NULL;
> + break;
> + }
>
> - /* Is this the one we are looking for? */
> - if (start < *next_offset || start == newp->start)
> - return newp;
> + /* Is this the one we are looking for? */
> + if (start < *next_offset || start == newp->start)
> + {
> + result = newp;
> + break;
> + }
> + }
> }
> - /* NOTREACHED */
> +
> + rwlock_unlock(next_tucu_offset_lock);
> + return result;
> }
This look OK.
> struct Dwarf_CU *
> @@ -283,7 +299,7 @@ __libdw_findcu_addr (Dwarf *dbg, void *addr)
> return NULL;
>
> struct Dwarf_CU fake = { .start = start, .end = 0 };
> - struct Dwarf_CU **found = tfind (&fake, tree, findcu_cb);
> + struct Dwarf_CU **found = eu_tfind (&fake, tree, findcu_cb);
>
> if (found != NULL)
> return *found;
> @@ -298,7 +314,7 @@ __libdw_find_split_dbg_addr (Dwarf *dbg, void *addr)
> /* XXX Assumes split DWARF only has CUs in main IDX_debug_info. */
> Elf_Data fake_data = { .d_buf = addr, .d_size = 0 };
> Dwarf fake = { .sectiondata[IDX_debug_info] = &fake_data };
> - Dwarf **found = tfind (&fake, &dbg->split_tree, __libdw_finddbg_cb);
> + Dwarf **found = eu_tfind (&fake, &dbg->split_tree, __libdw_finddbg_cb);
>
> if (found != NULL)
> return *found;
OK.
Thanks,
Mark
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH 14/16] libdw,libdwfl: Use eu-search for thread-safety
2023-10-10 13:42 ` [PATCH 14/16] libdw,libdwfl: Use eu-search for thread-safety Mark Wielaard
@ 2023-10-12 22:05 ` Mark Wielaard
0 siblings, 0 replies; 51+ messages in thread
From: Mark Wielaard @ 2023-10-12 22:05 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2
Hi Heather,
On Tue, Oct 10, 2023 at 03:42:58PM +0200, Mark Wielaard wrote:
>
> * libdw/cie.c: Add eu-search.h and remove search.h.
> Change calls of tsearch/tfind to eu_tsearch/eu_tfind.
> * libdw/dwarf_getlocation.c: Likewise.
> * libdw/dwarf_getmacros.c: Likewise.
> * libdw/dwarf_getsrclines.c: Likewise.
> * libdw/fde.c: Likewise.
> * libdwfl/cu.c: Likewise.
These all look good. Just have to think about whether the global
static lock currently used in eu-search is the right approach.
Cheers,
Mark
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH 16/16] configure: No longer mark --enable-thread-safety as EXPERIMENTAL
2023-10-10 13:43 ` [PATCH 16/16] configure: No longer mark --enable-thread-safety as EXPERIMENTAL Mark Wielaard
@ 2023-10-12 22:09 ` Mark Wielaard
0 siblings, 0 replies; 51+ messages in thread
From: Mark Wielaard @ 2023-10-12 22:09 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2
Hi Heather,
On Tue, Oct 10, 2023 at 03:43:00PM +0200, Mark Wielaard wrote:
>
> * configure.ac (--enable-thread-safety): Remove experimental
> warning.
>
This all looks good. We should push this once we have all libelf fixes
in.
BTW. Your original ChangeLog entry also mentioned a NEWS change, but
that wasn't in your original patch.
Cheers,
Mark
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH 15/16] tests: Add eu-search tests
2023-10-10 13:42 ` [PATCH 15/16] tests: Add eu-search tests Mark Wielaard
@ 2023-10-13 14:38 ` Mark Wielaard
0 siblings, 0 replies; 51+ messages in thread
From: Mark Wielaard @ 2023-10-13 14:38 UTC (permalink / raw)
To: elfutils-devel; +Cc: hsm2
Hi Heather,
On Tue, 2023-10-10 at 15:42 +0200, Mark Wielaard wrote:
> From: Heather McIntyre <hsm2@rice.edu>
>
> * tests/eu_search_cfi.c: New file.
> * tests/eu_search_die.c: New file.
> * tests/eu_search_lines.c: New file.
> * tests/eu_search_macros.c: New file.
> * tests/run-eu-search-tests.sh: New test.
> * tests/Makefile.am: Add USE_LOCKS condition for -pthread.
> (check_PROGRAMS): Add eu_search_cfi, eu_search_die,
> eu_search_lines, and eu_search_macros.
> (TESTS): Add run-eu-search-tests.sh
> (eu_search_cfi_LDADD): New variable.
> (eu_search_die_LDADD): New variable.
> (eu_search_lines_LDADD): New variable.
> (eu_search_macros_LDADD): New variable.
> (eu_search_cfi_LDFLAGS): New variable.
> Add -pthread if USE_LOCKS is not defined.
> (eu_search_die_LDFLAGS): Likewise.
> (eu_search_lines_LDFLAGS): Likewise.
> (eu_search_macros_LDFLAGS): Likewise.
>
> Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
> Signed-off-by: Mark Wielaard <mark@klomp.org>
> ---
> tests/Makefile.am | 16 ++-
> tests/eu_search_cfi.c | 234 +++++++++++++++++++++++++++++++
> tests/eu_search_die.c | 262 +++++++++++++++++++++++++++++++++++
> tests/eu_search_lines.c | 228 ++++++++++++++++++++++++++++++
> tests/eu_search_macros.c | 216 +++++++++++++++++++++++++++++
> tests/run-eu-search-tests.sh | 168 ++++++++++++++++++++++
> 6 files changed, 1123 insertions(+), 1 deletion(-)
> create mode 100644 tests/eu_search_cfi.c
> create mode 100644 tests/eu_search_die.c
> create mode 100644 tests/eu_search_lines.c
> create mode 100644 tests/eu_search_macros.c
> create mode 100755 tests/run-eu-search-tests.sh
>
> diff --git a/tests/Makefile.am b/tests/Makefile.am
> index 32b18e6e..a66b1033 100644
> --- a/tests/Makefile.am
> +++ b/tests/Makefile.am
> @@ -32,6 +32,10 @@ else
> tests_rpath = no
> endif
>
> +if USE_LOCKS
> + AM_CFLAGS += -pthread
> +endif
Below you add explicit -pthread to the new eu_search tests.
Do we need this globally in AM_CFLAGS as well?
> check_PROGRAMS = arextract arsymtest newfile saridx scnnames sectiondump \
> showptable update1 update2 update3 update4 test-nlist \
> show-die-info get-files next-files get-lines next-lines \
> @@ -63,6 +67,7 @@ check_PROGRAMS = arextract arsymtest newfile saridx scnnames sectiondump \
> getphdrnum leb128 read_unaligned \
> msg_tst system-elf-libelf-test system-elf-gelf-test \
> nvidia_extended_linemap_libdw \
> + eu_search_cfi eu_search_die eu_search_lines eu_search_macros \
> $(asm_TESTS)
>
> asm_TESTS = asm-tst1 asm-tst2 asm-tst3 asm-tst4 asm-tst5 \
> @@ -211,7 +216,8 @@ TESTS = run-arextract.sh run-arsymtest.sh run-ar.sh newfile test-nlist \
> $(asm_TESTS) run-disasm-bpf.sh run-low_high_pc-dw-form-indirect.sh \
> run-nvidia-extended-linemap-libdw.sh run-nvidia-extended-linemap-readelf.sh \
> run-readelf-dw-form-indirect.sh run-strip-largealign.sh \
> - run-readelf-Dd.sh
> + run-readelf-Dd.sh \
> + run-eu-search-tests.sh
Correct.
But run-eu-search-tests.sh also needs to be added to EXTRA_DISTS to
make sure it gets shipped when doing make dist (make distcheck would
point that out, although with an obscure error).
diff --git a/tests/Makefile.am b/tests/Makefile.am
index a66b1033..09bfe405 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -635,7 +631,7 @@ EXTRA_DIST = run-arextract.sh run-arsymtest.sh run-ar.sh \
run-nvidia-extended-linemap-libdw.sh run-nvidia-extended-linemap-readelf.sh \
testfile_nvidia_linemap.bz2 \
testfile-largealign.o.bz2 run-strip-largealign.sh \
- run-funcretval++11.sh
+ run-funcretval++11.sh run-eu-search-tests.sh
if USE_VALGRIND
> if !BIARCH
> export ELFUTILS_DISABLE_BIARCH = 1
> @@ -804,6 +810,14 @@ getphdrnum_LDADD = $(libelf) $(libdw)
> leb128_LDADD = $(libelf) $(libdw)
> read_unaligned_LDADD = $(libelf) $(libdw)
> nvidia_extended_linemap_libdw_LDADD = $(libelf) $(libdw)
> +eu_search_cfi_LDFLAGS = $(if $(filter undefined,$(origin USE_LOCKS)),-pthread) $(AM_LDFLAGS)
> +eu_search_die_LDFLAGS = $(if $(filter undefined,$(origin USE_LOCKS)),-pthread) $(AM_LDFLAGS)
> +eu_search_lines_LDFLAGS = $(if $(filter undefined,$(origin USE_LOCKS)),-pthread) $(AM_LDFLAGS)
> +eu_search_macros_LDFLAGS = $(if $(filter undefined,$(origin USE_LOCKS)),-pthread) $(AM_LDFLAGS)
> +eu_search_cfi_LDADD = $(libeu) $(libelf) $(libdw)
> +eu_search_die_LDADD = $(libdw)
> +eu_search_lines_LDADD = $(libdw) $(libelf)
> +eu_search_macros_LDADD = $(libdw)
>
> # We want to test the libelf headers against the system elf.h header.
> # Don't include any -I CPPFLAGS. Except when we install our own elf.h.
OK. Although you might not need to conditionalize it on USE_LOCKS. It
can always be compiled with -pthread, the test run will just be skipped
if USE_LOCKS isn't set.
> diff --git a/tests/eu_search_cfi.c b/tests/eu_search_cfi.c
> new file mode 100644
> index 00000000..0b63b213
> --- /dev/null
> +++ b/tests/eu_search_cfi.c
> @@ -0,0 +1,234 @@
> +/*Test program for eu_search_cfi
> + Copyright (C) 2023 Red Hat, Inc.
> + This file is part of elfutils.
Forgot to update the copyright notice as we discussed offlist. Sorry.
Will do so.
And maybe want to reindent to follow code style.
Skipping actual test code inspection, but see below on usage.
> + This file is free software; you can redistribute it and/or modify
> + it under the terms of the GNU General Public License as published by
> + the Free Software Foundation; either version 3 of the License, or
> + (at your option) any later version.
> +
> + elfutils is distributed in the hope that it will be useful, but
> + WITHOUT ANY WARRANTY; without even the implied warranty of
> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + GNU General Public License for more details.
> +
> + You should have received a copy of the GNU General Public License
> + along with this program. If not, see<http://www.gnu.org/licenses/>. */
> +
> +#include <config.h>
> +#include <assert.h>
> +#include <inttypes.h>
> +#include ELFUTILS_HEADER(dw)
> +#include <dwarf.h>
> +#include <argp.h>
> +#include <stdbool.h>
> +#include <stdio.h>
> +#include <sys/types.h>
> +#include <sys/stat.h>
> +#include <fcntl.h>
> +#include <locale.h>
> +#include <stdlib.h>
> +#include <string.h>
> +#include <unistd.h>
> +#include "system.h"
> +#include <pthread.h>
> +
> +static void handle_section(char *name, const unsigned char e_ident[], Elf_Scn *scn, const bool is_eh);
> +static void *thread_work(void *arg);
> +
> +typedef struct
> +{
> + char *name;
> + const unsigned char *e_ident;
> + Elf_Scn * scn;
> + bool is_eh;
> +}
> +
> +ThreadData;
> +
> +static void *thread_work(void *arg)
> +{
> + ThreadData *data = (ThreadData*) arg;
> + handle_section(data->name, data->e_ident, data->scn, data->is_eh);
> + free(data);
> + return NULL;
> +}
> +
> +static void handle_section(char *name, const unsigned char e_ident[],
> + Elf_Scn *scn, const bool is_eh)
> +{
> + if (is_eh)
> + printf(".eh_frame\n");
> + else
> + printf(".debug_frame\n");
> +
> + GElf_Shdr mem;
> + GElf_Shdr *shdr = gelf_getshdr(scn, &mem);
> + if (shdr == NULL)
> + error(EXIT_FAILURE, 0, "Couldn't get section header: %s",
> + elf_errmsg(-1));
> + if ((shdr->sh_flags &SHF_COMPRESSED) != 0)
> + {
> + if (elf_compress(scn, 0, 0) < 0)
> + error(EXIT_FAILURE, 0, "Couldn't decompress section: %s",
> + elf_errmsg(-1));
> + }
> + else if (name[0] == '.' && name[1] == 'z')
> + {
> + if (elf_compress_gnu(scn, 0, 0) < 0)
> + error(EXIT_FAILURE, 0, "Couldn't decompress section: %s",
> + elf_errmsg(-1));
> + }
> +
> + Elf_Data *data = elf_getdata(scn, NULL);
> + if (data == NULL || data->d_buf == NULL)
> + error(EXIT_FAILURE, 0, "no section data");
> +
> + int res;
> + Dwarf_Off off;
> + Dwarf_Off next_off = 0;
> + Dwarf_CFI_Entry entry;
> + while ((res = dwarf_next_cfi(e_ident, data, is_eh, off = next_off, &next_off, &entry)) == 0)
> + {
> + printf("[%" PRId64 "] ", off);
> + if (dwarf_cfi_cie_p(&entry))
> + printf("CIE augmentation=\"%s\"\n", entry.cie.augmentation);
> + else
> + {
> + printf("FDE cie=[%" PRId64 "]\n", entry.fde.CIE_pointer);
> +
> + Dwarf_Off cie_off = entry.fde.CIE_pointer;
> + Dwarf_Off cie_off_next;
> + Dwarf_CFI_Entry cie_entry;
> + if (dwarf_next_cfi(e_ident, data, is_eh, cie_off, &cie_off_next, &cie_entry) != 0 ||
> + !dwarf_cfi_cie_p(&cie_entry))
> + error(EXIT_FAILURE, 0, "FDE doesn't point to CIE");
> + }
> + }
> +
> + if (res < 0)
> + error(EXIT_FAILURE, 0, "dwarf_next_cfi failed: %s\n",
> + dwarf_errmsg(-1));
> +}
> +
> +int main(int argc, char *argv[])
> +{
> + if (argc != 2)
> + error(EXIT_FAILURE, 0, "need file name argument");
> +
> + const char *file = argv[1];
> + printf("%s\n", file);
> +
> + int fd = open(file, O_RDONLY);
> + if (fd == -1)
> + error(EXIT_FAILURE, errno, "cannot open input file `%s'", file);
> +
> + elf_version(EV_CURRENT);
> +
> + Elf *elf = elf_begin(fd, ELF_C_READ, NULL);
> + if (elf == NULL)
> + error(EXIT_FAILURE, 0, "cannot create ELF descriptor: %s", elf_errmsg(-1));
> +
> + size_t esize;
> + const unsigned char *ident = (const unsigned char *) elf_getident(elf, &esize);
> + if (ident == NULL || esize < EI_NIDENT)
> + error(EXIT_FAILURE, 0, "no, or too small, ELF ident");
> +
> + GElf_Ehdr ehdr;
> + if (gelf_getehdr(elf, &ehdr) == NULL)
> + error(EXIT_FAILURE, 0, "cannot get the ELF header: %s\n", elf_errmsg(-1));
> +
> + size_t strndx = ehdr.e_shstrndx;
> +
> + int num_threads = 4;
> + pthread_t *threads = (pthread_t*) malloc(num_threads * sizeof(pthread_t));
> + ThreadData **thread_data = (ThreadData**) malloc(num_threads * sizeof(ThreadData*));
> + int thread_count = 0;
> +
> + if (!threads || !thread_data)
> + {
> + fprintf(stderr, "Failed to allocate memory for threads.\n");
> + free(threads);
> + free(thread_data);
> + return 1;
> + }
> +
> + Elf_Scn *scn = NULL;
> + while ((scn = elf_nextscn(elf, scn)) != NULL)
> + {
> + GElf_Shdr shdr;
> + if (gelf_getshdr(scn, &shdr) != NULL)
> + {
> + char *name = elf_strptr(elf, strndx, (size_t) shdr.sh_name);
> + if (name != NULL && shdr.sh_type == SHT_PROGBITS)
> + {
> + bool is_eh = false;
> + if (strcmp(name, ".eh_frame") == 0)
> + {
> + is_eh = true;
> + }
> + else if (strcmp(name, ".debug_frame") == 0 || strcmp(name, ".zdebug_frame") == 0)
> + {
> + is_eh = false;
> + }
> + else
> + {
> + continue;
> + }
> +
> + if (thread_count >= num_threads)
> + {
> + num_threads *= 2;
> + threads = realloc(threads, num_threads * sizeof(pthread_t));
> + thread_data = realloc(thread_data, num_threads * sizeof(ThreadData*));
> + }
> +
> + thread_data[thread_count] = malloc(sizeof(ThreadData));
> + thread_data[thread_count]->name = name;
> + thread_data[thread_count]->e_ident = ident;
> + thread_data[thread_count]->scn = scn;
> + thread_data[thread_count]->is_eh = is_eh;
> +
> + if (pthread_create(&threads[thread_count], NULL, thread_work, thread_data[thread_count]) != 0)
> + {
> + perror("Failed to create thread");
> + for (int j = 0; j < thread_count; j++)
> + {
> + pthread_cancel(threads[j]);
> + }
> + free(threads);
> + free(thread_data);
> + return 1;
> + }
> + else
> + {
> + thread_count++;
> + }
> + }
> + }
> + }
> +
> + for (int i = 0; i < thread_count; i++)
> + {
> + if (pthread_join(threads[i], NULL) != 0)
> + {
> + perror("Failed to join thread");
> + free(threads);
> + free(thread_data);
> + return 1;
> + }
> + }
> +
> + for (int i = 0; i < thread_count; i++)
> + {
> + free(thread_data[i]);
> + }
> +
> + free(threads);
> + free(thread_data);
> +
> + elf_end(elf);
> + close(fd);
> +
> + return 0;
> +}
> \ No newline at end of file
> diff --git a/tests/eu_search_die.c b/tests/eu_search_die.c
> new file mode 100644
> index 00000000..a7f75521
> --- /dev/null
> +++ b/tests/eu_search_die.c
> @@ -0,0 +1,262 @@
> +/*Test program for eu_search_die.
> + Copyright (C) 2023 Red Hat, Inc.
> + This file is part of elfutils.
> +
> + This file is free software; you can redistribute it and/or modify
> + it under the terms of the GNU General Public License as published by
> + the Free Software Foundation; either version 3 of the License, or
> + (at your option) any later version.
> +
> + elfutils is distributed in the hope that it will be useful, but
> + WITHOUT ANY WARRANTY; without even the implied warranty of
> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + GNU General Public License for more details.
> +
> + You should have received a copy of the GNU General Public License
> + along with this program. If not, see<http://www.gnu.org/licenses/>. */
> +#include <config.h>
> +#include ELFUTILS_HEADER(dw)
> +#include <dwarf.h>
> +#include <sys/types.h>
> +#include <sys/stat.h>
> +#include <fcntl.h>
> +#include <stdio.h>
> +#include <stdint.h>
> +#include <stdlib.h>
> +#include <assert.h>
> +#include <inttypes.h>
> +#include <string.h>
> +#include <errno.h>
> +#include <unistd.h>
> +#include <pthread.h>
> +
> +static void *thread_work(void *arg);
> +static int check_die(Dwarf_Die *die);
> +static int check_dbg(Dwarf *dbg);
> +
> +/*The main Dwarf file. */
> +static Dwarf * dwarf;
> +
> +typedef struct
> +{
> + Dwarf * dbg;
> + Dwarf_Off start_offset;
> + Dwarf_Off end_offset;
> + int result;
> +}
> +
> +ThreadData;
> +
> +static void *thread_work(void *arg)
> +{
> + ThreadData *data = (ThreadData*) arg;
> + data->result = check_dbg(data->dbg);
> + return NULL;
> +}
> +
> +static int check_die(Dwarf_Die *die)
> +{
> + if (dwarf_tag(die) == DW_TAG_invalid)
> + {
> + printf("Invalid die\n");
> + return -1;
> + }
> +
> + int res = 0;
> + void *addr = die->addr;
> + Dwarf_Die die2;
> + if (dwarf_die_addr_die(dwarf, addr, &die2) == NULL)
> + {
> + printf("Bad die addr die at offset %" PRIx64 "\n", dwarf_dieoffset(die));
> + res = -1;
> + }
> +
> + if (dwarf_tag(die) != dwarf_tag(&die2))
> + {
> + printf("Tags differ for die at offset %" PRIx64 "\n", dwarf_dieoffset(die));
> + res = -1;
> + }
> +
> + if (dwarf_cuoffset(die) != dwarf_cuoffset(&die2))
> + {
> + printf("CU offsets differ for die at offset %" PRIx64 "\n", dwarf_dieoffset(die));
> + res = -1;
> + }
> +
> + Dwarf_Die child;
> + if (dwarf_child(die, &child) == 0)
> + res |= check_die(&child);
> +
> + Dwarf_Die sibling;
> + if (dwarf_siblingof(die, &sibling) == 0)
> + res |= check_die(&sibling);
> +
> + return res;
> +}
> +
> +static int check_dbg(Dwarf *dbg)
> +{
> + int res = 0;
> + Dwarf_Off off = 0;
> + Dwarf_Off old_off = 0;
> + size_t hsize;
> + Dwarf_Off abbrev;
> + uint8_t addresssize;
> + uint8_t offsetsize;
> +
> + while (dwarf_nextcu(dbg, off, &off, &hsize, &abbrev, &addresssize, &offsetsize) == 0)
> + {
> + Dwarf_Die die;
> + if (dwarf_offdie(dbg, old_off + hsize, &die) != NULL)
> + {
> + printf("checking CU at %" PRIx64 "\n", old_off);
> + res |= check_die(&die);
> + }
> +
> + old_off = off;
> + }
> +
> + // Same for type...
> + Dwarf_Half version;
> + uint64_t typesig;
> + Dwarf_Off typeoff;
> + off = 0;
> + old_off = 0;
> +
> + while (dwarf_next_unit(dbg, off, &off, &hsize, &version, &abbrev, &addresssize, &offsetsize, &typesig, &typeoff) == 0)
> + {
> + Dwarf_Die die;
> + if (dwarf_offdie_types(dbg, old_off + hsize, &die) != NULL)
> + {
> + printf("checking TU at %" PRIx64 "\n", old_off);
> + res |= check_die(&die);
> + }
> +
> + // We should have seen this already, but double check...
> + if (dwarf_offdie_types(dbg, old_off + typeoff, &die) != NULL)
> + {
> + printf("checking Type DIE at %" PRIx64 "\n", old_off + hsize + typeoff);
> + res |= check_die(&die);
> + }
> +
> + old_off = off;
> + }
> +
> + Dwarf *alt = dwarf_getalt(dbg);
> +
> + if (alt != NULL)
> + {
> + printf("checking alt debug\n");
> + res |= check_dbg(alt);
> + }
> +
> + // Split or Type Dwarf_Dies gotten through dwarf_get_units.
> + Dwarf_CU *cu = NULL;
> + Dwarf_Die subdie;
> + uint8_t unit_type;
> + while (dwarf_get_units(dbg, cu, &cu, NULL, &unit_type, NULL, &subdie) == 0)
> + {
> + if (dwarf_tag(&subdie) != DW_TAG_invalid)
> + {
> + printf("checking %"
> + PRIx8 " subdie\n", unit_type);
> + res |= check_die(&subdie);
> + }
> + }
> +
> + return res;
> +}
> +
> +int main(int argc, char *argv[])
> +{
> + if (argc < 2)
> + {
> + printf("No file given.\n");
> + return -1;
> + }
> +
> + const char *name = argv[1];
> + int fd = open(name, O_RDONLY);
> + if (fd < 0)
> + {
> + printf("Cannot open '%s': %s\n", name, strerror(errno));
> + return -1;
> + }
> +
> + dwarf = dwarf_begin(fd, DWARF_C_READ);
> + if (dwarf == NULL)
> + {
> + printf("Not a Dwarf file '%s': %s\n", name, dwarf_errmsg(-1));
> + close(fd);
> + return -1;
> + }
> +
> + printf("checking %s\n", name);
> +
> + int num_threads = 4;
> + pthread_t *threads = (pthread_t*) malloc(num_threads* sizeof(pthread_t));
> + ThreadData *thread_data = (ThreadData*) malloc(num_threads* sizeof(ThreadData));
> +
> + if (!threads || !thread_data)
> + {
> + fprintf(stderr, "Failed to allocate memory for threads.\n");
> + free(threads);
> + free(thread_data);
> + return 1;
> + }
> +
> + Dwarf_Off total_off = 0;
> + Dwarf_Off unit_off = 0;
> + size_t hsize;
> + Dwarf_Off abbrev;
> + uint8_t addresssize;
> + uint8_t offsetsize;
> +
> + while (dwarf_nextcu(dwarf, unit_off, &unit_off, &hsize, &abbrev, &addresssize, &offsetsize) == 0)
> + {
> + thread_data[total_off % num_threads].start_offset = unit_off;
> + thread_data[total_off % num_threads].end_offset = unit_off + hsize;
> + total_off++;
> + }
> +
> + for (int i = 0; i < num_threads; i++)
> + {
> + thread_data[i].dbg = dwarf;
> + if (pthread_create(&threads[i], NULL, thread_work, (void*) &thread_data[i]) != 0)
> + {
> + perror("Failed to create thread");
> + for (int j = 0; j < i; j++)
> + {
> + pthread_cancel(threads[j]);
> + }
> + free(threads);
> + free(thread_data);
> + return 1;
> + }
> + }
> +
> + for (int i = 0; i < num_threads; i++)
> + {
> + if (pthread_join(threads[i], NULL) != 0)
> + {
> + perror("Failed to join thread");
> + free(threads);
> + free(thread_data);
> + return 1;
> + }
> + }
> +
> + int res = 0;
> + for (int i = 0; i < num_threads; i++)
> + {
> + res |= thread_data[i].result;
> + }
> +
> + free(threads);
> + free(thread_data);
> +
> + dwarf_end(dwarf);
> + close(fd);
> +
> + return res;
> +}
> \ No newline at end of file
> diff --git a/tests/eu_search_lines.c b/tests/eu_search_lines.c
> new file mode 100644
> index 00000000..b7a875d8
> --- /dev/null
> +++ b/tests/eu_search_lines.c
> @@ -0,0 +1,228 @@
> +/*Test program for eu_search_lines.
> + Copyright (C) 2023 Red Hat, Inc.
> + This file is part of elfutils.
> +
> + This file is free software; you can redistribute it and/or modify
> + it under the terms of the GNU General Public License as published by
> + the Free Software Foundation; either version 3 of the License, or
> + (at your option) any later version.
> +
> + elfutils is distributed in the hope that it will be useful, but
> + WITHOUT ANY WARRANTY; without even the implied warranty of
> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + GNU General Public License for more details.
> +
> + You should have received a copy of the GNU General Public License
> + along with this program. If not, see<http://www.gnu.org/licenses/>. */
> +
> +#include <config.h>
> +#include <fcntl.h>
> +#include <inttypes.h>
> +#include <libelf.h>
> +#include ELFUTILS_HEADER(dw)
> +#include <stdio.h>
> +#include <string.h>
> +#include <unistd.h>
> +#include <stdlib.h>
> +#include <pthread.h>
> +
> +typedef struct
> +{
> + const char *filename;
> + int result;
> +}
> +
> +ThreadData;
> +
> +static void *thread_work(void *arg)
> +{
> + ThreadData *data = (ThreadData*) arg;
> +
> + int fd = open(data->filename, O_RDONLY);
> +
> + Dwarf *dbg = dwarf_begin(fd, DWARF_C_READ);
> + if (dbg == NULL)
> + {
> + printf("%s not usable: %s\n", data->filename, dwarf_errmsg(-1));
> + close(fd);
> + free(data);
> + pthread_exit(NULL);
> + }
> +
> + Dwarf_Off cuoff = 0;
> + Dwarf_Off old_cuoff = 0;
> + size_t hsize;
> + Dwarf_Off ao;
> + uint8_t asz;
> + uint8_t osz;
> + while (dwarf_nextcu(dbg, cuoff, &cuoff, &hsize, &ao, &asz, &osz) == 0)
> + {
> + printf("cuhl = %zu, o = %llu, asz = %hhu, osz = %hhu, ncu = %llu\n",
> + hsize, (unsigned long long int) ao,
> + asz, osz, (unsigned long long int) cuoff);
> +
> + // Get the DIE for the CU.
> + Dwarf_Die die;
> + if (dwarf_offdie(dbg, old_cuoff + hsize, &die) == NULL)
> + {
> + printf("%s: cannot get CU die\n", data->filename);
> + data->result = 1;
> + break;
> + }
> +
> + old_cuoff = cuoff;
> +
> + Dwarf_Lines * lb;
> + size_t nlb;
> + if (dwarf_getsrclines(&die, &lb, &nlb) != 0)
> + {
> + printf("%s: cannot get lines\n", data->filename);
> + data->result = 1;
> + break;
> + }
> +
> + printf(" %zu lines\n", nlb);
> +
> + for (size_t i = 0; i < nlb; ++i)
> + {
> + Dwarf_Line *l = dwarf_onesrcline(lb, i);
> + if (l == NULL)
> + {
> + printf("%s: cannot get individual line\n", data->filename);
> + data->result = 1;
> + break;
> + }
> +
> + Dwarf_Addr addr;
> + if (dwarf_lineaddr(l, &addr) != 0)
> + addr = 0;
> + const char *file = dwarf_linesrc(l, NULL, NULL);
> + int line;
> + if (dwarf_lineno(l, &line) != 0)
> + line = 0;
> +
> + printf("%" PRIx64 ": %s:%d:", (uint64_t) addr, file ? : "???", line);
> +
> + // Getting the file path through the Dwarf_Files should
> + // result in the same path.
> + Dwarf_Files * files;
> + size_t idx;
> + if (dwarf_line_file(l, &files, &idx) != 0)
> + {
> + printf("%s: cannot get file from line (%zd): %s\n",
> + data->filename, i, dwarf_errmsg(-1));
> + data->result = 1;
> + break;
> + }
> +
> + const char *path = dwarf_filesrc(files, idx, NULL, NULL);
> + if ((path == NULL && file != NULL) ||
> + (path != NULL && file == NULL) ||
> + (strcmp(file, path) != 0))
> + {
> + printf("%s: line %zd srcline (%s) != file srcline (%s)\n",
> + data->filename, i, file ? : "???", path ? : "???");
> + data->result = 1;
> + break;
> + }
> +
> + int column;
> + if (dwarf_linecol(l, &column) != 0)
> + column = 0;
> + if (column >= 0)
> + printf("%d:", column);
> +
> + bool is_stmt;
> + if (dwarf_linebeginstatement(l, &is_stmt) != 0)
> + is_stmt = false;
> + bool end_sequence;
> + if (dwarf_lineendsequence(l, &end_sequence) != 0)
> + end_sequence = false;
> + bool basic_block;
> + if (dwarf_lineblock(l, &basic_block) != 0)
> + basic_block = false;
> + bool prologue_end;
> + if (dwarf_lineprologueend(l, &prologue_end) != 0)
> + prologue_end = false;
> + bool epilogue_begin;
> + if (dwarf_lineepiloguebegin(l, &epilogue_begin) != 0)
> + epilogue_begin = false;
> +
> + printf(" is_stmt:%s, end_seq:%s, bb:%s, prologue:%s, epilogue:%s\n",
> + is_stmt ? "yes" : "no", end_sequence ? "yes" : "no",
> + basic_block ? "yes" : "no", prologue_end ? "yes" : "no",
> + epilogue_begin ? "yes" : "no");
> + }
> + }
> +
> + dwarf_end(dbg);
> + close(fd);
> + free(data);
> +
> + pthread_exit(NULL);
> +}
> +
> +int main(int argc, char *argv[])
> +{
> + int result = 0;
> + int cnt;
> +
> + if (argc < 2)
> + {
> + printf("Usage: %s<filename1>[<filename2> ...]\n", argv[0]);
> + return 1;
> + }
> +
> + pthread_t *threads = (pthread_t*) malloc((argc - 1) *sizeof(pthread_t));
> + ThreadData **thread_data = (ThreadData **) malloc((argc - 1) *sizeof(ThreadData*));
> +
> + if (!threads || !thread_data)
> + {
> + fprintf(stderr, "Failed to allocate memory for threads.\n");
> + free(threads);
> + free(thread_data);
> + return 1;
> + }
> +
> + for (cnt = 1; cnt < argc; ++cnt)
> + {
> + thread_data[cnt - 1] = (ThreadData*) malloc(sizeof(ThreadData));
> + thread_data[cnt - 1]->filename = argv[cnt];
> + thread_data[cnt - 1]->result = 0;
> +
> + if (pthread_create(&threads[cnt - 1], NULL, thread_work, thread_data[cnt - 1]) != 0)
> + {
> + perror("Failed to create thread");
> + for (int j = 0; j < cnt; j++)
> + {
> + pthread_cancel(threads[j]);
> + }
> + free(threads);
> + free(thread_data);
> + return 1;
> + }
> + }
> +
> + for (cnt = 0; cnt < argc - 1; ++cnt)
> + {
> + if (pthread_join(threads[cnt], NULL) != 0)
> + {
> + perror("Failed to join thread");
> + free(threads);
> + free(thread_data);
> + return 1;
> + }
> +
> + if (thread_data[cnt]->result != 0)
> + {
> + result = 1;
> + }
> +
> + free(thread_data[cnt]);
> + }
> +
> + free(threads);
> + free(thread_data);
> +
> + return result;
> +}
> \ No newline at end of file
> diff --git a/tests/eu_search_macros.c b/tests/eu_search_macros.c
> new file mode 100644
> index 00000000..3dca828e
> --- /dev/null
> +++ b/tests/eu_search_macros.c
> @@ -0,0 +1,216 @@
> +/*Test program for eu_search_macros
> + Copyright (C) 2023 Red Hat, Inc.
> + This file is part of elfutils.
> +
> + This file is free software; you can redistribute it and/or modify
> + it under the terms of the GNU General Public License as published by
> + the Free Software Foundation; either version 3 of the License, or
> + (at your option) any later version.
> +
> + elfutils is distributed in the hope that it will be useful, but
> + WITHOUT ANY WARRANTY; without even the implied warranty of
> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + GNU General Public License for more details.
> +
> + You should have received a copy of the GNU General Public License
> + along with this program. If not, see<http://www.gnu.org/licenses/>. */
> +
> +#include <config.h>
> +#include ELFUTILS_HEADER(dw)
> +#include <dwarf.h>
> +#include <sys/types.h>
> +#include <sys/stat.h>
> +#include <fcntl.h>
> +#include <stdio.h>
> +#include <stdint.h>
> +#include <stdlib.h>
> +#include <assert.h>
> +#include <inttypes.h>
> +#include <stdatomic.h>
> +#include <pthread.h>
> +
> +static void *thread_work(void *arg);
> +static int mac(Dwarf_Macro *macro, void *dbg);
> +static void include(Dwarf *dbg, Dwarf_Off macoff, ptrdiff_t token);
> +
> +typedef struct
> +{
> + Dwarf * dbg;
> + Dwarf_Die * cudie;
> + bool new_style;
> +}
> +
> +ThreadData;
> +
> +static void *thread_work(void *arg)
> +{
> + ThreadData *data = (ThreadData*) arg;
> + Dwarf *dbg = data->dbg;
> + Dwarf_Die *cudie = data->cudie;
> + bool new_style = data->new_style;
> +
> + for (ptrdiff_t off = new_style ? DWARF_GETMACROS_START : 0;
> + (off = dwarf_getmacros(cudie, mac, dbg, off));)
> + {
> + if (off == -1)
> + {
> + puts(dwarf_errmsg(dwarf_errno()));
> + break;
> + }
> + }
> +
> + return NULL;
> +}
> +
> +static void include(Dwarf *dbg, Dwarf_Off macoff, ptrdiff_t token)
> +{
> + while ((token = dwarf_getmacros_off(dbg, macoff, mac, dbg, token)) != 0)
> + {
> + if (token == -1)
> + {
> + puts(dwarf_errmsg(dwarf_errno()));
> + break;
> + }
> + }
> +}
> +
> +static int
> +mac(Dwarf_Macro *macro, void *dbg)
> +{
> + static atomic_int level = 0;
> +
> + unsigned int opcode;
> + dwarf_macro_opcode(macro, &opcode);
> + switch (opcode)
> + {
> + case DW_MACRO_import:
> + {
> + Dwarf_Attribute at;
> + int r = dwarf_macro_param(macro, 0, &at);
> + assert(r == 0);
> +
> + Dwarf_Word w;
> + r = dwarf_formudata(&at, &w);
> + assert(r == 0);
> +
> + printf ("%*sinclude %#" PRIx64 "\n", level, "", w);
> +
> + atomic_fetch_add(&level, 1);
> +
> + include(dbg, w, DWARF_GETMACROS_START);
> +
> + atomic_fetch_sub(&level, 1);
> +
> + printf ("%*s/include\n", level, "");
> + break;
> + }
> +
> + case DW_MACRO_start_file:
> + {
> + Dwarf_Files * files;
> + size_t nfiles;
> + if (dwarf_macro_getsrcfiles(dbg, macro, &files, &nfiles) < 0)
> + printf("dwarf_macro_getsrcfiles: %s\n", dwarf_errmsg(dwarf_errno()));
> +
> + Dwarf_Word w = 0;
> + dwarf_macro_param2(macro, &w, NULL);
> +
> + const char *name = dwarf_filesrc (files, (size_t) w, NULL, NULL);
> + printf ("%*sfile %s\n", level, "", name);
> + atomic_fetch_add(&level, 1);
> + break;
> + }
> +
> + case DW_MACRO_end_file:
> + {
> + atomic_fetch_sub(&level, 1);
> + printf ("%*s/file\n", level, "");
> + break;
> + }
> +
> + case DW_MACINFO_define:
> + case DW_MACRO_define_strp:
> + {
> + const char *value;
> + dwarf_macro_param2(macro, NULL, &value);
> + printf ("%*s%s\n", level, "", value);
> + break;
> + }
> +
> + case DW_MACINFO_undef:
> + case DW_MACRO_undef_strp:
> + break;
> +
> + default:
> + {
> + size_t paramcnt;
> + dwarf_macro_getparamcnt(macro, ¶mcnt);
> + printf ("%*sopcode %u with %zd arguments\n", level, "", opcode, paramcnt);
> + break;
> + }
> + }
> +
> + return DWARF_CB_ABORT;
> +}
> +
> +int main(int argc, char *argv[])
> +{
> + assert(argc >= 3);
> + const char *name = argv[1];
> + ptrdiff_t cuoff = strtol(argv[2], NULL, 0);
> + bool new_style = argc > 3;
See note below on usage in run-eu-search-tests.sh
> + int fd = open(name, O_RDONLY);
> + Dwarf *dbg = dwarf_begin(fd, DWARF_C_READ);
> +
> + Dwarf_Die cudie_mem, *cudie = dwarf_offdie(dbg, cuoff, &cudie_mem);
> +
> + int num_threads = 4;
> + pthread_t *threads = malloc(num_threads* sizeof(pthread_t));
> + ThreadData *thread_data = malloc(num_threads* sizeof(ThreadData));
> +
> + if (!threads || !thread_data)
> + {
> + fprintf(stderr, "Failed to allocate memory for threads.\n");
> + free(threads);
> + free(thread_data);
> + return 1;
> + }
> +
> + for (int i = 0; i < num_threads; i++)
> + {
> + thread_data[i].dbg = dbg;
> + thread_data[i].cudie = cudie;
> + thread_data[i].new_style = new_style;
> +
> + if (pthread_create(&threads[i], NULL, thread_work, (void*) &thread_data[i]) != 0)
> + {
> + perror("Failed to create thread");
> + for (int j = 0; j < i; j++)
> + {
> + pthread_cancel(threads[j]);
> + }
> + free(threads);
> + free(thread_data);
> + return 1;
> + }
> + }
> +
> + for (int i = 0; i < num_threads; i++)
> + {
> + if (pthread_join(threads[i], NULL) != 0)
> + {
> + perror("Failed to join thread");
> + free(threads);
> + free(thread_data);
> + return 1;
> + }
> + }
> +
> + free(threads);
> + free(thread_data);
> +
> + dwarf_end(dbg);
> +
> + return 0;
> +}
> \ No newline at end of file
> diff --git a/tests/run-eu-search-tests.sh b/tests/run-eu-search-tests.sh
> new file mode 100755
> index 00000000..84edc234
> --- /dev/null
> +++ b/tests/run-eu-search-tests.sh
> @@ -0,0 +1,168 @@
> +#! /bin/sh
> +# Copyright (C) 2015, 2018 Red Hat, Inc.
> +# This file is part of elfutils.
> +#
> +# This file is free software; you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License as published by
> +# the Free Software Foundation; either version 3 of the License, or
> +# (at your option) any later version.
> +#
> +# elfutils is distributed in the hope that it will be useful, but
> +# WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program. If not, see <http://www.gnu.org/licenses/>.
> +
> +. $srcdir/test-subr.sh
> +
> +# Extract the value of USE_ADDRESS_SANITIZER_TRUE from config.status
> +# Cannot use Helgrind and Address Sanitizer together.
> +# Test will be skipped if Address Sanitizer is enabled.
> +USE_ADDRESS_SANITIZER=$(grep 'USE_ADDRESS_SANITIZER_TRUE' ${abs_builddir}/../config.status | awk -F'=' '{print $2}')
> +
> +if [[ "$USE_ADDRESS_SANITIZER" == "\"#\"" ]]; then
> + echo "Address Sanitizer is disabled."
> +else
> + echo "Address Sanitizer is enabled. Skipping test."
> + exit 77
> +fi
> +
> +# Extract the value of USE_MEMORY_SANITIZER_TRUE from config.status
> +# Cannot use Helgrind and Memory Sanitizer together.
> +# Test will be skipped if Memory Sanitizer is enabled.
> +USE_MEMORY_SANITIZER=$(grep 'USE_MEMORY_SANITIZER_TRUE' ${abs_builddir}/../config.status | awk -F'=' '{print $2}')
> +
> +if [[ "$USE_MEMORY_SANITIZER" == "\"#\"" ]]; then
> + echo "Memory Sanitizer is disabled."
> +else
> + echo "Memory Sanitizer is enabled. Skipping test."
> + exit 77
> +fi
> +
> +# Extract the value of USE_LOCKS from config.h
> +# Test will only be run if USE_LOCKS is defined. Otherwise, skip.
> +USE_LOCKS=$(grep '^#define USE_LOCKS' ${abs_builddir}/../config.h | awk '{print $3}')
> +
> +if [[ "$USE_LOCKS" -eq 1 ]]; then
> + echo "USE_LOCKS is defined."
> +else
> + echo "USE_LOCKS is not defined. Skipping test."
> + exit 77
> +fi
Smart.
But you can also exclude the test in the tests/Makefile.am
which may be slightly simpler.
if USE_LOCKS
if !USE_ADDRESS_SANITIZER
if !USE_MEMORY_SANITIZER
TESTS += run-eu-search-tests.sh
endif
endif
endif
> +# Disable valgrind if configured, since we are already using it here.
> +SAVED_VALGRIND_CMD="$VALGRIND_CMD"
> +unset VALGRIND_CMD
Yes, but... see below.
> +echo "Begin tests..."
> +
> +# Begin data race test for parallelized dwarf-die-addr-die
> +# Tests thread safety for updated libdw_findcu.c and libdw_find_split_unit.c
> +testfiles testfile-debug-types
> +testfiles testfile_multi_main testfile_multi.dwz
> +testfiles testfilebazdbgppc64.debug
> +testfiles testfile-dwarf-4 testfile-dwarf-5
> +testfiles testfile-splitdwarf-4 testfile-hello4.dwo testfile-world4.dwo
> +testfiles testfile-splitdwarf-5 testfile-hello5.dwo testfile-world5.dwo
> +
> +die_test_files=("testfile-debug-types"
> + "testfile_multi_main" "testfile_multi.dwz"
> + "testfilebazdbgppc64.debug"
> + "testfile-dwarf-4" "testfile-dwarf-5"
> + "testfile-splitdwarf-4" "testfile-hello4.dwo" "testfile-world4.dwo"
> + "testfile-splitdwarf-5" "testfile-hello5.dwo" "testfile-world5.dwo")
Nice collection of test files.
> +echo -e "\nStarting data race test for dwarf-die-addr-die"
> +
> +for file in "${die_test_files[@]}"; do
> + helgrind_output=$(valgrind --tool=helgrind "${abs_builddir}/eu_search_die" "$file" 2>&1)
> +
> + if grep -q "ERROR SUMMARY: 0 errors" <<< "$helgrind_output"; then
> + echo "No data races found for $file. Test passed."
> + else
> + echo "Data races found for $file. Test failed."
> + echo "$helgrind_output"
> + exit 1
> + fi
> +done
This will run valgrind and the program under the system installed
libelf and libdw. So we will want to do something like
Also you may use valgrind --error-exitcode=1 to check for errors.
So you probably want something like
VALGRIND_CMD="valgrind -q --tool=helgrind --error-exitcode=1"
and then simply do:
for file in "${die_test_files[@]}"; do
testrun "${abs_builddir}/eu_search_die" "$file"
done
Same for the other tests.
> +echo -e "\nStarting data race test for dwarf-getmacros"
> +
> +for file in "${macro_test_files[@]}"; do
> +
> + helgrind_output=$(valgrind --tool=helgrind "${abs_builddir}/eu_search_macros" $file 2>&1)
> +
> + if grep -q "ERROR SUMMARY: 0 errors" <<< "$helgrind_output"; then
> + echo "No data races found for $file. Test passed."
> + else
> + echo "Data races found for $file. Test failed."
> + echo "$helgrind_output"
> + exit 1
> + fi
> +
> +done
>
Note you are passing just $file as argument to eu_search_macros but
that seems to require 3 arguments.
Cheers,
Mark
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH] Fix thread-safety for elfutils
2023-10-10 13:40 ` Mark Wielaard
2023-10-10 13:42 ` [PATCH 01/16] lib: Add new once_define and once macros to eu-config.h Mark Wielaard
@ 2023-10-14 15:39 ` Mark Wielaard
2023-10-14 18:29 ` Heather McIntyre
1 sibling, 1 reply; 51+ messages in thread
From: Mark Wielaard @ 2023-10-14 15:39 UTC (permalink / raw)
To: Heather McIntyre, elfutils-devel; +Cc: John Mellor-Crummey
Hi Heather,
On Tue, 2023-10-10 at 15:40 +0200, Mark Wielaard wrote:
> Very nice. That is a lot of work. And I must admit that I cannot hold
> that much work in my little head at the same time. So I have split up
> your commit into (what I hope are) logical independent parts. That will
> make it easier to review. I might have split it into too many parts,
> but that at least makes it easier to just add those parts that are
> trivially correct.
>
> The only changes I made were:
> 1. Move the ChangeLog entries into the commit message
> (This is something we do now and makes cherry picking small
> changes easier, but I see it isn't actually documented
> in CONTRIBUTING, sorry. I'll try to update that.)
> 2. Fixed up some Copyright notices as discussed off-list.
> 3. Made some whitespace/indentation changes which made the
> diffs slightly smaller (in most cases).
>
> I'll comment/review the individual commits. Which I'll post to the
> list.
So I commented on each one individually.
Below is the summary. I have pushed 4 patches to git trunk already.
I am suggesting to drop 2 changes, but please feel free to say we
really need them. I have some questions about the rest, but some are
minor issues. In general I really like the direction of this.
Hope my splitting of your patch isn't too confusing. But it really
helps me review separate smaller independent parts. And it makes it so
we can push more of your changes early.
> Heather McIntyre (16):
> lib: Add new once_define and once macros to eu-config.h
Looks good. Added to main branch.
> libelf: Make elf_version thread-safe
Looks good. Added to main branch.
Question whether this could also have been done with atomics?
> libelf: Fix deadlock in __libelf_readall
I think the locking here is subtly wrong in the case of
elf->map_address != NULL
Suggest to look into rewriting libelf_acquire_all and
libelf_release_all to libelf_acquire_all_children and
libelf_release_all_children (which would only be called
with the elf->lock already acquired).
> libelf: Fix deadlock in elf_cntl
Looks correct, but can probably be simplified by just relying on
if (__libelf_readall (elf) == NULL)
> libelf: Fix elf_end deadlock
Looks good. Added to main branch.
> libelf: Make elf32_getchdr and elf64_getchdr thread-safe
Looks good. Just needs elf32_getchdr.h added to noinst_HEADERS.
Pushed to the main branch with that change.
> lib: Add eu_tsearch and eu_tfind
I think this would work. But I really like to discuss whether we can
have more fine grained locking. The static global lock will likely
cause too much contention. I added a similar change to noinst_HEADERS
as above for eu-search.h.
> libcpu: Change calls for tsearch/tfind to eu_tsearch/eu_tfind.
Suggest to drop this for now. It will need a twalk wrapper. And I am
not sure it will be easy to make the bison parser concurrent as a
whole. This is for libasm, not libdw, so can imho be handled separately
if we really want to make libasm also completely thread-safe.
> src: Use eu-search in nm and findtextrel.
Although this works, I don't think it is needed because eu-nm and eu-
findtextrel are single threaded utilities.
> libdw: make dwarf_getalt thread-safe
I think this should also include dwarf_setalt. Also lets discuss
whether the locking is done correctly.
> libdw: Add locking around __libdw_dieabbrev for dwarf_hasattr
I think this is also incomplete. dwarf_child, dwarf_getattrs,
dwarf_haschildren and dwarf_tag also call __libdw_dieabbrev. I am also
concerned this locking is very "heavy". Lets discuss whether there is
an alternative way to update the die->abbrev in a thead-safe way.
> libdw: Make libdw_find_split_unit thread-safe
I have some concerns about whether the locking is complete/wide enough.
> libdw: Make libdw_findcu thread-safe
Looks mostly correct. Question about whether this could be a per struct
Dwarf lock and whether the lock should also be added for the
dwarf_formref_die call to __libdw_intern_next_unit.
> libdw,libdwfl: Use eu-search for thread-safety
These all look good. Just have to think about whether the global
static lock currently used in eu-search is the right approach.
> tests: Add eu-search tests
I am enthusiastic about having these new tests. Some suggestions to
simplify the mechanics. I also have updated the Copyright on the new
files. Could you look into the invocation of eu_search_macros. Maybe I
am misinterpreting how this should be used. But it looks like this will
always fail.
> configure: No longer mark --enable-thread-safety as EXPERIMENTAL
Looks good. Will push once at least all libelf patches are in.
Note, your NEWS entry seems missing.
> Which can also be found here:
> https://code.wildebeest.org/git/user/mjw/elfutils/log/?h=thread-safety
I have updated that git branch with some of the suggestions above and
removed the commits that have already been pushed.
Thanks,
Mark
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH] Fix thread-safety for elfutils
2023-10-14 15:39 ` [PATCH] Fix thread-safety for elfutils Mark Wielaard
@ 2023-10-14 18:29 ` Heather McIntyre
2023-10-17 15:04 ` Mark Wielaard
0 siblings, 1 reply; 51+ messages in thread
From: Heather McIntyre @ 2023-10-14 18:29 UTC (permalink / raw)
To: Mark Wielaard; +Cc: elfutils-devel, John Mellor-Crummey
[-- Attachment #1: Type: text/plain, Size: 6334 bytes --]
Hi Mark,
Thank you for taking the time to review the patch and for your thoughtful
feedback. I really appreciate your attention to detail and effort in
splitting the commit into smaller, more manageable parts. John and I went
over some of your comments and concerns yesterday, and I will be working on
addressing them ASAP.
I have a quick query regarding how you'd prefer to handle these changes.
Would you like me to implement some of the recommended modifications and
commit them (if possible), or would you prefer that I simply leave comments
so you can make the necessary adjustments? I noticed that the split-up
branch resides on what appears to be your personal space at
https://code.wildebeest.org/git/user/mjw/elfutils/log/?h=thread-safety. I'm
uncertain whether I have the necessary access for making commits there.
Best regards,
Heather McIntyre
On Sat, Oct 14, 2023 at 10:40 AM Mark Wielaard <mark@klomp.org> wrote:
> Hi Heather,
>
> On Tue, 2023-10-10 at 15:40 +0200, Mark Wielaard wrote:
> > Very nice. That is a lot of work. And I must admit that I cannot hold
> > that much work in my little head at the same time. So I have split up
> > your commit into (what I hope are) logical independent parts. That will
> > make it easier to review. I might have split it into too many parts,
> > but that at least makes it easier to just add those parts that are
> > trivially correct.
> >
> > The only changes I made were:
> > 1. Move the ChangeLog entries into the commit message
> > (This is something we do now and makes cherry picking small
> > changes easier, but I see it isn't actually documented
> > in CONTRIBUTING, sorry. I'll try to update that.)
> > 2. Fixed up some Copyright notices as discussed off-list.
> > 3. Made some whitespace/indentation changes which made the
> > diffs slightly smaller (in most cases).
> >
> > I'll comment/review the individual commits. Which I'll post to the
> > list.
>
> So I commented on each one individually.
> Below is the summary. I have pushed 4 patches to git trunk already.
> I am suggesting to drop 2 changes, but please feel free to say we
> really need them. I have some questions about the rest, but some are
> minor issues. In general I really like the direction of this.
>
> Hope my splitting of your patch isn't too confusing. But it really
> helps me review separate smaller independent parts. And it makes it so
> we can push more of your changes early.
>
> > Heather McIntyre (16):
> > lib: Add new once_define and once macros to eu-config.h
>
> Looks good. Added to main branch.
>
> > libelf: Make elf_version thread-safe
>
> Looks good. Added to main branch.
> Question whether this could also have been done with atomics?
>
> > libelf: Fix deadlock in __libelf_readall
>
> I think the locking here is subtly wrong in the case of
> elf->map_address != NULL
>
> Suggest to look into rewriting libelf_acquire_all and
> libelf_release_all to libelf_acquire_all_children and
> libelf_release_all_children (which would only be called
> with the elf->lock already acquired).
>
> > libelf: Fix deadlock in elf_cntl
>
> Looks correct, but can probably be simplified by just relying on
> if (__libelf_readall (elf) == NULL)
>
> > libelf: Fix elf_end deadlock
>
> Looks good. Added to main branch.
>
> > libelf: Make elf32_getchdr and elf64_getchdr thread-safe
>
> Looks good. Just needs elf32_getchdr.h added to noinst_HEADERS.
> Pushed to the main branch with that change.
>
> > lib: Add eu_tsearch and eu_tfind
>
> I think this would work. But I really like to discuss whether we can
> have more fine grained locking. The static global lock will likely
> cause too much contention. I added a similar change to noinst_HEADERS
> as above for eu-search.h.
>
> > libcpu: Change calls for tsearch/tfind to eu_tsearch/eu_tfind.
>
> Suggest to drop this for now. It will need a twalk wrapper. And I am
> not sure it will be easy to make the bison parser concurrent as a
> whole. This is for libasm, not libdw, so can imho be handled separately
> if we really want to make libasm also completely thread-safe.
>
> > src: Use eu-search in nm and findtextrel.
>
> Although this works, I don't think it is needed because eu-nm and eu-
> findtextrel are single threaded utilities.
>
> > libdw: make dwarf_getalt thread-safe
>
> I think this should also include dwarf_setalt. Also lets discuss
> whether the locking is done correctly.
>
> > libdw: Add locking around __libdw_dieabbrev for dwarf_hasattr
>
> I think this is also incomplete. dwarf_child, dwarf_getattrs,
> dwarf_haschildren and dwarf_tag also call __libdw_dieabbrev. I am also
> concerned this locking is very "heavy". Lets discuss whether there is
> an alternative way to update the die->abbrev in a thead-safe way.
>
> > libdw: Make libdw_find_split_unit thread-safe
>
> I have some concerns about whether the locking is complete/wide enough.
>
> > libdw: Make libdw_findcu thread-safe
>
> Looks mostly correct. Question about whether this could be a per struct
> Dwarf lock and whether the lock should also be added for the
> dwarf_formref_die call to __libdw_intern_next_unit.
>
> > libdw,libdwfl: Use eu-search for thread-safety
>
> These all look good. Just have to think about whether the global
> static lock currently used in eu-search is the right approach.
>
> > tests: Add eu-search tests
>
> I am enthusiastic about having these new tests. Some suggestions to
> simplify the mechanics. I also have updated the Copyright on the new
> files. Could you look into the invocation of eu_search_macros. Maybe I
> am misinterpreting how this should be used. But it looks like this will
> always fail.
>
> > configure: No longer mark --enable-thread-safety as EXPERIMENTAL
>
> Looks good. Will push once at least all libelf patches are in.
> Note, your NEWS entry seems missing.
>
> > Which can also be found here:
> > https://code.wildebeest.org/git/user/mjw/elfutils/log/?h=thread-safety
>
> I have updated that git branch with some of the suggestions above and
> removed the commits that have already been pushed.
>
> Thanks,
>
> Mark
>
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH] Fix thread-safety for elfutils
2023-10-14 18:29 ` Heather McIntyre
@ 2023-10-17 15:04 ` Mark Wielaard
0 siblings, 0 replies; 51+ messages in thread
From: Mark Wielaard @ 2023-10-17 15:04 UTC (permalink / raw)
To: Heather McIntyre; +Cc: elfutils-devel, John Mellor-Crummey
Hi Heather,
On Sat, 2023-10-14 at 13:29 -0500, Heather McIntyre wrote:
> I have a quick query regarding how you'd prefer to handle these changes.
> Would you like me to implement some of the recommended modifications and
> commit them (if possible), or would you prefer that I simply leave
> comments so you can make the necessary adjustments?
Both! :)
Ideally you'll go over the individual emails for each patch that hasn't
been committed yet and reply whether you agree or not with my
observations. I might have misinterpreted the goal of a patch or set
constraints that are unrealistic. So lets first make sure we agree on
the specific issue we are going to fix.
For the mechanics it depends a bit on being familiar with git and using
it for using email to sent out patches. I like this:
https://git-send-email.io/
But that is sometimes not the easiest to setup. In which case I would
recommend creating an account on sourcehut https://meta.sr.ht/register
and create a clone of elfutils there:
https://git.sr.ht/~sourceware/elfutils
You can push your commits on a branch on sr.ht and then you can use
"Prepare a patchset" to sent emails to the list (or anyone else) from
that server, so you don't need to setup git+email yourself locally.
https://man.sr.ht/git.sr.ht/#sending-patches-upstream
Cheers,
Mark
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH 02/16] libelf: Make elf_version thread-safe
2023-10-10 14:00 ` Mark Wielaard
@ 2023-10-17 19:05 ` Heather McIntyre
2023-10-19 21:00 ` Mark Wielaard
0 siblings, 1 reply; 51+ messages in thread
From: Heather McIntyre @ 2023-10-17 19:05 UTC (permalink / raw)
To: Mark Wielaard; +Cc: elfutils-devel
[-- Attachment #1: Type: text/plain, Size: 2669 bytes --]
John and I discussed that atomic_compare_exchange_strong could have been
used here. I see that this has been pushed to the main branch, but I can
make the change to the atomic operation if you think that is a better
option.
Best,
Heather
On Tue, Oct 10, 2023 at 9:00 AM Mark Wielaard <mark@klomp.org> wrote:
> Hi Heather,
>
> On Tue, 2023-10-10 at 15:42 +0200, Mark Wielaard wrote:
> > From: Heather McIntyre <hsm2@rice.edu>
> >
> > * elf_version.c (version_once): Define once.
> > (initialize_version): New static function.
> > (elf_version): Use initialize_version version_once.
> >
> > Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
> > Signed-off-by: Mark Wielaard <mark@klomp.org>
> > ---
> > libelf/elf_version.c | 11 ++++++++++-
> > 1 file changed, 10 insertions(+), 1 deletion(-)
> >
> > diff --git a/libelf/elf_version.c b/libelf/elf_version.c
> > index 6ec534ab..8296bb65 100644
> > --- a/libelf/elf_version.c
> > +++ b/libelf/elf_version.c
> > @@ -32,12 +32,21 @@
> > #endif
> >
> > #include <libelfP.h>
> > +#include <pthread.h>
> >
> > +/* Multiple threads may initialize __libelf_version.
> > + pthread_once() ensures that __libelf_version is initialized only
> once. */
> > +once_define(static, version_once);
> >
> > /* Currently selected version. Should be EV_CURRENT.
> > Will be EV_NONE if elf_version () has not been called yet. */
> > unsigned int __libelf_version = EV_NONE;
> >
> > +static void initialize_version(void)
> > +{
> > + __libelf_version = EV_CURRENT;
> > +}
> > +
> > unsigned int
> > elf_version (unsigned int version)
> > {
> > @@ -49,7 +58,7 @@ elf_version (unsigned int version)
> > /* Phew, we know this version. */
> >
> > /* Signal that the version is now initialized. */
> > - __libelf_version = EV_CURRENT;
> > + once(version_once, initialize_version);
> >
> > /* And return the last (or initial) version. */
> > return EV_CURRENT;
>
> This is an odd function. The intention clearly was to support more "ELF
> versions" at some point. But (luckily) that never happened and I doubt
> there will ever be a different (incompatible) ELF version that we'll
> have to support. So in the end this will always be EV_CURRENT == 1. But
> the function has to be called to make the rest of the library work.
>
> I think this works and is fine. There will most likely never be real
> contention calling elf_version because normally a program just calls it
> once at the start.
>
> But have you thought about using some atomic operation here instead?
>
> Cheers,
>
> Mark
>
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH 03/16] libelf: Fix deadlock in __libelf_readall
2023-10-10 15:06 ` Mark Wielaard
@ 2023-10-17 19:11 ` Heather McIntyre
2023-11-09 13:26 ` Mark Wielaard
0 siblings, 1 reply; 51+ messages in thread
From: Heather McIntyre @ 2023-10-17 19:11 UTC (permalink / raw)
To: Mark Wielaard; +Cc: elfutils-devel
[-- Attachment #1: Type: text/plain, Size: 3024 bytes --]
You are right that if elf->map_address != NULL then the acquired wrlock is
not unlocked. The rwlock_unlock that was there initially was removed due to
deadlocking when returning from inside the if statement, but this was not
correct. However, adding ‘else rwlock_unlock (elf->lock)’ at the end of the
if statement fixes this issue.
I rewrote libelf_acquire_all and libelf_release_all as per your suggestion.
Now, libelf_acquire_all_children does not acquire the lock again for the
current elf object, but it does acquire locks for all children. Similarly,
libelf_release_all_children releases the locks for all children under the
acquired elf->lock. In libelf_readall, the elf->lock for the current elf
object is released after the call to libelf_release_all_children before
returning from the function. All tests are still passing after I made these
changes.
I will push the changes after I am done testing other fixes since I want to
ensure that everything works together cohesively.
On Tue, Oct 10, 2023 at 10:06 AM Mark Wielaard <mark@klomp.org> wrote:
> Hi Heather,
>
> On Tue, 2023-10-10 at 15:42 +0200, Mark Wielaard wrote:
> > From: Heather McIntyre <hsm2@rice.edu>
> >
> > * libelf/elf_readall.c (__libelf_readall): Move rwlock_unlock
> > before libelf_acquire_all.
> >
> > Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
> > Signed-off-by: Mark Wielaard <mark@klomp.org>
> > ---
> > libelf/elf_readall.c | 5 ++---
> > 1 file changed, 2 insertions(+), 3 deletions(-)
> >
> > diff --git a/libelf/elf_readall.c b/libelf/elf_readall.c
> > index d0f9a28c..2d62d447 100644
> > --- a/libelf/elf_readall.c
> > +++ b/libelf/elf_readall.c
> > @@ -84,6 +84,7 @@ __libelf_readall (Elf *elf)
> >
> > /* If this is an archive and we have derived descriptors get the
> > locks for all of them. */
> > + rwlock_unlock(elf->lock); // lock will be reacquired next line
> > libelf_acquire_all (elf);
> >
> > if (elf->maximum_size == ~((size_t) 0))
> > @@ -141,10 +142,8 @@ __libelf_readall (Elf *elf)
> > __libelf_seterrno (ELF_E_NOMEM);
> >
> > /* Free the locks on the children. */
> > - libelf_release_all (elf);
> > + libelf_release_all (elf); // lock is released
> > }
> >
> > - rwlock_unlock (elf->lock);
> > -
> > return (char *) elf->map_address;
> > }
>
> I think this is wrong when this if statement, at the start of the
> block, fails:
>
> /* If the file is not mmap'ed and not previously loaded, do it now. */
> if (elf->map_address == NULL)
>
> So if elf->map_address != NULL we now never call
> rwlock_unlock (elf->lock).
>
> One way to simplify this locking might be to rewrite libelf_acquire_all
> and libelf_release_all to libelf_acquire_all_children and
> libelf_release_all_children (which would only be called with the elf-
> >lock already acquired).
>
> __libelf_readall is the only caller of these functions.
>
> Cheers,
>
> Mark
>
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH 04/16] libelf: Fix deadlock in elf_cntl
2023-10-10 15:23 ` Mark Wielaard
@ 2023-10-17 19:14 ` Heather McIntyre
0 siblings, 0 replies; 51+ messages in thread
From: Heather McIntyre @ 2023-10-17 19:14 UTC (permalink / raw)
To: Mark Wielaard; +Cc: elfutils-devel
[-- Attachment #1: Type: text/plain, Size: 2341 bytes --]
You are right. I changed the code to just rely on if (__libelf_readall
(elf) == NULL) and this seems to work just fine.
On Tue, Oct 10, 2023 at 10:23 AM Mark Wielaard <mark@klomp.org> wrote:
> Hi Heather,
>
> On Tue, 2023-10-10 at 15:42 +0200, Mark Wielaard wrote:
> > From: Heather McIntyre <hsm2@rice.edu>
> >
> > * libelf/elf_cntl.c (elf_cntl): Move rwlock_wrlock, rwlock_unlock,
> > inside case switch statements.
> >
> > Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
> > Signed-off-by: Mark Wielaard <mark@klomp.org>
> > ---
> > libelf/elf_cntl.c | 11 +++++++----
> > 1 file changed, 7 insertions(+), 4 deletions(-)
> >
> > diff --git a/libelf/elf_cntl.c b/libelf/elf_cntl.c
> > index 04aa9132..64087c7d 100644
> > --- a/libelf/elf_cntl.c
> > +++ b/libelf/elf_cntl.c
> > @@ -48,13 +48,16 @@ elf_cntl (Elf *elf, Elf_Cmd cmd)
> > return -1;
> > }
> >
> > - rwlock_wrlock (elf->lock);
> > +
> >
> > switch (cmd)
> > {
> > case ELF_C_FDREAD:
> > + rwlock_rdlock (elf->lock);
> > + int addr_isnull = elf->map_address == NULL;
> > + rwlock_unlock(elf->lock);
> > /* If not all of the file is in the memory read it now. */
> > - if (elf->map_address == NULL && __libelf_readall (elf) == NULL)
> > + if (addr_isnull && __libelf_readall (elf) == NULL)
> > {
> > /* We were not able to read everything. */
> > result = -1;
>
> Can't we just rely on if (__libelf_readall (elf) == NULL)?
>
> __libelf_readall already does locking and will return non-NULL if elf-
> >map_address is already set. So it looks like the extra check (and
> locking) to check addr_isnull is redundant and just make the code more
> complex.
>
> > @@ -64,7 +67,9 @@ elf_cntl (Elf *elf, Elf_Cmd cmd)
> >
> > case ELF_C_FDDONE:
> > /* Mark the file descriptor as not usable. */
> > + rwlock_wrlock (elf->lock);
> > elf->fildes = -1;
> > + rwlock_unlock (elf->lock);
> > break;
> >
> > default:
>
> This looks correct. All other accesses to elf->fildes seem to be done
> under the elf->lock too.
>
> > @@ -73,7 +78,5 @@ elf_cntl (Elf *elf, Elf_Cmd cmd)
> > break;
> > }
> >
> > - rwlock_unlock (elf->lock);
> > -
> > return result;
> > }
>
>
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH 09/16] src: Use eu-search in nm and findtextrel.
2023-10-10 21:25 ` Mark Wielaard
@ 2023-10-17 19:20 ` Heather McIntyre
0 siblings, 0 replies; 51+ messages in thread
From: Heather McIntyre @ 2023-10-17 19:20 UTC (permalink / raw)
To: Mark Wielaard; +Cc: elfutils-devel
[-- Attachment #1: Type: text/plain, Size: 5953 bytes --]
Thank you for pointing out that changes to these two files are not needed
due to them being single threaded utilities. I reverted the changes and ran
some tests. Works just fine with the original search.h, tsearch, and tfind.
On Tue, Oct 10, 2023 at 4:26 PM Mark Wielaard <mark@klomp.org> wrote:
> Hi Heather,
>
> On Tue, Oct 10, 2023 at 03:42:53PM +0200, Mark Wielaard wrote:
> > From: Heather McIntyre <hsm2@rice.edu>
> >
> > * src/Makefile.am: Add USE_LOCKS condition for -pthread.
> > * src/findtextrel.c: Add eu-search.h and remove search.h.
> > Change calls of tsearch/tfind to eu_tsearch/eu_tfind.
> > * src/nm.c: Likewise.
>
> This does look technically correct. But both nm and findtextrel are
> single threaded programs. It might be interesting to try to make them
> parallel. But currently I think this would just introduce unnecessary
> locking.
>
> Cheers,
>
> Mark
>
> > Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
> > Signed-off-by: Mark Wielaard <mark@klomp.org>
> > ---
> > src/Makefile.am | 3 +++
> > src/findtextrel.c | 10 +++++-----
> > src/nm.c | 10 +++++-----
> > 3 files changed, 13 insertions(+), 10 deletions(-)
> >
> > diff --git a/src/Makefile.am b/src/Makefile.am
> > index 10d59a48..fea5d43e 100644
> > --- a/src/Makefile.am
> > +++ b/src/Makefile.am
> > @@ -22,6 +22,9 @@ DEFS += $(YYDEBUG) -DDEBUGPRED=@DEBUGPRED@ \
> > AM_CPPFLAGS += -I$(srcdir)/../libelf -I$(srcdir)/../libebl \
> > -I$(srcdir)/../libdw -I$(srcdir)/../libdwelf \
> > -I$(srcdir)/../libdwfl -I$(srcdir)/../libasm
> > +if USE_LOCKS
> > + AM_CFLAGS += -pthread
> > +endif
> >
> > AM_LDFLAGS = -Wl,-rpath-link,../libelf:../libdw $(STACK_USAGE_NO_ERROR)
> >
> > diff --git a/src/findtextrel.c b/src/findtextrel.c
> > index d3021a3a..5ac4c29a 100644
> > --- a/src/findtextrel.c
> > +++ b/src/findtextrel.c
> > @@ -27,7 +27,7 @@
> > #include <gelf.h>
> > #include <libdw.h>
> > #include <locale.h>
> > -#include <search.h>
> > +#include <eu-search.h>
> > #include <stdbool.h>
> > #include <stdio.h>
> > #include <stdlib.h>
> > @@ -501,10 +501,10 @@ check_rel (size_t nsegments, struct segments
> segments[nsegments],
> > /* There can be more than one relocation against one file.
> > Try to avoid multiple messages. And yes, the code uses
> > pointer comparison. */
> > - if (tfind (src, knownsrcs, ptrcompare) == NULL)
> > + if (eu_tfind (src, knownsrcs, ptrcompare) == NULL)
> > {
> > printf (_("%s not compiled with -fpic/-fPIC\n"), src);
> > - tsearch (src, knownsrcs, ptrcompare);
> > + eu_tsearch (src, knownsrcs, ptrcompare);
> > }
> > return;
> > }
> > @@ -555,12 +555,12 @@ check_rel (size_t nsegments, struct segments
> segments[nsegments],
> > if (sym->st_value + sym->st_size > addr)
> > {
> > /* It is this function. */
> > - if (tfind (lowstr, knownsrcs, ptrcompare) == NULL)
> > + if (eu_tfind (lowstr, knownsrcs, ptrcompare) ==
> NULL)
> > {
> > printf (_("\
> > the file containing the function '%s' is not compiled with
> -fpic/-fPIC\n"),
> > lowstr);
> > - tsearch (lowstr, knownsrcs, ptrcompare);
> > + eu_tsearch (lowstr, knownsrcs, ptrcompare);
> > }
> > }
> > else if (highidx == -1)
> > diff --git a/src/nm.c b/src/nm.c
> > index fbdee8e1..44c20fb2 100644
> > --- a/src/nm.c
> > +++ b/src/nm.c
> > @@ -32,7 +32,7 @@
> > #include <libdw.h>
> > #include <locale.h>
> > #include <obstack.h>
> > -#include <search.h>
> > +#include <eu-search.h>
> > #include <stdbool.h>
> > #include <stdio.h>
> > #include <stdio_ext.h>
> > @@ -537,7 +537,7 @@ static int
> > get_global (Dwarf *dbg __attribute__ ((unused)), Dwarf_Global *global,
> > void *arg __attribute__ ((unused)))
> > {
> > - tsearch (memcpy (xmalloc (sizeof (Dwarf_Global)), global,
> > + eu_tsearch (memcpy (xmalloc (sizeof (Dwarf_Global)), global,
> > sizeof (Dwarf_Global)),
> > &global_root, global_compare);
> >
> > @@ -696,7 +696,7 @@ get_local_names (Dwarf *dbg)
> > /* Check whether a similar local_name is already in the
> > cache. That should not happen. But if it does, we
> > don't want to leak memory. */
> > - struct local_name **tres = tsearch (newp, &local_root,
> > + struct local_name **tres = eu_tsearch (newp, &local_root,
> > local_compare);
> > if (tres == NULL)
> > error_exit (errno, _("cannot create search tree"));
> > @@ -1387,7 +1387,7 @@ show_symbols (int fd, Ebl *ebl, GElf_Ehdr *ehdr,
> > && global_root != NULL)
> > {
> > Dwarf_Global fake = { .name = symstr };
> > - Dwarf_Global **found = tfind (&fake, &global_root,
> > + Dwarf_Global **found = eu_tfind (&fake, &global_root,
> > global_compare);
> > if (found != NULL)
> > {
> > @@ -1442,7 +1442,7 @@ show_symbols (int fd, Ebl *ebl, GElf_Ehdr *ehdr,
> > .lowpc = sym->st_value,
> > .highpc = sym->st_value,
> > };
> > - struct local_name **found = tfind (&fake, &local_root,
> > + struct local_name **found = eu_tfind (&fake, &local_root,
> > local_compare);
> > if (found != NULL)
> > {
> > --
> > 2.41.0
> >
>
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH 10/16] libdw: make dwarf_getalt thread-safe
2023-10-10 22:02 ` Mark Wielaard
@ 2023-10-17 19:25 ` Heather McIntyre
0 siblings, 0 replies; 51+ messages in thread
From: Heather McIntyre @ 2023-10-17 19:25 UTC (permalink / raw)
To: Mark Wielaard; +Cc: elfutils-devel
[-- Attachment #1: Type: text/plain, Size: 3737 bytes --]
Since it wasn't too complicated to do, I implemented a dwarf object lock
(per struct dwarf lock) instead of having the static global lock. As per
your suggestion, I placed the lock over the whole dwarf_getalt function, so
now find_debug_altlink is called with the lock already acquired. In
addition, I placed locking around dwarf_setalt. I am currently in the
process of testing these changes along with others to make sure everything
is ironed out.
On Tue, Oct 10, 2023 at 5:04 PM Mark Wielaard <mark@klomp.org> wrote:
> Hi Heather,
>
> On Tue, Oct 10, 2023 at 03:42:54PM +0200, Mark Wielaard wrote:
> > * libdw/dwarf_getalt.c: Add lock for
> > dbg->alt_dwarf/main->alt_dwarf.
>
> This takes care of dwarf_getalt. Shouldn't dwarf_setalt also use
> locking?
>
> > Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
> > Signed-off-by: Mark Wielaard <mark@klomp.org>
> > ---
> > libdw/dwarf_getalt.c | 27 ++++++++++++++++++++++-----
> > 1 file changed, 22 insertions(+), 5 deletions(-)
> >
> > diff --git a/libdw/dwarf_getalt.c b/libdw/dwarf_getalt.c
> > index 0a12dfae..e3894c8c 100644
> > --- a/libdw/dwarf_getalt.c
> > +++ b/libdw/dwarf_getalt.c
> > @@ -44,6 +44,10 @@
> > #include <sys/types.h>
> > #include <sys/stat.h>
> >
> > +/* find_debug_altlink() modifies "dbg->alt_dwarf".
> > + dwarf_getalt() reads "main->alt_dwarf".
> > + Mutual exclusion is enforced to prevent a race. */
> > +rwlock_define(static, alt_dwarf_lock);
>
> Probably overkill, but should we consider a Dwarf lock object instead
> of having a static global lock?
>
> > char *
> > internal_function
> > @@ -152,7 +156,9 @@ find_debug_altlink (Dwarf *dbg)
> > Dwarf *alt = dwarf_begin (fd, O_RDONLY);
> > if (alt != NULL)
> > {
> > + rwlock_wrlock(alt_dwarf_lock);
> > dbg->alt_dwarf = alt;
> > + rwlock_unlock(alt_dwarf_lock);
> > dbg->alt_fd = fd;
> > }
> > else
>
> Is this lock wide enough? See also below. It looks like multiple
> threads could arrive at this point at the same time. so alt_dwarf and
> alt_fd can be (re)set multiple times, causing leaks of the Dwarf and
> fd.
>
> > @@ -163,22 +169,33 @@ find_debug_altlink (Dwarf *dbg)
> > Dwarf *
> > dwarf_getalt (Dwarf *main)
> > {
> > + rwlock_rdlock(alt_dwarf_lock);
> > + Dwarf *alt_dwarf_local = main->alt_dwarf;
> > + rwlock_unlock(alt_dwarf_lock);
> > +
> > /* Only try once. */
> > - if (main == NULL || main->alt_dwarf == (void *) -1)
> > + if (main == NULL || alt_dwarf_local == (void *) -1)
> > return NULL;
> >
> > - if (main->alt_dwarf != NULL)
> > - return main->alt_dwarf;
> > + if (alt_dwarf_local != NULL)
> > + return alt_dwarf_local;
> >
>
> So at this point it looks like we can have multiple threads running
> (because the lock has been dropped above) all with alt_dwarf_local
> (and main->alt_dwarf) being NULL.
>
> > find_debug_altlink (main);
> >
> > + rwlock_rdlock(alt_dwarf_lock);
> > + alt_dwarf_local = main->alt_dwarf;
> > + rwlock_unlock(alt_dwarf_lock);
> > +
> > /* If we found nothing, make sure we don't try again. */
> > - if (main->alt_dwarf == NULL)
> > + if (alt_dwarf_local == NULL)
> > {
> > + rwlock_wrlock(alt_dwarf_lock);
> > main->alt_dwarf = (void *) -1;
> > + rwlock_unlock(alt_dwarf_lock);
> > +
> > return NULL;
> > }
> >
> > - return main->alt_dwarf;
> > + return alt_dwarf_local;
> > }
> > INTDEF (dwarf_getalt)
> > --
> > 2.41.0
> >
>
> It might be better to take the lock over the whole function (and only
> call find_debug_altlink with the lock held).
>
> Cheers,
>
> Mark
>
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH 11/16] libdw: Add locking around __libdw_dieabbrev for dwarf_hasattr
2023-10-10 13:42 ` [PATCH 11/16] libdw: Add locking around __libdw_dieabbrev for dwarf_hasattr Mark Wielaard
2023-10-11 15:10 ` Mark Wielaard
@ 2023-10-17 19:57 ` Heather McIntyre
2023-10-19 22:06 ` Mark Wielaard
1 sibling, 1 reply; 51+ messages in thread
From: Heather McIntyre @ 2023-10-17 19:57 UTC (permalink / raw)
To: Mark Wielaard; +Cc: elfutils-devel
[-- Attachment #1: Type: text/plain, Size: 2704 bytes --]
Hi Mark,
I see now that this is incomplete considering the other places that also
call this function. I do agree that global locking may be heavy if 1)
implemented in all of these locations, or 2) implemented directly in
__libdw_dieabbrev. We could use atomics here directly in __libdw_dieabbrev.
I have given this a try and it is currently passing all tests, including
the new ones I added for data race detection.
I know you mentioned that taking any pthread lock at all might be a big
overhead, but since I implemented a per dwarf struct lock, would using that
be a possibility? Assuming multiple calls to __libdw_dieabbrev will be
working on different dwarf objects.
On Tue, Oct 10, 2023 at 8:44 AM Mark Wielaard <mark@klomp.org> wrote:
> Hi Heather,
>
> On Tue, 2023-10-10 at 15:42 +0200, Mark Wielaard wrote:
> From: Heather McIntyre <hsm2@rice.edu>
>
> * libdw/dwarf_hasattr.c (dwarf_hasattr): Use die_abbrev_lock
> around __libdw_dieabbrev call.
>
> Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
> Signed-off-by: Mark Wielaard <mark@klomp.org>
> ---
> libdw/dwarf_hasattr.c | 9 +++++++++
> 1 file changed, 9 insertions(+)
>
> diff --git a/libdw/dwarf_hasattr.c b/libdw/dwarf_hasattr.c
> index eca08394..92f8de68 100644
> --- a/libdw/dwarf_hasattr.c
> +++ b/libdw/dwarf_hasattr.c
> @@ -34,6 +34,10 @@
> #include <dwarf.h>
> #include "libdwP.h"
>
> +/* dwarf_hasattr() calls __libdw_dieabbrev() in libdwP.h.
> + __libdw_dieabbrev() reads/writes "die->abbrev".
> + Mutual exclusion is enforced around the call to __libdw_dieabbrev to
> prevent a race. */
> +rwlock_define(static, die_abbrev_lock);
>
> dwarf_child, dwarf_getattrs, dwarf_haschildren and dwarf_tag also use
> __libdw_dieabbrev to get the Dwarf_Abbrev pointer for the given
> Dwarf_DIE. Shouldn't they also use such locking? Or have the locking
> inside __libdw_dieabbrev itself?
>
> Also there are many Dwarf_Dies which all start out "lazy" without
> abbrev set. So taking a global static lock, or even taking any pthread
> lock at all might be a big overhead. Is there some way we can do this
> with atomics instead?
>
>
> int
> dwarf_hasattr (Dwarf_Die *die, unsigned int search_name)
> @@ -41,8 +45,13 @@ dwarf_hasattr (Dwarf_Die *die, unsigned int
> search_name)
> if (die == NULL)
> return 0;
>
> + rwlock_wrlock(die_abbrev_lock);
> +
> /* Find the abbreviation entry. */
> Dwarf_Abbrev *abbrevp = __libdw_dieabbrev (die, NULL);
> +
> + rwlock_unlock(die_abbrev_lock);
> +
> if (unlikely (abbrevp == DWARF_END_ABBREV))
> {
> __libdw_seterrno (DWARF_E_INVALID_DWARF);
>
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH 12/16] libdw: Make libdw_find_split_unit thread-safe
2023-10-11 17:17 ` Mark Wielaard
@ 2023-10-17 20:01 ` Heather McIntyre
0 siblings, 0 replies; 51+ messages in thread
From: Heather McIntyre @ 2023-10-17 20:01 UTC (permalink / raw)
To: Mark Wielaard; +Cc: elfutils-devel
[-- Attachment #1: Type: text/plain, Size: 5714 bytes --]
Hi Mark,
As per your suggestion, I have placed the lock around the entire
__libdw_find_split_unit function call.
On Wed, Oct 11, 2023 at 12:17 PM Mark Wielaard <mark@klomp.org> wrote:
> Hi Heather,
>
> On Tue, Oct 10, 2023 at 03:42:56PM +0200, Mark Wielaard wrote:
> > From: Heather McIntyre <hsm2@rice.edu>
> >
> > * (try_split_file): Use eu_tsearch.
> > Add lock for cu->split.
> >
> > Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
> > Signed-off-by: Mark Wielaard <mark@klomp.org>
> > ---
> > libdw/libdw_find_split_unit.c | 43 +++++++++++++++++++++++++++++------
> > 1 file changed, 36 insertions(+), 7 deletions(-)
> >
> > diff --git a/libdw/libdw_find_split_unit.c
> b/libdw/libdw_find_split_unit.c
> > index 533f8072..a288e9bd 100644
> > --- a/libdw/libdw_find_split_unit.c
> > +++ b/libdw/libdw_find_split_unit.c
> > @@ -34,13 +34,19 @@
> > #include "libelfP.h"
> >
> > #include <limits.h>
> > -#include <search.h>
> > +#include <eu-search.h>
> > #include <stdlib.h>
> > #include <string.h>
> > #include <sys/types.h>
> > #include <sys/stat.h>
> > #include <fcntl.h>
> >
> > +/* __libdw_link_skel_split() modifies "skel->split = split"
> > + and "split->split = skel".
> > + "cu->split" is read at multiple locations and conditionally updated.
> > + Mutual exclusion is enforced to prevent a race. */
> > +rwlock_define(static, cu_split_lock);
>
> __libdw_link_skel_split is defined in libdw/libdwP.h and is called in
> try_split_file. (It is also called in src/readelf.c but that is a
> terrible hack, so ignore that for now.)
>
> A Dwarf_CU is created (see libdw/libdw_findcu.c) with split set to
> (Dwarf_CU *) -1 to indicate the split dwarf is not yet set. If
> cu->split is NULL it means the split dwarf was searched for, but not
> found.
>
> > static void
> > try_split_file (Dwarf_CU *cu, const char *dwo_path)
> > {
> > @@ -57,7 +63,7 @@ try_split_file (Dwarf_CU *cu, const char *dwo_path)
> > if (split->unit_type == DW_UT_split_compile
> > && cu->unit_id8 == split->unit_id8)
> > {
> > - if (tsearch (split->dbg, &cu->dbg->split_tree,
> > + if (eu_tsearch (split->dbg, &cu->dbg->split_tree,
> > __libdw_finddbg_cb) == NULL)
> > {
> > /* Something went wrong. Don't link. */
> > @@ -65,9 +71,13 @@ try_split_file (Dwarf_CU *cu, const char *dwo_path)
> > break;
> > }
> >
> > + rwlock_wrlock(cu_split_lock);
> > +
> > /* Link skeleton and split compile units. */
> > __libdw_link_skel_split (cu, split);
> >
> > + rwlock_unlock(cu_split_lock);
> > +
>
> Is this locking wide enough? It seems like multiple threads can race
> to this code and set different Dwarfs (which means they'll leak). See
> below.
>
> > /* We have everything we need from this ELF
> > file. And we are going to close the fd to
> > not run out of file descriptors. */
> > @@ -75,8 +85,13 @@ try_split_file (Dwarf_CU *cu, const char *dwo_path)
> > break;
> > }
> > }
> > +
> > + rwlock_rdlock(cu_split_lock);
> > +
> > if (cu->split == (Dwarf_CU *) -1)
> > dwarf_end (split_dwarf);
> > +
> > + rwlock_unlock(cu_split_lock);
>
> This means __libdw_link_skel_split wasn't called above (so the
> split_dwarf created by dwarf_begin didn't contain the expected CU).
>
> > }
> > /* Always close, because we don't want to run out of file
> > descriptors. See also the elf_fcntl ELF_C_FDDONE call
> > @@ -89,9 +104,13 @@ Dwarf_CU *
> > internal_function
> > __libdw_find_split_unit (Dwarf_CU *cu)
> > {
> > + rwlock_rdlock(cu_split_lock);
> > + Dwarf_CU *cu_split_local = cu->split;
> > + rwlock_unlock(cu_split_lock);
> > +
> > /* Only try once. */
> > - if (cu->split != (Dwarf_CU *) -1)
> > - return cu->split;
> > + if (cu_split_local != (Dwarf_CU *) -1)
> > + return cu_split_local;
> >
> > /* We need a skeleton unit with a comp_dir and [GNU_]dwo_name
> attributes.
> > The split unit will be the first in the dwo file and should have
> the
> > @@ -116,7 +135,11 @@ __libdw_find_split_unit (Dwarf_CU *cu)
> > free (dwo_path);
> > }
> >
> > - if (cu->split == (Dwarf_CU *) -1)
> > + rwlock_rdlock(cu_split_lock);
> > + cu_split_local = cu->split;
> > + rwlock_unlock(cu_split_lock);
> > +
> > + if (cu_split_local == (Dwarf_CU *) -1)
> > {
> > /* Try compdir plus dwo_name. */
> > Dwarf_Attribute compdir;
>
> It looks like multiple threads can end up here after having seen
> cu->split/cu_split_local == (Dwarf_CU *) -1) Which means multiple
> threads will enter try_split_file and might all call
> __libdw_link_skel_split as mentioned above.
>
> > @@ -138,9 +161,15 @@ __libdw_find_split_unit (Dwarf_CU *cu)
> > }
> > }
> >
> > + rwlock_wrlock(cu_split_lock);
> > +
> > /* If we found nothing, make sure we don't try again. */
> > if (cu->split == (Dwarf_CU *) -1)
> > - cu->split = NULL;
> > + {
> > + cu->split = NULL;
> > + cu_split_local = cu->split;
> > + }
> >
> > - return cu->split;
> > + rwlock_unlock(cu_split_lock);
> > + return cu_split_local;
> > }
>
> I think it would be better to keep the lock during the whole
> __libdw_find_split_unit call.
>
> Cheers,
>
> Mark
>
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH 13/16] libdw: Make libdw_findcu thread-safe
2023-10-12 22:02 ` Mark Wielaard
@ 2023-10-17 20:10 ` Heather McIntyre
0 siblings, 0 replies; 51+ messages in thread
From: Heather McIntyre @ 2023-10-17 20:10 UTC (permalink / raw)
To: Mark Wielaard; +Cc: elfutils-devel
[-- Attachment #1: Type: text/plain, Size: 5021 bytes --]
Hi Mark,
For the call to __libdw_intern_next_unit in __libdw_findcu, I have updated
the locking to the per struct drawf lock. Although I have not encountered a
data race report regarding the call to __libdw_intern_next_unit in
dwarf_formref_die, I have also placed the updated locking there as well.
On Thu, Oct 12, 2023 at 5:03 PM Mark Wielaard <mark@klomp.org> wrote:
> Hi Heather,
>
> On Tue, Oct 10, 2023 at 03:42:57PM +0200, Mark Wielaard wrote:
> > From: Heather McIntyre <hsm2@rice.edu>
> >
> > * libdw/libdw_findcu.c (findcu_cb): Use eu_tsearch.
> > (__libdw_findcu): Use eu_tfind and next_tucu_offset_lock.
> > (__libdw_findcu_addr): Use eu_tfind.
> > (__libdw_find_split_dbg_addr): Likewise.
> >
> > Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
> > Signed-off-by: Mark Wielaard <mark@klomp.org>
> > ---
> > libdw/libdw_findcu.c | 54 ++++++++++++++++++++++++++++----------------
> > 1 file changed, 35 insertions(+), 19 deletions(-)
> >
> > diff --git a/libdw/libdw_findcu.c b/libdw/libdw_findcu.c
> > index ed744231..e546fb0f 100644
> > --- a/libdw/libdw_findcu.c
> > +++ b/libdw/libdw_findcu.c
> > @@ -32,9 +32,13 @@
> > #endif
> >
> > #include <assert.h>
> > -#include <search.h>
> > +#include <eu-search.h>
> > #include "libdwP.h"
> >
> > +/* __libdw_findcu modifies "&dbg->next_tu_offset :
> &dbg->next_cu_offset".
> > + May read or write, so mutual exclusion is enforced to prevent a
> race. */
> > +rwlock_define(static, next_tucu_offset_lock);
> > +
>
> Could this be a per struct Dwarf lock?
>
> > static int
> > findcu_cb (const void *arg1, const void *arg2)
> > {
> > @@ -213,7 +217,7 @@ __libdw_intern_next_unit (Dwarf *dbg, bool
> debug_types)
> >
> > Dwarf_Sig8_Hash_insert (&dbg->sig8_hash, unit_id8, newp);
> >
> > /* Add the new entry to the search tree. */
> > - if (tsearch (newp, tree, findcu_cb) == NULL)
> > + if (eu_tsearch (newp, tree, findcu_cb) == NULL)
> > {
> > /* Something went wrong. Undo the operation. */
> > *offsetp = oldoff;
>
> This is OK since when __libdw_intern_next_unit is called from
> __libdw_findcu the next_tucu_offset_lock is held.
>
> But there is also a call to __libdw_intern_next_unit from
> dwarf_formref_die. So there should also be a lock there?
>
> > @@ -234,28 +238,40 @@ __libdw_findcu (Dwarf *dbg, Dwarf_Off start, bool
> v4_debug_types)
> >
> > /* Maybe we already know that CU. */
> > struct Dwarf_CU fake = { .start = start, .end = 0 };
> > - struct Dwarf_CU **found = tfind (&fake, tree, findcu_cb);
> > + struct Dwarf_CU **found = eu_tfind (&fake, tree, findcu_cb);
> > + struct Dwarf_CU *result = NULL;
> > +
> > if (found != NULL)
> > return *found;
> >
> > - if (start < *next_offset)
> > - {
> > - __libdw_seterrno (DWARF_E_INVALID_DWARF);
> > - return NULL;
> > - }
> > + rwlock_wrlock(next_tucu_offset_lock);
> >
> > - /* No. Then read more CUs. */
> > - while (1)
> > + if (start < *next_offset)
> > + __libdw_seterrno (DWARF_E_INVALID_DWARF);
> > + else
> > {
> > - struct Dwarf_CU *newp = __libdw_intern_next_unit (dbg,
> v4_debug_types);
> > - if (newp == NULL)
> > - return NULL;
> > + /* No. Then read more CUs. */
> > + while (1)
> > + {
> > + struct Dwarf_CU *newp = __libdw_intern_next_unit (dbg,
> > +
> v4_debug_types);
> > + if (newp == NULL)
> > + {
> > + result = NULL;
> > + break;
> > + }
> >
> > - /* Is this the one we are looking for? */
> > - if (start < *next_offset || start == newp->start)
> > - return newp;
> > + /* Is this the one we are looking for? */
> > + if (start < *next_offset || start == newp->start)
> > + {
> > + result = newp;
> > + break;
> > + }
> > + }
> > }
> > - /* NOTREACHED */
> > +
> > + rwlock_unlock(next_tucu_offset_lock);
> > + return result;
> > }
>
> This look OK.
>
> > struct Dwarf_CU *
> > @@ -283,7 +299,7 @@ __libdw_findcu_addr (Dwarf *dbg, void *addr)
> > return NULL;
> >
> > struct Dwarf_CU fake = { .start = start, .end = 0 };
> > - struct Dwarf_CU **found = tfind (&fake, tree, findcu_cb);
> > + struct Dwarf_CU **found = eu_tfind (&fake, tree, findcu_cb);
> >
> > if (found != NULL)
> > return *found;
> > @@ -298,7 +314,7 @@ __libdw_find_split_dbg_addr (Dwarf *dbg, void *addr)
> > /* XXX Assumes split DWARF only has CUs in main IDX_debug_info. */
> > Elf_Data fake_data = { .d_buf = addr, .d_size = 0 };
> > Dwarf fake = { .sectiondata[IDX_debug_info] = &fake_data };
> > - Dwarf **found = tfind (&fake, &dbg->split_tree, __libdw_finddbg_cb);
> > + Dwarf **found = eu_tfind (&fake, &dbg->split_tree,
> __libdw_finddbg_cb);
> >
> > if (found != NULL)
> > return *found;
>
> OK.
>
> Thanks,
>
> Mark
>
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH 07/16] lib: Add eu_tsearch and eu_tfind
2023-10-10 16:51 ` Mark Wielaard
@ 2023-10-17 20:52 ` Heather McIntyre
0 siblings, 0 replies; 51+ messages in thread
From: Heather McIntyre @ 2023-10-17 20:52 UTC (permalink / raw)
To: Mark Wielaard; +Cc: elfutils-devel
[-- Attachment #1: Type: text/plain, Size: 7697 bytes --]
Hi Mark,
I think having unique per-root locks might be a good idea. From a brief
test, I saw around 70 roots created when parsing the test file I have been
using. I have an initial idea for this that I don't think will be too
complicated. I will go over my idea with John to get his input and then get
back to you.
Best,
Heather
On Tue, Oct 10, 2023 at 11:51 AM Mark Wielaard <mark@klomp.org> wrote:
> Hi Heather,
>
> On Tue, 2023-10-10 at 15:42 +0200, Mark Wielaard wrote:
> > From: Heather McIntyre <hsm2@rice.edu>
> >
> > * lib/eu-search.h: New file.
> > Declarations for read/write locked eu_tsearch/eu_tfind.
>
> Like in the previous case, don't forget to add such a new header to
> noinst_HEADERS so make dist adds them.
>
> diff --git a/lib/Makefile.am b/lib/Makefile.am
> index ce8f3e1b..9df0a8c6 100644
> --- a/lib/Makefile.am
> +++ b/lib/Makefile.am
> @@ -39,5 +39,6 @@ libeu_a_SOURCES = xasprintf.c xstrdup.c xstrndup.c
> xmalloc.c next_prime.c \
>
> noinst_HEADERS = fixedsizehash.h libeu.h system.h dynamicsizehash.h
> list.h \
> eu-config.h color.h printversion.h bpf.h \
> - atomics.h stdatomic-fbsd.h dynamicsizehash_concurrent.h
> + atomics.h stdatomic-fbsd.h dynamicsizehash_concurrent.h \
> + eu-search.h
> EXTRA_DIST = dynamicsizehash.c dynamicsizehash_concurrent.c
>
> > * lib/eu-search.c: New file.
> > Definitions for read/write locked eu_tsearch/eu_tfind.
> > * Makefile.am (libeu_a_SOURCES): Add eu-search.c.
> >
> > Signed-off-by: Heather S. McIntyre <hsm2@rice.edu>
> > Signed-off-by: Mark Wielaard <mark@klomp.org>
> > ---
> > lib/Makefile.am | 2 +-
> > lib/eu-search.c | 60 +++++++++++++++++++++++++++++++++++++++++++++++++
> > lib/eu-search.h | 39 ++++++++++++++++++++++++++++++++
> > 3 files changed, 100 insertions(+), 1 deletion(-)
> > create mode 100644 lib/eu-search.c
> > create mode 100644 lib/eu-search.h
> >
> > diff --git a/lib/Makefile.am b/lib/Makefile.am
> > index b3bb929f..ce8f3e1b 100644
> > --- a/lib/Makefile.am
> > +++ b/lib/Makefile.am
> > @@ -34,7 +34,7 @@ AM_CPPFLAGS += -I$(srcdir)/../libelf
> > noinst_LIBRARIES = libeu.a
> >
> > libeu_a_SOURCES = xasprintf.c xstrdup.c xstrndup.c xmalloc.c
> next_prime.c \
> > - crc32.c crc32_file.c \
> > + crc32.c crc32_file.c eu-search.c \
> > color.c error.c printversion.c
>
> OK.
>
> > noinst_HEADERS = fixedsizehash.h libeu.h system.h dynamicsizehash.h
> list.h \
> > diff --git a/lib/eu-search.c b/lib/eu-search.c
> > new file mode 100644
> > index 00000000..a6b04f4f
> > --- /dev/null
> > +++ b/lib/eu-search.c
> > @@ -0,0 +1,60 @@
> > +/* Definitions for thread-safe tsearch/tfind
> > + Copyright (C) 2023 Rice University
> > + This file is part of elfutils.
> > +
> > + This file is free software; you can redistribute it and/or modify
> > + it under the terms of either
> > +
> > + * the GNU Lesser General Public License as published by the Free
> > + Software Foundation; either version 3 of the License, or (at
> > + your option) any later version
> > +
> > + or
> > +
> > + * the GNU General Public License as published by the Free
> > + Software Foundation; either version 2 of the License, or (at
> > + your option) any later version
> > +
> > + or both in parallel, as here.
> > +
> > + elfutils is distributed in the hope that it will be useful, but
> > + WITHOUT ANY WARRANTY; without even the implied warranty of
> > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> > + General Public License for more details.
> > +
> > + You should have received copies of the GNU General Public License and
> > + the GNU Lesser General Public License along with this program. If
> > + not, see <
> https://urldefense.com/v3/__http://www.gnu.org/licenses/__;!!BuQPrrmRaQ!j_izX_1hkv80LnClL5l2GD-1nTSH3dTTKjXsHtkYvr7TyudmXWLEZ7zOOkpQZFgo_0QKiWAkkvj7SBE54w$
> >. */
> > +
> > +#ifdef HAVE_CONFIG_H
> > +#include <config.h>
> > +#endif
> > +
> > +#include <stdlib.h>
> > +#include "eu-search.h"
> > +
> > +rwlock_define(static, search_find_lock);
> > +
> > +void *eu_tsearch(const void *key, void **rootp,
> > + int (*compar)(const void *, const void *))
> > +{
> > + void *ret = NULL;
> > + rwlock_wrlock(search_find_lock);
> > +
> > + ret = tsearch(key, rootp, compar);
> > +
> > + rwlock_unlock(search_find_lock);
> > + return ret;
> > +}
> > +
> > +void *eu_tfind(const void *key, void *const *rootp,
> > + int (*compar)(const void *, const void *))
> > +{
> > + void *ret = NULL;
> > + rwlock_rdlock(search_find_lock);
> > +
> > + ret = tfind(key, rootp, compar);
> > +
> > + rwlock_unlock(search_find_lock);
> > + return ret;
> > +}
>
> So this uses on static lock for all eu-tsearch/eu-tfind calls. But
> searches with different roots should be independent. Would it make
> sense to add different locks for different roots?
>
> That would make the code a little more complicated, but we could hide
> the root and lock in a new structure that will be passed the the
> eu_tsearch/eu_tfind calls.
>
> Maybe something like:
>
> struct eu_search_root
> {
> void *root;
> rwlock_define (,lock);
> };
>
> With helper functions to create/init and destroy/delete them?
> void eu_tinit (struct eu_search_root *search_root);
> (Or is there no real initing to do?)
>
> void eu_tdestroy (struct eu_search_root *search_root,
> void (*free_node)(void *nodep));
>
> Or is all this way too complicated and a global lock just fine?
>
> Cheers,
>
> Mark
>
> > diff --git a/lib/eu-search.h b/lib/eu-search.h
> > new file mode 100644
> > index 00000000..4ce0139a
> > --- /dev/null
> > +++ b/lib/eu-search.h
> > @@ -0,0 +1,39 @@
> > +/* Calls for thread-safe tsearch/tfind
> > + Copyright (C) 2023 Rice University
> > + This file is part of elfutils.
> > +
> > + This file is free software; you can redistribute it and/or modify
> > + it under the terms of either
> > +
> > + * the GNU Lesser General Public License as published by the Free
> > + Software Foundation; either version 3 of the License, or (at
> > + your option) any later version
> > +
> > + or
> > +
> > + * the GNU General Public License as published by the Free
> > + Software Foundation; either version 2 of the License, or (at
> > + your option) any later version
> > +
> > + or both in parallel, as here.
> > +
> > + elfutils is distributed in the hope that it will be useful, but
> > + WITHOUT ANY WARRANTY; without even the implied warranty of
> > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> > + General Public License for more details.
> > +
> > + You should have received copies of the GNU General Public License and
> > + the GNU Lesser General Public License along with this program. If
> > + not, see <
> https://urldefense.com/v3/__http://www.gnu.org/licenses/__;!!BuQPrrmRaQ!j_izX_1hkv80LnClL5l2GD-1nTSH3dTTKjXsHtkYvr7TyudmXWLEZ7zOOkpQZFgo_0QKiWAkkvj7SBE54w$
> >. */
> > +
> > +#ifndef EU_SEARCH_H
> > +#define EU_SEARCH_H 1
> > +
> > +#include <search.h>
> > +
> > +extern void *eu_tsearch(const void *key, void **rootp,
> > + int (*compar)(const void *, const void *));
> > +extern void *eu_tfind(const void *key, void *const *rootp,
> > + int (*compar)(const void *, const void *));
> > +
> > +#endif
>
>
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH 02/16] libelf: Make elf_version thread-safe
2023-10-17 19:05 ` Heather McIntyre
@ 2023-10-19 21:00 ` Mark Wielaard
0 siblings, 0 replies; 51+ messages in thread
From: Mark Wielaard @ 2023-10-19 21:00 UTC (permalink / raw)
To: Heather McIntyre; +Cc: elfutils-devel
Hi Heather,
On Tue, Oct 17, 2023 at 02:05:58PM -0500, Heather McIntyre wrote:
> John and I discussed that atomic_compare_exchange_strong could have been
> used here. I see that this has been pushed to the main branch, but I can
> make the change to the atomic operation if you think that is a better
> option.
It probably doesn't matter for this function since it isn't used
often. I was just wondering whether you thought of using atomics here
since there are really just two states here (__libelf_version ==
EV_NONE or __libelf_version == EV_CURRENT).
I do have one question though. __libelf_version is checked in
elf_begin. Should that check be guarded by a lock too?
Thanks,
Mark
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH 11/16] libdw: Add locking around __libdw_dieabbrev for dwarf_hasattr
2023-10-17 19:57 ` Heather McIntyre
@ 2023-10-19 22:06 ` Mark Wielaard
0 siblings, 0 replies; 51+ messages in thread
From: Mark Wielaard @ 2023-10-19 22:06 UTC (permalink / raw)
To: Heather McIntyre; +Cc: elfutils-devel
Hi Heather,
On Tue, Oct 17, 2023 at 02:57:39PM -0500, Heather McIntyre wrote:
> I see now that this is incomplete considering the other places that also
> call this function. I do agree that global locking may be heavy if 1)
> implemented in all of these locations, or 2) implemented directly in
> __libdw_dieabbrev. We could use atomics here directly in __libdw_dieabbrev.
> I have given this a try and it is currently passing all tests, including
> the new ones I added for data race detection.
>
> I know you mentioned that taking any pthread lock at all might be a big
> overhead, but since I implemented a per dwarf struct lock, would using that
> be a possibility? Assuming multiple calls to __libdw_dieabbrev will be
> working on different dwarf objects.
I have been thinking about this issue and I think we made a mistake in
designing how a Dwarf_Die is lazy initialized. The abbrev field of a
Dwarf_Die is only set when needed by calling __libdw_dieabbrev, which
means we need some kind of locking or atomic swapping whenever we try
to use that field. I assume the idea originally was that calling
__libdw_dieabbrev is fairly "heavy" (it is, potentially reading the
whole .debug_abbrev for the CU). So we try to postpone it till it is
really needed.
But in practice it is always needed. Without the abbrev field set you
can just call dwarf_dieoffset, dwarf_cuoffset, dwarf_diecu and
dwarf_getabbrev. In theory you could avoid adding the abbrev for the
initial CU DIE for a Dwarf_CU when you are iterating over all CUs and
know you don't need the CU without inspecting the initial CU DIE. But
the Dwarf_Abbrev_Hash for the Dwarf_CU will already be
initialized. And normally the abbrev for the first DIE will also be
the first abbrev, so searching for it should be really quick.
So I think setting the Dwarf_Die abbrev field lazy is not really
helpful and makes the code needlessly complex. If we set the abbrev
field when a Dwarf_Die is created we can simplify the code and don't
need all this locking when we just want to access the field.
This of course is still a lot of coding, we'll have to check every
place that initializes a new Dwarf_Die. Which will have to call
__libdw_dieabbrev directly. But I think that will not need any extra
locking because the Dwarf_Abbrev_Hash used is already thread-safe (it
was written by Srđan Milaković also from Rice University).
What do you think?
Cheers,
Mark
^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH 03/16] libelf: Fix deadlock in __libelf_readall
2023-10-17 19:11 ` Heather McIntyre
@ 2023-11-09 13:26 ` Mark Wielaard
0 siblings, 0 replies; 51+ messages in thread
From: Mark Wielaard @ 2023-11-09 13:26 UTC (permalink / raw)
To: Heather McIntyre; +Cc: elfutils-devel
Hi Heather,
On Tue, 2023-10-17 at 14:11 -0500, Heather McIntyre wrote:
> I will push the changes after I am done testing other fixes
> since I want to ensure that everything works together cohesively.
Please let us know if we can help update your patches. I realize we
have been patching some of the code you are also working on. So if
things don't apply anymore and you need help to figure out how to
adjust your patches please just ask.
Also please don't feel everything needs to be perfect and work together
completely. If you have parts ready that are independently useful
please just submit those parts and we can integrate them early.
Thanks,
Mark
^ permalink raw reply [flat|nested] 51+ messages in thread
end of thread, other threads:[~2023-11-09 13:26 UTC | newest]
Thread overview: 51+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-08-08 17:07 [PATCH] Fix thread-safety for elfutils Heather McIntyre
2023-08-21 22:08 ` John Mellor-Crummey
2023-08-25 14:10 ` Mark Wielaard
2023-10-10 13:40 ` Mark Wielaard
2023-10-10 13:42 ` [PATCH 01/16] lib: Add new once_define and once macros to eu-config.h Mark Wielaard
2023-10-10 13:42 ` [PATCH 02/16] libelf: Make elf_version thread-safe Mark Wielaard
2023-10-10 14:00 ` Mark Wielaard
2023-10-17 19:05 ` Heather McIntyre
2023-10-19 21:00 ` Mark Wielaard
2023-10-10 13:42 ` [PATCH 03/16] libelf: Fix deadlock in __libelf_readall Mark Wielaard
2023-10-10 15:06 ` Mark Wielaard
2023-10-17 19:11 ` Heather McIntyre
2023-11-09 13:26 ` Mark Wielaard
2023-10-10 13:42 ` [PATCH 04/16] libelf: Fix deadlock in elf_cntl Mark Wielaard
2023-10-10 15:23 ` Mark Wielaard
2023-10-17 19:14 ` Heather McIntyre
2023-10-10 13:42 ` [PATCH 05/16] libelf: Fix elf_end deadlock Mark Wielaard
2023-10-10 15:28 ` Mark Wielaard
2023-10-10 13:42 ` [PATCH 06/16] libelf: Make elf32_getchdr and elf64_getchdr thread-safe Mark Wielaard
2023-10-10 16:28 ` Mark Wielaard
2023-10-10 13:42 ` [PATCH 07/16] lib: Add eu_tsearch and eu_tfind Mark Wielaard
2023-10-10 16:51 ` Mark Wielaard
2023-10-17 20:52 ` Heather McIntyre
2023-10-10 13:42 ` [PATCH 08/16] libcpu: Change calls for tsearch/tfind to eu_tsearch/eu_tfind Mark Wielaard
2023-10-10 21:10 ` Mark Wielaard
2023-10-10 13:42 ` [PATCH 09/16] src: Use eu-search in nm and findtextrel Mark Wielaard
2023-10-10 21:25 ` Mark Wielaard
2023-10-17 19:20 ` Heather McIntyre
2023-10-10 13:42 ` [PATCH 10/16] libdw: make dwarf_getalt thread-safe Mark Wielaard
2023-10-10 22:02 ` Mark Wielaard
2023-10-17 19:25 ` Heather McIntyre
2023-10-10 13:42 ` [PATCH 11/16] libdw: Add locking around __libdw_dieabbrev for dwarf_hasattr Mark Wielaard
2023-10-11 15:10 ` Mark Wielaard
2023-10-17 19:57 ` Heather McIntyre
2023-10-19 22:06 ` Mark Wielaard
2023-10-10 13:42 ` [PATCH 12/16] libdw: Make libdw_find_split_unit thread-safe Mark Wielaard
2023-10-11 17:17 ` Mark Wielaard
2023-10-17 20:01 ` Heather McIntyre
2023-10-10 13:42 ` [PATCH 13/16] libdw: Make libdw_findcu thread-safe Mark Wielaard
2023-10-12 22:02 ` Mark Wielaard
2023-10-17 20:10 ` Heather McIntyre
2023-10-10 13:42 ` [PATCH 14/16] libdw,libdwfl: Use eu-search for thread-safety Mark Wielaard
2023-10-12 22:05 ` Mark Wielaard
2023-10-10 13:42 ` [PATCH 15/16] tests: Add eu-search tests Mark Wielaard
2023-10-13 14:38 ` Mark Wielaard
2023-10-10 13:43 ` [PATCH 16/16] configure: No longer mark --enable-thread-safety as EXPERIMENTAL Mark Wielaard
2023-10-12 22:09 ` Mark Wielaard
2023-10-10 13:54 ` [PATCH 01/16] lib: Add new once_define and once macros to eu-config.h Mark Wielaard
2023-10-14 15:39 ` [PATCH] Fix thread-safety for elfutils Mark Wielaard
2023-10-14 18:29 ` Heather McIntyre
2023-10-17 15:04 ` Mark Wielaard
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).