[Bug dynamic-link/30100] New: [patch] add dlmem()

[Bug dynamic-link/30100] New: [patch] add dlmem()

[stsp at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

            Bug ID: 30100
           Summary: [patch] add dlmem()
           Product: glibc
           Version: 2.38
            Status: UNCONFIRMED
          Severity: enhancement
          Priority: P2
         Component: dynamic-link
          Assignee: unassigned at sourceware dot org
          Reporter: stsp at users dot sourceforge.net
  Target Milestone: ---

Created attachment 14663
  --> https://sourceware.org/bugzilla/attachment.cgi?id=14663&action=edit
fix bad free()

Hi guys.

I coded up the dlmem() support discussed here:
https://sourceware.org/bugzilla/show_bug.cgi?id=11767

Currently it consists of 2 patches: small fix to the
pre-existing bad free() problem, and dlmem() itself.
I'll attach them to this ticket.
Yes, I hear your screams "this patch sucks!". But in fact,
when you spend 99% of time thinking how to refactor this
1000+ lines of code function with multiple gotos, and only
1% of time goes to the coding itself, the patch is not going
to be pretty. :)
Also I can't be sure it doesn't introduce a regression.
After I added a member to struct rtld_global_ro, I had
to change the interpreter with patchelf on a tests I need
to run, because they are using the system-wide interpreter
where the struct rtld_global_ro is different. So I have
some tests failures now, but manually patching the interpreter
allows the test to succeed. So I can't properly validate
my patch with all tests.

I really wish someone from glibc to take over that patch. :)
glibc code is exceptionally alien to me, but the patch
works and has a test-case within.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[stsp at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #1 from Stas Sergeev <stsp at users dot sourceforge.net> ---
Created attachment 14664
  --> https://sourceware.org/bugzilla/attachment.cgi?id=14664&action=edit
implement dlmem

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[adhemerval.zanella at linaro dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

Adhemerval Zanella <adhemerval.zanella at linaro dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |adhemerval.zanella at linaro dot o
                   |                            |rg

--- Comment #2 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> ---
Could you send the patch following glibc contribution guidelines [1]? Patches
are only reviewed once there are posted on the maillist.

[1] https://sourceware.org/glibc/wiki/Contribution%20checklist

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[stsp at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #3 from Stas Sergeev <stsp at users dot sourceforge.net> ---
But as indicated above, I have problems running this patch
through the test-suit. All built tests use the system-wide
elf interpreter, so after I changed the "struct rtld_global_ro"
I have to use patchelf on a test binaries to set the interpreter
to the newly-built one.
Please tell me how to solve that problem, because of course
running through the test-suit is a requirement for submission
to the mail list.

Please let me know if there are any alternatives, too, like
eg donating on patreon to someone who can do the remaining work.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[adhemerval.zanella at linaro dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #4 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> ---
(In reply to Stas Sergeev from comment #3)
> But as indicated above, I have problems running this patch
> through the test-suit. All built tests use the system-wide
> elf interpreter, so after I changed the "struct rtld_global_ro"
> I have to use patchelf on a test binaries to set the interpreter
> to the newly-built one.
> Please tell me how to solve that problem, because of course
> running through the test-suit is a requirement for submission
> to the mail list.
> 

The glibc testsuite has tests that either run through the built loader and
libraries (by invoking the loader directly with --library-path or by building
the tests to use the build loader when configure with
--enable-hardcoded-path-in-tests) or use a minimal container (where you can
override system-wide configuration). You can also test a binary with the glibc
loader by using the 'testrun.sh' script in the build folder.

The former is the default for tests, while the latter requires some extra care
(check the tests-container tests on Makefiles).

So first thing you would like to check is whether the new patch adds any
regression by running 'make check'.  Then you need to add any extra tests to
exercise the new feature, if they require changing any global state (such as
global files, etc.) it is preferable to provide them as tests-containers.

> Please let me know if there are any alternatives, too, like
> eg donating on patreon to someone who can do the remaining work.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[stsp at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #5 from Stas Sergeev <stsp at users dot sourceforge.net> ---
Thanks, getting closer!
Now under "make check" I have this crash:

112       assert (ptr == alloc_last_block);
(gdb) bt
#0  __minimal_realloc (ptr=0x7ffff7fc2000, n=1680) at dl-minimal-malloc.c:112
#1  0x00007ffff7fcf3b1 in realloc (size=1680, ptr=<optimized out>)
    at ../include/rtld-malloc.h:62
#2  filebuf_ensure (size=848, fb=0x7fffffffd4e0) at dl-load.c:149


My code calls realloc(), but for some reason it ends up in
__minimal_realloc(). Which is so minimal that it only supports
the realloc of a last-allocated pointer. If some other allocation
was in between, __minimal_realloc() asserts.
How can I use the maximum realloc instead?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[adhemerval.zanella at linaro dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #6 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> ---
(In reply to Stas Sergeev from comment #5)
> Thanks, getting closer!
> Now under "make check" I have this crash:
> 
> 112	  assert (ptr == alloc_last_block);
> (gdb) bt
> #0  __minimal_realloc (ptr=0x7ffff7fc2000, n=1680) at dl-minimal-malloc.c:112
> #1  0x00007ffff7fcf3b1 in realloc (size=1680, ptr=<optimized out>)
>     at ../include/rtld-malloc.h:62
> #2  filebuf_ensure (size=848, fb=0x7fffffffd4e0) at dl-load.c:149
> 
> 
> My code calls realloc(), but for some reason it ends up in
> __minimal_realloc(). Which is so minimal that it only supports
> the realloc of a last-allocated pointer. If some other allocation
> was in between, __minimal_realloc() asserts.
> How can I use the maximum realloc instead?

The minimal malloc should be only used by the loader on very specific cases, it
is a simple bump allocator without support advance features that generic malloc
provides (such as thread, fragmentation, etc).  If you need a more generic
memory allocator, it would be better to either restructure the patch to
pre-allocate the required memory or work with the bump allocator by trying to
use different algorithms that avoid fragmentation (I have not look into the
patch to comment what it would require).

You might try to extend the minimal malloc, but I am not sure if the complexity
would be worth.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[stsp at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #7 from Stas Sergeev <stsp at users dot sourceforge.net> ---
Will it make sense to add __minimal_realloc2(void *ptr, size_t old, size_t n)?
With old_size knowledge it will be able to allocate a new block
and perform a memcpy() if needed, or grow of the last block if suits.
This is not a big addition, should squeeze into 10-15 lines.
Or I can instead just do malloc/memcpy/free as a quick fix in my code.

The amounts of memory are very small: that buffer is used for ELF headers.
But I don't think I can reserve it beforehands, as probably there is no
upper limit on an elf header size, or is there?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[stsp at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #8 from Stas Sergeev <stsp at users dot sourceforge.net> ---
(In reply to Adhemerval Zanella from comment #6)
> trying to use different algorithms that avoid fragmentation

I ended up doing exactly that, got even a smaller code. :)
Testing now, will re-upload.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[stsp at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #9 from Stas Sergeev <stsp at users dot sourceforge.net> ---
"make check" doesn't seem to re-run tests.
It uses pre-existing outputs.
So after every update I need to do "make clean",
after which "make check" runs all tests again.
But that takes hours.
How can I solve that?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[adhemerval.zanella at linaro dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #10 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> ---
(In reply to Stas Sergeev from comment #7)
> Will it make sense to add __minimal_realloc2(void *ptr, size_t old, size_t
> n)?
> With old_size knowledge it will be able to allocate a new block
> and perform a memcpy() if needed, or grow of the last block if suits.
> This is not a big addition, should squeeze into 10-15 lines.
> Or I can instead just do malloc/memcpy/free as a quick fix in my code.
> 
> The amounts of memory are very small: that buffer is used for ELF headers.
> But I don't think I can reserve it beforehands, as probably there is no
> upper limit on an elf header size, or is there?

I am not sure it is possible, the minmalloc symbol are realocated to malloc one
that so the memory allocated can be free by default malloc.  The filebuf struct
is already a static scratch buffer used exactly to avoid realloc, why exactly
do you need a value larger than that?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[adhemerval.zanella at linaro dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #11 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> ---
(In reply to Stas Sergeev from comment #9)
> "make check" doesn't seem to re-run tests.
> It uses pre-existing outputs.
> So after every update I need to do "make clean",
> after which "make check" runs all tests again.
> But that takes hours.
> How can I solve that?

The make check should rerun the tests that the change affected, but
dependending of might indeed clean the *.out file and rerun.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[stsp at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #12 from Stas Sergeev <stsp at users dot sourceforge.net> ---
(In reply to Adhemerval Zanella from comment #10)
> The filebuf
> struct is already a static scratch buffer used exactly to avoid realloc, why
> exactly do you need a value larger than that?

It is used in glibc only for Ehdr.
Then both open_verify() and _dl_map_object_from_fd() have this
(duplicated) piece of code to read the rest:

  maplength = header->e_phnum * sizeof (ElfW(Phdr));
  if (header->e_phoff + maplength <= (size_t) fbp->len)
    phdr = (void *) (fbp->buf + header->e_phoff);
  else
    {
      phdr = alloca (maplength);
      if ((size_t) __pread64_nocancel (fd, (void *) phdr, maplength,
                                       header->e_phoff) != maplength)
        {
          errstring = N_("cannot read file data");
          goto lose_errno;
        }
    }

Because fbp only stores Ehdr, that condition always goes the
second branch, and reads the rest. It does so in both
open_verify() and _dl_map_object_from_fd().
I wanted to share/re-use most of the _dl_map_object_from_fd()
code, but for that I had to unbind it from "fd" as much as possible.
So I decided to de-duplicate the aforementioned piece and allow
it to read all elf headers to filebuf, rather than twice reading
into an alloca() space as in the aforementioned fragment.

But now I do that w/o reallocating because I can defer the allocation
until the header size is known.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[stsp at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #13 from Stas Sergeev <stsp at users dot sourceforge.net> ---
(In reply to Adhemerval Zanella from comment #11)
> The make check should rerun the tests that the change affected, but
> dependending of might indeed clean the *.out file and rerun.

The problem appears not in "clean" but in a fact that when
running the test, cwd is in srcdir! I.e. it runs my tst-dlmem from
glibc/dlfcn instead of build/dlfcn. So the test cannot find
glreflib1.so which it needs to mmap.
So how would I refer to the builddir in my tst-dlmem test-case?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[schwab@linux-m68k.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #14 from Andreas Schwab <schwab@linux-m68k.org> ---
There are several examples in elf/Makefile, use LDFLAGS-rpath-ORIGIN.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[stsp at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #15 from Stas Sergeev <stsp at users dot sourceforge.net> ---
(In reply to Andreas Schwab from comment #14)
> There are several examples in elf/Makefile, use LDFLAGS-rpath-ORIGIN.

I need something like
CFLAGS += -DBUILD_DIR=$(build_dir)
to refer to that from C code.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[stsp at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #16 from Stas Sergeev <stsp at users dot sourceforge.net> ---
Which should be:
CPPFLAGS-tst-dlmem.c += -DBUILDDIR=\"$(objpfx)\"
So found an example in Makefile.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[stsp at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

Stas Sergeev <stsp at users dot sourceforge.net> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #14664|0                           |1
        is obsolete|                            |

--- Comment #17 from Stas Sergeev <stsp at users dot sourceforge.net> ---
Created attachment 14668
  --> https://sourceware.org/bugzilla/attachment.cgi?id=14668&action=edit
implement dlmem

Tests feel much better now.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[stsp at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #18 from Stas Sergeev <stsp at users dot sourceforge.net> ---
Still have dozens of failures like this:

FAIL: conform/XPG42/sys/shm.h/linknamespace
FAIL: conform/XPG42/sys/socket.h/linknamespace
FAIL: conform/XPG42/sys/time.h/linknamespace
FAIL: conform/XPG42/sys/timeb.h/linknamespace
FAIL: conform/XPG42/sys/uio.h/linknamespace
FAIL: conform/XPG42/sys/wait.h/linknamespace
FAIL: conform/XPG42/syslog.h/linknamespace
FAIL: conform/XPG42/termios.h/linknamespace
FAIL: conform/XPG42/time.h/linknamespace
FAIL: conform/XPG42/ucontext.h/linknamespace
FAIL: conform/XPG42/ulimit.h/linknamespace
FAIL: conform/XPG42/unistd.h/linknamespace
FAIL: conform/XPG42/utmpx.h/linknamespace
FAIL: conform/XPG42/wordexp.h/linknamespace
FAIL: elf/check-abi-libc

Not sure if they are related to my patch or not...
To be figured out.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[adhemerval.zanella at linaro dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #19 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> ---
(In reply to Stas Sergeev from comment #18)
> Still have dozens of failures like this:
> 
> FAIL: conform/XPG42/sys/shm.h/linknamespace
> FAIL: conform/XPG42/sys/socket.h/linknamespace
> FAIL: conform/XPG42/sys/time.h/linknamespace
> FAIL: conform/XPG42/sys/timeb.h/linknamespace
> FAIL: conform/XPG42/sys/uio.h/linknamespace
> FAIL: conform/XPG42/sys/wait.h/linknamespace
> FAIL: conform/XPG42/syslog.h/linknamespace
> FAIL: conform/XPG42/termios.h/linknamespace
> FAIL: conform/XPG42/time.h/linknamespace
> FAIL: conform/XPG42/ucontext.h/linknamespace
> FAIL: conform/XPG42/ulimit.h/linknamespace
> FAIL: conform/XPG42/unistd.h/linknamespace
> FAIL: conform/XPG42/utmpx.h/linknamespace
> FAIL: conform/XPG42/wordexp.h/linknamespace
> FAIL: elf/check-abi-libc
> 
> Not sure if they are related to my patch or not...
> To be figured out.

Most likely you are issuing a libc function without using its internal alias
(for instance mmap64 instead of __mmap64).  The linknamespace.out file will
hint you which symbol is polluting the namespace.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[stsp at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #20 from Stas Sergeev <stsp at users dot sourceforge.net> ---

(In reply to Adhemerval Zanella from comment #19)
> Most likely you are issuing a libc function without using its internal alias
> (for instance mmap64 instead of __mmap64).  The linknamespace.out file will
> hint you which symbol is polluting the namespace.

Thanks!
The namespace problems were mostly with me forgetting to mark a
few functions static. I wonder why you put that on a checker, and
not on a -Wmissing-prototype or alike.
Anyway, thanks guys, will re-upload the patches that fully pass
tests now.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[stsp at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

Stas Sergeev <stsp at users dot sourceforge.net> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #14663|0                           |1
        is obsolete|                            |

--- Comment #21 from Stas Sergeev <stsp at users dot sourceforge.net> ---
Created attachment 14670
  --> https://sourceware.org/bugzilla/attachment.cgi?id=14670&action=edit
fix bad free()

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[stsp at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

Stas Sergeev <stsp at users dot sourceforge.net> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #14668|0                           |1
        is obsolete|                            |

--- Comment #22 from Stas Sergeev <stsp at users dot sourceforge.net> ---
Created attachment 14671
  --> https://sourceware.org/bugzilla/attachment.cgi?id=14671&action=edit
implement dlmem

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[stsp at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #23 from Stas Sergeev <stsp at users dot sourceforge.net> ---
Got this after "git pull --rebase":

FAIL: iconv/tst-iconv-opt
FAIL: iconv/tst-iconv9
FAIL: localedata/tst-wctype
FAIL: localedata/tst_mblen
FAIL: localedata/tst_mbrlen
FAIL: localedata/tst_mbsrtowcs
FAIL: localedata/tst_mbstowcs
FAIL: localedata/tst_mbtowc
FAIL: localedata/tst_wcrtomb
FAIL: localedata/tst_wcsrtombs
FAIL: localedata/tst_wcstombs
FAIL: localedata/tst_wctob
FAIL: localedata/tst_wctomb

Likely not related to my patch, as these failures were not
there before pull...

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[stsp at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #24 from Stas Sergeev <stsp at users dot sourceforge.net> ---
(In reply to Stas Sergeev from comment #23)
> Got this after "git pull --rebase":
> 
> FAIL: iconv/tst-iconv-opt
> FAIL: iconv/tst-iconv9
> FAIL: localedata/tst-wctype
> FAIL: localedata/tst_mblen
> FAIL: localedata/tst_mbrlen
> FAIL: localedata/tst_mbsrtowcs
> FAIL: localedata/tst_mbstowcs
> FAIL: localedata/tst_mbtowc
> FAIL: localedata/tst_wcrtomb
> FAIL: localedata/tst_wcsrtombs
> FAIL: localedata/tst_wcstombs
> FAIL: localedata/tst_wctob
> FAIL: localedata/tst_wctomb

Interesting: got that only from "make tests".
"make check" is still good. :)
So the patch might be ok.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[stsp at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #25 from Stas Sergeev <stsp at users dot sourceforge.net> ---
I don't think its possible to convince thunderbird to
send the attachment as plain-text. It will use base64.
Are base64 attachments acceptable as a patch in your
ML?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[adhemerval.zanella at linaro dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #26 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> ---
(In reply to Stas Sergeev from comment #25)
> I don't think its possible to convince thunderbird to
> send the attachment as plain-text. It will use base64.
> Are base64 attachments acceptable as a patch in your
> ML?

The https://sourceware.org/glibc/wiki/Contribution%20checklist describes that
the preferred way to send patches is through 'git send-email
--to=libc-alpha@sourceware.org'.  The is a brief walkthrough on how to
configure git send-mail:
https://www.freedesktop.org/wiki/Software/PulseAudio/HowToUseGitSendEmail/

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[stsp at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #27 from Stas Sergeev <stsp at users dot sourceforge.net> ---
I was asked to provide the symbol for the next
release. So I do this:

--- a/dlfcn/Versions
+++ b/dlfcn/Versions
@@ -28,6 +28,9 @@ libc {
     dlsym;
     dlvsym;
   }
+  GLIBC_2.37 {
+    dlmem;
+  }
   GLIBC_PRIVATE {
     __libc_dlerror_result;
     _dlerror_run;



But it doesn't appear in libc.so after that.
Actually nothing with @GLIBC_2.37 is there, the
newest symbol that gets in, is @GLIBC_2.36.

What should I do?
Version dlmem for 2.36?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[stsp at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #28 from Stas Sergeev <stsp at users dot sourceforge.net> ---
In fact, anything above GLIBC_2.34 in dlfcn/Versions
doesn't appear in libc.so. GLIBC_2.35 doesn't work, GLIBC_2.34 works.
Confused.
Its quite unlikely that I can solve this without a hint.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[stsp at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #29 from Stas Sergeev <stsp at users dot sourceforge.net> ---
Got it, dlmem.c was a copy/paste from dlopen.c which
had versioned_symbol() defs.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[stsp at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #30 from Stas Sergeev <stsp at users dot sourceforge.net> ---
So what should I do next, and how viable
are these patches? Of course I can keep
implementing things and throw in more
patches, but there is no indication that
anything is going to be applied.
So what to do next?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[stsp at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #31 from Stas Sergeev <stsp at users dot sourceforge.net> ---
https://sourceware.org/pipermail/libc-alpha/2023-March/146389.html
Posted a v8, which is mostly a full rewrite.
No more audit machinery.
And that's a huge relief, because all those
things like "pass fd to dlmem()" suddenly
completely disappeared from an implementation,
and remained only in a custom premap callback
in a test-case. The core impl no longer cares
what the premap callback does.

Replying to comment from
https://sourceware.org/bugzilla/show_bug.cgi?id=30007
This is almost as you "first mmap interception",
except that "first mmap interception" w/o dlmem
impl won't help more than to specify the reloc
address. The reason is that in the default impl
the mapping created by first mmap, is over-mapped
when loading segments.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[stsp at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

Stas Sergeev <stsp at users dot sourceforge.net> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |janderson at rice dot edu

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[stsp at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #32 from Stas Sergeev <stsp at users dot sourceforge.net> ---
https://sourceware.org/pipermail/libc-alpha/2023-March/146434.html
Posted a v9 with lots of new features.
Also its better structured for a review.
Namely the shm-dealing stuff is in the
patch 13, outside of the main dlmem()
impl. As this shm-dealing seems to be
the last part still not well understood,
I put it separately, with a separate
test-case. Which, as before, maps the
lib twice.

Also patch 12 has an fdlopen() impl as
a test-case:

static void *
fdlopen (int fd, int flags)
{
  off_t len;
  void *addr;
  void *handle;

  len = lseek (fd, 0, SEEK_END);
  lseek (fd, 0, SEEK_SET);
  addr = mmap (NULL, len, PROT_READ, MAP_PRIVATE, fd, 0);
  if (addr == MAP_FAILED)
    {
      printf ("cannot mmap, %s\n", strerror(errno));
      exit (EXIT_FAILURE);
    }
  handle = dlmem (addr, len, flags, NULL);
  munmap (addr, len);
  return handle;
}


Neither fdlopen() nor solib duplication can
be done with intercepting the first mmap.
Also I wonder if fdlopen() can ever be
implemented simpler than that. :) The test-case
for fdlopen() makes sure that /proc/self/maps
have the right references to the loaded solib,
rather than some anonymously mapped areas, which
would be a cheating.
So in fact dlmem() have suddenly grown into
an universal tool that exposes all the former
and new libdl features via the single small
call. It can handle file-based solibs not
any worse than the memory-based ones (you
can even specify the solib name now).
So basically, a huge libdl superset by the
price of just 1 call!

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[janderson at rice dot edu]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #33 from Jonathon Anderson <janderson at rice dot edu> ---
(In reply to Stas Sergeev from comment #32)
> Also I wonder if fdlopen() can ever be
> implemented simpler than that. :)
Well, since you asked. On Linux with /proc (vast majority of Linux distros):

static void *
fdlopen (int fd, int flags)
{
  char tmpbuf[100]; // > 35 == strlen("/proc/self/fd/") + log10(pow(2, 64)) + 1
  sprintf(tmpbuf, "/proc/self/fd/%d", fd);
  return dlopen(tmpbuf, flags);
}

No Glibc patch required, 60% shorter, and even non-Glibc compatible. :D

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[stsp at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #34 from Stas Sergeev <stsp at users dot sourceforge.net> ---
(In reply to Jonathon Anderson from comment #33)
> (In reply to Stas Sergeev from comment #32)
> > Also I wonder if fdlopen() can ever be
> > implemented simpler than that. :)
> Well, since you asked. On Linux with /proc (vast majority of Linux distros):

Deal. :)
Florian also asked about dlopen_with_offset.
So lets try again:

static void *
dlopen_with_offset (int fd, int flags, off_t offset)
{
  off_t len;
  void *addr;
  void *handle;

  len = lseek (fd, 0, SEEK_END);
  lseek (fd, 0, SEEK_SET);
  if (len <= offset)
    return NULL;
  len -= offset;
  addr = mmap (NULL, len, PROT_READ, MAP_PRIVATE, fd, offset);
  if (addr == MAP_FAILED)
    return NULL;
  handle = dlmem (addr, len, flags, NULL);
  munmap (addr, len);
  return handle;
}

How would that look like w/o dlmem()?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[stsp at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #35 from Stas Sergeev <stsp at users dot sourceforge.net> ---
Hi guys, just a quick question.
All libdl funcs have things like

static_link_warning (dlopen)

that produces the warning on static
link:

Using 'dlmem' in statically linked applications requires at runtime the shared
libraries from the glibc version used for linking

I've heard that libdl was merged
into libc itself, so is this warning
still relevant? Why it is needed now
and should dlmem() also print one?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[stsp at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #36 from Stas Sergeev <stsp at users dot sourceforge.net> ---
And does this static-link problem
exist even when dlmopen()ing into
a separate NS?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[sam at gentoo dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

Sam James <sam at gentoo dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           See Also|                            |https://sourceware.org/bugz
                   |                            |illa/show_bug.cgi?id=11767
                 CC|                            |sam at gentoo dot org

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/30100] [patch] add dlmem()

[stsp at users dot sourceforge.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=30100

--- Comment #37 from Stas Sergeev <stsp at users dot sourceforge.net> ---
Hmm, it seems like dlmem() indeed has
a difficult to solve problem. Its that
DT_NEEDED deps are not going to refer
to the memory buffer, but to an FS only.
It means that to dlmem() with deps, one
needs a premap per-dep callback or (hate
to say that) an elf parsing to get the
deps out.
It seems the one can't avoid both together,
and allowing the loader to get the deps
from an FS (which is what my impl does),
is a restricted solution.

The same problem applies to fdlopen()
and dlopen_with_offset(): DT_NEEDED entries
are not going to refer to an fd or an
fd with offset, so one would need a pre-open
per-dep callback or (hate to say that) an elf
parsing to get the deps out.
This is a cheap hint for you on where to
get the rejecting reason for such proposals
in the future, as you seem to be in a deep
need of the valid one. :)

-- 
You are receiving this mail because:
You are on the CC list for the bug.