Re: proof for dlmem() (Re: [PATCH v9 0/13] implement dlmem() function)

["Zack Weinberg" <zack@owlfolio.org>]

This discussion seems to have died down in the past two weeks but I'm
going to make one last attempt to bridge the divide.  I would hate to
see everyone walk away from this with a bad taste in their mouth.  Note
cc: list aggressively pruned.

On Fri, Apr 14, 2023, at 3:04 PM, stsp via Libc-alpha wrote:
> 14.04.2023 01:02, Adhemerval Zanella Netto пишет:
>> "It would be possible to require the caller to arrange all of these
>> things, but that's basically offloading A LOT of the ELF loading
>> process onto the calling program and I don't think that makes for a
>> reasonable public interface for glibc to provide."
> OK, in this case I am going to provide a very detailed, reproducible
> and undisputable proof that the above quote is false.

Here's the thing: stsp, no one is intentionally trying to spread lies
about your patch.  You have misunderstood Adhemerval here.  He was *not*
saying that *your patch* required the caller to parse ELF headers
themselves.  He was saying that your patch does a *different*
unacceptable thing -- running user-supplied code with the loader lock
held, if I understood correctly -- and then he was trying to warn you
that any *alternative* patch you might come up with would *also* be
unacceptable if it required the caller to parse the DLL themselves.

Do you understand the difference between what you thought Adhemerval was
saying, and what I am saying he was trying to communicate?  Please
confirm, or else describe why you don't see a difference so I can try to
clarify again.

Now. Another thing you must understand before you send in any further
revisions of the patch is that glibc is extremely reluctant to add new
APIs.  Witness how much we resisted adding strlcpy, a single function
with a clear specification and no interdependencies with any other
part of the C library.  This is why you are getting what probably
seems to you like an absurd level of negative feedback.  We aren't
trying to deny that you have a legitimate need, and we aren't saying
you haven't managed to come up with a proof of concept implementation
of how that need could be solved -- and yet we ARE saying, no, still
not good enough.

We do this because once we add something, we are stuck with it
*forever*. Not even if the C committee itself deletes something from the
standard do we drop it.  You can *still*, today, write a new program
that uses gets().  And we also do this because GNU libc is loaded into
the address space of *every program* running on (GNU/)Linux.  You're
asking to take up a chunk of (virtual) RAM in *every program* in order
to make your one specific program simpler.  This *will* make at least a
few programs' L1 cache utilization worse.  Not by much, but this happens
every time we add something and it adds up.

---

Having said all that, I can imagine other uses for dlmem() or something like it, besides your specific program, so let me try to meet you in the middle here.  Suppose we gave you *this* API:

/** Report the total amount of virtual address space required in order to load
 *  the shared object referred to by FD.  FD must be a readable and seekable
 *  file descriptor.  Returns a whole number of pages, or zero on error
 *  (in which case errno will provide details).
 */
size_t dl_total_vsize(int fd);

/** Load the shared object referred to by FD into this process's address
 *  space.  FD must be a readable and seekable file descriptor.
 *
 *  If LOAD_ADDR is not NULL, it specifies the address at which to load
 *  the shared object.  If this is not possible, for instance if there
 *  are already memory mappings in the range from LOAD_ADDR to LOAD_ADDR
 *  plus dl_total_vsize(FD), the operation will fail.  LOAD_ADDR must be
 *  a multiple of the system page size.
 *
 *  MODE accepts the same set of RTLD_* flags as are documented for dlopen(),
 *  plus one more: RTLD_READ directs the dynamic loader to read the shared
 *  object into the process's address space as-if with read() rather than
 *  with mmap().  When this flag is used, LOAD_ADDR may not be NULL, and must
 *  already point to at least dl_total_vsize(FD) bytes of writable memory.
 *  The dynamic loader will take ownership of that range of addresses and
 *  may subsequently (e.g. upon a call to dlclose()) deallocate them as-if
 *  with munmap().  Furthermore, the dynamic loader will change virtual
 *  memory protection settings for sub-ranges of this space, so that
 *  (for instance) the text segment of the shared object is read-only,
 *  as-if by calling mprotect().
 */
void *dl_fdopen(int fd, int mode, void *load_addr);

What would still be missing for your use case?

zw