On 20 Aug 2021 15:37, Richard Purdie via Libc-help wrote:
> Yocto Project is a cross compiling build system used to build customised
> Linux, RTOSs, firmware and more. It has some interesting technology. One piece
> of it is "pseudo" (http://git.yoctoproject.org/cgit.cgi/pseudo/) which is an 
> LD_PRELOAD which intercepts calls to libc and fakes root privileges. One way or
> another we've managed to keep that working with multiple libc versions for the
> last decade.

this sounds like Debian's fakeroot [1] & Gentoo's sandbox [2].  these have
always been a challenge to keep working, and to be complete.  they're always
"best effort" because they require:
(1) every program being dynamic.  any staticly linked programs won't work.
(2) every interesting call having an actual interception point by interposing
    known functions that the C library provides.  any calls the C library
    makes via PLT bypass cannot be captured, and any syscalls the C library
    or the app makes cannot be captured.
(3) a superset of every symbol (versioned & unversioned) that a C library
    might expose.  this in particular is messy.

[1] https://wiki.debian.org/FakeRoot
[2] https://wiki.gentoo.org/wiki/Sandbox_(Portage)

a tool using ptrace to intercept the syscall addresses all these, but at the
cost of being slower (since every syscall is notified).  if you can require
seccomp & Linux 3.5+, then things look a lot better here as you can setup a
seccomp filter for the process tree and only get notified for the exact set
of syscalls you're interested in.  it does mean you have to handle every
syscall wart across different architectures, but that's a bit easier to do
than all the variety of C library ABI warts.
(i'll ignore instability in the ptrace framework itself that we've observed
when using the interface at scale.)

i haven't reviewed the even newer seccomp notify [3] mechanism to see if
that's useful.  maybe that would offer even more relief.

[3] https://man7.org/linux/man-pages/man2/seccomp_unotify.2.html
[3] https://brauner.github.io/2020/07/23/seccomp-notify.html

> A second piece of technology is uninative. We build a lot binary artefacts, some
> for the target, some as tools running natively on the build system. We have a
> cache of these artefacts people can share and reuse. Uninative is our way of
> allowing one binary to run on any host distro. We do that by shipping a ld+libc
> binary shim and change the interpreter in the native binary to point at our own.
> As long as the shim is the same version or later than system version, it works.

we do this in CrOS with our toolchains by largely using pax-utils's lddtree [4].
it will bundle the active C library (and any other libs), and then generate a
wrapper shell script for each program that is actually executed at runtime.
that does the ldso & ldpath dance so it's all relocatable.

[4] https://gitweb.gentoo.org/proj/pax-utils.git/tree/lddtree.py
-mike