Re: arc4random - are you sure we want these?

[Rich Felker <dalias@libc.org>]

On Mon, Jul 25, 2022 at 12:11:27PM +0200, Florian Weimer via Libc-alpha wrote:
> * Jason A. Donenfeld via Libc-alpha:
> 
> > I really wonder whether this is a good idea, whether this is something
> > that glibc wants, and whether it's a design worth committing to in the
> > long term.
> 
> Do you object to the interface, or the implementation?

That was *exactly* my first question too.

> The implementation can be improved easily enough at a later date.
> 
> > Firstly, for what use cases does this actually help? As of recent
> > changes to the Linux kernels -- now backported all the way to 4.9! --
> > getrandom() and /dev/urandom are extremely fast and operate over per-cpu
> > states locklessly. Sure you avoid a syscall by doing that in userspace,
> > but does it really matter? Who exactly benefits from this?
> 
> getrandom may be fast for bulk generation.  It's not that great for
> generating a few bits here and there.  For example, shuffling a
> 1,000-element array takes 18 microseconds with arc4random_uniform in
> glibc, and 255 microseconds with the naïve getrandom-based
> implementation (with slightly biased results; measured on an Intel
> i9-10900T, Fedora's kernel-5.18.11-100.fc35.x86_64).
> 
> > You miss out on this with arc4random, and if that information _is_ to be
> > exported to userspace somehow in the future, it would be awfully nice to
> > design the userspace interface alongside the kernel one.
> 
> What is the kernel interface you are talking about?  From an interface
> standpoint, arc4random_buf and getrandom are very similar, with the main
> difference is that arc4random_buf cannot report failure (except by
> terminating the process).
> 
> > Seen from this perspective, going with OpenBSD's older paradigm might be
> > rather limiting. Why not work together, between the kernel and libc, to
> > see if we can come up with something better, before settling on an
> > interface with semantics that are hard to walk back later?
> 
> Historically, kernel developers were not interested in solving some of
> the hard problems (especially early seeding) that prevent the use of
> getrandom during early userspace stages.
> 
> > As-is, it's hard to recommend that anybody really use these functions.
> > Just keep using getrandom(2), which has mostly favorable semantics.
> 
> Some applications still need to run in configurations where getrandom is
> not available (either because the kernel is too old, or because it has
> been disabled via seccomp).
> 
> > Yes, I get it: it's fun to make a random number generator, and so lots
> > of projects figure out some way to make yet another one somewhere
> > somehow. But the tendency to do so feels like a weird computer tinkerer
> > disease rather something that has ever helped the overall ecosystem.
> 
> The performance numbers suggest that we benefit from buffering in user
> space.  It might not be necessary to implement expansion in userspace.
> getrandom (or /dev/urandom) with a moderately-sized buffer could be
> sufficient.

FWIW I'd rather have a few kB of shareable entropy-expansion .text in
userspace than a few kB per process (or even per thread? >_<) of
nonshareable data any day.

> But that's an implementation detail, and something we can revisit later.
> If we vDSO acceleration for getrandom (maybe using the userspace
> thread-specific data donation we discussed for rseq), we might
> eventually do way with the buffering in glibc.  Again this is an
> implementation detail we can change easily enough.

Exactly.

FWIW I've been kinda waiting to see what glibc would do on this after
the posix_random proposal failed, before considering much what we
should do in musl, but the value I see in either is not as an
optimization but as honoring a well-known interface so we have fewer
applications doing their own stupid YOLO stuff trying to get secure
entropy and botching it. So far the best we have is getentropy but it
fails on old kernels. At some point musl will probably implement both
arc4random and getentropy with secure fallback process for old
kernels -- certainly the fallback is needed for meeting the arc4random
contract and I'd like it on both places.

Rich