Re: On time64 and Large File Support

["Daniel P. Berrangé" <berrange@redhat.com>]

On Wed, Mar 01, 2023 at 07:29:01PM -0500, Demi Marie Obenour wrote:
> On 3/1/23 17:38, Eric Blake wrote:
> > [replying to the original post, because I'm not sure where else in the
> > more recent activity on this thread would be more appropriate]
> > 
> > On Fri, Nov 11, 2022 at 08:38:18AM +0000, Sam James wrote:
> >> Hi all,
> >>
> >> In Gentoo, we've been planning out what we should do for time64 on glibc [0]
> >> and concluded that we need some support in glibc for a newer option. I'll outline
> >> why below.
> >>
> > ...
> >>
> >> Indeed, the gnulib version of change #2 is exactly how we ended up with
> >> wget/gnutls breaking [1]. I feel this shows that the only approach
> >> "supported" by glibc right now is untenable.
> > 
> >> [1] https://bugs.gentoo.org/828001
> > 
> > Now Fedora is also being hit by the gnutls ABI change due to time_t in
> > public interfaces being silently changed.  From an IRC conversation I
> > had with Dan Berrange and Rich Jones (I think Rich mean i686 below):
> > 
> > <danpb> rjones (IRC): oh wow, the certificates created on i696 are not quite right .....
> > <danpb>         Validity:
> > <danpb>                 Not Before: Sat Sep 05 00:23:57 UTC 2703
> > <danpb>                 Not After: Sun Sep 06 00:23:57 UTC 2703
> > <danpb> just a few years too early
> > <danpb> i think this is looking like a gnutls regression,   downgrading gnutls makes it work
> > ...
> > <danpb> rjones (IRC): hmm, i'm beginning to think gnutls has been miscompiled by gcc
> > <danpb> gnutls_x509_crt_get_activation_time inside the gnutls verification api returns garbage
> > <danpb> but the very same call done from a demo program returns the right answer
> > ...
> > <danpb> OMG, gnulib-- has silently changed gnutls to use 64-bit time_t
> > <danpb> ...which is an ABI incompatibility because gnutls has public APIs which have time_t parameters
> > <danpb> so apps talking to gnutls will expect 32-bit time_t, but gnutls is processing 64-bit time_t
> > <danpb> this is utterly insane
> 
> Time to do a mass rebuild and mass SONAME bump of everything shipped as 32-bits?

A mass rebuild won't fix the problem, because the fundamental issue
is that apps using GNUTLS don't set _TIME_BITS=64 while GNUTLS does
set that, so they have incompatible views of what size time_t is.


GLibC sensibly hid the 64-bit time_t behind _TIME_BITS=64, so that
applications/libraries didn't get a silent ABI change when updating
to newer GLibC.

GNULIB's year2038 module will check for this _TIME_BITS=64 support
and if present, enable it. That would have been OK on its own as apps
would still be opting in to use of GNULIB's year2038 module.


The problem with GNUTLS arises because the pre-existing GNULIB largefile
module appears to also have been changed to set _TIME_BITS=64. Thus when
an application using the largefile module updates GNULIB, they unwittingly
get this change to time_t which impacts their ABI compatibility if they
export any APIs using time_t, or call into any other non-GLibC libraries
that use time_t in their APIs.

The compounding factor is that the GNULIB largefile module is something
probably any application has been using for decades. IOW, an opt-in GLiBC
change to 64-bit time_t has effectively become an always on change to
64-bit time_t for apps/libs using GNULIB.

I can understand the good intentions of GNULIB in making this change to
largefile expecting it to benefit apps, but I think it has a rather
unfortunate / undesirable ripple effect which makes it a net downside.

GNULIB did provide a --disable-year2038 flag to configure, however,
expecting every existing end user who builds apps/libs to consistently
know about and use --disable-year2038 is not practical IMHO.

Ideally GNUTLS would not have exposed time_t (or any other variable
size type) in its public API, but that's the unfortunately historical
baggage they can't address without an API+ABI change.


It is possible to argue that the situation with _TIME_BITS=64 is no
different to that of _FILE_OFFSET_BITS=64 usage, and from a purely
technical view that is correct. Both affect the size of certain GLibC
typedefs, and so can impact ABI compatibility of libararies whose public
APIs use such typedefs.

The difference I think is that _FILE_OFFSET_BITS=64 is ancient history.
Apps and libraries have been using _FILE_OFFSET_BITS=64 for 15-20 years
already. So while there is a chance of inconsistent usage, and thus
ABI incompatibility, in practice this is a non-issue since everything
of consequence has long ago opted in to _FILE_OFFSET_BITS=64. Also when
that was being adopted, 32-bit platforms were still in widespread use,
so developers & users fairly easily identified & fixed problems.

Today 32-bit is on its way out, so introducing a new _TIME_BITS=64 and
getting everything converted to use is a huge challenge, compounded by
the fact that very few developers ever test or use 32-bit anymore.

IMHO if distros really want to deal with this, they need to be able to
force _TIME_BITS=64 globally / unconditionally, and do a mass rebuild.

Having apps/libs incrementally adopt _TIME_BITS=64 in isolation is going
to drip feed incompatibility problems over years, many of which will go
undetected and silently cause problems. Some of these incompatibilities
could ultimately turn out to cause security flaws, because correctly
interpreting time is security critical in certain areas of code like
certificate validation. 

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|