Re: [PATCH 1/1] Y2038: add function __difftime64

[Albert ARIBAUD <albert.aribaud@3adev.fr>]

Hi Paul,

On Mon, 25 Jun 2018 16:56:14 -0700, Paul Eggert <eggert@cs.ucla.edu>
wrote :

> On 06/25/2018 03:32 PM, Albert ARIBAUD wrote:
> > I replaced the original 32-bit difftime with a wrapper
> > around the 64-bit time; and in both cases, there were two functions, on
> > for each time size.  
> 
> Yes, and all this is in the patch you posted in 
> <https://www.sourceware.org/ml/libc-alpha/2018-06/msg00605.html>. What 
> I'm not understanding is how the functions it defines (__difftime64 and 
> __difftime on 32-bit hosts, and just __difftime on 64-bit hosts) 
> interoperate the with user-defined macro (is it still _TIME_BITS? I 
> can't recall) that specifies whether the user wants time_t is 64 or 32 
> bits, and how they interoperate with the macro that Joseph Myers 
> suggested was needed by the system to say whether a 32-bit time_t 
> variant is supported.
> 
> In short, I'm still missing the bigger picture here.

OK, so let me try to sum things up (I'll update the design document
accordingly).

Right now, on 32-bit architectures, application source code which uses
glibc and manipulates time can only go as far as Y2038, because the
public API declares only 32-bit types (time_t and related structures)
and functions (difftime etc). Any time beyond Y2038 is undefined
behavior  -- I've documented behaviors ranging from just time rollover
(back to Dec. 1901 IIRC) to full system freeze.

My goal is:

- to make glibc provide, when _TIME_BITS is is defined equal to 64, the
  same set of type and function names but with 64-bit time support (and
  with _TIME_BITS undefined or defined equal to 32, glibc would stick
  with the old types and functions);

- to ensure that, at least for now, the existing ABI remain unchanged,
  i.e. application modules resulting from compiling against current
  glibc should link against a 64-bit-time-capable glibc and run
  without errors.

In other words, source code referring to difftime would result in
object code referring either to the existing __difftime symbol if
_TIME_BITS is not equal to 64, or to some new __difftime64 symbol if
_TIME_BITS is equal to 64; and the symbol __difftime in glibc should
always represent the existing 32-bit implementation so that existing,
already compiled, client code would execute the difftime they expect
to.

How to achieve this:

- since a single glibc header install must support compiling both
  32-bit-time and new 64-bit-time object code, glibc must provide two
  sets of public API symbol definitions, one for 32-bit-time, exposed
  if _TIME_BITS is defined equal to 32 or undefined, and one for 64-bit
  time, exposed if _TIME_BITS is defined equal to 64.

- since a single glibc library must support both existing 32-bit-time
  and new 64-bit-time object code, glibc must therefore provide two
  sets of implementation symbols at the same time: the current set for
  object client code which was compiled for 32-bit time, and a new set
  for object client code which was compiled for64-bit time.

Refining the how:

- we cannot introduce the public API and _TIME_BITS support in small
  batches, because if you switch time_t to 64-bit, that switches all
  functions using time_t to 64-bit too, and all types using time_t as
  well. Basically, _TIME_BITS and the switch of the API to 64-bit time
  has to be a single, atomic, patch.

- but we can introduce the new 64-bit symbols, including the
  publically visible ones, even though we won't expose them as they
  will be in the end. These new symbols do not need to be all introduced
  in a single patch; this can be done progressively.

For instance, in the end, we will have to map the public API's difftime
symbol either to the existing 32-bit __difftime or to a new 64-bit
__difftime64, depending on _TIME_BITS. So we will /have/ to provide a
public symbol __difftime64. Only in the final _TIME_BITS patch will
this symbol be aliased with 'difftime' -- for client code which compiles
with _TIME_BITS=64.

Further refining the how:

- We can create 64-bit implementations from existing 32-bit ones, with
  some hand-optimizing based on time_t assumptions which we know
  always / never apply to __time64_t (such as possibly being integer or
  being floating point) or size considerations (such as some precision
  loss risks which might not apply to __time64_t).

- We can/must rewrite some internals of glibc to handle __time64_t
  rather than time_t.

- Once there is a 64-bit implementation for a given functionality in
  32-bit architectures, then the 32-bit implementation can be turned
  into a wrapper around the 64-bit one (if the 32-bit implementation
  is even needed; for 64-bit architectures and x32, it does not need
  to be defined at all).

  This is where we need another macro, __TIME_SIZE. While _TIME_BITS
  tells us, at client code compile time, whether client code wants a
  64-bit time API or not, __TIMESIZE tells us, at glibc compile time,
  whether the time_t for the considered architecture is 64-bit or not.
  Architectures with 64-bit time_t do not need 32-bit wrappers, and can
  use time_t for their __time64_t; others need the 32-bit wrappers, and
  need separate time_t and __time64_t.

Final picture:

- the set of patches to support Y2038 will consist in new types and new
  functions (either visible from the public or purely internal to
  glibc) or internal glibc function conversions to 64 bit; these
  patches will have some of the implementation dependent on __TIMESIZE.
  Then in the end a single patch will expose the 'old' or 'new' set
  depending on _TIME_BITS. 

Is this clear enough? Don't hesitate to ask for further clarifications.

> > Since you want the changes in gnulib first, then I suspect I should
> > provide branches above gnulib as well as above glibc? If so, what
> > would you recommend as a good source on setting up a build and test
> > setup for gnulib, similar to build-many-glibcs.py is for glibc?
> 
> Yes, it would be helpful to have this in Gnulib too. The basic idea is 
> that Gnulib and glibc sources should be as close to each other as 
> possible; preferably identical. You can build and test a Gnulib module 
> by running './gnulib-tool --test modulename'; run './gnulib-tool --help' 
> for more.

Since gnulib is on Savannah, not Sourceware, I assume I will need to be
given some level of write access to the Savannah gnulib repository in
order to provide branches there too, similar to what is done in glibc.
Who should I ask this?

Cordialement,
Albert ARIBAUD
3ADEV