Fourth draft of the Y2038 design document

Fourth draft of the Y2038 design document

[Albert ARIBAUD]

Hi all,

I have produced a fourth draft of what will eventually become the Y2038
design document:

https://sourceware.org/glibc/wiki/Y2038ProofnessDesign?rev=83

Relative to the previous draft:

 * the scope clarifies that *only* Y2038 is considered, and no other
   doomsday such as Y2106 or Y9999;

 * all types directly or indirectly derived from time_t are now listed;

 * all APIs using these types are now listed;

 * all functions which use time_t internally are now listed;

 * also listed are types and APIs related to time but which are
   Y2038-safe (even though they might be unsafe for some other
   doomsday, e.g. struct rpc_timeval being Y2106-unsafe).

As always, comments welcome.

Cordialement,
Albert ARIBAUD
3ADEV

Re: Fourth draft of the Y2038 design document

[Arnd Bergmann]

On Wednesday, June 22, 2016 12:58:38 AM CEST Albert ARIBAUD wrote:
> Hi all,
> 
> I have produced a fourth draft of what will eventually become the Y2038
> design document:
> 
> https://sourceware.org/glibc/wiki/Y2038ProofnessDesign?rev=83
> 
> Relative to the previous draft:
> 
>  * the scope clarifies that *only* Y2038 is considered, and no other
>    doomsday such as Y2106 or Y9999;
> 
>  * all types directly or indirectly derived from time_t are now listed;
> 
>  * all APIs using these types are now listed;
> 
>  * all functions which use time_t internally are now listed;
> 
>  * also listed are types and APIs related to time but which are
>    Y2038-safe (even though they might be unsafe for some other
>    doomsday, e.g. struct rpc_timeval being Y2106-unsafe).
> 
> As always, comments welcome.

I've cross-checked your list of data structures with the one I have
for the kernel at
https://docs.google.com/spreadsheets/d/1HCYwHXxs48TsTb6IGUduNjQnmfRvMPzCN6T_0YiQwis/edit#gid=0

I noticed 'struct sysinfo' as another interface that holds a time,
even though this is documented as 'long' and I concluded that we
won't need to change it, that should probably be documented.

'struct rusage' is also interesting, as there is no overflow in 2038,
but instead it could overflow on large machines (hundreds of CPUs)
with very long-running tasks (months), so I'm unsure how to treat that
from the kernel side. It's also one of very few kernel interfaces using
'timeval' rather than 'timespec' (we generally replaced the other ones).
My current kernel patch series changes rusage by using 64-bit fields for
everything, with the intention of having the same binary layout for
32-bit and 64-bit processes, and reusing the 64-bit syscall entry point
for compat mode (running 32-bit tasks on a 64-bit kernel). Does that work
for you?

I think we should expand the ioctl section a bit: I can do most of it
in the kernel, but need help from glibc for a few things in which we
need to agree on calling conventions. Here is what I'd suggest having in
the document, feel free to take that into your document or edit as you
wish:

== IOCTLs and Y2038 ==

 Some Linux IOCTLs may be Y2038-unsafe, or use types defined by glibc
 that do not match the kernel internal types. Known important cases are:

 - An ioctl command number is defined using the _IOR/_IOW/_IORW macros
   by the kernel with a structure whose size changes based on glibc's
   time_t. The kernel can handle these transparently by implementing
   handlers for both command numbers with the correct structure format.

 - The binary ABI changes based on the glibc time_t type, but the
   command number does not change. In this case, the kernel header files
   defining the data structure will check the "__USE_TIME_BITS64"
   macro [do we need a new macro for the kernel headers?] to provide
   a different command number for the new data structure layout. glibc
   will define this macro at an appropriate location [where?] to make
   it visible before including kernel header files.

 - An ioctl command passes time information in a structure that is not
   based on time_t but another integer type that does not get changed.
   The kernel header files will provide both a new structure layout
   and command number when "__USE_TIME_BITS64" is set.

[I can find examples for ioctl commands in each of those
categories if needed.]

== Socket options ==

Like ioctl(), setsockopt()/getsockopt() has a few interfaces that are
passing time data:

SO_TIMESTAMP/SO_TIMESTAMPNS/SO_TIMESTAMPING: These enable the
timestamping infrastructure for a socket, which will consecutively
return data to user space using "cmsg" data on the socket. The kernel
does not know the layout of 'struct timespec' and 'struct timeval'
when filling the cmsg data, so we need to define new binary values
for the three flags, which then get used if __USE_TIME_BITS64
is set.

SO_RCVTIMEO/SO_SNTTIMEO: These pass a 'struct timeval' and a length.
Assuming that the 'optlen' argument of the setsockopt syscall always
matches the size of 'struct timeval', the kernel will be able to
access the data in the same format that was passed by glibc. [alternatively,
we could handle this the same way as SO_TIMESTAMP*, using new numbers
for the flags].

[end quote]

Regarding the "Support for non-Y2038-safe kernels" section, I'm not
sure if that can work at all: A kernel that does not have the appropriate
system calls will also not have the handlers for a lot of the ioctl
commands and possibly other interfaces that rely on a specific
structure layout. If we can instead enforce that __USE_TIME_BITS64
is only set with a minimal version of kernel headers and that it
implies binary compatibility with no older kernel version, we could
avoid those problems.

On your note "The implementation needs further thinking about, as
application code defining _TIME_BITS=64 and gets built against
new kernel headers and old GLIBC headers, then GLIBC will use 32-bit
time_t and kernel will expect 64-bit time_t, and there is no way
to ensure detection of this case.", I think that is covered by
having the kernel headers check __USE_TIME_BITS64 instead of
_TIME_BITS=64, as I described above.

	Arnd

Re: Fourth draft of the Y2038 design document

[Joseph Myers]

On Wed, 22 Jun 2016, Arnd Bergmann wrote:

> Regarding the "Support for non-Y2038-safe kernels" section, I'm not
> sure if that can work at all: A kernel that does not have the appropriate
> system calls will also not have the handlers for a lot of the ioctl
> commands and possibly other interfaces that rely on a specific
> structure layout. If we can instead enforce that __USE_TIME_BITS64

Maybe it's hard for ioctls, but I see no reason it shouldn't work for 
applications using normal non-ioctl glibc interfaces, with runtime 
fallbacks in glibc such as have been used for plenty of previous kernel 
features.

Of course eventually we remove support for the older kernels - when they 
are no longer on the list at https://www.kernel.org/category/releases.html 
unless something comes up like the issue with people expecting to use new 
glibc on old virtualization solutions only supporting 2.6.32 kernels - but 
it would be good for people and distributions to be using / testing 
_TIME_BITS=64 before they reach the point of ceasing to support any older 
kernels.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Fourth draft of the Y2038 design document

[Arnd Bergmann]

On Wednesday, June 22, 2016 2:30:10 PM CEST Joseph Myers wrote:
> On Wed, 22 Jun 2016, Arnd Bergmann wrote:
> 
> > Regarding the "Support for non-Y2038-safe kernels" section, I'm not
> > sure if that can work at all: A kernel that does not have the appropriate
> > system calls will also not have the handlers for a lot of the ioctl
> > commands and possibly other interfaces that rely on a specific
> > structure layout. If we can instead enforce that __USE_TIME_BITS64
> 
> Maybe it's hard for ioctls, but I see no reason it shouldn't work for 
> applications using normal non-ioctl glibc interfaces, with runtime 
> fallbacks in glibc such as have been used for plenty of previous kernel 
> features.
> 
> Of course eventually we remove support for the older kernels - when they 
> are no longer on the list at https://www.kernel.org/category/releases.html 
> unless something comes up like the issue with people expecting to use new 
> glibc on old virtualization solutions only supporting 2.6.32 kernels - but 
> it would be good for people and distributions to be using / testing 
> _TIME_BITS=64 before they reach the point of ceasing to support any older 
> kernels.

Testing is a good point, we crearly don't want to force people to
give up support for older kernels in their glibc binaries in order
to be able to build with _TIME_BITS=64.

At the same time I'd be worried about users building a binary
against a new libc with _TIME_BITS=64 and then running into data
corruption because they are accidentally executing the binaries on
kernels that interpret the some ABI differently.

We can avoid most of the problems if building with _TIME_BITS=64
has no effect unless both glibc and the kernel headers are new
enough to provide the definitions for 64-bit time_t. That way
we can at least ensure that calling an ioctl command or setsockopt
with an incompatible ABI will result in an error code rather than
wrong data.

	Arnd

Re: Fourth draft of the Y2038 design document

[Joseph Myers]

On Wed, 22 Jun 2016, Arnd Bergmann wrote:

> We can avoid most of the problems if building with _TIME_BITS=64
> has no effect unless both glibc and the kernel headers are new
> enough to provide the definitions for 64-bit time_t. That way
> we can at least ensure that calling an ioctl command or setsockopt
> with an incompatible ABI will result in an error code rather than
> wrong data.

I'd be a lot more comfortable with requiring new kernel headers to build 
and use glibc than with requiring a new kernel at runtime for 
_TIME_BITS=64 to work.  New kernel headers are one of the easiest things 
to use when building glibc, since we have the --with-headers option.  (In 
fact right now the headers requirement (3.2) is newer than the runtime 
requirement (2.6.32) on x86_64 / x86.)

(_TIME_BITS=64 should of course be an OS-independent API, supported for 
all glibc configurations.  Obviously exactly what Hurd does is up to the 
Hurd maintainers, as probably is fixing Hurd to keep it working with 
_TIME_BITS=64, but _TIME_BITS=64 should clearly enable 64-bit time_t for 
it even if some underlying Y2038-safety is missing.  NaCl already has 
64-bit time_t so _TIME_BITS=64 would have no effect there.)

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Fourth draft of the Y2038 design document

[Arnd Bergmann]

On Wednesday, June 22, 2016 8:19:36 PM CEST Joseph Myers wrote:
> On Wed, 22 Jun 2016, Arnd Bergmann wrote:
> 
> > We can avoid most of the problems if building with _TIME_BITS=64
> > has no effect unless both glibc and the kernel headers are new
> > enough to provide the definitions for 64-bit time_t. That way
> > we can at least ensure that calling an ioctl command or setsockopt
> > with an incompatible ABI will result in an error code rather than
> > wrong data.
> 
> I'd be a lot more comfortable with requiring new kernel headers to build 
> and use glibc than with requiring a new kernel at runtime for 
> _TIME_BITS=64 to work.  New kernel headers are one of the easiest things 
> to use when building glibc, since we have the --with-headers option.  (In 
> fact right now the headers requirement (3.2) is newer than the runtime 
> requirement (2.6.32) on x86_64 / x86.)

Just for my understanding: do you mean requiring new headers specifically
for building with _TIME_BITS=64 as I said, or would you change the minimum
kernel header version for building glibc in general when we merge 64-bit
time_t support?

> (_TIME_BITS=64 should of course be an OS-independent API, supported for 
> all glibc configurations.  Obviously exactly what Hurd does is up to the 
> Hurd maintainers, as probably is fixing Hurd to keep it working with 
> _TIME_BITS=64, but _TIME_BITS=64 should clearly enable 64-bit time_t for 
> it even if some underlying Y2038-safety is missing.  NaCl already has 
> 64-bit time_t so _TIME_BITS=64 would have no effect there.)

Fair enough. And this would also mean that we don't allow 32-bit
time_t on future architectures ports that never had an upstream Linux
kernel without 64-bit time_t interfaces, right?

On a related note, is there still a plan to allow building a glibc
with 32-bit time_t disabled? I asked for that to be included in the
past, but I don't see it in the Albert's document now, so I'm guessing
that it was intentionally removed again.

	Arnd

Re: Fourth draft of the Y2038 design document

[Joseph Myers]

On Thu, 23 Jun 2016, Arnd Bergmann wrote:

> > I'd be a lot more comfortable with requiring new kernel headers to build 
> > and use glibc than with requiring a new kernel at runtime for 
> > _TIME_BITS=64 to work.  New kernel headers are one of the easiest things 
> > to use when building glibc, since we have the --with-headers option.  (In 
> > fact right now the headers requirement (3.2) is newer than the runtime 
> > requirement (2.6.32) on x86_64 / x86.)
> 
> Just for my understanding: do you mean requiring new headers specifically
> for building with _TIME_BITS=64 as I said, or would you change the minimum
> kernel header version for building glibc in general when we merge 64-bit
> time_t support?

My view would be changing the minimum for building glibc in general, 
rather than trying to make installed libc headers check the kernel headers 
version and give errors conditional on _TIME_BITS.  (I think we can trust 
distribution maintainers to set appropriate dependencies so that in this 
case glibc doesn't get used with old headers if built with new headers.)

> Fair enough. And this would also mean that we don't allow 32-bit
> time_t on future architectures ports that never had an upstream Linux
> kernel without 64-bit time_t interfaces, right?

I'd expect such future architectures to be in the case where _TIME_BITS=64 
does nothing, much like existing 64-bit architecture ports and NaCl.

> On a related note, is there still a plan to allow building a glibc
> with 32-bit time_t disabled? I asked for that to be included in the
> past, but I don't see it in the Albert's document now, so I'm guessing
> that it was intentionally removed again.

A given glibc port has a given set of existing ABIs, which can't be 
removed without changing the SONAME.  That doesn't rule out making some of 
the old ABIs into compat symbols and disallowing building new binaries 
with _TIME_BITS=32, but that would be several years down the line if done 
at all.

(I expect we'll want to change the default for _FILE_OFFSET_BITS well 
before changing the _TIME_BITS default.  For _FILE_OFFSET_BITS, patches 
were posted a while back, and eventually evidence that 
_FILE_OFFSET_BITS=64 is the API most widely used for building libraries 
where it matters on GNU/Linux distributions; those patches need 
resurrecting.)

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Fourth draft of the Y2038 design document

[Arnd Bergmann]

On Thursday, June 23, 2016 2:32:50 PM CEST Joseph Myers wrote:
> On Thu, 23 Jun 2016, Arnd Bergmann wrote:
> 
> > > I'd be a lot more comfortable with requiring new kernel headers to build 
> > > and use glibc than with requiring a new kernel at runtime for 
> > > _TIME_BITS=64 to work.  New kernel headers are one of the easiest things 
> > > to use when building glibc, since we have the --with-headers option.  (In 
> > > fact right now the headers requirement (3.2) is newer than the runtime 
> > > requirement (2.6.32) on x86_64 / x86.)
> > 
> > Just for my understanding: do you mean requiring new headers specifically
> > for building with _TIME_BITS=64 as I said, or would you change the minimum
> > kernel header version for building glibc in general when we merge 64-bit
> > time_t support?
> 
> My view would be changing the minimum for building glibc in general, 
> rather than trying to make installed libc headers check the kernel headers 
> version and give errors conditional on _TIME_BITS.  (I think we can trust 
> distribution maintainers to set appropriate dependencies so that in this 
> case glibc doesn't get used with old headers if built with new headers.)

Ok, thanks for clarifying.

> > Fair enough. And this would also mean that we don't allow 32-bit
> > time_t on future architectures ports that never had an upstream Linux
> > kernel without 64-bit time_t interfaces, right?
> 
> I'd expect such future architectures to be in the case where _TIME_BITS=64 
> does nothing, much like existing 64-bit architecture ports and NaCl.

Ok.

> > On a related note, is there still a plan to allow building a glibc
> > with 32-bit time_t disabled? I asked for that to be included in the
> > past, but I don't see it in the Albert's document now, so I'm guessing
> > that it was intentionally removed again.
> 
> A given glibc port has a given set of existing ABIs, which can't be 
> removed without changing the SONAME.  That doesn't rule out making some of 
> the old ABIs into compat symbols and disallowing building new binaries 
> with _TIME_BITS=32, but that would be several years down the line if done 
> at all.

Ok, got it.

> (I expect we'll want to change the default for _FILE_OFFSET_BITS well 
> before changing the _TIME_BITS default.  For _FILE_OFFSET_BITS, patches 
> were posted a while back, and eventually evidence that 
> _FILE_OFFSET_BITS=64 is the API most widely used for building libraries 
> where it matters on GNU/Linux distributions; those patches need 
> resurrecting.)

I think the two flags are somewhat different from a user perspective:

_FILE_OFFSET_BITS=64 is generally a good idea and is required for some
applications, while others are fine without it. In contrast, _TIME_BITS=64
has zero benefit for most users of 32-bit machines, but for those people
that need it, it is essential that /all/ user space is built with that
option.

I would also guess that it's harder distros to do a gradual migration
for time_t than it is for off_t because there are more libraries that
expose interfaces with time_t and those libraries do not all use
symbol versioning but would get a silent ABI break from rebuilding
with _TIME_BITS=64.

I expect that most distros will not turn _TIME_BITS=64 at all because
it's easier to just stop supporting 32-bit binaries at some point
in the next 20 years, while the ones that plan to keep supporting them
will not do a gradual migration but instead force a rebuild of all
binaries at the same time.

For the second use case, a changed SONAME would make sense too, but
I understand this is not something you'd introduce lightly, and
it's clear from your explanation that you wouldn't add this for
the first version.

Is there a policy about what justifies such an ABI break? I.e. is it
possible to add a build-time configuration flag later that disables
32-bit time_t support for an existing architecture while setting
a different SONAME, assuming there is sufficient user interest for
this feature? I see a couple of different options that change SONAME
on https://sourceware.org/glibc/wiki/ABIList, but those are all
for fundamental incompatibilities, not for voluntary ABI breaks.

	Arnd

Re: Fourth draft of the Y2038 design document

[Joseph Myers]

On Thu, 23 Jun 2016, Arnd Bergmann wrote:

> Is there a policy about what justifies such an ABI break? I.e. is it

The clear implication from the discussions around C++11 support in 
libstdc++, which resulted in the complicated ABI tag infrastructure, is 
that changing the libstdc++ SONAME would now be too costly, which implies 
the same for the libc SONAME.  That is, incompatible ABIs may be 
introduced with new ports (including fundamentally different 
function-calling ABIs on an existing architecture), so under appropriate 
SHLIB_COMPAT conditionals, but we don't expect to make such changes on 
existing architectures (although as developers, it would be beneficial if 
we could change the libc and libstdc++ SONAMEs say once a decade and get 
rid of accumulated ABI peculiarities at that point - the problem is the 
effects of such transitions on users).

> possible to add a build-time configuration flag later that disables
> 32-bit time_t support for an existing architecture while setting
> a different SONAME, assuming there is sufficient user interest for
> this feature? I see a couple of different options that change SONAME
> on https://sourceware.org/glibc/wiki/ABIList, but those are all
> for fundamental incompatibilities, not for voluntary ABI breaks.

We got rid of --enable-oldest-abi as being extremely bitrotten.  I don't 
think new options for such incompatibility are likely, at least absent 
clear evidence of maintainability (and maybe builtbots using them, etc.).  
It is of course possible to use function lists developed in the course of 
designing and implementing _TIME_BITS=64 to produce a list of ABIs 
involving 32-bit time_t, which could be used to validate that a binary 
doesn't use such ABIs.  And a configuration option to turn the symbols 
into compat symbols wouldn't break the ABI - it would be more like 
--enable-obsolete-rpc.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Fourth draft of the Y2038 design document

[Arnd Bergmann]

On Thursday, June 23, 2016 4:56:00 PM CEST Joseph Myers wrote:
> On Thu, 23 Jun 2016, Arnd Bergmann wrote:
> 
> > Is there a policy about what justifies such an ABI break? I.e. is it
> 
> The clear implication from the discussions around C++11 support in 
> libstdc++, which resulted in the complicated ABI tag infrastructure, is 
> that changing the libstdc++ SONAME would now be too costly, which implies 
> the same for the libc SONAME.  That is, incompatible ABIs may be 
> introduced with new ports (including fundamentally different 
> function-calling ABIs on an existing architecture), so under appropriate 
> SHLIB_COMPAT conditionals, but we don't expect to make such changes on 
> existing architectures (although as developers, it would be beneficial if 
> we could change the libc and libstdc++ SONAMEs say once a decade and get 
> rid of accumulated ABI peculiarities at that point - the problem is the 
> effects of such transitions on users).
> 
> > possible to add a build-time configuration flag later that disables
> > 32-bit time_t support for an existing architecture while setting
> > a different SONAME, assuming there is sufficient user interest for
> > this feature? I see a couple of different options that change SONAME
> > on https://sourceware.org/glibc/wiki/ABIList, but those are all
> > for fundamental incompatibilities, not for voluntary ABI breaks.
> 
> We got rid of --enable-oldest-abi as being extremely bitrotten.  I don't 
> think new options for such incompatibility are likely, at least absent 
> clear evidence of maintainability (and maybe builtbots using them, etc.).  
> It is of course possible to use function lists developed in the course of 
> designing and implementing _TIME_BITS=64 to produce a list of ABIs 
> involving 32-bit time_t, which could be used to validate that a binary 
> doesn't use such ABIs.  And a configuration option to turn the symbols 
> into compat symbols wouldn't break the ABI - it would be more like 
> --enable-obsolete-rpc.

Thanks for the detailed explanation!

I think having a switch to turn the 32-bit time_t calls into compat
symbols would be great. For the goal of guaranteeing that all code
has been built with the _TIME_BITS=64 flag it's almost as good
as removing 32-bit time_t from the build entirely, and not having
to break SONAME gives a migration path for those that want it.

If we introduce that option, this would both mark the symbols
as compat and also change the default for time_t to a 64-bit type
without requiring an explicit _TIME_BITS=64, right?

Anyway, I don't think we need this configuration option right away,
but it may be good to document in the wiki that this would be
something that can be implemented if users ask for it.

For making sure we break runtime support on old binaries when we
want to, running on a kernel that has the old syscalls disabled
is probably sufficient, and we will need that kernel feature
anyway, but it doesn't capture the case where you have a library
that wraps a time_t based interface but doesn't have symbol
versioning itself and a caller gets built against the library
with a conflicting flag. 

	Arnd

Re: Fourth draft of the Y2038 design document

[Albert ARIBAUD]

Hi Arnd,

On Wed, 22 Jun 2016 13:55:16 +0200, Arnd Bergmann <arnd@arndb.de>
wrote :

> I noticed 'struct sysinfo' as another interface that holds a time,
> even though this is documented as 'long' and I concluded that we
> won't need to change it, that should probably be documented.

Added, thanks.

> 'struct rusage' is also interesting, as there is no overflow in 2038,
> but instead it could overflow on large machines (hundreds of CPUs)
> with very long-running tasks (months), so I'm unsure how to treat that
> from the kernel side. It's also one of very few kernel interfaces using
> 'timeval' rather than 'timespec' (we generally replaced the other ones).
> My current kernel patch series changes rusage by using 64-bit fields for
> everything, with the intention of having the same binary layout for
> 32-bit and 64-bit processes, and reusing the 64-bit syscall entry point
> for compat mode (running 32-bit tasks on a 64-bit kernel). Does that work
> for you?

Yes.

> I think we should expand the ioctl section a bit: I can do most of it
> in the kernel, but need help from glibc for a few things in which we
> need to agree on calling conventions. Here is what I'd suggest having in
> the document, feel free to take that into your document or edit as you
> wish:
> 
> == IOCTLs and Y2038 ==
> 
>  Some Linux IOCTLs may be Y2038-unsafe, or use types defined by glibc
>  that do not match the kernel internal types. Known important cases are:
> 
>  - An ioctl command number is defined using the _IOR/_IOW/_IORW macros
>    by the kernel with a structure whose size changes based on glibc's
>    time_t. The kernel can handle these transparently by implementing
>    handlers for both command numbers with the correct structure format.
> 
>  - The binary ABI changes based on the glibc time_t type, but the
>    command number does not change. In this case, the kernel header files
>    defining the data structure will check the "__USE_TIME_BITS64"
>    macro [do we need a new macro for the kernel headers?] to provide
>    a different command number for the new data structure layout. glibc
>    will define this macro at an appropriate location [where?] to make
>    it visible before including kernel header files.
> 
>  - An ioctl command passes time information in a structure that is not
>    based on time_t but another integer type that does not get changed.
>    The kernel header files will provide both a new structure layout
>    and command number when "__USE_TIME_BITS64" is set.
> 
> [I can find examples for ioctl commands in each of those
> categories if needed.]

Please do -- I understand the first two cases, but I am not sure I
understand the third one.

> == Socket options ==
> 
> Like ioctl(), setsockopt()/getsockopt() has a few interfaces that are
> passing time data:
> 
> SO_TIMESTAMP/SO_TIMESTAMPNS/SO_TIMESTAMPING: These enable the
> timestamping infrastructure for a socket, which will consecutively
> return data to user space using "cmsg" data on the socket. The kernel
> does not know the layout of 'struct timespec' and 'struct timeval'
> when filling the cmsg data, so we need to define new binary values
> for the three flags, which then get used if __USE_TIME_BITS64
> is set.

IOW, introduce three new optname values (say 51, 52, and 53 as on my
machine, /usr/include/asm-generic/socket.h stops at 50 right now) that
would behave similar to options 29, 35 and 37 but would use Y2038-safe
timestamps; and define option names SO_TIMESTAMP, SO_TIMESTAMPNS and
SO_TIMESTAMPING to be either 29, 35 and 37 (the Y2038-unsafe options) or
51, 52 and 53 (the Y2038-safe ones) depending on __USE_TIME_BITS64.
Right?

> SO_RCVTIMEO/SO_SNTTIMEO: These pass a 'struct timeval' and a length.
> Assuming that the 'optlen' argument of the setsockopt syscall always
> matches the size of 'struct timeval', the kernel will be able to
> access the data in the same format that was passed by glibc. [alternatively,
> we could handle this the same way as SO_TIMESTAMP*, using new numbers
> for the flags].

I would prefer new numbers, that's more explicit and slightly safer,
as if we rely on optlen, Murphy's Law will see to it that in practice
it will have the wrong value.

> [end quote]

Cordialement,
Albert ARIBAUD
3ADEV

Re: Fourth draft of the Y2038 design document

[Arnd Bergmann]

On Wednesday, June 29, 2016 11:58:38 AM CEST Albert ARIBAUD wrote:

> > == IOCTLs and Y2038 ==
> > 
> >  Some Linux IOCTLs may be Y2038-unsafe, or use types defined by glibc
> >  that do not match the kernel internal types. Known important cases are:
> > 
> >  - An ioctl command number is defined using the _IOR/_IOW/_IORW macros
> >    by the kernel with a structure whose size changes based on glibc's
> >    time_t. The kernel can handle these transparently by implementing
> >    handlers for both command numbers with the correct structure format.
> > 
> >  - The binary ABI changes based on the glibc time_t type, but the
> >    command number does not change. In this case, the kernel header files
> >    defining the data structure will check the "__USE_TIME_BITS64"
> >    macro [do we need a new macro for the kernel headers?] to provide
> >    a different command number for the new data structure layout. glibc
> >    will define this macro at an appropriate location [where?] to make
> >    it visible before including kernel header files.
> > 
> >  - An ioctl command passes time information in a structure that is not
> >    based on time_t but another integer type that does not get changed.
> >    The kernel header files will provide both a new structure layout
> >    and command number when "__USE_TIME_BITS64" is set.
> > 
> > [I can find examples for ioctl commands in each of those
> > categories if needed.]
> 
> Please do -- I understand the first two cases, but I am not sure I
> understand the third one.


a) include/linux/ppdev.h:

/* Set and get port timeout (struct timeval's) */
#define PPGETTIME       _IOR(PP_IOCTL, 0x95, struct timeval)
#define PPSETTIME       _IOW(PP_IOCTL, 0x96, struct timeval)

b) include/linux/lp.h: (this passes a timeval as well)
#define LPSETTIMEOUT 0x060f /* set parport timeout */

c) Actually I could not find a real case any more, it's possible they
are all gone. Two similar cases though:

linux/raid/md_u.h: (this one was "fixed" by ensuring the user space
side can deal with 32-bit time stamps)

typedef struct mdu_array_info_s {
       ...
        unsigned int utime;     /*  0 Superblock update time                  */
        ...
} mdu_array_info_t;
#define GET_ARRAY_INFO          _IOR (MD_MAJOR, 0x11, mdu_array_info_t)


drivers/staging/ft1000/ft1000-usb/ft1000_ioctl.h: (this file was
removed since, as the hardware is now obsolete)

struct IOCTL_GET_DSP_STAT {
       unsigned char DspVer[DSPVERSZ];        /* DSP version number */
       unsigned char HwSerNum[HWSERNUMSZ];    /* Hardware Serial Number */
       unsigned char Sku[SKUSZ];              /* SKU */
       unsigned char eui64[EUISZ];            /* EUI64 */
       unsigned short ConStat;                /* Connection Status */

       unsigned long nTxPkts;                /* Number of packets transmitted
                                              * from host to dsp
                                              */
       unsigned long nRxPkts;                /* Number of packets received from
                                              * dsp to host
                                              */
       unsigned long nTxBytes;               /* Number of bytes transmitted
                                              * from host to dsp
                                              */
       unsigned long nRxBytes;               /* Number of bytes received from
                                              * dsp to host
                                              */
       unsigned long ConTm;                  /* Current session connection time
                                              * in seconds
                                              */
      ....
} __packed;

#define IOCTL_FT1000_GET_DSP_STAT _IOR(FT1000_MAGIC_CODE,              \
                                      IOCTL_GET_DSP_STAT_CMD,          \
                                      struct IOCTL_GET_DSP_STAT)

> > == Socket options ==
> > 
> > Like ioctl(), setsockopt()/getsockopt() has a few interfaces that are
> > passing time data:
> > 
> > SO_TIMESTAMP/SO_TIMESTAMPNS/SO_TIMESTAMPING: These enable the
> > timestamping infrastructure for a socket, which will consecutively
> > return data to user space using "cmsg" data on the socket. The kernel
> > does not know the layout of 'struct timespec' and 'struct timeval'
> > when filling the cmsg data, so we need to define new binary values
> > for the three flags, which then get used if __USE_TIME_BITS64
> > is set.
> 
> IOW, introduce three new optname values (say 51, 52, and 53 as on my
> machine, /usr/include/asm-generic/socket.h stops at 50 right now) that
> would behave similar to options 29, 35 and 37 but would use Y2038-safe
> timestamps; and define option names SO_TIMESTAMP, SO_TIMESTAMPNS and
> SO_TIMESTAMPING to be either 29, 35 and 37 (the Y2038-unsafe options) or
> 51, 52 and 53 (the Y2038-safe ones) depending on __USE_TIME_BITS64.
> Right?

Correct.
 
> > SO_RCVTIMEO/SO_SNTTIMEO: These pass a 'struct timeval' and a length.
> > Assuming that the 'optlen' argument of the setsockopt syscall always
> > matches the size of 'struct timeval', the kernel will be able to
> > access the data in the same format that was passed by glibc.
> > [alternatively,
> > we could handle this the same way as SO_TIMESTAMP*, using new numbers
> > for the flags].
> 
> I would prefer new numbers, that's more explicit and slightly safer,
> as if we rely on optlen, Murphy's Law will see to it that in practice
> it will have the wrong value.

Fair enough.

	Arnd

Re: [Y2038] Fourth draft of the Y2038 design document

[Deepa Dinamani]

>> I think we should expand the ioctl section a bit: I can do most of it
>> in the kernel, but need help from glibc for a few things in which we
>> need to agree on calling conventions. Here is what I'd suggest having in
>> the document, feel free to take that into your document or edit as you
>> wish:
>>
>> == IOCTLs and Y2038 ==
>>
>>  Some Linux IOCTLs may be Y2038-unsafe, or use types defined by glibc
>>  that do not match the kernel internal types. Known important cases are:
>>
>>  - An ioctl command number is defined using the _IOR/_IOW/_IORW macros
>>    by the kernel with a structure whose size changes based on glibc's
>>    time_t. The kernel can handle these transparently by implementing
>>    handlers for both command numbers with the correct structure format.
>>
>>  - The binary ABI changes based on the glibc time_t type, but the
>>    command number does not change. In this case, the kernel header files
>>    defining the data structure will check the "__USE_TIME_BITS64"
>>    macro [do we need a new macro for the kernel headers?] to provide
>>    a different command number for the new data structure layout. glibc
>>    will define this macro at an appropriate location [where?] to make
>>    it visible before including kernel header files.
>>
>>  - An ioctl command passes time information in a structure that is not
>>    based on time_t but another integer type that does not get changed.
>>    The kernel header files will provide both a new structure layout
>>    and command number when "__USE_TIME_BITS64" is set.
>>
>> [I can find examples for ioctl commands in each of those
>> categories if needed.]
>
> Please do -- I understand the first two cases, but I am not sure I
> understand the third one.
>
>> == Socket options ==
>>
>> Like ioctl(), setsockopt()/getsockopt() has a few interfaces that are
>> passing time data:
>>
>> SO_TIMESTAMP/SO_TIMESTAMPNS/SO_TIMESTAMPING: These enable the
>> timestamping infrastructure for a socket, which will consecutively
>> return data to user space using "cmsg" data on the socket. The kernel
>> does not know the layout of 'struct timespec' and 'struct timeval'
>> when filling the cmsg data, so we need to define new binary values
>> for the three flags, which then get used if __USE_TIME_BITS64
>> is set.
>
> IOW, introduce three new optname values (say 51, 52, and 53 as on my
> machine, /usr/include/asm-generic/socket.h stops at 50 right now) that
> would behave similar to options 29, 35 and 37 but would use Y2038-safe
> timestamps; and define option names SO_TIMESTAMP, SO_TIMESTAMPNS and
> SO_TIMESTAMPING to be either 29, 35 and 37 (the Y2038-unsafe options) or
> 51, 52 and 53 (the Y2038-safe ones) depending on __USE_TIME_BITS64.
> Right?

Another way of handling this would be to use the flags in sendmsg/recvmsg.
Since cmsg is sent using these calls, at the time of call, sendmsg/recvmsg can
indicate whether 64 bit time_t or 32 bit time_t is used.
This will eliminate the need for new options and kernel need not depend on
__USE_TIME_BITS64.

>> SO_RCVTIMEO/SO_SNTTIMEO: These pass a 'struct timeval' and a length.
>> Assuming that the 'optlen' argument of the setsockopt syscall always
>> matches the size of 'struct timeval', the kernel will be able to
>> access the data in the same format that was passed by glibc. [alternatively,
>> we could handle this the same way as SO_TIMESTAMP*, using new numbers
>> for the flags].
>
> I would prefer new numbers, that's more explicit and slightly safer,
> as if we rely on optlen, Murphy's Law will see to it that in practice
> it will have the wrong value.

But, if we are not using optlen here, then probably just using new numbers for
timestamps also makes sense.

-Deepa

Re: [Y2038] Fourth draft of the Y2038 design document

[Deepa Dinamani]

On Wed, Jul 13, 2016 at 1:40 PM, Deepa Dinamani <deepa.kernel@gmail.com> wrote:
>>> I think we should expand the ioctl section a bit: I can do most of it
>>> in the kernel, but need help from glibc for a few things in which we
>>> need to agree on calling conventions. Here is what I'd suggest having in
>>> the document, feel free to take that into your document or edit as you
>>> wish:
>>>
>>> == IOCTLs and Y2038 ==
>>>
>>>  Some Linux IOCTLs may be Y2038-unsafe, or use types defined by glibc
>>>  that do not match the kernel internal types. Known important cases are:
>>>
>>>  - An ioctl command number is defined using the _IOR/_IOW/_IORW macros
>>>    by the kernel with a structure whose size changes based on glibc's
>>>    time_t. The kernel can handle these transparently by implementing
>>>    handlers for both command numbers with the correct structure format.
>>>
>>>  - The binary ABI changes based on the glibc time_t type, but the
>>>    command number does not change. In this case, the kernel header files
>>>    defining the data structure will check the "__USE_TIME_BITS64"
>>>    macro [do we need a new macro for the kernel headers?] to provide
>>>    a different command number for the new data structure layout. glibc
>>>    will define this macro at an appropriate location [where?] to make
>>>    it visible before including kernel header files.
>>>
>>>  - An ioctl command passes time information in a structure that is not
>>>    based on time_t but another integer type that does not get changed.
>>>    The kernel header files will provide both a new structure layout
>>>    and command number when "__USE_TIME_BITS64" is set.
>>>
>>> [I can find examples for ioctl commands in each of those
>>> categories if needed.]
>>
>> Please do -- I understand the first two cases, but I am not sure I
>> understand the third one.
>>
>>> == Socket options ==
>>>
>>> Like ioctl(), setsockopt()/getsockopt() has a few interfaces that are
>>> passing time data:
>>>
>>> SO_TIMESTAMP/SO_TIMESTAMPNS/SO_TIMESTAMPING: These enable the
>>> timestamping infrastructure for a socket, which will consecutively
>>> return data to user space using "cmsg" data on the socket. The kernel
>>> does not know the layout of 'struct timespec' and 'struct timeval'
>>> when filling the cmsg data, so we need to define new binary values
>>> for the three flags, which then get used if __USE_TIME_BITS64
>>> is set.
>>
>> IOW, introduce three new optname values (say 51, 52, and 53 as on my
>> machine, /usr/include/asm-generic/socket.h stops at 50 right now) that
>> would behave similar to options 29, 35 and 37 but would use Y2038-safe
>> timestamps; and define option names SO_TIMESTAMP, SO_TIMESTAMPNS and
>> SO_TIMESTAMPING to be either 29, 35 and 37 (the Y2038-unsafe options) or
>> 51, 52 and 53 (the Y2038-safe ones) depending on __USE_TIME_BITS64.
>> Right?
>
> Another way of handling this would be to use the flags in sendmsg/recvmsg.
> Since cmsg is sent using these calls, at the time of call, sendmsg/recvmsg can
> indicate whether 64 bit time_t or 32 bit time_t is used.
> This will eliminate the need for new options and kernel need not depend on
> __USE_TIME_BITS64.
>
>>> SO_RCVTIMEO/SO_SNTTIMEO: These pass a 'struct timeval' and a length.
>>> Assuming that the 'optlen' argument of the setsockopt syscall always
>>> matches the size of 'struct timeval', the kernel will be able to
>>> access the data in the same format that was passed by glibc. [alternatively,
>>> we could handle this the same way as SO_TIMESTAMP*, using new numbers
>>> for the flags].
>>
>> I would prefer new numbers, that's more explicit and slightly safer,
>> as if we rely on optlen, Murphy's Law will see to it that in practice
>> it will have the wrong value.
>
> But, if we are not using optlen here, then probably just using new numbers for
> timestamps also makes sense.

Forgot to note one more thing here.
Since we are already using struct sizes for ioctls, shouldn't this be similar.
So optlen should also be okay?
Just as in the example Arnd pointed out above:

/* Set and get port timeout (struct timeval's) */
#define PPGETTIME       _IOR(PP_IOCTL, 0x95, struct timeval)
#define PPSETTIME       _IOW(PP_IOCTL, 0x96, struct timeval)

-Deepa

Re: [Y2038] Fourth draft of the Y2038 design document

[Arnd Bergmann]

On Wednesday, July 13, 2016 7:38:25 PM CEST Deepa Dinamani wrote:
> On Wed, Jul 13, 2016 at 1:40 PM, Deepa Dinamani <deepa.kernel@gmail.com> wrote:

> >>> == Socket options ==
> >>>
> >>> Like ioctl(), setsockopt()/getsockopt() has a few interfaces that are
> >>> passing time data:
> >>>
> >>> SO_TIMESTAMP/SO_TIMESTAMPNS/SO_TIMESTAMPING: These enable the
> >>> timestamping infrastructure for a socket, which will consecutively
> >>> return data to user space using "cmsg" data on the socket. The kernel
> >>> does not know the layout of 'struct timespec' and 'struct timeval'
> >>> when filling the cmsg data, so we need to define new binary values
> >>> for the three flags, which then get used if __USE_TIME_BITS64
> >>> is set.
> >>
> >> IOW, introduce three new optname values (say 51, 52, and 53 as on my
> >> machine, /usr/include/asm-generic/socket.h stops at 50 right now) that
> >> would behave similar to options 29, 35 and 37 but would use Y2038-safe
> >> timestamps; and define option names SO_TIMESTAMP, SO_TIMESTAMPNS and
> >> SO_TIMESTAMPING to be either 29, 35 and 37 (the Y2038-unsafe options) or
> >> 51, 52 and 53 (the Y2038-safe ones) depending on __USE_TIME_BITS64.
> >> Right?
> >
> > Another way of handling this would be to use the flags in sendmsg/recvmsg.
> > Since cmsg is sent using these calls, at the time of call, sendmsg/recvmsg can
> > indicate whether 64 bit time_t or 32 bit time_t is used.
> > This will eliminate the need for new options and kernel need not depend on
> > __USE_TIME_BITS64.

Good point, I had forgotten how we discussed that a while ago.

> >>> SO_RCVTIMEO/SO_SNTTIMEO: These pass a 'struct timeval' and a length.
> >>> Assuming that the 'optlen' argument of the setsockopt syscall always
> >>> matches the size of 'struct timeval', the kernel will be able to
> >>> access the data in the same format that was passed by glibc. [alternatively,
> >>> we could handle this the same way as SO_TIMESTAMP*, using new numbers
> >>> for the flags].
> >>
> >> I would prefer new numbers, that's more explicit and slightly safer,
> >> as if we rely on optlen, Murphy's Law will see to it that in practice
> >> it will have the wrong value.
> >
> > But, if we are not using optlen here, then probably just using new numbers for
> > timestamps also makes sense.
> 
> Forgot to note one more thing here.
> Since we are already using struct sizes for ioctls, shouldn't this be similar.
> So optlen should also be okay?
> Just as in the example Arnd pointed out above:
> 
> /* Set and get port timeout (struct timeval's) */
> #define PPGETTIME       _IOR(PP_IOCTL, 0x95, struct timeval)
> #define PPSETTIME       _IOW(PP_IOCTL, 0x96, struct timeval)

I've added the netdev list and David Miller to Cc here, it seems
either way would work, and I don't have a strong preference.

a) add a flag to recvmsg() that glibc always passes when called
   using __USE_TIME_BITS64 to handle reading the timestamps, and
   rely on the optlen value for SO_RCVTIMEO/SO_SNTTIMEO to
   decide how to interpret the data

b) handle all five sockopts using conditional option numbers
   and don't rely on the recvmsg() flag or optlen.

Using either a) or b) is probably better than a combination of
them, so I've not listed that as another alternative.

To work around the header inclusion order problem we discussed
earlier for approach b), I suppose we can do this using
something like


#define SO_RCVTIMEO_TIME32	21
#define SO_RCVTIMEO_TIME64	53

#define SO_RCVTIMEO (__USE_TIME_BITS==64 ? SO_RCVTIMEO_TIME64 : SO_RCVTIMEO_TIME32)

where __USE_TIME_BITS is a macro defined by the libc. We cannot easily
check for whether a macro is defined in a conditional expression,
but I think the above should work, as long as we don't need use the
value from assembler code.

	Arnd

Re: [Y2038] Fourth draft of the Y2038 design document

[Albert ARIBAUD]

Hi Arnd,

On Thu, 14 Jul 2016 13:39:13 +0200, Arnd Bergmann <arnd@arndb.de>
wrote :

> On Wednesday, July 13, 2016 7:38:25 PM CEST Deepa Dinamani wrote:
> > On Wed, Jul 13, 2016 at 1:40 PM, Deepa Dinamani <deepa.kernel@gmail.com> wrote:  
> 
>  [...]  
>  [...]  
> > >
> > > Another way of handling this would be to use the flags in sendmsg/recvmsg.
> > > Since cmsg is sent using these calls, at the time of call, sendmsg/recvmsg can
> > > indicate whether 64 bit time_t or 32 bit time_t is used.
> > > This will eliminate the need for new options and kernel need not depend on
> > > __USE_TIME_BITS64.  
> 
> Good point, I had forgotten how we discussed that a while ago.
> 
>  [...]  
>  [...]  
> > >
> > > But, if we are not using optlen here, then probably just using new numbers for
> > > timestamps also makes sense.  
> > 
> > Forgot to note one more thing here.
> > Since we are already using struct sizes for ioctls, shouldn't this be similar.
> > So optlen should also be okay?
> > Just as in the example Arnd pointed out above:
> > 
> > /* Set and get port timeout (struct timeval's) */
> > #define PPGETTIME       _IOR(PP_IOCTL, 0x95, struct timeval)
> > #define PPSETTIME       _IOW(PP_IOCTL, 0x96, struct timeval)  
> 
> I've added the netdev list and David Miller to Cc here, it seems
> either way would work, and I don't have a strong preference.
> 
> a) add a flag to recvmsg() that glibc always passes when called
>    using __USE_TIME_BITS64 to handle reading the timestamps, and
>    rely on the optlen value for SO_RCVTIMEO/SO_SNTTIMEO to
>    decide how to interpret the data
> 
> b) handle all five sockopts using conditional option numbers
>    and don't rely on the recvmsg() flag or optlen.
> 
> Using either a) or b) is probably better than a combination of
> them, so I've not listed that as another alternative.
> 
> To work around the header inclusion order problem we discussed
> earlier for approach b), I suppose we can do this using
> something like
> 
> 
> #define SO_RCVTIMEO_TIME32	21
> #define SO_RCVTIMEO_TIME64	53
> 
> #define SO_RCVTIMEO (__USE_TIME_BITS==64 ? SO_RCVTIMEO_TIME64 : SO_RCVTIMEO_TIME32)
> 
> where __USE_TIME_BITS is a macro defined by the libc. We cannot easily
> check for whether a macro is defined in a conditional expression,
> but I think the above should work, as long as we don't need use the
> value from assembler code.
> 
> 	Arnd

Seems like there was no reply on the netdev or libc-alpha lists or from
David. I personally am not fond of relying on length to determine which
variant of an argument we are dealing with, therefore I prefer option b
over option a and will update the design doc accordingly.

Cordialement,
Albert ARIBAUD
3ADEV

Re: Fourth draft of the Y2038 design document

[Albert ARIBAUD]

Hi Arnd,

On Wed, 22 Jun 2016 13:55:16 +0200, Arnd Bergmann <arnd@arndb.de>
wrote :

> On your note "The implementation needs further thinking about, as
> application code defining _TIME_BITS=64 and gets built against
> new kernel headers and old GLIBC headers, then GLIBC will use 32-bit
> time_t and kernel will expect 64-bit time_t, and there is no way
> to ensure detection of this case.", I think that is covered by
> having the kernel headers check __USE_TIME_BITS64 instead of
> _TIME_BITS=64, as I described above.

What about application source code which includes kernel headers before
including any GLIBC header? The kernel header will see __TIME_BITS=64
but it won't see _USE_TIME_BITS64, and will make the wrong decision.
Don't we consider this a possible scenario?

Cordialement,
Albert ARIBAUD
3ADEV

Re: Fourth draft of the Y2038 design document

[Arnd Bergmann]

On Wednesday, June 29, 2016 1:16:11 PM CEST Albert ARIBAUD wrote:
> 
> On Wed, 22 Jun 2016 13:55:16 +0200, Arnd Bergmann <arnd@arndb.de>
> wrote :
> 
> > On your note "The implementation needs further thinking about, as
> > application code defining _TIME_BITS=64 and gets built against
> > new kernel headers and old GLIBC headers, then GLIBC will use 32-bit
> > time_t and kernel will expect 64-bit time_t, and there is no way
> > to ensure detection of this case.", I think that is covered by
> > having the kernel headers check __USE_TIME_BITS64 instead of
> > _TIME_BITS=64, as I described above.
> 
> What about application source code which includes kernel headers before
> including any GLIBC header? The kernel header will see __TIME_BITS=64
> but it won't see _USE_TIME_BITS64, and will make the wrong decision.
> Don't we consider this a possible scenario?

Good point, I think you are right that this is a problem. In particular,
the normal rule for kernel headers is that they should explictly not
include any libc headers or rely on them to be included first. We have
some exceptions from those rules for historic reasons.

Do you have any other ideas for how to solve it? We probably only
need a handful of those, so we could look at them case-by-case to
see what kind of exception we can make.

	Arnd

Re: Fourth draft of the Y2038 design document

[Albert ARIBAUD]

Hi Arnd,

On Wed, 29 Jun 2016 16:02:13 +0200, Arnd Bergmann <arnd@arndb.de>
wrote :

> On Wednesday, June 29, 2016 1:16:11 PM CEST Albert ARIBAUD wrote:
> > 
> > On Wed, 22 Jun 2016 13:55:16 +0200, Arnd Bergmann <arnd@arndb.de>
> > wrote :
> >   
> > > On your note "The implementation needs further thinking about, as
> > > application code defining _TIME_BITS=64 and gets built against
> > > new kernel headers and old GLIBC headers, then GLIBC will use 32-bit
> > > time_t and kernel will expect 64-bit time_t, and there is no way
> > > to ensure detection of this case.", I think that is covered by
> > > having the kernel headers check __USE_TIME_BITS64 instead of
> > > _TIME_BITS=64, as I described above.  
> > 
> > What about application source code which includes kernel headers before
> > including any GLIBC header? The kernel header will see __TIME_BITS=64
> > but it won't see _USE_TIME_BITS64, and will make the wrong decision.
> > Don't we consider this a possible scenario?  
> 
> Good point, I think you are right that this is a problem. In particular,
> the normal rule for kernel headers is that they should explictly not
> include any libc headers or rely on them to be included first. We have
> some exceptions from those rules for historic reasons.
> 
> Do you have any other ideas for how to solve it? We probably only
> need a handful of those, so we could look at them case-by-case to
> see what kind of exception we can make.

I don't think there is a perfect, or even good enough, compile-time
solution where header could be included in any order while keeping
kernel headers from referencing GLIBC macros.

But we can at least detect and explicitly flag bad kernel/glibc/app
time size combinations at runtime:

- make Y2038-safe kernel provide a new syscall which tells at run time
  what features the kernel supports, e.g. does it support 64-bit time,
  32-bit time, both? [the need could be more generic than just time
  support, so I tought such a syscall already existed, but I have not
  found any in the current syscalls list -- I might have missed it
  though]

- make Y2038-safe GLIBC use the syscall existence and returned value(s)
  to determine at runtime whether the running kernel, running GLIBC and
  running application all agree on 32-bit or 64-bit time.

- make Y2038-safe GLIBC unconditionally define a symbol to announce
  itself as such -- regardless from whether or not it is built for
  64-bit time.

A Y2038-safe GLIBC would thus detect when it runs on a non-Y2038-safe
kernel (syscall absent or returning a value which amounts to 'no
64-bit time support built in kernel') and could bail out with an
explicit message.

A Y2038-safe GLIBC on a kernel which supports the syscall (or syscall
extension) could compare the macros defined by the application code at
compile time with the actually supported kernel features and bail out
explicitly on mismatches.

Plus, *if it decides to, an application aspiring to be Y2038-safe
would be able to verify at build timethat the GLIBC it is compiling
against is indeed Y2038-safe, thus allowing detection of the scenario
above with new kernel headers and old GLIBC header. Of course, if the
application code does not check the macro, this will be missed -- one
can bring the horse to the river, etc. That will be a matter of good
practice.

Does this make any sense?

Cordialement,
Albert ARIBAUD
3ADEV

Re: Fourth draft of the Y2038 design document

[Joseph Myers]

On Wed, 28 Sep 2016, Albert ARIBAUD wrote:

> - make Y2038-safe kernel provide a new syscall which tells at run time
>   what features the kernel supports, e.g. does it support 64-bit time,
>   32-bit time, both? [the need could be more generic than just time
>   support, so I tought such a syscall already existed, but I have not
>   found any in the current syscalls list -- I might have missed it
>   though]

There will be lots of syscalls added with the 64-bit time support.  If 
they are all added in the same kernel version, glibc can have a single 
internal variable for whether 64-bit time support is available in the 
kernel (set the first time it tries using such a syscall, used to tell 
whether to try such syscalls in future).

> A Y2038-safe GLIBC would thus detect when it runs on a non-Y2038-safe
> kernel (syscall absent or returning a value which amounts to 'no
> 64-bit time support built in kernel') and could bail out with an
> explicit message.

But it shouldn't bail out; it should just use the existing syscalls and 
translate as needed when the user called the 64-bit time interfaces.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Fourth draft of the Y2038 design document

[Andrew Pinski]

On Tue, Jun 21, 2016 at 3:58 PM, Albert ARIBAUD <albert.aribaud@3adev.fr> wrote:
> Hi all,
>
> I have produced a fourth draft of what will eventually become the Y2038
> design document:
>
> https://sourceware.org/glibc/wiki/Y2038ProofnessDesign?rev=83
>
> Relative to the previous draft:
>
>  * the scope clarifies that *only* Y2038 is considered, and no other
>    doomsday such as Y2106 or Y9999;
>
>  * all types directly or indirectly derived from time_t are now listed;
>
>  * all APIs using these types are now listed;
>
>  * all functions which use time_t internally are now listed;
>
>  * also listed are types and APIs related to time but which are
>    Y2038-safe (even though they might be unsafe for some other
>    doomsday, e.g. struct rpc_timeval being Y2106-unsafe).
>
> As always, comments welcome.

In the aarch64 ILP32 thread, utmp  came up as something which needs to
looked into more.  Right now utmp struct is stored out in a file and
your specification does not specify how that is going to be handled.

Thanks,
Andrew

>
> Cordialement,
> Albert ARIBAUD
> 3ADEV