Is Y2038-proofing in a glibc roadmap somewhere?

Is Y2038-proofing in a glibc roadmap somewhere?

[Albert ARIBAUD]

Hello all,

I have read https://sourceware.org/glibc/wiki/HomePage especially
sections 5.9.1 (Project TODO) and 5.10 (Project Wishlist by Developer)
and found no indication that Y2038-proofing glibc was on any roadmap for
glibc. Is it really not, or did I just miss it?

Amicalement,
Albert.

Re: Is Y2038-proofing in a glibc roadmap somewhere?

[Szabolcs Nagy]

* Albert ARIBAUD <albert.aribaud@3adev.fr> [2015-06-18 15:08:35 +0200]:
> 
> I have read https://sourceware.org/glibc/wiki/HomePage especially
> sections 5.9.1 (Project TODO) and 5.10 (Project Wishlist by Developer)
> and found no indication that Y2038-proofing glibc was on any roadmap for
> glibc. Is it really not, or did I just miss it?
> 

because it is a kernel issue and glibc cannot do much until
the problematic syscall abis can handle 64bit time_t

for current state of the linux kernel side see
https://sourceware.org/ml/libc-alpha/2015-05/msg00430.html
http://lwn.net/Articles/643407/

Re: Is Y2038-proofing in a glibc roadmap somewhere?

[Albert ARIBAUD]

Hi Szabolcs,

Le Thu, 18 Jun 2015 15:25:33 +0200, Szabolcs Nagy <nsz@port70.net> a
écrit :

> * Albert ARIBAUD <albert.aribaud@3adev.fr> [2015-06-18 15:08:35 +0200]:
> > 
> > I have read https://sourceware.org/glibc/wiki/HomePage especially
> > sections 5.9.1 (Project TODO) and 5.10 (Project Wishlist by Developer)
> > and found no indication that Y2038-proofing glibc was on any roadmap for
> > glibc. Is it really not, or did I just miss it?
> > 
> 
> because it is a kernel issue and glibc cannot do much until
> the problematic syscall abis can handle 64bit time_t
> 
> for current state of the linux kernel side see
> https://sourceware.org/ml/libc-alpha/2015-05/msg00430.html
> http://lwn.net/Articles/643407/

Thanks -- I am already aware of Arnd Bergmann's patch series --
actually, I've discussed with him outside this list.

Of course one cannot make glibc y2038-proof before the kernel is, but I
was not asking if anyone was working on y2038 already, only if anyone
had any plans about working on y2038 in glibc once a suitable kernel
is available. Sorry for the misunderstanding.

So... Does anyone around know of any plans already for making glibc
y2038-proof? :)

Amicalement,
Albert.

Re: Is Y2038-proofing in a glibc roadmap somewhere?

[Albert ARIBAUD]

Bonjour Albert,

Le Thu, 18 Jun 2015 15:49:48 +0200, Albert ARIBAUD
<albert.aribaud@3adev.fr> a écrit :

> Hi Szabolcs,
> 
> Le Thu, 18 Jun 2015 15:25:33 +0200, Szabolcs Nagy <nsz@port70.net> a
> écrit :
> 
> > * Albert ARIBAUD <albert.aribaud@3adev.fr> [2015-06-18 15:08:35 +0200]:
> > > 
> > > I have read https://sourceware.org/glibc/wiki/HomePage especially
> > > sections 5.9.1 (Project TODO) and 5.10 (Project Wishlist by Developer)
> > > and found no indication that Y2038-proofing glibc was on any roadmap for
> > > glibc. Is it really not, or did I just miss it?
> > > 
> > 
> > because it is a kernel issue and glibc cannot do much until
> > the problematic syscall abis can handle 64bit time_t
> > 
> > for current state of the linux kernel side see
> > https://sourceware.org/ml/libc-alpha/2015-05/msg00430.html
> > http://lwn.net/Articles/643407/
> 
> Thanks -- I am already aware of Arnd Bergmann's patch series --
> actually, I've discussed with him outside this list.
> 
> Of course one cannot make glibc y2038-proof before the kernel is, but I
> was not asking if anyone was working on y2038 already, only if anyone
> had any plans about working on y2038 in glibc once a suitable kernel
> is available. Sorry for the misunderstanding.
> 
> So... Does anyone around know of any plans already for making glibc
> y2038-proof? :)

I'll take this as a no. :)

I will therefore start working on it myself, based on Arnd's input.

I intend to make my work available on a regular basis, probably as a git
repo as well as (RFC) patches to this list. Is this ok?

Cordialement,
Albert ARIBAUD
3ADEV

Re: Is Y2038-proofing in a glibc roadmap somewhere?

[Joseph Myers]

On Thu, 9 Jul 2015, Albert ARIBAUD wrote:

> I'll take this as a no. :)
> 
> I will therefore start working on it myself, based on Arnd's input.
> 
> I intend to make my work available on a regular basis, probably as a git
> repo as well as (RFC) patches to this list. Is this ok?

First, please see the contribution checklist on the wiki.  In particular, 
you should make sure your copyright assignment is in place at an early 
stage; glibc reviewers are unlikely to want to look at all at any 
substantial changes without an assignment in place.

Second, I advise starting by posting an extended design document 
describing your proposed design and how various issues will be addressed, 
to avoid expending large amounts of work on an approach with fundamental 
design issues.  I think the basic requirements are clear - a macro 
_TIME_BITS=64, that is only accepted in conjunction with 
_FILE_OFFSET_BITS=64, that causes affected functions and types to be 
mapped to versions using 64-bit time_t, while keeping all existing ABIs 
as-is; that is necessitated by the basic requirement of keeping full 
compatibility with all existing binaries.  But you need to expand that 
summary from sentence length to essay length, making a thorough analysis 
of all the issues involved.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Is Y2038-proofing in a glibc roadmap somewhere?

[Albert ARIBAUD]

Hi Joseph,

Thanks for the detailed answer on how to proceed.

Le Wed, 22 Jul 2015 14:26:10 +0000, Joseph Myers
<joseph@codesourcery.com> a écrit :

> On Thu, 9 Jul 2015, Albert ARIBAUD wrote:
> 
> > I'll take this as a no. :)
> > 
> > I will therefore start working on it myself, based on Arnd's input.
> > 
> > I intend to make my work available on a regular basis, probably as a git
> > repo as well as (RFC) patches to this list. Is this ok?
> 
> First, please see the contribution checklist on the wiki.  In particular, 
> you should make sure your copyright assignment is in place at an early 
> stage; glibc reviewers are unlikely to want to look at all at any 
> substantial changes without an assignment in place.

As I don't have a copyright assignment in place with the FSF yet, the
contribution checklist copyright assignment section
(https://sourceware.org/glibc/wiki/Contribution%20checklist#FSF_copyright_Assignment)
says someone from libc-alpha should direct me to a FSF copyright
assignment request form that's best suited to your contribution under
the guidelines. According to
http://www.gnu.org/prep/maintain/maintain.html#Copyright-Papers only
maintainers and important contributors can access the copyright papers
through their fencepost account, so obviously I don't qualify for
getting the papers myself.

Just in case, I have created a Savannah account (number 100585) with a
public SSH key in place.

> Second, I advise starting by posting an extended design document 
> describing your proposed design and how various issues will be addressed, 
> to avoid expending large amounts of work on an approach with fundamental 
> design issues.  I think the basic requirements are clear - a macro 
> _TIME_BITS=64, that is only accepted in conjunction with 
> _FILE_OFFSET_BITS=64, that causes affected functions and types to be 
> mapped to versions using 64-bit time_t, while keeping all existing ABIs 
> as-is; that is necessitated by the basic requirement of keeping full 
> compatibility with all existing binaries.  But you need to expand that 
> summary from sentence length to essay length, making a thorough analysis 
> of all the issues involved.

Hmm, why the relationship with _FILE_OFFSET_BITS? At least at first
sight, supporting 64-bit time seems pretty unrelated/orthogonal to
supporting 64-bit file sizes.

Regarding the design document, am I right in assuming it should exist
as a Wiki page similar to, for instance,
https://sourceware.org/glibc/wiki/PrintfHooksDesign ? In that case, I
suspect I would need some editor to add me to EditorGroup.

Cordialement,
Albert ARIBAUD
3ADEV

Re: Is Y2038-proofing in a glibc roadmap somewhere?

[Zack Weinberg]

On Mon, Aug 3, 2015 at 4:43 AM, Albert ARIBAUD <albert.aribaud@3adev.fr> wrote:
> As I don't have a copyright assignment in place with the FSF yet, the
> contribution checklist copyright assignment section
> (https://sourceware.org/glibc/wiki/Contribution%20checklist#FSF_copyright_Assignment)
> says someone from libc-alpha should direct me to a FSF copyright
> assignment request form that's best suited to your contribution under
> the guidelines.

Since you haven't done any work yet, you want the "past and future
changes" form:

http://git.savannah.gnu.org/cgit/gnulib.git/plain/doc/Copyright/request-assign.future

> According to
> http://www.gnu.org/prep/maintain/maintain.html#Copyright-Papers only
> maintainers and important contributors can access the copyright papers
> through their fencepost account, so obviously I don't qualify for
> getting the papers myself.

This is somewhat out of date - the process used to be more
complicated.  Just fill out the form above and mail it in as
instructed.

> Hmm, why the relationship with _FILE_OFFSET_BITS? At least at first
> sight, supporting 64-bit time seems pretty unrelated/orthogonal to
> supporting 64-bit file sizes.

I don't know for certain what Joseph had in mind, but I believe that's
just to cut down on the number of combinations the header files need
to support.  'struct stat', for instance, contains both time_t and
off_t quantities, so there would have to be four different definitions
of it if _FILE_OFFSET_BITS and _TIME_BITS were independent, but only
three if _TIME_BITS=64 requires _FILE_OFFSET_BITS=64.  And restricting
it to three variants also makes it more likely that the different
structures can be told apart by sizeof().

zw

Re: Is Y2038-proofing in a glibc roadmap somewhere?

[Albert ARIBAUD]

Bonjour Zack,

Le Mon, 3 Aug 2015 09:07:25 -0400, Zack Weinberg <zackw@panix.com> a
écrit :

> On Mon, Aug 3, 2015 at 4:43 AM, Albert ARIBAUD <albert.aribaud@3adev.fr> wrote:
> > As I don't have a copyright assignment in place with the FSF yet, the
> > contribution checklist copyright assignment section
> > (https://sourceware.org/glibc/wiki/Contribution%20checklist#FSF_copyright_Assignment)
> > says someone from libc-alpha should direct me to a FSF copyright
> > assignment request form that's best suited to your contribution under
> > the guidelines.
> 
> Since you haven't done any work yet, you want the "past and future
> changes" form:
> 
> http://git.savannah.gnu.org/cgit/gnulib.git/plain/doc/Copyright/request-assign.future

Thanks. I've sent the request accordingly.

> > Hmm, why the relationship with _FILE_OFFSET_BITS? At least at first
> > sight, supporting 64-bit time seems pretty unrelated/orthogonal to
> > supporting 64-bit file sizes.
> 
> I don't know for certain what Joseph had in mind, but I believe that's
> just to cut down on the number of combinations the header files need
> to support.  'struct stat', for instance, contains both time_t and
> off_t quantities, so there would have to be four different definitions
> of it if _FILE_OFFSET_BITS and _TIME_BITS were independent, but only
> three if _TIME_BITS=64 requires _FILE_OFFSET_BITS=64.  And restricting
> it to three variants also makes it more likely that the different
> structures can be told apart by sizeof().

I get the idea, although I am not sure why we would want to use
structure size to tell variants apart. If we know there are variants
to begin with, then we know on which macro these variants depend and we
can tell the variants apart based on which macros are defined, could
we not?

> zw

Cordialement,
Albert ARIBAUD
3ADEV

Re: Is Y2038-proofing in a glibc roadmap somewhere?

[Zack Weinberg]

On Mon, Aug 3, 2015 at 9:36 AM, Albert ARIBAUD <albert.aribaud@3adev.fr> wrote:
>> 'struct stat', for instance, contains both time_t and
>> off_t quantities, so there would have to be four different definitions
>> of it if _FILE_OFFSET_BITS and _TIME_BITS were independent, but only
>> three if _TIME_BITS=64 requires _FILE_OFFSET_BITS=64.  And restricting
>> it to three variants also makes it more likely that the different
>> structures can be told apart by sizeof().
>
> I get the idea, although I am not sure why we would want to use
> structure size to tell variants apart. If we know there are variants
> to begin with, then we know on which macro these variants depend and we
> can tell the variants apart based on which macros are defined, could
> we not?

This is only a relevant factor in cases when you can't tell them apart
any other way.  'struct stat' was a bad example because that's already
handled with the xstat mechanism and the '64' suffix, but imagine that
there's some poorly-documented ioctl() taking a structure containing
both an off_t and a struct timespec.

zw

Re: Is Y2038-proofing in a glibc roadmap somewhere?

[Andreas Schwab]

Albert ARIBAUD <albert.aribaud@3adev.fr> writes:

> Hmm, why the relationship with _FILE_OFFSET_BITS? At least at first
> sight, supporting 64-bit time seems pretty unrelated/orthogonal to
> supporting 64-bit file sizes.

No new interface should support anything less than 64bit off_t.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: Is Y2038-proofing in a glibc roadmap somewhere?

[Joseph Myers]

On Mon, 3 Aug 2015, Zack Weinberg wrote:

> > Hmm, why the relationship with _FILE_OFFSET_BITS? At least at first
> > sight, supporting 64-bit time seems pretty unrelated/orthogonal to
> > supporting 64-bit file sizes.
> 
> I don't know for certain what Joseph had in mind, but I believe that's
> just to cut down on the number of combinations the header files need
> to support.  'struct stat', for instance, contains both time_t and

And the number of ABI variants / function exports in the libraries.

One thing to consider in the design is whether you have separate functions 
exported from the shared libraries at all for those cases where time_t is 
already 64-bit.  _FILE_OFFSET_BITS=64 does have separate exports (and 
public API names such as stat64 for use with _LARGEFILE64_SOURCE; I don't 
see any clear need for API names like that for time_t); avoiding them for 
time_t would complicate the headers but simplify the ABI.

Issues with _FILE_OFFSET_BITS=64 that you should definitely seek to avoid 
for _TIME_BITS=64 include: (a) stat and stat64 having different layouts on 
at least one 64-bit platform (MIPS n64) (that is, whatever _TIME_BITS=64 
does on systems where time_t is already 64-bit, it should not change the 
layout of any structures); (b) link-time namespace violations (bug 14106); 
(c) _FILE_OFFSET_BITS=64 affecting the C++ ABI even when the layout of the 
types would otherwise be the same (bug 15766).

The evidence is that libraries affected by the _FILE_OFFSET_BITS value are 
more likely nowadays to be built with _FILE_OFFSET_BITS=64 than 
_FILE_OFFSET_BITS=32 on GNU/Linux distributions.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Is Y2038-proofing in a glibc roadmap somewhere?

[Paul Eggert]

Joseph Myers wrote:
> The evidence is that libraries affected by the _FILE_OFFSET_BITS value are
> more likely nowadays to be built with _FILE_OFFSET_BITS=64 than
> _FILE_OFFSET_BITS=32 on GNU/Linux distributions.

Even at the time _FILE_OFFSET_BITS was introduced, I thought that it was a 
mistake for it to default to 32.  An understandable mistake, but a mistake 
nonetheless.  What a hassle it was to arrange for every application to #define 
_FILE_OFFSET_BITS to 64!

Can we avoid the mistake this time around, and have _TIME_BITS default to 64?

Re: Is Y2038-proofing in a glibc roadmap somewhere?

[Joseph Myers]

On Mon, 3 Aug 2015, Paul Eggert wrote:

> Joseph Myers wrote:
> > The evidence is that libraries affected by the _FILE_OFFSET_BITS value are
> > more likely nowadays to be built with _FILE_OFFSET_BITS=64 than
> > _FILE_OFFSET_BITS=32 on GNU/Linux distributions.
> 
> Even at the time _FILE_OFFSET_BITS was introduced, I thought that it was a
> mistake for it to default to 32.  An understandable mistake, but a mistake
> nonetheless.  What a hassle it was to arrange for every application to #define
> _FILE_OFFSET_BITS to 64!
> 
> Can we avoid the mistake this time around, and have _TIME_BITS default to 64?

I think that's a recipe for the addition of _TIME_BITS=64 support not 
happening at all or being delayed by several years; changing the default 
is a lot trickier than adding new interfaces, and requires significant 
distribution work to coordinate ABI changes for shared libraries.

Work towards changing the _FILE_OFFSET_BITS default - such as obtaining 
and implementing a consensus on whether to deprecate the fts interface or 
add a 64-bit version of it, and fixing (maybe bit-by-bit rather than all 
at once) bug 14106 - would be welcome, and a prerequisite for being able 
to change the _TIME_BITS default.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Is Y2038-proofing in a glibc roadmap somewhere?

[Albert ARIBAUD]

Hi Joseph,

Le Wed, 22 Jul 2015 14:26:10 +0000, Joseph Myers
<joseph@codesourcery.com> a écrit :

> On Thu, 9 Jul 2015, Albert ARIBAUD wrote:
> 
> > I'll take this as a no. :)
> > 
> > I will therefore start working on it myself, based on Arnd's input.
> > 
> > I intend to make my work available on a regular basis, probably as a git
> > repo as well as (RFC) patches to this list. Is this ok?
> 
> First, please see the contribution checklist on the wiki.  In particular, 
> you should make sure your copyright assignment is in place at an early 
> stage; glibc reviewers are unlikely to want to look at all at any 
> substantial changes without an assignment in place.

I've just received my copy of the copyright assignment signed by the
FSF. Should I make it available somewhere for the glibc folks to see?

> Second, I advise starting by posting an extended design document 
> describing your proposed design and how various issues will be addressed, 
> to avoid expending large amounts of work on an approach with fundamental 
> design issues.  I think the basic requirements are clear - a macro 
> _TIME_BITS=64, that is only accepted in conjunction with 
> _FILE_OFFSET_BITS=64, that causes affected functions and types to be 
> mapped to versions using 64-bit time_t, while keeping all existing ABIs 
> as-is; that is necessitated by the basic requirement of keeping full 
> compatibility with all existing binaries.  But you need to expand that 
> summary from sentence length to essay length, making a thorough analysis 
> of all the issues involved.

I'd like to start working on the design document, but as I wrote
before, I think than, rather than posting it to the mailing list, it
would be better to host it on a Wiki page similar to, for instance,
<https://sourceware.org/glibc/wiki/PrintfHooksDesign>.

Is this approach acceptable? If so, then I would need some editor to
add me (just registered as "AlbertAribaud" on the Wiki) to EditorGroup.

Cordialement,
Albert ARIBAUD
3ADEV

Re: Is Y2038-proofing in a glibc roadmap somewhere?

[Joseph Myers]

On Wed, 2 Sep 2015, Albert ARIBAUD wrote:

> > First, please see the contribution checklist on the wiki.  In particular, 
> > you should make sure your copyright assignment is in place at an early 
> > stage; glibc reviewers are unlikely to want to look at all at any 
> > substantial changes without an assignment in place.
> 
> I've just received my copy of the copyright assignment signed by the
> FSF. Should I make it available somewhere for the glibc folks to see?

No, we can check the list of assignments on fencepost.

> > Second, I advise starting by posting an extended design document 
> > describing your proposed design and how various issues will be addressed, 
> > to avoid expending large amounts of work on an approach with fundamental 
> > design issues.  I think the basic requirements are clear - a macro 
> > _TIME_BITS=64, that is only accepted in conjunction with 
> > _FILE_OFFSET_BITS=64, that causes affected functions and types to be 
> > mapped to versions using 64-bit time_t, while keeping all existing ABIs 
> > as-is; that is necessitated by the basic requirement of keeping full 
> > compatibility with all existing binaries.  But you need to expand that 
> > summary from sentence length to essay length, making a thorough analysis 
> > of all the issues involved.
> 
> I'd like to start working on the design document, but as I wrote
> before, I think than, rather than posting it to the mailing list, it
> would be better to host it on a Wiki page similar to, for instance,
> <https://sourceware.org/glibc/wiki/PrintfHooksDesign>.
> 
> Is this approach acceptable? If so, then I would need some editor to
> add me (just registered as "AlbertAribaud" on the Wiki) to EditorGroup.

I've added you to EditorGroup.  If using this approach, I suggest starting 
each thread with a link to the specific version of the document being 
discussed in that thread, so it's clear everyone is commenting on the same 
document rather than each person looking at a different version.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Is Y2038-proofing in a glibc roadmap somewhere?

[Albert ARIBAUD]

Hi Joseph,

Le Wed, 2 Sep 2015 14:11:55 +0000, Joseph Myers
<joseph@codesourcery.com> a écrit :

> > I've just received my copy of the copyright assignment signed by the
> > FSF. Should I make it available somewhere for the glibc folks to see?
> 
> No, we can check the list of assignments on fencepost.

Ok.

> > > Second, I advise starting by posting an extended design document 
> > > describing your proposed design and how various issues will be addressed, 
> > > to avoid expending large amounts of work on an approach with fundamental 
> > > design issues.  I think the basic requirements are clear - a macro 
> > > _TIME_BITS=64, that is only accepted in conjunction with 
> > > _FILE_OFFSET_BITS=64, that causes affected functions and types to be 
> > > mapped to versions using 64-bit time_t, while keeping all existing ABIs 
> > > as-is; that is necessitated by the basic requirement of keeping full 
> > > compatibility with all existing binaries.  But you need to expand that 
> > > summary from sentence length to essay length, making a thorough analysis 
> > > of all the issues involved.
> > 
> > I'd like to start working on the design document, but as I wrote
> > before, I think than, rather than posting it to the mailing list, it
> > would be better to host it on a Wiki page similar to, for instance,
> > <https://sourceware.org/glibc/wiki/PrintfHooksDesign>.
> > 
> > Is this approach acceptable? If so, then I would need some editor to
> > add me (just registered as "AlbertAribaud" on the Wiki) to EditorGroup.
> 
> I've added you to EditorGroup.

Thanks a lot!

> If using this approach, I suggest starting 
> each thread with a link to the specific version of the document being 
> discussed in that thread, so it's clear everyone is commenting on the same 
> document rather than each person looking at a different version.

Agreed, and wilco.

Thanks again for your help!

Cordialement,
Albert ARIBAUD
3ADEV