The time64 and Large File Support mess

The time64 and Large File Support mess

[Andreas K. Huettel]

[-- Attachment #1: Type: text/plain, Size: 932 bytes --]

Dear all, 

now that the 2.37 release and with that another half year of delay is near, 
I'd like to remind everyone that we still need to find some decent solution
for the remaining problems associated with time64 and LFS as discussed
earlier [1].

As far as I can see at the moment there are two things that need a solution.

1) qemu usermode emulation is broken when emulating a 32bit arch from a 64bit
   arch, bug 23960
 
2) the transition to default time64 breaks legacy 32bit binaries in
   unpredictable ways

I'm going to reply with two separate e-mails for these issues, and would be
very happy for your opinions and contributions on the respective threads.

Cheers,
Andreas 
(Gentoo toolchain team)

[1] https://sourceware.org/pipermail/libc-alpha/2022-November/143386.html

-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer
(council, toolchain, base-system, perl, libreoffice)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

time64 / Large File Support: 1) [2.28 Regression]: New getdents{64} implementation breaks qemu-user

[Andreas K. Huettel]

[-- Attachment #1: Type: text/plain, Size: 2208 bytes --]

> 1) qemu usermode emulation is broken when emulating a 32bit arch from a 64bit
>    arch, bug 23960

This is bug 23960 [2] on the sourceware bugzilla. As you can see it's been around
for a bit, and breaks emulating a 32bit architecture on a 64bit architecture.

Fear not, there is a fix, a patchset by Adhemerval Zanella [3]. It's been part of
Gentoo's glibc for several years now, solves the problem fully, and has
not led to any problems. 
(We are in Gentoo Release Engineering heavy qemu usermode emulation users, building
for riscv, mips, loongarch64, m68k, alpha...)

Pre-2.37 the patchset stopped applying. To my best knowledge the 
azanella/bz23960-dirent branch [4] provides an updated version. I tried to
verify that with Adhemerval, however unfortunately he never replied to my
pings on IRC, so that confirmation is still missing.

* Is that the current fix?
* Can we please merge it?

[2] https://sourceware.org/bugzilla/show_bug.cgi?id=23960
[3] https://gitweb.gentoo.org/fork/glibc.git/log/?h=gentoo/glibc-2.36-1
[4] https://sourceware.org/git/?p=glibc.git;a=shortlog;h=refs/heads/azanella/bz23960-dirent


Am Donnerstag, 26. Januar 2023, 00:57:31 CET schrieb Andreas K. Huettel:
> Dear all, 
> 
> now that the 2.37 release and with that another half year of delay is near, 
> I'd like to remind everyone that we still need to find some decent solution
> for the remaining problems associated with time64 and LFS as discussed
> earlier [1].
> 
> As far as I can see at the moment there are two things that need a solution.
> 
> 1) qemu usermode emulation is broken when emulating a 32bit arch from a 64bit
>    arch, bug 23960
>  
> 2) the transition to default time64 breaks legacy 32bit binaries in
>    unpredictable ways
> 
> I'm going to reply with two separate e-mails for these issues, and would be
> very happy for your opinions and contributions on the respective threads.
> 
> Cheers,
> Andreas 
> (Gentoo toolchain team)
> 
> [1] https://sourceware.org/pipermail/libc-alpha/2022-November/143386.html
> 
> 


-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer
(council, toolchain, base-system, perl, libreoffice)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: time64 / Large File Support: 1) [2.28 Regression]: New getdents{64} implementation breaks qemu-user

[Adhemerval Zanella Netto]

Hi,

I did see your pings on IRC and I recall I rebased my tree against master after
it [1].  It is against master, so it should apply to 2.37.

And the main issue I did not get much replies on last iteration, I will send a
new version so we can track it for 2.38 (and maybe backport since it does not
interfere with ABI).

[1] https://sourceware.org/git/?p=glibc.git;a=shortlog;h=refs/heads/azanella/bz23960-dirent

On 25/01/23 20:58, Andreas K. Huettel via Libc-alpha wrote:
>> 1) qemu usermode emulation is broken when emulating a 32bit arch from a 64bit
>>    arch, bug 23960
> 
> This is bug 23960 [2] on the sourceware bugzilla. As you can see it's been around
> for a bit, and breaks emulating a 32bit architecture on a 64bit architecture.
> 
> Fear not, there is a fix, a patchset by Adhemerval Zanella [3]. It's been part of
> Gentoo's glibc for several years now, solves the problem fully, and has
> not led to any problems. 
> (We are in Gentoo Release Engineering heavy qemu usermode emulation users, building
> for riscv, mips, loongarch64, m68k, alpha...)
> 
> Pre-2.37 the patchset stopped applying. To my best knowledge the 
> azanella/bz23960-dirent branch [4] provides an updated version. I tried to
> verify that with Adhemerval, however unfortunately he never replied to my
> pings on IRC, so that confirmation is still missing.
> 
> * Is that the current fix?
> * Can we please merge it?
> 
> [2] https://sourceware.org/bugzilla/show_bug.cgi?id=23960
> [3] https://gitweb.gentoo.org/fork/glibc.git/log/?h=gentoo/glibc-2.36-1
> [4] https://sourceware.org/git/?p=glibc.git;a=shortlog;h=refs/heads/azanella/bz23960-dirent
> 
> 
> Am Donnerstag, 26. Januar 2023, 00:57:31 CET schrieb Andreas K. Huettel:
>> Dear all, 
>>
>> now that the 2.37 release and with that another half year of delay is near, 
>> I'd like to remind everyone that we still need to find some decent solution
>> for the remaining problems associated with time64 and LFS as discussed
>> earlier [1].
>>
>> As far as I can see at the moment there are two things that need a solution.
>>
>> 1) qemu usermode emulation is broken when emulating a 32bit arch from a 64bit
>>    arch, bug 23960
>>  
>> 2) the transition to default time64 breaks legacy 32bit binaries in
>>    unpredictable ways
>>
>> I'm going to reply with two separate e-mails for these issues, and would be
>> very happy for your opinions and contributions on the respective threads.
>>
>> Cheers,
>> Andreas 
>> (Gentoo toolchain team)
>>
>> [1] https://sourceware.org/pipermail/libc-alpha/2022-November/143386.html
>>
>>
> 
> 

Re: time64 / Large File Support: 1) [2.28 Regression]: New getdents{64} implementation breaks qemu-user

[Andreas K. Huettel]

[-- Attachment #1: Type: text/plain, Size: 583 bytes --]

>
> I did see your pings on IRC and I recall I rebased my tree against master after
> it [1].  It is against master, so it should apply to 2.37.
> 
> And the main issue I did not get much replies on last iteration, I will send a
> new version so we can track it for 2.38 (and maybe backport since it does not
> interfere with ABI).
> 
> [1] https://sourceware.org/git/?p=glibc.git;a=shortlog;h=refs/heads/azanella/bz23960-dirent
> 

Thanks a lot!

-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer
(council, toolchain, base-system, perl, libreoffice)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

time64 / Large File Support: 2) default time64 breaks legacy 32bit binaries

[Andreas K. Huettel]

[-- Attachment #1: Type: text/plain, Size: 2001 bytes --]

> 2) the transition to default time64 breaks legacy 32bit binaries in
>    unpredictable ways

Since the size of time_t is changing, a transition to a default 64bit interface
changes the public ABI of all libraries that integrate this type in any way.
Which means, if an application is linked to such a library, and the library is
later recompiled, undefined behaviour ensues.

This was discussed already in the previous thread on this list [1], with reactions
ranging from "need new triplet" via "need new libdir" to "meh".
The latter is a bit surprising given how much emphasis glibc usually places on 
backwards compatibility.

While we could certainly go ahead and invent a solution in Gentoo here, this
makes no sense at all for one distribution alone. This is an upstream problem
and should be solved here... pretty please? opinions?

[1] https://sourceware.org/pipermail/libc-alpha/2022-November/143386.html



Am Donnerstag, 26. Januar 2023, 00:57:31 CET schrieb Andreas K. Huettel:
> Dear all, 
> 
> now that the 2.37 release and with that another half year of delay is near, 
> I'd like to remind everyone that we still need to find some decent solution
> for the remaining problems associated with time64 and LFS as discussed
> earlier [1].
> 
> As far as I can see at the moment there are two things that need a solution.
> 
> 1) qemu usermode emulation is broken when emulating a 32bit arch from a 64bit
>    arch, bug 23960
>  
> 2) the transition to default time64 breaks legacy 32bit binaries in
>    unpredictable ways
> 
> I'm going to reply with two separate e-mails for these issues, and would be
> very happy for your opinions and contributions on the respective threads.
> 
> Cheers,
> Andreas 
> (Gentoo toolchain team)
> 
> [1] https://sourceware.org/pipermail/libc-alpha/2022-November/143386.html
> 
> 


-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer
(council, toolchain, base-system, perl, libreoffice)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: time64 / Large File Support: 2) default time64 breaks legacy 32bit binaries

[Paul Eggert]

On 1/25/23 15:59, Andreas K. Huettel via Libc-alpha wrote:

> This was discussed already in the previous thread on this list [1], with reactions
> ranging from "need new triplet" via "need new libdir" to "meh"....
> [1] https://sourceware.org/pipermail/libc-alpha/2022-November/143386.html

One thing new since that November email is that in bleeding-edge 
Autoconf we've scaled back AC_SYS_LARGEFILE so it no longer widens 
time_t by default. Instead, you need to pass a new option 
--enable-year2038 to 'configure' if you want 64-bit time_t on 32-bit 
glibc x86 and ARM platforms, which as I understand it are the only 
platforms that have this problem. If a package author wants 
--enable-year2038 to be the default, they need to use Autoconf's new 
AC_SYS_YEAR2038 macro. This change has also percolated into Gnulib so 
source packages using recent Gnulib will need to use the new Gnulib 
module year2038 if they want --enable-year2038 to be the default.

This change was done out of concern that although AC_SYS_LARGEFILE has 
long tweaked blkcnt_t, dev_t, ino_t, fsblkcnt_t, fsfilcnt_t and rlim_t 
(in addition to off_t of course), having it also tweak time_t was a 
compatibility bridge too far.

> Proposal: glibc gains two new build-time configure options:
> * --enable-hard-time64
> * --enable-hard-lfs

This sort of thing sounds like a good way to go. However, I suggest 
simplifying things, by having just one option (say, 
--enable-hard-sys-types64) that does both at once, because 
--enable-hard-time64 and --enable-hard-lfs would not be orthogonal and 
this would be confusing, and anyway nobody sane will want to use one 
option without also using the other - who wants the agony of *two* 
conversions?

Re: time64 / Large File Support: 2) default time64 breaks legacy 32bit binaries

[Adhemerval Zanella Netto]

On 26/01/23 01:13, Paul Eggert wrote:
> On 1/25/23 15:59, Andreas K. Huettel via Libc-alpha wrote:
> 
>> This was discussed already in the previous thread on this list [1], with reactions
>> ranging from "need new triplet" via "need new libdir" to "meh"....
>> [1] https://sourceware.org/pipermail/libc-alpha/2022-November/143386.html
> 
> One thing new since that November email is that in bleeding-edge Autoconf we've scaled back AC_SYS_LARGEFILE so it no longer widens time_t by default. Instead, you need to pass a new option --enable-year2038 to 'configure' if you want 64-bit time_t on 32-bit glibc x86 and ARM platforms, which as I understand it are the only platforms that have this problem. If a package author wants --enable-year2038 to be the default, they need to use Autoconf's new AC_SYS_YEAR2038 macro. This change has also percolated into Gnulib so source packages using recent Gnulib will need to use the new Gnulib module year2038 if they want --enable-year2038 to be the default.
> 
> This change was done out of concern that although AC_SYS_LARGEFILE has long tweaked blkcnt_t, dev_t, ino_t, fsblkcnt_t, fsfilcnt_t and rlim_t (in addition to off_t of course), having it also tweak time_t was a compatibility bridge too far.
> 
>> Proposal: glibc gains two new build-time configure options:
>> * --enable-hard-time64
>> * --enable-hard-lfs
> 
> This sort of thing sounds like a good way to go. However, I suggest simplifying things, by having just one option (say, --enable-hard-sys-types64) that does both at once, because --enable-hard-time64 and --enable-hard-lfs would not be orthogonal and this would be confusing, and anyway nobody sane will want to use one option without also using the other - who wants the agony of *two* conversions?

I agree a single option make sense, there is no good reason to add LFS-only
with 64-bit support. It also simplify build systems that are not autoconf 
based.

A minor problem, which is for all configure switch, it adds another build 
permutation that incur in more testing requirements and maintenance. 

However it does not help with the rest of plumbing that a system will need
to do for correct set library selection, since ldconfig will see both 32 and
64 bit time_t shared library essentially being the same ABI.  A mixed 
environment with legacy binaries/libraries will still incur in similar 
issue, albeit in a different direction.  So to run old binaries one will 
need to either setup LD_PRELOAD/LD_LIBRARY_PATH/RUNPATH or run it in an 
isolated environment (which itself has its own issues).

And the configure switch also adds a kind of fragmentation, but it is also
we already have when a projects enables time64_t anyway.

So although I am not quite against --enable-hard-sys-types64, I personally
think we should do something more drastically (which not all other glibc
developers agree) and flip the switch to enable 64-bit time_t *as default*
and document 32-bit is opt-in. If Fedora or any distro wants to keep the
*broken* non-LFS / 32 bit time_t, it is up to them to patch glibc to do so.

Re: time64 / Large File Support: 2) default time64 breaks legacy 32bit binaries

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 4204 bytes --]



> On 26 Jan 2023, at 13:21, Adhemerval Zanella Netto via Libc-alpha <libc-alpha@sourceware.org> wrote:
> 
> 
> 
> On 26/01/23 01:13, Paul Eggert wrote:
>> On 1/25/23 15:59, Andreas K. Huettel via Libc-alpha wrote:
>> 
>>> This was discussed already in the previous thread on this list [1], with reactions
>>> ranging from "need new triplet" via "need new libdir" to "meh"....
>>> [1] https://sourceware.org/pipermail/libc-alpha/2022-November/143386.html
>> 
>> One thing new since that November email is that in bleeding-edge Autoconf we've scaled back AC_SYS_LARGEFILE so it no longer widens time_t by default. Instead, you need to pass a new option --enable-year2038 to 'configure' if you want 64-bit time_t on 32-bit glibc x86 and ARM platforms, which as I understand it are the only platforms that have this problem. If a package author wants --enable-year2038 to be the default, they need to use Autoconf's new AC_SYS_YEAR2038 macro. This change has also percolated into Gnulib so source packages using recent Gnulib will need to use the new Gnulib module year2038 if they want --enable-year2038 to be the default.
>> 
>> This change was done out of concern that although AC_SYS_LARGEFILE has long tweaked blkcnt_t, dev_t, ino_t, fsblkcnt_t, fsfilcnt_t and rlim_t (in addition to off_t of course), having it also tweak time_t was a compatibility bridge too far.
>> 
>>> Proposal: glibc gains two new build-time configure options:
>>> * --enable-hard-time64
>>> * --enable-hard-lfs
>> 
>> This sort of thing sounds like a good way to go. However, I suggest simplifying things, by having just one option (say, --enable-hard-sys-types64) that does both at once, because --enable-hard-time64 and --enable-hard-lfs would not be orthogonal and this would be confusing, and anyway nobody sane will want to use one option without also using the other - who wants the agony of *two* conversions?
> 
> I agree a single option make sense, there is no good reason to add LFS-only
> with 64-bit support. It also simplify build systems that are not autoconf
> based.
> 

Single option is fine with me and I agree it makes more sense.

> A minor problem, which is for all configure switch, it adds another build
> permutation that incur in more testing requirements and maintenance.
> 
> However it does not help with the rest of plumbing that a system will need
> to do for correct set library selection, since ldconfig will see both 32 and
> 64 bit time_t shared library essentially being the same ABI.  A mixed
> environment with legacy binaries/libraries will still incur in similar
> issue, albeit in a different direction.  So to run old binaries one will
> need to either setup LD_PRELOAD/LD_LIBRARY_PATH/RUNPATH or run it in an
> isolated environment (which itself has its own issues).
> 

My feeling was that anyone who continues to need 32-bit time_t would just
run a system without such a glibc built and wouldn't contaminate it with
64-bit time_t binaries.

> And the configure switch also adds a kind of fragmentation, but it is also
> we already have when a projects enables time64_t anyway.
> 

Yeah, I think the ship has more-or-less sailed, but my hope is that we'd
all agree to do this as distros around the same time with new ABI
names to indicate it.

> So although I am not quite against --enable-hard-sys-types64, I personally
> think we should do something more drastically (which not all other glibc
> developers agree) and flip the switch to enable 64-bit time_t *as default*
> and document 32-bit is opt-in. If Fedora or any distro wants to keep the
> *broken* non-LFS / 32 bit time_t, it is up to them to patch glibc to do so.

Right, it seems RH has some needs due to supporting existing customers,
but I don't think this should unduly affect what glibc upstream does if there's
one clear technical path forward. Nobody seems to actually dispute that
the end-game here is a hard switch at some point. Just about when.

But I'm a bit less bothered about this if we're saying that we only need
to wait 2 years or so. I could live with that if we really have to. Not ideal,
but my hands are full at the moment, so...


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 358 bytes --]

Re: time64 / Large File Support: 2) default time64 breaks legacy 32bit binaries

[Adhemerval Zanella Netto]

On 26/01/23 20:35, Sam James wrote:
> 
> 
>> On 26 Jan 2023, at 13:21, Adhemerval Zanella Netto via Libc-alpha <libc-alpha@sourceware.org> wrote:
>>
>>
>>
>> On 26/01/23 01:13, Paul Eggert wrote:
>>> On 1/25/23 15:59, Andreas K. Huettel via Libc-alpha wrote:
>>>
>>>> This was discussed already in the previous thread on this list [1], with reactions
>>>> ranging from "need new triplet" via "need new libdir" to "meh"....
>>>> [1] https://sourceware.org/pipermail/libc-alpha/2022-November/143386.html
>>>
>>> One thing new since that November email is that in bleeding-edge Autoconf we've scaled back AC_SYS_LARGEFILE so it no longer widens time_t by default. Instead, you need to pass a new option --enable-year2038 to 'configure' if you want 64-bit time_t on 32-bit glibc x86 and ARM platforms, which as I understand it are the only platforms that have this problem. If a package author wants --enable-year2038 to be the default, they need to use Autoconf's new AC_SYS_YEAR2038 macro. This change has also percolated into Gnulib so source packages using recent Gnulib will need to use the new Gnulib module year2038 if they want --enable-year2038 to be the default.
>>>
>>> This change was done out of concern that although AC_SYS_LARGEFILE has long tweaked blkcnt_t, dev_t, ino_t, fsblkcnt_t, fsfilcnt_t and rlim_t (in addition to off_t of course), having it also tweak time_t was a compatibility bridge too far.
>>>
>>>> Proposal: glibc gains two new build-time configure options:
>>>> * --enable-hard-time64
>>>> * --enable-hard-lfs
>>>
>>> This sort of thing sounds like a good way to go. However, I suggest simplifying things, by having just one option (say, --enable-hard-sys-types64) that does both at once, because --enable-hard-time64 and --enable-hard-lfs would not be orthogonal and this would be confusing, and anyway nobody sane will want to use one option without also using the other - who wants the agony of *two* conversions?
>>
>> I agree a single option make sense, there is no good reason to add LFS-only
>> with 64-bit support. It also simplify build systems that are not autoconf
>> based.
>>
> 
> Single option is fine with me and I agree it makes more sense.
> 
>> A minor problem, which is for all configure switch, it adds another build
>> permutation that incur in more testing requirements and maintenance.
>>
>> However it does not help with the rest of plumbing that a system will need
>> to do for correct set library selection, since ldconfig will see both 32 and
>> 64 bit time_t shared library essentially being the same ABI.  A mixed
>> environment with legacy binaries/libraries will still incur in similar
>> issue, albeit in a different direction.  So to run old binaries one will
>> need to either setup LD_PRELOAD/LD_LIBRARY_PATH/RUNPATH or run it in an
>> isolated environment (which itself has its own issues).
>>
> 
> My feeling was that anyone who continues to need 32-bit time_t would just
> run a system without such a glibc built and wouldn't contaminate it with
> 64-bit time_t binaries.

That is my expectations as well, however it requires just a single broken 
library in the loop to trigger a cascade of issues.  I recall that we
consider to mark the library using 64-bit time_t with an ABI mark, but it
would be too troublesome to implement (with some corner cases as well). 

> 
>> And the configure switch also adds a kind of fragmentation, but it is also
>> we already have when a projects enables time64_t anyway.
>>
> 
> Yeah, I think the ship has more-or-less sailed, but my hope is that we'd
> all agree to do this as distros around the same time with new ABI
> names to indicate it.
> 
>> So although I am not quite against --enable-hard-sys-types64, I personally
>> think we should do something more drastically (which not all other glibc
>> developers agree) and flip the switch to enable 64-bit time_t *as default*
>> and document 32-bit is opt-in. If Fedora or any distro wants to keep the
>> *broken* non-LFS / 32 bit time_t, it is up to them to patch glibc to do so.
> 
> Right, it seems RH has some needs due to supporting existing customers,
> but I don't think this should unduly affect what glibc upstream does if there's
> one clear technical path forward. Nobody seems to actually dispute that
> the end-game here is a hard switch at some point. Just about when.
> 
> But I'm a bit less bothered about this if we're saying that we only need
> to wait 2 years or so. I could live with that if we really have to. Not ideal,
> but my hands are full at the moment, so...

I agree that a hard switch make sense if the idea is to keep providing 32 bit
environments (which is the whole point of 64 bit time_t support).  As Florian
has hinted, RH will eventually phase out 32-bit support and thus we can might
eventually move forward and make 64-bit time_t the default ABI.

Re: time64 / Large File Support: 2) default time64 breaks legacy 32bit binaries

[Florian Weimer]

* Sam James via Libc-alpha:

> Right, it seems RH has some needs due to supporting existing
> customers, but I don't think this should unduly affect what glibc
> upstream does if there's one clear technical path forward. Nobody
> seems to actually dispute that the end-game here is a hard switch at
> some point. Just about when.

I'm mostly worried about the glibc project issuing a statement that
distributions should change the i386 ABI of libraries layered on top of
glibc.  I don't quite see how this is in anyone's interest.

What's the reason for a 32-bit x86 distribution at this point?  Support
for old hardware which can't run 64-bit binaries?  Running 32-bit
binaries which can't be rebuilt?  Optimizing the footprint of workloads
that fit into a 32-bit address space?

Only the first reason (old hardware on current distributions) can
justify the ABI bump.  And it's in direct conflict with the second
reason.

For the third reason, it may be more approriate to revive the x32 port.
At least it doesn't interfere with legacy use.  There's a non-upstream
ILP32 port for aarch64, too, so the largely the same choice is available
over there (although 32-bit-only CPUs that may run glibc code are still
being manufactured in the Arm ecosystem, so the tradeoffs are different
over there, I assume).

My hunch is that the most likely outcome of switching distributions to
64-bit time_t is that the i386 port goes away more quickly than I would
expect otherwise because it makes the port rather useless for most
existing users (especially with those distributions that no longer
maintain a 32-bit-only kernel).  Personally, that wouldn't be a bad
outcome at all for me.  But I doubt that this is what people who push
for this change have in mind.

Thanks,
Florian

Re: time64 / Large File Support: 2) default time64 breaks legacy 32bit binaries

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 3763 bytes --]



> On 1 Feb 2023, at 16:26, Florian Weimer <fweimer@redhat.com> wrote:
> 
> * Sam James via Libc-alpha:
> 
>> Right, it seems RH has some needs due to supporting existing
>> customers, but I don't think this should unduly affect what glibc
>> upstream does if there's one clear technical path forward. Nobody
>> seems to actually dispute that the end-game here is a hard switch at
>> some point. Just about when.
> 
> I'm mostly worried about the glibc project issuing a statement that
> distributions should change the i386 ABI of libraries layered on top of
> glibc.  I don't quite see how this is in anyone's interest.

From my side, the issue is that glibc has enabled something which is
technically necessary, but given no guidance. This enables people to
footgun (and there's a risk of e.g. Debian moving first on this, they're
very keen to) and cause problems for compatibility, at least for arm.)

I get your perspective and if the result from this is "let's wait 2 more
years and no general-purpose distributions should be making the switch",
then that's not a terrible outcome. But providing no guidance whatsoever
Is dangerous, I feel.

Even "wait until we figure this out" as guidance would be useful.

> 
> What's the reason for a 32-bit x86 distribution at this point?  Support
> for old hardware which can't run 64-bit binaries?  Running 32-bit
> binaries which can't be rebuilt?  Optimizing the footprint of workloads
> that fit into a 32-bit address space?
> Only the first reason (old hardware on current distributions) can
> justify the ABI bump.  And it's in direct conflict with the second
> reason.

Right, my motivation is almost entirely old hardware on curren
distributions.

> 
> For the third reason, it may be more approriate to revive the x32 port.
> At least it doesn't interfere with legacy use.  There's a non-upstream
> ILP32 port for aarch64, too, so the largely the same choice is available
> over there (although 32-bit-only CPUs that may run glibc code are still
> being manufactured in the Arm ecosystem, so the tradeoffs are different
> over there, I assume).

Yes, a lot of my concerns come from either modern or somewhat modern
Arm hardware.

I too am not very sympathetic to the desire to run 32-bit userland
If the hardware is capable and software can be rebuilt if it's for
resource reasons - they can use x32 if they insist.

(As an aside, I love - and agree - with the acknowledgement that
x32 is presently dead ;))

> 
> My hunch is that the most likely outcome of switching distributions to
> 64-bit time_t is that the i386 port goes away more quickly than I would
> expect otherwise because it makes the port rather useless for most
> existing users (especially with those distributions that no longer
> maintain a 32-bit-only kernel).  Personally, that wouldn't be a bad
> outcome at all for me.  But I doubt that this is what people who push
> for this change have in mind.

Agreed.

Then we're back to "how do we enable people to make the transition
where they have the freedom to rebuild userspace?"

If glibc didn't see the value in that at all, it shouldn't have added
the options. But the options are there, they're in the wild,
and they're documented. So people have, are, and are going to
try to use them.

Then we're back to:
1. How do we stop people doing it dangerously?

2. How do we enable distributions interested in supporting
cases where the hardware only works for 32-bit & they can
rebuild their userland in its entirety?

If we don't want to do 2. yet, then we need to figure out
something for 1. Saying "wait because there aren't
the resources to make the migration sane yet" would
work.

Best,
sam


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 358 bytes --]

Re: time64 / Large File Support: 2) default time64 breaks legacy 32bit binaries

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 822 bytes --]



> On 1 Feb 2023, at 19:47, Sam James via Libc-alpha <libc-alpha@sourceware.org> wrote:
> [snip]
> 
> Then we're back to:
> 1. How do we stop people doing it dangerously?

> 2. How do we enable distributions interested in supporting
> cases where the hardware only works for 32-bit & they can
> rebuild their userland in its entirety?
> 
> If we don't want to do 2. yet, then we need to figure out
> something for 1. Saying "wait because there aren't
> the resources to make the migration sane yet" would
> work.

One more thing: I'm fine with us saying to wait, because
ideally we'd be done (lol) with Modern C porting first,
because implicit function declarations affect doing mass-
rebuilds anyway if someone is doing this via the package
manager rather than hard-changing the glibc default.



[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 358 bytes --]

Re: time64 / Large File Support: 2) default time64 breaks legacy 32bit binaries

[Florian Weimer]

* Sam James via Libc-alpha:

>> On 1 Feb 2023, at 16:26, Florian Weimer <fweimer@redhat.com> wrote:
>> 
>> * Sam James via Libc-alpha:
>> 
>>> Right, it seems RH has some needs due to supporting existing
>>> customers, but I don't think this should unduly affect what glibc
>>> upstream does if there's one clear technical path forward. Nobody
>>> seems to actually dispute that the end-game here is a hard switch at
>>> some point. Just about when.
>> 
>> I'm mostly worried about the glibc project issuing a statement that
>> distributions should change the i386 ABI of libraries layered on top of
>> glibc.  I don't quite see how this is in anyone's interest.
>
> From my side, the issue is that glibc has enabled something which is
> technically necessary, but given no guidance. This enables people to
> footgun (and there's a risk of e.g. Debian moving first on this, they're
> very keen to) and cause problems for compatibility, at least for arm.)
>
> I get your perspective and if the result from this is "let's wait 2 more
> years and no general-purpose distributions should be making the switch",
> then that's not a terrible outcome. But providing no guidance whatsoever
> Is dangerous, I feel.
>
> Even "wait until we figure this out" as guidance would be useful.

I can try to write a patch to amend the old NEWS entry.

> Then we're back to "how do we enable people to make the transition
> where they have the freedom to rebuild userspace?"

It depends on whether these people want to do actual work.

The full rebuild solves only some of the issues.

For example, we didn't add a __dlsym_time64 alias, so each time some FFI
(think Python ctypes, but pretty much anything that doesn't use the C
headers in the intended way) looks up “gettimeofday”, they get the
32-bit version.  This issue won't go away with a rebuild.  It will still
be there as long as the rebuild uses the standard i386 ABI, even if we
add a configure switch that enables 64-bit time_t by default and glibc
is built using this switch.  Maybe we can patch things up upstream so
that it is possible to build with

DEFAULT GLIBC_2.34

in sysdeps/unix/sysv/linux/i386/shlib-versions and the time32 interfaces
are just gone.  But that results in a really non-standard ABI.

> One more thing: I'm fine with us saying to wait, because
> ideally we'd be done (lol) with Modern C porting first,
> because implicit function declarations affect doing mass-
> rebuilds anyway if someone is doing this via the package
> manager rather than hard-changing the glibc default.

You are right, there is a definite connection here: if you call any of
the time64 functions through an implicit declaration, you won't get the
64-bit redirects, either.  It's similar to the dlsym problem.

Thanks,
Florian

Re: time64 / Large File Support: 2) default time64 breaks legacy 32bit binaries

[Michael Hudson-Doyle]

[-- Attachment #1: Type: text/plain, Size: 3232 bytes --]

On Thu, 2 Feb 2023 at 05:27, Florian Weimer via Libc-alpha <
libc-alpha@sourceware.org> wrote:

> * Sam James via Libc-alpha:
>
> > Right, it seems RH has some needs due to supporting existing
> > customers, but I don't think this should unduly affect what glibc
> > upstream does if there's one clear technical path forward. Nobody
> > seems to actually dispute that the end-game here is a hard switch at
> > some point. Just about when.
>
> I'm mostly worried about the glibc project issuing a statement that
> distributions should change the i386 ABI of libraries layered on top of
> glibc.  I don't quite see how this is in anyone's interest.
>
> What's the reason for a 32-bit x86 distribution at this point?  Support
> for old hardware which can't run 64-bit binaries?  Running 32-bit
> binaries which can't be rebuilt?  Optimizing the footprint of workloads
> that fit into a 32-bit address space?
>

The only reason Ubuntu still builds any packages on i386 at all is reason 2
here, to allow people to run old binaries that can't be rebuilt. As time
passes the binaries that people care about are increasingly games. I
presume for the cases where y2038 matters the default solution will be to
run these binaries in a namespace that lies about the time.

Only the first reason (old hardware on current distributions) can
> justify the ABI bump.  And it's in direct conflict with the second
> reason.
>

Exactly.


> For the third reason, it may be more approriate to revive the x32 port.
> At least it doesn't interfere with legacy use.  There's a non-upstream
> ILP32 port for aarch64, too, so the largely the same choice is available
> over there (although 32-bit-only CPUs that may run glibc code are still
> being manufactured in the Arm ecosystem, so the tradeoffs are different
> over there, I assume).
>

The 32-bit arm would is an entirely different calculation, yes. There are
definitely devices being manufactured today that run Linux 32-bit that have
the sort of expected lifetime that will bump into y2038 issues. I really
hope the manufacturers have a plan! OTOH, there isn't a large pile of old
armhf binaries lying around that people want to keep working in the same
way there is on x86.

For people who use yocto or buildroot they should just start passing
-D_TIME_BITS=64 and rebuild the world but general purpose, package based
distributions have a more delicate dance to perform (FWIW, we don't have a
fully fledged plan for Ubuntu yet but we're certainly starting to think
about it).


> My hunch is that the most likely outcome of switching distributions to
> 64-bit time_t is that the i386 port goes away more quickly than I would
> expect otherwise because it makes the port rather useless for most
> existing users (especially with those distributions that no longer
> maintain a 32-bit-only kernel).  Personally, that wouldn't be a bad
> outcome at all for me.  But I doubt that this is what people who push
> for this change have in mind.
>

Is anyone proposing that glibc will lose the ability to have the 32-bit
time_t ABI? As you note that would kill the Ubuntu i386 port entirely (at
some point this will probably happen anyway, people can always play their
games in VMs but still).

Cheers,
mwh

Re: time64 / Large File Support: 2) default time64 breaks legacy 32bit binaries

[Adhemerval Zanella Netto]

On 01/02/23 19:22, Michael Hudson-Doyle wrote:
> On Thu, 2 Feb 2023 at 05:27, Florian Weimer via Libc-alpha <libc-alpha@sourceware.org <mailto:libc-alpha@sourceware.org>> wrote:
> 
>     * Sam James via Libc-alpha:
> 
>     > Right, it seems RH has some needs due to supporting existing
>     > customers, but I don't think this should unduly affect what glibc
>     > upstream does if there's one clear technical path forward. Nobody
>     > seems to actually dispute that the end-game here is a hard switch at
>     > some point. Just about when.
> 
>     I'm mostly worried about the glibc project issuing a statement that
>     distributions should change the i386 ABI of libraries layered on top of
>     glibc.  I don't quite see how this is in anyone's interest.
> 
>     What's the reason for a 32-bit x86 distribution at this point?  Support
>     for old hardware which can't run 64-bit binaries?  Running 32-bit
>     binaries which can't be rebuilt?  Optimizing the footprint of workloads
>     that fit into a 32-bit address space?
> 
> 
> The only reason Ubuntu still builds any packages on i386 at all is reason 2 here, to allow people to run old binaries that can't be rebuilt. As time passes the binaries that people care about are increasingly games. I presume for the cases where y2038 matters the default solution will be to run these binaries in a namespace that lies about the time.

It is an option, although I am not sure what might really work since afaik
Linux does not support CLOCK_REALTIME on time namespace.

> 
>     Only the first reason (old hardware on current distributions) can
>     justify the ABI bump.  And it's in direct conflict with the second
>     reason.
> 
> 
> Exactly.
>  
> 
>     For the third reason, it may be more approriate to revive the x32 port.
>     At least it doesn't interfere with legacy use.  There's a non-upstream
>     ILP32 port for aarch64, too, so the largely the same choice is available
>     over there (although 32-bit-only CPUs that may run glibc code are still
>     being manufactured in the Arm ecosystem, so the tradeoffs are different
>     over there, I assume).
> 
> 
> The 32-bit arm would is an entirely different calculation, yes. There are definitely devices being manufactured today that run Linux 32-bit that have the sort of expected lifetime that will bump into y2038 issues. I really hope the manufacturers have a plan! OTOH, there isn't a large pile of old armhf binaries lying around that people want to keep working in the same way there is on x86.
> 
> For people who use yocto or buildroot they should just start passing -D_TIME_BITS=64 and rebuild the world but general purpose, package based distributions have a more delicate dance to perform (FWIW, we don't have a fully fledged plan for Ubuntu yet but we're certainly starting to think about it).
>  
> 
>     My hunch is that the most likely outcome of switching distributions to
>     64-bit time_t is that the i386 port goes away more quickly than I would
>     expect otherwise because it makes the port rather useless for most
>     existing users (especially with those distributions that no longer
>     maintain a 32-bit-only kernel).  Personally, that wouldn't be a bad
>     outcome at all for me.  But I doubt that this is what people who push
>     for this change have in mind.
> 
> 
> Is anyone proposing that glibc will lose the ability to have the 32-bit time_t ABI? As you note that would kill the Ubuntu i386 port entirely (at some point this will probably happen anyway, people can always play their games in VMs but still).
>

Not really, there are some idea on how to proceed with 64 time_t promotion:

  1. Add a configure option to enable LFS and 64 bit time_t support as default.
     It is basically promoting the port to a new ABI, although a program can
     still build with non LFS or 32 bit time_t by explicit use -D_TIME_BITS=32.

  2. Make this option the default on a future glibc version, meaning that for 
     distribution that want to keep the the compatibility mode as default (to
     run old binaries) will need to explicit enable it.

  3. Move the non-LFS and 32 bit time_t symbols compat one, meaning that newer
     binaries won't be able to use them and -D_TIME_BITS=32 won't be supported
     or just be a no-op.

The 1 and 2 should not be cause many trouble, even if 2 would make 32-bit time_t
opt-in.  At least it would focused on glibc itself, meaning that a distribution
could just ignore the transition by just adding a configure option on glibc build.

The 3. is what might cause i386 (and any 32-bit port where ABI also supports
64-bit) to wither.  I don't have a strong opinion, maybe we should focus on
only the 1. and 2. and see what community things about 3.

Re: time64 / Large File Support: 2) default time64 breaks legacy 32bit binaries

[Florian Weimer]

* Adhemerval Zanella Netto:

> It is an option, although I am not sure what might really work since afaik
> Linux does not support CLOCK_REALTIME on time namespace.

Not yet.  The pressure isn't there yet, but I totally see it happening.

> Not really, there are some idea on how to proceed with 64 time_t promotion:
> 
>   1. Add a configure option to enable LFS and 64 bit time_t support as default.
>      It is basically promoting the port to a new ABI, although a program can
>      still build with non LFS or 32 bit time_t by explicit use -D_TIME_BITS=32.
> 
>   2. Make this option the default on a future glibc version, meaning that for 
>      distribution that want to keep the the compatibility mode as default (to
>      run old binaries) will need to explicit enable it.
> 
>   3. Move the non-LFS and 32 bit time_t symbols compat one, meaning that newer
>      binaries won't be able to use them and -D_TIME_BITS=32 won't be supported
>      or just be a no-op.
> 
> The 1 and 2 should not be cause many trouble, even if 2 would make
> 32-bit time_t opt-in.  At least it would focused on glibc itself,
> meaning that a distribution could just ignore the transition by just
> adding a configure option on glibc build.

I tend to agree, except we'd have to make it clear that 1 (if enabled)
& 2 may require extensive porting work across the distribution to
maintain a consistent ABI.

> The 3. is what might cause i386 (and any 32-bit port where ABI also supports
> 64-bit) to wither.  I don't have a strong opinion, maybe we should focus on
> only the 1. and 2. and see what community things about 3.

This still doesn't solve the implicit function declaration/dlsym
problem, though, so I don't know how

Meanwhile, Ocaml has removed compiler (but not interpreter) support for
i386, and Openblas dropped 32-bit support completely.  I think it's
going to become more and more difficult to offer a general-purpose
distribution on i386 well before we hit Y2038 problems.

Thanks,
Florian

Re: time64 / Large File Support: 2) default time64 breaks legacy 32bit binaries

[Paul Eggert]

On 1/26/23 05:21, Adhemerval Zanella Netto via Libc-alpha wrote:
> So although I am not quite against --enable-hard-sys-types64, I personally
> think we should do something more drastically (which not all other glibc
> developers agree) and flip the switch to enable 64-bit time_t *as default*
> and document 32-bit is opt-in.

Just to clarify, there are two issues here:

A. Should _FILE_OFFSET_BITS and _TIME_BITS default to 64 instead of to 32?

B. Should _FILE_OFFSET_BITS=64 and _TIME_BITS=64 be the *only* values 
supported? That is, should the old API (with 32-bit time_t, off_t, etc.) 
be available only to old 32-bit executables, and not to newly-built 
32-bit executables?

As I understood it, the proposal was that (A) and (B) both default to 
"no", but '.../configure --enable-hard-sys-types64' would change both 
(A) and (B) to "yes".

It sounds like you're proposing a more drastic approach, in which (A) 
and (B) both default to "yes", but '.../configure 
--disable-hard-sys-types64' changes them both to "no". Is that right?

Another possibility is to adopt the milder approach for glibc version N, 
and switch to the more drastic approach in glibc version N+1.This would 
give distros time to test this new configure-time option before it's the 
default.

Re: time64 / Large File Support: 2) default time64 breaks legacy 32bit binaries

[Adhemerval Zanella Netto]

On 26/01/23 23:38, Paul Eggert wrote:
> On 1/26/23 05:21, Adhemerval Zanella Netto via Libc-alpha wrote:
>> So although I am not quite against --enable-hard-sys-types64, I personally
>> think we should do something more drastically (which not all other glibc
>> developers agree) and flip the switch to enable 64-bit time_t *as default*
>> and document 32-bit is opt-in.
> 
> Just to clarify, there are two issues here:
> 
> A. Should _FILE_OFFSET_BITS and _TIME_BITS default to 64 instead of to 32?

That is my understanding to make 64-bit time_t the default ABI.

> 
> B. Should _FILE_OFFSET_BITS=64 and _TIME_BITS=64 be the *only* values supported? That is, should the old API (with 32-bit time_t, off_t, etc.) be available only to old 32-bit executables, and not to newly-built 32-bit executables?

That's a good question and I think it does make sense to move the non-LFS
and 32-bit time_t API to compat code and only provide 64-bif off_t/time_t 
support.

> 
> As I understood it, the proposal was that (A) and (B) both default to "no", but '.../configure --enable-hard-sys-types64' would change both (A) and (B) to "yes".
> 
> It sounds like you're proposing a more drastic approach, in which (A) and (B) both default to "yes", but '.../configure --disable-hard-sys-types64' changes them both to "no". Is that right?

Not really, my proposal was initial to set (A) to 'yes' but still allow new
programs to be built with -D_FILE_OFFSET_BITS=32 and -D_TIME_BITS=32.  But
making the later compat symbols seems also a good thing to cement for good
the old interfaces.

I am really trying to avoid a configure switch (such as --disable-hard-sys-types64),
they tend to be become rotten and fail due lack of testing (and also add some
fragmentation due each distro using different switch).

> 
> Another possibility is to adopt the milder approach for glibc version N, and switch to the more drastic approach in glibc version N+1.This would give distros time to test this new configure-time option before it's the default.
Do you mean add the --enable-hard-sys-types64 on glibc N, and then remove it on
N+1 and make the 64-bit the default? 

Re: time64 / Large File Support: 2) default time64 breaks legacy 32bit binaries

[Paul Eggert]

On 2023-01-27 09:40, Adhemerval Zanella Netto wrote:
>> Another possibility is to adopt the milder approach for glibc version N, and switch to the more drastic approach in glibc version N+1.This would give distros time to test this new configure-time option before it's the default.
> Do you mean add the --enable-hard-sys-types64 on glibc N, and then remove it on
> N+1 and make the 64-bit the default?

Sure, that would work. If that's too fast we could do something more 
gradual, like this:

1. glibc version N supports --enable-hard-sys-types64 and 
--disable-hard-sys-types64 and the default is disabled.

2. glibc version N+1 is the same, but the default is enabled.

3. glibc version N+2 removes support for --disable-hard-sys-types64; 
--enable-hard-sys-type64 remains as a no-op.

However, your thought of skipping step (2) would be simpler.

Re: time64 / Large File Support: 2) default time64 breaks legacy 32bit binaries

[Joseph Myers]

On Fri, 27 Jan 2023, Paul Eggert wrote:

> On 2023-01-27 09:40, Adhemerval Zanella Netto wrote:
> > > Another possibility is to adopt the milder approach for glibc version N,
> > > and switch to the more drastic approach in glibc version N+1.This would
> > > give distros time to test this new configure-time option before it's the
> > > default.
> > Do you mean add the --enable-hard-sys-types64 on glibc N, and then remove it
> > on
> > N+1 and make the 64-bit the default?
> 
> Sure, that would work. If that's too fast we could do something more gradual,
> like this:
> 
> 1. glibc version N supports --enable-hard-sys-types64 and
> --disable-hard-sys-types64 and the default is disabled.
> 
> 2. glibc version N+1 is the same, but the default is enabled.
> 
> 3. glibc version N+2 removes support for --disable-hard-sys-types64;
> --enable-hard-sys-type64 remains as a no-op.

One release is an extremely short time period for glibc (six months).  A 
few years between steps would seem better for something like this.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: time64 / Large File Support: 2) default time64 breaks legacy 32bit binaries

[Adhemerval Zanella Netto]

On 27/01/23 20:58, Joseph Myers wrote:
> On Fri, 27 Jan 2023, Paul Eggert wrote:
> 
>> On 2023-01-27 09:40, Adhemerval Zanella Netto wrote:
>>>> Another possibility is to adopt the milder approach for glibc version N,
>>>> and switch to the more drastic approach in glibc version N+1.This would
>>>> give distros time to test this new configure-time option before it's the
>>>> default.
>>> Do you mean add the --enable-hard-sys-types64 on glibc N, and then remove it
>>> on
>>> N+1 and make the 64-bit the default?
>>
>> Sure, that would work. If that's too fast we could do something more gradual,
>> like this:
>>
>> 1. glibc version N supports --enable-hard-sys-types64 and
>> --disable-hard-sys-types64 and the default is disabled.
>>
>> 2. glibc version N+1 is the same, but the default is enabled.
>>
>> 3. glibc version N+2 removes support for --disable-hard-sys-types64;
>> --enable-hard-sys-type64 remains as a no-op.
> 
> One release is an extremely short time period for glibc (six months).  A 
> few years between steps would seem better for something like this.
> 

Between starting add the initial support to override the default 64-bit and making
it the default won't be at least 2 version, so I think the layout plan it feasible.

I try to move this plan forward, I think it past time we move from non-LFS and
32 bit time as default.

Re: time64 / Large File Support: 2) default time64 breaks legacy 32bit binaries

[Florian Weimer]

* Andreas K. Huettel via Libc-alpha:

> This was discussed already in the previous thread on this list [1],
> with reactions ranging from "need new triplet" via "need new libdir"
> to "meh".  The latter is a bit surprising given how much emphasis
> glibc usually places on backwards compatibility.

> [1] https://sourceware.org/pipermail/libc-alpha/2022-November/143386.html

For me, it's ”meh” because for our distributions, we need to keep time_t
at 32 bits.  We don't want to allocate resources to make all core
libraries dual-time_t, like it was contributed to glibc.  But such a
dual-ABI nature would be a prerequisite for offering libraries that
support 64-bit time_t.  It really has to be in addition to 32-bit
time_t.

In my world, 32-bit support is for running binaries built with GCC 2.95.
(Some of them built decades ago, some of them new.)  Fedora also used to
have a 32-bit time_t use case for Wine, but I think that is rapidly
going away, with Wine transitioning to a different technology for
running 32-bit applications (the dependency is purely in the
implementation of Wine).

I expect that our ABI compatibility requirement will go away in the
second half of 2024.  Until then, it puts us in conflict with certain
64-bit time_t approaches suggested by the community.  I still hope that
we can eliminate the requirement sooner, so that I don't have to argue
against those approaches, but these kind of decisions are hard.  I
really wish we could avoid this conflict.

> While we could certainly go ahead and invent a solution in Gentoo here, this
> makes no sense at all for one distribution alone. This is an upstream problem
> and should be solved here... pretty please? opinions?

There's some risk that several key upstreams will tell us, “just rebuild
everything with time_t as 64-bit”, at which point we would have to come
up with a custom, downstream-only ABI that has dual time_t support.

And I don't think glibc is the upstream project you need to work with
here.

Thanks,
Florian