Re: On time64 and Large File Support

["Arsen Arsenović" <arsen@gentoo.org>]

[-- Attachment #1: Type: text/plain, Size: 4743 bytes --]

Evening all,

[CC += distributions@lists.linux.dev as a thread about this popped up
 there]

Before anything else, I'd like to say that I'd prefer no distributor
acts on this issue until a smooth (or, at least, common) solution can be
established.

Bruno Haible <bruno@clisp.org> writes:

> Richard W.M. Jones wrote:
>> Another way to look at this is that it's a bug in gnutls and we should
>> fix it only in this package
>
> It's by far not just this one package. An 'fgrep -rl time_t /usr/include'
> search shows a number of libraries that use time_t in their API:
>   alsa, boost, libstdc++, glib-2.0, gtk+-3.0, libpng, nettle, openssl,
>   readline, libuuid, wxwidgets, X11, libxcb
> - and that's just the few that I happen to have installed.
>
> If a distro takes a package-by-package approach:
>   - Some of these packages use Gnulib, and are thus causing trouble now or
>     will soon.
>   - The other packages (and there are a number of them!) will either
>     require a manual switch to _TIME_BITS=64 or blow up in 2038.
>
> I agree with Daniel and Paul that a global switch to _TIME_BITS=64 + mass
> rebuild is
>   - more efficient than a package-by-package approach,
>   - also less likely to leave out some packages by mistake.

While I do too, and I believe this is the path forward for supporting
people with 32 bit hardware in y>=2038, this is *still* an ABI break,
and there is a regrettably large amount of proprietary software in the
world that users would expect to work (think games et al) which directly
depends on this ABI.  It is not as trivial as it may seem at first.  On
top of that, this still keeps the FFI problems Florian was talking
about.

I see two possibly slightly overlapping camps here:

1) AMD64 -m32 multilib users, who need this ABI for old, crusty non-Free
   binaries,
2) People running 32-bit hardware

(plus one that complains about 64 bit types consuming more memory which
I choose to ignore)

Supporting these both is a conflicting goal, (1) requires that time
remains 32 bit, and (2) does not care about that at all, and would
prefer a flag day which lets them have systems they can smoothly
continue to operate in fifteen years.  This is why I don't think
shoehorning _TIME_BITS=64 will work.

I (in personal capacity) think our best shot at supporting all of these
cases is to not opportunistically use _TIME_BITS (as it's difficult to
detect whether the rest of the system was built the same way), and
provide a hard break flag on glibc, with, potentially, a new
multilib/whathaveyou.

Let's establish a convention now, and just call the 64-bit time_t (et
al) gnuy39.  We can then use existing multilib practice we all have to
eventually migrate our i?86 systems to i?86-*-gnuy39 with an i?86-*-gnu
multilib for those that need it.  I imagine amd64 users won't ever need
i?86-*-gnuy39, but, in theory, this maps to that scenario, too.

Keep in mind, also, that we need to form a consensus on this.  Vendors
that produce the kind of software we're spending effort providing
compatibility for will just pick one or two distros to target, and so,
it's crucial that other distros follow the same conventions (including
source-based ones, because source-based needn't imply *fully* built from
source - we often need compatibility with software built for RHEL/Deb).

Enabling this requires the previously suggested ability to shift
"primary" symbols like gettimeofday to time64, rather than segregating
them to _TIME_BITS=64.  What I was thinking of above is having that be a
new $os value.  This should be easy to match.

>> In Fedora we have a
>> concept of global C/C++ flags which most C/C++ applications obey:
>>
>> $ rpm --eval '%{__global_cflags}'
>> -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe
>> -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2
>> -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
>> -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64
>> -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection
>> -fcf-protection
>>
>> We could stick -D_TIME_BITS=64 in there and then do a mass rebuild.
>
> How do you detect if a package (by mistake or intentionally) does
> '#undef _TIME_BITS'?
>
> A safer solution might be to hack the compilers (gcc and clang), so that
> they make _TIME_BITS evaluate to 64, regardless of any '#define _TIME_BITS 32'
> or '#undef _TIME_BITS' in the package's source code.

I'm not a huge fan of hacking cpp to do this, packagers could easily
check for this automatically, which is already a lot of "free"
"automated" testing.
-- 
Arsen Arsenović

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 381 bytes --]