Re: [RFC 00/10] y2038: nptl: futex: Provide support for futex_time64

[Lukasz Majewski <lukma@denx.de>]

[-- Attachment #1: Type: text/plain, Size: 7034 bytes --]

Hi Adhemerval,

> On 17/07/2020 05:11, Arnd Bergmann wrote:
> > On Fri, Jul 17, 2020 at 9:27 AM Lukasz Majewski <lukma@denx.de>
> > wrote:  
> >>>
> >>> Yes, it is on my list. I am trying to handle the mess of stat
> >>> functions  
> >>
> >> Regarding stat - it is indeed a mess - many versions of the same
> >> functions for generic, LFS, and architectures (like ARM).
> >> Conversion to 64 bit time support is challenging there as well.
> >>
> >> But as fair as I can tell - more than "code base" cleanup - the
> >> raise of glibc minimal kernel supported version would help much
> >> more in this situation as code for many "use cases" would be just
> >> removed.
> >>
> >> A side remark - now the oldest LTS kernel supported is 4.4 and
> >> oldest supported kernel for glibc is 3.2.
> >> Maybe it is a good time to bump the minimal supported kernel for
> >> glibc? 
> >>> before starting on the lowlevellock-futex.h.  
> >>
> >> If you don't have time to do the conversion in next two weeks then
> >> - I can in the meantime do a conversion for the above code snippet
> >> - futex_reltimed_wait_time64() (and up to
> >> pthread_mutex_{timed,clock}). This work shall be orthogonal to
> >> lowlevellock-futex.h cleanup.
> >>
> >> What do you think?  
> > 
> > I don't think it makes any difference for stat: linux-3.2 and
> > linux-5.0 appear to have the exact same kernel interface for
> > newstat/stat64 and newfstatat/fstatat64 on all architectures,
> > though there are two special cases that need to be handled:
> > 
> > - mips-n32 and ix86-x32 are lacking the stat64 version, so they only
> >   have the non-LFS newstat/newfstatat version but not
> > stat64/fstatat64. The x32 version is wide enough to support 64-bit
> > offsets but uses the non-LFS name for it.
> > 
> > - alpha, parisc and sparc are 64-bit architecture with
> > stat64/fstatat64, but they lack newstat/newfstatat.
> > 
> > While statx() was added in in linux-4.11, it was not enabled in ia64
> > until linux-5.1 when we sanitized the syscall tables across
> > architectures, so you can't rely on that until v5.1 is the minimum
> > kernel for glibc, and that will still be a long time.
> > 
> > There may be other reasons to increase the minimum runtimer kernel
> > level or the minimum kernel header version for glibc in the
> > meantime.  
> 
> Indeed raising the kernel does not help us much here, as you pointed
> out we need to handle some outliers. My current work to simplify the
> code base and keep me sane while adding y2038 support is to:
> 
>   1. Consolidate the {f}xstat{at}{64} routines.  With some ifdef
> checks to use the right syscall the only architectures that require
> special implementations are alpha and mips64 (which support either
> specific glibc _STAT_VER version or special function conversions.
> 
>   2. Remove internal usage of the {f}xstat{at}{64} routines and
> replace with {f}stat{at}64.
> 
>   3. Move the {f}xstat{at}{64} to compat symbols and add {f}stat{at}64
>      symbol for 2.33.  My plan is also do decouple the struct stat
>      definition from the kernel and define a glibc generic one for
>      *all* architectures (we can do it because there are new symbols).
> 
>   4. Implement fstat, lstat, and fstat on fstatat basis.  This will
> simplify the y2038 support since only the all syscall selection logic
> will at fstatat.
> 
>   5. Add y2038 support by adding a new internal struct
> __stat{64}_time{64}. These will require some for symbol for non-LFS
> architectures, but with the implementation consolidation their
> implementation should be way simpler.
> 
> For the generic fstatat{64} my plan is to  adjust the current struct 
> kernel_stat to defined for *all* architectures (not only when it is
> required, such as mips).  It will decouple the glibc from kernel,
> although it will require some routines to copy out the result (which
> we are do for some architectures anyway).  In a high level what I
> have devised so far is:
> 
> ---
> * fstatat.c
> 
> /* XSTAT_IS_XSTAT64 defines whether architecture support LFS or not.
> */ #if !XSTAT_IS_XSTAT64
> 
> /* Copies the fields from the kernel struct stat KST to glibc struct
> stat ST, returning EOVERFLOW if st_ino, st_size, or st_blocks does
> not fit in the non-LFS ST.  */
> extern int __kstat_to_stat (struct kernel_stat *kst, struct stat *st);
> 
> /* This is build only for ABI with non-LFS support: arm, csky, i386,
>    hppa, m68k, mips32, microblaze, nios2, s390, sh, powerpc32, and 
>    sparc32.  */
> int
> __fxstatat (int vers, int fd, const char *file, struct stat *st, int
> flag) {
>   struct kernel_stat kst;
>   int r = INLINE_SYSCALL_CALL (fstatat64, fd, file, &kst, flag);
>   return r ?: __kstat_to_stat (&kst, st);
> }
> 
> #endif /* XSTAT_IS_XSTAT64  */
> 
> --
> * fstatat64.c
> 
> int
> __fxstatat64 (int vers, int fd, const char *file, struct stat64 *st,
> int flag) {
> #ifdef __NR_newfstatat
>   /* 64-bit kABI, e.g. aarch64, ia64, mips64*, powerpc64*, s390x,
> riscv64, and x86_64.  */
>   r = INLINE_SYSCALL_CALL (newfstatat, fd, file, &kst, flag);
> #if defined __NR_fstatat64
>   /* 64-bit kABI outlier, e.g. alpha and sparc64.  */
>   r = INLINE_SYSCALL_CALL (fstatat64, fd, file, &kst, flag)
> #else
>   struct statx tmp;
>   int r = INLINE_SYSCALL_CALL (statx, fd, file, AT_NO_AUTOMOUNT |
> flag, STATX_BASIC_STATS, &tmp);
>   if (r == 0 || errno != ENOSYS)
>     {
>       if (r == 0)
>         __cp_stat64_statx (st, &tmp);
>       return r;
>     }
>   /* Maybe I need to handle mips64-n32 here... */
>   r =  INLINE_SYSCALL_CALL (fstatat64, fd, file, &kst, flag);
> #endif
>   return r ?: __kstat_to_stat64 (&kst, st);
> }
> --
> 
> I am currently working on the 4. item on the list above and this is
> just the part to consolidate the implementations and make it sane.
> The y2038 part should be way simpler than trying to work with current
> implementation: the __fxstatat64 would be the same, but for the statx
> fallback to use fstatat64 we will need an extra convention to check
> for overflow.

It would be great to have the "syscall decision logic" in one place.

Some of my "hackish" attempts to convert *stat* [1][2][3] to support
64 bit time were using statx as it is already available and supported
in glibc.

The only issue was to convert/emulate data for stat output.


Links:

[1] -
https://github.com/lmajewski/y2038_glibc/commit/57841b674d3fc007e31e7551a66963834cf096ac
[2] -
https://github.com/lmajewski/y2038_glibc/commit/d79192c729562f5734a4fbc058e933c2a715d893
[3] -
https://github.com/lmajewski/y2038_glibc/commit/c74c19ab74eef4dcba8bd48d5306d4b38c55648e


Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]