[Bug libc/27993] New: {f}stat{at} y2038 regression in 2.33

[Bug libc/27993] New: {f}stat{at} y2038 regression in 2.33

[adrian.ratiu at collabora dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27993

            Bug ID: 27993
           Summary: {f}stat{at} y2038 regression in 2.33
           Product: glibc
           Version: 2.33
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: libc
          Assignee: unassigned at sourceware dot org
          Reporter: adrian.ratiu at collabora dot com
                CC: drepper.fsp at gmail dot com
  Target Milestone: ---
              Host: x86_64
            Target: armv7a
             Build: GCC 10.2

Created attachment 13499
  --> https://sourceware.org/bugzilla/attachment.cgi?id=13499&action=edit
lsof strace log

Hello, I'm using ChromiumOS on armv7a-cros-linux-gnueabihf with LTS kernel 4.4,
kernel-headers 4.14 and when upgrading glibc from v2.32 to v2.33, lsof starts
failing.

See attachments for stderr and strace logs of `lsof /var/log/messages`. The
lsof version is 4.94.

I've identified the problem to be caused by the following commit added in glibc
2.33 and if I revert it then lsof works as expected.

aa03f722f3 linux: Add {f}stat{at} y2038 support

Any ideas?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/27993] {f}stat{at} y2038 regression in 2.33

[adrian.ratiu at collabora dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27993

--- Comment #1 from Adrian Ratiu <adrian.ratiu at collabora dot com> ---
Created attachment 13500
  --> https://sourceware.org/bugzilla/attachment.cgi?id=13500&action=edit
lsof stderr

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/27993] {f}stat{at} y2038 regression in 2.33

[adrian.ratiu at collabora dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27993

Adrian Ratiu <adrian.ratiu at collabora dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |adhemerval.zanella at linaro dot o
                   |                            |rg,
                   |                            |adrian.ratiu at collabora dot com

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/27993] {f}stat{at} y2038 regression in 2.33

[adrian.ratiu at collabora dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27993

Adrian Ratiu <adrian.ratiu at collabora dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |lukma at denx dot de

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/27993] {f}stat{at} y2038 regression in 2.33

[adhemerval.zanella at linaro dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27993

--- Comment #2 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> ---
I am seeing the stat failure even with older glibc on a chroot system on a
ChromeOS (kernel 3.8.11):

$ uname -a
Linux localhost 3.8.11 #1 SMP Sun Jul 14 00:09:57 PDT 2019 armv7l armv7l armv7l
GNU/Linux
$ /lib/arm-linux-gnueabihf/libc.so.6 -v
GNU C Library (Ubuntu GLIBC 2.27-3ubuntu1.4) stable release version 2.27.
[...]
$ /usr/bin/lsof | head -n10
lsof: WARNING: can't stat() ext4 file system /mnt/stateful_partition
      Output information may be incomplete.
lsof: WARNING: can't stat() ext4 file system /usr/share/oem
      Output information may be incomplete.
lsof: WARNING: can't stat() ext4 file system /mnt/stateful_partition/encrypted
      Output information may be incomplete.
lsof: WARNING: can't stat() ext4 file system /home/chronos
      Output information may be incomplete.
lsof: WARNING: can't stat() tmpfs file system /run/imageloader
      Output information may be incomplete.

Also, running with glibc-2.33 the output seems similar to glibc-227 from the
system, the main difference is since I am issuing lsof thorugh the loader it
should the ld-linux- instead of 'ldof'.

I don't think the issue is related, the 'stat' command does show the exactly
same information (meaning the compat symbols __fxstat64 is returning the
expected information):

$ /usr/bin/stat /usr/bin/lsof 
  File: /usr/bin/lsof
  Size: 133292          Blocks: 264        IO Block: 4096   regular file
Device: b301h/45825d    Inode: 551747      Links: 1
Access: (0755/-rwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2021-06-17 14:16:41.000000000 -0300
Modify: 2015-10-28 14:58:56.000000000 -0200
Change: 2021-06-17 14:16:42.060000319 -0300
 Birth: -
$ ./ld-linux-armhf.so.3 --library-path . /usr/bin/stat /usr/bin/lsof 
  File: /usr/bin/lsof
  Size: 133292          Blocks: 264        IO Block: 4096   regular file
Device: b301h/45825d    Inode: 551747      Links: 1
Access: (0755/-rwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2021-06-17 14:16:41.000000000 -0300
Modify: 2015-10-28 14:58:56.000000000 -0200
Change: 2021-06-17 14:16:42.060000319 -0300
 Birth: -

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/27993] {f}stat{at} y2038 regression in 2.33

[adrian.ratiu at collabora dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27993

--- Comment #3 from Adrian Ratiu <adrian.ratiu at collabora dot com> ---
@Adhemerval Zanella

Have you tried un-doing the folowing change in glibc 2.33? Does lsof work if
you locally revert it?

aa03f722f3 linux: Add {f}stat{at} y2038 support

For me lsof works before that commit and it also works if I undo / revert it in
glibc 2.33.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/27993] {f}stat{at} y2038 regression in 2.33

[adrian.ratiu at collabora dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27993

--- Comment #4 from Adrian Ratiu <adrian.ratiu at collabora dot com> ---
@Adhemerval Zanella

Also very important: please test against glibc 2.32. Latest ChromiumOS is using
glibc 2.32.

What I am saying is this regression was introduced between 2.32 (works) and
2.33 (breaks) due to commit:

aa03f722f3 linux: Add {f}stat{at} y2038 support

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/27993] {f}stat{at} y2038 regression in 2.33

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27993

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |fweimer at redhat dot com

--- Comment #5 from Florian Weimer <fweimer at redhat dot com> ---
Is the operating system establishing some seccomp filters for the chroot? Or
does it provide direct access to the kernel?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/27993] {f}stat{at} y2038 regression in 2.33

[adrian.ratiu at collabora dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27993

--- Comment #6 from Adrian Ratiu <adrian.ratiu at collabora dot com> ---
@Florian Weimer

I am running lsof directly as root without seccomp filtering.

Just food for thought: I could not find any other package exhibiting this issue
so it might be that lsof is doing something special.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/27993] {f}stat{at} y2038 regression in 2.33

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27993

--- Comment #7 from Florian Weimer <fweimer at redhat dot com> ---
(In reply to Adrian Ratiu from comment #6)
> Just food for thought: I could not find any other package exhibiting this
> issue so it might be that lsof is doing something special.

Can you run strace with the -f option? In your trace, the interesting stuff
happens in subprocess, so the trace doesn't contain the right data.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/27993] {f}stat{at} y2038 regression in 2.33

[adrian.ratiu at collabora dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27993

Adrian Ratiu <adrian.ratiu at collabora dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #13499|0                           |1
        is obsolete|                            |

--- Comment #8 from Adrian Ratiu <adrian.ratiu at collabora dot com> ---
Created attachment 13502
  --> https://sourceware.org/bugzilla/attachment.cgi?id=13502&action=edit
strace -f lsof glibc 2.33 failure

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/27993] {f}stat{at} y2038 regression in 2.33

[adrian.ratiu at collabora dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27993

--- Comment #9 from Adrian Ratiu <adrian.ratiu at collabora dot com> ---
Created attachment 13503
  --> https://sourceware.org/bugzilla/attachment.cgi?id=13503&action=edit
strace -f lsof glibc 2.33 with revert, working correctly

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/27993] {f}stat{at} y2038 regression in 2.33

[adrian.ratiu at collabora dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27993

--- Comment #10 from Adrian Ratiu <adrian.ratiu at collabora dot com> ---
(In reply to Florian Weimer from comment #7)
> Can you run strace with the -f option? In your trace, the interesting stuff
> happens in subprocess, so the trace doesn't contain the right data.

Yes, strace -f attachments added. One is with the failure the other is with
commit aa03f722f3 ("linux: Add {f}stat{at} y2038 support") reverted which works
as expected and how it used to work before the regression.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/27993] {f}stat{at} y2038 regression in 2.33

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27993

--- Comment #11 from Florian Weimer <fweimer at redhat dot com> ---
Comment on attachment 13502
  --> https://sourceware.org/bugzilla/attachment.cgi?id=13502
strace -f lsof glibc 2.33 failure

This is a bit strange:

[pid  2725] statx(AT_FDCWD, "/", AT_STATX_SYNC_AS_STAT|AT_NO_AUTOMOUNT,
STATX_BASIC_STATS, 0xffc0ad00) = -1 ENOSYS (Function not implemented)
[pid  2725] fstatat64(AT_FDCWD, "/", {st_mode=S_IFDIR|0755, st_size=4096, ...},
0) = 0
[pid  2725] --- SIGBUS {si_signo=SIGBUS, si_code=BUS_ADRALN,
si_addr=0xffc0aed3} ---
[pid  2725] +++ killed by SIGBUS (core dumped) +++
<... read resumed>"", 4)                = 0
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_DUMPED, si_pid=2725, si_uid=0,
si_status=SIGBUS, si_utime=0, si_stime=1} ---

Somewhere there should be a coredump, and hopefully you can get a backtrace
from it and see where it crashes. But it looks to me like the code in lsof is
invalid because the unmarshalling code in doinchild does not ensure that the
data in r_rbuf is properly aligned.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/27993] {f}stat{at} y2038 regression in 2.33

[adrian.ratiu at collabora dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27993

--- Comment #12 from Adrian Ratiu <adrian.ratiu at collabora dot com> ---
@Florian Weimer I think you are correct. Below is a stack trace.

So this means there is no bug in glibc and the breakage is to be expected?

In that case I'll have to report or attempt a fix in lsof itself and any other
package which might fail due to the new codepath.

#0  0xf396e1aa in __cp_stat64_t64_stat64 (st64_t64=0xffa4bc78, st64=<optimized
out>)
    at ../sysdeps/unix/sysv/linux/stat_t64_cp.c:40
#1  0xf39671a8 in __GI___stat64 (file=<optimized out>, buf=0xffa4bd2b)
    at ../sysdeps/unix/sysv/linux/stat64.c:38
#2  0x066cbf26 in doinchild (fn=<optimized out>, fp=<optimized out>,
rbuf=<optimized out>, 
    rbln=<optimized out>) at misc.c:364
#3  0x066cc09e in Readlink (arg=0x6b59378 "/proc") at misc.c:1109
#4  0x066bbe52 in readmnt () at dmnt.c:512
#5  0x066c6454 in ck_file_arg (i=1, ac=2, av=0xffa52224, fv=0, rs=0, sbp=0x0)
at arg.c:189
#6  0x066ca506 in main (argc=<optimized out>, argv=<optimized out>) at
main.c:1251

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/27993] {f}stat{at} y2038 regression in 2.33

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27993

--- Comment #13 from Florian Weimer <fweimer at redhat dot com> ---
(In reply to Adrian Ratiu from comment #12)
> @Florian Weimer I think you are correct. Below is a stack trace.

Very illuminating, thanks.

> So this means there is no bug in glibc and the breakage is to be expected?

I wouldn't say the breakage is expected, but rather that the application is
buggy.

> In that case I'll have to report or attempt a fix in lsof itself and any
> other package which might fail due to the new codepath.
> 
> #0  0xf396e1aa in __cp_stat64_t64_stat64 (st64_t64=0xffa4bc78,
> st64=<optimized out>)
>     at ../sysdeps/unix/sysv/linux/stat_t64_cp.c:40
> #1  0xf39671a8 in __GI___stat64 (file=<optimized out>, buf=0xffa4bd2b)
>     at ../sysdeps/unix/sysv/linux/stat64.c:38

If that buf pointer is correct, it's an odd address.  That clearly does not
satisfy the struct stat64 alignment requirements. Back when the kernel
performed the copy that didn't matter (because the copy over to userspace is a
bit like memcpy), but glibc assumes that incoming typed object pointers are
valid and (in particular) meet the alignment requirements of the actual data
pointers.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/27993] {f}stat{at} y2038 regression in 2.33

[adrian.ratiu at collabora dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27993

Adrian Ratiu <adrian.ratiu at collabora dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |NOTABUG
             Status|UNCONFIRMED                 |RESOLVED

--- Comment #14 from Adrian Ratiu <adrian.ratiu at collabora dot com> ---
Thanks for helping clarify the issue. Closing as not a bug.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/27993] {f}stat{at} y2038 regression in 2.33

[adrian.ratiu at collabora dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27993

--- Comment #15 from Adrian Ratiu <adrian.ratiu at collabora dot com> ---
This is just a comment for posterity in case other packages might stumble over
similar bugs.

lsof upstream commited a fix for the SIGBUS alignment crash:
https://github.com/masatake/lsof/commit/21cb1dad1243f4c0a427d893babab12e48b60f0e

Reported issue:
https://github.com/lsof-org/lsof/issues/160

-- 
You are receiving this mail because:
You are on the CC list for the bug.