[RFC v3 00/23] RISC-V glibc port for the 32-bit

[RFC v3 00/23] RISC-V glibc port for the 32-bit

[Alistair Francis]

This patch set contains the glibc port for the 32-bit RISC-V.

This is based on all of the work that Zong Li has done [1].

Unfortunately the Linux ABI has changed since Zong Li's latest
submission. The RISC-V 32-bit (RV32) ABI no longer defines
__ARCH_WANT_TIME32_SYSCALLS which means there are no 32-bit versions of
time_t, off_t or any struct resource or system calls that use them.

To mitigate this I have set the RV32 port to use 64-bit time_t and off_t
(as is done in x86-32) and also changed the syscall imvocation to handle
the missing syscalls.

This series applies ontop of Lukasz's __clock_settime64 iomplementation [2].

The ChangeLog will need to be updated, so don't worry too much
about that.

All testing has been done on a 5.1 kernel (it won't work on earlier
kernels).

Importantly glibc is not the only project that needs to be updated to
work with a 32-bit kernel that doesn't define
__ARCH_WANT_TIME32_SYSCALLS. So far OpenSSL, busybox and systemd all
need patches to work with RV32. My hope here is that we can get a
reasonable agreement on how to handle this in glibc and then use that to
update other projects. This is why I would still really like feedback
even though this won't go into the next release.

Feedback on this series is very welcome!

1: https://sourceware.org/ml/libc-alpha/2018-07/msg00892.html
2: https://sourceware.org/ml/libc-alpha/2019-05/msg00661.html

The latest version of my work can be found here: https://github.com/alistair23/glibc/tree/alistair/rv32.next

This specific version can be found here: https://github.com/alistair23/glibc/tree/alistair/rv32.rfc3

RFC v3:
 - Remove all "Hack" patches
 - Incorporate upstream comments
 - Ensure we don't break RV64
 - Lot's more testing and fixes
RFC v2:
 - Add Lukasz's patches
 - Update the non HACK syscalls after feedback
 - define __ASSUME_TIME64_SYSCALLS and __ASSUME_RLIM64_SYSCALLS
 - Remove lockf64.c
 - Other smaller changes from RFC v1

--END---

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 __COVER__ | 0
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 __COVER__

diff --git a/__COVER__ b/__COVER__
new file mode 100644
index 0000000000..e69de29bb2
-- 
2.22.0



Alistair Francis (13):
  sysdeps/nanosleep: Use clock_nanosleep_time64 if avaliable
  sysdeps/gettimeofday: Use clock_gettime64 if avaliable
  sysdeps/wait: Use waitid if avaliable
  sysdeps/clock_gettime: Use clock_gettime64 if avaliable
  sysdeps/timespec_get: Use clock_gettime64 if avaliable
  RISC-V: Use 64-bit time_t and off_t for RV32 and RV64
  RISC-V: define __NR_futex as __NR_futex_time64 for 32-bit
  RISC-V: define __NR_* as __NR_*_time64/64 for 32-bit
  RISC-V: define __NR_clock_getres as __NR_*_time64 for 32-bit
  RISC-V: define __vdso_clock_getres as __vdso_clock_getres_time64 for
    32-bit
  RISC-V: define __vdso_clock_gettime as __vdso_clock_gettime64 for
    32-bit
  RISC-V: Use 64-bit timespec in clock_gettime vdso calls
  RISC-V: Use 64-bit vdso syscalls

Zong Li (10):
  Documentation for the RISC-V 32-bit port
  RISC-V: Support dynamic loader for the 32-bit
  RISC-V: Add path of library directories for the 32-bit
  RISC-V: The ABI implementation for the 32-bit
  RISC-V: Hard float support for the 32 bit
  RISC-V: Regenerate ULPs of RISC-V
  RISC-V: Add ABI lists
  RISC-V: Build Infastructure for the 32-bit
  RISC-V: Fix llrint and llround missing exceptions on RV32
  Add RISC-V 32-bit target to build-many-glibcs.py

 ChangeLog                                     |  113 +
 NEWS                                          |    6 +
 README                                        |    1 +
 nptl/pthread_mutex_timedlock.c                |    7 +
 nptl/thrd_sleep.c                             |   41 +-
 scripts/build-many-glibcs.py                  |   15 +
 sysdeps/riscv/bits/wordsize.h                 |    4 +-
 sysdeps/riscv/nofpu/libm-test-ulps            |   16 +-
 sysdeps/riscv/nptl/bits/pthreadtypes-arch.h   |   25 +-
 sysdeps/riscv/preconfigure                    |    6 +-
 sysdeps/riscv/rv32/Implies-after              |    1 +
 .../riscv/rv32/fix-fp-int-convert-overflow.h  |   38 +
 sysdeps/riscv/rv32/rvd/Implies                |    3 +
 sysdeps/riscv/rv32/rvd/s_lrint.c              |   31 +
 sysdeps/riscv/rv32/rvd/s_lround.c             |   31 +
 sysdeps/riscv/rv32/rvf/Implies                |    1 +
 sysdeps/riscv/rv32/rvf/s_lrintf.c             |   31 +
 sysdeps/riscv/rv32/rvf/s_lroundf.c            |   31 +
 sysdeps/riscv/{rv64 => }/rvd/libm-test-ulps   |   56 +-
 .../riscv/{rv64 => }/rvd/libm-test-ulps-name  |    0
 sysdeps/riscv/sfp-machine.h                   |   27 +-
 sysdeps/riscv/sys/asm.h                       |    5 +-
 sysdeps/unix/sysv/linux/clock_gettime.c       |   30 +
 sysdeps/unix/sysv/linux/clock_nanosleep.c     |   42 +-
 sysdeps/unix/sysv/linux/gettimeofday.c        |   28 +
 sysdeps/unix/sysv/linux/nanosleep.c           |   50 +
 sysdeps/unix/sysv/linux/nanosleep_nocancel.c  |   11 +
 sysdeps/unix/sysv/linux/riscv/Makefile        |    4 +-
 .../unix/sysv/linux/riscv/bits/environments.h |   85 +
 sysdeps/unix/sysv/linux/riscv/bits/time64.h   |   36 +
 sysdeps/unix/sysv/linux/riscv/bits/timesize.h |   22 +
 .../unix/sysv/linux/riscv/bits/typesizes.h    |   84 +
 sysdeps/unix/sysv/linux/riscv/configure       |   39 +
 sysdeps/unix/sysv/linux/riscv/configure.ac    |    8 +
 sysdeps/unix/sysv/linux/riscv/dl-cache.h      |   17 +-
 sysdeps/unix/sysv/linux/riscv/init-first.c    |    6 +-
 sysdeps/unix/sysv/linux/riscv/ldconfig.h      |    2 +-
 sysdeps/unix/sysv/linux/riscv/libc-vdso.h     |    2 +-
 sysdeps/unix/sysv/linux/riscv/rv32/Implies    |    3 +
 .../unix/sysv/linux/riscv/rv32/c++-types.data |   67 +
 .../sysv/linux/riscv/rv32/jmp_buf-macros.h    |   53 +
 sysdeps/unix/sysv/linux/riscv/rv32/ld.abilist |    9 +
 .../linux/riscv/rv32/libBrokenLocale.abilist  |    1 +
 .../unix/sysv/linux/riscv/rv32/libanl.abilist |    4 +
 .../unix/sysv/linux/riscv/rv32/libc.abilist   | 2101 +++++++++++++++++
 .../sysv/linux/riscv/rv32/libcrypt.abilist    |    2 +
 .../unix/sysv/linux/riscv/rv32/libdl.abilist  |    9 +
 .../unix/sysv/linux/riscv/rv32/libm.abilist   | 1021 ++++++++
 .../sysv/linux/riscv/rv32/libpthread.abilist  |  235 ++
 .../sysv/linux/riscv/rv32/libresolv.abilist   |   79 +
 .../unix/sysv/linux/riscv/rv32/librt.abilist  |   35 +
 .../linux/riscv/rv32/libthread_db.abilist     |   40 +
 .../sysv/linux/riscv/rv32/libutil.abilist     |    6 +
 sysdeps/unix/sysv/linux/riscv/shlib-versions  |   10 +-
 sysdeps/unix/sysv/linux/riscv/sysdep.h        |   61 +
 sysdeps/unix/sysv/linux/timespec_get.c        |   37 +-
 sysdeps/unix/sysv/linux/wait.c                |   39 +-
 sysdeps/unix/sysv/linux/waitpid.c             |   46 +
 sysdeps/unix/sysv/linux/waitpid_nocancel.c    |   45 +
 59 files changed, 4796 insertions(+), 62 deletions(-)
 create mode 100644 sysdeps/riscv/rv32/Implies-after
 create mode 100644 sysdeps/riscv/rv32/fix-fp-int-convert-overflow.h
 create mode 100644 sysdeps/riscv/rv32/rvd/Implies
 create mode 100644 sysdeps/riscv/rv32/rvd/s_lrint.c
 create mode 100644 sysdeps/riscv/rv32/rvd/s_lround.c
 create mode 100644 sysdeps/riscv/rv32/rvf/Implies
 create mode 100644 sysdeps/riscv/rv32/rvf/s_lrintf.c
 create mode 100644 sysdeps/riscv/rv32/rvf/s_lroundf.c
 rename sysdeps/riscv/{rv64 => }/rvd/libm-test-ulps (98%)
 rename sysdeps/riscv/{rv64 => }/rvd/libm-test-ulps-name (100%)
 create mode 100644 sysdeps/unix/sysv/linux/riscv/bits/environments.h
 create mode 100644 sysdeps/unix/sysv/linux/riscv/bits/time64.h
 create mode 100644 sysdeps/unix/sysv/linux/riscv/bits/timesize.h
 create mode 100644 sysdeps/unix/sysv/linux/riscv/bits/typesizes.h
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/Implies
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/c++-types.data
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/jmp_buf-macros.h
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/ld.abilist
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libBrokenLocale.abilist
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libanl.abilist
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libc.abilist
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libcrypt.abilist
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libdl.abilist
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libm.abilist
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libpthread.abilist
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libresolv.abilist
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/librt.abilist
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libthread_db.abilist
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libutil.abilist

-- 
2.22.0

[RFC v3 08/23] RISC-V: define __NR_futex as __NR_futex_time64 for 32-bit

[Alistair Francis]

The futex syscall isn't avaliable on R32 so let's define it to
be futex_time64 instead.

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 sysdeps/unix/sysv/linux/riscv/sysdep.h | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/sysdeps/unix/sysv/linux/riscv/sysdep.h b/sysdeps/unix/sysv/linux/riscv/sysdep.h
index 5470ea3d2a..b7607ef9f1 100644
--- a/sysdeps/unix/sysv/linux/riscv/sysdep.h
+++ b/sysdeps/unix/sysv/linux/riscv/sysdep.h
@@ -116,6 +116,15 @@
 
 #include <sysdeps/unix/sysdep.h>
 
+#if __riscv_xlen == 32
+/* Define the __NR_futex as __NR_futex64 as RV32 doesn't have a
+ * __NR_futex syscall.
+ */
+# ifndef __NR_futex
+#  define __NR_futex __NR_futex_time64
+# endif
+#endif
+
 #undef SYS_ify
 #define SYS_ify(syscall_name)	__NR_##syscall_name
 
-- 
2.22.0

[RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Alistair Francis]

If the waitid syscall is avaliable let's use that as waitpid
and wait4 aren't always avaliable (they aren't avaliable on RV32).

Unfortunately waitid is substantially differnt to waitpid and wait4, so
the conversion ends up being complex.

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 ChangeLog                                  |  3 ++
 sysdeps/unix/sysv/linux/wait.c             | 39 ++++++++++++++++--
 sysdeps/unix/sysv/linux/waitpid.c          | 46 ++++++++++++++++++++++
 sysdeps/unix/sysv/linux/waitpid_nocancel.c | 45 +++++++++++++++++++++
 4 files changed, 130 insertions(+), 3 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index 9ca390a9c3..fb54a4aa61 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1029,6 +1029,9 @@
 	* sysdeps/unix/sysv/linux/nanosleep.c: Likewise.
 	* sysdeps/unix/sysv/linux/nanosleep_nocancel.c: Likewise.
 	* sysdeps/unix/sysv/linux/gettimeofday.c: Use clock_gettime64 syscall for gettimeofday.
+	* sysdeps/unix/sysv/linux/wait.c: Use __NR_waitid if avaliable.
+	* sysdeps/unix/sysv/linux/waitpid.c: Likewise.
+	* sysdeps/unix/sysv/linux/waitpid_nocancel.c: Likewise.
 
 2019-06-20  Dmitry V. Levin  <ldv@altlinux.org>
 	    Florian Weimer  <fweimer@redhat.com>
diff --git a/sysdeps/unix/sysv/linux/wait.c b/sysdeps/unix/sysv/linux/wait.c
index 498bd1c095..e727afa14b 100644
--- a/sysdeps/unix/sysv/linux/wait.c
+++ b/sysdeps/unix/sysv/linux/wait.c
@@ -26,9 +26,42 @@
 pid_t
 __libc_wait (int *stat_loc)
 {
-  pid_t result = SYSCALL_CANCEL (wait4, WAIT_ANY, stat_loc, 0,
-				 (struct rusage *) NULL);
-  return result;
+#ifdef __NR_waitid
+  siginfo_t infop;
+  __pid_t ret;
+
+  ret = SYSCALL_CANCEL (waitid, P_ALL, 0, &infop, WEXITED, NULL);
+
+  if (ret < 0) {
+    return ret;
+  }
+
+  if (stat_loc) {
+    *stat_loc = 0;
+    switch (infop.si_code) {
+    case CLD_EXITED:
+      *stat_loc = infop.si_status << 8;
+      break;
+    case CLD_DUMPED:
+      *stat_loc = 0x80;
+    case CLD_KILLED:
+      *stat_loc |= infop.si_status;
+      break;
+    case CLD_TRAPPED:
+    case CLD_STOPPED:
+      *stat_loc = infop.si_status << 8 | 0x7f;
+      break;
+    case CLD_CONTINUED:
+      *stat_loc = 0xffff;
+      break;
+    }
+  }
+
+  return infop.si_pid;
+#else
+  return SYSCALL_CANCEL (wait4, WAIT_ANY, stat_loc, 0,
+                         (struct rusage *) NULL);
+#endif
 }
 
 weak_alias (__libc_wait, __wait)
diff --git a/sysdeps/unix/sysv/linux/waitpid.c b/sysdeps/unix/sysv/linux/waitpid.c
index f0897574c0..7d4e0bb77d 100644
--- a/sysdeps/unix/sysv/linux/waitpid.c
+++ b/sysdeps/unix/sysv/linux/waitpid.c
@@ -20,12 +20,58 @@
 #include <sysdep-cancel.h>
 #include <stdlib.h>
 #include <sys/wait.h>
+#include <unistd.h>
 
 __pid_t
 __waitpid (__pid_t pid, int *stat_loc, int options)
 {
 #ifdef __NR_waitpid
   return SYSCALL_CANCEL (waitpid, pid, stat_loc, options);
+#elif defined(__NR_waitid)
+  __pid_t ret;
+  idtype_t idtype = P_PID;
+  siginfo_t infop;
+
+  if (pid < -1) {
+    idtype = P_PGID;
+    pid *= -1;
+  } else if (pid == -1) {
+    idtype = P_ALL;
+  } else if (pid == 0) {
+    idtype = P_PGID;
+    pid = getpgrp();
+  }
+
+  options |= WEXITED;
+
+  ret = SYSCALL_CANCEL (waitid, idtype, pid, &infop, options, NULL);
+
+  if (ret < 0) {
+    return ret;
+  }
+
+  if (stat_loc) {
+    *stat_loc = 0;
+    switch (infop.si_code) {
+    case CLD_EXITED:
+      *stat_loc = infop.si_status << 8;
+      break;
+    case CLD_DUMPED:
+      *stat_loc = 0x80;
+    case CLD_KILLED:
+      *stat_loc |= infop.si_status;
+      break;
+    case CLD_TRAPPED:
+    case CLD_STOPPED:
+      *stat_loc = infop.si_status << 8 | 0x7f;
+      break;
+    case CLD_CONTINUED:
+      *stat_loc = 0xffff;
+      break;
+    }
+  }
+
+  return infop.si_pid;
 #else
   return SYSCALL_CANCEL (wait4, pid, stat_loc, options, NULL);
 #endif
diff --git a/sysdeps/unix/sysv/linux/waitpid_nocancel.c b/sysdeps/unix/sysv/linux/waitpid_nocancel.c
index 89e36a5c0b..c92f05421b 100644
--- a/sysdeps/unix/sysv/linux/waitpid_nocancel.c
+++ b/sysdeps/unix/sysv/linux/waitpid_nocancel.c
@@ -27,6 +27,51 @@ __waitpid_nocancel (__pid_t pid, int *stat_loc, int options)
 {
 #ifdef __NR_waitpid
   return INLINE_SYSCALL_CALL (waitpid, pid, stat_loc, options);
+#elif defined(__NR_waitid)
+  __pid_t ret;
+  idtype_t idtype = P_PID;
+  siginfo_t infop;
+
+  if (pid < -1) {
+    idtype = P_PGID;
+    pid *= -1;
+  } else if (pid == -1) {
+    idtype = P_ALL;
+  } else if (pid == 0) {
+    idtype = P_PGID;
+    pid = getpgrp();
+  }
+
+  options |= WEXITED;
+
+  ret = INLINE_SYSCALL_CALL (waitid, idtype, pid, &infop, options, NULL);
+
+  if (ret < 0) {
+    return ret;
+  }
+
+  if (stat_loc) {
+    *stat_loc = 0;
+    switch (infop.si_code) {
+    case CLD_EXITED:
+      *stat_loc = infop.si_status << 8;
+      break;
+    case CLD_DUMPED:
+      *stat_loc = 0x80;
+    case CLD_KILLED:
+      *stat_loc |= infop.si_status;
+      break;
+    case CLD_TRAPPED:
+    case CLD_STOPPED:
+      *stat_loc = infop.si_status << 8 | 0x7f;
+      break;
+    case CLD_CONTINUED:
+      *stat_loc = 0xffff;
+      break;
+    }
+  }
+
+  return infop.si_pid;
 #else
   return INLINE_SYSCALL_CALL (wait4, pid, stat_loc, options, NULL);
 #endif
-- 
2.22.0

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Florian Weimer]

* Alistair Francis:

> +#ifdef __NR_waitid

I wonder if the condition should be

  #ifndef __NR_wait4

etc.  That would make it less risky for the existing architectures.

> +    case CLD_DUMPED:
> +      *stat_loc = 0x80;
> +    case CLD_KILLED:
> +      *stat_loc |= infop.si_status;
> +      break;

The kernel does this (in kernel/exit.c):

        status = (p->signal->flags & SIGNAL_GROUP_EXIT)
                ? p->signal->group_exit_code : p->exit_code;
        wo->wo_stat = status;
…
        if (infop) {
                if ((status & 0x7f) == 0) {
                        infop->cause = CLD_EXITED;
                        infop->status = status >> 8;
                } else {
                        infop->cause = (status & 0x80) ? CLD_DUMPED : CLD_KILLED;
                        infop->status = status & 0x7f;
                }
                infop->pid = pid;
                infop->uid = uid;
        }

Therefore, I wonder if you need to propagate the lower bits from
si_status for CLD_DUMPED, too.

For wait/wait3/waitpid, you could probably use the implementations for
sysdeps/unix/bsd, layered on top of wait4.

The patch also needs some formatting tweaks regarding the placement of
braces, and there are missing spaces before parentheses.

Thanks,
Florian

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Alistair Francis]

On Tue, Jul 16, 2019 at 10:31 PM Florian Weimer <fweimer@redhat.com> wrote:
>
> * Alistair Francis:
>
> > +#ifdef __NR_waitid
>
> I wonder if the condition should be
>
>   #ifndef __NR_wait4
>
> etc.  That would make it less risky for the existing architectures.

Seems fair, I have updated that.

>
> > +    case CLD_DUMPED:
> > +      *stat_loc = 0x80;
> > +    case CLD_KILLED:
> > +      *stat_loc |= infop.si_status;
> > +      break;
>
> The kernel does this (in kernel/exit.c):
>
>         status = (p->signal->flags & SIGNAL_GROUP_EXIT)
>                 ? p->signal->group_exit_code : p->exit_code;
>         wo->wo_stat = status;
> …
>         if (infop) {
>                 if ((status & 0x7f) == 0) {
>                         infop->cause = CLD_EXITED;
>                         infop->status = status >> 8;
>                 } else {
>                         infop->cause = (status & 0x80) ? CLD_DUMPED : CLD_KILLED;
>                         infop->status = status & 0x7f;
>                 }
>                 infop->pid = pid;
>                 infop->uid = uid;
>         }
>
> Therefore, I wonder if you need to propagate the lower bits from
> si_status for CLD_DUMPED, too.

We are propagating the bits due to a fall through in the switch
statement, I have added a comment to indicate that we are falling
through.

>
> For wait/wait3/waitpid, you could probably use the implementations for
> sysdeps/unix/bsd, layered on top of wait4.

I'm not sure what you mean here.

>
> The patch also needs some formatting tweaks regarding the placement of
> braces, and there are missing spaces before parentheses.

Yep, I have tried to fixup all the formatting issues.

Alistair

>
> Thanks,
> Florian

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Florian Weimer]

* Alistair Francis:

>> For wait/wait3/waitpid, you could probably use the implementations for
>> sysdeps/unix/bsd, layered on top of wait4.
>
> I'm not sure what you mean here.

Once you have wait4 (modulo the discussion about the process group), you
can use

#include <sysdeps/unix/bsd/waitpid.c>

to use the BSD layered implementation.  Also see
sysdeps/unix/sysv/linux/wait3.c.

Thanks,
Florian

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Alistair Francis]

On Mon, Jul 22, 2019 at 8:58 AM Florian Weimer <fweimer@redhat.com> wrote:
>
> * Alistair Francis:
>
> >> For wait/wait3/waitpid, you could probably use the implementations for
> >> sysdeps/unix/bsd, layered on top of wait4.
> >
> > I'm not sure what you mean here.
>
> Once you have wait4 (modulo the discussion about the process group), you
> can use
>
> #include <sysdeps/unix/bsd/waitpid.c>
>
> to use the BSD layered implementation.  Also see
> sysdeps/unix/sysv/linux/wait3.c.

Yep, if we get a wait4 implementation I can do that. Although if there
is a wait4 implementation for RV32 I can drop this patch altogether as
it isn't required if we have a wait4 syscall.

Alistair

>
> Thanks,
> Florian

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Rich Felker]

On Tue, Jul 16, 2019 at 05:08:48PM -0700, Alistair Francis wrote:
> If the waitid syscall is avaliable let's use that as waitpid
> and wait4 aren't always avaliable (they aren't avaliable on RV32).
> 
> Unfortunately waitid is substantially differnt to waitpid and wait4, so
> the conversion ends up being complex.
> 
> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
> ---
>  ChangeLog                                  |  3 ++
>  sysdeps/unix/sysv/linux/wait.c             | 39 ++++++++++++++++--
>  sysdeps/unix/sysv/linux/waitpid.c          | 46 ++++++++++++++++++++++
>  sysdeps/unix/sysv/linux/waitpid_nocancel.c | 45 +++++++++++++++++++++
>  4 files changed, 130 insertions(+), 3 deletions(-)
> [...]
>  
>  weak_alias (__libc_wait, __wait)
> diff --git a/sysdeps/unix/sysv/linux/waitpid.c b/sysdeps/unix/sysv/linux/waitpid.c
> index f0897574c0..7d4e0bb77d 100644
> --- a/sysdeps/unix/sysv/linux/waitpid.c
> +++ b/sysdeps/unix/sysv/linux/waitpid.c
> @@ -20,12 +20,58 @@
>  #include <sysdep-cancel.h>
>  #include <stdlib.h>
>  #include <sys/wait.h>
> +#include <unistd.h>
>  
>  __pid_t
>  __waitpid (__pid_t pid, int *stat_loc, int options)
>  {
>  #ifdef __NR_waitpid
>    return SYSCALL_CANCEL (waitpid, pid, stat_loc, options);
> +#elif defined(__NR_waitid)
> +  __pid_t ret;
> +  idtype_t idtype = P_PID;
> +  siginfo_t infop;
> +
> +  if (pid < -1) {
> +    idtype = P_PGID;
> +    pid *= -1;
> +  } else if (pid == -1) {
> +    idtype = P_ALL;
> +  } else if (pid == 0) {
> +    idtype = P_PGID;
> +    pid = getpgrp();
> +  }
> +
> +  options |= WEXITED;
> +
> +  ret = SYSCALL_CANCEL (waitid, idtype, pid, &infop, options, NULL);

This emulation has a fundamental race condition. Between getpgrp and
waitid, a signal handler may perform setpgrp, setsid, and/or fork in
ways that cause the wrong pgid to be passed to the waitid syscall.
There is no way around this because you cannot block signals for the
interval, since signals must be able to interrupt the waitid syscall.

Unless there's some trick I'm missing here, the kernel folks' removal
of the wait4 syscall is just a bug in the kernel that they need to
fix. It also makes it impossible to implement the wait4 function,
since there's no way to get rusage for the exited process.

Rich

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Rich Felker]

On Sun, Jul 21, 2019 at 12:03:10AM -0400, Rich Felker wrote:
> On Tue, Jul 16, 2019 at 05:08:48PM -0700, Alistair Francis wrote:
> > If the waitid syscall is avaliable let's use that as waitpid
> > and wait4 aren't always avaliable (they aren't avaliable on RV32).
> > 
> > Unfortunately waitid is substantially differnt to waitpid and wait4, so
> > the conversion ends up being complex.
> > 
> > Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
> > ---
> >  ChangeLog                                  |  3 ++
> >  sysdeps/unix/sysv/linux/wait.c             | 39 ++++++++++++++++--
> >  sysdeps/unix/sysv/linux/waitpid.c          | 46 ++++++++++++++++++++++
> >  sysdeps/unix/sysv/linux/waitpid_nocancel.c | 45 +++++++++++++++++++++
> >  4 files changed, 130 insertions(+), 3 deletions(-)
> > [...]
> >  
> >  weak_alias (__libc_wait, __wait)
> > diff --git a/sysdeps/unix/sysv/linux/waitpid.c b/sysdeps/unix/sysv/linux/waitpid.c
> > index f0897574c0..7d4e0bb77d 100644
> > --- a/sysdeps/unix/sysv/linux/waitpid.c
> > +++ b/sysdeps/unix/sysv/linux/waitpid.c
> > @@ -20,12 +20,58 @@
> >  #include <sysdep-cancel.h>
> >  #include <stdlib.h>
> >  #include <sys/wait.h>
> > +#include <unistd.h>
> >  
> >  __pid_t
> >  __waitpid (__pid_t pid, int *stat_loc, int options)
> >  {
> >  #ifdef __NR_waitpid
> >    return SYSCALL_CANCEL (waitpid, pid, stat_loc, options);
> > +#elif defined(__NR_waitid)
> > +  __pid_t ret;
> > +  idtype_t idtype = P_PID;
> > +  siginfo_t infop;
> > +
> > +  if (pid < -1) {
> > +    idtype = P_PGID;
> > +    pid *= -1;
> > +  } else if (pid == -1) {
> > +    idtype = P_ALL;
> > +  } else if (pid == 0) {
> > +    idtype = P_PGID;
> > +    pid = getpgrp();
> > +  }
> > +
> > +  options |= WEXITED;
> > +
> > +  ret = SYSCALL_CANCEL (waitid, idtype, pid, &infop, options, NULL);
> 
> This emulation has a fundamental race condition. Between getpgrp and
> waitid, a signal handler may perform setpgrp, setsid, and/or fork in
> ways that cause the wrong pgid to be passed to the waitid syscall.
> There is no way around this because you cannot block signals for the
> interval, since signals must be able to interrupt the waitid syscall.
> 
> Unless there's some trick I'm missing here, the kernel folks' removal
> of the wait4 syscall is just a bug in the kernel that they need to
> fix. It also makes it impossible to implement the wait4 function,
> since there's no way to get rusage for the exited process.

Reportedly (via Stefan O'Rear just now) there was a similar kernel bug
introduced in 161550d74c07303ffa6187ba776f62df5a906a21 that makes
wait4 fail to honor pgrp changes that happen while already in the
syscall (e.g. performed on the caller by another thread or even
another process). But the race condition here in userspace is even
more egregious I think, since it violates the contract in a case where
there is a clear observable order between the pgrp change and the
blocking wait -- for instance, the signal handler could change pgrp
of itself and a child process, and then whether or not the signal
handler had executed before the waitpid, the waitpid should catch the
child's exit. But with the above race, it fails to.

Rich

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Eric W. Biederman]

Rich Felker <dalias@libc.org> writes:

> On Sun, Jul 21, 2019 at 12:03:10AM -0400, Rich Felker wrote:
>> On Tue, Jul 16, 2019 at 05:08:48PM -0700, Alistair Francis wrote:
>> > If the waitid syscall is avaliable let's use that as waitpid
>> > and wait4 aren't always avaliable (they aren't avaliable on RV32).
>> > 
>> > Unfortunately waitid is substantially differnt to waitpid and wait4, so
>> > the conversion ends up being complex.
>> > 
>> > Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
>> > ---
>> >  ChangeLog                                  |  3 ++
>> >  sysdeps/unix/sysv/linux/wait.c             | 39 ++++++++++++++++--
>> >  sysdeps/unix/sysv/linux/waitpid.c          | 46 ++++++++++++++++++++++
>> >  sysdeps/unix/sysv/linux/waitpid_nocancel.c | 45 +++++++++++++++++++++
>> >  4 files changed, 130 insertions(+), 3 deletions(-)
>> > [...]
>> >  
>> >  weak_alias (__libc_wait, __wait)
>> > diff --git a/sysdeps/unix/sysv/linux/waitpid.c b/sysdeps/unix/sysv/linux/waitpid.c
>> > index f0897574c0..7d4e0bb77d 100644
>> > --- a/sysdeps/unix/sysv/linux/waitpid.c
>> > +++ b/sysdeps/unix/sysv/linux/waitpid.c
>> > @@ -20,12 +20,58 @@
>> >  #include <sysdep-cancel.h>
>> >  #include <stdlib.h>
>> >  #include <sys/wait.h>
>> > +#include <unistd.h>
>> >  
>> >  __pid_t
>> >  __waitpid (__pid_t pid, int *stat_loc, int options)
>> >  {
>> >  #ifdef __NR_waitpid
>> >    return SYSCALL_CANCEL (waitpid, pid, stat_loc, options);
>> > +#elif defined(__NR_waitid)
>> > +  __pid_t ret;
>> > +  idtype_t idtype = P_PID;
>> > +  siginfo_t infop;
>> > +
>> > +  if (pid < -1) {
>> > +    idtype = P_PGID;
>> > +    pid *= -1;
>> > +  } else if (pid == -1) {
>> > +    idtype = P_ALL;
>> > +  } else if (pid == 0) {
>> > +    idtype = P_PGID;
>> > +    pid = getpgrp();
>> > +  }
>> > +
>> > +  options |= WEXITED;
>> > +
>> > +  ret = SYSCALL_CANCEL (waitid, idtype, pid, &infop, options, NULL);
>> 
>> This emulation has a fundamental race condition. Between getpgrp and
>> waitid, a signal handler may perform setpgrp, setsid, and/or fork in
>> ways that cause the wrong pgid to be passed to the waitid syscall.
>> There is no way around this because you cannot block signals for the
>> interval, since signals must be able to interrupt the waitid syscall.
>> 
>> Unless there's some trick I'm missing here, the kernel folks' removal
>> of the wait4 syscall is just a bug in the kernel that they need to
>> fix. It also makes it impossible to implement the wait4 function,
>> since there's no way to get rusage for the exited process.
>
> Reportedly (via Stefan O'Rear just now) there was a similar kernel bug
> introduced in 161550d74c07303ffa6187ba776f62df5a906a21 that makes
> wait4 fail to honor pgrp changes that happen while already in the
> syscall (e.g. performed on the caller by another thread or even
> another process).

I could not find the report from Stefan O'Rear.

Does that result in actual problems for programs or is this a
theoretical race noticed upon code review?

> But the race condition here in userspace is even
> more egregious I think, since it violates the contract in a case where
> there is a clear observable order between the pgrp change and the
> blocking wait -- for instance, the signal handler could change pgrp
> of itself and a child process, and then whether or not the signal
> handler had executed before the waitpid, the waitpid should catch the
> child's exit. But with the above race, it fails to.

Definitely a bigger issue.  I believe posix requires waitpid is required
to be signal safe.

Eric

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Rich Felker]

On Sun, Jul 21, 2019 at 06:59:09AM -0500, Eric W. Biederman wrote:
> Rich Felker <dalias@libc.org> writes:
> 
> > On Sun, Jul 21, 2019 at 12:03:10AM -0400, Rich Felker wrote:
> >> On Tue, Jul 16, 2019 at 05:08:48PM -0700, Alistair Francis wrote:
> >> > If the waitid syscall is avaliable let's use that as waitpid
> >> > and wait4 aren't always avaliable (they aren't avaliable on RV32).
> >> > 
> >> > Unfortunately waitid is substantially differnt to waitpid and wait4, so
> >> > the conversion ends up being complex.
> >> > 
> >> > Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
> >> > ---
> >> >  ChangeLog                                  |  3 ++
> >> >  sysdeps/unix/sysv/linux/wait.c             | 39 ++++++++++++++++--
> >> >  sysdeps/unix/sysv/linux/waitpid.c          | 46 ++++++++++++++++++++++
> >> >  sysdeps/unix/sysv/linux/waitpid_nocancel.c | 45 +++++++++++++++++++++
> >> >  4 files changed, 130 insertions(+), 3 deletions(-)
> >> > [...]
> >> >  
> >> >  weak_alias (__libc_wait, __wait)
> >> > diff --git a/sysdeps/unix/sysv/linux/waitpid.c b/sysdeps/unix/sysv/linux/waitpid.c
> >> > index f0897574c0..7d4e0bb77d 100644
> >> > --- a/sysdeps/unix/sysv/linux/waitpid.c
> >> > +++ b/sysdeps/unix/sysv/linux/waitpid.c
> >> > @@ -20,12 +20,58 @@
> >> >  #include <sysdep-cancel.h>
> >> >  #include <stdlib.h>
> >> >  #include <sys/wait.h>
> >> > +#include <unistd.h>
> >> >  
> >> >  __pid_t
> >> >  __waitpid (__pid_t pid, int *stat_loc, int options)
> >> >  {
> >> >  #ifdef __NR_waitpid
> >> >    return SYSCALL_CANCEL (waitpid, pid, stat_loc, options);
> >> > +#elif defined(__NR_waitid)
> >> > +  __pid_t ret;
> >> > +  idtype_t idtype = P_PID;
> >> > +  siginfo_t infop;
> >> > +
> >> > +  if (pid < -1) {
> >> > +    idtype = P_PGID;
> >> > +    pid *= -1;
> >> > +  } else if (pid == -1) {
> >> > +    idtype = P_ALL;
> >> > +  } else if (pid == 0) {
> >> > +    idtype = P_PGID;
> >> > +    pid = getpgrp();
> >> > +  }
> >> > +
> >> > +  options |= WEXITED;
> >> > +
> >> > +  ret = SYSCALL_CANCEL (waitid, idtype, pid, &infop, options, NULL);
> >> 
> >> This emulation has a fundamental race condition. Between getpgrp and
> >> waitid, a signal handler may perform setpgrp, setsid, and/or fork in
> >> ways that cause the wrong pgid to be passed to the waitid syscall.
> >> There is no way around this because you cannot block signals for the
> >> interval, since signals must be able to interrupt the waitid syscall.
> >> 
> >> Unless there's some trick I'm missing here, the kernel folks' removal
> >> of the wait4 syscall is just a bug in the kernel that they need to
> >> fix. It also makes it impossible to implement the wait4 function,
> >> since there's no way to get rusage for the exited process.
> >
> > Reportedly (via Stefan O'Rear just now) there was a similar kernel bug
> > introduced in 161550d74c07303ffa6187ba776f62df5a906a21 that makes
> > wait4 fail to honor pgrp changes that happen while already in the
> > syscall (e.g. performed on the caller by another thread or even
> > another process).
> 
> I could not find the report from Stefan O'Rear.

There's no report yet as far as I know; it came up on #musl when I
mentioned the issue there. I'll inquire about getting it properly
reported. I just wanted to give credit to the original source when
conveying it here.

> Does that result in actual problems for programs or is this a
> theoretical race noticed upon code review?

I noticed it while looking at the glibc rv32 port proposal, where the
emulation caught my eye as something ugly and undesirable we'd have to
do for musl too. As soon as I looked at it in more detail, the race
condition was apparent, and this is exactly the kind of subtle bug we
don't like to reproduct in musl. I'm not aware of any particular
application that would be broken, but it's a cleae violation of the
AS-safety requirements of the interfaces involved.

Regarding resolving it, I would be perfectly happy with enabling wait4
even for archs that don't want time32 syscalls. musl will be
translating the wait4 rusage results to 64-bit time_t anyway, so
there's utterly no advantage to having to do it in a gratuitously
different way on rv32. The only problem with the fields in the kernel
struct rusage being 32-bit is that you can't represent usage of more
than 68 years of cpu time by a process, which is rather
inconsequential -- but if a working extension to the waitid syscall is
provided, we can use that too.

Rich

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Arnd Bergmann]

On Sun, Jul 21, 2019 at 6:03 AM Rich Felker <dalias@libc.org> wrote:
> On Tue, Jul 16, 2019 at 05:08:48PM -0700, Alistair Francis wrote:
> >  #ifdef __NR_waitpid
> >    return SYSCALL_CANCEL (waitpid, pid, stat_loc, options);
> > +#elif defined(__NR_waitid)
> > +  __pid_t ret;
> > +  idtype_t idtype = P_PID;
> > +  siginfo_t infop;
> > +
> > +  if (pid < -1) {
> > +    idtype = P_PGID;
> > +    pid *= -1;
> > +  } else if (pid == -1) {
> > +    idtype = P_ALL;
> > +  } else if (pid == 0) {
> > +    idtype = P_PGID;
> > +    pid = getpgrp();
> > +  }
> > +
> > +  options |= WEXITED;
> > +
> > +  ret = SYSCALL_CANCEL (waitid, idtype, pid, &infop, options, NULL);
>
> This emulation has a fundamental race condition. Between getpgrp and
> waitid, a signal handler may perform setpgrp, setsid, and/or fork in
> ways that cause the wrong pgid to be passed to the waitid syscall.
> There is no way around this because you cannot block signals for the
> interval, since signals must be able to interrupt the waitid syscall.

Interesting, I don't think anyone ever considered this race, as waitid()
was clearly always /intended/ as a superset of wait(), waitpid(), wait3()
and wait4(), as Roland McGrath described in the initial creation
of the waitid() syscall, see
https://lore.kernel.org/lkml/200408152303.i7FN3Vc3021030@magilla.sf.frob.com/

I originally planned to have a new waitid_time64() syscall to replace
them in the y2038 series, but that got deferred, so at the moment
the older 32-bit architectures don't have any changes for wait*(), and
riscv32 just removed wait4() but kept waitid().

It would be easy enough to restore wait4() for riscv32, or we could
decide to extend waitid() in a backward-compatible way to close
this race, e.g. by allowing P_PGID with pid=0 (this is currently
-EINVAL),  by adding a new P_SAME_PGID macro to cover that
case, or by creating a new waitid replacement that does other things
as well (nanosecond ru_*time, pidfd, ...).

Adding a few more kernel developers that may have an opinion on this.

> Unless there's some trick I'm missing here, the kernel folks' removal
> of the wait4 syscall is just a bug in the kernel that they need to
> fix. It also makes it impossible to implement the wait4 function,
> since there's no way to get rusage for the exited process.

The rusage is passed back from the kernel waitid() as an extension
to the posix interface, in order to allow implementing wait4().

      Arnd

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Eric W. Biederman]

Arnd Bergmann <arnd@arndb.de> writes:

> On Sun, Jul 21, 2019 at 6:03 AM Rich Felker <dalias@libc.org> wrote:
>> On Tue, Jul 16, 2019 at 05:08:48PM -0700, Alistair Francis wrote:
>> >  #ifdef __NR_waitpid
>> >    return SYSCALL_CANCEL (waitpid, pid, stat_loc, options);
>> > +#elif defined(__NR_waitid)
>> > +  __pid_t ret;
>> > +  idtype_t idtype = P_PID;
>> > +  siginfo_t infop;
>> > +
>> > +  if (pid < -1) {
>> > +    idtype = P_PGID;
>> > +    pid *= -1;
>> > +  } else if (pid == -1) {
>> > +    idtype = P_ALL;
>> > +  } else if (pid == 0) {
>> > +    idtype = P_PGID;
>> > +    pid = getpgrp();
>> > +  }
>> > +
>> > +  options |= WEXITED;
>> > +
>> > +  ret = SYSCALL_CANCEL (waitid, idtype, pid, &infop, options, NULL);
>>
>> This emulation has a fundamental race condition. Between getpgrp and
>> waitid, a signal handler may perform setpgrp, setsid, and/or fork in
>> ways that cause the wrong pgid to be passed to the waitid syscall.
>> There is no way around this because you cannot block signals for the
>> interval, since signals must be able to interrupt the waitid syscall.
>
> Interesting, I don't think anyone ever considered this race, as waitid()
> was clearly always /intended/ as a superset of wait(), waitpid(), wait3()
> and wait4(), as Roland McGrath described in the initial creation
> of the waitid() syscall, see
> https://lore.kernel.org/lkml/200408152303.i7FN3Vc3021030@magilla.sf.frob.com/
>

It is definitley a problem as setpid and setpgrp and setsid are all
required to be signal safe, and this waitpid emulation is clearly not.

> I originally planned to have a new waitid_time64() syscall to replace
> them in the y2038 series, but that got deferred, so at the moment
> the older 32-bit architectures don't have any changes for wait*(), and
> riscv32 just removed wait4() but kept waitid().
>
> It would be easy enough to restore wait4() for riscv32, or we could
> decide to extend waitid() in a backward-compatible way to close
> this race, e.g. by allowing P_PGID with pid=0 (this is currently
> -EINVAL),  by adding a new P_SAME_PGID macro to cover that
> case, or by creating a new waitid replacement that does other things
> as well (nanosecond ru_*time, pidfd, ...).
>
> Adding a few more kernel developers that may have an opinion on this.

In a different reply it was mentioned that wait4 has had an issue since
2008, where it does not track changes in the current processes pgrp.

I would like to get resolution on if that is a real problem for anyone
or if it is a something theoretical found on code review, before we deal
with this issue.  Because tracking pgrp changes will require a deeper
code change.

My gut feel says either fix waitid (with P_SAME_PGID) or clearly document
why the kernel's waitid is not a replacement for wait4, and add wait4 to
riscv.

Would adding wait4 to riscv allow us to get rid of the rusage parameter
of the kernel's waitid?

I don't oppose a new syscall to take advantage of pidfd but I don't
think there is any need to tie the two fixes together so I would rather
keep them separate.  Just so we don't rush through fixing one to deal
with the other.

Eric

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Christian Brauner]

On July 21, 2019 2:15:24 PM GMT+02:00, ebiederm@xmission.com wrote:
>Arnd Bergmann <arnd@arndb.de> writes:
>
>> On Sun, Jul 21, 2019 at 6:03 AM Rich Felker <dalias@libc.org> wrote:
>>> On Tue, Jul 16, 2019 at 05:08:48PM -0700, Alistair Francis wrote:
>>> >  #ifdef __NR_waitpid
>>> >    return SYSCALL_CANCEL (waitpid, pid, stat_loc, options);
>>> > +#elif defined(__NR_waitid)
>>> > +  __pid_t ret;
>>> > +  idtype_t idtype = P_PID;
>>> > +  siginfo_t infop;
>>> > +
>>> > +  if (pid < -1) {
>>> > +    idtype = P_PGID;
>>> > +    pid *= -1;
>>> > +  } else if (pid == -1) {
>>> > +    idtype = P_ALL;
>>> > +  } else if (pid == 0) {
>>> > +    idtype = P_PGID;
>>> > +    pid = getpgrp();
>>> > +  }
>>> > +
>>> > +  options |= WEXITED;
>>> > +
>>> > +  ret = SYSCALL_CANCEL (waitid, idtype, pid, &infop, options,
>NULL);
>>>
>>> This emulation has a fundamental race condition. Between getpgrp and
>>> waitid, a signal handler may perform setpgrp, setsid, and/or fork in
>>> ways that cause the wrong pgid to be passed to the waitid syscall.
>>> There is no way around this because you cannot block signals for the
>>> interval, since signals must be able to interrupt the waitid
>syscall.
>>
>> Interesting, I don't think anyone ever considered this race, as
>waitid()
>> was clearly always /intended/ as a superset of wait(), waitpid(),
>wait3()
>> and wait4(), as Roland McGrath described in the initial creation
>> of the waitid() syscall, see
>>
>https://lore.kernel.org/lkml/200408152303.i7FN3Vc3021030@magilla.sf.frob.com/
>>
>
>It is definitley a problem as setpid and setpgrp and setsid are all
>required to be signal safe, and this waitpid emulation is clearly not.
>
>> I originally planned to have a new waitid_time64() syscall to replace
>> them in the y2038 series, but that got deferred, so at the moment
>> the older 32-bit architectures don't have any changes for wait*(),
>and
>> riscv32 just removed wait4() but kept waitid().
>>
>> It would be easy enough to restore wait4() for riscv32, or we could
>> decide to extend waitid() in a backward-compatible way to close
>> this race, e.g. by allowing P_PGID with pid=0 (this is currently
>> -EINVAL),  by adding a new P_SAME_PGID macro to cover that
>> case, or by creating a new waitid replacement that does other things
>> as well (nanosecond ru_*time, pidfd, ...).
>>
>> Adding a few more kernel developers that may have an opinion on this.
>
>In a different reply it was mentioned that wait4 has had an issue since
>2008, where it does not track changes in the current processes pgrp.
>
>I would like to get resolution on if that is a real problem for anyone
>or if it is a something theoretical found on code review, before we
>deal
>with this issue.  Because tracking pgrp changes will require a deeper
>code change.
>
>My gut feel says either fix waitid (with P_SAME_PGID) or clearly
>document
>why the kernel's waitid is not a replacement for wait4, and add wait4
>to
>riscv.
>
>Would adding wait4 to riscv allow us to get rid of the rusage parameter
>of the kernel's waitid?
>
>I don't oppose a new syscall to take advantage of pidfd but I don't
>think there is any need to tie the two fixes together so I would rather
>keep them separate.  Just so we don't rush through fixing one to deal
>with the other.
>
>Eric

Fwiw, I have a patchset for pidfd_wait ready.
But I strongly oppose mixing pids and pidfds.
That syscall will only handle pidfds.

Christian

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Arnd Bergmann]

On Sun, Jul 21, 2019 at 2:15 PM Eric W. Biederman <ebiederm@xmission.com> wrote:
> Arnd Bergmann <arnd@arndb.de> writes:
> > On Sun, Jul 21, 2019 at 6:03 AM Rich Felker <dalias@libc.org> wrote:
> >> On Tue, Jul 16, 2019 at 05:08:48PM -0700, Alistair Francis wrote:
> >> >  #ifdef __NR_waitpid
> >> >    return SYSCALL_CANCEL (waitpid, pid, stat_loc, options);
> >> > +#elif defined(__NR_waitid)
> >> > +  __pid_t ret;
> >> > +  idtype_t idtype = P_PID;
> >> > +  siginfo_t infop;
> >> > +
> >> > +  if (pid < -1) {
> >> > +    idtype = P_PGID;
> >> > +    pid *= -1;
> >> > +  } else if (pid == -1) {
> >> > +    idtype = P_ALL;
> >> > +  } else if (pid == 0) {
> >> > +    idtype = P_PGID;
> >> > +    pid = getpgrp();
> >> > +  }
> >> > +
> >> > +  options |= WEXITED;
> >> > +
> >> > +  ret = SYSCALL_CANCEL (waitid, idtype, pid, &infop, options, NULL);
> >>
> >> This emulation has a fundamental race condition. Between getpgrp and
> >> waitid, a signal handler may perform setpgrp, setsid, and/or fork in
> >> ways that cause the wrong pgid to be passed to the waitid syscall.
> >> There is no way around this because you cannot block signals for the
> >> interval, since signals must be able to interrupt the waitid syscall.
> >
> > Interesting, I don't think anyone ever considered this race, as waitid()
> > was clearly always /intended/ as a superset of wait(), waitpid(), wait3()
> > and wait4(), as Roland McGrath described in the initial creation
> > of the waitid() syscall, see
> > https://lore.kernel.org/lkml/200408152303.i7FN3Vc3021030@magilla.sf.frob.com/
> >
>
> It is definitley a problem as setpid and setpgrp and setsid are all
> required to be signal safe, and this waitpid emulation is clearly not.

Ok.

> Would adding wait4 to riscv allow us to get rid of the rusage parameter
> of the kernel's waitid?

No, the kernel waitid() behavior is widely documented, and there are
likely to be applications that rely on it across architectures.

The only downside I can think of for adding wait4() is that it also
uses 32-bit time_t in its rusage, so if we create a time64 version
of rusage, we now have to add three more system calls (waitid,
getrusage, and wait4) instead of just two.

> I don't oppose a new syscall to take advantage of pidfd but I don't
> think there is any need to tie the two fixes together so I would rather
> keep them separate.  Just so we don't rush through fixing one to deal
> with the other.

I see at multiple independent issues with the current waitid():

- The race that Rich pointed out
- The lack of pidfd support
- The usage of a 32-bit timeval structure that is incompatible with
  the libc definition on rv32 and other 32-bit architectures with
  a y2038-safe libc (there is no y2038 overflow in rusage itself, but
  it requires an ugly wrapper to convert the structure)
- The lack of nanosecond resolution in rusage that someone asked for
- When we last talked about it, there was some debate over replacing
  siginfo_t with a different structure.

We don't have to address each one of those in a single new syscall
(or even at all), but addressing them one at a time would risk adding
even more system calls to do essentially just one thing that already
has at least five versions at the libc level (wait, waitpid, wait3, wait4
and waitid).

      Arnd

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Eric W. Biederman]

Arnd Bergmann <arnd@arndb.de> writes:

> On Sun, Jul 21, 2019 at 2:15 PM Eric W. Biederman <ebiederm@xmission.com> wrote:
>> Arnd Bergmann <arnd@arndb.de> writes:
>> > On Sun, Jul 21, 2019 at 6:03 AM Rich Felker <dalias@libc.org> wrote:
>> >> On Tue, Jul 16, 2019 at 05:08:48PM -0700, Alistair Francis wrote:
>> >> >  #ifdef __NR_waitpid
>> >> >    return SYSCALL_CANCEL (waitpid, pid, stat_loc, options);
>> >> > +#elif defined(__NR_waitid)
>> >> > +  __pid_t ret;
>> >> > +  idtype_t idtype = P_PID;
>> >> > +  siginfo_t infop;
>> >> > +
>> >> > +  if (pid < -1) {
>> >> > +    idtype = P_PGID;
>> >> > +    pid *= -1;
>> >> > +  } else if (pid == -1) {
>> >> > +    idtype = P_ALL;
>> >> > +  } else if (pid == 0) {
>> >> > +    idtype = P_PGID;
>> >> > +    pid = getpgrp();
>> >> > +  }
>> >> > +
>> >> > +  options |= WEXITED;
>> >> > +
>> >> > +  ret = SYSCALL_CANCEL (waitid, idtype, pid, &infop, options, NULL);
>> >>
>> >> This emulation has a fundamental race condition. Between getpgrp and
>> >> waitid, a signal handler may perform setpgrp, setsid, and/or fork in
>> >> ways that cause the wrong pgid to be passed to the waitid syscall.
>> >> There is no way around this because you cannot block signals for the
>> >> interval, since signals must be able to interrupt the waitid syscall.
>> >
>> > Interesting, I don't think anyone ever considered this race, as waitid()
>> > was clearly always /intended/ as a superset of wait(), waitpid(), wait3()
>> > and wait4(), as Roland McGrath described in the initial creation
>> > of the waitid() syscall, see
>> > https://lore.kernel.org/lkml/200408152303.i7FN3Vc3021030@magilla.sf.frob.com/
>> >
>>
>> It is definitley a problem as setpid and setpgrp and setsid are all
>> required to be signal safe, and this waitpid emulation is clearly not.
>
> Ok.
>
>> Would adding wait4 to riscv allow us to get rid of the rusage parameter
>> of the kernel's waitid?
>
> No, the kernel waitid() behavior is widely documented, and there are
> likely to be applications that rely on it across architectures.
>
> The only downside I can think of for adding wait4() is that it also
> uses 32-bit time_t in its rusage, so if we create a time64 version
> of rusage, we now have to add three more system calls (waitid,
> getrusage, and wait4) instead of just two.
>
>> I don't oppose a new syscall to take advantage of pidfd but I don't
>> think there is any need to tie the two fixes together so I would rather
>> keep them separate.  Just so we don't rush through fixing one to deal
>> with the other.
>
> I see at multiple independent issues with the current waitid():
>
> - The race that Rich pointed out
> - The lack of pidfd support
> - The usage of a 32-bit timeval structure that is incompatible with
>   the libc definition on rv32 and other 32-bit architectures with
>   a y2038-safe libc (there is no y2038 overflow in rusage itself, but
>   it requires an ugly wrapper to convert the structure)
> - The lack of nanosecond resolution in rusage that someone asked for
> - When we last talked about it, there was some debate over replacing
>   siginfo_t with a different structure.
>
> We don't have to address each one of those in a single new syscall
> (or even at all), but addressing them one at a time would risk adding
> even more system calls to do essentially just one thing that already
> has at least five versions at the libc level (wait, waitpid, wait3, wait4
> and waitid).

For the particular issue of 32bit riscv needing a working wait4 I see
one of two possibilities.

We either add P_PROCESS_PGID to the kernel's waitid or we add wait4.

I am leaning towards P_PROCESS_PGID as this is the tiny little bit
needed to make the kernel's waitid the one call that combines them all,
that it already tries to be.  Plus P_PROCESS_PGID or the equivalent in
the kernel internal version is needed if we choose to have wait4 handle
the process group id changing while wait4 is running.

Eric

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Arnd Bergmann]

On Sun, Jul 21, 2019 at 5:45 PM Eric W. Biederman <ebiederm@xmission.com> wrote:
> Arnd Bergmann <arnd@arndb.de> writes:
> > On Sun, Jul 21, 2019 at 2:15 PM Eric W. Biederman <ebiederm@xmission.com> wrote:
> >> Arnd Bergmann <arnd@arndb.de> writes:
> >> > On Sun, Jul 21, 2019 at 6:03 AM Rich Felker <dalias@libc.org> wrote:
> >> >> On Tue, Jul 16, 2019 at 05:08:48PM -0700, Alistair Francis wrote:
> > - The race that Rich pointed out
> > - The lack of pidfd support
> > - The usage of a 32-bit timeval structure that is incompatible with
> >   the libc definition on rv32 and other 32-bit architectures with
> >   a y2038-safe libc (there is no y2038 overflow in rusage itself, but
> >   it requires an ugly wrapper to convert the structure)
> > - The lack of nanosecond resolution in rusage that someone asked for
> > - When we last talked about it, there was some debate over replacing
> >   siginfo_t with a different structure.
> >
> > We don't have to address each one of those in a single new syscall
> > (or even at all), but addressing them one at a time would risk adding
> > even more system calls to do essentially just one thing that already
> > has at least five versions at the libc level (wait, waitpid, wait3, wait4
> > and waitid).
>
> For the particular issue of 32bit riscv needing a working wait4 I see
> one of two possibilities.
>
> We either add P_PROCESS_PGID to the kernel's waitid or we add wait4.
>
> I am leaning towards P_PROCESS_PGID as this is the tiny little bit
> needed to make the kernel's waitid the one call that combines them all,
> that it already tries to be.  Plus P_PROCESS_PGID or the equivalent in
> the kernel internal version is needed if we choose to have wait4 handle
> the process group id changing while wait4 is running.

Yes, that sounds good to me, both as a short-term and as a long-term
solution for the race, on all architectures. It would help if we could
have some extra review on the wait4() and waitpid() implementation
around waitid(). I first implemented this in my prototype for a y2038-safe
musl port, and Alistair write a corresponding version for glibc.

I did make sure that it passes LTP (and that found a number of bugs
in my first attempt), but it would still be good to know that the P_PGID
issue is the only problem in the underlying system call, aside from
any remaining bugs in the wrapper implementation.

        Arnd

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Linus Torvalds]

On Sun, Jul 21, 2019 at 8:45 AM Eric W. Biederman <ebiederm@xmission.com> wrote:
>
> We either add P_PROCESS_PGID to the kernel's waitid or we add wait4.

Do we actually need a new flag?

Why not just allow people to pass in pid = 0 with the existing P_PGID flag?

Those are the traditional and documented waitpid() semantics. Why
wouldn't we use those semantics for waitid() too?

And since our current (broken) waitid() returns EINVAL for that case,
it's even trivial to test for in user space (and fall back on the
broken racy code in user space - which you have to do anyway).

Honestly, that seems like the simplest soilution, but also like the
only sane model. The fact that P_PGID with a zero pid doesn't work
seems to simply be a bug right now, and keeps waitid() from being a
proper superset of waitpid().

              Linus

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Eric W. Biederman]

Linus Torvalds <torvalds@linux-foundation.org> writes:

> On Sun, Jul 21, 2019 at 8:45 AM Eric W. Biederman <ebiederm@xmission.com> wrote:
>>
>> We either add P_PROCESS_PGID to the kernel's waitid or we add wait4.
>
> Do we actually need a new flag?
>
> Why not just allow people to pass in pid = 0 with the existing P_PGID flag?
>
> Those are the traditional and documented waitpid() semantics. Why
> wouldn't we use those semantics for waitid() too?
>
> And since our current (broken) waitid() returns EINVAL for that case,
> it's even trivial to test for in user space (and fall back on the
> broken racy code in user space - which you have to do anyway).

Our current waitid implementation when serving as waitid is not broken.

What is broken is the waitpid emulation on top of waitid.

The way waitid is defined waitid P_PGID with a pid of 0 means
wait for the process group id of 0.  Which in theory and in limited
practice exists.  AKA that is the pid and process group of the idle
thread.

Further there is the outstanding question.  Should P_PROCESS_PGID refer
to the curent processes process group id at the time of the waitid call,
or should P_PROCESS_PGID refer to the process group id when a child is
found?

It has been suggested elswehere in this conversation that 161550d74c07
("pid: sys_wait... fixes") may have introduced a regression.  If so
the current wait4 behavior of capturing the process group at the time
of call is wrong and needs to be fixed.

Not capuring the pid at time time of call is a very different behavior
of P_PROCESS_PGID vs all of the other waitid cases which do have the pid
fixed at the time of the call which further argues a separate flag
because the behavior would be distinctly different.

> Honestly, that seems like the simplest soilution, but also like the
> only sane model. The fact that P_PGID with a zero pid doesn't work
> seems to simply be a bug right now, and keeps waitid() from being a
> proper superset of waitpid().

In the context of waitid it does not look sane.  Unlike the other wait
functions waitid does not have any magic ids and by having an idtype
field makes that kind of magic unnecessary.  Further it is trivial
to add one line to the architecture independent enumeration and have 0
risk of confusion or of regressing user space.

Eric

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Rich Felker]

On Sun, Jul 21, 2019 at 04:40:27PM -0500, Eric W. Biederman wrote:
> Linus Torvalds <torvalds@linux-foundation.org> writes:
> 
> > On Sun, Jul 21, 2019 at 8:45 AM Eric W. Biederman <ebiederm@xmission.com> wrote:
> >>
> >> We either add P_PROCESS_PGID to the kernel's waitid or we add wait4.
> >
> > Do we actually need a new flag?
> >
> > Why not just allow people to pass in pid = 0 with the existing P_PGID flag?
> >
> > Those are the traditional and documented waitpid() semantics. Why
> > wouldn't we use those semantics for waitid() too?
> >
> > And since our current (broken) waitid() returns EINVAL for that case,
> > it's even trivial to test for in user space (and fall back on the
> > broken racy code in user space - which you have to do anyway).
> 
> Our current waitid implementation when serving as waitid is not broken.
> 
> What is broken is the waitpid emulation on top of waitid.
> 
> The way waitid is defined waitid P_PGID with a pid of 0 means
> wait for the process group id of 0.  Which in theory and in limited
> practice exists.  AKA that is the pid and process group of the idle
> thread.
> 
> Further there is the outstanding question.  Should P_PROCESS_PGID refer
> to the curent processes process group id at the time of the waitid call,
> or should P_PROCESS_PGID refer to the process group id when a child is
> found?
> 
> It has been suggested elswehere in this conversation that 161550d74c07
> ("pid: sys_wait... fixes") may have introduced a regression.  If so
> the current wait4 behavior of capturing the process group at the time
> of call is wrong and needs to be fixed.
> 
> Not capuring the pid at time time of call is a very different behavior
> of P_PROCESS_PGID vs all of the other waitid cases which do have the pid
> fixed at the time of the call which further argues a separate flag
> because the behavior would be distinctly different.

I believe that is a regression, and that the "capturing it" line of
thinking is misleading. It's thinking in terms of implementation
rather than interface contract. The interface contract covers which
children it can reap and when it can block. If the call remains
blocking, there must be a possible ordering of events, with no
observable effects contradicting that order, by which there are no
reapable children in the process's current pgid. If the call returns
successfully, there must be a possible ordering of events, with no
observable effects contradicting that order, by which the caller's
pgid and the reaped process's pgid were equal.

> > Honestly, that seems like the simplest soilution, but also like the
> > only sane model. The fact that P_PGID with a zero pid doesn't work
> > seems to simply be a bug right now, and keeps waitid() from being a
> > proper superset of waitpid().
> 
> In the context of waitid it does not look sane.  Unlike the other wait
> functions waitid does not have any magic ids and by having an idtype
> field makes that kind of magic unnecessary.  Further it is trivial
> to add one line to the architecture independent enumeration and have 0
> risk of confusion or of regressing user space.

I'm in agreement that if an extension to the waitid syscall is added,
it should be P_PROCESS_PGID, not defining some special case for
pid==0. It's not entirely clear but arguable that the standard
requires EINVAL for P_PGID + pid==0, and plausible that there are
applications that depend on this. We could emulate the EINVAL case in
userspace, but assigning weird semantics to special cases is just a
mess of potential future headaches when it would be trivial to do it
right. And doing it right would also make programming userspace side
easier.

Rich

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Eric W. Biederman]

Rich Felker <dalias@libc.org> writes:

> On Sun, Jul 21, 2019 at 04:40:27PM -0500, Eric W. Biederman wrote:
>> Linus Torvalds <torvalds@linux-foundation.org> writes:
>> 
>> > On Sun, Jul 21, 2019 at 8:45 AM Eric W. Biederman <ebiederm@xmission.com> wrote:
>> >>
>> >> We either add P_PROCESS_PGID to the kernel's waitid or we add wait4.
>> >
>> > Do we actually need a new flag?
>> >
>> > Why not just allow people to pass in pid = 0 with the existing P_PGID flag?
>> >
>> > Those are the traditional and documented waitpid() semantics. Why
>> > wouldn't we use those semantics for waitid() too?
>> >
>> > And since our current (broken) waitid() returns EINVAL for that case,
>> > it's even trivial to test for in user space (and fall back on the
>> > broken racy code in user space - which you have to do anyway).
>> 
>> Our current waitid implementation when serving as waitid is not broken.
>> 
>> What is broken is the waitpid emulation on top of waitid.
>> 
>> The way waitid is defined waitid P_PGID with a pid of 0 means
>> wait for the process group id of 0.  Which in theory and in limited
>> practice exists.  AKA that is the pid and process group of the idle
>> thread.
>> 
>> Further there is the outstanding question.  Should P_PROCESS_PGID refer
>> to the curent processes process group id at the time of the waitid call,
>> or should P_PROCESS_PGID refer to the process group id when a child is
>> found?
>> 
>> It has been suggested elswehere in this conversation that 161550d74c07
>> ("pid: sys_wait... fixes") may have introduced a regression.  If so
>> the current wait4 behavior of capturing the process group at the time
>> of call is wrong and needs to be fixed.
>> 
>> Not capuring the pid at time time of call is a very different behavior
>> of P_PROCESS_PGID vs all of the other waitid cases which do have the pid
>> fixed at the time of the call which further argues a separate flag
>> because the behavior would be distinctly different.
>
> I believe that is a regression, and that the "capturing it" line of
> thinking is misleading. It's thinking in terms of implementation
> rather than interface contract. The interface contract covers which
> children it can reap and when it can block. If the call remains
> blocking, there must be a possible ordering of events, with no
> observable effects contradicting that order, by which there are no
> reapable children in the process's current pgid. If the call returns
> successfully, there must be a possible ordering of events, with no
> observable effects contradicting that order, by which the caller's
> pgid and the reaped process's pgid were equal.

A regression (as we talk about it in the linux kernel) requires there to
be an actual program that has problems because of the change in behavior
of the kernel.  Just it being possible for userspace to observe a
behavior change is not enough for something to be classified as a
regression.

Furthermore thinking about this and looking at it more closely this can
only be an issue in the presence of pthreads.  Except in a very narrow
special case setpgid is restricted to only changing the process group of
the calling process.  Which means that in unix without pthreads setpgid
can not run while a process is in wait, waitpid, or wait4.


I was really hoping we would have a pointer to a program that shows
a problem, or a clarification that someone was just reading the code.
I will wait a bit more but given that the code has worked without bug
reports for 11 years I am going to assume that there are no programs
that have regressed because of my old change.


>> > Honestly, that seems like the simplest soilution, but also like the
>> > only sane model. The fact that P_PGID with a zero pid doesn't work
>> > seems to simply be a bug right now, and keeps waitid() from being a
>> > proper superset of waitpid().
>> 
>> In the context of waitid it does not look sane.  Unlike the other wait
>> functions waitid does not have any magic ids and by having an idtype
>> field makes that kind of magic unnecessary.  Further it is trivial
>> to add one line to the architecture independent enumeration and have 0
>> risk of confusion or of regressing user space.
>
> I'm in agreement that if an extension to the waitid syscall is added,
> it should be P_PROCESS_PGID, not defining some special case for
> pid==0. It's not entirely clear but arguable that the standard
> requires EINVAL for P_PGID + pid==0, and plausible that there are
> applications that depend on this. We could emulate the EINVAL case in
> userspace, but assigning weird semantics to special cases is just a
> mess of potential future headaches when it would be trivial to do it
> right. And doing it right would also make programming userspace side
> easier.

Since this is a POSIX interface and shared between many unices I took
at look at a couple to see what they have done.  If possible it is
important to be able to write programs that work the same across
different unices.

The BSDs implemented wait6 as their common wait that does everything
system call.   It's large contribution is that wait6 will give both
the rusage of the child that exited and it's children separately.
To be the one system call that is a superset of them all wait6 implements
both P_PGID and P_PID with an id of 0 as a request to wait for any
process in the current process group.

The wait6 family also extends the idtype enumeration a bit with
things such as P_UID and P_GID that wait for the effective uid and gid
respectively.

I looked at the code on FreeBSD and they replace the id of 0 with
an the result of getpgid() internal to the system call, before any
waiting happens.   Resulting in the same semantics as current Linux.


I took a quick look at what Illumos does and they have narrowed their
kernel wait interface to just the posix waitid.  There is no rusage
nor is their any special case for waiting for the processes in the
current process group.


I looked at XNU and it still implements both wait4 and waitid as native
interfaces.  The waitid is not extended while their wait4 converts 0 to
-getpgrp() before waiting.



So my recommendation now is to avoid gratuitous incompatibilities.

1) For extending waitid.  Let's use P_PGID and and id of 0 to represent
   the callers process group.  That is compatible with the BSDs, and
   portable programs should not have any problem with it.

   I want to stay away from the BSD extention of P_PID with an id of 0
   meaning wait for the calling process's process group.  I see where
   it comes from but that is confusing.

2) It appears the popular definition of current process group is the
   current process group at the time of the system call.  Currently that
   is Linux, FreeBSD, Illumos, and XNU.  So short of a real program that
   cares, let's adopt that definition in linux.  Along with patches I
   will see about getting the linux manpage updated to clarify that
   point.

Eric

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Arnd Bergmann]

On Tue, Jul 23, 2019 at 2:01 AM Eric W. Biederman <ebiederm@xmission.com> wrote:
> Rich Felker <dalias@libc.org> writes:
> > On Sun, Jul 21, 2019 at 04:40:27PM -0500, Eric W. Biederman wrote:
> >> Linus Torvalds <torvalds@linux-foundation.org> writes:
> > I'm in agreement that if an extension to the waitid syscall is added,
> > it should be P_PROCESS_PGID, not defining some special case for
> > pid==0. It's not entirely clear but arguable that the standard
> > requires EINVAL for P_PGID + pid==0, and plausible that there are
> > applications that depend on this. We could emulate the EINVAL case in
> > userspace, but assigning weird semantics to special cases is just a
> > mess of potential future headaches when it would be trivial to do it
> > right. And doing it right would also make programming userspace side
> > easier.
>
> Since this is a POSIX interface and shared between many unices I took
> at look at a couple to see what they have done.  If possible it is
> important to be able to write programs that work the same across
> different unices.
>
> The BSDs implemented wait6 as their common wait that does everything
> system call.   It's large contribution is that wait6 will give both
> the rusage of the child that exited and it's children separately.

Ah right, that was one more thing I forgot on my earlier list. If we end up
creating a new wait4()/waitid() replacement  to replace the timeval
with __kernel_timespec, we may want that to return a wrusage instead
of rusage. Same for Christian's proposed pidfd_wait().

> So my recommendation now is to avoid gratuitous incompatibilities.
>
> 1) For extending waitid.  Let's use P_PGID and and id of 0 to represent
>    the callers process group.  That is compatible with the BSDs, and
>    portable programs should not have any problem with it.
>
>    I want to stay away from the BSD extention of P_PID with an id of 0
>    meaning wait for the calling process's process group.  I see where
>    it comes from but that is confusing.
>
> 2) It appears the popular definition of current process group is the
>    current process group at the time of the system call.  Currently that
>    is Linux, FreeBSD, Illumos, and XNU.  So short of a real program that
>    cares, let's adopt that definition in linux.  Along with patches I
>    will see about getting the linux manpage updated to clarify that
>    point.

That all sounds reasonable to me, thanks for researching the other
implementations!

         Arnd

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Christian Brauner]

On Tue, Jul 23, 2019 at 10:12:07AM +0200, Arnd Bergmann wrote:
> On Tue, Jul 23, 2019 at 2:01 AM Eric W. Biederman <ebiederm@xmission.com> wrote:
> > Rich Felker <dalias@libc.org> writes:
> > > On Sun, Jul 21, 2019 at 04:40:27PM -0500, Eric W. Biederman wrote:
> > >> Linus Torvalds <torvalds@linux-foundation.org> writes:
> > > I'm in agreement that if an extension to the waitid syscall is added,
> > > it should be P_PROCESS_PGID, not defining some special case for
> > > pid==0. It's not entirely clear but arguable that the standard
> > > requires EINVAL for P_PGID + pid==0, and plausible that there are
> > > applications that depend on this. We could emulate the EINVAL case in
> > > userspace, but assigning weird semantics to special cases is just a
> > > mess of potential future headaches when it would be trivial to do it
> > > right. And doing it right would also make programming userspace side
> > > easier.
> >
> > Since this is a POSIX interface and shared between many unices I took
> > at look at a couple to see what they have done.  If possible it is
> > important to be able to write programs that work the same across
> > different unices.
> >
> > The BSDs implemented wait6 as their common wait that does everything
> > system call.   It's large contribution is that wait6 will give both
> > the rusage of the child that exited and it's children separately.
> 
> Ah right, that was one more thing I forgot on my earlier list. If we end up
> creating a new wait4()/waitid() replacement  to replace the timeval
> with __kernel_timespec, we may want that to return a wrusage instead
> of rusage. Same for Christian's proposed pidfd_wait().

So I'm in the middle of writing tests for pidfd_wait. And I'm currently
using struct rusage. I would suggest sending out a first version using
struct rusage and then you can point out what you would like see as a
replacement. What do you think, Arnd?

Christian

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Arnd Bergmann]

On Tue, Jul 23, 2019 at 10:29 AM Christian Brauner <christian@brauner.io> wrote:
> On Tue, Jul 23, 2019 at 10:12:07AM +0200, Arnd Bergmann wrote:
> > On Tue, Jul 23, 2019 at 2:01 AM Eric W. Biederman <ebiederm@xmission.com> wrote:
> > > The BSDs implemented wait6 as their common wait that does everything
> > > system call.   It's large contribution is that wait6 will give both
> > > the rusage of the child that exited and it's children separately.
> >
> > Ah right, that was one more thing I forgot on my earlier list. If we end up
> > creating a new wait4()/waitid() replacement  to replace the timeval
> > with __kernel_timespec, we may want that to return a wrusage instead
> > of rusage. Same for Christian's proposed pidfd_wait().
>
> So I'm in the middle of writing tests for pidfd_wait. And I'm currently
> using struct rusage. I would suggest sending out a first version using
> struct rusage and then you can point out what you would like see as a
> replacement. What do you think, Arnd?

Sounds good to me, the debate over what rusage to use should not hold
up the review of the rest of that syscall.

       Arnd

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Alistair Francis]

On Tue, Jul 23, 2019 at 1:45 AM Arnd Bergmann <arnd@arndb.de> wrote:
>
> On Tue, Jul 23, 2019 at 10:29 AM Christian Brauner <christian@brauner.io> wrote:
> > On Tue, Jul 23, 2019 at 10:12:07AM +0200, Arnd Bergmann wrote:
> > > On Tue, Jul 23, 2019 at 2:01 AM Eric W. Biederman <ebiederm@xmission.com> wrote:
> > > > The BSDs implemented wait6 as their common wait that does everything
> > > > system call.   It's large contribution is that wait6 will give both
> > > > the rusage of the child that exited and it's children separately.
> > >
> > > Ah right, that was one more thing I forgot on my earlier list. If we end up
> > > creating a new wait4()/waitid() replacement  to replace the timeval
> > > with __kernel_timespec, we may want that to return a wrusage instead
> > > of rusage. Same for Christian's proposed pidfd_wait().
> >
> > So I'm in the middle of writing tests for pidfd_wait. And I'm currently
> > using struct rusage. I would suggest sending out a first version using
> > struct rusage and then you can point out what you would like see as a
> > replacement. What do you think, Arnd?
>
> Sounds good to me, the debate over what rusage to use should not hold
> up the review of the rest of that syscall.

I'm unclear what the final decision is here. What is the solution are
we going to have wait4() or add P_PROCESS_PGID to waitid()?

As well as that what is the solution to current implementations? If we
add wait4() then there isn't an issue (and I can drop this patch) but
if we add P_PROCESS_PGID then we will need a way to handle kernels
with waitid() but no P_PROCESS_PGID. Although my new plan is to only
use the waitid syscall if we don't have waitpid or wait4 so it seems
like this will only affect RV32 for the time being.

Alistair

>
>        Arnd

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Rich Felker]

On Wed, Jul 24, 2019 at 05:04:53PM -0700, Alistair Francis wrote:
> On Tue, Jul 23, 2019 at 1:45 AM Arnd Bergmann <arnd@arndb.de> wrote:
> >
> > On Tue, Jul 23, 2019 at 10:29 AM Christian Brauner <christian@brauner.io> wrote:
> > > On Tue, Jul 23, 2019 at 10:12:07AM +0200, Arnd Bergmann wrote:
> > > > On Tue, Jul 23, 2019 at 2:01 AM Eric W. Biederman <ebiederm@xmission.com> wrote:
> > > > > The BSDs implemented wait6 as their common wait that does everything
> > > > > system call.   It's large contribution is that wait6 will give both
> > > > > the rusage of the child that exited and it's children separately.
> > > >
> > > > Ah right, that was one more thing I forgot on my earlier list. If we end up
> > > > creating a new wait4()/waitid() replacement  to replace the timeval
> > > > with __kernel_timespec, we may want that to return a wrusage instead
> > > > of rusage. Same for Christian's proposed pidfd_wait().
> > >
> > > So I'm in the middle of writing tests for pidfd_wait. And I'm currently
> > > using struct rusage. I would suggest sending out a first version using
> > > struct rusage and then you can point out what you would like see as a
> > > replacement. What do you think, Arnd?
> >
> > Sounds good to me, the debate over what rusage to use should not hold
> > up the review of the rest of that syscall.
> 
> I'm unclear what the final decision is here. What is the solution are
> we going to have wait4() or add P_PROCESS_PGID to waitid()?
> 
> As well as that what is the solution to current implementations? If we
> add wait4() then there isn't an issue (and I can drop this patch) but
> if we add P_PROCESS_PGID then we will need a way to handle kernels
> with waitid() but no P_PROCESS_PGID. Although my new plan is to only
> use the waitid syscall if we don't have waitpid or wait4 so it seems
> like this will only affect RV32 for the time being.

I would really like some indication which solution will be taken,
since it impacts choices that will need to be made in musl very soon.
My favorite outcome would be bringing back wait4 for rv32 (and
no-time32 archs in general) *and* adding P_PROCESS_PGID. In the short
term, just using wait4 would be the simplest and cleanest for us (same
as all other archs, no extra case to deal with), but in the long term
there may be value in having rusage that can represent more than 68
cpu-years spent by a process (seems plausible with large numbers of
cores).

Rich

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Arnd Bergmann]

On Thu, Jul 25, 2019 at 6:40 AM Rich Felker <dalias@libc.org> wrote:
> On Wed, Jul 24, 2019 at 05:04:53PM -0700, Alistair Francis wrote:
> > On Tue, Jul 23, 2019 at 1:45 AM Arnd Bergmann <arnd@arndb.de> wrote:
> > >
> > > Sounds good to me, the debate over what rusage to use should not hold
> > > up the review of the rest of that syscall.
> >
> > I'm unclear what the final decision is here. What is the solution are
> > we going to have wait4() or add P_PROCESS_PGID to waitid()?
> >
> > As well as that what is the solution to current implementations? If we
> > add wait4() then there isn't an issue (and I can drop this patch) but
> > if we add P_PROCESS_PGID then we will need a way to handle kernels
> > with waitid() but no P_PROCESS_PGID. Although my new plan is to only
> > use the waitid syscall if we don't have waitpid or wait4 so it seems
> > like this will only affect RV32 for the time being.
>
> I would really like some indication which solution will be taken,
> since it impacts choices that will need to be made in musl very soon.
> My favorite outcome would be bringing back wait4 for rv32 (and
> no-time32 archs in general) *and* adding P_PROCESS_PGID. In the short
> term, just using wait4 would be the simplest and cleanest for us (same
> as all other archs, no extra case to deal with), but in the long term
> there may be value in having rusage that can represent more than 68
> cpu-years spent by a process (seems plausible with large numbers of
> cores).

Based on the feedback from Linus and Eric, the most likely outcome
at the moment seems to be an extension of waitid() to allow
P_PGID with id=0 like BSD does, and not bring back wait4() or
add P_PROCESS_PGID.

So far, I don't think anyone has proposed an actual kernel patch.
I was hoping that Eric would do it, but I could also send it if he's
otherwise busy.

      Arnd

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Christian Brauner]

On Thu, Jul 25, 2019 at 03:15:35PM +0200, Arnd Bergmann wrote:
> On Thu, Jul 25, 2019 at 6:40 AM Rich Felker <dalias@libc.org> wrote:
> > On Wed, Jul 24, 2019 at 05:04:53PM -0700, Alistair Francis wrote:
> > > On Tue, Jul 23, 2019 at 1:45 AM Arnd Bergmann <arnd@arndb.de> wrote:
> > > >
> > > > Sounds good to me, the debate over what rusage to use should not hold
> > > > up the review of the rest of that syscall.
> > >
> > > I'm unclear what the final decision is here. What is the solution are
> > > we going to have wait4() or add P_PROCESS_PGID to waitid()?
> > >
> > > As well as that what is the solution to current implementations? If we
> > > add wait4() then there isn't an issue (and I can drop this patch) but
> > > if we add P_PROCESS_PGID then we will need a way to handle kernels
> > > with waitid() but no P_PROCESS_PGID. Although my new plan is to only
> > > use the waitid syscall if we don't have waitpid or wait4 so it seems
> > > like this will only affect RV32 for the time being.
> >
> > I would really like some indication which solution will be taken,
> > since it impacts choices that will need to be made in musl very soon.
> > My favorite outcome would be bringing back wait4 for rv32 (and
> > no-time32 archs in general) *and* adding P_PROCESS_PGID. In the short
> > term, just using wait4 would be the simplest and cleanest for us (same
> > as all other archs, no extra case to deal with), but in the long term
> > there may be value in having rusage that can represent more than 68
> > cpu-years spent by a process (seems plausible with large numbers of
> > cores).
> 
> Based on the feedback from Linus and Eric, the most likely outcome
> at the moment seems to be an extension of waitid() to allow
> P_PGID with id=0 like BSD does, and not bring back wait4() or
> add P_PROCESS_PGID.
> 
> So far, I don't think anyone has proposed an actual kernel patch.
> I was hoping that Eric would do it, but I could also send it if he's
> otherwise busy.

I'm touching waitid() right now. I can pick this up and put this in
there too.

Christian

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Eric W. Biederman]

Arnd Bergmann <arnd@arndb.de> writes:

> On Thu, Jul 25, 2019 at 6:40 AM Rich Felker <dalias@libc.org> wrote:
>> On Wed, Jul 24, 2019 at 05:04:53PM -0700, Alistair Francis wrote:
>> > On Tue, Jul 23, 2019 at 1:45 AM Arnd Bergmann <arnd@arndb.de> wrote:
>> > >
>> > > Sounds good to me, the debate over what rusage to use should not hold
>> > > up the review of the rest of that syscall.
>> >
>> > I'm unclear what the final decision is here. What is the solution are
>> > we going to have wait4() or add P_PROCESS_PGID to waitid()?
>> >
>> > As well as that what is the solution to current implementations? If we
>> > add wait4() then there isn't an issue (and I can drop this patch) but
>> > if we add P_PROCESS_PGID then we will need a way to handle kernels
>> > with waitid() but no P_PROCESS_PGID. Although my new plan is to only
>> > use the waitid syscall if we don't have waitpid or wait4 so it seems
>> > like this will only affect RV32 for the time being.
>>
>> I would really like some indication which solution will be taken,
>> since it impacts choices that will need to be made in musl very soon.
>> My favorite outcome would be bringing back wait4 for rv32 (and
>> no-time32 archs in general) *and* adding P_PROCESS_PGID. In the short
>> term, just using wait4 would be the simplest and cleanest for us (same
>> as all other archs, no extra case to deal with), but in the long term
>> there may be value in having rusage that can represent more than 68
>> cpu-years spent by a process (seems plausible with large numbers of
>> cores).
>
> Based on the feedback from Linus and Eric, the most likely outcome
> at the moment seems to be an extension of waitid() to allow
> P_PGID with id=0 like BSD does, and not bring back wait4() or
> add P_PROCESS_PGID.
>
> So far, I don't think anyone has proposed an actual kernel patch.
> I was hoping that Eric would do it, but I could also send it if he's
> otherwise busy.

So here is what I am looking at.  It still needs to be tested
and the description needs to be improved so that it properly credits
everyone.  However I think I have the major stroeks correct.

From: "Eric W. Biederman" <ebiederm@xmission.com>
Date: Tue, 23 Jul 2019 07:44:46 -0500
Subject: [PATCH] waitid: Add support for waiting for the current process group

It was recently discovered that the linux version of waitid is not a
superset of the other wait functions because it does not include
support for waiting for the current process group.  This has two
downsides.  An extra system call is needed to get the current process
group, and a signal could come in between the system call that
retrieved the process gorup and the call to waitid that changes the
current process group.

Allow userspace to avoid both of those issues by defining
idtype == P_PGID and id == 0 to mean wait for the caller's process
group at the time of the call.

Arguments can be made for using a different choice of idtype and id
for this case but the BSDs already use this P_PGID and 0 to indicate
waiting for the current process's process group.  So be nice to user
space programmers and don't introduce an unnecessary incompatibility.

Some people have noted that the posix description is that
waitpid will wait for the current process group, and that in
the presence of pthreads that process group can change.  To get
clarity on this issue I looked at XNU, FreeBSD, and Luminos.  All of
those flavors of unix waited for the current process group at the
time of call and as written could not adapt to the process group
changing after the call.

At one point Linux did adapt to the current process group changing but
that stopped in 161550d74c07 ("pid: sys_wait... fixes").  It has been
over 11 years since Linux has that behavior, no programs that fail
with the change in behavior have been reported, and I could not
find any other unix that does this.  So I think it is safe to clarify
the definition of current process group, to current process group
at the time of the wait function.

Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
---
 kernel/exit.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/kernel/exit.c b/kernel/exit.c
index a75b6a7f458a..3d86930f035e 100644
--- a/kernel/exit.c
+++ b/kernel/exit.c
@@ -1577,14 +1577,16 @@ static long kernel_waitid(int which, pid_t upid, struct waitid_info *infop,
 		break;
 	case P_PGID:
 		type = PIDTYPE_PGID;
-		if (upid <= 0)
+		if (upid < 0)
 			return -EINVAL;
+		if (upid == 0)
+			pid = get_pid(task_pgrp(current));
 		break;
 	default:
 		return -EINVAL;
 	}
 
-	if (type < PIDTYPE_MAX)
+	if ((type < PIDTYPE_MAX) && !pid)
 		pid = find_get_pid(upid);
 
 	wo.wo_type	= type;
-- 
2.21.0.dirty

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Christian Brauner]

On July 25, 2019 7:14:05 PM GMT+02:00, ebiederm@xmission.com wrote:
>Arnd Bergmann <arnd@arndb.de> writes:
>
>> On Thu, Jul 25, 2019 at 6:40 AM Rich Felker <dalias@libc.org> wrote:
>>> On Wed, Jul 24, 2019 at 05:04:53PM -0700, Alistair Francis wrote:
>>> > On Tue, Jul 23, 2019 at 1:45 AM Arnd Bergmann <arnd@arndb.de>
>wrote:
>>> > >
>>> > > Sounds good to me, the debate over what rusage to use should not
>hold
>>> > > up the review of the rest of that syscall.
>>> >
>>> > I'm unclear what the final decision is here. What is the solution
>are
>>> > we going to have wait4() or add P_PROCESS_PGID to waitid()?
>>> >
>>> > As well as that what is the solution to current implementations?
>If we
>>> > add wait4() then there isn't an issue (and I can drop this patch)
>but
>>> > if we add P_PROCESS_PGID then we will need a way to handle kernels
>>> > with waitid() but no P_PROCESS_PGID. Although my new plan is to
>only
>>> > use the waitid syscall if we don't have waitpid or wait4 so it
>seems
>>> > like this will only affect RV32 for the time being.
>>>
>>> I would really like some indication which solution will be taken,
>>> since it impacts choices that will need to be made in musl very
>soon.
>>> My favorite outcome would be bringing back wait4 for rv32 (and
>>> no-time32 archs in general) *and* adding P_PROCESS_PGID. In the
>short
>>> term, just using wait4 would be the simplest and cleanest for us
>(same
>>> as all other archs, no extra case to deal with), but in the long
>term
>>> there may be value in having rusage that can represent more than 68
>>> cpu-years spent by a process (seems plausible with large numbers of
>>> cores).
>>
>> Based on the feedback from Linus and Eric, the most likely outcome
>> at the moment seems to be an extension of waitid() to allow
>> P_PGID with id=0 like BSD does, and not bring back wait4() or
>> add P_PROCESS_PGID.
>>
>> So far, I don't think anyone has proposed an actual kernel patch.
>> I was hoping that Eric would do it, but I could also send it if he's
>> otherwise busy.
>
>So here is what I am looking at.  It still needs to be tested
>and the description needs to be improved so that it properly credits
>everyone.  However I think I have the major stroeks correct.
>
>From: "Eric W. Biederman" <ebiederm@xmission.com>
>Date: Tue, 23 Jul 2019 07:44:46 -0500
>Subject: [PATCH] waitid: Add support for waiting for the current
>process group
>
>It was recently discovered that the linux version of waitid is not a
>superset of the other wait functions because it does not include
>support for waiting for the current process group.  This has two
>downsides.  An extra system call is needed to get the current process
>group, and a signal could come in between the system call that
>retrieved the process gorup and the call to waitid that changes the
>current process group.
>
>Allow userspace to avoid both of those issues by defining
>idtype == P_PGID and id == 0 to mean wait for the caller's process
>group at the time of the call.
>
>Arguments can be made for using a different choice of idtype and id
>for this case but the BSDs already use this P_PGID and 0 to indicate
>waiting for the current process's process group.  So be nice to user
>space programmers and don't introduce an unnecessary incompatibility.
>
>Some people have noted that the posix description is that
>waitpid will wait for the current process group, and that in
>the presence of pthreads that process group can change.  To get
>clarity on this issue I looked at XNU, FreeBSD, and Luminos.  All of
>those flavors of unix waited for the current process group at the
>time of call and as written could not adapt to the process group
>changing after the call.
>
>At one point Linux did adapt to the current process group changing but
>that stopped in 161550d74c07 ("pid: sys_wait... fixes").  It has been
>over 11 years since Linux has that behavior, no programs that fail
>with the change in behavior have been reported, and I could not
>find any other unix that does this.  So I think it is safe to clarify
>the definition of current process group, to current process group
>at the time of the wait function.
>
>Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
>---
> kernel/exit.c | 6 ++++--
> 1 file changed, 4 insertions(+), 2 deletions(-)
>
>diff --git a/kernel/exit.c b/kernel/exit.c
>index a75b6a7f458a..3d86930f035e 100644
>--- a/kernel/exit.c
>+++ b/kernel/exit.c
>@@ -1577,14 +1577,16 @@ static long kernel_waitid(int which, pid_t
>upid, struct waitid_info *infop,
> 		break;
> 	case P_PGID:
> 		type = PIDTYPE_PGID;
>-		if (upid <= 0)
>+		if (upid < 0)
> 			return -EINVAL;
>+		if (upid == 0)
>+			pid = get_pid(task_pgrp(current));
> 		break;
> 	default:
> 		return -EINVAL;
> 	}
> 
>-	if (type < PIDTYPE_MAX)
>+	if ((type < PIDTYPE_MAX) && !pid)
> 		pid = find_get_pid(upid);
> 
> 	wo.wo_type	= type;

Eric, mind if I send this out alongside the P_PIDFD patchset and put in a test for it?

Christian

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Alistair Francis]

On Thu, Jul 25, 2019 at 10:30 AM Christian Brauner <christian@brauner.io> wrote:
>
> On July 25, 2019 7:14:05 PM GMT+02:00, ebiederm@xmission.com wrote:
> >Arnd Bergmann <arnd@arndb.de> writes:
> >
> >> On Thu, Jul 25, 2019 at 6:40 AM Rich Felker <dalias@libc.org> wrote:
> >>> On Wed, Jul 24, 2019 at 05:04:53PM -0700, Alistair Francis wrote:
> >>> > On Tue, Jul 23, 2019 at 1:45 AM Arnd Bergmann <arnd@arndb.de>
> >wrote:
> >>> > >
> >>> > > Sounds good to me, the debate over what rusage to use should not
> >hold
> >>> > > up the review of the rest of that syscall.
> >>> >
> >>> > I'm unclear what the final decision is here. What is the solution
> >are
> >>> > we going to have wait4() or add P_PROCESS_PGID to waitid()?
> >>> >
> >>> > As well as that what is the solution to current implementations?
> >If we
> >>> > add wait4() then there isn't an issue (and I can drop this patch)
> >but
> >>> > if we add P_PROCESS_PGID then we will need a way to handle kernels
> >>> > with waitid() but no P_PROCESS_PGID. Although my new plan is to
> >only
> >>> > use the waitid syscall if we don't have waitpid or wait4 so it
> >seems
> >>> > like this will only affect RV32 for the time being.
> >>>
> >>> I would really like some indication which solution will be taken,
> >>> since it impacts choices that will need to be made in musl very
> >soon.
> >>> My favorite outcome would be bringing back wait4 for rv32 (and
> >>> no-time32 archs in general) *and* adding P_PROCESS_PGID. In the
> >short
> >>> term, just using wait4 would be the simplest and cleanest for us
> >(same
> >>> as all other archs, no extra case to deal with), but in the long
> >term
> >>> there may be value in having rusage that can represent more than 68
> >>> cpu-years spent by a process (seems plausible with large numbers of
> >>> cores).
> >>
> >> Based on the feedback from Linus and Eric, the most likely outcome
> >> at the moment seems to be an extension of waitid() to allow
> >> P_PGID with id=0 like BSD does, and not bring back wait4() or
> >> add P_PROCESS_PGID.
> >>
> >> So far, I don't think anyone has proposed an actual kernel patch.
> >> I was hoping that Eric would do it, but I could also send it if he's
> >> otherwise busy.
> >
> >So here is what I am looking at.  It still needs to be tested
> >and the description needs to be improved so that it properly credits
> >everyone.  However I think I have the major stroeks correct.
> >
> >From: "Eric W. Biederman" <ebiederm@xmission.com>
> >Date: Tue, 23 Jul 2019 07:44:46 -0500
> >Subject: [PATCH] waitid: Add support for waiting for the current
> >process group
> >
> >It was recently discovered that the linux version of waitid is not a
> >superset of the other wait functions because it does not include
> >support for waiting for the current process group.  This has two
> >downsides.  An extra system call is needed to get the current process
> >group, and a signal could come in between the system call that
> >retrieved the process gorup and the call to waitid that changes the
> >current process group.
> >
> >Allow userspace to avoid both of those issues by defining
> >idtype == P_PGID and id == 0 to mean wait for the caller's process
> >group at the time of the call.
> >
> >Arguments can be made for using a different choice of idtype and id
> >for this case but the BSDs already use this P_PGID and 0 to indicate
> >waiting for the current process's process group.  So be nice to user
> >space programmers and don't introduce an unnecessary incompatibility.
> >
> >Some people have noted that the posix description is that
> >waitpid will wait for the current process group, and that in
> >the presence of pthreads that process group can change.  To get
> >clarity on this issue I looked at XNU, FreeBSD, and Luminos.  All of
> >those flavors of unix waited for the current process group at the
> >time of call and as written could not adapt to the process group
> >changing after the call.
> >
> >At one point Linux did adapt to the current process group changing but
> >that stopped in 161550d74c07 ("pid: sys_wait... fixes").  It has been
> >over 11 years since Linux has that behavior, no programs that fail
> >with the change in behavior have been reported, and I could not
> >find any other unix that does this.  So I think it is safe to clarify
> >the definition of current process group, to current process group
> >at the time of the wait function.
> >
> >Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
> >---
> > kernel/exit.c | 6 ++++--
> > 1 file changed, 4 insertions(+), 2 deletions(-)
> >
> >diff --git a/kernel/exit.c b/kernel/exit.c
> >index a75b6a7f458a..3d86930f035e 100644
> >--- a/kernel/exit.c
> >+++ b/kernel/exit.c
> >@@ -1577,14 +1577,16 @@ static long kernel_waitid(int which, pid_t
> >upid, struct waitid_info *infop,
> >               break;
> >       case P_PGID:
> >               type = PIDTYPE_PGID;
> >-              if (upid <= 0)
> >+              if (upid < 0)
> >                       return -EINVAL;
> >+              if (upid == 0)
> >+                      pid = get_pid(task_pgrp(current));
> >               break;
> >       default:
> >               return -EINVAL;
> >       }
> >
> >-      if (type < PIDTYPE_MAX)
> >+      if ((type < PIDTYPE_MAX) && !pid)
> >               pid = find_get_pid(upid);
> >
> >       wo.wo_type      = type;
>
> Eric, mind if I send this out alongside the P_PIDFD patchset and put in a test for it?

Was this ever sent?

Alistair

>
> Christian

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Rich Felker]

On Tue, Aug 13, 2019 at 03:22:02PM -0700, Alistair Francis wrote:
> On Thu, Jul 25, 2019 at 10:30 AM Christian Brauner <christian@brauner.io> wrote:
> >
> > On July 25, 2019 7:14:05 PM GMT+02:00, ebiederm@xmission.com wrote:
> > >Arnd Bergmann <arnd@arndb.de> writes:
> > >
> > >> On Thu, Jul 25, 2019 at 6:40 AM Rich Felker <dalias@libc.org> wrote:
> > >>> On Wed, Jul 24, 2019 at 05:04:53PM -0700, Alistair Francis wrote:
> > >>> > On Tue, Jul 23, 2019 at 1:45 AM Arnd Bergmann <arnd@arndb.de>
> > >wrote:
> > >>> > >
> > >>> > > Sounds good to me, the debate over what rusage to use should not
> > >hold
> > >>> > > up the review of the rest of that syscall.
> > >>> >
> > >>> > I'm unclear what the final decision is here. What is the solution
> > >are
> > >>> > we going to have wait4() or add P_PROCESS_PGID to waitid()?
> > >>> >
> > >>> > As well as that what is the solution to current implementations?
> > >If we
> > >>> > add wait4() then there isn't an issue (and I can drop this patch)
> > >but
> > >>> > if we add P_PROCESS_PGID then we will need a way to handle kernels
> > >>> > with waitid() but no P_PROCESS_PGID. Although my new plan is to
> > >only
> > >>> > use the waitid syscall if we don't have waitpid or wait4 so it
> > >seems
> > >>> > like this will only affect RV32 for the time being.
> > >>>
> > >>> I would really like some indication which solution will be taken,
> > >>> since it impacts choices that will need to be made in musl very
> > >soon.
> > >>> My favorite outcome would be bringing back wait4 for rv32 (and
> > >>> no-time32 archs in general) *and* adding P_PROCESS_PGID. In the
> > >short
> > >>> term, just using wait4 would be the simplest and cleanest for us
> > >(same
> > >>> as all other archs, no extra case to deal with), but in the long
> > >term
> > >>> there may be value in having rusage that can represent more than 68
> > >>> cpu-years spent by a process (seems plausible with large numbers of
> > >>> cores).
> > >>
> > >> Based on the feedback from Linus and Eric, the most likely outcome
> > >> at the moment seems to be an extension of waitid() to allow
> > >> P_PGID with id=0 like BSD does, and not bring back wait4() or
> > >> add P_PROCESS_PGID.
> > >>
> > >> So far, I don't think anyone has proposed an actual kernel patch.
> > >> I was hoping that Eric would do it, but I could also send it if he's
> > >> otherwise busy.
> > >
> > >So here is what I am looking at.  It still needs to be tested
> > >and the description needs to be improved so that it properly credits
> > >everyone.  However I think I have the major stroeks correct.
> > >
> > >From: "Eric W. Biederman" <ebiederm@xmission.com>
> > >Date: Tue, 23 Jul 2019 07:44:46 -0500
> > >Subject: [PATCH] waitid: Add support for waiting for the current
> > >process group
> > >
> > >It was recently discovered that the linux version of waitid is not a
> > >superset of the other wait functions because it does not include
> > >support for waiting for the current process group.  This has two
> > >downsides.  An extra system call is needed to get the current process
> > >group, and a signal could come in between the system call that
> > >retrieved the process gorup and the call to waitid that changes the
> > >current process group.
> > >
> > >Allow userspace to avoid both of those issues by defining
> > >idtype == P_PGID and id == 0 to mean wait for the caller's process
> > >group at the time of the call.
> > >
> > >Arguments can be made for using a different choice of idtype and id
> > >for this case but the BSDs already use this P_PGID and 0 to indicate
> > >waiting for the current process's process group.  So be nice to user
> > >space programmers and don't introduce an unnecessary incompatibility.
> > >
> > >Some people have noted that the posix description is that
> > >waitpid will wait for the current process group, and that in
> > >the presence of pthreads that process group can change.  To get
> > >clarity on this issue I looked at XNU, FreeBSD, and Luminos.  All of
> > >those flavors of unix waited for the current process group at the
> > >time of call and as written could not adapt to the process group
> > >changing after the call.
> > >
> > >At one point Linux did adapt to the current process group changing but
> > >that stopped in 161550d74c07 ("pid: sys_wait... fixes").  It has been
> > >over 11 years since Linux has that behavior, no programs that fail
> > >with the change in behavior have been reported, and I could not
> > >find any other unix that does this.  So I think it is safe to clarify
> > >the definition of current process group, to current process group
> > >at the time of the wait function.
> > >
> > >Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
> > >---
> > > kernel/exit.c | 6 ++++--
> > > 1 file changed, 4 insertions(+), 2 deletions(-)
> > >
> > >diff --git a/kernel/exit.c b/kernel/exit.c
> > >index a75b6a7f458a..3d86930f035e 100644
> > >--- a/kernel/exit.c
> > >+++ b/kernel/exit.c
> > >@@ -1577,14 +1577,16 @@ static long kernel_waitid(int which, pid_t
> > >upid, struct waitid_info *infop,
> > >               break;
> > >       case P_PGID:
> > >               type = PIDTYPE_PGID;
> > >-              if (upid <= 0)
> > >+              if (upid < 0)
> > >                       return -EINVAL;
> > >+              if (upid == 0)
> > >+                      pid = get_pid(task_pgrp(current));
> > >               break;
> > >       default:
> > >               return -EINVAL;
> > >       }
> > >
> > >-      if (type < PIDTYPE_MAX)
> > >+      if ((type < PIDTYPE_MAX) && !pid)
> > >               pid = find_get_pid(upid);
> > >
> > >       wo.wo_type      = type;
> >
> > Eric, mind if I send this out alongside the P_PIDFD patchset and put in a test for it?
> 
> Was this ever sent?

It's not upstream. Would be really nice to get this fixed in 5.3 so
that RV32 will not be missing functionality...

Rich

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Christian Brauner]

On August 14, 2019 1:11:53 AM GMT+02:00, Rich Felker <dalias@libc.org> wrote:
>On Tue, Aug 13, 2019 at 03:22:02PM -0700, Alistair Francis wrote:
>> On Thu, Jul 25, 2019 at 10:30 AM Christian Brauner
><christian@brauner.io> wrote:
>> >
>> > On July 25, 2019 7:14:05 PM GMT+02:00, ebiederm@xmission.com wrote:
>> > >Arnd Bergmann <arnd@arndb.de> writes:
>> > >
>> > >> On Thu, Jul 25, 2019 at 6:40 AM Rich Felker <dalias@libc.org>
>wrote:
>> > >>> On Wed, Jul 24, 2019 at 05:04:53PM -0700, Alistair Francis
>wrote:
>> > >>> > On Tue, Jul 23, 2019 at 1:45 AM Arnd Bergmann <arnd@arndb.de>
>> > >wrote:
>> > >>> > >
>> > >>> > > Sounds good to me, the debate over what rusage to use
>should not
>> > >hold
>> > >>> > > up the review of the rest of that syscall.
>> > >>> >
>> > >>> > I'm unclear what the final decision is here. What is the
>solution
>> > >are
>> > >>> > we going to have wait4() or add P_PROCESS_PGID to waitid()?
>> > >>> >
>> > >>> > As well as that what is the solution to current
>implementations?
>> > >If we
>> > >>> > add wait4() then there isn't an issue (and I can drop this
>patch)
>> > >but
>> > >>> > if we add P_PROCESS_PGID then we will need a way to handle
>kernels
>> > >>> > with waitid() but no P_PROCESS_PGID. Although my new plan is
>to
>> > >only
>> > >>> > use the waitid syscall if we don't have waitpid or wait4 so
>it
>> > >seems
>> > >>> > like this will only affect RV32 for the time being.
>> > >>>
>> > >>> I would really like some indication which solution will be
>taken,
>> > >>> since it impacts choices that will need to be made in musl very
>> > >soon.
>> > >>> My favorite outcome would be bringing back wait4 for rv32 (and
>> > >>> no-time32 archs in general) *and* adding P_PROCESS_PGID. In the
>> > >short
>> > >>> term, just using wait4 would be the simplest and cleanest for
>us
>> > >(same
>> > >>> as all other archs, no extra case to deal with), but in the
>long
>> > >term
>> > >>> there may be value in having rusage that can represent more
>than 68
>> > >>> cpu-years spent by a process (seems plausible with large
>numbers of
>> > >>> cores).
>> > >>
>> > >> Based on the feedback from Linus and Eric, the most likely
>outcome
>> > >> at the moment seems to be an extension of waitid() to allow
>> > >> P_PGID with id=0 like BSD does, and not bring back wait4() or
>> > >> add P_PROCESS_PGID.
>> > >>
>> > >> So far, I don't think anyone has proposed an actual kernel
>patch.
>> > >> I was hoping that Eric would do it, but I could also send it if
>he's
>> > >> otherwise busy.
>> > >
>> > >So here is what I am looking at.  It still needs to be tested
>> > >and the description needs to be improved so that it properly
>credits
>> > >everyone.  However I think I have the major stroeks correct.
>> > >
>> > >From: "Eric W. Biederman" <ebiederm@xmission.com>
>> > >Date: Tue, 23 Jul 2019 07:44:46 -0500
>> > >Subject: [PATCH] waitid: Add support for waiting for the current
>> > >process group
>> > >
>> > >It was recently discovered that the linux version of waitid is not
>a
>> > >superset of the other wait functions because it does not include
>> > >support for waiting for the current process group.  This has two
>> > >downsides.  An extra system call is needed to get the current
>process
>> > >group, and a signal could come in between the system call that
>> > >retrieved the process gorup and the call to waitid that changes
>the
>> > >current process group.
>> > >
>> > >Allow userspace to avoid both of those issues by defining
>> > >idtype == P_PGID and id == 0 to mean wait for the caller's process
>> > >group at the time of the call.
>> > >
>> > >Arguments can be made for using a different choice of idtype and
>id
>> > >for this case but the BSDs already use this P_PGID and 0 to
>indicate
>> > >waiting for the current process's process group.  So be nice to
>user
>> > >space programmers and don't introduce an unnecessary
>incompatibility.
>> > >
>> > >Some people have noted that the posix description is that
>> > >waitpid will wait for the current process group, and that in
>> > >the presence of pthreads that process group can change.  To get
>> > >clarity on this issue I looked at XNU, FreeBSD, and Luminos.  All
>of
>> > >those flavors of unix waited for the current process group at the
>> > >time of call and as written could not adapt to the process group
>> > >changing after the call.
>> > >
>> > >At one point Linux did adapt to the current process group changing
>but
>> > >that stopped in 161550d74c07 ("pid: sys_wait... fixes").  It has
>been
>> > >over 11 years since Linux has that behavior, no programs that fail
>> > >with the change in behavior have been reported, and I could not
>> > >find any other unix that does this.  So I think it is safe to
>clarify
>> > >the definition of current process group, to current process group
>> > >at the time of the wait function.
>> > >
>> > >Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
>> > >---
>> > > kernel/exit.c | 6 ++++--
>> > > 1 file changed, 4 insertions(+), 2 deletions(-)
>> > >
>> > >diff --git a/kernel/exit.c b/kernel/exit.c
>> > >index a75b6a7f458a..3d86930f035e 100644
>> > >--- a/kernel/exit.c
>> > >+++ b/kernel/exit.c
>> > >@@ -1577,14 +1577,16 @@ static long kernel_waitid(int which, pid_t
>> > >upid, struct waitid_info *infop,
>> > >               break;
>> > >       case P_PGID:
>> > >               type = PIDTYPE_PGID;
>> > >-              if (upid <= 0)
>> > >+              if (upid < 0)
>> > >                       return -EINVAL;
>> > >+              if (upid == 0)
>> > >+                      pid = get_pid(task_pgrp(current));
>> > >               break;
>> > >       default:
>> > >               return -EINVAL;
>> > >       }
>> > >
>> > >-      if (type < PIDTYPE_MAX)
>> > >+      if ((type < PIDTYPE_MAX) && !pid)
>> > >               pid = find_get_pid(upid);
>> > >
>> > >       wo.wo_type      = type;
>> >
>> > Eric, mind if I send this out alongside the P_PIDFD patchset and
>put in a test for it?
>> 
>> Was this ever sent?
>
>It's not upstream. Would be really nice to get this fixed in 5.3 so
>that RV32 will not be missing functionality...
>
>Rich

I'll pick it up as is and send out for review later if I hear no objections.

Christian

[PATCH v1 0/1] waitid: process group enhancement

[christian.brauner]

From: Christian Brauner <christian.brauner@ubuntu.com>

Hey everyone,

This patch adds support for waiting on the current process group by
specifying waitid(P_PGID, 0, ...) as discussed in [1]. The details why
we need to do this are in the commit message of [PATCH 1/1] so I won't
repeat them here.

I've picked this up since the thread has gone stale and parts of
userspace are actually blocked by this.

Note that the patch has been marked with v1 because I've changed the
patch to be more closely aligned with the P_PIDFD changes to waitid() I
have sitting in my for-next branch (cf. [2]).
This makes the merge conflict a little simpler and picks up on the
coding style discussions that guided the P_PIDFD patchset.

There was some desire to get this feature in with 5.3 (cf. [3]).
But given that this is a new feature for waitid() and for the sake of
avoiding any merge conflicts I would prefer to land this in the 5.4
merge window together with the P_PIDFD changes.

Thanks!
Christian

/* References */
[1]: https://www.sourceware.org/ml/libc-alpha/2019-07/msg00587.html
[2]: https://lore.kernel.org/lkml/20190727222229.6516-1-christian@brauner.io/
[3]: https://www.sourceware.org/ml/libc-alpha/2019-08/msg00304.html

Eric W. Biederman (1):
  waitid: Add support for waiting for the current process group

 kernel/exit.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

-- 
2.22.0

[PATCH v1 1/1] waitid: Add support for waiting for the current process group

[christian.brauner]

From: "Eric W. Biederman" <ebiederm@xmission.com>

It was recently discovered that the linux version of waitid is not a
superset of the other wait functions because it does not include
support for waiting for the current process group.  This has two
downsides.  An extra system call is needed to get the current process
group, and a signal could come in between the system call that
retrieved the process gorup and the call to waitid that changes the
current process group.

Allow userspace to avoid both of those issues by defining
idtype == P_PGID and id == 0 to mean wait for the caller's process
group at the time of the call.

Arguments can be made for using a different choice of idtype and id
for this case but the BSDs already use this P_PGID and 0 to indicate
waiting for the current process's process group.  So be nice to user
space programmers and don't introduce an unnecessary incompatibility.

Some people have noted that the posix description is that
waitpid will wait for the current process group, and that in
the presence of pthreads that process group can change.  To get
clarity on this issue I looked at XNU, FreeBSD, and Luminos.  All of
those flavors of unix waited for the current process group at the
time of call and as written could not adapt to the process group
changing after the call.

At one point Linux did adapt to the current process group changing but
that stopped in 161550d74c07 ("pid: sys_wait... fixes").  It has been
over 11 years since Linux has that behavior, no programs that fail
with the change in behavior have been reported, and I could not
find any other unix that does this.  So I think it is safe to clarify
the definition of current process group, to current process group
at the time of the wait function.

Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
Signed-off-by: Christian Brauner <christian.brauner@ubuntu.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Palmer Dabbelt <palmer@sifive.com>
Cc: Rich Felker <dalias@libc.org>
Cc: Alistair Francis <alistair23@gmail.com>
Cc: Zong Li <zongbox@gmail.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Florian Weimer <fweimer@redhat.com>
Cc: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Cc: GNU C Library <libc-alpha@sourceware.org>
---
v1:
- Christian Brauner <christian.brauner@ubuntu.com>:
  - move find_get_pid() calls into the switch statements to minimize
    merge conflicts with P_PIDFD changes and adhere to coding style
    discussions we had for P_PIDFD
---
 kernel/exit.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/kernel/exit.c b/kernel/exit.c
index 5b4a5dcce8f8..e70083b14f31 100644
--- a/kernel/exit.c
+++ b/kernel/exit.c
@@ -1576,19 +1576,23 @@ static long kernel_waitid(int which, pid_t upid, struct waitid_info *infop,
 		type = PIDTYPE_PID;
 		if (upid <= 0)
 			return -EINVAL;
+
+		pid = find_get_pid(upid);
 		break;
 	case P_PGID:
 		type = PIDTYPE_PGID;
-		if (upid <= 0)
+		if (upid < 0)
 			return -EINVAL;
+
+		if (upid == 0)
+			pid = get_pid(task_pgrp(current));
+		else
+			pid = find_get_pid(upid);
 		break;
 	default:
 		return -EINVAL;
 	}
 
-	if (type < PIDTYPE_MAX)
-		pid = find_get_pid(upid);
-
 	wo.wo_type	= type;
 	wo.wo_pid	= pid;
 	wo.wo_flags	= options;
-- 
2.22.0

Re: [PATCH v1 1/1] waitid: Add support for waiting for the current process group

[Oleg Nesterov]

On 08/14, christian.brauner@ubuntu.com wrote:
>
>  	case P_PGID:
>  		type = PIDTYPE_PGID;
> -		if (upid <= 0)
> +		if (upid < 0)
>  			return -EINVAL;
> +
> +		if (upid == 0)
> +			pid = get_pid(task_pgrp(current));

this needs rcu lock or tasklist_lock, this can race with another thread
doing sys_setpgid/setsid (see change_pid(PIDTYPE_PGID)).

Oleg.

Re: [PATCH v1 1/1] waitid: Add support for waiting for the current process group

[Christian Brauner]

On Wed, Aug 14, 2019 at 02:29:10PM +0200, Oleg Nesterov wrote:
> On 08/14, christian.brauner@ubuntu.com wrote:
> >
> >  	case P_PGID:
> >  		type = PIDTYPE_PGID;
> > -		if (upid <= 0)
> > +		if (upid < 0)
> >  			return -EINVAL;
> > +
> > +		if (upid == 0)
> > +			pid = get_pid(task_pgrp(current));
> 
> this needs rcu lock or tasklist_lock, this can race with another thread
> doing sys_setpgid/setsid (see change_pid(PIDTYPE_PGID)).

Oh, I naively assumed task_pgrp() would take an rcu lock...

kernel/sys.c takes both, i.e.

rcu_read_lock();
write_lock_irq(&tasklist_lock);

though I think we should be fine with just rcu_read_lock(). setpgid()
indicates that it wants to check real_parent and needs the
write_lock_irq() because it might change behind its back which we don't
care about since we're not changing the pgrp.

Christian

Re: [PATCH v1 1/1] waitid: Add support for waiting for the current process group

[Oleg Nesterov]

On 08/14, Christian Brauner wrote:
>
> On Wed, Aug 14, 2019 at 02:29:10PM +0200, Oleg Nesterov wrote:
> > On 08/14, christian.brauner@ubuntu.com wrote:
> > >
> > >  	case P_PGID:
> > >  		type = PIDTYPE_PGID;
> > > -		if (upid <= 0)
> > > +		if (upid < 0)
> > >  			return -EINVAL;
> > > +
> > > +		if (upid == 0)
> > > +			pid = get_pid(task_pgrp(current));
> >
> > this needs rcu lock or tasklist_lock, this can race with another thread
> > doing sys_setpgid/setsid (see change_pid(PIDTYPE_PGID)).
>
> Oh, I naively assumed task_pgrp() would take an rcu lock...

but it would not help ;)

> though I think we should be fine with just rcu_read_lock().

Yes,

Oleg.

Re: [PATCH v1 1/1] waitid: Add support for waiting for the current process group

[Christian Brauner]

On Wed, Aug 14, 2019 at 02:50:12PM +0200, Oleg Nesterov wrote:
> On 08/14, Christian Brauner wrote:
> >
> > On Wed, Aug 14, 2019 at 02:29:10PM +0200, Oleg Nesterov wrote:
> > > On 08/14, christian.brauner@ubuntu.com wrote:
> > > >
> > > >  	case P_PGID:
> > > >  		type = PIDTYPE_PGID;
> > > > -		if (upid <= 0)
> > > > +		if (upid < 0)
> > > >  			return -EINVAL;
> > > > +
> > > > +		if (upid == 0)
> > > > +			pid = get_pid(task_pgrp(current));
> > >
> > > this needs rcu lock or tasklist_lock, this can race with another thread
> > > doing sys_setpgid/setsid (see change_pid(PIDTYPE_PGID)).
> >
> > Oh, I naively assumed task_pgrp() would take an rcu lock...
> 
> but it would not help ;)

Yeah, it doesn't do a get. :)

[PATCH v2 0/1] waitid: process group enhancement

[Christian Brauner]

Hey everyone,

This patch adds support for waiting on the current process group by
specifying waitid(P_PGID, 0, ...) as discussed in [1]. The details why
we need to do this are in the commit message of [PATCH 1/1] so I won't
repeat them here.

I've picked this up since the thread has gone stale and parts of
userspace are actually blocked by this.

Note that the patch has been changed to be more closely aligned with the
P_PIDFD changes to waitid() I have sitting in my for-next branch (cf. [2]).
This makes the merge conflict a little simpler and picks up on the
coding style discussions that guided the P_PIDFD patchset.

There was some desire to get this feature in with 5.3 (cf. [3]).
But given that this is a new feature for waitid() and for the sake of
avoiding any merge conflicts I would prefer to land this in the 5.4
merge window together with the P_PIDFD changes.

Thanks!
Christian

/* References */
[1]: https://www.sourceware.org/ml/libc-alpha/2019-07/msg00587.html
[2]: https://lore.kernel.org/lkml/20190727222229.6516-1-christian@brauner.io/
[3]: https://www.sourceware.org/ml/libc-alpha/2019-08/msg00304.html

Eric W. Biederman (1):
  waitid: Add support for waiting for the current process group

 kernel/exit.c | 23 +++++++++++++++++++----
 1 file changed, 19 insertions(+), 4 deletions(-)

-- 
2.22.0

[PATCH v2 1/1] waitid: Add support for waiting for the current process group

[Christian Brauner]

From: "Eric W. Biederman" <ebiederm@xmission.com>

It was recently discovered that the linux version of waitid is not a
superset of the other wait functions because it does not include
support for waiting for the current process group.  This has two
downsides.  An extra system call is needed to get the current process
group, and a signal could come in between the system call that
retrieved the process gorup and the call to waitid that changes the
current process group.

Allow userspace to avoid both of those issues by defining
idtype == P_PGID and id == 0 to mean wait for the caller's process
group at the time of the call.

Arguments can be made for using a different choice of idtype and id
for this case but the BSDs already use this P_PGID and 0 to indicate
waiting for the current process's process group.  So be nice to user
space programmers and don't introduce an unnecessary incompatibility.

Some people have noted that the posix description is that
waitpid will wait for the current process group, and that in
the presence of pthreads that process group can change.  To get
clarity on this issue I looked at XNU, FreeBSD, and Luminos.  All of
those flavors of unix waited for the current process group at the
time of call and as written could not adapt to the process group
changing after the call.

At one point Linux did adapt to the current process group changing but
that stopped in 161550d74c07 ("pid: sys_wait... fixes").  It has been
over 11 years since Linux has that behavior, no programs that fail
with the change in behavior have been reported, and I could not
find any other unix that does this.  So I think it is safe to clarify
the definition of current process group, to current process group
at the time of the wait function.

Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
Signed-off-by: Christian Brauner <christian.brauner@ubuntu.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Palmer Dabbelt <palmer@sifive.com>
Cc: Rich Felker <dalias@libc.org>
Cc: Alistair Francis <alistair23@gmail.com>
Cc: Zong Li <zongbox@gmail.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Florian Weimer <fweimer@redhat.com>
Cc: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Cc: GNU C Library <libc-alpha@sourceware.org>
---
v1:
- Christian Brauner <christian.brauner@ubuntu.com>:
  - move find_get_pid() calls into the switch statements to minimize
    merge conflicts with P_PIDFD changes and adhere to coding style
    discussions we had for P_PIDFD

v2:
- Oleg Nesterov <oleg@redhat.com>:
  - take rcu_read_lock() around task_pgrp() so that we don't race with
    another thread changing the pgrp
- Christian Brauner <christian.brauner@ubuntu.com>:
  - introduce find_get_pgrp()
---
 kernel/exit.c | 23 +++++++++++++++++++----
 1 file changed, 19 insertions(+), 4 deletions(-)

diff --git a/kernel/exit.c b/kernel/exit.c
index 5b4a5dcce8f8..d02715a39f68 100644
--- a/kernel/exit.c
+++ b/kernel/exit.c
@@ -1554,6 +1554,20 @@ static long do_wait(struct wait_opts *wo)
 	return retval;
 }
 
+static struct pid *find_get_pgrp(pid_t nr)
+{
+	struct pid *pid;
+
+	if (nr)
+		return find_get_pid(nr);
+
+	rcu_read_lock();
+	pid = get_pid(task_pgrp(current));
+	rcu_read_unlock();
+
+	return pid;
+}
+
 static long kernel_waitid(int which, pid_t upid, struct waitid_info *infop,
 			  int options, struct rusage *ru)
 {
@@ -1576,19 +1590,20 @@ static long kernel_waitid(int which, pid_t upid, struct waitid_info *infop,
 		type = PIDTYPE_PID;
 		if (upid <= 0)
 			return -EINVAL;
+
+		pid = find_get_pid(upid);
 		break;
 	case P_PGID:
 		type = PIDTYPE_PGID;
-		if (upid <= 0)
+		if (upid < 0)
 			return -EINVAL;
+
+		pid = find_get_pgrp(upid);
 		break;
 	default:
 		return -EINVAL;
 	}
 
-	if (type < PIDTYPE_MAX)
-		pid = find_get_pid(upid);
-
 	wo.wo_type	= type;
 	wo.wo_pid	= pid;
 	wo.wo_flags	= options;
-- 
2.22.0

Re: [PATCH v2 1/1] waitid: Add support for waiting for the current process group

[Oleg Nesterov]

On 08/14, Christian Brauner wrote:
>
> +static struct pid *find_get_pgrp(pid_t nr)
> +{
> +	struct pid *pid;
> +
> +	if (nr)
> +		return find_get_pid(nr);
> +
> +	rcu_read_lock();
> +	pid = get_pid(task_pgrp(current));
> +	rcu_read_unlock();
> +
> +	return pid;
> +}

I can't say I like this helper... even its name doesn't look good to me.

I forgot that we already have get_task_pid() when I replied to the previous
version... How about

	case P_PGID:

		if (upid)
			pid = find_get_pid(upid);
		else
			pid = get_task_pid(current, PIDTYPE_PGID);

?

Oleg.

Re: [PATCH v2 1/1] waitid: Add support for waiting for the current process group

[Christian Brauner]

On Wed, Aug 14, 2019 at 04:19:57PM +0200, Oleg Nesterov wrote:
> On 08/14, Christian Brauner wrote:
> >
> > +static struct pid *find_get_pgrp(pid_t nr)
> > +{
> > +	struct pid *pid;
> > +
> > +	if (nr)
> > +		return find_get_pid(nr);
> > +
> > +	rcu_read_lock();
> > +	pid = get_pid(task_pgrp(current));
> > +	rcu_read_unlock();
> > +
> > +	return pid;
> > +}
> 
> I can't say I like this helper... even its name doesn't look good to me.

Well, naming scheme obviously stolen from find_get_pid(). Not sure if
that doesn't look good as well. ;)

> 
> I forgot that we already have get_task_pid() when I replied to the previous
> version... How about
> 
> 	case P_PGID:
> 
> 		if (upid)
> 			pid = find_get_pid(upid);
> 		else
> 			pid = get_task_pid(current, PIDTYPE_PGID);
> 
> ?

Hmyeah, that works but wouldn't it still be nicer to simply have:

static struct pid *get_pgrp(pid_t nr)
{
	if (nr)
		return find_get_pid(nr);

	return get_task_pid(current, PIDTYPE_PGID);
}

That allows us to have all cases equivalent with a single call to a
function without any if-elses.
Imho, this becomes especially nice when considering that P_PIDFD goes on
top of that:

	switch (which) {
	case P_ALL:
		type = PIDTYPE_MAX;
		break;
	case P_PID:
		type = PIDTYPE_PID;
		if (upid <= 0)
			return -EINVAL;

		pid = find_get_pid(upid);
		break;
	case P_PGID:
		type = PIDTYPE_PGID;
		if (upid < 0)
			return -EINVAL;

		pid = find_get_pgrp(upid);
		break;
	case P_PIDFD:
		type = PIDTYPE_PID;
		if (upid < 0)
			return -EINVAL;

		pid = pidfd_get_pid(upid);
		if (IS_ERR(pid))
			return PTR_ERR(pid);
		break;
	default:
		return -EINVAL;
	}

Christian

Re: [PATCH v2 1/1] waitid: Add support for waiting for the current process group

[Oleg Nesterov]

On 08/14, Christian Brauner wrote:
>
> On Wed, Aug 14, 2019 at 04:19:57PM +0200, Oleg Nesterov wrote:
> > On 08/14, Christian Brauner wrote:
> > >
> > > +static struct pid *find_get_pgrp(pid_t nr)
> > > +{
> > > +	struct pid *pid;
> > > +
> > > +	if (nr)
> > > +		return find_get_pid(nr);
> > > +
> > > +	rcu_read_lock();
> > > +	pid = get_pid(task_pgrp(current));
> > > +	rcu_read_unlock();
> > > +
> > > +	return pid;
> > > +}
> >
> > I can't say I like this helper... even its name doesn't look good to me.
>
> Well, naming scheme obviously stolen from find_get_pid(). Not sure if
> that doesn't look good as well. ;)

find_get_pid() actually tries to find a pid. The helper above does "find"
or "use current" depending on nr != 0.

> > I forgot that we already have get_task_pid() when I replied to the previous
> > version... How about
> >
> > 	case P_PGID:
> >
> > 		if (upid)
> > 			pid = find_get_pid(upid);
> > 		else
> > 			pid = get_task_pid(current, PIDTYPE_PGID);
> >
> > ?
>
> Hmyeah, that works but wouldn't it still be nicer to simply have:
>
> static struct pid *get_pgrp(pid_t nr)
> {
> 	if (nr)
> 		return find_get_pid(nr);
>
> 	return get_task_pid(current, PIDTYPE_PGID);
> }

Who else can ever use it?

It saves 4 lines in kernel_waitid() but adds 7 lines outside, and you
need another ^] to see these lines if you try to understand what
PIDTYPE_PGID actually does. IOW, I think this helper will make waitid
less readable for no reason.


Christian, I try to never argue when it comes to cosmetic issues, and
in this case I won't insist too.

Oleg.

Re: [PATCH v2 1/1] waitid: Add support for waiting for the current process group

[Christian Brauner]

On Wed, Aug 14, 2019 at 05:27:12PM +0200, Oleg Nesterov wrote:
> On 08/14, Christian Brauner wrote:
> >
> > On Wed, Aug 14, 2019 at 04:19:57PM +0200, Oleg Nesterov wrote:
> > > On 08/14, Christian Brauner wrote:
> > > >
> > > > +static struct pid *find_get_pgrp(pid_t nr)
> > > > +{
> > > > +	struct pid *pid;
> > > > +
> > > > +	if (nr)
> > > > +		return find_get_pid(nr);
> > > > +
> > > > +	rcu_read_lock();
> > > > +	pid = get_pid(task_pgrp(current));
> > > > +	rcu_read_unlock();
> > > > +
> > > > +	return pid;
> > > > +}
> > >
> > > I can't say I like this helper... even its name doesn't look good to me.
> >
> > Well, naming scheme obviously stolen from find_get_pid(). Not sure if
> > that doesn't look good as well. ;)
> 
> find_get_pid() actually tries to find a pid. The helper above does "find"
> or "use current" depending on nr != 0.
> 
> > > I forgot that we already have get_task_pid() when I replied to the previous
> > > version... How about
> > >
> > > 	case P_PGID:
> > >
> > > 		if (upid)
> > > 			pid = find_get_pid(upid);
> > > 		else
> > > 			pid = get_task_pid(current, PIDTYPE_PGID);
> > >
> > > ?
> >
> > Hmyeah, that works but wouldn't it still be nicer to simply have:
> >
> > static struct pid *get_pgrp(pid_t nr)
> > {
> > 	if (nr)
> > 		return find_get_pid(nr);
> >
> > 	return get_task_pid(current, PIDTYPE_PGID);
> > }
> 
> Who else can ever use it?
> 
> It saves 4 lines in kernel_waitid() but adds 7 lines outside, and you
> need another ^] to see these lines if you try to understand what
> PIDTYPE_PGID actually does. IOW, I think this helper will make waitid
> less readable for no reason.
> 
> 
> Christian, I try to never argue when it comes to cosmetic issues, and
> in this case I won't insist too.

Yeah, I know. I'm not insisisting either. We can do your thing since you
do after all seem to care at least a tiny bit. ;)

Christian

[PATCH v3 0/1] waitid: process group enhancement

[Christian Brauner]

Hey everyone,

This patch adds support for waiting on the current process group by
specifying waitid(P_PGID, 0, ...) as discussed in [1]. The details why
we need to do this are in the commit message of [PATCH 1/1] so I won't
repeat them here.

I've picked this up since the thread has gone stale and parts of
userspace are actually blocked by this.

Note that the patch has been changed to be more closely aligned with the
P_PIDFD changes to waitid() I have sitting in my for-next branch (cf. [2]).
This makes the merge conflict a little simpler and picks up on the
coding style discussions that guided the P_PIDFD patchset.

There was some desire to get this feature in with 5.3 (cf. [3]).
But given that this is a new feature for waitid() and for the sake of
avoiding any merge conflicts I would prefer to land this in the 5.4
merge window together with the P_PIDFD changes.

Thanks!
Christian

/* v0 */
Link: https://www.sourceware.org/ml/libc-alpha/2019-07/msg00587.html

/* v1 */
Link: https://lore.kernel.org/lkml/20190814113822.9505-1-christian.brauner@ubuntu.com/

/* v2 */
Link: https://lore.kernel.org/lkml/20190814130732.23572-1-christian.brauner@ubuntu.com/

/* References */
[1]: https://www.sourceware.org/ml/libc-alpha/2019-07/msg00587.html
[2]: https://lore.kernel.org/lkml/20190727222229.6516-1-christian@brauner.io/
[3]: https://www.sourceware.org/ml/libc-alpha/2019-08/msg00304.html

Eric W. Biederman (1):
  waitid: Add support for waiting for the current process group

 kernel/exit.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

-- 
2.22.0

[PATCH v3 1/1] waitid: Add support for waiting for the current process group

[Christian Brauner]

From: "Eric W. Biederman" <ebiederm@xmission.com>

It was recently discovered that the linux version of waitid is not a
superset of the other wait functions because it does not include
support for waiting for the current process group.  This has two
downsides.  An extra system call is needed to get the current process
group, and a signal could come in between the system call that
retrieved the process gorup and the call to waitid that changes the
current process group.

Allow userspace to avoid both of those issues by defining
idtype == P_PGID and id == 0 to mean wait for the caller's process
group at the time of the call.

Arguments can be made for using a different choice of idtype and id
for this case but the BSDs already use this P_PGID and 0 to indicate
waiting for the current process's process group.  So be nice to user
space programmers and don't introduce an unnecessary incompatibility.

Some people have noted that the posix description is that
waitpid will wait for the current process group, and that in
the presence of pthreads that process group can change.  To get
clarity on this issue I looked at XNU, FreeBSD, and Luminos.  All of
those flavors of unix waited for the current process group at the
time of call and as written could not adapt to the process group
changing after the call.

At one point Linux did adapt to the current process group changing but
that stopped in 161550d74c07 ("pid: sys_wait... fixes").  It has been
over 11 years since Linux has that behavior, no programs that fail
with the change in behavior have been reported, and I could not
find any other unix that does this.  So I think it is safe to clarify
the definition of current process group, to current process group
at the time of the wait function.

Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
Signed-off-by: Christian Brauner <christian.brauner@ubuntu.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Palmer Dabbelt <palmer@sifive.com>
Cc: Rich Felker <dalias@libc.org>
Cc: Alistair Francis <alistair23@gmail.com>
Cc: Zong Li <zongbox@gmail.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Florian Weimer <fweimer@redhat.com>
Cc: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Cc: GNU C Library <libc-alpha@sourceware.org>
---
v1:
- Christian Brauner <christian.brauner@ubuntu.com>:
  - move find_get_pid() calls into the switch statements to minimize
    merge conflicts with P_PIDFD changes and adhere to coding style
    discussions we had for P_PIDFD

v2:
- Oleg Nesterov <oleg@redhat.com>:
  - take rcu_read_lock() around task_pgrp() so that we don't race with
    another thread changing the pgrp
- Christian Brauner <christian.brauner@ubuntu.com>:
  - introduce find_get_pgrp()

v3:
- Oleg Nesterov <oleg@redhat.com>:
  - use get_task_pid()
---
 kernel/exit.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/kernel/exit.c b/kernel/exit.c
index 5b4a5dcce8f8..8e4e357fddc8 100644
--- a/kernel/exit.c
+++ b/kernel/exit.c
@@ -1576,19 +1576,23 @@ static long kernel_waitid(int which, pid_t upid, struct waitid_info *infop,
 		type = PIDTYPE_PID;
 		if (upid <= 0)
 			return -EINVAL;
+
+		pid = find_get_pid(upid);
 		break;
 	case P_PGID:
 		type = PIDTYPE_PGID;
-		if (upid <= 0)
+		if (upid < 0)
 			return -EINVAL;
+
+		if (upid)
+			pid = find_get_pid(upid);
+		else
+			pid = get_task_pid(current, PIDTYPE_PGID);
 		break;
 	default:
 		return -EINVAL;
 	}
 
-	if (type < PIDTYPE_MAX)
-		pid = find_get_pid(upid);
-
 	wo.wo_type	= type;
 	wo.wo_pid	= pid;
 	wo.wo_flags	= options;
-- 
2.22.0

Re: [PATCH v3 1/1] waitid: Add support for waiting for the current process group

[Oleg Nesterov]

On 08/14, Christian Brauner wrote:
>
> and a signal could come in between the system call that
> retrieved the process gorup and the call to waitid that changes the
                        ^^^^^
> current process group.

I noticed this typo only because I spent 2 minutes or more trying to
understand this sentence ;) But yes, a signal handler or another thread
can change pgrp in between.

Reviewed-by: Oleg Nesterov <oleg@redhat.com>

Re: [PATCH v3 1/1] waitid: Add support for waiting for the current process group

[Christian Brauner]

On Wed, Aug 14, 2019 at 06:09:17PM +0200, Oleg Nesterov wrote:
> On 08/14, Christian Brauner wrote:
> >
> > and a signal could come in between the system call that
> > retrieved the process gorup and the call to waitid that changes the
>                         ^^^^^
> > current process group.
> 
> I noticed this typo only because I spent 2 minutes or more trying to
> understand this sentence ;) But yes, a signal handler or another thread

I'll try to rewrite it. :)

> can change pgrp in between.
> 
> Reviewed-by: Oleg Nesterov <oleg@redhat.com>
> 

Re: [PATCH v3 1/1] waitid: Add support for waiting for the current process group

[Christian Brauner]

On Wed, Aug 14, 2019 at 06:15:17PM +0200, Christian Brauner wrote:
> On Wed, Aug 14, 2019 at 06:09:17PM +0200, Oleg Nesterov wrote:
> > On 08/14, Christian Brauner wrote:
> > >
> > > and a signal could come in between the system call that
> > > retrieved the process gorup and the call to waitid that changes the
> >                         ^^^^^
> > > current process group.
> > 
> > I noticed this typo only because I spent 2 minutes or more trying to
> > understand this sentence ;) But yes, a signal handler or another thread
> 
> I'll try to rewrite it. :)

Ok, here's what I changed it to:

It was recently discovered that the linux version of waitid is not a
superset of the other wait functions because it does not include
support for waiting for the current process group. This has two
downsides:
1. An extra system call is needed to get the current process group.
2. After the current process group is received and before it is passed
   to waitid a signal could arrive causing the current process group to change.

Christian

Re: [PATCH v3 1/1] waitid: Add support for waiting for the current process group

[Rich Felker]

On Wed, Aug 14, 2019 at 06:34:44PM +0200, Christian Brauner wrote:
> On Wed, Aug 14, 2019 at 06:15:17PM +0200, Christian Brauner wrote:
> > On Wed, Aug 14, 2019 at 06:09:17PM +0200, Oleg Nesterov wrote:
> > > On 08/14, Christian Brauner wrote:
> > > >
> > > > and a signal could come in between the system call that
> > > > retrieved the process gorup and the call to waitid that changes the
> > >                         ^^^^^
> > > > current process group.
> > > 
> > > I noticed this typo only because I spent 2 minutes or more trying to
> > > understand this sentence ;) But yes, a signal handler or another thread
> > 
> > I'll try to rewrite it. :)
> 
> Ok, here's what I changed it to:
> 
> It was recently discovered that the linux version of waitid is not a
> superset of the other wait functions because it does not include
> support for waiting for the current process group. This has two
> downsides:
> 1. An extra system call is needed to get the current process group.
> 2. After the current process group is received and before it is passed
>    to waitid a signal could arrive causing the current process group to change.

I don't think "downsides" sufficiently conveys that this is hard
breakage of a requirement for waitpid. How about something like the
following?

"It was recently discovered that the linux version of waitid is not a
superset of the other wait functions because it does not include
support for waiting for the current process group. Userspace cannot
simply emulate this functionality with an additional getpgid syscall
due to inherent race conditions that violate the async-signal safety
requirements for waitpid."

Rich

Re: [PATCH v3 1/1] waitid: Add support for waiting for the current process group

[Christian Brauner]

On Wed, Aug 14, 2019 at 12:55:01PM -0400, Rich Felker wrote:
> On Wed, Aug 14, 2019 at 06:34:44PM +0200, Christian Brauner wrote:
> > On Wed, Aug 14, 2019 at 06:15:17PM +0200, Christian Brauner wrote:
> > > On Wed, Aug 14, 2019 at 06:09:17PM +0200, Oleg Nesterov wrote:
> > > > On 08/14, Christian Brauner wrote:
> > > > >
> > > > > and a signal could come in between the system call that
> > > > > retrieved the process gorup and the call to waitid that changes the
> > > >                         ^^^^^
> > > > > current process group.
> > > > 
> > > > I noticed this typo only because I spent 2 minutes or more trying to
> > > > understand this sentence ;) But yes, a signal handler or another thread
> > > 
> > > I'll try to rewrite it. :)
> > 
> > Ok, here's what I changed it to:
> > 
> > It was recently discovered that the linux version of waitid is not a
> > superset of the other wait functions because it does not include
> > support for waiting for the current process group. This has two
> > downsides:
> > 1. An extra system call is needed to get the current process group.
> > 2. After the current process group is received and before it is passed
> >    to waitid a signal could arrive causing the current process group to change.
> 
> I don't think "downsides" sufficiently conveys that this is hard
> breakage of a requirement for waitpid. How about something like the
> following?
> 
> "It was recently discovered that the linux version of waitid is not a
> superset of the other wait functions because it does not include
> support for waiting for the current process group. Userspace cannot
> simply emulate this functionality with an additional getpgid syscall
> due to inherent race conditions that violate the async-signal safety
> requirements for waitpid."

I like the rather specific example in there. How about we add that after
this section like so:

It was recently discovered that the linux version of waitid is not a
superset of the other wait functions because it does not include
support for waiting for the current process group. This has two
downsides:
1. An extra system call is needed to get the current process group.
2. After the current process group is received and before it is passed
   to waitid a signal could arrive causing the current process group to change.

Such inherent race-conditions as mentioned in 2. make it impossible for
userspace to emulate this functionaly and thus violate the async-signal
safety requirements for waitpid.

Christian

Re: [PATCH v3 1/1] waitid: Add support for waiting for the current process group

[Linus Torvalds]

On Wed, Aug 14, 2019 at 9:55 AM Rich Felker <dalias@libc.org> wrote:
>
> I don't think "downsides" sufficiently conveys that this is hard
> breakage of a requirement for waitpid.

Well, let's be honest here. Who has _ever_ seen a signal handler
changing the current process group?

In fact, the SYSV version of setpgid() takes a process ID to set it
*for somebody else*, so the signal safety is not even necessarily
relevant, since it might be racing with _another_ thread doing it
(which even the kernel side won't fix - it's just user space doing odd
things).

So yes - it's technically true that it's impossible to emulate
properly in user space.

But I doubt it makes _any_ difference what-so-ever, and glibc might as
well do something like

     ret = waitid(P_PGID, 0, ..);
     if (ret == -EINVAL) { do the emulation }

which makes it work with older kernels, and has zero downside in practice.

Hmm?

              Linus

Re: [PATCH v3 1/1] waitid: Add support for waiting for the current process group

[Rich Felker]

On Wed, Aug 14, 2019 at 10:06:19AM -0700, Linus Torvalds wrote:
> On Wed, Aug 14, 2019 at 9:55 AM Rich Felker <dalias@libc.org> wrote:
> >
> > I don't think "downsides" sufficiently conveys that this is hard
> > breakage of a requirement for waitpid.
> 
> Well, let's be honest here. Who has _ever_ seen a signal handler
> changing the current process group?
> 
> In fact, the SYSV version of setpgid() takes a process ID to set it
> *for somebody else*, so the signal safety is not even necessarily
> relevant, since it might be racing with _another_ thread doing it
> (which even the kernel side won't fix - it's just user space doing odd
> things).

For that case, the operations are inherently unordered with respect to
each other, and assuming the interpretation that waitpid is allowed to
wait on "the pgid at the time of the call" rather than at the time of
child exit/status-change -- which was discussed thoroughly in the
thread leading up to this patch -- there is no conformance
distinction.

On the other hand, with changing your own pgid from a signal handler,
there is a clear observable ordering between the events. For example,
if the signal handler changes the pgid and forks a child with the new
pgid, waitpid for "own pgid" can be assumed to include the new child
in its wait set.

I agree this is not common usage, so impact of breakage is probably
low, but I'd rather not have wrong/racy hacks be something we're
committed to supporting indefinitely on the userspace side.

> So yes - it's technically true that it's impossible to emulate
> properly in user space.
> 
> But I doubt it makes _any_ difference what-so-ever, and glibc might as
> well do something like
> 
>      ret = waitid(P_PGID, 0, ..);
>      if (ret == -EINVAL) { do the emulation }
> 
> which makes it work with older kernels, and has zero downside in practice.
> 
> Hmm?

It only affects RV32 anyway; other archs all have a waitpid syscall
that can be used. Since there's not yet any official libc release with
RV32 support and AIUI the ABI is not considered "frozen" yet,
emulation doesn't seem useful here. Whatever kernel version fixes this
(or some later one, if nobody gets things together on upstreaming libc
support of RV32) will just become the minimum version for RV32.

Rich

Re: [PATCH v3 1/1] waitid: Add support for waiting for the current process group

[Christian Brauner]

On Wed, Aug 14, 2019 at 06:09:17PM +0200, Oleg Nesterov wrote:
> On 08/14, Christian Brauner wrote:
> >
> > and a signal could come in between the system call that
> > retrieved the process gorup and the call to waitid that changes the
>                         ^^^^^
> > current process group.
> 
> I noticed this typo only because I spent 2 minutes or more trying to
> understand this sentence ;) But yes, a signal handler or another thread
> can change pgrp in between.
> 
> Reviewed-by: Oleg Nesterov <oleg@redhat.com>

Applied-to:
https://git.kernel.org/pub/scm/linux/kernel/git/brauner/linux.git/log/?h=pidfd
on top of P_PIDFD changes (cf. [1]) with merge conflict resolved (cf. [2]).
(All changes on top of v5.3-rc1.)

Merged-into:
https://git.kernel.org/pub/scm/linux/kernel/git/brauner/linux.git/log/?h=for-next
and should show up in linux-next tomorrow.

Thanks!
Christian

/* References */
[1]: https://lore.kernel.org/r/20190727222229.6516-2-christian@brauner.io
[2]: patch after resolved merge-conflict:
 kernel/exit.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/kernel/exit.c b/kernel/exit.c
index 64bb6893a37d..d2d74a7b81d1 100644
--- a/kernel/exit.c
+++ b/kernel/exit.c
@@ -1596,10 +1596,13 @@ static long kernel_waitid(int which, pid_t upid, struct waitid_info *infop,
 		break;
 	case P_PGID:
 		type = PIDTYPE_PGID;
-		if (upid <= 0)
+		if (upid < 0)
 			return -EINVAL;
 
-		pid = find_get_pid(upid);
+		if (upid)
+			pid = find_get_pid(upid);
+		else
+			pid = get_task_pid(current, PIDTYPE_PGID);
 		break;
 	case P_PIDFD:
 		type = PIDTYPE_PID;

Re: [PATCH v3 0/1] waitid: process group enhancement

[Rich Felker]

On Wed, Aug 14, 2019 at 05:43:59PM +0200, Christian Brauner wrote:
> Hey everyone,
> 
> This patch adds support for waiting on the current process group by
> specifying waitid(P_PGID, 0, ...) as discussed in [1]. The details why
> we need to do this are in the commit message of [PATCH 1/1] so I won't
> repeat them here.
> 
> I've picked this up since the thread has gone stale and parts of
> userspace are actually blocked by this.
> 
> Note that the patch has been changed to be more closely aligned with the
> P_PIDFD changes to waitid() I have sitting in my for-next branch (cf. [2]).
> This makes the merge conflict a little simpler and picks up on the
> coding style discussions that guided the P_PIDFD patchset.
> 
> There was some desire to get this feature in with 5.3 (cf. [3]).
> But given that this is a new feature for waitid() and for the sake of
> avoiding any merge conflicts I would prefer to land this in the 5.4
> merge window together with the P_PIDFD changes.

That makes 5.4 (or later, depending on other stuff) the hard minimum
for RV32 ABI. Is that acceptable? I was under the impression (perhaps
mistaken) that 5.3 was going to be next LTS series which is why I'd
like to have the necessary syscalls for a complete working RV32
userspace in it. If I'm wrong about that please ignore me. :-)

Rich

Re: [PATCH v3 0/1] waitid: process group enhancement

[Christian Brauner]

On Wed, Aug 14, 2019 at 11:58:22AM -0400, Rich Felker wrote:
> On Wed, Aug 14, 2019 at 05:43:59PM +0200, Christian Brauner wrote:
> > Hey everyone,
> > 
> > This patch adds support for waiting on the current process group by
> > specifying waitid(P_PGID, 0, ...) as discussed in [1]. The details why
> > we need to do this are in the commit message of [PATCH 1/1] so I won't
> > repeat them here.
> > 
> > I've picked this up since the thread has gone stale and parts of
> > userspace are actually blocked by this.
> > 
> > Note that the patch has been changed to be more closely aligned with the
> > P_PIDFD changes to waitid() I have sitting in my for-next branch (cf. [2]).
> > This makes the merge conflict a little simpler and picks up on the
> > coding style discussions that guided the P_PIDFD patchset.
> > 
> > There was some desire to get this feature in with 5.3 (cf. [3]).
> > But given that this is a new feature for waitid() and for the sake of
> > avoiding any merge conflicts I would prefer to land this in the 5.4
> > merge window together with the P_PIDFD changes.
> 
> That makes 5.4 (or later, depending on other stuff) the hard minimum
> for RV32 ABI. Is that acceptable? I was under the impression (perhaps
> mistaken) that 5.3 was going to be next LTS series which is why I'd
> like to have the necessary syscalls for a complete working RV32
> userspace in it. If I'm wrong about that please ignore me. :-)

5.3 is not going to be an LTS and we don't do new features after the
merge window is closed anyway. :)

Christian

Re: [RFC v3 03/23] sysdeps/wait: Use waitid if avaliable

[Alistair Francis]

On Thu, Jul 25, 2019 at 10:14 AM Eric W. Biederman
<ebiederm@xmission.com> wrote:
>
> Arnd Bergmann <arnd@arndb.de> writes:
>
> > On Thu, Jul 25, 2019 at 6:40 AM Rich Felker <dalias@libc.org> wrote:
> >> On Wed, Jul 24, 2019 at 05:04:53PM -0700, Alistair Francis wrote:
> >> > On Tue, Jul 23, 2019 at 1:45 AM Arnd Bergmann <arnd@arndb.de> wrote:
> >> > >
> >> > > Sounds good to me, the debate over what rusage to use should not hold
> >> > > up the review of the rest of that syscall.
> >> >
> >> > I'm unclear what the final decision is here. What is the solution are
> >> > we going to have wait4() or add P_PROCESS_PGID to waitid()?
> >> >
> >> > As well as that what is the solution to current implementations? If we
> >> > add wait4() then there isn't an issue (and I can drop this patch) but
> >> > if we add P_PROCESS_PGID then we will need a way to handle kernels
> >> > with waitid() but no P_PROCESS_PGID. Although my new plan is to only
> >> > use the waitid syscall if we don't have waitpid or wait4 so it seems
> >> > like this will only affect RV32 for the time being.
> >>
> >> I would really like some indication which solution will be taken,
> >> since it impacts choices that will need to be made in musl very soon.
> >> My favorite outcome would be bringing back wait4 for rv32 (and
> >> no-time32 archs in general) *and* adding P_PROCESS_PGID. In the short
> >> term, just using wait4 would be the simplest and cleanest for us (same
> >> as all other archs, no extra case to deal with), but in the long term
> >> there may be value in having rusage that can represent more than 68
> >> cpu-years spent by a process (seems plausible with large numbers of
> >> cores).
> >
> > Based on the feedback from Linus and Eric, the most likely outcome
> > at the moment seems to be an extension of waitid() to allow
> > P_PGID with id=0 like BSD does, and not bring back wait4() or
> > add P_PROCESS_PGID.
> >
> > So far, I don't think anyone has proposed an actual kernel patch.
> > I was hoping that Eric would do it, but I could also send it if he's
> > otherwise busy.
>
> So here is what I am looking at.  It still needs to be tested
> and the description needs to be improved so that it properly credits
> everyone.  However I think I have the major stroeks correct.
>
> From: "Eric W. Biederman" <ebiederm@xmission.com>
> Date: Tue, 23 Jul 2019 07:44:46 -0500
> Subject: [PATCH] waitid: Add support for waiting for the current process group
>
> It was recently discovered that the linux version of waitid is not a
> superset of the other wait functions because it does not include
> support for waiting for the current process group.  This has two
> downsides.  An extra system call is needed to get the current process
> group, and a signal could come in between the system call that
> retrieved the process gorup and the call to waitid that changes the
> current process group.
>
> Allow userspace to avoid both of those issues by defining
> idtype == P_PGID and id == 0 to mean wait for the caller's process
> group at the time of the call.
>
> Arguments can be made for using a different choice of idtype and id
> for this case but the BSDs already use this P_PGID and 0 to indicate
> waiting for the current process's process group.  So be nice to user
> space programmers and don't introduce an unnecessary incompatibility.
>
> Some people have noted that the posix description is that
> waitpid will wait for the current process group, and that in
> the presence of pthreads that process group can change.  To get
> clarity on this issue I looked at XNU, FreeBSD, and Luminos.  All of
> those flavors of unix waited for the current process group at the
> time of call and as written could not adapt to the process group
> changing after the call.
>
> At one point Linux did adapt to the current process group changing but
> that stopped in 161550d74c07 ("pid: sys_wait... fixes").  It has been
> over 11 years since Linux has that behavior, no programs that fail
> with the change in behavior have been reported, and I could not
> find any other unix that does this.  So I think it is safe to clarify
> the definition of current process group, to current process group
> at the time of the wait function.
>
> Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>

Will this land in 5.3? Also will it be back ported to stable kernels?

Do you mind keeping me in CC when you send the patch?

Alistair

> ---
>  kernel/exit.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/kernel/exit.c b/kernel/exit.c
> index a75b6a7f458a..3d86930f035e 100644
> --- a/kernel/exit.c
> +++ b/kernel/exit.c
> @@ -1577,14 +1577,16 @@ static long kernel_waitid(int which, pid_t upid, struct waitid_info *infop,
>                 break;
>         case P_PGID:
>                 type = PIDTYPE_PGID;
> -               if (upid <= 0)
> +               if (upid < 0)
>                         return -EINVAL;
> +               if (upid == 0)
> +                       pid = get_pid(task_pgrp(current));
>                 break;
>         default:
>                 return -EINVAL;
>         }
>
> -       if (type < PIDTYPE_MAX)
> +       if ((type < PIDTYPE_MAX) && !pid)
>                 pid = find_get_pid(upid);
>
>         wo.wo_type      = type;
> --
> 2.21.0.dirty
>

[RFC v3 02/23] sysdeps/gettimeofday: Use clock_gettime64 if avaliable

[Alistair Francis]

Not all architectures support the obsolete gettimeofday so use the
newer clock_gettime64 syscall if it is avaliable. This fixes RV32
build issues.

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 ChangeLog                              |  1 +
 sysdeps/unix/sysv/linux/gettimeofday.c | 28 ++++++++++++++++++++++++++
 2 files changed, 29 insertions(+)

diff --git a/ChangeLog b/ChangeLog
index 477b9b49b3..9ca390a9c3 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1028,6 +1028,7 @@
 	* nptl/thrd_sleep.c: Use clock_nanosleep_time64 instead of nanosleep.
 	* sysdeps/unix/sysv/linux/nanosleep.c: Likewise.
 	* sysdeps/unix/sysv/linux/nanosleep_nocancel.c: Likewise.
+	* sysdeps/unix/sysv/linux/gettimeofday.c: Use clock_gettime64 syscall for gettimeofday.
 
 2019-06-20  Dmitry V. Levin  <ldv@altlinux.org>
 	    Florian Weimer  <fweimer@redhat.com>
diff --git a/sysdeps/unix/sysv/linux/gettimeofday.c b/sysdeps/unix/sysv/linux/gettimeofday.c
index a74f03825a..151b1e606c 100644
--- a/sysdeps/unix/sysv/linux/gettimeofday.c
+++ b/sysdeps/unix/sysv/linux/gettimeofday.c
@@ -32,7 +32,35 @@
 int
 __gettimeofday (struct timeval *tv, struct timezone *tz)
 {
+#ifdef __ASSUME_TIME64_SYSCALLS
+  int ret;
+  struct __timespec64 now;
+
+  ret = INLINE_VSYSCALL (clock_gettime64, 2, CLOCK_REALTIME,
+                         &now);
+
+  /* Convert from timespec to timeval */
+  tv->tv_sec = now.tv_sec;
+  tv->tv_usec = now.tv_nsec / 1000;
+
+  return ret;
+#else
+# ifdef __NR_clock_gettime64
+  long int ret;
+  struct __timespec64 now;
+
+  ret = INLINE_VSYSCALL (clock_gettime64, 2, CLOCK_REALTIME,
+                         &now);
+
+  /* Convert from timespec to timeval */
+  tv->tv_sec = now.tv_sec;
+  tv->tv_usec = now.tv_nsec / 1000;
+
+  if (ret == 0 || errno != ENOSYS)
+    return ret;
+# endif
   return INLINE_VSYSCALL (gettimeofday, 2, tv, tz);
+#endif
 }
 libc_hidden_def (__gettimeofday)
 weak_alias (__gettimeofday, gettimeofday)
-- 
2.22.0

Re: [RFC v3 02/23] sysdeps/gettimeofday: Use clock_gettime64 if avaliable

[Florian Weimer]

* Alistair Francis:

> diff --git a/sysdeps/unix/sysv/linux/gettimeofday.c b/sysdeps/unix/sysv/linux/gettimeofday.c
> index a74f03825a..151b1e606c 100644
> --- a/sysdeps/unix/sysv/linux/gettimeofday.c
> +++ b/sysdeps/unix/sysv/linux/gettimeofday.c
> @@ -32,7 +32,35 @@
>  int
>  __gettimeofday (struct timeval *tv, struct timezone *tz)
>  {
> +#ifdef __ASSUME_TIME64_SYSCALLS
> +  int ret;
> +  struct __timespec64 now;
> +
> +  ret = INLINE_VSYSCALL (clock_gettime64, 2, CLOCK_REALTIME,
> +                         &now);
> +
> +  /* Convert from timespec to timeval */
> +  tv->tv_sec = now.tv_sec;
> +  tv->tv_usec = now.tv_nsec / 1000;
> +
> +  return ret;
> +#else
> +# ifdef __NR_clock_gettime64
> +  long int ret;
> +  struct __timespec64 now;
> +
> +  ret = INLINE_VSYSCALL (clock_gettime64, 2, CLOCK_REALTIME,
> +                         &now);
> +
> +  /* Convert from timespec to timeval */
> +  tv->tv_sec = now.tv_sec;
> +  tv->tv_usec = now.tv_nsec / 1000;
> +
> +  if (ret == 0 || errno != ENOSYS)
> +    return ret;
> +# endif
>    return INLINE_VSYSCALL (gettimeofday, 2, tv, tz);
> +#endif
>  }

This loses vDSO acceleration if glibc is compiled with kernel headers
which define __NR_clock_gettime64, but the run-time kernel does not have
clock_gettime64 in the vDSO.

And the kernel folks really want us to call clock_gettime when the user
calls the 32-bit function, for tracability of legacy processes.

Thanks,
Florian

Re: [RFC v3 02/23] sysdeps/gettimeofday: Use clock_gettime64 if avaliable

[Rich Felker]

On Wed, Jul 17, 2019 at 09:09:08AM +0200, Florian Weimer wrote:
> * Alistair Francis:
> 
> > diff --git a/sysdeps/unix/sysv/linux/gettimeofday.c b/sysdeps/unix/sysv/linux/gettimeofday.c
> > index a74f03825a..151b1e606c 100644
> > --- a/sysdeps/unix/sysv/linux/gettimeofday.c
> > +++ b/sysdeps/unix/sysv/linux/gettimeofday.c
> > @@ -32,7 +32,35 @@
> >  int
> >  __gettimeofday (struct timeval *tv, struct timezone *tz)
> >  {
> > +#ifdef __ASSUME_TIME64_SYSCALLS
> > +  int ret;
> > +  struct __timespec64 now;
> > +
> > +  ret = INLINE_VSYSCALL (clock_gettime64, 2, CLOCK_REALTIME,
> > +                         &now);
> > +
> > +  /* Convert from timespec to timeval */
> > +  tv->tv_sec = now.tv_sec;
> > +  tv->tv_usec = now.tv_nsec / 1000;
> > +
> > +  return ret;
> > +#else
> > +# ifdef __NR_clock_gettime64
> > +  long int ret;
> > +  struct __timespec64 now;
> > +
> > +  ret = INLINE_VSYSCALL (clock_gettime64, 2, CLOCK_REALTIME,
> > +                         &now);
> > +
> > +  /* Convert from timespec to timeval */
> > +  tv->tv_sec = now.tv_sec;
> > +  tv->tv_usec = now.tv_nsec / 1000;
> > +
> > +  if (ret == 0 || errno != ENOSYS)
> > +    return ret;
> > +# endif
> >    return INLINE_VSYSCALL (gettimeofday, 2, tv, tz);
> > +#endif
> >  }
> 
> This loses vDSO acceleration if glibc is compiled with kernel headers
> which define __NR_clock_gettime64, but the run-time kernel does not have
> clock_gettime64 in the vDSO.
> 
> And the kernel folks really want us to call clock_gettime when the user
> calls the 32-bit function, for tracability of legacy processes.

May or may not be relevant to glibc plans, but I want to chime in to
mention that this (kernel tracability of legacy processes) is not
going to work with musl, except possibly static-linked programs with
old libc. Our legacy compat functions are just going to be thin,
implementation-agnostic wrappers around the new functions, so even if
a program is still calling the old clock_gettime symbol with 32-bit
timespec, it's going to be executing the new one as its backend, and
thereby calling the 64-bit syscall or vdso if available.

In the case of new libc with old application on old kernel, there will
actually be two conversions: 32->64 in the core (new) function after
the 64-bit syscall fails with ENOSYS and we fallback to the old
syscall, and 64->32 in the old-ABI-compat wrapper. This is a
consequence of not duplicating functionality and ending up with two
versions of everything to maintain, and also avoids incentivizing
continued use of the old ABI for users with old kernels (as a way to
bypass conversion costs).

In any case, just grepping dynsym tables for references to legacy
symbols seems like a more effective way of finding problems than
grepping for syslog spam...

Rich

Re: [RFC v3 02/23] sysdeps/gettimeofday: Use clock_gettime64 if avaliable

[Joseph Myers]

On Fri, 19 Jul 2019, Rich Felker wrote:

> > And the kernel folks really want us to call clock_gettime when the user
> > calls the 32-bit function, for tracability of legacy processes.
> 
> May or may not be relevant to glibc plans, but I want to chime in to
> mention that this (kernel tracability of legacy processes) is not
> going to work with musl, except possibly static-linked programs with
> old libc. Our legacy compat functions are just going to be thin,
> implementation-agnostic wrappers around the new functions, so even if
> a program is still calling the old clock_gettime symbol with 32-bit
> timespec, it's going to be executing the new one as its backend, and
> thereby calling the 64-bit syscall or vdso if available.

And as far as I'm concerned glibc should do likewise (I have a sustained 
objection to duplication of nontrivial function implementations at either 
the source and binary level for time_t variants when thin wrappers would 
do the job just as well, given the principle that the particular choice of 
syscalls for a given function is not a stable interface).  (Where a 
function is defined entirely in syscalls.list, it might continue to use 
the old syscall, subject to working out the best way to set up function 
aliasing for that case.)

> In any case, just grepping dynsym tables for references to legacy
> symbols seems like a more effective way of finding problems than
> grepping for syslog spam...

Yes.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: [RFC v3 02/23] sysdeps/gettimeofday: Use clock_gettime64 if avaliable

[Lukasz Majewski]

[-- Attachment #1: Type: text/plain, Size: 3000 bytes --]

Hi Alistair,

> Not all architectures support the obsolete gettimeofday so use the
> newer clock_gettime64 syscall if it is avaliable. This fixes RV32
> build issues.
> 
> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
> ---
>  ChangeLog                              |  1 +
>  sysdeps/unix/sysv/linux/gettimeofday.c | 28
> ++++++++++++++++++++++++++ 2 files changed, 29 insertions(+)
> 
> diff --git a/ChangeLog b/ChangeLog
> index 477b9b49b3..9ca390a9c3 100644
> --- a/ChangeLog
> +++ b/ChangeLog
> @@ -1028,6 +1028,7 @@
>  	* nptl/thrd_sleep.c: Use clock_nanosleep_time64 instead of
> nanosleep.
>  	* sysdeps/unix/sysv/linux/nanosleep.c: Likewise.
>  	* sysdeps/unix/sysv/linux/nanosleep_nocancel.c: Likewise.
> +	* sysdeps/unix/sysv/linux/gettimeofday.c: Use
> clock_gettime64 syscall for gettimeofday. 
>  2019-06-20  Dmitry V. Levin  <ldv@altlinux.org>
>  	    Florian Weimer  <fweimer@redhat.com>
> diff --git a/sysdeps/unix/sysv/linux/gettimeofday.c
> b/sysdeps/unix/sysv/linux/gettimeofday.c index a74f03825a..151b1e606c
> 100644 --- a/sysdeps/unix/sysv/linux/gettimeofday.c
> +++ b/sysdeps/unix/sysv/linux/gettimeofday.c
> @@ -32,7 +32,35 @@
>  int
>  __gettimeofday (struct timeval *tv, struct timezone *tz)
>  {
> +#ifdef __ASSUME_TIME64_SYSCALLS

I'm not the glibc expert but according to [1], the
__ASSUME_TIME64_SYSCALLS will be defined also for __WORDSIZE = 64 archs.

This means that this code will be executed on x86_64 and return with an
ENOTSUPP error as those archs shall not define clock_gettime64 and
will just use the clock_settime.

Please consider re-using pattern from clock_settime conversion [2].

> +  int ret;
> +  struct __timespec64 now;
> +
> +  ret = INLINE_VSYSCALL (clock_gettime64, 2, CLOCK_REALTIME,
> +                         &now);
> +
> +  /* Convert from timespec to timeval */
> +  tv->tv_sec = now.tv_sec;
> +  tv->tv_usec = now.tv_nsec / 1000;
> +
> +  return ret;
> +#else
> +# ifdef __NR_clock_gettime64
> +  long int ret;
> +  struct __timespec64 now;
> +
> +  ret = INLINE_VSYSCALL (clock_gettime64, 2, CLOCK_REALTIME,
> +                         &now);
> +
> +  /* Convert from timespec to timeval */
> +  tv->tv_sec = now.tv_sec;
> +  tv->tv_usec = now.tv_nsec / 1000;
> +
> +  if (ret == 0 || errno != ENOSYS)
> +    return ret;
> +# endif
>    return INLINE_VSYSCALL (gettimeofday, 2, tv, tz);
> +#endif
>  }
>  libc_hidden_def (__gettimeofday)
>  weak_alias (__gettimeofday, gettimeofday)

Note:

[1] - 
https://github.com/lmajewski/y2038_glibc/commit/1fdbc6002101a78a8a6a076bbb642b3082c2225d

[2] -
https://github.com/lmajewski/y2038_glibc/commit/69f842a8519ca13ed11fab0ff1bcc6fa1a524192

Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [RFC v3 02/23] sysdeps/gettimeofday: Use clock_gettime64 if avaliable

[Lukasz Majewski]

[-- Attachment #1: Type: text/plain, Size: 3513 bytes --]

Hi,

> Hi Alistair,
> 
> > Not all architectures support the obsolete gettimeofday so use the
> > newer clock_gettime64 syscall if it is avaliable. This fixes RV32
> > build issues.
> > 
> > Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
> > ---
> >  ChangeLog                              |  1 +
> >  sysdeps/unix/sysv/linux/gettimeofday.c | 28
> > ++++++++++++++++++++++++++ 2 files changed, 29 insertions(+)
> > 
> > diff --git a/ChangeLog b/ChangeLog
> > index 477b9b49b3..9ca390a9c3 100644
> > --- a/ChangeLog
> > +++ b/ChangeLog
> > @@ -1028,6 +1028,7 @@
> >  	* nptl/thrd_sleep.c: Use clock_nanosleep_time64 instead of
> > nanosleep.
> >  	* sysdeps/unix/sysv/linux/nanosleep.c: Likewise.
> >  	* sysdeps/unix/sysv/linux/nanosleep_nocancel.c: Likewise.
> > +	* sysdeps/unix/sysv/linux/gettimeofday.c: Use
> > clock_gettime64 syscall for gettimeofday. 
> >  2019-06-20  Dmitry V. Levin  <ldv@altlinux.org>
> >  	    Florian Weimer  <fweimer@redhat.com>
> > diff --git a/sysdeps/unix/sysv/linux/gettimeofday.c
> > b/sysdeps/unix/sysv/linux/gettimeofday.c index
> > a74f03825a..151b1e606c 100644 ---
> > a/sysdeps/unix/sysv/linux/gettimeofday.c +++
> > b/sysdeps/unix/sysv/linux/gettimeofday.c @@ -32,7 +32,35 @@
> >  int
> >  __gettimeofday (struct timeval *tv, struct timezone *tz)
> >  {
> > +#ifdef __ASSUME_TIME64_SYSCALLS  
> 
> I'm not the glibc expert but according to [1], the
> __ASSUME_TIME64_SYSCALLS will be defined also for __WORDSIZE = 64
> archs.
> 
> This means that this code will be executed on x86_64 and return with
> an ENOTSUPP error as those archs shall not define clock_gettime64 and
     ^^^^^^^ - sorry s/ENOTSUPP/ENOSYS/g

> will just use the clock_settime.
> 
> Please consider re-using pattern from clock_settime conversion [2].
> 
> > +  int ret;
> > +  struct __timespec64 now;
> > +
> > +  ret = INLINE_VSYSCALL (clock_gettime64, 2, CLOCK_REALTIME,
> > +                         &now);
> > +
> > +  /* Convert from timespec to timeval */
> > +  tv->tv_sec = now.tv_sec;
> > +  tv->tv_usec = now.tv_nsec / 1000;
> > +
> > +  return ret;
> > +#else
> > +# ifdef __NR_clock_gettime64
> > +  long int ret;
> > +  struct __timespec64 now;
> > +
> > +  ret = INLINE_VSYSCALL (clock_gettime64, 2, CLOCK_REALTIME,
> > +                         &now);
> > +
> > +  /* Convert from timespec to timeval */
> > +  tv->tv_sec = now.tv_sec;
> > +  tv->tv_usec = now.tv_nsec / 1000;
> > +
> > +  if (ret == 0 || errno != ENOSYS)
> > +    return ret;
> > +# endif
> >    return INLINE_VSYSCALL (gettimeofday, 2, tv, tz);
> > +#endif
> >  }
> >  libc_hidden_def (__gettimeofday)
> >  weak_alias (__gettimeofday, gettimeofday)  
> 
> Note:
> 
> [1] - 
> https://github.com/lmajewski/y2038_glibc/commit/1fdbc6002101a78a8a6a076bbb642b3082c2225d
> 
> [2] -
> https://github.com/lmajewski/y2038_glibc/commit/69f842a8519ca13ed11fab0ff1bcc6fa1a524192
> 
> Best regards,
> 
> Lukasz Majewski
> 
> --
> 
> DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
> Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email:
> lukma@denx.de




Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [RFC v3 02/23] sysdeps/gettimeofday: Use clock_gettime64 if avaliable

[Alistair Francis]

On Wed, Jul 17, 2019 at 5:43 AM Lukasz Majewski <lukma@denx.de> wrote:
>
> Hi Alistair,
>
> > Not all architectures support the obsolete gettimeofday so use the
> > newer clock_gettime64 syscall if it is avaliable. This fixes RV32
> > build issues.
> >
> > Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
> > ---
> >  ChangeLog                              |  1 +
> >  sysdeps/unix/sysv/linux/gettimeofday.c | 28
> > ++++++++++++++++++++++++++ 2 files changed, 29 insertions(+)
> >
> > diff --git a/ChangeLog b/ChangeLog
> > index 477b9b49b3..9ca390a9c3 100644
> > --- a/ChangeLog
> > +++ b/ChangeLog
> > @@ -1028,6 +1028,7 @@
> >       * nptl/thrd_sleep.c: Use clock_nanosleep_time64 instead of
> > nanosleep.
> >       * sysdeps/unix/sysv/linux/nanosleep.c: Likewise.
> >       * sysdeps/unix/sysv/linux/nanosleep_nocancel.c: Likewise.
> > +     * sysdeps/unix/sysv/linux/gettimeofday.c: Use
> > clock_gettime64 syscall for gettimeofday.
> >  2019-06-20  Dmitry V. Levin  <ldv@altlinux.org>
> >           Florian Weimer  <fweimer@redhat.com>
> > diff --git a/sysdeps/unix/sysv/linux/gettimeofday.c
> > b/sysdeps/unix/sysv/linux/gettimeofday.c index a74f03825a..151b1e606c
> > 100644 --- a/sysdeps/unix/sysv/linux/gettimeofday.c
> > +++ b/sysdeps/unix/sysv/linux/gettimeofday.c
> > @@ -32,7 +32,35 @@
> >  int
> >  __gettimeofday (struct timeval *tv, struct timezone *tz)
> >  {
> > +#ifdef __ASSUME_TIME64_SYSCALLS
>
> I'm not the glibc expert but according to [1], the
> __ASSUME_TIME64_SYSCALLS will be defined also for __WORDSIZE = 64 archs.
>
> This means that this code will be executed on x86_64 and return with an
> ENOTSUPP error as those archs shall not define clock_gettime64 and
> will just use the clock_settime.
>
> Please consider re-using pattern from clock_settime conversion [2].

Yep, good point. I have updated this to better match what is being
discussed on the list and your patches.

Alistair

>
> > +  int ret;
> > +  struct __timespec64 now;
> > +
> > +  ret = INLINE_VSYSCALL (clock_gettime64, 2, CLOCK_REALTIME,
> > +                         &now);
> > +
> > +  /* Convert from timespec to timeval */
> > +  tv->tv_sec = now.tv_sec;
> > +  tv->tv_usec = now.tv_nsec / 1000;
> > +
> > +  return ret;
> > +#else
> > +# ifdef __NR_clock_gettime64
> > +  long int ret;
> > +  struct __timespec64 now;
> > +
> > +  ret = INLINE_VSYSCALL (clock_gettime64, 2, CLOCK_REALTIME,
> > +                         &now);
> > +
> > +  /* Convert from timespec to timeval */
> > +  tv->tv_sec = now.tv_sec;
> > +  tv->tv_usec = now.tv_nsec / 1000;
> > +
> > +  if (ret == 0 || errno != ENOSYS)
> > +    return ret;
> > +# endif
> >    return INLINE_VSYSCALL (gettimeofday, 2, tv, tz);
> > +#endif
> >  }
> >  libc_hidden_def (__gettimeofday)
> >  weak_alias (__gettimeofday, gettimeofday)
>
> Note:
>
> [1] -
> https://github.com/lmajewski/y2038_glibc/commit/1fdbc6002101a78a8a6a076bbb642b3082c2225d
>
> [2] -
> https://github.com/lmajewski/y2038_glibc/commit/69f842a8519ca13ed11fab0ff1bcc6fa1a524192
>
> Best regards,
>
> Lukasz Majewski
>
> --
>
> DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
> Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de

[RFC v3 04/23] sysdeps/clock_gettime: Use clock_gettime64 if avaliable

[Alistair Francis]

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 nptl/pthread_mutex_timedlock.c          |  7 ++++++
 sysdeps/unix/sysv/linux/clock_gettime.c | 30 +++++++++++++++++++++++++
 2 files changed, 37 insertions(+)

diff --git a/nptl/pthread_mutex_timedlock.c b/nptl/pthread_mutex_timedlock.c
index 52c258e33d..8d9cae7d87 100644
--- a/nptl/pthread_mutex_timedlock.c
+++ b/nptl/pthread_mutex_timedlock.c
@@ -402,10 +402,17 @@ __pthread_mutex_clocklock_common (pthread_mutex_t *mutex,
 		    /* Delay the thread until the timeout is reached.
 		       Then return ETIMEDOUT.  */
 		    struct timespec reltime;
+#ifdef __NR_clock_gettime64
+		    struct __timespec64 now;
+
+		    INTERNAL_SYSCALL (clock_gettime64, __err, 2, CLOCK_REALTIME,
+				      &now);
+#else
 		    struct timespec now;
 
 		    INTERNAL_SYSCALL (clock_gettime, __err, 2, clockid,
 				      &now);
+#endif
 		    reltime.tv_sec = abstime->tv_sec - now.tv_sec;
 		    reltime.tv_nsec = abstime->tv_nsec - now.tv_nsec;
 		    if (reltime.tv_nsec < 0)
diff --git a/sysdeps/unix/sysv/linux/clock_gettime.c b/sysdeps/unix/sysv/linux/clock_gettime.c
index 5fc47fb7dc..4832069c34 100644
--- a/sysdeps/unix/sysv/linux/clock_gettime.c
+++ b/sysdeps/unix/sysv/linux/clock_gettime.c
@@ -27,10 +27,40 @@
 #include <sysdep-vdso.h>
 
 /* Get current value of CLOCK and store it in TP.  */
+
+#if __WORDSIZE == 32
+int
+__clock_gettime (clockid_t clock_id, struct timespec *tp)
+{
+   int ret;
+
+#ifdef __NR_clock_gettime64
+  struct __timespec64 tp64;
+  ret = INLINE_VSYSCALL (clock_gettime64, 2, clock_id, &tp64);
+
+  tp->tv_sec = tp64.tv_sec;
+  tp->tv_nsec = tp64.tv_nsec;
+
+  if (! in_time_t_range (tp->tv_sec))
+    {
+      __set_errno (EOVERFLOW);
+      return -1;
+    }
+#endif
+
+#ifndef __ASSUME_TIME64_SYSCALLS
+  ret = INLINE_VSYSCALL (clock_gettime, 2, clock_id, tp);
+#endif
+
+  return ret;
+}
+#else
 int
 __clock_gettime (clockid_t clock_id, struct timespec *tp)
 {
   return INLINE_VSYSCALL (clock_gettime, 2, clock_id, tp);
 }
+#endif
+
 weak_alias (__clock_gettime, clock_gettime)
 libc_hidden_def (__clock_gettime)
-- 
2.22.0

Re: [RFC v3 04/23] sysdeps/clock_gettime: Use clock_gettime64 if avaliable

[Florian Weimer]

* Alistair Francis:

> diff --git a/nptl/pthread_mutex_timedlock.c b/nptl/pthread_mutex_timedlock.c
> index 52c258e33d..8d9cae7d87 100644
> --- a/nptl/pthread_mutex_timedlock.c
> +++ b/nptl/pthread_mutex_timedlock.c
> @@ -402,10 +402,17 @@ __pthread_mutex_clocklock_common (pthread_mutex_t *mutex,
>  		    /* Delay the thread until the timeout is reached.
>  		       Then return ETIMEDOUT.  */
>  		    struct timespec reltime;
> +#ifdef __NR_clock_gettime64
> +		    struct __timespec64 now;
> +
> +		    INTERNAL_SYSCALL (clock_gettime64, __err, 2, CLOCK_REALTIME,
> +				      &now);
> +#else
>  		    struct timespec now;
>  
>  		    INTERNAL_SYSCALL (clock_gettime, __err, 2, clockid,
>  				      &now);
> +#endif
>  		    reltime.tv_sec = abstime->tv_sec - now.tv_sec;
>  		    reltime.tv_nsec = abstime->tv_nsec - now.tv_nsec;
>  		    if (reltime.tv_nsec < 0)

I believe this needs to be updated for correctness (truncation of
tv_sec) if ever ported to architectures where __nanosleep_nocancel takes
a 32-bit time_t argument.  I don't know what our plans are regarding to
that.

If you had

#define __NR_clock_gettime64  __NR_clock_gettime

in <sysdep.h>, you wouldn't need this change.

> diff --git a/sysdeps/unix/sysv/linux/clock_gettime.c b/sysdeps/unix/sysv/linux/clock_gettime.c
> index 5fc47fb7dc..4832069c34 100644
> --- a/sysdeps/unix/sysv/linux/clock_gettime.c
> +++ b/sysdeps/unix/sysv/linux/clock_gettime.c
> @@ -27,10 +27,40 @@
>  #include <sysdep-vdso.h>
>  
>  /* Get current value of CLOCK and store it in TP.  */
> +
> +#if __WORDSIZE == 32
> +int
> +__clock_gettime (clockid_t clock_id, struct timespec *tp)
> +{
> +   int ret;
> +
> +#ifdef __NR_clock_gettime64
> +  struct __timespec64 tp64;
> +  ret = INLINE_VSYSCALL (clock_gettime64, 2, clock_id, &tp64);
> +
> +  tp->tv_sec = tp64.tv_sec;
> +  tp->tv_nsec = tp64.tv_nsec;
> +
> +  if (! in_time_t_range (tp->tv_sec))
> +    {
> +      __set_errno (EOVERFLOW);
> +      return -1;
> +    }
> +#endif
> +
> +#ifndef __ASSUME_TIME64_SYSCALLS
> +  ret = INLINE_VSYSCALL (clock_gettime, 2, clock_id, tp);
> +#endif
> +
> +  return ret;
> +}

I think this has the same problems as the timespec_get patch.

Thanks,
Florian

Re: [RFC v3 04/23] sysdeps/clock_gettime: Use clock_gettime64 if avaliable

[Arnd Bergmann]

On Wed, Jul 17, 2019 at 7:39 AM Florian Weimer <fweimer@redhat.com> wrote:
> > diff --git a/nptl/pthread_mutex_timedlock.c b/nptl/pthread_mutex_timedlock.c
> > index 52c258e33d..8d9cae7d87 100644
> > --- a/nptl/pthread_mutex_timedlock.c
> > +++ b/nptl/pthread_mutex_timedlock.c
> > @@ -402,10 +402,17 @@ __pthread_mutex_clocklock_common (pthread_mutex_t *mutex,
> >                   /* Delay the thread until the timeout is reached.
> >                      Then return ETIMEDOUT.  */
> >                   struct timespec reltime;
> > +#ifdef __NR_clock_gettime64
> > +                 struct __timespec64 now;
> > +
> > +                 INTERNAL_SYSCALL (clock_gettime64, __err, 2, CLOCK_REALTIME,
> > +                                   &now);
> > +#else
> >                   struct timespec now;
> >
> >                   INTERNAL_SYSCALL (clock_gettime, __err, 2, clockid,
> >                                     &now);
> > +#endif
> >                   reltime.tv_sec = abstime->tv_sec - now.tv_sec;
> >                   reltime.tv_nsec = abstime->tv_nsec - now.tv_nsec;
> >                   if (reltime.tv_nsec < 0)
>
> I believe this needs to be updated for correctness (truncation of
> tv_sec) if ever ported to architectures where __nanosleep_nocancel takes
> a 32-bit time_t argument.  I don't know what our plans are regarding to
> that.
>
> If you had
>
> #define __NR_clock_gettime64  __NR_clock_gettime
>
> in <sysdep.h>, you wouldn't need this change.

Could it be changed to just call internal __clock_gettime64() and
__nanosleep_nocancel_time64() functions that will then take care
of the architecture specifics like syscall number, vdso and
truncation?

        Arnd

Re: [RFC v3 04/23] sysdeps/clock_gettime: Use clock_gettime64 if avaliable

[Florian Weimer]

* Arnd Bergmann:

>> I believe this needs to be updated for correctness (truncation of
>> tv_sec) if ever ported to architectures where __nanosleep_nocancel takes
>> a 32-bit time_t argument.  I don't know what our plans are regarding to
>> that.
>>
>> If you had
>>
>> #define __NR_clock_gettime64  __NR_clock_gettime
>>
>> in <sysdep.h>, you wouldn't need this change.
>
> Could it be changed to just call internal __clock_gettime64() and
> __nanosleep_nocancel_time64() functions that will then take care
> of the architecture specifics like syscall number, vdso and
> truncation?

It depends on how these functions behave.  I think we need to figure out
if/how we maintain vDSO acceleration for clock_gettime and
clock_gettime64 first.

Thanks,
Florian

Re: [RFC v3 04/23] sysdeps/clock_gettime: Use clock_gettime64 if avaliable

[Arnd Bergmann]

On Wed, Jul 17, 2019 at 10:44 AM Florian Weimer <fweimer@redhat.com> wrote:
>
> * Arnd Bergmann:
>
> >> I believe this needs to be updated for correctness (truncation of
> >> tv_sec) if ever ported to architectures where __nanosleep_nocancel takes
> >> a 32-bit time_t argument.  I don't know what our plans are regarding to
> >> that.
> >>
> >> If you had
> >>
> >> #define __NR_clock_gettime64  __NR_clock_gettime
> >>
> >> in <sysdep.h>, you wouldn't need this change.
> >
> > Could it be changed to just call internal __clock_gettime64() and
> > __nanosleep_nocancel_time64() functions that will then take care
> > of the architecture specifics like syscall number, vdso and
> > truncation?
>
> It depends on how these functions behave.  I think we need to figure out
> if/how we maintain vDSO acceleration for clock_gettime and
> clock_gettime64 first.

Agreed. In the kernel, the generic clock_gettime64() support only just
landed for 5.3, and so far only arm32 and x86-32 have it, while at
least mips32, nds32, ppc32, s390 and sparc32 seem to have a the
old clock_gettime() and gettimeofday() vdso support.

For the internal __clock_gettime64() implementation, we clearly
want to call __vdso_clock_gettime64() whenever that is available,
but there does not seem to be a good fallback path from there
if it is not:

- if we try syscall(__NR_clock_gettime64) next, that is a performance
  regression in systems that don't care about 64-bit time_t
- if we try __vdso_clock_gettime() before __NR_clock_gettime64,
  that can result in incorrect behavior if user space actually relies
  on being able to work past 2038.

      Arnd

Re: [RFC v3 04/23] sysdeps/clock_gettime: Use clock_gettime64 if avaliable

[Florian Weimer]

* Arnd Bergmann:

> On Wed, Jul 17, 2019 at 10:44 AM Florian Weimer <fweimer@redhat.com> wrote:
>>
>> * Arnd Bergmann:
>>
>> >> I believe this needs to be updated for correctness (truncation of
>> >> tv_sec) if ever ported to architectures where __nanosleep_nocancel takes
>> >> a 32-bit time_t argument.  I don't know what our plans are regarding to
>> >> that.
>> >>
>> >> If you had
>> >>
>> >> #define __NR_clock_gettime64  __NR_clock_gettime
>> >>
>> >> in <sysdep.h>, you wouldn't need this change.
>> >
>> > Could it be changed to just call internal __clock_gettime64() and
>> > __nanosleep_nocancel_time64() functions that will then take care
>> > of the architecture specifics like syscall number, vdso and
>> > truncation?
>>
>> It depends on how these functions behave.  I think we need to figure out
>> if/how we maintain vDSO acceleration for clock_gettime and
>> clock_gettime64 first.
>
> Agreed. In the kernel, the generic clock_gettime64() support only just
> landed for 5.3, and so far only arm32 and x86-32 have it, while at
> least mips32, nds32, ppc32, s390 and sparc32 seem to have a the
> old clock_gettime() and gettimeofday() vdso support.
>
> For the internal __clock_gettime64() implementation, we clearly
> want to call __vdso_clock_gettime64() whenever that is available,
> but there does not seem to be a good fallback path from there
> if it is not:
>
> - if we try syscall(__NR_clock_gettime64) next, that is a performance
>   regression in systems that don't care about 64-bit time_t
> - if we try __vdso_clock_gettime() before __NR_clock_gettime64,
>   that can result in incorrect behavior if user space actually relies
>   on being able to work past 2038.

vDSO parsing happens ahead of time, during initialization, so the rules
are slightly different than with numbered syscall fallback.  System call
wrappers with both INLINE_VSYSCALL and fallback are really suspicious
and probably not a good idea in general.

The real question is whether vDSO and its constituent functions are
optional from a userspace ABI perspective.  If they are mandatory, we
can go straight to the 32-bit vDSO, and, possibly after that, the 32-bit
system call.  Instead of the vDSO implementation, we would install our
own emulation in the function pointer.  If the 64-bit vDSO is optional,
we would likely have to probe the 64-bit system call once, to keep
things sane and simple, and switch between the system call wrapper and
the fallback implementation (same as for the mandatory vDSO case).

Thanks,
Florian

Re: [RFC v3 04/23] sysdeps/clock_gettime: Use clock_gettime64 if avaliable

[Arnd Bergmann]

On Wed, Jul 17, 2019 at 5:16 PM Florian Weimer <fweimer@redhat.com> wrote:
>
> The real question is whether vDSO and its constituent functions are
> optional from a userspace ABI perspective.  If they are mandatory, we
> can go straight to the 32-bit vDSO, and, possibly after that, the 32-bit
> system call.  Instead of the vDSO implementation, we would install our
> own emulation in the function pointer.  If the 64-bit vDSO is optional,
> we would likely have to probe the 64-bit system call once, to keep
> things sane and simple, and switch between the system call wrapper and
> the fallback implementation (same as for the mandatory vDSO case).

Good question. I assumed that they would be optional because:

- Half the Linux architectures don't have any vdso support at all
- arm64 traditionally has no support for 32-bit vdso (added in
  linux-5.3), and still only contains it if an arm32 toolchain is available
  at kernel build time and we use gcc rather than clang (to be fixed
  in the future).
- I assumed that all libc implementations already need a fallback
  to plain syscalls to run on older kernels even for architectures that
  have it today.
- The recent rewrite of the vdso implementation in the kernel was not
  ready for 5.1, so no architecture has clock_gettime64() so far, but
  a lot of them clock_gettime()

I agree that assuming a vdso clock_gettime64() to be present on
all architectures would make this much easier for you, but I'd
still hope we can find a way to avoid truncating the times if you
do that.

I have two ideas for how that could be done:

- When building for a minimum kernel version of 5.1, don't
  fall back to __vdso_clock_gettime() or syscall(__NR_clock_gettime)
  but use the slow path for __clock_gettime64() if the vdso doesn't
  work.
- if __vdso_clock_gettime64() is unavailable and __vdso_clock_gettime()
  returns negative seconds, fall back to syscall(__NR_clock_gettime).

Would either of those meet your requirements?

       Arnd

Re: [RFC v3 04/23] sysdeps/clock_gettime: Use clock_gettime64 if avaliable

[Florian Weimer]

* Arnd Bergmann:

> I have two ideas for how that could be done:
>
> - When building for a minimum kernel version of 5.1, don't
>   fall back to __vdso_clock_gettime() or syscall(__NR_clock_gettime)
>   but use the slow path for __clock_gettime64() if the vdso doesn't
>   work.

Assuming that clock_gettime64 support is available, yes.

> - if __vdso_clock_gettime64() is unavailable and __vdso_clock_gettime()
>   returns negative seconds, fall back to syscall(__NR_clock_gettime).

I don't want to do anything like this.  I expect that some of us will
eventually use time namespaces to keep programs running (with incorrect
time).  If we make glibc itself time-sensitive, then things will get
horribly complex.

> Would either of those meet your requirements?

I don't have requirements.  I just want something that has limited
impact on 64-bit architectures.  I don't think probing at startup is too
bad, actually.

Thanks,
Florian

Re: [RFC v3 04/23] sysdeps/clock_gettime: Use clock_gettime64 if avaliable

[Arnd Bergmann]

On Thu, Jul 18, 2019 at 10:19 AM Florian Weimer <fweimer@redhat.com> wrote:
>
> * Arnd Bergmann:
>
> > I have two ideas for how that could be done:
> >
> > - When building for a minimum kernel version of 5.1, don't
> >   fall back to __vdso_clock_gettime() or syscall(__NR_clock_gettime)
> >   but use the slow path for __clock_gettime64() if the vdso doesn't
> >   work.
>
> Assuming that clock_gettime64 support is available, yes.

All 32-bit architectures have the clock_gettime64() syscall in 5.1.

> > Would either of those meet your requirements?
>
> I don't have requirements.  I just want something that has limited
> impact on 64-bit architectures.  I don't think probing at startup is too
> bad, actually.

Ok, fair enough.

      Arnd

Re: [RFC v3 04/23] sysdeps/clock_gettime: Use clock_gettime64 if avaliable

[Adhemerval Zanella]

On 18/07/2019 05:18, Florian Weimer wrote:
> * Arnd Bergmann:
> 
>> I have two ideas for how that could be done:
>>
>> - When building for a minimum kernel version of 5.1, don't
>>   fall back to __vdso_clock_gettime() or syscall(__NR_clock_gettime)
>>   but use the slow path for __clock_gettime64() if the vdso doesn't
>>   work.
> 
> Assuming that clock_gettime64 support is available, yes.
> 
>> - if __vdso_clock_gettime64() is unavailable and __vdso_clock_gettime()
>>   returns negative seconds, fall back to syscall(__NR_clock_gettime).
> 
> I don't want to do anything like this.  I expect that some of us will
> eventually use time namespaces to keep programs running (with incorrect
> time).  If we make glibc itself time-sensitive, then things will get
> horribly complex.
> 
>> Would either of those meet your requirements?
> 
> I don't have requirements.  I just want something that has limited
> impact on 64-bit architectures.  I don't think probing at startup is too
> bad, actually.

I hope that my vDSO refactor patchset [1] may simplify the support.
For the patchset it just a matter of define the expected vDSO symbol
name (HAVE_CLOCK_GETTIME_VSYSCALL for time32), and then
{INLINE,INTERNAL}_VSYSCALL will check, demangle, call, and fallback
to syscall.

I would expect that for time64_t it would be a matter to just add
a new define, HAVE_CLOCK_GETTIME64_VSYSCALL, where each architecture
would define if were the case.  

[1] https://sourceware.org/ml/libc-alpha/2019-06/msg00344.html

Re: [RFC v3 04/23] sysdeps/clock_gettime: Use clock_gettime64 if avaliable

[Alistair Francis]

On Tue, Jul 16, 2019 at 10:39 PM Florian Weimer <fweimer@redhat.com> wrote:
>
> * Alistair Francis:
>
> > diff --git a/nptl/pthread_mutex_timedlock.c b/nptl/pthread_mutex_timedlock.c
> > index 52c258e33d..8d9cae7d87 100644
> > --- a/nptl/pthread_mutex_timedlock.c
> > +++ b/nptl/pthread_mutex_timedlock.c
> > @@ -402,10 +402,17 @@ __pthread_mutex_clocklock_common (pthread_mutex_t *mutex,
> >                   /* Delay the thread until the timeout is reached.
> >                      Then return ETIMEDOUT.  */
> >                   struct timespec reltime;
> > +#ifdef __NR_clock_gettime64
> > +                 struct __timespec64 now;
> > +
> > +                 INTERNAL_SYSCALL (clock_gettime64, __err, 2, CLOCK_REALTIME,
> > +                                   &now);
> > +#else
> >                   struct timespec now;
> >
> >                   INTERNAL_SYSCALL (clock_gettime, __err, 2, clockid,
> >                                     &now);
> > +#endif
> >                   reltime.tv_sec = abstime->tv_sec - now.tv_sec;
> >                   reltime.tv_nsec = abstime->tv_nsec - now.tv_nsec;
> >                   if (reltime.tv_nsec < 0)
>
> I believe this needs to be updated for correctness (truncation of
> tv_sec) if ever ported to architectures where __nanosleep_nocancel takes
> a 32-bit time_t argument.  I don't know what our plans are regarding to
> that.
>
> If you had
>
> #define __NR_clock_gettime64  __NR_clock_gettime
>
> in <sysdep.h>, you wouldn't need this change.

Yep, you are right. I have used the #define and removed this change.

>
> > diff --git a/sysdeps/unix/sysv/linux/clock_gettime.c b/sysdeps/unix/sysv/linux/clock_gettime.c
> > index 5fc47fb7dc..4832069c34 100644
> > --- a/sysdeps/unix/sysv/linux/clock_gettime.c
> > +++ b/sysdeps/unix/sysv/linux/clock_gettime.c
> > @@ -27,10 +27,40 @@
> >  #include <sysdep-vdso.h>
> >
> >  /* Get current value of CLOCK and store it in TP.  */
> > +
> > +#if __WORDSIZE == 32
> > +int
> > +__clock_gettime (clockid_t clock_id, struct timespec *tp)
> > +{
> > +   int ret;
> > +
> > +#ifdef __NR_clock_gettime64
> > +  struct __timespec64 tp64;
> > +  ret = INLINE_VSYSCALL (clock_gettime64, 2, clock_id, &tp64);
> > +
> > +  tp->tv_sec = tp64.tv_sec;
> > +  tp->tv_nsec = tp64.tv_nsec;
> > +
> > +  if (! in_time_t_range (tp->tv_sec))
> > +    {
> > +      __set_errno (EOVERFLOW);
> > +      return -1;
> > +    }
> > +#endif
> > +
> > +#ifndef __ASSUME_TIME64_SYSCALLS
> > +  ret = INLINE_VSYSCALL (clock_gettime, 2, clock_id, tp);
> > +#endif
> > +
> > +  return ret;
> > +}
>
> I think this has the same problems as the timespec_get patch.

I'm still unclea what that problem is. Lukasz pointed out these don't
match the other time64_t conversions so I'll update it to use that
style anyway.

Alistair

>
> Thanks,
> Florian

Re: [RFC v3 04/23] sysdeps/clock_gettime: Use clock_gettime64 if avaliable

[Andreas Schwab]

On Jul 16 2019, Alistair Francis <alistair.francis@wdc.com> wrote:

> diff --git a/sysdeps/unix/sysv/linux/clock_gettime.c b/sysdeps/unix/sysv/linux/clock_gettime.c
> index 5fc47fb7dc..4832069c34 100644
> --- a/sysdeps/unix/sysv/linux/clock_gettime.c
> +++ b/sysdeps/unix/sysv/linux/clock_gettime.c
> @@ -27,10 +27,40 @@
>  #include <sysdep-vdso.h>
>  
>  /* Get current value of CLOCK and store it in TP.  */
> +
> +#if __WORDSIZE == 32
> +int
> +__clock_gettime (clockid_t clock_id, struct timespec *tp)
> +{
> +   int ret;
> +
> +#ifdef __NR_clock_gettime64
> +  struct __timespec64 tp64;
> +  ret = INLINE_VSYSCALL (clock_gettime64, 2, clock_id, &tp64);
> +
> +  tp->tv_sec = tp64.tv_sec;
> +  tp->tv_nsec = tp64.tv_nsec;

This needs to check if clock_gettime64 was successful.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: [RFC v3 04/23] sysdeps/clock_gettime: Use clock_gettime64 if avaliable

[Lukasz Majewski]

[-- Attachment #1: Type: text/plain, Size: 3848 bytes --]

Hi Alistair,

> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
> ---
>  nptl/pthread_mutex_timedlock.c          |  7 ++++++
>  sysdeps/unix/sysv/linux/clock_gettime.c | 30
> +++++++++++++++++++++++++ 2 files changed, 37 insertions(+)
> 
> diff --git a/nptl/pthread_mutex_timedlock.c
> b/nptl/pthread_mutex_timedlock.c index 52c258e33d..8d9cae7d87 100644
> --- a/nptl/pthread_mutex_timedlock.c
> +++ b/nptl/pthread_mutex_timedlock.c
> @@ -402,10 +402,17 @@ __pthread_mutex_clocklock_common
> (pthread_mutex_t *mutex, /* Delay the thread until the timeout is
> reached. Then return ETIMEDOUT.  */
>  		    struct timespec reltime;
> +#ifdef __NR_clock_gettime64
> +		    struct __timespec64 now;
> +
> +		    INTERNAL_SYSCALL (clock_gettime64, __err, 2,
> CLOCK_REALTIME,
> +				      &now);
> +#else
>  		    struct timespec now;
>  
>  		    INTERNAL_SYSCALL (clock_gettime, __err, 2,
> clockid, &now);
> +#endif
>  		    reltime.tv_sec = abstime->tv_sec - now.tv_sec;
>  		    reltime.tv_nsec = abstime->tv_nsec - now.tv_nsec;
>  		    if (reltime.tv_nsec < 0)
> diff --git a/sysdeps/unix/sysv/linux/clock_gettime.c
> b/sysdeps/unix/sysv/linux/clock_gettime.c index
> 5fc47fb7dc..4832069c34 100644 ---
> a/sysdeps/unix/sysv/linux/clock_gettime.c +++
> b/sysdeps/unix/sysv/linux/clock_gettime.c @@ -27,10 +27,40 @@
>  #include <sysdep-vdso.h>
>  
>  /* Get current value of CLOCK and store it in TP.  */
> +
> +#if __WORDSIZE == 32
> +int
> +__clock_gettime (clockid_t clock_id, struct timespec *tp)
> +{
> +   int ret;
> +
> +#ifdef __NR_clock_gettime64
> +  struct __timespec64 tp64;
> +  ret = INLINE_VSYSCALL (clock_gettime64, 2, clock_id, &tp64);
> +
> +  tp->tv_sec = tp64.tv_sec;
> +  tp->tv_nsec = tp64.tv_nsec;
> +
> +  if (! in_time_t_range (tp->tv_sec))
> +    {
> +      __set_errno (EOVERFLOW);
> +      return -1;
> +    }
> +#endif
> +
> +#ifndef __ASSUME_TIME64_SYSCALLS

I think that the context of using the __ASSUME_TIME64_SYSCALLS doesn't
comply with the semantics which was proposed in its introduction patch
[1].

In short:

This define means that the system is supporting 64 bit time, not that
it has (can use) introduced in Linux 5.1 64 bit time related syscalls
(i.e. clock_gettime64, clock_settime64, clock_nanosleep64).

Maybe you could consider using the "conversion" approach as proposed
for __clock_settime64 conversion [2]?

The version in [2] is more "developer/validator" friendly (as suggested
by Arnd) as it uses for __TIMESIZE != 64 archs the 32 bit clock_settime.

The version from [3] is the one recommended by Joseph (which does the
32 to 64 bit conversion and calls clock_settime64).



The full code for a reference with clock_settime conversion and Y2038
support [4].

> +  ret = INLINE_VSYSCALL (clock_gettime, 2, clock_id, tp);
> +#endif
> +
> +  return ret;
> +}
> +#else
>  int
>  __clock_gettime (clockid_t clock_id, struct timespec *tp)
>  {
>    return INLINE_VSYSCALL (clock_gettime, 2, clock_id, tp);
>  }
> +#endif
> +
>  weak_alias (__clock_gettime, clock_gettime)
>  libc_hidden_def (__clock_gettime)

Note:

[1] -
https://github.com/lmajewski/y2038_glibc/commit/1fdbc6002101a78a8a6a076bbb642b3082c2225d

[2] -
https://github.com/lmajewski/y2038_glibc/commit/69f842a8519ca13ed11fab0ff1bcc6fa1a524192

[3] -
https://github.com/lmajewski/y2038_glibc/commit/fa0f5ff6c942beca383daeff3d48829829ace5b1

[4] -

https://github.com/lmajewski/y2038_glibc/commits/Y2038-2.29-glibc-__clock-internal-struct-timespec-v6

Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[RFC v3 01/23] sysdeps/nanosleep: Use clock_nanosleep_time64 if avaliable

[Alistair Francis]

The nanosleep syscall is not supported on newer 32-bit platforms (such
as RV32). To fix this issue let's use clock_nanosleep_time64 if it is
avaliable.

Let's use CLOCK_REALTIME when calling clock_nanosleep_time64 as the
Linux specification says:
  "POSIX.1 specifies that nanosleep() should measure time against the
   CLOCK_REALTIME clock. However, Linux measures the time using the
   CLOCK_MONOTONIC clock. This probably does not matter, since the POSIX.1
   specification for clock_settime(2) says that discontinuous changes in
   CLOCK_REALTIME should not affect nanosleep()"

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 ChangeLog                                    |  6 +++
 nptl/thrd_sleep.c                            | 41 +++++++++++++++-
 sysdeps/unix/sysv/linux/clock_nanosleep.c    | 42 +++++++++++++++-
 sysdeps/unix/sysv/linux/nanosleep.c          | 50 ++++++++++++++++++++
 sysdeps/unix/sysv/linux/nanosleep_nocancel.c | 11 +++++
 5 files changed, 147 insertions(+), 3 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index bd57a815b0..477b9b49b3 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1023,6 +1023,12 @@
 
 	* support/xtime.h: Add xclock_now() helper function.
 
+2019-06-21  Alistair Francis  <alistair.francis@wdc.com>
+
+	* nptl/thrd_sleep.c: Use clock_nanosleep_time64 instead of nanosleep.
+	* sysdeps/unix/sysv/linux/nanosleep.c: Likewise.
+	* sysdeps/unix/sysv/linux/nanosleep_nocancel.c: Likewise.
+
 2019-06-20  Dmitry V. Levin  <ldv@altlinux.org>
 	    Florian Weimer  <fweimer@redhat.com>
 
diff --git a/nptl/thrd_sleep.c b/nptl/thrd_sleep.c
index 07a51808df..e7873a7170 100644
--- a/nptl/thrd_sleep.c
+++ b/nptl/thrd_sleep.c
@@ -21,11 +21,50 @@
 
 #include "thrd_priv.h"
 
+#if __TIMESIZE == 32
+struct timespec64
+{
+  long int tv_sec;   /* Seconds.  */
+  long int tv_nsec;  /* Nanoseconds.  */
+};
+#endif
+
 int
 thrd_sleep (const struct timespec* time_point, struct timespec* remaining)
 {
   INTERNAL_SYSCALL_DECL (err);
-  int ret = INTERNAL_SYSCALL_CANCEL (nanosleep, err, time_point, remaining);
+  int ret;
+
+#ifdef __ASSUME_TIME64_SYSCALLS
+  ret = INTERNAL_SYSCALL_CANCEL (clock_nanosleep_time64, err, CLOCK_REALTIME,
+                                 0, time_point, remaining);
+#else
+# ifdef __NR_clock_nanosleep_time64
+#  if __TIMESIZE == 64
+  long int ret_64;
+
+  ret_64 = INTERNAL_SYSCALL_CANCEL (clock_nanosleep_time64, err, CLOCK_REALTIME,
+                                    0, time_point, remaining);
+
+  if (ret_64 == 0 || errno != ENOSYS)
+    ret = ret_64;
+#  else
+  timespec64 ts;
+
+  ret = INTERNAL_SYSCALL_CANCEL (clock_nanosleep_time64, err,
+                                 CLOCK_REALTIME, 0, time_point,
+                                 ts);
+
+  if (ret == 0 || errno != ENOSYS) {
+    remaining->tv_sec = ts.tv_sec;
+    remaining->tv_nsec = ts.tv_nsec;
+    return ret;
+  }
+#  endif
+# endif
+  ret =  INTERNAL_SYSCALL_CANCEL (nanosleep, err, time_point, remaining);
+#endif
+
   if (INTERNAL_SYSCALL_ERROR_P (ret, err))
     {
       /* C11 states thrd_sleep function returns -1 if it has been interrupted
diff --git a/sysdeps/unix/sysv/linux/clock_nanosleep.c b/sysdeps/unix/sysv/linux/clock_nanosleep.c
index 0cb6614dc9..84bd5f8afc 100644
--- a/sysdeps/unix/sysv/linux/clock_nanosleep.c
+++ b/sysdeps/unix/sysv/linux/clock_nanosleep.c
@@ -21,6 +21,13 @@
 #include <sysdep-cancel.h>
 #include "kernel-posix-cpu-timers.h"
 
+#if __TIMESIZE == 32
+struct timespec64
+{
+  long int tv_sec;   /* Seconds.  */
+  long int tv_nsec;  /* Nanoseconds.  */
+};
+#endif
 
 /* We can simply use the syscall.  The CPU clocks are not supported
    with this function.  */
@@ -28,6 +35,8 @@ int
 __clock_nanosleep (clockid_t clock_id, int flags, const struct timespec *req,
 		   struct timespec *rem)
 {
+  int r;
+
   if (clock_id == CLOCK_THREAD_CPUTIME_ID)
     return EINVAL;
   if (clock_id == CLOCK_PROCESS_CPUTIME_ID)
@@ -36,8 +45,37 @@ __clock_nanosleep (clockid_t clock_id, int flags, const struct timespec *req,
   /* If the call is interrupted by a signal handler or encounters an error,
      it returns a positive value similar to errno.  */
   INTERNAL_SYSCALL_DECL (err);
-  int r = INTERNAL_SYSCALL_CANCEL (clock_nanosleep, err, clock_id, flags,
-				   req, rem);
+
+#ifdef __ASSUME_TIME64_SYSCALLS
+  r = INTERNAL_SYSCALL_CANCEL (clock_nanosleep_time64, err, clock_id,
+                               flags, req, rem);
+#else
+# ifdef __NR_clock_nanosleep_time64
+#  if __TIMESIZE == 64
+  long int ret_64;
+
+  ret_64 = INTERNAL_SYSCALL_CANCEL (clock_nanosleep_time64, err, clock_id,
+                                    flags, req, rem);
+
+  if (ret_64 == 0 || errno != ENOSYS)
+    r = ret_64;
+#  else
+  timespec64 ts;
+
+  r = INTERNAL_SYSCALL_CANCEL (clock_nanosleep_time64, err,
+                               clock_id, flags, req,
+                               ts);
+
+  if (r == 0 || errno != ENOSYS) {
+    rem->tv_sec = ts.tv_sec;
+    rem->tv_nsec = ts.tv_nsec;
+    return r;
+  }
+#  endif
+# endif
+  r =  INTERNAL_SYSCALL_CANCEL (clock_nanosleep, err, req, rem);
+#endif
+
   return (INTERNAL_SYSCALL_ERROR_P (r, err)
 	  ? INTERNAL_SYSCALL_ERRNO (r, err) : 0);
 }
diff --git a/sysdeps/unix/sysv/linux/nanosleep.c b/sysdeps/unix/sysv/linux/nanosleep.c
index f14ae565af..0c54f21e07 100644
--- a/sysdeps/unix/sysv/linux/nanosleep.c
+++ b/sysdeps/unix/sysv/linux/nanosleep.c
@@ -20,12 +20,62 @@
 #include <sysdep-cancel.h>
 #include <not-cancel.h>
 
+#if defined(__ASSUME_TIME64_SYSCALLS) || defined(__NR_clock_nanosleep_time64)
+static int
+__nanosleep_time64_64 (const struct timespec *requested_time,
+                       struct timespec *remaining)
+{
+  return SYSCALL_CANCEL (clock_nanosleep_time64, CLOCK_REALTIME, 0,
+                         requested_time, remaining);
+}
+
+#if __TIMESIZE == 32
+struct timespec64
+{
+  long int tv_sec;   /* Seconds.  */
+  long int tv_nsec;  /* Nanoseconds.  */
+};
+
+static int
+__nanosleep_time64_32 (const struct timespec *requested_time,
+                       struct timespec *remaining)
+{
+  timespec64 ts;
+
+  long int ret = SYSCALL_CANCEL (clock_nanosleep_time64, CLOCK_REALTIME, 0,
+                                 requested_time, &ts);
+
+  remaining->tv_sec = ts.tv_sec;
+  remaining->tv_nsec = ts.tv_nsec;
+
+  return ret;
+}
+#endif
+#endif
+
 /* Pause execution for a number of nanoseconds.  */
 int
 __nanosleep (const struct timespec *requested_time,
 	     struct timespec *remaining)
 {
+#ifdef __ASSUME_TIME64_SYSCALLS
+  return __nanosleep_time64_64 (requested_time, remaining);
+#else
+# ifdef __NR_clock_nanosleep_time64
+#  if __TIMESIZE == 64
+  long int ret = __nanosleep_time64_64 (requested_time, remaining);
+
+  if (ret == 0 || errno != ENOSYS)
+    return ret;
+#  else
+  long int ret = __nanosleep_time64_32 (requested_time, remaining);
+
+  if (ret == 0 || errno != ENOSYS)
+    return ret;
+#  endif
+# endif
   return SYSCALL_CANCEL (nanosleep, requested_time, remaining);
+#endif
 }
 hidden_def (__nanosleep)
 weak_alias (__nanosleep, nanosleep)
diff --git a/sysdeps/unix/sysv/linux/nanosleep_nocancel.c b/sysdeps/unix/sysv/linux/nanosleep_nocancel.c
index 122ba627ff..f411a1c6c2 100644
--- a/sysdeps/unix/sysv/linux/nanosleep_nocancel.c
+++ b/sysdeps/unix/sysv/linux/nanosleep_nocancel.c
@@ -24,6 +24,17 @@ int
 __nanosleep_nocancel (const struct timespec *requested_time,
 		      struct timespec *remaining)
 {
+#ifdef __ASSUME_TIME64_SYSCALLS
+  return INLINE_SYSCALL_CALL (clock_nanosleep_time64, CLOCK_REALTIME, 0,
+                         requested_time, remaining);
+#else
+# ifdef __NR_clock_nanosleep_time64
+  long int ret = INLINE_SYSCALL_CALL (clock_nanosleep_time64, CLOCK_REALTIME, 0,
+                                 requested_time, remaining);
+  if (ret == 0 || errno != ENOSYS)
+    return ret;
+# endif
   return INLINE_SYSCALL_CALL (nanosleep, requested_time, remaining);
+#endif
 }
 hidden_def (__nanosleep_nocancel)
-- 
2.22.0

Re: [RFC v3 01/23] sysdeps/nanosleep: Use clock_nanosleep_time64 if avaliable

[Florian Weimer]

* Alistair Francis:

> +#if __TIMESIZE == 32
> +struct timespec64
> +{
> +  long int tv_sec;   /* Seconds.  */
> +  long int tv_nsec;  /* Nanoseconds.  */
> +};
> +#endif
> +
>  int
>  thrd_sleep (const struct timespec* time_point, struct timespec* remaining)
>  {
>    INTERNAL_SYSCALL_DECL (err);
> -  int ret = INTERNAL_SYSCALL_CANCEL (nanosleep, err, time_point, remaining);
> +  int ret;
> +
> +#ifdef __ASSUME_TIME64_SYSCALLS
> +  ret = INTERNAL_SYSCALL_CANCEL (clock_nanosleep_time64, err, CLOCK_REALTIME,
> +                                 0, time_point, remaining);
> +#else
> +# ifdef __NR_clock_nanosleep_time64
> +#  if __TIMESIZE == 64
> +  long int ret_64;
> +
> +  ret_64 = INTERNAL_SYSCALL_CANCEL (clock_nanosleep_time64, err, CLOCK_REALTIME,
> +                                    0, time_point, remaining);
> +
> +  if (ret_64 == 0 || errno != ENOSYS)
> +    ret = ret_64;
> +#  else
> +  timespec64 ts;
> +
> +  ret = INTERNAL_SYSCALL_CANCEL (clock_nanosleep_time64, err,
> +                                 CLOCK_REALTIME, 0, time_point,
> +                                 ts);
> +
> +  if (ret == 0 || errno != ENOSYS) {
> +    remaining->tv_sec = ts.tv_sec;
> +    remaining->tv_nsec = ts.tv_nsec;
> +    return ret;
> +  }
> +#  endif
> +# endif
> +  ret =  INTERNAL_SYSCALL_CANCEL (nanosleep, err, time_point, remaining);
> +#endif
> +

I think this changes the ABI of thrd_sleep if
__NR_clock_nanosleep_time64 is defined (and __NR_nanosleep was defined
before).

Thanks,
Florian

Re: [RFC v3 01/23] sysdeps/nanosleep: Use clock_nanosleep_time64 if avaliable

[Alistair Francis]

On Tue, Jul 16, 2019 at 10:16 PM Florian Weimer <fweimer@redhat.com> wrote:
>
> * Alistair Francis:
>
> > +#if __TIMESIZE == 32
> > +struct timespec64
> > +{
> > +  long int tv_sec;   /* Seconds.  */
> > +  long int tv_nsec;  /* Nanoseconds.  */
> > +};
> > +#endif
> > +
> >  int
> >  thrd_sleep (const struct timespec* time_point, struct timespec* remaining)
> >  {
> >    INTERNAL_SYSCALL_DECL (err);
> > -  int ret = INTERNAL_SYSCALL_CANCEL (nanosleep, err, time_point, remaining);
> > +  int ret;
> > +
> > +#ifdef __ASSUME_TIME64_SYSCALLS
> > +  ret = INTERNAL_SYSCALL_CANCEL (clock_nanosleep_time64, err, CLOCK_REALTIME,
> > +                                 0, time_point, remaining);
> > +#else
> > +# ifdef __NR_clock_nanosleep_time64
> > +#  if __TIMESIZE == 64
> > +  long int ret_64;
> > +
> > +  ret_64 = INTERNAL_SYSCALL_CANCEL (clock_nanosleep_time64, err, CLOCK_REALTIME,
> > +                                    0, time_point, remaining);
> > +
> > +  if (ret_64 == 0 || errno != ENOSYS)
> > +    ret = ret_64;
> > +#  else
> > +  timespec64 ts;
> > +
> > +  ret = INTERNAL_SYSCALL_CANCEL (clock_nanosleep_time64, err,
> > +                                 CLOCK_REALTIME, 0, time_point,
> > +                                 ts);
> > +
> > +  if (ret == 0 || errno != ENOSYS) {
> > +    remaining->tv_sec = ts.tv_sec;
> > +    remaining->tv_nsec = ts.tv_nsec;
> > +    return ret;
> > +  }
> > +#  endif
> > +# endif
> > +  ret =  INTERNAL_SYSCALL_CANCEL (nanosleep, err, time_point, remaining);
> > +#endif
> > +
>
> I think this changes the ABI of thrd_sleep if
> __NR_clock_nanosleep_time64 is defined (and __NR_nanosleep was defined
> before).

I'm not sure I understand, which part changes the ABI?

Alistair

>
> Thanks,
> Florian

Re: [RFC v3 01/23] sysdeps/nanosleep: Use clock_nanosleep_time64 if avaliable

[Stepan Golosunov]

19.07.2019 ц≈ 10:25:46 -0700 Alistair Francis ц▌ц│ц░ц┴ц⌠ц│ц▄:
> On Tue, Jul 16, 2019 at 10:16 PM Florian Weimer <fweimer@redhat.com> wrote:
> >
> > * Alistair Francis:
> >
> > > +#if __TIMESIZE == 32
> > > +struct timespec64
> > > +{
> > > +  long int tv_sec;   /* Seconds.  */
> > > +  long int tv_nsec;  /* Nanoseconds.  */
> > > +};
> > > +#endif
> > > +
> > >  int
> > >  thrd_sleep (const struct timespec* time_point, struct timespec* remaining)
> > >  {
> > >    INTERNAL_SYSCALL_DECL (err);
> > > -  int ret = INTERNAL_SYSCALL_CANCEL (nanosleep, err, time_point, remaining);
> > > +  int ret;
> > > +
> > > +#ifdef __ASSUME_TIME64_SYSCALLS
> > > +  ret = INTERNAL_SYSCALL_CANCEL (clock_nanosleep_time64, err, CLOCK_REALTIME,
> > > +                                 0, time_point, remaining);
> > > +#else
> > > +# ifdef __NR_clock_nanosleep_time64
> > > +#  if __TIMESIZE == 64
> > > +  long int ret_64;
> > > +
> > > +  ret_64 = INTERNAL_SYSCALL_CANCEL (clock_nanosleep_time64, err, CLOCK_REALTIME,
> > > +                                    0, time_point, remaining);
> > > +
> > > +  if (ret_64 == 0 || errno != ENOSYS)
> > > +    ret = ret_64;
> > > +#  else
> > > +  timespec64 ts;
> > > +
> > > +  ret = INTERNAL_SYSCALL_CANCEL (clock_nanosleep_time64, err,
> > > +                                 CLOCK_REALTIME, 0, time_point,
> > > +                                 ts);
> > > +
> > > +  if (ret == 0 || errno != ENOSYS) {
> > > +    remaining->tv_sec = ts.tv_sec;
> > > +    remaining->tv_nsec = ts.tv_nsec;
> > > +    return ret;
> > > +  }
> > > +#  endif
> > > +# endif
> > > +  ret =  INTERNAL_SYSCALL_CANCEL (nanosleep, err, time_point, remaining);
> > > +#endif
> > > +
> >
> > I think this changes the ABI of thrd_sleep if
> > __NR_clock_nanosleep_time64 is defined (and __NR_nanosleep was defined
> > before).
> 
> I'm not sure I understand, which part changes the ABI?

Parts where time_point (2 cases) and remaining (1 case) are passed to
the clock_nanosleep_time64 syscall while __TIMESIZE == 32.

Re: [RFC v3 01/23] sysdeps/nanosleep: Use clock_nanosleep_time64 if avaliable

[Alistair Francis]

On Sat, Jul 20, 2019 at 7:24 AM Stepan Golosunov <stepan@golosunov.pp.ru> wrote:
>
> 19.07.2019 в 10:25:46 -0700 Alistair Francis написал:
> > On Tue, Jul 16, 2019 at 10:16 PM Florian Weimer <fweimer@redhat.com> wrote:
> > >
> > > * Alistair Francis:
> > >
> > > > +#if __TIMESIZE == 32
> > > > +struct timespec64
> > > > +{
> > > > +  long int tv_sec;   /* Seconds.  */
> > > > +  long int tv_nsec;  /* Nanoseconds.  */
> > > > +};
> > > > +#endif
> > > > +
> > > >  int
> > > >  thrd_sleep (const struct timespec* time_point, struct timespec* remaining)
> > > >  {
> > > >    INTERNAL_SYSCALL_DECL (err);
> > > > -  int ret = INTERNAL_SYSCALL_CANCEL (nanosleep, err, time_point, remaining);
> > > > +  int ret;
> > > > +
> > > > +#ifdef __ASSUME_TIME64_SYSCALLS
> > > > +  ret = INTERNAL_SYSCALL_CANCEL (clock_nanosleep_time64, err, CLOCK_REALTIME,
> > > > +                                 0, time_point, remaining);
> > > > +#else
> > > > +# ifdef __NR_clock_nanosleep_time64
> > > > +#  if __TIMESIZE == 64
> > > > +  long int ret_64;
> > > > +
> > > > +  ret_64 = INTERNAL_SYSCALL_CANCEL (clock_nanosleep_time64, err, CLOCK_REALTIME,
> > > > +                                    0, time_point, remaining);
> > > > +
> > > > +  if (ret_64 == 0 || errno != ENOSYS)
> > > > +    ret = ret_64;
> > > > +#  else
> > > > +  timespec64 ts;
> > > > +
> > > > +  ret = INTERNAL_SYSCALL_CANCEL (clock_nanosleep_time64, err,
> > > > +                                 CLOCK_REALTIME, 0, time_point,
> > > > +                                 ts);
> > > > +
> > > > +  if (ret == 0 || errno != ENOSYS) {
> > > > +    remaining->tv_sec = ts.tv_sec;
> > > > +    remaining->tv_nsec = ts.tv_nsec;
> > > > +    return ret;
> > > > +  }
> > > > +#  endif
> > > > +# endif
> > > > +  ret =  INTERNAL_SYSCALL_CANCEL (nanosleep, err, time_point, remaining);
> > > > +#endif
> > > > +
> > >
> > > I think this changes the ABI of thrd_sleep if
> > > __NR_clock_nanosleep_time64 is defined (and __NR_nanosleep was defined
> > > before).
> >
> > I'm not sure I understand, which part changes the ABI?
>
> Parts where time_point (2 cases) and remaining (1 case) are passed to
> the clock_nanosleep_time64 syscall while __TIMESIZE == 32.

Yep, I think I have fixed everything up in the next version.

Alistair

[RFC v3 22/23] Add RISC-V 32-bit target to build-many-glibcs.py

[Alistair Francis]

From: Zong Li <zongbox@gmail.com>

Support building three variant of 32 bit RISC-V glibc as follows:
- riscv32-linux-gnu-rv32imac-ilp32
- riscv32-linux-gnu-rv32imafdc-ilp32
- riscv32-linux-gnu-rv32imafdc-ilp32d

2018-11-29  Zong Li  <zong@andestech.com>

	* scripts/build-many-glibcs.py (Context): Add rv32 targets.
---
 ChangeLog                    | 11 +++++++++++
 scripts/build-many-glibcs.py | 15 +++++++++++++++
 2 files changed, 26 insertions(+)

diff --git a/ChangeLog b/ChangeLog
index 8b8b038b23..9d4c36f10a 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1124,6 +1124,17 @@
 	* sysdeps/unix/sysv/linux/riscv/shlib-versions: Likewise.
 	* sysdeps/riscv/preconfigure: Likewise.
 	* sysdeps/riscv/rv32/fix-fp-int-convert-overflow.h: New file.
+	* sysdeps/riscv/rv32/Implies-after: New file.
+	* sysdeps/riscv/rv32/rvd/Implies: Likewise.
+	* sysdeps/riscv/rv32/rvf/Implies: Likewise.
+	* sysdeps/unix/sysv/linux/riscv/rv32/Implies: Likewise.
+	* sysdeps/unix/sysv/linux/riscv/Makefile: Support rv32.
+	* sysdeps/unix/sysv/linux/riscv/configure: Likewise.
+	* sysdeps/unix/sysv/linux/riscv/configure.ac: Likewise.
+	* sysdeps/unix/sysv/linux/riscv/shlib-versions: Likewise.
+	* sysdeps/riscv/preconfigure: Likewise.
+	* sysdeps/riscv/rv32/fix-fp-int-convert-overflow.h: New file.
+	* scripts/build-many-glibcs.py (Context): Add rv32 targets.
 
 2019-06-20  Dmitry V. Levin  <ldv@altlinux.org>
 	    Florian Weimer  <fweimer@redhat.com>
diff --git a/scripts/build-many-glibcs.py b/scripts/build-many-glibcs.py
index dacd116f8e..5167dea9ce 100755
--- a/scripts/build-many-glibcs.py
+++ b/scripts/build-many-glibcs.py
@@ -320,6 +320,21 @@ class Context(object):
         self.add_config(arch='powerpc64le',
                         os_name='linux-gnu',
                         gcc_cfg=['--disable-multilib', '--enable-secureplt'])
+        self.add_config(arch='riscv32',
+                        os_name='linux-gnu',
+                        variant='rv32imac-ilp32',
+                        gcc_cfg=['--with-arch=rv32imac', '--with-abi=ilp32',
+                                 '--disable-multilib'])
+        self.add_config(arch='riscv32',
+                        os_name='linux-gnu',
+                        variant='rv32imafdc-ilp32',
+                        gcc_cfg=['--with-arch=rv32imafdc', '--with-abi=ilp32',
+                                 '--disable-multilib'])
+        self.add_config(arch='riscv32',
+                        os_name='linux-gnu',
+                        variant='rv32imafdc-ilp32d',
+                        gcc_cfg=['--with-arch=rv32imafdc', '--with-abi=ilp32d',
+                                 '--disable-multilib'])
         self.add_config(arch='riscv64',
                         os_name='linux-gnu',
                         variant='rv64imac-lp64',
-- 
2.22.0

[RFC v3 16/23] RISC-V: The ABI implementation for the 32-bit

[Alistair Francis]

From: Zong Li <zongbox@gmail.com>

This patch adds the ABI implementation about 32 bit version. It contains
the Linux-specific and RISC-V architecture code, I've collected here.

2018-11-29  Zong Li  <zong@andestech.com>

	* sysdeps/riscv/bits/wordsize.h: Supprt rv32.
	* sysdeps/riscv/nptl/bits/pthreadtypes-arch.h: Likewise.
	* sysdeps/riscv/sfp-machine.h: Likewise.
	* sysdeps/riscv/sys/asm.h: Likewise.
	* sysdeps/unix/sysv/linux/riscv/rv32/jmp_buf-macros.h: New file.

[ Changes by AF:
 - Remove lock64.c
]
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 ChangeLog                                     |  8 +++
 sysdeps/riscv/bits/wordsize.h                 |  4 +-
 sysdeps/riscv/nptl/bits/pthreadtypes-arch.h   | 25 ++++++++-
 sysdeps/riscv/sfp-machine.h                   | 27 +++++++++-
 sysdeps/riscv/sys/asm.h                       |  5 +-
 .../sysv/linux/riscv/rv32/jmp_buf-macros.h    | 53 +++++++++++++++++++
 6 files changed, 118 insertions(+), 4 deletions(-)
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/jmp_buf-macros.h

diff --git a/ChangeLog b/ChangeLog
index 4155ead682..e0446d70a0 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1043,6 +1043,14 @@
 	* sysdeps/unix/sysv/linux/riscv/ldconfig.h (LD_SO_ABI): Support rv32.
 	* sysdeps/unix/sysv/linux/riscv/dl-cache.h (add_system_dir): Add
 	libraries path for rv32.
+	* sysdeps/unix/sysv/linux/riscv/ldconfig.h (LD_SO_ABI): Support rv32.
+	* sysdeps/unix/sysv/linux/riscv/dl-cache.h (add_system_dir): Add
+	libraries path for rv32.
+	* sysdeps/riscv/bits/wordsize.h: Supprt rv32.
+	* sysdeps/riscv/nptl/bits/pthreadtypes-arch.h: Likewise.
+	* sysdeps/riscv/sfp-machine.h: Likewise.
+	* sysdeps/riscv/sys/asm.h: Likewise.
+	* sysdeps/unix/sysv/linux/riscv/rv32/jmp_buf-macros.h: New file.
 
 2019-06-20  Dmitry V. Levin  <ldv@altlinux.org>
 	    Florian Weimer  <fweimer@redhat.com>
diff --git a/sysdeps/riscv/bits/wordsize.h b/sysdeps/riscv/bits/wordsize.h
index 0b8cd8fefd..f10be0144c 100644
--- a/sysdeps/riscv/bits/wordsize.h
+++ b/sysdeps/riscv/bits/wordsize.h
@@ -25,5 +25,7 @@
 #if __riscv_xlen == 64
 # define __WORDSIZE_TIME64_COMPAT32 1
 #else
-# error "rv32i-based targets are not supported"
+# define __WORDSIZE_TIME64_COMPAT32 0
+# define __WORDSIZE32_SIZE_ULONG    0
+# define __WORDSIZE32_PTRDIFF_LONG  0
 #endif
diff --git a/sysdeps/riscv/nptl/bits/pthreadtypes-arch.h b/sysdeps/riscv/nptl/bits/pthreadtypes-arch.h
index e3fecc3208..4b08f7c692 100644
--- a/sysdeps/riscv/nptl/bits/pthreadtypes-arch.h
+++ b/sysdeps/riscv/nptl/bits/pthreadtypes-arch.h
@@ -32,7 +32,15 @@
 # define __SIZEOF_PTHREAD_BARRIER_T 		32
 # define __SIZEOF_PTHREAD_BARRIERATTR_T 	 4
 #else
-# error "rv32i-based systems are not supported"
+# define __SIZEOF_PTHREAD_ATTR_T 		32
+# define __SIZEOF_PTHREAD_MUTEX_T 		32
+# define __SIZEOF_PTHREAD_MUTEXATTR_T 		 4
+# define __SIZEOF_PTHREAD_COND_T 		48
+# define __SIZEOF_PTHREAD_CONDATTR_T 		 4
+# define __SIZEOF_PTHREAD_RWLOCK_T 		48
+# define __SIZEOF_PTHREAD_RWLOCKATTR_T 		 8
+# define __SIZEOF_PTHREAD_BARRIER_T 		20
+# define __SIZEOF_PTHREAD_BARRIERATTR_T 	 4
 #endif
 
 #define __PTHREAD_COMPAT_PADDING_MID
@@ -56,11 +64,26 @@ struct __pthread_rwlock_arch_t
   unsigned int __writers_futex;
   unsigned int __pad3;
   unsigned int __pad4;
+#if __riscv_xlen == 64
   int __cur_writer;
   int __shared;
   unsigned long int __pad1;
   unsigned long int __pad2;
   unsigned int __flags;
+#else
+# if __BYTE_ORDER == __BIG_ENDIAN
+  unsigned char __pad1;
+  unsigned char __pad2;
+  unsigned char __shared;
+  unsigned char __flags;
+# else
+  unsigned char __flags;
+  unsigned char __shared;
+  unsigned char __pad1;
+  unsigned char __pad2;
+# endif
+  int __cur_writer;
+#endif
 };
 
 #define __PTHREAD_RWLOCK_ELISION_EXTRA 		0
diff --git a/sysdeps/riscv/sfp-machine.h b/sysdeps/riscv/sfp-machine.h
index fa0b8fa41a..98e1f84370 100644
--- a/sysdeps/riscv/sfp-machine.h
+++ b/sysdeps/riscv/sfp-machine.h
@@ -22,7 +22,32 @@
 
 #if __riscv_xlen == 32
 
-# error "rv32i-based targets are not supported"
+# define _FP_W_TYPE_SIZE		32
+# define _FP_W_TYPE		unsigned long
+# define _FP_WS_TYPE		signed long
+# define _FP_I_TYPE		long
+
+# define _FP_MUL_MEAT_S(R, X, Y)				\
+  _FP_MUL_MEAT_1_wide (_FP_WFRACBITS_S, R, X, Y, umul_ppmm)
+# define _FP_MUL_MEAT_D(R, X, Y)				\
+  _FP_MUL_MEAT_2_wide (_FP_WFRACBITS_D, R, X, Y, umul_ppmm)
+# define _FP_MUL_MEAT_Q(R, X, Y)				\
+  _FP_MUL_MEAT_4_wide (_FP_WFRACBITS_Q, R, X, Y, umul_ppmm)
+
+# define _FP_MUL_MEAT_DW_S(R, X, Y)					\
+  _FP_MUL_MEAT_DW_1_wide (_FP_WFRACBITS_S, R, X, Y, umul_ppmm)
+# define _FP_MUL_MEAT_DW_D(R, X, Y)					\
+  _FP_MUL_MEAT_DW_2_wide (_FP_WFRACBITS_D, R, X, Y, umul_ppmm)
+# define _FP_MUL_MEAT_DW_Q(R, X, Y)					\
+  _FP_MUL_MEAT_DW_4_wide (_FP_WFRACBITS_Q, R, X, Y, umul_ppmm)
+
+# define _FP_DIV_MEAT_S(R, X, Y)	_FP_DIV_MEAT_1_udiv_norm (S, R, X, Y)
+# define _FP_DIV_MEAT_D(R, X, Y)	_FP_DIV_MEAT_2_udiv (D, R, X, Y)
+# define _FP_DIV_MEAT_Q(R, X, Y)	_FP_DIV_MEAT_4_udiv (Q, R, X, Y)
+
+# define _FP_NANFRAC_S		_FP_QNANBIT_S
+# define _FP_NANFRAC_D		_FP_QNANBIT_D, 0
+# define _FP_NANFRAC_Q		_FP_QNANBIT_Q, 0, 0, 0
 
 #else
 
diff --git a/sysdeps/riscv/sys/asm.h b/sysdeps/riscv/sys/asm.h
index b8f90a44ce..7ea34afd6d 100644
--- a/sysdeps/riscv/sys/asm.h
+++ b/sysdeps/riscv/sys/asm.h
@@ -26,7 +26,10 @@
 # define REG_S sd
 # define REG_L ld
 #elif __riscv_xlen == 32
-# error "rv32i-based targets are not supported"
+# define PTRLOG 2
+# define SZREG	4
+# define REG_S sw
+# define REG_L lw
 #else
 # error __riscv_xlen must equal 32 or 64
 #endif
diff --git a/sysdeps/unix/sysv/linux/riscv/rv32/jmp_buf-macros.h b/sysdeps/unix/sysv/linux/riscv/rv32/jmp_buf-macros.h
new file mode 100644
index 0000000000..e0042b9f01
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/riscv/rv32/jmp_buf-macros.h
@@ -0,0 +1,53 @@
+/* jump buffer constants for RISC-V
+   Copyright (C) 2017-2018 Free Software Foundation, Inc.
+
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library.  If not, see
+   <http://www.gnu.org/licenses/>.  */
+
+/* Produced by this program:
+
+   #include <stdio.h>
+   #include <unistd.h>
+   #include <setjmp.h>
+   #include <stddef.h>
+
+   int main (int argc, char **argv)
+   {
+       printf ("#define JMP_BUF_SIZE %d\n", sizeof (jmp_buf));
+       printf ("#define JMP_BUF_ALIGN %d\n", __alignof__ (jmp_buf));
+       printf ("#define SIGJMP_BUF_SIZE %d\n", sizeof (sigjmp_buf));
+       printf ("#define SIGJMP_BUF_ALIGN %d\n", __alignof__ (sigjmp_buf));
+       printf ("#define MASK_WAS_SAVED_OFFSET %d\n", offsetof (struct __jmp_buf_tag, __mask_was_saved));
+       printf ("#define SAVED_MASK_OFFSET %d\n", offsetof (struct __jmp_buf_tag, __saved_mask));
+   } */
+
+#if defined __riscv_float_abi_soft
+# define JMP_BUF_SIZE 188
+# define JMP_BUF_ALIGN 4
+# define SIGJMP_BUF_SIZE 188
+# define SIGJMP_BUF_ALIGN 4
+# define MASK_WAS_SAVED_OFFSET 56
+# define SAVED_MASK_OFFSET 60
+#elif defined __riscv_float_abi_double
+# define JMP_BUF_SIZE 288
+# define JMP_BUF_ALIGN 8
+# define SIGJMP_BUF_SIZE 288
+# define SIGJMP_BUF_ALIGN 8
+# define MASK_WAS_SAVED_OFFSET 152
+# define SAVED_MASK_OFFSET 156
+#else
+# error "Unknown RISC-V floating-point ABI"
+#endif
-- 
2.22.0

[RFC v3 06/23] Documentation for the RISC-V 32-bit port

[Alistair Francis]

From: Zong Li <zongbox@gmail.com>

There are RISC-V 64-bit port information in these documentations. I add
the small documentation entries about the RISC-V 32-bit port together.
---
 NEWS   | 6 ++++++
 README | 1 +
 2 files changed, 7 insertions(+)

diff --git a/NEWS b/NEWS
index 6d731b156d..7a897e2110 100644
--- a/NEWS
+++ b/NEWS
@@ -177,6 +177,12 @@ Major new features:
   "%EY" to control how the year number is formatted; they have the
   same effect that they would on "%Ey".
 
+* Support RISC-V port for 32-bit. The ISA and ABI pairs supported as follows:
+
+    - rv32imac ilp32
+    - rv32imafdc ilp32
+    - rv32imafdc ilp32d
+
 Deprecated and removed features, and other changes affecting compatibility:
 
 * The glibc.tune tunable namespace has been renamed to glibc.cpu and the
diff --git a/README b/README
index eb9b0b34a3..2dc80d0062 100644
--- a/README
+++ b/README
@@ -38,6 +38,7 @@ The GNU C Library supports these configurations for using Linux kernels:
 	powerpc64*-*-linux-gnu	Big-endian and little-endian.
 	s390-*-linux-gnu
 	s390x-*-linux-gnu
+	riscv32-*-linux-gnu
 	riscv64-*-linux-gnu
 	sh[34]-*-linux-gnu
 	sparc*-*-linux-gnu
-- 
2.22.0

[RFC v3 18/23] RISC-V: Regenerate ULPs of RISC-V

[Alistair Francis]

From: Zong Li <zongbox@gmail.com>

Use the regen-ulp to generate the ulp files of rvd and nofpu. The RV32
and RV64 use the same ulps.

2018-11-29  Zong Li  <zong@andestech.com>

	* sysdeps/riscv/nofpu/libm-test-ulps: Regenerate.
	* sysdeps/riscv/rvd/libm-test-ulps: New file.
	* sysdeps/riscv/rvd/libm-test-ulps-name: New file.
	* sysdeps/riscv/rv64/rvd/libm-test-ulps: Remove file.
	* sysdeps/riscv/rv64/rvd/libm-test-ulps-name: Remove file.
---
 ChangeLog                                     | 18 ++++++
 sysdeps/riscv/nofpu/libm-test-ulps            | 16 +++---
 sysdeps/riscv/{rv64 => }/rvd/libm-test-ulps   | 56 +++++++++----------
 .../riscv/{rv64 => }/rvd/libm-test-ulps-name  |  0
 4 files changed, 54 insertions(+), 36 deletions(-)
 rename sysdeps/riscv/{rv64 => }/rvd/libm-test-ulps (98%)
 rename sysdeps/riscv/{rv64 => }/rvd/libm-test-ulps-name (100%)

diff --git a/ChangeLog b/ChangeLog
index 9c61de6470..8f2224ee93 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1055,6 +1055,24 @@
 	* sysdeps/riscv/rv32/rvd/s_lround.c: Likewise.
 	* sysdeps/riscv/rv32/rvf/s_lrintf.c: Likewise.
 	* sysdeps/riscv/rv32/rvf/s_lroundf.c: Likewise.
+	* sysdeps/unix/sysv/linux/riscv/ldconfig.h (LD_SO_ABI): Support rv32.
+	* sysdeps/unix/sysv/linux/riscv/dl-cache.h (add_system_dir): Add
+	libraries path for rv32.
+	* sysdeps/riscv/bits/wordsize.h: Supprt rv32.
+	* sysdeps/riscv/nptl/bits/pthreadtypes-arch.h: Likewise.
+	* sysdeps/riscv/sfp-machine.h: Likewise.
+	* sysdeps/riscv/sys/asm.h: Likewise.
+	* sysdeps/unix/sysv/linux/riscv/rv32/jmp_buf-macros.h: New file.
+	* sysdeps/unix/sysv/linux/riscv/rv32/lockf64.c: Likewise.
+	* sysdeps/riscv/rv32/rvd/s_lrint.c: New file.
+	* sysdeps/riscv/rv32/rvd/s_lround.c: Likewise.
+	* sysdeps/riscv/rv32/rvf/s_lrintf.c: Likewise.
+	* sysdeps/riscv/rv32/rvf/s_lroundf.c: Likewise.
+	* sysdeps/riscv/nofpu/libm-test-ulps: Regenerate.
+	* sysdeps/riscv/rvd/libm-test-ulps: New file.
+	* sysdeps/riscv/rvd/libm-test-ulps-name: New file.
+	* sysdeps/riscv/rv64/rvd/libm-test-ulps: Remove file.
+	* sysdeps/riscv/rv64/rvd/libm-test-ulps-name: Remove file.
 
 2019-06-20  Dmitry V. Levin  <ldv@altlinux.org>
 	    Florian Weimer  <fweimer@redhat.com>
diff --git a/sysdeps/riscv/nofpu/libm-test-ulps b/sysdeps/riscv/nofpu/libm-test-ulps
index 700772a5bf..102144a428 100644
--- a/sysdeps/riscv/nofpu/libm-test-ulps
+++ b/sysdeps/riscv/nofpu/libm-test-ulps
@@ -532,16 +532,16 @@ double: 1
 float: 1
 idouble: 1
 ifloat: 1
-ildouble: 1
-ldouble: 1
+ildouble: 8
+ldouble: 8
 
 Function: Imaginary part of "catan":
 double: 1
 float: 1
 idouble: 1
 ifloat: 1
-ildouble: 1
-ldouble: 1
+ildouble: 4
+ldouble: 4
 
 Function: Real part of "catan_downward":
 double: 1
@@ -596,16 +596,16 @@ double: 1
 float: 1
 idouble: 1
 ifloat: 1
-ildouble: 1
-ldouble: 1
+ildouble: 4
+ldouble: 4
 
 Function: Imaginary part of "catanh":
 double: 1
 float: 1
 idouble: 1
 ifloat: 1
-ildouble: 1
-ldouble: 1
+ildouble: 8
+ldouble: 8
 
 Function: Real part of "catanh_downward":
 double: 2
diff --git a/sysdeps/riscv/rv64/rvd/libm-test-ulps b/sysdeps/riscv/rvd/libm-test-ulps
similarity index 98%
rename from sysdeps/riscv/rv64/rvd/libm-test-ulps
rename to sysdeps/riscv/rvd/libm-test-ulps
index 971373d87d..aff168f7bd 100644
--- a/sysdeps/riscv/rv64/rvd/libm-test-ulps
+++ b/sysdeps/riscv/rvd/libm-test-ulps
@@ -532,56 +532,56 @@ double: 1
 float: 1
 idouble: 1
 ifloat: 1
-ildouble: 1
-ldouble: 1
+ildouble: 8
+ldouble: 8
 
 Function: Imaginary part of "catan":
 double: 1
 float: 1
 idouble: 1
 ifloat: 1
-ildouble: 1
-ldouble: 1
+ildouble: 4
+ldouble: 4
 
 Function: Real part of "catan_downward":
 double: 1
 float: 2
 idouble: 1
 ifloat: 2
-ildouble: 2
-ldouble: 2
+ildouble: 9
+ldouble: 9
 
 Function: Imaginary part of "catan_downward":
 double: 2
 float: 2
 idouble: 2
 ifloat: 2
-ildouble: 2
-ldouble: 2
+ildouble: 3
+ldouble: 3
 
 Function: Real part of "catan_towardzero":
 double: 1
 float: 2
 idouble: 1
 ifloat: 2
-ildouble: 2
-ldouble: 2
+ildouble: 9
+ldouble: 9
 
 Function: Imaginary part of "catan_towardzero":
 double: 2
 float: 2
 idouble: 2
 ifloat: 2
-ildouble: 2
-ldouble: 2
+ildouble: 3
+ldouble: 3
 
 Function: Real part of "catan_upward":
 double: 1
 float: 1
 idouble: 1
 ifloat: 1
-ildouble: 2
-ldouble: 2
+ildouble: 8
+ldouble: 8
 
 Function: Imaginary part of "catan_upward":
 double: 2
@@ -596,48 +596,48 @@ double: 1
 float: 1
 idouble: 1
 ifloat: 1
-ildouble: 1
-ldouble: 1
+ildouble: 4
+ldouble: 4
 
 Function: Imaginary part of "catanh":
 double: 1
 float: 1
 idouble: 1
 ifloat: 1
-ildouble: 1
-ldouble: 1
+ildouble: 8
+ldouble: 8
 
 Function: Real part of "catanh_downward":
 double: 2
 float: 2
 idouble: 2
 ifloat: 2
-ildouble: 2
-ldouble: 2
+ildouble: 3
+ldouble: 3
 
 Function: Imaginary part of "catanh_downward":
 double: 1
 float: 2
 idouble: 1
 ifloat: 2
-ildouble: 2
-ldouble: 2
+ildouble: 9
+ldouble: 9
 
 Function: Real part of "catanh_towardzero":
 double: 2
 float: 2
 idouble: 2
 ifloat: 2
-ildouble: 2
-ldouble: 2
+ildouble: 3
+ldouble: 3
 
 Function: Imaginary part of "catanh_towardzero":
 double: 1
 float: 2
 idouble: 1
 ifloat: 2
-ildouble: 2
-ldouble: 2
+ildouble: 9
+ldouble: 9
 
 Function: Real part of "catanh_upward":
 double: 4
@@ -652,8 +652,8 @@ double: 1
 float: 1
 idouble: 1
 ifloat: 1
-ildouble: 2
-ldouble: 2
+ildouble: 8
+ldouble: 8
 
 Function: "cbrt":
 double: 3
diff --git a/sysdeps/riscv/rv64/rvd/libm-test-ulps-name b/sysdeps/riscv/rvd/libm-test-ulps-name
similarity index 100%
rename from sysdeps/riscv/rv64/rvd/libm-test-ulps-name
rename to sysdeps/riscv/rvd/libm-test-ulps-name
-- 
2.22.0

[RFC v3 13/23] RISC-V: Use 64-bit timespec in clock_gettime vdso calls

[Alistair Francis]

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 sysdeps/unix/sysv/linux/riscv/init-first.c | 2 +-
 sysdeps/unix/sysv/linux/riscv/libc-vdso.h  | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/sysdeps/unix/sysv/linux/riscv/init-first.c b/sysdeps/unix/sysv/linux/riscv/init-first.c
index 8134c79695..155a4a2c0c 100644
--- a/sysdeps/unix/sysv/linux/riscv/init-first.c
+++ b/sysdeps/unix/sysv/linux/riscv/init-first.c
@@ -25,7 +25,7 @@ long int (*VDSO_SYMBOL (getcpu)) (unsigned int *, unsigned int *, void *)
     attribute_hidden;
 long int (*VDSO_SYMBOL (gettimeofday)) (struct timeval *, void *)
     attribute_hidden;
-long int (*VDSO_SYMBOL (clock_gettime)) (clockid_t, struct timespec *)
+long int (*VDSO_SYMBOL (clock_gettime)) (clockid_t, struct __timespec64 *)
     attribute_hidden;
 long int (*VDSO_SYMBOL (clock_getres)) (clockid_t, struct timespec *)
     attribute_hidden;
diff --git a/sysdeps/unix/sysv/linux/riscv/libc-vdso.h b/sysdeps/unix/sysv/linux/riscv/libc-vdso.h
index b0c46f3cb7..406801746f 100644
--- a/sysdeps/unix/sysv/linux/riscv/libc-vdso.h
+++ b/sysdeps/unix/sysv/linux/riscv/libc-vdso.h
@@ -28,7 +28,7 @@ extern long int (*VDSO_SYMBOL (getcpu)) (unsigned int *, unsigned int *, void *)
     attribute_hidden;
 extern long int (*VDSO_SYMBOL (gettimeofday)) (struct timeval *, void *)
     attribute_hidden;
-extern long int (*VDSO_SYMBOL (clock_gettime)) (clockid_t, struct timespec *)
+extern long int (*VDSO_SYMBOL (clock_gettime)) (clockid_t, struct __timespec64 *)
     attribute_hidden;
 extern long int (*VDSO_SYMBOL (clock_getres)) (clockid_t, struct timespec *)
     attribute_hidden;
-- 
2.22.0

Re: [RFC v3 13/23] RISC-V: Use 64-bit timespec in clock_gettime vdso calls

[Arnd Bergmann]

On Wed, Jul 17, 2019 at 2:12 AM Alistair Francis
<alistair.francis@wdc.com> wrote:
>
> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
> ---
>  sysdeps/unix/sysv/linux/riscv/init-first.c | 2 +-
>  sysdeps/unix/sysv/linux/riscv/libc-vdso.h  | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/sysdeps/unix/sysv/linux/riscv/init-first.c b/sysdeps/unix/sysv/linux/riscv/init-first.c
> index 8134c79695..155a4a2c0c 100644
> --- a/sysdeps/unix/sysv/linux/riscv/init-first.c
> +++ b/sysdeps/unix/sysv/linux/riscv/init-first.c
> @@ -25,7 +25,7 @@ long int (*VDSO_SYMBOL (getcpu)) (unsigned int *, unsigned int *, void *)
>      attribute_hidden;
>  long int (*VDSO_SYMBOL (gettimeofday)) (struct timeval *, void *)
>      attribute_hidden;
> -long int (*VDSO_SYMBOL (clock_gettime)) (clockid_t, struct timespec *)
> +long int (*VDSO_SYMBOL (clock_gettime)) (clockid_t, struct __timespec64 *)

The vdso symbol in the kernel should be __clock_gettime64().

The gettimeofday() and clock_getres() functions should not have a
vdso symbol on 32-bit but instead get implemented by claling __clock_gettime64()
or the clock_getres() syscall.

If someone has a good reason for introducing a clock_getres_time64() vdso
symbol, we can do that as well, but so far I am not aware of any performance
critical caller of clock_getres().

      Arnd

[RFC v3 11/23] RISC-V: define __vdso_clock_getres as __vdso_clock_getres_time64 for 32-bit

[Alistair Francis]

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 sysdeps/unix/sysv/linux/riscv/sysdep.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/sysdeps/unix/sysv/linux/riscv/sysdep.h b/sysdeps/unix/sysv/linux/riscv/sysdep.h
index 9e07edfed3..e66e9f032a 100644
--- a/sysdeps/unix/sysv/linux/riscv/sysdep.h
+++ b/sysdeps/unix/sysv/linux/riscv/sysdep.h
@@ -167,6 +167,10 @@
 # ifndef __NR_clock_getres
 #  define __NR_clock_getres __NR_clock_getres_time64
 # endif
+
+# ifndef __vdso_clock_getres
+#  define __vdso_clock_getres __vdso_clock_getres_time64
+# endif
 #endif /* __riscv_xlen == 32 */
 
 #undef SYS_ify
-- 
2.22.0

[RFC v3 15/23] RISC-V: Add path of library directories for the 32-bit

[Alistair Francis]

From: Zong Li <zongbox@gmail.com>

For the recommand of 64 bit version, we add the libraries path of 32 bit
in this patch. This includes a fix to avoid an out of bound array check
when building with GCC 8.2.

2018-11-29  Zong Li  <zong@andestech.com>

	* sysdeps/unix/sysv/linux/riscv/dl-cache.h (add_system_dir): Add
	libraries path for rv32.

[ Changes by AF:
 - Fix bound array check when building with GCC 8.2
]
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 ChangeLog                                |  3 +++
 sysdeps/unix/sysv/linux/riscv/dl-cache.h | 17 +++++++++++++++--
 2 files changed, 18 insertions(+), 2 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index 940db8b92e..4155ead682 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1040,6 +1040,9 @@
 2018-06-20  Zong Li  <zong@andestech.com>
 
 	* sysdeps/unix/sysv/linux/riscv/ldconfig.h (LD_SO_ABI): Support rv32.
+	* sysdeps/unix/sysv/linux/riscv/ldconfig.h (LD_SO_ABI): Support rv32.
+	* sysdeps/unix/sysv/linux/riscv/dl-cache.h (add_system_dir): Add
+	libraries path for rv32.
 
 2019-06-20  Dmitry V. Levin  <ldv@altlinux.org>
 	    Florian Weimer  <fweimer@redhat.com>
diff --git a/sysdeps/unix/sysv/linux/riscv/dl-cache.h b/sysdeps/unix/sysv/linux/riscv/dl-cache.h
index 2a15e28e3b..5943060536 100644
--- a/sysdeps/unix/sysv/linux/riscv/dl-cache.h
+++ b/sysdeps/unix/sysv/linux/riscv/dl-cache.h
@@ -34,6 +34,8 @@
    RISC-V, libraries can be found in paths ending in:
      - /lib64/lp64d
      - /lib64/lp64
+     - /lib32/ilp32d
+     - /lib32/ilp32
      - /lib (only ld.so)
    so this will add all of those paths.
 
@@ -49,9 +51,16 @@
   do							    		\
     {									\
       size_t len = strlen (dir);					\
-      char path[len + 9];						\
+      char path[len + 10];						\
       memcpy (path, dir, len + 1);					\
-      if (len >= 12 && ! memcmp(path + len - 12, "/lib64/lp64d", 12))	\
+      if (len >= 13 && ! memcmp(path + len - 13, "/lib32/ilp32d", 13))	\
+        {								\
+          len -= 9;							\
+	  path[len] = '\0';						\
+        }								\
+      if (len >= 12							\
+          && (! memcmp(path + len - 12, "/lib32/ilp32", 12)		\
+              || ! memcmp(path + len - 12, "/lib64/lp64d", 12)))	\
 	{								\
 	  len -= 8;							\
 	  path[len] = '\0';						\
@@ -64,6 +73,10 @@
       add_dir (path);							\
       if (len >= 4 && ! memcmp(path + len - 4, "/lib", 4))		\
 	{								\
+	  memcpy (path + len, "32/ilp32d", 10);				\
+	  add_dir (path);						\
+	  memcpy (path + len, "32/ilp32", 9);				\
+	  add_dir (path);						\
 	  memcpy (path + len, "64/lp64d", 9);				\
 	  add_dir (path);						\
 	  memcpy (path + len, "64/lp64", 8);				\
-- 
2.22.0

Re: [RFC v3 15/23] RISC-V: Add path of library directories for the 32-bit

[Florian Weimer]

* Alistair Francis:

> diff --git a/ChangeLog b/ChangeLog
> index 940db8b92e..4155ead682 100644
> --- a/ChangeLog
> +++ b/ChangeLog
> @@ -1040,6 +1040,9 @@
>  2018-06-20  Zong Li  <zong@andestech.com>
>  
>  	* sysdeps/unix/sysv/linux/riscv/ldconfig.h (LD_SO_ABI): Support rv32.
> +	* sysdeps/unix/sysv/linux/riscv/ldconfig.h (LD_SO_ABI): Support rv32.
> +	* sysdeps/unix/sysv/linux/riscv/dl-cache.h (add_system_dir): Add
> +	libraries path for rv32.

Duplicate line, and the commit subject seems to be truncated.

Thanks,
Florian

[RFC v3 14/23] RISC-V: Support dynamic loader for the 32-bit

[Alistair Francis]

From: Zong Li <zongbox@gmail.com>

Add the LD_SO_ABI definition for RISC-V 32-bit.

2018-11-29  Zong Li  <zong@andestech.com>

	* sysdeps/unix/sysv/linux/riscv/ldconfig.h (LD_SO_ABI): Support
	rv32.
---
 ChangeLog                                | 4 ++++
 sysdeps/unix/sysv/linux/riscv/ldconfig.h | 2 +-
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/ChangeLog b/ChangeLog
index ec07b3ff27..940db8b92e 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1037,6 +1037,10 @@
 	* sysdeps/unix/sysv/linux/riscv/bits/timesize.h: Likewise.
 	* sysdeps/unix/sysv/linux/riscv/bits/typesizes.h: Likewise.
 
+2018-06-20  Zong Li  <zong@andestech.com>
+
+	* sysdeps/unix/sysv/linux/riscv/ldconfig.h (LD_SO_ABI): Support rv32.
+
 2019-06-20  Dmitry V. Levin  <ldv@altlinux.org>
 	    Florian Weimer  <fweimer@redhat.com>
 
diff --git a/sysdeps/unix/sysv/linux/riscv/ldconfig.h b/sysdeps/unix/sysv/linux/riscv/ldconfig.h
index 20fbd16c59..360f35fb28 100644
--- a/sysdeps/unix/sysv/linux/riscv/ldconfig.h
+++ b/sysdeps/unix/sysv/linux/riscv/ldconfig.h
@@ -24,7 +24,7 @@
 #if __riscv_xlen == 64
 # define LD_SO_ABI "riscv64-lp64"
 #else
-# error "rv32i-based targets are not supported"
+# define LD_SO_ABI "riscv32-ilp32"
 #endif
 
 #define SYSDEP_KNOWN_INTERPRETER_NAMES				\
-- 
2.22.0

[RFC v3 10/23] RISC-V: define __NR_clock_getres as __NR_*_time64 for 32-bit

[Alistair Francis]

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 sysdeps/unix/sysv/linux/riscv/sysdep.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/sysdeps/unix/sysv/linux/riscv/sysdep.h b/sysdeps/unix/sysv/linux/riscv/sysdep.h
index 2c06706701..9e07edfed3 100644
--- a/sysdeps/unix/sysv/linux/riscv/sysdep.h
+++ b/sysdeps/unix/sysv/linux/riscv/sysdep.h
@@ -163,6 +163,10 @@
 # ifndef __NR_timer_settime
 #  define __NR_timer_settime __NR_timer_settime64
 # endif
+
+# ifndef __NR_clock_getres
+#  define __NR_clock_getres __NR_clock_getres_time64
+# endif
 #endif /* __riscv_xlen == 32 */
 
 #undef SYS_ify
-- 
2.22.0

[RFC v3 12/23] RISC-V: define __vdso_clock_gettime as __vdso_clock_gettime64 for 32-bit

[Alistair Francis]

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 sysdeps/unix/sysv/linux/riscv/sysdep.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/sysdeps/unix/sysv/linux/riscv/sysdep.h b/sysdeps/unix/sysv/linux/riscv/sysdep.h
index e66e9f032a..ea47b9b82c 100644
--- a/sysdeps/unix/sysv/linux/riscv/sysdep.h
+++ b/sysdeps/unix/sysv/linux/riscv/sysdep.h
@@ -171,6 +171,10 @@
 # ifndef __vdso_clock_getres
 #  define __vdso_clock_getres __vdso_clock_getres_time64
 # endif
+
+# ifndef __vdso_clock_gettime
+#  define __vdso_clock_gettime __vdso_clock_gettime64
+# endif
 #endif /* __riscv_xlen == 32 */
 
 #undef SYS_ify
-- 
2.22.0

Re: [RFC v3 12/23] RISC-V: define __vdso_clock_gettime as __vdso_clock_gettime64 for 32-bit

[Arnd Bergmann]

On Wed, Jul 17, 2019 at 2:12 AM Alistair Francis
<alistair.francis@wdc.com> wrote:
>
> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
> ---
>  sysdeps/unix/sysv/linux/riscv/sysdep.h | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/sysdeps/unix/sysv/linux/riscv/sysdep.h b/sysdeps/unix/sysv/linux/riscv/sysdep.h
> index e66e9f032a..ea47b9b82c 100644
> --- a/sysdeps/unix/sysv/linux/riscv/sysdep.h
> +++ b/sysdeps/unix/sysv/linux/riscv/sysdep.h
> @@ -171,6 +171,10 @@
>  # ifndef __vdso_clock_getres
>  #  define __vdso_clock_getres __vdso_clock_getres_time64
>  # endif
> +
> +# ifndef __vdso_clock_gettime
> +#  define __vdso_clock_gettime __vdso_clock_gettime64
> +# endif
>  #endif /* __riscv_xlen == 32 */


I had not noticed this when commenting on the other patches. What is
the purpose of doing this?

As mentioned, I don't think we should have __vdso_clock_getres_time64()
at all (no caller cares about this being fast).
The "#define__vdso_clock_gettime __vdso_clock_gettime64" should
be harmless but leads to confusion because other architectures that
have both cannot do this.

      Arnd

Re: [RFC v3 12/23] RISC-V: define __vdso_clock_gettime as __vdso_clock_gettime64 for 32-bit

[Alistair Francis]

On Wed, Jul 17, 2019 at 1:16 AM Arnd Bergmann <arnd@arndb.de> wrote:
>
> On Wed, Jul 17, 2019 at 2:12 AM Alistair Francis
> <alistair.francis@wdc.com> wrote:
> >
> > Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
> > ---
> >  sysdeps/unix/sysv/linux/riscv/sysdep.h | 4 ++++
> >  1 file changed, 4 insertions(+)
> >
> > diff --git a/sysdeps/unix/sysv/linux/riscv/sysdep.h b/sysdeps/unix/sysv/linux/riscv/sysdep.h
> > index e66e9f032a..ea47b9b82c 100644
> > --- a/sysdeps/unix/sysv/linux/riscv/sysdep.h
> > +++ b/sysdeps/unix/sysv/linux/riscv/sysdep.h
> > @@ -171,6 +171,10 @@
> >  # ifndef __vdso_clock_getres
> >  #  define __vdso_clock_getres __vdso_clock_getres_time64
> >  # endif
> > +
> > +# ifndef __vdso_clock_gettime
> > +#  define __vdso_clock_gettime __vdso_clock_gettime64
> > +# endif
> >  #endif /* __riscv_xlen == 32 */
>
>
> I had not noticed this when commenting on the other patches. What is
> the purpose of doing this?
>
> As mentioned, I don't think we should have __vdso_clock_getres_time64()
> at all (no caller cares about this being fast).
> The "#define__vdso_clock_gettime __vdso_clock_gettime64" should
> be harmless but leads to confusion because other architectures that
> have both cannot do this.

These were required to avoid build failures, but I think I have a
better solution and have removed these patches.

Alistair

>
>       Arnd

[RFC v3 21/23] RISC-V: Fix llrint and llround missing exceptions on RV32

[Alistair Francis]

From: Zong Li <zongbox@gmail.com>

Similar to the recent fix for MIPS, ARM and S/390, RV32 is missing
correct exception on overflow from llrint and llround functions because
cast from floating-point types to long long do not result in correct
exceptions on overflow.

2018-11-29  Zong Li  <zong@andestech.com>

	* sysdeps/riscv/rv32/fix-fp-int-convert-overflow.h: New file.
---
 ChangeLog                                     | 10 +++++
 .../riscv/rv32/fix-fp-int-convert-overflow.h  | 38 +++++++++++++++++++
 2 files changed, 48 insertions(+)
 create mode 100644 sysdeps/riscv/rv32/fix-fp-int-convert-overflow.h

diff --git a/ChangeLog b/ChangeLog
index 131c1063fc..8b8b038b23 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1114,6 +1114,16 @@
 	* sysdeps/unix/sysv/linux/riscv/configure.ac: Likewise.
 	* sysdeps/unix/sysv/linux/riscv/shlib-versions: Likewise.
 	* sysdeps/riscv/preconfigure: Likewise.
+	* sysdeps/riscv/rv32/Implies-after: New file.
+	* sysdeps/riscv/rv32/rvd/Implies: Likewise.
+	* sysdeps/riscv/rv32/rvf/Implies: Likewise.
+	* sysdeps/unix/sysv/linux/riscv/rv32/Implies: Likewise.
+	* sysdeps/unix/sysv/linux/riscv/Makefile: Support rv32.
+	* sysdeps/unix/sysv/linux/riscv/configure: Likewise.
+	* sysdeps/unix/sysv/linux/riscv/configure.ac: Likewise.
+	* sysdeps/unix/sysv/linux/riscv/shlib-versions: Likewise.
+	* sysdeps/riscv/preconfigure: Likewise.
+	* sysdeps/riscv/rv32/fix-fp-int-convert-overflow.h: New file.
 
 2019-06-20  Dmitry V. Levin  <ldv@altlinux.org>
 	    Florian Weimer  <fweimer@redhat.com>
diff --git a/sysdeps/riscv/rv32/fix-fp-int-convert-overflow.h b/sysdeps/riscv/rv32/fix-fp-int-convert-overflow.h
new file mode 100644
index 0000000000..6b34a8415e
--- /dev/null
+++ b/sysdeps/riscv/rv32/fix-fp-int-convert-overflow.h
@@ -0,0 +1,38 @@
+/* Fix for conversion of floating point to integer overflow.  ARM version.
+   Copyright (C) 2015-2018 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#ifndef FIX_FP_INT_CONVERT_OVERFLOW_H
+#define FIX_FP_INT_CONVERT_OVERFLOW_H	1
+
+/* As of GCC 5, the generic libgcc2.c conversions from floating point
+   to long long may not raise the correct exceptions on overflow (and
+   may raise spurious "inexact" exceptions even in non-overflow cases,
+   see <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=59412>).  */
+#define FIX_FLT_LONG_CONVERT_OVERFLOW 0
+#define FIX_FLT_LLONG_CONVERT_OVERFLOW 1
+
+#define FIX_DBL_LONG_CONVERT_OVERFLOW 0
+#define FIX_DBL_LLONG_CONVERT_OVERFLOW 1
+
+#define FIX_LDBL_LONG_CONVERT_OVERFLOW 0
+#define FIX_LDBL_LLONG_CONVERT_OVERFLOW 0
+
+#define FIX_FLT128_LONG_CONVERT_OVERFLOW 0
+#define FIX_FLT128_LLONG_CONVERT_OVERFLOW 0
+
+#endif /* fix-fp-int-convert-overflow.h */
-- 
2.22.0

Re: [RFC v3 21/23] RISC-V: Fix llrint and llround missing exceptions on RV32

[Florian Weimer]

* Alistair Francis:

> +/* As of GCC 5, the generic libgcc2.c conversions from floating point
> +   to long long may not raise the correct exceptions on overflow (and
> +   may raise spurious "inexact" exceptions even in non-overflow cases,
> +   see <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=59412>).  */

Is there really a GCC 5 port for RV32?  If not, the comment is wrong.

Thanks,
Florian

Re: [RFC v3 21/23] RISC-V: Fix llrint and llround missing exceptions on RV32

[Alistair Francis]

On Wed, Jul 17, 2019 at 5:22 AM Florian Weimer <fweimer@redhat.com> wrote:
>
> * Alistair Francis:
>
> > +/* As of GCC 5, the generic libgcc2.c conversions from floating point
> > +   to long long may not raise the correct exceptions on overflow (and
> > +   may raise spurious "inexact" exceptions even in non-overflow cases,
> > +   see <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=59412>).  */
>
> Is there really a GCC 5 port for RV32?  If not, the comment is wrong.

There is not a GCC 5 port but I think the comment is referring to the
general issue which has existed since GCC 5. I have fixed this up
though.

Alistair

>
> Thanks,
> Florian

[RFC v3 05/23] sysdeps/timespec_get: Use clock_gettime64 if avaliable

[Alistair Francis]

This will break other 32-bit targets

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 sysdeps/unix/sysv/linux/timespec_get.c | 37 +++++++++++++++++++++++++-
 1 file changed, 36 insertions(+), 1 deletion(-)

diff --git a/sysdeps/unix/sysv/linux/timespec_get.c b/sysdeps/unix/sysv/linux/timespec_get.c
index 52080ddf08..78fa2aba1b 100644
--- a/sysdeps/unix/sysv/linux/timespec_get.c
+++ b/sysdeps/unix/sysv/linux/timespec_get.c
@@ -24,6 +24,36 @@
 #endif
 #include <sysdep-vdso.h>
 
+
+#if __WORDSIZE == 32
+int
+__timespec_get (struct timespec *ts, int base)
+{
+   int ret;
+
+#ifdef __NR_clock_gettime64
+  struct __timespec64 ts64;
+  ret = INTERNAL_VSYSCALL (clock_gettime64, err, 2, CLOCK_REALTIME, &ts64);
+
+  ts->tv_sec = ts64.tv_sec;
+  ts->tv_nsec = ts64.tv_nsec;
+
+  if (! in_time_t_range (ts->tv_sec))
+    {
+      __set_errno (EOVERFLOW);
+      return -1;
+    }
+#endif
+
+#ifndef __ASSUME_TIME64_SYSCALLS
+  ret = INTERNAL_VSYSCALL (clock_gettime, err, 2, CLOCK_REALTIME, ts);
+#endif
+
+  return ret;
+}
+
+#else
+
 /* Set TS to calendar time based in time base BASE.  */
 int
 timespec_get (struct timespec *ts, int base)
@@ -33,9 +63,13 @@ timespec_get (struct timespec *ts, int base)
       int res;
       INTERNAL_SYSCALL_DECL (err);
     case TIME_UTC:
+#if __WORDSIZE == 32
+      res = __timespec_get (*ts, base);
+#else
       res = INTERNAL_VSYSCALL (clock_gettime, err, 2, CLOCK_REALTIME, ts);
+#endif
       if (INTERNAL_SYSCALL_ERROR_P (res, err))
-	return 0;
+        return 0;
       break;
 
     default:
@@ -44,3 +78,4 @@ timespec_get (struct timespec *ts, int base)
 
   return base;
 }
+#endif
-- 
2.22.0

Re: [RFC v3 05/23] sysdeps/timespec_get: Use clock_gettime64 if avaliable

[Florian Weimer]

* Alistair Francis:

> +#if __WORDSIZE == 32
> +int
> +__timespec_get (struct timespec *ts, int base)
> +{
> +   int ret;
> +
> +#ifdef __NR_clock_gettime64
> +  struct __timespec64 ts64;
> +  ret = INTERNAL_VSYSCALL (clock_gettime64, err, 2, CLOCK_REALTIME, &ts64);
> +
> +  ts->tv_sec = ts64.tv_sec;
> +  ts->tv_nsec = ts64.tv_nsec;
> +
> +  if (! in_time_t_range (ts->tv_sec))
> +    {
> +      __set_errno (EOVERFLOW);
> +      return -1;
> +    }
> +#endif
> +
> +#ifndef __ASSUME_TIME64_SYSCALLS
> +  ret = INTERNAL_VSYSCALL (clock_gettime, err, 2, CLOCK_REALTIME, ts);
> +#endif

I don't think this is right.  I think you need to cache clock_gettime64
support somewhere and avoid trying to call the non-existing system call
again and again.  And the second system call will overwrite the results
of the first.

Thanks,
Florian

Re: [RFC v3 05/23] sysdeps/timespec_get: Use clock_gettime64 if avaliable

[Arnd Bergmann]

On Wed, Jul 17, 2019 at 7:08 AM Florian Weimer <fweimer@redhat.com> wrote:
>
> * Alistair Francis:
>
> > +#if __WORDSIZE == 32
> > +int
> > +__timespec_get (struct timespec *ts, int base)
> > +{
> > +   int ret;
> > +
> > +#ifdef __NR_clock_gettime64
> > +  struct __timespec64 ts64;
> > +  ret = INTERNAL_VSYSCALL (clock_gettime64, err, 2, CLOCK_REALTIME, &ts64);
> > +
> > +  ts->tv_sec = ts64.tv_sec;
> > +  ts->tv_nsec = ts64.tv_nsec;
> > +
> > +  if (! in_time_t_range (ts->tv_sec))
> > +    {
> > +      __set_errno (EOVERFLOW);
> > +      return -1;
> > +    }
> > +#endif
> > +
> > +#ifndef __ASSUME_TIME64_SYSCALLS
> > +  ret = INTERNAL_VSYSCALL (clock_gettime, err, 2, CLOCK_REALTIME, ts);
> > +#endif
>
> I don't think this is right.  I think you need to cache clock_gettime64
> support somewhere and avoid trying to call the non-existing system call
> again and again.

How important is this? It sounds like a micro-optimization for a case that
very few people are going to hit, given that running an application with
64-bit time_t on an old kernel will likely hit other bugs that glibc cannot
deal with.

      Arnd

Re: [RFC v3 05/23] sysdeps/timespec_get: Use clock_gettime64 if avaliable

[Florian Weimer]

* Arnd Bergmann:

> On Wed, Jul 17, 2019 at 7:08 AM Florian Weimer <fweimer@redhat.com> wrote:
>>
>> * Alistair Francis:
>>
>> > +#if __WORDSIZE == 32
>> > +int
>> > +__timespec_get (struct timespec *ts, int base)
>> > +{
>> > +   int ret;
>> > +
>> > +#ifdef __NR_clock_gettime64
>> > +  struct __timespec64 ts64;
>> > +  ret = INTERNAL_VSYSCALL (clock_gettime64, err, 2, CLOCK_REALTIME, &ts64);
>> > +
>> > +  ts->tv_sec = ts64.tv_sec;
>> > +  ts->tv_nsec = ts64.tv_nsec;
>> > +
>> > +  if (! in_time_t_range (ts->tv_sec))
>> > +    {
>> > +      __set_errno (EOVERFLOW);
>> > +      return -1;
>> > +    }
>> > +#endif
>> > +
>> > +#ifndef __ASSUME_TIME64_SYSCALLS
>> > +  ret = INTERNAL_VSYSCALL (clock_gettime, err, 2, CLOCK_REALTIME, ts);
>> > +#endif
>>
>> I don't think this is right.  I think you need to cache clock_gettime64
>> support somewhere and avoid trying to call the non-existing system call
>> again and again.
>
> How important is this? It sounds like a micro-optimization for a case that
> very few people are going to hit, given that running an application with
> 64-bit time_t on an old kernel will likely hit other bugs that glibc cannot
> deal with.

I don't think it's a microoptimization.  On old kernels without
clock_gettime64 in the vDSO, 32-bit timespec_get will always make the
system call, which fails.  Only then the 32-bit clock_gettime in the
vDSO is used.  This effectively negates the performance benefit of the
vDSO, I think.

(Not sure if this will even build on non-RV32 as posted, given that we
don't have a vDSO variable in glibc for clock_gettime64, but I assume
that's going to be fixed if necessary.)

Thanks,
Florian

Re: [RFC v3 05/23] sysdeps/timespec_get: Use clock_gettime64 if avaliable

[Arnd Bergmann]

On Wed, Jul 17, 2019 at 10:12 AM Florian Weimer <fweimer@redhat.com> wrote:
> * Arnd Bergmann:
> > On Wed, Jul 17, 2019 at 7:08 AM Florian Weimer <fweimer@redhat.com> wrote:
> >> * Alistair Francis:
> > How important is this? It sounds like a micro-optimization for a case that
> > very few people are going to hit, given that running an application with
> > 64-bit time_t on an old kernel will likely hit other bugs that glibc cannot
> > deal with.
>
> I don't think it's a microoptimization.  On old kernels without
> clock_gettime64 in the vDSO, 32-bit timespec_get will always make the
> system call, which fails.  Only then the 32-bit clock_gettime in the
> vDSO is used.  This effectively negates the performance benefit of the
> vDSO, I think.

I understand that it would be much slower on that particular combination,
I just can't think of anyone who would run into this in practice outside of
validation testing that makes sure glibc does run this way.

If you have any real-world binary built with 64-bit time_t, this will
require all library dependencies other than glibc to be built the same way
and that in turn won't happen unless someone builds a whole distro
for 64-bit time_t, which would be crazy unless they also use a modern
kernel.

I can understand the need to make it work in principle, but does it
have to be efficient?

      Arnd

Re: [RFC v3 05/23] sysdeps/timespec_get: Use clock_gettime64 if avaliable

[Florian Weimer]

* Arnd Bergmann:

> I understand that it would be much slower on that particular combination,
> I just can't think of anyone who would run into this in practice outside of
> validation testing that makes sure glibc does run this way.

I think it will happen if you run a glibc which has been built against
current kernel headers (and with this patch or something like it) on a
host which has an older kernel.  Due to the way the patch has been
written, it will also impact legacy applications which call the 32-bit
functions, not just 32-bit applications which use the 64-bit interfaces.
(I have not checked the impact on 64-bit kernels yet, hopefully they
will never define __NR_clock_gettime64.)

I could be totally wrong about this, or our disagreement could be about
the relevance of this scenario.  Not sure which.

(I think supporting old host kernels is important to our users.)

Thanks,
Florian

Re: [RFC v3 05/23] sysdeps/timespec_get: Use clock_gettime64 if avaliable

[Arnd Bergmann]

On Wed, Jul 17, 2019 at 10:41 AM Florian Weimer <fweimer@redhat.com> wrote:
>
> * Arnd Bergmann:
>
> > I understand that it would be much slower on that particular combination,
> > I just can't think of anyone who would run into this in practice outside of
> > validation testing that makes sure glibc does run this way.
>
> I think it will happen if you run a glibc which has been built against
> current kernel headers (and with this patch or something like it) on a
> host which has an older kernel.  Due to the way the patch has been
> written, it will also impact legacy applications which call the 32-bit
> functions, not just 32-bit applications which use the 64-bit interfaces.

Ah right, makes sense.

> I could be totally wrong about this, or our disagreement could be about
> the relevance of this scenario.  Not sure which.
>
> (I think supporting old host kernels is important to our users.)

No, I was just wrong and had only thought about applications calling
directly into clock_gettime() while using a 64-bit time_t, but not
the case of glibc-internal functions calling __clock_gettime64()
for convenience, which is something I even advocated for in one of
my other comments.

       Arnd

Re: [RFC v3 05/23] sysdeps/timespec_get: Use clock_gettime64 if avaliable

[Joseph Myers]

On Wed, 17 Jul 2019, Arnd Bergmann wrote:

> > I don't think this is right.  I think you need to cache clock_gettime64
> > support somewhere and avoid trying to call the non-existing system call
> > again and again.
> 
> How important is this? It sounds like a micro-optimization for a case that
> very few people are going to hit, given that running an application with
> 64-bit time_t on an old kernel will likely hit other bugs that glibc cannot
> deal with.

It's a generic design question for all the time64 patches that might end 
up using one or another syscall at runtime depending on kernel support - 
we'll need to arrive at a consensus on whether such caching (probably 
shared between all relevant syscalls) is desired.  It's not clear to me 
whether the case of _TIME_BITS=64 on an old kernel will end up as a rare 
case or not.  And as per my previous comments, many existing functions 
using 32-bit times should end up as thin wrappers round functions using 
64-bit times, so potentially making two syscalls every time unless there 
is such caching.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: [RFC v3 05/23] sysdeps/timespec_get: Use clock_gettime64 if avaliable

[Lukasz Majewski]

[-- Attachment #1: Type: text/plain, Size: 2334 bytes --]

Hi Alistair,

> This will break other 32-bit targets
> 
> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
> ---
>  sysdeps/unix/sysv/linux/timespec_get.c | 37
> +++++++++++++++++++++++++- 1 file changed, 36 insertions(+), 1
> deletion(-)
> 
> diff --git a/sysdeps/unix/sysv/linux/timespec_get.c
> b/sysdeps/unix/sysv/linux/timespec_get.c index 52080ddf08..78fa2aba1b
> 100644 --- a/sysdeps/unix/sysv/linux/timespec_get.c
> +++ b/sysdeps/unix/sysv/linux/timespec_get.c
> @@ -24,6 +24,36 @@
>  #endif
>  #include <sysdep-vdso.h>
>  
> +
> +#if __WORDSIZE == 32
> +int
> +__timespec_get (struct timespec *ts, int base)
> +{
> +   int ret;
> +
> +#ifdef __NR_clock_gettime64
> +  struct __timespec64 ts64;
> +  ret = INTERNAL_VSYSCALL (clock_gettime64, err, 2, CLOCK_REALTIME,
> &ts64); +
> +  ts->tv_sec = ts64.tv_sec;
> +  ts->tv_nsec = ts64.tv_nsec;
> +

You may consider using following helper functions (which are the part
of Y2038 support): [1]

> +  if (! in_time_t_range (ts->tv_sec))
> +    {
> +      __set_errno (EOVERFLOW);
> +      return -1;
> +    }
> +#endif
> +
> +#ifndef __ASSUME_TIME64_SYSCALLS
> +  ret = INTERNAL_VSYSCALL (clock_gettime, err, 2, CLOCK_REALTIME,
> ts); +#endif
> +
> +  return ret;
> +}
> +
> +#else
> +
>  /* Set TS to calendar time based in time base BASE.  */
>  int
>  timespec_get (struct timespec *ts, int base)
> @@ -33,9 +63,13 @@ timespec_get (struct timespec *ts, int base)
>        int res;
>        INTERNAL_SYSCALL_DECL (err);
>      case TIME_UTC:
> +#if __WORDSIZE == 32
> +      res = __timespec_get (*ts, base);
> +#else
>        res = INTERNAL_VSYSCALL (clock_gettime, err, 2,
> CLOCK_REALTIME, ts); +#endif
>        if (INTERNAL_SYSCALL_ERROR_P (res, err))
> -	return 0;
> +        return 0;
>        break;
>  
>      default:
> @@ -44,3 +78,4 @@ timespec_get (struct timespec *ts, int base)
>  
>    return base;
>  }
> +#endif

Note:
[1] - https://github.com/lmajewski/y2038_glibc/commit/12ac380db090219aac4ed8b0ef179b9fcc4c296e

Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[RFC v3 23/23] RISC-V: Use 64-bit vdso syscalls

[Alistair Francis]

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 sysdeps/unix/sysv/linux/riscv/init-first.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/sysdeps/unix/sysv/linux/riscv/init-first.c b/sysdeps/unix/sysv/linux/riscv/init-first.c
index 155a4a2c0c..fa9c690cdd 100644
--- a/sysdeps/unix/sysv/linux/riscv/init-first.c
+++ b/sysdeps/unix/sysv/linux/riscv/init-first.c
@@ -43,11 +43,11 @@ _libc_vdso_platform_setup (void)
   PTR_MANGLE (p);
   VDSO_SYMBOL (gettimeofday) = p;
 
-  p = _dl_vdso_vsym ("__vdso_clock_gettime", &linux_version);
+  p = _dl_vdso_vsym ("__vdso_clock_gettime64", &linux_version);
   PTR_MANGLE (p);
   VDSO_SYMBOL (clock_gettime) = p;
 
-  p = _dl_vdso_vsym ("__vdso_clock_getres", &linux_version);
+  p = _dl_vdso_vsym ("__vdso_clock_getres_time64", &linux_version);
   PTR_MANGLE (p);
   VDSO_SYMBOL (clock_getres) = p;
 }
-- 
2.22.0

Re: [RFC v3 23/23] RISC-V: Use 64-bit vdso syscalls

[Florian Weimer]

* Alistair Francis:

> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
> ---
>  sysdeps/unix/sysv/linux/riscv/init-first.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/sysdeps/unix/sysv/linux/riscv/init-first.c b/sysdeps/unix/sysv/linux/riscv/init-first.c
> index 155a4a2c0c..fa9c690cdd 100644
> --- a/sysdeps/unix/sysv/linux/riscv/init-first.c
> +++ b/sysdeps/unix/sysv/linux/riscv/init-first.c
> @@ -43,11 +43,11 @@ _libc_vdso_platform_setup (void)
>    PTR_MANGLE (p);
>    VDSO_SYMBOL (gettimeofday) = p;
>  
> -  p = _dl_vdso_vsym ("__vdso_clock_gettime", &linux_version);
> +  p = _dl_vdso_vsym ("__vdso_clock_gettime64", &linux_version);
>    PTR_MANGLE (p);
>    VDSO_SYMBOL (clock_gettime) = p;
>  
> -  p = _dl_vdso_vsym ("__vdso_clock_getres", &linux_version);
> +  p = _dl_vdso_vsym ("__vdso_clock_getres_time64", &linux_version);
>    PTR_MANGLE (p);
>    VDSO_SYMBOL (clock_getres) = p;
>  }

I believe this will lose vDSO acceleration for RV64 on older kernels
which do not define __vdso_clock_getres_time64.

Thanks,
Florian

Re: [RFC v3 23/23] RISC-V: Use 64-bit vdso syscalls

[Arnd Bergmann]

On Wed, Jul 17, 2019 at 7:33 AM Florian Weimer <fweimer@redhat.com> wrote:
>
> * Alistair Francis:
>
> > Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
> > ---
> >  sysdeps/unix/sysv/linux/riscv/init-first.c | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/sysdeps/unix/sysv/linux/riscv/init-first.c b/sysdeps/unix/sysv/linux/riscv/init-first.c
> > index 155a4a2c0c..fa9c690cdd 100644
> > --- a/sysdeps/unix/sysv/linux/riscv/init-first.c
> > +++ b/sysdeps/unix/sysv/linux/riscv/init-first.c
> > @@ -43,11 +43,11 @@ _libc_vdso_platform_setup (void)
> >    PTR_MANGLE (p);
> >    VDSO_SYMBOL (gettimeofday) = p;
> >
> > -  p = _dl_vdso_vsym ("__vdso_clock_gettime", &linux_version);
> > +  p = _dl_vdso_vsym ("__vdso_clock_gettime64", &linux_version);
> >    PTR_MANGLE (p);
> >    VDSO_SYMBOL (clock_gettime) = p;
> >
> > -  p = _dl_vdso_vsym ("__vdso_clock_getres", &linux_version);
> > +  p = _dl_vdso_vsym ("__vdso_clock_getres_time64", &linux_version);
> >    PTR_MANGLE (p);
> >    VDSO_SYMBOL (clock_getres) = p;
> >  }
>
> I believe this will lose vDSO acceleration for RV64 on older kernels
> which do not define __vdso_clock_getres_time64.

Also newer kernels, as 64-bit kernels only have __vdso_clock_getres,
not __vdso_clock_getres_time64.

       Arnd

Re: [RFC v3 23/23] RISC-V: Use 64-bit vdso syscalls

[Alistair Francis]

On Wed, Jul 17, 2019 at 1:02 AM Arnd Bergmann <arnd@arndb.de> wrote:
>
> On Wed, Jul 17, 2019 at 7:33 AM Florian Weimer <fweimer@redhat.com> wrote:
> >
> > * Alistair Francis:
> >
> > > Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
> > > ---
> > >  sysdeps/unix/sysv/linux/riscv/init-first.c | 4 ++--
> > >  1 file changed, 2 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/sysdeps/unix/sysv/linux/riscv/init-first.c b/sysdeps/unix/sysv/linux/riscv/init-first.c
> > > index 155a4a2c0c..fa9c690cdd 100644
> > > --- a/sysdeps/unix/sysv/linux/riscv/init-first.c
> > > +++ b/sysdeps/unix/sysv/linux/riscv/init-first.c
> > > @@ -43,11 +43,11 @@ _libc_vdso_platform_setup (void)
> > >    PTR_MANGLE (p);
> > >    VDSO_SYMBOL (gettimeofday) = p;
> > >
> > > -  p = _dl_vdso_vsym ("__vdso_clock_gettime", &linux_version);
> > > +  p = _dl_vdso_vsym ("__vdso_clock_gettime64", &linux_version);
> > >    PTR_MANGLE (p);
> > >    VDSO_SYMBOL (clock_gettime) = p;
> > >
> > > -  p = _dl_vdso_vsym ("__vdso_clock_getres", &linux_version);
> > > +  p = _dl_vdso_vsym ("__vdso_clock_getres_time64", &linux_version);
> > >    PTR_MANGLE (p);
> > >    VDSO_SYMBOL (clock_getres) = p;
> > >  }
> >
> > I believe this will lose vDSO acceleration for RV64 on older kernels
> > which do not define __vdso_clock_getres_time64.
>
> Also newer kernels, as 64-bit kernels only have __vdso_clock_getres,
> not __vdso_clock_getres_time64.

Ok, I can #ifdef this based on word size.

Alistair

>
>        Arnd

Re: [RFC v3 23/23] RISC-V: Use 64-bit vdso syscalls

[Alistair Francis]

 On Wed, Jul 17, 2019 at 3:23 PM Alistair Francis <alistair23@gmail.com> wrote:
>
> On Wed, Jul 17, 2019 at 1:02 AM Arnd Bergmann <arnd@arndb.de> wrote:
> >
> > On Wed, Jul 17, 2019 at 7:33 AM Florian Weimer <fweimer@redhat.com> wrote:
> > >
> > > * Alistair Francis:
> > >
> > > > Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
> > > > ---
> > > >  sysdeps/unix/sysv/linux/riscv/init-first.c | 4 ++--
> > > >  1 file changed, 2 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/sysdeps/unix/sysv/linux/riscv/init-first.c b/sysdeps/unix/sysv/linux/riscv/init-first.c
> > > > index 155a4a2c0c..fa9c690cdd 100644
> > > > --- a/sysdeps/unix/sysv/linux/riscv/init-first.c
> > > > +++ b/sysdeps/unix/sysv/linux/riscv/init-first.c
> > > > @@ -43,11 +43,11 @@ _libc_vdso_platform_setup (void)
> > > >    PTR_MANGLE (p);
> > > >    VDSO_SYMBOL (gettimeofday) = p;
> > > >
> > > > -  p = _dl_vdso_vsym ("__vdso_clock_gettime", &linux_version);
> > > > +  p = _dl_vdso_vsym ("__vdso_clock_gettime64", &linux_version);
> > > >    PTR_MANGLE (p);
> > > >    VDSO_SYMBOL (clock_gettime) = p;
> > > >
> > > > -  p = _dl_vdso_vsym ("__vdso_clock_getres", &linux_version);
> > > > +  p = _dl_vdso_vsym ("__vdso_clock_getres_time64", &linux_version);
> > > >    PTR_MANGLE (p);
> > > >    VDSO_SYMBOL (clock_getres) = p;
> > > >  }
> > >
> > > I believe this will lose vDSO acceleration for RV64 on older kernels
> > > which do not define __vdso_clock_getres_time64.
> >
> > Also newer kernels, as 64-bit kernels only have __vdso_clock_getres,
> > not __vdso_clock_getres_time64.

Just to confirm (based on this thread and "RISC-V: Use 64-bit timespec
in clock_gettime vdso calls"):
 - change the clock_gettime64 vdso symbol to  __clock_gettime64 for
RV32 to match the kernel
 - Remove clock_gettime, clock_getres and gettimeofday vdso calls for RV32

Alistair

>
> Ok, I can #ifdef this based on word size.
>
> Alistair
>
> >
> >        Arnd

Re: [RFC v3 23/23] RISC-V: Use 64-bit vdso syscalls

[Alistair Francis]

On Wed, Jul 17, 2019 at 4:42 PM Alistair Francis <alistair23@gmail.com> wrote:
>
>  On Wed, Jul 17, 2019 at 3:23 PM Alistair Francis <alistair23@gmail.com> wrote:
> >
> > On Wed, Jul 17, 2019 at 1:02 AM Arnd Bergmann <arnd@arndb.de> wrote:
> > >
> > > On Wed, Jul 17, 2019 at 7:33 AM Florian Weimer <fweimer@redhat.com> wrote:
> > > >
> > > > * Alistair Francis:
> > > >
> > > > > Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
> > > > > ---
> > > > >  sysdeps/unix/sysv/linux/riscv/init-first.c | 4 ++--
> > > > >  1 file changed, 2 insertions(+), 2 deletions(-)
> > > > >
> > > > > diff --git a/sysdeps/unix/sysv/linux/riscv/init-first.c b/sysdeps/unix/sysv/linux/riscv/init-first.c
> > > > > index 155a4a2c0c..fa9c690cdd 100644
> > > > > --- a/sysdeps/unix/sysv/linux/riscv/init-first.c
> > > > > +++ b/sysdeps/unix/sysv/linux/riscv/init-first.c
> > > > > @@ -43,11 +43,11 @@ _libc_vdso_platform_setup (void)
> > > > >    PTR_MANGLE (p);
> > > > >    VDSO_SYMBOL (gettimeofday) = p;
> > > > >
> > > > > -  p = _dl_vdso_vsym ("__vdso_clock_gettime", &linux_version);
> > > > > +  p = _dl_vdso_vsym ("__vdso_clock_gettime64", &linux_version);
> > > > >    PTR_MANGLE (p);
> > > > >    VDSO_SYMBOL (clock_gettime) = p;
> > > > >
> > > > > -  p = _dl_vdso_vsym ("__vdso_clock_getres", &linux_version);
> > > > > +  p = _dl_vdso_vsym ("__vdso_clock_getres_time64", &linux_version);
> > > > >    PTR_MANGLE (p);
> > > > >    VDSO_SYMBOL (clock_getres) = p;
> > > > >  }
> > > >
> > > > I believe this will lose vDSO acceleration for RV64 on older kernels
> > > > which do not define __vdso_clock_getres_time64.
> > >
> > > Also newer kernels, as 64-bit kernels only have __vdso_clock_getres,
> > > not __vdso_clock_getres_time64.
>
> Just to confirm (based on this thread and "RISC-V: Use 64-bit timespec
> in clock_gettime vdso calls"):
>  - change the clock_gettime64 vdso symbol to  __clock_gettime64 for
> RV32 to match the kernel

Wait this breaks all of the callers of INTERNAL_VSYSCALL() as the
syscall is just __NR_clock_gettime64 not __NR___clock_gettime64.

Alistair

>  - Remove clock_gettime, clock_getres and gettimeofday vdso calls for RV32
>
> Alistair
>
> >
> > Ok, I can #ifdef this based on word size.
> >
> > Alistair
> >
> > >
> > >        Arnd

[RFC v3 19/23] RISC-V: Add ABI lists

[Alistair Francis]

From: Zong Li <zongbox@gmail.com>

Use the check-api and update-abi to generate the abilist for rv32.

2018-11-29  Zong Li  <zong@andestech.com>

	* sysdeps/unix/sysv/linux/riscv/rv32/c++-types.data: New file.
	* sysdeps/unix/sysv/linux/riscv/rv32/ld.abilist: Likewise.
	* sysdeps/unix/sysv/linux/riscv/rv32/libanl.abilist: Likewise.
	* sysdeps/unix/sysv/linux/riscv/rv32/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/riscv/rv32/libcrypt.abilist: Likewise.
	* sysdeps/unix/sysv/linux/riscv/rv32/libdl.abilist: Likewise.
	* sysdeps/unix/sysv/linux/riscv/rv32/libm.abilist: Likewise.
	* sysdeps/unix/sysv/linux/riscv/rv32/libpthread.abilist: Likewise.
	* sysdeps/unix/sysv/linux/riscv/rv32/libresolv.abilist: Likewise.
	* sysdeps/unix/sysv/linux/riscv/rv32/librt.abilist: Likewise.
	* sysdeps/unix/sysv/linux/riscv/rv32/libthread_db.abilist: Likewise.
	* sysdeps/unix/sysv/linux/riscv/rv32/libutil.abilist: Likewise.
	* sysdeps/unix/sysv/linux/riscv/rv32/libBrokenLocale.abilist:
	Likewise.
---
 ChangeLog                                     |   32 +
 .../unix/sysv/linux/riscv/rv32/c++-types.data |   67 +
 sysdeps/unix/sysv/linux/riscv/rv32/ld.abilist |    9 +
 .../linux/riscv/rv32/libBrokenLocale.abilist  |    1 +
 .../unix/sysv/linux/riscv/rv32/libanl.abilist |    4 +
 .../unix/sysv/linux/riscv/rv32/libc.abilist   | 2101 +++++++++++++++++
 .../sysv/linux/riscv/rv32/libcrypt.abilist    |    2 +
 .../unix/sysv/linux/riscv/rv32/libdl.abilist  |    9 +
 .../unix/sysv/linux/riscv/rv32/libm.abilist   | 1021 ++++++++
 .../sysv/linux/riscv/rv32/libpthread.abilist  |  235 ++
 .../sysv/linux/riscv/rv32/libresolv.abilist   |   79 +
 .../unix/sysv/linux/riscv/rv32/librt.abilist  |   35 +
 .../linux/riscv/rv32/libthread_db.abilist     |   40 +
 .../sysv/linux/riscv/rv32/libutil.abilist     |    6 +
 14 files changed, 3641 insertions(+)
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/c++-types.data
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/ld.abilist
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libBrokenLocale.abilist
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libanl.abilist
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libc.abilist
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libcrypt.abilist
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libdl.abilist
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libm.abilist
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libpthread.abilist
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libresolv.abilist
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/librt.abilist
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libthread_db.abilist
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libutil.abilist

diff --git a/ChangeLog b/ChangeLog
index 8f2224ee93..c31e2dff87 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1073,6 +1073,38 @@
 	* sysdeps/riscv/rvd/libm-test-ulps-name: New file.
 	* sysdeps/riscv/rv64/rvd/libm-test-ulps: Remove file.
 	* sysdeps/riscv/rv64/rvd/libm-test-ulps-name: Remove file.
+	* sysdeps/unix/sysv/linux/riscv/ldconfig.h (LD_SO_ABI): Support rv32.
+	* sysdeps/unix/sysv/linux/riscv/dl-cache.h (add_system_dir): Add
+	libraries path for rv32.
+	* sysdeps/riscv/bits/wordsize.h: Supprt rv32.
+	* sysdeps/riscv/nptl/bits/pthreadtypes-arch.h: Likewise.
+	* sysdeps/riscv/sfp-machine.h: Likewise.
+	* sysdeps/riscv/sys/asm.h: Likewise.
+	* sysdeps/unix/sysv/linux/riscv/rv32/jmp_buf-macros.h: New file.
+	* sysdeps/unix/sysv/linux/riscv/rv32/lockf64.c: Likewise.
+	* sysdeps/riscv/rv32/rvd/s_lrint.c: New file.
+	* sysdeps/riscv/rv32/rvd/s_lround.c: Likewise.
+	* sysdeps/riscv/rv32/rvf/s_lrintf.c: Likewise.
+	* sysdeps/riscv/rv32/rvf/s_lroundf.c: Likewise.
+	* sysdeps/riscv/nofpu/libm-test-ulps: Regenerate.
+	* sysdeps/riscv/rvd/libm-test-ulps: New file.
+	* sysdeps/riscv/rvd/libm-test-ulps-name: New file.
+	* sysdeps/riscv/rv64/rvd/libm-test-ulps: Remove file.
+	* sysdeps/riscv/rv64/rvd/libm-test-ulps-name: Remove file.
+	* sysdeps/unix/sysv/linux/riscv/rv32/c++-types.data: New file.
+	* sysdeps/unix/sysv/linux/riscv/rv32/ld.abilist: Likewise.
+	* sysdeps/unix/sysv/linux/riscv/rv32/libanl.abilist: Likewise.
+	* sysdeps/unix/sysv/linux/riscv/rv32/libc.abilist: Likewise.
+	* sysdeps/unix/sysv/linux/riscv/rv32/libcrypt.abilist: Likewise.
+	* sysdeps/unix/sysv/linux/riscv/rv32/libdl.abilist: Likewise.
+	* sysdeps/unix/sysv/linux/riscv/rv32/libm.abilist: Likewise.
+	* sysdeps/unix/sysv/linux/riscv/rv32/libpthread.abilist: Likewise.
+	* sysdeps/unix/sysv/linux/riscv/rv32/libresolv.abilist: Likewise.
+	* sysdeps/unix/sysv/linux/riscv/rv32/librt.abilist: Likewise.
+	* sysdeps/unix/sysv/linux/riscv/rv32/libthread_db.abilist: Likewise.
+	* sysdeps/unix/sysv/linux/riscv/rv32/libutil.abilist: Likewise.
+	* sysdeps/unix/sysv/linux/riscv/rv32/libBrokenLocale.abilist:
+	Likewise.
 
 2019-06-20  Dmitry V. Levin  <ldv@altlinux.org>
 	    Florian Weimer  <fweimer@redhat.com>
diff --git a/sysdeps/unix/sysv/linux/riscv/rv32/c++-types.data b/sysdeps/unix/sysv/linux/riscv/rv32/c++-types.data
new file mode 100644
index 0000000000..303f4570c8
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/riscv/rv32/c++-types.data
@@ -0,0 +1,67 @@
+blkcnt64_t:x
+blkcnt_t:l
+blksize_t:i
+caddr_t:Pc
+clockid_t:i
+clock_t:l
+daddr_t:i
+dev_t:y
+fd_mask:l
+fsblkcnt64_t:y
+fsblkcnt_t:m
+fsfilcnt64_t:y
+fsfilcnt_t:m
+fsid_t:8__fsid_t
+gid_t:j
+id_t:j
+ino64_t:y
+ino_t:m
+int16_t:s
+int32_t:i
+int64_t:x
+int8_t:a
+intptr_t:i
+key_t:i
+loff_t:x
+mode_t:j
+nlink_t:j
+off64_t:x
+off_t:l
+pid_t:i
+pthread_attr_t:14pthread_attr_t
+pthread_barrier_t:17pthread_barrier_t
+pthread_barrierattr_t:21pthread_barrierattr_t
+pthread_cond_t:14pthread_cond_t
+pthread_condattr_t:18pthread_condattr_t
+pthread_key_t:j
+pthread_mutex_t:15pthread_mutex_t
+pthread_mutexattr_t:19pthread_mutexattr_t
+pthread_once_t:i
+pthread_rwlock_t:16pthread_rwlock_t
+pthread_rwlockattr_t:20pthread_rwlockattr_t
+pthread_spinlock_t:i
+pthread_t:m
+quad_t:x
+register_t:i
+rlim64_t:y
+rlim_t:m
+sigset_t:10__sigset_t
+size_t:j
+socklen_t:j
+ssize_t:i
+suseconds_t:l
+time_t:l
+u_char:h
+uid_t:j
+uint:j
+u_int:j
+u_int16_t:t
+u_int32_t:j
+u_int64_t:y
+u_int8_t:h
+ulong:m
+u_long:m
+u_quad_t:y
+useconds_t:j
+ushort:t
+u_short:t
diff --git a/sysdeps/unix/sysv/linux/riscv/rv32/ld.abilist b/sysdeps/unix/sysv/linux/riscv/rv32/ld.abilist
new file mode 100644
index 0000000000..71576160ed
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/riscv/rv32/ld.abilist
@@ -0,0 +1,9 @@
+GLIBC_2.29 __libc_stack_end D 0x4
+GLIBC_2.29 __stack_chk_guard D 0x4
+GLIBC_2.29 __tls_get_addr F
+GLIBC_2.29 _dl_mcount F
+GLIBC_2.29 _r_debug D 0x14
+GLIBC_2.29 calloc F
+GLIBC_2.29 free F
+GLIBC_2.29 malloc F
+GLIBC_2.29 realloc F
diff --git a/sysdeps/unix/sysv/linux/riscv/rv32/libBrokenLocale.abilist b/sysdeps/unix/sysv/linux/riscv/rv32/libBrokenLocale.abilist
new file mode 100644
index 0000000000..96b4163a25
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/riscv/rv32/libBrokenLocale.abilist
@@ -0,0 +1 @@
+GLIBC_2.29 __ctype_get_mb_cur_max F
diff --git a/sysdeps/unix/sysv/linux/riscv/rv32/libanl.abilist b/sysdeps/unix/sysv/linux/riscv/rv32/libanl.abilist
new file mode 100644
index 0000000000..416a6f8ddb
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/riscv/rv32/libanl.abilist
@@ -0,0 +1,4 @@
+GLIBC_2.29 gai_cancel F
+GLIBC_2.29 gai_error F
+GLIBC_2.29 gai_suspend F
+GLIBC_2.29 getaddrinfo_a F
diff --git a/sysdeps/unix/sysv/linux/riscv/rv32/libc.abilist b/sysdeps/unix/sysv/linux/riscv/rv32/libc.abilist
new file mode 100644
index 0000000000..d8035d72b2
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/riscv/rv32/libc.abilist
@@ -0,0 +1,2101 @@
+GLIBC_2.29 _Exit F
+GLIBC_2.29 _IO_2_1_stderr_ D 0xa0
+GLIBC_2.29 _IO_2_1_stdin_ D 0xa0
+GLIBC_2.29 _IO_2_1_stdout_ D 0xa0
+GLIBC_2.29 _IO_adjust_column F
+GLIBC_2.29 _IO_adjust_wcolumn F
+GLIBC_2.29 _IO_default_doallocate F
+GLIBC_2.29 _IO_default_finish F
+GLIBC_2.29 _IO_default_pbackfail F
+GLIBC_2.29 _IO_default_uflow F
+GLIBC_2.29 _IO_default_xsgetn F
+GLIBC_2.29 _IO_default_xsputn F
+GLIBC_2.29 _IO_do_write F
+GLIBC_2.29 _IO_doallocbuf F
+GLIBC_2.29 _IO_fclose F
+GLIBC_2.29 _IO_fdopen F
+GLIBC_2.29 _IO_feof F
+GLIBC_2.29 _IO_ferror F
+GLIBC_2.29 _IO_fflush F
+GLIBC_2.29 _IO_fgetpos F
+GLIBC_2.29 _IO_fgetpos64 F
+GLIBC_2.29 _IO_fgets F
+GLIBC_2.29 _IO_file_attach F
+GLIBC_2.29 _IO_file_close F
+GLIBC_2.29 _IO_file_close_it F
+GLIBC_2.29 _IO_file_doallocate F
+GLIBC_2.29 _IO_file_finish F
+GLIBC_2.29 _IO_file_fopen F
+GLIBC_2.29 _IO_file_init F
+GLIBC_2.29 _IO_file_jumps D 0x54
+GLIBC_2.29 _IO_file_open F
+GLIBC_2.29 _IO_file_overflow F
+GLIBC_2.29 _IO_file_read F
+GLIBC_2.29 _IO_file_seek F
+GLIBC_2.29 _IO_file_seekoff F
+GLIBC_2.29 _IO_file_setbuf F
+GLIBC_2.29 _IO_file_stat F
+GLIBC_2.29 _IO_file_sync F
+GLIBC_2.29 _IO_file_underflow F
+GLIBC_2.29 _IO_file_write F
+GLIBC_2.29 _IO_file_xsputn F
+GLIBC_2.29 _IO_flockfile F
+GLIBC_2.29 _IO_flush_all F
+GLIBC_2.29 _IO_flush_all_linebuffered F
+GLIBC_2.29 _IO_fopen F
+GLIBC_2.29 _IO_fprintf F
+GLIBC_2.29 _IO_fputs F
+GLIBC_2.29 _IO_fread F
+GLIBC_2.29 _IO_free_backup_area F
+GLIBC_2.29 _IO_free_wbackup_area F
+GLIBC_2.29 _IO_fsetpos F
+GLIBC_2.29 _IO_fsetpos64 F
+GLIBC_2.29 _IO_ftell F
+GLIBC_2.29 _IO_ftrylockfile F
+GLIBC_2.29 _IO_funlockfile F
+GLIBC_2.29 _IO_fwrite F
+GLIBC_2.29 _IO_getc F
+GLIBC_2.29 _IO_getline F
+GLIBC_2.29 _IO_getline_info F
+GLIBC_2.29 _IO_gets F
+GLIBC_2.29 _IO_init F
+GLIBC_2.29 _IO_init_marker F
+GLIBC_2.29 _IO_init_wmarker F
+GLIBC_2.29 _IO_iter_begin F
+GLIBC_2.29 _IO_iter_end F
+GLIBC_2.29 _IO_iter_file F
+GLIBC_2.29 _IO_iter_next F
+GLIBC_2.29 _IO_least_wmarker F
+GLIBC_2.29 _IO_link_in F
+GLIBC_2.29 _IO_list_all D 0x4
+GLIBC_2.29 _IO_list_lock F
+GLIBC_2.29 _IO_list_resetlock F
+GLIBC_2.29 _IO_list_unlock F
+GLIBC_2.29 _IO_marker_delta F
+GLIBC_2.29 _IO_marker_difference F
+GLIBC_2.29 _IO_padn F
+GLIBC_2.29 _IO_peekc_locked F
+GLIBC_2.29 _IO_popen F
+GLIBC_2.29 _IO_printf F
+GLIBC_2.29 _IO_proc_close F
+GLIBC_2.29 _IO_proc_open F
+GLIBC_2.29 _IO_putc F
+GLIBC_2.29 _IO_puts F
+GLIBC_2.29 _IO_remove_marker F
+GLIBC_2.29 _IO_seekmark F
+GLIBC_2.29 _IO_seekoff F
+GLIBC_2.29 _IO_seekpos F
+GLIBC_2.29 _IO_seekwmark F
+GLIBC_2.29 _IO_setb F
+GLIBC_2.29 _IO_setbuffer F
+GLIBC_2.29 _IO_setvbuf F
+GLIBC_2.29 _IO_sgetn F
+GLIBC_2.29 _IO_sprintf F
+GLIBC_2.29 _IO_sputbackc F
+GLIBC_2.29 _IO_sputbackwc F
+GLIBC_2.29 _IO_sscanf F
+GLIBC_2.29 _IO_str_init_readonly F
+GLIBC_2.29 _IO_str_init_static F
+GLIBC_2.29 _IO_str_overflow F
+GLIBC_2.29 _IO_str_pbackfail F
+GLIBC_2.29 _IO_str_seekoff F
+GLIBC_2.29 _IO_str_underflow F
+GLIBC_2.29 _IO_sungetc F
+GLIBC_2.29 _IO_sungetwc F
+GLIBC_2.29 _IO_switch_to_get_mode F
+GLIBC_2.29 _IO_switch_to_main_wget_area F
+GLIBC_2.29 _IO_switch_to_wbackup_area F
+GLIBC_2.29 _IO_switch_to_wget_mode F
+GLIBC_2.29 _IO_un_link F
+GLIBC_2.29 _IO_ungetc F
+GLIBC_2.29 _IO_unsave_markers F
+GLIBC_2.29 _IO_unsave_wmarkers F
+GLIBC_2.29 _IO_vfprintf F
+GLIBC_2.29 _IO_vsprintf F
+GLIBC_2.29 _IO_wdefault_doallocate F
+GLIBC_2.29 _IO_wdefault_finish F
+GLIBC_2.29 _IO_wdefault_pbackfail F
+GLIBC_2.29 _IO_wdefault_uflow F
+GLIBC_2.29 _IO_wdefault_xsgetn F
+GLIBC_2.29 _IO_wdefault_xsputn F
+GLIBC_2.29 _IO_wdo_write F
+GLIBC_2.29 _IO_wdoallocbuf F
+GLIBC_2.29 _IO_wfile_jumps D 0x54
+GLIBC_2.29 _IO_wfile_overflow F
+GLIBC_2.29 _IO_wfile_seekoff F
+GLIBC_2.29 _IO_wfile_sync F
+GLIBC_2.29 _IO_wfile_underflow F
+GLIBC_2.29 _IO_wfile_xsputn F
+GLIBC_2.29 _IO_wmarker_delta F
+GLIBC_2.29 _IO_wsetb F
+GLIBC_2.29 ___brk_addr D 0x4
+GLIBC_2.29 __adjtimex F
+GLIBC_2.29 __after_morecore_hook D 0x4
+GLIBC_2.29 __argz_count F
+GLIBC_2.29 __argz_next F
+GLIBC_2.29 __argz_stringify F
+GLIBC_2.29 __asprintf F
+GLIBC_2.29 __asprintf_chk F
+GLIBC_2.29 __assert F
+GLIBC_2.29 __assert_fail F
+GLIBC_2.29 __assert_perror_fail F
+GLIBC_2.29 __backtrace F
+GLIBC_2.29 __backtrace_symbols F
+GLIBC_2.29 __backtrace_symbols_fd F
+GLIBC_2.29 __bsd_getpgrp F
+GLIBC_2.29 __bzero F
+GLIBC_2.29 __check_rhosts_file D 0x4
+GLIBC_2.29 __chk_fail F
+GLIBC_2.29 __clone F
+GLIBC_2.29 __close F
+GLIBC_2.29 __cmsg_nxthdr F
+GLIBC_2.29 __confstr_chk F
+GLIBC_2.29 __connect F
+GLIBC_2.29 __ctype_b_loc F
+GLIBC_2.29 __ctype_get_mb_cur_max F
+GLIBC_2.29 __ctype_tolower_loc F
+GLIBC_2.29 __ctype_toupper_loc F
+GLIBC_2.29 __curbrk D 0x4
+GLIBC_2.29 __cxa_at_quick_exit F
+GLIBC_2.29 __cxa_atexit F
+GLIBC_2.29 __cxa_finalize F
+GLIBC_2.29 __cxa_thread_atexit_impl F
+GLIBC_2.29 __cyg_profile_func_enter F
+GLIBC_2.29 __cyg_profile_func_exit F
+GLIBC_2.29 __daylight D 0x4
+GLIBC_2.29 __dcgettext F
+GLIBC_2.29 __default_morecore F
+GLIBC_2.29 __dgettext F
+GLIBC_2.29 __dprintf_chk F
+GLIBC_2.29 __dup2 F
+GLIBC_2.29 __duplocale F
+GLIBC_2.29 __endmntent F
+GLIBC_2.29 __environ D 0x4
+GLIBC_2.29 __errno_location F
+GLIBC_2.29 __explicit_bzero_chk F
+GLIBC_2.29 __fbufsize F
+GLIBC_2.29 __fcntl F
+GLIBC_2.29 __fdelt_chk F
+GLIBC_2.29 __fdelt_warn F
+GLIBC_2.29 __ffs F
+GLIBC_2.29 __fgets_chk F
+GLIBC_2.29 __fgets_unlocked_chk F
+GLIBC_2.29 __fgetws_chk F
+GLIBC_2.29 __fgetws_unlocked_chk F
+GLIBC_2.29 __finite F
+GLIBC_2.29 __finitef F
+GLIBC_2.29 __finitel F
+GLIBC_2.29 __flbf F
+GLIBC_2.29 __fork F
+GLIBC_2.29 __fpending F
+GLIBC_2.29 __fprintf_chk F
+GLIBC_2.29 __fpu_control D 0x4
+GLIBC_2.29 __fpurge F
+GLIBC_2.29 __fread_chk F
+GLIBC_2.29 __fread_unlocked_chk F
+GLIBC_2.29 __freadable F
+GLIBC_2.29 __freading F
+GLIBC_2.29 __free_hook D 0x4
+GLIBC_2.29 __freelocale F
+GLIBC_2.29 __fsetlocking F
+GLIBC_2.29 __fwprintf_chk F
+GLIBC_2.29 __fwritable F
+GLIBC_2.29 __fwriting F
+GLIBC_2.29 __fxstat F
+GLIBC_2.29 __fxstat64 F
+GLIBC_2.29 __fxstatat F
+GLIBC_2.29 __fxstatat64 F
+GLIBC_2.29 __getauxval F
+GLIBC_2.29 __getcwd_chk F
+GLIBC_2.29 __getdelim F
+GLIBC_2.29 __getdomainname_chk F
+GLIBC_2.29 __getgroups_chk F
+GLIBC_2.29 __gethostname_chk F
+GLIBC_2.29 __getlogin_r_chk F
+GLIBC_2.29 __getmntent_r F
+GLIBC_2.29 __getpagesize F
+GLIBC_2.29 __getpgid F
+GLIBC_2.29 __getpid F
+GLIBC_2.29 __gets_chk F
+GLIBC_2.29 __gettimeofday F
+GLIBC_2.29 __getwd_chk F
+GLIBC_2.29 __gmtime_r F
+GLIBC_2.29 __h_errno_location F
+GLIBC_2.29 __isalnum_l F
+GLIBC_2.29 __isalpha_l F
+GLIBC_2.29 __isascii_l F
+GLIBC_2.29 __isblank_l F
+GLIBC_2.29 __iscntrl_l F
+GLIBC_2.29 __isctype F
+GLIBC_2.29 __isdigit_l F
+GLIBC_2.29 __isgraph_l F
+GLIBC_2.29 __isinf F
+GLIBC_2.29 __isinff F
+GLIBC_2.29 __isinfl F
+GLIBC_2.29 __islower_l F
+GLIBC_2.29 __isnan F
+GLIBC_2.29 __isnanf F
+GLIBC_2.29 __isnanl F
+GLIBC_2.29 __isoc99_fscanf F
+GLIBC_2.29 __isoc99_fwscanf F
+GLIBC_2.29 __isoc99_scanf F
+GLIBC_2.29 __isoc99_sscanf F
+GLIBC_2.29 __isoc99_swscanf F
+GLIBC_2.29 __isoc99_vfscanf F
+GLIBC_2.29 __isoc99_vfwscanf F
+GLIBC_2.29 __isoc99_vscanf F
+GLIBC_2.29 __isoc99_vsscanf F
+GLIBC_2.29 __isoc99_vswscanf F
+GLIBC_2.29 __isoc99_vwscanf F
+GLIBC_2.29 __isoc99_wscanf F
+GLIBC_2.29 __isprint_l F
+GLIBC_2.29 __ispunct_l F
+GLIBC_2.29 __isspace_l F
+GLIBC_2.29 __isupper_l F
+GLIBC_2.29 __iswalnum_l F
+GLIBC_2.29 __iswalpha_l F
+GLIBC_2.29 __iswblank_l F
+GLIBC_2.29 __iswcntrl_l F
+GLIBC_2.29 __iswctype F
+GLIBC_2.29 __iswctype_l F
+GLIBC_2.29 __iswdigit_l F
+GLIBC_2.29 __iswgraph_l F
+GLIBC_2.29 __iswlower_l F
+GLIBC_2.29 __iswprint_l F
+GLIBC_2.29 __iswpunct_l F
+GLIBC_2.29 __iswspace_l F
+GLIBC_2.29 __iswupper_l F
+GLIBC_2.29 __iswxdigit_l F
+GLIBC_2.29 __isxdigit_l F
+GLIBC_2.29 __ivaliduser F
+GLIBC_2.29 __key_decryptsession_pk_LOCAL D 0x4
+GLIBC_2.29 __key_encryptsession_pk_LOCAL D 0x4
+GLIBC_2.29 __key_gendes_LOCAL D 0x4
+GLIBC_2.29 __libc_allocate_rtsig F
+GLIBC_2.29 __libc_calloc F
+GLIBC_2.29 __libc_current_sigrtmax F
+GLIBC_2.29 __libc_current_sigrtmin F
+GLIBC_2.29 __libc_free F
+GLIBC_2.29 __libc_freeres F
+GLIBC_2.29 __libc_init_first F
+GLIBC_2.29 __libc_mallinfo F
+GLIBC_2.29 __libc_malloc F
+GLIBC_2.29 __libc_mallopt F
+GLIBC_2.29 __libc_memalign F
+GLIBC_2.29 __libc_pvalloc F
+GLIBC_2.29 __libc_realloc F
+GLIBC_2.29 __libc_sa_len F
+GLIBC_2.29 __libc_start_main F
+GLIBC_2.29 __libc_valloc F
+GLIBC_2.29 __longjmp_chk F
+GLIBC_2.29 __lseek F
+GLIBC_2.29 __lxstat F
+GLIBC_2.29 __lxstat64 F
+GLIBC_2.29 __malloc_hook D 0x4
+GLIBC_2.29 __mbrlen F
+GLIBC_2.29 __mbrtowc F
+GLIBC_2.29 __mbsnrtowcs_chk F
+GLIBC_2.29 __mbsrtowcs_chk F
+GLIBC_2.29 __mbstowcs_chk F
+GLIBC_2.29 __memalign_hook D 0x4
+GLIBC_2.29 __memcpy_chk F
+GLIBC_2.29 __memmove_chk F
+GLIBC_2.29 __mempcpy F
+GLIBC_2.29 __mempcpy_chk F
+GLIBC_2.29 __memset_chk F
+GLIBC_2.29 __monstartup F
+GLIBC_2.29 __morecore D 0x4
+GLIBC_2.29 __nanosleep F
+GLIBC_2.29 __newlocale F
+GLIBC_2.29 __nl_langinfo_l F
+GLIBC_2.29 __nss_configure_lookup F
+GLIBC_2.29 __nss_database_lookup F
+GLIBC_2.29 __nss_hostname_digits_dots F
+GLIBC_2.29 __nss_next F
+GLIBC_2.29 __obstack_printf_chk F
+GLIBC_2.29 __obstack_vprintf_chk F
+GLIBC_2.29 __open F
+GLIBC_2.29 __open64 F
+GLIBC_2.29 __open64_2 F
+GLIBC_2.29 __open_2 F
+GLIBC_2.29 __openat64_2 F
+GLIBC_2.29 __openat_2 F
+GLIBC_2.29 __overflow F
+GLIBC_2.29 __pipe F
+GLIBC_2.29 __poll F
+GLIBC_2.29 __poll_chk F
+GLIBC_2.29 __posix_getopt F
+GLIBC_2.29 __ppoll_chk F
+GLIBC_2.29 __pread64 F
+GLIBC_2.29 __pread64_chk F
+GLIBC_2.29 __pread_chk F
+GLIBC_2.29 __printf_chk F
+GLIBC_2.29 __printf_fp F
+GLIBC_2.29 __profile_frequency F
+GLIBC_2.29 __progname D 0x4
+GLIBC_2.29 __progname_full D 0x4
+GLIBC_2.29 __ptsname_r_chk F
+GLIBC_2.29 __pwrite64 F
+GLIBC_2.29 __rawmemchr F
+GLIBC_2.29 __rcmd_errstr D 0x4
+GLIBC_2.29 __read F
+GLIBC_2.29 __read_chk F
+GLIBC_2.29 __readlink_chk F
+GLIBC_2.29 __readlinkat_chk F
+GLIBC_2.29 __realloc_hook D 0x4
+GLIBC_2.29 __realpath_chk F
+GLIBC_2.29 __recv_chk F
+GLIBC_2.29 __recvfrom_chk F
+GLIBC_2.29 __register_atfork F
+GLIBC_2.29 __res_init F
+GLIBC_2.29 __res_nclose F
+GLIBC_2.29 __res_ninit F
+GLIBC_2.29 __res_randomid F
+GLIBC_2.29 __res_state F
+GLIBC_2.29 __riscv_flush_icache F
+GLIBC_2.29 __rpc_thread_createerr F
+GLIBC_2.29 __rpc_thread_svc_fdset F
+GLIBC_2.29 __rpc_thread_svc_max_pollfd F
+GLIBC_2.29 __rpc_thread_svc_pollfd F
+GLIBC_2.29 __sbrk F
+GLIBC_2.29 __sched_cpualloc F
+GLIBC_2.29 __sched_cpucount F
+GLIBC_2.29 __sched_cpufree F
+GLIBC_2.29 __sched_get_priority_max F
+GLIBC_2.29 __sched_get_priority_min F
+GLIBC_2.29 __sched_getparam F
+GLIBC_2.29 __sched_getscheduler F
+GLIBC_2.29 __sched_setscheduler F
+GLIBC_2.29 __sched_yield F
+GLIBC_2.29 __select F
+GLIBC_2.29 __send F
+GLIBC_2.29 __setmntent F
+GLIBC_2.29 __setpgid F
+GLIBC_2.29 __sigaction F
+GLIBC_2.29 __signbit F
+GLIBC_2.29 __signbitf F
+GLIBC_2.29 __signbitl F
+GLIBC_2.29 __sigpause F
+GLIBC_2.29 __sigsetjmp F
+GLIBC_2.29 __sigsuspend F
+GLIBC_2.29 __snprintf_chk F
+GLIBC_2.29 __sprintf_chk F
+GLIBC_2.29 __stack_chk_fail F
+GLIBC_2.29 __statfs F
+GLIBC_2.29 __stpcpy F
+GLIBC_2.29 __stpcpy_chk F
+GLIBC_2.29 __stpncpy F
+GLIBC_2.29 __stpncpy_chk F
+GLIBC_2.29 __strcasecmp F
+GLIBC_2.29 __strcasecmp_l F
+GLIBC_2.29 __strcasestr F
+GLIBC_2.29 __strcat_chk F
+GLIBC_2.29 __strcoll_l F
+GLIBC_2.29 __strcpy_chk F
+GLIBC_2.29 __strdup F
+GLIBC_2.29 __strerror_r F
+GLIBC_2.29 __strfmon_l F
+GLIBC_2.29 __strftime_l F
+GLIBC_2.29 __strncasecmp_l F
+GLIBC_2.29 __strncat_chk F
+GLIBC_2.29 __strncpy_chk F
+GLIBC_2.29 __strndup F
+GLIBC_2.29 __strsep_g F
+GLIBC_2.29 __strtod_internal F
+GLIBC_2.29 __strtod_l F
+GLIBC_2.29 __strtof_internal F
+GLIBC_2.29 __strtof_l F
+GLIBC_2.29 __strtok_r F
+GLIBC_2.29 __strtol_internal F
+GLIBC_2.29 __strtol_l F
+GLIBC_2.29 __strtold_internal F
+GLIBC_2.29 __strtold_l F
+GLIBC_2.29 __strtoll_internal F
+GLIBC_2.29 __strtoll_l F
+GLIBC_2.29 __strtoul_internal F
+GLIBC_2.29 __strtoul_l F
+GLIBC_2.29 __strtoull_internal F
+GLIBC_2.29 __strtoull_l F
+GLIBC_2.29 __strverscmp F
+GLIBC_2.29 __strxfrm_l F
+GLIBC_2.29 __swprintf_chk F
+GLIBC_2.29 __sysconf F
+GLIBC_2.29 __syslog_chk F
+GLIBC_2.29 __sysv_signal F
+GLIBC_2.29 __timezone D 0x4
+GLIBC_2.29 __toascii_l F
+GLIBC_2.29 __tolower_l F
+GLIBC_2.29 __toupper_l F
+GLIBC_2.29 __towctrans F
+GLIBC_2.29 __towctrans_l F
+GLIBC_2.29 __towlower_l F
+GLIBC_2.29 __towupper_l F
+GLIBC_2.29 __ttyname_r_chk F
+GLIBC_2.29 __tzname D 0x8
+GLIBC_2.29 __uflow F
+GLIBC_2.29 __underflow F
+GLIBC_2.29 __uselocale F
+GLIBC_2.29 __vasprintf_chk F
+GLIBC_2.29 __vdprintf_chk F
+GLIBC_2.29 __vfork F
+GLIBC_2.29 __vfprintf_chk F
+GLIBC_2.29 __vfscanf F
+GLIBC_2.29 __vfwprintf_chk F
+GLIBC_2.29 __vprintf_chk F
+GLIBC_2.29 __vsnprintf F
+GLIBC_2.29 __vsnprintf_chk F
+GLIBC_2.29 __vsprintf_chk F
+GLIBC_2.29 __vsscanf F
+GLIBC_2.29 __vswprintf_chk F
+GLIBC_2.29 __vsyslog_chk F
+GLIBC_2.29 __vwprintf_chk F
+GLIBC_2.29 __wait F
+GLIBC_2.29 __waitpid F
+GLIBC_2.29 __wcpcpy_chk F
+GLIBC_2.29 __wcpncpy_chk F
+GLIBC_2.29 __wcrtomb_chk F
+GLIBC_2.29 __wcscasecmp_l F
+GLIBC_2.29 __wcscat_chk F
+GLIBC_2.29 __wcscoll_l F
+GLIBC_2.29 __wcscpy_chk F
+GLIBC_2.29 __wcsftime_l F
+GLIBC_2.29 __wcsncasecmp_l F
+GLIBC_2.29 __wcsncat_chk F
+GLIBC_2.29 __wcsncpy_chk F
+GLIBC_2.29 __wcsnrtombs_chk F
+GLIBC_2.29 __wcsrtombs_chk F
+GLIBC_2.29 __wcstod_internal F
+GLIBC_2.29 __wcstod_l F
+GLIBC_2.29 __wcstof_internal F
+GLIBC_2.29 __wcstof_l F
+GLIBC_2.29 __wcstol_internal F
+GLIBC_2.29 __wcstol_l F
+GLIBC_2.29 __wcstold_internal F
+GLIBC_2.29 __wcstold_l F
+GLIBC_2.29 __wcstoll_internal F
+GLIBC_2.29 __wcstoll_l F
+GLIBC_2.29 __wcstombs_chk F
+GLIBC_2.29 __wcstoul_internal F
+GLIBC_2.29 __wcstoul_l F
+GLIBC_2.29 __wcstoull_internal F
+GLIBC_2.29 __wcstoull_l F
+GLIBC_2.29 __wcsxfrm_l F
+GLIBC_2.29 __wctomb_chk F
+GLIBC_2.29 __wctrans_l F
+GLIBC_2.29 __wctype_l F
+GLIBC_2.29 __wmemcpy_chk F
+GLIBC_2.29 __wmemmove_chk F
+GLIBC_2.29 __wmempcpy_chk F
+GLIBC_2.29 __wmemset_chk F
+GLIBC_2.29 __woverflow F
+GLIBC_2.29 __wprintf_chk F
+GLIBC_2.29 __write F
+GLIBC_2.29 __wuflow F
+GLIBC_2.29 __wunderflow F
+GLIBC_2.29 __xmknod F
+GLIBC_2.29 __xmknodat F
+GLIBC_2.29 __xpg_basename F
+GLIBC_2.29 __xpg_sigpause F
+GLIBC_2.29 __xpg_strerror_r F
+GLIBC_2.29 __xstat F
+GLIBC_2.29 __xstat64 F
+GLIBC_2.29 _authenticate F
+GLIBC_2.29 _dl_mcount_wrapper F
+GLIBC_2.29 _dl_mcount_wrapper_check F
+GLIBC_2.29 _environ D 0x4
+GLIBC_2.29 _exit F
+GLIBC_2.29 _flushlbf F
+GLIBC_2.29 _libc_intl_domainname D 0x5
+GLIBC_2.29 _longjmp F
+GLIBC_2.29 _mcleanup F
+GLIBC_2.29 _mcount F
+GLIBC_2.29 _nl_default_dirname D 0x12
+GLIBC_2.29 _nl_domain_bindings D 0x4
+GLIBC_2.29 _nl_msg_cat_cntr D 0x4
+GLIBC_2.29 _null_auth D 0xc
+GLIBC_2.29 _obstack_allocated_p F
+GLIBC_2.29 _obstack_begin F
+GLIBC_2.29 _obstack_begin_1 F
+GLIBC_2.29 _obstack_free F
+GLIBC_2.29 _obstack_memory_used F
+GLIBC_2.29 _obstack_newchunk F
+GLIBC_2.29 _res D 0x200
+GLIBC_2.29 _res_hconf D 0x30
+GLIBC_2.29 _rpc_dtablesize F
+GLIBC_2.29 _seterr_reply F
+GLIBC_2.29 _setjmp F
+GLIBC_2.29 _sys_errlist D 0x21c
+GLIBC_2.29 _sys_nerr D 0x4
+GLIBC_2.29 _sys_siglist D 0x104
+GLIBC_2.29 _tolower F
+GLIBC_2.29 _toupper F
+GLIBC_2.29 a64l F
+GLIBC_2.29 abort F
+GLIBC_2.29 abs F
+GLIBC_2.29 accept F
+GLIBC_2.29 accept4 F
+GLIBC_2.29 access F
+GLIBC_2.29 acct F
+GLIBC_2.29 addmntent F
+GLIBC_2.29 addseverity F
+GLIBC_2.29 adjtime F
+GLIBC_2.29 adjtimex F
+GLIBC_2.29 alarm F
+GLIBC_2.29 aligned_alloc F
+GLIBC_2.29 alphasort F
+GLIBC_2.29 alphasort64 F
+GLIBC_2.29 argp_err_exit_status D 0x4
+GLIBC_2.29 argp_error F
+GLIBC_2.29 argp_failure F
+GLIBC_2.29 argp_help F
+GLIBC_2.29 argp_parse F
+GLIBC_2.29 argp_program_bug_address D 0x4
+GLIBC_2.29 argp_program_version D 0x4
+GLIBC_2.29 argp_program_version_hook D 0x4
+GLIBC_2.29 argp_state_help F
+GLIBC_2.29 argp_usage F
+GLIBC_2.29 argz_add F
+GLIBC_2.29 argz_add_sep F
+GLIBC_2.29 argz_append F
+GLIBC_2.29 argz_count F
+GLIBC_2.29 argz_create F
+GLIBC_2.29 argz_create_sep F
+GLIBC_2.29 argz_delete F
+GLIBC_2.29 argz_extract F
+GLIBC_2.29 argz_insert F
+GLIBC_2.29 argz_next F
+GLIBC_2.29 argz_replace F
+GLIBC_2.29 argz_stringify F
+GLIBC_2.29 asctime F
+GLIBC_2.29 asctime_r F
+GLIBC_2.29 asprintf F
+GLIBC_2.29 atof F
+GLIBC_2.29 atoi F
+GLIBC_2.29 atol F
+GLIBC_2.29 atoll F
+GLIBC_2.29 authdes_create F
+GLIBC_2.29 authdes_getucred F
+GLIBC_2.29 authdes_pk_create F
+GLIBC_2.29 authnone_create F
+GLIBC_2.29 authunix_create F
+GLIBC_2.29 authunix_create_default F
+GLIBC_2.29 backtrace F
+GLIBC_2.29 backtrace_symbols F
+GLIBC_2.29 backtrace_symbols_fd F
+GLIBC_2.29 basename F
+GLIBC_2.29 bcmp F
+GLIBC_2.29 bcopy F
+GLIBC_2.29 bind F
+GLIBC_2.29 bind_textdomain_codeset F
+GLIBC_2.29 bindresvport F
+GLIBC_2.29 bindtextdomain F
+GLIBC_2.29 brk F
+GLIBC_2.29 bsd_signal F
+GLIBC_2.29 bsearch F
+GLIBC_2.29 btowc F
+GLIBC_2.29 bzero F
+GLIBC_2.29 c16rtomb F
+GLIBC_2.29 c32rtomb F
+GLIBC_2.29 calloc F
+GLIBC_2.29 callrpc F
+GLIBC_2.29 canonicalize_file_name F
+GLIBC_2.29 capget F
+GLIBC_2.29 capset F
+GLIBC_2.29 catclose F
+GLIBC_2.29 catgets F
+GLIBC_2.29 catopen F
+GLIBC_2.29 cbc_crypt F
+GLIBC_2.29 cfgetispeed F
+GLIBC_2.29 cfgetospeed F
+GLIBC_2.29 cfmakeraw F
+GLIBC_2.29 cfsetispeed F
+GLIBC_2.29 cfsetospeed F
+GLIBC_2.29 cfsetspeed F
+GLIBC_2.29 chdir F
+GLIBC_2.29 chflags F
+GLIBC_2.29 chmod F
+GLIBC_2.29 chown F
+GLIBC_2.29 chroot F
+GLIBC_2.29 clearenv F
+GLIBC_2.29 clearerr F
+GLIBC_2.29 clearerr_unlocked F
+GLIBC_2.29 clnt_broadcast F
+GLIBC_2.29 clnt_create F
+GLIBC_2.29 clnt_pcreateerror F
+GLIBC_2.29 clnt_perrno F
+GLIBC_2.29 clnt_perror F
+GLIBC_2.29 clnt_spcreateerror F
+GLIBC_2.29 clnt_sperrno F
+GLIBC_2.29 clnt_sperror F
+GLIBC_2.29 clntraw_create F
+GLIBC_2.29 clnttcp_create F
+GLIBC_2.29 clntudp_bufcreate F
+GLIBC_2.29 clntudp_create F
+GLIBC_2.29 clntunix_create F
+GLIBC_2.29 clock F
+GLIBC_2.29 clock_adjtime F
+GLIBC_2.29 clock_getcpuclockid F
+GLIBC_2.29 clock_getres F
+GLIBC_2.29 clock_gettime F
+GLIBC_2.29 clock_nanosleep F
+GLIBC_2.29 clock_settime F
+GLIBC_2.29 clone F
+GLIBC_2.29 close F
+GLIBC_2.29 closedir F
+GLIBC_2.29 closelog F
+GLIBC_2.29 confstr F
+GLIBC_2.29 connect F
+GLIBC_2.29 copy_file_range F
+GLIBC_2.29 copysign F
+GLIBC_2.29 copysignf F
+GLIBC_2.29 copysignl F
+GLIBC_2.29 creat F
+GLIBC_2.29 creat64 F
+GLIBC_2.29 ctermid F
+GLIBC_2.29 ctime F
+GLIBC_2.29 ctime_r F
+GLIBC_2.29 cuserid F
+GLIBC_2.29 daemon F
+GLIBC_2.29 daylight D 0x4
+GLIBC_2.29 dcgettext F
+GLIBC_2.29 dcngettext F
+GLIBC_2.29 delete_module F
+GLIBC_2.29 des_setparity F
+GLIBC_2.29 dgettext F
+GLIBC_2.29 difftime F
+GLIBC_2.29 dirfd F
+GLIBC_2.29 dirname F
+GLIBC_2.29 div F
+GLIBC_2.29 dl_iterate_phdr F
+GLIBC_2.29 dngettext F
+GLIBC_2.29 dprintf F
+GLIBC_2.29 drand48 F
+GLIBC_2.29 drand48_r F
+GLIBC_2.29 dup F
+GLIBC_2.29 dup2 F
+GLIBC_2.29 dup3 F
+GLIBC_2.29 duplocale F
+GLIBC_2.29 dysize F
+GLIBC_2.29 eaccess F
+GLIBC_2.29 ecb_crypt F
+GLIBC_2.29 ecvt F
+GLIBC_2.29 ecvt_r F
+GLIBC_2.29 endaliasent F
+GLIBC_2.29 endfsent F
+GLIBC_2.29 endgrent F
+GLIBC_2.29 endhostent F
+GLIBC_2.29 endmntent F
+GLIBC_2.29 endnetent F
+GLIBC_2.29 endnetgrent F
+GLIBC_2.29 endprotoent F
+GLIBC_2.29 endpwent F
+GLIBC_2.29 endrpcent F
+GLIBC_2.29 endservent F
+GLIBC_2.29 endsgent F
+GLIBC_2.29 endspent F
+GLIBC_2.29 endttyent F
+GLIBC_2.29 endusershell F
+GLIBC_2.29 endutent F
+GLIBC_2.29 endutxent F
+GLIBC_2.29 environ D 0x4
+GLIBC_2.29 envz_add F
+GLIBC_2.29 envz_entry F
+GLIBC_2.29 envz_get F
+GLIBC_2.29 envz_merge F
+GLIBC_2.29 envz_remove F
+GLIBC_2.29 envz_strip F
+GLIBC_2.29 epoll_create F
+GLIBC_2.29 epoll_create1 F
+GLIBC_2.29 epoll_ctl F
+GLIBC_2.29 epoll_pwait F
+GLIBC_2.29 epoll_wait F
+GLIBC_2.29 erand48 F
+GLIBC_2.29 erand48_r F
+GLIBC_2.29 err F
+GLIBC_2.29 error F
+GLIBC_2.29 error_at_line F
+GLIBC_2.29 error_message_count D 0x4
+GLIBC_2.29 error_one_per_line D 0x4
+GLIBC_2.29 error_print_progname D 0x4
+GLIBC_2.29 errx F
+GLIBC_2.29 ether_aton F
+GLIBC_2.29 ether_aton_r F
+GLIBC_2.29 ether_hostton F
+GLIBC_2.29 ether_line F
+GLIBC_2.29 ether_ntoa F
+GLIBC_2.29 ether_ntoa_r F
+GLIBC_2.29 ether_ntohost F
+GLIBC_2.29 euidaccess F
+GLIBC_2.29 eventfd F
+GLIBC_2.29 eventfd_read F
+GLIBC_2.29 eventfd_write F
+GLIBC_2.29 execl F
+GLIBC_2.29 execle F
+GLIBC_2.29 execlp F
+GLIBC_2.29 execv F
+GLIBC_2.29 execve F
+GLIBC_2.29 execvp F
+GLIBC_2.29 execvpe F
+GLIBC_2.29 exit F
+GLIBC_2.29 explicit_bzero F
+GLIBC_2.29 faccessat F
+GLIBC_2.29 fallocate F
+GLIBC_2.29 fallocate64 F
+GLIBC_2.29 fanotify_init F
+GLIBC_2.29 fanotify_mark F
+GLIBC_2.29 fattach F
+GLIBC_2.29 fchdir F
+GLIBC_2.29 fchflags F
+GLIBC_2.29 fchmod F
+GLIBC_2.29 fchmodat F
+GLIBC_2.29 fchown F
+GLIBC_2.29 fchownat F
+GLIBC_2.29 fclose F
+GLIBC_2.29 fcloseall F
+GLIBC_2.29 fcntl F
+GLIBC_2.29 fcntl64 F
+GLIBC_2.29 fcvt F
+GLIBC_2.29 fcvt_r F
+GLIBC_2.29 fdatasync F
+GLIBC_2.29 fdetach F
+GLIBC_2.29 fdopen F
+GLIBC_2.29 fdopendir F
+GLIBC_2.29 feof F
+GLIBC_2.29 feof_unlocked F
+GLIBC_2.29 ferror F
+GLIBC_2.29 ferror_unlocked F
+GLIBC_2.29 fexecve F
+GLIBC_2.29 fflush F
+GLIBC_2.29 fflush_unlocked F
+GLIBC_2.29 ffs F
+GLIBC_2.29 ffsl F
+GLIBC_2.29 ffsll F
+GLIBC_2.29 fgetc F
+GLIBC_2.29 fgetc_unlocked F
+GLIBC_2.29 fgetgrent F
+GLIBC_2.29 fgetgrent_r F
+GLIBC_2.29 fgetpos F
+GLIBC_2.29 fgetpos64 F
+GLIBC_2.29 fgetpwent F
+GLIBC_2.29 fgetpwent_r F
+GLIBC_2.29 fgets F
+GLIBC_2.29 fgets_unlocked F
+GLIBC_2.29 fgetsgent F
+GLIBC_2.29 fgetsgent_r F
+GLIBC_2.29 fgetspent F
+GLIBC_2.29 fgetspent_r F
+GLIBC_2.29 fgetwc F
+GLIBC_2.29 fgetwc_unlocked F
+GLIBC_2.29 fgetws F
+GLIBC_2.29 fgetws_unlocked F
+GLIBC_2.29 fgetxattr F
+GLIBC_2.29 fileno F
+GLIBC_2.29 fileno_unlocked F
+GLIBC_2.29 finite F
+GLIBC_2.29 finitef F
+GLIBC_2.29 finitel F
+GLIBC_2.29 flistxattr F
+GLIBC_2.29 flock F
+GLIBC_2.29 flockfile F
+GLIBC_2.29 fmemopen F
+GLIBC_2.29 fmtmsg F
+GLIBC_2.29 fnmatch F
+GLIBC_2.29 fopen F
+GLIBC_2.29 fopen64 F
+GLIBC_2.29 fopencookie F
+GLIBC_2.29 fork F
+GLIBC_2.29 fpathconf F
+GLIBC_2.29 fprintf F
+GLIBC_2.29 fputc F
+GLIBC_2.29 fputc_unlocked F
+GLIBC_2.29 fputs F
+GLIBC_2.29 fputs_unlocked F
+GLIBC_2.29 fputwc F
+GLIBC_2.29 fputwc_unlocked F
+GLIBC_2.29 fputws F
+GLIBC_2.29 fputws_unlocked F
+GLIBC_2.29 fread F
+GLIBC_2.29 fread_unlocked F
+GLIBC_2.29 free F
+GLIBC_2.29 freeaddrinfo F
+GLIBC_2.29 freeifaddrs F
+GLIBC_2.29 freelocale F
+GLIBC_2.29 fremovexattr F
+GLIBC_2.29 freopen F
+GLIBC_2.29 freopen64 F
+GLIBC_2.29 frexp F
+GLIBC_2.29 frexpf F
+GLIBC_2.29 frexpl F
+GLIBC_2.29 fscanf F
+GLIBC_2.29 fseek F
+GLIBC_2.29 fseeko F
+GLIBC_2.29 fseeko64 F
+GLIBC_2.29 fsetpos F
+GLIBC_2.29 fsetpos64 F
+GLIBC_2.29 fsetxattr F
+GLIBC_2.29 fstatfs F
+GLIBC_2.29 fstatfs64 F
+GLIBC_2.29 fstatvfs F
+GLIBC_2.29 fstatvfs64 F
+GLIBC_2.29 fsync F
+GLIBC_2.29 ftell F
+GLIBC_2.29 ftello F
+GLIBC_2.29 ftello64 F
+GLIBC_2.29 ftime F
+GLIBC_2.29 ftok F
+GLIBC_2.29 ftruncate F
+GLIBC_2.29 ftruncate64 F
+GLIBC_2.29 ftrylockfile F
+GLIBC_2.29 fts64_children F
+GLIBC_2.29 fts64_close F
+GLIBC_2.29 fts64_open F
+GLIBC_2.29 fts64_read F
+GLIBC_2.29 fts64_set F
+GLIBC_2.29 fts_children F
+GLIBC_2.29 fts_close F
+GLIBC_2.29 fts_open F
+GLIBC_2.29 fts_read F
+GLIBC_2.29 fts_set F
+GLIBC_2.29 ftw F
+GLIBC_2.29 ftw64 F
+GLIBC_2.29 funlockfile F
+GLIBC_2.29 futimens F
+GLIBC_2.29 futimes F
+GLIBC_2.29 futimesat F
+GLIBC_2.29 fwide F
+GLIBC_2.29 fwprintf F
+GLIBC_2.29 fwrite F
+GLIBC_2.29 fwrite_unlocked F
+GLIBC_2.29 fwscanf F
+GLIBC_2.29 gai_strerror F
+GLIBC_2.29 gcvt F
+GLIBC_2.29 get_avphys_pages F
+GLIBC_2.29 get_current_dir_name F
+GLIBC_2.29 get_myaddress F
+GLIBC_2.29 get_nprocs F
+GLIBC_2.29 get_nprocs_conf F
+GLIBC_2.29 get_phys_pages F
+GLIBC_2.29 getaddrinfo F
+GLIBC_2.29 getaliasbyname F
+GLIBC_2.29 getaliasbyname_r F
+GLIBC_2.29 getaliasent F
+GLIBC_2.29 getaliasent_r F
+GLIBC_2.29 getauxval F
+GLIBC_2.29 getc F
+GLIBC_2.29 getc_unlocked F
+GLIBC_2.29 getchar F
+GLIBC_2.29 getchar_unlocked F
+GLIBC_2.29 getcontext F
+GLIBC_2.29 getcpu F
+GLIBC_2.29 getcwd F
+GLIBC_2.29 getdate F
+GLIBC_2.29 getdate_err D 0x4
+GLIBC_2.29 getdate_r F
+GLIBC_2.29 getdelim F
+GLIBC_2.29 getdirentries F
+GLIBC_2.29 getdirentries64 F
+GLIBC_2.29 getdomainname F
+GLIBC_2.29 getdtablesize F
+GLIBC_2.29 getegid F
+GLIBC_2.29 getentropy F
+GLIBC_2.29 getenv F
+GLIBC_2.29 geteuid F
+GLIBC_2.29 getfsent F
+GLIBC_2.29 getfsfile F
+GLIBC_2.29 getfsspec F
+GLIBC_2.29 getgid F
+GLIBC_2.29 getgrent F
+GLIBC_2.29 getgrent_r F
+GLIBC_2.29 getgrgid F
+GLIBC_2.29 getgrgid_r F
+GLIBC_2.29 getgrnam F
+GLIBC_2.29 getgrnam_r F
+GLIBC_2.29 getgrouplist F
+GLIBC_2.29 getgroups F
+GLIBC_2.29 gethostbyaddr F
+GLIBC_2.29 gethostbyaddr_r F
+GLIBC_2.29 gethostbyname F
+GLIBC_2.29 gethostbyname2 F
+GLIBC_2.29 gethostbyname2_r F
+GLIBC_2.29 gethostbyname_r F
+GLIBC_2.29 gethostent F
+GLIBC_2.29 gethostent_r F
+GLIBC_2.29 gethostid F
+GLIBC_2.29 gethostname F
+GLIBC_2.29 getifaddrs F
+GLIBC_2.29 getipv4sourcefilter F
+GLIBC_2.29 getitimer F
+GLIBC_2.29 getline F
+GLIBC_2.29 getloadavg F
+GLIBC_2.29 getlogin F
+GLIBC_2.29 getlogin_r F
+GLIBC_2.29 getmntent F
+GLIBC_2.29 getmntent_r F
+GLIBC_2.29 getmsg F
+GLIBC_2.29 getnameinfo F
+GLIBC_2.29 getnetbyaddr F
+GLIBC_2.29 getnetbyaddr_r F
+GLIBC_2.29 getnetbyname F
+GLIBC_2.29 getnetbyname_r F
+GLIBC_2.29 getnetent F
+GLIBC_2.29 getnetent_r F
+GLIBC_2.29 getnetgrent F
+GLIBC_2.29 getnetgrent_r F
+GLIBC_2.29 getnetname F
+GLIBC_2.29 getopt F
+GLIBC_2.29 getopt_long F
+GLIBC_2.29 getopt_long_only F
+GLIBC_2.29 getpagesize F
+GLIBC_2.29 getpass F
+GLIBC_2.29 getpeername F
+GLIBC_2.29 getpgid F
+GLIBC_2.29 getpgrp F
+GLIBC_2.29 getpid F
+GLIBC_2.29 getpmsg F
+GLIBC_2.29 getppid F
+GLIBC_2.29 getpriority F
+GLIBC_2.29 getprotobyname F
+GLIBC_2.29 getprotobyname_r F
+GLIBC_2.29 getprotobynumber F
+GLIBC_2.29 getprotobynumber_r F
+GLIBC_2.29 getprotoent F
+GLIBC_2.29 getprotoent_r F
+GLIBC_2.29 getpt F
+GLIBC_2.29 getpublickey F
+GLIBC_2.29 getpw F
+GLIBC_2.29 getpwent F
+GLIBC_2.29 getpwent_r F
+GLIBC_2.29 getpwnam F
+GLIBC_2.29 getpwnam_r F
+GLIBC_2.29 getpwuid F
+GLIBC_2.29 getpwuid_r F
+GLIBC_2.29 getrandom F
+GLIBC_2.29 getresgid F
+GLIBC_2.29 getresuid F
+GLIBC_2.29 getrlimit F
+GLIBC_2.29 getrlimit64 F
+GLIBC_2.29 getrpcbyname F
+GLIBC_2.29 getrpcbyname_r F
+GLIBC_2.29 getrpcbynumber F
+GLIBC_2.29 getrpcbynumber_r F
+GLIBC_2.29 getrpcent F
+GLIBC_2.29 getrpcent_r F
+GLIBC_2.29 getrpcport F
+GLIBC_2.29 getrusage F
+GLIBC_2.29 gets F
+GLIBC_2.29 getsecretkey F
+GLIBC_2.29 getservbyname F
+GLIBC_2.29 getservbyname_r F
+GLIBC_2.29 getservbyport F
+GLIBC_2.29 getservbyport_r F
+GLIBC_2.29 getservent F
+GLIBC_2.29 getservent_r F
+GLIBC_2.29 getsgent F
+GLIBC_2.29 getsgent_r F
+GLIBC_2.29 getsgnam F
+GLIBC_2.29 getsgnam_r F
+GLIBC_2.29 getsid F
+GLIBC_2.29 getsockname F
+GLIBC_2.29 getsockopt F
+GLIBC_2.29 getsourcefilter F
+GLIBC_2.29 getspent F
+GLIBC_2.29 getspent_r F
+GLIBC_2.29 getspnam F
+GLIBC_2.29 getspnam_r F
+GLIBC_2.29 getsubopt F
+GLIBC_2.29 gettext F
+GLIBC_2.29 gettimeofday F
+GLIBC_2.29 getttyent F
+GLIBC_2.29 getttynam F
+GLIBC_2.29 getuid F
+GLIBC_2.29 getusershell F
+GLIBC_2.29 getutent F
+GLIBC_2.29 getutent_r F
+GLIBC_2.29 getutid F
+GLIBC_2.29 getutid_r F
+GLIBC_2.29 getutline F
+GLIBC_2.29 getutline_r F
+GLIBC_2.29 getutmp F
+GLIBC_2.29 getutmpx F
+GLIBC_2.29 getutxent F
+GLIBC_2.29 getutxid F
+GLIBC_2.29 getutxline F
+GLIBC_2.29 getw F
+GLIBC_2.29 getwc F
+GLIBC_2.29 getwc_unlocked F
+GLIBC_2.29 getwchar F
+GLIBC_2.29 getwchar_unlocked F
+GLIBC_2.29 getwd F
+GLIBC_2.29 getxattr F
+GLIBC_2.29 glob F
+GLIBC_2.29 glob64 F
+GLIBC_2.29 glob_pattern_p F
+GLIBC_2.29 globfree F
+GLIBC_2.29 globfree64 F
+GLIBC_2.29 gmtime F
+GLIBC_2.29 gmtime_r F
+GLIBC_2.29 gnu_dev_major F
+GLIBC_2.29 gnu_dev_makedev F
+GLIBC_2.29 gnu_dev_minor F
+GLIBC_2.29 gnu_get_libc_release F
+GLIBC_2.29 gnu_get_libc_version F
+GLIBC_2.29 grantpt F
+GLIBC_2.29 group_member F
+GLIBC_2.29 gsignal F
+GLIBC_2.29 gtty F
+GLIBC_2.29 h_errlist D 0x14
+GLIBC_2.29 h_nerr D 0x4
+GLIBC_2.29 hasmntopt F
+GLIBC_2.29 hcreate F
+GLIBC_2.29 hcreate_r F
+GLIBC_2.29 hdestroy F
+GLIBC_2.29 hdestroy_r F
+GLIBC_2.29 herror F
+GLIBC_2.29 host2netname F
+GLIBC_2.29 hsearch F
+GLIBC_2.29 hsearch_r F
+GLIBC_2.29 hstrerror F
+GLIBC_2.29 htonl F
+GLIBC_2.29 htons F
+GLIBC_2.29 iconv F
+GLIBC_2.29 iconv_close F
+GLIBC_2.29 iconv_open F
+GLIBC_2.29 if_freenameindex F
+GLIBC_2.29 if_indextoname F
+GLIBC_2.29 if_nameindex F
+GLIBC_2.29 if_nametoindex F
+GLIBC_2.29 imaxabs F
+GLIBC_2.29 imaxdiv F
+GLIBC_2.29 in6addr_any D 0x10
+GLIBC_2.29 in6addr_loopback D 0x10
+GLIBC_2.29 index F
+GLIBC_2.29 inet6_opt_append F
+GLIBC_2.29 inet6_opt_find F
+GLIBC_2.29 inet6_opt_finish F
+GLIBC_2.29 inet6_opt_get_val F
+GLIBC_2.29 inet6_opt_init F
+GLIBC_2.29 inet6_opt_next F
+GLIBC_2.29 inet6_opt_set_val F
+GLIBC_2.29 inet6_option_alloc F
+GLIBC_2.29 inet6_option_append F
+GLIBC_2.29 inet6_option_find F
+GLIBC_2.29 inet6_option_init F
+GLIBC_2.29 inet6_option_next F
+GLIBC_2.29 inet6_option_space F
+GLIBC_2.29 inet6_rth_add F
+GLIBC_2.29 inet6_rth_getaddr F
+GLIBC_2.29 inet6_rth_init F
+GLIBC_2.29 inet6_rth_reverse F
+GLIBC_2.29 inet6_rth_segments F
+GLIBC_2.29 inet6_rth_space F
+GLIBC_2.29 inet_addr F
+GLIBC_2.29 inet_aton F
+GLIBC_2.29 inet_lnaof F
+GLIBC_2.29 inet_makeaddr F
+GLIBC_2.29 inet_netof F
+GLIBC_2.29 inet_network F
+GLIBC_2.29 inet_nsap_addr F
+GLIBC_2.29 inet_nsap_ntoa F
+GLIBC_2.29 inet_ntoa F
+GLIBC_2.29 inet_ntop F
+GLIBC_2.29 inet_pton F
+GLIBC_2.29 init_module F
+GLIBC_2.29 initgroups F
+GLIBC_2.29 initstate F
+GLIBC_2.29 initstate_r F
+GLIBC_2.29 innetgr F
+GLIBC_2.29 inotify_add_watch F
+GLIBC_2.29 inotify_init F
+GLIBC_2.29 inotify_init1 F
+GLIBC_2.29 inotify_rm_watch F
+GLIBC_2.29 insque F
+GLIBC_2.29 ioctl F
+GLIBC_2.29 iruserok F
+GLIBC_2.29 iruserok_af F
+GLIBC_2.29 isalnum F
+GLIBC_2.29 isalnum_l F
+GLIBC_2.29 isalpha F
+GLIBC_2.29 isalpha_l F
+GLIBC_2.29 isascii F
+GLIBC_2.29 isastream F
+GLIBC_2.29 isatty F
+GLIBC_2.29 isblank F
+GLIBC_2.29 isblank_l F
+GLIBC_2.29 iscntrl F
+GLIBC_2.29 iscntrl_l F
+GLIBC_2.29 isctype F
+GLIBC_2.29 isdigit F
+GLIBC_2.29 isdigit_l F
+GLIBC_2.29 isfdtype F
+GLIBC_2.29 isgraph F
+GLIBC_2.29 isgraph_l F
+GLIBC_2.29 isinf F
+GLIBC_2.29 isinff F
+GLIBC_2.29 isinfl F
+GLIBC_2.29 islower F
+GLIBC_2.29 islower_l F
+GLIBC_2.29 isnan F
+GLIBC_2.29 isnanf F
+GLIBC_2.29 isnanl F
+GLIBC_2.29 isprint F
+GLIBC_2.29 isprint_l F
+GLIBC_2.29 ispunct F
+GLIBC_2.29 ispunct_l F
+GLIBC_2.29 isspace F
+GLIBC_2.29 isspace_l F
+GLIBC_2.29 isupper F
+GLIBC_2.29 isupper_l F
+GLIBC_2.29 iswalnum F
+GLIBC_2.29 iswalnum_l F
+GLIBC_2.29 iswalpha F
+GLIBC_2.29 iswalpha_l F
+GLIBC_2.29 iswblank F
+GLIBC_2.29 iswblank_l F
+GLIBC_2.29 iswcntrl F
+GLIBC_2.29 iswcntrl_l F
+GLIBC_2.29 iswctype F
+GLIBC_2.29 iswctype_l F
+GLIBC_2.29 iswdigit F
+GLIBC_2.29 iswdigit_l F
+GLIBC_2.29 iswgraph F
+GLIBC_2.29 iswgraph_l F
+GLIBC_2.29 iswlower F
+GLIBC_2.29 iswlower_l F
+GLIBC_2.29 iswprint F
+GLIBC_2.29 iswprint_l F
+GLIBC_2.29 iswpunct F
+GLIBC_2.29 iswpunct_l F
+GLIBC_2.29 iswspace F
+GLIBC_2.29 iswspace_l F
+GLIBC_2.29 iswupper F
+GLIBC_2.29 iswupper_l F
+GLIBC_2.29 iswxdigit F
+GLIBC_2.29 iswxdigit_l F
+GLIBC_2.29 isxdigit F
+GLIBC_2.29 isxdigit_l F
+GLIBC_2.29 jrand48 F
+GLIBC_2.29 jrand48_r F
+GLIBC_2.29 key_decryptsession F
+GLIBC_2.29 key_decryptsession_pk F
+GLIBC_2.29 key_encryptsession F
+GLIBC_2.29 key_encryptsession_pk F
+GLIBC_2.29 key_gendes F
+GLIBC_2.29 key_get_conv F
+GLIBC_2.29 key_secretkey_is_set F
+GLIBC_2.29 key_setnet F
+GLIBC_2.29 key_setsecret F
+GLIBC_2.29 kill F
+GLIBC_2.29 killpg F
+GLIBC_2.29 klogctl F
+GLIBC_2.29 l64a F
+GLIBC_2.29 labs F
+GLIBC_2.29 lchmod F
+GLIBC_2.29 lchown F
+GLIBC_2.29 lckpwdf F
+GLIBC_2.29 lcong48 F
+GLIBC_2.29 lcong48_r F
+GLIBC_2.29 ldexp F
+GLIBC_2.29 ldexpf F
+GLIBC_2.29 ldexpl F
+GLIBC_2.29 ldiv F
+GLIBC_2.29 lfind F
+GLIBC_2.29 lgetxattr F
+GLIBC_2.29 link F
+GLIBC_2.29 linkat F
+GLIBC_2.29 listen F
+GLIBC_2.29 listxattr F
+GLIBC_2.29 llabs F
+GLIBC_2.29 lldiv F
+GLIBC_2.29 llistxattr F
+GLIBC_2.29 localeconv F
+GLIBC_2.29 localtime F
+GLIBC_2.29 localtime_r F
+GLIBC_2.29 lockf F
+GLIBC_2.29 lockf64 F
+GLIBC_2.29 longjmp F
+GLIBC_2.29 lrand48 F
+GLIBC_2.29 lrand48_r F
+GLIBC_2.29 lremovexattr F
+GLIBC_2.29 lsearch F
+GLIBC_2.29 lseek F
+GLIBC_2.29 lseek64 F
+GLIBC_2.29 lsetxattr F
+GLIBC_2.29 lutimes F
+GLIBC_2.29 madvise F
+GLIBC_2.29 makecontext F
+GLIBC_2.29 mallinfo F
+GLIBC_2.29 malloc F
+GLIBC_2.29 malloc_info F
+GLIBC_2.29 malloc_stats F
+GLIBC_2.29 malloc_trim F
+GLIBC_2.29 malloc_usable_size F
+GLIBC_2.29 mallopt F
+GLIBC_2.29 mallwatch D 0x4
+GLIBC_2.29 mblen F
+GLIBC_2.29 mbrlen F
+GLIBC_2.29 mbrtoc16 F
+GLIBC_2.29 mbrtoc32 F
+GLIBC_2.29 mbrtowc F
+GLIBC_2.29 mbsinit F
+GLIBC_2.29 mbsnrtowcs F
+GLIBC_2.29 mbsrtowcs F
+GLIBC_2.29 mbstowcs F
+GLIBC_2.29 mbtowc F
+GLIBC_2.29 mcheck F
+GLIBC_2.29 mcheck_check_all F
+GLIBC_2.29 mcheck_pedantic F
+GLIBC_2.29 memalign F
+GLIBC_2.29 memccpy F
+GLIBC_2.29 memchr F
+GLIBC_2.29 memcmp F
+GLIBC_2.29 memcpy F
+GLIBC_2.29 memfd_create F
+GLIBC_2.29 memfrob F
+GLIBC_2.29 memmem F
+GLIBC_2.29 memmove F
+GLIBC_2.29 mempcpy F
+GLIBC_2.29 memrchr F
+GLIBC_2.29 memset F
+GLIBC_2.29 mincore F
+GLIBC_2.29 mkdir F
+GLIBC_2.29 mkdirat F
+GLIBC_2.29 mkdtemp F
+GLIBC_2.29 mkfifo F
+GLIBC_2.29 mkfifoat F
+GLIBC_2.29 mkostemp F
+GLIBC_2.29 mkostemp64 F
+GLIBC_2.29 mkostemps F
+GLIBC_2.29 mkostemps64 F
+GLIBC_2.29 mkstemp F
+GLIBC_2.29 mkstemp64 F
+GLIBC_2.29 mkstemps F
+GLIBC_2.29 mkstemps64 F
+GLIBC_2.29 mktemp F
+GLIBC_2.29 mktime F
+GLIBC_2.29 mlock F
+GLIBC_2.29 mlock2 F
+GLIBC_2.29 mlockall F
+GLIBC_2.29 mmap F
+GLIBC_2.29 mmap64 F
+GLIBC_2.29 modf F
+GLIBC_2.29 modff F
+GLIBC_2.29 modfl F
+GLIBC_2.29 moncontrol F
+GLIBC_2.29 monstartup F
+GLIBC_2.29 mount F
+GLIBC_2.29 mprobe F
+GLIBC_2.29 mprotect F
+GLIBC_2.29 mrand48 F
+GLIBC_2.29 mrand48_r F
+GLIBC_2.29 mremap F
+GLIBC_2.29 msgctl F
+GLIBC_2.29 msgget F
+GLIBC_2.29 msgrcv F
+GLIBC_2.29 msgsnd F
+GLIBC_2.29 msync F
+GLIBC_2.29 mtrace F
+GLIBC_2.29 munlock F
+GLIBC_2.29 munlockall F
+GLIBC_2.29 munmap F
+GLIBC_2.29 muntrace F
+GLIBC_2.29 name_to_handle_at F
+GLIBC_2.29 nanosleep F
+GLIBC_2.29 netname2host F
+GLIBC_2.29 netname2user F
+GLIBC_2.29 newlocale F
+GLIBC_2.29 nftw F
+GLIBC_2.29 nftw64 F
+GLIBC_2.29 ngettext F
+GLIBC_2.29 nice F
+GLIBC_2.29 nl_langinfo F
+GLIBC_2.29 nl_langinfo_l F
+GLIBC_2.29 nrand48 F
+GLIBC_2.29 nrand48_r F
+GLIBC_2.29 ntohl F
+GLIBC_2.29 ntohs F
+GLIBC_2.29 ntp_adjtime F
+GLIBC_2.29 ntp_gettime F
+GLIBC_2.29 ntp_gettimex F
+GLIBC_2.29 obstack_alloc_failed_handler D 0x4
+GLIBC_2.29 obstack_exit_failure D 0x4
+GLIBC_2.29 obstack_free F
+GLIBC_2.29 obstack_printf F
+GLIBC_2.29 obstack_vprintf F
+GLIBC_2.29 on_exit F
+GLIBC_2.29 open F
+GLIBC_2.29 open64 F
+GLIBC_2.29 open_by_handle_at F
+GLIBC_2.29 open_memstream F
+GLIBC_2.29 open_wmemstream F
+GLIBC_2.29 openat F
+GLIBC_2.29 openat64 F
+GLIBC_2.29 opendir F
+GLIBC_2.29 openlog F
+GLIBC_2.29 optarg D 0x4
+GLIBC_2.29 opterr D 0x4
+GLIBC_2.29 optind D 0x4
+GLIBC_2.29 optopt D 0x4
+GLIBC_2.29 parse_printf_format F
+GLIBC_2.29 passwd2des F
+GLIBC_2.29 pathconf F
+GLIBC_2.29 pause F
+GLIBC_2.29 pclose F
+GLIBC_2.29 perror F
+GLIBC_2.29 personality F
+GLIBC_2.29 pipe F
+GLIBC_2.29 pipe2 F
+GLIBC_2.29 pivot_root F
+GLIBC_2.29 pkey_alloc F
+GLIBC_2.29 pkey_free F
+GLIBC_2.29 pkey_get F
+GLIBC_2.29 pkey_mprotect F
+GLIBC_2.29 pkey_set F
+GLIBC_2.29 pmap_getmaps F
+GLIBC_2.29 pmap_getport F
+GLIBC_2.29 pmap_rmtcall F
+GLIBC_2.29 pmap_set F
+GLIBC_2.29 pmap_unset F
+GLIBC_2.29 poll F
+GLIBC_2.29 popen F
+GLIBC_2.29 posix_fadvise F
+GLIBC_2.29 posix_fadvise64 F
+GLIBC_2.29 posix_fallocate F
+GLIBC_2.29 posix_fallocate64 F
+GLIBC_2.29 posix_madvise F
+GLIBC_2.29 posix_memalign F
+GLIBC_2.29 posix_openpt F
+GLIBC_2.29 posix_spawn F
+GLIBC_2.29 posix_spawn_file_actions_addchdir_np F
+GLIBC_2.29 posix_spawn_file_actions_addclose F
+GLIBC_2.29 posix_spawn_file_actions_adddup2 F
+GLIBC_2.29 posix_spawn_file_actions_addfchdir_np F
+GLIBC_2.29 posix_spawn_file_actions_addopen F
+GLIBC_2.29 posix_spawn_file_actions_destroy F
+GLIBC_2.29 posix_spawn_file_actions_init F
+GLIBC_2.29 posix_spawnattr_destroy F
+GLIBC_2.29 posix_spawnattr_getflags F
+GLIBC_2.29 posix_spawnattr_getpgroup F
+GLIBC_2.29 posix_spawnattr_getschedparam F
+GLIBC_2.29 posix_spawnattr_getschedpolicy F
+GLIBC_2.29 posix_spawnattr_getsigdefault F
+GLIBC_2.29 posix_spawnattr_getsigmask F
+GLIBC_2.29 posix_spawnattr_init F
+GLIBC_2.29 posix_spawnattr_setflags F
+GLIBC_2.29 posix_spawnattr_setpgroup F
+GLIBC_2.29 posix_spawnattr_setschedparam F
+GLIBC_2.29 posix_spawnattr_setschedpolicy F
+GLIBC_2.29 posix_spawnattr_setsigdefault F
+GLIBC_2.29 posix_spawnattr_setsigmask F
+GLIBC_2.29 posix_spawnp F
+GLIBC_2.29 ppoll F
+GLIBC_2.29 prctl F
+GLIBC_2.29 pread F
+GLIBC_2.29 pread64 F
+GLIBC_2.29 preadv F
+GLIBC_2.29 preadv2 F
+GLIBC_2.29 preadv64 F
+GLIBC_2.29 preadv64v2 F
+GLIBC_2.29 printf F
+GLIBC_2.29 printf_size F
+GLIBC_2.29 printf_size_info F
+GLIBC_2.29 prlimit F
+GLIBC_2.29 prlimit64 F
+GLIBC_2.29 process_vm_readv F
+GLIBC_2.29 process_vm_writev F
+GLIBC_2.29 profil F
+GLIBC_2.29 program_invocation_name D 0x4
+GLIBC_2.29 program_invocation_short_name D 0x4
+GLIBC_2.29 pselect F
+GLIBC_2.29 psiginfo F
+GLIBC_2.29 psignal F
+GLIBC_2.29 pthread_attr_destroy F
+GLIBC_2.29 pthread_attr_getdetachstate F
+GLIBC_2.29 pthread_attr_getinheritsched F
+GLIBC_2.29 pthread_attr_getschedparam F
+GLIBC_2.29 pthread_attr_getschedpolicy F
+GLIBC_2.29 pthread_attr_getscope F
+GLIBC_2.29 pthread_attr_init F
+GLIBC_2.29 pthread_attr_setdetachstate F
+GLIBC_2.29 pthread_attr_setinheritsched F
+GLIBC_2.29 pthread_attr_setschedparam F
+GLIBC_2.29 pthread_attr_setschedpolicy F
+GLIBC_2.29 pthread_attr_setscope F
+GLIBC_2.29 pthread_cond_broadcast F
+GLIBC_2.29 pthread_cond_destroy F
+GLIBC_2.29 pthread_cond_init F
+GLIBC_2.29 pthread_cond_signal F
+GLIBC_2.29 pthread_cond_timedwait F
+GLIBC_2.29 pthread_cond_wait F
+GLIBC_2.29 pthread_condattr_destroy F
+GLIBC_2.29 pthread_condattr_init F
+GLIBC_2.29 pthread_equal F
+GLIBC_2.29 pthread_exit F
+GLIBC_2.29 pthread_getschedparam F
+GLIBC_2.29 pthread_mutex_destroy F
+GLIBC_2.29 pthread_mutex_init F
+GLIBC_2.29 pthread_mutex_lock F
+GLIBC_2.29 pthread_mutex_unlock F
+GLIBC_2.29 pthread_self F
+GLIBC_2.29 pthread_setcancelstate F
+GLIBC_2.29 pthread_setcanceltype F
+GLIBC_2.29 pthread_setschedparam F
+GLIBC_2.29 ptrace F
+GLIBC_2.29 ptsname F
+GLIBC_2.29 ptsname_r F
+GLIBC_2.29 putc F
+GLIBC_2.29 putc_unlocked F
+GLIBC_2.29 putchar F
+GLIBC_2.29 putchar_unlocked F
+GLIBC_2.29 putenv F
+GLIBC_2.29 putgrent F
+GLIBC_2.29 putmsg F
+GLIBC_2.29 putpmsg F
+GLIBC_2.29 putpwent F
+GLIBC_2.29 puts F
+GLIBC_2.29 putsgent F
+GLIBC_2.29 putspent F
+GLIBC_2.29 pututline F
+GLIBC_2.29 pututxline F
+GLIBC_2.29 putw F
+GLIBC_2.29 putwc F
+GLIBC_2.29 putwc_unlocked F
+GLIBC_2.29 putwchar F
+GLIBC_2.29 putwchar_unlocked F
+GLIBC_2.29 pvalloc F
+GLIBC_2.29 pwrite F
+GLIBC_2.29 pwrite64 F
+GLIBC_2.29 pwritev F
+GLIBC_2.29 pwritev2 F
+GLIBC_2.29 pwritev64 F
+GLIBC_2.29 pwritev64v2 F
+GLIBC_2.29 qecvt F
+GLIBC_2.29 qecvt_r F
+GLIBC_2.29 qfcvt F
+GLIBC_2.29 qfcvt_r F
+GLIBC_2.29 qgcvt F
+GLIBC_2.29 qsort F
+GLIBC_2.29 qsort_r F
+GLIBC_2.29 quick_exit F
+GLIBC_2.29 quotactl F
+GLIBC_2.29 raise F
+GLIBC_2.29 rand F
+GLIBC_2.29 rand_r F
+GLIBC_2.29 random F
+GLIBC_2.29 random_r F
+GLIBC_2.29 rawmemchr F
+GLIBC_2.29 rcmd F
+GLIBC_2.29 rcmd_af F
+GLIBC_2.29 re_comp F
+GLIBC_2.29 re_compile_fastmap F
+GLIBC_2.29 re_compile_pattern F
+GLIBC_2.29 re_exec F
+GLIBC_2.29 re_match F
+GLIBC_2.29 re_match_2 F
+GLIBC_2.29 re_search F
+GLIBC_2.29 re_search_2 F
+GLIBC_2.29 re_set_registers F
+GLIBC_2.29 re_set_syntax F
+GLIBC_2.29 re_syntax_options D 0x4
+GLIBC_2.29 read F
+GLIBC_2.29 readahead F
+GLIBC_2.29 readdir F
+GLIBC_2.29 readdir64 F
+GLIBC_2.29 readdir64_r F
+GLIBC_2.29 readdir_r F
+GLIBC_2.29 readlink F
+GLIBC_2.29 readlinkat F
+GLIBC_2.29 readv F
+GLIBC_2.29 realloc F
+GLIBC_2.29 reallocarray F
+GLIBC_2.29 realpath F
+GLIBC_2.29 reboot F
+GLIBC_2.29 recv F
+GLIBC_2.29 recvfrom F
+GLIBC_2.29 recvmmsg F
+GLIBC_2.29 recvmsg F
+GLIBC_2.29 regcomp F
+GLIBC_2.29 regerror F
+GLIBC_2.29 regexec F
+GLIBC_2.29 regfree F
+GLIBC_2.29 register_printf_function F
+GLIBC_2.29 register_printf_modifier F
+GLIBC_2.29 register_printf_specifier F
+GLIBC_2.29 register_printf_type F
+GLIBC_2.29 registerrpc F
+GLIBC_2.29 remap_file_pages F
+GLIBC_2.29 remove F
+GLIBC_2.29 removexattr F
+GLIBC_2.29 remque F
+GLIBC_2.29 rename F
+GLIBC_2.29 renameat F
+GLIBC_2.29 renameat2 F
+GLIBC_2.29 revoke F
+GLIBC_2.29 rewind F
+GLIBC_2.29 rewinddir F
+GLIBC_2.29 rexec F
+GLIBC_2.29 rexec_af F
+GLIBC_2.29 rexecoptions D 0x4
+GLIBC_2.29 rindex F
+GLIBC_2.29 rmdir F
+GLIBC_2.29 rpc_createerr D 0x10
+GLIBC_2.29 rpmatch F
+GLIBC_2.29 rresvport F
+GLIBC_2.29 rresvport_af F
+GLIBC_2.29 rtime F
+GLIBC_2.29 ruserok F
+GLIBC_2.29 ruserok_af F
+GLIBC_2.29 ruserpass F
+GLIBC_2.29 sbrk F
+GLIBC_2.29 scalbn F
+GLIBC_2.29 scalbnf F
+GLIBC_2.29 scalbnl F
+GLIBC_2.29 scandir F
+GLIBC_2.29 scandir64 F
+GLIBC_2.29 scandirat F
+GLIBC_2.29 scandirat64 F
+GLIBC_2.29 scanf F
+GLIBC_2.29 sched_get_priority_max F
+GLIBC_2.29 sched_get_priority_min F
+GLIBC_2.29 sched_getaffinity F
+GLIBC_2.29 sched_getcpu F
+GLIBC_2.29 sched_getparam F
+GLIBC_2.29 sched_getscheduler F
+GLIBC_2.29 sched_rr_get_interval F
+GLIBC_2.29 sched_setaffinity F
+GLIBC_2.29 sched_setparam F
+GLIBC_2.29 sched_setscheduler F
+GLIBC_2.29 sched_yield F
+GLIBC_2.29 secure_getenv F
+GLIBC_2.29 seed48 F
+GLIBC_2.29 seed48_r F
+GLIBC_2.29 seekdir F
+GLIBC_2.29 select F
+GLIBC_2.29 semctl F
+GLIBC_2.29 semget F
+GLIBC_2.29 semop F
+GLIBC_2.29 semtimedop F
+GLIBC_2.29 send F
+GLIBC_2.29 sendfile F
+GLIBC_2.29 sendfile64 F
+GLIBC_2.29 sendmmsg F
+GLIBC_2.29 sendmsg F
+GLIBC_2.29 sendto F
+GLIBC_2.29 setaliasent F
+GLIBC_2.29 setbuf F
+GLIBC_2.29 setbuffer F
+GLIBC_2.29 setcontext F
+GLIBC_2.29 setdomainname F
+GLIBC_2.29 setegid F
+GLIBC_2.29 setenv F
+GLIBC_2.29 seteuid F
+GLIBC_2.29 setfsent F
+GLIBC_2.29 setfsgid F
+GLIBC_2.29 setfsuid F
+GLIBC_2.29 setgid F
+GLIBC_2.29 setgrent F
+GLIBC_2.29 setgroups F
+GLIBC_2.29 sethostent F
+GLIBC_2.29 sethostid F
+GLIBC_2.29 sethostname F
+GLIBC_2.29 setipv4sourcefilter F
+GLIBC_2.29 setitimer F
+GLIBC_2.29 setjmp F
+GLIBC_2.29 setlinebuf F
+GLIBC_2.29 setlocale F
+GLIBC_2.29 setlogin F
+GLIBC_2.29 setlogmask F
+GLIBC_2.29 setmntent F
+GLIBC_2.29 setnetent F
+GLIBC_2.29 setnetgrent F
+GLIBC_2.29 setns F
+GLIBC_2.29 setpgid F
+GLIBC_2.29 setpgrp F
+GLIBC_2.29 setpriority F
+GLIBC_2.29 setprotoent F
+GLIBC_2.29 setpwent F
+GLIBC_2.29 setregid F
+GLIBC_2.29 setresgid F
+GLIBC_2.29 setresuid F
+GLIBC_2.29 setreuid F
+GLIBC_2.29 setrlimit F
+GLIBC_2.29 setrlimit64 F
+GLIBC_2.29 setrpcent F
+GLIBC_2.29 setservent F
+GLIBC_2.29 setsgent F
+GLIBC_2.29 setsid F
+GLIBC_2.29 setsockopt F
+GLIBC_2.29 setsourcefilter F
+GLIBC_2.29 setspent F
+GLIBC_2.29 setstate F
+GLIBC_2.29 setstate_r F
+GLIBC_2.29 settimeofday F
+GLIBC_2.29 setttyent F
+GLIBC_2.29 setuid F
+GLIBC_2.29 setusershell F
+GLIBC_2.29 setutent F
+GLIBC_2.29 setutxent F
+GLIBC_2.29 setvbuf F
+GLIBC_2.29 setxattr F
+GLIBC_2.29 sgetsgent F
+GLIBC_2.29 sgetsgent_r F
+GLIBC_2.29 sgetspent F
+GLIBC_2.29 sgetspent_r F
+GLIBC_2.29 shmat F
+GLIBC_2.29 shmctl F
+GLIBC_2.29 shmdt F
+GLIBC_2.29 shmget F
+GLIBC_2.29 shutdown F
+GLIBC_2.29 sigaction F
+GLIBC_2.29 sigaddset F
+GLIBC_2.29 sigaltstack F
+GLIBC_2.29 sigandset F
+GLIBC_2.29 sigblock F
+GLIBC_2.29 sigdelset F
+GLIBC_2.29 sigemptyset F
+GLIBC_2.29 sigfillset F
+GLIBC_2.29 siggetmask F
+GLIBC_2.29 sighold F
+GLIBC_2.29 sigignore F
+GLIBC_2.29 siginterrupt F
+GLIBC_2.29 sigisemptyset F
+GLIBC_2.29 sigismember F
+GLIBC_2.29 siglongjmp F
+GLIBC_2.29 signal F
+GLIBC_2.29 signalfd F
+GLIBC_2.29 sigorset F
+GLIBC_2.29 sigpause F
+GLIBC_2.29 sigpending F
+GLIBC_2.29 sigprocmask F
+GLIBC_2.29 sigqueue F
+GLIBC_2.29 sigrelse F
+GLIBC_2.29 sigreturn F
+GLIBC_2.29 sigset F
+GLIBC_2.29 sigsetmask F
+GLIBC_2.29 sigstack F
+GLIBC_2.29 sigsuspend F
+GLIBC_2.29 sigtimedwait F
+GLIBC_2.29 sigwait F
+GLIBC_2.29 sigwaitinfo F
+GLIBC_2.29 sleep F
+GLIBC_2.29 snprintf F
+GLIBC_2.29 sockatmark F
+GLIBC_2.29 socket F
+GLIBC_2.29 socketpair F
+GLIBC_2.29 splice F
+GLIBC_2.29 sprintf F
+GLIBC_2.29 sprofil F
+GLIBC_2.29 srand F
+GLIBC_2.29 srand48 F
+GLIBC_2.29 srand48_r F
+GLIBC_2.29 srandom F
+GLIBC_2.29 srandom_r F
+GLIBC_2.29 sscanf F
+GLIBC_2.29 ssignal F
+GLIBC_2.29 sstk F
+GLIBC_2.29 statfs F
+GLIBC_2.29 statfs64 F
+GLIBC_2.29 statvfs F
+GLIBC_2.29 statvfs64 F
+GLIBC_2.29 statx F
+GLIBC_2.29 stderr D 0x4
+GLIBC_2.29 stdin D 0x4
+GLIBC_2.29 stdout D 0x4
+GLIBC_2.29 stime F
+GLIBC_2.29 stpcpy F
+GLIBC_2.29 stpncpy F
+GLIBC_2.29 strcasecmp F
+GLIBC_2.29 strcasecmp_l F
+GLIBC_2.29 strcasestr F
+GLIBC_2.29 strcat F
+GLIBC_2.29 strchr F
+GLIBC_2.29 strchrnul F
+GLIBC_2.29 strcmp F
+GLIBC_2.29 strcoll F
+GLIBC_2.29 strcoll_l F
+GLIBC_2.29 strcpy F
+GLIBC_2.29 strcspn F
+GLIBC_2.29 strdup F
+GLIBC_2.29 strerror F
+GLIBC_2.29 strerror_l F
+GLIBC_2.29 strerror_r F
+GLIBC_2.29 strfmon F
+GLIBC_2.29 strfmon_l F
+GLIBC_2.29 strfromd F
+GLIBC_2.29 strfromf F
+GLIBC_2.29 strfromf128 F
+GLIBC_2.29 strfromf32 F
+GLIBC_2.29 strfromf32x F
+GLIBC_2.29 strfromf64 F
+GLIBC_2.29 strfromf64x F
+GLIBC_2.29 strfroml F
+GLIBC_2.29 strfry F
+GLIBC_2.29 strftime F
+GLIBC_2.29 strftime_l F
+GLIBC_2.29 strlen F
+GLIBC_2.29 strncasecmp F
+GLIBC_2.29 strncasecmp_l F
+GLIBC_2.29 strncat F
+GLIBC_2.29 strncmp F
+GLIBC_2.29 strncpy F
+GLIBC_2.29 strndup F
+GLIBC_2.29 strnlen F
+GLIBC_2.29 strpbrk F
+GLIBC_2.29 strptime F
+GLIBC_2.29 strptime_l F
+GLIBC_2.29 strrchr F
+GLIBC_2.29 strsep F
+GLIBC_2.29 strsignal F
+GLIBC_2.29 strspn F
+GLIBC_2.29 strstr F
+GLIBC_2.29 strtod F
+GLIBC_2.29 strtod_l F
+GLIBC_2.29 strtof F
+GLIBC_2.29 strtof128 F
+GLIBC_2.29 strtof128_l F
+GLIBC_2.29 strtof32 F
+GLIBC_2.29 strtof32_l F
+GLIBC_2.29 strtof32x F
+GLIBC_2.29 strtof32x_l F
+GLIBC_2.29 strtof64 F
+GLIBC_2.29 strtof64_l F
+GLIBC_2.29 strtof64x F
+GLIBC_2.29 strtof64x_l F
+GLIBC_2.29 strtof_l F
+GLIBC_2.29 strtoimax F
+GLIBC_2.29 strtok F
+GLIBC_2.29 strtok_r F
+GLIBC_2.29 strtol F
+GLIBC_2.29 strtol_l F
+GLIBC_2.29 strtold F
+GLIBC_2.29 strtold_l F
+GLIBC_2.29 strtoll F
+GLIBC_2.29 strtoll_l F
+GLIBC_2.29 strtoq F
+GLIBC_2.29 strtoul F
+GLIBC_2.29 strtoul_l F
+GLIBC_2.29 strtoull F
+GLIBC_2.29 strtoull_l F
+GLIBC_2.29 strtoumax F
+GLIBC_2.29 strtouq F
+GLIBC_2.29 strverscmp F
+GLIBC_2.29 strxfrm F
+GLIBC_2.29 strxfrm_l F
+GLIBC_2.29 stty F
+GLIBC_2.29 svc_exit F
+GLIBC_2.29 svc_fdset D 0x80
+GLIBC_2.29 svc_getreq F
+GLIBC_2.29 svc_getreq_common F
+GLIBC_2.29 svc_getreq_poll F
+GLIBC_2.29 svc_getreqset F
+GLIBC_2.29 svc_max_pollfd D 0x4
+GLIBC_2.29 svc_pollfd D 0x4
+GLIBC_2.29 svc_register F
+GLIBC_2.29 svc_run F
+GLIBC_2.29 svc_sendreply F
+GLIBC_2.29 svc_unregister F
+GLIBC_2.29 svcauthdes_stats D 0xc
+GLIBC_2.29 svcerr_auth F
+GLIBC_2.29 svcerr_decode F
+GLIBC_2.29 svcerr_noproc F
+GLIBC_2.29 svcerr_noprog F
+GLIBC_2.29 svcerr_progvers F
+GLIBC_2.29 svcerr_systemerr F
+GLIBC_2.29 svcerr_weakauth F
+GLIBC_2.29 svcfd_create F
+GLIBC_2.29 svcraw_create F
+GLIBC_2.29 svctcp_create F
+GLIBC_2.29 svcudp_bufcreate F
+GLIBC_2.29 svcudp_create F
+GLIBC_2.29 svcudp_enablecache F
+GLIBC_2.29 svcunix_create F
+GLIBC_2.29 svcunixfd_create F
+GLIBC_2.29 swab F
+GLIBC_2.29 swapcontext F
+GLIBC_2.29 swapoff F
+GLIBC_2.29 swapon F
+GLIBC_2.29 swprintf F
+GLIBC_2.29 swscanf F
+GLIBC_2.29 symlink F
+GLIBC_2.29 symlinkat F
+GLIBC_2.29 sync F
+GLIBC_2.29 sync_file_range F
+GLIBC_2.29 syncfs F
+GLIBC_2.29 sys_errlist D 0x21c
+GLIBC_2.29 sys_nerr D 0x4
+GLIBC_2.29 sys_sigabbrev D 0x104
+GLIBC_2.29 sys_siglist D 0x104
+GLIBC_2.29 syscall F
+GLIBC_2.29 sysconf F
+GLIBC_2.29 sysctl F
+GLIBC_2.29 sysinfo F
+GLIBC_2.29 syslog F
+GLIBC_2.29 system F
+GLIBC_2.29 sysv_signal F
+GLIBC_2.29 tcdrain F
+GLIBC_2.29 tcflow F
+GLIBC_2.29 tcflush F
+GLIBC_2.29 tcgetattr F
+GLIBC_2.29 tcgetpgrp F
+GLIBC_2.29 tcgetsid F
+GLIBC_2.29 tcsendbreak F
+GLIBC_2.29 tcsetattr F
+GLIBC_2.29 tcsetpgrp F
+GLIBC_2.29 tdelete F
+GLIBC_2.29 tdestroy F
+GLIBC_2.29 tee F
+GLIBC_2.29 telldir F
+GLIBC_2.29 tempnam F
+GLIBC_2.29 textdomain F
+GLIBC_2.29 tfind F
+GLIBC_2.29 thrd_current F
+GLIBC_2.29 thrd_equal F
+GLIBC_2.29 thrd_sleep F
+GLIBC_2.29 thrd_yield F
+GLIBC_2.29 time F
+GLIBC_2.29 timegm F
+GLIBC_2.29 timelocal F
+GLIBC_2.29 timerfd_create F
+GLIBC_2.29 timerfd_gettime F
+GLIBC_2.29 timerfd_settime F
+GLIBC_2.29 times F
+GLIBC_2.29 timespec_get F
+GLIBC_2.29 timezone D 0x4
+GLIBC_2.29 tmpfile F
+GLIBC_2.29 tmpfile64 F
+GLIBC_2.29 tmpnam F
+GLIBC_2.29 tmpnam_r F
+GLIBC_2.29 toascii F
+GLIBC_2.29 tolower F
+GLIBC_2.29 tolower_l F
+GLIBC_2.29 toupper F
+GLIBC_2.29 toupper_l F
+GLIBC_2.29 towctrans F
+GLIBC_2.29 towctrans_l F
+GLIBC_2.29 towlower F
+GLIBC_2.29 towlower_l F
+GLIBC_2.29 towupper F
+GLIBC_2.29 towupper_l F
+GLIBC_2.29 tr_break F
+GLIBC_2.29 truncate F
+GLIBC_2.29 truncate64 F
+GLIBC_2.29 tsearch F
+GLIBC_2.29 ttyname F
+GLIBC_2.29 ttyname_r F
+GLIBC_2.29 ttyslot F
+GLIBC_2.29 twalk F
+GLIBC_2.29 tzname D 0x8
+GLIBC_2.29 tzset F
+GLIBC_2.29 ualarm F
+GLIBC_2.29 ulckpwdf F
+GLIBC_2.29 ulimit F
+GLIBC_2.29 umask F
+GLIBC_2.29 umount F
+GLIBC_2.29 umount2 F
+GLIBC_2.29 uname F
+GLIBC_2.29 ungetc F
+GLIBC_2.29 ungetwc F
+GLIBC_2.29 unlink F
+GLIBC_2.29 unlinkat F
+GLIBC_2.29 unlockpt F
+GLIBC_2.29 unsetenv F
+GLIBC_2.29 unshare F
+GLIBC_2.29 updwtmp F
+GLIBC_2.29 updwtmpx F
+GLIBC_2.29 uselocale F
+GLIBC_2.29 user2netname F
+GLIBC_2.29 usleep F
+GLIBC_2.29 utime F
+GLIBC_2.29 utimensat F
+GLIBC_2.29 utimes F
+GLIBC_2.29 utmpname F
+GLIBC_2.29 utmpxname F
+GLIBC_2.29 valloc F
+GLIBC_2.29 vasprintf F
+GLIBC_2.29 vdprintf F
+GLIBC_2.29 verr F
+GLIBC_2.29 verrx F
+GLIBC_2.29 versionsort F
+GLIBC_2.29 versionsort64 F
+GLIBC_2.29 vfork F
+GLIBC_2.29 vfprintf F
+GLIBC_2.29 vfscanf F
+GLIBC_2.29 vfwprintf F
+GLIBC_2.29 vfwscanf F
+GLIBC_2.29 vhangup F
+GLIBC_2.29 vlimit F
+GLIBC_2.29 vmsplice F
+GLIBC_2.29 vprintf F
+GLIBC_2.29 vscanf F
+GLIBC_2.29 vsnprintf F
+GLIBC_2.29 vsprintf F
+GLIBC_2.29 vsscanf F
+GLIBC_2.29 vswprintf F
+GLIBC_2.29 vswscanf F
+GLIBC_2.29 vsyslog F
+GLIBC_2.29 vtimes F
+GLIBC_2.29 vwarn F
+GLIBC_2.29 vwarnx F
+GLIBC_2.29 vwprintf F
+GLIBC_2.29 vwscanf F
+GLIBC_2.29 wait F
+GLIBC_2.29 wait3 F
+GLIBC_2.29 wait4 F
+GLIBC_2.29 waitid F
+GLIBC_2.29 waitpid F
+GLIBC_2.29 warn F
+GLIBC_2.29 warnx F
+GLIBC_2.29 wcpcpy F
+GLIBC_2.29 wcpncpy F
+GLIBC_2.29 wcrtomb F
+GLIBC_2.29 wcscasecmp F
+GLIBC_2.29 wcscasecmp_l F
+GLIBC_2.29 wcscat F
+GLIBC_2.29 wcschr F
+GLIBC_2.29 wcschrnul F
+GLIBC_2.29 wcscmp F
+GLIBC_2.29 wcscoll F
+GLIBC_2.29 wcscoll_l F
+GLIBC_2.29 wcscpy F
+GLIBC_2.29 wcscspn F
+GLIBC_2.29 wcsdup F
+GLIBC_2.29 wcsftime F
+GLIBC_2.29 wcsftime_l F
+GLIBC_2.29 wcslen F
+GLIBC_2.29 wcsncasecmp F
+GLIBC_2.29 wcsncasecmp_l F
+GLIBC_2.29 wcsncat F
+GLIBC_2.29 wcsncmp F
+GLIBC_2.29 wcsncpy F
+GLIBC_2.29 wcsnlen F
+GLIBC_2.29 wcsnrtombs F
+GLIBC_2.29 wcspbrk F
+GLIBC_2.29 wcsrchr F
+GLIBC_2.29 wcsrtombs F
+GLIBC_2.29 wcsspn F
+GLIBC_2.29 wcsstr F
+GLIBC_2.29 wcstod F
+GLIBC_2.29 wcstod_l F
+GLIBC_2.29 wcstof F
+GLIBC_2.29 wcstof128 F
+GLIBC_2.29 wcstof128_l F
+GLIBC_2.29 wcstof32 F
+GLIBC_2.29 wcstof32_l F
+GLIBC_2.29 wcstof32x F
+GLIBC_2.29 wcstof32x_l F
+GLIBC_2.29 wcstof64 F
+GLIBC_2.29 wcstof64_l F
+GLIBC_2.29 wcstof64x F
+GLIBC_2.29 wcstof64x_l F
+GLIBC_2.29 wcstof_l F
+GLIBC_2.29 wcstoimax F
+GLIBC_2.29 wcstok F
+GLIBC_2.29 wcstol F
+GLIBC_2.29 wcstol_l F
+GLIBC_2.29 wcstold F
+GLIBC_2.29 wcstold_l F
+GLIBC_2.29 wcstoll F
+GLIBC_2.29 wcstoll_l F
+GLIBC_2.29 wcstombs F
+GLIBC_2.29 wcstoq F
+GLIBC_2.29 wcstoul F
+GLIBC_2.29 wcstoul_l F
+GLIBC_2.29 wcstoull F
+GLIBC_2.29 wcstoull_l F
+GLIBC_2.29 wcstoumax F
+GLIBC_2.29 wcstouq F
+GLIBC_2.29 wcswcs F
+GLIBC_2.29 wcswidth F
+GLIBC_2.29 wcsxfrm F
+GLIBC_2.29 wcsxfrm_l F
+GLIBC_2.29 wctob F
+GLIBC_2.29 wctomb F
+GLIBC_2.29 wctrans F
+GLIBC_2.29 wctrans_l F
+GLIBC_2.29 wctype F
+GLIBC_2.29 wctype_l F
+GLIBC_2.29 wcwidth F
+GLIBC_2.29 wmemchr F
+GLIBC_2.29 wmemcmp F
+GLIBC_2.29 wmemcpy F
+GLIBC_2.29 wmemmove F
+GLIBC_2.29 wmempcpy F
+GLIBC_2.29 wmemset F
+GLIBC_2.29 wordexp F
+GLIBC_2.29 wordfree F
+GLIBC_2.29 wprintf F
+GLIBC_2.29 write F
+GLIBC_2.29 writev F
+GLIBC_2.29 wscanf F
+GLIBC_2.29 xdecrypt F
+GLIBC_2.29 xdr_accepted_reply F
+GLIBC_2.29 xdr_array F
+GLIBC_2.29 xdr_authdes_cred F
+GLIBC_2.29 xdr_authdes_verf F
+GLIBC_2.29 xdr_authunix_parms F
+GLIBC_2.29 xdr_bool F
+GLIBC_2.29 xdr_bytes F
+GLIBC_2.29 xdr_callhdr F
+GLIBC_2.29 xdr_callmsg F
+GLIBC_2.29 xdr_char F
+GLIBC_2.29 xdr_cryptkeyarg F
+GLIBC_2.29 xdr_cryptkeyarg2 F
+GLIBC_2.29 xdr_cryptkeyres F
+GLIBC_2.29 xdr_des_block F
+GLIBC_2.29 xdr_double F
+GLIBC_2.29 xdr_enum F
+GLIBC_2.29 xdr_float F
+GLIBC_2.29 xdr_free F
+GLIBC_2.29 xdr_getcredres F
+GLIBC_2.29 xdr_hyper F
+GLIBC_2.29 xdr_int F
+GLIBC_2.29 xdr_int16_t F
+GLIBC_2.29 xdr_int32_t F
+GLIBC_2.29 xdr_int64_t F
+GLIBC_2.29 xdr_int8_t F
+GLIBC_2.29 xdr_key_netstarg F
+GLIBC_2.29 xdr_key_netstres F
+GLIBC_2.29 xdr_keybuf F
+GLIBC_2.29 xdr_keystatus F
+GLIBC_2.29 xdr_long F
+GLIBC_2.29 xdr_longlong_t F
+GLIBC_2.29 xdr_netnamestr F
+GLIBC_2.29 xdr_netobj F
+GLIBC_2.29 xdr_opaque F
+GLIBC_2.29 xdr_opaque_auth F
+GLIBC_2.29 xdr_pmap F
+GLIBC_2.29 xdr_pmaplist F
+GLIBC_2.29 xdr_pointer F
+GLIBC_2.29 xdr_quad_t F
+GLIBC_2.29 xdr_reference F
+GLIBC_2.29 xdr_rejected_reply F
+GLIBC_2.29 xdr_replymsg F
+GLIBC_2.29 xdr_rmtcall_args F
+GLIBC_2.29 xdr_rmtcallres F
+GLIBC_2.29 xdr_short F
+GLIBC_2.29 xdr_sizeof F
+GLIBC_2.29 xdr_string F
+GLIBC_2.29 xdr_u_char F
+GLIBC_2.29 xdr_u_hyper F
+GLIBC_2.29 xdr_u_int F
+GLIBC_2.29 xdr_u_long F
+GLIBC_2.29 xdr_u_longlong_t F
+GLIBC_2.29 xdr_u_quad_t F
+GLIBC_2.29 xdr_u_short F
+GLIBC_2.29 xdr_uint16_t F
+GLIBC_2.29 xdr_uint32_t F
+GLIBC_2.29 xdr_uint64_t F
+GLIBC_2.29 xdr_uint8_t F
+GLIBC_2.29 xdr_union F
+GLIBC_2.29 xdr_unixcred F
+GLIBC_2.29 xdr_vector F
+GLIBC_2.29 xdr_void F
+GLIBC_2.29 xdr_wrapstring F
+GLIBC_2.29 xdrmem_create F
+GLIBC_2.29 xdrrec_create F
+GLIBC_2.29 xdrrec_endofrecord F
+GLIBC_2.29 xdrrec_eof F
+GLIBC_2.29 xdrrec_skiprecord F
+GLIBC_2.29 xdrstdio_create F
+GLIBC_2.29 xencrypt F
+GLIBC_2.29 xprt_register F
+GLIBC_2.29 xprt_unregister F
diff --git a/sysdeps/unix/sysv/linux/riscv/rv32/libcrypt.abilist b/sysdeps/unix/sysv/linux/riscv/rv32/libcrypt.abilist
new file mode 100644
index 0000000000..da3abbf8fb
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/riscv/rv32/libcrypt.abilist
@@ -0,0 +1,2 @@
+GLIBC_2.29 crypt F
+GLIBC_2.29 crypt_r F
diff --git a/sysdeps/unix/sysv/linux/riscv/rv32/libdl.abilist b/sysdeps/unix/sysv/linux/riscv/rv32/libdl.abilist
new file mode 100644
index 0000000000..8fcb609ddc
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/riscv/rv32/libdl.abilist
@@ -0,0 +1,9 @@
+GLIBC_2.29 dladdr F
+GLIBC_2.29 dladdr1 F
+GLIBC_2.29 dlclose F
+GLIBC_2.29 dlerror F
+GLIBC_2.29 dlinfo F
+GLIBC_2.29 dlmopen F
+GLIBC_2.29 dlopen F
+GLIBC_2.29 dlsym F
+GLIBC_2.29 dlvsym F
diff --git a/sysdeps/unix/sysv/linux/riscv/rv32/libm.abilist b/sysdeps/unix/sysv/linux/riscv/rv32/libm.abilist
new file mode 100644
index 0000000000..f7e19f44ab
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/riscv/rv32/libm.abilist
@@ -0,0 +1,1021 @@
+GLIBC_2.29 __acos_finite F
+GLIBC_2.29 __acosf_finite F
+GLIBC_2.29 __acosh_finite F
+GLIBC_2.29 __acoshf_finite F
+GLIBC_2.29 __acoshl_finite F
+GLIBC_2.29 __acosl_finite F
+GLIBC_2.29 __asin_finite F
+GLIBC_2.29 __asinf_finite F
+GLIBC_2.29 __asinl_finite F
+GLIBC_2.29 __atan2_finite F
+GLIBC_2.29 __atan2f_finite F
+GLIBC_2.29 __atan2l_finite F
+GLIBC_2.29 __atanh_finite F
+GLIBC_2.29 __atanhf_finite F
+GLIBC_2.29 __atanhl_finite F
+GLIBC_2.29 __clog10 F
+GLIBC_2.29 __clog10f F
+GLIBC_2.29 __clog10l F
+GLIBC_2.29 __cosh_finite F
+GLIBC_2.29 __coshf_finite F
+GLIBC_2.29 __coshl_finite F
+GLIBC_2.29 __exp10_finite F
+GLIBC_2.29 __exp10f_finite F
+GLIBC_2.29 __exp10l_finite F
+GLIBC_2.29 __exp2_finite F
+GLIBC_2.29 __exp2f_finite F
+GLIBC_2.29 __exp2l_finite F
+GLIBC_2.29 __exp_finite F
+GLIBC_2.29 __expf_finite F
+GLIBC_2.29 __expl_finite F
+GLIBC_2.29 __finite F
+GLIBC_2.29 __finitef F
+GLIBC_2.29 __finitel F
+GLIBC_2.29 __fmod_finite F
+GLIBC_2.29 __fmodf_finite F
+GLIBC_2.29 __fmodl_finite F
+GLIBC_2.29 __fpclassify F
+GLIBC_2.29 __fpclassifyf F
+GLIBC_2.29 __fpclassifyl F
+GLIBC_2.29 __gamma_r_finite F
+GLIBC_2.29 __gammaf_r_finite F
+GLIBC_2.29 __gammal_r_finite F
+GLIBC_2.29 __hypot_finite F
+GLIBC_2.29 __hypotf_finite F
+GLIBC_2.29 __hypotl_finite F
+GLIBC_2.29 __iseqsig F
+GLIBC_2.29 __iseqsigf F
+GLIBC_2.29 __iseqsigl F
+GLIBC_2.29 __issignaling F
+GLIBC_2.29 __issignalingf F
+GLIBC_2.29 __issignalingl F
+GLIBC_2.29 __j0_finite F
+GLIBC_2.29 __j0f_finite F
+GLIBC_2.29 __j0l_finite F
+GLIBC_2.29 __j1_finite F
+GLIBC_2.29 __j1f_finite F
+GLIBC_2.29 __j1l_finite F
+GLIBC_2.29 __jn_finite F
+GLIBC_2.29 __jnf_finite F
+GLIBC_2.29 __jnl_finite F
+GLIBC_2.29 __lgamma_r_finite F
+GLIBC_2.29 __lgammaf_r_finite F
+GLIBC_2.29 __lgammal_r_finite F
+GLIBC_2.29 __log10_finite F
+GLIBC_2.29 __log10f_finite F
+GLIBC_2.29 __log10l_finite F
+GLIBC_2.29 __log2_finite F
+GLIBC_2.29 __log2f_finite F
+GLIBC_2.29 __log2l_finite F
+GLIBC_2.29 __log_finite F
+GLIBC_2.29 __logf_finite F
+GLIBC_2.29 __logl_finite F
+GLIBC_2.29 __pow_finite F
+GLIBC_2.29 __powf_finite F
+GLIBC_2.29 __powl_finite F
+GLIBC_2.29 __remainder_finite F
+GLIBC_2.29 __remainderf_finite F
+GLIBC_2.29 __remainderl_finite F
+GLIBC_2.29 __scalb_finite F
+GLIBC_2.29 __scalbf_finite F
+GLIBC_2.29 __scalbl_finite F
+GLIBC_2.29 __signbit F
+GLIBC_2.29 __signbitf F
+GLIBC_2.29 __signbitl F
+GLIBC_2.29 __signgam D 0x4
+GLIBC_2.29 __sinh_finite F
+GLIBC_2.29 __sinhf_finite F
+GLIBC_2.29 __sinhl_finite F
+GLIBC_2.29 __sqrt_finite F
+GLIBC_2.29 __sqrtf_finite F
+GLIBC_2.29 __sqrtl_finite F
+GLIBC_2.29 __y0_finite F
+GLIBC_2.29 __y0f_finite F
+GLIBC_2.29 __y0l_finite F
+GLIBC_2.29 __y1_finite F
+GLIBC_2.29 __y1f_finite F
+GLIBC_2.29 __y1l_finite F
+GLIBC_2.29 __yn_finite F
+GLIBC_2.29 __ynf_finite F
+GLIBC_2.29 __ynl_finite F
+GLIBC_2.29 acos F
+GLIBC_2.29 acosf F
+GLIBC_2.29 acosf128 F
+GLIBC_2.29 acosf32 F
+GLIBC_2.29 acosf32x F
+GLIBC_2.29 acosf64 F
+GLIBC_2.29 acosf64x F
+GLIBC_2.29 acosh F
+GLIBC_2.29 acoshf F
+GLIBC_2.29 acoshf128 F
+GLIBC_2.29 acoshf32 F
+GLIBC_2.29 acoshf32x F
+GLIBC_2.29 acoshf64 F
+GLIBC_2.29 acoshf64x F
+GLIBC_2.29 acoshl F
+GLIBC_2.29 acosl F
+GLIBC_2.29 asin F
+GLIBC_2.29 asinf F
+GLIBC_2.29 asinf128 F
+GLIBC_2.29 asinf32 F
+GLIBC_2.29 asinf32x F
+GLIBC_2.29 asinf64 F
+GLIBC_2.29 asinf64x F
+GLIBC_2.29 asinh F
+GLIBC_2.29 asinhf F
+GLIBC_2.29 asinhf128 F
+GLIBC_2.29 asinhf32 F
+GLIBC_2.29 asinhf32x F
+GLIBC_2.29 asinhf64 F
+GLIBC_2.29 asinhf64x F
+GLIBC_2.29 asinhl F
+GLIBC_2.29 asinl F
+GLIBC_2.29 atan F
+GLIBC_2.29 atan2 F
+GLIBC_2.29 atan2f F
+GLIBC_2.29 atan2f128 F
+GLIBC_2.29 atan2f32 F
+GLIBC_2.29 atan2f32x F
+GLIBC_2.29 atan2f64 F
+GLIBC_2.29 atan2f64x F
+GLIBC_2.29 atan2l F
+GLIBC_2.29 atanf F
+GLIBC_2.29 atanf128 F
+GLIBC_2.29 atanf32 F
+GLIBC_2.29 atanf32x F
+GLIBC_2.29 atanf64 F
+GLIBC_2.29 atanf64x F
+GLIBC_2.29 atanh F
+GLIBC_2.29 atanhf F
+GLIBC_2.29 atanhf128 F
+GLIBC_2.29 atanhf32 F
+GLIBC_2.29 atanhf32x F
+GLIBC_2.29 atanhf64 F
+GLIBC_2.29 atanhf64x F
+GLIBC_2.29 atanhl F
+GLIBC_2.29 atanl F
+GLIBC_2.29 cabs F
+GLIBC_2.29 cabsf F
+GLIBC_2.29 cabsf128 F
+GLIBC_2.29 cabsf32 F
+GLIBC_2.29 cabsf32x F
+GLIBC_2.29 cabsf64 F
+GLIBC_2.29 cabsf64x F
+GLIBC_2.29 cabsl F
+GLIBC_2.29 cacos F
+GLIBC_2.29 cacosf F
+GLIBC_2.29 cacosf128 F
+GLIBC_2.29 cacosf32 F
+GLIBC_2.29 cacosf32x F
+GLIBC_2.29 cacosf64 F
+GLIBC_2.29 cacosf64x F
+GLIBC_2.29 cacosh F
+GLIBC_2.29 cacoshf F
+GLIBC_2.29 cacoshf128 F
+GLIBC_2.29 cacoshf32 F
+GLIBC_2.29 cacoshf32x F
+GLIBC_2.29 cacoshf64 F
+GLIBC_2.29 cacoshf64x F
+GLIBC_2.29 cacoshl F
+GLIBC_2.29 cacosl F
+GLIBC_2.29 canonicalize F
+GLIBC_2.29 canonicalizef F
+GLIBC_2.29 canonicalizef128 F
+GLIBC_2.29 canonicalizef32 F
+GLIBC_2.29 canonicalizef32x F
+GLIBC_2.29 canonicalizef64 F
+GLIBC_2.29 canonicalizef64x F
+GLIBC_2.29 canonicalizel F
+GLIBC_2.29 carg F
+GLIBC_2.29 cargf F
+GLIBC_2.29 cargf128 F
+GLIBC_2.29 cargf32 F
+GLIBC_2.29 cargf32x F
+GLIBC_2.29 cargf64 F
+GLIBC_2.29 cargf64x F
+GLIBC_2.29 cargl F
+GLIBC_2.29 casin F
+GLIBC_2.29 casinf F
+GLIBC_2.29 casinf128 F
+GLIBC_2.29 casinf32 F
+GLIBC_2.29 casinf32x F
+GLIBC_2.29 casinf64 F
+GLIBC_2.29 casinf64x F
+GLIBC_2.29 casinh F
+GLIBC_2.29 casinhf F
+GLIBC_2.29 casinhf128 F
+GLIBC_2.29 casinhf32 F
+GLIBC_2.29 casinhf32x F
+GLIBC_2.29 casinhf64 F
+GLIBC_2.29 casinhf64x F
+GLIBC_2.29 casinhl F
+GLIBC_2.29 casinl F
+GLIBC_2.29 catan F
+GLIBC_2.29 catanf F
+GLIBC_2.29 catanf128 F
+GLIBC_2.29 catanf32 F
+GLIBC_2.29 catanf32x F
+GLIBC_2.29 catanf64 F
+GLIBC_2.29 catanf64x F
+GLIBC_2.29 catanh F
+GLIBC_2.29 catanhf F
+GLIBC_2.29 catanhf128 F
+GLIBC_2.29 catanhf32 F
+GLIBC_2.29 catanhf32x F
+GLIBC_2.29 catanhf64 F
+GLIBC_2.29 catanhf64x F
+GLIBC_2.29 catanhl F
+GLIBC_2.29 catanl F
+GLIBC_2.29 cbrt F
+GLIBC_2.29 cbrtf F
+GLIBC_2.29 cbrtf128 F
+GLIBC_2.29 cbrtf32 F
+GLIBC_2.29 cbrtf32x F
+GLIBC_2.29 cbrtf64 F
+GLIBC_2.29 cbrtf64x F
+GLIBC_2.29 cbrtl F
+GLIBC_2.29 ccos F
+GLIBC_2.29 ccosf F
+GLIBC_2.29 ccosf128 F
+GLIBC_2.29 ccosf32 F
+GLIBC_2.29 ccosf32x F
+GLIBC_2.29 ccosf64 F
+GLIBC_2.29 ccosf64x F
+GLIBC_2.29 ccosh F
+GLIBC_2.29 ccoshf F
+GLIBC_2.29 ccoshf128 F
+GLIBC_2.29 ccoshf32 F
+GLIBC_2.29 ccoshf32x F
+GLIBC_2.29 ccoshf64 F
+GLIBC_2.29 ccoshf64x F
+GLIBC_2.29 ccoshl F
+GLIBC_2.29 ccosl F
+GLIBC_2.29 ceil F
+GLIBC_2.29 ceilf F
+GLIBC_2.29 ceilf128 F
+GLIBC_2.29 ceilf32 F
+GLIBC_2.29 ceilf32x F
+GLIBC_2.29 ceilf64 F
+GLIBC_2.29 ceilf64x F
+GLIBC_2.29 ceill F
+GLIBC_2.29 cexp F
+GLIBC_2.29 cexpf F
+GLIBC_2.29 cexpf128 F
+GLIBC_2.29 cexpf32 F
+GLIBC_2.29 cexpf32x F
+GLIBC_2.29 cexpf64 F
+GLIBC_2.29 cexpf64x F
+GLIBC_2.29 cexpl F
+GLIBC_2.29 cimag F
+GLIBC_2.29 cimagf F
+GLIBC_2.29 cimagf128 F
+GLIBC_2.29 cimagf32 F
+GLIBC_2.29 cimagf32x F
+GLIBC_2.29 cimagf64 F
+GLIBC_2.29 cimagf64x F
+GLIBC_2.29 cimagl F
+GLIBC_2.29 clog F
+GLIBC_2.29 clog10 F
+GLIBC_2.29 clog10f F
+GLIBC_2.29 clog10f128 F
+GLIBC_2.29 clog10f32 F
+GLIBC_2.29 clog10f32x F
+GLIBC_2.29 clog10f64 F
+GLIBC_2.29 clog10f64x F
+GLIBC_2.29 clog10l F
+GLIBC_2.29 clogf F
+GLIBC_2.29 clogf128 F
+GLIBC_2.29 clogf32 F
+GLIBC_2.29 clogf32x F
+GLIBC_2.29 clogf64 F
+GLIBC_2.29 clogf64x F
+GLIBC_2.29 clogl F
+GLIBC_2.29 conj F
+GLIBC_2.29 conjf F
+GLIBC_2.29 conjf128 F
+GLIBC_2.29 conjf32 F
+GLIBC_2.29 conjf32x F
+GLIBC_2.29 conjf64 F
+GLIBC_2.29 conjf64x F
+GLIBC_2.29 conjl F
+GLIBC_2.29 copysign F
+GLIBC_2.29 copysignf F
+GLIBC_2.29 copysignf128 F
+GLIBC_2.29 copysignf32 F
+GLIBC_2.29 copysignf32x F
+GLIBC_2.29 copysignf64 F
+GLIBC_2.29 copysignf64x F
+GLIBC_2.29 copysignl F
+GLIBC_2.29 cos F
+GLIBC_2.29 cosf F
+GLIBC_2.29 cosf128 F
+GLIBC_2.29 cosf32 F
+GLIBC_2.29 cosf32x F
+GLIBC_2.29 cosf64 F
+GLIBC_2.29 cosf64x F
+GLIBC_2.29 cosh F
+GLIBC_2.29 coshf F
+GLIBC_2.29 coshf128 F
+GLIBC_2.29 coshf32 F
+GLIBC_2.29 coshf32x F
+GLIBC_2.29 coshf64 F
+GLIBC_2.29 coshf64x F
+GLIBC_2.29 coshl F
+GLIBC_2.29 cosl F
+GLIBC_2.29 cpow F
+GLIBC_2.29 cpowf F
+GLIBC_2.29 cpowf128 F
+GLIBC_2.29 cpowf32 F
+GLIBC_2.29 cpowf32x F
+GLIBC_2.29 cpowf64 F
+GLIBC_2.29 cpowf64x F
+GLIBC_2.29 cpowl F
+GLIBC_2.29 cproj F
+GLIBC_2.29 cprojf F
+GLIBC_2.29 cprojf128 F
+GLIBC_2.29 cprojf32 F
+GLIBC_2.29 cprojf32x F
+GLIBC_2.29 cprojf64 F
+GLIBC_2.29 cprojf64x F
+GLIBC_2.29 cprojl F
+GLIBC_2.29 creal F
+GLIBC_2.29 crealf F
+GLIBC_2.29 crealf128 F
+GLIBC_2.29 crealf32 F
+GLIBC_2.29 crealf32x F
+GLIBC_2.29 crealf64 F
+GLIBC_2.29 crealf64x F
+GLIBC_2.29 creall F
+GLIBC_2.29 csin F
+GLIBC_2.29 csinf F
+GLIBC_2.29 csinf128 F
+GLIBC_2.29 csinf32 F
+GLIBC_2.29 csinf32x F
+GLIBC_2.29 csinf64 F
+GLIBC_2.29 csinf64x F
+GLIBC_2.29 csinh F
+GLIBC_2.29 csinhf F
+GLIBC_2.29 csinhf128 F
+GLIBC_2.29 csinhf32 F
+GLIBC_2.29 csinhf32x F
+GLIBC_2.29 csinhf64 F
+GLIBC_2.29 csinhf64x F
+GLIBC_2.29 csinhl F
+GLIBC_2.29 csinl F
+GLIBC_2.29 csqrt F
+GLIBC_2.29 csqrtf F
+GLIBC_2.29 csqrtf128 F
+GLIBC_2.29 csqrtf32 F
+GLIBC_2.29 csqrtf32x F
+GLIBC_2.29 csqrtf64 F
+GLIBC_2.29 csqrtf64x F
+GLIBC_2.29 csqrtl F
+GLIBC_2.29 ctan F
+GLIBC_2.29 ctanf F
+GLIBC_2.29 ctanf128 F
+GLIBC_2.29 ctanf32 F
+GLIBC_2.29 ctanf32x F
+GLIBC_2.29 ctanf64 F
+GLIBC_2.29 ctanf64x F
+GLIBC_2.29 ctanh F
+GLIBC_2.29 ctanhf F
+GLIBC_2.29 ctanhf128 F
+GLIBC_2.29 ctanhf32 F
+GLIBC_2.29 ctanhf32x F
+GLIBC_2.29 ctanhf64 F
+GLIBC_2.29 ctanhf64x F
+GLIBC_2.29 ctanhl F
+GLIBC_2.29 ctanl F
+GLIBC_2.29 daddl F
+GLIBC_2.29 ddivl F
+GLIBC_2.29 dmull F
+GLIBC_2.29 drem F
+GLIBC_2.29 dremf F
+GLIBC_2.29 dreml F
+GLIBC_2.29 dsubl F
+GLIBC_2.29 erf F
+GLIBC_2.29 erfc F
+GLIBC_2.29 erfcf F
+GLIBC_2.29 erfcf128 F
+GLIBC_2.29 erfcf32 F
+GLIBC_2.29 erfcf32x F
+GLIBC_2.29 erfcf64 F
+GLIBC_2.29 erfcf64x F
+GLIBC_2.29 erfcl F
+GLIBC_2.29 erff F
+GLIBC_2.29 erff128 F
+GLIBC_2.29 erff32 F
+GLIBC_2.29 erff32x F
+GLIBC_2.29 erff64 F
+GLIBC_2.29 erff64x F
+GLIBC_2.29 erfl F
+GLIBC_2.29 exp F
+GLIBC_2.29 exp10 F
+GLIBC_2.29 exp10f F
+GLIBC_2.29 exp10f128 F
+GLIBC_2.29 exp10f32 F
+GLIBC_2.29 exp10f32x F
+GLIBC_2.29 exp10f64 F
+GLIBC_2.29 exp10f64x F
+GLIBC_2.29 exp10l F
+GLIBC_2.29 exp2 F
+GLIBC_2.29 exp2f F
+GLIBC_2.29 exp2f128 F
+GLIBC_2.29 exp2f32 F
+GLIBC_2.29 exp2f32x F
+GLIBC_2.29 exp2f64 F
+GLIBC_2.29 exp2f64x F
+GLIBC_2.29 exp2l F
+GLIBC_2.29 expf F
+GLIBC_2.29 expf128 F
+GLIBC_2.29 expf32 F
+GLIBC_2.29 expf32x F
+GLIBC_2.29 expf64 F
+GLIBC_2.29 expf64x F
+GLIBC_2.29 expl F
+GLIBC_2.29 expm1 F
+GLIBC_2.29 expm1f F
+GLIBC_2.29 expm1f128 F
+GLIBC_2.29 expm1f32 F
+GLIBC_2.29 expm1f32x F
+GLIBC_2.29 expm1f64 F
+GLIBC_2.29 expm1f64x F
+GLIBC_2.29 expm1l F
+GLIBC_2.29 f32addf128 F
+GLIBC_2.29 f32addf32x F
+GLIBC_2.29 f32addf64 F
+GLIBC_2.29 f32addf64x F
+GLIBC_2.29 f32divf128 F
+GLIBC_2.29 f32divf32x F
+GLIBC_2.29 f32divf64 F
+GLIBC_2.29 f32divf64x F
+GLIBC_2.29 f32mulf128 F
+GLIBC_2.29 f32mulf32x F
+GLIBC_2.29 f32mulf64 F
+GLIBC_2.29 f32mulf64x F
+GLIBC_2.29 f32subf128 F
+GLIBC_2.29 f32subf32x F
+GLIBC_2.29 f32subf64 F
+GLIBC_2.29 f32subf64x F
+GLIBC_2.29 f32xaddf128 F
+GLIBC_2.29 f32xaddf64 F
+GLIBC_2.29 f32xaddf64x F
+GLIBC_2.29 f32xdivf128 F
+GLIBC_2.29 f32xdivf64 F
+GLIBC_2.29 f32xdivf64x F
+GLIBC_2.29 f32xmulf128 F
+GLIBC_2.29 f32xmulf64 F
+GLIBC_2.29 f32xmulf64x F
+GLIBC_2.29 f32xsubf128 F
+GLIBC_2.29 f32xsubf64 F
+GLIBC_2.29 f32xsubf64x F
+GLIBC_2.29 f64addf128 F
+GLIBC_2.29 f64addf64x F
+GLIBC_2.29 f64divf128 F
+GLIBC_2.29 f64divf64x F
+GLIBC_2.29 f64mulf128 F
+GLIBC_2.29 f64mulf64x F
+GLIBC_2.29 f64subf128 F
+GLIBC_2.29 f64subf64x F
+GLIBC_2.29 f64xaddf128 F
+GLIBC_2.29 f64xdivf128 F
+GLIBC_2.29 f64xmulf128 F
+GLIBC_2.29 f64xsubf128 F
+GLIBC_2.29 fabs F
+GLIBC_2.29 fabsf F
+GLIBC_2.29 fabsf128 F
+GLIBC_2.29 fabsf32 F
+GLIBC_2.29 fabsf32x F
+GLIBC_2.29 fabsf64 F
+GLIBC_2.29 fabsf64x F
+GLIBC_2.29 fabsl F
+GLIBC_2.29 fadd F
+GLIBC_2.29 faddl F
+GLIBC_2.29 fdim F
+GLIBC_2.29 fdimf F
+GLIBC_2.29 fdimf128 F
+GLIBC_2.29 fdimf32 F
+GLIBC_2.29 fdimf32x F
+GLIBC_2.29 fdimf64 F
+GLIBC_2.29 fdimf64x F
+GLIBC_2.29 fdiml F
+GLIBC_2.29 fdiv F
+GLIBC_2.29 fdivl F
+GLIBC_2.29 feclearexcept F
+GLIBC_2.29 fedisableexcept F
+GLIBC_2.29 feenableexcept F
+GLIBC_2.29 fegetenv F
+GLIBC_2.29 fegetexcept F
+GLIBC_2.29 fegetexceptflag F
+GLIBC_2.29 fegetmode F
+GLIBC_2.29 fegetround F
+GLIBC_2.29 feholdexcept F
+GLIBC_2.29 feraiseexcept F
+GLIBC_2.29 fesetenv F
+GLIBC_2.29 fesetexcept F
+GLIBC_2.29 fesetexceptflag F
+GLIBC_2.29 fesetmode F
+GLIBC_2.29 fesetround F
+GLIBC_2.29 fetestexcept F
+GLIBC_2.29 fetestexceptflag F
+GLIBC_2.29 feupdateenv F
+GLIBC_2.29 finite F
+GLIBC_2.29 finitef F
+GLIBC_2.29 finitel F
+GLIBC_2.29 floor F
+GLIBC_2.29 floorf F
+GLIBC_2.29 floorf128 F
+GLIBC_2.29 floorf32 F
+GLIBC_2.29 floorf32x F
+GLIBC_2.29 floorf64 F
+GLIBC_2.29 floorf64x F
+GLIBC_2.29 floorl F
+GLIBC_2.29 fma F
+GLIBC_2.29 fmaf F
+GLIBC_2.29 fmaf128 F
+GLIBC_2.29 fmaf32 F
+GLIBC_2.29 fmaf32x F
+GLIBC_2.29 fmaf64 F
+GLIBC_2.29 fmaf64x F
+GLIBC_2.29 fmal F
+GLIBC_2.29 fmax F
+GLIBC_2.29 fmaxf F
+GLIBC_2.29 fmaxf128 F
+GLIBC_2.29 fmaxf32 F
+GLIBC_2.29 fmaxf32x F
+GLIBC_2.29 fmaxf64 F
+GLIBC_2.29 fmaxf64x F
+GLIBC_2.29 fmaxl F
+GLIBC_2.29 fmaxmag F
+GLIBC_2.29 fmaxmagf F
+GLIBC_2.29 fmaxmagf128 F
+GLIBC_2.29 fmaxmagf32 F
+GLIBC_2.29 fmaxmagf32x F
+GLIBC_2.29 fmaxmagf64 F
+GLIBC_2.29 fmaxmagf64x F
+GLIBC_2.29 fmaxmagl F
+GLIBC_2.29 fmin F
+GLIBC_2.29 fminf F
+GLIBC_2.29 fminf128 F
+GLIBC_2.29 fminf32 F
+GLIBC_2.29 fminf32x F
+GLIBC_2.29 fminf64 F
+GLIBC_2.29 fminf64x F
+GLIBC_2.29 fminl F
+GLIBC_2.29 fminmag F
+GLIBC_2.29 fminmagf F
+GLIBC_2.29 fminmagf128 F
+GLIBC_2.29 fminmagf32 F
+GLIBC_2.29 fminmagf32x F
+GLIBC_2.29 fminmagf64 F
+GLIBC_2.29 fminmagf64x F
+GLIBC_2.29 fminmagl F
+GLIBC_2.29 fmod F
+GLIBC_2.29 fmodf F
+GLIBC_2.29 fmodf128 F
+GLIBC_2.29 fmodf32 F
+GLIBC_2.29 fmodf32x F
+GLIBC_2.29 fmodf64 F
+GLIBC_2.29 fmodf64x F
+GLIBC_2.29 fmodl F
+GLIBC_2.29 fmul F
+GLIBC_2.29 fmull F
+GLIBC_2.29 frexp F
+GLIBC_2.29 frexpf F
+GLIBC_2.29 frexpf128 F
+GLIBC_2.29 frexpf32 F
+GLIBC_2.29 frexpf32x F
+GLIBC_2.29 frexpf64 F
+GLIBC_2.29 frexpf64x F
+GLIBC_2.29 frexpl F
+GLIBC_2.29 fromfp F
+GLIBC_2.29 fromfpf F
+GLIBC_2.29 fromfpf128 F
+GLIBC_2.29 fromfpf32 F
+GLIBC_2.29 fromfpf32x F
+GLIBC_2.29 fromfpf64 F
+GLIBC_2.29 fromfpf64x F
+GLIBC_2.29 fromfpl F
+GLIBC_2.29 fromfpx F
+GLIBC_2.29 fromfpxf F
+GLIBC_2.29 fromfpxf128 F
+GLIBC_2.29 fromfpxf32 F
+GLIBC_2.29 fromfpxf32x F
+GLIBC_2.29 fromfpxf64 F
+GLIBC_2.29 fromfpxf64x F
+GLIBC_2.29 fromfpxl F
+GLIBC_2.29 fsub F
+GLIBC_2.29 fsubl F
+GLIBC_2.29 gamma F
+GLIBC_2.29 gammaf F
+GLIBC_2.29 gammal F
+GLIBC_2.29 getpayload F
+GLIBC_2.29 getpayloadf F
+GLIBC_2.29 getpayloadf128 F
+GLIBC_2.29 getpayloadf32 F
+GLIBC_2.29 getpayloadf32x F
+GLIBC_2.29 getpayloadf64 F
+GLIBC_2.29 getpayloadf64x F
+GLIBC_2.29 getpayloadl F
+GLIBC_2.29 hypot F
+GLIBC_2.29 hypotf F
+GLIBC_2.29 hypotf128 F
+GLIBC_2.29 hypotf32 F
+GLIBC_2.29 hypotf32x F
+GLIBC_2.29 hypotf64 F
+GLIBC_2.29 hypotf64x F
+GLIBC_2.29 hypotl F
+GLIBC_2.29 ilogb F
+GLIBC_2.29 ilogbf F
+GLIBC_2.29 ilogbf128 F
+GLIBC_2.29 ilogbf32 F
+GLIBC_2.29 ilogbf32x F
+GLIBC_2.29 ilogbf64 F
+GLIBC_2.29 ilogbf64x F
+GLIBC_2.29 ilogbl F
+GLIBC_2.29 j0 F
+GLIBC_2.29 j0f F
+GLIBC_2.29 j0f128 F
+GLIBC_2.29 j0f32 F
+GLIBC_2.29 j0f32x F
+GLIBC_2.29 j0f64 F
+GLIBC_2.29 j0f64x F
+GLIBC_2.29 j0l F
+GLIBC_2.29 j1 F
+GLIBC_2.29 j1f F
+GLIBC_2.29 j1f128 F
+GLIBC_2.29 j1f32 F
+GLIBC_2.29 j1f32x F
+GLIBC_2.29 j1f64 F
+GLIBC_2.29 j1f64x F
+GLIBC_2.29 j1l F
+GLIBC_2.29 jn F
+GLIBC_2.29 jnf F
+GLIBC_2.29 jnf128 F
+GLIBC_2.29 jnf32 F
+GLIBC_2.29 jnf32x F
+GLIBC_2.29 jnf64 F
+GLIBC_2.29 jnf64x F
+GLIBC_2.29 jnl F
+GLIBC_2.29 ldexp F
+GLIBC_2.29 ldexpf F
+GLIBC_2.29 ldexpf128 F
+GLIBC_2.29 ldexpf32 F
+GLIBC_2.29 ldexpf32x F
+GLIBC_2.29 ldexpf64 F
+GLIBC_2.29 ldexpf64x F
+GLIBC_2.29 ldexpl F
+GLIBC_2.29 lgamma F
+GLIBC_2.29 lgamma_r F
+GLIBC_2.29 lgammaf F
+GLIBC_2.29 lgammaf128 F
+GLIBC_2.29 lgammaf128_r F
+GLIBC_2.29 lgammaf32 F
+GLIBC_2.29 lgammaf32_r F
+GLIBC_2.29 lgammaf32x F
+GLIBC_2.29 lgammaf32x_r F
+GLIBC_2.29 lgammaf64 F
+GLIBC_2.29 lgammaf64_r F
+GLIBC_2.29 lgammaf64x F
+GLIBC_2.29 lgammaf64x_r F
+GLIBC_2.29 lgammaf_r F
+GLIBC_2.29 lgammal F
+GLIBC_2.29 lgammal_r F
+GLIBC_2.29 llogb F
+GLIBC_2.29 llogbf F
+GLIBC_2.29 llogbf128 F
+GLIBC_2.29 llogbf32 F
+GLIBC_2.29 llogbf32x F
+GLIBC_2.29 llogbf64 F
+GLIBC_2.29 llogbf64x F
+GLIBC_2.29 llogbl F
+GLIBC_2.29 llrint F
+GLIBC_2.29 llrintf F
+GLIBC_2.29 llrintf128 F
+GLIBC_2.29 llrintf32 F
+GLIBC_2.29 llrintf32x F
+GLIBC_2.29 llrintf64 F
+GLIBC_2.29 llrintf64x F
+GLIBC_2.29 llrintl F
+GLIBC_2.29 llround F
+GLIBC_2.29 llroundf F
+GLIBC_2.29 llroundf128 F
+GLIBC_2.29 llroundf32 F
+GLIBC_2.29 llroundf32x F
+GLIBC_2.29 llroundf64 F
+GLIBC_2.29 llroundf64x F
+GLIBC_2.29 llroundl F
+GLIBC_2.29 log F
+GLIBC_2.29 log10 F
+GLIBC_2.29 log10f F
+GLIBC_2.29 log10f128 F
+GLIBC_2.29 log10f32 F
+GLIBC_2.29 log10f32x F
+GLIBC_2.29 log10f64 F
+GLIBC_2.29 log10f64x F
+GLIBC_2.29 log10l F
+GLIBC_2.29 log1p F
+GLIBC_2.29 log1pf F
+GLIBC_2.29 log1pf128 F
+GLIBC_2.29 log1pf32 F
+GLIBC_2.29 log1pf32x F
+GLIBC_2.29 log1pf64 F
+GLIBC_2.29 log1pf64x F
+GLIBC_2.29 log1pl F
+GLIBC_2.29 log2 F
+GLIBC_2.29 log2f F
+GLIBC_2.29 log2f128 F
+GLIBC_2.29 log2f32 F
+GLIBC_2.29 log2f32x F
+GLIBC_2.29 log2f64 F
+GLIBC_2.29 log2f64x F
+GLIBC_2.29 log2l F
+GLIBC_2.29 logb F
+GLIBC_2.29 logbf F
+GLIBC_2.29 logbf128 F
+GLIBC_2.29 logbf32 F
+GLIBC_2.29 logbf32x F
+GLIBC_2.29 logbf64 F
+GLIBC_2.29 logbf64x F
+GLIBC_2.29 logbl F
+GLIBC_2.29 logf F
+GLIBC_2.29 logf128 F
+GLIBC_2.29 logf32 F
+GLIBC_2.29 logf32x F
+GLIBC_2.29 logf64 F
+GLIBC_2.29 logf64x F
+GLIBC_2.29 logl F
+GLIBC_2.29 lrint F
+GLIBC_2.29 lrintf F
+GLIBC_2.29 lrintf128 F
+GLIBC_2.29 lrintf32 F
+GLIBC_2.29 lrintf32x F
+GLIBC_2.29 lrintf64 F
+GLIBC_2.29 lrintf64x F
+GLIBC_2.29 lrintl F
+GLIBC_2.29 lround F
+GLIBC_2.29 lroundf F
+GLIBC_2.29 lroundf128 F
+GLIBC_2.29 lroundf32 F
+GLIBC_2.29 lroundf32x F
+GLIBC_2.29 lroundf64 F
+GLIBC_2.29 lroundf64x F
+GLIBC_2.29 lroundl F
+GLIBC_2.29 modf F
+GLIBC_2.29 modff F
+GLIBC_2.29 modff128 F
+GLIBC_2.29 modff32 F
+GLIBC_2.29 modff32x F
+GLIBC_2.29 modff64 F
+GLIBC_2.29 modff64x F
+GLIBC_2.29 modfl F
+GLIBC_2.29 nan F
+GLIBC_2.29 nanf F
+GLIBC_2.29 nanf128 F
+GLIBC_2.29 nanf32 F
+GLIBC_2.29 nanf32x F
+GLIBC_2.29 nanf64 F
+GLIBC_2.29 nanf64x F
+GLIBC_2.29 nanl F
+GLIBC_2.29 nearbyint F
+GLIBC_2.29 nearbyintf F
+GLIBC_2.29 nearbyintf128 F
+GLIBC_2.29 nearbyintf32 F
+GLIBC_2.29 nearbyintf32x F
+GLIBC_2.29 nearbyintf64 F
+GLIBC_2.29 nearbyintf64x F
+GLIBC_2.29 nearbyintl F
+GLIBC_2.29 nextafter F
+GLIBC_2.29 nextafterf F
+GLIBC_2.29 nextafterf128 F
+GLIBC_2.29 nextafterf32 F
+GLIBC_2.29 nextafterf32x F
+GLIBC_2.29 nextafterf64 F
+GLIBC_2.29 nextafterf64x F
+GLIBC_2.29 nextafterl F
+GLIBC_2.29 nextdown F
+GLIBC_2.29 nextdownf F
+GLIBC_2.29 nextdownf128 F
+GLIBC_2.29 nextdownf32 F
+GLIBC_2.29 nextdownf32x F
+GLIBC_2.29 nextdownf64 F
+GLIBC_2.29 nextdownf64x F
+GLIBC_2.29 nextdownl F
+GLIBC_2.29 nexttoward F
+GLIBC_2.29 nexttowardf F
+GLIBC_2.29 nexttowardl F
+GLIBC_2.29 nextup F
+GLIBC_2.29 nextupf F
+GLIBC_2.29 nextupf128 F
+GLIBC_2.29 nextupf32 F
+GLIBC_2.29 nextupf32x F
+GLIBC_2.29 nextupf64 F
+GLIBC_2.29 nextupf64x F
+GLIBC_2.29 nextupl F
+GLIBC_2.29 pow F
+GLIBC_2.29 powf F
+GLIBC_2.29 powf128 F
+GLIBC_2.29 powf32 F
+GLIBC_2.29 powf32x F
+GLIBC_2.29 powf64 F
+GLIBC_2.29 powf64x F
+GLIBC_2.29 powl F
+GLIBC_2.29 remainder F
+GLIBC_2.29 remainderf F
+GLIBC_2.29 remainderf128 F
+GLIBC_2.29 remainderf32 F
+GLIBC_2.29 remainderf32x F
+GLIBC_2.29 remainderf64 F
+GLIBC_2.29 remainderf64x F
+GLIBC_2.29 remainderl F
+GLIBC_2.29 remquo F
+GLIBC_2.29 remquof F
+GLIBC_2.29 remquof128 F
+GLIBC_2.29 remquof32 F
+GLIBC_2.29 remquof32x F
+GLIBC_2.29 remquof64 F
+GLIBC_2.29 remquof64x F
+GLIBC_2.29 remquol F
+GLIBC_2.29 rint F
+GLIBC_2.29 rintf F
+GLIBC_2.29 rintf128 F
+GLIBC_2.29 rintf32 F
+GLIBC_2.29 rintf32x F
+GLIBC_2.29 rintf64 F
+GLIBC_2.29 rintf64x F
+GLIBC_2.29 rintl F
+GLIBC_2.29 round F
+GLIBC_2.29 roundeven F
+GLIBC_2.29 roundevenf F
+GLIBC_2.29 roundevenf128 F
+GLIBC_2.29 roundevenf32 F
+GLIBC_2.29 roundevenf32x F
+GLIBC_2.29 roundevenf64 F
+GLIBC_2.29 roundevenf64x F
+GLIBC_2.29 roundevenl F
+GLIBC_2.29 roundf F
+GLIBC_2.29 roundf128 F
+GLIBC_2.29 roundf32 F
+GLIBC_2.29 roundf32x F
+GLIBC_2.29 roundf64 F
+GLIBC_2.29 roundf64x F
+GLIBC_2.29 roundl F
+GLIBC_2.29 scalb F
+GLIBC_2.29 scalbf F
+GLIBC_2.29 scalbl F
+GLIBC_2.29 scalbln F
+GLIBC_2.29 scalblnf F
+GLIBC_2.29 scalblnf128 F
+GLIBC_2.29 scalblnf32 F
+GLIBC_2.29 scalblnf32x F
+GLIBC_2.29 scalblnf64 F
+GLIBC_2.29 scalblnf64x F
+GLIBC_2.29 scalblnl F
+GLIBC_2.29 scalbn F
+GLIBC_2.29 scalbnf F
+GLIBC_2.29 scalbnf128 F
+GLIBC_2.29 scalbnf32 F
+GLIBC_2.29 scalbnf32x F
+GLIBC_2.29 scalbnf64 F
+GLIBC_2.29 scalbnf64x F
+GLIBC_2.29 scalbnl F
+GLIBC_2.29 setpayload F
+GLIBC_2.29 setpayloadf F
+GLIBC_2.29 setpayloadf128 F
+GLIBC_2.29 setpayloadf32 F
+GLIBC_2.29 setpayloadf32x F
+GLIBC_2.29 setpayloadf64 F
+GLIBC_2.29 setpayloadf64x F
+GLIBC_2.29 setpayloadl F
+GLIBC_2.29 setpayloadsig F
+GLIBC_2.29 setpayloadsigf F
+GLIBC_2.29 setpayloadsigf128 F
+GLIBC_2.29 setpayloadsigf32 F
+GLIBC_2.29 setpayloadsigf32x F
+GLIBC_2.29 setpayloadsigf64 F
+GLIBC_2.29 setpayloadsigf64x F
+GLIBC_2.29 setpayloadsigl F
+GLIBC_2.29 signgam D 0x4
+GLIBC_2.29 significand F
+GLIBC_2.29 significandf F
+GLIBC_2.29 significandl F
+GLIBC_2.29 sin F
+GLIBC_2.29 sincos F
+GLIBC_2.29 sincosf F
+GLIBC_2.29 sincosf128 F
+GLIBC_2.29 sincosf32 F
+GLIBC_2.29 sincosf32x F
+GLIBC_2.29 sincosf64 F
+GLIBC_2.29 sincosf64x F
+GLIBC_2.29 sincosl F
+GLIBC_2.29 sinf F
+GLIBC_2.29 sinf128 F
+GLIBC_2.29 sinf32 F
+GLIBC_2.29 sinf32x F
+GLIBC_2.29 sinf64 F
+GLIBC_2.29 sinf64x F
+GLIBC_2.29 sinh F
+GLIBC_2.29 sinhf F
+GLIBC_2.29 sinhf128 F
+GLIBC_2.29 sinhf32 F
+GLIBC_2.29 sinhf32x F
+GLIBC_2.29 sinhf64 F
+GLIBC_2.29 sinhf64x F
+GLIBC_2.29 sinhl F
+GLIBC_2.29 sinl F
+GLIBC_2.29 sqrt F
+GLIBC_2.29 sqrtf F
+GLIBC_2.29 sqrtf128 F
+GLIBC_2.29 sqrtf32 F
+GLIBC_2.29 sqrtf32x F
+GLIBC_2.29 sqrtf64 F
+GLIBC_2.29 sqrtf64x F
+GLIBC_2.29 sqrtl F
+GLIBC_2.29 tan F
+GLIBC_2.29 tanf F
+GLIBC_2.29 tanf128 F
+GLIBC_2.29 tanf32 F
+GLIBC_2.29 tanf32x F
+GLIBC_2.29 tanf64 F
+GLIBC_2.29 tanf64x F
+GLIBC_2.29 tanh F
+GLIBC_2.29 tanhf F
+GLIBC_2.29 tanhf128 F
+GLIBC_2.29 tanhf32 F
+GLIBC_2.29 tanhf32x F
+GLIBC_2.29 tanhf64 F
+GLIBC_2.29 tanhf64x F
+GLIBC_2.29 tanhl F
+GLIBC_2.29 tanl F
+GLIBC_2.29 tgamma F
+GLIBC_2.29 tgammaf F
+GLIBC_2.29 tgammaf128 F
+GLIBC_2.29 tgammaf32 F
+GLIBC_2.29 tgammaf32x F
+GLIBC_2.29 tgammaf64 F
+GLIBC_2.29 tgammaf64x F
+GLIBC_2.29 tgammal F
+GLIBC_2.29 totalorder F
+GLIBC_2.29 totalorderf F
+GLIBC_2.29 totalorderf128 F
+GLIBC_2.29 totalorderf32 F
+GLIBC_2.29 totalorderf32x F
+GLIBC_2.29 totalorderf64 F
+GLIBC_2.29 totalorderf64x F
+GLIBC_2.29 totalorderl F
+GLIBC_2.29 totalordermag F
+GLIBC_2.29 totalordermagf F
+GLIBC_2.29 totalordermagf128 F
+GLIBC_2.29 totalordermagf32 F
+GLIBC_2.29 totalordermagf32x F
+GLIBC_2.29 totalordermagf64 F
+GLIBC_2.29 totalordermagf64x F
+GLIBC_2.29 totalordermagl F
+GLIBC_2.29 trunc F
+GLIBC_2.29 truncf F
+GLIBC_2.29 truncf128 F
+GLIBC_2.29 truncf32 F
+GLIBC_2.29 truncf32x F
+GLIBC_2.29 truncf64 F
+GLIBC_2.29 truncf64x F
+GLIBC_2.29 truncl F
+GLIBC_2.29 ufromfp F
+GLIBC_2.29 ufromfpf F
+GLIBC_2.29 ufromfpf128 F
+GLIBC_2.29 ufromfpf32 F
+GLIBC_2.29 ufromfpf32x F
+GLIBC_2.29 ufromfpf64 F
+GLIBC_2.29 ufromfpf64x F
+GLIBC_2.29 ufromfpl F
+GLIBC_2.29 ufromfpx F
+GLIBC_2.29 ufromfpxf F
+GLIBC_2.29 ufromfpxf128 F
+GLIBC_2.29 ufromfpxf32 F
+GLIBC_2.29 ufromfpxf32x F
+GLIBC_2.29 ufromfpxf64 F
+GLIBC_2.29 ufromfpxf64x F
+GLIBC_2.29 ufromfpxl F
+GLIBC_2.29 y0 F
+GLIBC_2.29 y0f F
+GLIBC_2.29 y0f128 F
+GLIBC_2.29 y0f32 F
+GLIBC_2.29 y0f32x F
+GLIBC_2.29 y0f64 F
+GLIBC_2.29 y0f64x F
+GLIBC_2.29 y0l F
+GLIBC_2.29 y1 F
+GLIBC_2.29 y1f F
+GLIBC_2.29 y1f128 F
+GLIBC_2.29 y1f32 F
+GLIBC_2.29 y1f32x F
+GLIBC_2.29 y1f64 F
+GLIBC_2.29 y1f64x F
+GLIBC_2.29 y1l F
+GLIBC_2.29 yn F
+GLIBC_2.29 ynf F
+GLIBC_2.29 ynf128 F
+GLIBC_2.29 ynf32 F
+GLIBC_2.29 ynf32x F
+GLIBC_2.29 ynf64 F
+GLIBC_2.29 ynf64x F
+GLIBC_2.29 ynl F
diff --git a/sysdeps/unix/sysv/linux/riscv/rv32/libpthread.abilist b/sysdeps/unix/sysv/linux/riscv/rv32/libpthread.abilist
new file mode 100644
index 0000000000..ea4b79a518
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/riscv/rv32/libpthread.abilist
@@ -0,0 +1,235 @@
+GLIBC_2.29 _IO_flockfile F
+GLIBC_2.29 _IO_ftrylockfile F
+GLIBC_2.29 _IO_funlockfile F
+GLIBC_2.29 __close F
+GLIBC_2.29 __connect F
+GLIBC_2.29 __errno_location F
+GLIBC_2.29 __h_errno_location F
+GLIBC_2.29 __libc_allocate_rtsig F
+GLIBC_2.29 __libc_current_sigrtmax F
+GLIBC_2.29 __libc_current_sigrtmin F
+GLIBC_2.29 __lseek F
+GLIBC_2.29 __nanosleep F
+GLIBC_2.29 __open F
+GLIBC_2.29 __open64 F
+GLIBC_2.29 __pread64 F
+GLIBC_2.29 __pthread_cleanup_routine F
+GLIBC_2.29 __pthread_getspecific F
+GLIBC_2.29 __pthread_key_create F
+GLIBC_2.29 __pthread_mutex_destroy F
+GLIBC_2.29 __pthread_mutex_init F
+GLIBC_2.29 __pthread_mutex_lock F
+GLIBC_2.29 __pthread_mutex_trylock F
+GLIBC_2.29 __pthread_mutex_unlock F
+GLIBC_2.29 __pthread_mutexattr_destroy F
+GLIBC_2.29 __pthread_mutexattr_init F
+GLIBC_2.29 __pthread_mutexattr_settype F
+GLIBC_2.29 __pthread_once F
+GLIBC_2.29 __pthread_register_cancel F
+GLIBC_2.29 __pthread_register_cancel_defer F
+GLIBC_2.29 __pthread_rwlock_destroy F
+GLIBC_2.29 __pthread_rwlock_init F
+GLIBC_2.29 __pthread_rwlock_rdlock F
+GLIBC_2.29 __pthread_rwlock_tryrdlock F
+GLIBC_2.29 __pthread_rwlock_trywrlock F
+GLIBC_2.29 __pthread_rwlock_unlock F
+GLIBC_2.29 __pthread_rwlock_wrlock F
+GLIBC_2.29 __pthread_setspecific F
+GLIBC_2.29 __pthread_unregister_cancel F
+GLIBC_2.29 __pthread_unregister_cancel_restore F
+GLIBC_2.29 __pthread_unwind_next F
+GLIBC_2.29 __pwrite64 F
+GLIBC_2.29 __read F
+GLIBC_2.29 __res_state F
+GLIBC_2.29 __send F
+GLIBC_2.29 __sigaction F
+GLIBC_2.29 __wait F
+GLIBC_2.29 __write F
+GLIBC_2.29 _pthread_cleanup_pop F
+GLIBC_2.29 _pthread_cleanup_pop_restore F
+GLIBC_2.29 _pthread_cleanup_push F
+GLIBC_2.29 _pthread_cleanup_push_defer F
+GLIBC_2.29 accept F
+GLIBC_2.29 call_once F
+GLIBC_2.29 close F
+GLIBC_2.29 cnd_broadcast F
+GLIBC_2.29 cnd_destroy F
+GLIBC_2.29 cnd_init F
+GLIBC_2.29 cnd_signal F
+GLIBC_2.29 cnd_timedwait F
+GLIBC_2.29 cnd_wait F
+GLIBC_2.29 connect F
+GLIBC_2.29 flockfile F
+GLIBC_2.29 fsync F
+GLIBC_2.29 ftrylockfile F
+GLIBC_2.29 funlockfile F
+GLIBC_2.29 lseek F
+GLIBC_2.29 lseek64 F
+GLIBC_2.29 msync F
+GLIBC_2.29 mtx_destroy F
+GLIBC_2.29 mtx_init F
+GLIBC_2.29 mtx_lock F
+GLIBC_2.29 mtx_timedlock F
+GLIBC_2.29 mtx_trylock F
+GLIBC_2.29 mtx_unlock F
+GLIBC_2.29 nanosleep F
+GLIBC_2.29 open F
+GLIBC_2.29 open64 F
+GLIBC_2.29 pause F
+GLIBC_2.29 pread F
+GLIBC_2.29 pread64 F
+GLIBC_2.29 pthread_attr_destroy F
+GLIBC_2.29 pthread_attr_getaffinity_np F
+GLIBC_2.29 pthread_attr_getdetachstate F
+GLIBC_2.29 pthread_attr_getguardsize F
+GLIBC_2.29 pthread_attr_getinheritsched F
+GLIBC_2.29 pthread_attr_getschedparam F
+GLIBC_2.29 pthread_attr_getschedpolicy F
+GLIBC_2.29 pthread_attr_getscope F
+GLIBC_2.29 pthread_attr_getstack F
+GLIBC_2.29 pthread_attr_getstackaddr F
+GLIBC_2.29 pthread_attr_getstacksize F
+GLIBC_2.29 pthread_attr_init F
+GLIBC_2.29 pthread_attr_setaffinity_np F
+GLIBC_2.29 pthread_attr_setdetachstate F
+GLIBC_2.29 pthread_attr_setguardsize F
+GLIBC_2.29 pthread_attr_setinheritsched F
+GLIBC_2.29 pthread_attr_setschedparam F
+GLIBC_2.29 pthread_attr_setschedpolicy F
+GLIBC_2.29 pthread_attr_setscope F
+GLIBC_2.29 pthread_attr_setstack F
+GLIBC_2.29 pthread_attr_setstackaddr F
+GLIBC_2.29 pthread_attr_setstacksize F
+GLIBC_2.29 pthread_barrier_destroy F
+GLIBC_2.29 pthread_barrier_init F
+GLIBC_2.29 pthread_barrier_wait F
+GLIBC_2.29 pthread_barrierattr_destroy F
+GLIBC_2.29 pthread_barrierattr_getpshared F
+GLIBC_2.29 pthread_barrierattr_init F
+GLIBC_2.29 pthread_barrierattr_setpshared F
+GLIBC_2.29 pthread_cancel F
+GLIBC_2.29 pthread_cond_broadcast F
+GLIBC_2.29 pthread_cond_destroy F
+GLIBC_2.29 pthread_cond_init F
+GLIBC_2.29 pthread_cond_signal F
+GLIBC_2.29 pthread_cond_timedwait F
+GLIBC_2.29 pthread_cond_wait F
+GLIBC_2.29 pthread_condattr_destroy F
+GLIBC_2.29 pthread_condattr_getclock F
+GLIBC_2.29 pthread_condattr_getpshared F
+GLIBC_2.29 pthread_condattr_init F
+GLIBC_2.29 pthread_condattr_setclock F
+GLIBC_2.29 pthread_condattr_setpshared F
+GLIBC_2.29 pthread_create F
+GLIBC_2.29 pthread_detach F
+GLIBC_2.29 pthread_equal F
+GLIBC_2.29 pthread_exit F
+GLIBC_2.29 pthread_getaffinity_np F
+GLIBC_2.29 pthread_getattr_default_np F
+GLIBC_2.29 pthread_getattr_np F
+GLIBC_2.29 pthread_getconcurrency F
+GLIBC_2.29 pthread_getcpuclockid F
+GLIBC_2.29 pthread_getname_np F
+GLIBC_2.29 pthread_getschedparam F
+GLIBC_2.29 pthread_getspecific F
+GLIBC_2.29 pthread_join F
+GLIBC_2.29 pthread_key_create F
+GLIBC_2.29 pthread_key_delete F
+GLIBC_2.29 pthread_kill F
+GLIBC_2.29 pthread_kill_other_threads_np F
+GLIBC_2.29 pthread_mutex_consistent F
+GLIBC_2.29 pthread_mutex_consistent_np F
+GLIBC_2.29 pthread_mutex_destroy F
+GLIBC_2.29 pthread_mutex_getprioceiling F
+GLIBC_2.29 pthread_mutex_init F
+GLIBC_2.29 pthread_mutex_lock F
+GLIBC_2.29 pthread_mutex_setprioceiling F
+GLIBC_2.29 pthread_mutex_timedlock F
+GLIBC_2.29 pthread_mutex_trylock F
+GLIBC_2.29 pthread_mutex_unlock F
+GLIBC_2.29 pthread_mutexattr_destroy F
+GLIBC_2.29 pthread_mutexattr_getkind_np F
+GLIBC_2.29 pthread_mutexattr_getprioceiling F
+GLIBC_2.29 pthread_mutexattr_getprotocol F
+GLIBC_2.29 pthread_mutexattr_getpshared F
+GLIBC_2.29 pthread_mutexattr_getrobust F
+GLIBC_2.29 pthread_mutexattr_getrobust_np F
+GLIBC_2.29 pthread_mutexattr_gettype F
+GLIBC_2.29 pthread_mutexattr_init F
+GLIBC_2.29 pthread_mutexattr_setkind_np F
+GLIBC_2.29 pthread_mutexattr_setprioceiling F
+GLIBC_2.29 pthread_mutexattr_setprotocol F
+GLIBC_2.29 pthread_mutexattr_setpshared F
+GLIBC_2.29 pthread_mutexattr_setrobust F
+GLIBC_2.29 pthread_mutexattr_setrobust_np F
+GLIBC_2.29 pthread_mutexattr_settype F
+GLIBC_2.29 pthread_once F
+GLIBC_2.29 pthread_rwlock_destroy F
+GLIBC_2.29 pthread_rwlock_init F
+GLIBC_2.29 pthread_rwlock_rdlock F
+GLIBC_2.29 pthread_rwlock_timedrdlock F
+GLIBC_2.29 pthread_rwlock_timedwrlock F
+GLIBC_2.29 pthread_rwlock_tryrdlock F
+GLIBC_2.29 pthread_rwlock_trywrlock F
+GLIBC_2.29 pthread_rwlock_unlock F
+GLIBC_2.29 pthread_rwlock_wrlock F
+GLIBC_2.29 pthread_rwlockattr_destroy F
+GLIBC_2.29 pthread_rwlockattr_getkind_np F
+GLIBC_2.29 pthread_rwlockattr_getpshared F
+GLIBC_2.29 pthread_rwlockattr_init F
+GLIBC_2.29 pthread_rwlockattr_setkind_np F
+GLIBC_2.29 pthread_rwlockattr_setpshared F
+GLIBC_2.29 pthread_setaffinity_np F
+GLIBC_2.29 pthread_setattr_default_np F
+GLIBC_2.29 pthread_setcancelstate F
+GLIBC_2.29 pthread_setcanceltype F
+GLIBC_2.29 pthread_setconcurrency F
+GLIBC_2.29 pthread_setname_np F
+GLIBC_2.29 pthread_setschedparam F
+GLIBC_2.29 pthread_setschedprio F
+GLIBC_2.29 pthread_setspecific F
+GLIBC_2.29 pthread_sigmask F
+GLIBC_2.29 pthread_sigqueue F
+GLIBC_2.29 pthread_spin_destroy F
+GLIBC_2.29 pthread_spin_init F
+GLIBC_2.29 pthread_spin_lock F
+GLIBC_2.29 pthread_spin_trylock F
+GLIBC_2.29 pthread_spin_unlock F
+GLIBC_2.29 pthread_testcancel F
+GLIBC_2.29 pthread_timedjoin_np F
+GLIBC_2.29 pthread_tryjoin_np F
+GLIBC_2.29 pthread_yield F
+GLIBC_2.29 pwrite F
+GLIBC_2.29 pwrite64 F
+GLIBC_2.29 raise F
+GLIBC_2.29 read F
+GLIBC_2.29 recv F
+GLIBC_2.29 recvfrom F
+GLIBC_2.29 recvmsg F
+GLIBC_2.29 sem_close F
+GLIBC_2.29 sem_destroy F
+GLIBC_2.29 sem_getvalue F
+GLIBC_2.29 sem_init F
+GLIBC_2.29 sem_open F
+GLIBC_2.29 sem_post F
+GLIBC_2.29 sem_timedwait F
+GLIBC_2.29 sem_trywait F
+GLIBC_2.29 sem_unlink F
+GLIBC_2.29 sem_wait F
+GLIBC_2.29 send F
+GLIBC_2.29 sendmsg F
+GLIBC_2.29 sendto F
+GLIBC_2.29 sigaction F
+GLIBC_2.29 sigwait F
+GLIBC_2.29 tcdrain F
+GLIBC_2.29 thrd_create F
+GLIBC_2.29 thrd_detach F
+GLIBC_2.29 thrd_exit F
+GLIBC_2.29 thrd_join F
+GLIBC_2.29 tss_create F
+GLIBC_2.29 tss_delete F
+GLIBC_2.29 tss_get F
+GLIBC_2.29 tss_set F
+GLIBC_2.29 wait F
+GLIBC_2.29 waitpid F
+GLIBC_2.29 write F
diff --git a/sysdeps/unix/sysv/linux/riscv/rv32/libresolv.abilist b/sysdeps/unix/sysv/linux/riscv/rv32/libresolv.abilist
new file mode 100644
index 0000000000..2830a7efd1
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/riscv/rv32/libresolv.abilist
@@ -0,0 +1,79 @@
+GLIBC_2.29 __b64_ntop F
+GLIBC_2.29 __b64_pton F
+GLIBC_2.29 __dn_comp F
+GLIBC_2.29 __dn_count_labels F
+GLIBC_2.29 __dn_expand F
+GLIBC_2.29 __dn_skipname F
+GLIBC_2.29 __fp_nquery F
+GLIBC_2.29 __fp_query F
+GLIBC_2.29 __fp_resstat F
+GLIBC_2.29 __hostalias F
+GLIBC_2.29 __loc_aton F
+GLIBC_2.29 __loc_ntoa F
+GLIBC_2.29 __p_cdname F
+GLIBC_2.29 __p_cdnname F
+GLIBC_2.29 __p_class F
+GLIBC_2.29 __p_class_syms D 0x54
+GLIBC_2.29 __p_fqname F
+GLIBC_2.29 __p_fqnname F
+GLIBC_2.29 __p_option F
+GLIBC_2.29 __p_query F
+GLIBC_2.29 __p_rcode F
+GLIBC_2.29 __p_time F
+GLIBC_2.29 __p_type F
+GLIBC_2.29 __p_type_syms D 0x228
+GLIBC_2.29 __putlong F
+GLIBC_2.29 __putshort F
+GLIBC_2.29 __res_close F
+GLIBC_2.29 __res_dnok F
+GLIBC_2.29 __res_hnok F
+GLIBC_2.29 __res_hostalias F
+GLIBC_2.29 __res_isourserver F
+GLIBC_2.29 __res_mailok F
+GLIBC_2.29 __res_mkquery F
+GLIBC_2.29 __res_nameinquery F
+GLIBC_2.29 __res_nmkquery F
+GLIBC_2.29 __res_nquery F
+GLIBC_2.29 __res_nquerydomain F
+GLIBC_2.29 __res_nsearch F
+GLIBC_2.29 __res_nsend F
+GLIBC_2.29 __res_ownok F
+GLIBC_2.29 __res_queriesmatch F
+GLIBC_2.29 __res_query F
+GLIBC_2.29 __res_querydomain F
+GLIBC_2.29 __res_search F
+GLIBC_2.29 __res_send F
+GLIBC_2.29 __sym_ntop F
+GLIBC_2.29 __sym_ntos F
+GLIBC_2.29 __sym_ston F
+GLIBC_2.29 _getlong F
+GLIBC_2.29 _getshort F
+GLIBC_2.29 inet_net_ntop F
+GLIBC_2.29 inet_net_pton F
+GLIBC_2.29 inet_neta F
+GLIBC_2.29 ns_datetosecs F
+GLIBC_2.29 ns_format_ttl F
+GLIBC_2.29 ns_get16 F
+GLIBC_2.29 ns_get32 F
+GLIBC_2.29 ns_initparse F
+GLIBC_2.29 ns_makecanon F
+GLIBC_2.29 ns_msg_getflag F
+GLIBC_2.29 ns_name_compress F
+GLIBC_2.29 ns_name_ntol F
+GLIBC_2.29 ns_name_ntop F
+GLIBC_2.29 ns_name_pack F
+GLIBC_2.29 ns_name_pton F
+GLIBC_2.29 ns_name_rollback F
+GLIBC_2.29 ns_name_skip F
+GLIBC_2.29 ns_name_uncompress F
+GLIBC_2.29 ns_name_unpack F
+GLIBC_2.29 ns_parse_ttl F
+GLIBC_2.29 ns_parserr F
+GLIBC_2.29 ns_put16 F
+GLIBC_2.29 ns_put32 F
+GLIBC_2.29 ns_samedomain F
+GLIBC_2.29 ns_samename F
+GLIBC_2.29 ns_skiprr F
+GLIBC_2.29 ns_sprintrr F
+GLIBC_2.29 ns_sprintrrf F
+GLIBC_2.29 ns_subdomain F
diff --git a/sysdeps/unix/sysv/linux/riscv/rv32/librt.abilist b/sysdeps/unix/sysv/linux/riscv/rv32/librt.abilist
new file mode 100644
index 0000000000..c6690ef7c1
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/riscv/rv32/librt.abilist
@@ -0,0 +1,35 @@
+GLIBC_2.29 __mq_open_2 F
+GLIBC_2.29 aio_cancel F
+GLIBC_2.29 aio_cancel64 F
+GLIBC_2.29 aio_error F
+GLIBC_2.29 aio_error64 F
+GLIBC_2.29 aio_fsync F
+GLIBC_2.29 aio_fsync64 F
+GLIBC_2.29 aio_init F
+GLIBC_2.29 aio_read F
+GLIBC_2.29 aio_read64 F
+GLIBC_2.29 aio_return F
+GLIBC_2.29 aio_return64 F
+GLIBC_2.29 aio_suspend F
+GLIBC_2.29 aio_suspend64 F
+GLIBC_2.29 aio_write F
+GLIBC_2.29 aio_write64 F
+GLIBC_2.29 lio_listio F
+GLIBC_2.29 lio_listio64 F
+GLIBC_2.29 mq_close F
+GLIBC_2.29 mq_getattr F
+GLIBC_2.29 mq_notify F
+GLIBC_2.29 mq_open F
+GLIBC_2.29 mq_receive F
+GLIBC_2.29 mq_send F
+GLIBC_2.29 mq_setattr F
+GLIBC_2.29 mq_timedreceive F
+GLIBC_2.29 mq_timedsend F
+GLIBC_2.29 mq_unlink F
+GLIBC_2.29 shm_open F
+GLIBC_2.29 shm_unlink F
+GLIBC_2.29 timer_create F
+GLIBC_2.29 timer_delete F
+GLIBC_2.29 timer_getoverrun F
+GLIBC_2.29 timer_gettime F
+GLIBC_2.29 timer_settime F
diff --git a/sysdeps/unix/sysv/linux/riscv/rv32/libthread_db.abilist b/sysdeps/unix/sysv/linux/riscv/rv32/libthread_db.abilist
new file mode 100644
index 0000000000..37e9bace55
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/riscv/rv32/libthread_db.abilist
@@ -0,0 +1,40 @@
+GLIBC_2.29 td_init F
+GLIBC_2.29 td_log F
+GLIBC_2.29 td_symbol_list F
+GLIBC_2.29 td_ta_clear_event F
+GLIBC_2.29 td_ta_delete F
+GLIBC_2.29 td_ta_enable_stats F
+GLIBC_2.29 td_ta_event_addr F
+GLIBC_2.29 td_ta_event_getmsg F
+GLIBC_2.29 td_ta_get_nthreads F
+GLIBC_2.29 td_ta_get_ph F
+GLIBC_2.29 td_ta_get_stats F
+GLIBC_2.29 td_ta_map_id2thr F
+GLIBC_2.29 td_ta_map_lwp2thr F
+GLIBC_2.29 td_ta_new F
+GLIBC_2.29 td_ta_reset_stats F
+GLIBC_2.29 td_ta_set_event F
+GLIBC_2.29 td_ta_setconcurrency F
+GLIBC_2.29 td_ta_thr_iter F
+GLIBC_2.29 td_ta_tsd_iter F
+GLIBC_2.29 td_thr_clear_event F
+GLIBC_2.29 td_thr_dbresume F
+GLIBC_2.29 td_thr_dbsuspend F
+GLIBC_2.29 td_thr_event_enable F
+GLIBC_2.29 td_thr_event_getmsg F
+GLIBC_2.29 td_thr_get_info F
+GLIBC_2.29 td_thr_getfpregs F
+GLIBC_2.29 td_thr_getgregs F
+GLIBC_2.29 td_thr_getxregs F
+GLIBC_2.29 td_thr_getxregsize F
+GLIBC_2.29 td_thr_set_event F
+GLIBC_2.29 td_thr_setfpregs F
+GLIBC_2.29 td_thr_setgregs F
+GLIBC_2.29 td_thr_setprio F
+GLIBC_2.29 td_thr_setsigpending F
+GLIBC_2.29 td_thr_setxregs F
+GLIBC_2.29 td_thr_sigsetmask F
+GLIBC_2.29 td_thr_tls_get_addr F
+GLIBC_2.29 td_thr_tlsbase F
+GLIBC_2.29 td_thr_tsd F
+GLIBC_2.29 td_thr_validate F
diff --git a/sysdeps/unix/sysv/linux/riscv/rv32/libutil.abilist b/sysdeps/unix/sysv/linux/riscv/rv32/libutil.abilist
new file mode 100644
index 0000000000..cbd11999a4
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/riscv/rv32/libutil.abilist
@@ -0,0 +1,6 @@
+GLIBC_2.29 forkpty F
+GLIBC_2.29 login F
+GLIBC_2.29 login_tty F
+GLIBC_2.29 logout F
+GLIBC_2.29 logwtmp F
+GLIBC_2.29 openpty F
-- 
2.22.0

[RFC v3 09/23] RISC-V: define __NR_* as __NR_*_time64/64 for 32-bit

[Alistair Francis]

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 sysdeps/unix/sysv/linux/riscv/sysdep.h | 42 +++++++++++++++++++++++++-
 1 file changed, 41 insertions(+), 1 deletion(-)

diff --git a/sysdeps/unix/sysv/linux/riscv/sysdep.h b/sysdeps/unix/sysv/linux/riscv/sysdep.h
index b7607ef9f1..2c06706701 100644
--- a/sysdeps/unix/sysv/linux/riscv/sysdep.h
+++ b/sysdeps/unix/sysv/linux/riscv/sysdep.h
@@ -123,7 +123,47 @@
 # ifndef __NR_futex
 #  define __NR_futex __NR_futex_time64
 # endif
-#endif
+
+# ifndef __NR_rt_sigtimedwait
+#  define __NR_rt_sigtimedwait __NR_rt_sigtimedwait_time64
+# endif
+
+# ifndef __NR_ppoll
+#  define __NR_ppoll __NR_ppoll_time64
+# endif
+
+# ifndef __NR_utimensat
+#  define __NR_utimensat __NR_utimensat_time64
+# endif
+
+# ifndef __NR_pselect6
+#  define __NR_pselect6 __NR_pselect6_time64
+# endif
+
+# ifndef __NR_recvmmsg
+#  define __NR_recvmmsg __NR_recvmmsg_time64
+# endif
+
+# ifndef __NR_semtimedop
+#  define __NR_semtimedop __NR_semtimedop_time64
+# endif
+
+# ifndef __NR_mq_timedreceive
+#  define __NR_mq_timedreceive __NR_mq_timedreceive_time64
+# endif
+
+# ifndef __NR_mq_timedsend
+#  define __NR_mq_timedsend __NR_mq_timedsend_time64
+# endif
+
+# ifndef __NR_timer_gettime
+#  define __NR_timer_gettime __NR_timer_gettime64
+# endif
+
+# ifndef __NR_timer_settime
+#  define __NR_timer_settime __NR_timer_settime64
+# endif
+#endif /* __riscv_xlen == 32 */
 
 #undef SYS_ify
 #define SYS_ify(syscall_name)	__NR_##syscall_name
-- 
2.22.0

[RFC v3 17/23] RISC-V: Hard float support for the 32 bit

[Alistair Francis]

From: Zong Li <zongbox@gmail.com>

This patch contains hardware floating-point support for the RV32IF and
RV32IFD

2018-11-29  Zong Li  <zong@andestech.com>

	* sysdeps/riscv/rv32/rvd/s_lrint.c: New file.
	* sysdeps/riscv/rv32/rvd/s_lround.c: Likewise.
	* sysdeps/riscv/rv32/rvf/s_lrintf.c: Likewise.
	* sysdeps/riscv/rv32/rvf/s_lroundf.c: Likewise.
---
 ChangeLog                          |  4 ++++
 sysdeps/riscv/rv32/rvd/s_lrint.c   | 31 ++++++++++++++++++++++++++++++
 sysdeps/riscv/rv32/rvd/s_lround.c  | 31 ++++++++++++++++++++++++++++++
 sysdeps/riscv/rv32/rvf/s_lrintf.c  | 31 ++++++++++++++++++++++++++++++
 sysdeps/riscv/rv32/rvf/s_lroundf.c | 31 ++++++++++++++++++++++++++++++
 5 files changed, 128 insertions(+)
 create mode 100644 sysdeps/riscv/rv32/rvd/s_lrint.c
 create mode 100644 sysdeps/riscv/rv32/rvd/s_lround.c
 create mode 100644 sysdeps/riscv/rv32/rvf/s_lrintf.c
 create mode 100644 sysdeps/riscv/rv32/rvf/s_lroundf.c

diff --git a/ChangeLog b/ChangeLog
index e0446d70a0..9c61de6470 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1051,6 +1051,10 @@
 	* sysdeps/riscv/sfp-machine.h: Likewise.
 	* sysdeps/riscv/sys/asm.h: Likewise.
 	* sysdeps/unix/sysv/linux/riscv/rv32/jmp_buf-macros.h: New file.
+	* sysdeps/riscv/rv32/rvd/s_lrint.c: New file.
+	* sysdeps/riscv/rv32/rvd/s_lround.c: Likewise.
+	* sysdeps/riscv/rv32/rvf/s_lrintf.c: Likewise.
+	* sysdeps/riscv/rv32/rvf/s_lroundf.c: Likewise.
 
 2019-06-20  Dmitry V. Levin  <ldv@altlinux.org>
 	    Florian Weimer  <fweimer@redhat.com>
diff --git a/sysdeps/riscv/rv32/rvd/s_lrint.c b/sysdeps/riscv/rv32/rvd/s_lrint.c
new file mode 100644
index 0000000000..4d5bdbc200
--- /dev/null
+++ b/sysdeps/riscv/rv32/rvd/s_lrint.c
@@ -0,0 +1,31 @@
+/* lrint().  RISC-V version.
+   Copyright (C) 2017-2018 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library.  If not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#include <math.h>
+#include <libm-alias-double.h>
+#include <stdint.h>
+
+long int
+__lrint (double x)
+{
+  int32_t res;
+  asm ("fcvt.w.d %0, %1" : "=r" (res) : "f" (x));
+  return res;
+}
+
+libm_alias_double (__lrint, lrint)
diff --git a/sysdeps/riscv/rv32/rvd/s_lround.c b/sysdeps/riscv/rv32/rvd/s_lround.c
new file mode 100644
index 0000000000..f5d9cf2e07
--- /dev/null
+++ b/sysdeps/riscv/rv32/rvd/s_lround.c
@@ -0,0 +1,31 @@
+/* lround().  RISC-V version.
+   Copyright (C) 2017-2018 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library.  If not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#include <math.h>
+#include <libm-alias-double.h>
+#include <stdint.h>
+
+long int
+__lround (double x)
+{
+  int32_t res;
+  asm ("fcvt.w.d %0, %1, rmm" : "=r" (res) : "f" (x));
+  return res;
+}
+
+libm_alias_double (__lround, lround)
diff --git a/sysdeps/riscv/rv32/rvf/s_lrintf.c b/sysdeps/riscv/rv32/rvf/s_lrintf.c
new file mode 100644
index 0000000000..08d44fa738
--- /dev/null
+++ b/sysdeps/riscv/rv32/rvf/s_lrintf.c
@@ -0,0 +1,31 @@
+/* lrintf().  RISC-V version.
+   Copyright (C) 2017-2018 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library.  If not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#include <math.h>
+#include <libm-alias-float.h>
+#include <stdint.h>
+
+long int
+__lrintf (float x)
+{
+  int32_t res;
+  asm ("fcvt.w.s %0, %1" : "=r" (res) : "f" (x));
+  return res;
+}
+
+libm_alias_float (__lrint, lrint)
diff --git a/sysdeps/riscv/rv32/rvf/s_lroundf.c b/sysdeps/riscv/rv32/rvf/s_lroundf.c
new file mode 100644
index 0000000000..f31b432936
--- /dev/null
+++ b/sysdeps/riscv/rv32/rvf/s_lroundf.c
@@ -0,0 +1,31 @@
+/* lroundf().  RISC-V version.
+   Copyright (C) 2017-2018 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library.  If not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#include <math.h>
+#include <libm-alias-float.h>
+#include <stdint.h>
+
+long int
+__lroundf (float x)
+{
+  int32_t res;
+  asm ("fcvt.w.s %0, %1, rmm" : "=r" (res) : "f" (x));
+  return res;
+}
+
+libm_alias_float (__lround, lround)
-- 
2.22.0

[RFC v3 07/23] RISC-V: Use 64-bit time_t and off_t for RV32 and RV64

[Alistair Francis]

Using the original glibc headers under bits/ let's make small
modifications to use 64-bit time_t and off_t for both RV32 and RV64.

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
---
 ChangeLog                                     |  4 +
 .../unix/sysv/linux/riscv/bits/environments.h | 85 +++++++++++++++++++
 sysdeps/unix/sysv/linux/riscv/bits/time64.h   | 36 ++++++++
 sysdeps/unix/sysv/linux/riscv/bits/timesize.h | 22 +++++
 .../unix/sysv/linux/riscv/bits/typesizes.h    | 84 ++++++++++++++++++
 5 files changed, 231 insertions(+)
 create mode 100644 sysdeps/unix/sysv/linux/riscv/bits/environments.h
 create mode 100644 sysdeps/unix/sysv/linux/riscv/bits/time64.h
 create mode 100644 sysdeps/unix/sysv/linux/riscv/bits/timesize.h
 create mode 100644 sysdeps/unix/sysv/linux/riscv/bits/typesizes.h

diff --git a/ChangeLog b/ChangeLog
index fb54a4aa61..ec07b3ff27 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1032,6 +1032,10 @@
 	* sysdeps/unix/sysv/linux/wait.c: Use __NR_waitid if avaliable.
 	* sysdeps/unix/sysv/linux/waitpid.c: Likewise.
 	* sysdeps/unix/sysv/linux/waitpid_nocancel.c: Likewise.
+	* sysdeps/unix/sysv/linux/riscv/bits/environments.h: Use 64-bit time_t and off_t for RV32 and RV64.
+	* sysdeps/unix/sysv/linux/riscv/bits/time64.h: Likewise.
+	* sysdeps/unix/sysv/linux/riscv/bits/timesize.h: Likewise.
+	* sysdeps/unix/sysv/linux/riscv/bits/typesizes.h: Likewise.
 
 2019-06-20  Dmitry V. Levin  <ldv@altlinux.org>
 	    Florian Weimer  <fweimer@redhat.com>
diff --git a/sysdeps/unix/sysv/linux/riscv/bits/environments.h b/sysdeps/unix/sysv/linux/riscv/bits/environments.h
new file mode 100644
index 0000000000..bdbf18fca0
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/riscv/bits/environments.h
@@ -0,0 +1,85 @@
+/* Copyright (C) 1999-2019 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#ifndef _UNISTD_H
+# error "Never include this file directly.  Use <unistd.h> instead"
+#endif
+
+#include <bits/wordsize.h>
+
+/* This header should define the following symbols under the described
+   situations.  A value `1' means that the model is always supported,
+   `-1' means it is never supported.  Undefined means it cannot be
+   statically decided.
+
+   _POSIX_V7_ILP32_OFF32   32bit int, long, pointers, and off_t type
+   _POSIX_V7_ILP32_OFFBIG  32bit int, long, and pointers and larger off_t type
+
+   _POSIX_V7_LP64_OFF32	   64bit long and pointers and 32bit off_t type
+   _POSIX_V7_LPBIG_OFFBIG  64bit long and pointers and large off_t type
+
+   The macros _POSIX_V6_ILP32_OFF32, _POSIX_V6_ILP32_OFFBIG,
+   _POSIX_V6_LP64_OFF32, _POSIX_V6_LPBIG_OFFBIG, _XBS5_ILP32_OFF32,
+   _XBS5_ILP32_OFFBIG, _XBS5_LP64_OFF32, and _XBS5_LPBIG_OFFBIG were
+   used in previous versions of the Unix standard and are available
+   only for compatibility.
+*/
+
+#if __WORDSIZE == 64
+
+/* We can never provide environments with 32-bit wide pointers.  */
+# define _POSIX_V7_ILP32_OFF32	-1
+# define _POSIX_V7_ILP32_OFFBIG	-1
+# define _POSIX_V6_ILP32_OFF32	-1
+# define _POSIX_V6_ILP32_OFFBIG	-1
+# define _XBS5_ILP32_OFF32	-1
+# define _XBS5_ILP32_OFFBIG	-1
+/* We also have no use (for now) for an environment with bigger pointers
+   and offsets.  */
+# define _POSIX_V7_LPBIG_OFFBIG	-1
+# define _POSIX_V6_LPBIG_OFFBIG	-1
+# define _XBS5_LPBIG_OFFBIG	-1
+
+/* By default we have 64-bit wide `long int', pointers and `off_t'.  */
+# define _POSIX_V7_LP64_OFF64	1
+# define _POSIX_V6_LP64_OFF64	1
+# define _XBS5_LP64_OFF64	1
+
+#else /* __WORDSIZE == 32 */
+
+/* RISC-V requires 64-bit off_t
+  # undef _POSIX_V7_ILP32_OFF32
+  # undef _POSIX_V6_ILP32_OFF32
+  # undef _XBS5_ILP32_OFF32
+ */
+
+# define _POSIX_V7_ILP32_OFFBIG  1
+# define _POSIX_V6_ILP32_OFFBIG  1
+# define _XBS5_ILP32_OFFBIG   1
+
+/* We can never provide environments with 64-bit wide pointers.  */
+# define _POSIX_V7_LP64_OFF64	-1
+# define _POSIX_V7_LPBIG_OFFBIG	-1
+# define _POSIX_V6_LP64_OFF64	-1
+# define _POSIX_V6_LPBIG_OFFBIG	-1
+# define _XBS5_LP64_OFF64	-1
+# define _XBS5_LPBIG_OFFBIG	-1
+
+/* CFLAGS.  */
+#define __ILP32_OFFBIG_CFLAGS   "-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64"
+
+#endif /* __WORDSIZE == 32 */
diff --git a/sysdeps/unix/sysv/linux/riscv/bits/time64.h b/sysdeps/unix/sysv/linux/riscv/bits/time64.h
new file mode 100644
index 0000000000..4be26d32d4
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/riscv/bits/time64.h
@@ -0,0 +1,36 @@
+/* bits/time64.h -- underlying types for __time64_t.  Generic version.
+   Copyright (C) 2018-2019 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#ifndef _BITS_TYPES_H
+# error "Never include <bits/time64.h> directly; use <sys/types.h> instead."
+#endif
+
+#ifndef	_BITS_TIME64_H
+#define	_BITS_TIME64_H	1
+
+/* Define __TIME64_T_TYPE so that it is always a 64-bit type.  */
+
+#if __WORDSIZE == 64
+/* If we already have 64-bit time type then use it.  */
+# define __TIME64_T_TYPE		__TIME_T_TYPE
+#else
+/* Define a 64-bit time type alongsize the 32-bit one.  */
+# define __TIME64_T_TYPE		__SQUAD_TYPE
+#endif
+
+#endif /* bits/time64.h */
diff --git a/sysdeps/unix/sysv/linux/riscv/bits/timesize.h b/sysdeps/unix/sysv/linux/riscv/bits/timesize.h
new file mode 100644
index 0000000000..150af80f86
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/riscv/bits/timesize.h
@@ -0,0 +1,22 @@
+/* Bit size of the time_t type at glibc build time, general case.
+   Copyright (C) 2018-2019 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#include <bits/wordsize.h>
+
+/* RV32 and RV64 both use 64-bit time_t */
+#define __TIMESIZE	64
diff --git a/sysdeps/unix/sysv/linux/riscv/bits/typesizes.h b/sysdeps/unix/sysv/linux/riscv/bits/typesizes.h
new file mode 100644
index 0000000000..221811b72e
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/riscv/bits/typesizes.h
@@ -0,0 +1,84 @@
+/* bits/typesizes.h -- underlying types for *_t.  Generic version.
+   Copyright (C) 2002-2019 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#ifndef _BITS_TYPES_H
+# error "Never include <bits/typesizes.h> directly; use <sys/types.h> instead."
+#endif
+
+#ifndef	_BITS_TYPESIZES_H
+#define	_BITS_TYPESIZES_H	1
+
+/* See <bits/types.h> for the meaning of these macros.  This file exists so
+   that <bits/types.h> need not vary across different GNU platforms.  */
+#define __DEV_T_TYPE    __UQUAD_TYPE
+#define __UID_T_TYPE    __U32_TYPE
+#define __GID_T_TYPE    __U32_TYPE
+#define __INO_T_TYPE    __UQUAD_TYPE
+#define __INO64_T_TYPE     __UQUAD_TYPE
+#define __MODE_T_TYPE      __U32_TYPE
+#define __NLINK_T_TYPE    __U32_TYPE
+#define __OFF_T_TYPE    __SQUAD_TYPE
+#define __OFF64_T_TYPE     __SQUAD_TYPE
+#define __PID_T_TYPE    __S32_TYPE
+#define __RLIM_T_TYPE      __UQUAD_TYPE
+#define __RLIM64_T_TYPE    __UQUAD_TYPE
+#define __BLKCNT_T_TYPE    __SQUAD_TYPE
+#define __BLKCNT64_T_TYPE  __SQUAD_TYPE
+#define __FSBLKCNT_T_TYPE  __UQUAD_TYPE
+#define __FSBLKCNT64_T_TYPE   __UQUAD_TYPE
+#define __FSFILCNT_T_TYPE  __UQUAD_TYPE
+#define __FSFILCNT64_T_TYPE   __UQUAD_TYPE
+#define __FSWORD_T_TYPE   __SWORD_TYPE
+#define __ID_T_TYPE     __U32_TYPE
+#define __CLOCK_T_TYPE     __SLONGWORD_TYPE
+#define __TIME_T_TYPE      __SQUAD_TYPE
+#define __USECONDS_T_TYPE  __U32_TYPE
+#define __SUSECONDS_T_TYPE __SQUAD_TYPE
+#define __DADDR_T_TYPE     __S32_TYPE
+#define __KEY_T_TYPE    __S32_TYPE
+#define __CLOCKID_T_TYPE   __S32_TYPE
+#define __TIMER_T_TYPE     void *
+#define __BLKSIZE_T_TYPE   __S32_TYPE
+#define __FSID_T_TYPE      struct { int __val[2]; }
+#define __SSIZE_T_TYPE     __SWORD_TYPE
+#define __SYSCALL_SLONG_TYPE __SQUAD_TYPE
+#define __SYSCALL_ULONG_TYPE __UQUAD_TYPE
+#define __CPU_MASK_TYPE    __UQUAD_TYPE
+
+/* Tell the libc code that off_t and off64_t are actually the same type
+   for all ABI purposes, even if possibly expressed as different base types
+   for C type-checking purposes.  */
+# define __OFF_T_MATCHES_OFF64_T 1
+
+/* Same for ino_t and ino64_t.  */
+# define __INO_T_MATCHES_INO64_T 1
+
+/* And for rlim_t and rlim64_t.  */
+# define __RLIM_T_MATCHES_RLIM64_T  1
+
+/* Define these as RV32 requires 64-bit syscall. */
+#if __riscv_xlen == 32
+#define __ASSUME_TIME64_SYSCALLS 1
+#define __ASSUME_RLIM64_SYSCALLS 1
+#endif
+
+/* Number of descriptors that can fit in an `fd_set'.  */
+#define	__FD_SETSIZE		1024
+
+
+#endif /* bits/typesizes.h */
-- 
2.22.0

Re: [RFC v3 07/23] RISC-V: Use 64-bit time_t and off_t for RV32 and RV64

[Arnd Bergmann]

On Wed, Jul 17, 2019 at 2:11 AM Alistair Francis
<alistair.francis@wdc.com> wrote:

> +#define __DEV_T_TYPE    __UQUAD_TYPE
> +#define __UID_T_TYPE    __U32_TYPE
> +#define __GID_T_TYPE    __U32_TYPE
> +#define __INO_T_TYPE    __UQUAD_TYPE
> +#define __INO64_T_TYPE     __UQUAD_TYPE
> +#define __MODE_T_TYPE      __U32_TYPE
> +#define __NLINK_T_TYPE    __U32_TYPE
> +#define __OFF_T_TYPE    __SQUAD_TYPE
> +#define __OFF64_T_TYPE     __SQUAD_TYPE
> +#define __PID_T_TYPE    __S32_TYPE
> +#define __RLIM_T_TYPE      __UQUAD_TYPE
> +#define __RLIM64_T_TYPE    __UQUAD_TYPE
> +#define __BLKCNT_T_TYPE    __SQUAD_TYPE
> +#define __BLKCNT64_T_TYPE  __SQUAD_TYPE
> +#define __FSBLKCNT_T_TYPE  __UQUAD_TYPE
> +#define __FSBLKCNT64_T_TYPE   __UQUAD_TYPE
> +#define __FSFILCNT_T_TYPE  __UQUAD_TYPE
> +#define __FSFILCNT64_T_TYPE   __UQUAD_TYPE
> +#define __FSWORD_T_TYPE   __SWORD_TYPE
> +#define __ID_T_TYPE     __U32_TYPE
> +#define __CLOCK_T_TYPE     __SLONGWORD_TYPE
> +#define __TIME_T_TYPE      __SQUAD_TYPE
> +#define __USECONDS_T_TYPE  __U32_TYPE
> +#define __SUSECONDS_T_TYPE __SQUAD_TYPE
> +#define __DADDR_T_TYPE     __S32_TYPE
> +#define __KEY_T_TYPE    __S32_TYPE
> +#define __CLOCKID_T_TYPE   __S32_TYPE
> +#define __TIMER_T_TYPE     void *
> +#define __BLKSIZE_T_TYPE   __S32_TYPE
> +#define __FSID_T_TYPE      struct { int __val[2]; }
> +#define __SSIZE_T_TYPE     __SWORD_TYPE
> +#define __SYSCALL_SLONG_TYPE __SQUAD_TYPE
> +#define __SYSCALL_ULONG_TYPE __UQUAD_TYPE
> +#define __CPU_MASK_TYPE    __UQUAD_TYPE

I see you fixed __CLOCK_T_TYPE, but you still have a number
of types that differ from the kernel ABI for no apparent reason.

Is this intentional?

       Arnd

Re: [RFC v3 07/23] RISC-V: Use 64-bit time_t and off_t for RV32 and RV64

[Alistair Francis]

On Wed, Jul 17, 2019 at 1:27 AM Arnd Bergmann <arnd@arndb.de> wrote:
>
> On Wed, Jul 17, 2019 at 2:11 AM Alistair Francis
> <alistair.francis@wdc.com> wrote:
>
> > +#define __DEV_T_TYPE    __UQUAD_TYPE
> > +#define __UID_T_TYPE    __U32_TYPE
> > +#define __GID_T_TYPE    __U32_TYPE
> > +#define __INO_T_TYPE    __UQUAD_TYPE
> > +#define __INO64_T_TYPE     __UQUAD_TYPE
> > +#define __MODE_T_TYPE      __U32_TYPE
> > +#define __NLINK_T_TYPE    __U32_TYPE
> > +#define __OFF_T_TYPE    __SQUAD_TYPE
> > +#define __OFF64_T_TYPE     __SQUAD_TYPE
> > +#define __PID_T_TYPE    __S32_TYPE
> > +#define __RLIM_T_TYPE      __UQUAD_TYPE
> > +#define __RLIM64_T_TYPE    __UQUAD_TYPE
> > +#define __BLKCNT_T_TYPE    __SQUAD_TYPE
> > +#define __BLKCNT64_T_TYPE  __SQUAD_TYPE
> > +#define __FSBLKCNT_T_TYPE  __UQUAD_TYPE
> > +#define __FSBLKCNT64_T_TYPE   __UQUAD_TYPE
> > +#define __FSFILCNT_T_TYPE  __UQUAD_TYPE
> > +#define __FSFILCNT64_T_TYPE   __UQUAD_TYPE
> > +#define __FSWORD_T_TYPE   __SWORD_TYPE
> > +#define __ID_T_TYPE     __U32_TYPE
> > +#define __CLOCK_T_TYPE     __SLONGWORD_TYPE
> > +#define __TIME_T_TYPE      __SQUAD_TYPE
> > +#define __USECONDS_T_TYPE  __U32_TYPE
> > +#define __SUSECONDS_T_TYPE __SQUAD_TYPE
> > +#define __DADDR_T_TYPE     __S32_TYPE
> > +#define __KEY_T_TYPE    __S32_TYPE
> > +#define __CLOCKID_T_TYPE   __S32_TYPE
> > +#define __TIMER_T_TYPE     void *
> > +#define __BLKSIZE_T_TYPE   __S32_TYPE
> > +#define __FSID_T_TYPE      struct { int __val[2]; }
> > +#define __SSIZE_T_TYPE     __SWORD_TYPE
> > +#define __SYSCALL_SLONG_TYPE __SQUAD_TYPE
> > +#define __SYSCALL_ULONG_TYPE __UQUAD_TYPE
> > +#define __CPU_MASK_TYPE    __UQUAD_TYPE
>
> I see you fixed __CLOCK_T_TYPE, but you still have a number
> of types that differ from the kernel ABI for no apparent reason.
>
> Is this intentional?

Yes, if I use the generic ones for RV32 I see build failures as some
of the structs don't align (I can't remember which ones now). So this
is required to build.

Alistair

>
>        Arnd

Re: [RFC v3 07/23] RISC-V: Use 64-bit time_t and off_t for RV32 and RV64

[Arnd Bergmann]

On Thu, Jul 18, 2019 at 12:43 AM Alistair Francis <alistair23@gmail.com> wrote:
> On Wed, Jul 17, 2019 at 1:27 AM Arnd Bergmann <arnd@arndb.de> wrote:
> > On Wed, Jul 17, 2019 at 2:11 AM Alistair Francis
> > <alistair.francis@wdc.com> wrote:
> >
> > > +#define __DEV_T_TYPE    __UQUAD_TYPE
> > > +#define __UID_T_TYPE    __U32_TYPE
> > > +#define __GID_T_TYPE    __U32_TYPE
> > > +#define __INO_T_TYPE    __UQUAD_TYPE
> > > +#define __INO64_T_TYPE     __UQUAD_TYPE
> > > +#define __MODE_T_TYPE      __U32_TYPE
> > > +#define __NLINK_T_TYPE    __U32_TYPE
> > > +#define __OFF_T_TYPE    __SQUAD_TYPE
> > > +#define __OFF64_T_TYPE     __SQUAD_TYPE
> > > +#define __PID_T_TYPE    __S32_TYPE
> > > +#define __RLIM_T_TYPE      __UQUAD_TYPE
> > > +#define __RLIM64_T_TYPE    __UQUAD_TYPE
> > > +#define __BLKCNT_T_TYPE    __SQUAD_TYPE
> > > +#define __BLKCNT64_T_TYPE  __SQUAD_TYPE
> > > +#define __FSBLKCNT_T_TYPE  __UQUAD_TYPE
> > > +#define __FSBLKCNT64_T_TYPE   __UQUAD_TYPE
> > > +#define __FSFILCNT_T_TYPE  __UQUAD_TYPE
> > > +#define __FSFILCNT64_T_TYPE   __UQUAD_TYPE
> > > +#define __FSWORD_T_TYPE   __SWORD_TYPE
> > > +#define __ID_T_TYPE     __U32_TYPE
> > > +#define __CLOCK_T_TYPE     __SLONGWORD_TYPE
> > > +#define __TIME_T_TYPE      __SQUAD_TYPE
> > > +#define __USECONDS_T_TYPE  __U32_TYPE
> > > +#define __SUSECONDS_T_TYPE __SQUAD_TYPE
> > > +#define __DADDR_T_TYPE     __S32_TYPE
> > > +#define __KEY_T_TYPE    __S32_TYPE
> > > +#define __CLOCKID_T_TYPE   __S32_TYPE
> > > +#define __TIMER_T_TYPE     void *
> > > +#define __BLKSIZE_T_TYPE   __S32_TYPE
> > > +#define __FSID_T_TYPE      struct { int __val[2]; }
> > > +#define __SSIZE_T_TYPE     __SWORD_TYPE
> > > +#define __SYSCALL_SLONG_TYPE __SQUAD_TYPE
> > > +#define __SYSCALL_ULONG_TYPE __UQUAD_TYPE
> > > +#define __CPU_MASK_TYPE    __UQUAD_TYPE
> >
> > I see you fixed __CLOCK_T_TYPE, but you still have a number
> > of types that differ from the kernel ABI for no apparent reason.
> >
> > Is this intentional?
>
> Yes, if I use the generic ones for RV32 I see build failures as some
> of the structs don't align (I can't remember which ones now). So this
> is required to build.

I think what it really means is that you have to fix up those build
failures by changing the broken code. Using mismatched types to
address build failures just replaces them with runtime failures but
does not result in a working systems.

    Arnd

Re: [RFC v3 07/23] RISC-V: Use 64-bit time_t and off_t for RV32 and RV64

[Alistair Francis]

On Thu, Jul 18, 2019 at 2:41 AM Arnd Bergmann <arnd@arndb.de> wrote:
>
> On Thu, Jul 18, 2019 at 12:43 AM Alistair Francis <alistair23@gmail.com> wrote:
> > On Wed, Jul 17, 2019 at 1:27 AM Arnd Bergmann <arnd@arndb.de> wrote:
> > > On Wed, Jul 17, 2019 at 2:11 AM Alistair Francis
> > > <alistair.francis@wdc.com> wrote:
> > >
> > > > +#define __DEV_T_TYPE    __UQUAD_TYPE
> > > > +#define __UID_T_TYPE    __U32_TYPE
> > > > +#define __GID_T_TYPE    __U32_TYPE
> > > > +#define __INO_T_TYPE    __UQUAD_TYPE
> > > > +#define __INO64_T_TYPE     __UQUAD_TYPE
> > > > +#define __MODE_T_TYPE      __U32_TYPE
> > > > +#define __NLINK_T_TYPE    __U32_TYPE
> > > > +#define __OFF_T_TYPE    __SQUAD_TYPE
> > > > +#define __OFF64_T_TYPE     __SQUAD_TYPE
> > > > +#define __PID_T_TYPE    __S32_TYPE
> > > > +#define __RLIM_T_TYPE      __UQUAD_TYPE
> > > > +#define __RLIM64_T_TYPE    __UQUAD_TYPE
> > > > +#define __BLKCNT_T_TYPE    __SQUAD_TYPE
> > > > +#define __BLKCNT64_T_TYPE  __SQUAD_TYPE
> > > > +#define __FSBLKCNT_T_TYPE  __UQUAD_TYPE
> > > > +#define __FSBLKCNT64_T_TYPE   __UQUAD_TYPE
> > > > +#define __FSFILCNT_T_TYPE  __UQUAD_TYPE
> > > > +#define __FSFILCNT64_T_TYPE   __UQUAD_TYPE
> > > > +#define __FSWORD_T_TYPE   __SWORD_TYPE
> > > > +#define __ID_T_TYPE     __U32_TYPE
> > > > +#define __CLOCK_T_TYPE     __SLONGWORD_TYPE
> > > > +#define __TIME_T_TYPE      __SQUAD_TYPE
> > > > +#define __USECONDS_T_TYPE  __U32_TYPE
> > > > +#define __SUSECONDS_T_TYPE __SQUAD_TYPE
> > > > +#define __DADDR_T_TYPE     __S32_TYPE
> > > > +#define __KEY_T_TYPE    __S32_TYPE
> > > > +#define __CLOCKID_T_TYPE   __S32_TYPE
> > > > +#define __TIMER_T_TYPE     void *
> > > > +#define __BLKSIZE_T_TYPE   __S32_TYPE
> > > > +#define __FSID_T_TYPE      struct { int __val[2]; }
> > > > +#define __SSIZE_T_TYPE     __SWORD_TYPE
> > > > +#define __SYSCALL_SLONG_TYPE __SQUAD_TYPE
> > > > +#define __SYSCALL_ULONG_TYPE __UQUAD_TYPE
> > > > +#define __CPU_MASK_TYPE    __UQUAD_TYPE
> > >
> > > I see you fixed __CLOCK_T_TYPE, but you still have a number
> > > of types that differ from the kernel ABI for no apparent reason.
> > >
> > > Is this intentional?
> >
> > Yes, if I use the generic ones for RV32 I see build failures as some
> > of the structs don't align (I can't remember which ones now). So this
> > is required to build.
>
> I think what it really means is that you have to fix up those build
> failures by changing the broken code. Using mismatched types to
> address build failures just replaces them with runtime failures but
> does not result in a working systems.

Good point, I have fixed the other issue. This is what I have now,
which matches the generic typesizes:

#define __DEV_T_TYPE    __UQUAD_TYPE
#define __UID_T_TYPE    __U32_TYPE
#define __GID_T_TYPE    __U32_TYPE
#define __INO_T_TYPE    __UQUAD_TYPE
#define __INO64_T_TYPE     __UQUAD_TYPE
#define __MODE_T_TYPE      __U32_TYPE
#define __NLINK_T_TYPE    __U32_TYPE
#define __OFF_T_TYPE    __SQUAD_TYPE
#define __OFF64_T_TYPE     __SQUAD_TYPE
#define __PID_T_TYPE    __S32_TYPE
#define __RLIM_T_TYPE      __UQUAD_TYPE
#define __RLIM64_T_TYPE    __UQUAD_TYPE
#define __BLKCNT_T_TYPE    __SQUAD_TYPE
#define __BLKCNT64_T_TYPE  __SQUAD_TYPE
#define __FSBLKCNT_T_TYPE  __UQUAD_TYPE
#define __FSBLKCNT64_T_TYPE   __UQUAD_TYPE
#define __FSFILCNT_T_TYPE  __UQUAD_TYPE
#define __FSFILCNT64_T_TYPE   __UQUAD_TYPE
#define __FSWORD_T_TYPE   __SWORD_TYPE
#define __ID_T_TYPE     __U32_TYPE
#define __CLOCK_T_TYPE     __SLONGWORD_TYPE
#define __TIME_T_TYPE      __SQUAD_TYPE
#define __USECONDS_T_TYPE  __U32_TYPE
#define __SUSECONDS_T_TYPE __SQUAD_TYPE
#define __DADDR_T_TYPE     __S32_TYPE
#define __KEY_T_TYPE    __S32_TYPE
#define __CLOCKID_T_TYPE   __S32_TYPE
#define __TIMER_T_TYPE     void *
#define __BLKSIZE_T_TYPE   __S32_TYPE
#define __FSID_T_TYPE      struct { int __val[2]; }
#define __SSIZE_T_TYPE     __SWORD_TYPE
#define __SYSCALL_SLONG_TYPE __SLONGWORD_TYPE
#define __SYSCALL_ULONG_TYPE __ULONGWORD_TYPE
#define __CPU_MASK_TYPE    __ULONGWORD_TYPE

Alistair

>
>     Arnd

Re: [RFC v3 07/23] RISC-V: Use 64-bit time_t and off_t for RV32 and RV64

[Arnd Bergmann]

On Thu, Jul 18, 2019 at 7:36 PM Alistair Francis <alistair23@gmail.com> wrote:
> On Thu, Jul 18, 2019 at 2:41 AM Arnd Bergmann <arnd@arndb.de> wrote:
> > On Thu, Jul 18, 2019 at 12:43 AM Alistair Francis <alistair23@gmail.com> wrote:
> > >
> > > Yes, if I use the generic ones for RV32 I see build failures as some
> > > of the structs don't align (I can't remember which ones now). So this
> > > is required to build.
> >
> > I think what it really means is that you have to fix up those build
> > failures by changing the broken code. Using mismatched types to
> > address build failures just replaces them with runtime failures but
> > does not result in a working systems.
>
> Good point, I have fixed the other issue. This is what I have now,
> which matches the generic typesizes:

Ok, just to confirm, I'm looking at the difference between your
version and sysdeps/unix/sysv/linux/generic/bits/typesizes.h:

--- sysdeps/unix/sysv/linux/generic/bits/typesizes.h 2019-06-25
13:02:42.301241279 +0200
+++ typesize.h +0200
...
 #define __DEV_T_TYPE __UQUAD_TYPE
 #define __UID_T_TYPE __U32_TYPE
 #define __GID_T_TYPE __U32_TYPE
-#define __INO_T_TYPE __ULONGWORD_TYPE
+#define __INO_T_TYPE    __UQUAD_TYPE
 #define __INO64_T_TYPE __UQUAD_TYPE
 #define __MODE_T_TYPE __U32_TYPE
 #define __NLINK_T_TYPE __U32_TYPE
-#define __OFF_T_TYPE __SLONGWORD_TYPE
+#define __OFF_T_TYPE    __SQUAD_TYPE
 #define __OFF64_T_TYPE __SQUAD_TYPE
 #define __PID_T_TYPE __S32_TYPE
-#define __RLIM_T_TYPE __ULONGWORD_TYPE
+#define __RLIM_T_TYPE      __UQUAD_TYPE
 #define __RLIM64_T_TYPE __UQUAD_TYPE
-#define __BLKCNT_T_TYPE __SLONGWORD_TYPE
+#define __BLKCNT_T_TYPE    __SQUAD_TYPE
 #define __BLKCNT64_T_TYPE __SQUAD_TYPE
-#define __FSBLKCNT_T_TYPE __ULONGWORD_TYPE
+#define __FSBLKCNT_T_TYPE  __UQUAD_TYPE
 #define __FSBLKCNT64_T_TYPE __UQUAD_TYPE
-#define __FSFILCNT_T_TYPE __ULONGWORD_TYPE
+#define __FSFILCNT_T_TYPE  __UQUAD_TYPE
 #define __FSFILCNT64_T_TYPE __UQUAD_TYPE
 #define __FSWORD_T_TYPE __SWORD_TYPE
 #define __ID_T_TYPE __U32_TYPE
 #define __CLOCK_T_TYPE __SLONGWORD_TYPE
-#define __TIME_T_TYPE __SLONGWORD_TYPE
+#define __TIME_T_TYPE      __SQUAD_TYPE
 #define __USECONDS_T_TYPE __U32_TYPE
-#define __SUSECONDS_T_TYPE __SLONGWORD_TYPE
+#define __SUSECONDS_T_TYPE __SQUAD_TYPE
 #define __DADDR_T_TYPE __S32_TYPE
 #define __KEY_T_TYPE __S32_TYPE
 #define __CLOCKID_T_TYPE __S32_TYPE
...

This all seems reasonable, but it's worth spelling out the decisions
that went into it:

-  All the !__USE_FILE_OFFSET64  types (__off_t, __ino_t, __rlim_t, ...) are
  changed to match the 64-bit replacements. I would have preferred to
  completely leave them out and always define __USE_FILE_OFFSET64,
  but I don't think anyone else liked that idea, so that's ok.

- __time_t is defined to 64 bit, but no __time64_t is added. This makes sense
  as we don't have the time64 support for other 32-bit architectures yet, and
  it will be easy to change when that happens.

- __suseconds_t is 64-bit. This matches what we use the kerne ABI for the
  few drivers that are relying on 'struct timeval' input arguments in
ioctl, as well as
  the adjtimex system call. It means that timeval has to be defined without the
  padding, unlike timespec, which needs padding.

      Arnd

Re: [RFC v3 07/23] RISC-V: Use 64-bit time_t and off_t for RV32 and RV64

[Alistair Francis]

On Thu, Jul 18, 2019 at 11:44 PM Arnd Bergmann <arnd@arndb.de> wrote:
>
> On Thu, Jul 18, 2019 at 7:36 PM Alistair Francis <alistair23@gmail.com> wrote:
> > On Thu, Jul 18, 2019 at 2:41 AM Arnd Bergmann <arnd@arndb.de> wrote:
> > > On Thu, Jul 18, 2019 at 12:43 AM Alistair Francis <alistair23@gmail.com> wrote:
> > > >
> > > > Yes, if I use the generic ones for RV32 I see build failures as some
> > > > of the structs don't align (I can't remember which ones now). So this
> > > > is required to build.
> > >
> > > I think what it really means is that you have to fix up those build
> > > failures by changing the broken code. Using mismatched types to
> > > address build failures just replaces them with runtime failures but
> > > does not result in a working systems.
> >
> > Good point, I have fixed the other issue. This is what I have now,
> > which matches the generic typesizes:
>
> Ok, just to confirm, I'm looking at the difference between your
> version and sysdeps/unix/sysv/linux/generic/bits/typesizes.h:
>
> --- sysdeps/unix/sysv/linux/generic/bits/typesizes.h 2019-06-25
> 13:02:42.301241279 +0200
> +++ typesize.h +0200
> ...
>  #define __DEV_T_TYPE __UQUAD_TYPE
>  #define __UID_T_TYPE __U32_TYPE
>  #define __GID_T_TYPE __U32_TYPE
> -#define __INO_T_TYPE __ULONGWORD_TYPE
> +#define __INO_T_TYPE    __UQUAD_TYPE
>  #define __INO64_T_TYPE __UQUAD_TYPE
>  #define __MODE_T_TYPE __U32_TYPE
>  #define __NLINK_T_TYPE __U32_TYPE
> -#define __OFF_T_TYPE __SLONGWORD_TYPE
> +#define __OFF_T_TYPE    __SQUAD_TYPE
>  #define __OFF64_T_TYPE __SQUAD_TYPE
>  #define __PID_T_TYPE __S32_TYPE
> -#define __RLIM_T_TYPE __ULONGWORD_TYPE
> +#define __RLIM_T_TYPE      __UQUAD_TYPE
>  #define __RLIM64_T_TYPE __UQUAD_TYPE
> -#define __BLKCNT_T_TYPE __SLONGWORD_TYPE
> +#define __BLKCNT_T_TYPE    __SQUAD_TYPE
>  #define __BLKCNT64_T_TYPE __SQUAD_TYPE
> -#define __FSBLKCNT_T_TYPE __ULONGWORD_TYPE
> +#define __FSBLKCNT_T_TYPE  __UQUAD_TYPE
>  #define __FSBLKCNT64_T_TYPE __UQUAD_TYPE
> -#define __FSFILCNT_T_TYPE __ULONGWORD_TYPE
> +#define __FSFILCNT_T_TYPE  __UQUAD_TYPE
>  #define __FSFILCNT64_T_TYPE __UQUAD_TYPE
>  #define __FSWORD_T_TYPE __SWORD_TYPE
>  #define __ID_T_TYPE __U32_TYPE
>  #define __CLOCK_T_TYPE __SLONGWORD_TYPE
> -#define __TIME_T_TYPE __SLONGWORD_TYPE
> +#define __TIME_T_TYPE      __SQUAD_TYPE
>  #define __USECONDS_T_TYPE __U32_TYPE
> -#define __SUSECONDS_T_TYPE __SLONGWORD_TYPE
> +#define __SUSECONDS_T_TYPE __SQUAD_TYPE
>  #define __DADDR_T_TYPE __S32_TYPE
>  #define __KEY_T_TYPE __S32_TYPE
>  #define __CLOCKID_T_TYPE __S32_TYPE
> ...
>
> This all seems reasonable, but it's worth spelling out the decisions
> that went into it:
>
> -  All the !__USE_FILE_OFFSET64  types (__off_t, __ino_t, __rlim_t, ...) are
>   changed to match the 64-bit replacements. I would have preferred to
>   completely leave them out and always define __USE_FILE_OFFSET64,
>   but I don't think anyone else liked that idea, so that's ok.
>
> - __time_t is defined to 64 bit, but no __time64_t is added. This makes sense
>   as we don't have the time64 support for other 32-bit architectures yet, and
>   it will be easy to change when that happens.
>
> - __suseconds_t is 64-bit. This matches what we use the kerne ABI for the
>   few drivers that are relying on 'struct timeval' input arguments in
> ioctl, as well as
>   the adjtimex system call. It means that timeval has to be defined without the
>   padding, unlike timespec, which needs padding.

Great! I have added all of this into the commit message (with some changes)

Alistair

>
>       Arnd

[RFC v3 20/23] RISC-V: Build Infastructure for the 32-bit

[Alistair Francis]

From: Zong Li <zongbox@gmail.com>

This patch lays out the top-level orginazition of the RISC-V 32-bit port. It
contains all the Implies files as well as various other fragments of
build infastructure for the RISC-V 32-bit port.

2018-11-29  Zong Li  <zong@andestech.com>

	* sysdeps/riscv/rv32/Implies-after: New file.
	* sysdeps/riscv/rv32/rvd/Implies: Likewise.
	* sysdeps/riscv/rv32/rvf/Implies: Likewise.
	* sysdeps/unix/sysv/linux/riscv/rv32/Implies: Likewise.
	* sysdeps/unix/sysv/linux/riscv/Makefile: Support rv32.
	* sysdeps/unix/sysv/linux/riscv/configure: Likewise.
	* sysdeps/unix/sysv/linux/riscv/configure.ac: Likewise.
	* sysdeps/unix/sysv/linux/riscv/shlib-versions: Likewise.
	* sysdeps/riscv/preconfigure: Likewise.
---
 ChangeLog                                    |  9 +++++
 sysdeps/riscv/preconfigure                   |  6 +--
 sysdeps/riscv/rv32/Implies-after             |  1 +
 sysdeps/riscv/rv32/rvd/Implies               |  3 ++
 sysdeps/riscv/rv32/rvf/Implies               |  1 +
 sysdeps/unix/sysv/linux/riscv/Makefile       |  4 +-
 sysdeps/unix/sysv/linux/riscv/configure      | 39 ++++++++++++++++++++
 sysdeps/unix/sysv/linux/riscv/configure.ac   |  8 ++++
 sysdeps/unix/sysv/linux/riscv/rv32/Implies   |  3 ++
 sysdeps/unix/sysv/linux/riscv/shlib-versions | 10 ++++-
 10 files changed, 76 insertions(+), 8 deletions(-)
 create mode 100644 sysdeps/riscv/rv32/Implies-after
 create mode 100644 sysdeps/riscv/rv32/rvd/Implies
 create mode 100644 sysdeps/riscv/rv32/rvf/Implies
 create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/Implies

diff --git a/ChangeLog b/ChangeLog
index c31e2dff87..131c1063fc 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1105,6 +1105,15 @@
 	* sysdeps/unix/sysv/linux/riscv/rv32/libutil.abilist: Likewise.
 	* sysdeps/unix/sysv/linux/riscv/rv32/libBrokenLocale.abilist:
 	Likewise.
+	* sysdeps/riscv/rv32/Implies-after: New file.
+	* sysdeps/riscv/rv32/rvd/Implies: Likewise.
+	* sysdeps/riscv/rv32/rvf/Implies: Likewise.
+	* sysdeps/unix/sysv/linux/riscv/rv32/Implies: Likewise.
+	* sysdeps/unix/sysv/linux/riscv/Makefile: Support rv32.
+	* sysdeps/unix/sysv/linux/riscv/configure: Likewise.
+	* sysdeps/unix/sysv/linux/riscv/configure.ac: Likewise.
+	* sysdeps/unix/sysv/linux/riscv/shlib-versions: Likewise.
+	* sysdeps/riscv/preconfigure: Likewise.
 
 2019-06-20  Dmitry V. Levin  <ldv@altlinux.org>
 	    Florian Weimer  <fweimer@redhat.com>
diff --git a/sysdeps/riscv/preconfigure b/sysdeps/riscv/preconfigure
index d9adb31b64..1ab5d20f0e 100644
--- a/sysdeps/riscv/preconfigure
+++ b/sysdeps/riscv/preconfigure
@@ -6,11 +6,7 @@ riscv*)
     atomic=`$CC $CFLAGS $CPPFLAGS -E -dM -xc /dev/null | grep '#define __riscv_atomic' | cut -d' ' -f2`
 
     case "$xlen" in
-    32)
-	echo "glibc does not yet support 32-bit systems" >&2
-	exit 1
-	;;
-    64)
+    64 | 32)
 	;;
     *)
 	echo "Unable to determine XLEN" >&2
diff --git a/sysdeps/riscv/rv32/Implies-after b/sysdeps/riscv/rv32/Implies-after
new file mode 100644
index 0000000000..39a34c5f57
--- /dev/null
+++ b/sysdeps/riscv/rv32/Implies-after
@@ -0,0 +1 @@
+wordsize-32
diff --git a/sysdeps/riscv/rv32/rvd/Implies b/sysdeps/riscv/rv32/rvd/Implies
new file mode 100644
index 0000000000..1151214e8f
--- /dev/null
+++ b/sysdeps/riscv/rv32/rvd/Implies
@@ -0,0 +1,3 @@
+riscv/rv32/rvf
+riscv/rvd
+riscv/rvf
diff --git a/sysdeps/riscv/rv32/rvf/Implies b/sysdeps/riscv/rv32/rvf/Implies
new file mode 100644
index 0000000000..66c401443b
--- /dev/null
+++ b/sysdeps/riscv/rv32/rvf/Implies
@@ -0,0 +1 @@
+riscv/rvf
diff --git a/sysdeps/unix/sysv/linux/riscv/Makefile b/sysdeps/unix/sysv/linux/riscv/Makefile
index b47858769f..bf9c24ad77 100644
--- a/sysdeps/unix/sysv/linux/riscv/Makefile
+++ b/sysdeps/unix/sysv/linux/riscv/Makefile
@@ -15,11 +15,13 @@ ifeq ($(subdir),stdlib)
 gen-as-const-headers += ucontext_i.sym
 endif
 
-abi-variants := lp64 lp64d
+abi-variants := ilp32 ilp32d lp64 lp64d
 
 ifeq (,$(filter $(default-abi),$(abi-variants)))
 $(error Unknown ABI $(default-abi), must be one of $(abi-variants))
 endif
 
+abi-ilp32-condition   := !defined __LP64__ && defined __riscv_float_abi_soft
+abi-ilp32d-condition  := !defined __LP64__ && defined __riscv_float_abi_double
 abi-lp64-condition    := defined __LP64__ && defined __riscv_float_abi_soft
 abi-lp64d-condition   := defined __LP64__ && defined __riscv_float_abi_double
diff --git a/sysdeps/unix/sysv/linux/riscv/configure b/sysdeps/unix/sysv/linux/riscv/configure
index 3018ca8f1b..2b3c77f18c 100755
--- a/sysdeps/unix/sysv/linux/riscv/configure
+++ b/sysdeps/unix/sysv/linux/riscv/configure
@@ -147,6 +147,17 @@ if (eval "$ac_cpp conftest.$ac_ext") 2>&5 |
 fi
 rm -f conftest*
 
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h.  */
+__SIZEOF_INT__ __SIZEOF_LONG__ __SIZEOF_POINTER__
+
+_ACEOF
+if (eval "$ac_cpp conftest.$ac_ext") 2>&5 |
+  $EGREP "4 4 4" >/dev/null 2>&1; then :
+  libc_cv_riscv_int_abi=ilp32
+fi
+rm -f conftest*
+
 if test $libc_cv_riscv_int_abi = no; then
   as_fn_error $? "Unable to determine integer ABI" "$LINENO" 5
 fi
@@ -214,6 +225,34 @@ case "$prefix" in
   ;;
 esac
   ;;
+ilp32-riscv/rv32/*)
+  test -n "$libc_cv_slibdir" ||
+case "$prefix" in
+/usr | /usr/)
+  libc_cv_slibdir='/lib32/ilp32'
+  libc_cv_rtlddir='/lib'
+  if test "$libdir" = '${exec_prefix}/lib'; then
+    libdir='${exec_prefix}/lib32/ilp32';
+    # Locale data can be shared between 32-bit and 64-bit libraries.
+    libc_cv_complocaledir='${exec_prefix}/lib/locale'
+  fi
+  ;;
+esac
+  ;;
+ilp32d-riscv/rv32/*)
+  test -n "$libc_cv_slibdir" ||
+case "$prefix" in
+/usr | /usr/)
+  libc_cv_slibdir='/lib32/ilp32d'
+  libc_cv_rtlddir='/lib'
+  if test "$libdir" = '${exec_prefix}/lib'; then
+    libdir='${exec_prefix}/lib32/ilp32d';
+    # Locale data can be shared between 32-bit and 64-bit libraries.
+    libc_cv_complocaledir='${exec_prefix}/lib/locale'
+  fi
+  ;;
+esac
+  ;;
 esac
 
 ldd_rewrite_script=sysdeps/unix/sysv/linux/riscv/ldd-rewrite.sed
diff --git a/sysdeps/unix/sysv/linux/riscv/configure.ac b/sysdeps/unix/sysv/linux/riscv/configure.ac
index d4819931ca..710d46afcd 100644
--- a/sysdeps/unix/sysv/linux/riscv/configure.ac
+++ b/sysdeps/unix/sysv/linux/riscv/configure.ac
@@ -7,6 +7,8 @@ arch_minimum_kernel=4.15.0
 libc_cv_riscv_int_abi=no
 AC_EGREP_CPP(4 8 8, [__SIZEOF_INT__ __SIZEOF_LONG__ __SIZEOF_POINTER__
   ], libc_cv_riscv_int_abi=lp64)
+AC_EGREP_CPP(4 4 4, [__SIZEOF_INT__ __SIZEOF_LONG__ __SIZEOF_POINTER__
+  ], libc_cv_riscv_int_abi=ilp32)
 if test $libc_cv_riscv_int_abi = no; then
   AC_MSG_ERROR([Unable to determine integer ABI])
 fi
@@ -33,6 +35,12 @@ lp64-riscv/rv64/*)
 lp64d-riscv/rv64/*)
   LIBC_SLIBDIR_RTLDDIR([lib64/lp64d], [lib])
   ;;
+ilp32-riscv/rv32/*)
+  LIBC_SLIBDIR_RTLDDIR([lib32/ilp32], [lib])
+  ;;
+ilp32d-riscv/rv32/*)
+  LIBC_SLIBDIR_RTLDDIR([lib32/ilp32d], [lib])
+  ;;
 esac
 
 ldd_rewrite_script=sysdeps/unix/sysv/linux/riscv/ldd-rewrite.sed
diff --git a/sysdeps/unix/sysv/linux/riscv/rv32/Implies b/sysdeps/unix/sysv/linux/riscv/rv32/Implies
new file mode 100644
index 0000000000..8b7deb33cd
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/riscv/rv32/Implies
@@ -0,0 +1,3 @@
+unix/sysv/linux/riscv
+unix/sysv/linux/generic/wordsize-32
+unix/sysv/linux/generic
diff --git a/sysdeps/unix/sysv/linux/riscv/shlib-versions b/sysdeps/unix/sysv/linux/riscv/shlib-versions
index 98c9b29cc4..a767d0f2ef 100644
--- a/sysdeps/unix/sysv/linux/riscv/shlib-versions
+++ b/sysdeps/unix/sysv/linux/riscv/shlib-versions
@@ -1,9 +1,15 @@
-DEFAULT		GLIBC_2.27
-
 %if RISCV_ABI_XLEN == 64 && RISCV_ABI_FLEN == 64
+DEFAULT		GLIBC_2.27
 ld=ld-linux-riscv64-lp64d.so.1
 %elif RISCV_ABI_XLEN == 64 && RISCV_ABI_FLEN == 0
+DEFAULT		GLIBC_2.27
 ld=ld-linux-riscv64-lp64.so.1
+%elif RISCV_ABI_XLEN == 32 && RISCV_ABI_FLEN == 64
+DEFAULT		GLIBC_2.29
+ld=ld-linux-riscv32-ilp32d.so.1
+%elif RISCV_ABI_XLEN == 32 && RISCV_ABI_FLEN == 0
+DEFAULT		GLIBC_2.29
+ld=ld-linux-riscv32-ilp32.so.1
 %else
 %error cannot determine ABI
 %endif
-- 
2.22.0

Re: [RFC v3 00/23] RISC-V glibc port for the 32-bit

[Alistair Francis]

On Tue, Jul 16, 2019 at 5:11 PM Alistair Francis
<alistair.francis@wdc.com> wrote:
>
> This patch set contains the glibc port for the 32-bit RISC-V.
>
> This is based on all of the work that Zong Li has done [1].
>
> Unfortunately the Linux ABI has changed since Zong Li's latest
> submission. The RISC-V 32-bit (RV32) ABI no longer defines
> __ARCH_WANT_TIME32_SYSCALLS which means there are no 32-bit versions of
> time_t, off_t or any struct resource or system calls that use them.
>
> To mitigate this I have set the RV32 port to use 64-bit time_t and off_t
> (as is done in x86-32) and also changed the syscall imvocation to handle
> the missing syscalls.
>
> This series applies ontop of Lukasz's __clock_settime64 iomplementation [2].
>
> The ChangeLog will need to be updated, so don't worry too much
> about that.
>
> All testing has been done on a 5.1 kernel (it won't work on earlier
> kernels).
>
> Importantly glibc is not the only project that needs to be updated to
> work with a 32-bit kernel that doesn't define
> __ARCH_WANT_TIME32_SYSCALLS. So far OpenSSL, busybox and systemd all
> need patches to work with RV32. My hope here is that we can get a
> reasonable agreement on how to handle this in glibc and then use that to
> update other projects. This is why I would still really like feedback
> even though this won't go into the next release.
>
> Feedback on this series is very welcome!
>
> 1: https://sourceware.org/ml/libc-alpha/2018-07/msg00892.html
> 2: https://sourceware.org/ml/libc-alpha/2019-05/msg00661.html
>
> The latest version of my work can be found here: https://github.com/alistair23/glibc/tree/alistair/rv32.next
>
> This specific version can be found here: https://github.com/alistair23/glibc/tree/alistair/rv32.rfc3

Hey,

I thought I would document what the ongoing discussions/problems/todos
are for the RV32 work. This is mostly for my own use so I don't have
to remember everything but it might also be useful for others.

These are listed roughly in order of importance/merge order.

 1. Lukasz's y2038 clock_settime patches need to be reviewed and
merged. RV32 relies on these patches but these patches also introduce
the __ASSUME_TIME64_SYSCALLS plumbing which we use thought the RV32
port.
 2. The timespec_get, clock_gettime, gettimeofday and nanosleep
patches need to be finished based on the final state of Lukasz's
patches and the current feedback on list.
 3. The waitid syscall implementation needs to be reviewed and merged.
 4. The define __NR_* as __NR_*_time64/64 for 32-bit needs to be
agreed upon (or something else)
 5. The vdso calls need to be sorted out.
 6. RV32 needs to be reviewed and merged.

There is probably other things, but that's rougly what has to happen.

Alistair

>
> RFC v3:
>  - Remove all "Hack" patches
>  - Incorporate upstream comments
>  - Ensure we don't break RV64
>  - Lot's more testing and fixes
> RFC v2:
>  - Add Lukasz's patches
>  - Update the non HACK syscalls after feedback
>  - define __ASSUME_TIME64_SYSCALLS and __ASSUME_RLIM64_SYSCALLS
>  - Remove lockf64.c
>  - Other smaller changes from RFC v1
>
> --END---
>
> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
> ---
>  __COVER__ | 0
>  1 file changed, 0 insertions(+), 0 deletions(-)
>  create mode 100644 __COVER__
>
> diff --git a/__COVER__ b/__COVER__
> new file mode 100644
> index 0000000000..e69de29bb2
> --
> 2.22.0
>
>
>
> Alistair Francis (13):
>   sysdeps/nanosleep: Use clock_nanosleep_time64 if avaliable
>   sysdeps/gettimeofday: Use clock_gettime64 if avaliable
>   sysdeps/wait: Use waitid if avaliable
>   sysdeps/clock_gettime: Use clock_gettime64 if avaliable
>   sysdeps/timespec_get: Use clock_gettime64 if avaliable
>   RISC-V: Use 64-bit time_t and off_t for RV32 and RV64
>   RISC-V: define __NR_futex as __NR_futex_time64 for 32-bit
>   RISC-V: define __NR_* as __NR_*_time64/64 for 32-bit
>   RISC-V: define __NR_clock_getres as __NR_*_time64 for 32-bit
>   RISC-V: define __vdso_clock_getres as __vdso_clock_getres_time64 for
>     32-bit
>   RISC-V: define __vdso_clock_gettime as __vdso_clock_gettime64 for
>     32-bit
>   RISC-V: Use 64-bit timespec in clock_gettime vdso calls
>   RISC-V: Use 64-bit vdso syscalls
>
> Zong Li (10):
>   Documentation for the RISC-V 32-bit port
>   RISC-V: Support dynamic loader for the 32-bit
>   RISC-V: Add path of library directories for the 32-bit
>   RISC-V: The ABI implementation for the 32-bit
>   RISC-V: Hard float support for the 32 bit
>   RISC-V: Regenerate ULPs of RISC-V
>   RISC-V: Add ABI lists
>   RISC-V: Build Infastructure for the 32-bit
>   RISC-V: Fix llrint and llround missing exceptions on RV32
>   Add RISC-V 32-bit target to build-many-glibcs.py
>
>  ChangeLog                                     |  113 +
>  NEWS                                          |    6 +
>  README                                        |    1 +
>  nptl/pthread_mutex_timedlock.c                |    7 +
>  nptl/thrd_sleep.c                             |   41 +-
>  scripts/build-many-glibcs.py                  |   15 +
>  sysdeps/riscv/bits/wordsize.h                 |    4 +-
>  sysdeps/riscv/nofpu/libm-test-ulps            |   16 +-
>  sysdeps/riscv/nptl/bits/pthreadtypes-arch.h   |   25 +-
>  sysdeps/riscv/preconfigure                    |    6 +-
>  sysdeps/riscv/rv32/Implies-after              |    1 +
>  .../riscv/rv32/fix-fp-int-convert-overflow.h  |   38 +
>  sysdeps/riscv/rv32/rvd/Implies                |    3 +
>  sysdeps/riscv/rv32/rvd/s_lrint.c              |   31 +
>  sysdeps/riscv/rv32/rvd/s_lround.c             |   31 +
>  sysdeps/riscv/rv32/rvf/Implies                |    1 +
>  sysdeps/riscv/rv32/rvf/s_lrintf.c             |   31 +
>  sysdeps/riscv/rv32/rvf/s_lroundf.c            |   31 +
>  sysdeps/riscv/{rv64 => }/rvd/libm-test-ulps   |   56 +-
>  .../riscv/{rv64 => }/rvd/libm-test-ulps-name  |    0
>  sysdeps/riscv/sfp-machine.h                   |   27 +-
>  sysdeps/riscv/sys/asm.h                       |    5 +-
>  sysdeps/unix/sysv/linux/clock_gettime.c       |   30 +
>  sysdeps/unix/sysv/linux/clock_nanosleep.c     |   42 +-
>  sysdeps/unix/sysv/linux/gettimeofday.c        |   28 +
>  sysdeps/unix/sysv/linux/nanosleep.c           |   50 +
>  sysdeps/unix/sysv/linux/nanosleep_nocancel.c  |   11 +
>  sysdeps/unix/sysv/linux/riscv/Makefile        |    4 +-
>  .../unix/sysv/linux/riscv/bits/environments.h |   85 +
>  sysdeps/unix/sysv/linux/riscv/bits/time64.h   |   36 +
>  sysdeps/unix/sysv/linux/riscv/bits/timesize.h |   22 +
>  .../unix/sysv/linux/riscv/bits/typesizes.h    |   84 +
>  sysdeps/unix/sysv/linux/riscv/configure       |   39 +
>  sysdeps/unix/sysv/linux/riscv/configure.ac    |    8 +
>  sysdeps/unix/sysv/linux/riscv/dl-cache.h      |   17 +-
>  sysdeps/unix/sysv/linux/riscv/init-first.c    |    6 +-
>  sysdeps/unix/sysv/linux/riscv/ldconfig.h      |    2 +-
>  sysdeps/unix/sysv/linux/riscv/libc-vdso.h     |    2 +-
>  sysdeps/unix/sysv/linux/riscv/rv32/Implies    |    3 +
>  .../unix/sysv/linux/riscv/rv32/c++-types.data |   67 +
>  .../sysv/linux/riscv/rv32/jmp_buf-macros.h    |   53 +
>  sysdeps/unix/sysv/linux/riscv/rv32/ld.abilist |    9 +
>  .../linux/riscv/rv32/libBrokenLocale.abilist  |    1 +
>  .../unix/sysv/linux/riscv/rv32/libanl.abilist |    4 +
>  .../unix/sysv/linux/riscv/rv32/libc.abilist   | 2101 +++++++++++++++++
>  .../sysv/linux/riscv/rv32/libcrypt.abilist    |    2 +
>  .../unix/sysv/linux/riscv/rv32/libdl.abilist  |    9 +
>  .../unix/sysv/linux/riscv/rv32/libm.abilist   | 1021 ++++++++
>  .../sysv/linux/riscv/rv32/libpthread.abilist  |  235 ++
>  .../sysv/linux/riscv/rv32/libresolv.abilist   |   79 +
>  .../unix/sysv/linux/riscv/rv32/librt.abilist  |   35 +
>  .../linux/riscv/rv32/libthread_db.abilist     |   40 +
>  .../sysv/linux/riscv/rv32/libutil.abilist     |    6 +
>  sysdeps/unix/sysv/linux/riscv/shlib-versions  |   10 +-
>  sysdeps/unix/sysv/linux/riscv/sysdep.h        |   61 +
>  sysdeps/unix/sysv/linux/timespec_get.c        |   37 +-
>  sysdeps/unix/sysv/linux/wait.c                |   39 +-
>  sysdeps/unix/sysv/linux/waitpid.c             |   46 +
>  sysdeps/unix/sysv/linux/waitpid_nocancel.c    |   45 +
>  59 files changed, 4796 insertions(+), 62 deletions(-)
>  create mode 100644 sysdeps/riscv/rv32/Implies-after
>  create mode 100644 sysdeps/riscv/rv32/fix-fp-int-convert-overflow.h
>  create mode 100644 sysdeps/riscv/rv32/rvd/Implies
>  create mode 100644 sysdeps/riscv/rv32/rvd/s_lrint.c
>  create mode 100644 sysdeps/riscv/rv32/rvd/s_lround.c
>  create mode 100644 sysdeps/riscv/rv32/rvf/Implies
>  create mode 100644 sysdeps/riscv/rv32/rvf/s_lrintf.c
>  create mode 100644 sysdeps/riscv/rv32/rvf/s_lroundf.c
>  rename sysdeps/riscv/{rv64 => }/rvd/libm-test-ulps (98%)
>  rename sysdeps/riscv/{rv64 => }/rvd/libm-test-ulps-name (100%)
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/bits/environments.h
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/bits/time64.h
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/bits/timesize.h
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/bits/typesizes.h
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/Implies
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/c++-types.data
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/jmp_buf-macros.h
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/ld.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libBrokenLocale.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libanl.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libc.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libcrypt.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libdl.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libm.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libpthread.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libresolv.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/librt.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libthread_db.abilist
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/rv32/libutil.abilist
>
> --
> 2.22.0
>