Fix Arm __ASSUME_COPY_FILE_RANGE (bug 23915) [committed]

Fix Arm __ASSUME_COPY_FILE_RANGE (bug 23915) [committed]

[Joseph Myers]

The generic kernel-features.h defines __ASSUME_COPY_FILE_RANGE for 4.5
and later kernels.  However, for 32-bit Arm binaries running on 64-bit
Arm kernels, the syscall was only wired up in the 4.7 kernel, although
the 32-bit Arm kernel had the syscall from 4.5 onwards.  This patch
corrects the Arm kernel-features.h to undefine the macro for
configured minimum kernel versions before 4.7.

Tested (compilation only) with a build-many-glibcs.py build for
arm-linux-gnueabi.  Committed.

2018-11-23  Joseph Myers  <joseph@codesourcery.com>

	[BZ #23915]
	* sysdeps/unix/sysv/linux/arm/kernel-features.h
	[__LINUX_KERNEL_VERSION < 0x040700] (__ASSUME_COPY_FILE_RANGE):
	Undefine.

diff --git a/sysdeps/unix/sysv/linux/arm/kernel-features.h b/sysdeps/unix/sysv/linux/arm/kernel-features.h
index c6e745227c..e4ca98bfea 100644
--- a/sysdeps/unix/sysv/linux/arm/kernel-features.h
+++ b/sysdeps/unix/sysv/linux/arm/kernel-features.h
@@ -40,11 +40,12 @@
 #define __ASSUME_RECV_SYSCALL   1
 #define __ASSUME_SEND_SYSCALL	1
 
-/* Support for the mlock2 syscall was added to the compat syscall
-   table for 64-bit kernels in 4.7, although present in 32-bit kernels
-   from 4.4.  */
+/* Support for the mlock2 and copy_file_range syscalls was added to
+   the compat syscall table for 64-bit kernels in 4.7, although
+   present in 32-bit kernels from 4.4 and 4.5 respectively.  */
 #if __LINUX_KERNEL_VERSION < 0x040700
 # undef __ASSUME_MLOCK2
+# undef __ASSUME_COPY_FILE_RANGE
 #endif
 
 #undef __ASSUME_CLONE_DEFAULT

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Fix Arm __ASSUME_COPY_FILE_RANGE (bug 23915) [committed]

[Florian Weimer]

* Joseph Myers:

> 2018-11-23  Joseph Myers  <joseph@codesourcery.com>
>
> 	[BZ #23915]
> 	* sysdeps/unix/sysv/linux/arm/kernel-features.h
> 	[__LINUX_KERNEL_VERSION < 0x040700] (__ASSUME_COPY_FILE_RANGE):
> 	Undefine.

So I feel kind of bad about this—I looked for such issues after the
first problems cropped up and just didn't seem them in the kernel source
tree.

Assuming that I'm going to implement a few system call wrappers in the
future, how can I do a better job here?

Thanks,
Florian

Re: Fix Arm __ASSUME_COPY_FILE_RANGE (bug 23915) [committed]

[Joseph Myers]

[-- Attachment #1: Type: text/plain, Size: 1582 bytes --]

On Mon, 26 Nov 2018, Florian Weimer wrote:

> * Joseph Myers:
> 
> > 2018-11-23  Joseph Myers  <joseph@codesourcery.com>
> >
> > 	[BZ #23915]
> > 	* sysdeps/unix/sysv/linux/arm/kernel-features.h
> > 	[__LINUX_KERNEL_VERSION < 0x040700] (__ASSUME_COPY_FILE_RANGE):
> > 	Undefine.
> 
> So I feel kind of bad about this—I looked for such issues after the
> first problems cropped up and just didn't seem them in the kernel source
> tree.
> 
> Assuming that I'm going to implement a few system call wrappers in the
> future, how can I do a better job here?

My approach is to examine the diffs for the kernel version that introduced 
the syscall, and check whether it's present (asm/unistd.h, and syscall 
table, and any relevant compat syscall table) for each (architecture, ABI) 
pair supported by glibc not using the asm-generic syscall ABI.

What we really need is for the kernel to do a better job at introducing 
syscalls at the same time on all architectures, rather than just adding 
them for one or two architectures, leaving it to architecture maintainers 
to fill in the rest and so resulting in long delays before glibc can 
actually assume a syscall to be available everywhere.

(I think kernel 4.4, when many socket syscalls can be assumed to be 
available everywhere, will be the next point where there are significant 
simplifications to be gained in glibc sources by increasing the global 
minimum kernel version supported - many of those functions will be able to 
move to just having a single entry in syscalls.list.)

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Fix Arm __ASSUME_COPY_FILE_RANGE (bug 23915) [committed]

[Adhemerval Zanella]

On 26/11/2018 14:46, Joseph Myers wrote:
> On Mon, 26 Nov 2018, Florian Weimer wrote:
> 
>> * Joseph Myers:
>>
>>> 2018-11-23  Joseph Myers  <joseph@codesourcery.com>
>>>
>>> 	[BZ #23915]
>>> 	* sysdeps/unix/sysv/linux/arm/kernel-features.h
>>> 	[__LINUX_KERNEL_VERSION < 0x040700] (__ASSUME_COPY_FILE_RANGE):
>>> 	Undefine.
>>
>> So I feel kind of bad about this—I looked for such issues after the
>> first problems cropped up and just didn't seem them in the kernel source
>> tree.
>>
>> Assuming that I'm going to implement a few system call wrappers in the
>> future, how can I do a better job here?
> 
> My approach is to examine the diffs for the kernel version that introduced 
> the syscall, and check whether it's present (asm/unistd.h, and syscall 
> table, and any relevant compat syscall table) for each (architecture, ABI) 
> pair supported by glibc not using the asm-generic syscall ABI.
> 
> What we really need is for the kernel to do a better job at introducing 
> syscalls at the same time on all architectures, rather than just adding 
> them for one or two architectures, leaving it to architecture maintainers 
> to fill in the rest and so resulting in long delays before glibc can 
> actually assume a syscall to be available everywhere.
> 
> (I think kernel 4.4, when many socket syscalls can be assumed to be 
> available everywhere, will be the next point where there are significant 
> simplifications to be gained in glibc sources by increasing the global 
> minimum kernel version supported - many of those functions will be able to 
> move to just having a single entry in syscalls.list.)
> 

I brought this issue to kernel developers on latest Linux Plumber two weeks
ago. We all agreed that Linux should avoid wiring up all-independent syscalls
in different releases for different architectures, but currently there is no
enforcing mechanism to do so.

On same discussion I also raised, from Florian's suggestion, about focal 
points relating to libc kernel iteration and Arnd Bergmann and Peter Anvin
volunteered. I am ccing them so we can improve the syscall additions for
new Linux releases.

Arnd also said the idea is to have the new 64-bit time_t syscalls for all 
affected architectures when it is accepted upstream.

Re: Fix Arm __ASSUME_COPY_FILE_RANGE (bug 23915) [committed]

[Arnd Bergmann]

On Mon, Nov 26, 2018 at 7:14 PM Adhemerval Zanella
<adhemerval.zanella@linaro.org> wrote:
> On 26/11/2018 14:46, Joseph Myers wrote:
> > On Mon, 26 Nov 2018, Florian Weimer wrote:
> >>> 2018-11-23  Joseph Myers  <joseph@codesourcery.com>
> >
> > (I think kernel 4.4, when many socket syscalls can be assumed to be
> > available everywhere, will be the next point where there are significant
> > simplifications to be gained in glibc sources by increasing the global
> > minimum kernel version supported - many of those functions will be able to
> > move to just having a single entry in syscalls.list.)
> >
> On same discussion I also raised, from Florian's suggestion, about focal
> points relating to libc kernel iteration and Arnd Bergmann and Peter Anvin
> volunteered. I am ccing them so we can improve the syscall additions for
> new Linux releases.
>
> Arnd also said the idea is to have the new 64-bit time_t syscalls for all
> affected architectures when it is accepted upstream.

Regarding the minimum releases, I assume the y2038 work will require
a split of the minimum runtime kernel version and the minimum kernel
header version. I'd aim for 4.21 or 4.22 to be the first kernel version that
has workable kernel headers for all sytem calls and ioctls, on all
architectures, so this would likely become the minimum version for
building glibc again for a long time.

As the runtime minimum version, 4.4 sounds reasonable, this is also
the oldest longterm stable kernel we currently support aside from the
ancient 3.16 release used primarily by Debian Jessie (oldstable). This
will be three years old by the time glibc-2.29 gets released. The last
time the minimum kernel version was raised in glibc-2.24, the
linux-3.2 release was 4.5 years old, the final stable 3.2.x release
was this summer after 6.5 years.If you wanted to go by those numbers
alone (ignoring technical benefits of picking a particular release),
glibc could move to 3.16 minimum now, and change to 4.4 in summer
2020 when 3.16 is expected to end.

When we synchronize the system calls during the y2038 work
and require a new header version 4.22 or something like that,
this would also make a good minimum binary version some three
to six years later, and then allow removing many of the remaining
differences between architectures, we just need to make sure
we can of the unintended differences now and fix them.

       Arnd

Re: Fix Arm __ASSUME_COPY_FILE_RANGE (bug 23915) [committed]

[Florian Weimer]

* Joseph Myers:

> On Mon, 26 Nov 2018, Florian Weimer wrote:
>
>> * Joseph Myers:
>> 
>> > 2018-11-23  Joseph Myers  <joseph@codesourcery.com>
>> >
>> > 	[BZ #23915]
>> > 	* sysdeps/unix/sysv/linux/arm/kernel-features.h
>> > 	[__LINUX_KERNEL_VERSION < 0x040700] (__ASSUME_COPY_FILE_RANGE):
>> > 	Undefine.
>> 
>> So I feel kind of bad about this—I looked for such issues after the
>> first problems cropped up and just didn't seem them in the kernel source
>> tree.
>> 
>> Assuming that I'm going to implement a few system call wrappers in the
>> future, how can I do a better job here?
>
> My approach is to examine the diffs for the kernel version that introduced 
> the syscall, and check whether it's present (asm/unistd.h, and syscall 
> table, and any relevant compat syscall table) for each (architecture, ABI) 
> pair supported by glibc not using the asm-generic syscall ABI.

Can we put a list somewhere of the files that need checking, for the
supported glibc architectures?

> What we really need is for the kernel to do a better job at introducing 
> syscalls at the same time on all architectures, rather than just adding 
> them for one or two architectures, leaving it to architecture maintainers 
> to fill in the rest and so resulting in long delays before glibc can 
> actually assume a syscall to be available everywhere.

Agreed.

Thanks,
Florian

Re: Fix Arm __ASSUME_COPY_FILE_RANGE (bug 23915) [committed]

[Joseph Myers]

On Wed, 9 Jan 2019, Florian Weimer wrote:

> > My approach is to examine the diffs for the kernel version that introduced 
> > the syscall, and check whether it's present (asm/unistd.h, and syscall 
> > table, and any relevant compat syscall table) for each (architecture, ABI) 
> > pair supported by glibc not using the asm-generic syscall ABI.
> 
> Can we put a list somewhere of the files that need checking, for the
> supported glibc architectures?

It tends to change with the Linux kernel version, as architectures change 
how they handle their syscall tables.  (Hopefully ongoing changes to unify 
the machinery more between different architectures will make such checks 
easier for future kernel versions - while not helping when it's past 
syscalls you're looking at.)

-- 
Joseph S. Myers
joseph@codesourcery.com