Re: socket: add support for MPTCP socket option from the Linux kernel?

[Adhemerval Zanella Netto <adhemerval.zanella@linaro.org>]

On 10/10/22 10:37, Matthieu Baerts via Libc-alpha wrote:
> Hello,
> 
> Thank you for maintaining and continuing to add new features to glibc!
> 
> 
> TL;DR: would it make sense to have a <netinet/mptcp.h> file?
> 
> 
> I'm contributing to MPTCP in the Linux kernel. MPTCP is an extension to
> TCP. The goal is to exchange data for a single connection over different
> paths, simultaneously (or not). Its support in the Linux kernel has
> started in Linux v5.6.
> 
> For the userspace, the usage is similar to TCP. The main difference is
> visible when creating the socket because the MPTCP protocol has to be set:
> 
>   socket(AF_INET(6), SOCK_STREAM, IPPROTO_MPTCP);
> 
> After that, the application uses the socket like it would do if it was
> TCP. It can also get and set socket options from different levels:
> socket, IP(v6), TCP and also a new one: SOL_MPTCP.
> 
> "IPPROTO_MPTCP" and "SOL_MPTCP" have been defined in glibc by Joseph
> Myers, thank you for that!
> 
> 
> With the MPTCP community, we were wondering if we can/should propose to
> add MPTCP headers to help userspace apps supporting MPTCP specific
> socket options. In other words, having something similar to
> <netinet/tcp.h> but for MPTCP.
> 
> MPTCP in Linux exposes different structures, enum and macros to userspace:
> 
>  https://elixir.bootlin.com/linux/v6.0/source/include/uapi/linux/mptcp.h
> 
> Most of the "linux/mptcp.h" content is linked to different Netlink API
> and we suppose this should not be exposed by glibc as well. The MPTCP
> socket options might be interesting to expose as it is similar to TCP.
> For the moment, there are 3 MPTCP specific socket options, defined at
> the end of the mptcp.h file. They are all linked somehow to MPTCP_INFO,
> similar to TCP_INFO.
> 
> When looking at the history of "sysdeps/gnu/netinet/tcp.h" file in
> glibc, it looks like TCP specific socket options are up to date. But
> that's not the case for the tcp_info structure for example which has not
> been updated since 2007 apparently.
> 
> 
> Considering the different elements mentioned above:
> 
> - Would it make sense to add a new "netinet/mptcp.h" file defining the
> different MPTCP specific socket options and linked structures?

It is a matter to maintainability, since any new header in glibc that is
bounded to a kernel specific features will need to be updated on every
release if there is any new definitions.

We already have Linux specific definition on sysdeps/unix/sysv/linux/netinet,
so I think it should be feasible to add a Linux specific MPTCP header.  
Besides it seems no to be a Linux specific features, it also seems that no 
other system does provide such header though (at least none I check, 
FreeBSD 13, MacOSX 12, Solaris 11, AIX 7.2).

> 
> - Or is the new "policy" to ask userspace app developers to include
> "linux/mptcp.h" from a recent enough kernel instead?

I still think this is not a settle topic, although with recent syscall
support we do tend to include kernel headers to get the latest definition.
It comes with its own burden, since the header might not be available and
dependant on the glibc header we need to add extra handling.

Usually on distro, the kernel headers seems to be update more often than
glibc; so a glibc header might not be updated with the latest kernel
definitions.  

I do not follow MCTCP development on Linux kernel, but if it is actively 
developed and newer features added in each release, I would say it would be
better to keep as a linux header so users know exactly what kind of features 
the underlying kernel supports.

> 
> - If it makes sense to add a new "netinet/mptcp.h" file, how should we
> handle conflicts between this new file and "linux/mptcp.h"? By using
> __USE_MISC?

That is not straightforward, depends of the feature you want to support,
and adds extra maintainability (and it is why I am not personally found 
on including kernel headers in glibc exported headers).

For instance for RSEQ (sysdeps/unix/sysv/linux/sys/rseq.h:

  #ifdef __has_include
  # if __has_include ("linux/rseq.h")
  #  define __GLIBC_HAVE_KERNEL_RSEQ
  # endif
  #else
  # include <linux/version.h>
  # if LINUX_VERSION_CODE >= KERNEL_VERSION (4, 18, 0)
  #  define __GLIBC_HAVE_KERNEL_RSEQ
  # endif
  #endif

  #ifdef __GLIBC_HAVE_KERNEL_RSEQ
  /* We use the structures declarations from the kernel headers.  */
  # include <linux/rseq.h>
  #else /* __GLIBC_HAVE_KERNEL_RSEQ */
  /* We use a copy of the include/uapi/linux/rseq.h kernel header.  */
  [...]
  #endif

And this is similar to close_range flags (sysdeps/unix/sysv/linux/bits/unistd_ext.h):

  #ifdef __has_include
  # if __has_include ("linux/close_range.h")
  #  include "linux/close_range.h"
  # endif
  #endif
  /* Unshare the file descriptor table before closing file descriptors.  */
  #ifndef CLOSE_RANGE_UNSHARE
  # define CLOSE_RANGE_UNSHARE (1U << 1)
  #endif
  /* Set the FD_CLOEXEC bit instead of closing the file descriptor.  */
  #ifndef CLOSE_RANGE_CLOEXEC
  # define CLOSE_RANGE_CLOEXEC (1U << 2)
  #endif

Since glibc can be built with kernel headers old as 3.2, you will need to
provide expected kernel flags and structures as a fallback if the kernel 
header can nto be included.

Since this is Linux specific header, there is no direct use of __USE_MISC
(which is used to adapt the definition to standard headers).