[RFC] Linux system call builtins

[RFC] Linux system call builtins

[Matheus Afonso Martins Moreira]

Hello! I'm a beginner when it comes to GCC development.
I want to learn how it works and start contributing.
Decided to start by implementing something relatively simple
but which would still be very useful for me: Linux builtins.
I sought help in the OFTC IRC channel and it was suggested
that I discuss it here first and obtain consensus before
spending more time on it since it might not be acceptable.

I'd like to add GCC builtins for generating Linux system call
code for all architectures supported by Linux.

They would look like this:

    __builtin_linux_system_call(long n, ...)
    __builtin_linux_system_call_1(long n, long _1)
    __builtin_linux_system_call_2(long n, long _1, long _2)
    /* More definitions, all the way up to 6 arguments */

Calling these builtins will make GCC place all the parameters
in the correct registers for the system call, emit the appropriate
instruction for the target architecture and return the result.
In other words, they would implement the calling convention[1] of
the Linux system calls.

I'm often asked why anyone should care about this system call stuff,
and I've been asked why I want this added to GCC in particular.
My rationale is as follows:

  + It's stable

        This is one of the things which makes Linux unique
        in the operating system landscape: applications
        can target the kernel directly. Unlike in virtually
        every other operating system out there, the Linux kernel
        to user space binary interface is documented[2] as stable.
        Breaking it is considered a regression in the kernel.
        Therefore it makes sense for a compiler to target it.
        The same is not true for any other operating system.

  + It's a calling convention

        GCC already supports many calling conventions
        via function attributes. On x86 alone[3] there's
        cdecl, fastcall, thiscall, stdcall, ms_abi, sysv_abi,
        Win32 specific hot patching hooks. So I believe this
        would not at all be a strange addition to the compiler.

  + It's becoming common

        Despite being specific to the Linux kernel,
        support for it is showing up in other systems.
        FreeBSD implements limited support[4] for Linux ABIs.
        Windows Subsystem for Linux started out[5] similarly,
        as an implementation of this system call ABI.
        Apparently it's becoming something of a lingua franca.
        Maybe one day Linux programs will actually become
        portable by virtue of this stable binary interface.

  + It doesn't make sense for libraries to support it

        There are libraries out there that provide
        system call functionality. The various libcs do.
        However they usually don't support the full set
        of Linux system calls. Using certain system calls
        could invalidate global state in these libraries
        which leads to them not being supported. Clone is
        the quintessential example. So I think libraries
        are not the proper place for this functionality.

  + It allows freestanding software to easily target Linux

        Freestanding code usually refers to bare metal
        targets but Linux is also a viable target.
        This will make it much easier for developers
        to create freestanding nolibc no dependency
        software targeting Linux without having to
        write any assembly code at all, making GCC
        ever more useful.

  + It centralizes functionality in the compiler

        Currently every programmer who wants to use
        these system calls must rely on libraries
        with incomplete support or recreate the
        system call machinery via inline assembly.
        Even the Linux kernel ended up doing it[6].
        It would be so much nicer if the compiler
        simply had support for it. I'm a huge fan
        of builtins like __builtin_frame_address,
        they make it very easy to solve difficult
        problems which would otherwise require tons
        of target specific assembly code. Getting
        the compiler to do that for Linux system
        calls is what this proposal is for.

  + It allows other languages to easily target Linux

        GCC is a compiler collection and has support
        for numerous languages. These builtins should
        allow all of them to target Linux directly
        in one fell swoop.

  + Compilers seem like the proper place for it

        The compiler knows everything about registers
        and instructions and calling conventions.
        It just seems like the right place for it.
        A just in time compiler could also generate
        this code instead of calling native functions.
        I really have no idea why they don't do that.
        Maybe this will prove that it's viable.

Implementation wise, I have managed to define the above builtins
in my GCC branch and compile it successfully. I have not yet
figured out how or even where to implement the code generation.
I was hoping to show up here with patches ready for review
but it really is a complex project. That's why I would like to
to see what the community thinks before proceeding.

A related proposal: hard register operand constraints[7]
for inline assembly code. Essentially, allowing the programmer
to specify the exact registers that must be used in the inline
assembly expression itself. This gets rid of numerous temporary
variables whose only purpose is to get GCC to put them in the
correct registers, as many as 7 local variables for system calls.

I've been told that implementing it would make this proposal
redundant. There is no doubt that this would make code much
simpler, easier to write and understand. It would be a valuable
enhancement to the compiler and I would certainly use it.
However, even with better inline assembly, I still believe
there's value in a simple system call builtin function.
The API is much nicer if nothing else.

Thanks for your attention,
  Matheus

[1]: https://www.man7.org/linux/man-pages/man2/syscall.2.html
[2]: https://www.kernel.org/doc/html/latest/admin-guide/abi-stable.html#the-kernel-syscall-interface
[3]: https://gcc.gnu.org/onlinedocs/gcc/x86-Function-Attributes.html
[4]: https://man.freebsd.org/cgi/man.cgi?linux
[5]: https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux
[6]: https://lwn.net/Articles/920158/
[7]: https://gcc.gnu.org/pipermail/gcc/2021-June/236269.html

Re: [RFC] Linux system call builtins

[Jonathan Wakely]

Hello,

On Mon, 8 Apr 2024 at 10:20, Matheus Afonso Martins Moreira via Gcc
<gcc@gcc.gnu.org> wrote:
>
> I'd like to add GCC builtins for generating Linux system call
> code for all architectures supported by Linux.
>
> They would look like this:
>
>     __builtin_linux_system_call(long n, ...)
>     __builtin_linux_system_call_1(long n, long _1)
>     __builtin_linux_system_call_2(long n, long _1, long _2)
>     /* More definitions, all the way up to 6 arguments */

What's the advantage of the _1, _2 etc. forms? The compiler knows how
many arguments you're passing, why can't there just be one built-in
handling all cases?



>
>   + It doesn't make sense for libraries to support it
>
>         There are libraries out there that provide
>         system call functionality. The various libcs do.
>         However they usually don't support the full set
>         of Linux system calls. Using certain system calls
>         could invalidate global state in these libraries
>         which leads to them not being supported. Clone is
>         the quintessential example. So I think libraries
>         are not the proper place for this functionality.

Your proposal doesn't seem to actually address the problem. If using
the clone syscall causes problems for glibc by not giving glibc a
chance to set up TLS etc for the new thread, how does making it easier
to use the clone syscall help?

Re: [RFC] Linux system call builtins

[Florian Weimer]

* Matheus Afonso Martins Moreira via Gcc:

>   + It's stable
>
>         This is one of the things which makes Linux unique
>         in the operating system landscape: applications
>         can target the kernel directly. Unlike in virtually
>         every other operating system out there, the Linux kernel
>         to user space binary interface is documented[2] as stable.
>         Breaking it is considered a regression in the kernel.
>         Therefore it makes sense for a compiler to target it.
>         The same is not true for any other operating system.

There is quite a bit of variance in how the kernel is entered.  On
x86-64, one once popular mechanism is longer present in widely-used
kernels.  For POWER, the preferred way changed over time.  Likewise for
i386.

>   + It's a calling convention
>
>         GCC already supports many calling conventions
>         via function attributes. On x86 alone[3] there's
>         cdecl, fastcall, thiscall, stdcall, ms_abi, sysv_abi,
>         Win32 specific hot patching hooks. So I believe this
>         would not at all be a strange addition to the compiler.

But using a builtin obfuscates that relationship.  There is no
__builtin_call_ms_abi, is there?

Thanks,
Florian

Re: [RFC] Linux system call builtins

[Alexander Monakov]

On Mon, 8 Apr 2024, Florian Weimer via Gcc wrote:

> * Matheus Afonso Martins Moreira via Gcc:
> 
> >   + It's stable
> >
> >         This is one of the things which makes Linux unique
> >         in the operating system landscape: applications
> >         can target the kernel directly. Unlike in virtually
> >         every other operating system out there, the Linux kernel
> >         to user space binary interface is documented[2] as stable.
> >         Breaking it is considered a regression in the kernel.
> >         Therefore it makes sense for a compiler to target it.
> >         The same is not true for any other operating system.
> 
> There is quite a bit of variance in how the kernel is entered.  On
> x86-64, one once popular mechanism is longer present in widely-used
> kernels.

I assume you're implicitly referencing the vsyscall mechanism, but on
amd64 it's not useful to *enter the kernel*, right? It was useful for
obtaining the result of certain syscalls without actually entering
the kernel, like with vdso.

Unlike i386, where the vdso (as well as vsyscall I guess) provides
the __kernel_vsyscall entrypoint, which provides whichever of
{ int 0x80, sysenter, syscall } methods is available and fastest.

Or am I missing something?

Alexander

Re: [RFC] Linux system call builtins

[Florian Weimer]

* Alexander Monakov:

> On Mon, 8 Apr 2024, Florian Weimer via Gcc wrote:
>
>> * Matheus Afonso Martins Moreira via Gcc:
>> 
>> >   + It's stable
>> >
>> >         This is one of the things which makes Linux unique
>> >         in the operating system landscape: applications
>> >         can target the kernel directly. Unlike in virtually
>> >         every other operating system out there, the Linux kernel
>> >         to user space binary interface is documented[2] as stable.
>> >         Breaking it is considered a regression in the kernel.
>> >         Therefore it makes sense for a compiler to target it.
>> >         The same is not true for any other operating system.
>> 
>> There is quite a bit of variance in how the kernel is entered.  On
>> x86-64, one once popular mechanism is longer present in widely-used
>> kernels.
>
> I assume you're implicitly referencing the vsyscall mechanism, but on
> amd64 it's not useful to *enter the kernel*, right? It was useful for
> obtaining the result of certain syscalls without actually entering
> the kernel, like with vdso.

The implementation performed a standard system call if a pure userspace
implementation wasn't possible.  It wasn't intended as a general-purpose
way to enter the kernel (although it could be used as such, hence the
desire to remove it in some cases).

Thanks,
Florian

Re: Re: [RFC] Linux system call builtins

[Matheus Afonso Martins Moreira]

> What's the advantage of the _1, _2 etc. forms?

Now that you mention it... I don't believe there are any.

> The compiler knows how many arguments you're passing,
> why can't there just be one built-in handling all cases?

You're right about that.

When I started working on this I just mirrored the existing
APIs which consist of one function for each possible arity.
I have always wanted a variadic system call function though
so when I saw that GCC had variadic builtins I threw one in.

A builtin doesn't have the overhead of variadic C functions though.
The compiler should be able to generate optimal code depending on
how many arguments it's called with.

It's the best possible interface and strictly superior to anything
libraries can offer.

Would other languages supported by GCC benefit from the other forms?
Perhaps variadic builtins are incompatible with them for some reason?
I can't think of any reason to keep the numbered forms otherwise.

> Your proposal doesn't seem to actually address the problem.
> If using the clone syscall causes problems for glibc by not
> giving glibc a chance to set up TLS etc for the new thread,
> how does making it easier to use the clone syscall help?

I think that problem can never be fully addressed, only worked around.
It seems to be inherent to how standard C libraries are implemented.
The C standards apparently make it impossible to avoid either global
or thread local state. Locales come to mind...

Compiler support for system calls help by eliminating the need for the
system call stub functions traditionally provided by these C libraries.
There's no need to link against the C libraries just for that anymore
and there's also no need to implement it ourselves since the compiler
provides it. The result is it makes it easier to develop freestanding
software targeting Linux.

Re: [RFC] Linux system call builtins

[Alexander Monakov]

On Mon, 8 Apr 2024, Florian Weimer wrote:

> * Alexander Monakov:
> 
> >> There is quite a bit of variance in how the kernel is entered.  On
> >> x86-64, one once popular mechanism is longer present in widely-used
> >> kernels.
> >
> > I assume you're implicitly referencing the vsyscall mechanism, but on
> > amd64 it's not useful to *enter the kernel*, right? It was useful for
> > obtaining the result of certain syscalls without actually entering
> > the kernel, like with vdso.
> 
> The implementation performed a standard system call if a pure userspace
> implementation wasn't possible.  It wasn't intended as a general-purpose
> way to enter the kernel (although it could be used as such, hence the
> desire to remove it in some cases).

I guess you meant it was "popular" only for the specific syscalls it 
provided acceleration for. I'm not sure how that is relevant to the
current discussion.

Alexander

Re: Re: [RFC] Linux system call builtins

[Matheus Afonso Martins Moreira]

> There is quite a bit of variance in how the kernel is entered.

I assume you mean the vDSO. It is also documented and stable.

https://www.kernel.org/doc/html/latest/admin-guide/abi-stable.html#vdso

> Unless otherwise noted, the set of symbols with any given version
> and the ABI of those symbols is considered stable.
> It may vary across architectures, though.

> As of this writing, this ABI documentation as been confirmed
> for x86_64. The maintainers of the other vDSO-using architectures
> should confirm that it is correct for their architecture.

It is an also entirely optional. The architecture specific system call
entry point is always available. The vDSO exists to provide much more
efficient ways to access frequently queried system information such
as the current time. While the optimized approaches are preferable,
the slower system call entry points are still available and stable.

> On x86-64, one once popular mechanism
> is longer present in widely-used kernels.

Please elaborate. Do you mean the vDSO?

Linux places a pointer to the vDSO in the auxiliary vector,
and a pointer to that vector is located immediately after
the program's environment. The program will have to walk that
vector in order to find the vDSO. If the vDSO is missing,
then the program will fail to find that pointer and hopefully
fall back to traditional system call entry points.

Perhaps GCC could also have builtins for accessing things like
argc, argv, envp and auxvec. They are part of the ABI too.
This would allow programs to access the vDSO via the auxvec.
It'd also allow implementation of ELF entry points entirely in C.

> But using a builtin obfuscates that relationship.
> There is no __builtin_call_ms_abi, is there?

That's true but that's because there's no need for those builtins.
Functions which conform to specific ABIs will have been marked with
the relevant attribute so GCC will know how what to do when it is called.
Linux system calls do not actually exist as functions so they can't be
marked that way.

It could be implemented that way by having a naked system call
function whose entire body is just the syscall instruction.
Then it could be marked with a linux_syscall_abi attribute
and the compiler would know to where to put the arguments
and from where to obtain the return value.

That just replicates what the builtin is supposed to do though.
The builtin would not need any declarations or attributes.
It would just work.

Re: Re: [RFC] Linux system call builtins

[Jonathan Wakely]

[-- Attachment #1: Type: text/plain, Size: 869 bytes --]

On Mon, 8 Apr 2024, 13:00 Matheus Afonso Martins Moreira via Gcc, <
gcc@gcc.gnu.org> wrote:

>
> Compiler support for system calls help by eliminating the need for the
> system call stub functions traditionally provided by these C libraries.
> There's no need to link against the C libraries just for that anymore
> and there's also no need to implement it ourselves since the compiler
> provides it. The result is it makes it easier to develop freestanding
> software targeting Linux.
>

It means you don't need newlib just for the syscall function, but then you
would also need to implement memcpy, etc. for yourself. Obviously some bare
metal code needs to do that anyway, but that wouldn't be using syscalls
either.

This isn't my field so maybe I'm totally wrong, but the use case seems like
a niche within a niche, but with a non-zero maintenance burden for GCC.

Re: [RFC] Linux system call builtins

[Paul Iannetta]

Hi,

On Mon, Apr 08, 2024 at 06:19:14AM -0300, Matheus Afonso Martins Moreira via Gcc wrote:
> Hello! I'm a beginner when it comes to GCC development.
> I want to learn how it works and start contributing.
> Decided to start by implementing something relatively simple
> but which would still be very useful for me: Linux builtins.
> I sought help in the OFTC IRC channel and it was suggested
> that I discuss it here first and obtain consensus before
> spending more time on it since it might not be acceptable.
> 
> I'd like to add GCC builtins for generating Linux system call
> code for all architectures supported by Linux.
> 
> They would look like this:
> 
>     __builtin_linux_system_call(long n, ...)
>     __builtin_linux_system_call_1(long n, long _1)
>     __builtin_linux_system_call_2(long n, long _1, long _2)
>     /* More definitions, all the way up to 6 arguments */
> 

As noted by J. Wakely, you don't need to have one variant for each
number of arguments.  By the way, even if you have multiple variants
you could unify them all under a macro __builtin_linux_system_call by
means such as "overloading macros based on the argument count." [1]

> Calling these builtins will make GCC place all the parameters
> in the correct registers for the system call, emit the appropriate
> instruction for the target architecture and return the result.
> In other words, they would implement the calling convention[1] of
> the Linux system calls.
> 
> I'm often asked why anyone should care about this system call stuff,
> and I've been asked why I want this added to GCC in particular.
> My rationale is as follows:
> 
>   + It's stable
>   [snip]

I assume you're talking about the interface which is often abstracted
by functions such as the following which are often found in libcs or
freestanding libraries. The musl is a typical example (cf syscall_arch.h)
for each architecture ( https://git.musl-libc.org/cgit/musl/tree/arch )

long linux_system_call_1(long number, long _1)
{
	register long rax __asm__("rax") = number;
	register long rdi __asm__("rdi") = _1;

	__asm__ volatile
	("syscall"

		: "+r" (rax)
		: "r" (rdi)
		: "rcx", "r11", "cc", "memory");

	return rax;
}

> 
>   + It's a calling convention
> 
>         GCC already supports many calling conventions
>         via function attributes. On x86 alone[3] there's
>         cdecl, fastcall, thiscall, stdcall, ms_abi, sysv_abi,
>         Win32 specific hot patching hooks. So I believe this
>         would not at all be a strange addition to the compiler.

I may be wrong, but I think that at least on sysv x86_64, syscalls have
the same calling conventions as regular functions.  However, the
function descriptor is not an address (or a symbol reference) but a
number.

> 
>   + It's becoming common
>  [snip]
> 
>   + It doesn't make sense for libraries to support it
>  [snip]

At least, it would be nice if not all freestanding libraries had to
reimplement those syscalls stubs.

> 
>   + It allows freestanding software to easily target Linux
> 
>   + It centralizes functionality in the compiler
> 
>   + It allows other languages to easily target Linux
> 
>   + Compilers seem like the proper place for it

I tend to agree with those points.

> Implementation wise, I have managed to define the above builtins
> in my GCC branch and compile it successfully. I have not yet
> figured out how or even where to implement the code generation.
> I was hoping to show up here with patches ready for review
> but it really is a complex project. That's why I would like to
> to see what the community thinks before proceeding.
> 

I think you could have a look at the function 'expand_call' in
calls.cc to see how regular calls are expanded to RTL and see what you
would need to do to support calls which use a number rather than an
address.

Cheers,
Paul

[1]: https://jadlevesque.github.io/PPMP-Iceberg/explanations#overloading-macros-based-on-argument-count

Re: [RFC] Linux system call builtins

[Andrew Pinski]

On Mon, Apr 8, 2024 at 11:20 AM Paul Iannetta via Gcc <gcc@gcc.gnu.org> wrote:
>
> Hi,
>
> On Mon, Apr 08, 2024 at 06:19:14AM -0300, Matheus Afonso Martins Moreira via Gcc wrote:
> > Hello! I'm a beginner when it comes to GCC development.
> > I want to learn how it works and start contributing.
> > Decided to start by implementing something relatively simple
> > but which would still be very useful for me: Linux builtins.
> > I sought help in the OFTC IRC channel and it was suggested
> > that I discuss it here first and obtain consensus before
> > spending more time on it since it might not be acceptable.
> >
> > I'd like to add GCC builtins for generating Linux system call
> > code for all architectures supported by Linux.
> >
> > They would look like this:
> >
> >     __builtin_linux_system_call(long n, ...)
> >     __builtin_linux_system_call_1(long n, long _1)
> >     __builtin_linux_system_call_2(long n, long _1, long _2)
> >     /* More definitions, all the way up to 6 arguments */
> >
>
> As noted by J. Wakely, you don't need to have one variant for each
> number of arguments.  By the way, even if you have multiple variants
> you could unify them all under a macro __builtin_linux_system_call by
> means such as "overloading macros based on the argument count." [1]

Actually you don't need a macro if implemented inside GCC. Can you can
count the number of arguments and expand it based on that. No reason
for macros. Now the question comes is the argument long or some other
type? E.g. for some 32bit ABIs built on top of 64bit ISA might always
just pass 32bits or they might allow passing the full 64bits. (x32
might fall under this and MIPS n32). Or do you split a 64bit argument
into the lower and upper half registers. Maybe you should warn/error
out if not passed the correct sized argument.
Also do you sign or zero extend a 32bit argument for LP64 targets?
Right now it is not obvious nor documented in your examples.



Thanks,
Andrew Pinski

>
> > Calling these builtins will make GCC place all the parameters
> > in the correct registers for the system call, emit the appropriate
> > instruction for the target architecture and return the result.
> > In other words, they would implement the calling convention[1] of
> > the Linux system calls.
> >
> > I'm often asked why anyone should care about this system call stuff,
> > and I've been asked why I want this added to GCC in particular.
> > My rationale is as follows:
> >
> >   + It's stable
> >   [snip]
>
> I assume you're talking about the interface which is often abstracted
> by functions such as the following which are often found in libcs or
> freestanding libraries. The musl is a typical example (cf syscall_arch.h)
> for each architecture ( https://git.musl-libc.org/cgit/musl/tree/arch )
>
> long linux_system_call_1(long number, long _1)
> {
>         register long rax __asm__("rax") = number;
>         register long rdi __asm__("rdi") = _1;
>
>         __asm__ volatile
>         ("syscall"
>
>                 : "+r" (rax)
>                 : "r" (rdi)
>                 : "rcx", "r11", "cc", "memory");
>
>         return rax;
> }
>
> >
> >   + It's a calling convention
> >
> >         GCC already supports many calling conventions
> >         via function attributes. On x86 alone[3] there's
> >         cdecl, fastcall, thiscall, stdcall, ms_abi, sysv_abi,
> >         Win32 specific hot patching hooks. So I believe this
> >         would not at all be a strange addition to the compiler.
>
> I may be wrong, but I think that at least on sysv x86_64, syscalls have
> the same calling conventions as regular functions.  However, the
> function descriptor is not an address (or a symbol reference) but a
> number.
>
> >
> >   + It's becoming common
> >  [snip]
> >
> >   + It doesn't make sense for libraries to support it
> >  [snip]
>
> At least, it would be nice if not all freestanding libraries had to
> reimplement those syscalls stubs.
>
> >
> >   + It allows freestanding software to easily target Linux
> >
> >   + It centralizes functionality in the compiler
> >
> >   + It allows other languages to easily target Linux
> >
> >   + Compilers seem like the proper place for it
>
> I tend to agree with those points.
>
> > Implementation wise, I have managed to define the above builtins
> > in my GCC branch and compile it successfully. I have not yet
> > figured out how or even where to implement the code generation.
> > I was hoping to show up here with patches ready for review
> > but it really is a complex project. That's why I would like to
> > to see what the community thinks before proceeding.
> >
>
> I think you could have a look at the function 'expand_call' in
> calls.cc to see how regular calls are expanded to RTL and see what you
> would need to do to support calls which use a number rather than an
> address.
>
> Cheers,
> Paul
>
> [1]: https://jadlevesque.github.io/PPMP-Iceberg/explanations#overloading-macros-based-on-argument-count
>
>
>
>

Re: [RFC] Linux system call builtins

[Paul Iannetta]

On Mon, Apr 08, 2024 at 11:26:40AM -0700, Andrew Pinski wrote:
> On Mon, Apr 8, 2024 at 11:20 AM Paul Iannetta via Gcc <gcc@gcc.gnu.org> wrote:
> >
> > Hi,
> >
> > On Mon, Apr 08, 2024 at 06:19:14AM -0300, Matheus Afonso Martins Moreira via Gcc wrote:
> > > Hello! I'm a beginner when it comes to GCC development.
> > > I want to learn how it works and start contributing.
> > > Decided to start by implementing something relatively simple
> > > but which would still be very useful for me: Linux builtins.
> > > I sought help in the OFTC IRC channel and it was suggested
> > > that I discuss it here first and obtain consensus before
> > > spending more time on it since it might not be acceptable.
> > >
> > > I'd like to add GCC builtins for generating Linux system call
> > > code for all architectures supported by Linux.
> > >
> > > They would look like this:
> > >
> > >     __builtin_linux_system_call(long n, ...)
> > >     __builtin_linux_system_call_1(long n, long _1)
> > >     __builtin_linux_system_call_2(long n, long _1, long _2)
> > >     /* More definitions, all the way up to 6 arguments */
> > >
> >
> > As noted by J. Wakely, you don't need to have one variant for each
> > number of arguments.  By the way, even if you have multiple variants
> > you could unify them all under a macro __builtin_linux_system_call by
> > means such as "overloading macros based on the argument count." [1]
> 
> Actually you don't need a macro if implemented inside GCC. Can you can
> count the number of arguments and expand it based on that. No reason
> for macros.

I fully agree here. I was mentioning the macro solution in the case
where it is supported outside the compiler.

> Now the question comes is the argument long or some other
> type? E.g. for some 32bit ABIs built on top of 64bit ISA might always
> just pass 32bits or they might allow passing the full 64bits. (x32
> might fall under this and MIPS n32). Or do you split a 64bit argument
> into the lower and upper half registers. Maybe you should warn/error
> out if not passed the correct sized argument.
> Also do you sign or zero extend a 32bit argument for LP64 targets?
> Right now it is not obvious nor documented in your examples.
> 

Another case would be targets allowing an immediate argument for their
syscall instruction.  Sign extend is probably always an error, zero
extend may give the expected results.  Emitting an error or a warning
seems a very good idea if the size does not match.  Syscalls can
receive both values or pointers (which may not have the same size as
regular values) which may complicate the handling and the types of the
arguments.
However, for most complex ABIs, all the cases you mentioned should be
addressed by each target backend by specializing the call/call_value
SPNs in their machine description files, and specifying the right
constraints.

> 
> Thanks,
> Andrew Pinski
> 
> >
> > > Calling these builtins will make GCC place all the parameters
> > > in the correct registers for the system call, emit the appropriate
> > > instruction for the target architecture and return the result.
> > > In other words, they would implement the calling convention[1] of
> > > the Linux system calls.
> > >
> > > I'm often asked why anyone should care about this system call stuff,
> > > and I've been asked why I want this added to GCC in particular.
> > > My rationale is as follows:
> > >
> > >   + It's stable
> > >   [snip]
> >
> > I assume you're talking about the interface which is often abstracted
> > by functions such as the following which are often found in libcs or
> > freestanding libraries. The musl is a typical example (cf syscall_arch.h)
> > for each architecture ( https://git.musl-libc.org/cgit/musl/tree/arch )
> >
> > long linux_system_call_1(long number, long _1)
> > {
> >         register long rax __asm__("rax") = number;
> >         register long rdi __asm__("rdi") = _1;
> >
> >         __asm__ volatile
> >         ("syscall"
> >
> >                 : "+r" (rax)
> >                 : "r" (rdi)
> >                 : "rcx", "r11", "cc", "memory");
> >
> >         return rax;
> > }
> >
> > >
> > >   + It's a calling convention
> > >
> > >         GCC already supports many calling conventions
> > >         via function attributes. On x86 alone[3] there's
> > >         cdecl, fastcall, thiscall, stdcall, ms_abi, sysv_abi,
> > >         Win32 specific hot patching hooks. So I believe this
> > >         would not at all be a strange addition to the compiler.
> >
> > I may be wrong, but I think that at least on sysv x86_64, syscalls have
> > the same calling conventions as regular functions.  However, the
> > function descriptor is not an address (or a symbol reference) but a
> > number.
> >
> > >
> > >   + It's becoming common
> > >  [snip]
> > >
> > >   + It doesn't make sense for libraries to support it
> > >  [snip]
> >
> > At least, it would be nice if not all freestanding libraries had to
> > reimplement those syscalls stubs.
> >
> > >
> > >   + It allows freestanding software to easily target Linux
> > >
> > >   + It centralizes functionality in the compiler
> > >
> > >   + It allows other languages to easily target Linux
> > >
> > >   + Compilers seem like the proper place for it
> >
> > I tend to agree with those points.
> >
> > > Implementation wise, I have managed to define the above builtins
> > > in my GCC branch and compile it successfully. I have not yet
> > > figured out how or even where to implement the code generation.
> > > I was hoping to show up here with patches ready for review
> > > but it really is a complex project. That's why I would like to
> > > to see what the community thinks before proceeding.
> > >
> >
> > I think you could have a look at the function 'expand_call' in
> > calls.cc to see how regular calls are expanded to RTL and see what you
> > would need to do to support calls which use a number rather than an
> > address.
> >
> > Cheers,
> > Paul
> >
> > [1]: https://jadlevesque.github.io/PPMP-Iceberg/explanations#overloading-macros-based-on-argument-count
> >
> >
> >
> >
> 
> 
> 
> 

Re: [RFC] Linux system call builtins

[Paul Koning]

> On Apr 8, 2024, at 4:01 PM, Paul Iannetta via Gcc <gcc@gcc.gnu.org> wrote:
> 
> On Mon, Apr 08, 2024 at 11:26:40AM -0700, Andrew Pinski wrote:
>> On Mon, Apr 8, 2024 at 11:20 AM Paul Iannetta via Gcc <gcc@gcc.gnu.org> wrote:
>>> ...
>> Also do you sign or zero extend a 32bit argument for LP64 targets?
>> Right now it is not obvious nor documented in your examples.
>> 
> 
> Another case would be targets allowing an immediate argument for their
> syscall instruction.  Sign extend is probably always an error, zero
> extend may give the expected results. 

It depends on the ABI.  For example, on MIPS, pointers are treated as signed when extending from 32 to 64 bits.

	paul

Re: [RFC] Linux system call builtins

[Paul Floyd]

On 08-04-24 09:19, Matheus Afonso Martins Moreira via Gcc wrote:

>    + It's becoming common
> 
>          Despite being specific to the Linux kernel,
>          support for it is showing up in other systems.
>          FreeBSD implements limited support[4] for Linux ABIs.
>          Windows Subsystem for Linux started out[5] similarly,
>          as an implementation of this system call ABI.
>          Apparently it's becoming something of a lingua franca.
>          Maybe one day Linux programs will actually become
>          portable by virtue of this stable binary interface.

I don't really buy your portability argument.

FreeBSD implements this as a syscall (two actually).
syscall 0:
int syscall(int number, ...);
syscall 198:
int __syscall(int64_t number, ...);

(I've never seen the second one used in anger, it's supposed to be for 
systems that have unusual argument padding).

The argument shuffling gets done in the kernel, not libc. In fact 
FreeBSD 15 just moved all syscall wrappers to a separate library, 
libsys, so it's there now.

Over in OpenBSD they are going to removing 'syscall' from libc.

https://lwn.net/Articles/949078/

 From what I've seen it has also been removed by Apple.

Whilst you aren't proposing the same thing, I see systems making it more 
difficult for code to make syscalls, not easier.

I also think that this could be misleading. There are sometimes subtle 
differences between the syscall interface and the interface exported by 
libc.

A+
Paul

Re: [RFC] Linux system call builtins

[Szabolcs Nagy]

The 04/08/2024 06:19, Matheus Afonso Martins Moreira via Gcc wrote:
>     __builtin_linux_system_call(long n, ...)
...
> Calling these builtins will make GCC place all the parameters
> in the correct registers for the system call, emit the appropriate
> instruction for the target architecture and return the result.
> In other words, they would implement the calling convention[1] of
> the Linux system calls.

note: some syscalls / features don't work without asm
(posix thread cancellation, vfork, signal return,..)

and using raw syscalls outside of the single runtime the
application is using is problematic (at least on linux).

>   + It doesn't make sense for libraries to support it
> 
>         There are libraries out there that provide
>         system call functionality. The various libcs do.
>         However they usually don't support the full set
>         of Linux system calls. Using certain system calls
>         could invalidate global state in these libraries
>         which leads to them not being supported. Clone is
>         the quintessential example. So I think libraries
>         are not the proper place for this functionality.

i don't follow the reasoning here, where should the
syscall be if not in a library like libc?

clone cannot even be used from c code in general as
CLONE_VM is not compatible with c semantics without
a new stack (child clobbers the parent stack), so
the c builtin would not always work, but it is also
a syscall that only freestanding application can use
not something that calls into the libc, and even in
a freestanding application it is tricky to use right
(especially in a portable way or with features like
shadow stack), so i don't see why clone is the
quintessential example.

>   + It allows freestanding software to easily target Linux
> 
>         Freestanding code usually refers to bare metal
>         targets but Linux is also a viable target.
>         This will make it much easier for developers
>         to create freestanding nolibc no dependency
>         software targeting Linux without having to
>         write any assembly code at all, making GCC
>         ever more useful.

i think the asm call convention bit is by far not the
hardest part in providing portable linux syscall wrappers.

my main worry is that the builtins encourage the use of raw
syscalls and outside of libc development it is not well
understood how to do that correctly, but i guess it's ok if
it is by default an error outside of -ffreestanding.

[RFC] Linux system call builtins

[Matheus Afonso Martins Moreira]

> As noted by J. Wakely, you don't need to have one variant
> for each number of arguments.

Yes, he is right about that. I have already deleted
all the variants from my code since the variadic
builtin will be able to generate optimal code,
unlike a variadic C function.

> I assume you're talking about the interface
> which is often abstracted by functions such as
> the following which are often found in libcs or
> freestanding libraries.

> long linux_system_call_1(long number, long _1)

Yes, that's exactly what I have in mind.
My goal is to implement those functions
inside the compiler itself as builtins.

> I think that at least on sysv x86_64, syscalls have
> the same calling conventions as regular functions.
However, the
function descriptor is not an address (or a symbol reference) but a
number.

They are similar but not equal.

x86_64 function calling convention passes the first 6 arguments
in the following registers: rdi, rsi, rdx, rcx, r8, r9.
x86_64 system call calling convention passes only 6 arguments
in the following registers: rdi, rsi, rdx, r10, r8, r9.

System calls use r10 instead of rcx
and don't support more than 6 arguments.

> At least, it would be nice if not all freestanding libraries
had to reimplement those syscalls stubs.

Completely agree!

> I think you could have a look at the function 'expand_call'
> in calls.cc to see how regular calls are expanded to RTL

Thanks for the pointer!! I had been looking
in the gcc/config/ directory for platform specific
code that implemented calling conventions.
Somehow that calls.cc file slipped past my attention.

My thinking is that I need to have the compiler
evaluate the input expressions and place the
results in specific registers, while also ensuring
all the input expressions are register sized and
emitting type errors in case they are not.
I looked for some kind of "register" type
but didn't find anything. So I used long int,
just like the libraries do.

Thanks,
    Matheus

[RFC] Linux system call builtins

[Matheus Afonso Martins Moreira]

> Now the question comes is the argument long or some other type?

I believe so. Every library I've seen and the kernel itself uses long.
Other types just get typecasted to long. I think it's just supposed to
mean "register type" since all the arguments must be in registers.

> E.g. for some 32bit ABIs built on top of 64bit ISA might always
> just pass 32bits or they might allow passing the full 64bits.
> (x32 might fall under this and MIPS n32).

The registers used are documented here:

https://www.man7.org/linux/man-pages/man2/syscall.2.html

x32 uses the exact same registers as x86_64, all 64 bit:

> x86-64    rdi rsi rdx r10 r8 r9
> x32       rdi rsi rdx r10 r8 r9

MIPS calling conventions work like this:

> mips/n32,64 a0 a1 a2 a3 a4 a5
> mips/o32    a0 a1 a2 a3 ...
> mips/o32    args5-8 are passed on the stack

> Maybe you should warn/error out
> if not passed the correct sized argument.

Yes. It should be an error if the inputs are not register sized.
As far as I know all system call inputs are either register sized
values or pointers to C strings, buffers or much larger structures.

> Also do you sign or zero extend a 32bit argument for LP64 targets?

I'm not sure. System call stubs just declare these register values as
signed long so whatever happens in that case seems to be appropriate.

[RFC] Linux system call builtins

[Matheus Afonso Martins Moreira]

> I see systems making it more  difficult for code to make syscalls,
> not easier.

That's true.

I think it's because other systems can afford to keep that ABI unstable.
Since Linux is an independently developed kernel, it _must_ be possible
to target the kernel directly with no user space component in-between.
Someone might write a freestanding program with nothing but system calls
and boot Linux directly into it.

This feature also makes it ideal for other programming languages.
On every other operating system, you need to link to some C library.
On Linux, that library is not actually necessary due to stable ABIs.
Rust programmers could conceivably recreate the entire Linux userspace
in Rust given enough time and effort. I created a programming language
based entirely around that concept, a lisp variant which uses nothing
but system calls and provides a system-call primitive to lisp code.
It's still in its infancy due to my limited free time but it's a fact
that with system calls it could do anything, it could mount disks.

> I also think that this could be misleading.

I don't see how. The __builtin_ prefix makes it clear
that it's a compiler feature rather than a libc function.

> There are sometimes subtle differences between the
> syscall interface and the interface exported by libc.

Yes. The C libraries seem to have some kind of cancellation mechanism
built right into it, for example. Using the system calls directly
eliminates them. This lets other languages build their own mechanisms.

[RFC] Linux system call builtins

[Matheus Afonso Martins Moreira]

> note: some syscalls / features don't work without asm
> (posix thread cancellation, vfork, signal return,..)

That's true. On the other hand, POSIX compliance
is not always a goal or a requirement.

> and using raw syscalls outside of the single runtime the
> application is using is problematic (at least on linux).

I understand why their use is problem on other operating systems:
since the ABI is not actually stable, programs will break eventually.

Why do you say they are problematic on Linux though? Please elaborate.

The ABI being stable should mean that I can for example
strace a program, analyze the system calls and implement
a new version of it that performs the same functions.

> where should the syscall be if not in a library like libc?

In the compiler. It should be able to generate the system call code.

> clone cannot even be used from c code in general
> as CLONE_VM is not compatible with c semantics
> without a new stack (child clobbers the parent stack)
> so the c builtin would not always work
> it is also a syscall that only freestanding
> application can use not something that calls
> into the libc

There are major projects out there which do use it regardless.
For example, systemd:

https://github.com/systemd/systemd/blob/main/src/basic/raw-clone.h
https://github.com/systemd/systemd/blob/main/src/shared/async.h
https://github.com/systemd/systemd/blob/main/src/shared/async.c
https://github.com/systemd/systemd/blob/main/docs/CODING_STYLE.md

> even in a freestanding application it is tricky to use right

No argument from me there. It is tricky...
The compiler should make it possible though.

> so i don't see why clone is the quintessential example.

I think it is the best example because attempting to use clone
is not actually supported by glibc.

https://sourceware.org/bugzilla/show_bug.cgi?id=10311

"If you use clone() you're on your own."

> my main worry is that the builtins encourage
> the use of raw syscalls and outside of libc
> development it is not well understood how to
> do that correctly

I don't think their use should be discouraged.
Linux system calls, at least.
The ones of other operating systems most definitely should.
Using them leads to breakage since the ABI is not stable.
The developers of the go programming language can attest to that.

> i guess it's ok if it is by default an error
> outside of -ffreestanding.

Hosted C programs could also make good use of them.
They could certainly start out exclusive to freestanding C
and then made available to general code if there's demand.

Re: [RFC] Linux system call builtins

[Szabolcs Nagy]

The 04/09/2024 23:59, Matheus Afonso Martins Moreira via Gcc wrote:
> > and using raw syscalls outside of the single runtime the
> > application is using is problematic (at least on linux).
> 
> Why do you say they are problematic on Linux though? Please elaborate.

because the portable c api layer and syscall abi layer
has a large enough gap that applications can break
libc internals by doing raw syscalls.

and it's not just the call convention that's target
specific (this makes the c syscall() function hard to
use on linux)

and linux evolves fast enough that raw syscalls have
to be adjusted over time (to support new features)
which is harder when they are all over the place
instead of in the libc only.

> 
> The ABI being stable should mean that I can for example
> strace a program, analyze the system calls and implement
> a new version of it that performs the same functions.

you could do that with syscall() but it is not very
useful as the state of the system is not the same
when you rerun a process so syscalls would likely
fail or do different things than in the first run.

> > clone cannot even be used from c code in general
> > as CLONE_VM is not compatible with c semantics
> > without a new stack (child clobbers the parent stack)
> > so the c builtin would not always work
> > it is also a syscall that only freestanding
> > application can use not something that calls
> > into the libc
> 
> There are major projects out there which do use it regardless.

that does not make it right.

> For example, systemd:
> 
> https://github.com/systemd/systemd/blob/main/src/basic/raw-clone.h
> https://github.com/systemd/systemd/blob/main/src/shared/async.h
> https://github.com/systemd/systemd/blob/main/src/shared/async.c
> https://github.com/systemd/systemd/blob/main/docs/CODING_STYLE.md
> 
> > even in a freestanding application it is tricky to use right
> 
> No argument from me there. It is tricky...
> The compiler should make it possible though.
> 
> > so i don't see why clone is the quintessential example.
> 
> I think it is the best example because attempting to use clone
> is not actually supported by glibc.
> 
> https://sourceware.org/bugzilla/show_bug.cgi?id=10311
> 
> "If you use clone() you're on your own."

should be

"if you use clone() *or* raw clone syscall then
 you're on your own"

which is roughly what i said in that discussion.

so your proposal does not fix this particular issue,
just provide a simpler footgun.

> > i guess it's ok if it is by default an error
> > outside of -ffreestanding.
> 
> Hosted C programs could also make good use of them.

they should not.

> They could certainly start out exclusive to freestanding C
> and then made available to general code if there's demand.

Re: [RFC] Linux system call builtins

[Paul Koning]

> On Apr 9, 2024, at 9:48 PM, Matheus Afonso Martins Moreira via Gcc <gcc@gcc.gnu.org> wrote:
> 
> ...
> MIPS calling conventions work like this:
> 
>> mips/n32,64 a0 a1 a2 a3 a4 a5
>> mips/o32    a0 a1 a2 a3 ...
>> mips/o32    args5-8 are passed on the stack

Yes, for regular function calls, but at least in the case of NetBSD, not for syscalls.  They have a somewhat odd calling convention that doesn't match any of the normal function call ABIs, though it's similar.

	paul

[RFC] Linux system call builtins

[Matheus Afonso Martins Moreira]

> because the portable c api layer and syscall abi layer
> has a large enough gap that applications can break
> libc internals by doing raw syscalls.

I think that problem cannot really be fixed.
System call users just have to be aware of it.

It's true that using certain system calls can clobber libc state.
However, it should be up to the programmer to decide whether or not
that's acceptable. The compiler should empower them regardless.

> and it's not just the call convention that's target
> specific (this makes the c syscall() function hard to
> use on linux)

Yes. On Linux, the ABIs are stable and the set of system calls
is only ever added to. However, this property does not hold
across different architectures. Some targets have several numbered
variants of a system call while others only have the latest version.
It's true that this is a source of complexity for system call users.

> and linux evolves fast enough that raw syscalls have
> to be adjusted over time (to support new features)
> which is harder when they are all over the place
> instead of in the libc only.

When those adjustments are done, they avoid breaking existing programs.
New versions of the system calls are created, old ones are maintained.
Existing binaries continue to work. Even statically linked binaries.
It's true that it's harder to update binaries all at once
when they are statically linked but Linux still supports them.

> that does not make it right.

I don't agree with that. I don't think the libc should be required
or that the system calls should be a privilege of the libc.
The libc should be entirely optional.

In the issue I linked you also noticed that the Linux manuals
frequently mix up the kernel perspective with that of the libc,
and that of glibc in particular.

> unfortunately the linux manuals mix the system call
> (linux behaviour) and libc api (glibc behaviour)
> in the same man page in general, and mainly focus
> on the linux behaviour, not on the c api semantics.

That's also something I intend to work on.

> so your proposal does not fix this particular issue,
> just provide a simpler footgun.

This issue cannot really be fixed. I believe the only solution to that
would be to impose a libc on all of Linux user space. I think that goes
against the spirit of Linux as a kernel that's completely independent
of its userspace.

The footgun already exists in inline assembly form regardless.

[RFC] Linux system call builtins

[Matheus Afonso Martins Moreira]

> Yes, for regular function calls,
> but at least in the case of NetBSD,
> not for syscalls.

Those are the registers Linux uses for system calls on MIPS.
They are documented as such here:

https://www.man7.org/linux/man-pages/man2/syscall.2.html

> The second table shows the registers used
> to pass the system call arguments.
>
> ...
>
> mips/o32       a0    a1    a2    a3
> mips/n32,64    a0    a1    a2    a3    a4    a5
>
> ...

So they match the normal function calling convention? That's neat.
I don't have much experience with MIPS so I didn't recognize it.
I'm not sure how NetBSD does system calls but I know the ABI
is not considered stable.