Re: [PATCH] Linux: Switch back to assembly syscall wrapper for prctl (bug 29770)

[Florian Weimer <fweimer@redhat.com>]

* Andreas Schwab:

> On Feb 13 2024, Florian Weimer wrote:
>
>> “
>> Commit ff026950e280bc3e9487b41b460fb31bc5b57721 ("Add a C wrapper for
>> prctl [BZ #25896]") replaced the assembler wrapper with a C function.
>> However, on powerpc64le-linux-gnu, the C variadic function
>> implementation requires extra work in the caller to set up the parameter
>> save area.  Calling a function that needs a parameter save area without
>> one (because the prototype used indicates the function is not variadic)
>> corrupts the caller's stack.  Switch back to the assembler
>> implementation on most targets and only keep the C implementation for
>> x86-64 x32.
>> ”
>
> That does not explain why the compiler did not set up the parameter save
> area even though the declaration in <sys/prctl.h> is varadic.  Do I
> understand correctly that some software uses a private declaration that
> is prototyped but non-variadic?

Yes, GCC and LLVM upstream contain an unprototyped prctl function
declaration somewhere:

  libasan uses incorrect prctl prototype 
  <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113728>

This is not the only piece of software with this problem.  It's easier
to fix this in glibc than to figure out how to change the libasan
sanitizer to change the prototype there, so that's why I prefer this
approach.

Thanks,
Florian