Re: Candidate Linux ABI for Intel AMX and hypothetical new related features

[Florian Weimer <fweimer@redhat.com>]

* Andy Lutomirski:

> Optionally we make AVX-512 also default off, which fixes what is
> arguably a serious ABI break with AVX-512: lots of programs, following
> POSIX (!), seem to think that they know much much space to allocate
> for sigaltstack().   AVX-512 is too big.
>
> Thoughts?

Maybe we could have done this in 2016 when I reported this for the first
time.  Now it is too late, as more and more software is using
CPUID-based detection for AVX-512.  Our users have been using AVX-512
hardware for quite some time now, and I haven't seen *that* many issues
resulting from the context size.  That isn't to say that problems do not
exist, but they are more of the kind that the increased stack usage
means that areas of the stack that used to be zero no longer are, so
users encounter different side effects from uninitialized-variable bugs.

How much software depends on the signal handler data layout?  The %zmm
state does not appear to be exposed today, so perhaps some savings could
be had there.

The suggestion to make CPUID trap doesn't sound workable to me.  At
least in the past, it's been suggested as a serializing instruction to
be used alongside RDTSC, which makes it rather time-critical for some
applications.

Even today, signal handlers do not really compose well in the sense that
multiple libraries can use them and collaborate without being aware of
each other (like they can divide up TLS memory with the help of the
dynamic linker, or carve out address space using mmap).  Proposals to
set special process-wide flags only make that situation worse.  Code
that installs a signal handler often does not have control on which
thread an asynchronous signal is delivered, or which code it interrupts.
A single process-wide flag cannot capture that accurately, even if it is
per signal number.

There is some work on handlers that do not save GP registers, for
userspace interrupts.  But I do not think that generalizes to existing
signal handlers because floating point is widely used in signal handlers
today (at least implicitly).

The rseq extension might work conceptually, but it requires to make
operations idempotent, with periodic checkpoint, and of course
inline/flatten all calls.  And it requires compiler work, the present
model based on inline asm hacks doesn't look workable.  Maybe that works
for AMX.  I have not checked if there is yet any public documentation of
the programming model.

I think someone expressed the sentiment (maybe on another thread) that
the current CPU feature enablement process does not work.  I do not
agree.  Currently it is only necessary to upgrade the kernel and maybe
glibc (but not in all cases), and then you are good to go.  You can keep
using your old libraries, old compilers, and even old assemblers if you
are okay with .byte hacks.  You do not need special userspace libraries,
new compilers for different languages, special firmware or binary blobs.
Overall, it just works.

On x86, we are really bad about actually using CPU features pervasively,
but that is a different story.

Thanks,
Florian