[RFC] Add createcontext and releasecontext to glibc 2.39

[RFC] Add createcontext and releasecontext to glibc 2.39

[H.J. Lu]

On Wed, Jan 10, 2024 at 3:49 PM Edgecombe, Rick P
<rick.p.edgecombe@intel.com> wrote:
>
> On Fri, 2023-12-29 at 08:43 -0800, H.J. Lu wrote:
> > +      long int ret
> > +       = __allocate_shadow_stack (((uintptr_t) sp
> > +                                   - (uintptr_t) ucp-
> > >uc_stack.ss_sp),
> > +                                  &ucp->__ssp[1]);
> > +      if (ret != 0)
> > +       {
> > +         /* FIXME: What should we do?  */
> > +         abort ();
> > +       }
>
> This is really because makecontext() doesn't normally do any
> allocations. The fix should be a new API that can return something for
> this failure, and also provides a mechanism to free the shadow stack.
> Or take a pointer to a shadow stack token, with the shadow stack
> allocated and managed by the caller.

I agree.  I think we should add

int createcontext(ucontext_t *ucp, void (*func)(), int argc, ...);

which is similar to makecontext, except that it can return an error
code.

void releasecontext(ucontext_t *ucp);

which should release all resources associated with UCP.

to the first version of the shadow stack enabled glibc as a GNU
extension.

I can create a patch this week.

--
H.J.

Re: [RFC] Add createcontext and releasecontext to glibc 2.39

[Edgecombe, Rick P]

On Wed, 2024-01-10 at 16:01 -0800, H.J. Lu wrote:
> > This is really because makecontext() doesn't normally do any
> > allocations. The fix should be a new API that can return something
> > for
> > this failure, and also provides a mechanism to free the shadow
> > stack.
> > Or take a pointer to a shadow stack token, with the shadow stack
> > allocated and managed by the caller.
> 
> I agree.  I think we should add
> 
> int createcontext(ucontext_t *ucp, void (*func)(), int argc, ...);
> 
> which is similar to makecontext, except that it can return an error
> code.
> 
> void releasecontext(ucontext_t *ucp);
> 
> which should release all resources associated with UCP.
> 
> to the first version of the shadow stack enabled glibc as a GNU
> extension.
> 
> I can create a patch this week.

createcontext() should probably take a shadow stack size in that case.
Please include the other arch's folks so we can try to make it work for
everyone.

Also, why have createcontext() do the allocation instead of having the
caller fill a pointer and size in ucontext_t (like for normal stacks)?
It might be possible for createcontext() callers to re-use shadow
stacks that have been swapped off of. Maybe naive question. But it is
not consistent with the ss_sp/sp_size interface, so wondering why favor
the other way.

Re: [RFC] Add createcontext and releasecontext to glibc 2.39

[H.J. Lu]

On Wed, Jan 10, 2024 at 4:12 PM Edgecombe, Rick P
<rick.p.edgecombe@intel.com> wrote:
>
> On Wed, 2024-01-10 at 16:01 -0800, H.J. Lu wrote:
> > > This is really because makecontext() doesn't normally do any
> > > allocations. The fix should be a new API that can return something
> > > for
> > > this failure, and also provides a mechanism to free the shadow
> > > stack.
> > > Or take a pointer to a shadow stack token, with the shadow stack
> > > allocated and managed by the caller.
> >
> > I agree.  I think we should add
> >
> > int createcontext(ucontext_t *ucp, void (*func)(), int argc, ...);
> >
> > which is similar to makecontext, except that it can return an error
> > code.
> >
> > void releasecontext(ucontext_t *ucp);
> >
> > which should release all resources associated with UCP.
> >
> > to the first version of the shadow stack enabled glibc as a GNU
> > extension.
> >
> > I can create a patch this week.
>
> createcontext() should probably take a shadow stack size in that case.
> Please include the other arch's folks so we can try to make it work for
> everyone.

The glibc developers are on this mailing list.  If you have additional people,
please add them.

> Also, why have createcontext() do the allocation instead of having the
> caller fill a pointer and size in ucontext_t (like for normal stacks)?
> It might be possible for createcontext() callers to re-use shadow
> stacks that have been swapped off of. Maybe naive question. But it is
> not consistent with the ss_sp/sp_size interface, so wondering why favor
> the other way.
>

The new interface should be easy to use.  If developers have to know
and manage shadow stack themselves, it may be hard to use and easy
to make mistakes.  If shadow stack is transparent to developers, it could
encourage adoption.

-- 
H.J.

Re: [RFC] Add createcontext and releasecontext to glibc 2.39

[Florian Weimer]

* H. J. Lu:

> I agree.  I think we should add
>
> int createcontext(ucontext_t *ucp, void (*func)(), int argc, ...);
>
> which is similar to makecontext, except that it can return an error
> code.
>
> void releasecontext(ucontext_t *ucp);
>
> which should release all resources associated with UCP.
>
> to the first version of the shadow stack enabled glibc as a GNU
> extension.

I'm not convinced that this is the interface that is most appropriate
for application needs.  It's weirdly asymmetric to allocate the shadow
stack, but not the regular stack.

We have more interfaces that accept user-allocated stacks:
pthread_create with suitable attributes, and sigaltstack come to my
mind.  It may be more natural to change the way stacks are allocated and
freed, and pass a stack descriptor object (providing information on
stack and shadow stack location) to all functions that need stacks.

The other question is what we are trying to do here.  Is it just achieve
full coverage for all glibc APIs, or are there applications we want to
port to shadow stack?  I suspect applications will generally shy away
from setcontext etc. for their couroutine/one-shot continuation/fiber
needs because they are too slow due to the unnecessary signal mask
update, and maybe due to floating point context updates as well.  (These
applications tend to use setjmp/longjmp instead.)  In the other
direction, other thread state that application expects to be switched is
not (like TLS or some other thread properties).

For most forms of userspace context switching, we also need a way to
inform GCC that the TCB address has changed in a live stack frame.  This
required to enable safe resumption on a different thread, not the
original suspending thread.  It probably requires a function attribute
(that has to propagate outwards).

These open issues suggest to me that this is not glibc 2.39 material.

Thanks,
Florian

Re: [RFC] Add createcontext and releasecontext to glibc 2.39

[Andreas K. Huettel]

[-- Attachment #1: Type: text/plain, Size: 1623 bytes --]

Am Donnerstag, 11. Januar 2024, 01:01:11 CET schrieb H.J. Lu:
> On Wed, Jan 10, 2024 at 3:49 PM Edgecombe, Rick P
> <rick.p.edgecombe@intel.com> wrote:
> >
> > On Fri, 2023-12-29 at 08:43 -0800, H.J. Lu wrote:
> > > +      long int ret
> > > +       = __allocate_shadow_stack (((uintptr_t) sp
> > > +                                   - (uintptr_t) ucp-
> > > >uc_stack.ss_sp),
> > > +                                  &ucp->__ssp[1]);
> > > +      if (ret != 0)
> > > +       {
> > > +         /* FIXME: What should we do?  */
> > > +         abort ();
> > > +       }
> >
> > This is really because makecontext() doesn't normally do any
> > allocations. The fix should be a new API that can return something for
> > this failure, and also provides a mechanism to free the shadow stack.
> > Or take a pointer to a shadow stack token, with the shadow stack
> > allocated and managed by the caller.
> 
> I agree.  I think we should add
> 
> int createcontext(ucontext_t *ucp, void (*func)(), int argc, ...);
> 
> which is similar to makecontext, except that it can return an error
> code.
> 
> void releasecontext(ucontext_t *ucp);
> 
> which should release all resources associated with UCP.
> 
> to the first version of the shadow stack enabled glibc as a GNU
> extension.
> 
> I can create a patch this week.

It's a but late in the cycle for completely new API ideas. 

I would suggest to postpone it for 2.40, then there is time for more
discussion.

-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer
(council, toolchain, base-system, perl, libreoffice)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [RFC] Add createcontext and releasecontext to glibc 2.39

[Edgecombe, Rick P]

On Thu, 2024-01-11 at 13:57 +0100, Florian Weimer wrote:
> The other question is what we are trying to do here.  Is it just
> achieve
> full coverage for all glibc APIs, or are there applications we want
> to
> port to shadow stack? 

Along these lines, since the current solution leaks shadow stacks, is
it better to try to keep the apps from crashing this way? Or, since the
loader now requires an environment variable to be set, just not support
ucontext for 2.39.

Or maybe it could warn somehow initially and use the leaky solution. So
people don't get confused why there app is leaking memory. I don't like
the silent leaking.