Pthread QOS interface

Pthread QOS interface

[H.J. Lu]

Hi,

MacOS has Pthread QOS interface:

https://opensource.apple.com/source/libpthread/libpthread-416.60.2/pthread/qos.h.auto.html

Can we do something similar or extend the current scheduling interface
to provide
QOS?

-- 
H.J.

Re: Pthread QOS interface

[Len Brown]

I have a more specific question.

Can we implement EXACTLY the same interface as Apple has -- so a
cross-platform application doesn't have to change?  (I see an Apple
source license on that qos.h -- is that an issue with an API?)

(I think the answer about the scheduling interface is simply "yes", we
can add new Linux-specific flags to the existing sched_setattr(2) --
but ideally applications should be using a more portable interface,
rather than directly invoking linux-specific system calls)

thanks,
-Len

On Tue, May 17, 2022 at 1:03 PM H.J. Lu <hjl.tools@gmail.com> wrote:
>
> Hi,
>
> MacOS has Pthread QOS interface:
>
> https://opensource.apple.com/source/libpthread/libpthread-416.60.2/pthread/qos.h.auto.html
>
> Can we do something similar or extend the current scheduling interface
> to provide
> QOS?
>
> --
> H.J.



-- 
Len Brown, Intel Open Source Technology Center

Re: Pthread QOS interface

[Florian Weimer]

* Len Brown via Libc-alpha:

> I have a more specific question.
>
> Can we implement EXACTLY the same interface as Apple has -- so a
> cross-platform application doesn't have to change?  (I see an Apple
> source license on that qos.h -- is that an issue with an API?)
>
> (I think the answer about the scheduling interface is simply "yes", we
> can add new Linux-specific flags to the existing sched_setattr(2) --
> but ideally applications should be using a more portable interface,
> rather than directly invoking linux-specific system calls)

Isn't this traditionally handled by cgroups and/or rtkit?

If we can get a syscall interface for this from the kernel, we could add
something in parallel to that to glibc, but I doubt it would have the
same semantics as the Darwin interface even if we keep the API the same.
For example, Linux may require different permissions for making these
changes, and apply them to different aspects of system behavior (CPU,
mass storage I/O, memory bandwidth, network traffic).

Thanks,
Florian

Re: Pthread QOS interface

[Len Brown]

On Wed, May 18, 2022 at 12:19 AM Florian Weimer <fweimer@redhat.com> wrote:

> > Can we implement EXACTLY the same interface as Apple has -- so a
> > cross-platform application doesn't have to change?  (I see an Apple
> > source license on that qos.h -- is that an issue with an API?)

> Isn't this traditionally handled by cgroups and/or rtkit?

Yes, cgroups have been used to address some of this in the past.  But
cgroups have their benefits and weaknesses.

The fundamental problem with cgroups is that somebody with a lot of
knowledge has to manage both what they do, and what tasks belong to
what group.  Further, cgroups only work in a strict hierarchy.

An API, on the other hand, allows a task to say something that it may
actually know about -- what kind of task it is -- and not require it
or anybody outside the kernel to know, for example, how the scheduler
may want to schedule a background task relative to other tasks at a
particular time.

What is rkit?

> If we can get a syscall interface for this from the kernel, we could add
> something in parallel to that to glibc, but I doubt it would have the
> same semantics as the Darwin interface even if we keep the API the same.
> For example, Linux may require different permissions for making these
> changes, and apply them to different aspects of system behavior (CPU,
> mass storage I/O, memory bandwidth, network traffic).

Yes, this is what I propose.  I can supply/propose the system call
interface.  My question is if there is hope that pthreads can be
plumbed to talk to it.

The semantics of the Darwin interface are documented, and Indeed, I
propose we match them EXACTLY.  For that is what the multi-platform
app wants.

Of course, under the hood, the implementation will be entirely
different, as the OS under the hood is entirely different, but he APP
will be giving each OS exactly the same message to work with.

Yes, under the hood on Linux, some scheduler attributes require more
than default permission.  We can grant that permission in the
important cases.  We can also still make a positive difference in
places where no permission is required.

thanks,
-Len Brown, Intel

Re: Pthread QOS interface

[Florian Weimer]

* Len Brown:

> On Wed, May 18, 2022 at 12:19 AM Florian Weimer <fweimer@redhat.com> wrote:
>
>> > Can we implement EXACTLY the same interface as Apple has -- so a
>> > cross-platform application doesn't have to change?  (I see an Apple
>> > source license on that qos.h -- is that an issue with an API?)
>
>> Isn't this traditionally handled by cgroups and/or rtkit?
>
> Yes, cgroups have been used to address some of this in the past.  But
> cgroups have their benefits and weaknesses.
>
> The fundamental problem with cgroups is that somebody with a lot of
> knowledge has to manage both what they do, and what tasks belong to
> what group.  Further, cgroups only work in a strict hierarchy.
>
> An API, on the other hand, allows a task to say something that it may
> actually know about -- what kind of task it is -- and not require it
> or anybody outside the kernel to know, for example, how the scheduler
> may want to schedule a background task relative to other tasks at a
> particular time.
>
> What is rkit?

rtkit is the workaround for the kernel permission check certain
scheduling policies.

>> If we can get a syscall interface for this from the kernel, we could add
>> something in parallel to that to glibc, but I doubt it would have the
>> same semantics as the Darwin interface even if we keep the API the same.
>> For example, Linux may require different permissions for making these
>> changes, and apply them to different aspects of system behavior (CPU,
>> mass storage I/O, memory bandwidth, network traffic).
>
> Yes, this is what I propose.  I can supply/propose the system call
> interface.  My question is if there is hope that pthreads can be
> plumbed to talk to it.

If there is a matching system call interface, I see no reason not to
wrap it in glibc.

What will be problematic is to emulate the Darwin interface on top of a
slightly incompatible or incomplete interface (e.g., something that is
missing pthread_override_qos_class_start_np).  As long as the kernel
maintains all the data, we should be fine.

We can store unlimited data in pthread_attr_t nowadays without impact
ABI, so there are no challenges in that area.

Thanks,
Florian