C11 threads ABI questions - enum values

C11 threads ABI questions - enum values

[Rich Felker]

Hi everyone,

Is there any progress on the C11 threads implementation for glibc? I
would at least like to solidify one ABI question, since we have
someone working on C11 threads for musl and I want to use matching
values for the constants in threads.h. This header defines several
enumeration constants for mutex types and result/error codes:

mtx_plain
mtx_recursive
mtx_timed
thrd_timedout
thrd_success
thrd_busy
thrd_error
thrd_nomem

Of the above, mtx_recursive is a flag which can be or'd with either
mtx_plain or mtx_timed. The rest are independent values.

For the result codes, Jens Gustedt has proposed matching their
definitions with "corresponding" POSIX errno codes. This potentially
allows some of the C11 functions to be implemented as aliases for (or
tail calls to) POSIX threads functions, but some care is needed since
the semantics for the error codes and which conditions they're
returned in may not be exactly the same. I'm generally not a fan of
this approach, since it pulls in a dependency of target-specific
values (the errno values vary widely per target) in a new header that
could otherwise be largely arch-generic and requires synchronizing
these values without actually including <errno.h> (which is outside
the namespace for this header). I would prefer just assigning enum
values sequentially from 0 with 0 being success. But I want to see
what glibc folks want to do.

For the mtx types, mtx_plain and mtx_timed really could both have the
same value; I can't see there being any reasonable implementation
where it would matter at initialization time whether you want to
perform timed operations on the mutex or not. And the standard does
not seem to preclude this. In that case, the values of mtx_plain and
mtx_timed could match the value of PTHREAD_MUTEX_NORMAL and the values
or'd with mtx_recursive could match PTHREAD_MUTEX_RECURSIVE. Of course
doing this is not necessary and I'm open to other choices of values.

Rich

Re: C11 threads ABI questions - enum values

[Juan Manuel Torres Palma]

I was supposed to be working on this but as I said before I'm not
having as much time as I would like to.

> For the result codes, Jens Gustedt has proposed matching their
> definitions with "corresponding" POSIX errno codes. This potentially
> allows some of the C11 functions to be implemented as aliases for (or
> tail calls to) POSIX threads functions, but some care is needed since
> the semantics for the error codes and which conditions they're
> returned in may not be exactly the same. I'm generally not a fan of
> this approach, since it pulls in a dependency of target-specific
> values (the errno values vary widely per target) in a new header that
> could otherwise be largely arch-generic and requires synchronizing
> these values without actually including <errno.h> (which is outside
> the namespace for this header). I would prefer just assigning enum
> values sequentially from 0 with 0 being success. But I want to see
> what glibc folks want to do.

In this topic I think the same way than you do, specially if we want
to forbid users to do things like matching and comparing types from
POSIX to C11 threads, so if we use POSIX as a base, we should then
isolate it to make sure we isolate both libraries.

> For the mtx types, mtx_plain and mtx_timed really could both have the
> same value; I can't see there being any reasonable implementation
> where it would matter at initialization time whether you want to
> perform timed operations on the mutex or not. And the standard does
> not seem to preclude this. In that case, the values of mtx_plain and
> mtx_timed could match the value of PTHREAD_MUTEX_NORMAL and the values
> or'd with mtx_recursive could match PTHREAD_MUTEX_RECURSIVE. Of course
> doing this is not necessary and I'm open to other choices of values.

Well in this case I'm not really sure what to do with mtx_plain and
mtx_timed, but if they are defined in a different way in the C11
standard, we need to stick to the standard definition, don't we? I
don't really understand why it's useful to match values between
libraries. I know it easier to implement and makes the coding way
easier but maybe we can get some not desired uses of them.

I'm not an expert so I just want to know what others think.

Cheers.


-- 
Juan Manuel "Lolo" Torres Palma.
Computer Science Student at Universidad de Granada.

Re: C11 threads ABI questions - enum values

[Alexander Monakov]

On Sun, 27 Jul 2014, Rich Felker wrote:
> For the mtx types, mtx_plain and mtx_timed really could both have the
> same value; I can't see there being any reasonable implementation
> where it would matter at initialization time whether you want to
> perform timed operations on the mutex or not. And the standard does
> not seem to preclude this. In that case, the values of mtx_plain and
> mtx_timed could match the value of PTHREAD_MUTEX_NORMAL and the values
> or'd with mtx_recursive could match PTHREAD_MUTEX_RECURSIVE. Of course
> doing this is not necessary and I'm open to other choices of values.

In case the ultimate decision is _not_ to match new enum values with existing
pthread constants, then in my opinion it's preferable to keep new enum values
distinct for the sake of future debug or emulation layers that might prefer to
know in advance the kind of mutex the application meant to use.

Alexander

Re: C11 threads ABI questions - enum values

[Rich Felker]

On Mon, Aug 18, 2014 at 04:25:57PM +0400, Alexander Monakov wrote:
> On Sun, 27 Jul 2014, Rich Felker wrote:
> > For the mtx types, mtx_plain and mtx_timed really could both have the
> > same value; I can't see there being any reasonable implementation
> > where it would matter at initialization time whether you want to
> > perform timed operations on the mutex or not. And the standard does
> > not seem to preclude this. In that case, the values of mtx_plain and
> > mtx_timed could match the value of PTHREAD_MUTEX_NORMAL and the values
> > or'd with mtx_recursive could match PTHREAD_MUTEX_RECURSIVE. Of course
> > doing this is not necessary and I'm open to other choices of values.
> 
> In case the ultimate decision is _not_ to match new enum values with existing
> pthread constants, then in my opinion it's preferable to keep new enum values
> distinct for the sake of future debug or emulation layers that might prefer to
> know in advance the kind of mutex the application meant to use.

Yes, I think there's some value in being able to track what the
calling application _asked_ for, e.g. in the debugger, even if the
request is meaningless from an implementation standpoint.

After thinking about it more, I'm probably mildly in favor of using
enum constants unrelated to the values of the pthread ones (at least
not in any official relationship).

Rich

Re: C11 threads ABI questions - enum values

[Rich Felker]

On Sun, Aug 17, 2014 at 09:15:25AM +0200, Juan Manuel Torres Palma wrote:
> I was supposed to be working on this but as I said before I'm not
> having as much time as I would like to.
> 
> > For the result codes, Jens Gustedt has proposed matching their
> > definitions with "corresponding" POSIX errno codes. This potentially
> > allows some of the C11 functions to be implemented as aliases for (or
> > tail calls to) POSIX threads functions, but some care is needed since
> > the semantics for the error codes and which conditions they're
> > returned in may not be exactly the same. I'm generally not a fan of
> > this approach, since it pulls in a dependency of target-specific
> > values (the errno values vary widely per target) in a new header that
> > could otherwise be largely arch-generic and requires synchronizing
> > these values without actually including <errno.h> (which is outside
> > the namespace for this header). I would prefer just assigning enum
> > values sequentially from 0 with 0 being success. But I want to see
> > what glibc folks want to do.
> 
> In this topic I think the same way than you do, specially if we want
> to forbid users to do things like matching and comparing types from
> POSIX to C11 threads, so if we use POSIX as a base, we should then
> isolate it to make sure we isolate both libraries.

Even if, implementation-internally, some types can be used
interchangibly, I think it was decided in earlier discussion that such
external usage should be explicitly unsupported, to allow for the
implementations to diverge or be completely independent in the future.

> > For the mtx types, mtx_plain and mtx_timed really could both have the
> > same value; I can't see there being any reasonable implementation
> > where it would matter at initialization time whether you want to
> > perform timed operations on the mutex or not. And the standard does
> > not seem to preclude this. In that case, the values of mtx_plain and
> > mtx_timed could match the value of PTHREAD_MUTEX_NORMAL and the values
> > or'd with mtx_recursive could match PTHREAD_MUTEX_RECURSIVE. Of course
> > doing this is not necessary and I'm open to other choices of values.
> 
> Well in this case I'm not really sure what to do with mtx_plain and
> mtx_timed, but if they are defined in a different way in the C11
> standard, we need to stick to the standard definition, don't we? I

The C11 standard just says that the enumeration constants need to be
defined, but nothing about their values, or even whether the values
are distinct, so that's really up to us. As Alexander Monakov noted,
however, there does seem to be value in having them be distinct.

> don't really understand why it's useful to match values between
> libraries. I know it easier to implement and makes the coding way
> easier but maybe we can get some not desired uses of them.

Being that mtx_init will presumably be a wrapper that basically does
something like:

__pthread_mutexattr_init
__pthread_mutexattr_settype
__pthread_mutex_init
__pthread_mutexattr_destroy

I don't see any problem having some trivial code to map the C11 type
codes to POSIX type codes before calling __pthread_mutexattr_settype.

To start making some progress, here's a proposed list of values for
enumeration and macro constants in C11 threads.h:

/* Matching what Jens Gustedt proposed for musl, and the order in C
 * standard; also ensuring non-plain types work as bit flags. */
mtx_plain = 0
mtx_recursive = 1
mtx_timed = 2

/* Matching order in the C standard, except that timedout is moved to
 * the end because success should obviously, morally, be zero. :) */
thrd_success = 0
thrd_busy = 1
thrd_error = 2
thrd_nomem = 3
thrd_timedout = 4

/* Matching current pthreads ABI. */
ONCE_FLAG_INIT = 0
TSS_DTOR_ITERATIONS = 4

Did I miss anything? I think the only ones that might be at all
controversial are the thrd_* error codes, and only from a bikeshed
standpoint. Any opinions?

As discussed before, the intent is for all _types_ to match the
corresponding pthread type in size/alignment.

Rich

Re: C11 threads ABI questions - enum values

[Juan Manuel Torres Palma]

I agree with the values chosen, and the error codes make sense to me.

The thrd_timedout should be the exit value when a thread can't finish
in a given time right?



2014-08-18 21:49 GMT+02:00 Rich Felker <dalias@libc.org>:
> On Sun, Aug 17, 2014 at 09:15:25AM +0200, Juan Manuel Torres Palma wrote:
>> I was supposed to be working on this but as I said before I'm not
>> having as much time as I would like to.
>>
>> > For the result codes, Jens Gustedt has proposed matching their
>> > definitions with "corresponding" POSIX errno codes. This potentially
>> > allows some of the C11 functions to be implemented as aliases for (or
>> > tail calls to) POSIX threads functions, but some care is needed since
>> > the semantics for the error codes and which conditions they're
>> > returned in may not be exactly the same. I'm generally not a fan of
>> > this approach, since it pulls in a dependency of target-specific
>> > values (the errno values vary widely per target) in a new header that
>> > could otherwise be largely arch-generic and requires synchronizing
>> > these values without actually including <errno.h> (which is outside
>> > the namespace for this header). I would prefer just assigning enum
>> > values sequentially from 0 with 0 being success. But I want to see
>> > what glibc folks want to do.
>>
>> In this topic I think the same way than you do, specially if we want
>> to forbid users to do things like matching and comparing types from
>> POSIX to C11 threads, so if we use POSIX as a base, we should then
>> isolate it to make sure we isolate both libraries.
>
> Even if, implementation-internally, some types can be used
> interchangibly, I think it was decided in earlier discussion that such
> external usage should be explicitly unsupported, to allow for the
> implementations to diverge or be completely independent in the future.
>
>> > For the mtx types, mtx_plain and mtx_timed really could both have the
>> > same value; I can't see there being any reasonable implementation
>> > where it would matter at initialization time whether you want to
>> > perform timed operations on the mutex or not. And the standard does
>> > not seem to preclude this. In that case, the values of mtx_plain and
>> > mtx_timed could match the value of PTHREAD_MUTEX_NORMAL and the values
>> > or'd with mtx_recursive could match PTHREAD_MUTEX_RECURSIVE. Of course
>> > doing this is not necessary and I'm open to other choices of values.
>>
>> Well in this case I'm not really sure what to do with mtx_plain and
>> mtx_timed, but if they are defined in a different way in the C11
>> standard, we need to stick to the standard definition, don't we? I
>
> The C11 standard just says that the enumeration constants need to be
> defined, but nothing about their values, or even whether the values
> are distinct, so that's really up to us. As Alexander Monakov noted,
> however, there does seem to be value in having them be distinct.
>
>> don't really understand why it's useful to match values between
>> libraries. I know it easier to implement and makes the coding way
>> easier but maybe we can get some not desired uses of them.
>
> Being that mtx_init will presumably be a wrapper that basically does
> something like:
>
> __pthread_mutexattr_init
> __pthread_mutexattr_settype
> __pthread_mutex_init
> __pthread_mutexattr_destroy
>
> I don't see any problem having some trivial code to map the C11 type
> codes to POSIX type codes before calling __pthread_mutexattr_settype.
>
> To start making some progress, here's a proposed list of values for
> enumeration and macro constants in C11 threads.h:
>
> /* Matching what Jens Gustedt proposed for musl, and the order in C
>  * standard; also ensuring non-plain types work as bit flags. */
> mtx_plain = 0
> mtx_recursive = 1
> mtx_timed = 2
>
> /* Matching order in the C standard, except that timedout is moved to
>  * the end because success should obviously, morally, be zero. :) */
> thrd_success = 0
> thrd_busy = 1
> thrd_error = 2
> thrd_nomem = 3
> thrd_timedout = 4
>
> /* Matching current pthreads ABI. */
> ONCE_FLAG_INIT = 0
> TSS_DTOR_ITERATIONS = 4
>
> Did I miss anything? I think the only ones that might be at all
> controversial are the thrd_* error codes, and only from a bikeshed
> standpoint. Any opinions?
>
> As discussed before, the intent is for all _types_ to match the
> corresponding pthread type in size/alignment.
>
> Rich



-- 
Juan Manuel "Lolo" Torres Palma.
Computer Science Student at Universidad de Granada.

Re: C11 threads ABI questions - enum values

[Rich Felker]

On Tue, Aug 26, 2014 at 01:19:27PM +0200, Juan Manuel Torres Palma wrote:
> I agree with the values chosen, and the error codes make sense to me.
> 
> The thrd_timedout should be the exit value when a thread can't finish
> in a given time right?

No, it's for timedwait operations that timeout.

Rich

C11 threads ABI - mtx_t and cnd_t types

[Rich Felker]

Another issue I have on the ABI for C11 threads pertains to the types
for mtx_t and cnd_t. My understanding, and I agree with this, is that
it was already decided to use the same underlying sizes/alignment, and
for now representations, as the corresponding POSIX types. However,
it's impossible to use the existing types directly due to namespace
issues. If the following were used:

typedef pthread_mutex_t mtx_t;
typedef pthread_cond_t cnd_t;

this would require pthread_mutex_t and pthread_cond_t to be visible
when threads.h is included.

It's possible to consider adding union tags (the underlying types of
pthread_mutex_t and pthread_cond_t are both unions without tags) in a
protected namespace, so that you could have:

union __mutex { ... };
typedef union __mutex pthread_mutex_t;
typedef union __mutex mtx_t;

but this is not acceptable because it would change the C++ ABI for the
pthread types. Since they currently do not have tags, the effective
tag name (for C++ ABI/manging purposes) for the union defining
pthread_mutex_t comes from the typedef name, pthread_mutex_t. Adding
an explicit tag would change this (unless the explicit tag were still
pthread_mutex_t, but then it's not in the namespace that can be used
in threads.h).

This issue has been discussed in detail (I might say, to exhaustion)
on the musl libc mailing list, and the conclusion we reached is that
pthread_mutex_t and mtx_t (and likewise pthread_cond_t and cnd_t)
should be defined with identical types (we happen to have a structure
rather than a union for the outermost layer in musl, but that's an
irrelevant difference), both without tags. This will give effective
C++ tag names (for name mangling ABI) of mtx_t and cnd_t for the C11
types.

The rationale for this conclusion is to allow a shared implementation.
If pthread_mutex_t and mtx_t are not compatible types, then it's an
aliasing violation for pthread_mutex_* functions to use the ->
operator to access the members of mtx_t as if it were an object of
type pthread_mutex_t -- this is what would happen when an application
declares an object of type mtx_t and passes it to mtx_*(), which in
turn calls pthread_mutex_*() and dereferences. By having both be
tagless with the same definitions, we make the instance of either type
in a given translation unit a compatible type with the instance of
either one in a separate translation unit (but neither is compatible
with the other within the same translation unit).

If anyone wants to check this reasoning, the relevant portions of C11
seem to be 6.2.7p1 and 6.5p7.

TL;DR:

mtx_t and cnd_t should be defined without tags and with definitions
identical to the corresponding pthread types. Other choices of
definition make it difficult to implement the C11 functions as pure
wrappers around the pthread functions without risking aliasing
violations which may result in actual breakage at some point in the
future with LTO enabled.

Rich

Re: C11 threads ABI - mtx_t and cnd_t types

[pinskia]

> On Aug 30, 2014, at 7:52 PM, Rich Felker <dalias@libc.org> wrote:
> 
> Another issue I have on the ABI for C11 threads pertains to the types
> for mtx_t and cnd_t. My understanding, and I agree with this, is that
> it was already decided to use the same underlying sizes/alignment, and
> for now representations, as the corresponding POSIX types. However,
> it's impossible to use the existing types directly due to namespace
> issues. If the following were used:
> 
> typedef pthread_mutex_t mtx_t;
> typedef pthread_cond_t cnd_t;
> 
> this would require pthread_mutex_t and pthread_cond_t to be visible
> when threads.h is included.
> 
> It's possible to consider adding union tags (the underlying types of
> pthread_mutex_t and pthread_cond_t are both unions without tags) in a
> protected namespace, so that you could have:
> 
> union __mutex { ... };
> typedef union __mutex pthread_mutex_t;
> typedef union __mutex mtx_t;

Why not do:
union __mutex {....}
Struct pthread_mutex_t {__mutex d; };
typedef union __mutex mtx_t;

That won't change the name mangling for pthread_mutex_t and allow for mtx_t not be a tagged one. 

Thanks,
Andrew


> 
> but this is not acceptable because it would change the C++ ABI for the
> pthread types. Since they currently do not have tags, the effective
> tag name (for C++ ABI/manging purposes) for the union defining
> pthread_mutex_t comes from the typedef name, pthread_mutex_t. Adding
> an explicit tag would change this (unless the explicit tag were still
> pthread_mutex_t, but then it's not in the namespace that can be used
> in threads.h).
> 
> This issue has been discussed in detail (I might say, to exhaustion)
> on the musl libc mailing list, and the conclusion we reached is that
> pthread_mutex_t and mtx_t (and likewise pthread_cond_t and cnd_t)
> should be defined with identical types (we happen to have a structure
> rather than a union for the outermost layer in musl, but that's an
> irrelevant difference), both without tags. This will give effective
> C++ tag names (for name mangling ABI) of mtx_t and cnd_t for the C11
> types.
> 
> The rationale for this conclusion is to allow a shared implementation.
> If pthread_mutex_t and mtx_t are not compatible types, then it's an
> aliasing violation for pthread_mutex_* functions to use the ->
> operator to access the members of mtx_t as if it were an object of
> type pthread_mutex_t -- this is what would happen when an application
> declares an object of type mtx_t and passes it to mtx_*(), which in
> turn calls pthread_mutex_*() and dereferences. By having both be
> tagless with the same definitions, we make the instance of either type
> in a given translation unit a compatible type with the instance of
> either one in a separate translation unit (but neither is compatible
> with the other within the same translation unit).
> 
> If anyone wants to check this reasoning, the relevant portions of C11
> seem to be 6.2.7p1 and 6.5p7.
> 
> TL;DR:
> 
> mtx_t and cnd_t should be defined without tags and with definitions
> identical to the corresponding pthread types. Other choices of
> definition make it difficult to implement the C11 functions as pure
> wrappers around the pthread functions without risking aliasing
> violations which may result in actual breakage at some point in the
> future with LTO enabled.
> 
> Rich

Re: C11 threads ABI - mtx_t and cnd_t types

[Rich Felker]

On Sun, Aug 31, 2014 at 01:48:35AM -0700, pinskia@gmail.com wrote:
> 
> 
> > On Aug 30, 2014, at 7:52 PM, Rich Felker <dalias@libc.org> wrote:
> > 
> > Another issue I have on the ABI for C11 threads pertains to the types
> > for mtx_t and cnd_t. My understanding, and I agree with this, is that
> > it was already decided to use the same underlying sizes/alignment, and
> > for now representations, as the corresponding POSIX types. However,
> > it's impossible to use the existing types directly due to namespace
> > issues. If the following were used:
> > 
> > typedef pthread_mutex_t mtx_t;
> > typedef pthread_cond_t cnd_t;
> > 
> > this would require pthread_mutex_t and pthread_cond_t to be visible
> > when threads.h is included.
> > 
> > It's possible to consider adding union tags (the underlying types of
> > pthread_mutex_t and pthread_cond_t are both unions without tags) in a
> > protected namespace, so that you could have:
> > 
> > union __mutex { ... };
> > typedef union __mutex pthread_mutex_t;
> > typedef union __mutex mtx_t;
> 
> Why not do:
> union __mutex {....}
> Struct pthread_mutex_t {__mutex d; };
> typedef union __mutex mtx_t;
> 
> That won't change the name mangling for pthread_mutex_t and allow for mtx_t not be a tagged one. 

This is potentially also legal (modulo some minor details you messed
up, like changing the definition of pthread_mutex_t from a tagless
union to a tagged structure), but unlike what I recommended it
requires internal changes to the implementation to avoid illegal
aliasing. If you have an object of type T and an aggregate type A
containing an object of type T as its only member, you cannot access
the object of type T as if it had type A. You would instead have to
cast the A* to T* before dereferencing it.

Note that what I proposed (tagless unions for mtx_t and cnd_t with the
same definitions as the pthread types) also supports an implementation
like the above where you cast to a pointer to the contained type
(e.g., currently, struct __pthread_mutex_s), but it's not necessary
since the original types are compatible due to beging tagless and
having the same members.

Rich

Re: C11 threads ABI questions - enum values

[Roland McGrath]

> Yes, I think there's some value in being able to track what the
> calling application _asked_ for, e.g. in the debugger, even if the
> request is meaningless from an implementation standpoint.
> 
> After thinking about it more, I'm probably mildly in favor of using
> enum constants unrelated to the values of the pthread ones (at least
> not in any official relationship).

I concur on both points.

Re: C11 threads ABI questions - enum values

[Rich Felker]

On Wed, Oct 01, 2014 at 02:13:08PM -0700, Roland McGrath wrote:
> > Yes, I think there's some value in being able to track what the
> > calling application _asked_ for, e.g. in the debugger, even if the
> > request is meaningless from an implementation standpoint.
> > 
> > After thinking about it more, I'm probably mildly in favor of using
> > enum constants unrelated to the values of the pthread ones (at least
> > not in any official relationship).
> 
> I concur on both points.

Thanks for the feedback. Since then we've got C11 threads support
committed in musl, with the following definitions. Are these
acceptable for glibc too? They're the same ones I proposed before.

enum {
	thrd_success  = 0,
	thrd_busy     = 1,
	thrd_error    = 2,
	thrd_nomem    = 3,
	thrd_timedout = 4,
};

enum {
	mtx_plain     = 0,
	mtx_recursive = 1,
	mtx_timed     = 2,
};

Rich

Re: C11 threads ABI questions - enum values

[Roland McGrath]

> Thanks for the feedback. Since then we've got C11 threads support
> committed in musl, with the following definitions. Are these
> acceptable for glibc too? They're the same ones I proposed before.

No guarantees until we actually implement something in glibc.  But they
seem fine to me.  The possible exception is that for interfaces where the
only valid thing is to pass in of the specified enum values, there is
something to be said for none of those values being zero (or -1) so that
applications can't sloppily use literal 0 (or uninitialized statics or the
like) and have it work but be non-portable and nonconforming.

Re: C11 threads ABI questions - enum values

[Rich Felker]

On Wed, Oct 01, 2014 at 02:30:22PM -0700, Roland McGrath wrote:
> > Thanks for the feedback. Since then we've got C11 threads support
> > committed in musl, with the following definitions. Are these
> > acceptable for glibc too? They're the same ones I proposed before.
> 
> No guarantees until we actually implement something in glibc.  But they
> seem fine to me.  The possible exception is that for interfaces where the
> only valid thing is to pass in of the specified enum values, there is
> something to be said for none of those values being zero (or -1) so that
> applications can't sloppily use literal 0 (or uninitialized statics or the
> like) and have it work but be non-portable and nonconforming.

My motive for wanting success to have a value of 0 is that it allows
some functions to be pure aliases or tail-calls where otherwise some
heavier wrapping would be needed. Everyone expects success to be 0
anyway. And since it's a return value, I don't think you'd have the
issue of passing literal zeros.

For mutex types, I suppose there's some risk of "sloppily" passing a
literal 0 without meaning mtx_plain, but I think it's a small issue
and I hope we can agree that keeping a common set of constants for ABI
purposes is of more value.

Rich

Re: C11 threads ABI questions - enum values

[Juan Manuel Torres Palma]

Hi guys.

I'm working on a early implementation at the moment since I'm done
with my internship but I still have some doubts:

-How can I use pthread_* functions for my threads.h functions? I
guessed that would be including the pthread.h in threads.c but then we
cannot map objects between both in header file.

-Is it some pthread refactoring needed? I remember a time ago somebody
suggested that idea.

I have a small and dummy standalone implementation working, not
integrated in glibc code yet, but still have the issues that user can
call pthread functions from main so i guess some help would be
appreciated. I know these doubts are pretty simple but I'm some kind
of newbie and I'm trying to learn the best way to solve it.

Cheers.

2014-10-02 2:17 GMT+02:00 Rich Felker <dalias@libc.org>:
> On Wed, Oct 01, 2014 at 02:30:22PM -0700, Roland McGrath wrote:
>> > Thanks for the feedback. Since then we've got C11 threads support
>> > committed in musl, with the following definitions. Are these
>> > acceptable for glibc too? They're the same ones I proposed before.
>>
>> No guarantees until we actually implement something in glibc.  But they
>> seem fine to me.  The possible exception is that for interfaces where the
>> only valid thing is to pass in of the specified enum values, there is
>> something to be said for none of those values being zero (or -1) so that
>> applications can't sloppily use literal 0 (or uninitialized statics or the
>> like) and have it work but be non-portable and nonconforming.
>
> My motive for wanting success to have a value of 0 is that it allows
> some functions to be pure aliases or tail-calls where otherwise some
> heavier wrapping would be needed. Everyone expects success to be 0
> anyway. And since it's a return value, I don't think you'd have the
> issue of passing literal zeros.
>
> For mutex types, I suppose there's some risk of "sloppily" passing a
> literal 0 without meaning mtx_plain, but I think it's a small issue
> and I hope we can agree that keeping a common set of constants for ABI
> purposes is of more value.
>
> Rich



-- 
Juan Manuel "Lolo" Torres Palma.
Computer Science Student at Universidad de Granada.

Re: C11 threads ABI questions - enum values

[Rich Felker]

On Thu, Oct 02, 2014 at 09:12:13AM +0200, Juan Manuel Torres Palma wrote:
> Hi guys.
> 
> I'm working on a early implementation at the moment since I'm done
> with my internship but I still have some doubts:
> 
> -How can I use pthread_* functions for my threads.h functions? I
> guessed that would be including the pthread.h in threads.c but then we
> cannot map objects between both in header file.

No, that can't be done, because the pthread names are not valid to be
exposed by threads.h. Also, the pthread function names cannot be
exposed in the external namespace as a result of using the threads.h
functions, so in order to implement C11 threads in terms of pthreads,
you need to make namespace-safe (e.g. prefixed by __) names for all of
the pthread functions you'll use, and call those instead from the
implementations of the C11 functions.

For the objects, you simply need to define them again with the same
definitions.

> -Is it some pthread refactoring needed? I remember a time ago somebody
> suggested that idea.

I think the glibc team would prefer that it be done without any actual
code refactoring in the same step as adding C11 threads. I'm not sure
if there's any that should be done as an earlier step, though. One
thing that comes to mind is the trouble of getting the 'int' return
value of threads out through the POSIX layer that uses 'void *' for
return values.

> I have a small and dummy standalone implementation working, not
> integrated in glibc code yet, but still have the issues that user can
> call pthread functions from main so i guess some help would be
> appreciated. I know these doubts are pretty simple but I'm some kind
> of newbie and I'm trying to learn the best way to solve it.

"Standalone implementations" like this are pretty easy to do because
they avoid all of the issues that make it difficult, but they can't
conform to the requirements of C. I don't mean to discourage you, just
to say that "most of the hard work is ahead". :-)

Rich

Re: C11 threads ABI questions - enum values

[Juan Manuel Torres Palma]

> "Standalone implementations" like this are pretty easy to do because
> they avoid all of the issues that make it difficult, but they can't
> conform to the requirements of C. I don't mean to discourage you, just
> to say that "most of the hard work is ahead". :-)

No, It doesn't discourage me at all but motivates me to keep learning
and trying to understand how it all works since I'm pretty stubborn ;)

> No, that can't be done, because the pthread names are not valid to be
> exposed by threads.h. Also, the pthread function names cannot be
> exposed in the external namespace as a result of using the threads.h
> functions, so in order to implement C11 threads in terms of pthreads,
> you need to make namespace-safe (e.g. prefixed by __) names for all of
> the pthread functions you'll use, and call those instead from the
> implementations of the C11 functions.
>
> For the objects, you simply need to define them again with the same
> definitions.
>

Ok so if I'm not mistaken I have to rename all the pthreads functions
that I'm using in my code, like for example pthread_create to
__pthread_create in all pthread related files, and also create macros
to keep compatibility with pthread old code like:

#define pthread_create __pthread_create

and then in the threads.h code i can just do #undef of all them. Would
that be the correct approach?

PS: Is there any guide or manual to learn to write system libraries?
I'm used to write user applications and it's getting a little tough ;)

Cheers.


-- 
Juan Manuel "Lolo" Torres Palma.
Computer Science Student at Universidad de Granada.

Re: C11 threads ABI questions - enum values

[Roland McGrath]

> My motive for wanting success to have a value of 0 is that it allows

Yeah, I'm fine with that one.  I meant the argument value cases.

> For mutex types, I suppose there's some risk of "sloppily" passing a
> literal 0 without meaning mtx_plain, but I think it's a small issue
> and I hope we can agree that keeping a common set of constants for ABI
> purposes is of more value.

We're not going to set a precedent of clearing ABI choices with another
implementation or letting another implementation's past choices dictate
to us.  Of course, harmonization is a good thing.  But you need to be
realistic about the relative positions of musl and glibc in terms of
installed base and de facto standards for GNU/Linux systems.  

This is a brand new feature and musl has few users even for features that
have existed for any length of time.  Are you really saying you cannot
change your recent ABI choices for new things that nobody is actually using?

Re: C11 threads ABI questions - enum values

[Rich Felker]

On Thu, Oct 02, 2014 at 02:42:29PM -0700, Roland McGrath wrote:
> > For mutex types, I suppose there's some risk of "sloppily" passing a
> > literal 0 without meaning mtx_plain, but I think it's a small issue
> > and I hope we can agree that keeping a common set of constants for ABI
> > purposes is of more value.
> 
> We're not going to set a precedent of clearing ABI choices with another
> implementation or letting another implementation's past choices dictate
> to us.  Of course, harmonization is a good thing.  But you need to be

That's understandable.

> realistic about the relative positions of musl and glibc in terms of
> installed base and de facto standards for GNU/Linux systems.  

Yes, that's why I've aimed to follow ABI rather than setting it. But
there are a few places where glibc is behind the current standards in
ways that require ABI decisions (O_SEARCH, O_EXEC, NI_NUMERICSCOPE, and
O_TTY_INIT are the examples that come to mind immediately) where it
would be really nice to have either some direction from glibc, or at
least a tenative assignment of values planned.

> This is a brand new feature and musl has few users even for features that
> have existed for any length of time.  Are you really saying you cannot
> change your recent ABI choices for new things that nobody is actually using?

If there were a strong reason at this point, where it's not being used
and not present in any release, we probably could. But it's rather
frowned upon and seen as unprofessional by much of our user community.

Rich

Re: C11 threads ABI questions - enum values

[Joseph S. Myers]

On Thu, 2 Oct 2014, Rich Felker wrote:

> ways that require ABI decisions (O_SEARCH, O_EXEC, NI_NUMERICSCOPE, and
> O_TTY_INIT are the examples that come to mind immediately) where it
> would be really nice to have either some direction from glibc, or at
> least a tenative assignment of values planned.

O_* are shared with the Linux kernel, so values need to be agreed there to 
avoid conflict with any new features added to the kernel in future (even 
if part of the semantics is implemented in userspace).  Once the values 
are defined in the kernel's uapi headers, that determines them for all C 
libraries.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: C11 threads ABI questions - enum values

[Rich Felker]

On Fri, Oct 03, 2014 at 10:40:23AM +0000, Joseph S. Myers wrote:
> On Thu, 2 Oct 2014, Rich Felker wrote:
> 
> > ways that require ABI decisions (O_SEARCH, O_EXEC, NI_NUMERICSCOPE, and
> > O_TTY_INIT are the examples that come to mind immediately) where it
> > would be really nice to have either some direction from glibc, or at
> > least a tenative assignment of values planned.
> 
> O_* are shared with the Linux kernel, so values need to be agreed there to 
> avoid conflict with any new features added to the kernel in future (even 
> if part of the semantics is implemented in userspace).  Once the values 
> are defined in the kernel's uapi headers, that determines them for all C 
> libraries.

The kernel position is that O_SEARCH and O_EXEC can be done by
userspace in terms of O_PATH, but I'm not sure that's correct. There's
an old thread (I think libc-alpha was CC'd on it but I'm not sure)
about that. O_TTY_INIT could probably be done in userspace too and
that might be what they expect us to do for it, too...

Rich

Re: C11 threads ABI questions - enum values

[Joseph S. Myers]

On Fri, 3 Oct 2014, Rich Felker wrote:

> The kernel position is that O_SEARCH and O_EXEC can be done by
> userspace in terms of O_PATH, but I'm not sure that's correct. There's

Looking back at the linux-api discussion from Aug 2013, I don't see that; 
it seems to have tailed off without a conclusion but with at least some 
suggestions that these features should be implemented in the kernel.

> an old thread (I think libc-alpha was CC'd on it but I'm not sure)
> about that. O_TTY_INIT could probably be done in userspace too and
> that might be what they expect us to do for it, too...

In both cases, even if things are to be implemented in userspace, the 
values need reserving in the kernel to ensure there aren't cases where the 
kernel is providing useful semantics for some value that aren't readily 
accessible from userspace because that value is used in libc for something 
else.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: C11 threads ABI questions - enum values

[Rich Felker]

On Fri, Oct 03, 2014 at 03:31:14PM +0000, Joseph S. Myers wrote:
> On Fri, 3 Oct 2014, Rich Felker wrote:
> 
> > The kernel position is that O_SEARCH and O_EXEC can be done by
> > userspace in terms of O_PATH, but I'm not sure that's correct. There's
> 
> Looking back at the linux-api discussion from Aug 2013, I don't see that; 
> it seems to have tailed off without a conclusion but with at least some 
> suggestions that these features should be implemented in the kernel.
> 
> > an old thread (I think libc-alpha was CC'd on it but I'm not sure)
> > about that. O_TTY_INIT could probably be done in userspace too and
> > that might be what they expect us to do for it, too...
> 
> In both cases, even if things are to be implemented in userspace, the 
> values need reserving in the kernel to ensure there aren't cases where the 
> kernel is providing useful semantics for some value that aren't readily 
> accessible from userspace because that value is used in libc for something 
> else.

Indeed, they need to be reserved on the kernel side, and during the
discussion on linux-api, almost everybody seemed to be failing to
understand why this is needed... :(

Rich

Re: C11 threads ABI questions - enum values

[Christoph Hellwig]

On Fri, Oct 03, 2014 at 11:32:48AM -0400, Rich Felker wrote:
> Indeed, they need to be reserved on the kernel side, and during the
> discussion on linux-api, almost everybody seemed to be failing to
> understand why this is needed... :(

At least of O_SEARCH and O_EXEC I strongly disagreed with the idea that
they should be implementated in userspace and still do.

Re: C11 threads ABI questions - enum values

[Rich Felker]

On Fri, Oct 03, 2014 at 08:39:10PM +0200, Christoph Hellwig wrote:
> On Fri, Oct 03, 2014 at 11:32:48AM -0400, Rich Felker wrote:
> > Indeed, they need to be reserved on the kernel side, and during the
> > discussion on linux-api, almost everybody seemed to be failing to
> > understand why this is needed... :(
> 
> At least of O_SEARCH and O_EXEC I strongly disagreed with the idea that
> they should be implementated in userspace and still do.

I do to, but for now it's the only way possible on Linux. For instance
it seems to be impossible to implement the POSIX rule (at least as I
understand it) that, when using an O_SEARCH or O_EXEC fd, the status
of the +x mode bit at open-time, rather than at the time of the
operation, dictates success or failure. There are a few other corner
cases where you want O_PATH and O_SEARCH/O_EXEC to behave differently,
too: for example, O_PATH|O_NOFOLLOW should open the symlink, while
O_SEARCH|O_NOFOLLOW or O_EXEC|O_NOFOLLOW should fail if the target is
a symlink.

Rich

Re: C11 threads ABI questions - enum values

[Christoph Hellwig]

On Fri, Oct 03, 2014 at 03:49:32PM -0400, Rich Felker wrote:
> On Fri, Oct 03, 2014 at 08:39:10PM +0200, Christoph Hellwig wrote:
> > On Fri, Oct 03, 2014 at 11:32:48AM -0400, Rich Felker wrote:
> > > Indeed, they need to be reserved on the kernel side, and during the
> > > discussion on linux-api, almost everybody seemed to be failing to
> > > understand why this is needed... :(
> > 
> > At least of O_SEARCH and O_EXEC I strongly disagreed with the idea that
> > they should be implementated in userspace and still do.
> 
> I do to, but for now it's the only way possible on Linux.

The other way would be to send patches, which I tried to encourage
you to do a few times.  It basically boils down to:

 - allow a few more operations on O_PATH fds, as listed by Posix
   for O_SEARCH and O_EXEC

 - add two defines that alias O_SEARCH and O_EXEC to (O_PATH|3),
   and..

> For instance
> it seems to be impossible to implement the POSIX rule (at least as I
> understand it) that, when using an O_SEARCH or O_EXEC fd, the status
> of the +x mode bit at open-time, rather than at the time of the
> operation, dictates success or failure. There are a few other corner
> cases where you want O_PATH and O_SEARCH/O_EXEC to behave differently,
> too: for example, O_PATH|O_NOFOLLOW should open the symlink, while
> O_SEARCH|O_NOFOLLOW or O_EXEC|O_NOFOLLOW should fail if the target is
> a symlink.

implement these quirks keyed off the (O_PATH|3) flag.

Re: C11 threads ABI questions - enum values

[Rich Felker]

On Sat, Oct 04, 2014 at 07:53:46PM +0200, Christoph Hellwig wrote:
> On Fri, Oct 03, 2014 at 03:49:32PM -0400, Rich Felker wrote:
> > On Fri, Oct 03, 2014 at 08:39:10PM +0200, Christoph Hellwig wrote:
> > > On Fri, Oct 03, 2014 at 11:32:48AM -0400, Rich Felker wrote:
> > > > Indeed, they need to be reserved on the kernel side, and during the
> > > > discussion on linux-api, almost everybody seemed to be failing to
> > > > understand why this is needed... :(
> > > 
> > > At least of O_SEARCH and O_EXEC I strongly disagreed with the idea that
> > > they should be implementated in userspace and still do.
> > 
> > I do to, but for now it's the only way possible on Linux.
> 
> The other way would be to send patches, which I tried to encourage
> you to do a few times.  It basically boils down to:
> 
>  - allow a few more operations on O_PATH fds, as listed by Posix
>    for O_SEARCH and O_EXEC
> 
>  - add two defines that alias O_SEARCH and O_EXEC to (O_PATH|3),
>    and..
> 
> > For instance
> > it seems to be impossible to implement the POSIX rule (at least as I
> > understand it) that, when using an O_SEARCH or O_EXEC fd, the status
> > of the +x mode bit at open-time, rather than at the time of the
> > operation, dictates success or failure. There are a few other corner
> > cases where you want O_PATH and O_SEARCH/O_EXEC to behave differently,
> > too: for example, O_PATH|O_NOFOLLOW should open the symlink, while
> > O_SEARCH|O_NOFOLLOW or O_EXEC|O_NOFOLLOW should fail if the target is
> > a symlink.
> 
> implement these quirks keyed off the (O_PATH|3) flag.

The symlink thing is easy to do, but the permissions thing is hard.
O_PATH does not open the actual file at all, just a reference to the
inode. It would be easy to just reject the open request if +x
permission is not available at open time, but the hard part is making
the +x permission stick at open time in the case where it's removed
from the underlying file after the open takes place. It would take
help from someone who knows the kernel internals a lot better than me
to make this work.

My thought is that O_SEARCH and O_EXEC should probably be defined as
3|O_PATH, but _not_ treated as O_PATH on the kernel side. Instead,
having the O_PATH bit set on top of the "secret" access mode 3 should
behave more like normal access mode 3, but with the usual permission
checks bypassed and replaced by checks for +x, and this +x access
right should be kept with the open file description. The reason for
using 3|O_PATH then is not to get O_PATH semantics, but for a graceful
fallback on older kernels that lack real O_SEARCH/O_EXEC support:
O_PATH is a reasonably close emulation, and old kernels will ignore
the access mode bits (3) when O_PATH is set.

Rich

Re: C11 threads ABI questions - enum values

[Christoph Hellwig]

On Sat, Oct 04, 2014 at 06:57:56PM -0400, Rich Felker wrote:
> The symlink thing is easy to do, but the permissions thing is hard.
> O_PATH does not open the actual file at all, just a reference to the
> inode. It would be easy to just reject the open request if +x
> permission is not available at open time, but the hard part is making
> the +x permission stick at open time in the case where it's removed
> from the underlying file after the open takes place. It would take
> help from someone who knows the kernel internals a lot better than me
> to make this work.

The kernel uses FMODE_* flags on file->f_mode for that.  There are
is no FMODE_EXEC right now, but it can be added along the lines of
FMODE_READ/FMODE_WRITE trivially.  If you need more detailed help
feel free to contact me off list.

> My thought is that O_SEARCH and O_EXEC should probably be defined as
> 3|O_PATH, but _not_ treated as O_PATH on the kernel side. Instead,
> having the O_PATH bit set on top of the "secret" access mode 3 should
> behave more like normal access mode 3, but with the usual permission
> checks bypassed and replaced by checks for +x, and this +x access
> right should be kept with the open file description. The reason for
> using 3|O_PATH then is not to get O_PATH semantics, but for a graceful
> fallback on older kernels that lack real O_SEARCH/O_EXEC support:
> O_PATH is a reasonably close emulation, and old kernels will ignore
> the access mode bits (3) when O_PATH is set.

Well, O_SEARCH and O_EXEC is very similar to O_PATH, so I would
treat it conceptually the same, that is make fdget fail on file
descriptors opened on it, and thus make all the normal operations
impossible with a high level exclusion.  It would suggest to initially
allow the additional operations like chown for all O_PATH descriptors.
There is precendence for allowing more operations on them than
initially supported, e.g. a year ago we started to allow fstatfs
on O_PATH descriptors.  If a good argument against that comes up
we'll need to use the new fdget variant that only allows to grab
a reference only if FMODE_PATH isn't set or FMODE_EXEC is set,
which we will need for lookup and the new fexecve syscall anyway.

Re: C11 threads ABI questions - enum values

[Rich Felker]

On Sun, Oct 05, 2014 at 11:49:13AM +0200, Christoph Hellwig wrote:
> On Sat, Oct 04, 2014 at 06:57:56PM -0400, Rich Felker wrote:
> > The symlink thing is easy to do, but the permissions thing is hard.
> > O_PATH does not open the actual file at all, just a reference to the
> > inode. It would be easy to just reject the open request if +x
> > permission is not available at open time, but the hard part is making
> > the +x permission stick at open time in the case where it's removed
> > from the underlying file after the open takes place. It would take
> > help from someone who knows the kernel internals a lot better than me
> > to make this work.
> 
> The kernel uses FMODE_* flags on file->f_mode for that.  There are
> is no FMODE_EXEC right now, but it can be added along the lines of
> FMODE_READ/FMODE_WRITE trivially.  If you need more detailed help
> feel free to contact me off list.

OK I'll look at it.

> > My thought is that O_SEARCH and O_EXEC should probably be defined as
> > 3|O_PATH, but _not_ treated as O_PATH on the kernel side. Instead,
> > having the O_PATH bit set on top of the "secret" access mode 3 should
> > behave more like normal access mode 3, but with the usual permission
> > checks bypassed and replaced by checks for +x, and this +x access
> > right should be kept with the open file description. The reason for
> > using 3|O_PATH then is not to get O_PATH semantics, but for a graceful
> > fallback on older kernels that lack real O_SEARCH/O_EXEC support:
> > O_PATH is a reasonably close emulation, and old kernels will ignore
> > the access mode bits (3) when O_PATH is set.
> 
> Well, O_SEARCH and O_EXEC is very similar to O_PATH, so I would
> treat it conceptually the same, that is make fdget fail on file
> descriptors opened on it, and thus make all the normal operations
> impossible with a high level exclusion.  It would suggest to initially
> allow the additional operations like chown for all O_PATH descriptors.

Yes, the lack of support for fchown, fchmod, fstat, fstatfs, fchdir,
and perhaps others on O_PATH file descriptors is another problem with
using O_PATH outright. In musl I have fallback support that uses
/proc/self/fd/%d for these operations when the initial syscall fails
with EBADF despite being a valid file descriptor, but it would be nice
if the kernel just supported them directly. Is it reasonably easy to
add support for O_PATH file descriptors on the kernel side? If not,
that would be an indication that O_EXEC/O_SEARCH should perhaps not
work like O_PATH but "actually open the file".

Rich

Re: C11 threads ABI questions - enum values

[Christoph Hellwig]

On Sun, Oct 05, 2014 at 12:13:58PM -0400, Rich Felker wrote:
> Yes, the lack of support for fchown, fchmod, fstat, fstatfs, fchdir,

fstat, fstatfs and fchdir are already support on O_PATH descriptos, and
below is an untested patch for fchmod and fchown.

diff --git a/fs/open.c b/fs/open.c
index d6fd3ac..ee24720 100644
--- a/fs/open.c
+++ b/fs/open.c
@@ -512,7 +512,7 @@ out_unlock:
 
 SYSCALL_DEFINE2(fchmod, unsigned int, fd, umode_t, mode)
 {
-	struct fd f = fdget(fd);
+	struct fd f = fdget_raw(fd);
 	int err = -EBADF;
 
 	if (f.file) {
@@ -633,7 +633,7 @@ SYSCALL_DEFINE3(lchown, const char __user *, filename, uid_t, user, gid_t, group
 
 SYSCALL_DEFINE3(fchown, unsigned int, fd, uid_t, user, gid_t, group)
 {
-	struct fd f = fdget(fd);
+	struct fd f = fdget_raw(fd);
 	int error = -EBADF;
 
 	if (!f.file)

Re: C11 threads ABI - mtx_t and cnd_t types

[Torvald Riegel]

On Sat, 2014-08-30 at 22:52 -0400, Rich Felker wrote:
> Another issue I have on the ABI for C11 threads pertains to the types
> for mtx_t and cnd_t. My understanding, and I agree with this, is that
> it was already decided to use the same underlying sizes/alignment, and
> for now representations, as the corresponding POSIX types.

I don't remember a decision being made rather than just people
expressing their opinion at that time, but maybe I'm wrong.

Anyway, for mtx_t I'm starting to wonder whether a fresh start would
indeed be better, with some additional room for expanding the lock
representation to state elsewhere.  (That is, mtx_t would at least be
pointer-sized.)
The reason for this is the current discussion around the barrier
semantics.  It is slowly moving, and I'm concerned about the Austin
Group deciding to stick to the less efficient spec for mutexes after a
long discussion -- which would leave us with larger-than-necessary mtx_t
while not using the pthread_mutex_t implementation anyway.

Re: C11 threads ABI - mtx_t and cnd_t types

[Joseph S. Myers]

On Mon, 6 Oct 2014, Torvald Riegel wrote:

> On Sat, 2014-08-30 at 22:52 -0400, Rich Felker wrote:
> > Another issue I have on the ABI for C11 threads pertains to the types
> > for mtx_t and cnd_t. My understanding, and I agree with this, is that
> > it was already decided to use the same underlying sizes/alignment, and
> > for now representations, as the corresponding POSIX types.
> 
> I don't remember a decision being made rather than just people
> expressing their opinion at that time, but maybe I'm wrong.
> 
> Anyway, for mtx_t I'm starting to wonder whether a fresh start would
> indeed be better, with some additional room for expanding the lock
> representation to state elsewhere.  (That is, mtx_t would at least be
> pointer-sized.)

If mtx_t isn't a thin wrapper round pthread_mutex_t, then doesn't that 
mean cnd_wait and cnd_timedwait can no longer be thin wrappers around the 
corresponding pthread functions?

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: C11 threads ABI - mtx_t and cnd_t types

[Torvald Riegel]

On Mon, 2014-10-06 at 15:52 +0000, Joseph S. Myers wrote:
> On Mon, 6 Oct 2014, Torvald Riegel wrote:
> 
> > On Sat, 2014-08-30 at 22:52 -0400, Rich Felker wrote:
> > > Another issue I have on the ABI for C11 threads pertains to the types
> > > for mtx_t and cnd_t. My understanding, and I agree with this, is that
> > > it was already decided to use the same underlying sizes/alignment, and
> > > for now representations, as the corresponding POSIX types.
> > 
> > I don't remember a decision being made rather than just people
> > expressing their opinion at that time, but maybe I'm wrong.
> > 
> > Anyway, for mtx_t I'm starting to wonder whether a fresh start would
> > indeed be better, with some additional room for expanding the lock
> > representation to state elsewhere.  (That is, mtx_t would at least be
> > pointer-sized.)
> 
> If mtx_t isn't a thin wrapper round pthread_mutex_t, then doesn't that 
> mean cnd_wait and cnd_timedwait can no longer be thin wrappers around the 
> corresponding pthread functions?

Right.  But I suspect we should still be able to reuse most of the
source.

Re: C11 threads ABI - mtx_t and cnd_t types

[Rich Felker]

On Mon, Oct 06, 2014 at 03:17:59PM +0200, Torvald Riegel wrote:
> On Sat, 2014-08-30 at 22:52 -0400, Rich Felker wrote:
> > Another issue I have on the ABI for C11 threads pertains to the types
> > for mtx_t and cnd_t. My understanding, and I agree with this, is that
> > it was already decided to use the same underlying sizes/alignment, and
> > for now representations, as the corresponding POSIX types.
> 
> I don't remember a decision being made rather than just people
> expressing their opinion at that time, but maybe I'm wrong.
> 
> Anyway, for mtx_t I'm starting to wonder whether a fresh start would
> indeed be better, with some additional room for expanding the lock
> representation to state elsewhere.  (That is, mtx_t would at least be
> pointer-sized.)
> The reason for this is the current discussion around the barrier
> semantics.  It is slowly moving, and I'm concerned about the Austin
> Group deciding to stick to the less efficient spec for mutexes after a
> long discussion -- which would leave us with larger-than-necessary mtx_t
> while not using the pthread_mutex_t implementation anyway.

I question whether it's "much larger than necessary", especially if
the C committee does not decide to make changes with regard to
orphaned mutexes (see http://austingroupbugs.net/view.php?id=755),
since you need room for a linked list to deal with the tid-reuse
issue. I don't see a way to make a mutex any smaller than what you
have on 32-bit targets now without some hacks like cramming the mutex
type into the upper bits of the lock count or something. If you were
talking about a 50% or 75% size saving, this might be appealing, but I
think at best you'd be looking at shaving off one field, in which case
the expandability is probably worth a lot more than the size savings
(expandability that requires allocation rather sucks, since there's
probably no way an application can deal with failure to create a
mutex, especially for mutexes that are intended to be "static" ones
but which, because C11 lacks static allocation, require call_once and
mtx_init).

Rich

Re: C11 threads ABI - mtx_t and cnd_t types

[Torvald Riegel]

On Mon, 2014-10-06 at 13:58 -0400, Rich Felker wrote:
> On Mon, Oct 06, 2014 at 03:17:59PM +0200, Torvald Riegel wrote:
> > On Sat, 2014-08-30 at 22:52 -0400, Rich Felker wrote:
> > > Another issue I have on the ABI for C11 threads pertains to the types
> > > for mtx_t and cnd_t. My understanding, and I agree with this, is that
> > > it was already decided to use the same underlying sizes/alignment, and
> > > for now representations, as the corresponding POSIX types.
> > 
> > I don't remember a decision being made rather than just people
> > expressing their opinion at that time, but maybe I'm wrong.
> > 
> > Anyway, for mtx_t I'm starting to wonder whether a fresh start would
> > indeed be better, with some additional room for expanding the lock
> > representation to state elsewhere.  (That is, mtx_t would at least be
> > pointer-sized.)
> > The reason for this is the current discussion around the barrier
> > semantics.  It is slowly moving, and I'm concerned about the Austin
> > Group deciding to stick to the less efficient spec for mutexes after a
> > long discussion -- which would leave us with larger-than-necessary mtx_t
> > while not using the pthread_mutex_t implementation anyway.
> 
> I question whether it's "much larger than necessary", especially if
> the C committee does not decide to make changes with regard to
> orphaned mutexes (see http://austingroupbugs.net/view.php?id=755),

I don't see anything in C11 that would agree with the opinion of the
Austin Group.  I think C11 should do what C++11 states explicitly: you
get undefined behvaior if a mutex is owned by a thread that terminates.
(That is, what you also stated in Note 0001848 in the POSIX BZ above.)

I think the resolution should be either an implementation of the
requested behavior for recursive locks (and non-robust PI too?), or,
preferably in my opinion, documentation that glibc does not implement
this requirement for non-robust locks.  This is now
https://sourceware.org/bugzilla/show_bug.cgi?id=17463

> since you need room for a linked list to deal with the tid-reuse
> issue. I don't see a way to make a mutex any smaller than what you
> have on 32-bit targets now without some hacks like cramming the mutex
> type into the upper bits of the lock count or something. If you were
> talking about a 50% or 75% size saving, this might be appealing, but I
> think at best you'd be looking at shaving off one field,

I think for C11 one pointer-sized field plus a 32b counter is
sufficient.  If we want to have extra space, like perhaps for spin-wait
or elision tuning state, we could give it one extra 32b field.  That's
8/12 -- 12/16B for C11 (32b -- 64b min/extra) vs. 24 -- 40 as we have
now on x86, for example.

> in which case
> the expandability is probably worth a lot more than the size savings
> (expandability that requires allocation rather sucks, since there's
> probably no way an application can deal with failure to create a
> mutex, especially for mutexes that are intended to be "static" ones
> but which, because C11 lacks static allocation, require call_once and
> mtx_init).

Expendability would be a best-effort thing feature, aimed at potential
performance increases rather than a necessity for correctness.

Re: C11 threads ABI - mtx_t and cnd_t types

[Rich Felker]

On Tue, Oct 07, 2014 at 02:51:47PM +0200, Torvald Riegel wrote:
> On Mon, 2014-10-06 at 13:58 -0400, Rich Felker wrote:
> > On Mon, Oct 06, 2014 at 03:17:59PM +0200, Torvald Riegel wrote:
> > > On Sat, 2014-08-30 at 22:52 -0400, Rich Felker wrote:
> > > > Another issue I have on the ABI for C11 threads pertains to the types
> > > > for mtx_t and cnd_t. My understanding, and I agree with this, is that
> > > > it was already decided to use the same underlying sizes/alignment, and
> > > > for now representations, as the corresponding POSIX types.
> > > 
> > > I don't remember a decision being made rather than just people
> > > expressing their opinion at that time, but maybe I'm wrong.
> > > 
> > > Anyway, for mtx_t I'm starting to wonder whether a fresh start would
> > > indeed be better, with some additional room for expanding the lock
> > > representation to state elsewhere.  (That is, mtx_t would at least be
> > > pointer-sized.)
> > > The reason for this is the current discussion around the barrier
> > > semantics.  It is slowly moving, and I'm concerned about the Austin
> > > Group deciding to stick to the less efficient spec for mutexes after a
> > > long discussion -- which would leave us with larger-than-necessary mtx_t
> > > while not using the pthread_mutex_t implementation anyway.
> > 
> > I question whether it's "much larger than necessary", especially if
> > the C committee does not decide to make changes with regard to
> > orphaned mutexes (see http://austingroupbugs.net/view.php?id=755),
> 
> I don't see anything in C11 that would agree with the opinion of the
> Austin Group.  I think C11 should do what C++11 states explicitly: you
> get undefined behvaior if a mutex is owned by a thread that terminates.
> (That is, what you also stated in Note 0001848 in the POSIX BZ above.)

It's just what you get when reading the requirements as written
without considering termination-while-owner as a special case:
mtx_lock cannot succeed unless the mutex is not already locked or the
current thread is already the owner. To nullify this (fundamental)
rule in the special case where the owner has terminated, you need
special language either making the whole thing UB, or specifically
addressing the possibility that mtx_lock will spuriously succeed in
this case.

> I think the resolution should be either an implementation of the
> requested behavior for recursive locks (and non-robust PI too?), or,

And for error-checking locks.

> preferably in my opinion, documentation that glibc does not implement
> this requirement for non-robust locks.  This is now
> https://sourceware.org/bugzilla/show_bug.cgi?id=17463

If it's a requirement and we can't get it relaxed for the next issue,
I think glibc should implement the required behavior rather than going
back to the old days of flaunting non-compliance whenever someone
disagrees with a decision made in the standards process. It's really
not hard to do, and the code is already there -- it's the exact same
code that's used for robust mutexes. If I remember correctly, glibc is
using separate code paths (and lock designs!) for robust and
non-robust, only putting the tid in the futex slot for robust and
using a separate owner slot for non-robust, so this may complicate
things somewhat; on the other hand, if there's no benefit to having
the two separate designs, it might actually help clean things up to
remove the redundant one. (My guess: maybe it was an attempt at
supporting targets without atomic compare-and-swap?)

> > since you need room for a linked list to deal with the tid-reuse
> > issue. I don't see a way to make a mutex any smaller than what you
> > have on 32-bit targets now without some hacks like cramming the mutex
> > type into the upper bits of the lock count or something. If you were
> > talking about a 50% or 75% size saving, this might be appealing, but I
> > think at best you'd be looking at shaving off one field,
> 
> I think for C11 one pointer-sized field plus a 32b counter is
> sufficient.  If we want to have extra space, like perhaps for spin-wait
> or elision tuning state, we could give it one extra 32b field.  That's
> 8/12 -- 12/16B for C11 (32b -- 64b min/extra) vs. 24 -- 40 as we have
> now on x86, for example.

I was assuming the need to keep a linked list for preventing false
ownership via tid reuse. BTW, in that case, since you'd be reusing the
robust mutex mechanism, you need the layouts (or at least the offset
between the linked lists pointers and the futex) to match between
pthread_mutex_t and mtx_t, or they can't reside together in a robust
list.

Rich

Re: C11 threads ABI - mtx_t and cnd_t types

[Torvald Riegel]

On Tue, 2014-10-07 at 09:26 -0400, Rich Felker wrote:
> On Tue, Oct 07, 2014 at 02:51:47PM +0200, Torvald Riegel wrote:
> > On Mon, 2014-10-06 at 13:58 -0400, Rich Felker wrote:
> > > On Mon, Oct 06, 2014 at 03:17:59PM +0200, Torvald Riegel wrote:
> > > > On Sat, 2014-08-30 at 22:52 -0400, Rich Felker wrote:
> > > > > Another issue I have on the ABI for C11 threads pertains to the types
> > > > > for mtx_t and cnd_t. My understanding, and I agree with this, is that
> > > > > it was already decided to use the same underlying sizes/alignment, and
> > > > > for now representations, as the corresponding POSIX types.
> > > > 
> > > > I don't remember a decision being made rather than just people
> > > > expressing their opinion at that time, but maybe I'm wrong.
> > > > 
> > > > Anyway, for mtx_t I'm starting to wonder whether a fresh start would
> > > > indeed be better, with some additional room for expanding the lock
> > > > representation to state elsewhere.  (That is, mtx_t would at least be
> > > > pointer-sized.)
> > > > The reason for this is the current discussion around the barrier
> > > > semantics.  It is slowly moving, and I'm concerned about the Austin
> > > > Group deciding to stick to the less efficient spec for mutexes after a
> > > > long discussion -- which would leave us with larger-than-necessary mtx_t
> > > > while not using the pthread_mutex_t implementation anyway.
> > > 
> > > I question whether it's "much larger than necessary", especially if
> > > the C committee does not decide to make changes with regard to
> > > orphaned mutexes (see http://austingroupbugs.net/view.php?id=755),
> > 
> > I don't see anything in C11 that would agree with the opinion of the
> > Austin Group.  I think C11 should do what C++11 states explicitly: you
> > get undefined behvaior if a mutex is owned by a thread that terminates.
> > (That is, what you also stated in Note 0001848 in the POSIX BZ above.)
> 
> It's just what you get when reading the requirements as written
> without considering termination-while-owner as a special case:
> mtx_lock cannot succeed unless the mutex is not already locked or the
> current thread is already the owner.

C11 states: "The mtx_lock function blocks until it locks the mutex
pointed to by mtx."  The case of what happens when a thread terminates
is simply undefined.

> To nullify this (fundamental)
> rule in the special case where the owner has terminated, you need
> special language either making the whole thing UB, or specifically
> addressing the possibility that mtx_lock will spuriously succeed in
> this case.

I disagree.  But WG14 will have to sort this out.

> > I think the resolution should be either an implementation of the
> > requested behavior for recursive locks (and non-robust PI too?), or,
> 
> And for error-checking locks.

Agreed.

> > preferably in my opinion, documentation that glibc does not implement
> > this requirement for non-robust locks.  This is now
> > https://sourceware.org/bugzilla/show_bug.cgi?id=17463
> 
> If it's a requirement and we can't get it relaxed for the next issue,
> I think glibc should implement the required behavior rather than going
> back to the old days of flaunting non-compliance whenever someone
> disagrees with a decision made in the standards process. It's really
> not hard to do, and the code is already there -- it's the exact same
> code that's used for robust mutexes. If I remember correctly, glibc is
> using separate code paths (and lock designs!) for robust and
> non-robust, only putting the tid in the futex slot for robust and
> using a separate owner slot for non-robust, so this may complicate
> things somewhat; on the other hand, if there's no benefit to having
> the two separate designs, it might actually help clean things up to
> remove the redundant one. (My guess: maybe it was an attempt at
> supporting targets without atomic compare-and-swap?)

I wouldn't use the algorithm of robust mutexes to implement the
thread-ID ABA avoidance.

Re: C11 threads ABI - mtx_t and cnd_t types

[Rich Felker]

On Tue, Oct 07, 2014 at 03:59:08PM +0200, Torvald Riegel wrote:
> > > I don't see anything in C11 that would agree with the opinion of the
> > > Austin Group.  I think C11 should do what C++11 states explicitly: you
> > > get undefined behvaior if a mutex is owned by a thread that terminates.
> > > (That is, what you also stated in Note 0001848 in the POSIX BZ above.)
> > 
> > It's just what you get when reading the requirements as written
> > without considering termination-while-owner as a special case:
> > mtx_lock cannot succeed unless the mutex is not already locked or the
> > current thread is already the owner.
> 
> C11 states: "The mtx_lock function blocks until it locks the mutex
> pointed to by mtx."  The case of what happens when a thread terminates
> is simply undefined.

In the absence of text saying something special happens when a thread
terminates, I don't think you can argue that this is UB; the mutex
just retains whatever state it already had (being locked) and there is
no longer any thread which is its owner. This is unfortunate and
undesirable, I agree, but I would approach WG14 with the idea that
this is an oversight and unintentional side effect of lack of
specification (note: there are already several bugs in C11 that are
presumably going to be fixed) rather than trying to weasel out some
interpretation that it's already undefined.

> > > preferably in my opinion, documentation that glibc does not implement
> > > this requirement for non-robust locks.  This is now
> > > https://sourceware.org/bugzilla/show_bug.cgi?id=17463
> > 
> > If it's a requirement and we can't get it relaxed for the next issue,
> > I think glibc should implement the required behavior rather than going
> > back to the old days of flaunting non-compliance whenever someone
> > disagrees with a decision made in the standards process. It's really
> > not hard to do, and the code is already there -- it's the exact same
> > code that's used for robust mutexes. If I remember correctly, glibc is
> > using separate code paths (and lock designs!) for robust and
> > non-robust, only putting the tid in the futex slot for robust and
> > using a separate owner slot for non-robust, so this may complicate
> > things somewhat; on the other hand, if there's no benefit to having
> > the two separate designs, it might actually help clean things up to
> > remove the redundant one. (My guess: maybe it was an attempt at
> > supporting targets without atomic compare-and-swap?)
> 
> I wouldn't use the algorithm of robust mutexes to implement the
> thread-ID ABA avoidance.

Do you have a reason in mind? IMO it's simple and it offloads all the
heavy work to the (unlikely) case of thread termination while owning
mutexes, where the kernel does the work for you (but it would be just
as easy to do in userspace, anyway, as part of pthread_exit).

Rich

Re: C11 threads ABI questions - enum values

[Christoph Hellwig]

On Sun, Oct 05, 2014 at 06:23:17PM +0200, Christoph Hellwig wrote:
> On Sun, Oct 05, 2014 at 12:13:58PM -0400, Rich Felker wrote:
> > Yes, the lack of support for fchown, fchmod, fstat, fstatfs, fchdir,
> 
> fstat, fstatfs and fchdir are already support on O_PATH descriptos, and
> below is an untested patch for fchmod and fchown.

Can you verify that this is enough to get full O_SEARCH semantics from
the kernel?  I can submit this one ASAP, and then we can try to help to
make the new execveat suitable for O_EXEC.

Re: C11 threads ABI questions - enum values

[Rich Felker]

On Thu, Oct 23, 2014 at 10:00:34AM +0200, Christoph Hellwig wrote:
> On Sun, Oct 05, 2014 at 06:23:17PM +0200, Christoph Hellwig wrote:
> > On Sun, Oct 05, 2014 at 12:13:58PM -0400, Rich Felker wrote:
> > > Yes, the lack of support for fchown, fchmod, fstat, fstatfs, fchdir,
> > 
> > fstat, fstatfs and fchdir are already support on O_PATH descriptos, and
> > below is an untested patch for fchmod and fchown.
> 
> Can you verify that this is enough to get full O_SEARCH semantics from
> the kernel?  I can submit this one ASAP, and then we can try to help to
> make the new execveat suitable for O_EXEC.

I've got a lot of things I'm working on at the moment, but I'll try to
give it a test maybe early next week. Is that soon enough?

Rich

Re: C11 threads ABI questions - enum values

[Rich Felker]

On Thu, Oct 23, 2014 at 12:37:23PM -0400, Rich Felker wrote:
> On Thu, Oct 23, 2014 at 10:00:34AM +0200, Christoph Hellwig wrote:
> > On Sun, Oct 05, 2014 at 06:23:17PM +0200, Christoph Hellwig wrote:
> > > On Sun, Oct 05, 2014 at 12:13:58PM -0400, Rich Felker wrote:
> > > > Yes, the lack of support for fchown, fchmod, fstat, fstatfs, fchdir,
> > > 
> > > fstat, fstatfs and fchdir are already support on O_PATH descriptos, and
> > > below is an untested patch for fchmod and fchown.
> > 
> > Can you verify that this is enough to get full O_SEARCH semantics from
> > the kernel?  I can submit this one ASAP, and then we can try to help to
> > make the new execveat suitable for O_EXEC.
> 
> I've got a lot of things I'm working on at the moment, but I'll try to
> give it a test maybe early next week. Is that soon enough?

Hmm, looking again it looks like it's not so much a matter of testing
the patch as verifying that there are no other missing cases. Is that
right? Let me know if I'm misunderstanding what you want.

Rich

Re: C11 threads ABI questions - enum values

[Christoph Hellwig]

On Thu, Oct 23, 2014 at 12:38:33PM -0400, Rich Felker wrote:
> > I've got a lot of things I'm working on at the moment, but I'll try to
> > give it a test maybe early next week. Is that soon enough?
> 
> Hmm, looking again it looks like it's not so much a matter of testing
> the patch as verifying that there are no other missing cases. Is that
> right? Let me know if I'm misunderstanding what you want.

I'm pretty sure that patch works for those two syscalls, but independent
verification is always welcome.  The more important bit would be a list
of remaining workaround you have to implement O_SEARCH (I think that should be
it) and O_EXEC.  For O_EXEC I suspect right now you don't have any other
workaround either given the /proc based fexecve implementation, but just
at the same time we're now getting a syscall for that one, so we'll have
to take care of that.  For O_EXEC with a syscall-based fexecve we'll
probably need the O_PATH|3 defintion and some hacky looking code in the
kernel I fear, so it would be great if you could bring that up in the
current discussion of the execveat system call.