Re: [PATCH 2/3]: C++20 P0482R6 and C2X N2653: Implement mbrtoc8, c8rtomb, char8_t

[Adhemerval Zanella <adhemerval.zanella@linaro.org>]

On 18/05/2022 14:26, Tom Honermann wrote:
> On 5/18/22 12:17 PM, Adhemerval Zanella wrote:
>> On 18/05/2022 12:32, Tom Honermann wrote:
>>> On 5/17/22 5:33 PM, Florian Weimer wrote:
>>>> * Adhemerval Zanella via Libc-alpha:
>>>>
>>>>> On 17/05/2022 15:12, Joseph Myers wrote:
>>>>>> On Tue, 17 May 2022, Adhemerval Zanella wrote:
>>>>>>
>>>>>>>> +/* This is the private state used if PS is NULL.  */
>>>>>>>> +static mbstate_t state;
>>>>>>> Although it was done for other convertion interfaces, I wonder if we should
>>>>>>> keep supporting this mt-unsafe usage for newer ones.  It was not clear from
>>>>>> In C23 it's implementation-defined whether the internal state for such
>>>>>> functions has static or thread storage duration (see the general
>>>>>> introduction to the uchar.h functions).
>>>>>>
>>>>> Right, so glibc still need to support either mode.
>>>> No, I think we can and should switch.  Maybe with new symbol versions
>>>> (but the same implementation) if we want to play it conservative.
>>>>
>>>> The intent in POSIX and C has been for a long time that thread-local
>>>> state is permitted for these functions, only the wording did not
>>>> technically allow it.  The phrase was “not required to avoid data
>>>> races”.  The problem is that it's possible to tell the difference
>>>> without data races (with external synchronization).  Many libcs already
>>>> use thread-local state for these functions, without any apparent ill
>>>> effects.
>>> I would err on the side of maintaining consistency across these functions for now and then transition them all at once if there is a desire to do so.
>> Do we need to keep distinct states for each one or can use the a shared
>> one?
> 
> Each is required to maintain its own state. The relevant wording from C17 7.28.1 (Restartable multibyte/wide character conversion functions) paragraph 1 states:
> 
> These functions have a parameter, ps, of type pointer to mbstate_t that points to an object that can completely describe the current conversion state of the associated multibyte character sequence, which the functions alter as necessary. *If ps is a null pointer, each function uses its own internal **mbstate_t**object instead, which is initialized at program startup to the initial conversion state; the****functions are not required to avoid data races with other calls to the same function in this case. The **implementation behaves as if no library function calls these functions with a null pointer for ps.*

Yeah, I forgot that I just read this very sentence yesterday reviewing you patch.
So it it currently 88 bytes per thread (104 considering the mbrtoc8, c8rtomb)
per thread state if we make it thread-local.  We can maybe lazy allocate the
whole wcsmbs state, so we just keep a pointer per thread.

We already have the internal scheme to allocate ion tcb (tls-internal.h) so
it does not exhaust static tls allocation.