Re: character class "alpha"

[Brian Inglis <Brian.Inglis@Shaw.ca>]

On 2023-07-31 15:12, Corinna Vinschen via Cygwin wrote:
> Hi Bruno,
> 
> On Jul 31 20:43, Bruno Haible via Cygwin wrote:
>> Corinna Vinschen wrote:
>>> there are more of those expressions which are disabled on glibc and
>>> fail on Cygwin, for instance in test-c32iscntrl.c.  Maybe it's actually
>>> the better idea to disable them on Cygwin, too, rather than to change
>>> a working system...
>>
>> Sure. There is no standard how to map the Unicode properties to POSIX
>> character classes. Other than the mentioned ISO C constraints for
>> 'digit' and 'xdigit' and a few POSIX constraints, you are free to
>> map them as you like. For glibc and gnulib, I mapped them in a way
>> that seemed to make most sense for applications. But different
>> people might come to different meanings of "make sense".
> 
> Ok, so I just pushed a patchset to Cygwin git, which should make GB18030
> support actually work.
> 
> Also, the C11 functions c16rtomb, c32rtomb, mbrtoc16, mbrtoc32 are now
> implemented in Cygwin and a uchar.h header exists now, too.
> 
> Assuming all gnulib tests disabled for GLibc in
> 
>    test-c32isalpha.c
>    test-c32iscntrl.c
>    test-c32isprint.c
>    test-c32isgraph.c
>    test-c32ispunct.c
>    test-c32islower.c
> 
> will be disabled for Cygwin as well, all gb18030 and c32 tests in gnulib
> work as desired now.

	https://www.iso.org/standard/86539.html		[ISO/IEC/IEEE 9945 CD]

Draft POSIX 2023 SUS V5 Issue 8 D3 CB2.1 proposes the following POSIX 
Subprofiling Option Group: POSIX_C_LANG_UCHAR: ISO C Unicode Utilities.

	https://www.iso.org/standard/82075.html		[ISO/IEC 9899 DIS]

Draft Standard C 2023 is being voted on as of 2023-07-14, and if no technical 
issues arise requiring tweaks, will become the new standard, in which Unicode 
utilities <uchar.h> has some additions which you may wish to add; from:

	https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3096.pdf#page=426

also:

	https://en.cppreference.com/w/c/string/multibyte

	https://en.cppreference.com/w/c/language/arithmetic_types

major additions (note November official standard publication date):

"7.30 Unicode utilities <uchar.h>
1 The header <uchar.h> declares one macro, a few types, and several functions 
for manipulating Unicode characters.
2 The macro
			__STDC_VERSION_UCHAR_H__
is an integer constant expression with a value equivalent to 202311L.

3 The types declared are mbstate_t (described in 7.31.1) and size_t (described 
in 7.21);
			char8_t
which is an unsigned integer type used for 8-bit characters and is the same type 
as unsigned char;
...

7.30.1 Restartable multibyte/wide character conversion functions
...
2 When used in the functions in this subclause, the encoding of char8_t,
char16_t, and char32_t objects, and sequences of such objects, is UTF-8, UTF-16, 
and UTF-32, respectively. Similarly, the encoding of char and wchar_t, and 
sequences of such objects, is the execution and wide execution encodings 
(6.2.9), respectively

7.30.1.1 The mbrtoc8 function
Synopsis
1	#include <uchar.h>
	size_t mbrtoc8(char8_t * restrict pc8, const char * restrict s,
			size_t n, mbstate_t * restrict ps);
Description
2 If s is a null pointer, the mbrtoc8 function is equivalent to the call:
			mbrtoc8(NULL, "", 1, ps)
In this case, the values of the parameters pc8 and n are ignored.
3 If s is not a null pointer, the mbrtoc8 function function inspects at most n 
bytes beginning with the byte pointed to by s to determine the number of bytes 
needed to complete the next multibyte character (including any shift sequences). 
If the function determines that the next multibyte character is complete and 
valid, it determines the values of the corresponding characters and then, if pc8 
is not a null pointer, stores the value of the first (or only) such character in 
the object pointed to by pc8.
Subsequent calls will store successive characters without consuming any 
additional input until all the characters have been stored. If the corresponding 
character is the null character, the resulting state described is the initial 
conversion state.
Returns
4 The mbrtoc8 function returns the first of the following that applies (given 
the current conversion state):
0	if the next n or fewer bytes complete the multibyte character that corresponds 
to the null character (which is the value stored).
between 1 and n inclusive	if the next n or fewer bytes complete a valid 
multibyte character (which is the value stored); the value returned is the 
number of bytes that complete the multibyte character.
(size_t)(-3)	if the next character resulting from a previous call has been 
stored (no bytes from the input have been consumed by this call).
(size_t)(-2)	if the next n bytes contribute to an incomplete (but potentially 
valid) multibyte character, and all n bytes have been processed (no value is 
stored).398)
(size_t)(-1)	if an encoding error occurs, in which case the next n or fewer 
bytes do not contribute to a complete and valid multibyte character (no value is 
stored); the value of the macro EILSEQ is stored in errno, and the conversion 
state is unspecified.

398)When n has at least the value of the MB_CUR_MAX macro, this case can only 
occur if s points at a sequence of redundant
shift sequences (for implementations with state-dependent encodings).

7.30.1.2 The c8rtomb function
Synopsis
1	#include <uchar.h>
	size_t c8rtomb(char * restrict s, char8_t c8, mbstate_t * restrict ps);
Description
2 If s is a null pointer, the c8rtomb function is equivalent to the call
			c8rtomb(buf, u8’\0’, ps)
where buf is an internal buffer.
3 If s is not a null pointer, the c8rtomb function determines the number of 
bytes needed to represent the multibyte character that corresponds to the 
character given or completed by c8 (including any shift sequences), and stores 
the multibyte character representation in the array whose first element is 
pointed to by s, or stores nothing if c8 does not represent a complete 
character. At most MB_CUR_MAX bytes are stored. If c8 is a null character, a 
null byte is stored, preceded by any shift sequence needed to restore the 
initial shift state; the resulting state described is the initial conversion state.
Returns
4 The c8rtomb function returns the number of bytes stored in the array object 
(including any shift sequences). When c8 is not a valid character, an encoding 
error occurs: the function stores the value of the macro EILSEQ in errno and 
returns (size_t)(-1); the conversion state is unspecified.
..."

-- 
Take care. Thanks, Brian Inglis              Calgary, Alberta, Canada

La perfection est atteinte                   Perfection is achieved
non pas lorsqu'il n'y a plus rien à ajouter  not when there is no more to add
mais lorsqu'il n'y a plus rien à retirer     but when there is no more to cut
                                 -- Antoine de Saint-Exupéry