[Bug libstdc++/63776] [C++11] Regex collate matching not working

["redi at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63776

--- Comment #4 from Jonathan Wakely <redi at gcc dot gnu.org> ---
(In reply to Tom Straub from comment #2)
> Hi Tim,
> 
> Okay, a program very similar to this using the Boost REGEX library and ICU
> 4.55 works just fine with this.
> 
> According to my understanding, the "char" data type and "std::string"
> classes were specifically set up in C++11 to handle UTF-8 sequences.

Nothing was done to char or std::string because they can already hold UTF-8
data. The new u8 literal prefix was added to produce UTF8-encoded string
literals.

> The "sequence of bytes" are actually valid UNICODE characters.

Right, and if your source character set wasn't UTF-8 you could still initialize
the std::string correctly with e.g.

  std::string s = u8"Jo\u00e3o M\u00e9ro\u00e7o";

> It is acting as if it is still in POSIX or C locale, since it doesn't
> recognize the accented characters as "[:alpha:]" class.

Yes, I don't know how it's supposed to work in <regex> but I agree there seems
to be a bug.

(In reply to Tim Shen from comment #1)
> I don't know much about unicode support status in the standard library.
> @Jon, can you put a comment?

We're missing the facilities for converting between different unicode encodings
which would allow us to convert multibyte char strings to wchar_t so that we
can use the ctype<wchar_t>::is(ctype_base::mask, wchar_t) function to match
non-ASCII characters.

Maybe a workaround for now would be to detect when we reach the first byte of a
UTF-8 character, read the rest of the multibyte character into a wchar_t and
use ctype<wchar_t> for that.

Or just wait for the rest of the ctype and codecvt features to be implemented.