[PATCH 1/9]: C++ P0482R5 char8_t: Documentation updates

[PATCH 1/9]: C++ P0482R5 char8_t: Documentation updates

[Tom Honermann]

[-- Attachment #1: Type: text/plain, Size: 202 bytes --]

This patch adds documentation for new -fchar8_t and -fno-char8_t options.

gcc/ChangeLog:

2018-11-04  Tom Honermann  <tom@honermann.net>
      * doc/invoke.texi (-fchar8_t): Document new option.

Tom.

[-- Attachment #2: p0482r5-1.patch --]
[-- Type: text/x-patch, Size: 2725 bytes --]

diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index 57491f1033c..cd3a2a715db 100644
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -206,7 +206,7 @@ in the following sections.
 @item C++ Language Options
 @xref{C++ Dialect Options,,Options Controlling C++ Dialect}.
 @gccoptlist{-fabi-version=@var{n}  -fno-access-control @gol
--faligned-new=@var{n}  -fargs-in-order=@var{n}  -fcheck-new @gol
+-faligned-new=@var{n}  -fargs-in-order=@var{n}  -fchar8_t -fcheck-new @gol
 -fconstexpr-depth=@var{n}  -fconstexpr-loop-limit=@var{n} @gol
 -fno-elide-constructors @gol
 -fno-enforce-eh-specs @gol
@@ -2432,6 +2432,53 @@ but few users will need to override the default of
 
 This flag is enabled by default for @option{-std=c++17}.
 
+@item -fchar8_t
+@itemx -fno-char8_t
+@opindex fchar8_t
+@opindex fno-char8_t
+Enable support for the P0482 proposal including the addition of a
+new @code{char8_t} fundamental type, changes to the types of UTF-8
+string and character literals, new signatures for user defined
+literals, and new specializations of standard library class templates
+@code{std::numeric_limits<char8_t>}, @code{std::char_traits<char8_t>},
+and @code{std::hash<char8_t>}.
+
+This option enables functions to be overloaded for ordinary and UTF-8
+strings:
+
+@smallexample
+int f(const char *);    // #1
+int f(const char8_t *); // #2
+int v1 = f("text");     // Calls #1
+int v2 = f(u8"text");   // Calls #2
+@end smallexample
+
+and introduces new signatures for user defined literals:
+
+@smallexample
+int operator""_udl1(char8_t);
+int v3 = u8'x'_udl1;
+int operator""_udl2(const char8_t*, std::size_t);
+int v4 = u8"text"_udl2;
+template<typename T, T...> int operator""_udl3();
+int v5 = u8"text"_udl3;
+@end smallexample
+
+The change to the types of UTF-8 string and character literals introduces
+incompatibilities with ISO C++11 and later standards.  For example, the
+following code is well-formed under ISO C++11, but is ill-formed when
+@option{-fchar8_t} is specified.
+
+@smallexample
+char ca[] = u8"text";       // error: char-array initialized from wide string
+const char *cp = u8"text";  // error: invalid conversion from 'const char8_t*' to 'const char*'
+int f(const char*);
+auto v = f(u8"text");       // error: invalid conversion from 'const char8_t*' to 'const char*'
+std::string s1@{u8"text"@};   // error: no matching function for call to 'std::basic_string<char>::basic_string()'
+using namespace std::literals;
+std::string s2 = u8"text"s; // error: conversion from 'basic_string<char8_t>' to non-scalar type 'basic_string<char>' requested
+@end smallexample
+
 @item -fcheck-new
 @opindex fcheck-new
 Check that the pointer returned by @code{operator new} is non-null

Re: [PATCH 1/9]: C++ P0482R5 char8_t: Documentation updates

[Jason Merrill]

On 11/5/18 2:39 PM, Tom Honermann wrote:
> This patch adds documentation for new -fchar8_t and -fno-char8_t options.
> 
> gcc/ChangeLog:
> 
> 2018-11-04  Tom Honermann  <tom@honermann.net>
>       * doc/invoke.texi (-fchar8_t): Document new option.

> +Enable support for the P0482 proposal including the addition of a
> +new @code{char8_t} fundamental type, changes to the types of UTF-8

Now that the proposal has been accepted, I'd refer to C++2a instead.

Jason

Re: [PATCH 1/9]: C++ P0482R5 char8_t: Documentation updates

[Tom Honermann]

On 12/3/18 2:59 PM, Jason Merrill wrote:
> On 11/5/18 2:39 PM, Tom Honermann wrote:
>> This patch adds documentation for new -fchar8_t and -fno-char8_t 
>> options.
>>
>> gcc/ChangeLog:
>>
>> 2018-11-04  Tom Honermann  <tom@honermann.net>
>>       * doc/invoke.texi (-fchar8_t): Document new option.
>
>> +Enable support for the P0482 proposal including the addition of a
>> +new @code{char8_t} fundamental type, changes to the types of UTF-8
>
> Now that the proposal has been accepted, I'd refer to C++2a instead.

Agreed.  I also need to make the changes to implicitly enable -fchar8_t 
with -std=c++2a.

The list of impacted standard library features was incomplete and I 
suspect it isn't worth mentioning them specifically.  Perhaps mentioning 
the feature test macros would be helpful as well?

How does the following sound?

Enable support for @code{char8_t} as adopted for C++2a.  This includes 
the addition of a new @code{char8_t} fundamental type, changes to the 
types of UTF-8 string and character literals, new signatures for user 
defined literals, associated standard library updates, and new 
@code{__cpp_char8_t} and @code{__cpp_lib_char8_t} feature test macros.

Tom.

>
> Jason

Re: [PATCH 1/9]: C++ P0482R5 char8_t: Documentation updates

[Jason Merrill]

On Wed, Dec 5, 2018 at 2:10 AM Tom Honermann <tom@honermann.net> wrote:
>
> On 12/3/18 2:59 PM, Jason Merrill wrote:
> > On 11/5/18 2:39 PM, Tom Honermann wrote:
> >> This patch adds documentation for new -fchar8_t and -fno-char8_t
> >> options.
> >>
> >> gcc/ChangeLog:
> >>
> >> 2018-11-04  Tom Honermann  <tom@honermann.net>
> >>       * doc/invoke.texi (-fchar8_t): Document new option.
> >
> >> +Enable support for the P0482 proposal including the addition of a
> >> +new @code{char8_t} fundamental type, changes to the types of UTF-8
> >
> > Now that the proposal has been accepted, I'd refer to C++2a instead.
>
> Agreed.  I also need to make the changes to implicitly enable -fchar8_t
> with -std=c++2a.
>
> The list of impacted standard library features was incomplete and I
> suspect it isn't worth mentioning them specifically.  Perhaps mentioning
> the feature test macros would be helpful as well?
>
> How does the following sound?
>
> Enable support for @code{char8_t} as adopted for C++2a.  This includes
> the addition of a new @code{char8_t} fundamental type, changes to the
> types of UTF-8 string and character literals, new signatures for user
> defined literals, associated standard library updates, and new
> @code{__cpp_char8_t} and @code{__cpp_lib_char8_t} feature test macros.

Sounds good.

Jason

Re: [PATCH 1/9]: C++ P0482R5 char8_t: Documentation updates

[Sandra Loosemore]

On 11/5/18 12:39 PM, Tom Honermann wrote:
> This patch adds documentation for new -fchar8_t and -fno-char8_t options.
> 
> gcc/ChangeLog:
> 
> 2018-11-04  Tom Honermann  <tom@honermann.net>
>       * doc/invoke.texi (-fchar8_t): Document new option.
> 

My comments are all about nitpicky formatting things.

> diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
> index 57491f1033c..cd3a2a715db 100644
> --- a/gcc/doc/invoke.texi
> +++ b/gcc/doc/invoke.texi
> @@ -206,7 +206,7 @@ in the following sections.
>  @item C++ Language Options
>  @xref{C++ Dialect Options,,Options Controlling C++ Dialect}.
>  @gccoptlist{-fabi-version=@var{n}  -fno-access-control @gol
> --faligned-new=@var{n}  -fargs-in-order=@var{n}  -fcheck-new @gol
> +-faligned-new=@var{n}  -fargs-in-order=@var{n}  -fchar8_t -fcheck-new @gol

Please consistently use 2 spaces (not just 1) to separate options on the 
same line in a @gccoptlist environment.

>  -fconstexpr-depth=@var{n}  -fconstexpr-loop-limit=@var{n} @gol
>  -fno-elide-constructors @gol
>  -fno-enforce-eh-specs @gol
> @@ -2432,6 +2432,53 @@ but few users will need to override the default of
>  
>  This flag is enabled by default for @option{-std=c++17}.
>  
> +@item -fchar8_t
> +@itemx -fno-char8_t
> +@opindex fchar8_t
> +@opindex fno-char8_t
> +Enable support for the P0482 proposal including the addition of a
> +new @code{char8_t} fundamental type, changes to the types of UTF-8
> +string and character literals, new signatures for user defined
> +literals, and new specializations of standard library class templates
> +@code{std::numeric_limits<char8_t>}, @code{std::char_traits<char8_t>},
> +and @code{std::hash<char8_t>}.
> +
> +This option enables functions to be overloaded for ordinary and UTF-8
> +strings:
> +
> +@smallexample
> +int f(const char *);    // #1
> +int f(const char8_t *); // #2
> +int v1 = f("text");     // Calls #1
> +int v2 = f(u8"text");   // Calls #2
> +@end smallexample
> +
> +and introduces new signatures for user defined literals:

@noindent immediately before the continued sentence of the paragraph 
before the example.

Also please hyphenate "user-defined" here.

> +
> +@smallexample
> +int operator""_udl1(char8_t);
> +int v3 = u8'x'_udl1;
> +int operator""_udl2(const char8_t*, std::size_t);
> +int v4 = u8"text"_udl2;
> +template<typename T, T...> int operator""_udl3();
> +int v5 = u8"text"_udl3;
> +@end smallexample
> +
> +The change to the types of UTF-8 string and character literals introduces
> +incompatibilities with ISO C++11 and later standards.  For example, the
> +following code is well-formed under ISO C++11, but is ill-formed when
> +@option{-fchar8_t} is specified.
> +
> +@smallexample
> +char ca[] = u8"text";       // error: char-array initialized from wide string
> +const char *cp = u8"text";  // error: invalid conversion from 'const char8_t*' to 'const char*'
> +int f(const char*);
> +auto v = f(u8"text");       // error: invalid conversion from 'const char8_t*' to 'const char*'
> +std::string s1@{u8"text"@};   // error: no matching function for call to 'std::basic_string<char>::basic_string()'
> +using namespace std::literals;
> +std::string s2 = u8"text"s; // error: conversion from 'basic_string<char8_t>' to non-scalar type 'basic_string<char>' requested
> +@end smallexample

The formatting of this code example is way too wide to fit on the page 
of the printed/PDF manual.  I suggest putting the comments on separate 
lines from the code and breaking them across multiple lines where 
necessary.  If you format the example for <80 columns it will probably 
fit, although you should check the PDF if at all possible.

> +
>  @item -fcheck-new
>  @opindex fcheck-new
>  Check that the pointer returned by @code{operator new} is non-null
> 

-Sandra

Re: [REVISED PATCH 1/9]: C++ P0482R5 char8_t: Documentation updates

[Tom Honermann]

[-- Attachment #1: Type: text/plain, Size: 440 bytes --]

Attached is a revised patch that addresses feedback provided by Jason 
and Sandra.  Changes from the prior patch include:
- Updates to the -fchar8_t option documentation as requested by Jason.
- Corrections for indentation, spacing, hyphenation, and wrapping as
   requested by Sandra.

Tested on x86_64-linux.

gcc/ChangeLog:

2018-11-04  Tom Honermann  <tom@honermann.net>
      * doc/invoke.texi (-fchar8_t): Document new option.

Tom.


[-- Attachment #2: p0482r5-1-2.patch --]
[-- Type: text/x-patch, Size: 2913 bytes --]

diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index 57491f1033c..95374951d98 100644
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -206,7 +206,7 @@ in the following sections.
 @item C++ Language Options
 @xref{C++ Dialect Options,,Options Controlling C++ Dialect}.
 @gccoptlist{-fabi-version=@var{n}  -fno-access-control @gol
--faligned-new=@var{n}  -fargs-in-order=@var{n}  -fcheck-new @gol
+-faligned-new=@var{n}  -fargs-in-order=@var{n}  -fchar8_t  -fcheck-new @gol
 -fconstexpr-depth=@var{n}  -fconstexpr-loop-limit=@var{n} @gol
 -fno-elide-constructors @gol
 -fno-enforce-eh-specs @gol
@@ -2432,6 +2432,60 @@ but few users will need to override the default of
 
 This flag is enabled by default for @option{-std=c++17}.
 
+@item -fchar8_t
+@itemx -fno-char8_t
+@opindex fchar8_t
+@opindex fno-char8_t
+Enable support for @code{char8_t} as adopted for C++2a.  This includes
+the addition of a new @code{char8_t} fundamental type, changes to the
+types of UTF-8 string and character literals, new signatures for
+user-defined literals, associated standard library updates, and new
+@code{__cpp_char8_t} and @code{__cpp_lib_char8_t} feature test macros.
+
+This option enables functions to be overloaded for ordinary and UTF-8
+strings:
+
+@smallexample
+int f(const char *);    // #1
+int f(const char8_t *); // #2
+int v1 = f("text");     // Calls #1
+int v2 = f(u8"text");   // Calls #2
+@end smallexample
+
+@noindent
+and introduces new signatures for user-defined literals:
+
+@smallexample
+int operator""_udl1(char8_t);
+int v3 = u8'x'_udl1;
+int operator""_udl2(const char8_t*, std::size_t);
+int v4 = u8"text"_udl2;
+template<typename T, T...> int operator""_udl3();
+int v5 = u8"text"_udl3;
+@end smallexample
+
+@noindent
+The change to the types of UTF-8 string and character literals introduces
+incompatibilities with ISO C++11 and later standards.  For example, the
+following code is well-formed under ISO C++11, but is ill-formed when
+@option{-fchar8_t} is specified.
+
+@smallexample
+char ca[] = u8"xx";     // error: char-array initialized from wide
+                        //        string
+const char *cp = u8"xx";// error: invalid conversion from
+                        //        `const char8_t*' to `const char*'
+int f(const char*);
+auto v = f(u8"xx");     // error: invalid conversion from
+                        //        `const char8_t*' to `const char*'
+std::string s@{u8"xx"@};  // error: no matching function for call to
+                        //        `std::basic_string<char>::basic_string()'
+using namespace std::literals;
+s = u8"xx"s;            // error: conversion from
+                        //        `basic_string<char8_t>' to non-scalar
+                        //        type `basic_string<char>' requested
+@end smallexample
+
 @item -fcheck-new
 @opindex fcheck-new
 Check that the pointer returned by @code{operator new} is non-null

Re: [PATCH 1/9]: C++ P0482R5 char8_t: Documentation updates

[Tom Honermann]

Thank you, Sandra!  I just sent a revised patch to the list that I 
believe addresses all of your comments.  Thanks for the suggestion to 
generate and check the pdf, that was helpful to ensure the changes 
rendered correctly.

Tom.

On 12/11/18 6:35 PM, Sandra Loosemore wrote:
> On 11/5/18 12:39 PM, Tom Honermann wrote:
>> This patch adds documentation for new -fchar8_t and -fno-char8_t 
>> options.
>>
>> gcc/ChangeLog:
>>
>> 2018-11-04  Tom Honermann  <tom@honermann.net>
>>       * doc/invoke.texi (-fchar8_t): Document new option.
>>
>
> My comments are all about nitpicky formatting things.
>
>> diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
>> index 57491f1033c..cd3a2a715db 100644
>> --- a/gcc/doc/invoke.texi
>> +++ b/gcc/doc/invoke.texi
>> @@ -206,7 +206,7 @@ in the following sections.
>>  @item C++ Language Options
>>  @xref{C++ Dialect Options,,Options Controlling C++ Dialect}.
>>  @gccoptlist{-fabi-version=@var{n}  -fno-access-control @gol
>> --faligned-new=@var{n}  -fargs-in-order=@var{n}  -fcheck-new @gol
>> +-faligned-new=@var{n}  -fargs-in-order=@var{n}  -fchar8_t 
>> -fcheck-new @gol
>
> Please consistently use 2 spaces (not just 1) to separate options on 
> the same line in a @gccoptlist environment.
>
>>  -fconstexpr-depth=@var{n} -fconstexpr-loop-limit=@var{n} @gol
>>  -fno-elide-constructors @gol
>>  -fno-enforce-eh-specs @gol
>> @@ -2432,6 +2432,53 @@ but few users will need to override the 
>> default of
>>
>>  This flag is enabled by default for @option{-std=c++17}.
>>
>> +@item -fchar8_t
>> +@itemx -fno-char8_t
>> +@opindex fchar8_t
>> +@opindex fno-char8_t
>> +Enable support for the P0482 proposal including the addition of a
>> +new @code{char8_t} fundamental type, changes to the types of UTF-8
>> +string and character literals, new signatures for user defined
>> +literals, and new specializations of standard library class templates
>> +@code{std::numeric_limits<char8_t>}, @code{std::char_traits<char8_t>},
>> +and @code{std::hash<char8_t>}.
>> +
>> +This option enables functions to be overloaded for ordinary and UTF-8
>> +strings:
>> +
>> +@smallexample
>> +int f(const char *);    // #1
>> +int f(const char8_t *); // #2
>> +int v1 = f("text");     // Calls #1
>> +int v2 = f(u8"text");   // Calls #2
>> +@end smallexample
>> +
>> +and introduces new signatures for user defined literals:
>
> @noindent immediately before the continued sentence of the paragraph 
> before the example.
>
> Also please hyphenate "user-defined" here.
>
>> +
>> +@smallexample
>> +int operator""_udl1(char8_t);
>> +int v3 = u8'x'_udl1;
>> +int operator""_udl2(const char8_t*, std::size_t);
>> +int v4 = u8"text"_udl2;
>> +template<typename T, T...> int operator""_udl3();
>> +int v5 = u8"text"_udl3;
>> +@end smallexample
>> +
>> +The change to the types of UTF-8 string and character literals 
>> introduces
>> +incompatibilities with ISO C++11 and later standards.  For example, the
>> +following code is well-formed under ISO C++11, but is ill-formed when
>> +@option{-fchar8_t} is specified.
>> +
>> +@smallexample
>> +char ca[] = u8"text";       // error: char-array initialized from 
>> wide string
>> +const char *cp = u8"text";  // error: invalid conversion from 'const 
>> char8_t*' to 'const char*'
>> +int f(const char*);
>> +auto v = f(u8"text");       // error: invalid conversion from 'const 
>> char8_t*' to 'const char*'
>> +std::string s1@{u8"text"@};   // error: no matching function for 
>> call to 'std::basic_string<char>::basic_string()'
>> +using namespace std::literals;
>> +std::string s2 = u8"text"s; // error: conversion from 
>> 'basic_string<char8_t>' to non-scalar type 'basic_string<char>' 
>> requested
>> +@end smallexample
>
> The formatting of this code example is way too wide to fit on the page 
> of the printed/PDF manual.  I suggest putting the comments on separate 
> lines from the code and breaking them across multiple lines where 
> necessary.  If you format the example for <80 columns it will probably 
> fit, although you should check the PDF if at all possible.
>
>> +
>>  @item -fcheck-new
>>  @opindex fcheck-new
>>  Check that the pointer returned by @code{operator new} is non-null
>>
>
> -Sandra

Re: [REVISED PATCH 1/9]: C++ P0482R5 char8_t: Documentation updates

[Martin Sebor]

On 12/23/18 7:27 PM, Tom Honermann wrote:
> Attached is a revised patch that addresses feedback provided by Jason 
> and Sandra.  Changes from the prior patch include:
> - Updates to the -fchar8_t option documentation as requested by Jason.
> - Corrections for indentation, spacing, hyphenation, and wrapping as
>    requested by Sandra.
> 

Just a minor nit that backticks in code examples should be avoided
(per the TexInfo manual, they can cause trouble when copying code
from PDF readers):

+@smallexample
+char ca[] = u8"xx";     // error: char-array initialized from wide
+                        //        string
+const char *cp = u8"xx";// error: invalid conversion from
+                        //        `const char8_t*' to `const char*'

Martin

Re: [REVISED PATCH 1/9]: C++ P0482R5 char8_t: Documentation updates

[Tom Honermann]

[-- Attachment #1: Type: text/plain, Size: 1179 bytes --]

On 1/4/19 7:40 PM, Martin Sebor wrote:
> On 12/23/18 7:27 PM, Tom Honermann wrote:
>> Attached is a revised patch that addresses feedback provided by Jason 
>> and Sandra.  Changes from the prior patch include:
>> - Updates to the -fchar8_t option documentation as requested by Jason.
>> - Corrections for indentation, spacing, hyphenation, and wrapping as
>>    requested by Sandra.
>>
>
> Just a minor nit that backticks in code examples should be avoided
> (per the TexInfo manual, they can cause trouble when copying code
> from PDF readers):
>
> +@smallexample
> +char ca[] = u8"xx";     // error: char-array initialized from wide
> +                        //        string
> +const char *cp = u8"xx";// error: invalid conversion from
> +                        //        `const char8_t*' to `const char*'

Thanks for catching that, Martin.  Patch relative to trunk (r267930) 
attached to correct this (Jason already committed the original change).

Tom.

>
> Martin



[-- Attachment #2: char8_t-doc-update.diff --]
[-- Type: text/x-patch, Size: 1360 bytes --]

Index: gcc/doc/invoke.texi
===================================================================
--- gcc/doc/invoke.texi	(revision 267930)
+++ gcc/doc/invoke.texi	(working copy)
@@ -2468,16 +2468,16 @@
 char ca[] = u8"xx";     // error: char-array initialized from wide
                         //        string
 const char *cp = u8"xx";// error: invalid conversion from
-                        //        `const char8_t*' to `const char*'
+                        //        'const char8_t*' to 'const char*'
 int f(const char*);
 auto v = f(u8"xx");     // error: invalid conversion from
-                        //        `const char8_t*' to `const char*'
+                        //        'const char8_t*' to 'const char*'
 std::string s@{u8"xx"@};  // error: no matching function for call to
-                        //        `std::basic_string<char>::basic_string()'
+                        //        'std::basic_string<char>::basic_string()'
 using namespace std::literals;
 s = u8"xx"s;            // error: conversion from
-                        //        `basic_string<char8_t>' to non-scalar
-                        //        type `basic_string<char>' requested
+                        //        'basic_string<char8_t>' to non-scalar
+                        //        type 'basic_string<char>' requested
 @end smallexample
 
 @item -fcheck-new