* [PATCH 1/2] libstdc++: Replace padding bits with a bit-field in __format::_Spec
@ 2024-02-01 15:36 Jonathan Wakely
2024-02-01 15:36 ` [PATCH 2/2] libstdc++: Handle encodings in localized chrono formatting [PR109162] Jonathan Wakely
` (2 more replies)
0 siblings, 3 replies; 11+ messages in thread
From: Jonathan Wakely @ 2024-02-01 15:36 UTC (permalink / raw)
To: libstdc++, gcc-patches; +Cc: Hans-Peter Nilsson
I plan to push this to trunk soon.
CC HP for visibility of the change affecting cris-elf. In practice it
shouldn't make any difference to any sensible code. It only affects
C++20 mode (and later), and only changes the size of std::formatter
objects which are typically only created by the library headers
themselves, and only on the stack (when using std::format and other new
C++20 APIs related to it).
-- >8 --
This ensures that the unused bits will be zero-initialized reliably, and
so can be used later by assigning them values in formatter
specializations. For example, formatters for std::chrono will need to
use an extra bit for a boolean to optimize the conversions between
locale encodings and UTF-8.
This will result in an ABI change for targets that use 1-byte alignment
for all integral types, e.g. cris-elf. We can't do that once C++20
support is non-experimental and ABI stable, so do it now before GCC 14
is released.
libstdc++-v3/ChangeLog:
* include/std/format (__format::_Spec::_M_reserved): Define a
new bit-field member in place of padding bits.
---
libstdc++-v3/include/std/format | 1 +
1 file changed, 1 insertion(+)
diff --git a/libstdc++-v3/include/std/format b/libstdc++-v3/include/std/format
index 0eca8b58bfa..6c958bc11a5 100644
--- a/libstdc++-v3/include/std/format
+++ b/libstdc++-v3/include/std/format
@@ -406,6 +406,7 @@ namespace __format
_WidthPrec _M_width_kind : 2;
_WidthPrec _M_prec_kind : 2;
_Pres_type _M_type : 4;
+ unsigned long _M_reserved : 17;
unsigned short _M_width;
unsigned short _M_prec;
char32_t _M_fill = ' ';
--
2.43.0
^ permalink raw reply [flat|nested] 11+ messages in thread* [PATCH 2/2] libstdc++: Handle encodings in localized chrono formatting [PR109162] 2024-02-01 15:36 [PATCH 1/2] libstdc++: Replace padding bits with a bit-field in __format::_Spec Jonathan Wakely @ 2024-02-01 15:36 ` Jonathan Wakely 2024-02-01 16:43 ` Arsen Arsenović 2024-02-01 16:16 ` [PATCH 1/2] libstdc++: Replace padding bits with a bit-field in __format::_Spec Hans-Peter Nilsson 2024-02-01 16:33 ` Andreas Schwab 2 siblings, 1 reply; 11+ messages in thread From: Jonathan Wakely @ 2024-02-01 15:36 UTC (permalink / raw) To: libstdc++, gcc-patches; +Cc: Hans-Peter Nilsson I am undecided about pushing this PATCH 2/2 to trunk. PATCH 1/2 needs to be done now due to the ABI impact on cris-elf. This one could wait for stage 1. (HP, this one probably isn't of interest to you, but I don't know how to tell git-send-email to only CC you on the first patch). I'd like to do this change now so people can experiment with the new feature by linking with -lstdc++exp, but the problem is that linking to that library becomes a hard requirement for any uses of std::chrono types with std::format in C++23. So code that compiles and links in C++20 mode will fail to link in C++23 mode unless you use the extra libstdc++exp.a library. The code might have worked fine in C++20 mode, or it might have just linked OK but produce mojibake at runtime, it depends on the locale. Declaring the new functions as weak and then checking them for non-null at runtime doesn't really help, because the non-weak symbols in the static lib won't be used without -Wl,--whole-archive -lstdc++exp because it's OK for weak symbols to be unresolved. We could play games with linker scripts so that libstdc++exp.a is a linker script which ensures the non-weak defs get used, but I'm not convinced it's worth the hassle for a feature that will probably get added to the main libstdc++.so in stage 1 anyway. What do others think? Should I just push it and require -lstdc++exp for chrono formatting in C++23 mode? -- >8 -- This implements the C++23 paper P2419R2 (Clarify handling of encodings in localized formatting of chrono types). The requirement is that when the literal encoding is "a Unicode encoding form" and the formatting locale uses a different encoding, any locale-specific strings such as "août" for std::chrono::August should be converted to the literal encoding. Using the recently-added std::locale::encoding() function we can check the locale's encoding and then use iconv if a conversion is needed. This requires a new non-inline function in the library. As this is currently experimental, the new symbol is added to libstdc++exp.a for now. This means that formatting std::chrono types requires -lstdc++exp in C++23 and later, at least until the new function is considered stable enough to add to the main library. When that happens, we might want to consider enabling these encoding conversions for C++20 too, treating it as a DR that resolves LWG 3656. Because nl_langinfo_l and iconv_open both allocate memory, a naive implementation would perform multiple allocations and deallocations for every snippet of locale-specific text that needs to be converted to UTF-8. To avoid that, a new internal locale::facet is defined to store the text_encoding and an iconv_t descriptor, which are then cached in the formatting locale. This requires access to the internals of a std::locale object in src/c++26/text_encoding.cc, so that file now needs to be compiled with -fno-access-control. When the new symbols move into libstdc++.so we should be able to avoid that by adding new member functions to std::locale. With this change we can increase the value of the __cpp_lib_format macro for C++23. The value should be 202207 for P2419R2, but we already implement P2510R3 (Formatting pointers) so can use the value 202304. libstdc++-v3/ChangeLog: PR libstdc++/109162 * include/bits/chrono_io.h (_ChronoSpec::_M_locale_specific): Add new accessor functions to use a reserved bit in _Spec. (__formatter_chrono::_M_parse): Use _M_locale_specific(true) when chrono-specs contains locale-dependent conversion specifiers. (__formatter_chrono::_M_format): Open iconv descriptor if conversion to UTF-8 will be needed. (__formatter_chrono::_M_write): New function to write a localized string with possible character conversion. (__formatter_chrono::_M_a_A, __formatter_chrono::_M_b_B) (__formatter_chrono::_M_p, __formatter_chrono::_M_r) (__formatter_chrono::_M_x, __formatter_chrono::_M_X) (__formatter_chrono::_M_locale_fmt): Use _M_write. * include/bits/version.def: Add C++23 values for format. * include/bits/version.h: Regenerate. * include/std/format (_GLIBCXX_P2518R3): Check feature test macro instead of __cplusplus. (basic_format_context): Declare __formatter_chrono as friend. * src/c++26/Makefile.am: Use -fno-access-control for text_encoding.o. * src/c++26/Makefile.in: Regenerate. * src/c++26/text_encoding.cc (__encoding): New locale facet. (locale::encoding): Check for __encoding facet first. Add fast path for "C" locale. (__with_encoding_conversion): New function to add __encoding facet to a locale. (__locale_encoding_to_utf8): New function to convert a string from a locale's encoding to UTF-8. * testsuite/20_util/duration/io.cc: Add -lstdc++exp for C++23. * testsuite/std/time/clock/file/io.cc: Likewise. * testsuite/std/time/clock/gps/io.cc: Likewise. * testsuite/std/time/clock/local/io.cc: Likewise. * testsuite/std/time/clock/system/io.cc: Likewise. * testsuite/std/time/clock/tai/io.cc: Likewise. * testsuite/std/time/clock/utc/io.cc: Likewise. * testsuite/std/time/day/io.cc: Likewise. * testsuite/std/time/exceptions.cc: Likewise. * testsuite/std/time/format.cc: Likewise. * testsuite/std/time/hh_mm_ss/io.cc: Likewise. * testsuite/std/time/month/io.cc: Likewise. * testsuite/std/time/month_day/io.cc: Likewise. * testsuite/std/time/month_day_last/io.cc: Likewise. * testsuite/std/time/month_weekday/io.cc: Likewise. * testsuite/std/time/month_weekday_last/io.cc: Likewise. * testsuite/std/time/time_zone/get_info_local.cc: Likewise. * testsuite/std/time/weekday/io.cc: Likewise. * testsuite/std/time/weekday_indexed/io.cc: Likewise. * testsuite/std/time/weekday_last/io.cc: Likewise. * testsuite/std/time/year/io.cc: Likewise. * testsuite/std/time/year_month/io.cc: Likewise. * testsuite/std/time/year_month_day/io.cc: Likewise. * testsuite/std/time/year_month_day_last/io.cc: Likewise. * testsuite/std/time/year_month_weekday/io.cc: Likewise. * testsuite/std/time/year_month_weekday_last/io.cc: Likewise. * testsuite/std/time/zoned_time/1.cc: Likewise. * testsuite/std/time/zoned_time/io.cc: Likewise. * testsuite/std/time/format_localized.cc: New test. --- libstdc++-v3/include/bits/chrono_io.h | 103 ++++++++++-- libstdc++-v3/include/bits/version.def | 34 +++- libstdc++-v3/include/bits/version.h | 7 +- libstdc++-v3/include/std/format | 16 +- libstdc++-v3/src/c++26/Makefile.am | 6 + libstdc++-v3/src/c++26/Makefile.in | 6 + libstdc++-v3/src/c++26/text_encoding.cc | 151 +++++++++++++++++- libstdc++-v3/testsuite/20_util/duration/io.cc | 1 + .../testsuite/std/time/clock/file/io.cc | 1 + .../testsuite/std/time/clock/gps/io.cc | 1 + .../testsuite/std/time/clock/local/io.cc | 1 + .../testsuite/std/time/clock/system/io.cc | 1 + .../testsuite/std/time/clock/tai/io.cc | 1 + .../testsuite/std/time/clock/utc/io.cc | 1 + libstdc++-v3/testsuite/std/time/day/io.cc | 1 + libstdc++-v3/testsuite/std/time/exceptions.cc | 1 + libstdc++-v3/testsuite/std/time/format.cc | 1 + .../testsuite/std/time/format_localized.cc | 48 ++++++ .../testsuite/std/time/hh_mm_ss/io.cc | 1 + libstdc++-v3/testsuite/std/time/month/io.cc | 1 + .../testsuite/std/time/month_day/io.cc | 1 + .../testsuite/std/time/month_day_last/io.cc | 1 + .../testsuite/std/time/month_weekday/io.cc | 1 + .../std/time/month_weekday_last/io.cc | 1 + .../std/time/time_zone/get_info_local.cc | 1 + libstdc++-v3/testsuite/std/time/weekday/io.cc | 1 + .../testsuite/std/time/weekday_indexed/io.cc | 1 + .../testsuite/std/time/weekday_last/io.cc | 1 + libstdc++-v3/testsuite/std/time/year/io.cc | 1 + .../testsuite/std/time/year_month/io.cc | 1 + .../testsuite/std/time/year_month_day/io.cc | 1 + .../std/time/year_month_day_last/io.cc | 1 + .../std/time/year_month_weekday/io.cc | 1 + .../std/time/year_month_weekday_last/io.cc | 1 + .../testsuite/std/time/zoned_time/1.cc | 1 + .../testsuite/std/time/zoned_time/io.cc | 1 + 36 files changed, 375 insertions(+), 24 deletions(-) create mode 100644 libstdc++-v3/testsuite/std/time/format_localized.cc diff --git a/libstdc++-v3/include/bits/chrono_io.h b/libstdc++-v3/include/bits/chrono_io.h index 82f2d39ec44..9f5b9e13a82 100644 --- a/libstdc++-v3/include/bits/chrono_io.h +++ b/libstdc++-v3/include/bits/chrono_io.h @@ -210,6 +210,20 @@ namespace __format struct _ChronoSpec : _Spec<_CharT> { basic_string_view<_CharT> _M_chrono_specs; + + // Use one of the reserved bits in __format::_Spec<C>. + // This indicates that a locale-dependent conversion specifier such as + // %a is used in the chrono-specs. This is not the same as the + // _Spec<C>::_M_localized member which indicates that "L" was present + // in the format-spec, e.g. "{:L%a}" is localized and locale-specific, + // but "{:L}" is only localized and "{:%a}" is only locale-specific. + constexpr bool + _M_locale_specific() const noexcept + { return this->_M_reserved & 1; } + + constexpr void + _M_locale_specific(bool __b) noexcept + { this->_M_reserved |= __b; } }; // Represents the information provided by a chrono type. @@ -304,11 +318,12 @@ namespace __format const auto __chrono_specs = __first++; // Skip leading '%' if (*__chrono_specs != '%') __throw_format_error("chrono format error: no '%' at start of " - "chrono-specs"); + "chrono-specs"); _CharT __mod{}; bool __conv = true; int __needed = 0; + bool __locale_specific = false; while (__first != __last) { @@ -321,15 +336,18 @@ namespace __format case 'a': case 'A': __needed = _Weekday; + __locale_specific = true; break; case 'b': case 'h': case 'B': __needed = _Month; + __locale_specific = true; break; case 'c': __needed = _DateTime; __allowed_mods = _Mod_E; + __locale_specific = true; break; case 'C': __needed = _Year; @@ -367,6 +385,8 @@ namespace __format break; case 'p': case 'r': + __locale_specific = true; + [[fallthrough]]; case 'R': case 'T': __needed = _TimeOfDay; @@ -392,10 +412,12 @@ namespace __format break; case 'x': __needed = _Date; + __locale_specific = true; __allowed_mods = _Mod_E; break; case 'X': __needed = _TimeOfDay; + __locale_specific = true; __allowed_mods = _Mod_E; break; case 'y': @@ -435,6 +457,8 @@ namespace __format || (__mod == 'O' && !(__allowed_mods & _Mod_O))) __throw_format_error("chrono format error: invalid " " modifier in chrono-specs"); + if (__mod && __c != 'z') + __locale_specific = true; __mod = _CharT(); if ((__parts & __needed) != __needed) @@ -466,6 +490,7 @@ namespace __format _M_spec = __spec; _M_spec._M_chrono_specs = __string_view(__chrono_specs, __first - __chrono_specs); + _M_spec._M_locale_specific(__locale_specific); return __first; } @@ -485,6 +510,26 @@ namespace __format if (__first == __last) return _M_format_to_ostream(__t, __fc, __is_neg); +#if __glibcxx_format >= 202207L // C++ >= 23 +#if _GLIBCXX_USE_CXX11_ABI + // _GLIBCXX_RESOLVE_LIB_DEFECTS + // 3565. Handling of encodings in localized formatting + // of chrono types is underspecified + if constexpr (is_same_v<_CharT, char>) + if constexpr (__unicode::__literal_encoding_is_utf8()) + if (_M_spec._M_localized && _M_spec._M_locale_specific()) + { + extern locale __with_encoding_conversion(const locale&); + + // Allocate and cache the necessary state to convert strings + // in the locale's encoding to UTF-8. + locale __loc = __fc.locale(); + if (__loc != locale::classic()) + __fc._M_loc = __with_encoding_conversion(__loc); + } +#endif +#endif + _Sink_iter<_CharT> __out; __format::_Str_sink<_CharT> __sink; bool __write_direct = false; @@ -741,6 +786,37 @@ namespace __format static constexpr _CharT _S_space = _S_chars[14]; static constexpr const _CharT* _S_empty_spec = _S_chars + 15; + template<typename _OutIter> + _OutIter + _M_write(_OutIter __out, const locale& __loc, __string_view __s) const + { + // FIXME: It would be nice to treat this as a DR for C++20 + // but it currently requires new symbols in libstdc++exp.a + // which we don't want to require for C++20. Could do it when + // those symbols move into the non-experimental library. +#if __glibcxx_format >= 202207L // C++ >= 23 +#if _GLIBCXX_USE_CXX11_ABI + // _GLIBCXX_RESOLVE_LIB_DEFECTS + // 3565. Handling of encodings in localized formatting + // of chrono types is underspecified + using _StringType = string; + string __buf; + if constexpr (is_same_v<_CharT, char>) + if constexpr (__unicode::__literal_encoding_is_utf8()) + if (_M_spec._M_localized && _M_spec._M_locale_specific() + && __loc != locale::classic()) + { + extern string_view + __locale_encoding_to_utf8(const std::locale&, string_view, + _StringType&); + + __s = __locale_encoding_to_utf8(__loc, __s, __buf); + } +#endif +#endif + return __format::__write(std::move(__out), __s); + } + template<typename _Tp, typename _FormatContext> typename _FormatContext::iterator _M_a_A(const _Tp& __t, typename _FormatContext::iterator __out, @@ -760,7 +836,7 @@ namespace __format else __tp._M_days_abbreviated(__days); __string_view __str(__days[__wd.c_encoding()]); - return __format::__write(std::move(__out), __str); + return _M_write(std::move(__out), __loc, __str); } template<typename _Tp, typename _FormatContext> @@ -781,7 +857,7 @@ namespace __format else __tp._M_months_abbreviated(__months); __string_view __str(__months[(unsigned)__m - 1]); - return __format::__write(std::move(__out), __str); + return _M_write(std::move(__out), __loc, __str); } template<typename _Tp, typename _FormatContext> @@ -1058,8 +1134,8 @@ namespace __format const auto& __tp = use_facet<__timepunct<_CharT>>(__loc); const _CharT* __ampm[2]; __tp._M_am_pm(__ampm); - return std::format_to(std::move(__out), _S_empty_spec, - __ampm[__hms.hours().count() >= 12]); + return _M_write(std::move(__out), __loc, + __ampm[__hms.hours().count() >= 12]); } template<typename _Tp, typename _FormatContext> @@ -1094,8 +1170,9 @@ namespace __format basic_string<_CharT> __fmt(_S_empty_spec); __fmt.insert(1u, 1u, _S_colon); __fmt.insert(2u, __ampm_fmt); - return std::vformat_to(std::move(__out), __fmt, - std::make_format_args<_FormatContext>(__t)); + using _FmtStr = _Runtime_format_string<_CharT>; + return _M_write(std::move(__out), __loc, + std::format(__loc, _FmtStr(__fmt), __t)); } template<typename _Tp, typename _FormatContext> @@ -1278,8 +1355,9 @@ namespace __format basic_string<_CharT> __fmt(_S_empty_spec); __fmt.insert(1u, 1u, _S_colon); __fmt.insert(2u, __rep); - return std::vformat_to(std::move(__out), __fmt, - std::make_format_args<_FormatContext>(__t)); + using _FmtStr = _Runtime_format_string<_CharT>; + return _M_write(std::move(__out), __loc, + std::format(__loc, _FmtStr(__fmt), __t)); } template<typename _Tp, typename _FormatContext> @@ -1301,8 +1379,9 @@ namespace __format basic_string<_CharT> __fmt(_S_empty_spec); __fmt.insert(1u, 1u, _S_colon); __fmt.insert(2u, __rep); - return std::vformat_to(std::move(__out), __fmt, - std::make_format_args<_FormatContext>(__t)); + using _FmtStr = _Runtime_format_string<_CharT>; + return _M_write(std::move(__out), __loc, + std::format(__loc, _FmtStr(__fmt), __t)); } template<typename _Tp, typename _FormatContext> @@ -1579,7 +1658,7 @@ namespace __format const auto& __tp = use_facet<time_put<_CharT>>(__loc); __tp.put(__os, __os, _S_space, &__tm, __fmt, __mod); if (__os) - __out = __format::__write(std::move(__out), __os.view()); + __out = _M_write(std::move(__out), __loc, __os.view()); return __out; } }; diff --git a/libstdc++-v3/include/bits/version.def b/libstdc++-v3/include/bits/version.def index 8fb8a2877ee..29ca973fe43 100644 --- a/libstdc++-v3/include/bits/version.def +++ b/libstdc++-v3/include/bits/version.def @@ -1160,14 +1160,25 @@ ftms = { }; ftms = { + name = format; + // 202311 P2918R2 Runtime format strings II + // values = { + // v = 202311; + // cxxmin = 26; + // hosted = yes; + // }; + // 202207 Encodings in localized formatting of chrono, basic-format-string. + // 202304 P2510R3 Formatting pointers + // 202305 P2757R3 Type checking format args + // 202306 P2637R3 Member visit + values = { + v = 202304; + cxxmin = 23; + hosted = yes; + }; // 201907 Text Formatting, Integration of chrono, printf corner cases. // 202106 std::format improvements. // 202110 Fixing locale handling in chrono formatters, generator-like types. - // 202207 Encodings in localized formatting of chrono, basic-format-string. - // 202207 P2286R8 Formatting Ranges - // 202207 P2585R1 Improving default container formatting - // TODO: #define __cpp_lib_format_ranges 202207L - name = format; values = { v = 202110; cxxmin = 20; @@ -1364,6 +1375,19 @@ ftms = { }; }; +// ftms = { + // name = format_ranges; + // 202207 P2286R8 Formatting Ranges + // 202207 P2585R1 Improving default container formatting + // LWG3750 Too many papers bump __cpp_lib_format + // TODO: #define __cpp_lib_format_ranges 202207L + // values = { + // v = 202207; + // cxxmin = 23; + // hosted = yes; + // }; +// }; + ftms = { name = freestanding_algorithm; values = { diff --git a/libstdc++-v3/include/bits/version.h b/libstdc++-v3/include/bits/version.h index fa4e89cf845..714bf064950 100644 --- a/libstdc++-v3/include/bits/version.h +++ b/libstdc++-v3/include/bits/version.h @@ -1304,7 +1304,12 @@ #undef __glibcxx_want_barrier #if !defined(__cpp_lib_format) -# if (__cplusplus >= 202002L) && _GLIBCXX_HOSTED +# if (__cplusplus >= 202100L) && _GLIBCXX_HOSTED +# define __glibcxx_format 202304L +# if defined(__glibcxx_want_all) || defined(__glibcxx_want_format) +# define __cpp_lib_format 202304L +# endif +# elif (__cplusplus >= 202002L) && _GLIBCXX_HOSTED # define __glibcxx_format 202110L # if defined(__glibcxx_want_all) || defined(__glibcxx_want_format) # define __cpp_lib_format 202110L diff --git a/libstdc++-v3/include/std/format b/libstdc++-v3/include/std/format index 6c958bc11a5..1f72cb9b9ff 100644 --- a/libstdc++-v3/include/std/format +++ b/libstdc++-v3/include/std/format @@ -2341,10 +2341,10 @@ namespace __format // _GLIBCXX_RESOLVE_LIB_DEFECTS // P2510R3 Formatting pointers -#if __cplusplus > 202302L || ! defined __STRICT_ANSI__ -#define _GLIBCXX_P2518R3 1 +#if __glibcxx_format >= 202304L || ! defined __STRICT_ANSI__ +# define _GLIBCXX_P2518R3 1 #else -#define _GLIBCXX_P2518R3 0 +# define _GLIBCXX_P2518R3 0 #endif #if _GLIBCXX_P2518R3 @@ -3788,6 +3788,9 @@ namespace __format __do_vformat_to(_Out, basic_string_view<_CharT>, const basic_format_args<_Context>&, const locale* = nullptr); + + template<typename _CharT> struct __formatter_chrono; + } // namespace __format /// @endcond @@ -3798,6 +3801,11 @@ namespace __format * this class template explicitly. For typical uses of `std::format` the * library will use the specializations `std::format_context` (for `char`) * and `std::wformat_context` (for `wchar_t`). + * + * You are not allowed to define partial or explicit specializations of + * this class template. + * + * @since C++20 */ template<typename _Out, typename _CharT> class basic_format_context @@ -3824,6 +3832,8 @@ namespace __format const basic_format_args<_Context2>&, const locale*); + friend __format::__formatter_chrono<_CharT>; + public: basic_format_context() = default; ~basic_format_context() = default; diff --git a/libstdc++-v3/src/c++26/Makefile.am b/libstdc++-v3/src/c++26/Makefile.am index 000ced1f501..1f9dff7e737 100644 --- a/libstdc++-v3/src/c++26/Makefile.am +++ b/libstdc++-v3/src/c++26/Makefile.am @@ -46,6 +46,12 @@ else libc__26convenience_la_SOURCES = endif +# This needs access to the internals of std::locale. +text_encoding.lo: text_encoding.cc + $(LTCXXCOMPILE) -fno-access-control -c $< +text_encoding.o: text_encoding.cc + $(CXXCOMPILE) -fno-access-control -c $< + # AM_CXXFLAGS needs to be in each subdirectory so that it can be # modified in a per-library or per-sub-library way. Need to manually # set this option because CONFIG_CXXFLAGS has to be after diff --git a/libstdc++-v3/src/c++26/Makefile.in b/libstdc++-v3/src/c++26/Makefile.in index 77e73b2b265..96947ccdd0c 100644 --- a/libstdc++-v3/src/c++26/Makefile.in +++ b/libstdc++-v3/src/c++26/Makefile.in @@ -742,6 +742,12 @@ uninstall-am: vpath % $(top_srcdir)/src/c++26 +# This needs access to the internals of std::locale. +text_encoding.lo: text_encoding.cc + $(LTCXXCOMPILE) -fno-access-control -c $< +text_encoding.o: text_encoding.cc + $(CXXCOMPILE) -fno-access-control -c $< + # Tell versions [3.59,3.63) of GNU make to not export all variables. # Otherwise a system limit (for SysV at least) may be exceeded. .NOEXPORT: diff --git a/libstdc++-v3/src/c++26/text_encoding.cc b/libstdc++-v3/src/c++26/text_encoding.cc index b9a50ef1a00..fc5141e02c7 100644 --- a/libstdc++-v3/src/c++26/text_encoding.cc +++ b/libstdc++-v3/src/c++26/text_encoding.cc @@ -26,17 +26,28 @@ #include <locale> #ifdef _GLIBCXX_USE_NL_LANGINFO_L +#include <memory> // make_unique +#include <string.h> // strlen, strcpy #include <locale.h> #if __has_include(<xlocale.h>) # include <xlocale.h> #endif #include <langinfo.h> +#ifdef _GLIBCXX_HAVE_ICONV +# include <format> +# include <chrono> +# include <iconv.h> +# include <errno.h> +#endif + #if __CHAR_BIT__ == 8 namespace std { _GLIBCXX_BEGIN_NAMESPACE_VERSION - +namespace +{ +// Attempt to determine the text_encoding used by the named locale. text_encoding __locale_encoding(const char* name) { @@ -54,6 +65,48 @@ __locale_encoding(const char* name) return enc; } +// A non-standard locale::facet that caches the locale's std::text_encoding +// and an iconv descriptor for converting from that encoding to UTF-8. +struct __encoding : locale::facet +{ + static locale::id id; + + explicit + __encoding(const text_encoding& enc, size_t refs = 0) + : facet(refs), _M_enc(enc) + { +#if defined _GLIBCXX_HAVE_ICONV + if (enc != text_encoding::UTF8 && enc != text_encoding::ASCII) + _M_cd = ::iconv_open("UTF-8", enc.name()); +#endif + } + + ~__encoding() + { +#if defined _GLIBCXX_HAVE_ICONV + if (_M_has_desc()) + ::iconv_close(_M_cd); +#endif + } + + bool _M_has_desc() const + { +#if defined _GLIBCXX_HAVE_ICONV + return _M_cd != (::iconv_t)-1; +#else + return false; +#endif + } + + text_encoding _M_enc; +#if defined _GLIBCXX_HAVE_ICONV + ::iconv_t _M_cd = (::iconv_t)-1; +#endif +}; + +locale::id __encoding::id; + +} // namespace _GLIBCXX_END_NAMESPACE_VERSION } // namespace std @@ -87,8 +140,100 @@ std::text_encoding::_M_is_environment() const std::text_encoding std::locale::encoding() const { - return std::__locale_encoding(name().c_str()); + if (auto enc_facet = std::__try_use_facet<__encoding>(*this)) + return enc_facet->_M_enc; + string name = this->name(); + if (name == "*") + return {}; + if (name == "C") + return text_encoding(text_encoding::ASCII); + return __locale_encoding(name.c_str()); } #endif // CHAR_BIT == 8 - #endif // _GLIBCXX_USE_NL_LANGINFO_L + +namespace std +{ +_GLIBCXX_BEGIN_NAMESPACE_VERSION +namespace __format +{ +// Helpers for P2419R2 +// (Clarify handling of encodings in localized formatting of chrono types) +// Convert a string from the locale's charset to UTF-8. + +std::locale +__with_encoding_conversion(const std::locale& loc) +{ +#if defined _GLIBCXX_USE_NL_LANGINFO_L && __CHAR_BIT__ == 8 + const text_encoding locenc = loc.encoding(); + + if (locenc == text_encoding::UTF8 || locenc == text_encoding::ASCII + || locenc == text_encoding::unknown) + return loc; + + auto impl = std::make_unique<locale::_Impl>(*loc._M_impl, 1); + auto facetp = std::make_unique<__encoding>(locenc); + locale loc2(loc, facetp.get()); // FIXME: PR libstdc++/113704 + facetp.release(); + // FIXME: Ideally we wouldn't need to reallocate this string again, + // just don't delete[] it in the locale(locale, Facet*) constructor. + if (const char* name = loc._M_impl->_M_names[0]) + { + loc2._M_impl->_M_names[0] = new char[strlen(name) + 1]; + strcpy(loc2._M_impl->_M_names[0], name); + } + return loc2; +#else + return loc; +#endif +} + +string_view +__locale_encoding_to_utf8(const std::locale& loc, string_view str, + string& outbuf) +{ +#ifdef _GLIBCXX_HAVE_ICONV + // Don't need to use __try_use_facet with its dynamic_cast<__encoding*>, + // since we know there are no types derived from __encoding. If the array + // element is non-null, we have the facet. + auto id = __encoding::id._M_id(); + auto enc_facet = static_cast<const __encoding*>(loc._M_impl->_M_facets[id]); + if (!enc_facet || !enc_facet->_M_has_desc()) + return str; + + size_t inbytesleft = str.size(); + size_t written = 0; + bool done = false; + + auto overwrite = [&](char* p, size_t n) { + auto inbytes = const_cast<char*>(str.data()) + str.size() - inbytesleft; + char* outbytes = p + written; + size_t outbytesleft = n - written; + size_t res = ::iconv(enc_facet->_M_cd, &inbytes, &inbytesleft, + &outbytes, &outbytesleft); + if (res == (size_t)-1) + { + if (errno != E2BIG) + { + done = true; + return 0zu; + } + } + else + done = true; + written = outbytes - p; + return written; + }; + do + outbuf.resize_and_overwrite(outbuf.capacity() + (inbytesleft * 3 / 2), + overwrite); + while (!done); + if (outbuf.size()) + str = outbuf; +#endif // HAVE_ICONV + + return str; +} +} // namespace __format +_GLIBCXX_END_NAMESPACE_VERSION +} // namespace std diff --git a/libstdc++-v3/testsuite/20_util/duration/io.cc b/libstdc++-v3/testsuite/20_util/duration/io.cc index e141baf42dc..30469fb861d 100644 --- a/libstdc++-v3/testsuite/20_util/duration/io.cc +++ b/libstdc++-v3/testsuite/20_util/duration/io.cc @@ -1,3 +1,4 @@ +// { dg-options "-lstdc++exp" { target c++23 } } // { dg-do run { target c++20 } } // { dg-timeout-factor 2 } diff --git a/libstdc++-v3/testsuite/std/time/clock/file/io.cc b/libstdc++-v3/testsuite/std/time/clock/file/io.cc index 9ab9f10ec77..2ab4a43c7e3 100644 --- a/libstdc++-v3/testsuite/std/time/clock/file/io.cc +++ b/libstdc++-v3/testsuite/std/time/clock/file/io.cc @@ -1,3 +1,4 @@ +// { dg-options "-lstdc++exp" { target c++23 } } // { dg-do run { target c++20 } } // { dg-timeout-factor 2 } diff --git a/libstdc++-v3/testsuite/std/time/clock/gps/io.cc b/libstdc++-v3/testsuite/std/time/clock/gps/io.cc index d5405f61520..c3db63ab524 100644 --- a/libstdc++-v3/testsuite/std/time/clock/gps/io.cc +++ b/libstdc++-v3/testsuite/std/time/clock/gps/io.cc @@ -1,3 +1,4 @@ +// { dg-options "-lstdc++exp" { target c++23 } } // { dg-do run { target c++20 } } // { dg-timeout-factor 2 } diff --git a/libstdc++-v3/testsuite/std/time/clock/local/io.cc b/libstdc++-v3/testsuite/std/time/clock/local/io.cc index bb682cd40cf..bb0ba0df419 100644 --- a/libstdc++-v3/testsuite/std/time/clock/local/io.cc +++ b/libstdc++-v3/testsuite/std/time/clock/local/io.cc @@ -1,3 +1,4 @@ +// { dg-options "-lstdc++exp" { target c++23 } } // { dg-do run { target c++20 } } // { dg-timeout-factor 2 } diff --git a/libstdc++-v3/testsuite/std/time/clock/system/io.cc b/libstdc++-v3/testsuite/std/time/clock/system/io.cc index 2cc116156f2..dcb28dd51cf 100644 --- a/libstdc++-v3/testsuite/std/time/clock/system/io.cc +++ b/libstdc++-v3/testsuite/std/time/clock/system/io.cc @@ -1,3 +1,4 @@ +// { dg-options "-lstdc++exp" { target c++23 } } // { dg-do run { target c++20 } } // { dg-timeout-factor 2 } diff --git a/libstdc++-v3/testsuite/std/time/clock/tai/io.cc b/libstdc++-v3/testsuite/std/time/clock/tai/io.cc index 0fd61c0e612..b85fb761d99 100644 --- a/libstdc++-v3/testsuite/std/time/clock/tai/io.cc +++ b/libstdc++-v3/testsuite/std/time/clock/tai/io.cc @@ -1,3 +1,4 @@ +// { dg-options "-lstdc++exp" { target c++23 } } // { dg-do run { target c++20 } } // { dg-timeout-factor 2 } diff --git a/libstdc++-v3/testsuite/std/time/clock/utc/io.cc b/libstdc++-v3/testsuite/std/time/clock/utc/io.cc index 55c53dc4057..867c912cfdd 100644 --- a/libstdc++-v3/testsuite/std/time/clock/utc/io.cc +++ b/libstdc++-v3/testsuite/std/time/clock/utc/io.cc @@ -1,3 +1,4 @@ +// { dg-options "-lstdc++exp" { target c++23 } } // { dg-do run { target c++20 } } // { dg-timeout-factor 2 } diff --git a/libstdc++-v3/testsuite/std/time/day/io.cc b/libstdc++-v3/testsuite/std/time/day/io.cc index 36ce7ec7d17..eb9be402e6f 100644 --- a/libstdc++-v3/testsuite/std/time/day/io.cc +++ b/libstdc++-v3/testsuite/std/time/day/io.cc @@ -1,3 +1,4 @@ +// { dg-options "-lstdc++exp" { target c++23 } } // { dg-do run { target c++20 } } // { dg-require-namedlocale "fr_FR.ISO8859-15" } // { dg-timeout-factor 2 } diff --git a/libstdc++-v3/testsuite/std/time/exceptions.cc b/libstdc++-v3/testsuite/std/time/exceptions.cc index 06b8a53c353..1ecf502b62d 100644 --- a/libstdc++-v3/testsuite/std/time/exceptions.cc +++ b/libstdc++-v3/testsuite/std/time/exceptions.cc @@ -1,3 +1,4 @@ +// { dg-options "-lstdc++exp" { target c++23 } } // { dg-do run { target c++20 } } // { dg-require-effective-target tzdb } diff --git a/libstdc++-v3/testsuite/std/time/format.cc b/libstdc++-v3/testsuite/std/time/format.cc index d6e35832cb5..18d2171c117 100644 --- a/libstdc++-v3/testsuite/std/time/format.cc +++ b/libstdc++-v3/testsuite/std/time/format.cc @@ -1,3 +1,4 @@ +// { dg-options "-lstdc++exp" { target c++23 } } // { dg-do run { target c++20 } } // { dg-timeout-factor 2 } diff --git a/libstdc++-v3/testsuite/std/time/format_localized.cc b/libstdc++-v3/testsuite/std/time/format_localized.cc new file mode 100644 index 00000000000..f1bfe94fddc --- /dev/null +++ b/libstdc++-v3/testsuite/std/time/format_localized.cc @@ -0,0 +1,48 @@ +// { dg-options "-fexec-charset=UTF-8 -lstdc++exp" } +// { dg-do run { target c++23 } } +// { dg-require-namedlocale "ru_UA.koi8u" } +// { dg-require-namedlocale "es_ES.ISO8859-1" } +// { dg-require-namedlocale "fr_FR.ISO8859-1" } +// { dg-require-effective-target cxx11_abi } + +// P2419R2 +// Clarify handling of encodings in localized formatting of chrono types + +// Localized date-time strings such as "février" should be converted to UTF-8 +// if the locale uses a different encoding. + +#include <chrono> +#include <format> +#include <testsuite_hooks.h> + +void +test_ru() +{ + std::locale loc("ru_UA.koi8u"); + auto s = std::format(loc, "День недели: {:L}", std::chrono::Monday); + VERIFY( s == "День недели: Пн" ); +} + +void +test_es() +{ + std::locale loc(ISO_8859(1,es_ES)); + auto s = std::format(loc, "Día de la semana: {:L%A %a}", std::chrono::Wednesday); + VERIFY( s == "Día de la semana: miércoles mié" ); +} + +void +test_fr() +{ + std::locale loc(ISO_8859(1,fr_FR)); + auto s = std::format(loc, "Six mois après {0:L%b}, c'est {1:L%B}.", + std::chrono::February, std::chrono::August); + VERIFY( s == "Six mois après févr., c'est août." ); +} + +int main() +{ + test_ru(); + test_es(); + test_fr(); +} diff --git a/libstdc++-v3/testsuite/std/time/hh_mm_ss/io.cc b/libstdc++-v3/testsuite/std/time/hh_mm_ss/io.cc index d651dfbcdc8..551e4ed9a37 100644 --- a/libstdc++-v3/testsuite/std/time/hh_mm_ss/io.cc +++ b/libstdc++-v3/testsuite/std/time/hh_mm_ss/io.cc @@ -1,3 +1,4 @@ +// { dg-options "-lstdc++exp" { target c++23 } } // { dg-do run { target c++20 } } // { dg-timeout-factor 2 } diff --git a/libstdc++-v3/testsuite/std/time/month/io.cc b/libstdc++-v3/testsuite/std/time/month/io.cc index 99ec0737305..564eed54706 100644 --- a/libstdc++-v3/testsuite/std/time/month/io.cc +++ b/libstdc++-v3/testsuite/std/time/month/io.cc @@ -1,3 +1,4 @@ +// { dg-options "-lstdc++exp" { target c++23 } } // { dg-do run { target c++20 } } // { dg-require-namedlocale "fr_FR.ISO8859-15" } // { dg-timeout-factor 2 } diff --git a/libstdc++-v3/testsuite/std/time/month_day/io.cc b/libstdc++-v3/testsuite/std/time/month_day/io.cc index 30aa5881356..7f34f027ba9 100644 --- a/libstdc++-v3/testsuite/std/time/month_day/io.cc +++ b/libstdc++-v3/testsuite/std/time/month_day/io.cc @@ -1,3 +1,4 @@ +// { dg-options "-lstdc++exp" { target c++23 } } // { dg-do run { target c++20 } } // { dg-require-namedlocale "fr_FR.ISO8859-15" } // { dg-timeout-factor 2 } diff --git a/libstdc++-v3/testsuite/std/time/month_day_last/io.cc b/libstdc++-v3/testsuite/std/time/month_day_last/io.cc index c12cee848ed..94d54b9b124 100644 --- a/libstdc++-v3/testsuite/std/time/month_day_last/io.cc +++ b/libstdc++-v3/testsuite/std/time/month_day_last/io.cc @@ -1,3 +1,4 @@ +// { dg-options "-lstdc++exp" { target c++23 } } // { dg-do run { target c++20 } } // { dg-require-namedlocale "fr_FR.ISO8859-15" } // { dg-timeout-factor 2 } diff --git a/libstdc++-v3/testsuite/std/time/month_weekday/io.cc b/libstdc++-v3/testsuite/std/time/month_weekday/io.cc index 82cac648905..0fee88bb3a3 100644 --- a/libstdc++-v3/testsuite/std/time/month_weekday/io.cc +++ b/libstdc++-v3/testsuite/std/time/month_weekday/io.cc @@ -1,3 +1,4 @@ +// { dg-options "-lstdc++exp" { target c++23 } } // { dg-do run { target c++20 } } // { dg-require-namedlocale "fr_FR.ISO8859-15" } // { dg-timeout-factor 2 } diff --git a/libstdc++-v3/testsuite/std/time/month_weekday_last/io.cc b/libstdc++-v3/testsuite/std/time/month_weekday_last/io.cc index 47968d0e06d..e6a6751bb4a 100644 --- a/libstdc++-v3/testsuite/std/time/month_weekday_last/io.cc +++ b/libstdc++-v3/testsuite/std/time/month_weekday_last/io.cc @@ -1,3 +1,4 @@ +// { dg-options "-lstdc++exp" { target c++23 } } // { dg-do run { target c++20 } } // { dg-require-namedlocale "fr_FR.ISO8859-15" } // { dg-timeout-factor 2 } diff --git a/libstdc++-v3/testsuite/std/time/time_zone/get_info_local.cc b/libstdc++-v3/testsuite/std/time/time_zone/get_info_local.cc index d2972aca7df..af830374700 100644 --- a/libstdc++-v3/testsuite/std/time/time_zone/get_info_local.cc +++ b/libstdc++-v3/testsuite/std/time/time_zone/get_info_local.cc @@ -1,3 +1,4 @@ +// { dg-options "-lstdc++exp" { target c++23 } } // { dg-do run { target c++20 } } // { dg-require-effective-target tzdb } diff --git a/libstdc++-v3/testsuite/std/time/weekday/io.cc b/libstdc++-v3/testsuite/std/time/weekday/io.cc index a56cdaef88c..f102c9250e9 100644 --- a/libstdc++-v3/testsuite/std/time/weekday/io.cc +++ b/libstdc++-v3/testsuite/std/time/weekday/io.cc @@ -1,3 +1,4 @@ +// { dg-options "-lstdc++exp" { target c++23 } } // { dg-do run { target c++20 } } // { dg-require-namedlocale "fr_FR.ISO8859-15" } // { dg-timeout-factor 2 } diff --git a/libstdc++-v3/testsuite/std/time/weekday_indexed/io.cc b/libstdc++-v3/testsuite/std/time/weekday_indexed/io.cc index 29255ad5c06..42fc9b914a1 100644 --- a/libstdc++-v3/testsuite/std/time/weekday_indexed/io.cc +++ b/libstdc++-v3/testsuite/std/time/weekday_indexed/io.cc @@ -1,3 +1,4 @@ +// { dg-options "-lstdc++exp" { target c++23 } } // { dg-do run { target c++20 } } // { dg-require-namedlocale "fr_FR.ISO8859-15" } // { dg-timeout-factor 2 } diff --git a/libstdc++-v3/testsuite/std/time/weekday_last/io.cc b/libstdc++-v3/testsuite/std/time/weekday_last/io.cc index 6f76922195d..d069d857ff0 100644 --- a/libstdc++-v3/testsuite/std/time/weekday_last/io.cc +++ b/libstdc++-v3/testsuite/std/time/weekday_last/io.cc @@ -1,3 +1,4 @@ +// { dg-options "-lstdc++exp" { target c++23 } } // { dg-do run { target c++20 } } // { dg-require-namedlocale "fr_FR.ISO8859-15" } // { dg-timeout-factor 2 } diff --git a/libstdc++-v3/testsuite/std/time/year/io.cc b/libstdc++-v3/testsuite/std/time/year/io.cc index bcaa57faeb7..1bdc95ddbd1 100644 --- a/libstdc++-v3/testsuite/std/time/year/io.cc +++ b/libstdc++-v3/testsuite/std/time/year/io.cc @@ -1,3 +1,4 @@ +// { dg-options "-lstdc++exp" { target c++23 } } // { dg-do run { target c++20 } } // { dg-require-namedlocale "fr_FR.ISO8859-15" } // { dg-timeout-factor 2 } diff --git a/libstdc++-v3/testsuite/std/time/year_month/io.cc b/libstdc++-v3/testsuite/std/time/year_month/io.cc index 7bb3442e299..e7963d31ae6 100644 --- a/libstdc++-v3/testsuite/std/time/year_month/io.cc +++ b/libstdc++-v3/testsuite/std/time/year_month/io.cc @@ -1,3 +1,4 @@ +// { dg-options "-lstdc++exp" { target c++23 } } // { dg-do run { target c++20 } } // { dg-require-namedlocale "fr_FR.ISO8859-15" } // { dg-timeout-factor 2 } diff --git a/libstdc++-v3/testsuite/std/time/year_month_day/io.cc b/libstdc++-v3/testsuite/std/time/year_month_day/io.cc index cb82ef3b612..e7dc60b9dea 100644 --- a/libstdc++-v3/testsuite/std/time/year_month_day/io.cc +++ b/libstdc++-v3/testsuite/std/time/year_month_day/io.cc @@ -1,3 +1,4 @@ +// { dg-options "-lstdc++exp" { target c++23 } } // { dg-do run { target c++20 } } // { dg-require-namedlocale "fr_FR.ISO8859-15" } // { dg-timeout-factor 2 } diff --git a/libstdc++-v3/testsuite/std/time/year_month_day_last/io.cc b/libstdc++-v3/testsuite/std/time/year_month_day_last/io.cc index 3241536a2e6..a57e56a1805 100644 --- a/libstdc++-v3/testsuite/std/time/year_month_day_last/io.cc +++ b/libstdc++-v3/testsuite/std/time/year_month_day_last/io.cc @@ -1,3 +1,4 @@ +// { dg-options "-lstdc++exp" { target c++23 } } // { dg-do run { target c++20 } } // { dg-require-namedlocale "fr_FR.ISO8859-15" } // { dg-timeout-factor 2 } diff --git a/libstdc++-v3/testsuite/std/time/year_month_weekday/io.cc b/libstdc++-v3/testsuite/std/time/year_month_weekday/io.cc index 65baf1d37ae..e9a5bee7666 100644 --- a/libstdc++-v3/testsuite/std/time/year_month_weekday/io.cc +++ b/libstdc++-v3/testsuite/std/time/year_month_weekday/io.cc @@ -1,3 +1,4 @@ +// { dg-options "-lstdc++exp" { target c++23 } } // { dg-do run { target c++20 } } // { dg-require-namedlocale "fr_FR.ISO8859-15" } // { dg-timeout-factor 2 } diff --git a/libstdc++-v3/testsuite/std/time/year_month_weekday_last/io.cc b/libstdc++-v3/testsuite/std/time/year_month_weekday_last/io.cc index 17f2244420d..fc2328ac001 100644 --- a/libstdc++-v3/testsuite/std/time/year_month_weekday_last/io.cc +++ b/libstdc++-v3/testsuite/std/time/year_month_weekday_last/io.cc @@ -1,3 +1,4 @@ +// { dg-options "-lstdc++exp" { target c++23 } } // { dg-do run { target c++20 } } // { dg-require-namedlocale "fr_FR.ISO8859-15" } // { dg-timeout-factor 2 } diff --git a/libstdc++-v3/testsuite/std/time/zoned_time/1.cc b/libstdc++-v3/testsuite/std/time/zoned_time/1.cc index 1623aca1c7a..e773f50ae73 100644 --- a/libstdc++-v3/testsuite/std/time/zoned_time/1.cc +++ b/libstdc++-v3/testsuite/std/time/zoned_time/1.cc @@ -1,3 +1,4 @@ +// { dg-options "-lstdc++exp" { target c++23 } } // { dg-do run { target c++20 } } // { dg-require-effective-target tzdb } // { dg-require-effective-target cxx11_abi } diff --git a/libstdc++-v3/testsuite/std/time/zoned_time/io.cc b/libstdc++-v3/testsuite/std/time/zoned_time/io.cc index 376b2734f19..c19e2a3c882 100644 --- a/libstdc++-v3/testsuite/std/time/zoned_time/io.cc +++ b/libstdc++-v3/testsuite/std/time/zoned_time/io.cc @@ -1,3 +1,4 @@ +// { dg-options "-lstdc++exp" { target c++23 } } // { dg-do run { target c++20 } } // { dg-require-effective-target cxx11_abi } // { dg-timeout-factor 2 } -- 2.43.0 ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH 2/2] libstdc++: Handle encodings in localized chrono formatting [PR109162] 2024-02-01 15:36 ` [PATCH 2/2] libstdc++: Handle encodings in localized chrono formatting [PR109162] Jonathan Wakely @ 2024-02-01 16:43 ` Arsen Arsenović 0 siblings, 0 replies; 11+ messages in thread From: Arsen Arsenović @ 2024-02-01 16:43 UTC (permalink / raw) To: Jonathan Wakely; +Cc: gcc-patches, libstdc++ Hi, Jonathan Wakely <jwakely@redhat.com> writes: > I am undecided about pushing this PATCH 2/2 to trunk. PATCH 1/2 needs to > be done now due to the ABI impact on cris-elf. This one could wait for > stage 1. (HP, this one probably isn't of interest to you, but I don't > know how to tell git-send-email to only CC you on the first patch). (haven't read the patchset, just weighing in with Git tips) You can, instead of specifying git send-email --cc=..., just place Cc: <whoever> in the patch file to Cc just that patch. G-S-E will aggregate them together before shipping out. Hope that helps, have a lovely day. > I'd like to do this change now so people can experiment with the new > feature by linking with -lstdc++exp, but the problem is that linking to > that library becomes a hard requirement for any uses of std::chrono > types with std::format in C++23. So code that compiles and links in > C++20 mode will fail to link in C++23 mode unless you use the extra > libstdc++exp.a library. The code might have worked fine in C++20 mode, > or it might have just linked OK but produce mojibake at runtime, it > depends on the locale. > > Declaring the new functions as weak and then checking them for non-null > at runtime doesn't really help, because the non-weak symbols in the > static lib won't be used without -Wl,--whole-archive -lstdc++exp because > it's OK for weak symbols to be unresolved. We could play games with > linker scripts so that libstdc++exp.a is a linker script which ensures > the non-weak defs get used, but I'm not convinced it's worth the hassle > for a feature that will probably get added to the main libstdc++.so in > stage 1 anyway. > > What do others think? Should I just push it and require -lstdc++exp for > chrono formatting in C++23 mode? > > -- >8 -- > > This implements the C++23 paper P2419R2 (Clarify handling of encodings > in localized formatting of chrono types). The requirement is that when > the literal encoding is "a Unicode encoding form" and the formatting > locale uses a different encoding, any locale-specific strings such as > "août" for std::chrono::August should be converted to the literal > encoding. > > Using the recently-added std::locale::encoding() function we can check > the locale's encoding and then use iconv if a conversion is needed. This > requires a new non-inline function in the library. As this is currently > experimental, the new symbol is added to libstdc++exp.a for now. This > means that formatting std::chrono types requires -lstdc++exp in C++23 > and later, at least until the new function is considered stable enough > to add to the main library. When that happens, we might want to consider > enabling these encoding conversions for C++20 too, treating it as a DR > that resolves LWG 3656. > > Because nl_langinfo_l and iconv_open both allocate memory, a naive > implementation would perform multiple allocations and deallocations for > every snippet of locale-specific text that needs to be converted to > UTF-8. To avoid that, a new internal locale::facet is defined to store > the text_encoding and an iconv_t descriptor, which are then cached in > the formatting locale. This requires access to the internals of a > std::locale object in src/c++26/text_encoding.cc, so that file now needs > to be compiled with -fno-access-control. When the new symbols move into > libstdc++.so we should be able to avoid that by adding new member > functions to std::locale. > > With this change we can increase the value of the __cpp_lib_format macro > for C++23. The value should be 202207 for P2419R2, but we already > implement P2510R3 (Formatting pointers) so can use the value 202304. > > libstdc++-v3/ChangeLog: > > PR libstdc++/109162 > * include/bits/chrono_io.h (_ChronoSpec::_M_locale_specific): > Add new accessor functions to use a reserved bit in _Spec. > (__formatter_chrono::_M_parse): Use _M_locale_specific(true) > when chrono-specs contains locale-dependent conversion > specifiers. > (__formatter_chrono::_M_format): Open iconv descriptor if > conversion to UTF-8 will be needed. > (__formatter_chrono::_M_write): New function to write a > localized string with possible character conversion. > (__formatter_chrono::_M_a_A, __formatter_chrono::_M_b_B) > (__formatter_chrono::_M_p, __formatter_chrono::_M_r) > (__formatter_chrono::_M_x, __formatter_chrono::_M_X) > (__formatter_chrono::_M_locale_fmt): Use _M_write. > * include/bits/version.def: Add C++23 values for format. > * include/bits/version.h: Regenerate. > * include/std/format (_GLIBCXX_P2518R3): Check feature test > macro instead of __cplusplus. > (basic_format_context): Declare __formatter_chrono as friend. > * src/c++26/Makefile.am: Use -fno-access-control for > text_encoding.o. > * src/c++26/Makefile.in: Regenerate. > * src/c++26/text_encoding.cc (__encoding): New locale facet. > (locale::encoding): Check for __encoding facet first. Add fast > path for "C" locale. > (__with_encoding_conversion): New function to add __encoding > facet to a locale. > (__locale_encoding_to_utf8): New function to convert a string > from a locale's encoding to UTF-8. > * testsuite/20_util/duration/io.cc: Add -lstdc++exp for C++23. > * testsuite/std/time/clock/file/io.cc: Likewise. > * testsuite/std/time/clock/gps/io.cc: Likewise. > * testsuite/std/time/clock/local/io.cc: Likewise. > * testsuite/std/time/clock/system/io.cc: Likewise. > * testsuite/std/time/clock/tai/io.cc: Likewise. > * testsuite/std/time/clock/utc/io.cc: Likewise. > * testsuite/std/time/day/io.cc: Likewise. > * testsuite/std/time/exceptions.cc: Likewise. > * testsuite/std/time/format.cc: Likewise. > * testsuite/std/time/hh_mm_ss/io.cc: Likewise. > * testsuite/std/time/month/io.cc: Likewise. > * testsuite/std/time/month_day/io.cc: Likewise. > * testsuite/std/time/month_day_last/io.cc: Likewise. > * testsuite/std/time/month_weekday/io.cc: Likewise. > * testsuite/std/time/month_weekday_last/io.cc: Likewise. > * testsuite/std/time/time_zone/get_info_local.cc: Likewise. > * testsuite/std/time/weekday/io.cc: Likewise. > * testsuite/std/time/weekday_indexed/io.cc: Likewise. > * testsuite/std/time/weekday_last/io.cc: Likewise. > * testsuite/std/time/year/io.cc: Likewise. > * testsuite/std/time/year_month/io.cc: Likewise. > * testsuite/std/time/year_month_day/io.cc: Likewise. > * testsuite/std/time/year_month_day_last/io.cc: Likewise. > * testsuite/std/time/year_month_weekday/io.cc: Likewise. > * testsuite/std/time/year_month_weekday_last/io.cc: Likewise. > * testsuite/std/time/zoned_time/1.cc: Likewise. > * testsuite/std/time/zoned_time/io.cc: Likewise. > * testsuite/std/time/format_localized.cc: New test. > --- > libstdc++-v3/include/bits/chrono_io.h | 103 ++++++++++-- > libstdc++-v3/include/bits/version.def | 34 +++- > libstdc++-v3/include/bits/version.h | 7 +- > libstdc++-v3/include/std/format | 16 +- > libstdc++-v3/src/c++26/Makefile.am | 6 + > libstdc++-v3/src/c++26/Makefile.in | 6 + > libstdc++-v3/src/c++26/text_encoding.cc | 151 +++++++++++++++++- > libstdc++-v3/testsuite/20_util/duration/io.cc | 1 + > .../testsuite/std/time/clock/file/io.cc | 1 + > .../testsuite/std/time/clock/gps/io.cc | 1 + > .../testsuite/std/time/clock/local/io.cc | 1 + > .../testsuite/std/time/clock/system/io.cc | 1 + > .../testsuite/std/time/clock/tai/io.cc | 1 + > .../testsuite/std/time/clock/utc/io.cc | 1 + > libstdc++-v3/testsuite/std/time/day/io.cc | 1 + > libstdc++-v3/testsuite/std/time/exceptions.cc | 1 + > libstdc++-v3/testsuite/std/time/format.cc | 1 + > .../testsuite/std/time/format_localized.cc | 48 ++++++ > .../testsuite/std/time/hh_mm_ss/io.cc | 1 + > libstdc++-v3/testsuite/std/time/month/io.cc | 1 + > .../testsuite/std/time/month_day/io.cc | 1 + > .../testsuite/std/time/month_day_last/io.cc | 1 + > .../testsuite/std/time/month_weekday/io.cc | 1 + > .../std/time/month_weekday_last/io.cc | 1 + > .../std/time/time_zone/get_info_local.cc | 1 + > libstdc++-v3/testsuite/std/time/weekday/io.cc | 1 + > .../testsuite/std/time/weekday_indexed/io.cc | 1 + > .../testsuite/std/time/weekday_last/io.cc | 1 + > libstdc++-v3/testsuite/std/time/year/io.cc | 1 + > .../testsuite/std/time/year_month/io.cc | 1 + > .../testsuite/std/time/year_month_day/io.cc | 1 + > .../std/time/year_month_day_last/io.cc | 1 + > .../std/time/year_month_weekday/io.cc | 1 + > .../std/time/year_month_weekday_last/io.cc | 1 + > .../testsuite/std/time/zoned_time/1.cc | 1 + > .../testsuite/std/time/zoned_time/io.cc | 1 + > 36 files changed, 375 insertions(+), 24 deletions(-) > create mode 100644 libstdc++-v3/testsuite/std/time/format_localized.cc > > diff --git a/libstdc++-v3/include/bits/chrono_io.h b/libstdc++-v3/include/bits/chrono_io.h > index 82f2d39ec44..9f5b9e13a82 100644 > --- a/libstdc++-v3/include/bits/chrono_io.h > +++ b/libstdc++-v3/include/bits/chrono_io.h > @@ -210,6 +210,20 @@ namespace __format > struct _ChronoSpec : _Spec<_CharT> > { > basic_string_view<_CharT> _M_chrono_specs; > + > + // Use one of the reserved bits in __format::_Spec<C>. > + // This indicates that a locale-dependent conversion specifier such as > + // %a is used in the chrono-specs. This is not the same as the > + // _Spec<C>::_M_localized member which indicates that "L" was present > + // in the format-spec, e.g. "{:L%a}" is localized and locale-specific, > + // but "{:L}" is only localized and "{:%a}" is only locale-specific. > + constexpr bool > + _M_locale_specific() const noexcept > + { return this->_M_reserved & 1; } > + > + constexpr void > + _M_locale_specific(bool __b) noexcept > + { this->_M_reserved |= __b; } > }; > > // Represents the information provided by a chrono type. > @@ -304,11 +318,12 @@ namespace __format > const auto __chrono_specs = __first++; // Skip leading '%' > if (*__chrono_specs != '%') > __throw_format_error("chrono format error: no '%' at start of " > - "chrono-specs"); > + "chrono-specs"); > > _CharT __mod{}; > bool __conv = true; > int __needed = 0; > + bool __locale_specific = false; > > while (__first != __last) > { > @@ -321,15 +336,18 @@ namespace __format > case 'a': > case 'A': > __needed = _Weekday; > + __locale_specific = true; > break; > case 'b': > case 'h': > case 'B': > __needed = _Month; > + __locale_specific = true; > break; > case 'c': > __needed = _DateTime; > __allowed_mods = _Mod_E; > + __locale_specific = true; > break; > case 'C': > __needed = _Year; > @@ -367,6 +385,8 @@ namespace __format > break; > case 'p': > case 'r': > + __locale_specific = true; > + [[fallthrough]]; > case 'R': > case 'T': > __needed = _TimeOfDay; > @@ -392,10 +412,12 @@ namespace __format > break; > case 'x': > __needed = _Date; > + __locale_specific = true; > __allowed_mods = _Mod_E; > break; > case 'X': > __needed = _TimeOfDay; > + __locale_specific = true; > __allowed_mods = _Mod_E; > break; > case 'y': > @@ -435,6 +457,8 @@ namespace __format > || (__mod == 'O' && !(__allowed_mods & _Mod_O))) > __throw_format_error("chrono format error: invalid " > " modifier in chrono-specs"); > + if (__mod && __c != 'z') > + __locale_specific = true; > __mod = _CharT(); > > if ((__parts & __needed) != __needed) > @@ -466,6 +490,7 @@ namespace __format > _M_spec = __spec; > _M_spec._M_chrono_specs > = __string_view(__chrono_specs, __first - __chrono_specs); > + _M_spec._M_locale_specific(__locale_specific); > > return __first; > } > @@ -485,6 +510,26 @@ namespace __format > if (__first == __last) > return _M_format_to_ostream(__t, __fc, __is_neg); > > +#if __glibcxx_format >= 202207L // C++ >= 23 > +#if _GLIBCXX_USE_CXX11_ABI > + // _GLIBCXX_RESOLVE_LIB_DEFECTS > + // 3565. Handling of encodings in localized formatting > + // of chrono types is underspecified > + if constexpr (is_same_v<_CharT, char>) > + if constexpr (__unicode::__literal_encoding_is_utf8()) > + if (_M_spec._M_localized && _M_spec._M_locale_specific()) > + { > + extern locale __with_encoding_conversion(const locale&); > + > + // Allocate and cache the necessary state to convert strings > + // in the locale's encoding to UTF-8. > + locale __loc = __fc.locale(); > + if (__loc != locale::classic()) > + __fc._M_loc = __with_encoding_conversion(__loc); > + } > +#endif > +#endif > + > _Sink_iter<_CharT> __out; > __format::_Str_sink<_CharT> __sink; > bool __write_direct = false; > @@ -741,6 +786,37 @@ namespace __format > static constexpr _CharT _S_space = _S_chars[14]; > static constexpr const _CharT* _S_empty_spec = _S_chars + 15; > > + template<typename _OutIter> > + _OutIter > + _M_write(_OutIter __out, const locale& __loc, __string_view __s) const > + { > + // FIXME: It would be nice to treat this as a DR for C++20 > + // but it currently requires new symbols in libstdc++exp.a > + // which we don't want to require for C++20. Could do it when > + // those symbols move into the non-experimental library. > +#if __glibcxx_format >= 202207L // C++ >= 23 > +#if _GLIBCXX_USE_CXX11_ABI > + // _GLIBCXX_RESOLVE_LIB_DEFECTS > + // 3565. Handling of encodings in localized formatting > + // of chrono types is underspecified > + using _StringType = string; > + string __buf; > + if constexpr (is_same_v<_CharT, char>) > + if constexpr (__unicode::__literal_encoding_is_utf8()) > + if (_M_spec._M_localized && _M_spec._M_locale_specific() > + && __loc != locale::classic()) > + { > + extern string_view > + __locale_encoding_to_utf8(const std::locale&, string_view, > + _StringType&); > + > + __s = __locale_encoding_to_utf8(__loc, __s, __buf); > + } > +#endif > +#endif > + return __format::__write(std::move(__out), __s); > + } > + > template<typename _Tp, typename _FormatContext> > typename _FormatContext::iterator > _M_a_A(const _Tp& __t, typename _FormatContext::iterator __out, > @@ -760,7 +836,7 @@ namespace __format > else > __tp._M_days_abbreviated(__days); > __string_view __str(__days[__wd.c_encoding()]); > - return __format::__write(std::move(__out), __str); > + return _M_write(std::move(__out), __loc, __str); > } > > template<typename _Tp, typename _FormatContext> > @@ -781,7 +857,7 @@ namespace __format > else > __tp._M_months_abbreviated(__months); > __string_view __str(__months[(unsigned)__m - 1]); > - return __format::__write(std::move(__out), __str); > + return _M_write(std::move(__out), __loc, __str); > } > > template<typename _Tp, typename _FormatContext> > @@ -1058,8 +1134,8 @@ namespace __format > const auto& __tp = use_facet<__timepunct<_CharT>>(__loc); > const _CharT* __ampm[2]; > __tp._M_am_pm(__ampm); > - return std::format_to(std::move(__out), _S_empty_spec, > - __ampm[__hms.hours().count() >= 12]); > + return _M_write(std::move(__out), __loc, > + __ampm[__hms.hours().count() >= 12]); > } > > template<typename _Tp, typename _FormatContext> > @@ -1094,8 +1170,9 @@ namespace __format > basic_string<_CharT> __fmt(_S_empty_spec); > __fmt.insert(1u, 1u, _S_colon); > __fmt.insert(2u, __ampm_fmt); > - return std::vformat_to(std::move(__out), __fmt, > - std::make_format_args<_FormatContext>(__t)); > + using _FmtStr = _Runtime_format_string<_CharT>; > + return _M_write(std::move(__out), __loc, > + std::format(__loc, _FmtStr(__fmt), __t)); > } > > template<typename _Tp, typename _FormatContext> > @@ -1278,8 +1355,9 @@ namespace __format > basic_string<_CharT> __fmt(_S_empty_spec); > __fmt.insert(1u, 1u, _S_colon); > __fmt.insert(2u, __rep); > - return std::vformat_to(std::move(__out), __fmt, > - std::make_format_args<_FormatContext>(__t)); > + using _FmtStr = _Runtime_format_string<_CharT>; > + return _M_write(std::move(__out), __loc, > + std::format(__loc, _FmtStr(__fmt), __t)); > } > > template<typename _Tp, typename _FormatContext> > @@ -1301,8 +1379,9 @@ namespace __format > basic_string<_CharT> __fmt(_S_empty_spec); > __fmt.insert(1u, 1u, _S_colon); > __fmt.insert(2u, __rep); > - return std::vformat_to(std::move(__out), __fmt, > - std::make_format_args<_FormatContext>(__t)); > + using _FmtStr = _Runtime_format_string<_CharT>; > + return _M_write(std::move(__out), __loc, > + std::format(__loc, _FmtStr(__fmt), __t)); > } > > template<typename _Tp, typename _FormatContext> > @@ -1579,7 +1658,7 @@ namespace __format > const auto& __tp = use_facet<time_put<_CharT>>(__loc); > __tp.put(__os, __os, _S_space, &__tm, __fmt, __mod); > if (__os) > - __out = __format::__write(std::move(__out), __os.view()); > + __out = _M_write(std::move(__out), __loc, __os.view()); > return __out; > } > }; > diff --git a/libstdc++-v3/include/bits/version.def b/libstdc++-v3/include/bits/version.def > index 8fb8a2877ee..29ca973fe43 100644 > --- a/libstdc++-v3/include/bits/version.def > +++ b/libstdc++-v3/include/bits/version.def > @@ -1160,14 +1160,25 @@ ftms = { > }; > > ftms = { > + name = format; > + // 202311 P2918R2 Runtime format strings II > + // values = { > + // v = 202311; > + // cxxmin = 26; > + // hosted = yes; > + // }; > + // 202207 Encodings in localized formatting of chrono, basic-format-string. > + // 202304 P2510R3 Formatting pointers > + // 202305 P2757R3 Type checking format args > + // 202306 P2637R3 Member visit > + values = { > + v = 202304; > + cxxmin = 23; > + hosted = yes; > + }; > // 201907 Text Formatting, Integration of chrono, printf corner cases. > // 202106 std::format improvements. > // 202110 Fixing locale handling in chrono formatters, generator-like types. > - // 202207 Encodings in localized formatting of chrono, basic-format-string. > - // 202207 P2286R8 Formatting Ranges > - // 202207 P2585R1 Improving default container formatting > - // TODO: #define __cpp_lib_format_ranges 202207L > - name = format; > values = { > v = 202110; > cxxmin = 20; > @@ -1364,6 +1375,19 @@ ftms = { > }; > }; > > +// ftms = { > + // name = format_ranges; > + // 202207 P2286R8 Formatting Ranges > + // 202207 P2585R1 Improving default container formatting > + // LWG3750 Too many papers bump __cpp_lib_format > + // TODO: #define __cpp_lib_format_ranges 202207L > + // values = { > + // v = 202207; > + // cxxmin = 23; > + // hosted = yes; > + // }; > +// }; > + > ftms = { > name = freestanding_algorithm; > values = { > diff --git a/libstdc++-v3/include/bits/version.h b/libstdc++-v3/include/bits/version.h > index fa4e89cf845..714bf064950 100644 > --- a/libstdc++-v3/include/bits/version.h > +++ b/libstdc++-v3/include/bits/version.h > @@ -1304,7 +1304,12 @@ > #undef __glibcxx_want_barrier > > #if !defined(__cpp_lib_format) > -# if (__cplusplus >= 202002L) && _GLIBCXX_HOSTED > +# if (__cplusplus >= 202100L) && _GLIBCXX_HOSTED > +# define __glibcxx_format 202304L > +# if defined(__glibcxx_want_all) || defined(__glibcxx_want_format) > +# define __cpp_lib_format 202304L > +# endif > +# elif (__cplusplus >= 202002L) && _GLIBCXX_HOSTED > # define __glibcxx_format 202110L > # if defined(__glibcxx_want_all) || defined(__glibcxx_want_format) > # define __cpp_lib_format 202110L > diff --git a/libstdc++-v3/include/std/format b/libstdc++-v3/include/std/format > index 6c958bc11a5..1f72cb9b9ff 100644 > --- a/libstdc++-v3/include/std/format > +++ b/libstdc++-v3/include/std/format > @@ -2341,10 +2341,10 @@ namespace __format > > // _GLIBCXX_RESOLVE_LIB_DEFECTS > // P2510R3 Formatting pointers > -#if __cplusplus > 202302L || ! defined __STRICT_ANSI__ > -#define _GLIBCXX_P2518R3 1 > +#if __glibcxx_format >= 202304L || ! defined __STRICT_ANSI__ > +# define _GLIBCXX_P2518R3 1 > #else > -#define _GLIBCXX_P2518R3 0 > +# define _GLIBCXX_P2518R3 0 > #endif > > #if _GLIBCXX_P2518R3 > @@ -3788,6 +3788,9 @@ namespace __format > __do_vformat_to(_Out, basic_string_view<_CharT>, > const basic_format_args<_Context>&, > const locale* = nullptr); > + > + template<typename _CharT> struct __formatter_chrono; > + > } // namespace __format > /// @endcond > > @@ -3798,6 +3801,11 @@ namespace __format > * this class template explicitly. For typical uses of `std::format` the > * library will use the specializations `std::format_context` (for `char`) > * and `std::wformat_context` (for `wchar_t`). > + * > + * You are not allowed to define partial or explicit specializations of > + * this class template. > + * > + * @since C++20 > */ > template<typename _Out, typename _CharT> > class basic_format_context > @@ -3824,6 +3832,8 @@ namespace __format > const basic_format_args<_Context2>&, > const locale*); > > + friend __format::__formatter_chrono<_CharT>; > + > public: > basic_format_context() = default; > ~basic_format_context() = default; > diff --git a/libstdc++-v3/src/c++26/Makefile.am b/libstdc++-v3/src/c++26/Makefile.am > index 000ced1f501..1f9dff7e737 100644 > --- a/libstdc++-v3/src/c++26/Makefile.am > +++ b/libstdc++-v3/src/c++26/Makefile.am > @@ -46,6 +46,12 @@ else > libc__26convenience_la_SOURCES = > endif > > +# This needs access to the internals of std::locale. > +text_encoding.lo: text_encoding.cc > + $(LTCXXCOMPILE) -fno-access-control -c $< > +text_encoding.o: text_encoding.cc > + $(CXXCOMPILE) -fno-access-control -c $< > + > # AM_CXXFLAGS needs to be in each subdirectory so that it can be > # modified in a per-library or per-sub-library way. Need to manually > # set this option because CONFIG_CXXFLAGS has to be after > diff --git a/libstdc++-v3/src/c++26/Makefile.in b/libstdc++-v3/src/c++26/Makefile.in > index 77e73b2b265..96947ccdd0c 100644 > --- a/libstdc++-v3/src/c++26/Makefile.in > +++ b/libstdc++-v3/src/c++26/Makefile.in > @@ -742,6 +742,12 @@ uninstall-am: > > vpath % $(top_srcdir)/src/c++26 > > +# This needs access to the internals of std::locale. > +text_encoding.lo: text_encoding.cc > + $(LTCXXCOMPILE) -fno-access-control -c $< > +text_encoding.o: text_encoding.cc > + $(CXXCOMPILE) -fno-access-control -c $< > + > # Tell versions [3.59,3.63) of GNU make to not export all variables. > # Otherwise a system limit (for SysV at least) may be exceeded. > .NOEXPORT: > diff --git a/libstdc++-v3/src/c++26/text_encoding.cc b/libstdc++-v3/src/c++26/text_encoding.cc > index b9a50ef1a00..fc5141e02c7 100644 > --- a/libstdc++-v3/src/c++26/text_encoding.cc > +++ b/libstdc++-v3/src/c++26/text_encoding.cc > @@ -26,17 +26,28 @@ > #include <locale> > > #ifdef _GLIBCXX_USE_NL_LANGINFO_L > +#include <memory> // make_unique > +#include <string.h> // strlen, strcpy > #include <locale.h> > #if __has_include(<xlocale.h>) > # include <xlocale.h> > #endif > #include <langinfo.h> > > +#ifdef _GLIBCXX_HAVE_ICONV > +# include <format> > +# include <chrono> > +# include <iconv.h> > +# include <errno.h> > +#endif > + > #if __CHAR_BIT__ == 8 > namespace std > { > _GLIBCXX_BEGIN_NAMESPACE_VERSION > - > +namespace > +{ > +// Attempt to determine the text_encoding used by the named locale. > text_encoding > __locale_encoding(const char* name) > { > @@ -54,6 +65,48 @@ __locale_encoding(const char* name) > return enc; > } > > +// A non-standard locale::facet that caches the locale's std::text_encoding > +// and an iconv descriptor for converting from that encoding to UTF-8. > +struct __encoding : locale::facet > +{ > + static locale::id id; > + > + explicit > + __encoding(const text_encoding& enc, size_t refs = 0) > + : facet(refs), _M_enc(enc) > + { > +#if defined _GLIBCXX_HAVE_ICONV > + if (enc != text_encoding::UTF8 && enc != text_encoding::ASCII) > + _M_cd = ::iconv_open("UTF-8", enc.name()); > +#endif > + } > + > + ~__encoding() > + { > +#if defined _GLIBCXX_HAVE_ICONV > + if (_M_has_desc()) > + ::iconv_close(_M_cd); > +#endif > + } > + > + bool _M_has_desc() const > + { > +#if defined _GLIBCXX_HAVE_ICONV > + return _M_cd != (::iconv_t)-1; > +#else > + return false; > +#endif > + } > + > + text_encoding _M_enc; > +#if defined _GLIBCXX_HAVE_ICONV > + ::iconv_t _M_cd = (::iconv_t)-1; > +#endif > +}; > + > +locale::id __encoding::id; > + > +} // namespace > _GLIBCXX_END_NAMESPACE_VERSION > } // namespace std > > @@ -87,8 +140,100 @@ std::text_encoding::_M_is_environment() const > std::text_encoding > std::locale::encoding() const > { > - return std::__locale_encoding(name().c_str()); > + if (auto enc_facet = std::__try_use_facet<__encoding>(*this)) > + return enc_facet->_M_enc; > + string name = this->name(); > + if (name == "*") > + return {}; > + if (name == "C") > + return text_encoding(text_encoding::ASCII); > + return __locale_encoding(name.c_str()); > } > #endif // CHAR_BIT == 8 > - > #endif // _GLIBCXX_USE_NL_LANGINFO_L > + > +namespace std > +{ > +_GLIBCXX_BEGIN_NAMESPACE_VERSION > +namespace __format > +{ > +// Helpers for P2419R2 > +// (Clarify handling of encodings in localized formatting of chrono types) > +// Convert a string from the locale's charset to UTF-8. > + > +std::locale > +__with_encoding_conversion(const std::locale& loc) > +{ > +#if defined _GLIBCXX_USE_NL_LANGINFO_L && __CHAR_BIT__ == 8 > + const text_encoding locenc = loc.encoding(); > + > + if (locenc == text_encoding::UTF8 || locenc == text_encoding::ASCII > + || locenc == text_encoding::unknown) > + return loc; > + > + auto impl = std::make_unique<locale::_Impl>(*loc._M_impl, 1); > + auto facetp = std::make_unique<__encoding>(locenc); > + locale loc2(loc, facetp.get()); // FIXME: PR libstdc++/113704 > + facetp.release(); > + // FIXME: Ideally we wouldn't need to reallocate this string again, > + // just don't delete[] it in the locale(locale, Facet*) constructor. > + if (const char* name = loc._M_impl->_M_names[0]) > + { > + loc2._M_impl->_M_names[0] = new char[strlen(name) + 1]; > + strcpy(loc2._M_impl->_M_names[0], name); > + } > + return loc2; > +#else > + return loc; > +#endif > +} > + > +string_view > +__locale_encoding_to_utf8(const std::locale& loc, string_view str, > + string& outbuf) > +{ > +#ifdef _GLIBCXX_HAVE_ICONV > + // Don't need to use __try_use_facet with its dynamic_cast<__encoding*>, > + // since we know there are no types derived from __encoding. If the array > + // element is non-null, we have the facet. > + auto id = __encoding::id._M_id(); > + auto enc_facet = static_cast<const __encoding*>(loc._M_impl->_M_facets[id]); > + if (!enc_facet || !enc_facet->_M_has_desc()) > + return str; > + > + size_t inbytesleft = str.size(); > + size_t written = 0; > + bool done = false; > + > + auto overwrite = [&](char* p, size_t n) { > + auto inbytes = const_cast<char*>(str.data()) + str.size() - inbytesleft; > + char* outbytes = p + written; > + size_t outbytesleft = n - written; > + size_t res = ::iconv(enc_facet->_M_cd, &inbytes, &inbytesleft, > + &outbytes, &outbytesleft); > + if (res == (size_t)-1) > + { > + if (errno != E2BIG) > + { > + done = true; > + return 0zu; > + } > + } > + else > + done = true; > + written = outbytes - p; > + return written; > + }; > + do > + outbuf.resize_and_overwrite(outbuf.capacity() + (inbytesleft * 3 / 2), > + overwrite); > + while (!done); > + if (outbuf.size()) > + str = outbuf; > +#endif // HAVE_ICONV > + > + return str; > +} > +} // namespace __format > +_GLIBCXX_END_NAMESPACE_VERSION > +} // namespace std > diff --git a/libstdc++-v3/testsuite/20_util/duration/io.cc b/libstdc++-v3/testsuite/20_util/duration/io.cc > index e141baf42dc..30469fb861d 100644 > --- a/libstdc++-v3/testsuite/20_util/duration/io.cc > +++ b/libstdc++-v3/testsuite/20_util/duration/io.cc > @@ -1,3 +1,4 @@ > +// { dg-options "-lstdc++exp" { target c++23 } } > // { dg-do run { target c++20 } } > // { dg-timeout-factor 2 } > > diff --git a/libstdc++-v3/testsuite/std/time/clock/file/io.cc b/libstdc++-v3/testsuite/std/time/clock/file/io.cc > index 9ab9f10ec77..2ab4a43c7e3 100644 > --- a/libstdc++-v3/testsuite/std/time/clock/file/io.cc > +++ b/libstdc++-v3/testsuite/std/time/clock/file/io.cc > @@ -1,3 +1,4 @@ > +// { dg-options "-lstdc++exp" { target c++23 } } > // { dg-do run { target c++20 } } > // { dg-timeout-factor 2 } > > diff --git a/libstdc++-v3/testsuite/std/time/clock/gps/io.cc b/libstdc++-v3/testsuite/std/time/clock/gps/io.cc > index d5405f61520..c3db63ab524 100644 > --- a/libstdc++-v3/testsuite/std/time/clock/gps/io.cc > +++ b/libstdc++-v3/testsuite/std/time/clock/gps/io.cc > @@ -1,3 +1,4 @@ > +// { dg-options "-lstdc++exp" { target c++23 } } > // { dg-do run { target c++20 } } > // { dg-timeout-factor 2 } > > diff --git a/libstdc++-v3/testsuite/std/time/clock/local/io.cc b/libstdc++-v3/testsuite/std/time/clock/local/io.cc > index bb682cd40cf..bb0ba0df419 100644 > --- a/libstdc++-v3/testsuite/std/time/clock/local/io.cc > +++ b/libstdc++-v3/testsuite/std/time/clock/local/io.cc > @@ -1,3 +1,4 @@ > +// { dg-options "-lstdc++exp" { target c++23 } } > // { dg-do run { target c++20 } } > // { dg-timeout-factor 2 } > > diff --git a/libstdc++-v3/testsuite/std/time/clock/system/io.cc b/libstdc++-v3/testsuite/std/time/clock/system/io.cc > index 2cc116156f2..dcb28dd51cf 100644 > --- a/libstdc++-v3/testsuite/std/time/clock/system/io.cc > +++ b/libstdc++-v3/testsuite/std/time/clock/system/io.cc > @@ -1,3 +1,4 @@ > +// { dg-options "-lstdc++exp" { target c++23 } } > // { dg-do run { target c++20 } } > // { dg-timeout-factor 2 } > > diff --git a/libstdc++-v3/testsuite/std/time/clock/tai/io.cc b/libstdc++-v3/testsuite/std/time/clock/tai/io.cc > index 0fd61c0e612..b85fb761d99 100644 > --- a/libstdc++-v3/testsuite/std/time/clock/tai/io.cc > +++ b/libstdc++-v3/testsuite/std/time/clock/tai/io.cc > @@ -1,3 +1,4 @@ > +// { dg-options "-lstdc++exp" { target c++23 } } > // { dg-do run { target c++20 } } > // { dg-timeout-factor 2 } > > diff --git a/libstdc++-v3/testsuite/std/time/clock/utc/io.cc b/libstdc++-v3/testsuite/std/time/clock/utc/io.cc > index 55c53dc4057..867c912cfdd 100644 > --- a/libstdc++-v3/testsuite/std/time/clock/utc/io.cc > +++ b/libstdc++-v3/testsuite/std/time/clock/utc/io.cc > @@ -1,3 +1,4 @@ > +// { dg-options "-lstdc++exp" { target c++23 } } > // { dg-do run { target c++20 } } > // { dg-timeout-factor 2 } > > diff --git a/libstdc++-v3/testsuite/std/time/day/io.cc b/libstdc++-v3/testsuite/std/time/day/io.cc > index 36ce7ec7d17..eb9be402e6f 100644 > --- a/libstdc++-v3/testsuite/std/time/day/io.cc > +++ b/libstdc++-v3/testsuite/std/time/day/io.cc > @@ -1,3 +1,4 @@ > +// { dg-options "-lstdc++exp" { target c++23 } } > // { dg-do run { target c++20 } } > // { dg-require-namedlocale "fr_FR.ISO8859-15" } > // { dg-timeout-factor 2 } > diff --git a/libstdc++-v3/testsuite/std/time/exceptions.cc b/libstdc++-v3/testsuite/std/time/exceptions.cc > index 06b8a53c353..1ecf502b62d 100644 > --- a/libstdc++-v3/testsuite/std/time/exceptions.cc > +++ b/libstdc++-v3/testsuite/std/time/exceptions.cc > @@ -1,3 +1,4 @@ > +// { dg-options "-lstdc++exp" { target c++23 } } > // { dg-do run { target c++20 } } > // { dg-require-effective-target tzdb } > > diff --git a/libstdc++-v3/testsuite/std/time/format.cc b/libstdc++-v3/testsuite/std/time/format.cc > index d6e35832cb5..18d2171c117 100644 > --- a/libstdc++-v3/testsuite/std/time/format.cc > +++ b/libstdc++-v3/testsuite/std/time/format.cc > @@ -1,3 +1,4 @@ > +// { dg-options "-lstdc++exp" { target c++23 } } > // { dg-do run { target c++20 } } > // { dg-timeout-factor 2 } > > diff --git a/libstdc++-v3/testsuite/std/time/format_localized.cc b/libstdc++-v3/testsuite/std/time/format_localized.cc > new file mode 100644 > index 00000000000..f1bfe94fddc > --- /dev/null > +++ b/libstdc++-v3/testsuite/std/time/format_localized.cc > @@ -0,0 +1,48 @@ > +// { dg-options "-fexec-charset=UTF-8 -lstdc++exp" } > +// { dg-do run { target c++23 } } > +// { dg-require-namedlocale "ru_UA.koi8u" } > +// { dg-require-namedlocale "es_ES.ISO8859-1" } > +// { dg-require-namedlocale "fr_FR.ISO8859-1" } > +// { dg-require-effective-target cxx11_abi } > + > +// P2419R2 > +// Clarify handling of encodings in localized formatting of chrono types > + > +// Localized date-time strings such as "février" should be converted to UTF-8 > +// if the locale uses a different encoding. > + > +#include <chrono> > +#include <format> > +#include <testsuite_hooks.h> > + > +void > +test_ru() > +{ > + std::locale loc("ru_UA.koi8u"); > + auto s = std::format(loc, "День недели: {:L}", std::chrono::Monday); > + VERIFY( s == "День недели: Пн" ); > +} > + > +void > +test_es() > +{ > + std::locale loc(ISO_8859(1,es_ES)); > + auto s = std::format(loc, "Día de la semana: {:L%A %a}", std::chrono::Wednesday); > + VERIFY( s == "Día de la semana: miércoles mié" ); > +} > + > +void > +test_fr() > +{ > + std::locale loc(ISO_8859(1,fr_FR)); > + auto s = std::format(loc, "Six mois après {0:L%b}, c'est {1:L%B}.", > + std::chrono::February, std::chrono::August); > + VERIFY( s == "Six mois après févr., c'est août." ); > +} > + > +int main() > +{ > + test_ru(); > + test_es(); > + test_fr(); > +} > diff --git a/libstdc++-v3/testsuite/std/time/hh_mm_ss/io.cc b/libstdc++-v3/testsuite/std/time/hh_mm_ss/io.cc > index d651dfbcdc8..551e4ed9a37 100644 > --- a/libstdc++-v3/testsuite/std/time/hh_mm_ss/io.cc > +++ b/libstdc++-v3/testsuite/std/time/hh_mm_ss/io.cc > @@ -1,3 +1,4 @@ > +// { dg-options "-lstdc++exp" { target c++23 } } > // { dg-do run { target c++20 } } > // { dg-timeout-factor 2 } > > diff --git a/libstdc++-v3/testsuite/std/time/month/io.cc b/libstdc++-v3/testsuite/std/time/month/io.cc > index 99ec0737305..564eed54706 100644 > --- a/libstdc++-v3/testsuite/std/time/month/io.cc > +++ b/libstdc++-v3/testsuite/std/time/month/io.cc > @@ -1,3 +1,4 @@ > +// { dg-options "-lstdc++exp" { target c++23 } } > // { dg-do run { target c++20 } } > // { dg-require-namedlocale "fr_FR.ISO8859-15" } > // { dg-timeout-factor 2 } > diff --git a/libstdc++-v3/testsuite/std/time/month_day/io.cc b/libstdc++-v3/testsuite/std/time/month_day/io.cc > index 30aa5881356..7f34f027ba9 100644 > --- a/libstdc++-v3/testsuite/std/time/month_day/io.cc > +++ b/libstdc++-v3/testsuite/std/time/month_day/io.cc > @@ -1,3 +1,4 @@ > +// { dg-options "-lstdc++exp" { target c++23 } } > // { dg-do run { target c++20 } } > // { dg-require-namedlocale "fr_FR.ISO8859-15" } > // { dg-timeout-factor 2 } > diff --git a/libstdc++-v3/testsuite/std/time/month_day_last/io.cc b/libstdc++-v3/testsuite/std/time/month_day_last/io.cc > index c12cee848ed..94d54b9b124 100644 > --- a/libstdc++-v3/testsuite/std/time/month_day_last/io.cc > +++ b/libstdc++-v3/testsuite/std/time/month_day_last/io.cc > @@ -1,3 +1,4 @@ > +// { dg-options "-lstdc++exp" { target c++23 } } > // { dg-do run { target c++20 } } > // { dg-require-namedlocale "fr_FR.ISO8859-15" } > // { dg-timeout-factor 2 } > diff --git a/libstdc++-v3/testsuite/std/time/month_weekday/io.cc b/libstdc++-v3/testsuite/std/time/month_weekday/io.cc > index 82cac648905..0fee88bb3a3 100644 > --- a/libstdc++-v3/testsuite/std/time/month_weekday/io.cc > +++ b/libstdc++-v3/testsuite/std/time/month_weekday/io.cc > @@ -1,3 +1,4 @@ > +// { dg-options "-lstdc++exp" { target c++23 } } > // { dg-do run { target c++20 } } > // { dg-require-namedlocale "fr_FR.ISO8859-15" } > // { dg-timeout-factor 2 } > diff --git a/libstdc++-v3/testsuite/std/time/month_weekday_last/io.cc b/libstdc++-v3/testsuite/std/time/month_weekday_last/io.cc > index 47968d0e06d..e6a6751bb4a 100644 > --- a/libstdc++-v3/testsuite/std/time/month_weekday_last/io.cc > +++ b/libstdc++-v3/testsuite/std/time/month_weekday_last/io.cc > @@ -1,3 +1,4 @@ > +// { dg-options "-lstdc++exp" { target c++23 } } > // { dg-do run { target c++20 } } > // { dg-require-namedlocale "fr_FR.ISO8859-15" } > // { dg-timeout-factor 2 } > diff --git a/libstdc++-v3/testsuite/std/time/time_zone/get_info_local.cc b/libstdc++-v3/testsuite/std/time/time_zone/get_info_local.cc > index d2972aca7df..af830374700 100644 > --- a/libstdc++-v3/testsuite/std/time/time_zone/get_info_local.cc > +++ b/libstdc++-v3/testsuite/std/time/time_zone/get_info_local.cc > @@ -1,3 +1,4 @@ > +// { dg-options "-lstdc++exp" { target c++23 } } > // { dg-do run { target c++20 } } > // { dg-require-effective-target tzdb } > > diff --git a/libstdc++-v3/testsuite/std/time/weekday/io.cc b/libstdc++-v3/testsuite/std/time/weekday/io.cc > index a56cdaef88c..f102c9250e9 100644 > --- a/libstdc++-v3/testsuite/std/time/weekday/io.cc > +++ b/libstdc++-v3/testsuite/std/time/weekday/io.cc > @@ -1,3 +1,4 @@ > +// { dg-options "-lstdc++exp" { target c++23 } } > // { dg-do run { target c++20 } } > // { dg-require-namedlocale "fr_FR.ISO8859-15" } > // { dg-timeout-factor 2 } > diff --git a/libstdc++-v3/testsuite/std/time/weekday_indexed/io.cc b/libstdc++-v3/testsuite/std/time/weekday_indexed/io.cc > index 29255ad5c06..42fc9b914a1 100644 > --- a/libstdc++-v3/testsuite/std/time/weekday_indexed/io.cc > +++ b/libstdc++-v3/testsuite/std/time/weekday_indexed/io.cc > @@ -1,3 +1,4 @@ > +// { dg-options "-lstdc++exp" { target c++23 } } > // { dg-do run { target c++20 } } > // { dg-require-namedlocale "fr_FR.ISO8859-15" } > // { dg-timeout-factor 2 } > diff --git a/libstdc++-v3/testsuite/std/time/weekday_last/io.cc b/libstdc++-v3/testsuite/std/time/weekday_last/io.cc > index 6f76922195d..d069d857ff0 100644 > --- a/libstdc++-v3/testsuite/std/time/weekday_last/io.cc > +++ b/libstdc++-v3/testsuite/std/time/weekday_last/io.cc > @@ -1,3 +1,4 @@ > +// { dg-options "-lstdc++exp" { target c++23 } } > // { dg-do run { target c++20 } } > // { dg-require-namedlocale "fr_FR.ISO8859-15" } > // { dg-timeout-factor 2 } > diff --git a/libstdc++-v3/testsuite/std/time/year/io.cc b/libstdc++-v3/testsuite/std/time/year/io.cc > index bcaa57faeb7..1bdc95ddbd1 100644 > --- a/libstdc++-v3/testsuite/std/time/year/io.cc > +++ b/libstdc++-v3/testsuite/std/time/year/io.cc > @@ -1,3 +1,4 @@ > +// { dg-options "-lstdc++exp" { target c++23 } } > // { dg-do run { target c++20 } } > // { dg-require-namedlocale "fr_FR.ISO8859-15" } > // { dg-timeout-factor 2 } > diff --git a/libstdc++-v3/testsuite/std/time/year_month/io.cc b/libstdc++-v3/testsuite/std/time/year_month/io.cc > index 7bb3442e299..e7963d31ae6 100644 > --- a/libstdc++-v3/testsuite/std/time/year_month/io.cc > +++ b/libstdc++-v3/testsuite/std/time/year_month/io.cc > @@ -1,3 +1,4 @@ > +// { dg-options "-lstdc++exp" { target c++23 } } > // { dg-do run { target c++20 } } > // { dg-require-namedlocale "fr_FR.ISO8859-15" } > // { dg-timeout-factor 2 } > diff --git a/libstdc++-v3/testsuite/std/time/year_month_day/io.cc b/libstdc++-v3/testsuite/std/time/year_month_day/io.cc > index cb82ef3b612..e7dc60b9dea 100644 > --- a/libstdc++-v3/testsuite/std/time/year_month_day/io.cc > +++ b/libstdc++-v3/testsuite/std/time/year_month_day/io.cc > @@ -1,3 +1,4 @@ > +// { dg-options "-lstdc++exp" { target c++23 } } > // { dg-do run { target c++20 } } > // { dg-require-namedlocale "fr_FR.ISO8859-15" } > // { dg-timeout-factor 2 } > diff --git a/libstdc++-v3/testsuite/std/time/year_month_day_last/io.cc b/libstdc++-v3/testsuite/std/time/year_month_day_last/io.cc > index 3241536a2e6..a57e56a1805 100644 > --- a/libstdc++-v3/testsuite/std/time/year_month_day_last/io.cc > +++ b/libstdc++-v3/testsuite/std/time/year_month_day_last/io.cc > @@ -1,3 +1,4 @@ > +// { dg-options "-lstdc++exp" { target c++23 } } > // { dg-do run { target c++20 } } > // { dg-require-namedlocale "fr_FR.ISO8859-15" } > // { dg-timeout-factor 2 } > diff --git a/libstdc++-v3/testsuite/std/time/year_month_weekday/io.cc b/libstdc++-v3/testsuite/std/time/year_month_weekday/io.cc > index 65baf1d37ae..e9a5bee7666 100644 > --- a/libstdc++-v3/testsuite/std/time/year_month_weekday/io.cc > +++ b/libstdc++-v3/testsuite/std/time/year_month_weekday/io.cc > @@ -1,3 +1,4 @@ > +// { dg-options "-lstdc++exp" { target c++23 } } > // { dg-do run { target c++20 } } > // { dg-require-namedlocale "fr_FR.ISO8859-15" } > // { dg-timeout-factor 2 } > diff --git a/libstdc++-v3/testsuite/std/time/year_month_weekday_last/io.cc b/libstdc++-v3/testsuite/std/time/year_month_weekday_last/io.cc > index 17f2244420d..fc2328ac001 100644 > --- a/libstdc++-v3/testsuite/std/time/year_month_weekday_last/io.cc > +++ b/libstdc++-v3/testsuite/std/time/year_month_weekday_last/io.cc > @@ -1,3 +1,4 @@ > +// { dg-options "-lstdc++exp" { target c++23 } } > // { dg-do run { target c++20 } } > // { dg-require-namedlocale "fr_FR.ISO8859-15" } > // { dg-timeout-factor 2 } > diff --git a/libstdc++-v3/testsuite/std/time/zoned_time/1.cc b/libstdc++-v3/testsuite/std/time/zoned_time/1.cc > index 1623aca1c7a..e773f50ae73 100644 > --- a/libstdc++-v3/testsuite/std/time/zoned_time/1.cc > +++ b/libstdc++-v3/testsuite/std/time/zoned_time/1.cc > @@ -1,3 +1,4 @@ > +// { dg-options "-lstdc++exp" { target c++23 } } > // { dg-do run { target c++20 } } > // { dg-require-effective-target tzdb } > // { dg-require-effective-target cxx11_abi } > diff --git a/libstdc++-v3/testsuite/std/time/zoned_time/io.cc b/libstdc++-v3/testsuite/std/time/zoned_time/io.cc > index 376b2734f19..c19e2a3c882 100644 > --- a/libstdc++-v3/testsuite/std/time/zoned_time/io.cc > +++ b/libstdc++-v3/testsuite/std/time/zoned_time/io.cc > @@ -1,3 +1,4 @@ > +// { dg-options "-lstdc++exp" { target c++23 } } > // { dg-do run { target c++20 } } > // { dg-require-effective-target cxx11_abi } > // { dg-timeout-factor 2 } -- Arsen Arsenović ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH 1/2] libstdc++: Replace padding bits with a bit-field in __format::_Spec 2024-02-01 15:36 [PATCH 1/2] libstdc++: Replace padding bits with a bit-field in __format::_Spec Jonathan Wakely 2024-02-01 15:36 ` [PATCH 2/2] libstdc++: Handle encodings in localized chrono formatting [PR109162] Jonathan Wakely @ 2024-02-01 16:16 ` Hans-Peter Nilsson 2024-02-01 17:08 ` Hans-Peter Nilsson 2024-02-01 16:33 ` Andreas Schwab 2 siblings, 1 reply; 11+ messages in thread From: Hans-Peter Nilsson @ 2024-02-01 16:16 UTC (permalink / raw) To: Jonathan Wakely; +Cc: libstdc++, gcc-patches > From: Jonathan Wakely <jwakely@redhat.com> > Cc: Hans-Peter Nilsson <hp@axis.com> > Date: Thu, 1 Feb 2024 15:36:50 +0000 > I plan to push this to trunk soon. > > CC HP for visibility of the change affecting cris-elf. In practice it > shouldn't make any difference to any sensible code. It only affects > C++20 mode (and later), and only changes the size of std::formatter > objects which are typically only created by the library headers > themselves, and only on the stack (when using std::format and other new > C++20 APIs related to it). > > -- >8 -- > > This ensures that the unused bits will be zero-initialized reliably, and > so can be used later by assigning them values in formatter > specializations. For example, formatters for std::chrono will need to > use an extra bit for a boolean to optimize the conversions between > locale encodings and UTF-8. > > This will result in an ABI change for targets that use 1-byte alignment > for all integral types, e.g. cris-elf. Thanks for the heads-up, but don't worry about ABI changes for cris-elf. Not speaking for other platforms with default-packed layout or where ABI structure layout alignment implies a change due to PCC_BITFIELD_TYPE_MATTERS and the "unsigned long" bitfield type. That last one may matter though. > We can't do that once C++20 > support is non-experimental and ABI stable, so do it now before GCC 14 > is released. > > libstdc++-v3/ChangeLog: > > * include/std/format (__format::_Spec::_M_reserved): Define a > new bit-field member in place of padding bits. > --- > libstdc++-v3/include/std/format | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/libstdc++-v3/include/std/format b/libstdc++-v3/include/std/format > index 0eca8b58bfa..6c958bc11a5 100644 > --- a/libstdc++-v3/include/std/format > +++ b/libstdc++-v3/include/std/format > @@ -406,6 +406,7 @@ namespace __format > _WidthPrec _M_width_kind : 2; > _WidthPrec _M_prec_kind : 2; > _Pres_type _M_type : 4; > + unsigned long _M_reserved : 17; > unsigned short _M_width; > unsigned short _M_prec; > char32_t _M_fill = ' '; > -- > 2.43.0 > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH 1/2] libstdc++: Replace padding bits with a bit-field in __format::_Spec 2024-02-01 16:16 ` [PATCH 1/2] libstdc++: Replace padding bits with a bit-field in __format::_Spec Hans-Peter Nilsson @ 2024-02-01 17:08 ` Hans-Peter Nilsson 2024-02-01 17:21 ` Jonathan Wakely 0 siblings, 1 reply; 11+ messages in thread From: Hans-Peter Nilsson @ 2024-02-01 17:08 UTC (permalink / raw) To: jwakely; +Cc: libstdc++, gcc-patches > From: Hans-Peter Nilsson <hp@axis.com> > Date: Thu, 1 Feb 2024 17:16:47 +0100 > Not speaking for other platforms with default-packed layout > or where ABI structure layout alignment implies a change due > to PCC_BITFIELD_TYPE_MATTERS and the "unsigned long" > bitfield type. > > That last one may matter though. > > diff --git a/libstdc++-v3/include/std/format b/libstdc++-v3/include/std/format > > index 0eca8b58bfa..6c958bc11a5 100644 > > --- a/libstdc++-v3/include/std/format > > +++ b/libstdc++-v3/include/std/format > > @@ -406,6 +406,7 @@ namespace __format > > _WidthPrec _M_width_kind : 2; > > _WidthPrec _M_prec_kind : 2; > > _Pres_type _M_type : 4; > > + unsigned long _M_reserved : 17; FAOD (no doubt you got this already, but...) I'd suggest making that "unsigned long" only for 16-bitters (i.e. where you'd need a type larger than "unsigned int" to cover 17 bits) and "unsigned" elsewhere, so at least you have no impact from PCC_BITFIELD_TYPE_MATTERS. brgds, H-P ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH 1/2] libstdc++: Replace padding bits with a bit-field in __format::_Spec 2024-02-01 17:08 ` Hans-Peter Nilsson @ 2024-02-01 17:21 ` Jonathan Wakely 0 siblings, 0 replies; 11+ messages in thread From: Jonathan Wakely @ 2024-02-01 17:21 UTC (permalink / raw) To: Hans-Peter Nilsson; +Cc: libstdc++, gcc-patches On Thu, 1 Feb 2024 at 17:08, Hans-Peter Nilsson <hp@axis.com> wrote: > > > From: Hans-Peter Nilsson <hp@axis.com> > > Date: Thu, 1 Feb 2024 17:16:47 +0100 > > > Not speaking for other platforms with default-packed layout > > or where ABI structure layout alignment implies a change due > > to PCC_BITFIELD_TYPE_MATTERS and the "unsigned long" > > bitfield type. > > > > That last one may matter though. > > > > diff --git a/libstdc++-v3/include/std/format b/libstdc++-v3/include/std/format > > > index 0eca8b58bfa..6c958bc11a5 100644 > > > --- a/libstdc++-v3/include/std/format > > > +++ b/libstdc++-v3/include/std/format > > > @@ -406,6 +406,7 @@ namespace __format > > > _WidthPrec _M_width_kind : 2; > > > _WidthPrec _M_prec_kind : 2; > > > _Pres_type _M_type : 4; > > > + unsigned long _M_reserved : 17; > > FAOD (no doubt you got this already, but...) > > I'd suggest making that "unsigned long" only for 16-bitters (i.e. where > you'd need a type larger than "unsigned int" to cover 17 bits) and > "unsigned" elsewhere, so at least you have no impact from > PCC_BITFIELD_TYPE_MATTERS. Ah, I have never heard of that. I guess using uint32_t would be OK. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH 1/2] libstdc++: Replace padding bits with a bit-field in __format::_Spec 2024-02-01 15:36 [PATCH 1/2] libstdc++: Replace padding bits with a bit-field in __format::_Spec Jonathan Wakely 2024-02-01 15:36 ` [PATCH 2/2] libstdc++: Handle encodings in localized chrono formatting [PR109162] Jonathan Wakely 2024-02-01 16:16 ` [PATCH 1/2] libstdc++: Replace padding bits with a bit-field in __format::_Spec Hans-Peter Nilsson @ 2024-02-01 16:33 ` Andreas Schwab 2024-02-01 17:22 ` Jonathan Wakely 2 siblings, 1 reply; 11+ messages in thread From: Andreas Schwab @ 2024-02-01 16:33 UTC (permalink / raw) To: Jonathan Wakely; +Cc: libstdc++, gcc-patches, Hans-Peter Nilsson On Feb 01 2024, Jonathan Wakely wrote: > This will result in an ABI change for targets that use 1-byte alignment > for all integral types, e.g. cris-elf. Or 2-byte alignment as on m68k. -- Andreas Schwab, SUSE Labs, schwab@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different." ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH 1/2] libstdc++: Replace padding bits with a bit-field in __format::_Spec 2024-02-01 16:33 ` Andreas Schwab @ 2024-02-01 17:22 ` Jonathan Wakely 2024-02-01 17:28 ` Jonathan Wakely 2024-02-01 19:24 ` Jonathan Wakely 0 siblings, 2 replies; 11+ messages in thread From: Jonathan Wakely @ 2024-02-01 17:22 UTC (permalink / raw) To: Andreas Schwab; +Cc: libstdc++, gcc-patches, Hans-Peter Nilsson On Thu, 1 Feb 2024 at 16:34, Andreas Schwab <schwab@suse.de> wrote: > > On Feb 01 2024, Jonathan Wakely wrote: > > > This will result in an ABI change for targets that use 1-byte alignment > > for all integral types, e.g. cris-elf. > > Or 2-byte alignment as on m68k. Ah yes. In fact it's a change for everybody, because previously it was 15 bits of bit-fields, 1 bit padding, 2 x 16-bit short, 16 bits padding, then a char32_t, but now it's 15+17 bits of bit-fields, 2x 16-bit short, char32_t. So the shorts moved internally even if the size didn't change. I could make it 15+1 bit-fields, 2x16-bit short, 16 bit-field to avoid moving the shorts. template<typename _CharT> struct _Spec { _Align _M_align : 2; _Sign _M_sign : 2; unsigned _M_alt : 1; unsigned _M_localized : 1; unsigned _M_zero_fill : 1; _WidthPrec _M_width_kind : 2; _WidthPrec _M_prec_kind : 2; _Pres_type _M_type : 4; unsigned _M_reserved : 1; unsigned short _M_width; unsigned short _M_prec; unsigned _M_reserved2 : 16; char32_t _M_fill = ' '; That would also address the PCC_BITFIELD_TYPE_MATTERS point H-P made. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH 1/2] libstdc++: Replace padding bits with a bit-field in __format::_Spec 2024-02-01 17:22 ` Jonathan Wakely @ 2024-02-01 17:28 ` Jonathan Wakely 2024-02-01 19:24 ` Jonathan Wakely 1 sibling, 0 replies; 11+ messages in thread From: Jonathan Wakely @ 2024-02-01 17:28 UTC (permalink / raw) To: Jonathan Wakely Cc: Andreas Schwab, libstdc++, gcc-patches, Hans-Peter Nilsson [-- Attachment #1: Type: text/plain, Size: 1430 bytes --] On Thu, 1 Feb 2024, 17:23 Jonathan Wakely, <jwakely@redhat.com> wrote: > On Thu, 1 Feb 2024 at 16:34, Andreas Schwab <schwab@suse.de> wrote: > > > > On Feb 01 2024, Jonathan Wakely wrote: > > > > > This will result in an ABI change for targets that use 1-byte alignment > > > for all integral types, e.g. cris-elf. > > > > Or 2-byte alignment as on m68k. > > Ah yes. > > In fact it's a change for everybody, because previously it was 15 bits > of bit-fields, 1 bit padding, 2 x 16-bit short, 16 bits padding, (Except for targets with 1- or 2-byte alignment, which didn't have the second padding) then > a char32_t, but now it's 15+17 bits of bit-fields, 2x 16-bit short, > char32_t. So the shorts moved internally even if the size didn't > change. > > I could make it 15+1 bit-fields, 2x16-bit short, 16 bit-field to avoid > moving the shorts. > > template<typename _CharT> > struct _Spec > { > _Align _M_align : 2; > _Sign _M_sign : 2; > unsigned _M_alt : 1; > unsigned _M_localized : 1; > unsigned _M_zero_fill : 1; > _WidthPrec _M_width_kind : 2; > _WidthPrec _M_prec_kind : 2; > _Pres_type _M_type : 4; > unsigned _M_reserved : 1; > unsigned short _M_width; > unsigned short _M_prec; > unsigned _M_reserved2 : 16; > char32_t _M_fill = ' '; > > That would also address the PCC_BITFIELD_TYPE_MATTERS point H-P made. > > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH 1/2] libstdc++: Replace padding bits with a bit-field in __format::_Spec 2024-02-01 17:22 ` Jonathan Wakely 2024-02-01 17:28 ` Jonathan Wakely @ 2024-02-01 19:24 ` Jonathan Wakely 2024-02-01 19:50 ` Hans-Peter Nilsson 1 sibling, 1 reply; 11+ messages in thread From: Jonathan Wakely @ 2024-02-01 19:24 UTC (permalink / raw) To: Andreas Schwab; +Cc: libstdc++, gcc-patches, Hans-Peter Nilsson On Thu, 1 Feb 2024 at 17:22, Jonathan Wakely <jwakely@redhat.com> wrote: > > On Thu, 1 Feb 2024 at 16:34, Andreas Schwab <schwab@suse.de> wrote: > > > > On Feb 01 2024, Jonathan Wakely wrote: > > > > > This will result in an ABI change for targets that use 1-byte alignment > > > for all integral types, e.g. cris-elf. > > > > Or 2-byte alignment as on m68k. > > Ah yes. > > In fact it's a change for everybody, because previously it was 15 bits > of bit-fields, 1 bit padding, 2 x 16-bit short, 16 bits padding, then > a char32_t, but now it's 15+17 bits of bit-fields, 2x 16-bit short, > char32_t. So the shorts moved internally even if the size didn't > change. > > I could make it 15+1 bit-fields, 2x16-bit short, 16 bit-field to avoid > moving the shorts. > > template<typename _CharT> > struct _Spec > { > _Align _M_align : 2; > _Sign _M_sign : 2; > unsigned _M_alt : 1; > unsigned _M_localized : 1; > unsigned _M_zero_fill : 1; > _WidthPrec _M_width_kind : 2; > _WidthPrec _M_prec_kind : 2; > _Pres_type _M_type : 4; > unsigned _M_reserved : 1; > unsigned short _M_width; > unsigned short _M_prec; > unsigned _M_reserved2 : 16; > char32_t _M_fill = ' '; > > That would also address the PCC_BITFIELD_TYPE_MATTERS point H-P made. Oh, and I forgot to mention in the first email that I already changed the ABI of that type in r14-6991-g37a4c5c23a270c so it's already different from gcc-13 anyway. I think I'd prefer to keep the reserved bits together, but a simpler way to avoid 'unsigned long' making a difference for PCC_BITFIELD_TYPE_MATTERS targets would be to use no more than 16 bits but do: unsigned _M_reserved : 1; unsigned _M_reserved2 : 16; ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH 1/2] libstdc++: Replace padding bits with a bit-field in __format::_Spec 2024-02-01 19:24 ` Jonathan Wakely @ 2024-02-01 19:50 ` Hans-Peter Nilsson 0 siblings, 0 replies; 11+ messages in thread From: Hans-Peter Nilsson @ 2024-02-01 19:50 UTC (permalink / raw) To: Jonathan Wakely; +Cc: schwab, libstdc++, gcc-patches > From: Jonathan Wakely <jwakely@redhat.com> > Date: Thu, 1 Feb 2024 19:24:49 +0000 > I think I'd prefer to keep the reserved bits together, but a simpler > way to avoid 'unsigned long' making a difference for > PCC_BITFIELD_TYPE_MATTERS targets would be to use no more than 16 bits > but do: > > unsigned _M_reserved : 1; > unsigned _M_reserved2 : 16; > Hah! Genious! :-) brgds, H-P ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2024-02-01 19:50 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2024-02-01 15:36 [PATCH 1/2] libstdc++: Replace padding bits with a bit-field in __format::_Spec Jonathan Wakely 2024-02-01 15:36 ` [PATCH 2/2] libstdc++: Handle encodings in localized chrono formatting [PR109162] Jonathan Wakely 2024-02-01 16:43 ` Arsen Arsenović 2024-02-01 16:16 ` [PATCH 1/2] libstdc++: Replace padding bits with a bit-field in __format::_Spec Hans-Peter Nilsson 2024-02-01 17:08 ` Hans-Peter Nilsson 2024-02-01 17:21 ` Jonathan Wakely 2024-02-01 16:33 ` Andreas Schwab 2024-02-01 17:22 ` Jonathan Wakely 2024-02-01 17:28 ` Jonathan Wakely 2024-02-01 19:24 ` Jonathan Wakely 2024-02-01 19:50 ` Hans-Peter Nilsson
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).