Re: [TCWG CI] Regression caused by gcc: libstdc++: Support old and new T_FMT for en_HK locale [PR103687]

[Maxim Kuvyrkov <maxim.kuvyrkov@linaro.org>]

Hi Jonathan,

This was sent out due to a bug in our CI.  Sorry for the noise.

Regards,

--
Maxim Kuvyrkov
https://www.linaro.org

> On 20 Dec 2021, at 15:38, ci_notify@linaro.org wrote:
> 
> [TCWG CI] Regression caused by gcc: libstdc++: Support old and new T_FMT for en_HK locale [PR103687]:
> commit 9a4b4514bde2fe2f287f6549ef51326fb8918008
> Author: Jonathan Wakely <jwakely@redhat.com>
> 
>    libstdc++: Support old and new T_FMT for en_HK locale [PR103687]
> 
> Results regressed to
> # reset_artifacts:
> -10
> # build_abe binutils:
> -9
> # build_abe stage1 -- --set gcc_override_configure=--disable-libsanitizer --set gcc_override_configure=--disable-multilib --set gcc_override_configure=--with-cpu=cortex-m4 --set gcc_override_configure=--with-mode=thumb --set gcc_override_configure=--with-float=hard:
> -8
> # build_abe newlib:
> -6
> # build_abe stage2 -- --set gcc_override_configure=--disable-libsanitizer --set gcc_override_configure=--disable-multilib --set gcc_override_configure=--with-cpu=cortex-m4 --set gcc_override_configure=--with-mode=thumb --set gcc_override_configure=--with-float=hard:
> -5
> # benchmark -- -O2_LTO_mthumb artifacts/build-9a4b4514bde2fe2f287f6549ef51326fb8918008/results_id:
> 1
> 
> from
> # reset_artifacts:
> -10
> # build_abe binutils:
> -9
> # build_abe stage1 -- --set gcc_override_configure=--disable-libsanitizer --set gcc_override_configure=--disable-multilib --set gcc_override_configure=--with-cpu=cortex-m4 --set gcc_override_configure=--with-mode=thumb --set gcc_override_configure=--with-float=hard:
> -8
> # build_abe newlib:
> -6
> # build_abe stage2 -- --set gcc_override_configure=--disable-libsanitizer --set gcc_override_configure=--disable-multilib --set gcc_override_configure=--with-cpu=cortex-m4 --set gcc_override_configure=--with-mode=thumb --set gcc_override_configure=--with-float=hard:
> -5
> # benchmark -- -O2_LTO_mthumb artifacts/build-baseline/results_id:
> 1
> 
> THIS IS THE END OF INTERESTING STUFF.  BELOW ARE LINKS TO BUILDS, REPRODUCTION INSTRUCTIONS, AND THE RAW COMMIT.
> 
> This commit has regressed these CI configurations:
> - tcwg_bmk_gnu_eabi_stm32/gnu_eabi-master-arm_eabi-coremark-O2_LTO
> 
> First_bad build: https://ci.linaro.org/job/tcwg_bmk_ci_gnu_eabi-bisect-tcwg_bmk_stm32-gnu_eabi-master-arm_eabi-coremark-O2_LTO/15/artifact/artifacts/build-9a4b4514bde2fe2f287f6549ef51326fb8918008/
> Last_good build: https://ci.linaro.org/job/tcwg_bmk_ci_gnu_eabi-bisect-tcwg_bmk_stm32-gnu_eabi-master-arm_eabi-coremark-O2_LTO/15/artifact/artifacts/build-85a438fc78dd12249ca854a3e5c577fefeb1a5cd/
> Baseline build: https://ci.linaro.org/job/tcwg_bmk_ci_gnu_eabi-bisect-tcwg_bmk_stm32-gnu_eabi-master-arm_eabi-coremark-O2_LTO/15/artifact/artifacts/build-baseline/
> Even more details: https://ci.linaro.org/job/tcwg_bmk_ci_gnu_eabi-bisect-tcwg_bmk_stm32-gnu_eabi-master-arm_eabi-coremark-O2_LTO/15/artifact/artifacts/
> 
> Reproduce builds:
> <cut>
> mkdir investigate-gcc-9a4b4514bde2fe2f287f6549ef51326fb8918008
> cd investigate-gcc-9a4b4514bde2fe2f287f6549ef51326fb8918008
> 
> # Fetch scripts
> git clone https://git.linaro.org/toolchain/jenkins-scripts
> 
> # Fetch manifests and test.sh script
> mkdir -p artifacts/manifests
> curl -o artifacts/manifests/build-baseline.sh https://ci.linaro.org/job/tcwg_bmk_ci_gnu_eabi-bisect-tcwg_bmk_stm32-gnu_eabi-master-arm_eabi-coremark-O2_LTO/15/artifact/artifacts/manifests/build-baseline.sh --fail
> curl -o artifacts/manifests/build-parameters.sh https://ci.linaro.org/job/tcwg_bmk_ci_gnu_eabi-bisect-tcwg_bmk_stm32-gnu_eabi-master-arm_eabi-coremark-O2_LTO/15/artifact/artifacts/manifests/build-parameters.sh --fail
> curl -o artifacts/test.sh https://ci.linaro.org/job/tcwg_bmk_ci_gnu_eabi-bisect-tcwg_bmk_stm32-gnu_eabi-master-arm_eabi-coremark-O2_LTO/15/artifact/artifacts/test.sh --fail
> chmod +x artifacts/test.sh
> 
> # Reproduce the baseline build (build all pre-requisites)
> ./jenkins-scripts/tcwg_bmk-build.sh @@ artifacts/manifests/build-baseline.sh
> 
> # Save baseline build state (which is then restored in artifacts/test.sh)
> mkdir -p ./bisect
> rsync -a --del --delete-excluded --exclude /bisect/ --exclude /artifacts/ --exclude /gcc/ ./ ./bisect/baseline/
> 
> cd gcc
> 
> # Reproduce first_bad build
> git checkout --detach 9a4b4514bde2fe2f287f6549ef51326fb8918008
> ../artifacts/test.sh
> 
> # Reproduce last_good build
> git checkout --detach 85a438fc78dd12249ca854a3e5c577fefeb1a5cd
> ../artifacts/test.sh
> 
> cd ..
> </cut>
> 
> Full commit (up to 1000 lines):
> <cut>
> commit 9a4b4514bde2fe2f287f6549ef51326fb8918008
> Author: Jonathan Wakely <jwakely@redhat.com>
> Date:   Tue Dec 14 22:14:48 2021 +0000
> 
>    libstdc++: Support old and new T_FMT for en_HK locale [PR103687]
> 
>    This checks whether the locale data for en_HK includes %p and adjusts
>    the string being tested accordingly. To account for Jakub's fix to make
>    %I parse "12" as 0 instead of 12, we need to change the expected value
>    for the case where the locale format doesn't include %p. Also change the
>    time from 12:00:00 to 12:02:01 so we can tell if the minutes and seconds
>    get mixed up.
> 
>    libstdc++-v3/ChangeLog:
> 
>            PR libstdc++/103687
>            * testsuite/22_locale/time_get/get_date/wchar_t/4.cc: Restore
>            original locale before returning.
>            * testsuite/22_locale/time_get/get_time/char/2.cc: Check for %p
>            in locale's T_FMT and adjust accordingly.
>            * testsuite/22_locale/time_get/get_time/wchar_t/2.cc: Likewise.
> ---
> .../22_locale/time_get/get_date/wchar_t/4.cc       |  9 +++---
> .../22_locale/time_get/get_time/char/2.cc          | 33 ++++++++++++++++++++--
> .../22_locale/time_get/get_time/wchar_t/2.cc       | 33 ++++++++++++++++++++--
> 3 files changed, 65 insertions(+), 10 deletions(-)
> 
> diff --git a/libstdc++-v3/testsuite/22_locale/time_get/get_date/wchar_t/4.cc b/libstdc++-v3/testsuite/22_locale/time_get/get_date/wchar_t/4.cc
> index d227d4b1ce0..f6de882e4bd 100644
> --- a/libstdc++-v3/testsuite/22_locale/time_get/get_date/wchar_t/4.cc
> +++ b/libstdc++-v3/testsuite/22_locale/time_get/get_date/wchar_t/4.cc
> @@ -39,7 +39,7 @@ void test01()
> 
>   wistringstream iss;
>   iss.imbue(loc_tw);
> -  const time_get<wchar_t>& tim_get = use_facet<time_get<wchar_t> >(iss.getloc()); 
> +  const time_get<wchar_t>& tim_get = use_facet<time_get<wchar_t> >(iss.getloc());
> 
>   const ios_base::iostate good = ios_base::goodbit;
>   ios_base::iostate errorstate = good;
> @@ -66,13 +66,14 @@ void test01()
> static bool debian_date_format()
> {
> #ifdef D_FMT
> +  std::string orig = setlocale(LC_TIME, NULL);
>   if (setlocale(LC_TIME, "zh_TW.UTF-8") != NULL)
>   {
>     // See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=31413
>     // and https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71641#c2
> -    if (*nl_langinfo(D_FMT) == '%')
> -      return true;
> -    setlocale(LC_TIME, "C");
> +    std::string d_fmt = nl_langinfo(D_FMT);
> +    setlocale(LC_TIME, orig.c_str());
> +    return d_fmt[0] == '%';
>   }
> #endif
>   return false;
> diff --git a/libstdc++-v3/testsuite/22_locale/time_get/get_time/char/2.cc b/libstdc++-v3/testsuite/22_locale/time_get/get_time/char/2.cc
> index a847748dc27..b40971a9bf7 100644
> --- a/libstdc++-v3/testsuite/22_locale/time_get/get_time/char/2.cc
> +++ b/libstdc++-v3/testsuite/22_locale/time_get/get_time/char/2.cc
> @@ -25,6 +25,8 @@
> #include <sstream>
> #include <testsuite_hooks.h>
> 
> +static bool ampm_time_format();
> +
> void test02()
> {
>   using namespace std;
> @@ -36,19 +38,23 @@ void test02()
>   locale loc_hk = locale(ISO_8859(1,en_HK));
>   VERIFY( loc_hk != loc_c );
> 
> +  const int pm = ampm_time_format() ? 12 : 0;
>   const string empty;
> -  const tm time_bday = __gnu_test::test_tm(0, 0, 12, 4, 3, 71, 0, 93, 0);
> +  const tm time_bday = __gnu_test::test_tm(1, 2, 0+pm, 4, 3, 71, 0, 93, 0);
> 
>   // create an ostream-derived object, cache the time_get facet
>   iterator_type end;
>   istringstream iss;
> -  const time_get<char>& tim_get = use_facet<time_get<char> >(iss.getloc()); 
> +  const time_get<char>& tim_get = use_facet<time_get<char> >(iss.getloc());
>   const ios_base::iostate good = ios_base::goodbit;
>   ios_base::iostate errorstate = good;
> 
>   // inspection of named locales, en_HK
>   iss.imbue(loc_hk);
> -  iss.str("12:00:00 PM PST"); 
> +  if (pm)
> +    iss.str("12:02:01 PM PST");
> +  else
> +    iss.str("12:02:01 PST"); // %I means 12-hour clock, so parsed as 12am
>   // Hong Kong in California! Well, they have Paris in Vegas... this
>   // is all a little disney-esque anyway. Besides, you can get decent
>   // Dim Sum in San Francisco.
> @@ -62,6 +68,27 @@ void test02()
>   VERIFY( errorstate == ios_base::eofbit );
> }
> 
> +#include <locale.h>
> +#if __has_include(<langinfo.h>)
> +# include <langinfo.h>
> +#endif
> +#include <string.h>
> +
> +static bool ampm_time_format()
> +{
> +#ifdef T_FMT
> +  std::string orig = setlocale(LC_TIME, NULL);
> +  if (setlocale(LC_TIME, ISO_8859(1,en_HK)) != NULL)
> +  {
> +    // See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103687
> +    std::string t_fmt = nl_langinfo(T_FMT);
> +    setlocale(LC_TIME, orig.c_str());
> +    return t_fmt.find("%p") != std::string::npos;
> +  }
> +#endif
> +  return false;
> +}
> +
> int main()
> {
>   test02();
> diff --git a/libstdc++-v3/testsuite/22_locale/time_get/get_time/wchar_t/2.cc b/libstdc++-v3/testsuite/22_locale/time_get/get_time/wchar_t/2.cc
> index b5d61e1afdb..b74604ef472 100644
> --- a/libstdc++-v3/testsuite/22_locale/time_get/get_time/wchar_t/2.cc
> +++ b/libstdc++-v3/testsuite/22_locale/time_get/get_time/wchar_t/2.cc
> @@ -25,6 +25,8 @@
> #include <sstream>
> #include <testsuite_hooks.h>
> 
> +static bool ampm_time_format();
> +
> void test02()
> {
>   using namespace std;
> @@ -36,19 +38,23 @@ void test02()
>   locale loc_hk = locale(ISO_8859(1,en_HK));
>   VERIFY( loc_hk != loc_c );
> 
> +  const int pm = ampm_time_format() ? 12 : 0;
>   const wstring empty;
> -  const tm time_bday = __gnu_test::test_tm(0, 0, 12, 4, 3, 71, 0, 93, 0);
> +  const tm time_bday = __gnu_test::test_tm(1, 2, 0+pm, 4, 3, 71, 0, 93, 0);
> 
>   // create an ostream-derived object, cache the time_get facet
>   iterator_type end;
>   wistringstream iss;
> -  const time_get<wchar_t>& tim_get = use_facet<time_get<wchar_t> >(iss.getloc()); 
> +  const time_get<wchar_t>& tim_get = use_facet<time_get<wchar_t> >(iss.getloc());
>   const ios_base::iostate good = ios_base::goodbit;
>   ios_base::iostate errorstate = good;
> 
>   // inspection of named locales, en_HK
>   iss.imbue(loc_hk);
> -  iss.str(L"12:00:00 PM PST"); 
> +  if (pm)
> +    iss.str(L"12:02:01 PM PST");
> +  else
> +    iss.str(L"12:02:01 PST"); // %I means 12-hour clock, so parsed as 12am
>   // Hong Kong in California! Well, they have Paris in Vegas... this
>   // is all a little disney-esque anyway. Besides, you can get decent
>   // Dim Sum in San Francisco.
> @@ -62,6 +68,27 @@ void test02()
>   VERIFY( errorstate == ios_base::eofbit );
> }
> 
> +#include <locale.h>
> +#if __has_include(<langinfo.h>)
> +# include <langinfo.h>
> +#endif
> +#include <string.h>
> +
> +static bool ampm_time_format()
> +{
> +#ifdef T_FMT
> +  std::string orig = setlocale(LC_TIME, NULL);
> +  if (setlocale(LC_TIME, ISO_8859(1,en_HK)) != NULL)
> +  {
> +    // See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103687
> +    std::string t_fmt = nl_langinfo(T_FMT);
> +    setlocale(LC_TIME, orig.c_str());
> +    return t_fmt.find("%p") != std::string::npos;
> +  }
> +#endif
> +  return false;
> +}
> +
> int main()
> {
>   test02();
> </cut>