Re: [TCWG CI] 436.cactusADM:[.] PUGH_ReductionMinVal slowed down by 83% after gcc: Darwin: Future-proof and homogeneize detection of darwin versions

[Prathamesh Kulkarni <prathamesh.kulkarni@linaro.org>]

On Wed, 22 Dec 2021 at 04:54, <ci_notify@linaro.org> wrote:
>
> After gcc commit f18cbc1ee1f421a0dd79dc389bef9a23dd4a761d
> Author: François-Xavier Coudert <fxcoudert@gcc.gnu.org>
>
>     Darwin: Future-proof and homogeneize detection of darwin versions
>
> the following hot functions slowed down by more than 10% (but their benchmarks slowed down by less than 2%):
> - 436.cactusADM:[.] PUGH_ReductionMinVal slowed down by 83% from 6 to 11 perf samples
> - 403.gcc:[.] sbitmap_intersection_of_preds slowed down by 52% from 23 to 35 perf samples
> - 403.gcc:[.] canon_hash slowed down by 39% from 18 to 25 perf samples
>
Hi,
Unfortunately this is a false positive caused due to an issue in our
CI, which we are looking into.
I apologize for the inconvenience, please ignore the report.

Best Regards,
Prathamesh
> Below reproducer instructions can be used to re-build both "first_bad" and "last_good" cross-toolchains used in this bisection.  Naturally, the scripts will fail when triggerring benchmarking jobs if you don't have access to Linaro TCWG CI.
>
> For your convenience, we have uploaded tarballs with pre-processed source and assembly files at:
> - First_bad save-temps: https://ci.linaro.org/job/tcwg_bmk_ci_gnu-bisect-tcwg_bmk_tx1-gnu-master-aarch64-spec2k6-O3/44/artifact/artifacts/build-f18cbc1ee1f421a0dd79dc389bef9a23dd4a761d/save-temps/
> - Last_good save-temps: https://ci.linaro.org/job/tcwg_bmk_ci_gnu-bisect-tcwg_bmk_tx1-gnu-master-aarch64-spec2k6-O3/44/artifact/artifacts/build-2554e2da9263e4e26a164318f8041b19b8e54c21/save-temps/
> - Baseline save-temps: https://ci.linaro.org/job/tcwg_bmk_ci_gnu-bisect-tcwg_bmk_tx1-gnu-master-aarch64-spec2k6-O3/44/artifact/artifacts/build-baseline/save-temps/
>
> Configuration:
> - Benchmark: SPEC CPU2006
> - Toolchain: GCC + Glibc + GNU Linker
> - Version: all components were built from their tip of trunk
> - Target: aarch64-linux-gnu
> - Compiler flags: -O3
> - Hardware: NVidia TX1 4x Cortex-A57
>
> This benchmarking CI is work-in-progress, and we welcome feedback and suggestions at linaro-toolchain@lists.linaro.org .  In our improvement plans is to add support for SPEC CPU2017 benchmarks and provide "perf report/annotate" data behind these reports.
>
> THIS IS THE END OF INTERESTING STUFF.  BELOW ARE LINKS TO BUILDS, REPRODUCTION INSTRUCTIONS, AND THE RAW COMMIT.
>
> This commit has regressed these CI configurations:
>  - tcwg_bmk_gnu_tx1/gnu-master-aarch64-spec2k6-O3
>
> First_bad build: https://ci.linaro.org/job/tcwg_bmk_ci_gnu-bisect-tcwg_bmk_tx1-gnu-master-aarch64-spec2k6-O3/44/artifact/artifacts/build-f18cbc1ee1f421a0dd79dc389bef9a23dd4a761d/
> Last_good build: https://ci.linaro.org/job/tcwg_bmk_ci_gnu-bisect-tcwg_bmk_tx1-gnu-master-aarch64-spec2k6-O3/44/artifact/artifacts/build-2554e2da9263e4e26a164318f8041b19b8e54c21/
> Baseline build: https://ci.linaro.org/job/tcwg_bmk_ci_gnu-bisect-tcwg_bmk_tx1-gnu-master-aarch64-spec2k6-O3/44/artifact/artifacts/build-baseline/
> Even more details: https://ci.linaro.org/job/tcwg_bmk_ci_gnu-bisect-tcwg_bmk_tx1-gnu-master-aarch64-spec2k6-O3/44/artifact/artifacts/
>
> Reproduce builds:
> <cut>
> mkdir investigate-gcc-f18cbc1ee1f421a0dd79dc389bef9a23dd4a761d
> cd investigate-gcc-f18cbc1ee1f421a0dd79dc389bef9a23dd4a761d
>
> # Fetch scripts
> git clone https://git.linaro.org/toolchain/jenkins-scripts
>
> # Fetch manifests and test.sh script
> mkdir -p artifacts/manifests
> curl -o artifacts/manifests/build-baseline.sh https://ci.linaro.org/job/tcwg_bmk_ci_gnu-bisect-tcwg_bmk_tx1-gnu-master-aarch64-spec2k6-O3/44/artifact/artifacts/manifests/build-baseline.sh --fail
> curl -o artifacts/manifests/build-parameters.sh https://ci.linaro.org/job/tcwg_bmk_ci_gnu-bisect-tcwg_bmk_tx1-gnu-master-aarch64-spec2k6-O3/44/artifact/artifacts/manifests/build-parameters.sh --fail
> curl -o artifacts/test.sh https://ci.linaro.org/job/tcwg_bmk_ci_gnu-bisect-tcwg_bmk_tx1-gnu-master-aarch64-spec2k6-O3/44/artifact/artifacts/test.sh --fail
> chmod +x artifacts/test.sh
>
> # Reproduce the baseline build (build all pre-requisites)
> ./jenkins-scripts/tcwg_bmk-build.sh @@ artifacts/manifests/build-baseline.sh
>
> # Save baseline build state (which is then restored in artifacts/test.sh)
> mkdir -p ./bisect
> rsync -a --del --delete-excluded --exclude /bisect/ --exclude /artifacts/ --exclude /gcc/ ./ ./bisect/baseline/
>
> cd gcc
>
> # Reproduce first_bad build
> git checkout --detach f18cbc1ee1f421a0dd79dc389bef9a23dd4a761d
> ../artifacts/test.sh
>
> # Reproduce last_good build
> git checkout --detach 2554e2da9263e4e26a164318f8041b19b8e54c21
> ../artifacts/test.sh
>
> cd ..
> </cut>
>
> Full commit (up to 1000 lines):
> <cut>
> commit f18cbc1ee1f421a0dd79dc389bef9a23dd4a761d
> Author: François-Xavier Coudert <fxcoudert@gcc.gnu.org>
> Date:   Fri Dec 17 19:30:36 2021 +0100
>
>     Darwin: Future-proof and homogeneize detection of darwin versions
>
>     The current GCC branch will become 12.1.0, which will be the stable
>     version of GCC when the next macOS version is released. There are some
>     places in GCC that don’t handle darwin22 as a version, so we need to
>     future-proof it (gcc/config.gcc and gcc/config/darwin-driver.c). We
>     align that code with what Apple clang does, i.e. accept all potential
>     major macOS versions until 99.
>
>     This patch also homogenises the handling of darwin version numbers,
>     where the majority of places use darwin2*, but some used darwin2[0-9]*.
>     Since there never was a darwin2.x version, the two are equivalent, and
>     we prefer the simpler darwin2*
>
>     gcc/ChangeLog:
>
>             * config/darwin-driver.c: Make version code more future-proof.
>             * config.gcc: Homogeneize darwin versions.
>             * configure.ac: Homogeneize darwin versions.
>             * configure: Regenerate.
>
>     gcc/testsuite/ChangeLog:
>
>             * gcc.dg/darwin-minversion-link.c: Test darwin21.
>             * obj-c++.dg/cxx-ivars-3.mm: Homogeneize darwin versions.
>             * obj-c++.dg/objc-gc-3.mm: Homogeneize darwin versions.
>             * objc.dg/objc-gc-4.m: Homogeneize darwin versions.
> ---
>  gcc/config.gcc                                | 4 ++--
>  gcc/config/darwin-driver.c                    | 6 +++---
>  gcc/configure                                 | 2 +-
>  gcc/configure.ac                              | 2 +-
>  gcc/testsuite/gcc.dg/darwin-minversion-link.c | 3 ++-
>  gcc/testsuite/obj-c++.dg/cxx-ivars-3.mm       | 2 +-
>  gcc/testsuite/obj-c++.dg/objc-gc-3.mm         | 2 +-
>  gcc/testsuite/objc.dg/objc-gc-4.m             | 2 +-
>  8 files changed, 12 insertions(+), 11 deletions(-)
>
> diff --git a/gcc/config.gcc b/gcc/config.gcc
> index c8824367b13..c8c0ef9aa05 100644
> --- a/gcc/config.gcc
> +++ b/gcc/config.gcc
> @@ -1847,7 +1847,7 @@ hppa[12]*-*-hpux11*)
>                 dwarf2=no
>         fi
>         ;;
> -i[34567]86-*-darwin1[89]* | i[34567]86-*-darwin2[0-9]*)
> +i[34567]86-*-darwin1[89]* | i[34567]86-*-darwin2*)
>         echo "Error: 32bit target is not supported after Darwin17" 1>&2
>         ;;
>  i[34567]86-*-darwin*)
> @@ -1857,7 +1857,7 @@ i[34567]86-*-darwin*)
>         tmake_file="${tmake_file} ${cpu_type}/t-darwin32-biarch t-slibgcc"
>         tm_file="${cpu_type}/darwin32-biarch.h ${tm_file} "
>         ;;
> -x86_64-*-darwin1[89]* | x86_64-*-darwin2[01]*)
> +x86_64-*-darwin1[89]* | x86_64-*-darwin2*)
>         # Only 64b from now
>         tm_defines="${tm_defines} TARGET_64BIT_DEFAULT=(OPTION_MASK_ISA_64BIT|OPTION_MASK_ABI_64)"
>         tm_defines="${tm_defines} TARGET_BI_ARCH=0"
> diff --git a/gcc/config/darwin-driver.c b/gcc/config/darwin-driver.c
> index 688fe8fabc0..e459a72f907 100644
> --- a/gcc/config/darwin-driver.c
> +++ b/gcc/config/darwin-driver.c
> @@ -64,7 +64,8 @@ validate_macosx_version_min (const char *version_str)
>
>    major = strtoul (version_str, &end, 10);
>
> -  if (major < 10 || major > 12 ) /* macOS 10, 11, and 12 are known. */
> +  /* macOS 10, 11, and 12 are known. clang accepts up to 99.  */
> +  if (major < 10 || major > 99)
>      return NULL;
>
>    /* Skip a separating period, if there's one.  */
> @@ -160,8 +161,7 @@ darwin_find_version_from_kernel (void)
>
>    /* Darwin20 sees a transition to macOS 11.  In this, it seems that the
>       mapping to macOS minor version is now shifted to the kernel minor
> -     version - 1 (at least for the initial releases).  At this stage, we
> -     don't know what macOS version will correspond to Darwin21.  */
> +     version - 1 (at least for the initial releases).  */
>    if (major_vers >= 20)
>      {
>        int minor_vers = *version_p++ - '0';
> diff --git a/gcc/configure b/gcc/configure
> index 5470987b62f..992a9d70092 100755
> --- a/gcc/configure
> +++ b/gcc/configure
> @@ -26947,7 +26947,7 @@ $as_echo "$as_me: WARNING: LTO for $target requires binutils >= 2.20.1, but vers
>         ;;
>      esac
>      case $target_os in
> -       darwin2[0-9]* | darwin19*)
> +       darwin2* | darwin19*)
>          { $as_echo "$as_me:${as_lineno-$LINENO}: checking assembler for llvm assembler x86-pad-for-align option" >&5
>  $as_echo_n "checking assembler for llvm assembler x86-pad-for-align option... " >&6; }
>  if ${gcc_cv_as_mllvm_x86_pad_for_align+:} false; then :
> diff --git a/gcc/configure.ac b/gcc/configure.ac
> index 2b4914b6ec1..0db9c4cb027 100644
> --- a/gcc/configure.ac
> +++ b/gcc/configure.ac
> @@ -4692,7 +4692,7 @@ foo:      nop
>         ;;
>      esac
>      case $target_os in
> -       darwin2[[0-9]]* | darwin19*)
> +       darwin2* | darwin19*)
>          gcc_GAS_CHECK_FEATURE([llvm assembler x86-pad-for-align option],
>            gcc_cv_as_mllvm_x86_pad_for_align,
>            [-mllvm -x86-pad-for-align=false], [.text],,
> diff --git a/gcc/testsuite/gcc.dg/darwin-minversion-link.c b/gcc/testsuite/gcc.dg/darwin-minversion-link.c
> index 765fb799a91..b6ede31c985 100644
> --- a/gcc/testsuite/gcc.dg/darwin-minversion-link.c
> +++ b/gcc/testsuite/gcc.dg/darwin-minversion-link.c
> @@ -15,7 +15,8 @@
>  /* { dg-additional-options "-mmacosx-version-min=010.013.06 -DCHECK=101306" { target *-*-darwin17* } } */
>  /* { dg-additional-options "-mmacosx-version-min=010.014.05 -DCHECK=101405" { target *-*-darwin18* } } */
>  /* { dg-additional-options "-mmacosx-version-min=010.015.06 -DCHECK=101506" { target *-*-darwin19* } } */
> -/* { dg-additional-options "-mmacosx-version-min=011.000.00 -DCHECK=110000" { target *-*-darwin20 } } */
> +/* { dg-additional-options "-mmacosx-version-min=011.000.00 -DCHECK=110000" { target *-*-darwin20* } } */
> +/* { dg-additional-options "-mmacosx-version-min=012.000.00 -DCHECK=120000" { target *-*-darwin21* } } */
>
>  int
>  main ()
> diff --git a/gcc/testsuite/obj-c++.dg/cxx-ivars-3.mm b/gcc/testsuite/obj-c++.dg/cxx-ivars-3.mm
> index 07123559d72..27bae630ce8 100644
> --- a/gcc/testsuite/obj-c++.dg/cxx-ivars-3.mm
> +++ b/gcc/testsuite/obj-c++.dg/cxx-ivars-3.mm
> @@ -2,7 +2,7 @@
>
>  // { dg-do run { target *-*-darwin* } }
>  // { dg-skip-if "" { *-*-* } { "-fgnu-runtime" } { "" } }
> -// { dg-skip-if "Headers incompatible with 10.4 APIs" { *-*-darwin1[1-9]* *-*-darwin2[0-9]* } { "-fnext-runtime" } { "" } }
> +// { dg-skip-if "Headers incompatible with 10.4 APIs" { *-*-darwin1[1-9]* *-*-darwin2* } { "-fnext-runtime" } { "" } }
>  // { dg-additional-options "-fobjc-call-cxx-cdtors -mmacosx-version-min=10.4 -framework Foundation" }
>  // This test has no equivalent or meaning for m64/ABI V2
>  // { dg-xfail-run-if "No Test Avail" {  *-*-darwin* && lp64 } { "-fnext-runtime" } { "" } }
> diff --git a/gcc/testsuite/obj-c++.dg/objc-gc-3.mm b/gcc/testsuite/obj-c++.dg/objc-gc-3.mm
> index 45ffbc5553d..18f2cbe8869 100644
> --- a/gcc/testsuite/obj-c++.dg/objc-gc-3.mm
> +++ b/gcc/testsuite/obj-c++.dg/objc-gc-3.mm
> @@ -3,7 +3,7 @@
>  /* Contributed by Ziemowit Laski <zlaski@apple.com>  */
>
>  /* { dg-do compile } */
> -/* { dg-skip-if "GC API is an error from Darwin16." { *-*-darwin1[6-9]* *-*-darwin2[0-9]* } { "-fnext-runtime" } { "" } } */
> +/* { dg-skip-if "GC API is an error from Darwin16." { *-*-darwin1[6-9]* *-*-darwin2* } { "-fnext-runtime" } { "" } } */
>  /* { dg-options "-fobjc-gc" } */
>  /* { dg-prune-output "cc1objplus: warning: '-fobjc-gc' is ignored for '-fgnu-runtime'" } */
>
> diff --git a/gcc/testsuite/objc.dg/objc-gc-4.m b/gcc/testsuite/objc.dg/objc-gc-4.m
> index 8102a5a532f..1b2d9674969 100644
> --- a/gcc/testsuite/objc.dg/objc-gc-4.m
> +++ b/gcc/testsuite/objc.dg/objc-gc-4.m
> @@ -3,7 +3,7 @@
>  /* Contributed by Ziemowit Laski <zlaski@apple.com>  */
>
>  /* { dg-do compile } */
> -/* { dg-skip-if "GC API is an error from Darwin16." { *-*-darwin1[6-9]* *-*-darwin2[0-9]* } { "-fnext-runtime" } { "" } } */
> +/* { dg-skip-if "GC API is an error from Darwin16." { *-*-darwin1[6-9]* *-*-darwin2* } { "-fnext-runtime" } { "" } } */
>  /* { dg-options "-fobjc-gc" } */
>  /* { dg-prune-output "cc1obj: warning: '-fobjc-gc' is ignored for '-fgnu-runtime'" } */
>
> </cut>