From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by sourceware.org (Postfix) with ESMTPS id 7567D385843E for ; Wed, 26 Oct 2022 05:52:53 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 7567D385843E Authentication-Results: sourceware.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: sourceware.org; spf=fail smtp.mailfrom=gmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666763573; x=1698299573; h=from:to:cc:subject:date:message-id; bh=lR7xWIZOYX3IB8s02pd+L/yVJVFNpalvdEZWAWiy2ZE=; b=V4wujWIOUDft/e164m3BP+kMhALWv2rsePERWLNHrgl+4ldrD7uGu/a+ uJ4TosgbvMeASgVDe6BqOVqAswMmjxqvXIdBPYGrZQbf1yJWSE7i31y8I qQy3MrbuUYehPXr+0CGtDjSGN4NDcJXDzDlSMu59b6MYOF1moiG220x90 H4In6peq35Dth9IUh7kKuv5KFAVwELJCJzCemJwwbsYct2Z6dLBm0HuGb mTzGX+XEoYMBZ0qb4jYcjImsdJpDQa8meQLnqJ7EXi2O06ebgnA5GsfVd 5gjQlefH/F2ndLsDj3aBd1sw9MB02K7/NCd0sD56l+vjeyZaTdwbOsttY w==; X-IronPort-AV: E=McAfee;i="6500,9779,10511"; a="291169437" X-IronPort-AV: E=Sophos;i="5.95,213,1661842800"; d="scan'208";a="291169437" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Oct 2022 22:52:51 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10511"; a="665170560" X-IronPort-AV: E=Sophos;i="5.95,213,1661842800"; d="scan'208";a="665170560" Received: from shvmail03.sh.intel.com ([10.239.245.20]) by orsmga001.jf.intel.com with ESMTP; 25 Oct 2022 22:52:49 -0700 Received: from shliclel320.sh.intel.com (shliclel320.sh.intel.com [10.239.240.127]) by shvmail03.sh.intel.com (Postfix) with ESMTP id 8E37010056B1; Wed, 26 Oct 2022 13:52:48 +0800 (CST) From: Hongyu Wang To: gcc-patches@gcc.gnu.org Cc: richard.guenther@gmail.com, ubizjak@gmail.com, hongtao.liu@intel.com Subject: [PATCH] i386: Enable small loop unrolling for O2 Date: Wed, 26 Oct 2022 13:52:48 +0800 Message-Id: <20221026055248.94100-1-hongyu.wang@intel.com> X-Mailer: git-send-email 2.18.1 X-Spam-Status: No, score=-10.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM,GIT_PATCH_0,HEADER_FROM_DIFFERENT_DOMAINS,KAM_NUMSUBJECT,KAM_SHORT,SPF_HELO_NONE,SPF_SOFTFAIL,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Hi, Inspired by rs6000 and s390 port changes, this patch enables loop unrolling for small size loop at O2 by default. The default behavior is to unroll loop with unknown trip-count and less than 4 insns by 1 time. This improves 548.exchange2 by 3.5% on icelake and 6% on zen3 with 1.2% codesize increment. For other benchmarks the variants are minor and overall codesize increased by 0.2%. The kernel image size increased by 0.06%, and no impact on eembc. Bootstrapped & regrtested on x86_64-pc-linux-gnu. Ok for trunk? gcc/ChangeLog: * common/config/i386/i386-common.cc (ix86_optimization_table): Enable loop unroll and small loop unroll at O2 by default. * config/i386/i386-options.cc (ix86_override_options_after_change): Disable small loop unroll when funroll-loops enabled, reset cunroll_grow_size when it is not explicitly enabled. (ix86_option_override_internal): Call ix86_override_options_after_change instead of calling ix86_recompute_optlev_based_flags and ix86_default_align separately. * config/i386/i386.cc (ix86_loop_unroll_adjust): Adjust unroll factor if -munroll-only-small-loops enabled. * config/i386/i386.opt: Add -munroll-only-small-loops, -param=x86-small-unroll-ninsns= for loop insn limit, -param=x86-small-unroll-factor= for unroll factor. * doc/invoke.texi: Document -munroll-only-small-loops, x86-small-unroll-ninsns and x86-small-unroll-factor. gcc/testsuite/ChangeLog: * gcc.target/i386/pr86270.c: Add -mno-unroll-only-small-loops. * gcc.target/i386/pr93002.c: Likewise. --- gcc/common/config/i386/i386-common.cc | 6 ++++ gcc/config/i386/i386-options.cc | 40 ++++++++++++++++++++++--- gcc/config/i386/i386.cc | 13 ++++++++ gcc/config/i386/i386.opt | 13 ++++++++ gcc/doc/invoke.texi | 14 +++++++++ gcc/testsuite/gcc.target/i386/pr86270.c | 2 +- gcc/testsuite/gcc.target/i386/pr93002.c | 2 +- 7 files changed, 84 insertions(+), 6 deletions(-) diff --git a/gcc/common/config/i386/i386-common.cc b/gcc/common/config/i386/i386-common.cc index d6a68dc9b1d..0e580b39d14 100644 --- a/gcc/common/config/i386/i386-common.cc +++ b/gcc/common/config/i386/i386-common.cc @@ -1686,6 +1686,12 @@ static const struct default_options ix86_option_optimization_table[] = /* The STC algorithm produces the smallest code at -Os, for x86. */ { OPT_LEVELS_2_PLUS, OPT_freorder_blocks_algorithm_, NULL, REORDER_BLOCKS_ALGORITHM_STC }, + { OPT_LEVELS_2_PLUS_SPEED_ONLY, OPT_funroll_loops, NULL, 1 }, + { OPT_LEVELS_2_PLUS_SPEED_ONLY, OPT_munroll_only_small_loops, NULL, 1 }, + /* Turns off -frename-registers and -fweb which are enabled by + funroll-loops. */ + { OPT_LEVELS_ALL, OPT_frename_registers, NULL, 0 }, + { OPT_LEVELS_ALL, OPT_fweb, NULL, 0 }, /* Turn off -fschedule-insns by default. It tends to make the problem with not enough registers even worse. */ { OPT_LEVELS_ALL, OPT_fschedule_insns, NULL, 0 }, diff --git a/gcc/config/i386/i386-options.cc b/gcc/config/i386/i386-options.cc index acb2291e70f..6ea347c32e1 100644 --- a/gcc/config/i386/i386-options.cc +++ b/gcc/config/i386/i386-options.cc @@ -1819,8 +1819,43 @@ ix86_recompute_optlev_based_flags (struct gcc_options *opts, void ix86_override_options_after_change (void) { + /* Default align_* from the processor table. */ ix86_default_align (&global_options); + ix86_recompute_optlev_based_flags (&global_options, &global_options_set); + + /* Disable unrolling small loops when there's explicit + -f{,no}unroll-loop. */ + if ((OPTION_SET_P (flag_unroll_loops)) + || (OPTION_SET_P (flag_unroll_all_loops) + && flag_unroll_all_loops)) + { + if (!OPTION_SET_P (ix86_unroll_only_small_loops)) + ix86_unroll_only_small_loops = 0; + /* Re-enable -frename-registers and -fweb if funroll-loops + enabled. */ + if (!OPTION_SET_P (flag_web)) + flag_web = flag_unroll_loops; + if (!OPTION_SET_P (flag_rename_registers)) + flag_rename_registers = flag_unroll_loops; + if (!OPTION_SET_P (flag_cunroll_grow_size)) + flag_cunroll_grow_size = flag_unroll_loops + || flag_peel_loops + || optimize >= 3; + } + else + { + if (!OPTION_SET_P (flag_cunroll_grow_size)) + flag_cunroll_grow_size = flag_peel_loops || optimize >= 3; + /* Disables loop unrolling if -mno-unroll-only-small-loops is + explicitly set and -funroll-loops is not enabled. */ + if (OPTION_SET_P (ix86_unroll_only_small_loops) + && !ix86_unroll_only_small_loops + && !(OPTION_SET_P (flag_unroll_loops) + || OPTION_SET_P (flag_unroll_all_loops))) + flag_unroll_loops = flag_unroll_all_loops = 0; + } + } /* Clear stack slot assignments remembered from previous functions. @@ -2332,7 +2367,7 @@ ix86_option_override_internal (bool main_args_p, set_ix86_tune_features (opts, ix86_tune, opts->x_ix86_dump_tunes); - ix86_recompute_optlev_based_flags (opts, opts_set); + ix86_override_options_after_change (); ix86_tune_cost = processor_cost_table[ix86_tune]; /* TODO: ix86_cost should be chosen at instruction or function granuality @@ -2363,9 +2398,6 @@ ix86_option_override_internal (bool main_args_p, || TARGET_64BIT_P (opts->x_ix86_isa_flags)) opts->x_ix86_regparm = REGPARM_MAX; - /* Default align_* from the processor table. */ - ix86_default_align (opts); - /* Provide default for -mbranch-cost= value. */ SET_OPTION_IF_UNSET (opts, opts_set, ix86_branch_cost, ix86_tune_cost->branch_cost); diff --git a/gcc/config/i386/i386.cc b/gcc/config/i386/i386.cc index 480db35f6cd..75829a5d0f4 100644 --- a/gcc/config/i386/i386.cc +++ b/gcc/config/i386/i386.cc @@ -23820,6 +23820,19 @@ ix86_loop_unroll_adjust (unsigned nunroll, class loop *loop) unsigned i; unsigned mem_count = 0; + /* Unroll small size loop when unroll factor is not explicitly + specified. */ + if (ix86_unroll_only_small_loops && !loop->unroll) + { + int small_unroll = 0; + if (loop->ninsns <= (unsigned) ix86_small_unroll_ninsns) + small_unroll = MIN ((unsigned) ix86_small_unroll_factor, + nunroll); + else + small_unroll = 1; + return small_unroll; + } + if (!TARGET_ADJUST_UNROLL) return nunroll; diff --git a/gcc/config/i386/i386.opt b/gcc/config/i386/i386.opt index 0dbaacb57ed..a724c73c0c4 100644 --- a/gcc/config/i386/i386.opt +++ b/gcc/config/i386/i386.opt @@ -1214,3 +1214,16 @@ Do not use GOT to access external symbols. -param=x86-stlf-window-ninsns= Target Joined UInteger Var(x86_stlf_window_ninsns) Init(64) Param Instructions number above which STFL stall penalty can be compensated. + +munroll-only-small-loops +Target Var(ix86_unroll_only_small_loops) Init(0) Save +Enable conservative small loop unrolling. + +-param=x86-small-unroll-ninsns= +Target Joined UInteger Var(ix86_small_unroll_ninsns) Init(4) Param +Insturctions number limit for loop to be unrolled under +-munroll-only-small-loops. + +-param=x86-small-unroll-factor= +Target Joined UInteger Var(ix86_small_unroll_factor) Init(2) Param +Unroll factor for -munroll-only-small-loops. diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi index cd4d3c1d72c..b6fa79eccc3 100644 --- a/gcc/doc/invoke.texi +++ b/gcc/doc/invoke.texi @@ -15779,6 +15779,14 @@ The following choices of @var{name} are available on i386 and x86_64 targets: @item x86-stlf-window-ninsns Instructions number above which STFL stall penalty can be compensated. +@item x86-small-unroll-ninsns +If -munroll-only-small-loops is enabled, only unroll loops with instruction +count less than this parameter. The default value is 4. + +@item x86-small-unroll-factor +If -munroll-only-small-loops is enabled, reset the unroll factor with this +value. The default value is 2 which means the loop will be unrolled once. + @end table @end table @@ -25186,6 +25194,12 @@ environments where no dynamic link is performed, like firmwares, OS kernels, executables linked with @option{-static} or @option{-static-pie}. @option{-mdirect-extern-access} is not compatible with @option{-fPIC} or @option{-fpic}. + +@item -munroll-only-small-loops +@itemx -mno-unroll-only-small-loops +@opindex munroll-only-small-loops +Controls conservative small loop unrolling. It is default enbaled by +O2, and unrolls loop with less than 4 insns by 1 time. @end table @node M32C Options diff --git a/gcc/testsuite/gcc.target/i386/pr86270.c b/gcc/testsuite/gcc.target/i386/pr86270.c index 81841ef5bd7..cbc9fbb0450 100644 --- a/gcc/testsuite/gcc.target/i386/pr86270.c +++ b/gcc/testsuite/gcc.target/i386/pr86270.c @@ -1,5 +1,5 @@ /* { dg-do compile } */ -/* { dg-options "-O2" } */ +/* { dg-options "-O2 -mno-unroll-only-small-loops" } */ int *a; long len; diff --git a/gcc/testsuite/gcc.target/i386/pr93002.c b/gcc/testsuite/gcc.target/i386/pr93002.c index 0248fcc00a5..f75a847f75d 100644 --- a/gcc/testsuite/gcc.target/i386/pr93002.c +++ b/gcc/testsuite/gcc.target/i386/pr93002.c @@ -1,6 +1,6 @@ /* PR target/93002 */ /* { dg-do compile } */ -/* { dg-options "-O2" } */ +/* { dg-options "-O2 -mno-unroll-only-small-loops" } */ /* { dg-final { scan-assembler-not "cmp\[^\n\r]*-1" } } */ volatile int sink; -- 2.18.1