Re: [PATCH V2] Enable small loop unrolling for O2

[Richard Biener <richard.guenther@gmail.com>]

On Wed, Nov 2, 2022 at 4:37 AM Hongyu Wang <hongyu.wang@intel.com> wrote:
>
> Hi, this is the updated patch of
> https://gcc.gnu.org/pipermail/gcc-patches/2022-October/604345.html,
> which uses targetm.loop_unroll_adjust as gate to enable small loop unroll.
>
> This patch does not change rs6000/s390 since I don't have machine to
> test them, but I suppose the default behavior is the same since they
> enable flag_unroll_loops at O2.
>
> Bootstrapped & regrtested on x86_64-pc-linux-gnu.
>
> Ok for trunk?
>
> ---------- Patch content --------
>
> Modern processors has multiple way instruction decoders
> For x86, icelake/zen3 has 5 uops, so for small loop with <= 4
> instructions (usually has 3 uops with a cmp/jmp pair that can be
> macro-fused), the decoder would have 2 uops bubble for each iteration
> and the pipeline could not be fully utilized.
>
> Therefore, this patch enables loop unrolling for small size loop at O2
> to fullfill the decoder as much as possible. It turns on rtl loop
> unrolling when targetm.loop_unroll_adjust exists and O2 plus speed only.
> In x86 backend the default behavior is to unroll small loops with less
> than 4 insns by 1 time.
>
> This improves 548.exchange2 by 9% on icelake and 7.4% on zen3 with
> 0.9% codesize increment. For other benchmarks the variants are minor
> and overall codesize increased by 0.2%.
>
> The kernel image size increased by 0.06%, and no impact on eembc.
>
> gcc/ChangeLog:
>
>         * common/config/i386/i386-common.cc (ix86_optimization_table):
>         Enable small loop unroll at O2 by default.
>         * config/i386/i386.cc (ix86_loop_unroll_adjust): Adjust unroll
>         factor if -munroll-only-small-loops enabled and -funroll-loops/
>         -funroll-all-loops are disabled.
>         * config/i386/i386.opt: Add -munroll-only-small-loops,
>         -param=x86-small-unroll-ninsns= for loop insn limit,
>         -param=x86-small-unroll-factor= for unroll factor.
>         * doc/invoke.texi: Document -munroll-only-small-loops,
>         x86-small-unroll-ninsns and x86-small-unroll-factor.
>         * loop-init.cc (pass_rtl_unroll_loops::gate): Enable rtl
>         loop unrolling for -O2-speed and above if target hook
>         loop_unroll_adjust exists.
>
> gcc/testsuite/ChangeLog:
>
>         * gcc.dg/guality/loop-1.c: Add additional option
>           -mno-unroll-only-small-loops.
>         * gcc.target/i386/pr86270.c: Add -mno-unroll-only-small-loops.
>         * gcc.target/i386/pr93002.c: Likewise.
> ---
>  gcc/common/config/i386/i386-common.cc   |  1 +
>  gcc/config/i386/i386.cc                 | 18 ++++++++++++++++++
>  gcc/config/i386/i386.opt                | 13 +++++++++++++
>  gcc/doc/invoke.texi                     | 16 ++++++++++++++++
>  gcc/loop-init.cc                        | 10 +++++++---
>  gcc/testsuite/gcc.dg/guality/loop-1.c   |  2 ++
>  gcc/testsuite/gcc.target/i386/pr86270.c |  2 +-
>  gcc/testsuite/gcc.target/i386/pr93002.c |  2 +-
>  8 files changed, 59 insertions(+), 5 deletions(-)
>
> diff --git a/gcc/common/config/i386/i386-common.cc b/gcc/common/config/i386/i386-common.cc
> index f66bdd5a2af..c6891486078 100644
> --- a/gcc/common/config/i386/i386-common.cc
> +++ b/gcc/common/config/i386/i386-common.cc
> @@ -1724,6 +1724,7 @@ static const struct default_options ix86_option_optimization_table[] =
>      /* The STC algorithm produces the smallest code at -Os, for x86.  */
>      { OPT_LEVELS_2_PLUS, OPT_freorder_blocks_algorithm_, NULL,
>        REORDER_BLOCKS_ALGORITHM_STC },
> +    { OPT_LEVELS_2_PLUS_SPEED_ONLY, OPT_munroll_only_small_loops, NULL, 1 },
>      /* Turn off -fschedule-insns by default.  It tends to make the
>         problem with not enough registers even worse.  */
>      { OPT_LEVELS_ALL, OPT_fschedule_insns, NULL, 0 },
> diff --git a/gcc/config/i386/i386.cc b/gcc/config/i386/i386.cc
> index c0f37149ed0..0f94a3b609e 100644
> --- a/gcc/config/i386/i386.cc
> +++ b/gcc/config/i386/i386.cc
> @@ -23827,6 +23827,24 @@ ix86_loop_unroll_adjust (unsigned nunroll, class loop *loop)
>    unsigned i;
>    unsigned mem_count = 0;
>
> +  /* Unroll small size loop when unroll factor is not explicitly
> +     specified.  */
> +  if (!(flag_unroll_loops
> +       || flag_unroll_all_loops
> +       || loop->unroll))
> +    {
> +      nunroll = 1;
> +
> +      /* Any explicit -f{no-}unroll-{all-}loops turns off
> +        -munroll-only-small-loops.  */
> +      if (ix86_unroll_only_small_loops
> +         && !OPTION_SET_P (flag_unroll_loops))
> +       if (loop->ninsns <= (unsigned) ix86_small_unroll_ninsns)

either add braces or combine the two if's

Otherwise the middle-end changes look OK.  The target maintainers need to decide
whether the two --params should be core tunings instead - I would assume that
given your rationale the decode and issue widths of the core plays an important
role here.  That might also suggest a single parameter instead and unrolling
(factor * issue_width) / loop->ninsns times instead of a static unroll_factor?

Thanks,
Richard.

> +         nunroll = (unsigned) ix86_small_unroll_factor;
> +
> +      return nunroll;
> +    }
> +
>    if (!TARGET_ADJUST_UNROLL)
>       return nunroll;
>
> diff --git a/gcc/config/i386/i386.opt b/gcc/config/i386/i386.opt
> index 53d534f6392..6da9c8d670d 100644
> --- a/gcc/config/i386/i386.opt
> +++ b/gcc/config/i386/i386.opt
> @@ -1224,3 +1224,16 @@ mavxvnniint8
>  Target Mask(ISA2_AVXVNNIINT8) Var(ix86_isa_flags2) Save
>  Support MMX, SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, AVX, AVX2 and
>  AVXVNNIINT8 built-in functions and code generation.
> +
> +munroll-only-small-loops
> +Target Var(ix86_unroll_only_small_loops) Init(0) Save
> +Enable conservative small loop unrolling.
> +
> +-param=x86-small-unroll-ninsns=
> +Target Joined UInteger Var(ix86_small_unroll_ninsns) Init(4) Param
> +Insturctions number limit for loop to be unrolled under
> +-munroll-only-small-loops.
> +
> +-param=x86-small-unroll-factor=
> +Target Joined UInteger Var(ix86_small_unroll_factor) Init(2) Param
> +Unroll factor for -munroll-only-small-loops.
> diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
> index 550aec87809..487218bd0ce 100644
> --- a/gcc/doc/invoke.texi
> +++ b/gcc/doc/invoke.texi
> @@ -15821,6 +15821,14 @@ The following choices of @var{name} are available on i386 and x86_64 targets:
>  @item x86-stlf-window-ninsns
>  Instructions number above which STFL stall penalty can be compensated.
>
> +@item x86-small-unroll-ninsns
> +If -munroll-only-small-loops is enabled, only unroll loops with instruction
> +count less than this parameter. The default value is 4.
> +
> +@item x86-small-unroll-factor
> +If -munroll-only-small-loops is enabled, reset the unroll factor with this
> +value. The default value is 2 which means the loop will be unrolled once.
> +
>  @end table
>
>  @end table
> @@ -25232,6 +25240,14 @@ environments where no dynamic link is performed, like firmwares, OS
>  kernels, executables linked with @option{-static} or @option{-static-pie}.
>  @option{-mdirect-extern-access} is not compatible with @option{-fPIC} or
>  @option{-fpic}.
> +
> +@item -munroll-only-small-loops
> +@itemx -mno-unroll-only-small-loops
> +@opindex munroll-only-small-loops
> +Controls conservative small loop unrolling. It is default enbaled by
> +O2, and unrolls loop with less than 4 insns by 1 time. Explicit
> +-f[no-]unroll-[all-]loops would disable this flag to avoid any
> +unintended unrolling behavior that user does not want.
>  @end table
>
>  @node M32C Options
> diff --git a/gcc/loop-init.cc b/gcc/loop-init.cc
> index b9e07973dd6..9789efa1e11 100644
> --- a/gcc/loop-init.cc
> +++ b/gcc/loop-init.cc
> @@ -565,9 +565,12 @@ public:
>    {}
>
>    /* opt_pass methods: */
> -  bool gate (function *) final override
> +  bool gate (function *fun) final override
>      {
> -      return (flag_unroll_loops || flag_unroll_all_loops || cfun->has_unroll);
> +      return (flag_unroll_loops || flag_unroll_all_loops || cfun->has_unroll
> +             || (targetm.loop_unroll_adjust
> +                 && optimize >= 2
> +                 && optimize_function_for_speed_p (fun)));
>      }
>
>    unsigned int execute (function *) final override;
> @@ -583,7 +586,8 @@ pass_rtl_unroll_loops::execute (function *fun)
>        if (dump_file)
>         df_dump (dump_file);
>
> -      if (flag_unroll_loops)
> +      if (flag_unroll_loops
> +         || targetm.loop_unroll_adjust)
>         flags |= UAP_UNROLL;
>        if (flag_unroll_all_loops)
>         flags |= UAP_UNROLL_ALL;
> diff --git a/gcc/testsuite/gcc.dg/guality/loop-1.c b/gcc/testsuite/gcc.dg/guality/loop-1.c
> index 1b1f6d32271..a32ea445a3f 100644
> --- a/gcc/testsuite/gcc.dg/guality/loop-1.c
> +++ b/gcc/testsuite/gcc.dg/guality/loop-1.c
> @@ -1,5 +1,7 @@
>  /* { dg-do run } */
>  /* { dg-options "-fno-tree-scev-cprop -fno-tree-vectorize -g" } */
> +/* { dg-additional-options "-mno-unroll-only-small-loops" { target ia32 } } */
> +
>
>  #include "../nop.h"
>
> diff --git a/gcc/testsuite/gcc.target/i386/pr86270.c b/gcc/testsuite/gcc.target/i386/pr86270.c
> index 81841ef5bd7..cbc9fbb0450 100644
> --- a/gcc/testsuite/gcc.target/i386/pr86270.c
> +++ b/gcc/testsuite/gcc.target/i386/pr86270.c
> @@ -1,5 +1,5 @@
>  /* { dg-do compile } */
> -/* { dg-options "-O2" } */
> +/* { dg-options "-O2 -mno-unroll-only-small-loops" } */
>
>  int *a;
>  long len;
> diff --git a/gcc/testsuite/gcc.target/i386/pr93002.c b/gcc/testsuite/gcc.target/i386/pr93002.c
> index 0248fcc00a5..f75a847f75d 100644
> --- a/gcc/testsuite/gcc.target/i386/pr93002.c
> +++ b/gcc/testsuite/gcc.target/i386/pr93002.c
> @@ -1,6 +1,6 @@
>  /* PR target/93002 */
>  /* { dg-do compile } */
> -/* { dg-options "-O2" } */
> +/* { dg-options "-O2 -mno-unroll-only-small-loops" } */
>  /* { dg-final { scan-assembler-not "cmp\[^\n\r]*-1" } } */
>
>  volatile int sink;
> --
> 2.18.1
>