[Bug middle-end/64909] New: [4.8/5 regression] Missed vectorization

[Bug middle-end/64909] New: [4.8/5 regression] Missed vectorization

[hubicka at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64909

            Bug ID: 64909
           Summary: [4.8/5 regression] Missed vectorization
           Product: gcc
           Version: 5.0
            Status: UNCONFIRMED
          Severity: enhancement
          Priority: P3
         Component: middle-end
          Assignee: unassigned at gcc dot gnu.org
          Reporter: hubicka at gcc dot gnu.org

Hi,
the following loop (taken from firefox unicode stuff)
unsigned short a[32];
unsigned int b[32];
t()
{
  int i;
  for (i=0;i<12;i++)
    b[i]=a[i];
}

compiles by clang to:
t:                                      # @t
        .cfi_startproc
# BB#0:
        vpmovzxwd       a(%rip), %xmm0
        vmovdqa .LCPI0_0(%rip), %xmm1   # xmm1 = [65535,65535,65535,65535]
        vpand   %xmm1, %xmm0, %xmm0
        vmovdqa %xmm0, b(%rip)
        vpmovzxwd       a+8(%rip), %xmm0
        vpand   %xmm1, %xmm0, %xmm0
        vmovdqa %xmm0, b+16(%rip)
        vpmovzxwd       a+16(%rip), %xmm0
        vpand   %xmm1, %xmm0, %xmm0
        vmovdqa %xmm0, b+32(%rip)
        retq

GCC 4.7 does:
t:
.LFB0:
        .cfi_startproc
        movzwl  a+16(%rip), %eax
        vmovaps a(%rip), %xmm0
        vpmovzxwd       %xmm0, %xmm1
        vpsrldq $8, %xmm0, %xmm0
        vpmovzxwd       %xmm0, %xmm0
        movl    %eax, b+32(%rip)
        movzwl  a+18(%rip), %eax
        vmovaps %xmm1, b(%rip)
        vmovaps %xmm0, b+16(%rip)
        movl    %eax, b+36(%rip)
        movzwl  a+20(%rip), %eax
        movl    %eax, b+40(%rip)
        movzwl  a+22(%rip), %eax
        movl    %eax, b+44(%rip)
        ret

while 4.8 and mainline unrolls and keeps it that way:

t:
.LFB0:
        .cfi_startproc
        movzwl  a(%rip), %eax
        movl    %eax, b(%rip)
        movzwl  a+2(%rip), %eax
        movl    %eax, b+4(%rip)
        movzwl  a+4(%rip), %eax
        movl    %eax, b+8(%rip)
        movzwl  a+6(%rip), %eax
        movl    %eax, b+12(%rip)
        movzwl  a+8(%rip), %eax
        movl    %eax, b+16(%rip)
        movzwl  a+10(%rip), %eax
        movl    %eax, b+20(%rip)
        movzwl  a+12(%rip), %eax
        movl    %eax, b+24(%rip)
        movzwl  a+14(%rip), %eax
        movl    %eax, b+28(%rip)
        movzwl  a+16(%rip), %eax
        movl    %eax, b+32(%rip)
        movzwl  a+18(%rip), %eax
        movl    %eax, b+36(%rip)
        movzwl  a+20(%rip), %eax
        movl    %eax, b+40(%rip)
        movzwl  a+22(%rip), %eax
        movl    %eax, b+44(%rip)
        ret

[Bug middle-end/64909] [4.8/5 regression] Missed vectorization

[hubicka at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64909

Jan Hubicka <hubicka at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |hubicka at gcc dot gnu.org,
                   |                            |tsaunders at mozilla dot com

--- Comment #1 from Jan Hubicka <hubicka at gcc dot gnu.org> ---
Forgot to mention, seen in the disassembly of Firefox's
AssignJSString<nsAutoJSString>(JSContext*, nsAutoJSString&, JSString*) :)

[Bug middle-end/64909] [4.8/5 regression] Missed vectorization

[hjl.tools at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64909

H.J. Lu <hjl.tools at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |WAITING
   Last reconfirmed|                            |2015-02-02
     Ever confirmed|0                           |1

--- Comment #2 from H.J. Lu <hjl.tools at gmail dot com> ---
Today's mainline generates:

    movzwl    a+16(%rip), %eax
    pxor    %xmm1, %xmm1
    movdqa    a(%rip), %xmm0
    movdqa    %xmm0, %xmm2
    movl    %eax, b+32(%rip)
    movzwl    a+18(%rip), %eax
    punpcklwd    %xmm1, %xmm2
    punpckhwd    %xmm1, %xmm0
    movl    %eax, b+36(%rip)
    movzwl    a+20(%rip), %eax
    movaps    %xmm2, b(%rip)
    movaps    %xmm0, b+16(%rip)
    movl    %eax, b+40(%rip)
    movzwl    a+22(%rip), %eax
    movl    %eax, b+44(%rip)
    ret

at -O3.

[Bug middle-end/64909] [4.8/5 regression] Missed vectorization

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64909

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |5.0

[Bug middle-end/64909] [4.8/5 regression] Missed vectorization

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64909

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jakub at gcc dot gnu.org,
                   |                            |kyukhin at gcc dot gnu.org
             Blocks|                            |53947

--- Comment #3 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
I'm with H.J. here, can't reproduce any kind of code you are showing, the loop
is normally vectorized.

But, what can we see there is that for e.g. -O3 -mavx we choose vectorization
factor of 8, which is based on the fact that there are 16-bit and 32-bit types
used in the loop, and before AVX2 we can mostly use V4SImode and V8HImode.

Compared to that, clang vectorizes it probably with vectorization factor 4
instead of 8, and as the loop has constant 12 iterations, doing it that way is
beneficial.

So, perhaps the question is why slp after cunroll hasn't vectorized the
unrolled scalar tail loop with vectorization factor 4.
pr64909.c:8:11: note: not vectorized: not enough data-refs in basic block.
Although it is true that for HImode we indeed can't fill the V8HImode, it is
only used immediately in an extension, which normally looks like:
  vect__4.7_30 = MEM[(short unsigned int *)vectp_a.6_27];
  vect__5.8_31 = [vec_unpack_lo_expr] vect__4.7_30;
  vect__5.8_32 = [vec_unpack_hi_expr] vect__4.7_30;
so all we'd need is the ability to emit a V4HImode load followed solely by
vec_unpack_lo_expr from it instead of both vec_unpack_{lo,hi}_expr.

[Bug middle-end/64909] [4.8/5 regression] Missed vectorization

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64909

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |missed-optimization
                 CC|                            |rguenth at gcc dot gnu.org

--- Comment #4 from Richard Biener <rguenth at gcc dot gnu.org> ---
I think that SLP doesn't consider this kind of operation (in loops it wouldn't
help and the non-loop case has not many cases that exceed those handled by the
loop code).

Any standalone testcase only showing the BB SLP opportunity?  That is, non-loop
code from the start?

[Bug middle-end/64909] [4.8/5 regression] Missed vectorization

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64909

--- Comment #5 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
E.g.
unsigned short a[32];
unsigned int b[32];

void
foo ()
{
  b[0] = a[0];
  b[1] = a[1];
  b[2] = a[2];
  b[3] = a[3];
}

will do, or b[0] = a[0] + 5; b[1] = a[1] + 5; b[2] = a[2] + 5; b[3] = a[3] + 5;
and similar.

[Bug middle-end/64909] [4.8/5 regression] Missed vectorization

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64909

--- Comment #6 from Richard Biener <rguenth at gcc dot gnu.org> ---
(In reply to Jakub Jelinek from comment #5)
> E.g.
> unsigned short a[32];
> unsigned int b[32];
> 
> void
> foo ()
> {
>   b[0] = a[0];
>   b[1] = a[1];
>   b[2] = a[2];
>   b[3] = a[3];
> }
> 
> will do, or b[0] = a[0] + 5; b[1] = a[1] + 5; b[2] = a[2] + 5; b[3] = a[3] +
> 5;
> and similar.

So it first of all boils down to the vectorizer not handling vectors of
different
size (V4HI and V4SI) at the same time.  The same thing happens when you
vectorize
double<->int conversion (for which there is already a bug somewhere).  For loop
vectorization this usually causes unnecessary unrolling (though that may be
profitable in the end).

[Bug middle-end/64909] Missed vectorization

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64909

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|5.0                         |6.0
            Summary|[4.8/5 regression] Missed   |Missed vectorization
                   |vectorization               |

--- Comment #7 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
In any case, I don't see this as a regression.

[Bug middle-end/64909] Missed vectorization

[hubicka at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64909

--- Comment #8 from Jan Hubicka <hubicka at gcc dot gnu.org> ---
Aha, reproduces for me with -O3 -march=bdver1; -march=core2 vectorizes.

[Bug target/64909] [4.8/4.9/5 Regression] Missed vectorization with bdver1

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64909

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Target|                            |x86_64-*-*, i?86-*-*
           Priority|P3                          |P2
             Status|WAITING                     |NEW
          Component|middle-end                  |target
      Known to work|                            |4.7.4
   Target Milestone|6.0                         |4.8.5
            Summary|Missed vectorization        |[4.8/4.9/5 Regression]
                   |                            |Missed vectorization with
                   |                            |bdver1
      Known to fail|                            |4.8.3, 4.9.2, 5.0

--- Comment #9 from Richard Biener <rguenth at gcc dot gnu.org> ---
Confirmed with -march=bdver1.  A cost model issue:

t.c:6:3: note: Cost model analysis:
  Vector inside of loop cost: 24
  Vector prologue cost: 0
  Vector epilogue cost: 336
  Scalar iteration cost: 14
  Scalar outside cost: 0
  Vector outside cost: 336
  prologue iterations: 0
  epilogue iterations: 4
  Calculated minimum iters for profitability: 30
t.c:6:3: note:   Runtime profitability threshold = 29
t.c:6:3: note:   Static estimate profitability threshold = 29
t.c:6:3: note: not vectorized: vectorization not profitable.

the vector epilogue cost looks odd to me.  Without -march=bdver1 I get 12.

[Bug target/64909] [4.8/4.9/5 Regression] Missed vectorization with bdver1

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64909

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
           Assignee|unassigned at gcc dot gnu.org      |rguenth at gcc dot gnu.org

--- Comment #10 from Richard Biener <rguenth at gcc dot gnu.org> ---
We call

Breakpoint 6, ix86_add_stmt_cost (data=0x21f34b0, count=56, kind=scalar_stmt, 
    stmt_info=0x0, misalign=0, where=vect_epilogue)

which is because we estimate epilogue cost to # of peeled iterations times
scalar iteration cost (4 * 14).  The way we do that,

  if (*peel_iters_epilogue)
    retval += record_stmt_cost (epilogue_cost_vec,
                                *peel_iters_epilogue * scalar_single_iter_cost,
                                scalar_stmt, NULL, 0, vect_epilogue);

is slightly off (we record 4 * 14 scalar stmts but for that we'd need to
use scalar_single_iter_num_stmts, not their cost, but well - scalar stmt cost
is
1 even for bdver1).

So the issue is that scalar iteration cost is somehow very high for bdver1 (14)
compared to generic (3).  It looks like bdver1 uses scaled costs (not based on
1):

  6,                                    /* scalar_stmt_cost.  */
  4,                                    /* scalar load_cost.  */
  4,                                    /* scalar_store_cost.  */
  6,                                    /* vec_stmt_cost.  */
  0,                                    /* vec_to_scalar_cost.  */
  2,                                    /* scalar_to_vec_cost.  */
  4,                                    /* vec_align_load_cost.  */
  4,                                    /* vec_unalign_load_cost.  */
  4,                                    /* vec_store_cost.  */
  2,                                    /* cond_taken_branch_cost.  */
  1,                                    /* cond_not_taken_branch_cost.  

and thus runs into the aforementioned issue.

[Bug tree-optimization/64909] [4.8/4.9/5 Regression] Missed vectorization with bdver1

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64909

--- Comment #11 from Richard Biener <rguenth at gcc dot gnu.org> ---
Author: rguenth
Date: Tue Feb 10 11:48:47 2015
New Revision: 220580

URL: https://gcc.gnu.org/viewcvs?rev=220580&root=gcc&view=rev
Log:
2015-02-10  Richard Biener  <rguenther@suse.de>

    PR tree-optimization/64909
    * tree-vect-loop.c (vect_estimate_min_profitable_iters): Properly
    pass a scalar-stmt count estimate to the cost model.
    * tree-vect-data-refs.c (vect_peeling_hash_get_lowest_cost): Likewise.

    * gcc.dg/vect/costmodel/x86_64/costmodel-pr64909.c: New testcase.

Added:
    trunk/gcc/testsuite/gcc.dg/vect/costmodel/x86_64/costmodel-pr64909.c
Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/testsuite/ChangeLog
    trunk/gcc/tree-vect-data-refs.c
    trunk/gcc/tree-vect-loop.c

[Bug tree-optimization/64909] [4.8/4.9 Regression] Missed vectorization with bdver1

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64909

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
      Known to work|                            |5.0
            Summary|[4.8/4.9/5 Regression]      |[4.8/4.9 Regression] Missed
                   |Missed vectorization with   |vectorization with bdver1
                   |bdver1                      |
      Known to fail|5.0                         |
           Severity|enhancement                 |normal

--- Comment #12 from Richard Biener <rguenth at gcc dot gnu.org> ---
Fixed on trunk sofar (for the SLP case please open another bugreport or amend
the existing double <-> int convert one).

[Bug tree-optimization/64909] [4.8/4.9 Regression] Missed vectorization with bdver1

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64909

--- Comment #13 from Richard Biener <rguenth at gcc dot gnu.org> ---
Author: rguenth
Date: Mon Feb 23 11:14:25 2015
New Revision: 220912

URL: https://gcc.gnu.org/viewcvs?rev=220912&root=gcc&view=rev
Log:
2015-02-23  Richard Biener  <rguenther@suse.de>

    Backport from mainline
    2014-11-19  Richard Biener  <rguenther@suse.de>

    PR tree-optimization/63844
    * omp-low.c (fixup_child_record_type): Use a restrict qualified
    referece type for the receiver parameter.

    2014-11-27  Richard Biener  <rguenther@suse.de>

    PR tree-optimization/61634
    * tree-vect-slp.c: Include gimple-walk.h.
    (vect_detect_hybrid_slp_stmts): Rewrite to propagate hybrid
    down the SLP tree for one scalar statement.
    (vect_detect_hybrid_slp_1): New walker function.
    (vect_detect_hybrid_slp_2): Likewise.
    (vect_detect_hybrid_slp): Properly handle pattern statements
    in a pre-scan over all loop stmts.

    * gcc.dg/vect/pr61634.c: New testcase.

    2015-01-14  Richard Biener  <rguenther@suse.de>

    PR tree-optimization/59354
    * tree-vect-slp.c (vect_build_slp_tree_1): Treat loads from
    groups larger than the slp group size as having gaps.

    * gcc.dg/vect/pr59354.c: New testcase.

    2015-02-10  Richard Biener  <rguenther@suse.de>

    PR tree-optimization/64909
    * tree-vect-loop.c (vect_estimate_min_profitable_iters): Properly
    pass a scalar-stmt count estimate to the cost model.
    * tree-vect-data-refs.c (vect_peeling_hash_get_lowest_cost): Likewise.

    * gcc.dg/vect/costmodel/x86_64/costmodel-pr64909.c: New testcase.

Added:
   
branches/gcc-4_9-branch/gcc/testsuite/gcc.dg/vect/costmodel/x86_64/costmodel-pr64909.c
    branches/gcc-4_9-branch/gcc/testsuite/gcc.dg/vect/pr59354.c
    branches/gcc-4_9-branch/gcc/testsuite/gcc.dg/vect/pr61634.c
Modified:
    branches/gcc-4_9-branch/gcc/ChangeLog
    branches/gcc-4_9-branch/gcc/omp-low.c
    branches/gcc-4_9-branch/gcc/testsuite/ChangeLog
    branches/gcc-4_9-branch/gcc/tree-vect-data-refs.c
    branches/gcc-4_9-branch/gcc/tree-vect-loop.c
    branches/gcc-4_9-branch/gcc/tree-vect-slp.c

[Bug tree-optimization/64909] [4.8 Regression] Missed vectorization with bdver1

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64909

--- Comment #14 from Richard Biener <rguenth at gcc dot gnu.org> ---
Author: rguenth
Date: Sat Apr  4 10:47:08 2015
New Revision: 221866

URL: https://gcc.gnu.org/viewcvs?rev=221866&root=gcc&view=rev
Log:
2015-04-04  Richard Biener  <rguenther@suse.de>

    PR tree-optimization/64909
    PR tree-optimization/65660
    * tree-vectorizer.h (vect_get_known_peeling_cost): Adjust
    to take a cost vector for scalar iteration cost.
    (vect_get_single_scalar_iteration_cost): Likewise.
    * tree-vect-loop.c (vect_get_single_scalar_iteration_cost):
    Compute the scalar iteration cost into a cost vector.
    (vect_get_known_peeling_cost): Use the scalar cost vector to
    account for the cost of the peeled iterations.
    (vect_estimate_min_profitable_iters): Likewise.
    * tree-vect-data-refs.c (vect_peeling_hash_get_lowest_cost):
    Likewise.

Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/tree-vect-data-refs.c
    trunk/gcc/tree-vect-loop.c
    trunk/gcc/tree-vectorizer.h

[Bug tree-optimization/64909] [4.8 Regression] Missed vectorization with bdver1

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64909

--- Comment #15 from Richard Biener <rguenth at gcc dot gnu.org> ---
Author: rguenth
Date: Tue Apr  7 13:40:24 2015
New Revision: 221895

URL: https://gcc.gnu.org/viewcvs?rev=221895&root=gcc&view=rev
Log:
2015-04-07  Richard Biener  <rguenther@suse.de>

    Backport from mainline
    2015-04-04  Richard Biener  <rguenther@suse.de>

    PR tree-optimization/64909
    PR tree-optimization/65660
    * tree-vectorizer.h (vect_get_known_peeling_cost): Adjust
    to take a cost vector for scalar iteration cost.
    (vect_get_single_scalar_iteration_cost): Likewise.
    * tree-vect-loop.c (vect_get_single_scalar_iteration_cost):
    Compute the scalar iteration cost into a cost vector.
    (vect_get_known_peeling_cost): Use the scalar cost vector to
    account for the cost of the peeled iterations.
    (vect_estimate_min_profitable_iters): Likewise.
    * tree-vect-data-refs.c (vect_peeling_hash_get_lowest_cost):
    Likewise.

Modified:
    branches/gcc-4_9-branch/gcc/ChangeLog
    branches/gcc-4_9-branch/gcc/tree-vect-data-refs.c
    branches/gcc-4_9-branch/gcc/tree-vect-loop.c
    branches/gcc-4_9-branch/gcc/tree-vectorizer.h

[Bug tree-optimization/64909] [4.8 Regression] Missed vectorization with bdver1

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64909

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|---                         |FIXED
   Target Milestone|4.8.5                       |4.9.3
      Known to fail|                            |4.8.5

--- Comment #16 from Richard Biener <rguenth at gcc dot gnu.org> ---
Fixed for 4.9.3.