[Bug tree-optimization/110020] New: [13/14 Regression] SHA2 misscompilation at -O3

[Bug tree-optimization/110020] New: [13/14 Regression] SHA2 misscompilation at -O3

[chfast at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110020

            Bug ID: 110020
           Summary: [13/14 Regression] SHA2 misscompilation at -O3
           Product: gcc
           Version: 13.1.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: tree-optimization
          Assignee: unassigned at gcc dot gnu.org
          Reporter: chfast at gmail dot com
  Target Milestone: ---

This is a test case reduced from a C implementation of SHA256.

void test(unsigned h[8]) {
    for (unsigned i = 0; i < 2; i++) {

        unsigned w[16];
        for (unsigned j = 0; j < 16; j++) {
            if (i == 0)
                w[j] = 0;

            h[7] = h[6];
            h[6] = h[5];
            h[5] = h[4];
            h[4] = h[3];
            h[3] = h[2];
            h[2] = h[1];
            h[1] = h[0];
            h[0] += w[j];
        }
    }
}

It looks that at -O3 compiler looses track of w[j] = 0 and uses uninitialized
stack storage.

test:
        movl    -36(%rsp), %ecx
        movl    -68(%rsp), %eax
        movq    %rdi, %rdx
        movl    -32(%rsp), %esi
        addl    -72(%rsp), %eax
        addl    -64(%rsp), %eax
        addl    -60(%rsp), %eax
        addl    -56(%rsp), %eax
        addl    -52(%rsp), %eax
        addl    -48(%rsp), %eax
        addl    -44(%rsp), %eax
        addl    -40(%rsp), %eax
        addl    (%rdi), %eax
        addl    %eax, %ecx
        movl    -28(%rsp), %edi
        movl    -24(%rsp), %r8d
        movl    %eax, 28(%rdx)
        addl    %ecx, %esi
        movl    -20(%rsp), %r9d
        movl    -16(%rsp), %r10d
        movl    %ecx, 24(%rdx)
        addl    %esi, %edi
        movl    -12(%rsp), %r11d
        movl    %esi, 20(%rdx)
        addl    %edi, %r8d
        movl    %edi, 16(%rdx)
        addl    %r8d, %r9d
        movl    %r8d, 12(%rdx)
        addl    %r9d, %r10d
        movl    %r9d, 8(%rdx)
        addl    %r10d, %r11d
        movl    %r10d, 4(%rdx)
        movl    %r11d, (%rdx)
        ret 


https://godbolt.org/z/ff7E9sd94

[Bug tree-optimization/110020] [13/14 Regression] SHA2 misscompilation at -O3

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110020

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jakub at gcc dot gnu.org
         Resolution|---                         |INVALID
             Status|UNCONFIRMED                 |RESOLVED

--- Comment #1 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
This is clearly invalid testcase.
It initializes the w array with i == 0, then w goes out of scope, then it
reappears uninitialized in the scope with i == 1 and the uninitialized value is
used.
You'd need to move the w declaration before the outer loop to make it well
defined.

[Bug tree-optimization/110020] [13/14 Regression] SHA2 misscompilation at -O3

[chfast at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110020

--- Comment #2 from Paweł Bylica <chfast at gmail dot com> ---
Yes, you are right. Sorry for taking your time.