[Bug c/95666] New: gcc 9.3.0 generates incorrect code with -O3 on HP-UX IA64

[Bug c/95666] New: gcc 9.3.0 generates incorrect code with -O3 on HP-UX IA64

[aerofeev at commvault dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95666

            Bug ID: 95666
           Summary: gcc 9.3.0 generates incorrect code with -O3 on HP-UX
                    IA64
           Product: gcc
           Version: 9.3.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c
          Assignee: unassigned at gcc dot gnu.org
          Reporter: aerofeev at commvault dot com
  Target Milestone: ---

Created attachment 48726
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=48726&action=edit
Source code, assembly and compiled test program

Hi,

While upgrading our compilers from gcc-4.9.3 to gcc-9.3.0 we have found that
openssl started crashing with the following stack:

#0  0x40000000002bbf00:1 in x509_name_ex_d2i+0x641 ()
#1  0x400000000026bd40:0 in ASN1_item_ex_d2i+0x8c0 ()
#2  0x400000000026eea0:0 in asn1_template_noexp_d2i+0xd00 ()
#3  0x400000000026e110:0 in asn1_template_ex_d2i+0x590 ()
#4  0x400000000026d340:0 in ASN1_item_ex_d2i+0x1ec0 ()
#5  0x400000000026eea0:0 in asn1_template_noexp_d2i+0xd00 ()
#6  0x400000000026e110:0 in asn1_template_ex_d2i+0x590 ()
#7  0x400000000026d340:0 in ASN1_item_ex_d2i+0x1ec0 ()
#8  0x400000000026b330:0 in ASN1_item_d2i+0x110 ()
#9  0x40000000002c14c0:0 in d2i_X509+0x80 ()
#10 0x40000000003946b0:0 in PEM_ASN1_read_bio+0xf0 ()
#11 0x40000000001f9610:0 in PEM_read_bio_X509+0x50 ()

The stack by itself is not very relevant to this report, I'm mentioning it here
only in case if someone else runs into the same issue and tries googling the
stack.

The real issue lies in another file, stack.c where a function called
sk_insert() corrupts OpenSSL's struct STACK by inserting a new element into
STACK::data[] array at incorrect location. The corruption is caused by gcc
producing incorrect assembly for the body of sk_insert() with -O3.

The purpose of sk_insert(struct STACK *st, void *data, int loc) is to insert a
new element "data" into st->data[] array at the specified location "loc". If
necessary, sk_insert() can reallocate st->data[] and can shift elements in the
array to free up the space for the new element, but all of that code is not
what triggers the bug. It's the simplest insertion of a new element into empty
preallocated array at location 0 where things break, and the element ends up
being inserted at location 1 instead.

Regretfully, I couldn't trim the reallocation piece and the piece that moves
existing elements from the function, because without them the assembly changes,
and the bug disappears.

I have extracted sk_insert() into a separate file stack.c (attached to this
report), removed some of the fat and constructed a test case demonstrating how
the code puts first element at location st->data[1] instead of location
st->data[0].

If compiled with -O2, the test program will produce a single "OK" word. If
compiled with -O3, the post-insertion checks yield these messages:

Bad st.data[0]=0000000000000000 (87fffffffffff228 expected)
Bad st.data[1]=87fffffffffff228 (0 expected)

I have reviewed the assembly code and found that at line 101 r41 is incorrectly
incremented after having been set to st->num at line 88. Later at line 133 the
value of r41 is assigned to r15. r15 is then multiplied by 8 (size of a single
element in st->data[]) and is added to st->data. Finally the new element is
assigned to the location pointed to by r15 at line 152. Net result is that
instead of storing new element at location st->data[0], the code stores it at
location st->data[0+1].

Like I've said, the issue disappears if the code is compiled with -O2. I am not
confident however that it's related to the optimization level at all. Any
change to sk_insert(), sometimes as subtle as a simple printf() or removal of
loc<0 alters assembly enough to make the bug go away.

Please advise on whether this is something known and what other information I
can provide to help troubleshoot the problem.