[Bug d/96153] d: struct literals have non-deterministic hash values

["ibuclaw at gdcproject dot org" <gcc-bugzilla@gcc.gnu.org>]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96153

--- Comment #4 from Iain Buclaw <ibuclaw at gdcproject dot org> ---
Created attachment 49006
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=49006&action=edit
inline memset

(In reply to Iain Buclaw from comment #3)
> (In reply to Rainer Orth from comment #2)
> > The new test FAILs on 64-bit Solaris/SPARC:
> > 
> > The test PASSes with -O0 and -Os, as well as with -m32.
> 
> Thanks, I've checked Linux/SPARC64 and see the same thing.

DSE is removing the call to memset as dead code, so the optimizer is not
understanding the hint.

Inlining the memset using a bunch of MEM_REFs yields the correct result, and
infact the optimizer is able to fully const-fold the hash computation at -O3 so
all assert contracts are removed.

Though not sure about the approach, are there any pitfalls to this?

This is what optimized trees look like at -O1 and -O2 levels, with comments
inserted describing which bits are set.

x86_64 -O1:
  // 0..128 = 0
  MEM <uint128_t *> [(uint128_t *)&D.4244] = 0B;
  // 128..192 = 0
  MEM[(ulong *)&D.4244 + 16B] = 0B;
  // 0..32 = 12
  D.4244.payload = 12;
  // 128..160 = 4294967295
  D.4244.hook.var2 = 4294967295;
  // 64..128 = 18446744073709551615
  D.4244.hook.var1 = 18446744073709551615;

x86_64 -O2:
  // 0..64 = 12
  MEM <unsigned long> [(void *)&D.4244] = 12;
  // 64..128 = 18446744073709551615
  D.4244.hook.var1 = 18446744073709551615;
  // 128..192 = 4294967295
  MEM <unsigned long> [(void *)&D.4244 + 16B] = 4294967295;

SPARC64 -O1:
  // 0..128 = 51539607552
  MEM <uint128_t *> [(void *)&D.1048] = 51539607552B;
  // 64..128 = 18446744073709551615
  MEM <ulong> [(void *)&D.1048 + 8B] = 18446744073709551615;
  // 128..192 = -4294967296
  MEM <ulong *> [(void *)&D.1048 + 16B] = -4294967296B;

SPARC -O2
  // 0..128 = 51539607552
  MEM <uint128_t *> [(void *)&D.1048] = 51539607552B;
  // 64..128 = 18446744073709551615
  MEM <ulong> [(void *)&D.1048 + 8B] = 18446744073709551615;
  // 128..192 = -4294967296
  MEM <ulong *> [(void *)&D.1048 + 16B] = -4294967296B;


Both SPARC and x86_64 yield the same hash values.

  assert(-1444610504 != -1444610503);
  assert(-1444610503 == -1444610503);