[Bug target/109933] New: __atomic_test_and_set is broken for BIG ENDIAN riscv targets

["rory.bolt at gmail dot com" <gcc-bugzilla@gcc.gnu.org>]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109933

            Bug ID: 109933
           Summary: __atomic_test_and_set is broken for BIG ENDIAN riscv
                    targets
           Product: gcc
           Version: 12.2.1
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: target
          Assignee: unassigned at gcc dot gnu.org
          Reporter: rory.bolt at gmail dot com
  Target Milestone: ---
            Target: riscv64be-unknown-linux-gnu

Created attachment 55136
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=55136&action=edit
Simple test program illustrating the bug

PLEASE NOTE: this bug is in reference to BIG ENDIAN riscv targets...

Although reported on riscv64be, it most likely exists for 32 bit targets too.

Please see the attached test program.

The issue appears to be that the value atomically or'ed should be 0x01000000
not 1 for big endian systems, since we want the non-zero value in the lowest
memory address and the amoor.w command is writing a 32 bit value to memory.

The current (broken) implementation will pass a simple unit test, the problem
only manifests itself on a call to __atomic_test_and_set AFTER a previous call
to __atomic_test_and_set to the same address and a __atomic_clear to the same
address.

This problem originally manifested itself in the linux libcap library, which
uses __atomic_test_and_set in conjunction with __atomic_clear to implement a
mutex to guard the manipulation of capabilities.