Andrew Stubbs wrote:
> I've got so confused trying to figure out this stuff and how it works 
> with different LLVM, different defaults, different devices.
> 
> I think this patch is fine, but we should wait until we can test it on 
> all those devices.

Well, we can test the compiling part without the need for any device.

I have now run the attached script and the result running yesterday's 
build with both my patch and your patch applied.


That's with Ubuntu 22's system LLVM 15 and readelf.

The "2>/dev/null" silences the warning:
"Missing knowledge of 32-bit reloc types used in DWARF sections of 
machine number 224"

It looks as if some cleanup is needed - both for consistency between the 
code generated for the '.s' file vs. the 'llvm-mc' ("as") command line 
and also "mkoffload.cc" does not seem to be fully correct, either.

* * *

RESULTS:

* Expected:

lto1: error: '-mxnack=on' is incompatible with '-march=fiji'
lto1: error: '-mxnack=on' is incompatible with '-march=gfx1030'
lto1: error: '-mxnack=on' is incompatible with '-march=gfx1100'


Compiler bugs (ASM_SPEC vs. gcn.cc), i.e. causing llvm-mc assember errors:

* gfx900 and gfx906 for
     -foffload-options=-mxnack=any
   or
     -foffload-options=-mxnack=on
as that sets
     ".amdhsa_reserve_xnack_mask    1"
but passes
     -mattr=-xnack


* For gfx908 and gfx90a, the SRAM argument is wrongly passed, e.g.,
      -mattr=-xnack-mattr=-sramecc
such that one gets the messages:
'-xnack-mattr=-sramecc' is not a recognized feature for this target
'-xnack-mattr=+sramecc' is not a recognized feature for this target
'-xnack-mattr=-sramecc' is not a recognized feature for this target
'-xnack-mattr=+sramecc' is not a recognized feature for this target
'-xnack-mattr=-sramecc' is not a recognized feature for this target
'-xnack-mattr=+sramecc' is not a recognized feature for this target
'-xnack-mattr=-sramecc' is not a recognized feature for this target
'-xnack-mattr=+sramecc' is not a recognized feature for this target
'-xnack-mattr=-sramecc' is not a recognized feature for this target
'-xnack-mattr=+sramecc' is not a recognized feature for this target
'+xnack-mattr=-sramecc' is not a recognized feature for this target
'+xnack-mattr=+sramecc' is not a recognized feature for this target


Now to my patch for mkoffload:

* For the debug-file generation, it fails with
   "incompatible sramecc"
(a) for gfx900:
    -foffload-options=-msram-ecc=off
    -foffload-options=-msram-ecc=any
    -foffload-options=-msram-ecc=on
those cause that SRAM ECC is set but it shouldn't:
   Flags:   0x22c  - a.xamdgcn-amdhsa.mkoffload.hsaco
   Flags:   0x62c  - a.xamdgcn-amdhsa.mkoffload.dbg0.o

=> FIX: unconditional use
     case EF_AMDGPU_MACH_AMDGCN_GFX900:
       SET_SRAM_ECC_UNSET (elf_flags);

(b) for gfx906, here is the reverse true:
   Flags:   0x62f  - a.xamdgcn-
   Flags:   0x22f  - a.xamdgcn-amdhsa.mkoffload.dbg0.o
Namely, it should set SRAMECC_ANY_V4 (0x400) independent of the passed 
-msram=... value.

=> FIX: unconditional use
     case EF_AMDGPU_MACH_AMDGCN_GFX906:
       SET_XNACK_ANY (elf_flags);

And for gfx908/gfx90a, I think it makes sense to fix first the compiler
issue; I think gfx90a should be fine, for gfx908 I am less sure.

Tobias