Issue with pointer types marked with scalar_storage_order

[Ulrich Weigand <uweigand@de.ibm.com>]

Hello,

for a few years now, GCC has supported the scalar_storage_order
attribute (and pragma) that allows specifying the storage order
(endianness) of structures.  This affects both the code GCC
generates to access variables declared using the attribute,
and also the debug info: GCC emits the DW_AT_endianity attribute
to allow debuggers to work correctly with those variables as well.

However, in one specific scenario this doesn't work correctly
right now: pointer types.  Currently, GCC *will* swap the storage
order for members of pointer type in a structure declared using
scalar_storage_order, but it will *not* emit DW_AT_endianity
for these fields -- and it really cannot, because the DWARF
standard allows DW_AT_endianity only for DW_TAG_base_type
type entries and not for DW_TAG_pointer_type entries.

I'm not really sure where exactly the bug is, because I'm not
quite sure if pointer types actually *should* be byte swapped.

On the one hand, the typical use case of scalar_storage_order
is to simplify accessing binary data (read from a file or the
network) that was generated on a "foreign" architecture that
uses a different byte order.  Those use cases are unlikely
to involve any pointer types, since pointer values from a
foreign system are typically not usable on the current
system anyway.

On the other hand, even the name of the attribute specifically
refers to *scalar* types, and the C standard does classsify
pointer types amongst the scalar type.  So maybe this was
originally intended?

If we do want to byte-swap pointer types, then I guess we need
to still fix the debug info problem, which I guess would at a
minimum require extending DWARF to allow DW_AT_endianity as an
attribute to DW_TAG_pointer_type (and then probably also
DW_TAG_reference_type, DW_TAG_rvalue_reference_type,
DW_TAG_ptr_to_member_type and possibly others).  Also, the
implementation in GDB would need to be changed accordingly.

Any comments or suggestions on what to do here?

Thanks,
Ulrich

-- 
  Dr. Ulrich Weigand
  GNU/Linux compilers and toolchain
  Ulrich.Weigand@de.ibm.com