[Bug tree-optimization/43197] New: Endianness and Optimization

[Bug tree-optimization/43197] New: Endianness and Optimization

[kai dot extern at googlemail dot com]

The attached code (which tries to generically load given-endianness values of
varying width from memory) shows some interesting optimization quirks. It's
especially pussling why optimization quality varies so greatly with width, and
is actually worst for the native word width.

For example:

$ gcc -Wall -Wextra bad.cc -lstdc++ -O3 -###
Using built-in specs.
Target: x86_64-linux-gnu                
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 4.4.1-4ubuntu9'
--with-bugurl=file:///usr/share/doc/gcc-4.4/README.Bugs
--enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --enable-shared
--enable-multiarch --enable-linker-build-id --with-system-zlib
--libexecdir=/usr/lib --without-included-gettext --enable-threads=posix
--with-gxx-include-dir=/usr/include/c++/4.4 --program-suffix=-4.4 --enable-nls
--enable-clocale=gnu --enable-libstdcxx-debug --enable-objc-gc --disable-werror
--with-arch-32=i486 --with-tune=generic --enable-checking=release
--build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.4.1 (Ubuntu 4.4.1-4ubuntu9) 
COLLECT_GCC_OPTIONS='-Wall' '-Wextra' '-O3' '-mtune=generic'
 "/usr/lib/gcc/x86_64-linux-gnu/4.4.1/cc1plus" "-quiet" "-D_GNU_SOURCE"
"bad.cc" "-D_FORTIFY_SOURCE=2" "-quiet" "-dumpbase" "bad.cc" "-mtune=generic"
"-auxbase" "bad" "-O3" "-Wall" "-Wextra" "-fstack-protector" "-o"
"/tmp/ccnHgEb9.s"
COLLECT_GCC_OPTIONS='-Wall' '-Wextra' '-O3' '-mtune=generic'
 "as" "-Qy" "-o" "/tmp/cceJWVC8.o" "/tmp/ccnHgEb9.s"
COMPILER_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.4.1/:/usr/lib/gcc/x86_64-linux-gnu/4.4.1/:/usr/lib/gcc/x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/4.4.1/:/usr/lib/gcc/x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/4.4.1/:/usr/lib/gcc/x86_64-linux-gnu/
LIBRARY_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.4.1/:/usr/lib/gcc/x86_64-linux-gnu/4.4.1/:/usr/lib/gcc/x86_64-linux-gnu/4.4.1/../../../../lib/:/lib/../lib/:/usr/lib/../lib/:/usr/lib/gcc/x86_64-linux-gnu/4.4.1/../../../:/lib/:/usr/lib/:/usr/lib/x86_64-linux-gnu/
COLLECT_GCC_OPTIONS='-Wall' '-Wextra' '-O3' '-mtune=generic'
 "/usr/lib/gcc/x86_64-linux-gnu/4.4.1/collect2" "--build-id" "--eh-frame-hdr"
"-m" "elf_x86_64" "--hash-style=both" "-dynamic-linker"
"/lib64/ld-linux-x86-64.so.2" "-z" "relro"
"/usr/lib/gcc/x86_64-linux-gnu/4.4.1/../../../../lib/crt1.o"
"/usr/lib/gcc/x86_64-linux-gnu/4.4.1/../../../../lib/crti.o"
"/usr/lib/gcc/x86_64-linux-gnu/4.4.1/crtbegin.o"
"-L/usr/lib/gcc/x86_64-linux-gnu/4.4.1" "-L/usr/lib/gcc/x86_64-linux-gnu/4.4.1"
"-L/usr/lib/gcc/x86_64-linux-gnu/4.4.1/../../../../lib" "-L/lib/../lib"
"-L/usr/lib/../lib" "-L/usr/lib/gcc/x86_64-linux-gnu/4.4.1/../../.."
"-L/usr/lib/x86_64-linux-gnu" "/tmp/cceJWVC8.o" "-lstdc++" "-lgcc"
"--as-needed" "-lgcc_s" "--no-as-needed" "-lc" "-lgcc" "--as-needed" "-lgcc_s"
"--no-as-needed" "/usr/lib/gcc/x86_64-linux-gnu/4.4.1/crtend.o"
"/usr/lib/gcc/x86_64-linux-gnu/4.4.1/../../../../lib/crtn.o"
$ objdump -d a.out | c++filt | sed -n '/Test_/,/constructors/ p' > out

(See attachments for source and output.)


-- 
           Summary: Endianness and Optimization
           Product: gcc
           Version: 4.4.1
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: tree-optimization
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: kai dot extern at googlemail dot com
 GCC build triplet: x86_64-linux-gnu
  GCC host triplet: x86_64-linux-gnu
GCC target triplet: x86_64-linux-gnu


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43197

[Bug tree-optimization/43197] Endianness and Optimization

[kai dot extern at googlemail dot com]

------- Comment #1 from kai dot extern at googlemail dot com  2010-02-27 10:59 -------
Created an attachment (id=19974)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=19974&action=view)
C++ Source

The source file to demonstrate the problem


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43197

[Bug tree-optimization/43197] Endianness and Optimization

[kai dot extern at googlemail dot com]

------- Comment #2 from kai dot extern at googlemail dot com  2010-02-27 11:01 -------
Created an attachment (id=19975)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=19975&action=view)
Disassembled output

The results from compiling the source 


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43197

Re: [Bug tree-optimization/43197] New: Endianness and Optimization

[Andrew Pinski]

Sent from my iPhone

On Feb 27, 2010, at 2:56 AM, "kai dot extern at googlemail dot com" <gcc-bugzilla@gcc.gnu.org 
 > wrote:

> The attached code (which tries to generically load given-endianness  
> values of
> varying width from memory) shows some interesting optimization  
> quirks. It's
> especially pussling why optimization quality varies so greatly with  
> width, and
> is actually worst for the native word width.


You are violating c++ aliasing rules. You access a uint8_t via  
different types.


>
> For example:
>
> $ gcc -Wall -Wextra bad.cc -lstdc++ -O3 -###
> Using built-in specs.
> Target: x86_64-linux-gnu
> Configured with: ../src/configure -v --with-pkgversion='Ubuntu  
> 4.4.1-4ubuntu9'
> --with-bugurl=file:///usr/share/doc/gcc-4.4/README.Bugs
> --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --enable- 
> shared
> --enable-multiarch --enable-linker-build-id --with-system-zlib
> --libexecdir=/usr/lib --without-included-gettext --enable- 
> threads=posix
> --with-gxx-include-dir=/usr/include/c++/4.4 --program-suffix=-4.4 -- 
> enable-nls
> --enable-clocale=gnu --enable-libstdcxx-debug --enable-objc-gc -- 
> disable-werror
> --with-arch-32=i486 --with-tune=generic --enable-checking=release
> --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64- 
> linux-gnu
> Thread model: posix
> gcc version 4.4.1 (Ubuntu 4.4.1-4ubuntu9)
> COLLECT_GCC_OPTIONS='-Wall' '-Wextra' '-O3' '-mtune=generic'
> "/usr/lib/gcc/x86_64-linux-gnu/4.4.1/cc1plus" "-quiet" "-D_GNU_SOURCE"
> "bad.cc" "-D_FORTIFY_SOURCE=2" "-quiet" "-dumpbase" "bad.cc" "- 
> mtune=generic"
> "-auxbase" "bad" "-O3" "-Wall" "-Wextra" "-fstack-protector" "-o"
> "/tmp/ccnHgEb9.s"
> COLLECT_GCC_OPTIONS='-Wall' '-Wextra' '-O3' '-mtune=generic'
> "as" "-Qy" "-o" "/tmp/cceJWVC8.o" "/tmp/ccnHgEb9.s"
> COMPILER_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.4.1/:/usr/lib/gcc/ 
> x86_64-linux-gnu/4.4.1/:/usr/lib/gcc/x86_64-linux-gnu/:/usr/lib/gcc/ 
> x86_64-linux-gnu/4.4.1/:/usr/lib/gcc/x86_64-linux-gnu/:/usr/lib/gcc/ 
> x86_64-linux-gnu/4.4.1/:/usr/lib/gcc/x86_64-linux-gnu/
> LIBRARY_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.4.1/:/usr/lib/gcc/ 
> x86_64-linux-gnu/4.4.1/:/usr/lib/gcc/x86_64-linux-gnu/ 
> 4.4.1/../../../../lib/:/lib/../lib/:/usr/lib/../lib/:/usr/lib/gcc/ 
> x86_64-linux-gnu/4.4.1/../../../:/lib/:/usr/lib/:/usr/lib/x86_64- 
> linux-gnu/
> COLLECT_GCC_OPTIONS='-Wall' '-Wextra' '-O3' '-mtune=generic'
> "/usr/lib/gcc/x86_64-linux-gnu/4.4.1/collect2" "--build-id" "--eh- 
> frame-hdr"
> "-m" "elf_x86_64" "--hash-style=both" "-dynamic-linker"
> "/lib64/ld-linux-x86-64.so.2" "-z" "relro"
> "/usr/lib/gcc/x86_64-linux-gnu/4.4.1/../../../../lib/crt1.o"
> "/usr/lib/gcc/x86_64-linux-gnu/4.4.1/../../../../lib/crti.o"
> "/usr/lib/gcc/x86_64-linux-gnu/4.4.1/crtbegin.o"
> "-L/usr/lib/gcc/x86_64-linux-gnu/4.4.1" "-L/usr/lib/gcc/x86_64-linux- 
> gnu/4.4.1"
> "-L/usr/lib/gcc/x86_64-linux-gnu/4.4.1/../../../../lib" "-L/lib/../ 
> lib"
> "-L/usr/lib/../lib" "-L/usr/lib/gcc/x86_64-linux-gnu/4.4.1/../../.."
> "-L/usr/lib/x86_64-linux-gnu" "/tmp/cceJWVC8.o" "-lstdc++" "-lgcc"
> "--as-needed" "-lgcc_s" "--no-as-needed" "-lc" "-lgcc" "--as-needed"  
> "-lgcc_s"
> "--no-as-needed" "/usr/lib/gcc/x86_64-linux-gnu/4.4.1/crtend.o"
> "/usr/lib/gcc/x86_64-linux-gnu/4.4.1/../../../../lib/crtn.o"
> $ objdump -d a.out | c++filt | sed -n '/Test_/,/constructors/ p' > out
>
> (See attachments for source and output.)
>
>
> -- 
>           Summary: Endianness and Optimization
>           Product: gcc
>           Version: 4.4.1
>            Status: UNCONFIRMED
>          Severity: normal
>          Priority: P3
>         Component: tree-optimization
>        AssignedTo: unassigned at gcc dot gnu dot org
>        ReportedBy: kai dot extern at googlemail dot com
> GCC build triplet: x86_64-linux-gnu
>  GCC host triplet: x86_64-linux-gnu
> GCC target triplet: x86_64-linux-gnu
>
>
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43197
>

[Bug tree-optimization/43197] Endianness and Optimization

[pinskia at gmail dot com]

------- Comment #3 from pinskia at gmail dot com  2010-02-27 13:06 -------
Subject: Re:   New: Endianness and Optimization



Sent from my iPhone

On Feb 27, 2010, at 2:56 AM, "kai dot extern at googlemail dot com"
<gcc-bugzilla@gcc.gnu.org 
 > wrote:

> The attached code (which tries to generically load given-endianness  
> values of
> varying width from memory) shows some interesting optimization  
> quirks. It's
> especially pussling why optimization quality varies so greatly with  
> width, and
> is actually worst for the native word width.


You are violating c++ aliasing rules. You access a uint8_t via  
different types.


>
> For example:
>
> $ gcc -Wall -Wextra bad.cc -lstdc++ -O3 -###
> Using built-in specs.
> Target: x86_64-linux-gnu
> Configured with: ../src/configure -v --with-pkgversion='Ubuntu  
> 4.4.1-4ubuntu9'
> --with-bugurl=file:///usr/share/doc/gcc-4.4/README.Bugs
> --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --enable- 
> shared
> --enable-multiarch --enable-linker-build-id --with-system-zlib
> --libexecdir=/usr/lib --without-included-gettext --enable- 
> threads=posix
> --with-gxx-include-dir=/usr/include/c++/4.4 --program-suffix=-4.4 -- 
> enable-nls
> --enable-clocale=gnu --enable-libstdcxx-debug --enable-objc-gc -- 
> disable-werror
> --with-arch-32=i486 --with-tune=generic --enable-checking=release
> --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64- 
> linux-gnu
> Thread model: posix
> gcc version 4.4.1 (Ubuntu 4.4.1-4ubuntu9)
> COLLECT_GCC_OPTIONS='-Wall' '-Wextra' '-O3' '-mtune=generic'
> "/usr/lib/gcc/x86_64-linux-gnu/4.4.1/cc1plus" "-quiet" "-D_GNU_SOURCE"
> "bad.cc" "-D_FORTIFY_SOURCE=2" "-quiet" "-dumpbase" "bad.cc" "- 
> mtune=generic"
> "-auxbase" "bad" "-O3" "-Wall" "-Wextra" "-fstack-protector" "-o"
> "/tmp/ccnHgEb9.s"
> COLLECT_GCC_OPTIONS='-Wall' '-Wextra' '-O3' '-mtune=generic'
> "as" "-Qy" "-o" "/tmp/cceJWVC8.o" "/tmp/ccnHgEb9.s"
> COMPILER_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.4.1/:/usr/lib/gcc/ 
> x86_64-linux-gnu/4.4.1/:/usr/lib/gcc/x86_64-linux-gnu/:/usr/lib/gcc/ 
> x86_64-linux-gnu/4.4.1/:/usr/lib/gcc/x86_64-linux-gnu/:/usr/lib/gcc/ 
> x86_64-linux-gnu/4.4.1/:/usr/lib/gcc/x86_64-linux-gnu/
> LIBRARY_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.4.1/:/usr/lib/gcc/ 
> x86_64-linux-gnu/4.4.1/:/usr/lib/gcc/x86_64-linux-gnu/ 
> 4.4.1/../../../../lib/:/lib/../lib/:/usr/lib/../lib/:/usr/lib/gcc/ 
> x86_64-linux-gnu/4.4.1/../../../:/lib/:/usr/lib/:/usr/lib/x86_64- 
> linux-gnu/
> COLLECT_GCC_OPTIONS='-Wall' '-Wextra' '-O3' '-mtune=generic'
> "/usr/lib/gcc/x86_64-linux-gnu/4.4.1/collect2" "--build-id" "--eh- 
> frame-hdr"
> "-m" "elf_x86_64" "--hash-style=both" "-dynamic-linker"
> "/lib64/ld-linux-x86-64.so.2" "-z" "relro"
> "/usr/lib/gcc/x86_64-linux-gnu/4.4.1/../../../../lib/crt1.o"
> "/usr/lib/gcc/x86_64-linux-gnu/4.4.1/../../../../lib/crti.o"
> "/usr/lib/gcc/x86_64-linux-gnu/4.4.1/crtbegin.o"
> "-L/usr/lib/gcc/x86_64-linux-gnu/4.4.1" "-L/usr/lib/gcc/x86_64-linux- 
> gnu/4.4.1"
> "-L/usr/lib/gcc/x86_64-linux-gnu/4.4.1/../../../../lib" "-L/lib/../ 
> lib"
> "-L/usr/lib/../lib" "-L/usr/lib/gcc/x86_64-linux-gnu/4.4.1/../../.."
> "-L/usr/lib/x86_64-linux-gnu" "/tmp/cceJWVC8.o" "-lstdc++" "-lgcc"
> "--as-needed" "-lgcc_s" "--no-as-needed" "-lc" "-lgcc" "--as-needed"  
> "-lgcc_s"
> "--no-as-needed" "/usr/lib/gcc/x86_64-linux-gnu/4.4.1/crtend.o"
> "/usr/lib/gcc/x86_64-linux-gnu/4.4.1/../../../../lib/crtn.o"
> $ objdump -d a.out | c++filt | sed -n '/Test_/,/constructors/ p' > out
>
> (See attachments for source and output.)
>
>
> -- 
>           Summary: Endianness and Optimization
>           Product: gcc
>           Version: 4.4.1
>            Status: UNCONFIRMED
>          Severity: normal
>          Priority: P3
>         Component: tree-optimization
>        AssignedTo: unassigned at gcc dot gnu dot org
>        ReportedBy: kai dot extern at googlemail dot com
> GCC build triplet: x86_64-linux-gnu
>  GCC host triplet: x86_64-linux-gnu
> GCC target triplet: x86_64-linux-gnu
>
>
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43197
>


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43197

[Bug tree-optimization/43197] Endianness and Optimization

[kai dot extern at googlemail dot com]

------- Comment #4 from kai dot extern at googlemail dot com  2010-02-27 13:46 -------
> You are violating c++ aliasing rules. You access a uint8_t via  
> different types.un 

Actually, I address other types via uint8_t, but that is neither here nor
there. (Oh, I just realized you probably didn't mean the union but the load in
mem2int. But the following comments apply just as well to that part.)

First, as far as I can tell, the code produced is clearly *correct*.

Second, the optimization of everything except uint64_t shows that gcc clearly
understands what I'm trying to do.

In fact, my first attempts had no type-punning whatsoever; however, the
resulting code demonstrated that gcc had no clue what I was trying to do - it
was pretty much completely unoptimized. So I went looking forways to describe
the problem that gcc would actually understand.

This particular solution grew out of some other gcc bug which compared various
versions of determining endianess, and showed that currently, the version with
unions is the only one gcc can optimize - that seems to be a regression
introduced with 4.0, and the bug seems to be still open.

Anyway, my point here is that gcc *does* understand this idiom. I see two
problems:

1. For some reason, gcc can optimize the byte_sex function to a constant -
except when the integer is 64 bits long. It is not obvious if the problem is in
the 64 bits, or in the 8 loop iterations, but something does not work there
which works in the smaller cases.

2. The actual byte swapping code (which has nothing whatsoever to do with any
aliasing) is clearly suboptimal in some cases. Again, it is not obvious what
property causes gcc to generate this one just fine for some cases, and not very
fine for others. The basic logic is obviously the same.

Anyway, if you can point out a way to write this that is completely
standards-conformant, generates decent code (at least as good as this version),
and does not rely on me telling the compiler what the endianness is, I'm
interested in learning. (I should probably point out that this ought to work
even for inconsistent endianness - I don't recall exactly, but I rember hearing
about a cpu that did something like 3412 byte ordering.)

Just for comparision, my first attempt looked like this:

    template < int n, typename X > struct xword {
        void operator=(X x) {
            set(x);
        };
        operator X() {
            return get();
        };
      protected:
        void set(register X x) {
            for (register int i = 0; i < n; i++) {
                m_x[TRAITS::le ? i : n - i - 1] = x & 0xff;
                x >>= 8;
            };
        };
        X get() {
            register X r;
            for (register int i = 0; i < n; i++) {
                r <<= 8;
                r |= m_x[TRAITS::le ? n - i - 1 : i];
            };
            return r;
        };
      private:
        uint8_t m_x[n];
    };

No aliasing issues. Also, no optimization whatsoever.


-- 

kai dot extern at googlemail dot com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |kai dot extern at googlemail
                   |                            |dot com


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43197

[Bug tree-optimization/43197] Endianness and Optimization

[pinskia at gcc dot gnu dot org]

------- Comment #5 from pinskia at gcc dot gnu dot org  2010-02-27 17:25 -------
I just think your endian swap is not recognized by the bswap pass.  The main
reason is because of byte_sex function which is not optimized at the tree
level.  But really there are better ways of writing endian swaps.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43197

[Bug tree-optimization/43197] Endianness and Optimization

[rguenth at gcc dot gnu dot org]

------- Comment #6 from rguenth at gcc dot gnu dot org  2010-02-27 20:04 -------


*** This bug has been marked as a duplicate of 42587 ***


-- 

rguenth at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|                            |DUPLICATE


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43197

[Bug tree-optimization/43197] Endianness and Optimization

[kai dot extern at googlemail dot com]

------- Comment #7 from kai dot extern at googlemail dot com  2010-02-27 20:34 -------
(In reply to comment #6)
> 
> *** This bug has been marked as a duplicate of 42587 ***
> 

Oh? 42587 seems to be about not recognising memory bswap, which explains why my
first attempt didn't work. But that wasn't what this bug was about - in this
bug, the bswap was register-to-register, and some cases were recognized just
fine. Also, the other half was about a different optimization failing for 64
bit types.

Doesn't look like a duplicate to me.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43197