[Bug tree-optimization/98856] [11 Regression] botan AES-128/XTS is slower by ~17% since r11-6649-g285fa338b06b804e72997c4d876ecf08a9c083af

["jakub at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98856

--- Comment #15 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
The needed permutations for this boil down to
typedef int V __attribute__((vector_size (16)));
typedef int W __attribute__((vector_size (32)));

#ifdef __clang__
V f1 (V x) { return __builtin_shufflevector (x, x, 1, 1, 3, 3); }
V f2 (V x, V y) { return __builtin_shufflevector (x, y, 1, 5, 3, 7); }
V f3 (V x, V y) { return __builtin_shufflevector (x, y, 0, 5, 2, 7); }
#ifdef __AVX2__
W f4 (W x, W y) { return __builtin_shufflevector (x, y, 1, 9, 3, 11, 5, 13, 7,
15); }
W f5 (W x, W y) { return __builtin_shufflevector (x, y, 0, 9, 2, 11, 4, 13, 6,
15); }
W f6 (W x) { return __builtin_shufflevector (x, x, 1, 1, 3, 3, 5, 5, 7, 7); }
#endif
V f7 (V x) { return __builtin_shufflevector (x, x, 1, 3, 2, 3); }
V f8 (V x) { return __builtin_shufflevector (x, x, 0, 2, 2, 3); }
V f9 (V x, V y) { return __builtin_shufflevector (x, y, 0, 4, 1, 5); }
#else
V f1 (V x) { return __builtin_shuffle (x, (V) { 1, 1, 3, 3 }); }
V f2 (V x, V y) { return __builtin_shuffle (x, y, (V) { 1, 5, 3, 7 }); }
V f3 (V x, V y) { return __builtin_shuffle (x, y, (V) { 0, 5, 2, 7 }); }
#ifdef __AVX2__
W f4 (W x, W y) { return __builtin_shuffle (x, y, (W) { 1, 9, 3, 11, 5, 13, 7,
15 }); }
W f5 (W x, W y) { return __builtin_shuffle (x, y, (W) { 0, 9, 2, 11, 4, 13, 6,
15 }); }
W f6 (W x, W y) { return __builtin_shuffle (x, (W) { 1, 1, 3, 3, 5, 5, 7, 7 });
}
#endif
V f7 (V x) { return __builtin_shuffle (x, (V) { 1, 3, 2, 3 }); }
V f8 (V x) { return __builtin_shuffle (x, (V) { 0, 2, 2, 3 }); }
V f9 (V x, V y) { return __builtin_shuffle (x, y, (V) { 0, 4, 1, 5 }); }
#endif

With -msse2, LLVM emits 2 x pshufd $237 + punpckldq for f2 and pshufd $237 +
pshufd $232 + punpckldq, we give up or emit very large code.
With -msse4, we handle everything, and f1/f3 are the same/comparable, but for
f2 we emit 2 x pshufb (with memory operands) + por while
LLVM emits pshufd $245 + pblendw $204.
With -mavx2, the f2 inefficiency remains, and for f4 we emit 2x vpshufb with
memory operands + vpor while LLVM emits vpermilps $245 + vblendps $170.
f6-f9 are all insns that we handle through a single insn and that plus f3 are
the roadblocks to build the f2 and f4 permutations more efficiently.