[Bug tree-optimization/107009] [13 Regression] massive unnecessary code blowup in vectorizer

["aldyh at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107009

--- Comment #4 from Aldy Hernandez <aldyh at gcc dot gnu.org> ---
(In reply to Richard Biener from comment #3)
>   # RANGE [irange] size_t [1, +INF]
>   size_t n_12(D) = n;
> 
> the nonzero bits info on 'n' is gone.  DOM2 used to produce that and
> CCP3 elides the __builtin_unreachable () but if DOM2 is disabled that
> doesn't produce the nonzero bits.
> 
> DOM2 from trunk no longer sets nonzero bits.  CCP doesn't do any
> conditional non-zero bits (or constant equivalence) propagation.

Indeed, there is a missing nonzero mask on n_12(D).  Thanks for the analysis,
Richi.  It saved me a ton of work.

This is the problematic BB, where the 2->3 edge is unreachable:

    <bb 2> [local count: 118111600]:
    _1 = n_12(D) & 7;
    if (_1 != 0)
      goto <bb 3>; [0.00%]
    else
      goto <bb 4>; [100.00%]

_1 : [irange] long unsigned int [0, 0]
2->3  (T) _1 :  [irange] UNDEFINED
2->3  (T) n_12(D) :     [irange] UNDEFINED
2->4  (F) _1 :  [irange] long unsigned int [0, 0]
2->4  (F) n_12(D) :     [irange] size_t [1, 18446744073709551608] <-- BOO
HISS!!!!

Notice there is no nonzero mask on n_12(D) in the 2->4 edge which is
unreachable.

So this is actually our fault, not because of this patch, but because we didn't
track nonzero bits back then.  I keep saying, this patch is a red herring :-P.

However, we do track nonzero bits now, and the testcase still fails so...

The culprit is operator_bitwise_and::op1_range() which is not setting the
nonzero mask.  This should be quite easy to fix.  What we're trying to solve is
n_12 on the 2->4 edge:

    _1 = n_12(D) & 7;

which _1 = 0 because we're talking about the 2->4 edge:

     0 = n_12(D) & 7;

So the mask should be everything but the lower 3 bits, 0xffff...f8 if my math
is correct.