[Bug tree-optimization/80574] GCC fail to optimize nested ternary

["pinskia at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=80574

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|8.0                         |---

--- Comment #8 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
(In reply to Andrew Pinski from comment #7)
> The original testcase in comment #0 is fixed in GCC 8, I don't know what
> caused the improvement though.

Well actually if you use the C++ front-end, it still fails.

for f2_signed, we start out as:
  _1 = MAX_EXPR <a3_4(D), a2_5(D)>;
  if (_1 >= a1_6(D))
    goto <bb 3>; [INV]
  else
    goto <bb 6>; [INV]

  <bb 3> :
  if (a3_4(D) < a2_5(D))
    goto <bb 6>; [INV]
  else
    goto <bb 4>; [INV]

  <bb 4> :

  <bb 6> :
  # iftmp.5_2 = PHI <a3_4(D)(4), a1_6(D)(2), a2_5(D)(3)>
  return iftmp.5_2;

phiopt1 transforms it to:
  _1 = MAX_EXPR <a3_4(D), a2_5(D)>;
  if (_1 >= a1_6(D))
    goto <bb 3>; [INV]
  else
    goto <bb 4>; [INV]

  <bb 3> :
  _3 = MAX_EXPR <a3_4(D), a2_5(D)>;

  <bb 4> :
  # iftmp.12_2 = PHI <_3(3), a1_6(D)(2)>

Which is perfect.
But then we don't exactly patch that _1 and _3 are the same though we do try to
simplify it at least on the trunk:
phiopt match-simplify trying:
        _1 >= a1_6(D) ? _3 : a1_6(D)

phiopt match-simplify trying:
        _1 < a1_6(D) ? a1_6(D) : _3

What happens afterwards is fre (or is it pre) figures out _1 and _3 are the
same and get:
  if (_1 >= a1_6(D))
    goto <bb 3>; [INV]
  else
    goto <bb 4>; [INV]

  <bb 3> :

  <bb 4> :
  # iftmp.12_2 = PHI <_1(3), a1_6(D)(2)>

Which then phiopt2 is able to simplify.
So if we iterate phiopt and fre we should able to handle all of these but that
is NOT a reasonable solution.
I have to think of a good way of solving these really.