[Bug tree-optimization/98514] New: ICE in insert_operand_rank

["jakub at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98514

            Bug ID: 98514
           Summary: ICE in insert_operand_rank
           Product: gcc
           Version: 11.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: tree-optimization
          Assignee: unassigned at gcc dot gnu.org
          Reporter: jakub at gcc dot gnu.org
  Target Milestone: ---

When building gmic on i686-linux (or other 32-bit arches) with LTO, one gets:
gmic.cpp: In member function '_run.isra':
gmic.cpp:4981:7: internal compiler error: in insert_operand_rank, at
tree-ssa-reassoc.c:367
 4981 | gmic& gmic::_run(const CImgList<char>& commands_line, unsigned int&
position,
      |       ^
It is in 101th partition and the function is huge, so am not going to reduce,
but it seems a general reassoc issue on 32-bit hosts.

The function in question has 104549 SSA_NAMEs and 36954 basic blocks.
Now, the bb ranks are computed as:
  /* Give each default definition a distinct rank.  This includes
     parameters and the static chain.  Walk backwards over all
     SSA names so that we get proper rank ordering according
     to tree_swap_operands_p.  */
  for (i = num_ssa_names - 1; i > 0; --i)
    {
      tree name = ssa_name (i);
      if (name && SSA_NAME_IS_DEFAULT_DEF (name))
        insert_operand_rank (name, ++rank);
    }

  /* Set up rank for each BB  */
  for (i = 0; i < n_basic_blocks_for_fn (cfun) - NUM_FIXED_BLOCKS; i++)
    bb_rank[bbs[i]] = ++rank << 16;
and bb_rank as well as everything else that deals with ranks is long,
so obviously it can't work properly in any function which has more than
32767 (num_ssa_names + n_basic_blocks_for_fn (cfun)).
Either we should punt on trying to reassociate such functions, but that would
make 32-bit hosts behave differently from 64-bit hosts even when targeting the
same target, or my preferred way out of this is make the ranks just
HOST_WIDE_INTs, yes, it will need more memory on 32-bit hosts and will be
slightly slower, but it will be consistent with 64-bit hosts.  We have vectors
of ssa names and basic blocks and those have int length, so even on 64-bit
hosts we can't support more than 4 billion ssa names and more than 4 billion
basic blocks.