[gcc r14-6305] haifa-sched: Avoid overflows in extend_h_i_d [PR112411]

[gcc r14-6305] haifa-sched: Avoid overflows in extend_h_i_d [PR112411]

[Jakub Jelinek]

https://gcc.gnu.org/g:8f60f5499e10d19218cada082e0909516ebf0e74

commit r14-6305-g8f60f5499e10d19218cada082e0909516ebf0e74
Author: Jakub Jelinek <jakub@redhat.com>
Date:   Fri Dec 8 08:56:33 2023 +0100

    haifa-sched: Avoid overflows in extend_h_i_d [PR112411]
    
    On Thu, Dec 07, 2023 at 09:36:23AM +0100, Jakub Jelinek wrote:
    > Without the dg-skip-if I got on 64-bit host with
    > -O3 --param min-nondebug-insn-uid=0x40000000:
    > cc1: out of memory allocating 571230784744 bytes after a total of 2772992 bytes
    
    I've looked at this and the problem is in haifa-sched.cc:
    9047        h_i_d.safe_grow_cleared (3 * get_max_uid () / 2, true);
    get_max_uid () is 0x4000024d with the --param min-nondebug-insn-uid=0x40000000
    and so 3 * get_max_uid () / 2 actually overflows to -536870028 but as vec.h
    then treats the value as unsigned, it attempts to allocate
    0xe0000374U * 152UL bytes, i.e. those 532GB.  If the above is fixed to do
    3U * get_max_uid () / 2 instead, it will get slightly better and will only
    need 0x60000373U * 152UL bytes, i.e. 228GB.
    
    Perhaps more could be helped by making the vector indirect (contain pointers
    to haifa_insn_data_def rather than the structures themselves) and pool allocate
    those, but the more important question is how sparse are uids in normal
    compilations without those large --param min-nondebug-insn-uid= parameters.
    Because if they aren't enough, such a change would increase compile time memory
    just to help the unusual case.
    
    2023-12-08  Jakub Jelinek  <jakub@redhat.com>
    
            PR middle-end/112411
            * haifa-sched.cc (extend_h_i_d): Use 3U instead of 3 in
            3 * get_max_uid () / 2 calculation.

Diff:
---
 gcc/haifa-sched.cc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gcc/haifa-sched.cc b/gcc/haifa-sched.cc
index 8e8add709b3..35c2c9f2bdc 100644
--- a/gcc/haifa-sched.cc
+++ b/gcc/haifa-sched.cc
@@ -9044,7 +9044,7 @@ extend_h_i_d (void)
   if (reserve > 0
       && ! h_i_d.space (reserve))
     {
-      h_i_d.safe_grow_cleared (3 * get_max_uid () / 2, true);
+      h_i_d.safe_grow_cleared (3U * get_max_uid () / 2, true);
       sched_extend_target ();
     }
 }