[Bug c/36262] New: Extreme memory usage compared to older versions

[Bug c/36262] New: Extreme memory usage compared to older versions

[hakonrk at fys dot uio dot no]

While compiling the NES emulator FCE Ultra for my new Fedora 9 system (x86-64),
I noticed that gcc 4.3.0 (or at least Fedora's version of it) used so much
memory on some of the files that my system almost crashed.  The preprocessed
source code that trigger this bug is attached here.  This seems to be a
regression, as I have not been able to reproduce it on Fedora's gcc 3.4.6
(compat-gcc-34-3.4.6-9.x86_64), nor on gcc 4.1.2 on my old system (Slackware
12).

I search for other memory related bugs in gcc 4.3, and found a couple that
mention similar problems, so it's possible (or even probable) that this is a
duplicate.  I'm not sure, though, so I've created this bug report just so you
can take a quick look at the problem and dismiss it if there's nothing new
here.

Sorry about the length of the code that reproduces the bug, by the way.  It's a
20 thousand line file with, what looks like, machine-generated C code.


-- 
           Summary: Extreme memory usage compared to older versions
           Product: gcc
           Version: 4.3.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: hakonrk at fys dot uio dot no


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36262

[Bug c/36262] Extreme memory usage compared to older versions

[hakonrk at fys dot uio dot no]

------- Comment #1 from hakonrk at fys dot uio dot no  2008-05-18 23:07 -------
Created an attachment (id=15653)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=15653&action=view)
Uses gigabytes of memory when compiled with optimizations on 4.3.0


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36262

[Bug middle-end/36262] [4.3/4.4 Regression] Extreme memory usage of VRP compared to older versions

[rguenth at gcc dot gnu dot org]

------- Comment #2 from rguenth at gcc dot gnu dot org  2008-05-19 09:18 -------
Confirmed.

4.3.1 -O: 177MB
4.3.1 -O2: 1.3GB
4.3.1 -O2 -fno-tree-vrp: 230MB
4.2.3 -O2: 230MB

 tree VRP              :  42.06 (55%) usr   1.33 (53%) sys  43.52 (55%) wall
2319231 kB (94%) ggc

whoooo ;)  Looks like sth new.


-- 

rguenth at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |rguenth at gcc dot gnu dot
                   |                            |org
             Status|UNCONFIRMED                 |NEW
     Ever Confirmed|0                           |1
      Known to fail|                            |4.3.0
      Known to work|                            |4.2.3
   Last reconfirmed|0000-00-00 00:00:00         |2008-05-19 09:18:21
               date|                            |
            Summary|Extreme memory usage        |[4.3/4.4 Regression] Extreme
                   |compared to older versions  |memory usage of VRP compared
                   |                            |to older versions
   Target Milestone|---                         |4.3.1


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36262

[Bug middle-end/36262] [4.3/4.4 Regression] Extreme memory usage of VRP compared to older versions

[rguenth at gcc dot gnu dot org]

------- Comment #3 from rguenth at gcc dot gnu dot org  2008-05-19 09:42 -------
This testcase has 1025 loops we create pre-headers for.  We insert tons of
asserts for non-NULL pointers due to dereferences (>3074) which all cause
new PHI nodes to be registered during insertion (and nearly the whole program
is rewritten).  But most of the time/memory is probably spent within SCEV
analysis (indeed, starting with 4.3 we reset the SCEV cache on each
invocation of adjust_range_with_scev).

>From detailed mem-report we see that also reassociation is causing quite some
garbage:

tree-phinodes.c:155 (allocate_phi_node)            11491552: 0.5%          0:
0.0%          0: 0.0%     454880: 0.4%      29364
fold-const.c:6254 (extract_muldiv_1)               35618816: 1.7%          0:
0.0%          0: 0.0%          0: 0.0%     556544
fold-const.c:2516 (fold_convert)                   36274032: 1.7%          0:
0.0%          0: 0.0%    4030448: 4.0%     503806
fold-const.c:7473 (fold_plusminus_mult_expr)       45185024: 2.1%          0:
0.0%          0: 0.0%          0: 0.0%     706016
fold-const.c:1592 (associate_trees)               131797504: 6.2%          0:
0.0%          0: 0.0%          0: 0.0%    2059336
fold-const.c:9743 (fold_binary)                   131862976: 6.2%          0:
0.0%          0: 0.0%          0: 0.0%    2060359
tree-chrec.h:149 (build_polynomial_chrec)         183160824: 8.6%          0:
0.0%          0: 0.0%   16650984:16.4%    2081373
tree-chrec.h:149 (build_polynomial_chrec)         198801152: 9.3%          0:
0.0%          0: 0.0%   18072832:17.8%    2259104
fold-const.c:1453 (negate_expr)                   445240224:20.9%          0:
0.0%          0: 0.0%   49471136:48.6%    6183892
tree-chrec.c:325 (chrec_fold_plus_1)              782040448:36.7%          0:
0.0%          0: 0.0%          0: 0.0%   12219382
Total                                            2132008624        486260114   
      4194416        101771994         59795791
source location                                     Garbage            Freed   
         Leak         Overhead            Times


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36262

[Bug middle-end/36262] [4.3/4.4 Regression] Extreme memory usage of VRP compared to older versions

[rguenth at gcc dot gnu dot org]

------- Comment #4 from rguenth at gcc dot gnu dot org  2008-05-19 11:00 -------
Collecting after clearing the SCEV cache brings down peak memory usage to about
450MB, the question is whether this is safe.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36262

[Bug middle-end/36262] [4.3/4.4 Regression] Extreme memory usage of VRP compared to older versions

[rguenth at gcc dot gnu dot org]

------- Comment #5 from rguenth at gcc dot gnu dot org  2008-05-20 13:00 -------
It is not safe.  Probably the best thing would be to not ask SCEV during the
propagation but instead at ASSERT_EXPR insertion time.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36262

[Bug middle-end/36262] [4.3/4.4 Regression] Extreme memory usage of VRP compared to older versions

[rguenth at gcc dot gnu dot org]

-- 

rguenth at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         AssignedTo|unassigned at gcc dot gnu   |rguenth at gcc dot gnu dot
                   |dot org                     |org
             Status|NEW                         |ASSIGNED
   Last reconfirmed|2008-05-19 09:18:21         |2008-05-27 08:32:02
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36262

[Bug middle-end/36262] [4.3/4.4 Regression] Extreme memory usage of VRP compared to older versions

[rguenth at gcc dot gnu dot org]

-- 

rguenth at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Priority|P3                          |P2


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36262

[Bug middle-end/36262] [4.3 Regression] Extreme memory usage of VRP compared to older versions

[rguenth at gcc dot gnu dot org]

------- Comment #7 from rguenth at gcc dot gnu dot org  2008-05-31 13:01 -------
Subject: Bug 36262

Author: rguenth
Date: Sat May 31 13:01:10 2008
New Revision: 136237

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=136237
Log:
2008-05-31  Richard Guenther  <rguenther@suse.de>

        PR tree-optimization/34244
        * fold-const.c (tree_expr_nonnegative_warnv_p): Do not ask VRP.
        (tree_expr_nonzero_warnv_p): Likewise.
        * tree-vrp.c (vrp_expr_computes_nonnegative): Call
        ssa_name_nonnegative_p.
        (vrp_expr_computes_nonzero): Call ssa_name_nonzero_p.
        (extract_range_from_unary_expr): Use vrp_expr_computes_nonzero,
        not tree_expr_nonzero_warnv_p.

        PR tree-optimization/36262
        Revert
        2007-11-29  Zdenek Dvorak  <ook@ucw.cz>

        PR tree-optimization/34244
        * tree-vrp.c (adjust_range_with_scev): Clear scev cache.
        (record_numbers_of_iterations): New function.
        (execute_vrp): Cache the numbers of iterations of loops.
        * tree-scalar-evolution.c (scev_reset_except_niters):
        New function.
        (scev_reset): Use scev_reset_except_niters.
        * tree-scalar-evolution.h (scev_reset_except_niters): Declare.

Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/fold-const.c
    trunk/gcc/tree-scalar-evolution.c
    trunk/gcc/tree-scalar-evolution.h
    trunk/gcc/tree-vrp.c


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36262

[Bug middle-end/36262] [4.3 Regression] Extreme memory usage of VRP compared to older versions

[rguenth at gcc dot gnu dot org]

------- Comment #6 from rguenth at gcc dot gnu dot org  2008-05-31 13:01 -------
Fixed for 4.4.0.


-- 

rguenth at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
      Known to work|4.2.3                       |4.2.3 4.4.0
            Summary|[4.3/4.4 Regression] Extreme|[4.3 Regression] Extreme
                   |memory usage of VRP compared|memory usage of VRP compared
                   |to older versions           |to older versions


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36262

[Bug middle-end/36262] [4.3 Regression] Extreme memory usage of VRP compared to older versions

[rguenth at gcc dot gnu dot org]

------- Comment #8 from rguenth at gcc dot gnu dot org  2008-06-06 14:59 -------
4.3.1 is being released, adjusting target milestone.


-- 

rguenth at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|4.3.1                       |4.3.2


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36262

[Bug middle-end/36262] [4.3 Regression] Extreme memory usage of VRP compared to older versions

[rguenth at gcc dot gnu dot org]

------- Comment #9 from rguenth at gcc dot gnu dot org  2008-06-06 20:07 -------
Fixed.


-- 

rguenth at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
      Known to fail|4.3.0                       |4.3.0 4.3.1
         Resolution|                            |FIXED


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36262

[Bug middle-end/36262] [4.3 Regression] Extreme memory usage of VRP compared to older versions

[rguenth at gcc dot gnu dot org]

------- Comment #10 from rguenth at gcc dot gnu dot org  2008-06-06 20:07 -------
Subject: Bug 36262

Author: rguenth
Date: Fri Jun  6 20:06:40 2008
New Revision: 136501

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=136501
Log:
2008-06-06  Richard Guenther  <rguenther@suse.de>

        PR tree-optimization/34244
        * fold-const.c (tree_expr_nonnegative_warnv_p): Do not ask VRP.
        (tree_expr_nonzero_warnv_p): Likewise.
        * tree-vrp.c (vrp_expr_computes_nonnegative): Call
        ssa_name_nonnegative_p.
        (vrp_expr_computes_nonzero): Call ssa_name_nonzero_p.
        (extract_range_from_unary_expr): Use vrp_expr_computes_nonzero,
        not tree_expr_nonzero_warnv_p.

        PR tree-optimization/36262
        Revert
        2007-11-29  Zdenek Dvorak  <ook@ucw.cz>

        PR tree-optimization/34244
        * tree-vrp.c (adjust_range_with_scev): Clear scev cache.
        (record_numbers_of_iterations): New function.
        (execute_vrp): Cache the numbers of iterations of loops.
        * tree-scalar-evolution.c (scev_reset_except_niters):
        New function.
        (scev_reset): Use scev_reset_except_niters.
        * tree-scalar-evolution.h (scev_reset_except_niters): Declare.

Modified:
    branches/gcc-4_3-branch/gcc/ChangeLog
    branches/gcc-4_3-branch/gcc/fold-const.c
    branches/gcc-4_3-branch/gcc/tree-scalar-evolution.c
    branches/gcc-4_3-branch/gcc/tree-scalar-evolution.h
    branches/gcc-4_3-branch/gcc/tree-vrp.c


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36262

[Bug middle-end/36262] [4.3 Regression] Extreme memory usage of VRP compared to older versions

[steven at gcc dot gnu dot org]

------- Comment #11 from steven at gcc dot gnu dot org  2008-07-06 09:59 -------
It looks like we don't use a known number of loop iterations at all anymore
after this patch.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36262

[Bug middle-end/36262] [4.3 Regression] Extreme memory usage of VRP compared to older versions

[rguenth at gcc dot gnu dot org]

------- Comment #12 from rguenth at gcc dot gnu dot org  2008-07-06 12:27 -------
I don't think we "used" it before either?  Still the _computing_ of niters
can be easily re-instantiated - it wasn't the expensive thing here.  But I
had the impression SCEV computes niters itself when needed, so the removal
of the upfront computation was just an "optimization".  Note that Zdenek
added it to not do this expensive thing multiple times.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36262