[Bug c++/19614] New: Excessive memory consumption

[Bug c++/19614] New: Excessive memory consumption

[dmartin at cliftonlabs dot com]

Processing the attached file causes g++ to consume about 1.4G of virtual memory
on x86_64.  The same file caused problems on x86 in an earlier release - see bug
#15320.  Also, on my project's mailing list I have seen the implication that
similar issues exist on powerpc:
http://www.cliftonlabs.com/savant/list-archives/savant-users/2004-December/000013.html

Please let me know if there is anything I can do to help sort this out.

Here is the output of g++ -v:
Reading specs from /usr/lib/gcc/x86_64-linux/3.4.4/specs
Configured with: ../src/configure -v
--enable-languages=c,c++,java,f77,pascal,objc,ada,treelang --prefix=/usr
--libexecdir=/usr/lib --with-gxx-include-dir=/usr/include/c++/3.4
--enable-shared --with-system-zlib --enable-nls --without-included-gettext
--program-suffix=-3.4 --enable-__cxa_atexit --enable-libstdcxx-allocator=mt
--enable-clocale=gnu --enable-libstdcxx-debug --enable-java-gc=boehm
--enable-java-awt=gtk --disable-werror x86_64-linux
Thread model: posix
gcc version 3.4.4 20041218 (prerelease) (Debian 3.4.3-7)

-- 
           Summary: Excessive memory consumption
           Product: gcc
           Version: 3.4.4
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: c++
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: dmartin at cliftonlabs dot com
                CC: gcc-bugs at gcc dot gnu dot org
  GCC host triplet: x86_64-unknown-linux-gnu
GCC target triplet: x86_64-unknown-linux-gnu


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=19614

[Bug c++/19614] Excessive memory consumption

[dmartin at cliftonlabs dot com]

------- Additional Comments From dmartin at cliftonlabs dot com  2005-01-24 21:00 -------
Created an attachment (id=8056)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=8056&action=view)
A file requiring 1.4G of virtual memory to process on g++ 3.4.3


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=19614

[Bug c++/19614] Excessive memory consumption

[pinskia at gcc dot gnu dot org]

------- Additional Comments From pinskia at gcc dot gnu dot org  2005-01-24 21:07 -------
For me on the mainline on powerpc-darwin this takes a max of 500MB, at -O0 and -O2.

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=19614

[Bug c++/19614] Excessive memory consumption

[pinskia at gcc dot gnu dot org]

------- Additional Comments From pinskia at gcc dot gnu dot org  2005-01-25 00:07 -------
cp/class.c:7224 (build_vtbl_initializer)            6525792: 2.6%          0: 0.0%   22705120:17.7%          0: 
0.0%     913466
cp/class.c:7600 (add_vcall_offset)                  9266784: 3.7%          0: 0.0%   27057192:21.1%          0: 
0.0%    1513499
cp/class.c:7247 (build_vtbl_initializer)            9402024: 3.7%          0: 0.0%   27433944:21.4%          0: 
0.0%    1534832
tree.c:476 (copy_list)                               780332: 0.3%          0: 0.0%   13252456:10.3%          0: 0.0%     
545496

That is where we allocate alot of memory on the mainline.

exprs                4274758  137709384
random kinds         5811214  139520464

There are also a compile time issues too (we spent 30 seconds in the "parser").

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
     Ever Confirmed|                            |1
           Keywords|                            |compile-time-hog, memory-hog
   Last reconfirmed|0000-00-00 00:00:00         |2005-01-25 00:07:34
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=19614

[Bug c++/19614] Excessive memory consumption with a class with large (200) virtual (pure?) function and derived classes

[pinskia at gcc dot gnu dot org]

------- Additional Comments From pinskia at gcc dot gnu dot org  2005-01-25 04:26 -------
All of the memory seems to becoming from virtual functions, some from pure virtual functions (which 
we could optimizate a little better).
          /* You can't call an abstract virtual function; it's abstract.
             So, we replace these functions with __pure_virtual.  */
          if (DECL_PURE_VIRTUAL_P (fn_original))
            fn = abort_fndecl;
          else if (!integer_zerop (delta) || vcall_index)
            {
              fn = make_thunk (fn, /*this_adjusting=*/1, delta, vcall_index);
              if (!DECL_NAME (fn))
                finish_thunk (fn);
            }
          /* Take the address of the function, considering it to be of an
             appropriate generic type.  */
          init = build1 (ADDR_EXPR, vfunc_ptr_type_node, fn);

We could create one ADDR_EXPR for the pure virtual case which will both speed up this and create a 
huge amount of less memory (how much I don't know yet but I will test it soon).

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|Excessive memory consumption|Excessive memory consumption
                   |                            |with a class with large
                   |                            |(200) virtual (pure?)
                   |                            |function and derived classes


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=19614

[Bug c++/19614] Excessive memory consumption with a class with large (200) virtual (pure?) function and derived classes

[pinskia at gcc dot gnu dot org]

------- Additional Comments From pinskia at gcc dot gnu dot org  2005-01-25 06:45 -------
This explains a lot:
vtables = 917; vtable searches = 0
vtable entries = 0; vtable elems = 82020
(In reply to comment #4)
> We could create one ADDR_EXPR for the pure virtual case which will both speed up this and create a 
> huge amount of less memory (how much I don't know yet but I will test it soon).
This saves us about 5M-7M on powerpc-darwin but could be more on x86_64 and other targets, I will 
submit this now.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=19614

[Bug c++/19614] Excessive memory consumption with a class with large (200) virtual (pure?) function and derived classes

[dmartin at cliftonlabs dot com]

------- Additional Comments From dmartin at cliftonlabs dot com  2005-01-27 13:04 -------
I checked out CVS head on 1/27/2005.  The memory consumption on this file on
x86_64 was "only" 750M.  I don't know if this qualifies as excessive or not so
I'll let someone else close it.

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |WAITING


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=19614

[Bug c++/19614] Excessive memory consumption with a class with large (200) virtual (pure?) function and derived classes

[pinskia at gcc dot gnu dot org]

------- Additional Comments From pinskia at gcc dot gnu dot org  2005-01-27 13:24 -------
(In reply to comment #6)
> I checked out CVS head on 1/27/2005.  The memory consumption on this file on
> x86_64 was "only" 750M.  I don't know if this qualifies as excessive or not so
> I'll let someone else close it.
750M for me is still excessive.  I and other people will look into this soon (again).

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |NEW
   Last reconfirmed|2005-01-25 00:07:34         |2005-01-27 13:24:29
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=19614

[Bug c++/19614] Excessive memory consumption with a class with large (200) virtual (pure?) function and derived classes

[dmartin at cliftonlabs dot com]

------- Additional Comments From dmartin at cliftonlabs dot com  2005-01-27 13:43 -------
As a point of reference, it's using around 550M on x86.  A 50% expansion on a 64
bit architecture (with more registers and opcodes) does not seem completely
unreasonable to me.  So perhaps it's not a backend-specific issue at this point.

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=19614

[Bug c++/19614] Excessive memory consumption with a class with large (200) virtual (pure?) function and derived classes

[pinskia at gcc dot gnu dot org]

------- Additional Comments From pinskia at gcc dot gnu dot org  2005-04-28 00:52 -------
On the mainline, (on x86) we use about 300 megs which is an improvement.

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=19614

[Bug c++/19614] Excessive memory consumption with a class with large (200) virtual (pure?) function and derived classes

[pinskia at gcc dot gnu dot org]

------- Additional Comments From pinskia at gcc dot gnu dot org  2005-07-25 01:40 -------
cp/parser.c:285 (cp_lexer_new_main)                       0: 0.0%   22585856:63.1%          0: 0.0%    
6332928:24.0%          5
tree.c:966 (build_constructor_from_list)              28488: 0.0%          0: 0.0%   38444792:40.2%    
9123452:34.6%       7212
cp/decl.c:11123 (cxx_push_function_context)          396000: 0.2%          0: 0.0%     147600: 0.2%      
65232: 0.2%       5436
cp/pt.c:5954 (tsubst_template_args)                  449196: 0.2%          0: 0.0%     115304: 0.1%      31156: 
0.1%      18671
cp/decl.c:533 (poplevel)                             390540: 0.2%          0: 0.0%     178440: 0.2%      75864: 0.3%       
9483
cp/class.c:7042 (dfs_accumulate_vtbl_inits)          107680: 0.0%          0: 0.0%     480640: 0.5%          0: 
0.0%      18385
tree.c:3643 (build_distinct_type_copy)                69888: 0.0%          0: 0.0%     532608: 0.6%          0: 
0.0%       6276
cp/method.c:131 (make_thunk)                              0: 0.0%          0: 0.0%     605200: 0.6%      35600: 
0.1%       4450
tree.c:607 (copy_list)                                 1408: 0.0%          0: 0.0%     622904: 0.7%          0: 0.0%      
11501
c-semantics.c:120 (build_stmt)                       436032: 0.2%          0: 0.0%     221680: 0.2%       2116: 
0.0%      18967
cp/class.c:7049 (dfs_accumulate_vtbl_inits)          121140: 0.0%          0: 0.0%     540720: 0.6%          0: 
0.0%      18385
ggc-common.c:193 (ggc_calloc)                        383944: 0.2%      30792: 0.1%     280544: 0.3%      
13732: 0.1%       1953
cp/lex.c:693 (retrofit_lang_decl)                     54180: 0.0%      17640: 0.0%     683496: 0.7%          0: 
0.0%      26793
tree-inline.c:632 (copy_body_r)                      746080: 0.3%          0: 0.0%          0: 0.0%          0: 0.0%      
23315
tree.c:4813 (build_index_type)                       702624: 0.3%          0: 0.0%      48576: 0.1%          0: 0.0%       
7825
tree.c:4910 (build_array_type)                       696480: 0.3%          0: 0.0%      55488: 0.1%          0: 0.0%       
7833
cp/typeck.c:3701 (build_address)                     307424: 0.1%          0: 0.0%     498912: 0.5%          0: 
0.0%      25198
gimplify.c:476 (internal_get_tmp_var)               1062792: 0.4%          0: 0.0%          0: 0.0%          0: 0.0%      
29522
cp/class.c:7381 (build_vbase_offset_vtbl_entries    1064712: 0.4%          0: 0.0%          0: 0.0%          0: 
0.0%      44363
tree.c:4963 (build_function_type)                   1093632: 0.4%          0: 0.0%     121536: 0.1%          0: 
0.0%      12658
cp/lex.c:748 (copy_decl)                              13104: 0.0%          0: 0.0%    1242932: 1.3%      62528: 0.2%       
9619
cp/class.c:7571 (add_vcall_offset)                        0: 0.0%    1461232: 4.1%    1612536: 1.7%     830176: 
3.1%       4408
stringpool.c:77 (alloc_node)                              0: 0.0%          0: 0.0%    1769144: 1.8%          0: 0.0%      
34022
tree.c:5023 (build_method_type_directly)             827520: 0.3%          0: 0.0%    1007520: 1.1%          0: 
0.0%      19115
cp/decl2.c:142 (cp_build_parm_decl)                  114048: 0.0%          0: 0.0%    1881088: 2.0%          0: 
0.0%      22672
cp/typeck.c:3713 (build_nop)                         854016: 0.3%          0: 0.0%    1187744: 1.2%          0: 
0.0%      63805
tree-iterator.c:47 (alloc_stmt_list)                2320176: 0.9%          0: 0.0%      38952: 0.0%          0: 0.0%      
98297
tree-iterator.c:165 (tsi_link_after)                2895444: 1.2%          0: 0.0%      59172: 0.1%          0: 0.0%     
246218
cp/lex.c:668 (build_lang_decl)                       171560: 0.1%      71888: 0.2%    2905044: 3.0%     
102068: 0.4%      26599
gimplify.c:337 (create_tmp_var_raw)                 3679584: 1.5%          0: 0.0%       1152: 0.0%          0: 
0.0%      38341
function.c:3782 (allocate_struct_function)          2851200: 1.1%          0: 0.0%    1062720: 1.1%    
1130688: 4.3%       5436
integrate.c:115 (copy_decl_for_inlining)            7672180: 3.1%          0: 0.0%       1152: 0.0%          0: 
0.0%      79887
tree-inline.c:2380 (copy_tree_r)                   11958124: 4.8%          0: 0.0%          0: 0.0%      46920: 0.2%     
332081
tree.c:611 (copy_list)                               590920: 0.2%          0: 0.0%   13154456:13.8%          0: 0.0%     
545484
cp/class.c:7582 (add_vcall_offset)                 14411760: 5.8%    1305304: 3.6%          0: 0.0%    
4582600:17.4%      19954
cp/class.c:7233 (build_vtbl_initializer)            4921824: 2.0%          0: 0.0%   16785472:17.6%          0: 
0.0%     678353
cp/class.c:7608 (add_vcall_offset)                 36323976:14.5%          0: 0.0%          0: 0.0%          0: 0.0%    
1513499
cp/class.c:7257 (build_vtbl_initializer)           36835968:14.7%          0: 0.0%          0: 0.0%          0: 0.0%    
1534832
tree.c:966 (build_constructor_from_list)              28488: 0.0%          0: 0.0%   38444792:40.2%    
9123452:34.6%       7212
convert.c:457 (convert_to_integer)                 45943936:18.4%          0: 0.0%          0: 0.0%          0: 0.0%    
1435748
cp/class.c:1835 (dfs_find_final_overrider_1)       52468296:21.0%          0: 0.0%          0: 0.0%          0: 
0.0%    2186179

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=19614

[Bug c++/19614] Excessive memory consumption with a class with large (200) virtual (pure?) function and derived classes

[phython at gcc dot gnu dot org]

------- Additional Comments From phython at gcc dot gnu dot org  2005-08-09 05:05 -------
 Wow, compiling this on ia64-linux at -O2 gives:
 parser                :  50.14 (80%) usr   0.58 (49%) sys  50.75 (79%) wall  
797327 kB (93%) ggc

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=19614

[Bug c++/19614] Excessive memory consumption with a class with large (200) virtual (pure?) function and derived classes

[pinskia at gcc dot gnu dot org]

------- Additional Comments From pinskia at gcc dot gnu dot org  2005-09-18 23:57 -------
This is still only a front-end issue:
 parser                :  14.19 (84%) usr   0.49 (51%) sys  14.64 (82%) wall  560492 kB (91%) ggc


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=19614

[Bug c++/19614] Excessive memory consumption with a class with large (200) virtual (pure?) function and derived classes

[bangerth at dealii dot org]

------- Comment #14 from bangerth at dealii dot org  2007-09-21 02:29 -------
We are now again on par with gcc 3.3 in terms of memory use (3.3 uses 298 MB
whereas 4.1 uses 280MB). This is a factor of 2 or 3 less than what we used
to use. We are also about a factor of 2 faster than in the 3.3 and 3.4 times.

I think we can close this one now.

W.


-- 

bangerth at dealii dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |bangerth at dealii dot org
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=19614

[Bug c++/19614] Excessive memory consumption with a class with large (200) virtual (pure?) function and derived classes

[rguenth at gcc dot gnu dot org]

------- Comment #13 from rguenth at gcc dot gnu dot org  2007-09-20 14:24 -------
Still a frontend only issue.  At -O2 we have

 parser                :  13.57 (85%) usr   0.62 (58%) sys  14.08 (83%) wall 
536975 kB (90%) ggc
 TOTAL                 :  15.91             1.07            17.00            
595601 kB

and virtual memory peaks at 370MB.


-- 

rguenth at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Last reconfirmed|2005-01-27 13:24:29         |2007-09-20 14:24:50
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=19614