* [Bug c++/12850] memory consumption for heavy template instantiations tripled since 3.3
[not found] <bug-12850-1824@http.gcc.gnu.org/bugzilla/>
@ 2007-09-20 14:09 ` rguenth at gcc dot gnu dot org
2007-09-20 14:10 ` rguenth at gcc dot gnu dot org
` (3 subsequent siblings)
4 siblings, 0 replies; 26+ messages in thread
From: rguenth at gcc dot gnu dot org @ 2007-09-20 14:09 UTC (permalink / raw)
To: gcc-bugs
------- Comment #43 from rguenth at gcc dot gnu dot org 2007-09-20 14:09 -------
Mainline with release checking uses 520MB ram again on the testcase with -O0
on x86_64 and 650MB with -O2. time-report with -O2 shows
df live regs : 4.84 ( 5%) usr 0.06 ( 1%) sys 4.76 ( 5%) wall
0 kB ( 0%) ggc
parser : 3.97 ( 4%) usr 0.49 (10%) sys 4.82 ( 5%) wall
246991 kB (15%) ggc
expand : 7.94 ( 8%) usr 0.22 ( 4%) sys 8.30 ( 8%) wall
169886 kB (10%) ggc
CSE : 3.72 ( 4%) usr 0.03 ( 1%) sys 3.94 ( 4%) wall
5135 kB ( 0%) ggc
global alloc : 4.91 ( 5%) usr 0.07 ( 1%) sys 5.02 ( 5%) wall
43268 kB ( 3%) ggc
scheduling 2 : 4.00 ( 4%) usr 0.08 ( 2%) sys 4.07 ( 4%) wall
3173 kB ( 0%) ggc
TOTAL : 95.64 5.07 101.29
1657755 kB
that is, nothing really outstanding.
--
rguenth at gcc dot gnu dot org changed:
What |Removed |Added
----------------------------------------------------------------------------
Keywords|ice-on-valid-code |
Last reconfirmed|2003-10-31 01:13:03 |2007-09-20 14:09:06
date| |
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12850
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug c++/12850] memory consumption for heavy template instantiations tripled since 3.3
[not found] <bug-12850-1824@http.gcc.gnu.org/bugzilla/>
2007-09-20 14:09 ` [Bug c++/12850] memory consumption for heavy template instantiations tripled since 3.3 rguenth at gcc dot gnu dot org
@ 2007-09-20 14:10 ` rguenth at gcc dot gnu dot org
2007-09-20 14:11 ` rguenth at gcc dot gnu dot org
` (2 subsequent siblings)
4 siblings, 0 replies; 26+ messages in thread
From: rguenth at gcc dot gnu dot org @ 2007-09-20 14:10 UTC (permalink / raw)
To: gcc-bugs
------- Comment #44 from rguenth at gcc dot gnu dot org 2007-09-20 14:10 -------
Created an attachment (id=14232)
--> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=14232&action=view)
unincluded testcase
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12850
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug c++/12850] memory consumption for heavy template instantiations tripled since 3.3
[not found] <bug-12850-1824@http.gcc.gnu.org/bugzilla/>
2007-09-20 14:09 ` [Bug c++/12850] memory consumption for heavy template instantiations tripled since 3.3 rguenth at gcc dot gnu dot org
2007-09-20 14:10 ` rguenth at gcc dot gnu dot org
@ 2007-09-20 14:11 ` rguenth at gcc dot gnu dot org
2008-09-06 14:13 ` hubicka at gcc dot gnu dot org
2009-02-23 16:29 ` hubicka at gcc dot gnu dot org
4 siblings, 0 replies; 26+ messages in thread
From: rguenth at gcc dot gnu dot org @ 2007-09-20 14:11 UTC (permalink / raw)
To: gcc-bugs
--
rguenth at gcc dot gnu dot org changed:
What |Removed |Added
----------------------------------------------------------------------------
AssignedTo|hubicka at gcc dot gnu dot |unassigned at gcc dot gnu
|org |dot org
Status|ASSIGNED |NEW
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12850
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug c++/12850] memory consumption for heavy template instantiations tripled since 3.3
[not found] <bug-12850-1824@http.gcc.gnu.org/bugzilla/>
` (2 preceding siblings ...)
2007-09-20 14:11 ` rguenth at gcc dot gnu dot org
@ 2008-09-06 14:13 ` hubicka at gcc dot gnu dot org
2009-02-23 16:29 ` hubicka at gcc dot gnu dot org
4 siblings, 0 replies; 26+ messages in thread
From: hubicka at gcc dot gnu dot org @ 2008-09-06 14:13 UTC (permalink / raw)
To: gcc-bugs
------- Comment #45 from hubicka at gcc dot gnu dot org 2008-09-06 14:12 -------
Memory footprint in TOP is about 430MB (64bit machine).
On current mainline we need 191MB before IPA. Top consumers
cfg.c:226 (connect_dest) 598696: 0.2% 180224:
0.5% 3484960: 1.8% 594504: 1.5% 73663
gimple-low.c:806 (record_vars_into) 0: 0.0% 0:
0.0% 3825552: 2.0% 0: 0.0% 79699
cp/pt.c:8316 (tsubst_decl) 2244888: 0.9% 0:
0.0% 4552704: 2.4% 357768: 0.9% 44721
tree.c:6061 (build_method_type_directly) 1946600: 0.8% 0:
0.0% 4703200: 2.5% 265992: 0.7% 33249
tree-inline.c:3589 (copy_tree_r) 9450136: 3.6% 0:
0.0% 4820840: 2.5% 1248128: 3.2% 187483
cfg.c:142 (alloc_block) 1046016: 0.4% 0:
0.0% 4988448: 2.6% 0: 0.0% 62859
cgraph.c:638 (cgraph_create_edge) 0: 0.0% 0:
0.0% 5183328: 2.7% 0: 0.0% 53993
gimplify.c:4314 (gimplify_modify_expr) 1185040: 0.5% 0:
0.0% 5570160: 2.9% 304112: 0.8% 57599
gimple-iterator.c:446 (gsi_insert_after_without_ 4904480: 1.9% 0:
0.0% 5843840: 3.1% 2149664: 5.5% 268708
cfg.c:280 (unchecked_make_edge) 0: 0.0% 783288:
2.2% 5930352: 3.1% 745960: 1.9% 93245
gimple.c:287 (gimple_build_call_1) 871144: 0.3% 0:
0.0% 6066056: 3.2% 247408: 0.6% 51874
tree.c:962 (build_int_cst_wide) 6096: 0.0% 0:
0.0% 9716432: 5.1% 3187680: 8.1% 2221
gimplify.c:521 (create_tmp_var_raw) 452760: 0.2% 0:
0.0% 10597944: 5.5% 526224: 1.3% 65778
cp/lex.c:590 (copy_decl) 26304: 0.0% 0:
0.0% 13586520: 7.1% 1326296: 3.4% 56894
Total 258936448 34882576
191255621 39440157 5928571
source location Garbage Freed
Leak Overhead Times
Apparently largest are the gimple temporaries
after IPA:
cp/lex.c:573 (cxx_dup_lang_specific_decl) 384: 0.0% 896:
0.0% 2770736: 0.9% 2992: 0.0% 43453
cp/lex.c:510 (build_lang_decl) 805432: 0.2% 209648:
0.2% 3196488: 1.1% 349552: 0.5% 18896
stringpool.c:74 (alloc_node) 1994400: 0.4% 0:
0.0% 3287712: 1.1% 0: 0.0% 55022
cfg.c:142 (alloc_block) 10005792: 2.1% 0:
0.0% 3966048: 1.4% 0: 0.0% 145540
cfg.c:280 (unchecked_make_edge) 4507272: 0.9% 7134984:
5.8% 4456296: 1.5% 1788728: 2.5% 223591
cgraph.c:408 (cgraph_create_node) 7802208: 1.6% 0:
0.0% 4477248: 1.5% 1364384: 1.9% 42637
cp/pt.c:8316 (tsubst_decl) 2244888: 0.5% 0:
0.0% 4552704: 1.6% 357768: 0.5% 44721
tree.c:6061 (build_method_type_directly) 1946600: 0.4% 0:
0.0% 4703200: 1.6% 265992: 0.4% 33249
cgraph.c:638 (cgraph_create_edge) 17196288: 3.5% 0:
0.0% 5254848: 1.8% 0: 0.0% 233866
tree-inline.c:4045 (copy_decl_to_var) 145488: 0.0% 0:
0.0% 5593392: 1.9% 273280: 0.4% 34160
gimple-iterator.c:446 (gsi_insert_after_without_ 14342320: 2.9% 0:
0.0% 5650120: 1.9% 3998488: 5.7% 499811
ggc-common.c:187 (ggc_calloc) 16950080: 3.5% 3025816:
2.4% 6151656: 2.1% 459072: 0.6% 69247
tree-ssanames.c:141 (make_ssa_name_fn) 16930080: 3.5% 0:
0.0% 8363760: 2.9% 1686256: 2.4% 210782
gimplify.c:521 (create_tmp_var_raw) 5453784: 1.1% 0:
0.0% 9020256: 3.1% 689240: 1.0% 86155
tree.c:962 (build_int_cst_wide) 6096: 0.0% 0:
0.0% 10131688: 3.5% 3323928: 4.7% 2299
tree-inline.c:3589 (copy_tree_r) 49631032:10.2% 0:
0.0% 12553384: 4.3% 5797568: 8.2% 800223
tree-dfa.c:150 (create_var_ann) 0: 0.0%
27303320:22.0% 12672616: 4.3% 3634176: 5.1% 454272
gimple.c:2106 (gimple_copy) 11226992: 2.3% 0:
0.0% 13146032: 4.5% 1196784: 1.7% 209491
cp/lex.c:590 (copy_decl) 64104: 0.0% 0:
0.0% 13548720: 4.6% 1326296: 1.9% 56894
tree-inline.c:484 (remap_block) 1928264: 0.4% 0:
0.0% 14843088: 5.1% 1290104: 1.8% 161263
tree-ssa-operands.c:499 (ssa_operand_alloc) 0: 0.0%
34199342:27.6% 18090837: 6.2% 3566211: 5.0% 11251
tree-inline.c:4088 (copy_decl_no_change) 11756840: 2.4% 0:
0.0% 40988416:14.0% 2425144: 3.4% 317455
Total 487237966 123888014
293018044 70674672 9669114
source location Garbage Freed
Leak Overhead Times
so debug info and declarations are quite top. This is with my DECL_INGORED_P
fix I plan to commit to mainline soon.
5MB are also bitmaps
tree-ssa-operands.c:2381 (add_to_addressa 73585 9052240 5946000
4181320 173579
I suspect most of the rest are operand caches, since they are so ineffective
for small functions.
at end of compilation:
tree-inline.c:484 (remap_block) 29218176: 2.1% 0:
0.0% 104: 0.0% 2247560: 1.3% 280945
cselib.c:1155 (cselib_subst_to_values) 31320504: 2.3% 0:
0.0% 0: 0.0% 5958648: 3.4% 838942
cp/call.c:2346 (add_template_candidate_real) 31457040: 2.3% 0:
0.0% 0: 0.0% 3096816: 1.8% 457682
gimple-iterator.c:446 (gsi_insert_after_without_ 32515440: 2.3% 0:
0.0% 0: 0.0% 6503088: 3.7% 812886
tree-phinodes.c:157 (allocate_phi_node) 33375352: 2.4% 0:
0.0% 0: 0.0% 1120888: 0.6% 108792
ggc-common.c:187 (ggc_calloc) 34614992: 2.5% 9072016:
2.6% 1895328: 2.0% 671680: 0.4% 102129
rtl.c:269 (copy_rtx) 42322896: 3.1% 0:
0.0% 0: 0.0% 8318000: 4.7% 1083689
emit-rtl.c:3348 (make_insn_raw) 42838312: 3.1% 0:
0.0% 88: 0.0% 3894400: 2.2% 486800
gimple.c:2106 (gimple_copy) 43173352: 3.1% 0:
0.0% 0: 0.0% 2063688: 1.2% 368502
tree-ssanames.c:141 (make_ssa_name_fn) 73506000: 5.3% 0:
0.0% 26640: 0.0% 4902176: 2.8% 612772
tree-inline.c:4088 (copy_decl_no_change) 93714848: 6.8% 0:
0.0% 176464: 0.2% 4370928: 2.5% 562896
tree-inline.c:3589 (copy_tree_r) 98165464: 7.1% 0:
0.0% 2352: 0.0% 9178696: 5.2% 1250363
Total 1385145407 354509964
93434594 175468533 23822336
source location Garbage Freed
Leak Overhead Times
positive thing is that there are no leaked gimple statements at all. Most of
alocation at the end is:
cp/lex.c:590 (copy_decl) 1532928: 0.1% 0:
0.0% 12079896:12.9% 1326296: 0.8% 56894
tree.c:962 (build_int_cst_wide) 6096: 0.0% 0:
0.0% 10359544:11.1% 3388072: 1.9% 3011
tree.c:6061 (build_method_type_directly) 1947800: 0.1% 0:
0.0% 4703200: 5.0% 266040: 0.2% 33255
cp/pt.c:8316 (tsubst_decl) 2244888: 0.2% 0:
0.0% 4552704: 4.9% 357768: 0.2% 44721
DF and PRE allocate some giant bitmaps:
df-problems.c:308 (df_rd_alloc) 145581 12612800 11870840
11870840 597073
df-problems.c:309 (df_rd_alloc) 145581 8655600 8293080
8293080 108099
df-problems.c:310 (df_rd_alloc) 145581 15585520 14869800
14869800 1391724
tree-ssa-pre.c:584 (bitmap_set_new) 987262 68922080 53349440
53349440 2631124
tree-ssa-pre.c:585 (bitmap_set_new) 987262 69386800 53918200
53918200 3978100
df-problems.c:311 (df_rd_alloc) 145581 74605440 73361600
73361600 0
df-problems.c:539 (df_rd_transfer_functio 100011 63125520 42433280
42433280 148378
My guess is that ssa-operands can be easiest to track if I am right about their
memory usage.
Honza
--
hubicka at gcc dot gnu dot org changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |amacleod at redhat dot com
Last reconfirmed|2007-09-20 14:09:06 |2008-09-06 14:12:03
date| |
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12850
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug c++/12850] memory consumption for heavy template instantiations tripled since 3.3
[not found] <bug-12850-1824@http.gcc.gnu.org/bugzilla/>
` (3 preceding siblings ...)
2008-09-06 14:13 ` hubicka at gcc dot gnu dot org
@ 2009-02-23 16:29 ` hubicka at gcc dot gnu dot org
4 siblings, 0 replies; 26+ messages in thread
From: hubicka at gcc dot gnu dot org @ 2009-02-23 16:29 UTC (permalink / raw)
To: gcc-bugs
------- Comment #46 from hubicka at gcc dot gnu dot org 2009-02-23 16:29 -------
So with brand new tuplified world, we need new statistics ;)
After parsing we are still the same:
cfg.c:216 (connect_src) 608608: 0.2% 520:
0.0% 3028808: 1.6% 519680: 1.3% 64954
cp/lex.c:511 (build_lang_decl) 805432: 0.3% 209648:
0.6% 3196488: 1.7% 349552: 0.9% 18896
stringpool.c:73 (alloc_node) 65088: 0.0% 0:
0.0% 3208992: 1.7% 0: 0.0% 34105
fold-const.c:7969 (build_fold_addr_expr_with_typ 530352: 0.2% 0:
0.0% 3440880: 1.8% 441248: 1.1% 55156
cfg.c:226 (connect_dest) 598696: 0.2% 180224:
0.5% 3484960: 1.8% 594504: 1.5% 73663
cgraph.c:432 (cgraph_create_node) 0: 0.0% 0:
0.0% 3712320: 1.9% 412480: 1.1% 12890
gimple-low.c:888 (record_vars_into) 0: 0.0% 0:
0.0% 3808032: 2.0% 0: 0.0% 79334
cp/pt.c:8398 (tsubst_decl) 2244888: 0.9% 0:
0.0% 4552704: 2.4% 357768: 0.9% 44721
tree.c:6101 (build_method_type_directly) 1946800: 0.8% 0:
0.0% 4704000: 2.5% 266032: 0.7% 33254
tree-inline.c:3595 (copy_tree_r) 9428112: 3.7% 0:
0.0% 4793248: 2.5% 1243776: 3.2% 186815
cfg.c:142 (alloc_block) 1046016: 0.4% 0:
0.0% 4988448: 2.6% 0: 0.0% 62859
cgraph.c:681 (cgraph_create_edge) 0: 0.0% 0:
0.0% 5183328: 2.7% 0: 0.0% 53993
cfg.c:280 (unchecked_make_edge) 0: 0.0% 696256:
1.9% 5271424: 2.8% 0: 0.0% 93245
gimplify.c:4295 (gimplify_modify_expr) 1183600: 0.5% 0:
0.0% 5519400: 2.9% 300632: 0.8% 57164
gimple-iterator.c:446 (gsi_insert_after_without_ 4903960: 1.9% 0:
0.0% 5826440: 3.1% 2146080: 5.5% 268260
gimple.c:287 (gimple_build_call_1) 871144: 0.3% 0:
0.0% 6066056: 3.2% 247408: 0.6% 51874
tree.c:964 (build_int_cst_wide) 6096: 0.0% 0:
0.0% 10089256: 5.3% 3310168: 8.6% 2292
gimplify.c:522 (create_tmp_var_raw) 453768: 0.2% 0:
0.0% 10537632: 5.5% 523400: 1.4% 65425
cp/lex.c:591 (copy_decl) 26304: 0.0% 0:
0.0% 13586520: 7.1% 1326296: 3.4% 56894
Total 254844605 37217584
190405469 38695562 5886026
source location Garbage Freed
Leak Overhead Times
After early optimizations:
tree-inline.c:4051 (copy_decl_to_var) 145488: 0.0% 0:
0.0% 5595072: 2.0% 273360: 0.4% 34170
gimple-iterator.c:446 (gsi_insert_after_without_ 14165480: 3.2% 0:
0.0% 5633680: 2.0% 3959832: 5.9% 494979
ggc-common.c:187 (ggc_calloc) 565872: 0.1%
17553976:12.6% 5692736: 2.1% 423928: 0.6% 67925
cgraph.c:681 (cgraph_create_edge) 0: 0.0% 0:
0.0% 5958624: 2.2% 0: 0.0% 62069
tree-ssanames.c:141 (make_ssa_name_fn) 16722480: 3.8% 0:
0.0% 8318040: 3.0% 1669368: 2.5% 208671
gimplify.c:522 (create_tmp_var_raw) 5067048: 1.2% 0:
0.0% 8897280: 3.2% 664968: 1.0% 83121
tree.c:964 (build_int_cst_wide) 6096: 0.0% 0:
0.0% 10327480: 3.8% 3388168: 5.1% 2341
tree-dfa.c:150 (create_var_ann) 0: 0.0%
23279784:16.7% 10578392: 3.8% 3078016: 4.6% 384752
tree-inline.c:3595 (copy_tree_r) 49340008:11.2% 0:
0.0% 12558984: 4.6% 5762144: 8.6% 796535
gimple.c:2071 (gimple_copy) 11219128: 2.6% 0:
0.0% 13100280: 4.8% 1193408: 1.8% 209046
cp/lex.c:591 (copy_decl) 62424: 0.0% 0:
0.0% 13550400: 4.9% 1326296: 2.0% 56894
tree-inline.c:484 (remap_block) 1924416: 0.4% 0:
0.0% 14794832: 5.4% 1286096: 1.9% 160762
tree-ssa-operands.c:499 (ssa_operand_alloc) 0: 0.0%
33737566:24.3% 17976728: 6.5% 3547382: 5.3% 11222
tree-inline.c:4094 (copy_decl_no_change) 12372728: 2.8% 0:
0.0% 29185744:10.6% 1892440: 2.8% 250867
Total 439703478 139072686
274931123 66651575 9262384
source location Garbage Freed
Leak Overhead Times
Declarations and debug info being major consumer. We improved 293->274
and final compilation:
cp/call.c:2348 (add_template_candidate_real) 31457040: 2.5% 0:
0.0% 0: 0.0% 3096816: 1.8% 457682
gimple-iterator.c:446 (gsi_insert_after_without_ 32206200: 2.5% 0:
0.0% 0: 0.0% 6441240: 3.8% 805155
tree-phinodes.c:157 (allocate_phi_node) 33346952: 2.6% 0:
0.0% 0: 0.0% 1121800: 0.7% 108439
rtl.c:269 (copy_rtx) 41145992: 3.2% 0:
0.0% 0: 0.0% 8084728: 4.7% 1053375
emit-rtl.c:3502 (make_insn_raw) 41827016: 3.3% 0:
0.0% 88: 0.0% 3802464: 2.2% 475308
gimple.c:2071 (gimple_copy) 43033384: 3.4% 0:
0.0% 0: 0.0% 2056248: 1.2% 367296
tree-ssanames.c:141 (make_ssa_name_fn) 72524280: 5.7% 0:
0.0% 148440: 0.1% 4844848: 2.8% 605606
tree-inline.c:4094 (copy_decl_no_change) 74338752: 5.8% 0:
0.0% 226240: 0.2% 3451120: 2.0% 447839
tree-inline.c:3595 (copy_tree_r) 97555488: 7.7% 0:
0.0% 2792: 0.0% 9116248: 5.4% 1242196
Total 1271569773 404722617
103497642 170223016 23365837
source location Garbage Freed
Leak Overhead Times
1.38GB to 1.27GB...
so not much change, but some progress ;)
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12850
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug c++/12850] memory consumption for heavy template instantiations tripled since 3.3
2003-10-30 23:42 [Bug c++/12850] New: " jens dot maurer at gmx dot net
@ 2003-10-30 23:43 ` jens dot maurer at gmx dot net
2003-10-30 23:44 ` jens dot maurer at gmx dot net
` (17 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: jens dot maurer at gmx dot net @ 2003-10-30 23:43 UTC (permalink / raw)
To: gcc-bugs
PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12850
------- Additional Comments From jens dot maurer at gmx dot net 2003-10-30 23:42 -------
Created an attachment (id=5021)
--> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=5021&action=view)
preprocessed random_test.cpp for g++ 3.3
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug c++/12850] memory consumption for heavy template instantiations tripled since 3.3
2003-10-30 23:42 [Bug c++/12850] New: " jens dot maurer at gmx dot net
2003-10-30 23:43 ` [Bug c++/12850] " jens dot maurer at gmx dot net
@ 2003-10-30 23:44 ` jens dot maurer at gmx dot net
2003-10-31 0:04 ` jens dot maurer at gmx dot net
` (16 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: jens dot maurer at gmx dot net @ 2003-10-30 23:44 UTC (permalink / raw)
To: gcc-bugs
PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12850
------- Additional Comments From jens dot maurer at gmx dot net 2003-10-30 23:43 -------
Created an attachment (id=5022)
--> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=5022&action=view)
preprocessed random_test.cpp for g++ 3.4
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug c++/12850] memory consumption for heavy template instantiations tripled since 3.3
2003-10-30 23:42 [Bug c++/12850] New: " jens dot maurer at gmx dot net
2003-10-30 23:43 ` [Bug c++/12850] " jens dot maurer at gmx dot net
2003-10-30 23:44 ` jens dot maurer at gmx dot net
@ 2003-10-31 0:04 ` jens dot maurer at gmx dot net
2004-03-21 18:58 ` mmitchel at gcc dot gnu dot org
` (15 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: jens dot maurer at gmx dot net @ 2003-10-31 0:04 UTC (permalink / raw)
To: gcc-bugs
PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12850
jens dot maurer at gmx dot net changed:
What |Removed |Added
----------------------------------------------------------------------------
Attachment #5021|text/plain |application/x-gunzip
mime type| |
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug c++/12850] memory consumption for heavy template instantiations tripled since 3.3
2003-10-30 23:42 [Bug c++/12850] New: " jens dot maurer at gmx dot net
` (2 preceding siblings ...)
2003-10-31 0:04 ` jens dot maurer at gmx dot net
@ 2004-03-21 18:58 ` mmitchel at gcc dot gnu dot org
2004-05-05 23:31 ` hubicka at gcc dot gnu dot org
` (14 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: mmitchel at gcc dot gnu dot org @ 2004-03-21 18:58 UTC (permalink / raw)
To: gcc-bugs
------- Additional Comments From mmitchel at gcc dot gnu dot org 2004-03-21 18:58 -------
There's nothing more to be fixed here for 3.4.x, so I've retargeted this at 3.5.
--
What |Removed |Added
----------------------------------------------------------------------------
Summary|[3.4/3.5 Regression] memory |memory consumption for heavy
|consumption for heavy |template instantiations
|template instantiations |tripled since 3.3
|tripled since 3.3 |
Target Milestone|3.4.0 |3.5.0
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12850
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug c++/12850] memory consumption for heavy template instantiations tripled since 3.3
2003-10-30 23:42 [Bug c++/12850] New: " jens dot maurer at gmx dot net
` (3 preceding siblings ...)
2004-03-21 18:58 ` mmitchel at gcc dot gnu dot org
@ 2004-05-05 23:31 ` hubicka at gcc dot gnu dot org
2004-07-04 23:35 ` pinskia at gcc dot gnu dot org
` (13 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: hubicka at gcc dot gnu dot org @ 2004-05-05 23:31 UTC (permalink / raw)
To: gcc-bugs
------- Additional Comments From hubicka at gcc dot gnu dot org 2004-05-05 23:31 -------
We no longer have major memory consumption regression here. I don't want to
see it red ;)
--
What |Removed |Added
----------------------------------------------------------------------------
Severity|critical |normal
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12850
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug c++/12850] memory consumption for heavy template instantiations tripled since 3.3
2003-10-30 23:42 [Bug c++/12850] New: " jens dot maurer at gmx dot net
` (4 preceding siblings ...)
2004-05-05 23:31 ` hubicka at gcc dot gnu dot org
@ 2004-07-04 23:35 ` pinskia at gcc dot gnu dot org
2004-09-23 18:17 ` pinskia at gcc dot gnu dot org
` (12 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: pinskia at gcc dot gnu dot org @ 2004-07-04 23:35 UTC (permalink / raw)
To: gcc-bugs
--
What |Removed |Added
----------------------------------------------------------------------------
Target Milestone|3.5.0 |---
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12850
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug c++/12850] memory consumption for heavy template instantiations tripled since 3.3
2003-10-30 23:42 [Bug c++/12850] New: " jens dot maurer at gmx dot net
` (5 preceding siblings ...)
2004-07-04 23:35 ` pinskia at gcc dot gnu dot org
@ 2004-09-23 18:17 ` pinskia at gcc dot gnu dot org
2004-09-27 4:02 ` pinskia at gcc dot gnu dot org
` (11 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: pinskia at gcc dot gnu dot org @ 2004-09-23 18:17 UTC (permalink / raw)
To: gcc-bugs
------- Additional Comments From pinskia at gcc dot gnu dot org 2004-09-23 18:17 -------
could someone test this again (I think Jan's memory tester has the numbers for the mainline but I could
be wrong).
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12850
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug c++/12850] memory consumption for heavy template instantiations tripled since 3.3
2003-10-30 23:42 [Bug c++/12850] New: " jens dot maurer at gmx dot net
` (6 preceding siblings ...)
2004-09-23 18:17 ` pinskia at gcc dot gnu dot org
@ 2004-09-27 4:02 ` pinskia at gcc dot gnu dot org
2004-12-03 3:02 ` pinskia at gcc dot gnu dot org
` (10 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: pinskia at gcc dot gnu dot org @ 2004-09-27 4:02 UTC (permalink / raw)
To: gcc-bugs
------- Additional Comments From pinskia at gcc dot gnu dot org 2004-09-27 04:02 -------
Removing the patch keyword since all the patches referenced here have been applied.
--
What |Removed |Added
----------------------------------------------------------------------------
Keywords|patch |
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12850
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug c++/12850] memory consumption for heavy template instantiations tripled since 3.3
2003-10-30 23:42 [Bug c++/12850] New: " jens dot maurer at gmx dot net
` (7 preceding siblings ...)
2004-09-27 4:02 ` pinskia at gcc dot gnu dot org
@ 2004-12-03 3:02 ` pinskia at gcc dot gnu dot org
2004-12-11 17:43 ` pinskia at gcc dot gnu dot org
` (9 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: pinskia at gcc dot gnu dot org @ 2004-12-03 3:02 UTC (permalink / raw)
To: gcc-bugs
------- Additional Comments From pinskia at gcc dot gnu dot org 2004-12-03 03:02 -------
I should note that 4.0.0 is like 3x faster than 3.3.2 at -O1 on this test.
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12850
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug c++/12850] memory consumption for heavy template instantiations tripled since 3.3
2003-10-30 23:42 [Bug c++/12850] New: " jens dot maurer at gmx dot net
` (8 preceding siblings ...)
2004-12-03 3:02 ` pinskia at gcc dot gnu dot org
@ 2004-12-11 17:43 ` pinskia at gcc dot gnu dot org
2004-12-21 3:33 ` pinskia at gcc dot gnu dot org
` (8 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: pinskia at gcc dot gnu dot org @ 2004-12-11 17:43 UTC (permalink / raw)
To: gcc-bugs
------- Additional Comments From pinskia at gcc dot gnu dot org 2004-12-11 17:42 -------
On the mainline at -O1 (since I cannot compile at -O0 but that is a different bug which I already filed):
cp/lex.c:716 (copy_decl) 910284: 0.1% 0: 0.0% 6083812:11.3% 0: 0.0%
56404
ggc-common.c:193 (ggc_calloc) 3299632: 0.4% 11884228: 2.5% 1207832: 2.2%
2736868: 1.6% 22853
tree.c:4530 (build_method_type_directly) 1173592: 0.1% 0: 0.0% 2903048: 5.4%
750960: 0.4% 26820
tree.c:4266 (build_reference_type_for_mode) 456: 0.0% 0: 0.0% 602376: 1.1%
111048: 0.1% 3966
cp/class.c:2455 (maybe_add_class_template_decl_l 0: 0.0% 0: 0.0% 1061232: 2.0%
0: 0.0% 44218
tree.c:472 (copy_list) 2232: 0.0% 0: 0.0% 1083164: 2.0% 0: 0.0%
8854
Those are ones which leak still.
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12850
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug c++/12850] memory consumption for heavy template instantiations tripled since 3.3
2003-10-30 23:42 [Bug c++/12850] New: " jens dot maurer at gmx dot net
` (9 preceding siblings ...)
2004-12-11 17:43 ` pinskia at gcc dot gnu dot org
@ 2004-12-21 3:33 ` pinskia at gcc dot gnu dot org
2004-12-21 5:53 ` pinskia at gcc dot gnu dot org
` (7 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: pinskia at gcc dot gnu dot org @ 2004-12-21 3:33 UTC (permalink / raw)
To: gcc-bugs
--
Bug 12850 depends on bug 18683, which changed state.
Bug 18683 Summary: [4.0 Regression] seg fault in local allocate
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=18683
What |Old Value |New Value
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution| |FIXED
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12850
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug c++/12850] memory consumption for heavy template instantiations tripled since 3.3
2003-10-30 23:42 [Bug c++/12850] New: " jens dot maurer at gmx dot net
` (10 preceding siblings ...)
2004-12-21 3:33 ` pinskia at gcc dot gnu dot org
@ 2004-12-21 5:53 ` pinskia at gcc dot gnu dot org
2004-12-23 12:20 ` steven at gcc dot gnu dot org
` (6 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: pinskia at gcc dot gnu dot org @ 2004-12-21 5:53 UTC (permalink / raw)
To: gcc-bugs
------- Additional Comments From pinskia at gcc dot gnu dot org 2004-12-21 05:53 -------
Here are the results for -O0, now that PR 18683 is now fixed:
cp/lex.c:716 (copy_decl) 1087604: 0.3% 0: 0.0% 5906492:10.6% 0: 0.0%
56404
cp/pt.c:3978 (coerce_template_parms) 41586524: 9.6% 0: 0.0% 136540: 0.2%
3865680: 7.4% 1138236
Though we do create a lot:
cp/parser.c:278 (cp_lexer_new_main) 0: 0.0% 22585856:36.1% 0: 0.0%
6332928:12.1% 5
Which is mostly a ggc_realloc of a buffer of all the tokens, maybe there is a better way of allocating this
buffer as it seems like we create a lot of overhead because ot it.
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12850
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug c++/12850] memory consumption for heavy template instantiations tripled since 3.3
2003-10-30 23:42 [Bug c++/12850] New: " jens dot maurer at gmx dot net
` (11 preceding siblings ...)
2004-12-21 5:53 ` pinskia at gcc dot gnu dot org
@ 2004-12-23 12:20 ` steven at gcc dot gnu dot org
2004-12-23 14:42 ` gdr at integrable-solutions dot net
` (5 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: steven at gcc dot gnu dot org @ 2004-12-23 12:20 UTC (permalink / raw)
To: gcc-bugs
------- Additional Comments From steven at gcc dot gnu dot org 2004-12-23 12:19 -------
The initial CP lexer bugger size is 10000:
#define CP_LEXER_BUFFER_SIZE 10000
That came in with the lex-all-ahead patch from Matt and Zack,
on 2004-09-20 (parser.c rev. 1.250 for the CVS history diggers)
but it seems a bit low to me if you're going to lex the whole
file up front. I would not be surprised if the average C++
code with lots of templates has several 100,000 tokens... Let
me see:
- preprocessed sources for generate.ii from PR8361, blank and
pound lines stripped: 36200 lines
- an average of 7 tokens per line in the first 500 lines, let's
assume that's a reasonable average for the whole file (it's
easy to instrument g++ to get the exact number of tokens, if
you want more accurate numbers ;-)
That makes it >250,000 tokens for this file.
Since we double the buffer, we have:
10,000 + 20,000 + 40,000 + 80,000 + 160,000 + 320,000 = 630000
That is the number of tokens we have allocate room for, with no
ggc-collect in the middle. With ggc-page, which has power-of-2
based page sizes, it's safe to assume that each previous buffer
is too small to be reallocated, so a full new buffer is allocated
and the old one is memcpy-ed to the new one. With checking off,
we ggc_free the old buffer, but with checking enabled we don't
so after finishing the whole lexing process, we have keep around
a buffer of ~380,000*sizeof(cp_token), so that's roughly 10MB
of memory we can't reclaim until the first ggc_collect call.
Maybe buffer should not be in GC memory at all? We know the
exact live time of buffer, and as far as I can tell we never
ggc_collect while it is live. According to the comments for
cp_lexer, "Tokens are never added to the cp_lexer after it is
created." So it may be cheaper to have the buffer xmalloced,
and memcpy-ed to a buffer in GC space just before saving it
in the new cp_lexer object.
So two suggestions for a person who wants to make g++ a little
faster here:
- make CP_LEXER_BUFFER_SIZE larger. To make it use pages more
efficiently, look for some ratio of pagesize/(sizeof (cp_token))
- see buffer in parser.c:cp_lexer_new_main can be moved out of GC
space as suggested above.
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12850
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug c++/12850] memory consumption for heavy template instantiations tripled since 3.3
2003-10-30 23:42 [Bug c++/12850] New: " jens dot maurer at gmx dot net
` (12 preceding siblings ...)
2004-12-23 12:20 ` steven at gcc dot gnu dot org
@ 2004-12-23 14:42 ` gdr at integrable-solutions dot net
2004-12-28 16:43 ` pinskia at gcc dot gnu dot org
` (4 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: gdr at integrable-solutions dot net @ 2004-12-23 14:42 UTC (permalink / raw)
To: gcc-bugs
------- Additional Comments From gdr at integrable-solutions dot net 2004-12-23 14:42 -------
Subject: Re: memory consumption for heavy template instantiations tripled since 3.3
"steven at gcc dot gnu dot org" <gcc-bugzilla@gcc.gnu.org> writes:
[...]
| Maybe buffer should not be in GC memory at all? We know the
| exact live time of buffer, and as far as I can tell we never
| ggc_collect while it is live. According to the comments for
| cp_lexer, "Tokens are never added to the cp_lexer after it is
| created." So it may be cheaper to have the buffer xmalloced,
| and memcpy-ed to a buffer in GC space just before saving it
| in the new cp_lexer object.
Your analysis makes sense to me. I never quite understood the
addiction to GC-allocated memory throughout the compiler.
-- Gaby
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12850
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug c++/12850] memory consumption for heavy template instantiations tripled since 3.3
2003-10-30 23:42 [Bug c++/12850] New: " jens dot maurer at gmx dot net
` (13 preceding siblings ...)
2004-12-23 14:42 ` gdr at integrable-solutions dot net
@ 2004-12-28 16:43 ` pinskia at gcc dot gnu dot org
2004-12-29 12:57 ` steven at gcc dot gnu dot org
` (3 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: pinskia at gcc dot gnu dot org @ 2004-12-28 16:43 UTC (permalink / raw)
To: gcc-bugs
------- Additional Comments From pinskia at gcc dot gnu dot org 2004-12-28 16:43 -------
(In reply to comment #36)
> The initial CP lexer bugger size is 10000:
The same amount of garbage is also done for PR 8361.
Also note I could not compile this source again becuase of the use of long double which causes an ICE
for ppc-darwin but that has been fixed already.
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12850
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug c++/12850] memory consumption for heavy template instantiations tripled since 3.3
2003-10-30 23:42 [Bug c++/12850] New: " jens dot maurer at gmx dot net
` (14 preceding siblings ...)
2004-12-28 16:43 ` pinskia at gcc dot gnu dot org
@ 2004-12-29 12:57 ` steven at gcc dot gnu dot org
2004-12-29 13:10 ` steven at gcc dot gnu dot org
` (2 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: steven at gcc dot gnu dot org @ 2004-12-29 12:57 UTC (permalink / raw)
To: gcc-bugs
------- Additional Comments From steven at gcc dot gnu dot org 2004-12-29 12:57 -------
I've been looking at a bunch of C++ codes, 160000 or 320000 seems like
a reasonable value for CP_LEXER_BUFFER_SIZE.
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12850
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug c++/12850] memory consumption for heavy template instantiations tripled since 3.3
2003-10-30 23:42 [Bug c++/12850] New: " jens dot maurer at gmx dot net
` (15 preceding siblings ...)
2004-12-29 12:57 ` steven at gcc dot gnu dot org
@ 2004-12-29 13:10 ` steven at gcc dot gnu dot org
2005-07-23 22:32 ` pinskia at gcc dot gnu dot org
2005-07-24 4:50 ` gdr at integrable-solutions dot net
18 siblings, 0 replies; 26+ messages in thread
From: steven at gcc dot gnu dot org @ 2004-12-29 13:10 UTC (permalink / raw)
To: gcc-bugs
------- Additional Comments From steven at gcc dot gnu dot org 2004-12-29 13:09 -------
Trivial 6MB win:
Index: parser.c
===================================================================
RCS file: /cvs/gcc/gcc/gcc/cp/parser.c,v
retrieving revision 1.298
diff -u -r1.298 parser.c
--- parser.c 23 Dec 2004 22:07:01 -0000 1.298
+++ parser.c 29 Dec 2004 13:06:30 -0000
@@ -190,7 +190,7 @@
(cp_token *, cp_token *);
/* Manifest constants. */
-#define CP_LEXER_BUFFER_SIZE 10000
+#define CP_LEXER_BUFFER_SIZE 160000
#define CP_SAVED_TOKEN_STACK 5
/* A token type for keywords, as opposed to ordinary identifiers. */
This does not fix the underlying problem that the buffer resizing in GC
space gives a quadratic behavior in storage allocation, but it avoids it
for most files, and it gives a ~2% speedup at -O0 on my box.
Stats for cp_lexer_new_main for the test case from PR8361 (-O0):
Before:
source location Freed Leak Overhead Times
cp/parser.c:263 728576: 1.2% 0: 0.0% 204288: 0.5% 1
cp/parser.c:278 45171712:71.7% 0: 0.0% 12665856:29.2% 5
cp/parser.c:253 72: 0.0% 0: 0.0% 8: 0.0% 1
After:
source location Freed Leak Overhead Times
cp/parser.c:263 11657216:22.4% 0: 0.0% 3268608: 8.1% 1
cp/parser.c:278 23314432:44.8% 0: 0.0% 6537216:16.2% 1
cp/parser.c:253 72: 0.0% 0: 0.0% 8: 0.0% 1
Perhaps we should look for an altogether different data structure for the
token buffer - some kind of vector of smaller buffers perhaps.
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12850
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug c++/12850] memory consumption for heavy template instantiations tripled since 3.3
2003-10-30 23:42 [Bug c++/12850] New: " jens dot maurer at gmx dot net
` (16 preceding siblings ...)
2004-12-29 13:10 ` steven at gcc dot gnu dot org
@ 2005-07-23 22:32 ` pinskia at gcc dot gnu dot org
2005-07-24 4:50 ` gdr at integrable-solutions dot net
18 siblings, 0 replies; 26+ messages in thread
From: pinskia at gcc dot gnu dot org @ 2005-07-23 22:32 UTC (permalink / raw)
To: gcc-bugs
------- Additional Comments From pinskia at gcc dot gnu dot org 2005-07-23 22:14 -------
cp/tree.c:827 (ovl_cons) 11464712: 3.2% 0: 0.0% 660240: 1.4% 1732136:
5.2% 433034
Hmm OVERLOAD tree takes 3% of the Garbage which seems like too big, though I don't know how big
long the OVERLOAD trees are, I might add something to count that.
--
What |Removed |Added
----------------------------------------------------------------------------
BugsThisDependsOn| |22635
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12850
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug c++/12850] memory consumption for heavy template instantiations tripled since 3.3
2003-10-30 23:42 [Bug c++/12850] New: " jens dot maurer at gmx dot net
` (17 preceding siblings ...)
2005-07-23 22:32 ` pinskia at gcc dot gnu dot org
@ 2005-07-24 4:50 ` gdr at integrable-solutions dot net
18 siblings, 0 replies; 26+ messages in thread
From: gdr at integrable-solutions dot net @ 2005-07-24 4:50 UTC (permalink / raw)
To: gcc-bugs
------- Additional Comments From gdr at integrable-solutions dot net 2005-07-24 03:36 -------
Subject: Re: memory consumption for heavy template instantiations tripled since 3.3
"pinskia at gcc dot gnu dot org" <gcc-bugzilla@gcc.gnu.org> writes:
| cp/tree.c:827 (ovl_cons) 11464712: 3.2% 0: 0.0% 660240: 1.4% 1732136:
| 5.2% 433034
|
| Hmm OVERLOAD tree takes 3% of the Garbage which seems like too big,
| though I don't know how big
| long the OVERLOAD trees are, I might add something to count that.
It is not uncommon to have large overload sets in C++ -- that is what
people do when they discover that they can overload in the literal
sense ;-)
-- Gaby
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12850
^ permalink raw reply [flat|nested] 26+ messages in thread