* [Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile
2012-08-20 18:06 [Bug fortran/54332] New: [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile hjl.tools at gmail dot com
@ 2012-08-20 20:53 ` hjl.tools at gmail dot com
2012-08-20 21:18 ` hjl.tools at gmail dot com
` (27 subsequent siblings)
28 siblings, 0 replies; 30+ messages in thread
From: hjl.tools at gmail dot com @ 2012-08-20 20:53 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
H.J. Lu <hjl.tools at gmail dot com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |areg.melikadamyan at gmail
| |dot com, dnovillo at google
| |dot com
--- Comment #1 from H.J. Lu <hjl.tools at gmail dot com> 2012-08-20 20:53:14 UTC ---
It is caused by revision 190402:
http://gcc.gnu.org/ml/gcc-cvs/2012-08/msg00379.html
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile
2012-08-20 18:06 [Bug fortran/54332] New: [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile hjl.tools at gmail dot com
2012-08-20 20:53 ` [Bug fortran/54332] " hjl.tools at gmail dot com
@ 2012-08-20 21:18 ` hjl.tools at gmail dot com
2012-08-20 21:21 ` steven at gcc dot gnu.org
` (26 subsequent siblings)
28 siblings, 0 replies; 30+ messages in thread
From: hjl.tools at gmail dot com @ 2012-08-20 21:18 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
--- Comment #2 from H.J. Lu <hjl.tools at gmail dot com> 2012-08-20 21:18:00 UTC ---
Revision 190401 takes 512MB virtual memory to compile module_domain.fppized.f90
while revision 190402 takes 10GB. This is a 20x increase.
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile
2012-08-20 18:06 [Bug fortran/54332] New: [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile hjl.tools at gmail dot com
2012-08-20 20:53 ` [Bug fortran/54332] " hjl.tools at gmail dot com
2012-08-20 21:18 ` hjl.tools at gmail dot com
@ 2012-08-20 21:21 ` steven at gcc dot gnu.org
2012-08-20 21:42 ` hjl.tools at gmail dot com
` (25 subsequent siblings)
28 siblings, 0 replies; 30+ messages in thread
From: steven at gcc dot gnu.org @ 2012-08-20 21:21 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
Steven Bosscher <steven at gcc dot gnu.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Keywords| |memory-hog
Status|UNCONFIRMED |NEW
Last reconfirmed| |2012-08-20
CC| |steven at gcc dot gnu.org
Ever Confirmed|0 |1
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile
2012-08-20 18:06 [Bug fortran/54332] New: [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile hjl.tools at gmail dot com
` (2 preceding siblings ...)
2012-08-20 21:21 ` steven at gcc dot gnu.org
@ 2012-08-20 21:42 ` hjl.tools at gmail dot com
2012-08-21 2:59 ` hjl.tools at gmail dot com
` (24 subsequent siblings)
28 siblings, 0 replies; 30+ messages in thread
From: hjl.tools at gmail dot com @ 2012-08-20 21:42 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
H.J. Lu <hjl.tools at gmail dot com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |UNCONFIRMED
Ever Confirmed|1 |0
--- Comment #3 from H.J. Lu <hjl.tools at gmail dot com> 2012-08-20 21:41:46 UTC ---
Revision 190402 memory stat:
Memory consumption before IPA
Number of expanded macros: 0
Line Table allocations during the compilation process
Number of ordinary maps used: 3
Ordinary map used size: 120
Number of ordinary maps allocated: 409
Ordinary maps allocated size: 15k
Number of macro maps used: 0
Macro maps used size: 0
Macro maps locations size: 0
Macro maps size: 0
Duplicated maps locations size: 0
Total allocated maps size: 15k
Total used maps size: 120
Memory still allocated at the end of the compilation process
Size Allocated Used Overhead
8 36k 11k 1080
16 96k 82k 2112
32 132k 62k 2376
64 1460k 1338k 22k
256 440k 432k 6160
512 8192 3584 112
1024 16k 11k 224
2048 12k 8192 168
4096 84k 84k 1176
8192 32k 32k 224
16384 48k 48k 168
65536 64k 64k 56
131072 128k 128k 56
262144 512k 512k 112
24 68k 32k 1224
40 3740k 1482k 58k
48 2108k 876k 32k
56 1396k 913k 21k
72 28k 2232 392
80 104k 101k 1456
88 16k 2200 224
96 4252k 4140k 58k
112 16k 12k 224
120 8192 5160 112
184 72k 67k 1008
128 36k 14k 504
152 4324k 4168k 59k
168 788k 362k 10k
160 504k 487k 7056
104 2836k 2770k 38k
312 212k 208k 2968
136 4096 2448 56
Total 23M 18M 331k
String pool
entries 10710
identifiers 7074 (66.05%)
slots 16384
deleted 3636
bytes 86k (17592186044415M overhead)
table size 128k
coll/search 0.6146
ins/search 0.2503
avg. entry 8.25 bytes (+/- 8.27)
longest entry 46
(No per-node statistics)
Type hash: size 2039, 996 elements, 1.024948 collisions
DECL_DEBUG_EXPR hash: size 1021, 294 elements, 0.128814 collisions
DECL_VALUE_EXPR hash: size 1021, 0 elements, 0.000000 collisions
No gimple statistics
No RTX statistics
Alias oracle query stats:
refs_may_alias_p: 0 disambiguations, 0 queries
ref_maybe_used_by_call_p: 0 disambiguations, 0 queries
call_may_clobber_ref_p: 0 disambiguations, 0 queries
PTA query stats:
pt_solution_includes: 0 disambiguations, 0 queries
pt_solutions_intersect: 0 disambiguations, 0 queries
Memory consumption after IPA
Number of expanded macros: 0
Line Table allocations during the compilation process
Number of ordinary maps used: 3
Ordinary map used size: 120
Number of ordinary maps allocated: 409
Ordinary maps allocated size: 15k
Number of macro maps used: 0
Macro maps used size: 0
Macro maps locations size: 0
Macro maps size: 0
Duplicated maps locations size: 0
Total allocated maps size: 15k
Total used maps size: 120
Memory still allocated at the end of the compilation process
Size Allocated Used Overhead
8 36k 11k 1080
16 96k 82k 2112
32 132k 70k 2376
64 1460k 1160k 22k
256 1724k 1722k 23k
512 8192 5120 112
1024 28k 27k 392
2048 12k 8192 168
4096 116k 116k 1624
8192 32k 32k 224
16384 4544k 4544k 15k
65536 64k 64k 56
131072 128k 128k 56
262144 512k 512k 112
524288 512k 512k 56
24 200k 74k 3600
40 3740k 1280k 58k
48 2108k 425k 32k
56 1396k 910k 21k
72 28k 2232 392
80 4824k 2840k 65k
88 16k 2024 224
96 4252k 1954k 58k
112 16k 13k 224
120 8192 6240 112
184 72k 67k 1008
128 36k 14k 504
152 4320k 3675k 59k
168 788k 362k 10k
160 504k 441k 7056
104 2836k 2327k 38k
312 212k 209k 2968
136 4096 2448 56
Total 33M 23M 431k
String pool
entries 10723
identifiers 7066 (65.90%)
slots 16384
deleted 3656
bytes 86k (17592186044415M overhead)
table size 128k
coll/search 0.6149
ins/search 0.2505
avg. entry 8.23 bytes (+/- 8.27)
longest entry 46
(No per-node statistics)
Type hash: size 2039, 996 elements, 1.024948 collisions
DECL_DEBUG_EXPR hash: size 1021, 290 elements, 0.128814 collisions
DECL_VALUE_EXPR hash: size 1021, 0 elements, 0.000000 collisions
No gimple statistics
No RTX statistics
Alias oracle query stats:
refs_may_alias_p: 0 disambiguations, 0 queries
ref_maybe_used_by_call_p: 1022 disambiguations, 1 queries
call_may_clobber_ref_p: 1022 disambiguations, 1022 queries
PTA query stats:
pt_solution_includes: 294 disambiguations, 1048 queries
pt_solutions_intersect: 2887 disambiguations, 289744 queries
Number of expanded macros: 0
Line Table allocations during the compilation process
Number of ordinary maps used: 3
Ordinary map used size: 120
Number of ordinary maps allocated: 409
Ordinary maps allocated size: 15k
Number of macro maps used: 0
Macro maps used size: 0
Macro maps locations size: 0
Macro maps size: 0
Duplicated maps locations size: 0
Total allocated maps size: 15k
Total used maps size: 120
Memory still allocated at the end of the compilation process
Size Allocated Used Overhead
8 24k 10k 720
16 1132k 250k 24k
32 2648k 534k 46k
64 3884k 1108k 60k
256 504k 439k 7056
512 4096 2048 56
1024 12k 6144 168
4096 88k 88k 1232
8192 32k 32k 224
16384 32k 32k 112
32768 32k 32k 56
65536 128k 128k 112
262144 256k 256k 56
524288 512k 512k 56
24 7648k 1015k 134k
40 4100k 1059k 64k
48 1704k 43k 26k
56 1340k 67k 20k
72 4640k 540k 63k
80 2900k 622k 39k
88 8192 1320 112
96 1404k 102k 19k
112 20k 12k 280
120 12k 5160 168
184 72k 67k 1008
128 20k 14k 280
152 4428k 3615k 60k
168 788k 356k 10k
160 8192 320 112
104 3812k 1237k 52k
312 212k 209k 2968
136 16k 2448 224
Total 41M 12M 637k
String pool
entries 15310
identifiers 7154 (46.73%)
slots 32768
deleted 4403
bytes 89k (17592186044415M overhead)
table size 256k
coll/search 0.6751
ins/search 0.2777
avg. entry 5.97 bytes (+/- 7.90)
longest entry 46
(No per-node statistics)
Type hash: size 2039, 989 elements, 0.520436 collisions
DECL_DEBUG_EXPR hash: size 1021, 289 elements, 0.128814 collisions
DECL_VALUE_EXPR hash: size 1021, 0 elements, 0.000000 collisions
No gimple statistics
No RTX statistics
Alias oracle query stats:
refs_may_alias_p: 37013 disambiguations, 46240 queries
ref_maybe_used_by_call_p: 3806 disambiguations, 37502 queries
call_may_clobber_ref_p: 3777 disambiguations, 3777 queries
PTA query stats:
pt_solution_includes: 805 disambiguations, 4942 queries
pt_solutions_intersect: 15712 disambiguations, 3621558 queries
Revision 190401 memory stat:
[hjl@gnu-ivb-1 build_peak_lnx32e-gcc.0000]$ cat nohup.out
Memory consumption before IPA
Number of expanded macros: 0
Line Table allocations during the compilation process
Number of ordinary maps used: 3
Ordinary map used size: 120
Number of ordinary maps allocated: 409
Ordinary maps allocated size: 15k
Number of macro maps used: 0
Macro maps used size: 0
Macro maps locations size: 0
Macro maps size: 0
Duplicated maps locations size: 0
Total allocated maps size: 15k
Total used maps size: 120
Memory still allocated at the end of the compilation process
Size Allocated Used Overhead
8 36k 11k 1080
16 96k 82k 2112
32 132k 62k 2376
64 1460k 1338k 22k
256 440k 432k 6160
512 8192 3584 112
1024 16k 11k 224
2048 12k 8192 168
4096 84k 84k 1176
8192 32k 32k 224
16384 48k 48k 168
65536 64k 64k 56
131072 128k 128k 56
262144 512k 512k 112
24 68k 32k 1224
40 3740k 1482k 58k
48 2108k 876k 32k
56 1396k 913k 21k
72 28k 2232 392
80 104k 101k 1456
88 16k 2200 224
96 4252k 4140k 58k
112 16k 12k 224
120 8192 5160 112
184 72k 67k 1008
128 36k 14k 504
152 4324k 4168k 59k
168 788k 362k 10k
160 504k 487k 7056
104 2836k 2770k 38k
312 212k 208k 2968
136 4096 2448 56
Total 23M 18M 331k
String pool
entries 10710
identifiers 7074 (66.05%)
slots 16384
deleted 3636
bytes 86k (17592186044415M overhead)
table size 128k
coll/search 0.6146
ins/search 0.2503
avg. entry 8.25 bytes (+/- 8.27)
longest entry 46
(No per-node statistics)
Type hash: size 2039, 996 elements, 1.024948 collisions
DECL_DEBUG_EXPR hash: size 1021, 294 elements, 0.128814 collisions
DECL_VALUE_EXPR hash: size 1021, 0 elements, 0.000000 collisions
No gimple statistics
No RTX statistics
Alias oracle query stats:
refs_may_alias_p: 0 disambiguations, 0 queries
ref_maybe_used_by_call_p: 0 disambiguations, 0 queries
call_may_clobber_ref_p: 0 disambiguations, 0 queries
PTA query stats:
pt_solution_includes: 0 disambiguations, 0 queries
pt_solutions_intersect: 0 disambiguations, 0 queries
Memory consumption after IPA
Number of expanded macros: 0
Line Table allocations during the compilation process
Number of ordinary maps used: 3
Ordinary map used size: 120
Number of ordinary maps allocated: 409
Ordinary maps allocated size: 15k
Number of macro maps used: 0
Macro maps used size: 0
Macro maps locations size: 0
Macro maps size: 0
Duplicated maps locations size: 0
Total allocated maps size: 15k
Total used maps size: 120
Memory still allocated at the end of the compilation process
Size Allocated Used Overhead
8 36k 11k 1080
16 96k 82k 2112
32 132k 70k 2376
64 1460k 1160k 22k
256 1724k 1722k 23k
512 8192 5120 112
1024 28k 27k 392
2048 12k 8192 168
4096 116k 116k 1624
8192 32k 32k 224
16384 4544k 4544k 15k
65536 64k 64k 56
131072 128k 128k 56
262144 512k 512k 112
524288 512k 512k 56
24 200k 74k 3600
40 3740k 1280k 58k
48 2108k 425k 32k
56 1396k 910k 21k
72 28k 2232 392
80 4824k 2840k 65k
88 16k 2024 224
96 4252k 1954k 58k
112 16k 13k 224
120 8192 6240 112
184 72k 67k 1008
128 36k 14k 504
152 4320k 3675k 59k
168 788k 362k 10k
160 504k 441k 7056
104 2836k 2327k 38k
312 212k 209k 2968
136 4096 2448 56
Total 33M 23M 431k
String pool
entries 10723
identifiers 7066 (65.90%)
slots 16384
deleted 3656
bytes 86k (17592186044415M overhead)
table size 128k
coll/search 0.6149
ins/search 0.2505
avg. entry 8.23 bytes (+/- 8.27)
longest entry 46
(No per-node statistics)
Type hash: size 2039, 996 elements, 1.024948 collisions
DECL_DEBUG_EXPR hash: size 1021, 290 elements, 0.128814 collisions
DECL_VALUE_EXPR hash: size 1021, 0 elements, 0.000000 collisions
No gimple statistics
No RTX statistics
Alias oracle query stats:
refs_may_alias_p: 0 disambiguations, 0 queries
ref_maybe_used_by_call_p: 1022 disambiguations, 1 queries
call_may_clobber_ref_p: 1022 disambiguations, 1022 queries
PTA query stats:
pt_solution_includes: 294 disambiguations, 1048 queries
pt_solutions_intersect: 2887 disambiguations, 289744 queries
Number of expanded macros: 0
Line Table allocations during the compilation process
Number of ordinary maps used: 3
Ordinary map used size: 120
Number of ordinary maps allocated: 409
Ordinary maps allocated size: 15k
Number of macro maps used: 0
Macro maps used size: 0
Macro maps locations size: 0
Macro maps size: 0
Duplicated maps locations size: 0
Total allocated maps size: 15k
Total used maps size: 120
Memory still allocated at the end of the compilation process
Size Allocated Used Overhead
8 24k 10k 720
16 1132k 250k 24k
32 2648k 534k 46k
64 3884k 1108k 60k
256 504k 439k 7056
512 4096 2048 56
1024 12k 6144 168
4096 88k 88k 1232
8192 32k 32k 224
16384 32k 32k 112
32768 32k 32k 56
65536 128k 128k 112
262144 256k 256k 56
524288 512k 512k 56
24 7648k 1015k 134k
40 4100k 1059k 64k
48 1704k 43k 26k
56 1340k 67k 20k
72 4640k 540k 63k
80 2900k 622k 39k
88 8192 1320 112
96 1404k 102k 19k
112 20k 12k 280
120 12k 5160 168
184 72k 67k 1008
128 20k 14k 280
152 4428k 3615k 60k
168 788k 356k 10k
160 8192 320 112
104 3812k 1237k 52k
312 212k 209k 2968
136 16k 2448 224
Total 41M 12M 637k
String pool
entries 15310
identifiers 7154 (46.73%)
slots 32768
deleted 4403
bytes 89k (17592186044415M overhead)
table size 256k
coll/search 0.6751
ins/search 0.2777
avg. entry 5.97 bytes (+/- 7.90)
longest entry 46
(No per-node statistics)
Type hash: size 2039, 989 elements, 0.520436 collisions
DECL_DEBUG_EXPR hash: size 1021, 289 elements, 0.128814 collisions
DECL_VALUE_EXPR hash: size 1021, 0 elements, 0.000000 collisions
No gimple statistics
No RTX statistics
Alias oracle query stats:
refs_may_alias_p: 37013 disambiguations, 46240 queries
ref_maybe_used_by_call_p: 3806 disambiguations, 37502 queries
call_may_clobber_ref_p: 3777 disambiguations, 3777 queries
PTA query stats:
pt_solution_includes: 805 disambiguations, 4942 queries
pt_solutions_intersect: 15712 disambiguations, 3621558 queries
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile
2012-08-20 18:06 [Bug fortran/54332] New: [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile hjl.tools at gmail dot com
` (3 preceding siblings ...)
2012-08-20 21:42 ` hjl.tools at gmail dot com
@ 2012-08-21 2:59 ` hjl.tools at gmail dot com
2012-08-21 8:26 ` rguenth at gcc dot gnu.org
` (23 subsequent siblings)
28 siblings, 0 replies; 30+ messages in thread
From: hjl.tools at gmail dot com @ 2012-08-21 2:59 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
--- Comment #4 from H.J. Lu <hjl.tools at gmail dot com> 2012-08-21 02:59:15 UTC ---
It was introduced between revision 189101 and revision 189664
on cxx-conversion branch. Unfortunately, since branch was broken
between those 2 revisions, I can't bisect further.
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile
2012-08-20 18:06 [Bug fortran/54332] New: [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile hjl.tools at gmail dot com
` (4 preceding siblings ...)
2012-08-21 2:59 ` hjl.tools at gmail dot com
@ 2012-08-21 8:26 ` rguenth at gcc dot gnu.org
2012-08-21 13:39 ` dnovillo at google dot com
` (22 subsequent siblings)
28 siblings, 0 replies; 30+ messages in thread
From: rguenth at gcc dot gnu.org @ 2012-08-21 8:26 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
Richard Guenther <rguenth at gcc dot gnu.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Target Milestone|--- |4.8.0
--- Comment #5 from Richard Guenther <rguenth at gcc dot gnu.org> 2012-08-21 08:26:03 UTC ---
(In reply to comment #4)
> It was introduced between revision 189101 and revision 189664
> on cxx-conversion branch. Unfortunately, since branch was broken
> between those 2 revisions, I can't bisect further.
I only see
r189664 | dnovillo | 2012-07-19 16:40:11 +0200 (Thu, 19 Jul 2012) | 3 lines
Merge from trunk rev 189106.
"inbetween" those two revisions. r189668 is another merge from trunk,
so is r188963. So I suspect a mismerge in r189106, r188963 was
r188963 | dnovillo | 2012-06-25 23:10:33 +0200 (Mon, 25 Jun 2012) | 7 lines
Merge from trunk
Merged revisions
188725-188729,188731,188733,188738-188749,188751-188755,188759,
188764-188765,188771,188776,188778,188780-188789,188791-188796,188802-188808,188
814-188815,188818,188820-188824,188826-188827,188829,188833,188838,188840-188843
,188847,188849,188852-188853,188856-188860,188865-188872,188874-188876,188880-18
8881,188884-188885,188891,188893,188900-188902,188906-188909,188913,188915-18891
8,188922 via svnmerge from
svn+ssh://gcc.gnu.org/svn/gcc/trunk
HJ, maybe you can help narrowing down things by trying different optimization
levels? I see that using a memleak checker may not be possible with
10G memory usage.
Note that -fmem-report does not track all allocations, esp. heap hashtables
are not tracked.
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile
2012-08-20 18:06 [Bug fortran/54332] New: [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile hjl.tools at gmail dot com
` (5 preceding siblings ...)
2012-08-21 8:26 ` rguenth at gcc dot gnu.org
@ 2012-08-21 13:39 ` dnovillo at google dot com
2012-08-21 13:58 ` hjl.tools at gmail dot com
` (21 subsequent siblings)
28 siblings, 0 replies; 30+ messages in thread
From: dnovillo at google dot com @ 2012-08-21 13:39 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
--- Comment #6 from dnovillo at google dot com <dnovillo at google dot com> 2012-08-21 13:38:24 UTC ---
On 2012-08-20 22:59 , hjl.tools at gmail dot com wrote:
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
>
> --- Comment #4 from H.J. Lu <hjl.tools at gmail dot com> 2012-08-21 02:59:15 UTC ---
> It was introduced between revision 189101 and revision 189664
> on cxx-conversion branch. Unfortunately, since branch was broken
> between those 2 revisions, I can't bisect further.
>
There was no rev 189101 in cxx-conversion. That is a trunk revision.
In that range of revisions, there are only merges from trunk until rev
188129, which introduces the new hash table.
Prior to that, we have rev 188059, which makes cosmetic changes to
configure.ac.
If it's related to the hash table, then comparing rev 188059 vs rev
188129 may show the regression.
Diego.
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile
2012-08-20 18:06 [Bug fortran/54332] New: [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile hjl.tools at gmail dot com
` (6 preceding siblings ...)
2012-08-21 13:39 ` dnovillo at google dot com
@ 2012-08-21 13:58 ` hjl.tools at gmail dot com
2012-08-21 14:07 ` dnovillo at google dot com
` (20 subsequent siblings)
28 siblings, 0 replies; 30+ messages in thread
From: hjl.tools at gmail dot com @ 2012-08-21 13:58 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
--- Comment #7 from H.J. Lu <hjl.tools at gmail dot com> 2012-08-21 13:58:05 UTC ---
(In reply to comment #6)
>
> If it's related to the hash table, then comparing rev 188059 vs rev
> 188129 may show the regression.
>
Neither rev 188059 nor rev 188129 will build:
../../../../gcc/gcc/graphite-sese-to-poly.c: In function \u2018void
build_sese_conditions_before(dom_walk_data*, basic_block)\u2019:
../../../../gcc/gcc/graphite-sese-to-poly.c:1357:2: error: call of overloaded
\u2018VEC_safe_push_1(vec_t<gimple_statement_d*>**, NULL, const char [44], int,
const char [29])\u2019 is ambiguous
../../../../gcc/gcc/graphite-sese-to-poly.c:1357:2: note: candidates are:
In file included from ../../../../gcc/gcc/basic-block.h:25:0,
from ../../../../gcc/gcc/tree-flow.h:27,
from ../../../../gcc/gcc/graphite-sese-to-poly.c:24:
../../../../gcc/gcc/vec.h:674:1: note: T& VEC_safe_push_1(vec_t<T>**, T, const
char*, unsigned int, const char*) [with T = gimple_statement_d*;
vec_allocation_t A = (vec_allocation_t)0u]
../../../../gcc/gcc/vec.h:682:1: note: T* VEC_safe_push_1(vec_t<T>**, const T*,
const char*, unsigned int, const char*) [with T = gimple_statement_d*;
vec_allocation_t A = (vec_allocation_t)0u]
make[2]: *** [graphite-sese-to-poly.o] Error 1
2692,40 99%
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile
2012-08-20 18:06 [Bug fortran/54332] New: [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile hjl.tools at gmail dot com
` (7 preceding siblings ...)
2012-08-21 13:58 ` hjl.tools at gmail dot com
@ 2012-08-21 14:07 ` dnovillo at google dot com
2012-08-21 16:21 ` hjl.tools at gmail dot com
` (19 subsequent siblings)
28 siblings, 0 replies; 30+ messages in thread
From: dnovillo at google dot com @ 2012-08-21 14:07 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
--- Comment #8 from dnovillo at google dot com <dnovillo at google dot com> 2012-08-21 14:06:34 UTC ---
On 2012-08-21 09:58 , hjl.tools at gmail dot com wrote:
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
>
> --- Comment #7 from H.J. Lu <hjl.tools at gmail dot com> 2012-08-21 13:58:05 UTC ---
> (In reply to comment #6)
>>
>> If it's related to the hash table, then comparing rev 188059 vs rev
>> 188129 may show the regression.
>>
>
> Neither rev 188059 nor rev 188129 will build:
>
> ../../../../gcc/gcc/graphite-sese-to-poly.c: In function \u2018void
> build_sese_conditions_before(dom_walk_data*, basic_block)\u2019:
> ../../../../gcc/gcc/graphite-sese-to-poly.c:1357:2: error: call of overloaded
> \u2018VEC_safe_push_1(vec_t<gimple_statement_d*>**, NULL, const char [44], int,
> const char [29])\u2019 is ambiguous
> ../../../../gcc/gcc/graphite-sese-to-poly.c:1357:2: note: candidates are:
> In file included from ../../../../gcc/gcc/basic-block.h:25:0,
> from ../../../../gcc/gcc/tree-flow.h:27,
> from ../../../../gcc/gcc/graphite-sese-to-poly.c:24:
> ../../../../gcc/gcc/vec.h:674:1: note: T& VEC_safe_push_1(vec_t<T>**, T, const
> char*, unsigned int, const char*) [with T = gimple_statement_d*;
> vec_allocation_t A = (vec_allocation_t)0u]
> ../../../../gcc/gcc/vec.h:682:1: note: T* VEC_safe_push_1(vec_t<T>**, const T*,
> const char*, unsigned int, const char*) [with T = gimple_statement_d*;
> vec_allocation_t A = (vec_allocation_t)0u]
> make[2]: *** [graphite-sese-to-poly.o] Error 1
> 2692,40 99%
>
Huh, odd. Can you try this patchlet on top of those revs? It builds
for me with this applied:
diff --git a/gcc/graphite-sese-to-poly.c b/gcc/graphite-sese-to-poly.c
index cdabd73..5712e58 100644
--- a/gcc/graphite-sese-to-poly.c
+++ b/gcc/graphite-sese-to-poly.c
@@ -1354,7 +1354,7 @@ build_sese_conditions_before (struct dom_walk_data
*dw_data,
if (e->flags & EDGE_TRUE_VALUE)
VEC_safe_push (gimple, heap, *cases, stmt);
else
- VEC_safe_push (gimple, heap, *cases, NULL);
+ VEC_safe_push (gimple, heap, *cases, (gimple) NULL);
}
gbb = gbb_from_bb (bb);
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile
2012-08-20 18:06 [Bug fortran/54332] New: [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile hjl.tools at gmail dot com
` (8 preceding siblings ...)
2012-08-21 14:07 ` dnovillo at google dot com
@ 2012-08-21 16:21 ` hjl.tools at gmail dot com
2012-08-21 16:44 ` dnovillo at google dot com
` (18 subsequent siblings)
28 siblings, 0 replies; 30+ messages in thread
From: hjl.tools at gmail dot com @ 2012-08-21 16:21 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
--- Comment #9 from H.J. Lu <hjl.tools at gmail dot com> 2012-08-21 16:20:37 UTC ---
Revision 188059 is bad:
f951: out of memory allocating 36872 bytes after a total of 583266304 bytes
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile
2012-08-20 18:06 [Bug fortran/54332] New: [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile hjl.tools at gmail dot com
` (9 preceding siblings ...)
2012-08-21 16:21 ` hjl.tools at gmail dot com
@ 2012-08-21 16:44 ` dnovillo at google dot com
2012-08-21 16:52 ` hjl.tools at gmail dot com
` (17 subsequent siblings)
28 siblings, 0 replies; 30+ messages in thread
From: dnovillo at google dot com @ 2012-08-21 16:44 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
--- Comment #10 from dnovillo at google dot com <dnovillo at google dot com> 2012-08-21 16:44:10 UTC ---
On Tue, Aug 21, 2012 at 12:20 PM, hjl.tools at gmail dot com
<gcc-bugzilla@gcc.gnu.org> wrote:
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
>
> --- Comment #9 from H.J. Lu <hjl.tools at gmail dot com> 2012-08-21 16:20:37 UTC ---
> Revision 188059 is bad:
>
> f951: out of memory allocating 36872 bytes after a total of 583266304 bytes
Thanks. Does rev 188129 show the same thing? The next revisions to try are:
188040 (TREE_CHECK macros)
187954 (merge from trunk)
187836 (initial VEC conversion)
187735 (merge from trunk)
I now have access to SPEC2006, I'll try a build.
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile
2012-08-20 18:06 [Bug fortran/54332] New: [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile hjl.tools at gmail dot com
` (10 preceding siblings ...)
2012-08-21 16:44 ` dnovillo at google dot com
@ 2012-08-21 16:52 ` hjl.tools at gmail dot com
2012-08-21 16:56 ` dnovillo at gcc dot gnu.org
` (16 subsequent siblings)
28 siblings, 0 replies; 30+ messages in thread
From: hjl.tools at gmail dot com @ 2012-08-21 16:52 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
H.J. Lu <hjl.tools at gmail dot com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|UNCONFIRMED |NEW
Last reconfirmed|2012-08-20 00:00:00 |2012-08-21
Ever Confirmed|0 |1
--- Comment #11 from H.J. Lu <hjl.tools at gmail dot com> 2012-08-21 16:51:24 UTC ---
It is caused by revision 187836:
http://gcc.gnu.org/ml/gcc-cvs/2012-05/msg00833.html
The C++ implementation of vec.[ch] has a massive memory leak.
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile
2012-08-20 18:06 [Bug fortran/54332] New: [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile hjl.tools at gmail dot com
` (11 preceding siblings ...)
2012-08-21 16:52 ` hjl.tools at gmail dot com
@ 2012-08-21 16:56 ` dnovillo at gcc dot gnu.org
2012-08-21 17:10 ` hjl.tools at gmail dot com
` (15 subsequent siblings)
28 siblings, 0 replies; 30+ messages in thread
From: dnovillo at gcc dot gnu.org @ 2012-08-21 16:56 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
Diego Novillo <dnovillo at gcc dot gnu.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |ASSIGNED
--- Comment #12 from Diego Novillo <dnovillo at gcc dot gnu.org> 2012-08-21 16:55:34 UTC ---
Thanks. I'll work on a fix.
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile
2012-08-20 18:06 [Bug fortran/54332] New: [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile hjl.tools at gmail dot com
` (12 preceding siblings ...)
2012-08-21 16:56 ` dnovillo at gcc dot gnu.org
@ 2012-08-21 17:10 ` hjl.tools at gmail dot com
2012-08-21 17:41 ` hjl.tools at gmail dot com
` (14 subsequent siblings)
28 siblings, 0 replies; 30+ messages in thread
From: hjl.tools at gmail dot com @ 2012-08-21 17:10 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
--- Comment #13 from H.J. Lu <hjl.tools at gmail dot com> 2012-08-21 17:10:09 UTC ---
It can be reproduced with -frecord-marker=4 -O -funswitch-loops.
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile
2012-08-20 18:06 [Bug fortran/54332] New: [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile hjl.tools at gmail dot com
` (13 preceding siblings ...)
2012-08-21 17:10 ` hjl.tools at gmail dot com
@ 2012-08-21 17:41 ` hjl.tools at gmail dot com
2012-08-21 17:58 ` hjl.tools at gmail dot com
` (13 subsequent siblings)
28 siblings, 0 replies; 30+ messages in thread
From: hjl.tools at gmail dot com @ 2012-08-21 17:41 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
--- Comment #14 from H.J. Lu <hjl.tools at gmail dot com> 2012-08-21 17:41:10 UTC ---
It failed even with
diff --git a/gcc/tree-ssa-loop.c b/gcc/tree-ssa-loop.c
index 3d650bf..30ac4b5 100644
--- a/gcc/tree-ssa-loop.c
+++ b/gcc/tree-ssa-loop.c
@@ -149,7 +149,7 @@ tree_ssa_loop_unswitch (void)
static bool
gate_tree_ssa_loop_unswitch (void)
{
- return flag_unswitch_loops != 0;
+ return false;
}
struct gimple_opt_pass pass_tree_unswitch =
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile
2012-08-20 18:06 [Bug fortran/54332] New: [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile hjl.tools at gmail dot com
` (14 preceding siblings ...)
2012-08-21 17:41 ` hjl.tools at gmail dot com
@ 2012-08-21 17:58 ` hjl.tools at gmail dot com
2012-08-21 18:09 ` hjl.tools at gmail dot com
` (12 subsequent siblings)
28 siblings, 0 replies; 30+ messages in thread
From: hjl.tools at gmail dot com @ 2012-08-21 17:58 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
--- Comment #15 from H.J. Lu <hjl.tools at gmail dot com> 2012-08-21 17:57:59 UTC ---
It failed with
diff --git a/gcc/passes.c b/gcc/passes.c
index b6fe18e..10174c4 100644
--- a/gcc/passes.c
+++ b/gcc/passes.c
@@ -1449,7 +1449,6 @@ init_optimization_passes (void)
NEXT_PASS (pass_lim);
NEXT_PASS (pass_copy_prop);
NEXT_PASS (pass_dce_loop);
- NEXT_PASS (pass_tree_unswitch);
NEXT_PASS (pass_scev_cprop);
NEXT_PASS (pass_record_bounds);
NEXT_PASS (pass_check_data_deps);
Somehow just processing the -funswitch-loops command-line option
triggers this problem.
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile
2012-08-20 18:06 [Bug fortran/54332] New: [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile hjl.tools at gmail dot com
` (15 preceding siblings ...)
2012-08-21 17:58 ` hjl.tools at gmail dot com
@ 2012-08-21 18:09 ` hjl.tools at gmail dot com
2012-08-21 18:19 ` dnovillo at google dot com
` (11 subsequent siblings)
28 siblings, 0 replies; 30+ messages in thread
From: hjl.tools at gmail dot com @ 2012-08-21 18:09 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
--- Comment #16 from H.J. Lu <hjl.tools at gmail dot com> 2012-08-21 18:08:49 UTC ---
There are:
opts.c:typedef char *char_p; /* For DEF_VEC_P. */
opts.c:DEF_VEC_P(char_p);
opts.c:DEF_VEC_ALLOC_P(char_p,heap);
opts-global.c:typedef const char *const_char_p; /* For DEF_VEC_P. */
opts-global.c:DEF_VEC_P(const_char_p);
opts-global.c:DEF_VEC_ALLOC_P(const_char_p,heap);
Will they cause problems if other files define similar types?
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile
2012-08-20 18:06 [Bug fortran/54332] New: [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile hjl.tools at gmail dot com
` (16 preceding siblings ...)
2012-08-21 18:09 ` hjl.tools at gmail dot com
@ 2012-08-21 18:19 ` dnovillo at google dot com
2012-08-21 18:32 ` dnovillo at google dot com
` (10 subsequent siblings)
28 siblings, 0 replies; 30+ messages in thread
From: dnovillo at google dot com @ 2012-08-21 18:19 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
--- Comment #17 from dnovillo at google dot com <dnovillo at google dot com> 2012-08-21 18:19:10 UTC ---
On 2012-08-21 14:08 , hjl.tools at gmail dot com wrote:
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
>
> --- Comment #16 from H.J. Lu <hjl.tools at gmail dot com> 2012-08-21 18:08:49 UTC ---
> There are:
>
> opts.c:typedef char *char_p; /* For DEF_VEC_P. */
> opts.c:DEF_VEC_P(char_p);
> opts.c:DEF_VEC_ALLOC_P(char_p,heap);
> opts-global.c:typedef const char *const_char_p; /* For DEF_VEC_P. */
> opts-global.c:DEF_VEC_P(const_char_p);
> opts-global.c:DEF_VEC_ALLOC_P(const_char_p,heap);
>
> Will they cause problems if other files define similar types?
>
They shouldn't. DEF_VEC_* expands to a NOP now. The allocation
strategy is only needed during the actual allocation call. So, in the
case of opts.c, that would be in add_comma_separated_to_vector() (the
call to VEC_safe_push).
Those two vectors are only used for -finstrument-options..., though. So
that does not seem related.
The call to postpone_unknown_option_warning in opts-global.c seems also
unrelated. It's only used when processing unknown options. That vector
is popped when the unknown options are freed, so that can't be it either.
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile
2012-08-20 18:06 [Bug fortran/54332] New: [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile hjl.tools at gmail dot com
` (17 preceding siblings ...)
2012-08-21 18:19 ` dnovillo at google dot com
@ 2012-08-21 18:32 ` dnovillo at google dot com
2012-08-21 18:55 ` hjl.tools at gmail dot com
` (9 subsequent siblings)
28 siblings, 0 replies; 30+ messages in thread
From: dnovillo at google dot com @ 2012-08-21 18:32 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
--- Comment #18 from dnovillo at google dot com <dnovillo at google dot com> 2012-08-21 18:31:51 UTC ---
OK, I think this is the hunk that's causing grief:
diff --git a/gcc/df-scan.c b/gcc/df-scan.c
index 39f444f..35100d1 100644
--- a/gcc/df-scan.c
+++ b/gcc/df-scan.c
@@ -4392,7 +4392,6 @@ df_bb_verify (basic_block bb)
if (!INSN_P (insn))
continue;
df_insn_refs_verify (&collection_rec, bb, insn, true);
- df_free_collection_rec (&collection_rec);
}
/* Do the artificial defs and uses. */
I remember that I ran into this during the VEC conversion
(http://gcc.gnu.org/ml/gcc/2012-05/msg00271.html) and after some
discussion I ended up convincing myself that taking it out was harmless.
Clearly, I was wrong.
I've hooked gdb to the running f951 and it's stuck in df_bb_verify().
Odd that this has not triggered anywhere else.
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile
2012-08-20 18:06 [Bug fortran/54332] New: [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile hjl.tools at gmail dot com
` (18 preceding siblings ...)
2012-08-21 18:32 ` dnovillo at google dot com
@ 2012-08-21 18:55 ` hjl.tools at gmail dot com
2012-08-21 19:08 ` dnovillo at google dot com
` (8 subsequent siblings)
28 siblings, 0 replies; 30+ messages in thread
From: hjl.tools at gmail dot com @ 2012-08-21 18:55 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
--- Comment #19 from H.J. Lu <hjl.tools at gmail dot com> 2012-08-21 18:54:45 UTC ---
(In reply to comment #15)
> It failed with
>
> diff --git a/gcc/passes.c b/gcc/passes.c
> index b6fe18e..10174c4 100644
> --- a/gcc/passes.c
> +++ b/gcc/passes.c
> @@ -1449,7 +1449,6 @@ init_optimization_passes (void)
> NEXT_PASS (pass_lim);
> NEXT_PASS (pass_copy_prop);
> NEXT_PASS (pass_dce_loop);
> - NEXT_PASS (pass_tree_unswitch);
> NEXT_PASS (pass_scev_cprop);
> NEXT_PASS (pass_record_bounds);
> NEXT_PASS (pass_check_data_deps);
>
> Somehow just processing the -funswitch-loops command-line option
> triggers this problem.
With --enable-gather-detailed-mem-stats, I got
Alloc-pool Kind Elt size Pools Allocated (elts) Peak
(elts) Leak (elts)
-df_scan ref base 64 18 24808192( 387628) 11869056(
185454) 0( 0)
-df_scan ref artificial 72 18 15168528( 210674) 2044944(
28402) 0( 0)
+df_scan ref base 64 18 513091264( 8017051) 500077440(
7813710) 0( 0)
+df_scan ref artificial 72 18 599905368( 8332019) 2044944(
28402) 0( 0)
elt_loc_list 32 27 7982112( 249441) 2399488(
74984) 0( 0)
-df_scan ref regular 72 18 71483184( 992822) 45955584(
638272) 0( 0)
+df_scan ref regular 72 18 2091195360( 29044380) 2065579776(
28688608) 0( 0)
df_scan insn 56 18 7681016( 137161) 3340848(
59658) 0( 0)
-Total 15775 253131240
+Total 16067 3345899232
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile
2012-08-20 18:06 [Bug fortran/54332] New: [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile hjl.tools at gmail dot com
` (19 preceding siblings ...)
2012-08-21 18:55 ` hjl.tools at gmail dot com
@ 2012-08-21 19:08 ` dnovillo at google dot com
2012-08-21 19:20 ` steven at gcc dot gnu.org
` (7 subsequent siblings)
28 siblings, 0 replies; 30+ messages in thread
From: dnovillo at google dot com @ 2012-08-21 19:08 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
--- Comment #20 from dnovillo at google dot com <dnovillo at google dot com> 2012-08-21 19:07:33 UTC ---
On 2012-08-21 14:54 , hjl.tools at gmail dot com wrote:
> With --enable-gather-detailed-mem-stats, I got
>
> Alloc-pool Kind Elt size Pools Allocated (elts) Peak
> (elts) Leak (elts)
>
> -df_scan ref base 64 18 24808192( 387628) 11869056(
> 185454) 0( 0)
> -df_scan ref artificial 72 18 15168528( 210674) 2044944(
> 28402) 0( 0)
> +df_scan ref base 64 18 513091264( 8017051) 500077440(
> 7813710) 0( 0)
> +df_scan ref artificial 72 18 599905368( 8332019) 2044944(
> 28402) 0( 0)
> elt_loc_list 32 27 7982112( 249441) 2399488(
> 74984) 0( 0)
> -df_scan ref regular 72 18 71483184( 992822) 45955584(
> 638272) 0( 0)
> +df_scan ref regular 72 18 2091195360( 29044380) 2065579776(
> 28688608) 0( 0)
> df_scan insn 56 18 7681016( 137161) 3340848(
> 59658) 0( 0)
>
> -Total 15775 253131240
> +Total 16067 3345899232
>
That agrees with what I found, thanks. I've added a link to the
discussion about the df verifier. The vectors need to be cleared, but I
can't just free the vectors:
Stack vectors must be initially allocated with VEC_stack_alloc.
gcc/df-scan.c: In function 'unsigned int df_count_refs(bool, bool, bool)':
gcc/df-scan.c:1507:1: internal compiler error: in vec_reserve, at vec.h:1111
}
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile
2012-08-20 18:06 [Bug fortran/54332] New: [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile hjl.tools at gmail dot com
` (20 preceding siblings ...)
2012-08-21 19:08 ` dnovillo at google dot com
@ 2012-08-21 19:20 ` steven at gcc dot gnu.org
2012-08-21 19:28 ` hjl.tools at gmail dot com
` (6 subsequent siblings)
28 siblings, 0 replies; 30+ messages in thread
From: steven at gcc dot gnu.org @ 2012-08-21 19:20 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
--- Comment #21 from Steven Bosscher <steven at gcc dot gnu.org> 2012-08-21 19:19:58 UTC ---
(In reply to comment #18)
> Odd that this has not triggered anywhere else.
It may have triggered elsewhere, see PR54343 ...
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile
2012-08-20 18:06 [Bug fortran/54332] New: [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile hjl.tools at gmail dot com
` (21 preceding siblings ...)
2012-08-21 19:20 ` steven at gcc dot gnu.org
@ 2012-08-21 19:28 ` hjl.tools at gmail dot com
2012-08-21 19:50 ` dnovillo at google dot com
` (5 subsequent siblings)
28 siblings, 0 replies; 30+ messages in thread
From: hjl.tools at gmail dot com @ 2012-08-21 19:28 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
--- Comment #22 from H.J. Lu <hjl.tools at gmail dot com> 2012-08-21 19:27:50 UTC ---
This seems to work:
diff --git a/gcc/df-scan.c b/gcc/df-scan.c
index 35100d1..39f444f 100644
--- a/gcc/df-scan.c
+++ b/gcc/df-scan.c
@@ -4392,6 +4392,7 @@ df_bb_verify (basic_block bb)
if (!INSN_P (insn))
continue;
df_insn_refs_verify (&collection_rec, bb, insn, true);
+ df_free_collection_rec (&collection_rec);
}
/* Do the artificial defs and uses. */
diff --git a/gcc/vec.h b/gcc/vec.h
index cc7e819..3a298ff 100644
--- a/gcc/vec.h
+++ b/gcc/vec.h
@@ -1031,21 +1031,9 @@ vec_reserve (vec_t<T> *vec_, int reserve MEM_STAT_DECL)
sizeof (T), false
PASS_MEM_STAT);
else
- {
- /* Only allow stack vectors when re-growing them. The initial
- allocation of stack vectors must be done with the
- VEC_stack_alloc macro, because it uses alloca() for the
- allocation. */
- if (vec_ == NULL)
- {
- fprintf (stderr, "Stack vectors must be initially allocated "
- "with VEC_stack_alloc.\n");
- gcc_unreachable ();
- }
- return (vec_t<T> *) vec_stack_o_reserve (vec_, reserve,
- offsetof (vec_t<T>, vec),
- sizeof (T) PASS_MEM_STAT);
- }
+ return (vec_t<T> *) vec_stack_o_reserve (vec_, reserve,
+ offsetof (vec_t<T>, vec),
+ sizeof (T) PASS_MEM_STAT);
}
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile
2012-08-20 18:06 [Bug fortran/54332] New: [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile hjl.tools at gmail dot com
` (22 preceding siblings ...)
2012-08-21 19:28 ` hjl.tools at gmail dot com
@ 2012-08-21 19:50 ` dnovillo at google dot com
2012-08-21 19:53 ` hjl.tools at gmail dot com
` (4 subsequent siblings)
28 siblings, 0 replies; 30+ messages in thread
From: dnovillo at google dot com @ 2012-08-21 19:50 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
--- Comment #23 from dnovillo at google dot com <dnovillo at google dot com> 2012-08-21 19:50:12 UTC ---
On 2012-08-21 15:27 , hjl.tools at gmail dot com wrote:
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
>
> --- Comment #22 from H.J. Lu <hjl.tools at gmail dot com> 2012-08-21 19:27:50 UTC ---
> This seems to work:
>
> diff --git a/gcc/df-scan.c b/gcc/df-scan.c
> index 35100d1..39f444f 100644
> --- a/gcc/df-scan.c
> +++ b/gcc/df-scan.c
> @@ -4392,6 +4392,7 @@ df_bb_verify (basic_block bb)
> if (!INSN_P (insn))
> continue;
> df_insn_refs_verify (&collection_rec, bb, insn, true);
> + df_free_collection_rec (&collection_rec);
> }
>
> /* Do the artificial defs and uses. */
> diff --git a/gcc/vec.h b/gcc/vec.h
> index cc7e819..3a298ff 100644
> --- a/gcc/vec.h
> +++ b/gcc/vec.h
> @@ -1031,21 +1031,9 @@ vec_reserve (vec_t<T> *vec_, int reserve MEM_STAT_DECL)
> sizeof (T), false
> PASS_MEM_STAT);
> else
> - {
> - /* Only allow stack vectors when re-growing them. The initial
> - allocation of stack vectors must be done with the
> - VEC_stack_alloc macro, because it uses alloca() for the
> - allocation. */
> - if (vec_ == NULL)
> - {
> - fprintf (stderr, "Stack vectors must be initially allocated "
> - "with VEC_stack_alloc.\n");
> - gcc_unreachable ();
> - }
> - return (vec_t<T> *) vec_stack_o_reserve (vec_, reserve,
> - offsetof (vec_t<T>, vec),
> - sizeof (T) PASS_MEM_STAT);
> - }
> + return (vec_t<T> *) vec_stack_o_reserve (vec_, reserve,
> + offsetof (vec_t<T>, vec),
> + sizeof (T) PASS_MEM_STAT);
> }
>
The problem with this is that you are switching a stack vec into a heap
vec. This may not always be what the caller wanted.
The other alternative is to truncate the vectors after the call to
df_insn_refs_verify().
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile
2012-08-20 18:06 [Bug fortran/54332] New: [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile hjl.tools at gmail dot com
` (23 preceding siblings ...)
2012-08-21 19:50 ` dnovillo at google dot com
@ 2012-08-21 19:53 ` hjl.tools at gmail dot com
2012-08-21 20:49 ` dnovillo at google dot com
` (3 subsequent siblings)
28 siblings, 0 replies; 30+ messages in thread
From: hjl.tools at gmail dot com @ 2012-08-21 19:53 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
--- Comment #24 from H.J. Lu <hjl.tools at gmail dot com> 2012-08-21 19:53:14 UTC ---
(In reply to comment #23)
>
> The problem with this is that you are switching a stack vec into a heap
> vec. This may not always be what the caller wanted.
My patch just restores the old behavior.
>
> The other alternative is to truncate the vectors after the call to
> df_insn_refs_verify().
This should be a separate patch, not the part of C++ conversion.
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile
2012-08-20 18:06 [Bug fortran/54332] New: [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile hjl.tools at gmail dot com
` (24 preceding siblings ...)
2012-08-21 19:53 ` hjl.tools at gmail dot com
@ 2012-08-21 20:49 ` dnovillo at google dot com
2012-08-21 21:07 ` hjl at gcc dot gnu.org
` (2 subsequent siblings)
28 siblings, 0 replies; 30+ messages in thread
From: dnovillo at google dot com @ 2012-08-21 20:49 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
--- Comment #25 from dnovillo at google dot com <dnovillo at google dot com> 2012-08-21 20:49:16 UTC ---
On 2012-08-21 15:53 , hjl.tools at gmail dot com wrote:
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
>
> --- Comment #24 from H.J. Lu <hjl.tools at gmail dot com> 2012-08-21 19:53:14 UTC ---
> (In reply to comment #23)
>>
>> The problem with this is that you are switching a stack vec into a heap
>> vec. This may not always be what the caller wanted.
>
> My patch just restores the old behavior.
You are right. This was always the case. I added the extra check to
guard against inadvertent *initial* heap allocations for stack vectors.
But now that I see the old code, this was always the case. The
subsequent stack operations after the first round around the loop will
move the stacks into the heap.
The patch is OK with a ChangeLog and bootstrap testing.
Thanks! Diego.
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile
2012-08-20 18:06 [Bug fortran/54332] New: [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile hjl.tools at gmail dot com
` (25 preceding siblings ...)
2012-08-21 20:49 ` dnovillo at google dot com
@ 2012-08-21 21:07 ` hjl at gcc dot gnu.org
2012-08-21 21:11 ` hjl.tools at gmail dot com
2012-08-22 9:02 ` [Bug middle-end/54332] " steven at gcc dot gnu.org
28 siblings, 0 replies; 30+ messages in thread
From: hjl at gcc dot gnu.org @ 2012-08-21 21:07 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
--- Comment #26 from hjl at gcc dot gnu.org <hjl at gcc dot gnu.org> 2012-08-21 21:07:07 UTC ---
Author: hjl
Date: Tue Aug 21 21:07:01 2012
New Revision: 190576
URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=190576
Log:
Restore df_free_collection_rec call in df_bb_verify
PR middle-end/54332
* df-scan.c (df_bb_verify): Restore df_free_collection_rec call
inside the insn traversal loop.
* vec.h (vec_reserve): Remove the stack allocation check.
Modified:
trunk/gcc/ChangeLog
trunk/gcc/df-scan.c
trunk/gcc/vec.h
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile
2012-08-20 18:06 [Bug fortran/54332] New: [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile hjl.tools at gmail dot com
` (26 preceding siblings ...)
2012-08-21 21:07 ` hjl at gcc dot gnu.org
@ 2012-08-21 21:11 ` hjl.tools at gmail dot com
2012-08-22 9:02 ` [Bug middle-end/54332] " steven at gcc dot gnu.org
28 siblings, 0 replies; 30+ messages in thread
From: hjl.tools at gmail dot com @ 2012-08-21 21:11 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
H.J. Lu <hjl.tools at gmail dot com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|ASSIGNED |RESOLVED
Resolution| |FIXED
--- Comment #27 from H.J. Lu <hjl.tools at gmail dot com> 2012-08-21 21:11:11 UTC ---
Fixed.
^ permalink raw reply [flat|nested] 30+ messages in thread
* [Bug middle-end/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile
2012-08-20 18:06 [Bug fortran/54332] New: [4.8 Regression] 481.wrf in SPEC CPU 2006 takes > 10GB memory to compile hjl.tools at gmail dot com
` (27 preceding siblings ...)
2012-08-21 21:11 ` hjl.tools at gmail dot com
@ 2012-08-22 9:02 ` steven at gcc dot gnu.org
28 siblings, 0 replies; 30+ messages in thread
From: steven at gcc dot gnu.org @ 2012-08-22 9:02 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332
Steven Bosscher <steven at gcc dot gnu.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |Joost.VandeVondele at mat
| |dot ethz.ch
--- Comment #28 from Steven Bosscher <steven at gcc dot gnu.org> 2012-08-22 08:59:04 UTC ---
*** Bug 54269 has been marked as a duplicate of this bug. ***
^ permalink raw reply [flat|nested] 30+ messages in thread