[Bug c++/54056] New: Fast run out of memory compiling template specialization

[Bug c++/54056] New: Fast run out of memory compiling template specialization

[sergey.prokhorenko at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54056

             Bug #: 54056
           Summary: Fast run out of memory compiling template
                    specialization
    Classification: Unclassified
           Product: gcc
           Version: 4.7.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c++
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: sergey.prokhorenko@gmail.com


Created attachment 27848
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=27848
Test case to reproduce.

The attached file, compiled with both g++ 4.6.3 and g++ 4.7.0, quickly eats up
all memory and swap space.

The similar templates source in a real project quickly (in 10-20 seconds) eat
16GB of memory, swaps out and die. But when compiled with g++ 4.5.2 it does not
run out of memory. We found this trying to move to 4.6 or 4.7.

Also, the bug does not happen if the attached test case is compiled with
-DNOSWAPBUG. In this case a template without extra parameter is used. This was
found in the real project, where I compiled 30MB preprocessed file and it did
not eat memory if the "buggy" templates form (with 2 parameters) was replaced
with the NOSWAPBUG one.

I used a generator to produce test case. On my simple machine that I use now to
test with 1GB of memory, max number of templates it can handle is 200. With 300
it quickly takes up all memory and swap space.

Here's -ftime-report for 200 structs:

Execution times (seconds)
 phase setup             :   0.01 ( 1%) usr   0.00 ( 0%) sys   0.02 ( 1%) wall 
  1438 kB ( 1%) ggc
 phase parsing           :   1.36 (98%) usr   0.16 (94%) sys   1.52 (81%) wall 
234458 kB (99%) ggc
 phase lang. deferred    :   0.02 ( 1%) usr   0.00 ( 0%) sys   0.17 ( 9%) wall 
     0 kB ( 0%) ggc
 phase cgraph            :   0.00 ( 0%) usr   0.01 ( 6%) sys   0.13 ( 7%) wall 
   131 kB ( 0%) ggc
 phase generate          :   0.02 ( 1%) usr   0.01 ( 6%) sys   0.33 (18%) wall 
   132 kB ( 0%) ggc
 |name lookup            :   0.00 ( 0%) usr   0.00 ( 0%) sys   0.01 ( 1%) wall 
   307 kB ( 0%) ggc
 garbage collection      :   0.02 ( 1%) usr   0.00 ( 0%) sys   0.15 ( 8%) wall 
     0 kB ( 0%) ggc
 callgraph construction  :   0.00 ( 0%) usr   0.01 ( 6%) sys   0.04 ( 2%) wall 
   131 kB ( 0%) ggc
 callgraph optimization  :   0.00 ( 0%) usr   0.00 ( 0%) sys   0.06 ( 3%) wall 
     0 kB ( 0%) ggc
 varpool construction    :   0.00 ( 0%) usr   0.00 ( 0%) sys   0.03 ( 2%) wall 
     0 kB ( 0%) ggc
 preprocessing           :   0.00 ( 0%) usr   0.00 ( 0%) sys   0.01 ( 1%) wall 
    29 kB ( 0%) ggc
 parser (global)         :   0.00 ( 0%) usr   0.02 (12%) sys   0.02 ( 1%) wall 
   767 kB ( 0%) ggc
 parser struct body      :   1.31 (94%) usr   0.14 (82%) sys   1.42 (76%) wall 
231197 kB (98%) ggc
 template instantiation  :   0.05 ( 4%) usr   0.00 ( 0%) sys   0.07 ( 4%) wall 
  2442 kB ( 1%) ggc
 TOTAL                 :   1.39             0.17             1.87            
236046 kB

And to compare here's exactly same 200 struct report but with -DNOSWAPBUG:

 phase setup             :   0.01 (20%) usr   0.00 ( 0%) sys   0.02 (33%) wall 
  1438 kB (55%) ggc
 phase parsing           :   0.04 (80%) usr   0.01 (100%) sys   0.04 (67%) wall
   1024 kB (39%) ggc
 |name lookup            :   0.02 (40%) usr   0.00 ( 0%) sys   0.01 (17%) wall 
   216 kB ( 8%) ggc
 preprocessing           :   0.00 ( 0%) usr   0.01 (100%) sys   0.01 (17%) wall
     29 kB ( 1%) ggc
 parser (global)         :   0.02 (40%) usr   0.00 ( 0%) sys   0.02 (33%) wall 
   551 kB (21%) ggc
 parser struct body      :   0.02 (40%) usr   0.00 ( 0%) sys   0.01 (17%) wall 
   282 kB (11%) ggc
 TOTAL                 :   0.05             0.01             0.06              
2607 kB

Not only memory 230KB vs 2KB, but compilation time is 1.39s vs 0.05s.

My environment at work is Ubuntu 11.10 x86_64, so 64 bit compiler, and Ubuntu
12.04 at home, also 64 bit.

[Bug c++/54056] Fast run out of memory compiling template specialization

[sergey.prokhorenko at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54056

--- Comment #1 from Sergey Prokhorenko <sergey.prokhorenko at gmail dot com> 2012-07-21 10:58:04 UTC ---
I also asked few people to test, this could be reproduced on:
g++ (GCC) 4.7.0 20120507 (Red Hat 4.7.0-5)
g++-mp-4.6 (MacPorts gcc46 4.6.3_3) 4.6.3
g++-mp-4.7 (MacPorts gcc47 4.7.1_1) 4.7.1
CentOS 6.3 / GCC 4.7.1 custom build

They also tested on gcc 4.4.6, 4.2.1, 4.5.3 on the same systems, and with these
earlier gcc versions the bug does not happen.

[Bug c++/54056] Fast run out of memory compiling template specialization

[ppalka at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54056

Patrick Palka <ppalka at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |compile-time-hog
     Ever confirmed|0                           |1
      Known to work|                            |4.5.3
             Status|UNCONFIRMED                 |NEW
      Known to fail|                            |14.0
   Last reconfirmed|                            |2023-07-27
           See Also|                            |https://gcc.gnu.org/bugzill
                   |                            |a/show_bug.cgi?id=32505
                 CC|                            |jason at gcc dot gnu.org,
                   |                            |ppalka at gcc dot gnu.org

--- Comment #2 from Patrick Palka <ppalka at gcc dot gnu.org> ---
FWIW, this seems to be an old regression caused by the fix for PR32505; the
loop over all implicit instantiations at the end of
process_partial_specialization introduces quadraticness in the number of
DEFINE_STRUCTs in the testcase.