[Bug fortran/59811] New: Huge increase in memory usage and compile time with gfortran 4.8

[Bug fortran/59811] New: Huge increase in memory usage and compile time with gfortran 4.8

[bastian.feigl at kit dot edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59811

            Bug ID: 59811
           Summary: Huge increase in memory usage and compile time with
                    gfortran 4.8
           Product: gcc
           Version: 4.8.2
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: fortran
          Assignee: unassigned at gcc dot gnu.org
          Reporter: bastian.feigl at kit dot edu

Created attachment 31833
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=31833&action=edit
Example fortran source code file with problems with gfortran 4.8

When switching from gfortran 4.7 to gfortran 4.8 the memory usage and compile
time vastly increases for some files in our project, e.g. for the attached
example file. gfortran 4.8.2 needs 50s to compile, using up to 1.7 GB of RAM,
while gfortran 4.7 compiles it in 7s with a memory usage of 136 MB.

The command line of the gfortran-call for the attached example is
/opt/gcc/4.8.2/bin/gfortran -fno-automatic -ffixed-line-length-none -O2 -c
FermionBoxEventempCoupling_Div.F -o output.o 

The problem seems to be partly linked to the option -fno-automatic, since
omitting it inhibits the memory increase, but the compile time is still 14s
with gfortran 4.8, compared to 6s with gfortran 4.7.

The problem arises with optimizations -O2 and -O1 lead to a similar
discrepancy, with -O0 the problem does not exist.

The gfortran 4.8 which shows the problematic behaviour is built with:
Target: x86_64-unknown-linux-gnu
Configured with: ../gcc-4.8.2/configure --prefix=/opt/gcc/4.8.2
--enable-languages=c,c++,fortran
gcc version 4.8.2 (GCC) 

The gfortran 4.7 is the built-in from openSUSE 12.2:
Target: x86_64-suse-linux
Configured with: ../configure --prefix=/usr --infodir=/usr/share/info
--mandir=/usr/share/man --libdir=/usr/lib64 --libexecdir=/usr/lib64
--enable-languages=c,c++,objc,fortran,obj-c++,java,ada
--enable-checking=release --with-gxx-include-dir=/usr/include/c++/4.7
--enable-ssp --disable-libssp --disable-libitm --disable-plugin
--with-bugurl=http://bugs.opensuse.org/ --with-pkgversion='SUSE Linux'
--disable-libgcj --disable-libmudflap --with-slibdir=/lib64 --with-system-zlib
--enable-__cxa_atexit --enable-libstdcxx-allocator=new --disable-libstdcxx-pch
--enable-version-specific-runtime-libs --enable-linker-build-id
--program-suffix=-4.7 --enable-linux-futex --without-system-libunwind
--with-arch-32=i586 --with-tune=generic --build=x86_64-suse-linux
gcc version 4.7.1 20120723 [gcc-4_7-branch revision 189773] (SUSE Linux)

[Bug rtl-optimization/59811] [4.8/4.9 Regression] Huge increase in memory usage and compile time with gfortran 4.8

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59811

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
      Known to work|                            |4.7.3
           Keywords|                            |compile-time-hog
   Last reconfirmed|                            |2014-01-15
          Component|fortran                     |rtl-optimization
     Ever confirmed|0                           |1
            Summary|Huge increase in memory     |[4.8/4.9 Regression] Huge
                   |usage and compile time with |increase in memory usage
                   |gfortran 4.8                |and compile time with
                   |                            |gfortran 4.8
   Target Milestone|---                         |4.8.3
      Known to fail|                            |4.8.2, 4.9.0

--- Comment #1 from Richard Biener <rguenth at gcc dot gnu.org> ---
Confirmed.  At -O2 I see for 4.8

 alias stmt walking      :   4.90 ( 8%) usr   0.02 ( 3%) sys   5.05 ( 8%) wall 
     0 kB ( 0%) ggc
 tree copy propagation   :   0.02 ( 0%) usr   0.00 ( 0%) sys   0.02 ( 0%) wall 
   513 kB ( 0%) ggc
 combiner                :  45.94 (71%) usr   0.57 (83%) sys  46.57 (71%) wall
1612812 kB (96%) ggc
 integrated RA           :   0.59 ( 1%) usr   0.01 ( 1%) sys   0.58 ( 1%) wall 
 12481 kB ( 1%) ggc
 LRA hard reg assignment :   7.53 (12%) usr   0.03 ( 4%) sys   7.56 (11%) wall 
     0 kB ( 0%) ggc
 TOTAL                 :  65.03             0.69            65.92           
1681500 kB

and for 4.7

 alias stmt walking      :   2.77 (28%) usr   0.02 (17%) sys   2.80 (28%) wall 
     0 kB ( 0%) ggc
 tree copy propagation   :   1.64 (17%) usr   0.01 ( 8%) sys   1.67 (17%) wall 
   513 kB ( 1%) ggc
 integrated RA           :   0.66 ( 7%) usr   0.02 (17%) sys   0.68 ( 7%) wall 
  6295 kB (10%) ggc
 reload                  :   0.45 ( 5%) usr   0.01 ( 8%) sys   0.46 ( 5%) wall 
  1593 kB ( 3%) ggc
 reload CSE regs         :   0.74 ( 7%) usr   0.00 ( 0%) sys   0.75 ( 7%) wall 
  1103 kB ( 2%) ggc
 TOTAL                 :   9.87             0.12            10.01             
61922 kB

the combiner slowdown and the LRA thing are worth investigating.  I suppose
the generated code differs quite a bit ;)  It's a very large function
with some very large basic-block(s).  Current trunk behaves comparable
to GCC 4.8.

[Bug rtl-optimization/59811] [4.8/4.9 Regression] Huge increase in memory usage and compile time in combine

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59811

--- Comment #2 from Richard Biener <rguenth at gcc dot gnu.org> ---
No clear offender in combine.  Points at if_then_else_cond (14% of combine
time).
LRA slowness is calls to find_hard_regno_for, 93% of lra_assign ().

[Bug rtl-optimization/59811] [4.8/4.9 Regression] Huge increase in memory usage and compile time in combine

[jakub at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59811

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jakub at gcc dot gnu.org

--- Comment #3 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Started with r190594 which apparently made big changes in some of the GIMPLE
passes, e.g. the optimized dump linecount went down from 65782 lines (r190593)
to 52107 lines (r190594).  But, for some reason that change results in
uncomparably more log links and combiner opportunities.
In -fdump-rtl-combine-details dump, 'Successfully match' matches went up from
9742 up to 3921927, and 'Failed to match' matches went up from 53193 to
22538586.
So, the combiner success rate is approximately the same, around 17.5%, just on
400 times more opportunities.
For GIMPLE passes, looking just at the sizes of the gimple dumps, until esra
the sizes are the same, from fre1 the dump with r190594 is slightly to
significantly larger than corresponding r190593 dump until crited, and then
surprisingly the pre dump is on the other side half the size of r190593 dump
and from sink till optimized the dump sizes roughly match the 65782 to 52107
lines.

[Bug rtl-optimization/59811] [4.8/4.9 Regression] Huge increase in memory usage and compile time in combine

[jakub at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59811

--- Comment #4 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
With -O2 -fno-automatic -ffixed-line-length-none --param
sccvn-max-alias-queries-per-access=1300 the combiner completely disappears out
of the picture, supposedly because then FRE/PRE doesn't give up, with 1200 the
combiner already takes 17% of compile time and 77% of memory, with the default
1000 it takes 58% of compile time and 96% of memory.

[Bug rtl-optimization/59811] [4.8/4.9 Regression] Huge increase in memory usage and compile time in combine

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59811

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Priority|P3                          |P2
                 CC|                            |rguenth at gcc dot gnu.org

--- Comment #5 from Richard Biener <rguenth at gcc dot gnu.org> ---
Bah.  Add a limiting --param to combine? ....

[Bug rtl-optimization/59811] [4.8/4.9 Regression] Huge increase in memory usage and compile time in combine

[jakub at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59811

--- Comment #6 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Dunno what to limit there though.

BTW, the 1000 default for sccvn-max-alias-queries-per-access param was some
carefully chosen value, e.g. does the test that was fixed by the addition of
the param fail if it is bumped to say 1500 or 2000, or can we just bump
slightly the default?

[Bug rtl-optimization/59811] [4.8/4.9 Regression] Huge increase in memory usage and compile time in combine

[rguenther at suse dot de]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59811

--- Comment #7 from rguenther at suse dot de <rguenther at suse dot de> ---
On Fri, 31 Jan 2014, jakub at gcc dot gnu.org wrote:

> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59811
> 
> --- Comment #6 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
> Dunno what to limit there though.
> 
> BTW, the 1000 default for sccvn-max-alias-queries-per-access param was some
> carefully chosen value, e.g. does the test that was fixed by the addition of
> the param fail if it is bumped to say 1500 or 2000, or can we just bump
> slightly the default?

It was reasonably chosen - the testcase I added it for still uses
a lot of time in alias-walking (~30% I think which I decided was
reasonable for the testcase).  We sure can slightly bump it,
but that only requires a slightly larger testcase to re-trigger
this bug?

[Bug rtl-optimization/59811] [4.8/4.9/4.10 Regression] Huge increase in memory usage and compile time in combine

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=59811

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|4.8.3                       |4.8.4

--- Comment #8 from Richard Biener <rguenth at gcc dot gnu.org> ---
GCC 4.8.3 is being released, adjusting target milestone.

[Bug rtl-optimization/59811] [4.8/4.9/5 Regression] Huge increase in memory usage and compile time in combine

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=59811

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|4.8.4                       |4.8.5

--- Comment #9 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
GCC 4.8.4 has been released.

[Bug rtl-optimization/59811] [4.8/4.9/5/6 Regression] Huge increase in memory usage and compile time in combine

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=59811

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|4.8.5                       |4.9.3

--- Comment #10 from Richard Biener <rguenth at gcc dot gnu.org> ---
The gcc-4_8-branch is being closed, re-targeting regressions to 4.9.3.

[Bug rtl-optimization/59811] [4.9/5/6 Regression] Huge increase in memory usage and compile time in combine

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=59811

--- Comment #11 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
GCC 4.9.3 has been released.

[Bug rtl-optimization/59811] [4.9/5/6 Regression] Huge increase in memory usage and compile time in combine

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=59811

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|4.9.3                       |4.9.4