[Bug fortran/63577] New: [4.8/4.9/5? Regression]: Huge compile time and memory usage with -O and not -fPIC

[Bug fortran/63577] New: [4.8/4.9/5? Regression]: Huge compile time and memory usage with -O and not -fPIC

[jfsoden at mpp dot mpg.de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63577

            Bug ID: 63577
           Summary: [4.8/4.9/5? Regression]: Huge compile time and memory
                    usage with -O and not -fPIC
           Product: gcc
           Version: unknown
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: fortran
          Assignee: unassigned at gcc dot gnu.org
          Reporter: jfsoden at mpp dot mpg.de

Created attachment 33749
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=33749&action=edit
Test program.

The attached Fortran programs compiles slowly and needs lots of memory
if compiled with basic optimization (-O) but without -fPIC.
With -fPIC, memory usage and compile time are much shorter.


$ gfortran abbrevd408h0.f90 -c -O  
needs 45s and 630MB RAM

but

$ gfortran abbrevd408h0.f90 -c -O -fPIC
needs only 6s and 105MB RAM.

$ gfortran abbrevd408h0.f90 -c
works also quickly.

The attached file is shortened. On the full file, the effect is more distinct
(with gfortran 4.8.1: >8GB RAM usage, several minutes).

With gfortran 4.7, the issue does not appear.

Used version:
GNU Fortran (Debian 4.9.1-17) 4.9.1     [~ SVN r216240]

Also
GNU Fortran (SUSE Linux) 4.8.1 20130909 [gcc-4_8-branch revision 202388]
GNU Fortran (SUSE Linux) 4.8.3 20140627 [gcc-4_8-branch revision 212064] 
are affected.

[Bug rtl-optimization/63577] [4.8/4.9/5? Regression]: Huge compile time and memory usage with -O and not -fPIC

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63577

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Target|                            |x86_64-*-*
             Status|UNCONFIRMED                 |NEW
   Last reconfirmed|                            |2014-10-20
          Component|fortran                     |rtl-optimization
            Version|unknown                     |4.9.1
             Blocks|                            |47344
   Target Milestone|---                         |4.8.4
     Ever confirmed|0                           |1

--- Comment #1 from Richard Biener <rguenth at gcc dot gnu.org> ---
Confirmed with 4.9:

 combiner                :  31.14 (79%) usr   0.46 (74%) sys  31.65 (78%) wall
1029289 kB (96%) ggc
 TOTAL                 :  39.48             0.62            40.77           
1071504 kB

[Bug rtl-optimization/63577] [4.8/4.9/5 Regression]: Huge compile time and memory usage with -O and not -fPIC

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63577

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|[4.8/4.9/5? Regression]:    |[4.8/4.9/5 Regression]:
                   |Huge compile time and       |Huge compile time and
                   |memory usage with -O and    |memory usage with -O and
                   |not -fPIC                   |not -fPIC

--- Comment #2 from Richard Biener <rguenth at gcc dot gnu.org> ---
--param max-combine-insns=2 helps a bit compile-time wise but not fully
memory-usage-wise (I suppose log-links are expensive and of course still set
up).
Only available on trunk, of course.

[Bug rtl-optimization/63577] [4.8/4.9/5 Regression]: Huge compile time and memory usage with -O and not -fPIC

[segher at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63577

Segher Boessenkool <segher at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |segher at gcc dot gnu.org

--- Comment #3 from Segher Boessenkool <segher at gcc dot gnu.org> ---
The LOG_LINKS take up only a few hundred kB, tops; the gigantic memory
use is from of all the garbage RTL produced for all the failed combine
attempts.

[Bug rtl-optimization/63577] [4.8/4.9/5 Regression]: Huge compile time and memory usage with -O and not -fPIC

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63577

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Priority|P3                          |P2

[Bug rtl-optimization/63577] [4.8/4.9/5 Regression]: Huge compile time and memory usage with -O and not -fPIC

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63577

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|4.8.4                       |4.8.5

--- Comment #4 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
GCC 4.8.4 has been released.

[Bug rtl-optimization/63577] [4.8/4.9/5 Regression]: Huge compile time and memory usage with -O and not -fPIC

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63577

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jakub at gcc dot gnu.org

--- Comment #5 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Does the combiner have any GC pointers stored in non-GTY memory?  I mean e.g.
LOG_LINKS, ...  If we could ggc_collect within the pass, the memory consumption
problem would be fixed.

[Bug rtl-optimization/63577] [4.8/4.9/5 Regression]: Huge compile time and memory usage with -O and not -fPIC

[segher at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63577

--- Comment #6 from Segher Boessenkool <segher at gcc dot gnu.org> ---
[ I missed your last comment, sorry. ]

Both the log_links and the reg_stat point to insns in the insn stream,
(all of those are either live or never again referred to), so that
might be fine, but you really should make sure you only GC between
try_combine calls, never inside one -- that would be rather disastrous.

Do you want to try this for GCC 5?

[Bug rtl-optimization/63577] [4.8/4.9/5 Regression]: Huge compile time and memory usage with -O and not -fPIC

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63577

--- Comment #7 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
It is a regression, so perhaps, depends on how risky the patch would be.
Most likely it would need to be tested with always-collect params on a few
larger testcases.

[Bug rtl-optimization/63577] [4.8/4.9/5 Regression]: Huge compile time and memory usage with -O and not -fPIC

[segher at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63577

--- Comment #8 from Segher Boessenkool <segher at gcc dot gnu.org> ---
It's not a very new regression, and it is quite risky in my opinion;
I prefer to have this dealt with in stage1.

[Bug rtl-optimization/63577] [4.8/4.9/5 Regression]: Huge compile time and memory usage with -O and not -fPIC

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63577

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|4.8.5                       |6.0

--- Comment #9 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Ok then.

[Bug rtl-optimization/63577] [4.8/4.9/5 Regression]: Huge compile time and memory usage with -O and not -fPIC

[segher at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63577

--- Comment #10 from Segher Boessenkool <segher at gcc dot gnu.org> ---
Also note that doing GC during the pass will not reduce the compile
time or the amount of garbage created at all, so won't fix the actual
problem; it does of course make it more bearable on smaller machines.

I'll have another look at what causes this; from what I remember last
time I looked there simply *are* very many opportunities to combine
some insns (most of which fail, maybe we could short-circuit some).