[Bug ada/34898] New: Excessive memory consumption during compilation

[Bug ada/34898] New: Excessive memory consumption during compilation

[oliver dot kellogg at eads dot com]

For reference, here's a compile with gcc-3.3.5:

$ gcc -c -v pkg001u.adb
Reading specs from /usr/lib/gcc-lib/i586-suse-linux/3.3.5/specs
Configured with: ../configure --enable-threads=posix --prefix=/usr
--with-local-prefix=/usr/local --infodir=/usr/share/info
--mandir=/usr/share/man --enable-languages=c,c++,f77,objc,java,ada
--disable-checking --libdir=/usr/lib --enable-libgcj --with-slibdir=/lib
--with-system-zlib --enable-shared --enable-__cxa_atexit i586-suse-linux
Thread model: posix
gcc version 3.3.5 20050117 (prerelease) (SUSE Linux)
 /usr/lib/gcc-lib/i586-suse-linux/3.3.5/gnat1 -quiet -dumpbase pkg001u.adb
pkg001u.adb -o /tmp/cckqHQka.s

The file compiles successfully and 'top' shows a peak RES requirement of 77 Meg
using no VIRT.
Thus let's limit the VIRT to a half gig,

$ ulimit -v 512000

and compile with a newer gcc:

$ gcc -c -v -gnat95 pkg001u.adb
Using built-in specs.
Target: i686-pc-linux-gnu
Configured with: ../../SOURCES/gcc/configure --prefix=/opt/gccsnap
--enable-debug --enable-languages=c,ada,c++
Thread model: posix
gcc version 4.3.0 20080114 (experimental) (GCC) 
COLLECT_GCC_OPTIONS='-c' '-v' '-gnat95' '-mtune=generic'
 /opt/gccsnap/libexec/gcc/i686-pc-linux-gnu/4.3.0/gnat1 -quiet -dumpbase
pkg001u.adb -gnat95 -mtune=generic pkg001u.adb -o /tmp/ccmBlEVU.s
+===========================GNAT BUG DETECTED==============================+
| 4.3.0 20080114 (experimental) (i686-pc-linux-gnu) Storage_Error heap
exhausted|
| Error detected at pkg001u.adb:296:35                                     |
| Please submit a bug report; see http://gcc.gnu.org/bugs.html.            |
| Use a subject line meaningful to you and us to track the bug.            |
| Include the entire contents of this bug box in the report.               |
| Include the exact gcc or gnatmake command that you entered.              |
| Also include sources listed below in gnatchop format                     |
| (concatenated together with no headers between files).                   |
+==========================================================================+

Please include these source files with error report
Note that list may not be accurate in some cases,
so please double check that the problem can still
be reproduced with the set of files listed.

pkg001u.adb
pkg001u.ads
p_portable.ads
corba.ads
pkg001z.ads
pkg0013.ads
pkg000x.ads
pkg001o.ads
pkg000q.ads
pkg000v.ads
pkg000y.ads
pkg001l.ads
pkg0025.ads
pkg001e.ads
pkg000w.ads
pkg001r.ads
pkg000u.ads
pkg002n.ads
pkg002f.ads
pkg0020.ads
pkg0019.ads
pkg001w.ads
pkg000t.ads
pkg001v.ads
pkg001m.ads
pkg000a.ads
pkg000d.ads
pkg000c.ads
pkg0007.ads
pkg0009.ads
pkg000k.ads
pkg0001.ads
pkg000g.ads
pkg000b.ads
pkg000l.ads
pkg000f.ads
pkg002g.ads
pkg0006.ads
pkg001k.ads
pkg0000.ads
pkg0005.ads
pkg002o.ads
pkg0028.ads
pkg0029.ads
pkg000h.ads
pkg002p.ads
pkg000j.ads
pkg000i.ads
pkg0027.ads
pkg0002.ads
pkg0022.ads
pkg0023.ads
pkg0012.ads
pkg0026.ads
pkg000r.ads
pkg000s.ads
c_types.ads
pkg000m.ads
pkg001a.ads
pkg0003.ads
pkg001n.ads
pkg0021.ads
pkg001t.ads
pkg001p.ads
pkg001c.ads
pkg0004.ads
pkg0016.ads
pkg001s.ads
pkg001x.ads
pkg001z-o_protokoll.ads
pkg0018.ads
pkg000z.ads
pkg001j.ads
pkg0011.ads
pkg001d.ads
pkg002m.ads
pkg002e.ads
pkg002d.ads
pkg000o.ads
pkg001b.ads
pkg000p.ads
pkg001f.ads
pkg0015.ads
pkg000n.ads
pkg0014.ads
pkg0017.ads
pkg0024.ads

compilation abandoned

(Tried limiting the VIRT to 1.5 gig, same result.)


-- 
           Summary: Excessive memory consumption during compilation
           Product: gcc
           Version: 4.3.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: ada
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: oliver dot kellogg at eads dot com
  GCC host triplet: i686-pc-linux-gnu


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=34898

[Bug ada/34898] Excessive memory consumption during compilation

[oliver dot kellogg at eads dot com]

------- Comment #1 from oliver dot kellogg at eads dot com  2008-01-21 07:30 -------
Created an attachment (id=14985)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=14985&action=view)
source code for producing the bug

I tried this with gcc-4.1, 4.2, and 4.3.

If I remove the aggregate assignments such as

   Fumessage.O000X := (Pkg000W.E08EY, Aktergsatz);

from pkg001u.adb then compilation proceeds at normal memory consumption.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=34898

[Bug ada/34898] Excessive memory consumption during compilation

[oliver dot kellogg at eads dot com]

------- Comment #2 from oliver dot kellogg at eads dot com  2008-05-22 11:23 -------
Still happens with 4.4.0 20080522.
Please advise if there is any further info that I could provide
to help track the problem down.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=34898

[Bug ada/34898] Excessive memory consumption during compilation

[oliver dot kellogg at eads dot com]

------- Comment #3 from oliver dot kellogg at eads dot com  2008-05-25 10:38 -------
Does not happen with -gnatc (syntax and semantics check only.)


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=34898

[Bug ada/34898] Excessive memory consumption during compilation

[rguenth at gcc dot gnu dot org]

------- Comment #4 from rguenth at gcc dot gnu dot org  2008-05-25 12:03 -------
Does enabling optimization (-O) fix the problem?  My guess is that the
gimplification of the aggregate assignments creates lots of overhead, but that
needs to be investigated by Ada people - stats with a compiler configured with
--enable-gather-detailed-mem-stats would also be useful.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=34898

[Bug ada/34898] Excessive memory consumption during compilation

[oliver dot kellogg at eads dot com]

------- Comment #5 from oliver dot kellogg at eads dot com  2008-05-25 13:31 -------
Created an attachment (id=15679)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=15679&action=view)
statistics output from gnat1 on pkg001u.adb without aggregate assignments


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=34898

[Bug ada/34898] Excessive memory consumption during compilation

[oliver dot kellogg at eads dot com]

------- Comment #6 from oliver dot kellogg at eads dot com  2008-05-25 13:38 -------
Created an attachment (id=15680)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=15680&action=view)
statistics output from gnat1 on pkg001u.adb with one assignment

Here, I enabled the assignment in line 377,

   Tramessage.O000X := (Pkg000W.E08EY, Aktergsatz);

and invoked gnat1 directly as follows:

/opt/gccsnap/libexec/gcc/i686-pc-linux-gnu/4.4.0/gnat1 -gnat95 -mtune=generic \
   pkg001u.adb

(Notice the increase in 'expand'. Is that within expected limits?)


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=34898

[Bug ada/34898] Excessive memory consumption during compilation

[rguenth at gcc dot gnu dot org]

------- Comment #7 from rguenth at gcc dot gnu dot org  2008-05-25 13:48 -------
Well, this assignment seems to be _very_ expensive both in terms of parsing
time
and size of the IL to expand.  It certainly looks unreasonable.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=34898

[Bug ada/34898] Excessive memory consumption during compilation

[oliver dot kellogg at eads dot com]

------- Comment #8 from oliver dot kellogg at eads dot com  2008-05-25 15:42 -------
(in reply to comment #4)
> Does enabling optimization (-O) fix the problem?

No, does not change the behavior (other than taking even longer)

> [...] stats with a compiler configured with
> --enable-gather-detailed-mem-stats would also be useful.

Coming up.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=34898

[Bug ada/34898] Excessive memory consumption during compilation

[oliver dot kellogg at eads dot com]

------- Comment #9 from oliver dot kellogg at eads dot com  2008-05-25 18:12 -------
Created an attachment (id=15681)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=15681&action=view)
gnat1 (trunk r135848) output from -fmem-report, no aggregate assignments


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=34898

[Bug ada/34898] Excessive memory consumption during compilation

[oliver dot kellogg at eads dot com]

------- Comment #10 from oliver dot kellogg at eads dot com  2008-05-25 18:17 -------
Created an attachment (id=15682)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=15682&action=view)
same as above but with assignments in pkg001u.adb lines 296 and 377 enabled


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=34898

[Bug ada/34898] Excessive memory consumption during compilation

[oliver dot kellogg at eads dot com]

------- Comment #11 from oliver dot kellogg at eads dot com  2008-05-25 18:43 -------
Created an attachment (id=15683)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=15683&action=view)
att15682 was incorrect, two assignments already exhaust the memory. memreport
for _one_ assignmt.


-- 

oliver dot kellogg at eads dot com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #15682|0                           |1
        is obsolete|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=34898

[Bug ada/34898] Excessive memory consumption during compilation

[rguenth at gcc dot gnu dot org]

------- Comment #12 from rguenth at gcc dot gnu dot org  2008-05-25 18:56 -------
ada/utils2.c:1774 (build_simple_component_ref)    111547200:71.1%

clearly a frontend issue.


-- 

rguenth at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
     Ever Confirmed|0                           |1
   Last reconfirmed|0000-00-00 00:00:00         |2008-05-25 18:56:20
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=34898

[Bug ada/34898] Excessive memory consumption during compilation

[oliver dot kellogg at eads dot com]

------- Comment #13 from oliver dot kellogg at eads dot com  2008-06-01 18:10 -------
Created an attachment (id=15708)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=15708&action=view)
instrumentation of build_simple_component_ref (for analysis only)

I put in some putchar calls of different letters to see where the code is
going.
The code formatting looks bad but maintains the original line numbers.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=34898

[Bug ada/34898] Excessive memory consumption during compilation

[oliver dot kellogg at eads dot com]

------- Comment #14 from oliver dot kellogg at eads dot com  2008-06-01 18:28 -------
Created an attachment (id=15709)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=15709&action=view)
console output from gnat1 with the above instrumentation

The following pattern occurs extremely often:

   'C'  (DECL_CONTEXT (field) != record_type), line 1716
   'D'  (!new_field), line 1735
   'J'  (!field), line 1753, return NULL_TREE
 'H'  retval from call at line 1744 was NULL
 'E'  (DECL_INTERNAL_P (new_field)) on next iteration of loop at line 1736
   'M'  now in new call to build_simple_component_ref
   'P'  return fold (ref);   at line 1800
 'F'  just before next call to build_simple_component_ref at line 1744

and back up to 'C'.

(I am still trying to understand what is actually happening.)


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=34898

[Bug ada/34898] Excessive memory consumption during compilation

[rguenth at gcc dot gnu dot org]

------- Comment #15 from rguenth at gcc dot gnu dot org  2008-06-01 18:56 -------
You can look at the original IL generated, I guess the assignments simply
contain
millions of element assignments (-fdump-tree-original).

??? tree nodes created

Kind                   Nodes      Bytes
...
refs                 2903482  115769372

some Ada guy needs to look into this.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=34898

[Bug ada/34898] Excessive memory consumption during compilation

[oliver dot kellogg at eads dot com]

------- Comment #16 from oliver dot kellogg at eads dot com  2008-06-02 19:16 -------
Created an attachment (id=15715)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=15715&action=view)
output from -fdump-tree-original of gnat1 compiling pkg001u.adb

(in reply to comment #15)
> You can look at the original IL generated, I guess the assignments simply
> contain
> millions of element assignments (-fdump-tree-original).

Ah, thanks.
I believe the code at line 29417 represents the problematic assignment
(pkg001u.adb line 296), could somebody check?

Some of this stuff looks strange to the layman's eye, for example
around line 29402:

                    <<< Unknown tree: loop_stmt


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=34898

[Bug ada/34898] Excessive memory consumption during compilation

[oliver dot kellogg at eads dot com]

------- Comment #17 from oliver dot kellogg at eads dot com  2008-08-21 05:41 -------
Created an attachment (id=16118)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=16118&action=view)
regenerated statistics: trunk r139367 gnat1-gnat95 -fmem-report -fdump-tree-all
pkg001u.adb


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=34898