[Bug c++/14179] New: out of memory

[Bug c++/14179] New: out of memory

[debora dot j dot estey at lmco dot com]

We are trying to move from gcc 2.95 to 3.3.1. We are using cygwin and the code 
that will not compile is code in a library generated by another group. we must 
use it as is. The other group also uses the library but they use a greenhills 
compiler. The code will compile with gcc 2.95 but gcc 3.3.1 give us this error: 

cc1plus: out of memory allocating 65536 bytes after a total of 402620416 bytes

The gcc -v produces this information : 
Reading specs from /bin/../lib/gcc-lib/i686-pc-cygwin/3.3.1/specs
Configured with: /netrel/src/gcc-3.3.1-2/configure --enable-
languages=c,c++,f77,java --enable-libgcj --enable-threads=posix --with-system-
zlib --enable-nls --without-included-gettext --enable-interpreter --enable-sjlj-
exceptions --disable-version-specific-runtime-libs --enable-shared --build=i686-
pc-linux --host=i686-pc-cygwin --target=i686-pc-cygwin --prefix=/usr --exec-
prefix=/usr --sysconfdir=/etc --libdir=/usr/lib --
includedir=/nonexistent/include --libexecdir=/usr/sbin
Thread model: posix
gcc version 3.3.1 (cygming special)

-- 
           Summary: out of memory
           Product: gcc
           Version: 3.3.3
            Status: UNCONFIRMED
          Severity: normal
          Priority: P1
         Component: c++
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: debora dot j dot estey at lmco dot com
                CC: gcc-bugs at gcc dot gnu dot org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] out of memory

[debora dot j dot estey at lmco dot com]

------- Additional Comments From debora dot j dot estey at lmco dot com  2004-02-17 17:15 -------
Created an attachment (id=5760)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=5760&action=view)
file created with the -save-temps

the command line that was run to produce this error was: 
g++ -I/usr/include/GL -c -I/RhapsodyCustomizations/Share/LangCpp
-fno-schedule-insns -fno-schedule-insns2
-/RapsodyCustomizations/Share/LangCpp/osconfig/Cygwin
-I/RhapsodyCustomizations/Share -I/RhapsodyCustomizations/Share/LangCpp/oxf
-I/uaenfs_ASE/base/uae_dev/b2v6_1_0/uae//models/src/cds/include/
-I/uaenfs_ASE/base/uae_dev/b2v6_1_0/uae//models/src/cds/include/mfdSym/
-I/uaenfs_ASE/base/uae_dev/b2v6_1_0/uae//include
-I/uaenfs_ASE/base/uae_dev/b2v6_1_0/universal/include
-I/uaenfs_ASE/base/uae_dev/b2v6_1_0/universal/include/i386-pc-cygwin32
-I/uaenfs_ASE/base/uae_dev/b2v6_1_0universal/include/i386-pc-cygwin32
-I/uaenfs_ASE/base/uae_dev/b2v6_1_0//universal/lib/src/libsdn/include
-I/usr/local/mysql/include/mysql -I/Exceed/xdk/motif21/include
-I/Exceed/xdk/include -I/usr/include/GL -I/Exceed/xdk/motif21/include
-DARCH_X86 -DOS_CYGWIN -DOS_VERSION=1322 -DWIN32 -DNT -DMOTIFAPP
-DBO_LITTLE_ENDIAN -DOM_USE_STL -DDBTYPE_MYSQL -DUSE_IOSTREAM -DARCH_X86
-DCYGWIN -DOS_VERSION=1322 -DWIN32 -DNT -DXMSTATIC -c ../SymSymbolGwb.cpp -o
SymSymbolGwb.o

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] out of memory

[debora dot j dot estey at lmco dot com]

------- Additional Comments From debora dot j dot estey at lmco dot com  2004-02-17 17:25 -------
Created an attachment (id=5763)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=5763&action=view)
file created with -save-temps

file was to large so it was compressed using bzip2

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
Attachment #5760 is|0                           |1
           obsolete|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] out of memory

[giovannibajo at libero dot it]

------- Additional Comments From giovannibajo at libero dot it  2004-02-17 17:52 -------
Debora,

Your problem is that G++ needs more RAM to be able to compile that file.

Lately, there has been a lot of work on making the C++ compiler consumes less 
and less memory, and on making on faster, but this is work cannot be backported 
into the 3.3 stable release serie. Currently G++ 3.4.0 compiles faster than 
2.95 on most tests, and consumes around the same amount of memory. 

I would suggest you to try again upgrading your compiler to 3.4.0. You will 
have to pull it from CVS and recompile it. Also, you may want to take a look at 
http://gcc.gnu.org/gcc-3.4/changes.html, since the new compiler is much more 
strict to the C++ standard, so old code might need to be modified. 

If this is feasable for you, I will be looking forward to a new report from you 
on how 3.4.0 behaves with your code. Alas, it's probably too late to improve 
the 3.3 serie so much. If you really need to use it, you can probably trying 
adding more RAM to your computer, it should probably allow it to finish 
compilation.

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |WAITING
            Summary|out of memory               |out of memory


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/3.5 Regression] out of memory

[pinskia at gcc dot gnu dot org]

------- Additional Comments From pinskia at gcc dot gnu dot org  2004-02-17 18:00 -------
Confirmed., the problem is the large array for some reason GCC now likes to take huge amount 
memory for the array, mega_TextureSymbolData.

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |NEW
     Ever Confirmed|                            |1
           Keywords|                            |memory-hog
   Last reconfirmed|0000-00-00 00:00:00         |2004-02-17 18:00:05
               date|                            |
            Summary|out of memory               |[3.3/3.4/3.5 Regression] out
                   |                            |of memory
   Target Milestone|---                         |3.4.0


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/3.5 Regression] out of memory

[pinskia at gcc dot gnu dot org]

------- Additional Comments From pinskia at gcc dot gnu dot org  2004-02-17 18:10 -------
Related to bug 12245.

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/3.5 Regression] out of memory

[mmitchel at gcc dot gnu dot org]

------- Additional Comments From mmitchel at gcc dot gnu dot org  2004-03-08 23:08 -------
I've investigated this problem.

To some extent, this problem is inevitable.  In the old days, we used to output
assembly code for a global array element-by-element as we saw it.  That's not
what we want to do: we want to store up the array so that we can optimize loads
from fixed indices, etc.

However, we do waste a ton of memory.  We allocate a separate INTEGER_CST for
each occurrence of the same integer constant, even though there are only a few
of them.  We allocate an entirely new list of constants when we refactor the
brace-enclosed form (essentially adding braces that C++ says you can omit).  We
allocate an integer constant for every possible index into the array, which has
8 million entries.

We should be representing the initializer with a structure like this:

  struct init_group { 
    struct init_group *next;
    tree designator; /* The designator for the first element in the array.  */
    tree elts[];
  };

rather than a linked-list per element.  That would be far more efficient for
large arrays and a win even for small arrays.

None of this is going to get fixed until 3.5, however.

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|3.4.0                       |3.5.0


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/3.5 Regression] out of memory

[pinskia at gcc dot gnu dot org]

------- Additional Comments From pinskia at gcc dot gnu dot org  2004-06-07 03:25 -------
I will take a look at this next week because of things which need to improve with respect to compile 
time.

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |pinskia at gcc dot gnu dot
                   |                            |org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory

[giovannibajo at libero dot it]

------- Additional Comments From giovannibajo at libero dot it  2004-09-15 14:26 -------
Mark, one question: are you suggesting the special structure for the 
constructor only before reshape_init or also after it? Because we need to build 
a CONSTRUCTOR sooner or later for the gimplifier.

Anyway, would such a change acceptable for Stage 3, being a bugfix towards 
better memory allocation (and fixing this regression)?

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |mmitchel at gcc dot gnu dot
                   |                            |org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory

[mark at codesourcery dot com]

------- Additional Comments From mark at codesourcery dot com  2004-09-15 16:29 -------
Subject: Re:  [3.3/3.4/4.0 Regression] out of memory

giovannibajo at libero dot it wrote:

>------- Additional Comments From giovannibajo at libero dot it  2004-09-15 14:26 -------
>Mark, one question: are you suggesting the special structure for the 
>constructor only before reshape_init or also after it? Because we need to build 
>a CONSTRUCTOR sooner or later for the gimplifier.
>  
>
Both.  After reshape_init is even more critical because that memory 
cannot be collected.

>Anyway, would such a change acceptable for Stage 3, being a bugfix towards 
>better memory allocation (and fixing this regression)?
>
Perhaps, but it seems unlikely.  I've thought about it, but I'm scared 
of how much impact there would be through the compiler.  There are some 
smaller things that we could do more locally, though, like reusing the 
TREE_LISTs from before reshape_init after reshape_init.  Also, I suspect 
that Nathan's integer-sharing work has already reduced this problem 
somewhat; part of the problem was that we made multiple copies of every 
INTEGER_CST frpm zero up to the upper bound of the array.  Now, we 
should have only one at least.



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory

[giovannibajo at libero dot it]

------- Additional Comments From giovannibajo at libero dot it  2004-09-17 18:50 -------
It looks like we do not destroy and recreate initializers in reshape_init, 
elements are moved from the old CONSTRUCTOR to the new one.

Instead, while investigating the code, I noticed this in reshape_init:

          /* Loop through the array elements, gathering initializers.  */
          for (index = size_zero_node;
               *initp && (!max_index || !tree_int_cst_lt (max_index, index));
               index = size_binop (PLUS_EXPR, index, size_one_node))
            {

We are constructing a *different* INTEGER_CST for each index, and we never use 
it. This generates a lot of garbage.

I do not know if it is enough to switch to HOST_WIDE_INT only, we may want to 
handle arrays larger than HWI (e.g. crosscompiling from 16bit to 32bit). My 
solution for mainline is to use HWI whenever possible, and falling back to 
trees when the indices get too high. Mark, does this make sense?

Dunno if this will be acceptable for 3.3 and 3.4 too, but let's have this fixed 
in mainline, as a start.

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
         AssignedTo|unassigned at gcc dot gnu   |giovannibajo at libero dot
                   |dot org                     |it
             Status|NEW                         |ASSIGNED


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory

[giovannibajo at libero dot it]

------- Additional Comments From giovannibajo at libero dot it  2004-09-18 13:10 -------
Two patches posted, waiting for review:
http://gcc.gnu.org/ml/gcc-patches/2004-09/msg01839.html
http://gcc.gnu.org/ml/gcc-patches/2004-09/msg01840.html


-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |patch


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory

[cvs-commit at gcc dot gnu dot org]

------- Additional Comments From cvs-commit at gcc dot gnu dot org  2004-09-20 23:05 -------
Subject: Bug 14179

CVSROOT:	/cvs/gcc
Module name:	gcc
Changes by:	giovannibajo@gcc.gnu.org	2004-09-20 23:05:43

Modified files:
	gcc/cp         : ChangeLog decl.c 

Log message:
	PR c++/14179
	* decl.c (reshape_init): Extract array handling into...
	(reshape_init_array): New function. Use integers instead of trees
	for indices. Handle out-of-range designated initializers.

Patches:
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/cp/ChangeLog.diff?cvsroot=gcc&r1=1.4366&r2=1.4367
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/cp/decl.c.diff?cvsroot=gcc&r1=1.1296&r2=1.1297



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory

[cvs-commit at gcc dot gnu dot org]

------- Additional Comments From cvs-commit at gcc dot gnu dot org  2004-09-21 21:12 -------
Subject: Bug 14179

CVSROOT:	/cvs/gcc
Module name:	gcc
Branch: 	gcc-3_3-branch
Changes by:	giovannibajo@gcc.gnu.org	2004-09-21 21:12:51

Modified files:
	gcc/cp         : ChangeLog decl.c 

Log message:
	PR c++/14179
	* decl.c (reshape_init): Extract array handling into...
	(reshape_init_array): New function. Use integers instead of trees
	for indices.

Patches:
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/cp/ChangeLog.diff?cvsroot=gcc&only_with_tag=gcc-3_3-branch&r1=1.3076.2.274&r2=1.3076.2.275
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/cp/decl.c.diff?cvsroot=gcc&only_with_tag=gcc-3_3-branch&r1=1.965.2.84&r2=1.965.2.85



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory

[cvs-commit at gcc dot gnu dot org]

------- Additional Comments From cvs-commit at gcc dot gnu dot org  2004-09-21 22:50 -------
Subject: Bug 14179

CVSROOT:	/cvs/gcc
Module name:	gcc
Branch: 	gcc-3_4-branch
Changes by:	giovannibajo@gcc.gnu.org	2004-09-21 22:49:56

Modified files:
	gcc/cp         : ChangeLog decl.c 

Log message:
	PR c++/14179
	* decl.c (reshape_init): Extract array handling into...
	(reshape_init_array): New function. Use integers instead of trees
	for indices. Handle out-of-range designated initializers.

Patches:
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/cp/ChangeLog.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=1.3892.2.158&r2=1.3892.2.159
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/cp/decl.c.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=1.1174.2.23&r2=1.1174.2.24



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory

[cvs-commit at gcc dot gnu dot org]

------- Additional Comments From cvs-commit at gcc dot gnu dot org  2004-09-21 23:46 -------
Subject: Bug 14179

CVSROOT:	/cvs/gcc
Module name:	gcc
Branch: 	gcc-3_4-branch
Changes by:	giovannibajo@gcc.gnu.org	2004-09-21 23:46:08

Modified files:
	gcc/cp         : ChangeLog parser.c 

Log message:
	PR c++/14179
	* parser.c (cp_parser_initializer): Speed up parsing of simple
	literals as initializers.

Patches:
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/cp/ChangeLog.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=1.3892.2.159&r2=1.3892.2.160
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/cp/parser.c.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=1.157.2.40&r2=1.157.2.41



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory

[giovannibajo at libero dot it]

------- Additional Comments From giovannibajo at libero dot it  2004-09-22 00:36 -------
OK, let me update the situation. I'm using a testcase which has an array of 4 
millions of initializers (about 1/4th of the original testcase). These are the 
three patches I'm considering important for this patch till now:

(1) Nathan's INTEGER_CST sharing
(2) my patch to reshape_init_array using HOST_WIDE_INTEGER
(3) my patch to speedup initializer list parsing

(1) and (2) affects memory occupation, (3) affects compilation time. This is 
the situations for our supported compilers:

GCC-3.3.5
---------
(1) not present
(2) applied
(3) not required (old parser)

Testcase: killed after 0m31s (420MB is the kill threshold).



GCC-3.4.3
---------
(1) not present
(2) applied
(3) applied

Testcase: killed after 0m23s (420MB is the kill threshold).


GCC-4.0.0
---------
(1) applied
(2) applied
(3) will be applied (or rewritten in a better form).

Testcase: 0m26s, 220MB
(this is with (3) applied, and checking disabled)


For comparison, GCC-2.95: 0m5s, 175MB.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory

[giovannibajo at libero dot it]

------- Additional Comments From giovannibajo at libero dot it  2004-09-22 00:38 -------
So, situation is getting better, but we are not there yet. My next target is 
process_init_constructor, where we are being very dumb.

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|patch                       |compile-time-hog


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory

[giovannibajo at libero dot it]

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
  BugsThisDependsOn|                            |17596


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory

[mark at codesourcery dot com]

------- Additional Comments From mark at codesourcery dot com  2004-09-22 00:54 -------
Subject: Re:  [3.3/3.4/4.0 Regression] out of memory

giovannibajo at libero dot it wrote:

>Testcase: 0m26s, 220MB
>(this is with (3) applied, and checking disabled)
>
>
>For comparison, GCC-2.95: 0m5s, 175MB.
>  
>
For the record, I don't expect we will get all the way back there.  I 
consider it a design feature that we are keeping the initializer around 
in the compiler: that allows us to (in theory) pull elements out of it 
if they are referenced.  With that capabitility comes a cost.

That is not to say that we should not keep going with your work of course!

FYI, Jeffrey Oldham is working on operator-precedence parsing; hopefully 
he'll have a patch soon.



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory

[bonzini at gnu dot org]

------- Additional Comments From bonzini at gnu dot org  2004-09-22 13:51 -------
Actually precedence parsing is not a big job, incrementally what I did for my
patch was just a matter of
1) making a single map of all the productions instead of the per-function maps
we have currently
2) making cp_parser_binary_expression use it, still keeping the recursive call
and passing around the precedence we are interested in
3) turn recursion into an explicit stack
4) optimize to eliminate unneeded (simulated) recursion

It would be nice if the wiki were used to track in-house CodeSourcery projects.
 It looks like Jeffrey and I duplicated work, and we almost did the same on the
tree class codes projects (and I'm not mocking CodeSourcery's work, I'm just
reading the wiki and using common sense on where to optimize the compiler).

Paolo

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |oldham at codesourcery dot
                   |                            |com


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory

[mark at codesourcery dot com]

------- Additional Comments From mark at codesourcery dot com  2004-09-22 15:33 -------
Subject: Re:  [3.3/3.4/4.0 Regression] out of memory

bonzini at gnu dot org wrote:

>------- Additional Comments From bonzini at gnu dot org  2004-09-22 13:51 -------
>Actually precedence parsing is not a big job, incrementally what I did for my
>patch was just a matter of
>1) making a single map of all the productions instead of the per-function maps
>we have currently
>2) making cp_parser_binary_expression use it, still keeping the recursive call
>and passing around the precedence we are interested in
>3) turn recursion into an explicit stack
>4) optimize to eliminate unneeded (simulated) recursion
>
>It would be nice if the wiki were used to track in-house CodeSourcery projects.
> It looks like Jeffrey and I duplicated work, and we almost did the same on the
>tree class codes projects (and I'm not mocking CodeSourcery's work, I'm just
>reading the wiki and using common sense on where to optimize the compiler).
>  
>
I'm sorry that happened, but we did mention the operator-precedence 
parsing and tree-code conversion work publicly as we started on it.  
We'll try to be louder.  I am certainly no happier than you that we are 
duplicating each other's work!



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory

[mmitchel at gcc dot gnu dot org]

------- Additional Comments From mmitchel at gcc dot gnu dot org  2004-09-22 15:39 -------
Paolo --

This patch is great.  My only comment is that I would like the grammar entries
that used to be above the various functions (pm-expression: ..., etc.) to be
preserved above the rewritten binary_expression function.  Please check in!

Thanks,

-- Mark

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory

[paolo dot bonzini at polimi dot it]

------- Additional Comments From paolo dot bonzini at polimi dot it  2004-09-22 15:43 -------
Subject: Re:  [3.3/3.4/4.0 Regression] out of memory

> I'm sorry that happened, but we did mention the operator-precedence 
> parsing and tree-code conversion work publicly as we started on it.  

Though I only knew of tree-code conversion from private mail by Zack 
(and you said you were about to finish it when I said I'd do that on 
faster-compiler-branch); and the operator-precedence was placed on the 
wiki a week ago as a generic "speedup area", not a project that is 
actively worked on unlike Matt's lexer overhaul:

http://www.dberlin.org/gccwiki/index.php/Speedup%20areas

Since that page also mentions Nathan's "Add contains-repeated-base and 
is diamond-shaped flags to classes" project, maybe it is *that* page 
that needs to be louder.

 > We'll try to be louder.

No problem.  You're doing a good work.

Paolo




-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory

[mark at codesourcery dot com]

------- Additional Comments From mark at codesourcery dot com  2004-09-22 15:54 -------
Subject: Re:  [3.3/3.4/4.0 Regression] out of memory

paolo dot bonzini at polimi dot it wrote:

> and the operator-precedence was placed on the 
>wiki a week ago as a generic "speedup area", not a project that is 
>actively worked on unlike Matt's lexer overhaul:
>  
>
Honestly, we didn't know we'd be working on until the early part of this 
week.  We didn't decided until Monday around noon in California.  But, 
the fundamental point is that it's in nobody's best interest to 
duplicate work, so we will try to make as much noise as possible about  
projects.

>http://www.dberlin.org/gccwiki/index.php/Speedup%20areas
>
>Since that page also mentions Nathan's "Add contains-repeated-base and 
>is diamond-shaped flags to classes" project, maybe it is *that* page 
>that needs to be louder.
>  
>
Just for the record, I do not know if Nathan is presently working on 
that or not.  I think he may have concluded there is not enough win there.

> > We'll try to be louder.
>
>No problem.  You're doing a good work.
>  
>
Thanks for your understanding -- and you, likewise, are doing good work!

After you get timing numbers for 14179, it would be interesting to 
consider whether or not we should try to extend the operator-precedence 
parser, or do other short-circuiting tricks, when getting down to the 
bottom of binary expressions.  For example, if a unary expression is an 
integer literal or identifier followed by ")" or "," or ";" we know that 
it's just a primary expression.  In other words, we could use two tokens 
of lookahead to zip straight to cp_parser_primary_expression.  Would you 
like to take a look at that as well?



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory

[paolo dot bonzini at polimi dot it]

------- Additional Comments From paolo dot bonzini at polimi dot it  2004-09-22 15:56 -------
Subject: Re:  [3.3/3.4/4.0 Regression] out of memory

> In other words, we could use two tokens 
> of lookahead to zip straight to cp_parser_primary_expression.  Would you 
> like to take a look at that as well?

I had mailed a prototype of the patch to Giovanni as well, hoping that 
he got to that before me.  We'll see after the regtest has concluded, 
though.

Paolo



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory

[giovannibajo at libero dot it]

------- Additional Comments From giovannibajo at libero dot it  2004-09-23 00:01 -------
Mark: process_init_constructor builds new TREE_LISTs for each new initializer. 
This is pretty easy to get rid of, at least for arrays, and will be taken of 
with a patch I will be testing soon. The mainline version will likely extract 
and cleanup array handling into a separate function.

But process_init_constructor also calls digest_init for each and every 
initializer, which makes the initializer goes through the conversion machinery. 
For the example in this PR, we build an IDENTITY_CONV and a couple of 
STANDARD_CONV for each initializer.

For mainline, I could try to modify the conversion engine to not use trees to 
keep track of conversions. I think a specific struct kept in a local Vec would 
be good enough.

For 3.4/3.3, is there a way to avoid calling digest_init if we detect that we 
can just fold_convert (or similar) the initializer? Or maybe put such a speedup 
check within digest_init directly? I am thinking of simple default promotions, 
for which building 3-4 trees and throwing them away doesn't look too smart. I 
am not expert in this kind of type conversions stuff, so I can't devise what a 
correct check for this would be, without making it too specific for the case in 
this PR. Can you suggest me something to get me started?


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory

[mark at codesourcery dot com]

------- Additional Comments From mark at codesourcery dot com  2004-09-23 00:16 -------
Subject: Re:  [3.3/3.4/4.0 Regression] out of memory

giovannibajo at libero dot it wrote:

>------- Additional Comments From giovannibajo at libero dot it  2004-09-23 00:01 -------
>Mark: process_init_constructor builds new TREE_LISTs for each new initializer. 
>This is pretty easy to get rid of, at least for arrays, and will be taken of 
>with a patch I will be testing soon. The mainline version will likely extract 
>and cleanup array handling into a separate function.
>
>But process_init_constructor also calls digest_init for each and every 
>initializer, which makes the initializer goes through the conversion machinery. 
>For the example in this PR, we build an IDENTITY_CONV and a couple of 
>STANDARD_CONV for each initializer.
>  
>
On the mainline, this should be much cheaper because we do not build any 
trees for conversions.  We build "struct conversion" instead, and those 
are allocated on an obstack.  So, you should confirm that this is still 
a bottleneck on the mainline.

>For 3.4/3.3, is there a way to avoid calling digest_init if we detect that we 
>can just fold_convert (or similar) the initializer? Or maybe put such a speedup 
>check within digest_init directly? I am thinking of simple default promotions, 
>for which building 3-4 trees and throwing them away doesn't look too smart. I 
>am not expert in this kind of type conversions stuff, so I can't devise what a 
>correct check for this would be, without making it too specific for the case in 
>this PR. Can you suggest me something to get me started?
>  
>
I think it would be better to try to do this in a way that could be used 
on the mainline too.  If conversions are still a bottleneck,  then we 
could try to optimize.  The most common case is probably that the "from" 
and "to" types are the same.  So, you could try having 
implicit_conversion do "if same_type_p (to, from) && !class_type return 
identity conversion".  (Might even be better just to check pointer 
equality of "to" and "from", so as to avoid the cost of same_type_p if 
they are *not* the same.)  That would short-circuit a lot of the work, 
and might win for other test cases as well, because you save not only on 
digest_init, but with function calls like:

  void f(int);
  void g() { f(3); }



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory

[giovannibajo at libero dot it]

------- Additional Comments From giovannibajo at libero dot it  2004-09-23 01:00 -------
(In reply to comment #27)

> On the mainline, this should be much cheaper because we do not build any 
> trees for conversions.  We build "struct conversion" instead, and those 
> are allocated on an obstack.  So, you should confirm that this is still 
> a bottleneck on the mainline.

Ah right. I did forget that this cleanup was already done. I can confirm this 
is not a bottleneck on the mainline anymore.

BTW, preliminar testing of my patch to process_init_constructor is *very* 
promising: on the mainline, compared to comment #16, we now save an additional 
100MB of RAM. We can compile the quarter of the testcase with 120MB of RAM (and 
GCC 2.95 uses 175MB)!



>>For 3.4/3.3, is there a way to avoid calling digest_init if we detect that we 
>>can just fold_convert (or similar) the initializer? 

> I think it would be better to try to do this in a way that could be used 
> on the mainline too.  If conversions are still a bottleneck,  then we 
> could try to optimize.  

It turned out I was wrong, and we don't need to do this on mainline.

> The most common case is probably that the "from" 
> and "to" types are the same.  So, you could try having 
> implicit_conversion do "if same_type_p (to, from) && !class_type return 
> identity conversion".  (Might even be better just to check pointer 
> equality of "to" and "from", so as to avoid the cost of same_type_p if 
> they are *not* the same.)  That would short-circuit a lot of the work, 
> and might win for other test cases as well, because you save not only on 
> digest_init, but with function calls like:
>   void f(int);
>   void g() { f(3); }

Yes, but the problem is that also default promotions are very common:

void f(char);
void g() { f(3); }

and this is what we need to short-circuit for the testcase to start saving 
memory. I tried something like:

  if (INTEGRAL_TYPE_P (to) && INTEGRAL_TYPE_P (from)
      && same_type_p (type_promotes_to (to), type_promotes_to (from)))
      return ocp_convert (to, expr, CONV_IMPLICIT, flags);

but I'm not sure about those type_promotes_to, plus it segfaults for some 
reason I'm investigating...



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory

[mark at codesourcery dot com]

------- Additional Comments From mark at codesourcery dot com  2004-09-23 01:37 -------
Subject: Re:  [3.3/3.4/4.0 Regression] out of memory

giovannibajo at libero dot it wrote:

>>The most common case is probably that the "from" 
>>and "to" types are the same.  So, you could try having 
>>implicit_conversion do "if same_type_p (to, from) && !class_type return 
>>identity conversion".  (Might even be better just to check pointer 
>>equality of "to" and "from", so as to avoid the cost of same_type_p if 
>>they are *not* the same.)  That would short-circuit a lot of the work, 
>>and might win for other test cases as well, because you save not only on 
>>digest_init, but with function calls like:
>>  void f(int);
>>  void g() { f(3); }
>>    
>>
>
>Yes, but the problem is that also default promotions are very common:
>
>void f(char);
>void g() { f(3); }
>
>and this is what we need to short-circuit for the testcase to start saving 
>memory. I tried something like:
>
>  if (INTEGRAL_TYPE_P (to) && INTEGRAL_TYPE_P (from)
>      && same_type_p (type_promotes_to (to), type_promotes_to (from)))
>      return ocp_convert (to, expr, CONV_IMPLICIT, flags);
>
>but I'm not sure about those type_promotes_to, plus it segfaults for some 
>reason I'm investigating...
>  
>
I don't know about the segfault, but I'd worry that you might not win 
much once the tests get that complex, at least for code other than this 
one test case.  Giant arrays with huge initializers are not the typical 
case, thankfully.



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory

[bonzini at gcc dot gnu dot org]

-- 
Bug 14179 depends on bug 17596, which changed state.

Bug 17596 Summary: [3.4/4.0 Regression] expression parser is too slow, should be rewritten
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=17596

           What    |Old Value                   |New Value
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
             Status|ASSIGNED                    |RESOLVED
         Resolution|                            |FIXED

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory

[bonzini at gcc dot gnu dot org]

------- Additional Comments From bonzini at gcc dot gnu dot org  2004-09-24 16:05 -------
Great news.  Thanks to fixing PR17596, we now outperform 3.3.4 by 25% for a
reduced testcase (with an array of 240000 elements).  Every additional element
costs us a mere 44 instructions in cp_parser_binary_expression according to
cachegrind.

I'm not closing this because Giovanni's patch to lookahead for a comma may still
make some difference, but I'm degrading its priority.

Paolo

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|normal                      |minor
           Priority|P1                          |P2


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory

[mark at codesourcery dot com]

------- Additional Comments From mark at codesourcery dot com  2004-09-24 16:53 -------
Subject: Re:  [3.3/3.4/4.0 Regression] out of memory

bonzini at gcc dot gnu dot org wrote:

>------- Additional Comments From bonzini at gcc dot gnu dot org  2004-09-24 16:05 -------
>Great news.  Thanks to fixing PR17596, we now outperform 3.3.4 by 25% for a
>reduced testcase (with an array of 240000 elements).  Every additional element
>costs us a mere 44 instructions in cp_parser_binary_expression according to
>cachegrind.
>  
>
That is fabulous news!

>I'm not closing this because Giovanni's patch to lookahead for a comma may still
>make some difference, but I'm degrading its priority.
>
Please also remove any regression tags, and remove any release target 
markings.  (This is now an opportunity for improvement, not a regresison.)



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory

[bonzini at gnu dot org]

------- Additional Comments From bonzini at gnu dot org  2004-09-24 18:53 -------
Subject: Re:  [3.3/3.4/4.0 Regression] out of memory

 >>Every additional element costs us a mere 44 instructions
>>in cp_parser_binary_expression according to cachegrind.
> 
> That is fabulous news!

(Of course it does not count instructions elsewhere).

> Please also remove any regression tags, and remove any release target 
> markings.  (This is now an opportunity for improvement, not a regresison.)

Done.



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] array parsing could be made faster

[bonzini at gcc dot gnu dot org]

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|minor                       |enhancement
           Keywords|memory-hog                  |
      Known to fail|3.3.5 3.4.3 4.0.0           |
      Known to work|2.95.3                      |
            Summary|[3.3/3.4/4.0 Regression] out|array parsing could be made
                   |of memory                   |faster
            Version|3.3.3                       |4.0.0


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] array parsing could be made faster

[giovannibajo at libero dot it]

------- Additional Comments From giovannibajo at libero dot it  2004-09-25 00:50 -------
No, sorry, this is wrong. This bug still shows a big memory regression. As I am 
explaining in comment #26 and comment #28, I am working on a patch to 
process_init_constructor to fix it, but we are not there yet.

(when I said "I confirm this is not a bottleneck on mainline anymore" I meant 
that the standard conversion code was not eating too much memory -- the 
testcase still cannot be compiled on my computer because of useless TREE_LISTs 
we build in process_init_constructor).

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|compile-time-hog            |memory-hog
            Summary|array parsing could be made |[3.3/3.4/4.0 Regression]
                   |faster                      |array parsing could be made
                   |                            |faster


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory while parsing array with many initializers

[debora dot j dot estey at lmco dot com]

------- Additional Comments From debora dot j dot estey at lmco dot com  2004-10-26 16:11 -------
Now that this is an enhancement is there any chance of getting it fixed

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory while parsing array with many initializers

[pinskia at gcc dot gnu dot org]

------- Additional Comments From pinskia at gcc dot gnu dot org  2004-10-26 16:14 -------
Yes because this is still a regression.  Note the mainline is already better than what 3.3.3 was in terms 
of memory usage.

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|enhancement                 |minor
            Version|4.0.0                       |3.3.3


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory while parsing array with many initializers

[giovannibajo at libero dot it]

------- Additional Comments From giovannibajo at libero dot it  2004-10-27 15:43 -------
Debora, I'm working on a patch which should definitely fix this bug. I hope to 
be able to finish it before 4.0 gets out and/or 3.3 is definitely closed.

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory while parsing array with many initializers

[steven at gcc dot gnu dot org]

------- Additional Comments From steven at gcc dot gnu dot org  2004-12-23 12:35 -------
Giovanni, any news?

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory while parsing array with many initializers

[giovannibajo at libero dot it]

------- Additional Comments From giovannibajo at libero dot it  2004-12-24 02:23 -------
Subject: Re:  [3.3/3.4/4.0 Regression] out of memory while parsing array with many initializers

steven at gcc dot gnu dot org <gcc-bugzilla@gcc.gnu.org> wrote:

> Giovanni, any news?

I have a patch around for a long time already, but I cannot find the time to do
much GCC work right now (as you may have noticed). I would still like to tackle
this bug unless somebody is in a hurry, so I guess you'll have to wait a little
bit more.

Giovanni Bajo



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0 Regression] out of memory while parsing array with many initializers

[steven at gcc dot gnu dot org]

------- Additional Comments From steven at gcc dot gnu dot org  2004-12-24 07:07 -------
Everyone is in a hurry, this is a regression ;-) 
Can you attach the patch so someone can have a look and maybe 
finish it for you? 
 

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0/4.1 Regression] out of memory while parsing array with many initializers

[mmitchel at gcc dot gnu dot org]

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|4.0.0                       |4.0.1


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.3/3.4/4.0/4.1 Regression] out of memory while parsing array with many initializers

[debora dot j dot estey at lmco dot com]

------- Additional Comments From debora dot j dot estey at lmco dot com  2005-05-03 22:44 -------
any word on this?

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.4/4.0/4.1 Regression] out of memory while parsing array with many initializers

[mmitchel at gcc dot gnu dot org]

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|4.0.1                       |4.0.2


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.4/4.0/4.1 Regression] out of memory while parsing array with many initializers

[pinskia at gcc dot gnu dot org]

------- Additional Comments From pinskia at gcc dot gnu dot org  2005-07-25 01:34 -------
Using the testcase from PR 12245:
cp/parser.c:285 (cp_lexer_new_main)                       0: 0.0%  372302336:88.5%          0: 0.0%  
104391168:79.7%          9
cp/parser.c:270 (cp_lexer_new_main)                       0: 0.0%     364288: 0.1%          0: 0.0%     102144: 
0.1%          1
cp/parser.c:12407 (cp_parser_initializer_list)     22770552:24.8%   23955160: 5.7%          0: 0.0%   
13171408:10.1%         19
cp/decl.c:4182 (reshape_init_array_1)                     0: 0.0%   23955160: 5.7%   22770552:24.6%   
13171408:10.1%         19
ggc-common.c:193 (ggc_calloc)                      16770368:18.3%          0: 0.0%   16805272:18.1%        
612: 0.0%         51
tree.c:828 (build_int_cst_wide)                         480: 0.0%          0: 0.0%   52180672:56.3%          0: 0.0%    
1630661
convert.c:671 (convert_to_integer)                 52187584:56.8%          0: 0.0%          0: 0.0%          0: 0.0%    
1630862


This is worse than the C front-end.

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179

[Bug c++/14179] [3.4/4.0/4.1 Regression] out of memory while parsing array with many initializers

[mmitchel at gcc dot gnu dot org]

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|4.0.2                       |4.0.3


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14179