public inbox for gcc-bugs@sourceware.org
help / color / mirror / Atom feed
* [Bug c/19513] New: High memory usage when compiling multiple files at a time
@ 2005-01-18 21:56 ch at csh-consult dot dk
2005-01-18 22:02 ` [Bug c/19513] [IMA] " pinskia at gcc dot gnu dot org
` (7 more replies)
0 siblings, 8 replies; 9+ messages in thread
From: ch at csh-consult dot dk @ 2005-01-18 21:56 UTC (permalink / raw)
To: gcc-bugs
Passing 250 or so files from a larger software project (about 3MB of
sourcecode) to gcc at a time makes gcc use more than 400MB of memory. Possible
more as I had to stop the compilation.
I created a much simpler example. 100 equal .c files each containing:
static void mainX() {}
where X varies from 1 to 10000. This gives me a slightly different behaviour.
Compiling only one of the files consumes about 60MB memory. Compiling all of
the 100 files consumes up to 142MB of memory. Then gcc crashes.
If I'm understanding how gcc compiles multiple files at the same time
correctly, then gcc should release most of the memory used for compiling one
file before it begins compiling the next. Is this correct? If so, then I would
expect overall memory usage during compilation of the 100 equal .c files to be
not much more than 60MB.
Reading specs from C:/MinGW/bin/../lib/gcc/mingw32/3.4.2/specs
Configured with: ../gcc/configure --with-gcc --with-gnu-ld --with-gnu-as --host=
mingw32 --target=mingw32 --prefix=/mingw --enable-threads --disable-nls --enable
-languages=c,c++,f77,ada,objc,java --disable-win32-registry --disable-shared --e
nable-sjlj-exceptions --enable-libgcj --disable-java-awt --without-x --enable-ja
va-gc=boehm --disable-libgcj-debug --enable-interpreter --enable-hash-synchroniz
ation --enable-libstdcxx-debug
Thread model: win32
gcc version 3.4.2 (mingw-special)
Let me know if you need more information.
--
Summary: High memory usage when compiling multiple files at a
time
Product: gcc
Version: 3.4.2
Status: UNCONFIRMED
Severity: normal
Priority: P2
Component: c
AssignedTo: unassigned at gcc dot gnu dot org
ReportedBy: ch at csh-consult dot dk
CC: gcc-bugs at gcc dot gnu dot org
GCC build triplet: i386-pc-mingw32
GCC host triplet: i386-pc-mingw32
GCC target triplet: i386-pc-mingw32
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=19513
^ permalink raw reply [flat|nested] 9+ messages in thread
* [Bug c/19513] [IMA] High memory usage when compiling multiple files at a time
2005-01-18 21:56 [Bug c/19513] New: High memory usage when compiling multiple files at a time ch at csh-consult dot dk
@ 2005-01-18 22:02 ` pinskia at gcc dot gnu dot org
2005-01-18 22:32 ` ch at csh-consult dot dk
` (6 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: pinskia at gcc dot gnu dot org @ 2005-01-18 22:02 UTC (permalink / raw)
To: gcc-bugs
------- Additional Comments From pinskia at gcc dot gnu dot org 2005-01-18 22:02 -------
Do you have a program which generates those files?
Also is this at -O0 or -O2?
--
What |Removed |Added
----------------------------------------------------------------------------
Keywords| |memory-hog
Summary|High memory usage when |[IMA] High memory usage when
|compiling multiple files at |compiling multiple files at
|a time |a time
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=19513
^ permalink raw reply [flat|nested] 9+ messages in thread
* [Bug c/19513] [IMA] High memory usage when compiling multiple files at a time
2005-01-18 21:56 [Bug c/19513] New: High memory usage when compiling multiple files at a time ch at csh-consult dot dk
2005-01-18 22:02 ` [Bug c/19513] [IMA] " pinskia at gcc dot gnu dot org
@ 2005-01-18 22:32 ` ch at csh-consult dot dk
2005-01-19 7:38 ` pinskia at gcc dot gnu dot org
` (5 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: ch at csh-consult dot dk @ 2005-01-18 22:32 UTC (permalink / raw)
To: gcc-bugs
------- Additional Comments From ch at csh-consult dot dk 2005-01-18 22:32 -------
Yes I have, but I was lazy and wrote it in C#. I've put them up for download
here: http://212.242.245.122/100files.tar.gz (2.5MB)
There is also the command to invoke gcc (run.bat)
No -O flag is used.
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=19513
^ permalink raw reply [flat|nested] 9+ messages in thread
* [Bug c/19513] [IMA] High memory usage when compiling multiple files at a time
2005-01-18 21:56 [Bug c/19513] New: High memory usage when compiling multiple files at a time ch at csh-consult dot dk
2005-01-18 22:02 ` [Bug c/19513] [IMA] " pinskia at gcc dot gnu dot org
2005-01-18 22:32 ` ch at csh-consult dot dk
@ 2005-01-19 7:38 ` pinskia at gcc dot gnu dot org
2005-01-19 7:58 ` pinskia at gcc dot gnu dot org
` (4 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: pinskia at gcc dot gnu dot org @ 2005-01-19 7:38 UTC (permalink / raw)
To: gcc-bugs
------- Additional Comments From pinskia at gcc dot gnu dot org 2005-01-19 07:38 -------
on the mainline I get an stack overflow in the GC.
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=19513
^ permalink raw reply [flat|nested] 9+ messages in thread
* [Bug c/19513] [IMA] High memory usage when compiling multiple files at a time
2005-01-18 21:56 [Bug c/19513] New: High memory usage when compiling multiple files at a time ch at csh-consult dot dk
` (2 preceding siblings ...)
2005-01-19 7:38 ` pinskia at gcc dot gnu dot org
@ 2005-01-19 7:58 ` pinskia at gcc dot gnu dot org
2005-01-19 7:59 ` pinskia at gcc dot gnu dot org
` (3 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: pinskia at gcc dot gnu dot org @ 2005-01-19 7:58 UTC (permalink / raw)
To: gcc-bugs
------- Additional Comments From pinskia at gcc dot gnu dot org 2005-01-19 07:58 -------
(In reply to comment #3)
> on the mainline I get an stack overflow in the GC.
And I have a work around for that. But that is only a work around, we are creating too many varients of
FUNCTION_TYPE which seems wrong.
We really should be sharing these FUNCTION_TYPEs. This will greatly reduce the memory usage overall
in all compilation.
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=19513
^ permalink raw reply [flat|nested] 9+ messages in thread
* [Bug c/19513] [IMA] High memory usage when compiling multiple files at a time
2005-01-18 21:56 [Bug c/19513] New: High memory usage when compiling multiple files at a time ch at csh-consult dot dk
` (3 preceding siblings ...)
2005-01-19 7:58 ` pinskia at gcc dot gnu dot org
@ 2005-01-19 7:59 ` pinskia at gcc dot gnu dot org
2005-01-19 13:06 ` ch at csh-consult dot dk
` (2 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: pinskia at gcc dot gnu dot org @ 2005-01-19 7:59 UTC (permalink / raw)
To: gcc-bugs
--
What |Removed |Added
----------------------------------------------------------------------------
CC| |pinskia at gcc dot gnu dot
| |org
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=19513
^ permalink raw reply [flat|nested] 9+ messages in thread
* [Bug c/19513] [IMA] High memory usage when compiling multiple files at a time
2005-01-18 21:56 [Bug c/19513] New: High memory usage when compiling multiple files at a time ch at csh-consult dot dk
` (4 preceding siblings ...)
2005-01-19 7:59 ` pinskia at gcc dot gnu dot org
@ 2005-01-19 13:06 ` ch at csh-consult dot dk
2005-01-19 13:57 ` pinskia at gcc dot gnu dot org
2005-01-25 6:17 ` pinskia at gcc dot gnu dot org
7 siblings, 0 replies; 9+ messages in thread
From: ch at csh-consult dot dk @ 2005-01-19 13:06 UTC (permalink / raw)
To: gcc-bugs
------- Additional Comments From ch at csh-consult dot dk 2005-01-19 13:06 -------
Does this mean that there is a limit to how many files you can compile at a
time (due to limited memory)? Can't the garbage collector run between each
compilation?
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=19513
^ permalink raw reply [flat|nested] 9+ messages in thread
* [Bug c/19513] [IMA] High memory usage when compiling multiple files at a time
2005-01-18 21:56 [Bug c/19513] New: High memory usage when compiling multiple files at a time ch at csh-consult dot dk
` (5 preceding siblings ...)
2005-01-19 13:06 ` ch at csh-consult dot dk
@ 2005-01-19 13:57 ` pinskia at gcc dot gnu dot org
2005-01-25 6:17 ` pinskia at gcc dot gnu dot org
7 siblings, 0 replies; 9+ messages in thread
From: pinskia at gcc dot gnu dot org @ 2005-01-19 13:57 UTC (permalink / raw)
To: gcc-bugs
------- Additional Comments From pinskia at gcc dot gnu dot org 2005-01-19 13:57 -------
(In reply to comment #5)
> Does this mean that there is a limit to how many files you can compile at a
> time (due to limited memory)? Can't the garbage collector run between each
> compilation?
Yes for limited by memory but GCC should not be as a hug of memory usage as shown by me looking
into how to solve memory usage.
Yes the garbage collector runs but the problem is we keep around too much still after the compiler like
FUNCTION_TYPEs which seems like most of those should be able to be shared.
I will attach later today the a tar ball with one of the files since that is really all that is needed to
reproduce this bug.
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=19513
^ permalink raw reply [flat|nested] 9+ messages in thread
* [Bug c/19513] [IMA] High memory usage when compiling multiple files at a time
2005-01-18 21:56 [Bug c/19513] New: High memory usage when compiling multiple files at a time ch at csh-consult dot dk
` (6 preceding siblings ...)
2005-01-19 13:57 ` pinskia at gcc dot gnu dot org
@ 2005-01-25 6:17 ` pinskia at gcc dot gnu dot org
7 siblings, 0 replies; 9+ messages in thread
From: pinskia at gcc dot gnu dot org @ 2005-01-25 6:17 UTC (permalink / raw)
To: gcc-bugs
------- Additional Comments From pinskia at gcc dot gnu dot org 2005-01-25 06:17 -------
Confirmed, mainly comment #6.
--
What |Removed |Added
----------------------------------------------------------------------------
Status|UNCONFIRMED |NEW
Ever Confirmed| |1
Last reconfirmed|0000-00-00 00:00:00 |2005-01-25 06:17:06
date| |
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=19513
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2005-01-25 6:17 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2005-01-18 21:56 [Bug c/19513] New: High memory usage when compiling multiple files at a time ch at csh-consult dot dk
2005-01-18 22:02 ` [Bug c/19513] [IMA] " pinskia at gcc dot gnu dot org
2005-01-18 22:32 ` ch at csh-consult dot dk
2005-01-19 7:38 ` pinskia at gcc dot gnu dot org
2005-01-19 7:58 ` pinskia at gcc dot gnu dot org
2005-01-19 7:59 ` pinskia at gcc dot gnu dot org
2005-01-19 13:06 ` ch at csh-consult dot dk
2005-01-19 13:57 ` pinskia at gcc dot gnu dot org
2005-01-25 6:17 ` pinskia at gcc dot gnu dot org
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).