public inbox for gcc-bugs@sourceware.org
help / color / mirror / Atom feed
* [Bug fortran/30285] New: gfortran huge (excessive?) memory usage with large modules
@ 2006-12-24 0:26 anlauf at gmx dot de
2006-12-24 0:28 ` [Bug fortran/30285] " anlauf at gmx dot de
` (24 more replies)
0 siblings, 25 replies; 26+ messages in thread
From: anlauf at gmx dot de @ 2006-12-24 0:26 UTC (permalink / raw)
To: gcc-bugs
Hi,
gfortran seems to use much more memory at compile time
when I "use" larger modules that contain many symbols,
even if I "use, only" selected one. In the described
situation it needs significantly more memory than
"competitors".
The attached archive contains the following main program
and three (3) large modules to demonstrate this.
The variant with the commented lines uncommented
goes beyond 500 MB virtual memory in a Linux/x86 machine.
program main
use mo_psas, only: setup_psas
use mo_3dvar, only: obs, bg
! use mo_t_enkf, only: enkf_driver ! Uncomment to go beyond 500 MB
virt
ual mem.
implicit none
call setup_psas (obs% o, bg% grid% a, bg)
! call enkf_driver
end program main
Compiling this program with:
% gfortran -S -fmem-report -ftime-report
Memory still allocated at the end of the compilation process
Size Allocated Used Overhead
8 4096 2816 96
16 16k 13k 256
64 4096 1088 40
128 4096 768 36
256 4096 2304 32
512 64k 61k 512
2048 36k 36k 288
4096 16k 16k 128
8192 8192 8192 32
32768 416k 416k 416
8388608 8192k 8192k 32
56 4096 280 40
104 20k 17k 180
92 106M 105M 958k
80 4096 80 36
88 73M 72M 661k
24 24k 15k 312
72 4096 288 36
28 9208k 9189k 107k
112 44k 37k 396
36 4096 288 44
12 10164k 10146k 178k
40 33M 33M 364k
Total 240M 238M 2274k
String pool
entries 336041
identifiers 336041 (100.00%)
slots 524288
bytes 4934k (350k overhead)
table size 2048k
coll/search 0.6655
ins/search 0.2790
avg. entry 15.04 bytes (+/- 0.99)
longest entry 34
??? tree nodes created
(No per-node statistics)
Type hash: size 1021, 175 elements, 0.084121 collisions
DECL_DEBUG_EXPR hash: size 1021, 0 elements, 0.000000 collisions
DECL_VALUE_EXPR hash: size 2097143, 865473 elements, 1.209451 collisions
Execution times (seconds)
garbage collection : 1.56 (19%) usr 0.00 ( 0%) sys 1.54 (17%) wall
0 kB ( 0%) ggc
parser : 6.29 (78%) usr 0.78 (99%) sys 7.08 (79%) wall
25
1744 kB (100%) ggc
symout : 0.00 ( 0%) usr 0.00 ( 0%) sys 0.01 ( 0%) wall
0 kB ( 0%) ggc
TOTAL : 8.11 0.79 8.91
25
2445 kB
Extra diagnostic checks enabled; compiler may run slowly.
Configure with --disable-checking to disable checks.
The modules are extracted from a larger project and should
be sufficient to just compile the example. The project
uses MPI (mpich) which is why these symbols occur quite
frequently in the module files. Gfortran seems to handle
the present case quite inefficiently. Maybe there is a
solution that handles used modules more economically...
Cheers,
-ha
--
Summary: gfortran huge (excessive?) memory usage with large
modules
Product: gcc
Version: 4.3.0
Status: UNCONFIRMED
Severity: normal
Priority: P3
Component: fortran
AssignedTo: unassigned at gcc dot gnu dot org
ReportedBy: anlauf at gmx dot de
GCC host triplet: i686-pc-linux-gnu
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30285
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug fortran/30285] gfortran huge (excessive?) memory usage with large modules
2006-12-24 0:26 [Bug fortran/30285] New: gfortran huge (excessive?) memory usage with large modules anlauf at gmx dot de
@ 2006-12-24 0:28 ` anlauf at gmx dot de
2006-12-24 10:47 ` [Bug fortran/30285] gfortran excessive " steven at gcc dot gnu dot org
` (23 subsequent siblings)
24 siblings, 0 replies; 26+ messages in thread
From: anlauf at gmx dot de @ 2006-12-24 0:28 UTC (permalink / raw)
To: gcc-bugs
------- Comment #1 from anlauf at gmx dot de 2006-12-24 00:28 -------
Created an attachment (id=12841)
--> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=12841&action=view)
main.f90 + 3 modules included my main
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30285
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug fortran/30285] gfortran excessive memory usage with large modules
2006-12-24 0:26 [Bug fortran/30285] New: gfortran huge (excessive?) memory usage with large modules anlauf at gmx dot de
2006-12-24 0:28 ` [Bug fortran/30285] " anlauf at gmx dot de
@ 2006-12-24 10:47 ` steven at gcc dot gnu dot org
2007-02-02 1:50 ` bdavis at gcc dot gnu dot org
` (22 subsequent siblings)
24 siblings, 0 replies; 26+ messages in thread
From: steven at gcc dot gnu dot org @ 2006-12-24 10:47 UTC (permalink / raw)
To: gcc-bugs
------- Comment #2 from steven at gcc dot gnu dot org 2006-12-24 10:47 -------
Confirmed. Definitely excessive.
--
steven at gcc dot gnu dot org changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |steven at gcc dot gnu dot
| |org
Status|UNCONFIRMED |NEW
Ever Confirmed|0 |1
Last reconfirmed|0000-00-00 00:00:00 |2006-12-24 10:47:43
date| |
Summary|gfortran huge (excessive?) |gfortran excessive memory
|memory usage with large |usage with large modules
|modules |
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30285
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug fortran/30285] gfortran excessive memory usage with large modules
2006-12-24 0:26 [Bug fortran/30285] New: gfortran huge (excessive?) memory usage with large modules anlauf at gmx dot de
2006-12-24 0:28 ` [Bug fortran/30285] " anlauf at gmx dot de
2006-12-24 10:47 ` [Bug fortran/30285] gfortran excessive " steven at gcc dot gnu dot org
@ 2007-02-02 1:50 ` bdavis at gcc dot gnu dot org
2007-02-02 9:35 ` bdavis at gcc dot gnu dot org
` (21 subsequent siblings)
24 siblings, 0 replies; 26+ messages in thread
From: bdavis at gcc dot gnu dot org @ 2007-02-02 1:50 UTC (permalink / raw)
To: gcc-bugs
------- Comment #3 from bdavis at gcc dot gnu dot org 2007-02-02 01:50 -------
if you try the example, f951 may exit with a segfault. reason is this code
takes a lot of stack space.
for tcsh, "ulimit stacksize unlimited" was required.
(just to save 10 minutes for the next person who takes a look at this !!)
--
bdavis at gcc dot gnu dot org changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |bdavis at gcc dot gnu dot
| |org
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30285
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug fortran/30285] gfortran excessive memory usage with large modules
2006-12-24 0:26 [Bug fortran/30285] New: gfortran huge (excessive?) memory usage with large modules anlauf at gmx dot de
` (2 preceding siblings ...)
2007-02-02 1:50 ` bdavis at gcc dot gnu dot org
@ 2007-02-02 9:35 ` bdavis at gcc dot gnu dot org
2007-02-02 11:10 ` anlauf at gmx dot de
` (20 subsequent siblings)
24 siblings, 0 replies; 26+ messages in thread
From: bdavis at gcc dot gnu dot org @ 2007-02-02 9:35 UTC (permalink / raw)
To: gcc-bugs
------- Comment #4 from bdavis at gcc dot gnu dot org 2007-02-02 09:35 -------
from what i see so far, the problem is in the .mod files, not in the reading of
them. there are hundreds of thousands of 'commons' defined, which is silly.
any way to get source to the attached mod files ?
--bud
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30285
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug fortran/30285] gfortran excessive memory usage with large modules
2006-12-24 0:26 [Bug fortran/30285] New: gfortran huge (excessive?) memory usage with large modules anlauf at gmx dot de
` (3 preceding siblings ...)
2007-02-02 9:35 ` bdavis at gcc dot gnu dot org
@ 2007-02-02 11:10 ` anlauf at gmx dot de
2007-02-02 21:44 ` anlauf at gmx dot de
` (19 subsequent siblings)
24 siblings, 0 replies; 26+ messages in thread
From: anlauf at gmx dot de @ 2007-02-02 11:10 UTC (permalink / raw)
To: gcc-bugs
------- Comment #5 from anlauf at gmx dot de 2007-02-02 11:09 -------
> from what i see so far, the problem is in the .mod files, not in the reading of
> them. there are hundreds of thousands of 'commons' defined, which is silly.
This gave me the idea to the code below. Try compiling:
module mod0
implicit none
INCLUDE 'mpif.h'
private
public :: sub0
contains
subroutine sub0 ()
end subroutine sub0
end module mod0
module mod1
use mod0, only : sub0
implicit none
private
public :: sub1
contains
subroutine sub1 ()
end subroutine sub1
end module mod1
module mod2
use mod0, only : sub0
use mod1, only : sub1
implicit none
private
public :: sub2
contains
subroutine sub2 ()
end subroutine sub2
end module mod2
with mpif.h cut down to:
INTEGER MPI_BOTTOM
COMMON /MPIPRIV/ MPI_BOTTOM
and look at the module files. If you repeat the above,
the module files will explode.
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30285
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug fortran/30285] gfortran excessive memory usage with large modules
2006-12-24 0:26 [Bug fortran/30285] New: gfortran huge (excessive?) memory usage with large modules anlauf at gmx dot de
` (4 preceding siblings ...)
2007-02-02 11:10 ` anlauf at gmx dot de
@ 2007-02-02 21:44 ` anlauf at gmx dot de
2007-02-02 22:37 ` bdavis at gcc dot gnu dot org
` (18 subsequent siblings)
24 siblings, 0 replies; 26+ messages in thread
From: anlauf at gmx dot de @ 2007-02-02 21:44 UTC (permalink / raw)
To: gcc-bugs
------- Comment #6 from anlauf at gmx dot de 2007-02-02 21:44 -------
Created an attachment (id=12998)
--> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=12998&action=view)
Module file size explosion demo
I have rewritten the demo code. See how the size of
the module files explodes with gfortran.
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30285
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug fortran/30285] gfortran excessive memory usage with large modules
2006-12-24 0:26 [Bug fortran/30285] New: gfortran huge (excessive?) memory usage with large modules anlauf at gmx dot de
` (5 preceding siblings ...)
2007-02-02 21:44 ` anlauf at gmx dot de
@ 2007-02-02 22:37 ` bdavis at gcc dot gnu dot org
2007-02-25 4:42 ` bdavis at gcc dot gnu dot org
` (17 subsequent siblings)
24 siblings, 0 replies; 26+ messages in thread
From: bdavis at gcc dot gnu dot org @ 2007-02-02 22:37 UTC (permalink / raw)
To: gcc-bugs
------- Comment #7 from bdavis at gcc dot gnu dot org 2007-02-02 22:36 -------
perfect. that seems to duplicate the problem.
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30285
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug fortran/30285] gfortran excessive memory usage with large modules
2006-12-24 0:26 [Bug fortran/30285] New: gfortran huge (excessive?) memory usage with large modules anlauf at gmx dot de
` (6 preceding siblings ...)
2007-02-02 22:37 ` bdavis at gcc dot gnu dot org
@ 2007-02-25 4:42 ` bdavis at gcc dot gnu dot org
2007-02-26 13:05 ` anlauf at gmx dot de
` (16 subsequent siblings)
24 siblings, 0 replies; 26+ messages in thread
From: bdavis at gcc dot gnu dot org @ 2007-02-25 4:42 UTC (permalink / raw)
To: gcc-bugs
------- Comment #8 from bdavis at gcc dot gnu dot org 2007-02-25 04:42 -------
the below patch looks like it fixes the problem. any chance this could be
tryed on the reported problem ?
--bud
Index: gcc/gcc/fortran/module.c
===================================================================
--- gcc/gcc/fortran/module.c (revision 122310)
+++ gcc/gcc/fortran/module.c (working copy)
@@ -3598,6 +3598,7 @@
write_common (gfc_symtree *st)
{
gfc_common_head *p;
+ pointer_info *pinfo;
const char * name;
int flags;
@@ -3607,6 +3608,14 @@
write_common (st->left);
write_common (st->right);
+ /* only need to write a given symtree entry once for
+ common blocks */
+ p = st->n.common;
+ pinfo = find_pointer(p);
+
+ if (pinfo != NULL && pinfo->u.wsym.state != UNREFERENCED)
+ return;
+
mio_lparen ();
/* Write the unmangled name. */
@@ -3614,7 +3623,6 @@
mio_pool_string (&name);
- p = st->n.common;
mio_symbol_ref (&p->head);
flags = p->saved ? 1 : 0;
if (p->threadprivate) flags |= 2;
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30285
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug fortran/30285] gfortran excessive memory usage with large modules
2006-12-24 0:26 [Bug fortran/30285] New: gfortran huge (excessive?) memory usage with large modules anlauf at gmx dot de
` (7 preceding siblings ...)
2007-02-25 4:42 ` bdavis at gcc dot gnu dot org
@ 2007-02-26 13:05 ` anlauf at gmx dot de
2007-04-13 18:44 ` tobi at gcc dot gnu dot org
` (15 subsequent siblings)
24 siblings, 0 replies; 26+ messages in thread
From: anlauf at gmx dot de @ 2007-02-26 13:05 UTC (permalink / raw)
To: gcc-bugs
------- Comment #9 from anlauf at gmx dot de 2007-02-26 13:05 -------
(In reply to comment #8)
> the below patch looks like it fixes the problem. any chance this could be
> tryed on the reported problem ?
If somebody with sufficient resources can provide a binary
(like FX's snapshots), I will try to recompile the project.
-ha
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30285
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug fortran/30285] gfortran excessive memory usage with large modules
2006-12-24 0:26 [Bug fortran/30285] New: gfortran huge (excessive?) memory usage with large modules anlauf at gmx dot de
` (8 preceding siblings ...)
2007-02-26 13:05 ` anlauf at gmx dot de
@ 2007-04-13 18:44 ` tobi at gcc dot gnu dot org
2007-04-20 2:41 ` bdavis at gcc dot gnu dot org
` (14 subsequent siblings)
24 siblings, 0 replies; 26+ messages in thread
From: tobi at gcc dot gnu dot org @ 2007-04-13 18:44 UTC (permalink / raw)
To: gcc-bugs
------- Comment #10 from tobi at gcc dot gnu dot org 2007-04-13 19:44 -------
Bud, I tried your patch, but it doesn't seem to make a difference. Is it the
right patch?
--
tobi at gcc dot gnu dot org changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |tobi at gcc dot gnu dot org
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30285
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug fortran/30285] gfortran excessive memory usage with large modules
2006-12-24 0:26 [Bug fortran/30285] New: gfortran huge (excessive?) memory usage with large modules anlauf at gmx dot de
` (9 preceding siblings ...)
2007-04-13 18:44 ` tobi at gcc dot gnu dot org
@ 2007-04-20 2:41 ` bdavis at gcc dot gnu dot org
2007-04-20 19:56 ` bdavis at gcc dot gnu dot org
` (13 subsequent siblings)
24 siblings, 0 replies; 26+ messages in thread
From: bdavis at gcc dot gnu dot org @ 2007-04-20 2:41 UTC (permalink / raw)
To: gcc-bugs
------- Comment #11 from bdavis at gcc dot gnu dot org 2007-04-20 03:41 -------
i think not. i must have confued myself (rather easy to do!). will dig
through the source this weekend.
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30285
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug fortran/30285] gfortran excessive memory usage with large modules
2006-12-24 0:26 [Bug fortran/30285] New: gfortran huge (excessive?) memory usage with large modules anlauf at gmx dot de
` (10 preceding siblings ...)
2007-04-20 2:41 ` bdavis at gcc dot gnu dot org
@ 2007-04-20 19:56 ` bdavis at gcc dot gnu dot org
2007-10-22 7:35 ` [Bug fortran/30285] gfortran excessive memory usage with COMMON blocks in modules anlauf at gmx dot de
` (12 subsequent siblings)
24 siblings, 0 replies; 26+ messages in thread
From: bdavis at gcc dot gnu dot org @ 2007-04-20 19:56 UTC (permalink / raw)
To: gcc-bugs
------- Comment #12 from bdavis at gcc dot gnu dot org 2007-04-20 20:56 -------
i can confirm the attached patch is wrong.
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30285
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug fortran/30285] gfortran excessive memory usage with COMMON blocks in modules
2006-12-24 0:26 [Bug fortran/30285] New: gfortran huge (excessive?) memory usage with large modules anlauf at gmx dot de
` (11 preceding siblings ...)
2007-04-20 19:56 ` bdavis at gcc dot gnu dot org
@ 2007-10-22 7:35 ` anlauf at gmx dot de
2007-11-09 12:36 ` fxcoudert at gcc dot gnu dot org
` (11 subsequent siblings)
24 siblings, 0 replies; 26+ messages in thread
From: anlauf at gmx dot de @ 2007-10-22 7:35 UTC (permalink / raw)
To: gcc-bugs
------- Comment #13 from anlauf at gmx dot de 2007-10-22 07:35 -------
I've adjusted the title slightly to refer to the use of
COMMON blocks (in MPICH, OpenMPI) because the problem of
.mod size excursion depends only on the "nesting depth",
not on the actual size of the modules. I hope this bug
will be fixed someday when somebody has a look either at
the module writing routines or at the way the structures
containing module information are held internally.
I am unable to compile a reasonably large project (just in
terms of the number of files, not total source code size!)
using MPI.
The problem appears even worse with OpenMPI than with MPICH.
--
anlauf at gmx dot de changed:
What |Removed |Added
----------------------------------------------------------------------------
Summary|gfortran excessive memory |gfortran excessive memory
|usage with large modules |usage with COMMON blocks in
| |modules
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30285
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug fortran/30285] gfortran excessive memory usage with COMMON blocks in modules
2006-12-24 0:26 [Bug fortran/30285] New: gfortran huge (excessive?) memory usage with large modules anlauf at gmx dot de
` (12 preceding siblings ...)
2007-10-22 7:35 ` [Bug fortran/30285] gfortran excessive memory usage with COMMON blocks in modules anlauf at gmx dot de
@ 2007-11-09 12:36 ` fxcoudert at gcc dot gnu dot org
2007-11-09 12:44 ` Tobias dot Schlueter at physik dot uni-muenchen dot de
` (10 subsequent siblings)
24 siblings, 0 replies; 26+ messages in thread
From: fxcoudert at gcc dot gnu dot org @ 2007-11-09 12:36 UTC (permalink / raw)
To: gcc-bugs
------- Comment #14 from fxcoudert at gcc dot gnu dot org 2007-11-09 12:36 -------
I think the problem starts with:
module mod0
integer mpi_bottom
common /mpipriv/ mpi_bottom
end module mod0
module mod1
use mod0
end module mod1
module mod2
use mod0
use mod1
end module mod2
Because in that case, mod2.mod already has two copies of the common:
(('mpipriv' 2 0 0 'mpipriv') ('mpipriv' 2 0 0 'mpipriv'))
and I don't think that's desirable. I think that the module loading is actually
wrong here: the code in gfc_get_common (match.c) takes special care to
duplicate this common name by creating a unique name for it. While I believe
that mangling is necessary, the mangled name shouldn't be unique but simply
prefixed, so that of the same name are merged, while prevented to clash with
the namespace of the use'ing procedure.
I'm regtesting the following patch:
Index: match.c
===================================================================
--- match.c (revision 129869)
+++ match.c (working copy)
@@ -2608,23 +2608,19 @@ gfc_common_head *
gfc_get_common (const char *name, int from_module)
{
gfc_symtree *st;
- static int serial = 0;
- char mangled_name[GFC_MAX_SYMBOL_LEN + 1];
+ char mangled_name[GFC_MAX_SYMBOL_LEN + 12];
if (from_module)
- {
- /* A use associated common block is only needed to correctly layout
- the variables it contains. */
- snprintf (mangled_name, GFC_MAX_SYMBOL_LEN, "_%d_%s", serial++, name);
- st = gfc_new_symtree (&gfc_current_ns->common_root, mangled_name);
- }
- else
- {
- st = gfc_find_symtree (gfc_current_ns->common_root, name);
-
- if (st == NULL)
- st = gfc_new_symtree (&gfc_current_ns->common_root, name);
- }
+ /* A use associated common block is only needed to correctly layout
+ the variables it contains. */
+ snprintf (mangled_name, GFC_MAX_SYMBOL_LEN, "_frommodule_%s", name);
+
+ st = gfc_find_symtree (gfc_current_ns->common_root,
+ from_module ? mangled_name : name);
+
+ if (st == NULL)
+ st = gfc_new_symtree (&gfc_current_ns->common_root,
+ from_module ? mangled_name : name);
if (st->n.common == NULL)
{
--
fxcoudert at gcc dot gnu dot org changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |fxcoudert at gcc dot gnu dot
| |org
AssignedTo|unassigned at gcc dot gnu |fxcoudert at gcc dot gnu dot
|dot org |org
Status|NEW |ASSIGNED
GCC host triplet|i686-pc-linux-gnu |
Last reconfirmed|2006-12-24 10:47:43 |2007-11-09 12:36:15
date| |
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30285
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug fortran/30285] gfortran excessive memory usage with COMMON blocks in modules
2006-12-24 0:26 [Bug fortran/30285] New: gfortran huge (excessive?) memory usage with large modules anlauf at gmx dot de
` (13 preceding siblings ...)
2007-11-09 12:36 ` fxcoudert at gcc dot gnu dot org
@ 2007-11-09 12:44 ` Tobias dot Schlueter at physik dot uni-muenchen dot de
2007-11-10 0:00 ` fxcoudert at gcc dot gnu dot org
` (9 subsequent siblings)
24 siblings, 0 replies; 26+ messages in thread
From: Tobias dot Schlueter at physik dot uni-muenchen dot de @ 2007-11-09 12:44 UTC (permalink / raw)
To: gcc-bugs
------- Comment #15 from Tobias dot Schlueter at physik dot uni-muenchen dot de 2007-11-09 12:44 -------
Subject: Re: gfortran excessive memory usage with COMMON
blocks in modules
fxcoudert at gcc dot gnu dot org wrote:
> Because in that case, mod2.mod already has two copies of the common:
>
> (('mpipriv' 2 0 0 'mpipriv') ('mpipriv' 2 0 0 'mpipriv'))
>
> and I don't think that's desirable. I think that the module loading is actually
> wrong here: the code in gfc_get_common (match.c) takes special care to
> duplicate this common name by creating a unique name for it. While I believe
> that mangling is necessary, the mangled name shouldn't be unique but simply
> prefixed, so that of the same name are merged, while prevented to clash with
> the namespace of the use'ing procedure.
I wrote this code originally, and I agree with your analysis.
> Index: match.c
> ===================================================================
> --- match.c (revision 129869)
> +++ match.c (working copy)
> @@ -2608,23 +2608,19 @@ gfc_common_head *
> gfc_get_common (const char *name, int from_module)
> {
> gfc_symtree *st;
> - static int serial = 0;
> - char mangled_name[GFC_MAX_SYMBOL_LEN + 1];
> + char mangled_name[GFC_MAX_SYMBOL_LEN + 12];
Should be + 13 (need one char for '\0').
> + /* A use associated common block is only needed to correctly layout
> + the variables it contains. */
> + snprintf (mangled_name, GFC_MAX_SYMBOL_LEN, "_frommodule_%s", name);
GFC_MAX_SYMBOL_LEN + 12, otherwise you could create ambiguities with
really long common names. Previously this wasn't possible due to the
serial number.
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30285
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug fortran/30285] gfortran excessive memory usage with COMMON blocks in modules
2006-12-24 0:26 [Bug fortran/30285] New: gfortran huge (excessive?) memory usage with large modules anlauf at gmx dot de
` (14 preceding siblings ...)
2007-11-09 12:44 ` Tobias dot Schlueter at physik dot uni-muenchen dot de
@ 2007-11-10 0:00 ` fxcoudert at gcc dot gnu dot org
2007-11-10 1:16 ` Tobias dot Schlueter at physik dot uni-muenchen dot de
` (8 subsequent siblings)
24 siblings, 0 replies; 26+ messages in thread
From: fxcoudert at gcc dot gnu dot org @ 2007-11-10 0:00 UTC (permalink / raw)
To: gcc-bugs
------- Comment #16 from fxcoudert at gcc dot gnu dot org 2007-11-09 23:59 -------
(In reply to comment #15)
> I wrote this code originally, and I agree with your analysis.
But regtesting doesn't agree with my analysis... in case of common with
bind(c,name="..."), this patch hampers the diagnosis of commons with the same
name but different labels. I've tried hard to get it rolling that way, because
I agree it's cleaner, but I couldn't. I'll propose a different approach (a
hack, to avoid writing twice the same combination of name and binding label) to
the list when it finishes regtesting.
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30285
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug fortran/30285] gfortran excessive memory usage with COMMON blocks in modules
2006-12-24 0:26 [Bug fortran/30285] New: gfortran huge (excessive?) memory usage with large modules anlauf at gmx dot de
` (15 preceding siblings ...)
2007-11-10 0:00 ` fxcoudert at gcc dot gnu dot org
@ 2007-11-10 1:16 ` Tobias dot Schlueter at physik dot uni-muenchen dot de
2007-11-10 4:01 ` patchapp at dberlin dot org
` (7 subsequent siblings)
24 siblings, 0 replies; 26+ messages in thread
From: Tobias dot Schlueter at physik dot uni-muenchen dot de @ 2007-11-10 1:16 UTC (permalink / raw)
To: gcc-bugs
------- Comment #17 from Tobias dot Schlueter at physik dot uni-muenchen dot de 2007-11-10 01:16 -------
Subject: Re: gfortran excessive memory usage with COMMON
blocks in modules
fxcoudert at gcc dot gnu dot org wrote:
> ------- Comment #16 from fxcoudert at gcc dot gnu dot org 2007-11-09 23:59 -------
> (In reply to comment #15)
>> I wrote this code originally, and I agree with your analysis.
>
> But regtesting doesn't agree with my analysis... in case of common with
> bind(c,name="..."), this patch hampers the diagnosis of commons with the same
> name but different labels. I've tried hard to get it rolling that way, because
> I agree it's cleaner, but I couldn't. I'll propose a different approach (a
> hack, to avoid writing twice the same combination of name and binding label) to
> the list when it finishes regtesting.
There was no BIND(C) when I wrote it :-)
I'll give it a look tomorrow.
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30285
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug fortran/30285] gfortran excessive memory usage with COMMON blocks in modules
2006-12-24 0:26 [Bug fortran/30285] New: gfortran huge (excessive?) memory usage with large modules anlauf at gmx dot de
` (16 preceding siblings ...)
2007-11-10 1:16 ` Tobias dot Schlueter at physik dot uni-muenchen dot de
@ 2007-11-10 4:01 ` patchapp at dberlin dot org
2007-11-10 12:29 ` burnus at gcc dot gnu dot org
` (6 subsequent siblings)
24 siblings, 0 replies; 26+ messages in thread
From: patchapp at dberlin dot org @ 2007-11-10 4:01 UTC (permalink / raw)
To: gcc-bugs
------- Comment #18 from patchapp at dberlin dot org 2007-11-10 04:01 -------
Subject: Bug number PR 30285
A patch for this bug has been added to the patch tracker.
The mailing list url for the patch is
http://gcc.gnu.org/ml/gcc-patches/2007-11/msg00552.html
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30285
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug fortran/30285] gfortran excessive memory usage with COMMON blocks in modules
2006-12-24 0:26 [Bug fortran/30285] New: gfortran huge (excessive?) memory usage with large modules anlauf at gmx dot de
` (17 preceding siblings ...)
2007-11-10 4:01 ` patchapp at dberlin dot org
@ 2007-11-10 12:29 ` burnus at gcc dot gnu dot org
2007-11-10 15:18 ` anlauf at gmx dot de
` (5 subsequent siblings)
24 siblings, 0 replies; 26+ messages in thread
From: burnus at gcc dot gnu dot org @ 2007-11-10 12:29 UTC (permalink / raw)
To: gcc-bugs
------- Comment #19 from burnus at gcc dot gnu dot org 2007-11-10 12:29 -------
Harald, can you test the patch at
http://gcc.gnu.org/ml/gcc-patches/2007-11/msg00552.html
with your real-world program? Due to .mod changes, the "gfc-excessive.tar.gz"
does not work. And for the other tests I have, the compile time does not
significantly change (too much noise to see the effect).
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30285
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug fortran/30285] gfortran excessive memory usage with COMMON blocks in modules
2006-12-24 0:26 [Bug fortran/30285] New: gfortran huge (excessive?) memory usage with large modules anlauf at gmx dot de
` (18 preceding siblings ...)
2007-11-10 12:29 ` burnus at gcc dot gnu dot org
@ 2007-11-10 15:18 ` anlauf at gmx dot de
2007-11-10 18:09 ` fxcoudert at gcc dot gnu dot org
` (4 subsequent siblings)
24 siblings, 0 replies; 26+ messages in thread
From: anlauf at gmx dot de @ 2007-11-10 15:18 UTC (permalink / raw)
To: gcc-bugs
------- Comment #20 from anlauf at gmx dot de 2007-11-10 15:17 -------
Tobias,
> Harald, can you test the patch at
> http://gcc.gnu.org/ml/gcc-patches/2007-11/msg00552.html
> with your real-world program?
I would love to test it with a i586/i686-compatible build. (f951 should
be sufficient that can be used as a replacement in one of those regular
builds provided on FX's web page. Unfortunately I don't have the
resourced to build gfortran myself.)
> And for the other tests I have, the compile time does not
> significantly change (too much noise to see the effect).
The essential problem was actually the virtual memory used by f951
when I last tried to compile the app with MPI enabled. I think the
first "benchmark" is to have the module files of the reworked
example to all have the same size and essentially same contents.
>From comment #14 I presume that this will indeed be the case.
Thanks,
-ha
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30285
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug fortran/30285] gfortran excessive memory usage with COMMON blocks in modules
2006-12-24 0:26 [Bug fortran/30285] New: gfortran huge (excessive?) memory usage with large modules anlauf at gmx dot de
` (19 preceding siblings ...)
2007-11-10 15:18 ` anlauf at gmx dot de
@ 2007-11-10 18:09 ` fxcoudert at gcc dot gnu dot org
2007-11-11 22:18 ` anlauf at gmx dot de
` (3 subsequent siblings)
24 siblings, 0 replies; 26+ messages in thread
From: fxcoudert at gcc dot gnu dot org @ 2007-11-10 18:09 UTC (permalink / raw)
To: gcc-bugs
------- Comment #21 from fxcoudert at gcc dot gnu dot org 2007-11-10 18:09 -------
(In reply to comment #20)
> I would love to test it with a i586/i686-compatible build. (f951 should
> be sufficient that can be used as a replacement in one of those regular
> builds provided on FX's web page. Unfortunately I don't have the
> resourced to build gfortran myself.)
I made a build of the patched compiler that you can download from
http://www.coudert.name/tmp/gfortran-i686-linux-20071110.tar.gz
> I think the
> first "benchmark" is to have the module files of the reworked
> example to all have the same size and essentially same contents.
> From comment #14 I presume that this will indeed be the case.
Yes, it is the case. Please let us know how your real-life testing goes.
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30285
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug fortran/30285] gfortran excessive memory usage with COMMON blocks in modules
2006-12-24 0:26 [Bug fortran/30285] New: gfortran huge (excessive?) memory usage with large modules anlauf at gmx dot de
` (20 preceding siblings ...)
2007-11-10 18:09 ` fxcoudert at gcc dot gnu dot org
@ 2007-11-11 22:18 ` anlauf at gmx dot de
2007-11-12 5:55 ` patchapp at dberlin dot org
` (2 subsequent siblings)
24 siblings, 0 replies; 26+ messages in thread
From: anlauf at gmx dot de @ 2007-11-11 22:18 UTC (permalink / raw)
To: gcc-bugs
------- Comment #22 from anlauf at gmx dot de 2007-11-11 22:18 -------
FX,
> I made a build of the patched compiler that you can download from
> http://www.coudert.name/tmp/gfortran-i686-linux-20071110.tar.gz
I successfully recompiled the app with OpenMPI enabled, which uses
several COMMON blocks. Not only are the module files about a factor
100 smaller for me ;-), the compilation actually completes and the
executable works. I am currently investigating the output, but the
problem described in this PR is solved for me.
Many thanks,
-ha
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30285
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug fortran/30285] gfortran excessive memory usage with COMMON blocks in modules
2006-12-24 0:26 [Bug fortran/30285] New: gfortran huge (excessive?) memory usage with large modules anlauf at gmx dot de
` (21 preceding siblings ...)
2007-11-11 22:18 ` anlauf at gmx dot de
@ 2007-11-12 5:55 ` patchapp at dberlin dot org
2007-11-17 13:47 ` fxcoudert at gcc dot gnu dot org
2007-11-17 13:49 ` fxcoudert at gcc dot gnu dot org
24 siblings, 0 replies; 26+ messages in thread
From: patchapp at dberlin dot org @ 2007-11-12 5:55 UTC (permalink / raw)
To: gcc-bugs
------- Comment #23 from patchapp at dberlin dot org 2007-11-12 05:55 -------
Subject: Bug number PR 30285
A patch for this bug has been added to the patch tracker.
The mailing list url for the patch is
http://gcc.gnu.org/ml/gcc-patches/2007-11/msg00552.html
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30285
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug fortran/30285] gfortran excessive memory usage with COMMON blocks in modules
2006-12-24 0:26 [Bug fortran/30285] New: gfortran huge (excessive?) memory usage with large modules anlauf at gmx dot de
` (22 preceding siblings ...)
2007-11-12 5:55 ` patchapp at dberlin dot org
@ 2007-11-17 13:47 ` fxcoudert at gcc dot gnu dot org
2007-11-17 13:49 ` fxcoudert at gcc dot gnu dot org
24 siblings, 0 replies; 26+ messages in thread
From: fxcoudert at gcc dot gnu dot org @ 2007-11-17 13:47 UTC (permalink / raw)
To: gcc-bugs
------- Comment #24 from fxcoudert at gcc dot gnu dot org 2007-11-17 13:47 -------
Subject: Bug 30285
Author: fxcoudert
Date: Sat Nov 17 13:46:53 2007
New Revision: 130257
URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=130257
Log:
PR fortran/30285
* module.c (struct written_common, written_commons): New structure.
(compare_written_commons, free_written_common, write_common_0):
New functions.
(write_common): Call recursive function write_common_0.
Modified:
trunk/gcc/fortran/ChangeLog
trunk/gcc/fortran/module.c
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30285
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Bug fortran/30285] gfortran excessive memory usage with COMMON blocks in modules
2006-12-24 0:26 [Bug fortran/30285] New: gfortran huge (excessive?) memory usage with large modules anlauf at gmx dot de
` (23 preceding siblings ...)
2007-11-17 13:47 ` fxcoudert at gcc dot gnu dot org
@ 2007-11-17 13:49 ` fxcoudert at gcc dot gnu dot org
24 siblings, 0 replies; 26+ messages in thread
From: fxcoudert at gcc dot gnu dot org @ 2007-11-17 13:49 UTC (permalink / raw)
To: gcc-bugs
------- Comment #25 from fxcoudert at gcc dot gnu dot org 2007-11-17 13:49 -------
Fixed.
--
fxcoudert at gcc dot gnu dot org changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|ASSIGNED |RESOLVED
Resolution| |FIXED
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30285
^ permalink raw reply [flat|nested] 26+ messages in thread
end of thread, other threads:[~2007-11-17 13:49 UTC | newest]
Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2006-12-24 0:26 [Bug fortran/30285] New: gfortran huge (excessive?) memory usage with large modules anlauf at gmx dot de
2006-12-24 0:28 ` [Bug fortran/30285] " anlauf at gmx dot de
2006-12-24 10:47 ` [Bug fortran/30285] gfortran excessive " steven at gcc dot gnu dot org
2007-02-02 1:50 ` bdavis at gcc dot gnu dot org
2007-02-02 9:35 ` bdavis at gcc dot gnu dot org
2007-02-02 11:10 ` anlauf at gmx dot de
2007-02-02 21:44 ` anlauf at gmx dot de
2007-02-02 22:37 ` bdavis at gcc dot gnu dot org
2007-02-25 4:42 ` bdavis at gcc dot gnu dot org
2007-02-26 13:05 ` anlauf at gmx dot de
2007-04-13 18:44 ` tobi at gcc dot gnu dot org
2007-04-20 2:41 ` bdavis at gcc dot gnu dot org
2007-04-20 19:56 ` bdavis at gcc dot gnu dot org
2007-10-22 7:35 ` [Bug fortran/30285] gfortran excessive memory usage with COMMON blocks in modules anlauf at gmx dot de
2007-11-09 12:36 ` fxcoudert at gcc dot gnu dot org
2007-11-09 12:44 ` Tobias dot Schlueter at physik dot uni-muenchen dot de
2007-11-10 0:00 ` fxcoudert at gcc dot gnu dot org
2007-11-10 1:16 ` Tobias dot Schlueter at physik dot uni-muenchen dot de
2007-11-10 4:01 ` patchapp at dberlin dot org
2007-11-10 12:29 ` burnus at gcc dot gnu dot org
2007-11-10 15:18 ` anlauf at gmx dot de
2007-11-10 18:09 ` fxcoudert at gcc dot gnu dot org
2007-11-11 22:18 ` anlauf at gmx dot de
2007-11-12 5:55 ` patchapp at dberlin dot org
2007-11-17 13:47 ` fxcoudert at gcc dot gnu dot org
2007-11-17 13:49 ` fxcoudert at gcc dot gnu dot org
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).