public inbox for gcc-bugs@sourceware.org
help / color / mirror / Atom feed
* [Bug fortran/25708] Module loading is not good at all
       [not found] <bug-25708-4@http.gcc.gnu.org/bugzilla/>
@ 2010-09-29 15:12 ` burnus at gcc dot gnu.org
  2011-04-03 18:33 ` jvdelisle at gcc dot gnu.org
                   ` (16 subsequent siblings)
  17 siblings, 0 replies; 26+ messages in thread
From: burnus at gcc dot gnu.org @ 2010-09-29 15:12 UTC (permalink / raw)
  To: gcc-bugs

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25708

Tobias Burnus <burnus at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |burnus at gcc dot gnu.org

--- Comment #9 from Tobias Burnus <burnus at gcc dot gnu.org> 2010-09-29 12:48:16 UTC ---
Cf. also thread started at
http://gcc.gnu.org/ml/fortran/2010-09/threads.html#00499


^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug fortran/25708] Module loading is not good at all
       [not found] <bug-25708-4@http.gcc.gnu.org/bugzilla/>
  2010-09-29 15:12 ` [Bug fortran/25708] Module loading is not good at all burnus at gcc dot gnu.org
@ 2011-04-03 18:33 ` jvdelisle at gcc dot gnu.org
  2011-04-04 11:51 ` pault at gcc dot gnu.org
                   ` (15 subsequent siblings)
  17 siblings, 0 replies; 26+ messages in thread
From: jvdelisle at gcc dot gnu.org @ 2011-04-03 18:33 UTC (permalink / raw)
  To: gcc-bugs

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25708

Jerry DeLisle <jvdelisle at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jvdelisle at gcc dot
                   |                            |gnu.org
           Severity|enhancement                 |normal

--- Comment #10 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> 2011-04-03 18:32:40 UTC ---
I would like to promote this one.  I have run into an application that is
taking about 2 to 3 minutes to compile while other compilers can do so in a
matter of seconds.  The delta here is huge.

I would also like to take a crack at it. Paul any progress on your end?


^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug fortran/25708] Module loading is not good at all
       [not found] <bug-25708-4@http.gcc.gnu.org/bugzilla/>
  2010-09-29 15:12 ` [Bug fortran/25708] Module loading is not good at all burnus at gcc dot gnu.org
  2011-04-03 18:33 ` jvdelisle at gcc dot gnu.org
@ 2011-04-04 11:51 ` pault at gcc dot gnu.org
  2011-04-04 12:35 ` burnus at gcc dot gnu.org
                   ` (14 subsequent siblings)
  17 siblings, 0 replies; 26+ messages in thread
From: pault at gcc dot gnu.org @ 2011-04-04 11:51 UTC (permalink / raw)
  To: gcc-bugs

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25708

--- Comment #11 from Paul Thomas <pault at gcc dot gnu.org> 2011-04-04 11:49:52 UTC ---
(In reply to comment #10)
> I would like to promote this one.  I have run into an application that is
> taking about 2 to 3 minutes to compile while other compilers can do so in a
> matter of seconds.  The delta here is huge.
> 
> I would also like to take a crack at it. Paul any progress on your end?

Dear Jerry,

I have to confess that I have not thought about it for quite a while.

It strikes me that it would be useful to profile module reading to find out
what takes the time.  In the case of the original example, it is clear that
repeating the same use statement cannot help; however, that begs that question
why it is so slow for each time.  Is it the IO, the lexing, or.....?

An associated issue is the size of module files.  Clearly, where a module uses
another module, we could help by inserting use statements in the module file.

Cheers

Paul


^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug fortran/25708] Module loading is not good at all
       [not found] <bug-25708-4@http.gcc.gnu.org/bugzilla/>
                   ` (2 preceding siblings ...)
  2011-04-04 11:51 ` pault at gcc dot gnu.org
@ 2011-04-04 12:35 ` burnus at gcc dot gnu.org
  2011-04-05  4:47 ` jvdelisle at gcc dot gnu.org
                   ` (13 subsequent siblings)
  17 siblings, 0 replies; 26+ messages in thread
From: burnus at gcc dot gnu.org @ 2011-04-04 12:35 UTC (permalink / raw)
  To: gcc-bugs

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25708

--- Comment #12 from Tobias Burnus <burnus at gcc dot gnu.org> 2011-04-04 12:27:28 UTC ---
(In reply to comment #11)
> An associated issue is the size of module files.

Joost suggested to cut down the string tags "d" instead of "dimension", "al"
instead of "allocatable" or something like that. That would help with the file
size, the I/O (also: I/O caching), and the string parsing.

> Clearly, where a module uses
> another module, we could help by inserting use statements in the module file.

There are pro and cons doing so. For binary-only code, it is much more
convenient to ship only a single ".mod" instead of several - that was at least
a complaint I have heard about NAG's compiler. Though, having by default a
simple "use" in the .mod file would help.

Another thing which could help is to read a .mod file only once per translation
unit (source file) instead of every time a USE statement is encountered.


^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug fortran/25708] Module loading is not good at all
       [not found] <bug-25708-4@http.gcc.gnu.org/bugzilla/>
                   ` (3 preceding siblings ...)
  2011-04-04 12:35 ` burnus at gcc dot gnu.org
@ 2011-04-05  4:47 ` jvdelisle at gcc dot gnu.org
  2011-04-05  5:25 ` jvdelisle at gcc dot gnu.org
                   ` (12 subsequent siblings)
  17 siblings, 0 replies; 26+ messages in thread
From: jvdelisle at gcc dot gnu.org @ 2011-04-05  4:47 UTC (permalink / raw)
  To: gcc-bugs

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25708

Jerry DeLisle <jvdelisle at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         AssignedTo|pault at gcc dot gnu.org    |jvdelisle at gcc dot
                   |                            |gnu.org

--- Comment #13 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> 2011-04-05 04:47:35 UTC ---
Believe it or not, I have an 8.2 megabyte .mod file.  We are working on
splitting this up into more files.  The module(s) contain interfaces for
existing c libraries which are quite large.

Since I have a good test case, I can use it to profile as well as do some basic
things to reduce the file size. I will take assignment for a while.


^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug fortran/25708] Module loading is not good at all
       [not found] <bug-25708-4@http.gcc.gnu.org/bugzilla/>
                   ` (4 preceding siblings ...)
  2011-04-05  4:47 ` jvdelisle at gcc dot gnu.org
@ 2011-04-05  5:25 ` jvdelisle at gcc dot gnu.org
  2011-04-05  5:35 ` jb at gcc dot gnu.org
                   ` (11 subsequent siblings)
  17 siblings, 0 replies; 26+ messages in thread
From: jvdelisle at gcc dot gnu.org @ 2011-04-05  5:25 UTC (permalink / raw)
  To: gcc-bugs

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25708

--- Comment #14 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> 2011-04-05 05:24:46 UTC ---
Here is an strace summary.  There is something afoot here:

% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 54.41    0.052854           3     17229     17218 stat
 18.41    0.017886           8      2277           write
  7.24    0.007028           2      4651           times
  6.55    0.006364           4      1473           brk
  6.18    0.006000        6000         1           unlink
  5.65    0.005488           8       664           read
  0.98    0.000948         948         1           execve
  0.23    0.000227           3        68        52 open
  0.20    0.000190           4        45           mmap
  0.16    0.000152          19         8           mprotect
  0.00    0.000000           0        16           close
  0.00    0.000000           0        17           fstat
  0.00    0.000000           0         3           lseek
  0.00    0.000000           0        11           munmap
  0.00    0.000000           0         5           rt_sigaction
  0.00    0.000000           0         1         1 access
  0.00    0.000000           0         1           arch_prctl
------ ----------- ----------- --------- --------- ----------------
100.00    0.097137                 26471     17271 total


^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug fortran/25708] Module loading is not good at all
       [not found] <bug-25708-4@http.gcc.gnu.org/bugzilla/>
                   ` (5 preceding siblings ...)
  2011-04-05  5:25 ` jvdelisle at gcc dot gnu.org
@ 2011-04-05  5:35 ` jb at gcc dot gnu.org
  2011-04-10 17:47 ` jvdelisle at gcc dot gnu.org
                   ` (10 subsequent siblings)
  17 siblings, 0 replies; 26+ messages in thread
From: jb at gcc dot gnu.org @ 2011-04-05  5:35 UTC (permalink / raw)
  To: gcc-bugs

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25708

Janne Blomqvist <jb at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         AssignedTo|jvdelisle at gcc dot        |pault at gcc dot gnu.org
                   |gnu.org                     |

--- Comment #15 from Janne Blomqvist <jb at gcc dot gnu.org> 2011-04-05 05:34:57 UTC ---
(In reply to comment #12)
> (In reply to comment #11)
> > An associated issue is the size of module files.
> 
> Joost suggested to cut down the string tags "d" instead of "dimension", "al"
> instead of "allocatable" or something like that. That would help with the file
> size, the I/O (also: I/O caching), and the string parsing.

FWIW, this is http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40958

Another option would be to compress the mod file with zlib; there is a copy in
the gcc tree already.

There also seems to be a bit of redundancy in the module files, e.g. things
like null atoms "()" which I guess can be removed? I'm not sure how big an
effect this would have, though.

> > Clearly, where a module uses
> > another module, we could help by inserting use statements in the module file.
> 
> There are pro and cons doing so. For binary-only code, it is much more
> convenient to ship only a single ".mod" instead of several - that was at least
> a complaint I have heard about NAG's compiler. Though, having by default a
> simple "use" in the .mod file would help.

To reiterate my comments from 

http://gcc.gnu.org/ml/fortran/2010-09/msg00512.html 

the go developers claim that including the transitive dependencies is a major
reason why go compilation is fast. See pages 9-10 on 

http://assets.en.oreilly.com/1/event/45/Another%20Go%20at%20Language%20Design%20Presentation.pdf

In the Fortran case, additionally by doing it this way one can take care of
ONLY and renamed symbols when generating the .mod file.

> Another thing which could help is to read a .mod file only once per translation
> unit (source file) instead of every time a USE statement is encountered.

I think this is the major issue, yes.


^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug fortran/25708] Module loading is not good at all
       [not found] <bug-25708-4@http.gcc.gnu.org/bugzilla/>
                   ` (6 preceding siblings ...)
  2011-04-05  5:35 ` jb at gcc dot gnu.org
@ 2011-04-10 17:47 ` jvdelisle at gcc dot gnu.org
  2011-11-30 19:57 ` Joost.VandeVondele at mat dot ethz.ch
                   ` (9 subsequent siblings)
  17 siblings, 0 replies; 26+ messages in thread
From: jvdelisle at gcc dot gnu.org @ 2011-04-10 17:47 UTC (permalink / raw)
  To: gcc-bugs

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25708

Jerry DeLisle <jvdelisle at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         AssignedTo|pault at gcc dot gnu.org    |jvdelisle at gcc dot
                   |                            |gnu.org

--- Comment #16 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> 2011-04-10 17:47:07 UTC ---
Paul, taking this one if you do not mind.

I have run some tests.  I replaced the long strings in the various minit
invocations with a 2 or 3 character mnemonic. For a particularly large module
the size of the .mod file created is:

un-patched ->         8210691 bytes

gzipped ->             724854 bytes

patched ->            6280606 bytes

patched and zipped ->  714777 bytes

Compressing the file is quite good.  There is no particular advantage to
changing the minit strings if one is going to compress the file.  The question
is then what cost in time do we have of actually compressing and decompressing.

The above just deals with raw file size and I think compression is a good idea.
 If we leave the strings alone, we could allow manual decompressing the files
to look at them for debugging purposes.

The next thing I would like to consider is some sort of module caching. Let's
say we create a module namespace for each module file.  I would suggest
allowing a fixed number of these to conserve memory usage.  Modules that are
USEed repeatedly would be retained, free up ones not used if more are needed. 
I am thinking of some sort of leased recently used scheme.

Another thing I wonder about is how efficiently do we retrieve from this name
space. (I have more to study on the internals of module.c)


^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug fortran/25708] Module loading is not good at all
       [not found] <bug-25708-4@http.gcc.gnu.org/bugzilla/>
                   ` (7 preceding siblings ...)
  2011-04-10 17:47 ` jvdelisle at gcc dot gnu.org
@ 2011-11-30 19:57 ` Joost.VandeVondele at mat dot ethz.ch
  2011-12-01  7:30 ` Joost.VandeVondele at mat dot ethz.ch
                   ` (8 subsequent siblings)
  17 siblings, 0 replies; 26+ messages in thread
From: Joost.VandeVondele at mat dot ethz.ch @ 2011-11-30 19:57 UTC (permalink / raw)
  To: gcc-bugs

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25708

--- Comment #17 from Joost VandeVondele <Joost.VandeVondele at mat dot ethz.ch> 2011-11-30 19:50:37 UTC ---
Janne's lseek patch:
http://gcc.gnu.org/ml/fortran/2011-11/msg00251.html
has further nice results on CP2K (CP2K_2009-05-01.f90)

Thomas (trunk):
 92.08    4.963429           0  19557182           lseek
  5.91    0.318514           1    523064           read
  0.61    0.032888           3     11208           munmap
  0.37    0.020212           2     11969       757 open
  0.24    0.012753           1     11212           close
  0.21    0.011314           1     10533        21 stat
  0.17    0.009117           0     25154           mmap
  0.16    0.008425           0     56715           write
  0.15    0.008353           1     12138           brk
  0.05    0.002811           0     11211           fstat
  0.02    0.001068           2       684           rename
Janne (trunk+patch):
 77.60    1.316715           0   5265206           lseek
  9.12    0.154767           0    466059           read
  4.07    0.069073           0    242658           madvise
  2.77    0.046965           4     11969        74 open
  1.82    0.030845           3     11891           munmap
  1.47    0.024943          36       684           unlink
  0.72    0.012244           1     11895           close
  0.63    0.010689           1     10533        21 stat
  0.56    0.009533           0     56715           write
  0.51    0.008707           0     25837           mmap
  0.40    0.006794           1     12117           brk
  0.15    0.002542           0     11894           fstat


^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug fortran/25708] Module loading is not good at all
       [not found] <bug-25708-4@http.gcc.gnu.org/bugzilla/>
                   ` (8 preceding siblings ...)
  2011-11-30 19:57 ` Joost.VandeVondele at mat dot ethz.ch
@ 2011-12-01  7:30 ` Joost.VandeVondele at mat dot ethz.ch
  2011-12-01 14:13 ` jb at gcc dot gnu.org
                   ` (7 subsequent siblings)
  17 siblings, 0 replies; 26+ messages in thread
From: Joost.VandeVondele at mat dot ethz.ch @ 2011-12-01  7:30 UTC (permalink / raw)
  To: gcc-bugs

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25708

--- Comment #18 from Joost VandeVondele <Joost.VandeVondele at mat dot ethz.ch> 2011-12-01 07:29:25 UTC ---
Janne's latest patch now effectively 'removes' lseek:

 26.84    0.108906           0    242658           madvise
 20.12    0.081608           0    459999           read
 19.27    0.078198           0    512288           lseek
 12.33    0.050038          73       684           unlink
  5.99    0.024315           2     11969        74 open
  4.57    0.018544           2     11891           munmap

(512288 down from >198000000 a few days ago).


^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug fortran/25708] Module loading is not good at all
       [not found] <bug-25708-4@http.gcc.gnu.org/bugzilla/>
                   ` (9 preceding siblings ...)
  2011-12-01  7:30 ` Joost.VandeVondele at mat dot ethz.ch
@ 2011-12-01 14:13 ` jb at gcc dot gnu.org
  2011-12-03 13:29 ` jvdelisle at gcc dot gnu.org
                   ` (6 subsequent siblings)
  17 siblings, 0 replies; 26+ messages in thread
From: jb at gcc dot gnu.org @ 2011-12-01 14:13 UTC (permalink / raw)
  To: gcc-bugs

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25708

--- Comment #19 from Janne Blomqvist <jb at gcc dot gnu.org> 2011-12-01 14:12:45 UTC ---
Author: jb
Date: Thu Dec  1 14:12:37 2011
New Revision: 181879

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=181879
Log:
PR 25708 Avoid seeking when parsing strings and when peeking.

2011-12-01  Janne Blomqvist  <jb@gcc.gnu.org>

    PR fortran/25708
    * module.c (parse_string): Read string into resizable array
    instead of parsing twice and seeking.
    (peek_atom): New implementation avoiding seeks.
    (require_atom): Save and set column and line explicitly for error
    handling.


Modified:
    trunk/gcc/fortran/ChangeLog
    trunk/gcc/fortran/module.c


^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug fortran/25708] Module loading is not good at all
       [not found] <bug-25708-4@http.gcc.gnu.org/bugzilla/>
                   ` (10 preceding siblings ...)
  2011-12-01 14:13 ` jb at gcc dot gnu.org
@ 2011-12-03 13:29 ` jvdelisle at gcc dot gnu.org
  2012-08-24 14:02 ` Joost.VandeVondele at mat dot ethz.ch
                   ` (5 subsequent siblings)
  17 siblings, 0 replies; 26+ messages in thread
From: jvdelisle at gcc dot gnu.org @ 2011-12-03 13:29 UTC (permalink / raw)
  To: gcc-bugs

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25708

Jerry DeLisle <jvdelisle at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         AssignedTo|jvdelisle at gcc dot        |unassigned at gcc dot
                   |gnu.org                     |gnu.org

--- Comment #20 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> 2011-12-03 13:28:30 UTC ---
I don't have time now and others are making good progress.  Unassigning myself.


^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug fortran/25708] Module loading is not good at all
       [not found] <bug-25708-4@http.gcc.gnu.org/bugzilla/>
                   ` (11 preceding siblings ...)
  2011-12-03 13:29 ` jvdelisle at gcc dot gnu.org
@ 2012-08-24 14:02 ` Joost.VandeVondele at mat dot ethz.ch
  2013-03-29  8:33 ` [Bug fortran/25708] [F95] " Joost.VandeVondele at mat dot ethz.ch
                   ` (4 subsequent siblings)
  17 siblings, 0 replies; 26+ messages in thread
From: Joost.VandeVondele at mat dot ethz.ch @ 2012-08-24 14:02 UTC (permalink / raw)
  To: gcc-bugs

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25708

Joost VandeVondele <Joost.VandeVondele at mat dot ethz.ch> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Depends on|                            |40958

--- Comment #21 from Joost VandeVondele <Joost.VandeVondele at mat dot ethz.ch> 2012-08-24 14:00:40 UTC ---
I did another timing experiment on compiling CP2K. I found that on my server,
compiling with -fsyntax-only is as fast as just compiling at -O0. I believe the
reason for this is that module reading is dominating the compile time. In CP2K
each module is included only once per file, so I think it is the efficiency of
reading the module that matters most. My guess would be that the human readable
format of the .mod file is the source of most inefficiency. Is it still
important to the development of gfortran that the .mod file is in this form ?
If I count the number of times a module is used, and multiply that with the
size, I have about 1Gb of .mod files being parsed per CP2K compile (for about
35Mb of Fortran).


^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug fortran/25708] [F95] Module loading is not good at all
       [not found] <bug-25708-4@http.gcc.gnu.org/bugzilla/>
                   ` (12 preceding siblings ...)
  2012-08-24 14:02 ` Joost.VandeVondele at mat dot ethz.ch
@ 2013-03-29  8:33 ` Joost.VandeVondele at mat dot ethz.ch
  2013-06-24  7:00 ` dominiq at lps dot ens.fr
                   ` (3 subsequent siblings)
  17 siblings, 0 replies; 26+ messages in thread
From: Joost.VandeVondele at mat dot ethz.ch @ 2013-03-29  8:33 UTC (permalink / raw)
  To: gcc-bugs


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25708

--- Comment #22 from Joost VandeVondele <Joost.VandeVondele at mat dot ethz.ch> 2013-03-29 08:33:31 UTC ---
Improved in part by

http://gcc.gnu.org/ml/fortran/2013-03/msg00143.html

as r197124 for 4.9.0


^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug fortran/25708] [F95] Module loading is not good at all
       [not found] <bug-25708-4@http.gcc.gnu.org/bugzilla/>
                   ` (13 preceding siblings ...)
  2013-03-29  8:33 ` [Bug fortran/25708] [F95] " Joost.VandeVondele at mat dot ethz.ch
@ 2013-06-24  7:00 ` dominiq at lps dot ens.fr
  2015-08-08 20:23 ` fxcoudert at gcc dot gnu.org
                   ` (2 subsequent siblings)
  17 siblings, 0 replies; 26+ messages in thread
From: dominiq at lps dot ens.fr @ 2013-06-24  7:00 UTC (permalink / raw)
  To: gcc-bugs

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25708

Dominique d'Humieres <dominiq at lps dot ens.fr> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |NEW

--- Comment #23 from Dominique d'Humieres <dominiq at lps dot ens.fr> ---
Was marked as ASSIGNED, but actually "Not yet assigned to anyone". Set to NEW.


^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug fortran/25708] [F95] Module loading is not good at all
       [not found] <bug-25708-4@http.gcc.gnu.org/bugzilla/>
                   ` (14 preceding siblings ...)
  2013-06-24  7:00 ` dominiq at lps dot ens.fr
@ 2015-08-08 20:23 ` fxcoudert at gcc dot gnu.org
  2015-09-08  9:35 ` dominiq at lps dot ens.fr
  2015-10-10  9:11 ` dominiq at lps dot ens.fr
  17 siblings, 0 replies; 26+ messages in thread
From: fxcoudert at gcc dot gnu.org @ 2015-08-08 20:23 UTC (permalink / raw)
  To: gcc-bugs

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=25708
Bug 25708 depends on bug 40958, which changed state.

Bug 40958 Summary: module files too large
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=40958

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED


^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug fortran/25708] [F95] Module loading is not good at all
       [not found] <bug-25708-4@http.gcc.gnu.org/bugzilla/>
                   ` (15 preceding siblings ...)
  2015-08-08 20:23 ` fxcoudert at gcc dot gnu.org
@ 2015-09-08  9:35 ` dominiq at lps dot ens.fr
  2015-10-10  9:11 ` dominiq at lps dot ens.fr
  17 siblings, 0 replies; 26+ messages in thread
From: dominiq at lps dot ens.fr @ 2015-09-08  9:35 UTC (permalink / raw)
  To: gcc-bugs

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=25708

Dominique d'Humieres <dominiq at lps dot ens.fr> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |WAITING

--- Comment #24 from Dominique d'Humieres <dominiq at lps dot ens.fr> ---
I wonder if this PR has not been fixed by recent changes.


^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug fortran/25708] [F95] Module loading is not good at all
       [not found] <bug-25708-4@http.gcc.gnu.org/bugzilla/>
                   ` (16 preceding siblings ...)
  2015-09-08  9:35 ` dominiq at lps dot ens.fr
@ 2015-10-10  9:11 ` dominiq at lps dot ens.fr
  17 siblings, 0 replies; 26+ messages in thread
From: dominiq at lps dot ens.fr @ 2015-10-10  9:11 UTC (permalink / raw)
  To: gcc-bugs

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=25708

Dominique d'Humieres <dominiq at lps dot ens.fr> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |RESOLVED
         Resolution|---                         |FIXED

--- Comment #25 from Dominique d'Humieres <dominiq at lps dot ens.fr> ---
> I wonder if this PR has not been fixed by recent changes.

No feedback, Closing as FIXED.


^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug fortran/25708] Module loading is not good at all
  2006-01-07  5:05 [Bug fortran/25708] New: " pinskia at gcc dot gnu dot org
                   ` (6 preceding siblings ...)
  2007-04-25  8:42 ` jb at gcc dot gnu dot org
@ 2007-05-02  7:41 ` pault at gcc dot gnu dot org
  7 siblings, 0 replies; 26+ messages in thread
From: pault at gcc dot gnu dot org @ 2007-05-02  7:41 UTC (permalink / raw)
  To: gcc-bugs



------- Comment #8 from pault at gcc dot gnu dot org  2007-05-02 08:41 -------
(In reply to comment #7)
> BTW, here's some slides describing NAG:s experience, they use lazy symbol
> lookup combined with caching, and claim it is up to 1000 times faster than
> non-lazy (which gfortran uses AFAICS).
> http://www.fortran.bcs.org/2007/jubilee/f50.pdf
Janne,

I had noted that, when I read Cohen's talk.  Your comment led me to research
lazy and non-lazy symbols and to have a think about how they might apply to
module.c.

What I had a mind to do was to load and decode the .mod file into its own
namespace. Then, use association consists of copying and, if needed, renaming
the symbols into the target namespace.  This requires that the formal and other
secondary namespaces be scanned to see if they are required.

The lazy symbol mechanism would retain the existing pointer_info tree for each
module , resetting the NEEDED and USED flags before use.  The existing loading
mechanism could then be recycled.

It is my belief that building module namespaces will involve the least work and
will be the most efficient way to load symbols but I will need to analyse this
more thoroughly.  In either case, symbols have to be copied.

Watch this space - I am working on it, albeit slowly.

Paul


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25708


^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug fortran/25708] Module loading is not good at all
  2006-01-07  5:05 [Bug fortran/25708] New: " pinskia at gcc dot gnu dot org
                   ` (5 preceding siblings ...)
  2007-01-15 10:28 ` pault at gcc dot gnu dot org
@ 2007-04-25  8:42 ` jb at gcc dot gnu dot org
  2007-05-02  7:41 ` pault at gcc dot gnu dot org
  7 siblings, 0 replies; 26+ messages in thread
From: jb at gcc dot gnu dot org @ 2007-04-25  8:42 UTC (permalink / raw)
  To: gcc-bugs



------- Comment #7 from jb at gcc dot gnu dot org  2007-04-25 09:41 -------
BTW, here's some slides describing NAG:s experience, they use lazy symbol
lookup combined with caching, and claim it is up to 1000 times faster than
non-lazy (which gfortran uses AFAICS).

http://www.fortran.bcs.org/2007/jubilee/f50.pdf


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25708


^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug fortran/25708] Module loading is not good at all
  2006-01-07  5:05 [Bug fortran/25708] New: " pinskia at gcc dot gnu dot org
                   ` (4 preceding siblings ...)
  2006-10-31  8:29 ` paul dot richard dot thomas at cea dot fr
@ 2007-01-15 10:28 ` pault at gcc dot gnu dot org
  2007-04-25  8:42 ` jb at gcc dot gnu dot org
  2007-05-02  7:41 ` pault at gcc dot gnu dot org
  7 siblings, 0 replies; 26+ messages in thread
From: pault at gcc dot gnu dot org @ 2007-01-15 10:28 UTC (permalink / raw)
  To: gcc-bugs



------- Comment #6 from pault at gcc dot gnu dot org  2007-01-15 10:27 -------
This is the one that I said that I would take on for the next few months.  I
will try to implement the manifesto in #5 and have the F2003 specification for
sub-modules with me.

Paul


-- 

pault at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         AssignedTo|unassigned at gcc dot gnu   |pault at gcc dot gnu dot org
                   |dot org                     |
             Status|NEW                         |ASSIGNED
   Last reconfirmed|2006-10-31 07:00:40         |2007-01-15 10:27:52
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25708


^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug fortran/25708] Module loading is not good at all
  2006-01-07  5:05 [Bug fortran/25708] New: " pinskia at gcc dot gnu dot org
                   ` (3 preceding siblings ...)
  2006-10-31  7:00 ` fxcoudert at gcc dot gnu dot org
@ 2006-10-31  8:29 ` paul dot richard dot thomas at cea dot fr
  2007-01-15 10:28 ` pault at gcc dot gnu dot org
                   ` (2 subsequent siblings)
  7 siblings, 0 replies; 26+ messages in thread
From: paul dot richard dot thomas at cea dot fr @ 2006-10-31  8:29 UTC (permalink / raw)
  To: gcc-bugs

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 2575 bytes --]



------- Comment #5 from paul dot richard dot thomas at cea dot fr  2006-10-31 08:29 -------
Subject: RE:  Module loading is not good at all

FX

> -----Message d'origine-----
> De : fxcoudert at gcc dot gnu dot org 
> [mailto:gcc-bugzilla@gcc.gnu.org]
> Envoyé : mardi 31 octobre 2006 08:01
>
> I like that idea. I've looked at the code in symbol.c, and 
> don't see clearly
> how to get it done (where do you get the master symtree from, eg?)
> 

I haven't really thought about this enough yet but.....

(i) I think that a namespace will have to be built for each module.  This will
allow the initial part of read_module to be left completely unmodified. *grin*

(ii) The namespaces should be contained in a linked list or binary tree; the
structures should be something like

typedef struct gfc_module_namespace
{
  const char *name;
  struct gfc_module_namespace *next;
  maybe some attributes?;
}
gfc_module_namespace;

I think that a list should be sufficient, since numbers of modules are likely
to remain limited for real-life codes.... aren't they?  Or is your 25Mb monster
an indication that I am wrong?

(iii) As the modules get USEd, new symtrees are added to the current namespace
and I think that it is OK to point to the symbols in the module namespaces.
This would simplify derived type association a lot.  I have not the foggiest
idea what to do about interfaces; they are on my list of urgent things to try
to understand.  I am about to create a meta-bug of interface PRs. I had a
notion to look at modules after I had had a stab at that.

(There is a PR on doubly USEd interfaces that I had a stab at in the airport
last Friday - I got absolutely nowhere; even though I thought that I was
touching the right places, it had no effect on the fault!  This is one and the
same problem of doubly USEd variables that I fixed by explicitly going through
and checking each against the other.  To do likewise for interfaces requires
some understanding!  I note, however, that these problems are fixed in g95 and
there is no sign whatsoever of the corresponding fixes... It is a crying shame
that Andy Vaught is not on-side.) 

(iv) As at present, the symtree carries the local name and the symbol the true
name - that's why I think that the symbols can be shared.

Regards

Paul

(v) Clearly, a clean-up at the end of the compilation of the file will have to
be done but all the mechanisms to do that exist already.

As I say, the problems are a) interfaces, b) interfaces and c) interfaces.

Ciao

Paul


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25708


^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug fortran/25708] Module loading is not good at all
  2006-01-07  5:05 [Bug fortran/25708] New: " pinskia at gcc dot gnu dot org
                   ` (2 preceding siblings ...)
  2006-10-02 11:24 ` fxcoudert at gcc dot gnu dot org
@ 2006-10-31  7:00 ` fxcoudert at gcc dot gnu dot org
  2006-10-31  8:29 ` paul dot richard dot thomas at cea dot fr
                   ` (3 subsequent siblings)
  7 siblings, 0 replies; 26+ messages in thread
From: fxcoudert at gcc dot gnu dot org @ 2006-10-31  7:00 UTC (permalink / raw)
  To: gcc-bugs



------- Comment #4 from fxcoudert at gcc dot gnu dot org  2006-10-31 07:00 -------
(In reply to comment #2)
> I have proposed to introduce module namespaces that are built just once per
> compiled file per module; either from source or a mod file.  Subsequent usage
> can lift the symbol information from the appropriate namespace.

I like that idea. I've looked at the code in symbol.c, and don't see clearly
how to get it done (where do you get the master symtree from, eg?)


-- 

fxcoudert at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
OtherBugsDependingO|29669                       |
              nThis|                            |
   Last reconfirmed|2006-10-02 11:23:54         |2006-10-31 07:00:40
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25708


^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug fortran/25708] Module loading is not good at all
  2006-01-07  5:05 [Bug fortran/25708] New: " pinskia at gcc dot gnu dot org
  2006-01-07  5:10 ` [Bug fortran/25708] " pinskia at gcc dot gnu dot org
  2006-01-09  7:37 ` paul dot richard dot thomas at cea dot fr
@ 2006-10-02 11:24 ` fxcoudert at gcc dot gnu dot org
  2006-10-31  7:00 ` fxcoudert at gcc dot gnu dot org
                   ` (4 subsequent siblings)
  7 siblings, 0 replies; 26+ messages in thread
From: fxcoudert at gcc dot gnu dot org @ 2006-10-02 11:24 UTC (permalink / raw)
  To: gcc-bugs



------- Comment #3 from fxcoudert at gcc dot gnu dot org  2006-10-02 11:23 -------
Confirmed and marked as an enhancement. After all, it's working :)


-- 

fxcoudert at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|normal                      |enhancement
             Status|UNCONFIRMED                 |NEW
     Ever Confirmed|0                           |1
   Last reconfirmed|0000-00-00 00:00:00         |2006-10-02 11:23:54
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25708


^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug fortran/25708] Module loading is not good at all
  2006-01-07  5:05 [Bug fortran/25708] New: " pinskia at gcc dot gnu dot org
  2006-01-07  5:10 ` [Bug fortran/25708] " pinskia at gcc dot gnu dot org
@ 2006-01-09  7:37 ` paul dot richard dot thomas at cea dot fr
  2006-10-02 11:24 ` fxcoudert at gcc dot gnu dot org
                   ` (5 subsequent siblings)
  7 siblings, 0 replies; 26+ messages in thread
From: paul dot richard dot thomas at cea dot fr @ 2006-01-09  7:37 UTC (permalink / raw)
  To: gcc-bugs



------- Comment #2 from paul dot richard dot thomas at cea dot fr  2006-01-09 07:36 -------
Subject: RE:  Module loading is not good at all

Andrew,

> ------- Comment #1 from pinskia at gcc dot gnu dot org  
> 2006-01-07 05:10 -------
> Looking at the profile for PR 21130 makes me think fixing 
> this bug will also
> fix that one.
> Oh. it is just as bad if we have module modulef declared in 
> the same file as we
> have to save it and then reload it which is just a waste of time. 
>

I have proposed to introduce module namespaces that are built just once per
compiled file per module; either from source or a mod file.  Subsequent usage
can lift the symbol information from the appropriate namespace.

> To Paul:
> Also fixing up the derived types after the fact is just wrong 
> that was PR
> 25391.
> 

What makes you say that?  We have a choice about the way that this should be
done:

(i) Catch equal derived types at the read_module + match stage;
(ii) At resolution; or
(iii) At the translation stage.

I came to the conclusion that (iii) was the most economical, in terms of code,
and the simplest to implement.  There is nothing "wrong" in having separate
symbols for the same object in different scopes, just as long as the tree
declaration is the same.

Paul


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25708



^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Bug fortran/25708] Module loading is not good at all
  2006-01-07  5:05 [Bug fortran/25708] New: " pinskia at gcc dot gnu dot org
@ 2006-01-07  5:10 ` pinskia at gcc dot gnu dot org
  2006-01-09  7:37 ` paul dot richard dot thomas at cea dot fr
                   ` (6 subsequent siblings)
  7 siblings, 0 replies; 26+ messages in thread
From: pinskia at gcc dot gnu dot org @ 2006-01-07  5:10 UTC (permalink / raw)
  To: gcc-bugs



------- Comment #1 from pinskia at gcc dot gnu dot org  2006-01-07 05:10 -------
Looking at the profile for PR 21130 makes me think fixing this bug will also
fix that one.
Oh. it is just as bad if we have module modulef declared in the same file as we
have to save it and then reload it which is just a waste of time. 

To Paul:
Also fixing up the derived types after the fact is just wrong that was PR
25391.


-- 

pinskia at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |paul dot richard dot thomas
                   |                            |at cea dot fr
OtherBugsDependingO|                            |21130, 25391
              nThis|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25708



^ permalink raw reply	[flat|nested] 26+ messages in thread

end of thread, other threads:[~2015-10-10  9:11 UTC | newest]

Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <bug-25708-4@http.gcc.gnu.org/bugzilla/>
2010-09-29 15:12 ` [Bug fortran/25708] Module loading is not good at all burnus at gcc dot gnu.org
2011-04-03 18:33 ` jvdelisle at gcc dot gnu.org
2011-04-04 11:51 ` pault at gcc dot gnu.org
2011-04-04 12:35 ` burnus at gcc dot gnu.org
2011-04-05  4:47 ` jvdelisle at gcc dot gnu.org
2011-04-05  5:25 ` jvdelisle at gcc dot gnu.org
2011-04-05  5:35 ` jb at gcc dot gnu.org
2011-04-10 17:47 ` jvdelisle at gcc dot gnu.org
2011-11-30 19:57 ` Joost.VandeVondele at mat dot ethz.ch
2011-12-01  7:30 ` Joost.VandeVondele at mat dot ethz.ch
2011-12-01 14:13 ` jb at gcc dot gnu.org
2011-12-03 13:29 ` jvdelisle at gcc dot gnu.org
2012-08-24 14:02 ` Joost.VandeVondele at mat dot ethz.ch
2013-03-29  8:33 ` [Bug fortran/25708] [F95] " Joost.VandeVondele at mat dot ethz.ch
2013-06-24  7:00 ` dominiq at lps dot ens.fr
2015-08-08 20:23 ` fxcoudert at gcc dot gnu.org
2015-09-08  9:35 ` dominiq at lps dot ens.fr
2015-10-10  9:11 ` dominiq at lps dot ens.fr
2006-01-07  5:05 [Bug fortran/25708] New: " pinskia at gcc dot gnu dot org
2006-01-07  5:10 ` [Bug fortran/25708] " pinskia at gcc dot gnu dot org
2006-01-09  7:37 ` paul dot richard dot thomas at cea dot fr
2006-10-02 11:24 ` fxcoudert at gcc dot gnu dot org
2006-10-31  7:00 ` fxcoudert at gcc dot gnu dot org
2006-10-31  8:29 ` paul dot richard dot thomas at cea dot fr
2007-01-15 10:28 ` pault at gcc dot gnu dot org
2007-04-25  8:42 ` jb at gcc dot gnu dot org
2007-05-02  7:41 ` pault at gcc dot gnu dot org

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).