[Bug libfortran/31501] New: libgfortran I/O performance issues

[Bug libfortran/31501] New: libgfortran I/O performance issues

[jvdelisle at gcc dot gnu dot org]

Keeping in mind that correct is better than fast.  I would like to start
looking at refactoring the I/O library to eliminate some of the overhead. 
Beginning with internal units where there is no need for alloc buffering like
we do now,

Flat profile:

Each sample counts as 0.01 seconds.
  %   cumulative   self              self     total           
 time   seconds   seconds    calls  ms/call  ms/call  name    
 18.02      1.84     1.84                             next_char
 10.87      2.95     1.11                             _gfortrani_read_sf
  7.84      3.75     0.80                             _gfortrani_is_array_io
  5.83      4.35     0.60                             memcpy
  5.83      4.94     0.60                             push_char
  5.68      5.52     0.58                             read_real
  5.58      6.09     0.57                             ____strtod_l_internal
  5.19      6.62     0.53                             fd_alloc_r_at
  4.60      7.09     0.47                             mem_alloc_r_at
  3.92      7.49     0.40                             eat_spaces
  3.77      7.88     0.39                             _gfortrani_is_stream_io
  2.89      8.17     0.30                            
list_formatted_read_scalar
  2.55      8.43     0.26                            
_gfortrani_is_internal_unit
  2.25      8.66     0.23                             __read_nocancel
  1.13      8.78     0.12                             _gfortran_transfer_array
  1.08      8.89     0.11                             _gfortrani_convert_real
  1.08      9.00     0.11                             nml_bad_return
  0.88      9.09     0.09                             eat_line
  0.88      9.18     0.09                             str_to_mpn
  0.83      9.26     0.09                            
_gfortrani_list_formatted_read
  0.78      9.34     0.08                             __lseek_nocancel


-- 
           Summary: libgfortran I/O performance issues
           Product: gcc
           Version: 4.3.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: libfortran
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: jvdelisle at gcc dot gnu dot org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31501

[Bug libfortran/31501] libgfortran I/O performance issues

[jvdelisle at gcc dot gnu dot org]

------- Comment #1 from jvdelisle at gcc dot gnu dot org  2007-04-07 01:45 -------
Created an attachment (id=13336)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=13336&action=view)
Test case used to get a profile

This is a reference test case we can use to measure progress.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31501

[Bug libfortran/31501] libgfortran I/O performance issues

[jvdelisle at gcc dot gnu dot org]

-- 

jvdelisle at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         AssignedTo|unassigned at gcc dot gnu   |jvdelisle at gcc dot gnu dot
                   |dot org                     |org
             Status|UNCONFIRMED                 |ASSIGNED
     Ever Confirmed|0                           |1
   Last reconfirmed|0000-00-00 00:00:00         |2007-04-07 18:28:02
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31501

[Bug libfortran/31501] libgfortran I/O performance issues

[fxcoudert at gcc dot gnu dot org]

------- Comment #2 from fxcoudert at gcc dot gnu dot org  2007-04-09 11:07 -------
See PR20278. Do we really want to separate PR for these?


-- 

fxcoudert at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  BugsThisDependsOn|                            |20278


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31501

[Bug libfortran/31501] libgfortran internal unit I/O performance issues

[jvdelisle at gcc dot gnu dot org]

------- Comment #3 from jvdelisle at gcc dot gnu dot org  2007-04-10 00:19 -------
I am not sure this is formatted I/O related.  I will investigate further, but I
suspect we are allocating buffer memory to write to memory and we should not
have to do that.


-- 

jvdelisle at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|libgfortran I/O performance |libgfortran internal unit
                   |issues                      |I/O performance issues


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31501

[Bug libfortran/31501] libgfortran internal unit I/O performance issues

[jvdelisle at gcc dot gnu dot org]

------- Comment #4 from jvdelisle at gcc dot gnu dot org  2007-04-18 06:12 -------
Looking at sf_read in transfer.c, we can see that we are reading one character
at a time with internal units.  This was done for external units because we can
not anticipate where the end of the file is until we hit it.  This does not
make sense with internal units because we know the record length at run time.

I should be able to improve this quite a bit by doing one larger read.  I also
may be able to do a straight memcpy which will further improve this.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31501

[Bug libfortran/31501] libgfortran internal unit I/O performance issues

[jvdelisle at gcc dot gnu dot org]

------- Comment #5 from jvdelisle at gcc dot gnu dot org  2007-04-22 20:24 -------
Created an attachment (id=13425)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=13425&action=view)
Patch to improve read-sf

This patch knocks read_sf off the profile.  Thats a start.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31501

[Bug libfortran/31501] libgfortran internal unit I/O performance issues

[jvdelisle at gcc dot gnu dot org]

------- Comment #6 from jvdelisle at gcc dot gnu dot org  2007-04-22 21:18 -------
Created an attachment (id=13426)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=13426&action=view)
Modified patch for further improvement

This modified patch, gets the time for the test case on my system from about 12
seconds down to about 7.6.

before$ time ./a.out

real    0m11.162s
user    0m10.845s
sys     0m0.306s

after$ time ./a.out
real    0m7.680s
user    0m7.666s
sys     0m0.002s

We probably want to make is_array_io a macro so we can retain code readability.
 The big culprit here is now push_char in list_read.c


-- 

jvdelisle at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #13425|0                           |1
        is obsolete|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31501

[Bug libfortran/31501] libgfortran internal unit I/O performance issues

[jvdelisle at gcc dot gnu dot org]

------- Comment #7 from jvdelisle at gcc dot gnu dot org  2007-04-22 22:09 -------
Created an attachment (id=13427)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=13427&action=view)
Refinement on the previous, using macros

With this patch I replaced is_array_io, is_internal_unit, and is_stream_io with
macros.  Now on my system I get this down to 7 seconds.

$ time ./a.out

real    0m6.954s
user    0m6.951s
sys     0m0.001s


-- 

jvdelisle at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #13426|0                           |1
        is obsolete|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31501

[Bug libfortran/31501] libgfortran internal unit I/O performance issues

[patchapp at dberlin dot org]

------- Comment #8 from patchapp at dberlin dot org  2007-04-22 23:40 -------
Subject: Bug number PR31501

A patch for this bug has been added to the patch tracker.
The mailing list url for the patch is
http://gcc.gnu.org/ml/gcc-patches/2007-04/msg01439.html


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31501

[Bug libfortran/31501] libgfortran internal unit I/O performance issues

[jvdelisle at gcc dot gnu dot org]

------- Comment #9 from jvdelisle at gcc dot gnu dot org  2007-04-29 00:23 -------
Subject: Bug 31501

Author: jvdelisle
Date: Sun Apr 29 00:23:35 2007
New Revision: 124266

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=124266
Log:
2007-04-28  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

        PR libfortran/31501
        * io/list_read.c (next_char): Fix whitespace.
        * io/io.h: Remove prototypes and define macros for is_array_io,
        is_stream_io, and is_internal_unit.
        * io/unit.c (is_array_io), (is_internal_unit), (is_stream_io): Delete
        these functions.
        * io/transfer.c (read_sf): Change handling of internal_unit to make a
        single call to salloc_r and use memcpy to transfer the data. 

Modified:
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/io/io.h
    trunk/libgfortran/io/list_read.c
    trunk/libgfortran/io/transfer.c
    trunk/libgfortran/io/unit.c


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31501

[Bug libfortran/31501] libgfortran internal unit I/O performance issues

[jvdelisle at gcc dot gnu dot org]

------- Comment #10 from jvdelisle at gcc dot gnu dot org  2007-06-22 20:20 -------
Keeping track of these here.


-- 

jvdelisle at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  BugsThisDependsOn|20278                       |32382


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31501

[Bug libfortran/31501] libgfortran internal unit I/O performance issues

[jvdelisle at gcc dot gnu dot org]

-- 

jvdelisle at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|normal                      |enhancement


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31501

[Bug libfortran/31501] libgfortran internal unit I/O performance issues

[fxcoudert at gcc dot gnu dot org]

------- Comment #11 from fxcoudert at gcc dot gnu dot org  2007-10-26 22:43 -------
Probably too generic to be useful, and we're not doing so bad. Closing (with
Jerry's approval).


-- 

fxcoudert at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|                            |FIXED


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31501