[Bug fortran/38199] New: missed optimization, regression: I/O performance

[Bug fortran/38199] New: missed optimization, regression: I/O performance

[manfred99 at gmx dot ch]

With

!234567
      character buffer*10000000
      integer i,j

      DO j=1,20
        write(buffer,'(i2)') j
        write(*,*) buffer(1:2)
        read(buffer,*) i
        write(*,*) i
      ENDDO
      end

I get the following timings:
ifort11 (64bit):               0.306s
ifort9 (32bit):                0.562s
g77 (32bit):                   2.786s
gfortran4.3 (64bit):           3.906s
gfortran4.4 (20081120, 64bit): 4.832s


Even worse:
!234567
      character buffer*100000
      integer i,j

      DO j=1,9999
        write(buffer,'(i4)') j
        write(*,*) buffer(1:4)
        read(buffer,*) i
        write(*,*) i
      ENDDO
      end

ifort11 (64bit):                0.458s
g77 (32bit):                   13.283s
gfortran4.3 (64bit):           19.362s
gfortran4.4 (20081120, 64bit): 23.917s


This is a very realistic real-world scenario when reading
in a flat-file of unknown width (<100000 assumed), and
then processing the received string buffer, i.e. doing
    100 read(10,'(a)',END=999) buffer
        ---> 
        read(buffer,*,END=101,ERR=101) array
    101 do something when unexpected content
        GOTO 100
    999 end


-- 
           Summary: missed optimization, regression: I/O performance
           Product: gcc
           Version: 4.4.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: fortran
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: manfred99 at gmx dot ch


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

[Bug fortran/38199] missed optimization, regression: I/O performance

[jb at gcc dot gnu dot org]

------- Comment #1 from jb at gcc dot gnu dot org  2008-11-20 14:04 -------
I'm looking at I/O performance as part of PR 25561 (see also PR 37754, perhaps
this is a dup?), but my changes are invasive enough that they are 4.5 material.
Thanks for the report.


-- 

jb at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jb at gcc dot gnu dot org
             Status|UNCONFIRMED                 |NEW
     Ever Confirmed|0                           |1
   Last reconfirmed|0000-00-00 00:00:00         |2008-11-20 14:04:55
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

[Bug fortran/38199] missed optimization, regression: I/O performance

[manfred99 at gmx dot ch]

------- Comment #2 from manfred99 at gmx dot ch  2008-11-20 14:59 -------
The profiling of the second testcase gives
  %   cumulative   self              self     total
 time   seconds   seconds    calls  Ts/call  Ts/call  name
 44.20      8.34     8.34                             next_char
 24.85     13.03     4.69                             mem_read
 17.04     16.25     3.22                             memcpy
  4.98     17.19     0.94                             eat_spaces
  4.03     17.95     0.76                            
_gfortrani_empty_internal_buffer
  2.44     18.41     0.46                             nml_query
  2.09     18.80     0.40                             strncasecmp_l
  0.21     18.84     0.04                             memset
  0.05     18.85     0.01                             __divti3
  0.05     18.86     0.01                             _gfortrani_write_block
  0.05     18.87     0.01                             next_char
  0.00     18.87     0.00        1     0.00     0.00  MAIN__


It seems gfortran reads character by character and does not take
any opportunities to shortcut the read process.

For this case, one could imagine that the read routine would scan for the
last non-blank character and would stop reading at this position.

Perhaps ifort is doing exactly this.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

[Bug fortran/38199] [4.4 regression] I/O performance

[burnus at gcc dot gnu dot org]

------- Comment #3 from burnus at gcc dot gnu dot org  2008-11-20 15:56 -------
Jerry, regarding the suggestion in comment 2: Do you see that we can do there
some optimization, esp. when reading a number or with "*" a string (for '(a)'
one presumably cannot do any optimization and has to read past all blanks).  


-- 

burnus at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jvdelisle at gcc dot gnu dot
                   |                            |org
           Keywords|                            |missed-optimization
           Priority|P3                          |P4
            Summary|missed optimization,        |[4.4 regression] I/O
                   |regression: I/O performance |performance


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

[Bug fortran/38199] [4.4 regression] I/O performance

[manfred99 at gmx dot ch]

------- Comment #4 from manfred99 at gmx dot ch  2008-11-20 16:09 -------
Consider
!234567
      program internalread3
      implicit none
      character value*100000
      integer i,j

      DO j=1,9999
        write(value,'(i4)') j
        write(*,*) value(1:4)
        read(value(1:LEN_TRIM(value)),*) i
        write(*,*) i
      ENDDO
      end program internalread3

gfortran4.4 (20081120, 64bit): 1.079s

i.e. speedup by factor 23 ...
but there are cases where the user can't solve the issue like this.
And such basic optimizations are more efficiently done
by the compiler.

Besides this: This is an internal read, so the compiler has full
control over the contents of this string variable. It can e.g.
null terminate the string or similar, or it can carry on an index 
of the last character in string (set on assignment), or ... 


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

[Bug fortran/38199] [4.4 regression] I/O performance

[jvdelisle at gcc dot gnu dot org]

------- Comment #5 from jvdelisle at gcc dot gnu dot org  2008-11-21 04:14 -------
Regarding comment #2.  This is exactly the area I have been investigating, but
I don't have anything solid yet.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

[Bug fortran/38199] missed optimization: I/O performance

[pault at gcc dot gnu dot org]

------- Comment #6 from pault at gcc dot gnu dot org  2008-11-21 13:54 -------
I do not believe that it is a regression, so I have removed that from the
summary.

The profiling that you have done tells a story - I think that it is fairly
clear where the problem lies; not in making a spurious copy as I first thought
but rather in looking for an end of record, or some such foolishness.

I had a brief look in the library to try to understand why so much time is
spent in next_char, mem_read and memcpy and cannot immediately see what is
happening.  Jerry knows the terrain better than me.

Cheers

Paul


-- 

pault at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|[4.4 regression] I/O        |missed optimization: I/O
                   |performance                 |performance


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

[Bug fortran/38199] missed optimization: I/O performance

[jvdelisle at gcc dot gnu dot org]

------- Comment #7 from jvdelisle at gcc dot gnu dot org  2008-11-22 05:34 -------
I think this is then a dup of 37754.  Janne is working some ideas and these are
similar to my thoughts.  This fix here is in a high bug domain so we think we
should hold for 4.5, get it resolved and tested, then possibly backport to 4.4

Marking as duplicate.

*** This bug has been marked as a duplicate of 37754 ***


-- 

jvdelisle at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |DUPLICATE


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

[Bug fortran/38199] missed optimization: I/O performance

[jvdelisle at gcc dot gnu dot org]

------- Comment #8 from jvdelisle at gcc dot gnu dot org  2010-02-13 06:39 -------
I am reopening this bug.  I stumbled upon it searching testcases from Manfred, 
Running the test case here with 4.5 has not substantially improved.  Time to
put on my thinking cap.


-- 

jvdelisle at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
         Resolution|DUPLICATE                   |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

[Bug fortran/38199] missed optimization: I/O performance

[jvdelisle at gcc dot gnu dot org]

------- Comment #9 from jvdelisle at gcc dot gnu dot org  2010-02-13 06:40 -------
Assigning to myself.


-- 

jvdelisle at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         AssignedTo|unassigned at gcc dot gnu   |jvdelisle at gcc dot gnu dot
                   |dot org                     |org
             Status|REOPENED                    |ASSIGNED
   Last reconfirmed|2008-11-20 14:04:55         |2010-02-13 06:40:14
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

[Bug fortran/38199] missed optimization: I/O performance

[jvdelisle at gcc dot gnu dot org]

------- Comment #10 from jvdelisle at gcc dot gnu dot org  2010-02-21 15:59 -------
An update.  I have a patch developing.  Conceptually, it requires handling of
separators in list_read.c to be moved to the beginning of each invocation of
list_formatted_read_scalar.  This avoids trying to eat_spaces or separators
after a value is read.  This change is actually the right thing to do since it
takes code scattered in several places and consolidates it to one place.

The result is execution time on the original testcase is reduced to less then
.2 seconds. I don't expect significant performance gains in general, though the
code should be more maintainable

The patch is a bit intrusive, especially in namelist areas, so may be best for
4.6 at this stage. (assuming I get it to work completely at all :) )


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199