[Bug fortran/38199] missed optimization: I/O performance

[Bug fortran/38199] missed optimization: I/O performance

[tkoenig at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

--- Comment #11 from Thomas Koenig <tkoenig at gcc dot gnu.org> 2010-10-23 10:02:46 UTC ---
Hi Jerry,

> An update.  I have a patch developing. 

> The patch is a bit intrusive, especially in namelist areas, so may be best for
> 4.6 at this stage. (assuming I get it to work completely at all :) )

Still in time for 4.6? :-)

[Bug fortran/38199] missed optimization: I/O performance

[jvdelisle at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

--- Comment #12 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> 2010-10-23 13:03:08 UTC ---
I got sidetracked by other more serious bugs. I am not even sure I have the
draft patch any more, but I will look for it. Not too sure about 4.6

[Bug fortran/38199] missed optimization: I/O performance

[jb at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

--- Comment #13 from Janne Blomqvist <jb at gcc dot gnu.org> 2011-05-10 09:41:08 UTC ---
Here's something for formatted writes; consider the write-many.f (from some
other PR, I'm too lazy to check which now)

program main
  open(10,status='SCRATCH')
  a = 0.3858204
  do i=1,1000000
     a = a + 0.4761748164
     write(10, '(G12.5)'),a
  end do
end program main

Profiling this with 'perf' shows the top offenders as

# Overhead         Command                                                     
Shared Object  Symbol
# ........  .............. 
.................................................................  ......
#
    21.56%      write-many  /lib/libc-2.11.1.so                                
               [.] __mpn_divrem
    14.72%      write-many  /lib/libc-2.11.1.so                                
               [.] ___printf_fp
    13.42%      write-many  /lib/libc-2.11.1.so                                
               [.] hack_digit.15661
     7.75%      write-many  /lib/libc-2.11.1.so                                
               [.] __GI_vfprintf
     3.81%      write-many 
/home/janne/src/gfortran/trunk/install/lib64/libgfortran.so.3.0.0  [.]
output_float.isra.7.constprop.16
     2.81%      write-many 
/home/janne/src/gfortran/trunk/install/lib64/libgfortran.so.3.0.0  [.]
write_float
     2.38%      write-many  /lib/libc-2.11.1.so                                
               [.] _IO_default_xsputn_internal
     2.10%      write-many 
/home/janne/src/gfortran/trunk/install/lib64/libgfortran.so.3.0.0  [.]
data_transfer_init
     1.96%      write-many 
/home/janne/src/gfortran/trunk/install/lib64/libgfortran.so.3.0.0  [.]
formatted_transfer
     1.37%      write-many 
/home/janne/src/gfortran/trunk/install/lib64/libgfortran.so.3.0.0  [.]
next_format0


That is, most of the time seems to be spent somewhere related to the libc
formatting (as we're using snprintf to convert the real numbers to ascii).
Next, consider

#include <stdio.h>
#include <stdlib.h>

int main(int argc, char *argv[])
{
        int ndigits = atoi(argv[1]);
        printf("Doing test with %d digits\n", ndigits);
        size_t bufsz = ndigits + 9;
        char *buf = malloc(bufsz);
        for (int i = 0; i < 10000000; i++)
                snprintf(buf, bufsz, "%#-.*e", ndigits, 1./3);
        printf("%s\n", buf);
        return 0;
}

$ time ./snprintfbench 0
Doing test with 0 digits
3.e-01

real    0m2.608s
user    0m2.610s
sys     0m0.000s

$ time ./snprintfbench 20
Doing test with 20 digits
3.33333333333333314830e-01

real    0m4.746s
user    0m4.740s
sys     0m0.010s

$ time ./snprintfbench 40
Doing test with 40 digits
3.3333333333333331482961625624739099293947e-01

real    0m6.362s
user    0m6.360s
sys     0m0.000s

$ time ./snprintfbench 60
Doing test with 60 digits
3.333333333333333148296162562473909929394721984863281250000000e-01

real    0m8.155s
user    0m8.160s
sys     0m0.000s

That is, while there is a constant cost for snprintf(), each additional digit
increases the time approximately linearly. 

Now, in io/write_float.def we always convert with a constant 41 digits (when
REAL(16) is available). Instead, we could first figure out how many digits we
need, and only then call snprintf(), generating only as many digits as needed.
Or as many as requested + 1, if the user has chosen a non-default rounding
mode, that is we need an extra digit in order to do the rounding in that case.

[Bug fortran/38199] missed optimization: I/O performance

[steven at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

Steven Bosscher <steven at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |steven at gcc dot gnu.org

--- Comment #14 from Steven Bosscher <steven at gcc dot gnu.org> 2012-03-11 22:41:25 UTC ---
Jerry, are you still working on this?

[Bug libfortran/38199] missed optimization: I/O performance

[jb at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

--- Comment #15 from Janne Blomqvist <jb at gcc dot gnu.org> 2012-03-15 15:14:51 UTC ---
Author: jb
Date: Thu Mar 15 15:14:43 2012
New Revision: 185433

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=185433
Log:
2012-03-15  Janne Blomqvist  <jb@gcc.gnu.org>

        PR libfortran/52434
        PR libfortran/48878
        PR libfortran/38199
        * io/unit.c (get_internal_unit): Default to ROUND_UNSPECIFIED.
        (init_units): Likewise.
        * io/write_float.def (determine_precision): New function.
        (output_float): Take into account buffer with %f format, no need
        for our own rounding if unspecified or processor specified
        rounding.
        (DTOA): Simplify format string, add parameters.
        (FDTOA): New macros similar to DTOA, but using %f format.
        (OUTPUT_FLOAT_FMT_G): Stack allocate newf, determine correct
        precision and fill buffer.
        (EN_PREC): New macro.
        (determine_en_precision): New function.
        (WRITE_FLOAT): For G format, move buffer filling into
        output_float_FMT_G, use FDTOA for F format.
        (write_float): Increase buffer due to F format.

testsuite ChangeLog:

2012-03-15  Janne Blomqvist  <jb@gcc.gnu.org>

        PR libfortran/52434
        PR libfortran/48878
        PR libfortran/38199
        * gfortran.dg/edit_real_1.f90: Don't assume roundTiesToAway.
        * gfortran.dg/round_1.f03: Likewise.


Modified:
    trunk/gcc/testsuite/ChangeLog
    trunk/gcc/testsuite/gfortran.dg/edit_real_1.f90
    trunk/gcc/testsuite/gfortran.dg/round_1.f03
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/io/unit.c
    trunk/libgfortran/io/write_float.def

[Bug libfortran/38199] missed optimization: I/O performance

[manfred99 at gmx dot ch]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

--- Comment #16 from Manfred Schwarb <manfred99 at gmx dot ch> 2012-03-19 09:42:29 UTC ---
Thanks, Janne, for your patch. It does not help very much, though.
[ As expected, as the reading part is the bottleneck ]

My current timings of the second test case:
g77: 12.41s
4.3: 14.60s
4.5: 18.55s
4.7: 17.21s
4.8: 16.80s (inkl. Janne's patch)
len_trim variant, 4.3: 0.92s
len_trim variant, 4.8: 0.24s

That's still a factor 70 (sic!) away from the optimized variant.
And we are still slower than GCC 4.3 and g77.

The len_trim variant improved a lot, probably due
to Janne's patch.


Now, the interesting thing is:
I tried to figure out at which point the len_trim variant
is slower due to the len_trim overhead.
On my box, this point does not exist, i.e. the len_trim
variant is always faster!

I.e. even for
      character buffer*10

the len_trim variant is faster!

Which suggests that a brute-force approach to unconditionally 
issue a "len_trim" in the relevant gfortran reading routine 
would always be a win!

[Bug libfortran/38199] missed optimization: I/O performance

[tkoenig at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

--- Comment #17 from Thomas Koenig <tkoenig at gcc dot gnu.org> 2012-04-15 11:52:49 UTC ---
Author: tkoenig
Date: Sun Apr 15 11:52:44 2012
New Revision: 186466

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=186466
Log:
2012-04-15  Thomas Koenig  <tkoenig@gcc.gnu.org>

    PR libfortran/38199
    PR libfortran/50673
    * intrinsics/string_intriniscs_inc.c (string_len_trim):
    Remove prototypes for string_len_trim and move to...
    * libgfortran.h (string_len_trim): ... here and
    (string_len_trim_char4): ...here.
    * io/unit.c: For non-array internal arrays where we do reading,
    adjust the record length to the last non-blank character.
    * io/unix.c:  Fix typo.


Modified:
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/intrinsics/string_intrinsics_inc.c
    trunk/libgfortran/io/unit.c
    trunk/libgfortran/io/unix.c
    trunk/libgfortran/libgfortran.h

[Bug libfortran/38199] missed optimization: I/O performance

[tkoenig at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

--- Comment #18 from Thomas Koenig <tkoenig at gcc dot gnu.org> 2012-04-15 11:55:25 UTC ---
Test case from comment #16 has been fixed.

Is there anything else to do?

[Bug libfortran/38199] missed optimization: I/O performance

[jvdelisle at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

--- Comment #19 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> 2012-04-15 14:17:43 UTC ---
I think this can be closed.  Thanks Thomas! The last patch is simple enough to
consider a backport.

[Bug libfortran/38199] missed optimization: I/O performance

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

--- Comment #20 from Tobias Burnus <burnus at gcc dot gnu.org> 2012-04-17 19:30:39 UTC ---
Author: burnus
Date: Tue Apr 17 19:30:29 2012
New Revision: 186548

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=186548
Log:
2012-04-17  Tobias Burnus  <burnus@net-b.de>

        PR libfortran/38199
        PR libfortran/50673
        * io/unit.c (get_internal_unit): Properly check for the presence
        of the format string.


Modified:
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/io/unit.c

[Bug libfortran/38199] missed optimization: I/O performance

[manfred99 at gmx dot ch]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

--- Comment #21 from Manfred Schwarb <manfred99 at gmx dot ch> 2012-04-18 09:01:46 UTC ---
This new version does fix it, timings are around 0.2s for the
above test-case (exactly as fast as the user-optimized len_trim
variant). Thanks a lot!

However, I don't see why this fix is restricted to list-directed
reads. I.e. consider a slight variation:

!234567
      character buffer*100000
      integer i,j

      DO j=1,9999
        write(buffer,'(i4)') j
        write(*,*) buffer(1:4)
!!        read(buffer(1:len_trim(buffer)),'(i100000)') i
        read(buffer,'(i100000)') i
        write(*,*) i
      ENDDO
      end

Here, I get 4.5s for the above variant and 0.2s for the len_trim variant.
But perhaps this is a different issue (separate bug report needed)?

[Bug libfortran/38199] missed optimization: I/O performance

[manfred99 at gmx dot ch]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

--- Comment #22 from Manfred Schwarb <manfred99 at gmx dot ch> 2013-02-15 10:20:46 UTC ---
Last month I had a private communication with Jerry, whether this bug
can be closed. I decided to add a summary to the bugzilla page:

The fix of Thomas is restricted to scalars and list-io. One could simply
replace buffer by buffer(1) in my test case 2 in comment #1 (using list-io):
!234567
      character buffer(1)*100000
      integer i,j

      DO j=1,9999
        write(buffer(1),'(i4)') j
        write(*,*) buffer(1)(1:4)
        read(buffer,*) i
        write(*,*) i
      ENDDO
      end

And we are again dog slow.
For formatted IO (comment #21), this patch does not help either.

Timings of the above test case:
gfortran4.3:     10.3s
gfortran4.7:     13.5s
gfortran4.8:     15.8s

as comparison, test case 2 of comment #1:
gfortran4.8:     0.26s


Timings for testcase in comment #21:
gfortran4.3:    2.7s
gfortran4.6:    2.7s
gfortran4.7:    4.7s  <---
gfortran4.8:    4.0s

[Bug libfortran/38199] [4.7/4.8 Regression] missed optimization: I/O performance

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

Tobias Burnus <burnus at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |burnus at gcc dot gnu.org
   Target Milestone|---                         |4.7.3
            Summary|missed optimization: I/O    |[4.7/4.8 Regression] missed
                   |performance                 |optimization: I/O
                   |                            |performance

--- Comment #23 from Tobias Burnus <burnus at gcc dot gnu.org> 2013-02-15 11:28:19 UTC ---
(In reply to comment #22)
> Timings for testcase in comment #21:
> gfortran4.6:    2.7s
> gfortran4.7:    4.7s  <---
> gfortran4.8:    4.0s

Based on those timings, it is a regression - hence, I marked it as such.

[Bug libfortran/38199] [4.7/4.8 Regression] missed optimization: I/O performance

[tkoenig at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

--- Comment #24 from Thomas Koenig <tkoenig at gcc dot gnu.org> 2013-02-17 11:25:15 UTC ---

> Based on those timings, it is a regression - hence, I marked it as such.

Non-trivial to fix. To implement array I/O, we would have to call
len_trim on every record at the right place, then update it.

I am tempted to put something original_recl into gfc_unit and set
recl to the length of the current record for internal array I/O
in next_record_r.

I'll play arond a bit with this idea, but it is probably more 4.9
material.

[Bug libfortran/38199] [4.7/4.8/4.9 Regression] missed optimization: I/O performance

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|4.7.3                       |4.7.4

--- Comment #25 from Richard Biener <rguenth at gcc dot gnu.org> 2013-04-11 07:58:45 UTC ---
GCC 4.7.3 is being released, adjusting target milestone.

[Bug libfortran/38199] [4.7/4.8/4.9 Regression] missed optimization: I/O performance

[jvdelisle at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

--- Comment #26 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
Test case from Comment #22

ifort:

real    0m0.608s
user    0m0.464s
sys    0m0.080s


gfortran:

real    0m22.893s
user    0m21.059s
sys    0m0.152s

Obviously we could do better.

[Bug libfortran/38199] [4.7/4.8/4.9 Regression] missed optimization: I/O performance

[jvdelisle at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

--- Comment #27 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
Getting my self up to speed here.  I reviewed what we have going on with this. 
Currently we call a function called eat_spaces to do what is needed. 
Naturally, to keep track of all the flags and counters, eat_spaces just calls
next_char.

next_char handles all the details, but is not optimized to just skip over
spaces.

If we wanted to break our paradigm, we could go straight to memory and just
scan the string and skip the spaces.  Another option is to use the sread call
which goes straight to mem_read in unix.c

Another option would be to not even store the spaces and use a NULL marker and
then pad spaces on writes when we see it and stop reads.  This would be most
efficient, but is a very intrusive and could lead to problems when accessing
the middle of strings, etc, etc, etc.

I will be attempting to use sread in a better way for eat_spaces and see how it
turns out.

[Bug libfortran/38199] [4.7/4.8/4.9 Regression] missed optimization: I/O performance

[jvdelisle at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

--- Comment #28 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
Patch submitted for the case in Comment #22.

$ gfc48 22.f                   gfortran version 4.8
$ time ./a.out 

real    0m26.320s
user    0m26.184s
sys    0m0.018s

$ gfc 22.f                     gfortran patched trunk
$ time ./a.out 

real    0m2.917s
user    0m2.902s
sys    0m0.001s

[Bug libfortran/38199] [4.7/4.8/4.9 Regression] missed optimization: I/O performance

[manfred99 at gmx dot ch]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

--- Comment #29 from Manfred Schwarb <manfred99 at gmx dot ch> ---
The regression flag was re-added by Tobias in comment 23 due to
a regression in the testcase of comment 21:

!234567
      character buffer*100000
      integer i,j

      DO j=1,9999
        write(buffer,'(i4)') j
        write(*,*) buffer(1:4)
        read(buffer,'(i100000)') i
        write(*,*) i
      ENDDO
      end

Here are the profiles of gfortran 4.6:
Each sample counts as 0.01 seconds.
  %   cumulative   self              self     total
 time   seconds   seconds    calls  Ts/call  Ts/call  name
 97.45      3.06     3.06                             _gfortrani_read_decimal
  1.59      3.11     0.05                             memset
  0.64      3.13     0.02                             __write_nocancel
  0.32      3.14     0.01                             format_lex
  0.00      3.14     0.00        1     0.00     0.00  main


and gfortran trunk:
Each sample counts as 0.01 seconds.
  %   cumulative   self              self     total
 time   seconds   seconds    calls  Ts/call  Ts/call  name
 59.96      2.71     2.71                             read_utf8
 38.72      4.46     1.75                             _gfortrani_read_decimal
  0.66      4.49     0.03                             memset
  0.22      4.50     0.01                             fflush
  0.22      4.51     0.01                             formatted_transfer
  0.22      4.52     0.01                             malloc_consolidate
  0.00      4.52     0.00        1     0.00     0.00  main


so the difference is obvious, the utf8 treatment makes the difference.
Is there a possibility to speed up reading the trivial cases?
99.99% of fortran input will be ASCII, I guess...

And again, the magical approach is to add LEN_TRIM to avoid reading
100000 space characters and interpreting them as utf8.

So if something like Thomas' patch could be applied also for formatted
reads, this would speed up things dramatically as well.

[Bug libfortran/38199] [4.7/4.8/4.9 Regression] missed optimization: I/O performance

[jvdelisle at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

--- Comment #30 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
The read:

read(buffer,'(i100000)') i

is an entirely different code path then the example in Comment #22 because of
the format specifier.  This is handled very differently from a list directed
read.  In read_block_form we take the users intent of i100000 very literally. 
I also see how we can do a trim on it.  I will work on this as a separate
patch, keeping in mind this test case is well contrived.  :)

[Bug libfortran/38199] [4.7/4.8/4.9 Regression] missed optimization: I/O performance

[jvdelisle at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

--- Comment #31 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
Author: jvdelisle
Date: Sun Mar  9 03:17:16 2014
New Revision: 208438

URL: http://gcc.gnu.org/viewcvs?rev=208438&root=gcc&view=rev
Log:
2014-03-08  Jerry DeLisle  <jvdelisle@gcc.gnu>

    PR libfortran/38199
    * io/list_read.c (next_char): Delete unuseful error checks.
    (eat_spaces): For character array reading, skip ahead over
    spaces rather than call next_char multiple times.

Modified:
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/io/list_read.c

[Bug libfortran/38199] [4.7/4.8/4.9 Regression] missed optimization: I/O performance

[jvdelisle at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

--- Comment #32 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
Created attachment 32312
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=32312&action=edit
New patch for case in comment 21

This is just a preliminary runtime patch.

Without:

real    0m7.519s
user    0m7.496s
sys    0m0.000s

With patch:

real    0m1.433s
user    0m1.424s
sys    0m0.002s

Which is not a bad improvement.  I am still looking at what we could do on the
front-end.

[Bug libfortran/38199] [4.7/4.8/4.9 Regression] missed optimization: I/O performance

[jvdelisle at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

--- Comment #33 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
Author: jvdelisle
Date: Sun Mar  9 05:34:34 2014
New Revision: 208439

URL: http://gcc.gnu.org/viewcvs?rev=208439&root=gcc&view=rev
Log:
2014-03-08  Jerry DeLisle  <jvdelisle@gcc.gnu>

    PR libfortran/38199
    * io/list_read.c (next_char): Mark unlikely error checks.
    (eat_spaces): For character array reading, skip ahead over
    spaces rather than call next_char multiple times.

Modified:
    trunk/libgfortran/ChangeLog

[Bug libfortran/38199] [4.7/4.8/4.9 Regression] missed optimization: I/O performance

[jvdelisle at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

--- Comment #34 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
http://gcc.gnu.org/ml/fortran/2014-03/msg00079.html

[Bug libfortran/38199] [4.7/4.8/4.9 Regression] missed optimization: I/O performance

[dominiq at lps dot ens.fr]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

--- Comment #35 from Dominique d'Humieres <dominiq at lps dot ens.fr> ---
> http://gcc.gnu.org/ml/fortran/2014-03/msg00079.html

The test

      character buffer*100000
      integer i,j

      DO j=1,9999
        write(buffer,'(i4)') j
        read(buffer,'(i100000)') i
      ENDDO
      end

takes ~3.5s without the patch (and 4.7.4 or 4.8.3) and ~0.7s with it (~2s with
4.6). The test

      character buffer(1)*100000
      integer i,j

      DO j=1,9999
        write(buffer(1),'(i4)') j
        read(buffer,*) i
      ENDDO
      end

takes ~1.8s with/without the patch (~11.3s with 4.6, ~10.2s with 4.7.4, and
~13.6 with 4.8.3, i7 2.8Ghz, turbo 3.8Ghz).

[Bug libfortran/38199] [4.7/4.8/4.9 Regression] missed optimization: I/O performance

[dominiq at lps dot ens.fr]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

--- Comment #36 from Dominique d'Humieres <dominiq at lps dot ens.fr> ---
When running the NIST tests from a script, I see the following failure with the
patch:

FM908 fails to run
At line 727 of file /Users/dominiq/Documents/Fortran/NISTtest/NIST/FM908.f
Fortran runtime error: Bad value on logical read

However the test succeeds when run manually.

[Bug libfortran/38199] [4.7/4.8/4.9 Regression] missed optimization: I/O performance

[dominiq at lps dot ens.fr]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

--- Comment #38 from Dominique d'Humieres <dominiq at lps dot ens.fr> ---
> Dominiq, can you try FM908 with something like Valgrind and also try different 
> optimization levels and see if anything odd pops up.

I don't have access to valgrind on this machine, but if I compile with
-fsanitize=address, I get

...
     7     FAIL  
                 COMPUTED=  0.3400000000D-03
                 CORRECT=   0.3438457000D+03
     8     FAIL  
                 COMPUTED=  0.0000000000D+00
                 CORRECT=   0.3438457000D+03
     9     FAIL  
                 COMPUTED=  0.0000000000D+00
                 CORRECT=   0.3438457000D+03
...
    27     FAIL  
                 COMPUTED= (    34.84000,      0.00000)
                 CORRECT=  (    34.84000,    349.88699)
...
    43     PASS  
1
  TEST   PASS/FAIL       DISPLAYED RESULTS       REMARKS

-------------------------------------------------------------------------------
At line 727 of file /Users/dominiq/Documents/Fortran/NISTtest/NIST/FM908.f
Fortran runtime error: Bad value on logical read

[Bug libfortran/38199] [4.7/4.8/4.9 Regression] missed optimization: I/O performance

[jvdelisle at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

--- Comment #39 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
OK, I can reproduce this here.  The problem is the read is trying to go past
where we adjusted the string length due to things like 1X and / in the format.

Good catch, thanks

[Bug libfortran/38199] [4.7/4.8/4.9 Regression] missed optimization: I/O performance

[jvdelisle at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

--- Comment #40 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
Created attachment 32335
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=32335&action=edit
Updated patch taking care of NIST failures and cleaned up

This patch regression tests and passes NIST on x86-64.  I have restructured it
and took care of leaping on uninitialized values.

Dominiq. Can you give it a spin? Thanks

[Bug libfortran/38199] [4.7/4.8/4.9 Regression] missed optimization: I/O performance

[dominiq at lps dot ens.fr]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

--- Comment #41 from Dominique d'Humieres <dominiq at lps dot ens.fr> ---
> Created attachment 32335 [details]
> Updated patch taking care of NIST failures and cleaned up
>
> This patch regression tests and passes NIST on x86-64.  I have restructured it 
> and took care of leaping on uninitialized values.
>
> Dominiq. Can you give it a spin? Thanks

Confirmed. Now the two test in comment 35 execute in ~0.1s. Note that the test

      character buffer(1)*100000
      integer i,j

      buffer = ''
      buffer(1)(6:100000) = repeat('a',99995)

      DO j=1,9999
        write(buffer(1)(1:4),'(i4)') j
        read(buffer,*) i
      ENDDO
      print *, "'",buffer(1)(1:10), '...', buffer(1)(99990:100000), "'"
      end

obviously does not benefit of the speed-up due to the trimming:

[Book15] f90/bug% time a.out
 '9999 aaaaa...aaaaaaaaaaa'
11.380u 0.007s 0:11.38 100.0%    0+0k 0+1io 0pf+0w

Without the patch, the run time is slightly shorter:

[Book15] f90/bug% time a.out
 '9999 aaaaa...aaaaaaaaaaa'
10.617u 0.001s 0:10.61 100.0%    0+0k 0+0io 0pf+0w

One possibility would be to "go to end" as soon as the value(s) is(are) read.
This would also fix pr58324 and pr59727.

[Bug libfortran/38199] [4.7/4.8/4.9 Regression] missed optimization: I/O performance

[jvdelisle at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

--- Comment #42 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
Author: jvdelisle
Date: Thu Mar 13 05:06:57 2014
New Revision: 208528

URL: http://gcc.gnu.org/viewcvs?rev=208528&root=gcc&view=rev
Log:
2014-03-12  Jerry DeLisle  <jvdelisle@gcc.gnu>

    PR libfortran/38199
    * io/read.c (read_decimal): Quickly skip spaces to avoid calls
    to next_char.
    * io/unit.c (is_trim_ok): New helper function to check various
    conditions to see if its OK to trim the internal unit string.
    (get_internal_unit): Use LEN_TRIM to shorten selected internal
    unit strings for optimizing READ. Enable this optimization for
    formatted READ.
    * io/list_read.c (finish_list_read): Don't call eat_line for
    internal units.

Modified:
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/io/list_read.c
    trunk/libgfortran/io/read.c
    trunk/libgfortran/io/unit.c

[Bug libfortran/38199] [4.7/4.8/4.9 Regression] missed optimization: I/O performance

[jvdelisle at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

--- Comment #43 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
Author: jvdelisle
Date: Sat Mar 15 20:31:33 2014
New Revision: 208595

URL: http://gcc.gnu.org/viewcvs?rev=208595&root=gcc&view=rev
Log:
2014-03-15  Jerry DeLisle  <jvdelisle@gcc.gnu>

    Backport from mainline
    PR libfortran/58324
    PR libfortran/38199
    * io/list_read.c (finish_list_read): Read one character to check
    for the end of the file.  If it is the end, then issue the file
    end error message.  If not, use eat_line to reach the end
    without giving error.  The next attempt to read will then
    issue the error as described above.
    * io/read.c (read_decimal): Quickly skip spaces to avoid calls
    to next_char.
    * io/unit.c (is_trim_ok): New helper function to check various
    conditions to see if its OK to trim the internal unit string.
    (get_internal_unit): Use LEN_TRIM to shorten selected internal
    unit strings for optimizing READ. Enable this optimization for
    formatted READ.

Modified:
    branches/gcc-4_8-branch/libgfortran/ChangeLog
    branches/gcc-4_8-branch/libgfortran/io/list_read.c
    branches/gcc-4_8-branch/libgfortran/io/read.c
    branches/gcc-4_8-branch/libgfortran/io/unit.c

[Bug libfortran/38199] [4.7/4.8/4.9 Regression] missed optimization: I/O performance

[jvdelisle at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

--- Comment #44 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
Author: jvdelisle
Date: Sat Mar 15 23:06:44 2014
New Revision: 208599

URL: http://gcc.gnu.org/viewcvs?rev=208599&root=gcc&view=rev
Log:
2014-03-15  Jerry DeLisle  <jvdelisle@gcc.gnu>

    Backport from mainline
    PR libfortran/58324
    PR libfortran/38199
    * intrinsics/string_intriniscs_inc.c (string_len_trim):
    Remove prototypes for string_len_trim and move to...
    * libgfortran.h (string_len_trim): ... here and
    (string_len_trim_char4): ...here.
    * io/list_read.c (finish_list_read): Read one character to check
    for the end of the file.  If it is the end, then issue the file
    end error message.  If not, use eat_line to reach the end
    without giving error.  The next attempt to read will then
    issue the error as described above.
    * io/read.c (read_decimal): Quickly skip spaces to avoid calls
    to next_char.
    * io/unit.c (is_trim_ok): New helper function to check various
    conditions to see if its OK to trim the internal unit string.
    (get_internal_unit): Use LEN_TRIM to shorten selected internal
    unit strings for optimizing READ. Enable this optimization for
    formatted READ.

    Backport from mainline
    PR libfortran/58324
    * gfortran.dg/list_read_12.f90: New test.

Added:
    branches/gcc-4_7-branch/gcc/testsuite/gfortran.dg/list_read_12.f90
Modified:
    branches/gcc-4_7-branch/gcc/testsuite/ChangeLog
    branches/gcc-4_7-branch/libgfortran/ChangeLog
    branches/gcc-4_7-branch/libgfortran/intrinsics/string_intrinsics_inc.c
    branches/gcc-4_7-branch/libgfortran/io/list_read.c
    branches/gcc-4_7-branch/libgfortran/io/read.c
    branches/gcc-4_7-branch/libgfortran/io/unit.c
    branches/gcc-4_7-branch/libgfortran/libgfortran.h

[Bug libfortran/38199] [4.7/4.8/4.9 Regression] missed optimization: I/O performance

[jvdelisle at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38199

Jerry DeLisle <jvdelisle at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #45 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
Fixed and closing.