[Bug libfortran/38654] New: Fortran I/O speedup

[Bug libfortran/38654] New: Fortran I/O speedup

[tkoenig at gcc dot gnu dot org]

A bug to hang a few ideas for I/O speedup from.

We have Jerry's first I/O refactoring results (splitting
into formatted read and write) at

http://gcc.gnu.org/ml/fortran/2008-12/msg00114.html

We also would want to pre-parse format strings.

I would really like to pre-parse the option strings
like advance="NO".  If we handle that in the front-end,
we should usually be able to avoid any run-time overhead
at all, and people who use variables should be no worse
off than now.  This should also reduce the size requirements
of the dt structures by a lot.

Finally, the way we parse and write our floating point
values appears to be slow.

What we also should have is a way to hang information
private to the library off the unit number, so we can
extend that.


-- 
           Summary: Fortran I/O speedup
           Product: gcc
           Version: 4.4.0
            Status: UNCONFIRMED
          Keywords: missed-optimization
          Severity: enhancement
          Priority: P3
         Component: libfortran
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: tkoenig at gcc dot gnu dot org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38654

[Bug libfortran/38654] Fortran I/O speedup

[domob at gcc dot gnu dot org]

------- Comment #1 from domob at gcc dot gnu dot org  2008-12-28 17:38 -------
I did once write a floating-point parser for FreeWRL.  I can dig it out so we
can try to compare it to gfortran's current one, but I've no idea whether it is
fast or not (although I tried at that time to write it as fast as possible). 
Would that help?


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38654

[Bug libfortran/38654] Fortran I/O speedup

[jvdelisle at gcc dot gnu dot org]

------- Comment #2 from jvdelisle at gcc dot gnu dot org  2008-12-28 20:55 -------
Daniel, it can't hurt to look.

Also, I have a format data hashing/caching patch in the works.  This avoids
re-parsing format strings if they have already been parsed once such as in a
loop containing I/O statements.  Preliminary results I am seeing about a 17%
speedup on at least one test case.  The patch is fairly straightforward, but I
do need to find a Gnu-able hash function.  I am using one out of wikipedia at
the moment.  I will post the patch as soon as I get it to a bug free state. 
Proof of principle is there for this.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38654

[Bug libfortran/38654] Fortran I/O speedup

[jvdelisle at gcc dot gnu dot org]

------- Comment #3 from jvdelisle at gcc dot gnu dot org  2008-12-28 21:02 -------
Created an attachment (id=16997)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=16997&action=view)
Patch that splits formatted read and write 

This is the patch mentioned in comment 0.  This patch touches some areas that
Janne is doing.  My thought is to apply this to 4.4.1 to regain some of the 4.3
performance back. I want to wait until branching so I do not interfere with
what Janne is doing at the moment which is fairly intrusive and will probably
go into 4.5


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38654

[Bug libfortran/38654] Fortran I/O speedup

[domob at gcc dot gnu dot org]

------- Comment #4 from domob at gcc dot gnu dot org  2008-12-28 21:27 -------
Created an attachment (id=16998)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=16998&action=view)
Number parsing routines

Sorry for the spam, but this is the parser-code for numbers I promised; it's
just a part out of FreeWRL's CFuncs/CParseLexer.c which was mostly written by
myself.  The code cut here is, except the two JAS-comments which are trivial
and will be reverted anyways, fully written by myself and under my copyright,
so I think this should not give any problems.  I'm going to rework it anyway,
though :)

If I find some time and the right places, I will make a full patch for gfortran
and do some benchmarks.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38654

[Bug libfortran/38654] Fortran I/O speedup

[tkoenig at gcc dot gnu dot org]

------- Comment #5 from tkoenig at gcc dot gnu dot org  2008-12-29 12:24 -------
Some simple test cases:

$ cat write-dollar.f
      program main
      do i=1,10**6
         write (10,fmt='(A$)') 'a'
      end do
      end
$ g77-3.4 -O write-dollar.f && time ./a.out

real    0m3.356s
user    0m1.068s
sys     0m2.280s
$ gfortran -O write-dollar.f && time ./a.out

real    0m12.198s
user    0m2.172s
sys     0m10.009s

$ gfortran -O write-advance.f && time ./a.out

real    0m10.737s
user    0m2.828s
sys     0m7.292s
$ ifort write-advance.f && time ./a.out

real    0m0.488s
user    0m0.480s
sys     0m0.008s


-- 

tkoenig at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
     Ever Confirmed|0                           |1
   Last reconfirmed|0000-00-00 00:00:00         |2008-12-29 12:24:52
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38654

[Bug libfortran/38654] Fortran I/O speedup

[domob at gcc dot gnu dot org]

------- Comment #6 from domob at gcc dot gnu dot org  2009-03-29 19:50 -------
Subject: Bug 38654

Author: domob
Date: Sun Mar 29 19:50:15 2009
New Revision: 145259

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=145259
Log:
2009-03-29  Daniel Kraft  <d@domob.eu>

        PR fortran/38654
        * io/read.c (read_f): Reworked to speed up floating point parsing.
        (convert_real): Use pointer-casting instead of memcpy and temporaries.

2009-03-29  Daniel Kraft  <d@domob.eu>

        PR fortran/38654
        * gfortran.dg/read_float_2.f03
        * gfortran.dg/read_float_3.f90

Added:
    branches/fortran-dev/gcc/testsuite/ChangeLog.dev
    branches/fortran-dev/gcc/testsuite/gfortran.dg/read_float_2.f03
    branches/fortran-dev/gcc/testsuite/gfortran.dg/read_float_3.f90
Modified:
    branches/fortran-dev/libgfortran/ChangeLog.dev
    branches/fortran-dev/libgfortran/io/read.c


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38654

[Bug libfortran/38654] Fortran I/O speedup

[jvdelisle at gcc dot gnu dot org]

------- Comment #7 from jvdelisle at gcc dot gnu dot org  2009-04-05 20:14 -------
Subject: Bug 38654

Author: jvdelisle
Date: Sun Apr  5 20:13:56 2009
New Revision: 145571

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=145571
Log:
2009-04-05  Daniel Kraft  <d@domob.eu>

        PR fortran/38654
        * io/read.c (read_f): Reworked to speed up floating point parsing.
        (convert_real): Use pointer-casting instead of memcpy and temporaries.

2009-04-05  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

        PR libfortran/37754
        * io/io.h (format_hash_entry): New structure for hash table.
        (format_hash_table): The hash table itself.
        (free_format_data): Revise function prototype.
        (free_format_hash_table, init_format_hash,
        free_format_hash): New function prototypes.
        * io/unit.c (close_unit_1): Use free_format_hash_table.
        * io/transfer.c (st_read_done, st_write_done): Free format data if
        internal unit.
        * io/format.c (free_format_hash_table): New function that frees any
        memory allocated previously for cached format data.
        (reset_node): New static helper function to reset the format counters
        for a format node.
        (reset_fnode_counters): New static function recursively calls
reset_node
        to traverse the fnode tree.
        (format_hash): New simple hash function based on XOR, probabalistic,
        tosses collisions.
        (save_parsed_format): New static function to save the parsed format
        data to use again.
        (find_parsed_format): New static function searches the hash table
        looking for a match.
        (free_format_data): Revised to accept pointer to format data rather
than
        the dtp pointer so that the function can be used in more places.
        (format_lex): Editorial.
        (parse_format_list): Set flag used to determine of format data hashing
        is to be used.  Internal units are not persistent enough for this.
        (revert): Move to ne location in file.
        (parse_format): Use new functions to look for previously parsed
        format strings and use them rather than re-parse.  If not found, saves
        the parsed format data for later use.

2009-04-05  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

        PR libfortran/37754
        * io/transfer.c (formatted_transfer_scalar): Remove this function by
        factoring it into two new functions, one for read and one for write,
        eliminating all the conditionals for read or write mode.
        (formatted transfer_scalar_read): New function.
        (formatted transfer_scalar_write): New function.
        (formatted_transfer): Use new functions.

2009-04-05  Janne Blomqvist  <jb@gcc.gnu.org>

        PR libfortran/25561 libfortran/37754
        * io/io.h (struct stream): Define new stream interface function
        pointers, and inline functions for accessing it.
        (struct fbuf): Use int instead of size_t, remove flushed element.
        (mem_alloc_w): New prototype.
        (mem_alloc_r): New prototype.
        (stream_at_bof): Remove prototype.
        (stream_at_eof): Remove prototype.
        (file_position): Remove prototype.
        (flush): Remove prototype.
        (stream_offset): Remove prototype.
        (unit_truncate): New prototype.
        (read_block_form): Change to return pointer, int* argument.
        (hit_eof): New prototype.
        (fbuf_init): Change prototype.
        (fbuf_reset): Change prototype.
        (fbuf_alloc): Change prototype.
        (fbuf_flush): Change prototype.
        (fbuf_seek): Change prototype.
        (fbuf_read): New prototype.
        (fbuf_getc_refill): New prototype.
        (fbuf_getc): New inline function.
        * io/fbuf.c (fbuf_init): Use int, get rid of flushed.
        (fbuf_debug): New function.
        (fbuf_reset): Flush, and return position offset.
        (fbuf_alloc): Simplify, don't flush, just realloc.
        (fbuf_flush): Make usable for read mode, salvage remaining bytes.
        (fbuf_seek): New whence argument.
        (fbuf_read): New function.
        (fbuf_getc_refill): New function.
        * io/file_pos.c (formatted_backspace): Use new stream interface.
        (unformatted_backspace): Likewise.
        (st_backspace): Make sure format buffer is reset, use new stream
        interface, use unit_truncate.
        (st_endfile): Likewise.
        (st_rewind): Likewise.
        * io/intrinsics.c: Use new stream interface.
        * io/list_read.c (push_char): Don't use u.p.scratch, use realloc
        to resize.
        (free_saved): Don't check u.p.scratch.
        (next_char): Use new stream interface, use fbuf_getc() for external
files.
        (finish_list_read): flush format buffer.
        (nml_query): Update to use modified interface:s
        * io/open.c (test_endfile): Use new stream interface.
        (edit_modes): Likewise.
        (new_unit): Likewise, set bytes_left to 1 for stream files.
        * io/read.c (read_l): Use new read_block_form interface.
        (read_utf8): Likewise.
        (read_utf8_char1): Likewise.
        (read_default_char1): Likewise.
        (read_utf8_char4): Likewise.
        (read_default_char4): Likewise.
        (read_a): Likewise.
        (read_a_char4): Likewise.
        (read_decimal): Likewise.
        (read_radix): Likewise.
        (read_f): Likewise.
        * io/transfer.c (read_sf): Use fbuf_read and mem_alloc_r, remove
        usage of u.p.line_buffer.
        (read_block_form): Update interface to return pointer, use
        fbuf_read for direct access.
        (read_block_direct): Update to new stream interface.
        (write_block): Use mem_alloc_w for internal I/O.
        (write_buf): Update to new stream interface.
        (formatted_transfer_scalar): Don't use u.p.line_buffer, use
        fbuf_seek for external files.
        (us_read): Update to new stream interface.
        (us_write): Likewise.
        (data_transfer_init): Always check if we switch modes and flush.
        (skip_record): Use new stream interface, fix comparison.
        (next_record_r): Check for and reset u.p.at_eof, use new stream
        interface, use fbuf_getc for spacing.
        (write_us_marker): Update to new stream interface, don't inline.
        (next_record_w_unf): Likewise.
        (sset): New function.
        (next_record_w): Use new stream interface, use fbuf for printing
        newline.
        (next_record): Use new stream interface.
        (finalize_transfer): Remove sfree call, use new stream interface.
        (st_iolength_done): Don't use u.p.scratch.
        (st_read): Don't check for end of file.
        (st_read_done): Don't use u.p.scratch, use unit_truncate.
        (hit_eof): New function.
        * io/unit.c (init_units): Always init fbuf for formatted units.
        (update_position): Use new stream interface.
        (unit_truncate): New function.
        (finish_last_advance_record): Use fbuf to print newline.
        * io/unix.c: Remove unused SSIZE_MAX macro.
        (BUFFER_SIZE): Make static const variable rather than macro.
        (struct unix_stream): Remove dirty_offset, len, method,
        small_buffer. Order elements by decreasing size.
        (struct int_stream): Remove.
        (move_pos_offset): Remove usage of dirty_offset.
        (reset_stream): Remove.
        (do_read): Rename to raw_read, update to match new stream
        interface.
        (do_write): Rename to raw_write, update to new stream interface.
        (raw_seek): New function.
        (raw_tell): New function.
        (raw_truncate): New function.
        (raw_close): New function.
        (raw_flush): New function.
        (raw_init): New function.
        (fd_alloc): Remove.
        (fd_alloc_r_at): Remove.
        (fd_alloc_w_at): Remove.
        (fd_sfree): Remove.
        (fd_seek): Remove.
        (fd_truncate): Remove.
        (fd_sset): Remove.
        (fd_read): Remove.
        (fd_write): Remove.
        (fd_close): Remove.
        (fd_open): Remove.
        (fd_flush): Rename to buf_flush, update to new stream interface
        and unix_stream.
        (buf_read): New function.
        (buf_write): New function.
        (buf_seek): New function.
        (buf_tell): New function.
        (buf_truncate): New function.
        (buf_close): New function.
        (buf_init): New function.
        (mem_alloc_r_at): Rename to mem_alloc_r, change prototype.
        (mem_alloc_w_at): Rename to mem_alloc_w, change prototype.
        (mem_read): Change to match new stream interface.
        (mem_write): Likewise.
        (mem_seek): Likewise.
        (mem_tell): Likewise.
        (mem_truncate): Likewise.
        (mem_close): Likewise.
        (mem_flush): New function.
        (mem_sfree): Remove.
        (empty_internal_buffer): Cast to correct type.
        (open_internal): Use correct type, init function pointers.
        (fd_to_stream): Test whether to open file as buffered or raw.
        (output_stream): Remove mode set.
        (error_stream): Likewise.
        (flush_all_units_1): Use new stream interface.
        (flush_all_units): Likewise.
        (stream_at_bof): Remove.
        (stream_at_eof): Remove.
        (file_position): Remove.
        (file_length): Update logic to use stream interface.
        (flush): Remove.
        (stream_offset): Remove.
        * io/write.c (write_utf8_char4): Use int instead of size_t.
        (write_x): Extra safety check.
        (namelist_write_newline): Use new stream interface.

Modified:
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/io/fbuf.c
    trunk/libgfortran/io/file_pos.c
    trunk/libgfortran/io/format.c
    trunk/libgfortran/io/intrinsics.c
    trunk/libgfortran/io/io.h
    trunk/libgfortran/io/list_read.c
    trunk/libgfortran/io/open.c
    trunk/libgfortran/io/read.c
    trunk/libgfortran/io/transfer.c
    trunk/libgfortran/io/unit.c
    trunk/libgfortran/io/unix.c
    trunk/libgfortran/io/write.c


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38654

[Bug libfortran/38654] Fortran I/O speedup

[jvdelisle at gcc dot gnu dot org]

------- Comment #8 from jvdelisle at gcc dot gnu dot org  2009-04-05 20:23 -------
Subject: Bug 38654

Author: jvdelisle
Date: Sun Apr  5 20:23:21 2009
New Revision: 145572

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=145572
Log:
2009-04-05  Daniel Kraft  <d@domob.eu>

        PR fortran/38654
        * gfortran.dg/read_float_2.f03
        * gfortran.dg/read_float_3.f90

Added:
    trunk/gcc/testsuite/gfortran.dg/read_float_2.f03
    trunk/gcc/testsuite/gfortran.dg/read_float_3.f90
Modified:
    trunk/gcc/testsuite/ChangeLog


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38654

[Bug libfortran/38654] Fortran I/O speedup

[jvdelisle at gcc dot gnu dot org]

------- Comment #9 from jvdelisle at gcc dot gnu dot org  2009-05-27 01:22 -------
Subject: Bug 38654

Author: jvdelisle
Date: Wed May 27 01:21:22 2009
New Revision: 147887

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=147887
Log:
2009-05-23  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

        Backport from mainline:
        PR libfortran/37754
        * io/write_float.def: Simplify format calculation.

2009-05-23  Francois-Xavier Coudert  <fxcoudert@gcc.gnu.org>

        Backport from mainline:
        PR fortran/22423
        * io/transfer.c (read_block_direct): Avoid warning.

2009-05-23  Janne Blomqvist  <jb@gcc.gnu.org>

        Backport from mainline:
        PR libfortran/39667
        * io/file_pos.c (st_rewind): Don't truncate or flush.
        * io/intrinsics.c (fgetc): Flush if switching mode.
        (fputc): Likewise.

2009-05-23  Janne Blomqvist  <jb@gcc.gnu.org>

        Backport from mainline:
        PR libfortran/39782
        * io/transfer.c (data_transfer_init): Don't flush before seek.

2009-05-23  Janne Blomqvist  <jb@gcc.gnu.org>

        Backport from mainline:
        * io/io.h (is_preconnected): Remove prototype.
        * io/unix.c (is_preconnected): Remove function.

2009-05-23  Janne Blomqvist  <jb@gcc.gnu.org>

        Backport from mainline:
        PR libfortran/38668
        * io/transfer.c (finalize_transfer): Don't flush for advance='no'.

2009-05-23 Danny Smith  <dannysmith@clear.net.nz>

        Backport from mainline:
        * io/write.c (itoa) : Rename back to gfc_itoa.
        (write_i): Adjust call to write_decimal.
        (write_integer):  Use gfc_itoa.

2009-05-23  Janne Blomqvist  <jb@gcc.gnu.org>

        Backport from mainline:
        * io/io.h (move_pos_offset): Remove prototype.
        * io/transfer.c (formatted_transfer_scalar_read): Use sseek
        instead of move_pos_offset.
        * io/unix.c (move_pos_offset): Remove.

2009-05-23  Janne Blomqvist  <jb@gcc.gnu.org>

        Backport from mainline:
        PR libfortran/39665 libfortran/39702 libfortran/39709
        * io/io.h (st_parameter_dt): Revert aligned attribute from u.p.value.
        * io/list_read.c (read_complex): Read directly into user pointer.
        (read_real): Likewise.
        (list_formatted_read_scalar): Update read_complex and read_real calls.
        (nml_read_obj): Read directly into user pointer.

2009-05-23  Janne Blomqvist  <jb@gcc.gnu.org>

        Backport from mainline:
        PR libfortran/39665
        * io/io.h (st_parameter_dt): Add aligned attribute to u.p.value.
        * io/read.c (convert_real): Add note about alignment requirements.

2009-05-23  Janne Blomqvist  <jb@gcc.gnu.org>

        Backport from mainline:
        * io/open.c (already_open): Test for POSIX close return value.
        * io/unit.c (close_unit_1): Likewise.
        * io/unix.c (raw_close): Return 0 for success for preconnected units.

2009-05-23  Janne Blomqvist  <jb@gcc.gnu.org>

        Backport from mainline:
        * runtime/error.c (gfc_itoa): Move to io/write.c
        (xtoa): Rename to gfc_xtoa.
        * runtime/backtrace.c (show_backtrace): Call gfc_xtoa.
        * libgfortran.h (gfc_itoa): Remove prototype.
        (xtoa): Rename prototype to gfc_xtoa.
        * io/list_read.c (nml_read_obj): Use size_t for string length.
        * io/transfer.c (read_block_direct): Change nbytes arg from
        pointer to value.
        (unformatted_read): Minor cleanup, call read_block_directly properly.
        (skip_record): Use ssize_t.
        (next_record_w_unf): Avoid stell() call by calling sseek with SEEK_CUR.
        (iolength_transfer): Make sure to multiply before cast.
        * io/intrinsics.c (fgetc): Remove unnecessary variable.
        * io/format.c (format_hash): Use gfc_charlen_type.
        * io/write.c (itoa): Move from runtime/error.c:gfc_itoa, rename,
        make static.
        (write_i): Call with pointer to itoa.
        (write_z): Call with pointer to gfc_xtoa.
        (write_integer): Pointer to itoa.
        (nml_write_obj): Type cleanup, don't call strlen in loop.

2009-05-23  H.J. Lu  <hongjiu.lu@intel.com>

        Backport from mainline:
        PR libgfortran/39664
        * io/unix.c (raw_close): Don't close STDOUT_FILENO,
        STDERR_FILENO nor STDIN_FILENO.

2009-05-23  David Edelsohn  <edelsohn@gnu.org>

        Backport from mainline:
        * io/io.h (struct stream): Rename truncate to trunc.
        (struncate): Same.
        * io/unix.c (raw_init): Rename truncate to trunc.
        (buf_init): Same.
        (open_internal): Same.

2009-05-23  Daniel Kraft  <d@domob.eu>

        Backport from mainline:
        PR fortran/38654
        * io/read.c (read_f): Reworked to speed up floating point parsing.
        (convert_real): Use pointer-casting instead of memcpy and temporaries.

2009-05-23  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

        Backport from mainline:
        PR libfortran/37754
        * io/io.h (format_hash_entry): New structure for hash table.
        (format_hash_table): The hash table itself.
        (free_format_data): Revise function prototype.
        (free_format_hash_table, init_format_hash,
        free_format_hash): New function prototypes.
        * io/unit.c (close_unit_1): Use free_format_hash_table.
        * io/transfer.c (st_read_done, st_write_done): Free format data if
        internal unit.
        * io/format.c (free_format_hash_table): New function that frees any
        memory allocated previously for cached format data.
        (reset_node): New static helper function to reset the format counters
        for a format node.
        (reset_fnode_counters): New static function recursively calls
reset_node
        to traverse the fnode tree.
        (format_hash): New simple hash function based on XOR, probabalistic,
        tosses collisions.
        (save_parsed_format): New static function to save the parsed format
        data to use again.
        (find_parsed_format): New static function searches the hash table
        looking for a match.
        (free_format_data): Revised to accept pointer to format data rather
than
        the dtp pointer so that the function can be used in more places.
        (format_lex): Editorial.
        (parse_format_list): Set flag used to determine of format data hashing
        is to be used.  Internal units are not persistent enough for this.
        (revert): Move to ne location in file.
        (parse_format): Use new functions to look for previously parsed
        format strings and use them rather than re-parse.  If not found, saves
        the parsed format data for later use.

2009-05-23  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

        Backport from mainline:
        PR libfortran/37754
        * io/transfer.c (formatted_transfer_scalar): Remove this function by
        factoring it into two new functions, one for read and one for write,
        eliminating all the conditionals for read or write mode.
        (formatted transfer_scalar_read): New function.
        (formatted transfer_scalar_write): New function.
        (formatted_transfer): Use new functions.

2009-05-23  Janne Blomqvist  <jb@gcc.gnu.org>

        Backport from mainline:
        PR libfortran/25561 libfortran/37754
        * io/io.h (struct stream): Define new stream interface function
        pointers, and inline functions for accessing it.
        (struct fbuf): Use int instead of size_t, remove flushed element.
        (mem_alloc_w): New prototype.
        (mem_alloc_r): New prototype.
        (stream_at_bof): Remove prototype.
        (stream_at_eof): Remove prototype.
        (file_position): Remove prototype.
        (flush): Remove prototype.
        (stream_offset): Remove prototype.
        (unit_truncate): New prototype.
        (read_block_form): Change to return pointer, int* argument.
        (hit_eof): New prototype.
        (fbuf_init): Change prototype.
        (fbuf_reset): Change prototype.
        (fbuf_alloc): Change prototype.
        (fbuf_flush): Change prototype.
        (fbuf_seek): Change prototype.
        (fbuf_read): New prototype.
        (fbuf_getc_refill): New prototype.
        (fbuf_getc): New inline function.
        * io/fbuf.c (fbuf_init): Use int, get rid of flushed.
        (fbuf_debug): New function.
        (fbuf_reset): Flush, and return position offset.
        (fbuf_alloc): Simplify, don't flush, just realloc.
        (fbuf_flush): Make usable for read mode, salvage remaining bytes.
        (fbuf_seek): New whence argument.
        (fbuf_read): New function.
        (fbuf_getc_refill): New function.
        * io/file_pos.c (formatted_backspace): Use new stream interface.
        (unformatted_backspace): Likewise.
        (st_backspace): Make sure format buffer is reset, use new stream
        interface, use unit_truncate.
        (st_endfile): Likewise.
        (st_rewind): Likewise.
        * io/intrinsics.c: Use new stream interface.
        * io/list_read.c (push_char): Don't use u.p.scratch, use realloc
        to resize.
        (free_saved): Don't check u.p.scratch.
        (next_char): Use new stream interface, use fbuf_getc() for external
files.
        (finish_list_read): flush format buffer.
        (nml_query): Update to use modified interface:s
        * io/open.c (test_endfile): Use new stream interface.
        (edit_modes): Likewise.
        (new_unit): Likewise, set bytes_left to 1 for stream files.
        * io/read.c (read_l): Use new read_block_form interface.
        (read_utf8): Likewise.
        (read_utf8_char1): Likewise.
        (read_default_char1): Likewise.
        (read_utf8_char4): Likewise.
        (read_default_char4): Likewise.
        (read_a): Likewise.
        (read_a_char4): Likewise.
        (read_decimal): Likewise.
        (read_radix): Likewise.
        (read_f): Likewise.
        * io/transfer.c (read_sf): Use fbuf_read and mem_alloc_r, remove
        usage of u.p.line_buffer.
        (read_block_form): Update interface to return pointer, use
        fbuf_read for direct access.
        (read_block_direct): Update to new stream interface.
        (write_block): Use mem_alloc_w for internal I/O.
        (write_buf): Update to new stream interface.
        (formatted_transfer_scalar): Don't use u.p.line_buffer, use
        fbuf_seek for external files.
        (us_read): Update to new stream interface.
        (us_write): Likewise.
        (data_transfer_init): Always check if we switch modes and flush.
        (skip_record): Use new stream interface, fix comparison.
        (next_record_r): Check for and reset u.p.at_eof, use new stream
        interface, use fbuf_getc for spacing.
        (write_us_marker): Update to new stream interface, don't inline.
        (next_record_w_unf): Likewise.
        (sset): New function.
        (next_record_w): Use new stream interface, use fbuf for printing
        newline.
        (next_record): Use new stream interface.
        (finalize_transfer): Remove sfree call, use new stream interface.
        (st_iolength_done): Don't use u.p.scratch.
        (st_read): Don't check for end of file.
        (st_read_done): Don't use u.p.scratch, use unit_truncate.
        (hit_eof): New function.
        * io/unit.c (init_units): Always init fbuf for formatted units.
        (update_position): Use new stream interface.
        (unit_truncate): New function.
        (finish_last_advance_record): Use fbuf to print newline.
        * io/unix.c: Remove unused SSIZE_MAX macro.
        (BUFFER_SIZE): Make static const variable rather than macro.
        (struct unix_stream): Remove dirty_offset, len, method,
        small_buffer. Order elements by decreasing size.
        (struct int_stream): Remove.
        (move_pos_offset): Remove usage of dirty_offset.
        (reset_stream): Remove.
        (do_read): Rename to raw_read, update to match new stream
        interface.
        (do_write): Rename to raw_write, update to new stream interface.
        (raw_seek): New function.
        (raw_tell): New function.
        (raw_truncate): New function.
        (raw_close): New function.
        (raw_flush): New function.
        (raw_init): New function.
        (fd_alloc): Remove.
        (fd_alloc_r_at): Remove.
        (fd_alloc_w_at): Remove.
        (fd_sfree): Remove.
        (fd_seek): Remove.
        (fd_truncate): Remove.
        (fd_sset): Remove.
        (fd_read): Remove.
        (fd_write): Remove.
        (fd_close): Remove.
        (fd_open): Remove.
        (fd_flush): Rename to buf_flush, update to new stream interface
        and unix_stream.
        (buf_read): New function.
        (buf_write): New function.
        (buf_seek): New function.
        (buf_tell): New function.
        (buf_truncate): New function.
        (buf_close): New function.
        (buf_init): New function.
        (mem_alloc_r_at): Rename to mem_alloc_r, change prototype.
        (mem_alloc_w_at): Rename to mem_alloc_w, change prototype.
        (mem_read): Change to match new stream interface.
        (mem_write): Likewise.
        (mem_seek): Likewise.
        (mem_tell): Likewise.
        (mem_truncate): Likewise.
        (mem_close): Likewise.
        (mem_flush): New function.
        (mem_sfree): Remove.
        (empty_internal_buffer): Cast to correct type.
        (open_internal): Use correct type, init function pointers.
        (fd_to_stream): Test whether to open file as buffered or raw.
        (output_stream): Remove mode set.
        (error_stream): Likewise.
        (flush_all_units_1): Use new stream interface.
        (flush_all_units): Likewise.
        (stream_at_bof): Remove.
        (stream_at_eof): Remove.
        (file_position): Remove.
        (file_length): Update logic to use stream interface.
        (flush): Remove.
        (stream_offset): Remove.
        * io/write.c (write_utf8_char4): Use int instead of size_t.
        (write_x): Extra safety check.
        (namelist_write_newline): Use new stream interface.


Modified:
    branches/gcc-4_4-branch/libgfortran/ChangeLog
    branches/gcc-4_4-branch/libgfortran/io/fbuf.c
    branches/gcc-4_4-branch/libgfortran/io/file_pos.c
    branches/gcc-4_4-branch/libgfortran/io/format.c
    branches/gcc-4_4-branch/libgfortran/io/intrinsics.c
    branches/gcc-4_4-branch/libgfortran/io/io.h
    branches/gcc-4_4-branch/libgfortran/io/list_read.c
    branches/gcc-4_4-branch/libgfortran/io/open.c
    branches/gcc-4_4-branch/libgfortran/io/read.c
    branches/gcc-4_4-branch/libgfortran/io/transfer.c
    branches/gcc-4_4-branch/libgfortran/io/unit.c
    branches/gcc-4_4-branch/libgfortran/io/unix.c
    branches/gcc-4_4-branch/libgfortran/io/write.c
    branches/gcc-4_4-branch/libgfortran/io/write_float.def
    branches/gcc-4_4-branch/libgfortran/libgfortran.h
    branches/gcc-4_4-branch/libgfortran/runtime/backtrace.c
    branches/gcc-4_4-branch/libgfortran/runtime/error.c


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38654

[Bug libfortran/38654] Fortran I/O speedup

[jvdelisle at gcc dot gnu dot org]

------- Comment #10 from jvdelisle at gcc dot gnu dot org  2009-06-09 03:21 -------
I think we are done with this.  Reopen if more suggestions.


-- 

jvdelisle at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38654

[Bug libfortran/38654] Fortran I/O speedup

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=38654
Bug 38654 depends on bug 41137, which changed state.

Bug 41137 Summary: inefficient zeroing of an array
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=41137

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |RESOLVED
         Resolution|---                         |FIXED