[Bug libfortran/25949] New: Unbounded I/O buffer memory usage for formatted IO

[Bug libfortran/25949] New: Unbounded I/O buffer memory usage for formatted IO

[jb at gcc dot gnu dot org]

In some places the IO library simply assumes that the IO request is 'small
enough', and happily allocates a buffer big enough for the request to be
handled in one go. Unformatted IO already uses sread()/swrite() which bypasses
buffering if the request is bigger than the buffer size. However, this has not
been done for formatted IO. In some cases this results in excessive memory
usage. A rather extreme (and admittedly contrived) example is

program bigformdir
  integer :: i = 10**9
  open (10, file="foo", form="formatted", access="direct", recl=i)
  write (10, '(i3)', rec=1) 1
  close (10)
end program bigformdir

The above program will cause libgfortran to allocate 1 GB, fill it with ' ',
and then write it. Clearly the allocation is not necessary.


-- 
           Summary: Unbounded I/O buffer memory usage for formatted IO
           Product: gcc
           Version: 4.2.0
            Status: UNCONFIRMED
          Severity: minor
          Priority: P3
         Component: libfortran
        AssignedTo: jb at gcc dot gnu dot org
        ReportedBy: jb at gcc dot gnu dot org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25949

[Bug libfortran/25949] Unbounded I/O buffer memory usage for formatted IO

[pinskia at gcc dot gnu dot org]

------- Comment #1 from pinskia at gcc dot gnu dot org  2006-01-24 21:22 -------
It is even worse when opening two of these :) (or even five).

Confirmed.


-- 

pinskia at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
     Ever Confirmed|0                           |1
   Last reconfirmed|0000-00-00 00:00:00         |2006-01-24 21:22:56
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25949

[Bug libfortran/25949] Unbounded I/O buffer memory usage for formatted IO

[jb at gcc dot gnu dot org]

------- Comment #2 from jb at gcc dot gnu dot org  2006-01-25 21:35 -------
Patch here: http://gcc.gnu.org/ml/gcc-patches/2006-01/msg01765.html


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25949

[Bug libfortran/25949] Unbounded I/O buffer memory usage for formatted IO

[jb at gcc dot gnu dot org]

------- Comment #3 from jb at gcc dot gnu dot org  2006-02-12 19:59 -------
Subject: Bug 25949

Author: jb
Date: Sun Feb 12 19:59:32 2006
New Revision: 110895

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=110895
Log:
2006-02-12  Janne Blomqvist  <jb@gcc.gnu.org>

        PR libgfortran/25949
        * io/io.h: Add set function pointer to struct stream.  
        * io/unix.c (fd_seek): Only update offset, don't seek.
        (fd_sset): New function.
        (fd_read): Call lseek directly if necessary.
        (fd_write): Likewise.
        (fd_open): Set pointer to fd_sset.
        (mem_set): New function.
        (open_internal): Set pointer to mem_set.
        * io/transfer.c (write_block_direct): Rename to write_buf, add
        error return, non-pointer length argument.
        (unformatted_write): Update to use write_buf.
        (us_write): Simplify by using swrite instead of salloc_w.
        (write_us_marker): New function.
        (new_record_w): Use sset instead of memset, use write_us_marker,
        simplify by using swrite instead of salloc_w.

Modified:
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/io/io.h
    trunk/libgfortran/io/transfer.c
    trunk/libgfortran/io/unix.c


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25949

[Bug libfortran/25949] Unbounded I/O buffer memory usage for formatted IO

[jb at gcc dot gnu dot org]

------- Comment #4 from jb at gcc dot gnu dot org  2006-02-14 20:21 -------
Subject: Bug 25949

Author: jb
Date: Tue Feb 14 20:21:15 2006
New Revision: 110995

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=110995
Log:
2006-02-14  Janne Blomqvist  <jb@gcc.gnu.org>

        PR libgfortran/25949
        * io/io.h: Add set function pointer to struct stream.  
        * io/unix.c (fd_seek): Only update offset, don't seek.
        (fd_sset): New function.
        (fd_read): Call lseek directly if necessary.
        (fd_write): Likewise.
        (fd_open): Set pointer to fd_sset.
        (mem_set): New function.
        (open_internal): Set pointer to mem_set.
        * io/transfer.c (write_block_direct): Rename to write_buf, add
        error return, non-pointer length argument.
        (unformatted_write): Update to use write_buf.
        (us_write): Simplify by using swrite instead of salloc_w.
        (write_us_marker): New function.
        (new_record_w): Use sset instead of memset, use write_us_marker,
        simplify by using swrite instead of salloc_w.


Modified:
    branches/gcc-4_1-branch/libgfortran/ChangeLog
    branches/gcc-4_1-branch/libgfortran/io/io.h
    branches/gcc-4_1-branch/libgfortran/io/transfer.c
    branches/gcc-4_1-branch/libgfortran/io/unix.c


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25949

[Bug libfortran/25949] Unbounded I/O buffer memory usage for formatted IO

[fxcoudert at gcc dot gnu dot org]

------- Comment #5 from fxcoudert at gcc dot gnu dot org  2006-05-19 16:45 -------
I think we can close this one, now, can't we?


-- 

fxcoudert at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25949