[Bug fortran/20257] New: Fortran runtime error: End of record occurs when writing large arrays

[Bug fortran/20257] New: Fortran runtime error: End of record occurs when writing large arrays

[kamaraju at gmail dot com]

Consider the following test program (fortran 90)

program explore
 implicit none

 integer, parameter :: NX = 32, NR = 16, NPHI=2

 real :: input(0:NX-1, 0:NR-1, 0:NPHI-1)

 write(*,*) size(input)

 call random_number(input)
 write(*,*) input
end program explore


This program compiles fine. It executes fine. But when the output is redirected
to a file, it produces the following error.

$gfortran explore.f90
$./a.out > temp
At line 11 of file explore.f90
Fortran runtime error: End of record




system & build information:

$gfortran -v
Using built-in specs.
Target: i486-linux-gnu
Configured with: ../src/configure -v
--enable-languages=c,c++,java,f95,objc,ada--prefix=/usr/lib/gcc-snapshot
--enable-shared --with-system-zlib --enable-nls --enable-threads=posix
--without-included-gettext --disable-werror --enable-__cxa_atexit
--enable-libstdcxx-allocator=mt --enable-clocale=gnu
--enable-libstdcxx-debug --enable-mpfr --enable-java-gc=boehm
--enable-java-awt=gtk --enable-gtk-cairo --enable-checking=release
i486-linux-gnu
Thread model: posix
gcc version 4.1.0 20050227 (experimental)

$gcc -v
Using built-in specs.
Target: i486-linux-gnu
Configured with: ../src/configure -v --enable-languages=c,c++,java,f95,objc,ada
--prefix=/usr/lib/gcc-snapshot --enable-shared --with-system-zlib --enable-nls
--enable-threads=posix --without-included-gettext --disable-werror
--enable-__cxa_atexit --enable-libstdcxx-allocator=mt --enable-clocale=gnu
--enable-libstdcxx-debug --enable-mpfr --enable-java-gc=boehm
--enable-java-awt=gtk --enable-gtk-cairo --enable-checking=release i486-linux-gnu
Thread model: posix
gcc version 4.1.0 20050227 (experimental)

$uname -a
Linux kusumanchi 2.6.9-1-686 #1 Thu Nov 25 03:48:29 EST 2004 i686 GNU/Linux

Other information:
  System is Debian unstable (sid).

-- 
           Summary: Fortran runtime error: End of record occurs when writing
                    large arrays
           Product: gcc
           Version: 4.1.0
            Status: UNCONFIRMED
          Severity: critical
          Priority: P2
         Component: fortran
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: kamaraju at gmail dot com
                CC: gcc-bugs at gcc dot gnu dot org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=20257

[Bug fortran/20257] Fortran runtime error: End of record occurs when writing large arrays

[jvdelisle at verizon dot net]

------- Additional Comments From jvdelisle at verizon dot net  2005-03-01 03:14 -------
I noticed on a the list mention that the error occurs only when the output is
redirected.  When I run ./a.out the error message is embedded in the middle of
the output before the last numbers are printed.

The error message gets printed out before the stdout buffer finishes.

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=20257

[Bug fortran/20257] Fortran runtime error: End of record occurs when writing large arrays

[coudert at clipper dot ens dot fr]

------- Additional Comments From coudert at clipper dot ens dot fr  2005-03-01 13:10 -------
This has indeed nothing to do with the output being redirected to a file, or
stdout, or even being connected (in Fortran way) to a file:

$ cat pr20257.f90 
 integer, parameter :: NX = 32, NR = 16, NPHI=2
 real :: input(0:NX-1, 0:NR-1, 0:NPHI-1)

 input(:,:,:) = 0
 open (10, file='foo')
 write(10,*) input
end
$ gfortran pr20257.f90 && ./a.out
At line 6 of file pr20257.f90
Fortran runtime error: End of record

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=20257

[Bug fortran/20257] Fortran runtime error: End of record occurs when writing large arrays

[coudert at clipper dot ens dot fr]

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |coudert at clipper dot ens
                   |                            |dot fr


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=20257

[Bug libfortran/20257] Fortran runtime error: End of record occurs when writing large arrays

[pinskia at gcc dot gnu dot org]

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|critical                    |normal
             Status|UNCONFIRMED                 |NEW
          Component|fortran                     |libfortran
     Ever Confirmed|                            |1
   Last reconfirmed|0000-00-00 00:00:00         |2005-03-01 13:59:17
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=20257

[Bug libfortran/20257] Fortran runtime error: End of record occurs when writing large arrays

[coudert at clipper dot ens dot fr]

------- Additional Comments From coudert at clipper dot ens dot fr  2005-03-02 09:34 -------
It happens for all kind of output (formatted (direct or sequential) or
unformatted), and is due to gfortran setting an arbitrary maximal length of
10000 chars (this is in current_unit->recl). Since this code doesn't specify a
recl, we use the default:

io.h:#define DEFAULT_RECL 10000

I don't see a reason for setting so low a value. I will ask later today on the
gfortran mailing-list about the reasons behind this choice. Hope we can fix this
quickly.

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=20257

[Bug libfortran/20257] Fortran runtime error: End of record occurs when writing large arrays

[coudert at clipper dot ens dot fr]

------- Additional Comments From coudert at clipper dot ens dot fr  2005-03-03 10:34 -------
Partial fix proposed here: http://gcc.gnu.org/ml/fortran/2005-03/msg00023.html

This fixes the bug when writing to files, but there still is the case of
preconnected units (stdout, stderr). This will need a little more thinking.

Reduced test case for preconnected units:

 real :: array(10000)
 array(:) = 0
 write(*,*) array
end

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=20257

[Bug libfortran/20257] Fortran runtime error: End of record occurs when writing large arrays

[coudert at clipper dot ens dot fr]

------- Additional Comments From coudert at clipper dot ens dot fr  2005-03-03 13:08 -------
Complete patch is here: http://gcc.gnu.org/ml/fortran/2005-03/msg00024.html

Note: this just makes the limit higher (1 Gb), but there still is a limit on the
size of things you can write to preconnected units.

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=20257

[Bug libfortran/20257] Fortran runtime error: End of record occurs when writing large arrays

[fxcoudert at gcc dot gnu dot org]

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|coudert at clipper dot ens  |fxcoudert at gcc dot gnu dot
                   |dot fr                      |org
   Last reconfirmed|2005-03-01 13:59:17         |2005-03-18 07:11:28
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=20257

[Bug libfortran/20257] Fortran runtime error: End of record occurs when writing large arrays

[pinskia at gcc dot gnu dot org]

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|normal                      |minor
   Last reconfirmed|2005-06-17 15:14:59         |2005-09-25 20:28:59
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=20257

[Bug libfortran/20257] Fortran runtime error: End of record occurs when writing large arrays

[jvdelisle at gcc dot gnu dot org]

------- Comment #12 from jvdelisle at gcc dot gnu dot org  2006-04-29 05:47 -------
Fixed on Trunk and now 4.1 branch


-- 

jvdelisle at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|                            |FIXED


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=20257

[Bug libfortran/20257] Fortran runtime error: End of record occurs when writing large arrays

[jvdelisle at gcc dot gnu dot org]

------- Comment #11 from jvdelisle at gcc dot gnu dot org  2006-04-29 04:27 -------
Subject: Bug 20257

Author: jvdelisle
Date: Sat Apr 29 04:27:09 2006
New Revision: 113373

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=113373
Log:
2006-04-28  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

        PR libgfortran/20257
        * io/io.h: Add prototypes for get_internal_unit and free_internal_unit.
        * io/unit.c (get_internal_unit): Initialize unit number, not zero.
        (free_internal_unit): New function to consolidate freeing memory.
        (get_unit): Initialize internal_unit_desc to NULL when unit is
        external.
        * io/unix.c (mem_close): Check for not NULL before freeing memory.
        * io/transfer.c (read_block): Reset bytes_left and skip error if unit
        is preconnected and default record length is reached.
        (read_block_direct): Ditto.
        (write_block): Ditto.
        (write_buf): Ditto.
        (data_transfer_init): Only flush if not internal unit.
        (finalize_transfer): Ditto and delete code to free memory used by
        internal units.
        (st_read_done): Use new function - free_internal_unit.
        (st_write_done): Use new function - free_internal unit.

Modified:
    branches/gcc-4_1-branch/libgfortran/ChangeLog
    branches/gcc-4_1-branch/libgfortran/io/io.h
    branches/gcc-4_1-branch/libgfortran/io/transfer.c
    branches/gcc-4_1-branch/libgfortran/io/unit.c
    branches/gcc-4_1-branch/libgfortran/io/unix.c


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=20257

[Bug libfortran/20257] Fortran runtime error: End of record occurs when writing large arrays

[jvdelisle at gcc dot gnu dot org]

------- Comment #10 from jvdelisle at gcc dot gnu dot org  2006-04-23 02:05 -------
Subject: Bug 20257

Author: jvdelisle
Date: Sun Apr 23 02:04:58 2006
New Revision: 113190

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=113190
Log:
2006-04-22  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

        PR libgfortran/20257
        * io/io.h: Add prototypes for get_internal_unit and free_internal_unit.
        * io/unit.c (get_internal_unit): Initialize unit number, not zero.
        (free_internal_unit): New function to consolidate freeing memory.
        (get_unit): Initialize internal_unit_desc to NULL when unit is
        external.
        * io/unix.c (mem_close): Check for not NULL before freeing memory.
        * io/transfer.c (read_block): Reset bytes_left and skip error if unit
        is preconnected and default record length is reached.
        (read_block_direct): Ditto.
        (write_block): Ditto.
        (write_buf): Ditto.
        (data_transfer_init): Only flush if not internal unit.
        (finalize_transfer): Ditto and delete code to free memory used by
        internal units.
        (st_read_done): Use new function - free_internal_unit.
        (st_write_done): Use new function - free_internal unit.

Modified:
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/io/io.h
    trunk/libgfortran/io/transfer.c
    trunk/libgfortran/io/unit.c
    trunk/libgfortran/io/unix.c


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=20257

[Bug libfortran/20257] Fortran runtime error: End of record occurs when writing large arrays

[patchapp at dberlin dot org]

------- Comment #9 from patchapp at dberlin dot org  2006-04-20 23:20 -------
Subject: Bug number PR20257

A patch for this bug has been added to the patch tracker.
The mailing list url for the patch is
http://gcc.gnu.org/ml/gcc-patches/2006-04/msg00785.html


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=20257

[Bug libfortran/20257] Fortran runtime error: End of record occurs when writing large arrays

[jvdelisle at gcc dot gnu dot org]

------- Comment #8 from jvdelisle at gcc dot gnu dot org  2006-04-15 07:15 -------
I have the beginnings of a patch going.  It covers formatted write to stdout. 
Seems to work OK.  I just need to extend the concept to the other cases.


-- 

jvdelisle at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         AssignedTo|unassigned at gcc dot gnu   |jvdelisle at gcc dot gnu dot
                   |dot org                     |org
             Status|NEW                         |ASSIGNED
   Last reconfirmed|2005-12-28 06:36:18         |2006-04-15 07:15:20
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=20257

[Bug libfortran/20257] Fortran runtime error: End of record occurs when writing large arrays

[fxcoudert at gcc dot gnu dot org]

------- Comment #7 from fxcoudert at gcc dot gnu dot org  2006-01-25 09:32 -------
(In reply to comment #6)
> I don't see this problem any more.  Was this fixed?  Is the default record
> length an issue any more?  Can we close this PR?  I wonder if the fix to P25835
> fixed this?  If so I propose we close this unless it is related to the new PR
> Janne is working on, large records with little writes, 25949.

As I said in comment #5, the limit was just pushed higher, but big writes on
preconnected units still fail:

$ gfortran a.f90 && ./a.out > bigfile
At line 3 of file a.f90
Fortran runtime error: End of record
$ cat a.f90
 real :: array(100000000)
 array(:) = 0
 write(*,*) array
end
$ ll bigfile 
-rw-r--r--  1 coudert users 1.0G Jan 25 10:28 bigfile
$ gfortran -v
Using built-in specs.
Target: i386-linux
Configured with: ../gcc/configure --prefix=/tmp/gfortran-20060124/irun
--enable-languages=c,fortran --host=i386-linux
--with-gmp=/tmp/gfortran-20060124/gfortran_libs
Thread model: posix
gcc version 4.2.0 20060124 (experimental)


You can look at the patch cited in comment #5 for where the problem lies. And
this is not really related to Janne's PR, I guess. It's specific to
preconnected units.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=20257

[Bug libfortran/20257] Fortran runtime error: End of record occurs when writing large arrays

[jvdelisle at gcc dot gnu dot org]

------- Comment #6 from jvdelisle at gcc dot gnu dot org  2006-01-25 07:30 -------
I don't see this problem any more.  Was this fixed?  Is the default record
length an issue any more?  Can we close this PR?  I wonder if the fix to P25835
fixed this?  If so I propose we close this unless it is related to the new PR
Janne is working on, large records with little writes, 25949.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=20257