* gfortran-4.1.0 file io bug fix
@ 2006-03-09 12:23 Georgy Salnikov
2006-03-09 14:28 ` Jerry DeLisle
0 siblings, 1 reply; 4+ messages in thread
From: Georgy Salnikov @ 2006-03-09 12:23 UTC (permalink / raw)
To: jvdelisle, sgk; +Cc: fortran, gcc-patches
Dear Jerry, dear Steve,
It's me again, this time dealing with another Fortran file IO bug found in
the version gcc-4.1.0.
The file IO in gfortran-4.1.0 is now good fast, and backspace after
end-of-file condition I reported in november 2005 is now working correctly.
The problem I'd like to report now is following.
One creates a sequential unformatted file, writes several records with
various data there, and then repeatedly rewinds the file and reads these
data back again and again.
If all the records are short enough, the IO will be buffered inside the
libgfortran runtime (for the reason of speed), and this works correctly.
If the records are much longer than the buffer size (8K at the moment), the
IO requests will be done directly via libc's read/write/lseek. This seems
also to work correctly, although I am not 100% sure.
My program writes firstly a few short records, one of them containing the
size of an array which is to be written next. All these records are written
buffered. Then a rather long array is written (unbuffered because of its
length). Then again several short records, and again a long array (of
different size). And so on.
While writing all the information is written correctly, I have proven, the
file contents is right.
After rewinding the first short records and the first long array are read
correctly. But immediately after reading the long array the file pointer
s->logical_offset (libgfortran/io/unix.c) gets corrupted. As the result, the
following short records are filled with garbage, and while trying to read
the array of wrong length the program usually gets SIGSEGV or something like
this.
The following patch against gcc-4.1.0 corrects this wrong behaviour:
diff -Naur gcc-4.1.0.orig/libgfortran/io/unix.c gcc-4.1.0/libgfortran/io/unix.c
--- gcc-4.1.0.orig/libgfortran/io/unix.c 2006-02-15 02:21:15.000000000 +0600
+++ gcc-4.1.0/libgfortran/io/unix.c 2006-03-09 16:31:44.000000000 +0600
@@ -224,8 +224,8 @@
inline static void
reset_stream (unix_stream * s, size_t bytes_rw)
{
- s->physical_offset += bytes_rw;
- s->logical_offset = s->physical_offset;
+ s->logical_offset += bytes_rw;
+ s->physical_offset = s->logical_offset;
if (s->file_length != -1 && s->physical_offset > s->file_length)
s->file_length = s->physical_offset;
}
The following sample program demonstrates the bug. It runs correctly when
compiled, let's say, with g77-3.3.6, wrong when compiled with unpatched
gfortran-4.1.0, and again correctly after patching it as shown above.
With kind regards,
Georgy Salnikov, Novosibirsk.
parameter (isize=50000)
character*80 name
integer array(isize)
call getarg (1, name)
print *, 'opening ', name
open (unit=7, file=name, status='new', access='sequential',
*form='unformatted', err=100)
print *, 'ok'
do 1 i=1,isize
1 array(i) = i
print *, 'writing ', name
write (unit=7, err=100) name
print *, 'writing ', isize
write (unit=7, err=100) isize
print *, 'ok'
print *, 'writing ', array(1), ' ... ', array(isize)
call wr (array, isize, iret)
if (iret.eq.0) then
print *, 'ok'
else
print *, 'bad'
endif
do 2 i=1,isize
2 array(i) = isize-i
print *, 'writing ', name
write (unit=7, err=100) name
print *, 'writing ', isize
write (unit=7, err=100) isize
print *, 'ok'
print *, 'writing ', array(1), ' ... ', array(isize)
call wr (array, isize, iret)
if (iret.eq.0) then
print *, 'ok'
else
print *, 'bad'
endif
print *, 'rewinding'
rewind (unit=7, err=100)
print *, 'ok'
print *, 'reading name'
read (unit=7, err=100) name
print *, 'name ', name, ' ok'
print *, 'reading nsize'
read (unit=7, err=100) nsize
print *, 'nsize ', nsize, ' ok'
do 3 i=1,nsize
3 array(i) = 0
print *, 'reading array'
call rd (array, nsize, iret)
if (iret.eq.0) then
print *, 'array ', array(1), ' ... ', array(nsize), ' ok'
else
print *, 'bad'
endif
print *, 'reading name'
read (unit=7, err=100) name
print *, 'name ', name, ' ok'
print *, 'reading nsize'
read (unit=7, err=100) nsize
print *, 'nsize ', nsize, ' ok'
do 4 i=1,nsize
4 array(i) = 0
print *, 'reading array'
call rd (array, nsize, iret)
if (iret.eq.0) then
print *, 'array ', array(1), ' ... ', array(nsize), ' ok'
else
print *, 'bad'
endif
stop
100 print *, 'bad'
stop
end
subroutine rd (a, n, iret)
integer a(n)
read (unit=7, err=100) a
iret = 0
return
100 iret = 1
return
end
subroutine wr (a, n, iret)
integer a(n)
write (unit=7, err=100) a
iret = 0
return
100 iret = 1
return
end
_______________________________________________________________________________
Georgy Salnikov
NMR Group
Novosibirsk Institute of Organic Chemistry
Lavrentjeva, 9, 630090 Novosibirsk, Russia
Tel. +7-383-3307864 +7-383-3331456
Fax +7-383-3331456
Email sge@nmr.nioch.nsc.ru
_______________________________________________________________________________
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: gfortran-4.1.0 file io bug fix
2006-03-09 12:23 gfortran-4.1.0 file io bug fix Georgy Salnikov
@ 2006-03-09 14:28 ` Jerry DeLisle
2006-03-09 15:09 ` Jerry DeLisle
2007-03-30 10:40 ` gfortran-4.1.2 " Georgy Salnikov
0 siblings, 2 replies; 4+ messages in thread
From: Jerry DeLisle @ 2006-03-09 14:28 UTC (permalink / raw)
To: Georgy Salnikov; +Cc: sgk, fortran, gcc-patches
Georgy Salnikov wrote:
> Dear Jerry, dear Steve,
>
> It's me again, this time dealing with another Fortran file IO bug found in
> the version gcc-4.1.0.
>
> The file IO in gfortran-4.1.0 is now good fast, and backspace after
> end-of-file condition I reported in november 2005 is now working correctly.
>
> The problem I'd like to report now is following.
>
> One creates a sequential unformatted file, writes several records with
> various data there, and then repeatedly rewinds the file and reads these
> data back again and again.
>
> If all the records are short enough, the IO will be buffered inside the
> libgfortran runtime (for the reason of speed), and this works correctly.
>
> If the records are much longer than the buffer size (8K at the moment), the
> IO requests will be done directly via libc's read/write/lseek. This seems
> also to work correctly, although I am not 100% sure.
>
> My program writes firstly a few short records, one of them containing the
> size of an array which is to be written next. All these records are written
> buffered. Then a rather long array is written (unbuffered because of its
> length). Then again several short records, and again a long array (of
> different size). And so on.
>
> While writing all the information is written correctly, I have proven, the
> file contents is right.
>
> After rewinding the first short records and the first long array are read
> correctly. But immediately after reading the long array the file pointer
> s->logical_offset (libgfortran/io/unix.c) gets corrupted. As the result, the
> following short records are filled with garbage, and while trying to read
> the array of wrong length the program usually gets SIGSEGV or something like
> this.
>
Georgy,
This may be PR26499.
Let me check this out this evening with the patch to pr26499 first and see if
that resolves this. Then we'll go from there. Thanks for the report and test case.
Regards,
Jerry
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: gfortran-4.1.0 file io bug fix
2006-03-09 14:28 ` Jerry DeLisle
@ 2006-03-09 15:09 ` Jerry DeLisle
2007-03-30 10:40 ` gfortran-4.1.2 " Georgy Salnikov
1 sibling, 0 replies; 4+ messages in thread
From: Jerry DeLisle @ 2006-03-09 15:09 UTC (permalink / raw)
Cc: Georgy Salnikov, sgk, fortran, gcc-patches
Jerry DeLisle wrote:
>
> Georgy,
>
> This may be PR26499.
>
> Let me check this out this evening with the patch to pr26499 first and
> see if that resolves this. Then we'll go from there. Thanks for the
> report and test case.
>
> Regards,
>
> Jerry
>
Well, I am late for work. I tried the test case this morning and it works fine
with patch for pr26499. This will be committed to trunk soon and 4.1.1 in about
a week.
Regards,
Jerry
^ permalink raw reply [flat|nested] 4+ messages in thread
* gfortran-4.1.2 file io bug fix
2006-03-09 14:28 ` Jerry DeLisle
2006-03-09 15:09 ` Jerry DeLisle
@ 2007-03-30 10:40 ` Georgy Salnikov
1 sibling, 0 replies; 4+ messages in thread
From: Georgy Salnikov @ 2007-03-30 10:40 UTC (permalink / raw)
To: Jerry DeLisle; +Cc: sgk, fortran, gcc-patches
Dear Jerry, dear Steve,
It's me again, this time about a patch against another Fortran file IO bug
found in the newest release gcc-4.1.2.
The program I am working on reads a previously created sequential
unformatted file. It reads the file record by record searching for a
specific contents. The records have different lengths and they are sometimes
shorter than the array they are read to. The algorithm worked fine under
gfortran-4.1.1. But gfortran-4.1.2 introduced a new kind of error:
ERROR_SHORT_RECORD whose condition was simply ignored by gfortran-4.1.1. The
existence of this new feature itself is not a problem at all. But just after
ERROR_SHORT_RECORD libgfortran gets out of sync, looses the record markers
and gives an end-of-file condition on all subsequent reads although there
should be a lot of further records present.
I have reduced the test to a small demo program. This program writes
sequentially five records: 32, 16, 32, 16, 32 bytes long. Then it rewinds
the file and reads all them again, this procedure goes fine under both
gfortran-4.1.1 and 4.1.2. Then it rewinds again and tries to read 32 bytes
five times. It goes fine under gfortran-4.1.1 as it ignores short records.
Gfortran-4.1.2 gives a proper error condition at the second read. I dont
know the Fortran 95 standard so good, I rely on the supposition that there
must be really an error condition in this case. But I'd expect the following
record (32 bytes long) should be read correctly. But it isn't. The program
gets two times end-of-file, and again an error for the very last read.
After some debugging I created a patch against gcc-4.1.2 which corrects my
problem. Now libgfortran gives ERROR_SHORT_RECORD for the records number 2
and 4 (16 bytes) and continues to read correctly all the other records. And
the real program where I have seen the problem for the first time works now
also.
But I am not so experienced gfortran hacker, and the place I tried to
correct looks rather complex. I am not sure if my patch corrects all the
possible cases. Could you please review this peace of code?
The proposed patch against gcc-4.1.2 is here:
diff -Naur gcc-4.1.2.orig/libgfortran/io/transfer.c gcc-4.1.2/libgfortran/io/transfer.c
--- gcc-4.1.2.orig/libgfortran/io/transfer.c 2006-12-26 04:56:54.000000000 +0600
+++ gcc-4.1.2/libgfortran/io/transfer.c 2007-03-29 18:54:02.000000000 +0700
@@ -348,6 +348,8 @@
if (short_record)
{
generate_error (&dtp->common, ERROR_SHORT_RECORD, NULL);
+ dtp->u.p.current_unit->current_record = 0;
+ next_record (dtp, 1);
return;
}
The test program which demonstrates the bug is here (compile with
--std=legacy).
PROGRAM TEST
CHARACTER*8 A(2), B(4), C
A(1) = '***A1***'
A(2) = '***A2***'
B(1) = '***B1***'
B(2) = '***B2***'
B(3) = '***B3***'
B(4) = '***B4***'
OPEN (1, FILE='TEST', STATUS='UNKNOWN', FORM='UNFORMATTED')
PRINT *, 'WRITING FIVE ARRAYS: B(4), A(2), B(4), A(2), B(4)'
WRITE (1) B
WRITE (1) A
WRITE (1) B
WRITE (1) A
WRITE (1) B
PRINT *, 'REWINDING FILE'
REWIND 1
PRINT *, 'READING FIVE ARRAYS: B(4), A(2), B(4), A(2), B(4)'
ASSIGN 1 TO M
C = 'B(4)'
READ (1, ERR=2, END=3) B
PRINT 4, C, B
4 FORMAT (A/4(A8/))
1 ASSIGN 5 TO M
C = 'A(2)'
READ (1, ERR=2, END=3) A
PRINT 4, C, A
5 ASSIGN 6 TO M
C = 'B(4)'
READ (1, ERR=2, END=3) B
PRINT 4, C, B
6 ASSIGN 7 TO M
C = 'A(2)'
READ (1, ERR=2, END=3) A
PRINT 4, C, A
7 ASSIGN 8 TO M
C = 'B(4)'
READ (1, ERR=2, END=3) B
PRINT 4, C, B
8 PRINT *, 'REWINDING FILE'
REWIND 1
PRINT *, 'READING FIVE ARRAYS: B(4), B(4), B(4), B(4), B(4)'
ASSIGN 9 TO M
READ (1, ERR=2, END=3) B
PRINT 4, C, B
9 ASSIGN 10 TO M
READ (1, ERR=2, END=3) B
PRINT 4, C, B
10 ASSIGN 11 TO M
READ (1, ERR=2, END=3) B
PRINT 4, C, B
11 ASSIGN 12 TO M
READ (1, ERR=2, END=3) B
PRINT 4, C, B
12 ASSIGN 13 TO M
READ (1, ERR=2, END=3) B
PRINT 4, C, B
13 STOP
2 PRINT 14, C, A, B
14 FORMAT ('ERROR READING ',A/'A(2)'/2(A8/)'B(4)'/4(A8/))
GOTO M
3 PRINT 15, C, A, B
15 FORMAT ('ENDFILE READING ',A/'A(2)'/2(A8/)'B(4)'/4(A8/))
GOTO M
END
The output of this program under gcc-4.1.1 (no errors at all) is:
WRITING FIVE ARRAYS: B(4), A(2), B(4), A(2), B(4)
REWINDING FILE
READING FIVE ARRAYS: B(4), A(2), B(4), A(2), B(4)
B(4)
***B1***
***B2***
***B3***
***B4***
A(2)
***A1***
***A2***
B(4)
***B1***
***B2***
***B3***
***B4***
A(2)
***A1***
***A2***
B(4)
***B1***
***B2***
***B3***
***B4***
REWINDING FILE
READING FIVE ARRAYS: B(4), B(4), B(4), B(4), B(4)
B(4)
***B1***
***B2***
***B3***
***B4***
B(4)
***A1***
***A2***
***B3***
***B4***
B(4)
***B1***
***B2***
***B3***
***B4***
B(4)
***A1***
***A2***
***B3***
***B4***
B(4)
***B1***
***B2***
***B3***
***B4***
Under unpatched gcc-4.1.2 (a complete mess after the first short record) is:
WRITING FIVE ARRAYS: B(4), A(2), B(4), A(2), B(4)
REWINDING FILE
READING FIVE ARRAYS: B(4), A(2), B(4), A(2), B(4)
B(4)
***B1***
***B2***
***B3***
***B4***
A(2)
***A1***
***A2***
B(4)
***B1***
***B2***
***B3***
***B4***
A(2)
***A1***
***A2***
B(4)
***B1***
***B2***
***B3***
***B4***
REWINDING FILE
READING FIVE ARRAYS: B(4), B(4), B(4), B(4), B(4)
B(4)
***B1***
***B2***
***B3***
***B4***
ERROR READING B(4)
A(2)
***A1***
***A2***
B(4)
***A1***
***A2***
***B3***
***B4***
ENDFILE READING B(4)
A(2)
***A1***
***A2***
B(4)
***A1***
***A2***
***B3***
***B4***
ENDFILE READING B(4)
A(2)
***A1***
***A2***
B(4)
***A1***
***A2***
***B3***
***B4***
ERROR READING B(4)
A(2)
***A1***
***A2***
B(4)
***A1***
***A2***
***B3***
***B4***
And under patched gcc-4.1.2 (errors on short records, otherwise fine) is:
WRITING FIVE ARRAYS: B(4), A(2), B(4), A(2), B(4)
REWINDING FILE
READING FIVE ARRAYS: B(4), A(2), B(4), A(2), B(4)
B(4)
***B1***
***B2***
***B3***
***B4***
A(2)
***A1***
***A2***
B(4)
***B1***
***B2***
***B3***
***B4***
A(2)
***A1***
***A2***
B(4)
***B1***
***B2***
***B3***
***B4***
REWINDING FILE
READING FIVE ARRAYS: B(4), B(4), B(4), B(4), B(4)
B(4)
***B1***
***B2***
***B3***
***B4***
ERROR READING B(4)
A(2)
***A1***
***A2***
B(4)
***A1***
***A2***
***B3***
***B4***
B(4)
***B1***
***B2***
***B3***
***B4***
ERROR READING B(4)
A(2)
***A1***
***A2***
B(4)
***A1***
***A2***
***B3***
***B4***
B(4)
***B1***
***B2***
***B3***
***B4***
Kind regards,
Georgy.
_______________________________________________________________________________
Georgy Salnikov
NMR Group
Novosibirsk Institute of Organic Chemistry
Lavrentjeva, 9, 630090 Novosibirsk, Russia
Tel. +7-383-3307864 +7-383-3331456
Fax +7-383-3331456
Email sge@nmr.nioch.nsc.ru
_______________________________________________________________________________
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2007-03-30 9:36 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2006-03-09 12:23 gfortran-4.1.0 file io bug fix Georgy Salnikov
2006-03-09 14:28 ` Jerry DeLisle
2006-03-09 15:09 ` Jerry DeLisle
2007-03-30 10:40 ` gfortran-4.1.2 " Georgy Salnikov
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).