[Bug fortran/35632] New: [Regression] stream io broken on FreeBSD due to ftruncate changes.

[Bug fortran/35632] New: [Regression] stream io broken on FreeBSD due to ftruncate changes.

[kargl at gcc dot gnu dot org]

This commit:

2008-03-05  Hans-Peter Nilsson  <hp@axis.com>

        PR libfortran/35293
        * io/unix.c (fd_truncate): Fold s->special_file case into
        success case of ftruncate/chsize call instead of the failure case.
        Make failure case actually return failure.  Properly update stream
        pointers on failure.  Call runtime_error for targets without
        neither ftruncate nor chsize where such a call would be needed.

leads to

WARNING: program timed out.
FAIL: gfortran.dg/streamio_4.f90  -O0  execution test
WARNING: program timed out.
FAIL: gfortran.dg/streamio_4.f90  -O1  execution test

WARNING: program timed out.
FAIL: gfortran.dg/streamio_4.f90  -O2  execution test
WARNING: program timed out.
FAIL: gfortran.dg/streamio_4.f90  -O3 -fomit-frame-pointer  execution test

A ktrace on one of the failing cases shows

 19649 streamio_4.exe RET   ftruncate 0
 19649 streamio_4.exe CALL  sigprocmask(SIG_BLOCK,0,0xbfbfd7a8)
 19649 streamio_4.exe RET   sigprocmask 0
 19649 streamio_4.exe CALL  lseek(0x3,0x188b2,SEEK_SET,0)
 19649 streamio_4.exe RET   lseek 100530/0x188b2
 19649 streamio_4.exe CALL  ftruncate(0x3,0x188b2,0)
 19649 streamio_4.exe RET   ftruncate 0
 19649 streamio_4.exe CALL  sigprocmask(SIG_BLOCK,0,0xbfbfd7a8)
 19649 streamio_4.exe RET   sigprocmask 0
 19649 streamio_4.exe CALL  lseek(0x3,0x188b8,SEEK_SET,0)
 19649 streamio_4.exe RET   lseek 100536/0x188b8
 19649 streamio_4.exe CALL  ftruncate(0x3,0x188b8,0)
 19649 streamio_4.exe RET   ftruncate 0
 19649 streamio_4.exe CALL  sigprocmask(SIG_BLOCK,0,0xbfbfd7a8)
 19649 streamio_4.exe RET   sigprocmask 0
 19649 streamio_4.exe CALL  lseek(0x3,0x188be,SEEK_SET,0)
 19649 streamio_4.exe RET   lseek 100542/0x188be
 19649 streamio_4.exe CALL  ftruncate(0x3,0x188be,0)
 19649 streamio_4.exe RET   ftruncate 0
 19649 streamio_4.exe CALL  sigprocmask(SIG_BLOCK,0,0xbfbfd7a8)
 19649 streamio_4.exe RET   sigprocmask 0
 19649 streamio_4.exe CALL  lseek(0x3,0x188c4,SEEK_SET,0)
 19649 streamio_4.exe RET   lseek 100548/0x188c4
 19649 streamio_4.exe CALL  ftruncate(0x3,0x188c4,0)
 19649 streamio_4.exe RET   ftruncate 0
 19649 streamio_4.exe CALL  sigprocmask(SIG_BLOCK,0,0xbfbfd7a8)
 19649 streamio_4.exe RET   sigprocmask 0
 19649 streamio_4.exe CALL  lseek(0x3,0x188ca,SEEK_SET,0)
 19649 streamio_4.exe RET   lseek 100554/0x188ca
 19649 streamio_4.exe CALL  ftruncate(0x3,0x188ca,0)
 19649 streamio_4.exe RET   ftruncate 0

ad nausem.

Please revert your patch.


-- 
           Summary: [Regression] stream io broken on FreeBSD due to
                    ftruncate changes.
           Product: gcc
           Version: 4.4.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: fortran
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: kargl at gcc dot gnu dot org
GCC target triplet:  i386-unknown-freebsd8.0


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35632

[Bug fortran/35632] [Regression] stream io broken on FreeBSD due to ftruncate changes.

[jvdelisle at gcc dot gnu dot org]

------- Comment #1 from jvdelisle at gcc dot gnu dot org  2008-03-19 03:46 -------
*** Bug 35633 has been marked as a duplicate of this bug. ***


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35632

[Bug fortran/35632] [Regression] stream io broken on FreeBSD due to ftruncate changes.

[kargl at gcc dot gnu dot org]

------- Comment #2 from kargl at gcc dot gnu dot org  2008-03-19 04:09 -------
Hmmm, reverting Hans-Peter's patch does not fix the problem. Apologies
to Hans-Peter for jumping the gun.

Going back further in time, reverting r132512 fixes the problem with
streamio_4.f90, but
leads to 

Running /usr/home/kargl/gcc/trunk/gcc/testsuite/gfortran.dg/dg.exp ...
FAIL: gfortran.dg/streamio_15.f90  -O0  execution test
FAIL: gfortran.dg/streamio_15.f90  -O1  execution test
FAIL: gfortran.dg/streamio_15.f90  -O2  execution test
FAIL: gfortran.dg/streamio_15.f90  -O3 -fomit-frame-pointer  execution test
FAIL: gfortran.dg/streamio_15.f90  -O3 -fomit-frame-pointer -funroll-loops 
execution test
FAIL: gfortran.dg/streamio_15.f90  -O3 -fomit-frame-pointer -funroll-all-loops
-finline-functions  execution test
FAIL: gfortran.dg/streamio_15.f90  -O3 -g  execution test
FAIL: gfortran.dg/streamio_15.f90  -Os  execution test

This revision is 2008-02-20  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

        PR libfortran/35132
        * io/transfer.c (next_record_w): Truncate after the last record for
        STREAM I/O.

        PR libfortran/34954
        * io/transfer.c (data_transfer_init): Initialize dtp->rec if writing.

        PR libfortran/34974
        * io/transfer.c (formatted_transfer_scalar): Flush the buffer if skips
        is less than zero. (next_record_w): Use sseek to position the file to
        the max position reached.


-- 

kargl at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|hp at axis dot com          |
 GCC target triplet| i386-unknown-freebsd8.0    |i386-unknown-freebsd8.0


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35632

[Bug fortran/35632] [Regression] stream io broken on FreeBSD due to ftruncate changes.

[hp at gcc dot gnu dot org]

------- Comment #3 from hp at gcc dot gnu dot org  2008-03-19 04:43 -------
(In reply to comment #2)
> Hmmm, reverting Hans-Peter's patch does not fix the problem. Apologies
> to Hans-Peter for jumping the gun.

Apology accepted. FWIW, looking at your strace output and streamio_4.f90, it
seems this fd 0x3 isn't a special file and the ftruncate calls succeed, so my
patch shouldn't have had any effect.  But you know that now.  I did a similar
strace for streamio_4 on x86_64-unknown-linux-gnu with r132965 and came to the
same conclusions; the lots-of-small-increment ftruncate-calls were introduced
with r132512 and not affected by my patch.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35632

[Bug fortran/35632] [Regression] stream io broken on FreeBSD due to ftruncate changes.

[jvdelisle at gcc dot gnu dot org]

------- Comment #4 from jvdelisle at gcc dot gnu dot org  2008-03-19 05:25 -------
Since I do not have access to this system, can you try this:

Index: transfer.c
===================================================================
--- transfer.c  (revision 133275)
+++ transfer.c  (working copy)
@@ -2604,7 +2604,8 @@ next_record_w (st_parameter_dt *dtp, int
          if (is_stream_io (dtp))
            {
              dtp->u.p.current_unit->strm_pos += len;
-             struncate(dtp->u.p.current_unit->s);
+             if (done)
+               struncate(dtp->u.p.current_unit->s);
            }
        }

streamio_15.f90 passes with this and it should reduce the number of calls to
truncate.  If this fixes it, then I would think we have some other latent bug
and not really fixed it.


-- 

jvdelisle at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
     Ever Confirmed|0                           |1
   Last reconfirmed|0000-00-00 00:00:00         |2008-03-19 05:25:01
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35632

[Bug fortran/35632] [Regression] stream io broken on FreeBSD due to ftruncate changes.

[jvdelisle at gcc dot gnu dot org]

------- Comment #5 from jvdelisle at gcc dot gnu dot org  2008-03-19 05:40 -------
Steve, also please replace the aborts in the test case and see what is really
happening. streamio_4.f90 is doing a large number of individual WRITEs in the
loop and your ktrace shows no failure of truncate.  This test case also
exercises the buffering and it could be a READ error as well.  Need more data. 
Thanks


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35632

[Bug fortran/35632] [Regression] stream io broken on FreeBSD due to ftruncate changes.

[kargl at gcc dot gnu dot org]

------- Comment #6 from kargl at gcc dot gnu dot org  2008-03-20 00:56 -------
(In reply to comment #4)
> Since I do not have access to this system, can you try this:
> 
> Index: transfer.c
> ===================================================================
> --- transfer.c  (revision 133275)
> +++ transfer.c  (working copy)
> @@ -2604,7 +2604,8 @@ next_record_w (st_parameter_dt *dtp, int
>           if (is_stream_io (dtp))
>             {
>               dtp->u.p.current_unit->strm_pos += len;
> -             struncate(dtp->u.p.current_unit->s);
> +             if (done)
> +               struncate(dtp->u.p.current_unit->s);
>             }
>         }
> 
> streamio_15.f90 passes with this and it should reduce the number of calls to
> truncate.  If this fixes it, then I would think we have some other latent bug
> and not really fixed it.
> 

The patch does not fix the problem.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35632

[Bug fortran/35632] [Regression] stream io broken on FreeBSD due to ftruncate changes.

[kargl at gcc dot gnu dot org]

------- Comment #7 from kargl at gcc dot gnu dot org  2008-03-20 03:36 -------
With a clean trunk as of 2 minutes ago, this program 

program streamtest
  implicit none
  character(1)   :: lf = char(10)
  character(1)   :: tchar
  integer        :: i,j,k
  integer, parameter :: lines = 5231
  open(10, file="teststream", access="stream", form="formatted")
  do i=1,lines
    do j=0,9
      write(10,"(i5)") j
    end do
    if (mod(i,23) == 0) print *, 'i = ', i
  end do
  close(10)
  close(10,status="delete")
end program streamtest

yields

~/work/bin/gfortran -static -O -o z streamio_4.f90
time ./z
      158.72 real         0.46 user         5.90 sys

If I revert Jerry's commit,

svn merge -r 132512:132511 .

I then see

~/work/bin/gfortran -static -O -o z streamio_4.f90
time ./z
        0.94 real         0.64 user         0.26 sys

158.72 / 0.94 = 169

Any chance, you'll revert the offending patch?


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35632

[Bug fortran/35632] [Regression] stream io broken on FreeBSD due to ftruncate changes.

[tkoenig at gcc dot gnu dot org]

------- Comment #8 from tkoenig at gcc dot gnu dot org  2008-03-20 14:47 -------
(In reply to comment #7)
>
> Any chance, you'll revert the offending patch?
> 

The patch fixed a wrong-code PR, so by reverting the patch, we would
introduce a wrong-code regression.  I think it would be better to see what
the underlying issue is for FreeBSD, then fix that. 


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35632

[Bug fortran/35632] [Regression] stream io broken on FreeBSD due to ftruncate changes.

[kargl at gcc dot gnu dot org]

------- Comment #9 from kargl at gcc dot gnu dot org  2008-03-21 00:16 -------
(In reply to comment #8)
> (In reply to comment #7)
> >
> > Any chance, you'll revert the offending patch?
> > 
> 
> The patch fixed a wrong-code PR, so by reverting the patch, we would
> introduce a wrong-code regression.  I think it would be better to see what
> the underlying issue is for FreeBSD, then fix that. 

Your first sentence is also illogical.  gfortran 4.2 and 4.3 work fine on
FreeBSD.  This patch breaks gfortran on FreeBSD.  It is therefore a regression.
The patch should be reverted and the wrong-code PR re-opened.  You can't have
a regression within trunk.

If I understood Jerry correctly on IRC, the committed code actually
addresses 3 independent issues, which makes it rather hard for me to
undo only portions of what was committed because I don't know which
parts address which problem.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35632

[Bug fortran/35632] [Regression] stream io broken on FreeBSD due to ftruncate changes.

[jvdelisle at gcc dot gnu dot org]

------- Comment #10 from jvdelisle at gcc dot gnu dot org  2008-03-21 00:41 -------
I have been working with Steve on IRC in the evenings when we can.  We are
attempting to isolate this issue.  Stay tuned.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35632

[Bug fortran/35632] [Regression] stream io broken on FreeBSD due to ftruncate changes.

[jvdelisle at gcc dot gnu dot org]

------- Comment #11 from jvdelisle at gcc dot gnu dot org  2008-03-22 21:14 -------
I think this patch solves this issue.  Regression tested on x86-64.  Waiting
for Steve to confirm on FreeBSD.

Index: transfer.c
===================================================================
--- transfer.c  (revision 133436)
+++ transfer.c  (working copy)
@@ -1985,12 +1985,12 @@ data_transfer_init (st_parameter_dt *dtp
       if (dtp->u.p.mode == READING
          && dtp->u.p.current_unit->mode == WRITING
          && !is_internal_unit (dtp))
-        flush(dtp->u.p.current_unit->s);
+       flush(dtp->u.p.current_unit->s);

       /* Check whether the record exists to be read.  Only
         a partial record needs to exist.  */

-      if (dtp->u.p.mode == READING && (dtp->rec -1)
+      if (dtp->u.p.mode == READING && (dtp->rec - 1)
          * dtp->u.p.current_unit->recl >= file_length
(dtp->u.p.current_unit->s))
        {
          generate_error (&dtp->common, LIBERROR_BAD_OPTION,
@@ -2604,7 +2604,9 @@ next_record_w (st_parameter_dt *dtp, int
          if (is_stream_io (dtp))
            {
              dtp->u.p.current_unit->strm_pos += len;
-             struncate(dtp->u.p.current_unit->s);
+             if (dtp->u.p.current_unit->strm_pos
+                 < file_length (dtp->u.p.current_unit->s))
+               struncate (dtp->u.p.current_unit->s);
            }
        }



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35632

[Bug fortran/35632] [Regression] stream io broken on FreeBSD due to ftruncate changes.

[jvdelisle at gcc dot gnu dot org]

------- Comment #12 from jvdelisle at gcc dot gnu dot org  2008-03-22 22:03 -------
Subject: Bug 35632

Author: jvdelisle
Date: Sat Mar 22 22:03:13 2008
New Revision: 133454

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=133454
Log:
2008-03-22  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

        PR libfortran/35632
        * io/transfer.c (data_transfer_init):  Fix whitespace.
        (next_record_w): Truncate the file only if the stream
        position is short of the file end.

Modified:
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/io/transfer.c


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35632

[Bug fortran/35632] [Regression] stream io broken on FreeBSD due to ftruncate changes.

[jvdelisle at gcc dot gnu dot org]

------- Comment #13 from jvdelisle at gcc dot gnu dot org  2008-03-23 17:15 -------
Fixed on trunk.  No need to back port. The struncate in question is not in 4.3

For the record.  The code before the patch was fine except it called struncate
more often then needed.  On most systems, this was not noticeable.  However, on
FreeBSD, the ftruncate is performed sychronously and there for took too long to
do for the streamio_4.90 test case.  The patch avoids the issue by only calling
struncate when necessary.  Overall performance is improved.


-- 

jvdelisle at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED
   Target Milestone|---                         |4.4.0


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35632

[Bug fortran/35632] [Regression] stream io broken on FreeBSD due to ftruncate changes.

[jb at gcc dot gnu dot org]

------- Comment #14 from jb at gcc dot gnu dot org  2008-05-16 17:38 -------
Subject: Bug 35632

Author: jb
Date: Fri May 16 17:37:30 2008
New Revision: 135432

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=135432
Log:
Fix fallout from part 1 of PR25561 patch.

2008-05-16  Janne Blomqvist  <jb@gcc.gnu.org>

        PR libfortran/35632
        * io/open.c (new_unit): Set stream position to correct value.

Modified:
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/io/open.c


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35632