[Bug libfortran/30162] I/O with named pipes does not work

[Bug libfortran/30162] I/O with named pipes does not work

[iliev at rz dot rwth-aachen.de]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

Hristo Iliev <iliev at rz dot rwth-aachen.de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |iliev at rz dot
                   |                            |rwth-aachen.de

--- Comment #22 from Hristo Iliev <iliev at rz dot rwth-aachen.de> 2012-08-02 17:34:03 UTC ---
Revision 180701 removed all checks for special files in unit.c:unit_truncate().
As a consequence programs compiled with gfortran 4.6.3 and newer cannot to
write (formatted) to Unix named pipes as ftruncate() fails with EINVAL:

(64-bit code)

lseek(3, 0, SEEK_CUR)                   = -1 ESPIPE (Illegal seek)
write(3, " Hello, world!\n", 15)        = 15
lseek(3, 0, SEEK_CUR)                   = -1 ESPIPE (Illegal seek)
ftruncate(3, 18446744073709551615)      = -1 EINVAL (Invalid argument)

(32-bit code)

_llseek(3, 0, 0xffba8fa0, SEEK_CUR)     = -1 ESPIPE (Illegal seek)
write(3, " Hello, world!\n", 15)        = 15
_llseek(3, 0, 0xffba9080, SEEK_CUR)     = -1 ESPIPE (Illegal seek)
ftruncate64(3, 18446744073709551615)    = -1 EINVAL (Invalid argument)

[Bug libfortran/30162] I/O with named pipes does not work

[jb at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

--- Comment #23 from Janne Blomqvist <jb at gcc dot gnu.org> 2012-08-03 19:45:10 UTC ---
(In reply to comment #22)
> Revision 180701 removed all checks for special files in unit.c:unit_truncate().

Yes. As the email message introducing the patch explained, special files are
special in many ways, and trying to impose some kind of uniform behavior on
them is bound to fail in more or less surprising and inconvenient ways. For the
same reason, the code for handling special files were removed from the buffered
IO functions, and all non-regular files now just use the raw IO functions
instead. The only sane way is to seek/truncate only when the user code requires
such a behavior, and if that then fails, return an error and let the user
handle it.

The real bug is thus that we shouldn't try to seek or truncate the fd at all,
as it's not necessary in this case.

I recall that I tried some simple programs with named pipes when I implemented
the patch, but maybe some subsequent patch then broke it again.

[Bug libfortran/30162] I/O with named pipes does not work

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

Tobias Burnus <burnus at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
         Resolution|WONTFIX                     |

--- Comment #24 from Tobias Burnus <burnus at gcc dot gnu.org> 2012-08-06 13:43:05 UTC ---
(In reply to comment #23)
> The real bug is thus that we shouldn't try to seek or truncate the fd at all,
> as it's not necessary in this case.

I think the issue occurs for:

st_write_done (st_parameter_dt *dtp)
...
  if (dtp->u.p.current_unit != NULL
      && dtp->u.p.current_unit->flags.access == ACCESS_SEQUENTIAL)
    switch (dtp->u.p.current_unit->endfile)
...
      case NO_ENDFILE:
        /* Get rid of whatever is after this record.  */
        if (!is_internal_unit (dtp))
          unit_truncate (dtp->u.p.current_unit,
                         stell (dtp->u.p.current_unit->s),
                         &dtp->common);
        dtp->u.p.current_unit->endfile = AT_ENDFILE;

with a file as simple as

  open (99, file="foo.dat")
  write (99, '(a)') 'Hello world'
  end


$ mkfifo foo.dat
$ tail -f foo.dat &
[2] 342
$ gfortran test.f90 && ./a.out
At line 2 of file test.f90 (unit = 99, file = 'foo.dat')
Fortran runtime error: Invalid argument
Hello world
$ fg
tail -f foo.dat
^C

[Bug libfortran/30162] [4.7/4.8 Regression] I/O with named pipes does not work

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

Tobias Burnus <burnus at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |burnus at gcc dot gnu.org
   Target Milestone|---                         |4.7.3
            Summary|I/O with named pipes does   |[4.7/4.8 Regression] I/O
                   |not work                    |with named pipes does not
                   |                            |work

--- Comment #25 from Tobias Burnus <burnus at gcc dot gnu.org> 2012-10-30 11:37:18 UTC ---
As Janne just mentioned on the gfortran mailing list, this PR is a regression.
Thus, I have marked it as such. Based on the information below, It fails since
GCC 4.7.

In reply to comment #22)
> Revision 180701 removed all checks for special files in unit.c:unit_truncate().

[Bug libfortran/30162] [4.7/4.8 Regression] I/O with named pipes does not work

[tkoenig at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

--- Comment #26 from Thomas Koenig <tkoenig at gcc dot gnu.org> 2012-11-10 14:38:03 UTC ---
Is this caused by

http://gcc.gnu.org/viewcvs?view=revision&revision=180701

?

Maybe we need to remember if we have a special file after all, or just ignore
the error on the truncate.

[Bug libfortran/30162] [4.7/4.8 Regression] I/O with named pipes does not work

[jb at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

--- Comment #27 from Janne Blomqvist <jb at gcc dot gnu.org> 2012-11-10 20:21:41 UTC ---
(In reply to comment #26)
> Is this caused by
> 
> http://gcc.gnu.org/viewcvs?view=revision&revision=180701
> 
> ?
> 
> Maybe we need to remember if we have a special file after all, or just ignore
> the error on the truncate.

IMHO the correct fix is to not seek and/or truncate the file unless the Fortran
semantics require it; that way libgfortran does not need to care whether the
file is special or not. As explained in #c23, special files are special in
different ways (also different on different OS'es), and trying to enumerate all
the ways in which they are special is bound to fail. 

I think Tobias comment #c24 pinpoints the place which needs to be fixed, but
unfortunately I haven't had time to look into it.

[Bug libfortran/30162] [4.7/4.8 Regression] I/O with named pipes does not work

[jakub at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Priority|P3                          |P4
                 CC|                            |jakub at gcc dot gnu.org

[Bug libfortran/30162] [4.7/4.8 Regression] I/O with named pipes does not work

[tkoenig at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

--- Comment #28 from Thomas Koenig <tkoenig at gcc dot gnu.org> 2012-11-24 12:05:57 UTC ---
(In reply to comment #27)
> (In reply to comment #26)
> > Is this caused by
> > 
> > http://gcc.gnu.org/viewcvs?view=revision&revision=180701
> > 
> > ?
> > 
> > Maybe we need to remember if we have a special file after all, or just ignore
> > the error on the truncate.
> 
> IMHO the correct fix is to not seek and/or truncate the file unless the Fortran
> semantics require it; that way libgfortran does not need to care whether the
> file is special or not. As explained in #c23, special files are special in
> different ways (also different on different OS'es), and trying to enumerate all
> the ways in which they are special is bound to fail. 
> 
> I think Tobias comment #c24 pinpoints the place which needs to be fixed, but
> unfortunately I haven't had time to look into it.

Well, we need to make sure that the (very basic) program

      program main
      character*10 x
      open(10,file="foo.dat")
      write (10,'(A)') '1111'
      write (10,'(A)') '2222'
      close (10)
      open(10,file="foo.dat")
      write (10,'(A)') '3333'
      close (10)
      open(10,file="foo.dat")
 100  continue
        read (10,'(A)',end=200) x
        write (*,'(A)') x
      goto 100
 200  continue
      close(10,status="delete")
      end

continues to work as expected: That probably means truncating on
close and rewind.

I can see what I can do, but I have little time... (as always)

[Bug libfortran/30162] [4.7/4.8 Regression] I/O with named pipes does not work

[jb at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

--- Comment #29 from Janne Blomqvist <jb at gcc dot gnu.org> 2012-11-25 15:08:24 UTC ---
(In reply to comment #28)
> (In reply to comment #27)
> > (In reply to comment #26)
> > > Is this caused by
> > > 
> > > http://gcc.gnu.org/viewcvs?view=revision&revision=180701
> > > 
> > > ?
> > > 
> > > Maybe we need to remember if we have a special file after all, or just ignore
> > > the error on the truncate.
> > 
> > IMHO the correct fix is to not seek and/or truncate the file unless the Fortran
> > semantics require it; that way libgfortran does not need to care whether the
> > file is special or not. As explained in #c23, special files are special in
> > different ways (also different on different OS'es), and trying to enumerate all
> > the ways in which they are special is bound to fail. 
> > 
> > I think Tobias comment #c24 pinpoints the place which needs to be fixed, but
> > unfortunately I haven't had time to look into it.
> 
> Well, we need to make sure that the (very basic) program
> 
>       program main
>       character*10 x
>       open(10,file="foo.dat")
>       write (10,'(A)') '1111'
>       write (10,'(A)') '2222'
>       close (10)
>       open(10,file="foo.dat")
>       write (10,'(A)') '3333'
>       close (10)
>       open(10,file="foo.dat")
>  100  continue
>         read (10,'(A)',end=200) x
>         write (*,'(A)') x
>       goto 100
>  200  continue
>       close(10,status="delete")
>       end
> 
> continues to work as expected: That probably means truncating on
> close and rewind.

I'm not sure about truncation being necessary here, actually. Where we do need
to truncate is 

1) Following a write in access='sequential' mode, when we are not at the end of
the file (that is, we have opened an existing file and are at the beginning, we
have previously used rewind/backspace or such). This might also apply to
formatted writes in access='stream' mode, need to check the standard.

2) Executing an ENDFILE statement.

3) Opening a file with status='replace' (actually, probably not, as a sensible
implementation would be to open() with O_TRUNC instead of a separate
ftruncate() call following open(), and while I'm too lazy to look it up right
now I'm quite sure this is exactly what we already do)

I think the current code goes wrong with pipes because as they are reported to
be of size 0, it is undefined whether we're at the beginning, middle, or end of
the file, and hence we have made some arbitrary choice, and hit_eof() think's
we're not at the end, and tries to truncate. Perhaps we need something more
sophisticated such as some need_to_truncate_after_next_write flag which is set
and tested at appropriate places..

[Bug libfortran/30162] [4.7/4.8 Regression] I/O with named pipes does not work

[tkoenig at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

--- Comment #30 from Thomas Koenig <tkoenig at gcc dot gnu.org> 2012-12-14 23:07:34 UTC ---
This seems to do the trick.

Index: unix.c
===================================================================
--- unix.c      (Revision 194507)
+++ unix.c      (Arbeitskopie)
@@ -344,7 +344,15 @@
 static gfc_offset
 raw_tell (unix_stream * s)
 {
-  return lseek (s->fd, 0, SEEK_CUR);
+  gfc_offset x;
+  x = lseek (s->fd, 0, SEEK_CUR);
+
+  /* Non-seekable files should always be assumed to be at
+     current position.  */
+  if (x == -1 && errno == ESPIPE)
+    x = 0;
+
+  return x;
 }

 static gfc_offset

[Bug libfortran/30162] [4.7/4.8 Regression] I/O with named pipes does not work

[tkoenig at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

Thomas Koenig <tkoenig at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |ASSIGNED
         AssignedTo|unassigned at gcc dot       |tkoenig at gcc dot gnu.org
                   |gnu.org                     |

[Bug libfortran/30162] [4.7/4.8 Regression] I/O with named pipes does not work

[tkoenig at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

--- Comment #31 from Thomas Koenig <tkoenig at gcc dot gnu.org> 2012-12-21 20:50:52 UTC ---
Author: tkoenig
Date: Fri Dec 21 20:50:48 2012
New Revision: 194679

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=194679
Log:
2012-12-21  Thomas Koenig  <tkoenig@gcc.gnu.org>

    PR libfortran/30162
    * io/unix.c (raw_tell):  If the lseek is done on a
    non-seekable file, return 0.


Modified:
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/io/unix.c

[Bug libfortran/30162] [4.7/4.8 Regression] I/O with named pipes does not work

[tkoenig at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

--- Comment #32 from Thomas Koenig <tkoenig at gcc dot gnu.org> 2012-12-22 10:46:42 UTC ---
Author: tkoenig
Date: Sat Dec 22 10:46:37 2012
New Revision: 194694

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=194694
Log:
2012-12-22  Thomas Koenig  <tkoenig@gcc.gnu.org>

    PR libfortran/30162
    Backport from trunk
    * io/unix.c (raw_tell):  If the lseek is done on a
    non-seekable file, return 0.


Modified:
    branches/gcc-4_7-branch/libgfortran/ChangeLog
    branches/gcc-4_7-branch/libgfortran/io/unix.c

[Bug libfortran/30162] [4.7/4.8 Regression] I/O with named pipes does not work

[tkoenig at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

Thomas Koenig <tkoenig at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|                            |FIXED

--- Comment #33 from Thomas Koenig <tkoenig at gcc dot gnu.org> 2012-12-22 10:49:08 UTC ---
Fixed on trunk and 4.7, closing.

[Bug libfortran/30162] [4.7/4.8 Regression] I/O with named pipes does not work

[dominiq at lps dot ens.fr]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

--- Comment #34 from Dominique d'Humieres <dominiq at lps dot ens.fr> 2012-12-22 12:02:52 UTC ---
I still see

 status from open is            0
 status from open is            0
At line 7 of file pr30162_1.f (unit = 20, file = 'np')
Fortran runtime error: Illegal seek

[1]    Exit 2                        a.exe
At line 8 of file pr30162_2.f (unit = 21, file = 'np')
Fortran runtime error: I/O past end of record on unformatted file

on x86_64-apple-darwin10 r194681.

[Bug libfortran/30162] [4.7/4.8 Regression] I/O with named pipes does not work

[tkoenig at netcologne dot de]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

--- Comment #35 from tkoenig at netcologne dot de <tkoenig at netcologne dot de> 2012-12-22 15:38:08 UTC ---
> I still see
>
>   status from open is            0
>   status from open is            0
> At line 7 of file pr30162_1.f (unit = 20, file = 'np')
> Fortran runtime error: Illegal seek
>
> [1]    Exit 2                        a.exe
> At line 8 of file pr30162_2.f (unit = 21, file = 'np')
> Fortran runtime error: I/O past end of record on unformatted file
>
> on x86_64-apple-darwin10 r194681.

This is strange.

Can you strace (or whatever) the code and tell me what lseek()
returns?  Is it really EINVAL like the error message suggests?
If so, this would be strange, because

http://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man2/lseek.2.html

tells me (as a return value)

      [EINVAL]           Whence is not a proper value.

      [ESPIPE]           Fildes is associated with a pipe, socket, or FIFO.

which indicates a bug in the library, but maybe I am looking
at the wrong manpage here.

[Bug libfortran/30162] [4.7/4.8 Regression] I/O with named pipes does not work

[dominiq at lps dot ens.fr]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

--- Comment #36 from Dominique d'Humieres <dominiq at lps dot ens.fr> 2012-12-22 19:13:23 UTC ---
> Can you strace (or whatever) the code and tell me what lseek()
> returns?  Is it really EINVAL like the error message suggests?

The best I can do (without understanding it) is

(1) if I use dtruss (should be equivalent to strace) on a, I get

[macbook] f90/bug% 25210/0x2baef9e:  lseek(0x3, 0x0, 0x1)         = -1 Err#29
25210/0x2baef9e:  lseek(0x3, 0xFFFFFFFFFFFFFFF4, 0x1)         = -1 Err#29

(2) if I use it on b, I get

[macbook] f90/bug% 25218/0x2baf031:  lseek(0x3, 0x0, 0x1)         = -1 Err#29
25218/0x2baf031:  lseek(0x3, 0x4, 0x1)         = -1 Err#29

[Bug libfortran/30162] [4.7/4.8 Regression] I/O with named pipes does not work

[tkoenig at netcologne dot de]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

--- Comment #37 from tkoenig at netcologne dot de <tkoenig at netcologne dot de> 2012-12-22 20:16:07 UTC ---
Hi Dominique,

> The best I can do (without understanding it) is
>
> (1) if I use dtruss (should be equivalent to strace) on a, I get
>
> [macbook] f90/bug% 25210/0x2baef9e:  lseek(0x3, 0x0, 0x1)         = -1 Err#29
> 25210/0x2baef9e:  lseek(0x3, 0xFFFFFFFFFFFFFFF4, 0x1)         = -1 Err#29
>
> (2) if I use it on b, I get
>
> [macbook] f90/bug% 25218/0x2baf031:  lseek(0x3, 0x0, 0x1)         = -1 Err#29
> 25218/0x2baf031:  lseek(0x3, 0x4, 0x1)         = -1 Err#29

That doesn't point to anything suspicious.

Can you tell me what happens if you apply this patch?

Index: io/unix.c
===================================================================
--- io/unix.c   (Revision 194679)
+++ io/unix.c   (Arbeitskopie)
@@ -349,7 +349,7 @@

    /* Non-seekable files should always be assumed to be at
       current position.  */
-  if (x == -1 && errno == ESPIPE)
+  if (x == -1)
      x = 0;

    return x;

[Bug libfortran/30162] [4.7/4.8 Regression] I/O with named pipes does not work

[dominiq at lps dot ens.fr]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

--- Comment #38 from Dominique d'Humieres <dominiq at lps dot ens.fr> 2012-12-22 21:23:07 UTC ---
> Can you tell me what happens if you apply this patch?

The behavior stays unchanged;-(

[Bug libfortran/30162] [4.7/4.8 Regression] I/O with named pipes does not work

[tkoenig at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

Thomas Koenig <tkoenig at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Target|                            |x86_64-apple-darwin10
             Status|RESOLVED                    |REOPENED
         Resolution|FIXED                       |

--- Comment #39 from Thomas Koenig <tkoenig at gcc dot gnu.org> 2012-12-22 22:14:48 UTC ---
Unfortunately, I cannot really debug this without access to a machine
where it fails.

Does this also fail on another BSD-based machine?  Is such a machine
available on the gcc compile farm?

[Bug libfortran/30162] [4.7/4.8 Regression] I/O with named pipes does not work

[dominiq at lps dot ens.fr]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

--- Comment #40 from Dominique d'Humieres <dominiq at lps dot ens.fr> 2012-12-23 14:16:52 UTC ---
> Unfortunately, I cannot really debug this without access to a machine
> where it fails.

Would it be possible to write a simple C test that can be run on the different
platforms in order to check the behavior of lseek?

Note that I'll be off the hook for the next ten days , so don't expect feedback
before the beginning of next year.

[Bug libfortran/30162] [4.7/4.8 Regression] I/O with named pipes does not work

[tkoenig at netcologne dot de]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

--- Comment #41 from tkoenig at netcologne dot de <tkoenig at netcologne dot de> 2012-12-23 15:09:59 UTC ---
Am 23.12.2012 15:16, schrieb dominiq at lps dot ens.fr:
>
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162
>
> --- Comment #40 from Dominique d'Humieres <dominiq at lps dot ens.fr> 2012-12-23 14:16:52 UTC ---
>> Unfortunately, I cannot really debug this without access to a machine
>> where it fails.
>
> Would it be possible to write a simple C test that can be run on the different
> platforms in order to check the behavior of lseek?

The tests that you ran with the patchlet show that the error is not
in setting errno on the lseek.  It has to be somewhere else.

If you still have the time, could you attach the truss output of

program main
   open (10,file="pipe",status="old")
   write (10,'(A)') 'Hello, world!'
end program main

where pipe is a FIFO, and the same for

program read
   character(len=100) c
   open (10,file="pipe", status="old")
   read (10,'(A)') c
   print *,trim(c)
end program read

(first starting read, then write).

[Bug libfortran/30162] [4.7/4.8 Regression] I/O with named pipes does not work

[tkoenig at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

Thomas Koenig <tkoenig at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |NEW
         AssignedTo|tkoenig at gcc dot gnu.org  |unassigned at gcc dot
                   |                            |gnu.org

--- Comment #42 from Thomas Koenig <tkoenig at gcc dot gnu.org> 2012-12-25 15:26:24 UTC ---
I'll try to find a system I have access to where this also fails;
unassigning myself until then.

[Bug libfortran/30162] [4.7/4.8 Regression] I/O with named pipes does not work on Darwin

[dominiq at lps dot ens.fr]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

--- Comment #43 from Dominique d'Humieres <dominiq at lps dot ens.fr> 2013-02-17 20:10:52 UTC ---
I finally had a look at this PR.
First the test on comment #41 works on x86_64-apple-darwin10 r196109:

macbook] f90/bug% mkfifo pipe
[macbook] f90/bug% read_f &
[1] 6998
[macbook] f90/bug% write_f
 Hello, world!
[macbook] f90/bug% 
[1]    Done                          read_f

I have done the following changes to the original test:

[macbook] f90/bug% cat pr30162_1_db.f
      integer status
      integer i
      open(unit=20,file='np',action='write',
     &     form='unformatted',iostat=status);
      print*,'status for write from open is ',status
      do i = 1,5
         print *, "write ", i
         write(20)i
      end do
      end
[macbook] f90/bug% cat pr30162_2_db.f
      integer status
      integer i
      integer val
      open(unit=21,file='np',action='read',
     &     form='unformatted',iostat=status);
      print*,'status for read from open is ',status
      do i = 1,5
         read(21)val
         print*,val
      end do
      end

and I got

[macbook] f90/bug% write_f
 status for write from open is            0
 write            1
At line 8 of file pr30162_1_db.f (unit = 20, file = 'np')
Fortran runtime error: Illegal seek
 status for read from open is            0
At line 8 of file pr30162_2_db.f (unit = 21, file = 'np')
Fortran runtime error: I/O past end of record on unformatted file
[macbook] f90/bug% 
[1]    Exit 2                        read_f

Then I have applied the following patch (based on r194679):

--- ../_clean/libgfortran/io/unix.c    2013-01-14 19:25:10.000000000 +0100
+++ libgfortran/io/unix.c    2013-02-17 20:23:43.000000000 +0100
@@ -336,7 +336,14 @@ raw_write (unix_stream * s, const void *
 static gfc_offset
 raw_seek (unix_stream * s, gfc_offset offset, int whence)
 {
-  return lseek (s->fd, offset, whence);
+  gfc_offset x;
+  x = lseek (s->fd, offset, whence);
+  /* Non-seekable files should always be assumed to be at
+     current position.  */
+  if (x == -1 && errno == ESPIPE)
+    x = 0;
+  return x;
+  /* return lseek (s->fd, offset, whence); */
 }

 static gfc_offset

and now I get

[macbook] f90/bug% write_f
 status for write from open is            0
 write            1
 write            2
 write            3
 write            4
 write            5
 status for read from open is            0
At line 8 of file pr30162_2_db.f (unit = 21, file = 'np')
Fortran runtime error: I/O past end of record on unformatted file
[macbook] f90/bug% 
[1]    Exit 2                        read_f

So the write seems now OK. For the read, dtrace gives

fstat64(0x3, 0x7FFF5FBFD580, 0x50)         = 0 0
fstat64(0x3, 0x7FFF5FBFD5C0, 0x0)         = 0 0
lseek(0x3, 0x0, 0x1)         = -1 Err#29
write(0x1, " status for read from open is            0\n\0", 0x2B)         = 43
0
read(0x3, "\0", 0x4)         = 4 0
read(0x3, "\0", 0x0)         = 0 0
lseek(0x3, 0x4, 0x1)         = -1 Err#29
write(0x2, "At line 8 of file pr30162_2_db.f (unit = 21, file = 'np')\n\0",
0x3A)         = 58 0
write(0x2, "Fortran runtime error: \0", 0x17)         = 23 0
write(0x2, "I/O past end of record on unformatted file\0", 0x2A)         = 42 0
write(0x2, "\n\0", 0x1)         = 1 0
close(0x3)         = 0 0

i.e., 2 errors #29 on lseek. I did not find out how this is connected to the
error "I/O past end of record on unformatted file".

[Bug libfortran/30162] [4.7/4.8 Regression] I/O with named pipes does not work on Darwin

[tkoenig at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

--- Comment #44 from Thomas Koenig <tkoenig at gcc dot gnu.org> 2013-02-18 18:44:02 UTC ---
If unformatted sequential ever worked, it was by chance.

Look at this piece of code:

  /* Seek to the head and overwrite the bogus length with the real
     length.  */

  if (unlikely (sseek (dtp->u.p.current_unit->s, - m - 2 * record_marker, 
               SEEK_CUR) < 0))
    goto io_error;

This works if it happens within a buffer, but you cannot rely on that.

I would therefore suggest to resolve this PR by issuing a well-defined
error if we encounter a pipe on opening.

I'll prepare a patch.

[Bug libfortran/30162] [4.7/4.8 Regression] I/O with named pipes does not work on Darwin

[dominiq at lps dot ens.fr]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

--- Comment #45 from Dominique d'Humieres <dominiq at lps dot ens.fr> 2013-02-18 19:15:43 UTC ---
> ...
> I would therefore suggest to resolve this PR by issuing a well-defined
> error if we encounter a pipe on opening.

Are you suggesting to "fix" the non working named pipes on Darwin by removing
them everywhere?
If yes, isn't it a little bit too radical?

If there are pieces of code working only by chance, would not it be better to
fix them?

[Bug libfortran/30162] [4.7/4.8 Regression] I/O with named pipes does not work on Darwin

[jb at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

--- Comment #46 from Janne Blomqvist <jb at gcc dot gnu.org> 2013-02-18 20:16:28 UTC ---
(In reply to comment #44)
> If unformatted sequential ever worked, it was by chance.
> 
> Look at this piece of code:
> 
>   /* Seek to the head and overwrite the bogus length with the real
>      length.  */
> 
>   if (unlikely (sseek (dtp->u.p.current_unit->s, - m - 2 * record_marker, 
>                SEEK_CUR) < 0))
>     goto io_error;
> 
> This works if it happens within a buffer, but you cannot rely on that.
> 
> I would therefore suggest to resolve this PR by issuing a well-defined
> error if we encounter a pipe on opening.
> 
> I'll prepare a patch.

As I explained in comment #23, this is already handled. When opening a file, we
stat() the fd, and use the buffered I/O functions only if it's a regular file,
otherwise the unbuffered raw I/O functions are used.

[Bug libfortran/30162] [4.7/4.8 Regression] I/O with named pipes does not work on Darwin

[tkoenig at netcologne dot de]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

--- Comment #47 from tkoenig at netcologne dot de <tkoenig at netcologne dot de> 2013-02-18 21:24:31 UTC ---
Am 18.02.2013 21:16, schrieb jb at gcc dot gnu.org:
>> Look at this piece of code:
>> >
>> >   /* Seek to the head and overwrite the bogus length with the real
>> >      length.  */
>> >
>> >   if (unlikely (sseek (dtp->u.p.current_unit->s, - m - 2 * record_marker,
>> >                SEEK_CUR) < 0))
>> >     goto io_error;
>> >
>> >This works if it happens within a buffer, but you cannot rely on that.
>> >
>> >I would therefore suggest to resolve this PR by issuing a well-defined
>> >error if we encounter a pipe on opening.
>> >
>> >I'll prepare a patch.
> As I explained in comment #23, this is already handled. When opening a file, we
> stat() the fd, and use the buffered I/O functions only if it's a regular file,
> otherwise the unbuffered raw I/O functions are used.

This is part of the precipitate, not part of the solution ;-)

For unformatted sequential, we first write a dummy record marker, write
the data and the final record marker, then seek to the first marker
and re-write that. Of course, this fails for an unbuffered fifo.

If we want this to work, we should make sure we always use
_buffered_ I/O for unformatted sequential, setting a maximum
record length on open that we can handle with our buffer.

[Bug libfortran/30162] [4.7/4.8 Regression] I/O with named pipes does not work on Darwin

[tkoenig at netcologne dot de]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

--- Comment #48 from tkoenig at netcologne dot de <tkoenig at netcologne dot de> 2013-02-18 21:29:22 UTC ---
Am 18.02.2013 20:15, schrieb dominiq at lps dot ens.fr:
> Are you suggesting to "fix" the non working named pipes on Darwin by removing
> them everywhere?

Actually, the error happens everywhere, I could reproduce the error with
unformatted sequential on my Linux box.

> If yes, isn't it a little bit too radical?

Formatted sequential, unformatted stream and formatted stream
would continue to work. As BACKSPACE most certainly will not work
on a pipe, I don't see the advantage of using unformatted sequential.

[Bug libfortran/30162] [4.7/4.8 Regression] Unformatted sequential I/O with named pipes does not work

[tkoenig at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

--- Comment #49 from Thomas Koenig <tkoenig at gcc dot gnu.org> 2013-02-18 21:46:58 UTC ---
Here's a proof of concept patch which lets the
test cases "work" due to buffering.

Index: unix.c
===================================================================
--- unix.c      (Revision 195922)
+++ unix.c      (Arbeitskopie)
@@ -995,7 +995,8 @@
   s->file_length = statbuf.st_size;

   /* Only use buffered IO for regular files.  */
-  if (S_ISREG (statbuf.st_mode)
+  if ((S_ISREG (statbuf.st_mode)
+       || S_ISFIFO (statbuf.st_mode))
       && !options.all_unbuffered
       && !(options.unbuffered_preconnected && 
           (s->fd == STDIN_FILENO 

What it doesn't do is handle writes larger than
BUFSIZE, and this limitation is not communicated up
from the stream library to the record length, and also
not the other way.  This would break the abstraction
(but maybe we would not care, would we?)

[Bug libfortran/30162] [4.7/4.8 Regression] Unformatted sequential I/O with named pipes does not work

[jb at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

--- Comment #50 from Janne Blomqvist <jb at gcc dot gnu.org> 2013-02-19 10:59:30 UTC ---
(In reply to comment #47)
> Am 18.02.2013 21:16, schrieb jb at gcc dot gnu.org:
> >> Look at this piece of code:
> >> >
> >> >   /* Seek to the head and overwrite the bogus length with the real
> >> >      length.  */
> >> >
> >> >   if (unlikely (sseek (dtp->u.p.current_unit->s, - m - 2 * record_marker,
> >> >                SEEK_CUR) < 0))
> >> >     goto io_error;
> >> >
> >> >This works if it happens within a buffer, but you cannot rely on that.
> >> >
> >> >I would therefore suggest to resolve this PR by issuing a well-defined
> >> >error if we encounter a pipe on opening.
> >> >
> >> >I'll prepare a patch.
> > As I explained in comment #23, this is already handled. When opening a file, we
> > stat() the fd, and use the buffered I/O functions only if it's a regular file,
> > otherwise the unbuffered raw I/O functions are used.
> 
> This is part of the precipitate, not part of the solution ;-)
> 
> For unformatted sequential, we first write a dummy record marker, write
> the data and the final record marker, then seek to the first marker
> and re-write that. Of course, this fails for an unbuffered fifo.
> 
> If we want this to work, we should make sure we always use
> _buffered_ I/O for unformatted sequential, setting a maximum
> record length on open that we can handle with our buffer.

The buffered I/O functions generally assume that the underlying fd is a regular
file; if you break that be prepared to handle a bunch of subtle fallout. I
tried playing that whack-a-mole game a few years ago until I got tired of it
and restricted buffered I/O to regular files.

Wrt to pipes, one regression we had (I don't remember the PR number) was a
deadlock due to the buffering. E.g. one process writes "hello" to the pipe, the
other one answers "hello, neighbor". The write part could be handled by
executing the FLUSH statement, but I recall there was some problem on the read
side as well, something to do with read() hanging due to there not being any
data available in the pipe when it tried to fill the buffer.

Anyway, I don't think this is something libgfortran can, or should, try to fix.
If the user tries to do something requiring seeking on a special file that
doesn't support seeking, they get what they deserve. The real bug here is that
we try to seek even though the Fortran semantics don't require it (mind that
the complaint in comment #22 which reopened this PR refers to formatted I/O).

[Bug libfortran/30162] [4.7/4.8 Regression] Unformatted sequential I/O with named pipes does not work

[jb at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

--- Comment #51 from Janne Blomqvist <jb at gcc dot gnu.org> 2013-02-19 22:48:50 UTC ---
(In reply to comment #22)
> Revision 180701 removed all checks for special files in unit.c:unit_truncate().
> As a consequence programs compiled with gfortran 4.6.3 and newer cannot to
> write (formatted) to Unix named pipes as ftruncate() fails with EINVAL:
> 
> (64-bit code)
> 
> lseek(3, 0, SEEK_CUR)                   = -1 ESPIPE (Illegal seek)
> write(3, " Hello, world!\n", 15)        = 15
> lseek(3, 0, SEEK_CUR)                   = -1 ESPIPE (Illegal seek)
> ftruncate(3, 18446744073709551615)      = -1 EINVAL (Invalid argument)
> 
> (32-bit code)
> 
> _llseek(3, 0, 0xffba8fa0, SEEK_CUR)     = -1 ESPIPE (Illegal seek)
> write(3, " Hello, world!\n", 15)        = 15
> _llseek(3, 0, 0xffba9080, SEEK_CUR)     = -1 ESPIPE (Illegal seek)
> ftruncate64(3, 18446744073709551615)    = -1 EINVAL (Invalid argument)

This bug (http://stackoverflow.com/questions/11780556/write-to-fifo-named-pipe
, presumably) should be fixed by the patch at

http://gcc.gnu.org/ml/gcc-patches/2013-02/msg00923.html

[Bug libfortran/30162] [4.7/4.8 Regression] Unformatted sequential I/O with named pipes does not work

[jb at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

--- Comment #52 from Janne Blomqvist <jb at gcc dot gnu.org> 2013-02-21 19:03:19 UTC ---
Author: jb
Date: Thu Feb 21 19:03:10 2013
New Revision: 196210

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=196210
Log:
Fix regression when writing formatted sequential to a pipe.

2013-02-21  Janne Blomqvist  <jb@gcc.gnu.org>

    PR libfortran/30162
    * io/open.c (test_endfile): Call stell only if size != 0.
    * io/unix.c (raw_tell): Revert r194679.
    (raw_size): Return size field only for regular files, otherwise 0.

Modified:
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/io/open.c
    trunk/libgfortran/io/unix.c

[Bug libfortran/30162] [4.7/4.8 Regression] Unformatted sequential I/O with named pipes does not work

[jb at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

--- Comment #53 from Janne Blomqvist <jb at gcc dot gnu.org> 2013-02-21 20:13:12 UTC ---
Author: jb
Date: Thu Feb 21 20:13:04 2013
New Revision: 196212

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=196212
Log:
Fix regression when writing formatted sequential to a pipe.

2013-02-21  Janne Blomqvist  <jb@gcc.gnu.org>

    PR libfortran/30162
    * io/open.c (test_endfile): Call stell only if size != 0.
    * io/unix.c (raw_tell): Revert r194694.
    (raw_size): Return size field only for regular files, otherwise 0.

Modified:
    branches/gcc-4_7-branch/libgfortran/ChangeLog
    branches/gcc-4_7-branch/libgfortran/io/open.c
    branches/gcc-4_7-branch/libgfortran/io/unix.c

[Bug libfortran/30162] [4.7/4.8 Regression] Unformatted sequential I/O with named pipes does not work

[jb at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

--- Comment #54 from Janne Blomqvist <jb at gcc dot gnu.org> 2013-02-21 20:39:58 UTC ---
I'd suggest to close the unformatted sequential part of this PR as WONTFIX.
Maybe it worked sometime in the past for small record sizes when buffered IO
was used for pipes, but that was by accident not by design. I think re-enabling
buffering for pipes would open up a can of worms for very little benefit.

A way to "properly" enable this would be to rework the IO library so that we
know how much we're writing before writing the initial record marker (so that
we don't need to seek back and fill in the correct value later), however that
implies a rather major rework of how we do IO, so IMHO the benefit is not worth
it.

[Bug libfortran/30162] [4.7/4.8 Regression] Unformatted sequential I/O with named pipes does not work

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

--- Comment #55 from Tobias Burnus <burnus at gcc dot gnu.org> 2013-02-22 08:54:30 UTC ---
(In reply to comment #54)
> I'd suggest to close the unformatted sequential part of this PR as WONTFIX. [...]

Should we document in http://gcc.gnu.org/onlinedocs/gfortran/ that I/O via
pipes (and other nonseekable devices) is only supported for STREAM and for
FORMATTED+SEQUENTIAL?

[Bug libfortran/30162] [4.7/4.8/4.9 Regression] Unformatted sequential I/O with named pipes does not work

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|4.7.3                       |4.7.4

--- Comment #56 from Richard Biener <rguenth at gcc dot gnu.org> 2013-04-11 07:58:51 UTC ---
GCC 4.7.3 is being released, adjusting target milestone.

[Bug libfortran/30162] Document when sequential I/O with named pipes works

[tkoenig at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

Thomas Koenig <tkoenig at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |documentation
   Target Milestone|4.7.4                       |---
            Summary|[4.7/4.8/4.9 Regression]    |Document when sequential
                   |Unformatted sequential I/O  |I/O with named pipes works
                   |with named pipes does not   |
                   |work                        |
           Severity|normal                      |enhancement

--- Comment #57 from Thomas Koenig <tkoenig at gcc dot gnu.org> ---
Downgrading to a documentation issue.

[Bug libfortran/30162] Document when sequential I/O with named pipes works

[jvdelisle at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=30162

Jerry DeLisle <jvdelisle at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED

--- Comment #58 from Jerry DeLisle <jvdelisle at gcc dot gnu.org> ---
Closing.