[Bug fortran/16970] New: long write times via NFS

[Bug fortran/16970] New: long write times via NFS

[e-frx at arnone dot de dot unifi dot it]

When a g77-compiled standard FORTRAN77 program writes out a file to an
NFS-mounted remote partition, I/O performance dramatically drops.
Same program, on the same machine, writes out the same file in a very short time
when writing to a local partition.
Once created on the local filesystem, the output file can be written to the
NFS-mounted remote partition in a very short time using cp(1) or mv(1).

This behaviour seems g77-specific, since other [proprietary... :( ] FORTRAN
compilers don't have the same issue.

I reproduced the behaviour with several versions of g77, among with:

$ g77 -v 
g77 version 2.95.4 20011002 (Debian prerelease) (from FSF-g77 version 0.5.25 200
10319 (prerelease))
Driving: g77 -v -c -xf77-version /dev/null -xnone
Reading specs from /usr/lib/gcc-lib/i386-linux/2.95.4/specs
gcc version 2.95.4 20011002 (Debian prerelease)
 /usr/lib/gcc-lib/i386-linux/2.95.4/cpp0 -lang-c -v -D__GNUC__=2 -D__GNUC_MINOR_
_=95 -D__ELF__ -D__unix__ -D__i386__ -D__linux__ -D__unix -D__linux -Asystem(pos
ix) -D_LANGUAGE_FORTRAN -traditional -Acpu(i386) -Amachine(i386) -Di386 -D__i386
 -D__i386__ /dev/null /dev/null
GNU CPP version 2.95.4 20011002 (Debian prerelease) (i386 Linux/ELF)
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/include
 /usr/lib/gcc-lib/i386-linux/2.95.4/include
 /usr/include
End of search list.
The following default directories have been omitted from the search path:
 /usr/lib/gcc-lib/i386-linux/2.95.4/../../../../include/g++-3
 /usr/lib/gcc-lib/i386-linux/2.95.4/../../../../i386-linux/include
End of omitted list.
 /usr/lib/gcc-lib/i386-linux/2.95.4/f771 -fnull-version -quiet -dumpbase g77-ver
sion.f -version -fversion -o /tmp/cc7Qj77b.s /dev/null
GNU F77 version 2.95.4 20011002 (Debian prerelease) (i386-linux) compiled by GNU
 C version 2.95.4 20011002 (Debian prerelease).
GNU Fortran Front End version 0.5.25 20010319 (prerelease)
 as -V -Qy -o /tmp/cc6Ymqqe.o /tmp/cc7Qj77b.s
GNU assembler version 2.13.90.0.18 (i386-linux) using BFD version 2.13.90.0.18 2
0030121 Debian GNU/Linux
 ld -m elf_i386 -dynamic-linker /lib/ld-linux.so.2 -o /tmp/ccpZKWDe /tmp/cc6Ymqq
e.o /usr/lib/crt1.o /usr/lib/crti.o /usr/lib/gcc-lib/i386-linux/2.95.4/crtbegin.
o -L/usr/lib/gcc-lib/i386-linux/2.95.4 -lg2c -lm -lgcc -lc -lgcc /usr/lib/gcc-li
b/i386-linux/2.95.4/crtend.o /usr/lib/crtn.o
 /tmp/ccpZKWDe
__G77_LIBF77_VERSION__: 0.5.25 20010319 (prerelease)
@(#)LIBF77 VERSION 19990503
__G77_LIBI77_VERSION__: 0.5.25 20010319 (prerelease)
@(#) LIBI77 VERSION pjw,dmg-mods 19990503
__G77_LIBU77_VERSION__: 0.5.25 20010319 (prerelease)
@(#) LIBU77 VERSION 19980709

$ g77 -v
Reading specs from /usr/lib/gcc-lib/i386-redhat-linux/3.2.3/specs
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man
--infodir=/usr/share/info --enable-shared --enable-threads=posix
--disable-checking --with-system-zlib --enable-__cxa_atexit --host=i386-redhat-linux
Thread model: posix
gcc version 3.2.3 20030502 (Red Hat Linux 3.2.3-34)

$ g77-3.3 -v
Reading specs from /usr/lib/gcc-lib/i486-linux/3.3.3/specs
Configured with: ../src/configure -v
--enable-languages=c,c++,java,f77,objc,ada,treelang --prefix=/usr
--mandir=/usr/share/man --infodir=/usr/share/info
--with-gxx-include-dir=/usr/include/c++/3.3 --enable-shared --with-system-zlib
--enable-nls --without-included-gettext --enable-__cxa_atexit
--enable-clocale=gnu --enable-debug --enable-java-gc=boehm
--enable-java-awt=xlib --enable-objc-gc i486-linux
Thread model: posix
gcc version 3.3.3 20040110 (prerelease) (Debian)


I searched the web about this issue and found other people that asked about this
issue on different mailing lists, but no one seems to have a solution or a work
around.

By digging in the g77 doc I found the ALWAYS_FLUSH dissertation, but it seems
(at least to me) that this macro is no longer defined in libg2c, just to
*increase* NFS I/O performance. So, I'm a bit puzzled...


How to reproduce the strange behaviour:

The following is done on a RedHat Enterprise Linux 3.0 machine having other
RHEL3 filesystems mounted via NFS.

Let's start on the local filesystem:

$ df --si .
Filesystem             Size   Used  Avail Use% Mounted on
/dev/cciss/c0d0p2     139GB  8.1GB  124GB   7% /
$ ls
mywrite.f
$ cat mywrite.f 
      program mywrite

c     constant definitions
c     """"""""""""""""""""
      integer n
      parameter (n=800000)

c     variable declarations
c     """""""""""""""""""""
      integer i
      complex cvar

c     write out unformatted file
c     """"""""""""""""""""""""""
      open (unit=3,file='myout.dat',form='unformatted')
      close(unit=3,status='delete')
      open (unit=3,file='myout.dat',form='unformatted')
      rewind 3

      cvar = (1.23e20,0.31e20)

      write (3) n
      do 10 i = 1, n
        cvar = cvar/1.1
        write (3) cvar
 10   continue

      close (unit=3)

      stop
      end
$ g77 -static -o mywrite mywrite.f 
$ \time -o localtime ./mywrite
$ cat localtime 
3.52user 12.06system 0:15.88elapsed 98%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (33major+16minor)pagefaults 0swaps


Now let's turn to the remote, NFS-mounted filesystem:

$ df --si .
Filesystem             Size   Used  Avail Use% Mounted on
192.168.0.10:/home10b
                       79GB  199MB   75GB   1% /home10b
$ cat /proc/mounts | grep 10b
192.168.0.10:/home10b /home10b nfs rw,v3,rsize=8192,wsize=8192,hard,udp,lock,add
r=192.168.0.10 0 0
$ ls 
mywrite.f
$ g77 -static -o mywrite mywrite.f 
$ \time -o nfstime ./mywrite


Well, after about *one hour*, it's still running: the output file is about *one
half* of its final size and still (slowly) growing...   :-(
I'll add the result when it's finished...


Any ideas?
Why do I obtain unreasonable times (two hours for a 13 Mbyte output file that
can be copied to the same NFS-mounted filesystem in less than one half of a
second with cp)?
Any help will be much appreciated.

TNX

-- 
           Summary: long write times via NFS
           Product: gcc
           Version: unknown
            Status: UNCONFIRMED
          Severity: critical
          Priority: P2
         Component: fortran
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: e-frx at arnone dot de dot unifi dot it
                CC: gcc-bugs at gcc dot gnu dot org
 GCC build triplet: i386-pc-linux-gnu
  GCC host triplet: i386-pc-linux-gnu
GCC target triplet: i386-pc-linux-gnu


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=16970

[Bug libf2c/16970] long write times via NFS

[pinskia at gcc dot gnu dot org]

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|fortran                     |libf2c


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=16970

[Bug fortran/16970] long write times via NFS

[pinskia at gcc dot gnu dot org]

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|critical                    |normal
            Version|unknown                     |3.3


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=16970

[Bug libf2c/16970] long write times via NFS

[e-frx at arnone dot de dot unifi dot it]

------- Additional Comments From e-frx at arnone dot de dot unifi dot it  2004-08-11 07:06 -------
For the record, now I have the NFS-writing time:

$ cat nfstime 
3.29user 11.90system 2:16:53elapsed 0%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (33major+15minor)pagefaults 0swaps

Note also that the two output files are (obviously) identical:

$ diff -sq myout.dat ../mydata/nfstest/
Files myout.dat and ../mydata/nfstest/myout.dat are identical


Important: I managed to reproduce the same strange behaviour (long NFS write
times) with a recent version of g77:

$ g77-ssa -v
Reading specs from /lib/ssa/gcc-lib/i386-redhat-linux-gnu/3.5-tree-ssa/specs
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man
--infodir=/usr/share/info --libdir=/lib/ssa --host=i386-redhat-linux-gnu
--with-gxx-include-dir=/lib/ssa/gcc-lib/i386-redhat-linux-gnu/3.5-tree-ssa/include/c++
--enable-languages=c,c++,java,objc,f77 --enable-shared --enable-threads=posix
--with-system-zlib --enable-__cxa_atexit --disable-multilib
Thread model: posix
gcc version 3.5-tree-ssa 20030617 (Red Hat Linux Rawhide 3.5ssa-47)

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=16970

[Bug libf2c/16970] long write times via NFS

[e-frx at arnone dot de dot unifi dot it]

------- Additional Comments From e-frx at arnone dot de dot unifi dot it  2004-08-11 15:50 -------
Has this explanation something to do with the issue?

See section 5. [performance] in:
http://www.xraylith.wisc.edu/~khan/software/gnu-win32/perf-comments.txt

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=16970

[Bug libf2c/16970] long write times via NFS

[e-frx at arnone dot de dot unifi dot it]

------- Additional Comments From e-frx at arnone dot de dot unifi dot it  2004-08-12 09:22 -------
I also noted that this NFS-performance killer behaviour shows up *only* with
unformatted files. If the program is modified so that the output file is a
formatted (text) file, the writing time drops to a reasonable value, despite the
output file higher size (about 26 Mbyte, rather than 13 Mbyte).

$ df --si .
Filesystem             Size   Used  Avail Use% Mounted on
192.168.0.10:/home10b
                       79GB  550MB   75GB   1% /home10b
$ cat mywrite2.f 
      program mywrite2

c     constant definitions
c     """"""""""""""""""""
      integer n
      parameter (n=800000)

c     variable declarations
c     """""""""""""""""""""
      integer i
      complex cvar

c     write out unformatted file
c     """"""""""""""""""""""""""
      open (unit=3,file='myout2.dat',form='formatted')
      close(unit=3,status='delete')
      open (unit=3,file='myout2.dat',form='formatted')
      rewind 3

      cvar = (1.23e20,0.31e20)

      write (3,*) n
      do 10 i = 1, n
        cvar = cvar/1.1
        write (3,*) cvar
 10   continue

      close (unit=3)

      stop
      end
$ g77 -static -o mywrite2 mywrite2.f
$ \time -o nfstime2 ./mywrite2
$ cat nfstime2 
4.50user 0.17system 0:05.25elapsed 88%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (50major+17minor)pagefaults 0swaps


That's fairly similar to the time I can get on the local filesystem:

$ df --si .
Filesystem             Size   Used  Avail Use% Mounted on
/dev/cciss/c0d0p2     139GB  8.6GB  123GB   7% /
$ \time -o localtime2 ./mywrite2
$ cat localtime2
4.47user 0.30system 0:05.23elapsed 91%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (50major+17minor)pagefaults 0swaps

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=16970

[Bug libf2c/16970] long write times via NFS

[bdavis at gcc dot gnu dot org]

------- Additional Comments From bdavis at gcc dot gnu dot org  2004-08-14 03:19 -------
i can duplicate this on a i686/linux/gnu FC1 system, with the NFS server being
an i686/ FreeBSD system.

the analysis above is probably correct as to the culprit.

 

--bud davis

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=16970

[Bug libf2c/16970] long write times via NFS

[pinskia at gcc dot gnu dot org]

------- Additional Comments From pinskia at gcc dot gnu dot org  2004-08-15 05:45 -------
Confirmed in 3.4.0 but I see that this is fixed in libgfortran with gfortran, a Fortran 95 compiler which 
will come with 3.5.0.

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|                            |FIXED
   Target Milestone|---                         |3.5.0


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=16970

[Bug libf2c/16970] long write times via NFS

[e-frx at arnone dot de dot unifi dot it]

------- Additional Comments From e-frx at arnone dot de dot unifi dot it  2004-08-23 08:27 -------
OK, so this is fixed in the active development tree, with the new gfortran
compiler (that, IIUC, will be used for FORTRAN95, as well as for FORTRAN77, and
for FORTRAN90 perhaps). Is that right?

Is there a simple way to backport the bugfix to
gcc version 3.5-tree-ssa 20030617 (Red Hat Linux Rawhide 3.5ssa-47)
?

I mean: a little patch to apply to this version in order to get the bug resolved
without upgrading to a development version.
Is that asking for too much?

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=16970

[Bug libf2c/16970] long write times via NFS

[bdavis at gcc dot gnu dot org]

------- Additional Comments From bdavis at gcc dot gnu dot org  2004-08-28 03:51 -------
The version of gfortran you would like a backport for is very old. 

gfortran has progressed a lot in the past year, there have probably been
a dozen bug fixes that pertain just to unformatted I/O since then.

building from source is not terribly difficult.  if you use the default
installation location (/usr/local/bin) then the "development" compiler
will co-exist with the versions of gcc installed by your distribution.

A linux binary of gfortran is also available from gfortran.org.

HTH,
bud davis

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=16970