[Bug libfortran/21820] Really, really, horrible IO performance

[Bug libfortran/21820] Really, really, horrible IO performance

[tobi at gcc dot gnu dot org]

------- Comment #14 from tobi at gcc dot gnu dot org  2005-11-01 21:22 -------
(In reply to comment #13)
> The patch from #12 has been committed to mainline.

So should this bug be closed?


-- 

tobi at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |tobi at gcc dot gnu dot org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21820

[Bug libfortran/21820] Really, really, horrible IO performance

[jblomqvi at cc dot hut dot fi]

------- Comment #15 from jblomqvi at cc dot hut dot fi  2005-11-01 22:09 -------
(In reply to comment #14)
> (In reply to comment #13)
> > The patch from #12 has been committed to mainline.
> 
> So should this bug be closed?
> 

It depends on what you consider "really, really horrible IO performance". ;-)
Getting rid of mmap (the patch referred to above) improved performance by a
factor of 25, and I think before I made those measurements there were some
patches committed which made the mmap window bigger or somesuch, improving
performance compared to the situation when the bug was filed. So IMHO we have
made huge improvements.

OTOH we still lose to ifort by a factor of 6 or so. The major reason is that
ifort does bulk transfers for implied do loops, while gfortran doesn't.
Changing the code to use array transfers makes gfortran only about a factor of
1.5 slower than ifort.

Personally, I think we can keep the bug around for reference, but change the
priority to "enhancement". 


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21820

[Bug libfortran/21820] Really, really, horrible IO performance

[Tobias dot Schlueter at physik dot uni-muenchen dot de]

------- Comment #16 from Tobias dot Schlueter at physik dot uni-muenchen dot de  2005-11-01 22:31 -------
Subject: Re:  Really, really, horrible IO performance

jblomqvi at cc dot hut dot fi wrote:
> It depends on what you consider "really, really horrible IO performance". ;-)
> Getting rid of mmap (the patch referred to above) improved performance by a
> factor of 25, and I think before I made those measurements there were some
> patches committed which made the mmap window bigger or somesuch, improving
> performance compared to the situation when the bug was filed. So IMHO we have
> made huge improvements.
> 
> OTOH we still lose to ifort by a factor of 6 or so. The major reason is that
> ifort does bulk transfers for implied do loops, while gfortran doesn't.
> Changing the code to use array transfers makes gfortran only about a factor of
> 1.5 slower than ifort.
> 
> Personally, I think we can keep the bug around for reference, but change the
> priority to "enhancement". 

We could probably close this as a duplicate of PR16339.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21820

[Bug libfortran/21820] Really, really, horrible IO performance

[fxcoudert at gcc dot gnu dot org]

------- Comment #17 from fxcoudert at gcc dot gnu dot org  2005-12-09 09:24 -------
On this one, we now (due to Janne's array I/O transfer) perform better than
Intel and Portland compilers (on i686-linux, ext3 filesystem). I think this can
be closed as dup of 16339. For formatted I/O performance, we have PR 20278.

*** This bug has been marked as a duplicate of 16339 ***


-- 

fxcoudert at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |DUPLICATE


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21820

[Bug libfortran/21820] Really, really, horrible IO performance

[sgk at troutmask dot apl dot washington dot edu]

------- Comment #18 from sgk at troutmask dot apl dot washington dot edu  2005-12-09 15:09 -------
Subject: Re:  Really, really, horrible IO performance

On Fri, Dec 09, 2005 at 09:24:29AM -0000, fxcoudert at gcc dot gnu dot org
wrote:
> On this one, we now (due to Janne's array I/O transfer) perform better
> than Intel and Portland compilers (on i686-linux, ext3 filesystem). I
> think this can be closed as dup of 16339. For formatted I/O performance,
> we have PR 20278.

IIRC, Janne's patch did not deal with IO that involves implied-do
loops.  I'll check this later today.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21820

[Bug libfortran/21820] Really, really, horrible IO performance

[jblomqvi at cc dot hut dot fi]

------- Additional Comments From jblomqvi at cc dot hut dot fi  2005-09-11 11:10 -------
The patch from #12 has been committed to mainline.

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21820

[Bug libfortran/21820] Really, really, horrible IO performance

[jblomqvi at cc dot hut dot fi]

------- Additional Comments From jblomqvi at cc dot hut dot fi  2005-09-04 09:49 -------
Removing mmap improves performance, patch:

http://gcc.gnu.org/ml/gcc-patches/2005-09/msg00176.html

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21820

[Bug libfortran/21820] Really, really, horrible IO performance

[tobi at gcc dot gnu dot org]

------- Additional Comments From tobi at gcc dot gnu dot org  2005-08-16 19:12 -------
I've set this to blocking PR23363, which looks related.

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
OtherBugsDependingO|                            |23363
              nThis|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21820

[Bug libfortran/21820] Really, really, horrible IO performance

[sgk at troutmask dot apl dot washington dot edu]

------- Additional Comments From sgk at troutmask dot apl dot washington dot edu  2005-05-31 16:48 -------
Subject: Re:  Really, really, horrible IO performance

On Tue, May 31, 2005 at 07:38:41AM -0000, fxcoudert at gcc dot gnu dot org wrote:
> First thing: I can reproduce the timings differences on my i386-linux,
> with ext3 filesystem.
> 
> Second thing: Steve, your patch truncates files, which is not correct!
> Given the following modified testcase

Yes, I know.  See http://gcc.gnu.org/ml/fortran/2005-05/msg00420.html.

Note, I played with various combination of mmap flags with 
no notice difference in performance.  Also, I run FreeBSD with
debugging kernel features, which causes a significant increase
over non-debugging kernels.  I've observed that if the file
does not exist, the execution time is on order of 0.01 s.  With
a debugging kernel, I see execution times on the order of 70 s.
With a non-debugging kernel, I see times on the order of 1 s.



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21820

[Bug libfortran/21820] Really, really, horrible IO performance

[fxcoudert at gcc dot gnu dot org]

------- Additional Comments From fxcoudert at gcc dot gnu dot org  2005-05-31 10:46 -------
Additional info: disabling MMAP (#undefining HAVE_MMAP in unix.c) gives far
better performance (in fact, I can see no performance penatly at all).

Perhaps a survey of the cases where mmap induces performance gains and losses
would be usefull...

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21820

[Bug libfortran/21820] Really, really, horrible IO performance

[fxcoudert at gcc dot gnu dot org]

------- Additional Comments From fxcoudert at gcc dot gnu dot org  2005-05-31 07:38 -------
First thing: I can reproduce the timings differences on my i386-linux, with ext3
filesystem.

Second thing: Steve, your patch truncates files, which is not correct! Given the
following modified testcase:

$ cat pr21820.f90
   program kl
   integer i,j,k
   integer, parameter :: m = 1000, n = 300, n2 = 20
   real x(m,n)
   x = 1.e0
   inquire(iolength=k) (x(i,1), i=1, m)
   open(unit=1, file='foo.dat', access='direct', recl=k)
   do j = 1, n
      write(1,rec=j) (x(i,j), i=1, n)
   end do
   close(1)
   open(unit=1, file='foo.dat', access='direct', recl=k)
   do j = 1, n2
      write(1,rec=j) (x(i,j), i=1, n)
   end do
   close(1)
   end program kl

With an unpatched gfortran, I get:
$ ./a.out && ll foo.dat                  
-rw-r--r--  1 fx fx 1.2M May 31 09:38 foo.dat

while your patch yields to:

$ ./a.out && ll foo.dat
-rw-r--r--  1 fx fx 76K May 31 09:35 foo.dat

See the difference in sizes!

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
     Ever Confirmed|                            |1
   Last reconfirmed|0000-00-00 00:00:00         |2005-05-31 07:38:40
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21820

[Bug libfortran/21820] Really, really, horrible IO performance

[sgk at troutmask dot apl dot washington dot edu]

------- Additional Comments From sgk at troutmask dot apl dot washington dot edu  2005-05-30 16:20 -------
Subject: Re:  Really, really, horrible IO performance

On Mon, May 30, 2005 at 03:20:18PM -0000, pinskia at gcc dot gnu dot org wrote:
> 
> I see the same thing on powerpc-darwin but it seems like on freebsd,
> it is actually paging in the memory which seems wrong, I almost want
> to say you should report it to freebsd as their performance bug
> (Yes working around in libgfortran is still a good idea).

It appears to be a faulty assumption in how gfortran uses mmap.
gfortran appears to assume Linux's behavior:

   (From some version of Redhat)
   MAP_SHARED   Share this mapping with all other processes that map this
                object.  Storing to the region is equivalent to writing to
                the file.  The file may not actually be updated until
                msync(2) or munmap(2) are called.

   (From FreeBSD)
   MAP_SHARED   Modifications are shared.

   MAP_NOSYNC   Causes data dirtied via this VM map to be flushed to
                physical media only when necessary (usually by the
                pager) rather than gratuitously.  Typically this pre-
                vents the update daemons from flushing pages dirtied
                through such maps and thus allows efficient sharing of
                memory across unassociated processes using a file-
                backed shared memory map.  (Much snipped, including a
                big warning about ftruncate extending a file).

   (From http://www.opengroup.org/onlinepubs/7990989775/xsh/mmap.html)
   If MAP_SHARED is specified, write references change the underlying object. 

It appears that Linux's MAP_SHARED is equivalent to FreeBSD's
(MAP_SHARED | MAP_NOSYNC).  NetBSD and OpenBSD man pages do not
have the MAP_NOSYNC flag, so they may not be affected.  Note 
POSIX doesn't state anything about when and how the underlying object 
is updated.  POSIX also doesn't include a MAP_NOSYNC flag.

I've verified that the following patch works and improves
performance, but I prefer the original O_TRUNC patch because
O_TRUNC is a POSIX flag and AFAIK most OS's implement it.

Index: unix.c
===================================================================
RCS file: /cvs/gcc/gcc/libgfortran/io/unix.c,v
retrieving revision 1.26
diff -c -p -r1.26 unix.c
*** unix.c      17 May 2005 16:54:51 -0000      1.26
--- unix.c      30 May 2005 16:06:54 -0000
*************** mmap_alloc (unix_stream * s, gfc_offset 
*** 622,628 ****
  
    length = ((where - offset) & page_mask) + 2 * page_size;
  
!   p = mmap (NULL, length, s->prot, MAP_SHARED, s->fd, offset);
    if (p == (char *) MAP_FAILED)
      return FAILURE;
  
--- 622,628 ----
  
    length = ((where - offset) & page_mask) + 2 * page_size;
  
!   p = mmap (NULL, length, s->prot, MAP_SHARED | MAP_NOSYNC, s->fd, offset);
    if (p == (char *) MAP_FAILED)
      return FAILURE;




-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21820

[Bug libfortran/21820] Really, really, horrible IO performance

[pinskia at gcc dot gnu dot org]

------- Additional Comments From pinskia at gcc dot gnu dot org  2005-05-30 15:20 -------
I see the same thing on powerpc-darwin but it seems like on freebsd, it is actually paging in the 
memory which seems wrong, I almost want to say you should report it to freebsd as their performance 
bug (Yes working around in libgfortran is still a good idea).

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21820

[Bug libfortran/21820] Really, really, horrible IO performance

[sgk at troutmask dot apl dot washington dot edu]

------- Additional Comments From sgk at troutmask dot apl dot washington dot edu  2005-05-30 15:15 -------
Subject: Re:  Really, really, horrible IO performance

On Mon, May 30, 2005 at 03:00:41PM -0000, pinskia at gcc dot gnu dot org wrote:
> 
> > Note, the use of O_TRUNC on replacing a file should 
> > not hurt performance on other OS's.
> 
> On powerpc-darwin I am using HFS+.
> Hmm, I think I should drag my firewire drive out and try it on the
> UFS partition.
> 

A little more investigate shows that this may be a mmap problem.

If foo.dat does not exist
   ktrace ./z
   kdump -f ktrace.out | grep mmap | wc -l 
   4

   If foo.dat exists, then I see
   ktrace ./z
   kdump -f ktrace.out | grep mmap | wc -l
   246054

In looking at the ktrace, I see a bunch of segments like

 95157 z        CALL  mmap(0,0x2000,0x3,0x1,0x3,0,0,0)
 95157 z        RET   mmap 1208418304/0x48070000
 95157 z        CALL  munmap(0x48070000,0x2000)
 95157 z        RET   munmap 0
 95157 z        CALL  mmap(0,0x2000,0x3,0x1,0x3,0,0,0)
 95157 z        RET   mmap 1208418304/0x48070000
 95157 z        CALL  munmap(0x48070000,0x2000)
 95157 z        RET   munmap 0
 95157 z        CALL  mmap(0,0x2000,0x3,0x1,0x3,0,0,0)
 95157 z        RET   mmap 1208418304/0x48070000
 95157 z        Events dropped.



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21820

[Bug libfortran/21820] Really, really, horrible IO performance

[pinskia at gcc dot gnu dot org]

------- Additional Comments From pinskia at gcc dot gnu dot org  2005-05-30 15:00 -------
(In reply to comment #3)
> Note, the use of O_TRUNC on replacing a file should 
> not hurt performance on other OS's.

On powerpc-darwin I am using HFS+.
Hmm, I think I should drag my firewire drive out and try it on the UFS partition.

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21820

[Bug libfortran/21820] Really, really, horrible IO performance

[sgk at troutmask dot apl dot washington dot edu]

------- Additional Comments From sgk at troutmask dot apl dot washington dot edu  2005-05-30 14:58 -------
Subject: Re:  Really, really, horrible IO performance

On Mon, May 30, 2005 at 02:49:04PM -0000, pinskia at gcc dot gnu dot org wrote:
> 
> I cannot reproduce this on either powerpc-darwin or i686-pc-linux-gnu,
> what target are you using?
> 

i386-*-freebsd and amd64-*-freebsd.  The i386 system
is using softupdates on a UFS1 filesystem.  The amd64
system is using softupdates on a UFS2 filesystem.  This
certainly can be a OS dependent problem, and I would 
guess it may affect OpenBSD and NetBSD.

Note, the use of O_TRUNC on replacing a file should 
not hurt performance on other OS's.



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21820

[Bug libfortran/21820] Really, really, horrible IO performance

[pinskia at gcc dot gnu dot org]

------- Additional Comments From pinskia at gcc dot gnu dot org  2005-05-30 14:49 -------
I cannot reproduce this on either powerpc-darwin or i686-pc-linux-gnu, what target are you using?

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21820

[Bug libfortran/21820] Really, really, horrible IO performance

[kargl at gcc dot gnu dot org]

------- Additional Comments From kargl at gcc dot gnu dot org  2005-05-30 14:33 -------
Created an attachment (id=8994)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=8994&action=view)
patch to fix problem


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21820