From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 29288 invoked by alias); 17 Apr 2013 14:50:21 -0000 Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Archive: List-Post: List-Help: Sender: gcc-bugs-owner@gcc.gnu.org Received: (qmail 29028 invoked by uid 48); 17 Apr 2013 14:50:17 -0000 From: "burnus at gcc dot gnu.org" To: gcc-bugs@gcc.gnu.org Subject: [Bug fortran/56981] Slow I/O: Unformatted 5x slower, large sys component; formatted slow as well Date: Wed, 17 Apr 2013 14:50:00 -0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: gcc X-Bugzilla-Component: fortran X-Bugzilla-Keywords: missed-optimization X-Bugzilla-Severity: normal X-Bugzilla-Who: burnus at gcc dot gnu.org X-Bugzilla-Status: UNCONFIRMED X-Bugzilla-Priority: P3 X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/ Auto-Submitted: auto-generated Content-Type: text/plain; charset="UTF-8" MIME-Version: 1.0 X-SW-Source: 2013-04/txt/msg01620.txt.bz2 http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56981 --- Comment #5 from Tobias Burnus 2013-04-17 14:50:16 UTC --- (In reply to comment #4) > The reason why gfortran is slow here is that for non-regular files we use > unbuffered I/O. If you write to a regular file instead of /dev/null, you'll > see us doing ~8 KB writes at a time. > > The reason for this is that non-regular files (a.k.a. special files) are > special in many ways wrt seeking. Some allow seeking just fine, some always > return 0, some return an error (and which special files behave in which way is > to some extent different on different OS'es). I do not understand the argument regarding seek. If seek doesn't work - why should there be a problem with buffering but not without? At least with SEQUENTIAL one cannot do without (buffer exceeded or no buffering) and with STREAM no seek should be required. > Also, for special files users often expect non-buffered IO, e.g. they want > output on the terminal directly instead of waiting until the 8 KB buffer fills > up, programs communicating via pipes can deadlock if data sits in the buffers, > etc. But the code should be able to wait until a complete record has been written? That should be rather quick, unless one write a 2GB array. I am not talking about flushing the data only when 8kB are filled or when the file is closed. And doing buffering within a record avoids seeks. > One could of course make "unbuffered" I/O in gfortran really mean "flush > the buffer at the end of each I/O statement" rather than not using a buffer at > all. We should consider this. * * * I have now updated timings with writing to a file. Results for the example in comment 0, but writing to a file ("test.dat", tmpfs). Unformatted is much faster with a normal file, but some others compilers are still significantly faster. And for formatted, all other compilers are significantly faster. ---- Timing in sec ------------------------------------------------ Unformatted Formatted real / user real / user Compiler ----------- ----------- ----------------------------------------- 0.378/0.352 2.815/2.804 GCC 4.8.0 (-Ofast, 20130308, Rev. 196547) 0.307/0.296 1.303/1.288 g95 4.0.3 (g95 0.93!) Aug 17 2010 (-O3) 0.210/0.196 0.555/0.532 Sun Fortran 95 8.3 Linux_i386 2007/05/03 0.208/0.184 0.920/0.888 PathScale 3.2.99 0.176/0.152 2.185/2.168 NAGWare Fortran 5.1 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 0.127/0.125 1.091/1.080 GCC 4.9 (trunk, -Ofast) 0.120/0.118 0.465/0.459 g95 4.0.3 (g95 0.94!) Dec 17 2012 0.136/0.131 0.527/0.524 PathScale EKOPath 4.9.0 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 0.335/0.316 2.866/2.860 GCC 4.7.2 20120920 (Cray Inc.) 0.204/0.188 0.659/0.628 Cray Fortran : Version 8.1.6 0.881/0.328 1.281/0.672 Intel 64, Version 13.1.1.163 0.444/0.432 0.884/0.864 pgf90 12.10-0 -------------------------------------------------------------------