[Bug fortran/22519] Memory and binary disk layout disagree for real*10

[Bug fortran/22519] Memory and binary disk layout disagree for real*10

[fxcoudert at gcc dot gnu dot org]

------- Comment #5 from fxcoudert at gcc dot gnu dot org  2005-10-03 10:36 -------
(In reply to comment #4)
> We need to settle what kind of disk image we want for real(kind=10)
> before resolving this for complex.

I am strongly in favour of real(kind=10) being written as 12 bytes on disk.
This will make life much easier for all of us, and might speed things up.


-- 

fxcoudert at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
     Ever Confirmed|0                           |1
   Last reconfirmed|0000-00-00 00:00:00         |2005-10-03 10:36:38
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=22519

[Bug fortran/22519] Memory and binary disk layout disagree for real*10

[jblomqvi at cc dot hut dot fi]

------- Comment #6 from jblomqvi at cc dot hut dot fi  2005-11-05 18:12 -------
(In reply to comment #5)
> (In reply to comment #4)
> > We need to settle what kind of disk image we want for real(kind=10)
> > before resolving this for complex.
> 
> I am strongly in favour of real(kind=10) being written as 12 bytes on disk.
> This will make life much easier for all of us, and might speed things up.
> 

The patch I have posted for PR 24174 and PR 24305 writes the padded size (i.e.
12 bytes on x86, and AFAIK 16 bytes on x86-64 and IA-64). Now that we have
gotten rid of mmap and support bulk transfers of arrays, this stuff actually
makes a difference.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=22519

[Bug fortran/22519] Memory and binary disk layout disagree for real*10

[fxcoudert at gcc dot gnu dot org]

------- Comment #7 from fxcoudert at gcc dot gnu dot org  2006-02-14 16:27 -------
I think this one is fixed, isn't it?


-- 

fxcoudert at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |fxcoudert at gcc dot gnu dot
                   |                            |org
             Status|NEW                         |SUSPENDED


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=22519

[Bug fortran/22519] Memory and binary disk layout disagree for real*10

[jb at gcc dot gnu dot org]

------- Comment #8 from jb at gcc dot gnu dot org  2006-03-04 16:02 -------
(In reply to comment #7)
> I think this one is fixed, isn't it?
> 

Yes, I think so too. Fixed.


-- 

jb at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|SUSPENDED                   |RESOLVED
         Resolution|                            |FIXED


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=22519

[Bug fortran/22519] Memory and binary disk layout disagree for real*10

[pinskia at gcc dot gnu dot org]

-- 

pinskia at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |4.1.0


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=22519

[Bug fortran/22519] New: Memory and binary disk layout disagree for real*10

[schnetter at aei dot mpg dot de]

The memory layout of a real*10 variable on an i386 processor requires 12 or 16 
bytes per variable, depending on the flag -m128bit-long-double.  The disk 
layout for unformatted (binary) writing occupies 10 bytes per variable. 
 
This has several disadvantages.  It is different from I would naively expect 
(which does not mean much), and it is different from binary C I/O (which could 
also be ignored).  However, it is likely also rather inefficient if large 
arrays are written to disk, because a single call to write() or fwrite() now 
has to be split up into one call per array element, so that the padding is 
removed. 
 
It is also questionable whether having a real*10 variable occupy 12 or 16 
bytes in memory is what people expect; again, this could be ignored since the 
standard permits this.  However, this is also very different from what other 
compilers do.

-- 
           Summary: Memory and binary disk layout disagree for real*10
           Product: gcc
           Version: 4.1.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: fortran
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: schnetter at aei dot mpg dot de
                CC: gcc-bugs at gcc dot gnu dot org
 GCC build triplet: i686-pc-linux-gnu
  GCC host triplet: i686-pc-linux-gnu
GCC target triplet: i686-pc-linux-gnu


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=22519

[Bug fortran/22519] Memory and binary disk layout disagree for real*10

[pinskia at gcc dot gnu dot org]

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|normal                      |enhancement
  GCC build triplet|i686-pc-linux-gnu           |
   GCC host triplet|i686-pc-linux-gnu           |
 GCC target triplet|i686-pc-linux-gnu           |i?86-*-*, x86_64-*-*


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=22519

[Bug fortran/22519] Memory and binary disk layout disagree for real*10

[tkoenig at gcc dot gnu dot org]

------- Additional Comments From tkoenig at gcc dot gnu dot org  2005-07-17 19:45 -------
I don't think the timing issue is valid.

Look at these benchmarks:

The first one simulates copying 12-byte values to 10-bit values,
the second one a compact memcpy of a larger field.

$ cat foo.c
#include <string.h>

#define NELEM 10000000

#define SA 12
#define SB 10

void foo(char *a, char *b, int n);

int main()
{
    char *a, *b;
    int n;
    a = malloc(NELEM*SA);
    b = malloc(NELEM*SB);
    for (n=0; n<10; n++)
        foo(a,b,NELEM);

    return 0;
}

void foo(char *a, char *b, int n)
{
    int i;
    for (i=0; i<n; i++)
    {
        memcpy(a, b, 10);
        a += SA;
        b += SB;
    }
}
$ gcc -O3 foo.c
$ time ./a.out

real    0m2.628s
user    0m2.523s
sys     0m0.096s
$ cat foo2.c
#include <string.h>

#define NELEM 10000000

#define SA 12

void foo(char *a, char *b, int n);

int main()
{
    char *a, *b;
    int n;
    a = malloc(NELEM*SA);
    b = malloc(NELEM*SA);
    for (n=0; n<10; n++)
        memcpy(a, b, NELEM*SA);

    return 0;
}
$ gcc foo2.c
foo2.c: In function 'main':
foo2.c:13: warning: incompatible implicit declaration of built-in function 'malloc'
$ time ./a.out

real    0m2.876s
user    0m2.777s
sys     0m0.093s

We also have slow disk I/O to deal with.

This is on i686-pc-linux-gnu.  Timings on other systems may differ,
of course.

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=22519

[Bug fortran/22519] Memory and binary disk layout disagree for real*10

[tkoenig at gcc dot gnu dot org]

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |tkoenig at gcc dot gnu dot
                   |                            |org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=22519

[Bug fortran/22519] Memory and binary disk layout disagree for real*10

[schnetter at uni-tuebingen dot de]

------- Additional Comments From schnetter at uni-tuebingen dot de  2005-07-17 20:01 -------
My argument -- which I had in my head, but didn't put down -- went as follows: 
ultimately, things have to be written by a system call, and a system call is 
expensive.  (One system call per array element is out of the question.)  Thus 
we either dump the whole array to disk, byte for byte as it is in memory, or 
we have to transform it into a temporary buffer and then write this buffer.  
C's fwrite uses a buffer (probably), fprintf does for sure. 
 
If you remove the padding, then you have to allocate and deallocate these 
buffers and pack or unpack the array elements explicitly.  Depending on your 
I/O library, that may be a lot of work.  And in the end you are still 
incompatibly to C. 
 
(I'm afraid we may be going to repeat a discussion here that was already held 
on the mailing list.) 
 

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=22519

[Bug fortran/22519] Memory and binary disk layout disagree for real*10

[tkoenig at gcc dot gnu dot org]

------- Additional Comments From tkoenig at gcc dot gnu dot org  2005-07-18 14:35 -------
(In reply to comment #2)


> ultimately, things have to be written by a system call, and a system call is 
> expensive.  (One system call per array element is out of the question.)

With the current implementation (which may not be the best for performance)
we use memory-mapped I/O.  It then becomes a question of how efficient
you transfer your data into the memory you mmapped.

> Thus 
> we either dump the whole array to disk, byte for byte as it is in memory, or 
> we have to transform it into a temporary buffer and then write this buffer. 

Agreed.
 
> C's fwrite uses a buffer (probably), fprintf does for sure. 
>  
> If you remove the padding, then you have to allocate and deallocate these 
> buffers

If you use read() instead of mmap(), it is enough to allocate a
single buffer, which can be reused.

>and pack or unpack the array elements explicitly.  Depending on your 
> I/O library, that may be a lot of work.

I tried to show in my little benchmark above that packing things may
be a little less work than actually copying the whole thing.  This doesn't
take disk I/O into account, which is also fairly slow.

> And in the end you are still 
> incompatibly to C. 

Fortran unformatted I/O is incompatible to C anyway, because of the record
markers, this is not really a big deal.

I think a few benchmarks could help there.

> (I'm afraid we may be going to repeat a discussion here that was already held 
> on the mailing list.) 

If there's a PR, I think it's better if people can read the discussion in its
audit trail.

The biggest problems in I/O performance at the moment are the overhead
of individual function calls for unformatted array I/O.  I think the best
way to address that would be to let the library use array descriptors.
For the general case, this would require pack / unpack operations, so
a conversion to 10 bytes for numbers aligned to 16 bytes would only
help.

Hmm....

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=22519

[Bug fortran/22519] Memory and binary disk layout disagree for real*10

[tkoenig at gcc dot gnu dot org]

------- Additional Comments From tkoenig at gcc dot gnu dot org  2005-08-16 19:52 -------
We need to settle what kind of disk image we want for real(kind=10)
before resolving this for complex.

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
OtherBugsDependingO|                            |23419
              nThis|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=22519