Re: Problem setting buffer size for gfortran ( v 11.2)

[Harald Anlauf <anlauf@gmx.de>]

Hi Denis,

Am 23.02.22 um 21:28 schrieb Bertini, Denis Dr. via Fortran:
> Hi Harald,
> 
> In attachment you will find a typical strace output when running with
> 
> gfortran8 on lustre filsystem.

unfortunately I do not see any attachment to your mail.

Note: GFORTRAN_FORMATTED_BUFFER_SIZE & friends was added in 10.1,
and possibly backported to some 9.y, but *not* to 8.x.

> As i said for gfortran version 8.x, 10.x, 11.2 setting the runtime
> 
> env. variable to change buffer size did not work for me as you could
> 
> see in the strace output.
> 
> You could also see that "mmap" is used by the lustre client.
> 
> The optimal blocksize given back by the STAT() routine indeed
> 
> favor a 4 MiB buffer size.

It might make sense to show your issue to the vendor of your system.

Parallel file systems - not only Lustre - require special handling
not only to get optimum performance, but also correct behavior.
(E.g. the HDF5 library is known to hit bugs in our Lustre.)

Cheers,
Harald

> 
> Best
> 
> Denis
> 
> 
> ________________________________
> From: Harald Anlauf <anlauf@gmx.de>
> Sent: Wednesday, February 23, 2022 7:26 PM
> To: Bertini, Denis Dr.; fortran@gcc.gnu.org
> Subject: Re: Problem setting buffer size for gfortran ( v 11.2)
> 
> Hi Denis,
> 
> Am 23.02.22 um 17:10 schrieb Bertini, Denis Dr. via Fortran:
>> Hi
>> Investigating some sporadical performance drop when using Fortran IO on our Lustre
>> shared file system, we realized that the buffer size used as default by the gnu fortran
>> compiler is 8 kiB only.
>> We wanted to change this default and according to the documentation, one has just to set
>> the environment variables:
>>
>>    GFORTRAN_UNFORMATTED_BUFFER_SIZE
>>    GFORTRAN_FORMATTED_BUFFER_SIZE=
>>    GFORTRAN_UNBUFFERED_ALL=n
>>
>> to increase the buffer size  during runtime.
> 
> this works for me on gcc-12 development for the following
> test program on Linux/x86:
> 
> open(10,file="bigfile")
> write(10,*) "data"
> close(10)
> end
> 
> Compile, link, and run under strace with
> 
> GFORTRAN_FORMATTED_BUFFER_SIZE=4194304 strace ./a.out
> 
> [...]
> openat(AT_FDCWD, "bigfile", O_RDWR|O_CREAT|O_CLOEXEC, 0666) = 3
> fstat(3, {st_mode=S_IFREG|0644, st_size=6, ...}) = 0
> mmap(NULL, 4198400, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> 0) = 0x146b69b1c000
> write(3, " data\n", 6)                  = 6
> ftruncate(3, 6)                         = 0
> munmap(0x146b69b1c000, 4198400)         = 0
> close(3)                                = 0
> 
>> Unfortunately isetting these variables  does not  change the default buffer size.
>> I tried with gfortran 8, 10, and the new 11.2.
>> Looking a the code, i found in
>>    io/unix.c
>>    that the default is set to the following value,
>>
>> static const int FORMATTED_BUFFER_SIZE_DEFAULT = 8192;
>> static const int UNFORMATTED_BUFFER_SIZE_DEFAULT = 128*1024;
>>
>> Of course, changing the default value and recompiling works and change accordingly the buffer size
>> It should nevertheless be possible to change these values without recompiling.
>> Is there something one should be aware of when setting these variables?
> 
> You said you're using Lustre.  I guess you use the default
> blocksize (4MB).  Does your Lustre client support mmap?
> What does strace report?
> 
> I remember I once saw a system where we had to disable the use
> of mmap in a library.
> 
> Cheers,
> Harald
> 
>> Best regards,
>> Denis
>>
>> ---------
>> Denis Bertini
>> Abteilung: CIT
>> Ort: SB3 2.265a
>>
>> Tel: +49 6159 71 2240
>> Fax: +49 6159 71 2986
>> E-Mail: d.bertini@gsi.de
>>
>> GSI Helmholtzzentrum für Schwerionenforschung GmbH
>> Planckstraße 1, 64291 Darmstadt, Germany, www.gsi.de
>>
>> Commercial Register / Handelsregister: Amtsgericht Darmstadt, HRB 1528
>> Managing Directors / Geschäftsführung:
>> Professor Dr. Paolo Giubellino, Dr. Ulrich Breuer, Jörg Blaurock
>> Chairman of the GSI Supervisory Board / Vorsitzender des GSI-Aufsichtsrats:
>> Ministerialdirigent Dr. Volkmar Dietz
>>
> 
>