Re: Problem setting buffer size for gfortran ( v 11.2)

[Harald Anlauf <anlauf@gmx.de>]

Hi Denis,

Am 23.02.22 um 17:10 schrieb Bertini, Denis Dr. via Fortran:
> Hi
> Investigating some sporadical performance drop when using Fortran IO on our Lustre
> shared file system, we realized that the buffer size used as default by the gnu fortran
> compiler is 8 kiB only.
> We wanted to change this default and according to the documentation, one has just to set
> the environment variables:
> 
>   GFORTRAN_UNFORMATTED_BUFFER_SIZE
>   GFORTRAN_FORMATTED_BUFFER_SIZE=
>   GFORTRAN_UNBUFFERED_ALL=n
> 
> to increase the buffer size  during runtime.

this works for me on gcc-12 development for the following
test program on Linux/x86:

open(10,file="bigfile")
write(10,*) "data"
close(10)
end

Compile, link, and run under strace with

GFORTRAN_FORMATTED_BUFFER_SIZE=4194304 strace ./a.out

[...]
openat(AT_FDCWD, "bigfile", O_RDWR|O_CREAT|O_CLOEXEC, 0666) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=6, ...}) = 0
mmap(NULL, 4198400, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 
0) = 0x146b69b1c000
write(3, " data\n", 6)                  = 6
ftruncate(3, 6)                         = 0
munmap(0x146b69b1c000, 4198400)         = 0
close(3)                                = 0

> Unfortunately isetting these variables  does not  change the default buffer size.
> I tried with gfortran 8, 10, and the new 11.2.
> Looking a the code, i found in
>   io/unix.c
>   that the default is set to the following value,
> 
> static const int FORMATTED_BUFFER_SIZE_DEFAULT = 8192;
> static const int UNFORMATTED_BUFFER_SIZE_DEFAULT = 128*1024;
> 
> Of course, changing the default value and recompiling works and change accordingly the buffer size
> It should nevertheless be possible to change these values without recompiling.
> Is there something one should be aware of when setting these variables?

You said you're using Lustre.  I guess you use the default
blocksize (4MB).  Does your Lustre client support mmap?
What does strace report?

I remember I once saw a system where we had to disable the use
of mmap in a library.

Cheers,
Harald

> Best regards,
> Denis
> 
> ---------
> Denis Bertini
> Abteilung: CIT
> Ort: SB3 2.265a
> 
> Tel: +49 6159 71 2240
> Fax: +49 6159 71 2986
> E-Mail: d.bertini@gsi.de
> 
> GSI Helmholtzzentrum für Schwerionenforschung GmbH
> Planckstraße 1, 64291 Darmstadt, Germany, www.gsi.de
> 
> Commercial Register / Handelsregister: Amtsgericht Darmstadt, HRB 1528
> Managing Directors / Geschäftsführung:
> Professor Dr. Paolo Giubellino, Dr. Ulrich Breuer, Jörg Blaurock
> Chairman of the GSI Supervisory Board / Vorsitzender des GSI-Aufsichtsrats:
> Ministerialdirigent Dr. Volkmar Dietz
>