Re: Possible funding of gfortran work

[Mikael Morin <morin-mikael@orange.fr>]

Hello,

some more comments below:

Le 01/06/2023 à 11:18, Andre Vehreschild a écrit :
> Hi Damian, all,
> 
> thank you for your input. I have incorporated most of it. Due to Germany
> stepping out of nuclear use, I have reduced the cites on these to a minimum. I
> don't know anything about the people evaluating the proposal and don't want to
> be rejected just because of ideological reasons. Here is the proposal so far.
> Has any one a good url for the FAST project?
> 
> ---
> - Title:
> 
> GFortran-Improvement
> 
> - Abstract:
> 
> Enable the free gfortran compiler to support contemporary language paradigms.
> 
> - Dependencies (on the project as well as projects that depend on the
>    technology; max 300 words)
> 
> Exemplarily these codes make use of language paradigms or want to, but can not
> due to lack of support in gfortran:
> 
> CP2K: https://www.cp2k.org/ -- Quantum chemistry and solid state physics
> ICAR: https://github.com/ncar/icar -- Simplified atmospheric model
> FEATS: https://github.com/sourceryinstitute/feats -- Asynchronous task
> scheduling framework FAVOR:
> https://www.nrc.gov/reading-rm/doc-collections/nuregs/staff/sr1795/index.html
> -- Reactor security NWChem: https://www.nwchem-sw.org/ -- Computational
> chemistry software FUN3D: https://fun3d.larc.nasa.gov/ -- Computational fluid
> dynamics software from NASA MSC NASTRAN:
> https://simulatemore.mscsoftware.com/category/products/msc-nastran/ --
> Structural engineering software from MSC Software WRF:
> https://www.mmm.ucar.edu/models/wrf -- Weather forecasting software from NCAR
> FAST: ??? -- Nuclear fuel performance software from the U.S. Nuclear Regulatory
> Commission (NRC)
> 
> Some references:
> 
> Mozdzynski, G., Hamrud, M., & Wedi, N. (2015) A Partitioned Global Address
> Space implementation of the European Centre for Medium Range Weather
> Forecasts Integrated Forecasting System. International Journal of High
> Performance Computing Applications.
> 
> Garain, S., Balsara, D. S., & Reid, J. (2015) Comparing Coarray Fortran
> (CAF) with MPI for several structured mesh PDE applications. Journal of
> Computational Physics.
> 
> Preissl, R., Wichmann, N., Long, B., Shalf, J., Ethier, S., & Koniges, A.
> (2011) Multithreaded global address space communication techniques for
> gyrokinetic fusion applications on ultra-scale platforms. In Proceedings of
> 2011 International Conference for High Performance Computing, Networking,
> Storage and Analysis (p. 78). ACM.
> 
> - Why is it critical to fund this project (300 words max)?
> 
> Fortran remains one of the premier language for science, especially for
> high-performance computing and fields like quantum chemistry or
> computational fluid dynamics.
> 
> gfortran is the default Fortran compiler on many Linux systems, and lack of
> features and bugs in gfortran hinder adoption of more modern, safer and more
> efficient language features. The project has been almost entirely
> volunteer-driven so far, but is currently suffering from a lack of active
> developers for larger features. Funding will enable the project to pay some
> gfortran experienced developers to implement some of the missing/incomplete
> features, that are too large to tackle for a single volunteer. Payed developers
> are to contribute a significant part of the features.
> 
The latter paragraph seems more an answer to the question "why is it 
critical for gfortran to get funding" than "why is it critical for a 
funding body to choose gfortran"?

One idea about the latter question:
so that there is always a free solution:
  - for engineers to make best usage of the hardware available to them 
without hassle and spend their time at what they are best: making science
  - for decades-old proven science codes to be adapted to current 
parallel computing architectures

> - Target of the projects in the sense of users (max 300 words)
> 
> This project targets the high-performance computing community, esp. but not
> limited to fluid and thermo dynamics, wheater forecasting and climate models.
> Most if not all of those are Fortran codes. Maintaining them using explicit
> parallelism is tedious and deviates from the domain to address. Other users are
> aerospace, e.g. NASA or SpaceX, as well as aircraft and automotive
> manufacturers.
> 
> - How was the project funded in the past (max 300 words)
> 
> Foundational work on the coarray implementation was funded by Sourcery
> Institute. Small extensions and selected bug fixes have been donated by
> companies having a minor impact only. Most of the work was done on a
> voluntary basis.
> 
There is also Siemens (Tobias, etc) working on OMP and OpenACC.  Not 
sure whether it should me mentioned here.

> - Project goal (max 900 words!):
> 
> * Fortran has a safe and intuitive method for parallel execution,
>    coarrays. There is currently no complete and efficient implementation for
>    multi-core CPUs on a freely available compiler. The goal is to bring
>    the existing, process-based shared memory implementation on a branch
>    into gfortran mainline as a feature for further evaluation and hardening.
> 
I'm not very found of the last part of the sentence, which sounds like 
the project is targetting half-unfinished state.  I understand Thomas 
not willing to engage to something without being sure it can be 
delivered on time.  But the goal is always to be somehow successful; I 
mean, delivering something that doesn't happen to be useful can't be a goal.

I propose this instead:
The goal is to improve and extend on the previous work on a 
process-based shared memory coarray implementation, so that the feature 
can be made available in the next release of gfortran.

This is a goal, not a promise of succes.  And remember that reallocation 
on assignment was made available behind a flag for quite some time 
before being enabled by default.  We could do the same here if the 
feature is not ready yet, so the above is not a great commitment.

> * GFortran's coarrays for distributed memory lack support for data structures
>    provided by modules that have not been compiled with coarray support (or are
>    distributed in binary form only). Research and prototypical implementation
>    in gfortran and the OpenCoarrays library shall be conducted with the goal to
>    find a general and well performing solution. This could become an outstanding
>    feature for a free compiler.
> 
> * Enhance the support for teams and failed images in coarrays to a level where
>    it gets usable. Teams in coarrays allow for grouping workers logically. These
>    then can colaborate without interference or the user needing to take care.
>    The support is rather basic and shall be made usable to a level where the
>    most popular calls work. The failed images concept allows a program to react
>    on one of its processes failing without terminating the program. The present
>    support shall be extended to enable the restart of the process instead of
>    just reporting the fail and then having to quit anyway as it is currently.
> 
> * Enhance standard compliance from Fortran 2003 onwards. Esp. fixing
>    finalization of partially derived types (PDTs) and issues in the associate
>    command.
> 
> * Ensure maintainability of gfortran by cleaning up/refactoring APIs including
>    the scalarizer. Improve the single responsibility pattern's (SRP) use by,
>    e.g., ensuring the parser does no longer parts of the resolve stage. The goal
>    is not only to separate responsibilities but also to get clearer error
>    messages and with that improve user-friendliness.
> 
> - How many FTEs are you requesting?
> 
> - What is the amount of funding you are requesting, approximately?
> 
> - In what timeframe will you perform the activities?
> 
> - Who (maintainer, contributor, organization) would be most qualified to
>    implement this work/receive the support and why?
> ---
> 
> Any input welcome.
> 
> Regards,
> 	Andre
> --
> Andre Vehreschild * Email: vehre ad gmx dot de

Regarding the time estimates, it's a bit difficult as we can't foresee 
at this stage the amount of regressions that will need to be fixed, and 
how difficult they will be.  I'm not even sure that the process of 
picking one regression and fixing it will eventually converge to a 
zero-regression state.  It doesn't mean much, but I expect to spend 
between 3 and 6 months on every item I have proposed.  But I expected 
those items to be discussed, prioritized, and either acknowledged or 
refused by the contributors before.

Mikael