Setting up a proper modelling environment in Cygwin

Setting up a proper modelling environment in Cygwin

[Lawrence Bernardo]

Dear Cygwin users and developers,

I am trying to run some models based on the ROMS ocean modeling system. And
although I mainly run such models on Linux supercomputing systems, I
sometimes use Cygwin to make test runs. However, whenever I install Cygwin
on a new PC, I often find that my set up on an older PC I had been using
doesn't work properly. After looking deeper into the issues, it seems that
proper compilation requires that the same fortran compiler (gfortran in
this case) be used to compile the necessary components.

And as far as I understand, these components would mainly be:
- zlib
- hdf5
- netcdf
- netcdf-fortran
- openmpi

As the gfortran compiler version number used for these different components
is not indicated whenever I run cygwin setup-x86_64.exe, I am left to guess
which version numbers for each to select. Of course, another option would
be to accept the defaults for some components, and then compile my own
version of say, netcdf, but this attempt almost always fails, and I'm not
sure if this is due to conflicts with the already existing cygwin setup.

Given the issues above, would anyone have any suggestions or
recommendations on how to best achieve the setup I would like to have? I
will greatly appreciate any help.

Best regards,
Lawrence

Re: Setting up a proper modelling environment in Cygwin

[Eliot Moss]

On 9/10/2020 10:37 PM, Lawrence Bernardo via Cygwin wrote:
> Dear Cygwin users and developers,
> 
> I am trying to run some models based on the ROMS ocean modeling system. And
> although I mainly run such models on Linux supercomputing systems, I
> sometimes use Cygwin to make test runs. However, whenever I install Cygwin
> on a new PC, I often find that my set up on an older PC I had been using
> doesn't work properly. After looking deeper into the issues, it seems that
> proper compilation requires that the same fortran compiler (gfortran in
> this case) be used to compile the necessary components.
> 
> And as far as I understand, these components would mainly be:
> - zlib
> - hdf5
> - netcdf
> - netcdf-fortran
> - openmpi
> 
> As the gfortran compiler version number used for these different components
> is not indicated whenever I run cygwin setup-x86_64.exe, I am left to guess
> which version numbers for each to select. Of course, another option would
> be to accept the defaults for some components, and then compile my own
> version of say, netcdf, but this attempt almost always fails, and I'm not
> sure if this is due to conflicts with the already existing cygwin setup.
> 
> Given the issues above, would anyone have any suggestions or
> recommendations on how to best achieve the setup I would like to have? I
> will greatly appreciate any help.

Not sure if anyone has responded yet ...

You can:

- Install the specific version of gfortran (gcc suite) that you need.
- Build the other things from sources in Cygwin, using the source packages.

There might be rough edges if there are cygwin-specific adjustments that have been
made and they don't quite match the version you want.

There is also the cygwin time machine that you can use to get any old versions.
(I've not used it, but know it exists and sometimes helps folks with situations
like this.)

Regards - Eliot Moss

Re: Setting up a proper modelling environment in Cygwin

[Marco Atzeri]

On 11.09.2020 04:37, Lawrence Bernardo via Cygwin wrote:
> Dear Cygwin users and developers,
> 
> I am trying to run some models based on the ROMS ocean modeling system. And
> although I mainly run such models on Linux supercomputing systems, I
> sometimes use Cygwin to make test runs. However, whenever I install Cygwin
> on a new PC, I often find that my set up on an older PC I had been using
> doesn't work properly. After looking deeper into the issues, it seems that
> proper compilation requires that the same fortran compiler (gfortran in
> this case) be used to compile the necessary components.
> 
> And as far as I understand, these components would mainly be:
> - zlib
> - hdf5
> - netcdf
> - netcdf-fortran
> - openmpi
> 
> As the gfortran compiler version number used for these different components
> is not indicated whenever I run cygwin setup-x86_64.exe, I am left to guess
> which version numbers for each to select. Of course, another option would
> be to accept the defaults for some components, and then compile my own
> version of say, netcdf, but this attempt almost always fails, and I'm not
> sure if this is due to conflicts with the already existing cygwin setup.
> 
> Given the issues above, would anyone have any suggestions or
> recommendations on how to best achieve the setup I would like to have? I
> will greatly appreciate any help.
> 
> Best regards,
> Lawrence
> --


for what I am aware the problem is only present with
fortran modules if you are using them.
C interface are not involved

zlib should not be involved as it has no fortran specific modules
same for hdf5.

Now the current gfortran has

$ zcat 
/usr/lib/gcc/x86_64-pc-cygwin/9.3.0/finclude/ieee_arithmetic.mod|head -n 1
GFORTRAN module version '15' created from ieee_arithmetic.F90

openmpi was built with last gcc so it is aligned

$ zcat /usr/lib/mpi.mod |head -n 1
GFORTRAN module version '15' created from mpi-ignore-tkr.F90

netcdf-fortran needs a rebuild

$ zcat /usr/include/netcdf.mod | head -n 1
GFORTRAN module version '14' created from netcdf4.f90

Give me some days for an update of the package
And I will need to re-build for test gcc 10 also,
but it is less urgent


Regards
Marco

Re: Setting up a proper modelling environment in Cygwin

[Brian Inglis]

On 2020-09-11 08:59, Marco Atzeri via Cygwin wrote:
> On 11.09.2020 04:37, Lawrence Bernardo via Cygwin wrote:
>> Dear Cygwin users and developers,
>>
>> I am trying to run some models based on the ROMS ocean modeling system. And
>> although I mainly run such models on Linux supercomputing systems, I
>> sometimes use Cygwin to make test runs. However, whenever I install Cygwin
>> on a new PC, I often find that my set up on an older PC I had been using
>> doesn't work properly. After looking deeper into the issues, it seems that
>> proper compilation requires that the same fortran compiler (gfortran in
>> this case) be used to compile the necessary components.
>>
>> And as far as I understand, these components would mainly be:

>> - netcdf-fortran

>> As the gfortran compiler version number used for these different components
>> is not indicated whenever I run cygwin setup-x86_64.exe, I am left to guess
>> which version numbers for each to select. Of course, another option would
>> be to accept the defaults for some components, and then compile my own
>> version of say, netcdf, but this attempt almost always fails, and I'm not
>> sure if this is due to conflicts with the already existing cygwin setup.
>>
>> Given the issues above, would anyone have any suggestions or
>> recommendations on how to best achieve the setup I would like to have? I
>> will greatly appreciate any help.

> netcdf-fortran needs a rebuild
> 
> $ zcat /usr/include/netcdf.mod | head -n 1
> GFORTRAN module version '14' created from netcdf4.f90

Worse, setup.ini somehow now contains component package netcdf-fortran-debuginfo
but not netcdf-fortran itself!

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.
[Data in IEC units and prefixes, physical quantities in SI.]

Re: Setting up a proper modelling environment in Cygwin

[ASSI]

Marco Atzeri via Cygwin writes:
> Now the current gfortran has
>
> $ zcat
> /usr/lib/gcc/x86_64-pc-cygwin/9.3.0/finclude/ieee_arithmetic.mod|head
> -n 1
> GFORTRAN module version '15' created from ieee_arithmetic.F90
>
> openmpi was built with last gcc so it is aligned

That points to some larger problem with the way dependencies are
extracted for packages using Fortran modules.

> Give me some days for an update of the package
> And I will need to re-build for test gcc 10 also,
> but it is less urgent

No.  :-)


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Terratec KOMPLEXER:
http://Synth.Stromeko.net/Downloads.html#KomplexerWaves

Re: Setting up a proper modelling environment in Cygwin

[Marco Atzeri]

On 12.09.2020 07:39, Brian Inglis wrote:
> On 2020-09-11 08:59, Marco Atzeri via Cygwin wrote:
>> On 11.09.2020 04:37, Lawrence Bernardo via Cygwin wrote:
>>> Dear Cygwin users and developers,
>>>
>>> I am trying to run some models based on the ROMS ocean modeling system. And
>>> although I mainly run such models on Linux supercomputing systems, I
>>> sometimes use Cygwin to make test runs. However, whenever I install Cygwin
>>> on a new PC, I often find that my set up on an older PC I had been using
>>> doesn't work properly. After looking deeper into the issues, it seems that
>>> proper compilation requires that the same fortran compiler (gfortran in
>>> this case) be used to compile the necessary components.
>>>
>>> And as far as I understand, these components would mainly be:
> 
>>> - netcdf-fortran
> 
>>> As the gfortran compiler version number used for these different components
>>> is not indicated whenever I run cygwin setup-x86_64.exe, I am left to guess
>>> which version numbers for each to select. Of course, another option would
>>> be to accept the defaults for some components, and then compile my own
>>> version of say, netcdf, but this attempt almost always fails, and I'm not
>>> sure if this is due to conflicts with the already existing cygwin setup.
>>>
>>> Given the issues above, would anyone have any suggestions or
>>> recommendations on how to best achieve the setup I would like to have? I
>>> will greatly appreciate any help.
> 
>> netcdf-fortran needs a rebuild
>>
>> $ zcat /usr/include/netcdf.mod | head -n 1
>> GFORTRAN module version '14' created from netcdf4.f90
> 
> Worse, setup.ini somehow now contains component package netcdf-fortran-debuginfo
> but not netcdf-fortran itself!
> 

of course they are called a bit differently


libnetcdf-fortran-devel                 4.5.2-1
libnetcdf-fortran_7                     4.5.2-1

Re: Setting up a proper modelling environment in Cygwin

[Marco Atzeri]

On 12.09.2020 08:13, ASSI wrote:
> Marco Atzeri via Cygwin writes:
>> Now the current gfortran has
>>
>> $ zcat
>> /usr/lib/gcc/x86_64-pc-cygwin/9.3.0/finclude/ieee_arithmetic.mod|head
>> -n 1
>> GFORTRAN module version '15' created from ieee_arithmetic.F90
>>
>> openmpi was built with last gcc so it is aligned
> 
> That points to some larger problem with the way dependencies are
> extracted for packages using Fortran modules.
> 
>> Give me some days for an update of the package
>> And I will need to re-build for test gcc 10 also,
>> but it is less urgent
> 
> No.  :-)

No as "I do not need for GCC 10" or
no "it is really urgent to build a test version to match the
gcc test version" ?

$ zcat 
/usr/lib/gcc/x86_64-pc-cygwin/10.2.0/finclude/ieee_arithmetic.mod| head -n 1
GFORTRAN module version '15' created from ieee_arithmetic.F90

ok, it is "I do not need for GCC 10"

> Regards,
> Achim.
> 

Regards
Marco

Re: Setting up a proper modelling environment in Cygwin

[Achim Gratz]

Marco Atzeri via Cygwin writes:
> On 12.09.2020 08:13, ASSI wrote:
> No as "I do not need for GCC 10" or
> no "it is really urgent to build a test version to match the
> gcc test version" ?

The Fortran module version has last been incremented at gcc8.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Q+, Q and microQ:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: Setting up a proper modelling environment in Cygwin

[Achim Gratz]

Marco Atzeri via Cygwin writes:
> On 11.09.2020 04:37, Lawrence Bernardo via Cygwin wrote:
> for what I am aware the problem is only present with
> fortran modules if you are using them.
> Now the current gfortran has
>
> $ zcat
> /usr/lib/gcc/x86_64-pc-cygwin/9.3.0/finclude/ieee_arithmetic.mod|head
> -n 1
> GFORTRAN module version '15' created from ieee_arithmetic.F90
>
> openmpi was built with last gcc so it is aligned
>
> $ zcat /usr/lib/mpi.mod |head -n 1
> GFORTRAN module version '15' created from mpi-ignore-tkr.F90

If these are dependent on the compiler version / module ABI version,
then they should be moved to some version dependent directory, then.  I
have no idea yet how to name that.  The Linux distributions I've looked
at have a very complicated folder structure that caters to the HPC folks
that want multiple parallel installed versions.

With gcc10 the gfortran package will provide "gfm15" and that number
will be incremented in lockstep with the Fortran module version so that
setup can figure out if things match.  In order for this to work any
packages delivering Fortran modules would need to have a manual require
on that.

> netcdf-fortran needs a rebuild
>
> $ zcat /usr/include/netcdf.mod | head -n 1
> GFORTRAN module version '14' created from netcdf4.f90

Certainly none of these files should live in /usr/include.

These seem to be the only two packages in the distribution that provide
Fortran modules, so the impact is currently limited.



Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: Setting up a proper modelling environment in Cygwin

[Marco Atzeri]

On 12.09.2020 08:13, ASSI wrote:
> Marco Atzeri via Cygwin writes:
>> Now the current gfortran has
>>
>> $ zcat
>> /usr/lib/gcc/x86_64-pc-cygwin/9.3.0/finclude/ieee_arithmetic.mod|head
>> -n 1
>> GFORTRAN module version '15' created from ieee_arithmetic.F90
>>
>> openmpi was built with last gcc so it is aligned
> 
> That points to some larger problem with the way dependencies are
> extracted for packages using Fortran modules.

I doubt we have any dependencies from Fortran modules
in our packages.

They are mainly used by others building their own binaries

Re: Setting up a proper modelling environment in Cygwin

[Lawrence Bernardo]

Dear Eliot,

Thank you for your response. Indeed, after trying a number of different
combinations of component versions using the setup application, I found a
working set based on gcc/gfortran version 7.4.0. And I get what you mean
about Cygwin-specifc adjustments, as one of the codes I tried to compile
didn't succeed, and after a google search, found that it was mainly a
Cygwin issue. Thankfully, someone was already able to find a fix.

The cygwin time machine sounds interesting too, for the case of some really
old codes that might need to be dug up and reused. But thank you again for
your reply, it has been really helpful.

Best,
Lawrence


On Fri, Sep 11, 2020 at 10:33 PM Eliot Moss <moss@cs.umass.edu> wrote:

> On 9/10/2020 10:37 PM, Lawrence Bernardo via Cygwin wrote:
> > Dear Cygwin users and developers,
> >
> > I am trying to run some models based on the ROMS ocean modeling system.
> And
> > although I mainly run such models on Linux supercomputing systems, I
> > sometimes use Cygwin to make test runs. However, whenever I install
> Cygwin
> > on a new PC, I often find that my set up on an older PC I had been using
> > doesn't work properly. After looking deeper into the issues, it seems
> that
> > proper compilation requires that the same fortran compiler (gfortran in
> > this case) be used to compile the necessary components.
> >
> > And as far as I understand, these components would mainly be:
> > - zlib
> > - hdf5
> > - netcdf
> > - netcdf-fortran
> > - openmpi
> >
> > As the gfortran compiler version number used for these different
> components
> > is not indicated whenever I run cygwin setup-x86_64.exe, I am left to
> guess
> > which version numbers for each to select. Of course, another option would
> > be to accept the defaults for some components, and then compile my own
> > version of say, netcdf, but this attempt almost always fails, and I'm not
> > sure if this is due to conflicts with the already existing cygwin setup.
> >
> > Given the issues above, would anyone have any suggestions or
> > recommendations on how to best achieve the setup I would like to have? I
> > will greatly appreciate any help.
>
> Not sure if anyone has responded yet ...
>
> You can:
>
> - Install the specific version of gfortran (gcc suite) that you need.
> - Build the other things from sources in Cygwin, using the source packages.
>
> There might be rough edges if there are cygwin-specific adjustments that
> have been
> made and they don't quite match the version you want.
>
> There is also the cygwin time machine that you can use to get any old
> versions.
> (I've not used it, but know it exists and sometimes helps folks with
> situations
> like this.)
>
> Regards - Eliot Moss
>

Re: Setting up a proper modelling environment in Cygwin

[Lawrence Bernardo]

Thank you Marco for addressing my concerns. And I think checking for the
fortran module consistency will be really helpful for me to assess issues I
run into in the future. I have rolled back the versions I have been using
to all be based on gfortran/gcc version 7.4.0, but an update, although not
urgent for now, would indeed be helpful. Lately, some other codes I've been
running have been having problems with this older version, so I'll stay
tuned for your updates to the package.

Best regards,
Lawrence

On Fri, Sep 11, 2020 at 11:59 PM Marco Atzeri via Cygwin <cygwin@cygwin.com>
wrote:

> On 11.09.2020 04:37, Lawrence Bernardo via Cygwin wrote:
> > Dear Cygwin users and developers,
> >
> > I am trying to run some models based on the ROMS ocean modeling system.
> And
> > although I mainly run such models on Linux supercomputing systems, I
> > sometimes use Cygwin to make test runs. However, whenever I install
> Cygwin
> > on a new PC, I often find that my set up on an older PC I had been using
> > doesn't work properly. After looking deeper into the issues, it seems
> that
> > proper compilation requires that the same fortran compiler (gfortran in
> > this case) be used to compile the necessary components.
> >
> > And as far as I understand, these components would mainly be:
> > - zlib
> > - hdf5
> > - netcdf
> > - netcdf-fortran
> > - openmpi
> >
> > As the gfortran compiler version number used for these different
> components
> > is not indicated whenever I run cygwin setup-x86_64.exe, I am left to
> guess
> > which version numbers for each to select. Of course, another option would
> > be to accept the defaults for some components, and then compile my own
> > version of say, netcdf, but this attempt almost always fails, and I'm not
> > sure if this is due to conflicts with the already existing cygwin setup.
> >
> > Given the issues above, would anyone have any suggestions or
> > recommendations on how to best achieve the setup I would like to have? I
> > will greatly appreciate any help.
> >
> > Best regards,
> > Lawrence
> > --
>
>
> for what I am aware the problem is only present with
> fortran modules if you are using them.
> C interface are not involved
>
> zlib should not be involved as it has no fortran specific modules
> same for hdf5.
>
> Now the current gfortran has
>
> $ zcat
> /usr/lib/gcc/x86_64-pc-cygwin/9.3.0/finclude/ieee_arithmetic.mod|head -n 1
> GFORTRAN module version '15' created from ieee_arithmetic.F90
>
> openmpi was built with last gcc so it is aligned
>
> $ zcat /usr/lib/mpi.mod |head -n 1
> GFORTRAN module version '15' created from mpi-ignore-tkr.F90
>
> netcdf-fortran needs a rebuild
>
> $ zcat /usr/include/netcdf.mod | head -n 1
> GFORTRAN module version '14' created from netcdf4.f90
>
> Give me some days for an update of the package
> And I will need to re-build for test gcc 10 also,
> but it is less urgent
>
>
> Regards
> Marco
>
>
>
>
> --
> Problem reports:      https://cygwin.com/problems.html
> FAQ:                  https://cygwin.com/faq/
> Documentation:        https://cygwin.com/docs.html
> Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple
>

Re: Setting up a proper modelling environment in Cygwin

[Marco Atzeri]

On 14.09.2020 04:07, Lawrence Bernardo wrote:
> Thank you Marco for addressing my concerns. And I think checking for the 
> fortran module consistency will be really helpful for me to assess 
> issues I run into in the future. I have rolled back the versions I have 
> been using to all be based on gfortran/gcc version 7.4.0, but an update, 
> although not urgent for now, would indeed be helpful. Lately, some 
> other codes I've been running have been having problems with this older 
> version, so I'll stay tuned for your updates to the package.
> 
> Best regards,
> Lawrence
> 

bottom post on this mailing list, please.

The netcdf-fortran is already aligned

https://sourceware.org/pipermail/cygwin-announce/2020-September/009700.html

  $ zcat /usr/include/netcdf.mod | head -n 1
GFORTRAN module version '15' created from netcdf4.f90

Regards
Marco

Re: Setting up a proper modelling environment in Cygwin

[Lawrence Bernardo]

Thanks Marco! I'll attempt to reinstall the latest netcdf-fortran and see
if it will work for me.

Cheers,
Lawrence

On Tue, Sep 15, 2020 at 3:25 AM Marco Atzeri <marco.atzeri@gmail.com> wrote:

> On 14.09.2020 04:07, Lawrence Bernardo wrote:
> > Thank you Marco for addressing my concerns. And I think checking for the
> > fortran module consistency will be really helpful for me to assess
> > issues I run into in the future. I have rolled back the versions I have
> > been using to all be based on gfortran/gcc version 7.4.0, but an update,
> > although not urgent for now, would indeed be helpful. Lately, some
> > other codes I've been running have been having problems with this older
> > version, so I'll stay tuned for your updates to the package.
> >
> > Best regards,
> > Lawrence
> >
>
> bottom post on this mailing list, please.
>
> The netcdf-fortran is already aligned
>
> https://sourceware.org/pipermail/cygwin-announce/2020-September/009700.html
>
>   $ zcat /usr/include/netcdf.mod | head -n 1
> GFORTRAN module version '15' created from netcdf4.f90
>
> Regards
> Marco
>
>