g++ and c++17 filesystem

g++ and c++17 filesystem

[sten.kristian.ivarsson]

[-- Attachment #1: Type: text/plain, Size: 1900 bytes --]

Hi folks

The filesystem-library as a part of C++17 seems to have some defects and
flaws in the cygwin-package and pretty much every lexical- and canonical
operation works in mysterious ways (or not at all)


Following output with g++cygwin

 $ uname -a
 CYGWIN_NT-10.0 JOKK 3.1.7(0.340/5/3) 2020-08-22 17:48 x86_64 Cygwin

 $ g++ --version
 g++ (GCC) 10.2.0
 Copyright (C) 2020 Free Software Foundation, Inc.
 This is free software; see the source for copying conditions.  There is NO
 warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

 $ g++ -std=c++17 main.cpp

 $ ./a.exe
 true
 C:/Temp
 filesystem error: cannot make canonical path: No such file or directory
[C:/Temp]


Following output with mingw (that also is a bit strange (note the last
"generic" (i.e. posix) output))

 $ .\g++.exe --version
 g++.exe (MinGW.org GCC Build-2) 9.2.0
 Copyright (C) 2019 Free Software Foundation, Inc.
 This is free software; see the source for copying conditions.  There is NO
 warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

 $ .\g++.exe -std=c++17 main.cpp 

 $ .\a.exe
 true
 C:/Temp
 C:\Temp


Following output with cl (that seems to be the most standard-conformant)

 C:\>cl /std:c++17 main.cpp
 Microsoft (R) C/C++ Optimizing Compiler Version 19.27.29112 for x86
 Copyright (C) Microsoft Corporation.  All rights reserved.

 main.cpp

 /out:main.exe
 main.obj

 C:\>main.exe
 true
 C:/Temp
 C:/Temp



I have failed to find out exactly where the defect are though and I also do
have a hard time to see where/how/if there's a gcc/g++/mingw- or
libstdc++-fork for the cygwin package where this breaks


Does anyone have any ideas about this and know of any patches or (hot)fixes
to this ? 

I haven't tried out all the mingw generic/lexical/canonical functions yet,
but maybe they're flawed as well


Thanx in advance



Best regards,
Kristian





[-- Attachment #2: main.cpp --]
[-- Type: text/plain, Size: 549 bytes --]

#include <filesystem>
#include <iostream>
#include <stdexcept>

// g++ -std=c++17 main.cpp


int main()
{

   try
   {
      const std::filesystem::path path{"C:\\Temp"};
      std::cout << std::boolalpha << std::filesystem::exists(path) << std::noboolalpha << std::endl;
      std::cout << path.generic_u8string() << std::endl;
      std::cout << std::filesystem::canonical(path).generic_u8string() << std::endl;
   }
   catch(const std::exception& e)
   {
      std::cerr << e.what() << std::endl;
   }


   return 0;
}

Re: g++ and c++17 filesystem

[René Berber]

On 11/17/2020 9:15 AM, Kristian Ivarsson via Cygwin wrote:

> The filesystem-library as a part of C++17 seems to have some defects and
> flaws in the cygwin-package and pretty much every lexical- and canonical
> operation works in mysterious ways (or not at all)
[snip]

https://cygwin.com/cygwin-ug-net/using.html#pathnames-win32

-- 
R.Berber

Sv: g++ and c++17 filesystem

[sten.kristian.ivarsson]

> On 11/17/2020 9:15 AM, Kristian Ivarsson via Cygwin wrote:
> 
> > The filesystem-library as a part of C++17 seems to have some defects
> > and flaws in the cygwin-package and pretty much every lexical- and
> > canonical operation works in mysterious ways (or not at all)
> [snip]
> 
> https://cygwin.com/cygwin-ug-net/using.html#pathnames-win32

So by this you're saying that cygwin-applications cannot handle the
filesystem it is supposed to handle ? 

How come std::filesystem first say "It's a valid file" followed by "It's not
a valid file" ? Is that part of the "circumvention" ?

In the current use case, the path is even returned from
std::filesystem::directory_iterator where it says "Hey, here's a path to a
file I found in the folder you pointed out" followed by "The path I just
gave to you is not a valid path"

It's gonna be tricky to implement applications in that way




Best regards,
Kristian



> --
> R.Berber
> --
> Problem reports:      https://cygwin.com/problems.html
> FAQ:                  https://cygwin.com/faq/
> Documentation:        https://cygwin.com/docs.html
> Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple

Re: Sv: g++ and c++17 filesystem

[René Berber]

On 11/18/2020 3:00 AM, Kristian Ivarsson via Cygwin wrote:

>> On 11/17/2020 9:15 AM, Kristian Ivarsson via Cygwin wrote:
>>
>>> The filesystem-library as a part of C++17 seems to have some defects
>>> and flaws in the cygwin-package and pretty much every lexical- and
>>> canonical operation works in mysterious ways (or not at all)
>> [snip]
>>
>> https://cygwin.com/cygwin-ug-net/using.html#pathnames-win32
> 
> So by this you're saying that cygwin-applications cannot handle the
> filesystem it is supposed to handle ?

I'm not saying anything, I'm pointing to the relevant documentation.

> How come std::filesystem first say "It's a valid file" followed by "It's not
> a valid file" ? Is that part of the "circumvention" ?

The documentation states that using Windows notation is not supported 
and may result in unexpected results (i.e. sometimes work, sometimes 
doesn't).

> In the current use case, the path is even returned from
> std::filesystem::directory_iterator where it says "Hey, here's a path to a
> file I found in the folder you pointed out" followed by "The path I just
> gave to you is not a valid path"
> 
> It's gonna be tricky to implement applications in that way

The problem is that your assumptions and expectations are wrong.  Not 
knowing Cygwin that is expected.

Cygwin handles the file system with no problem, but using Posix-like 
notation, not Windows-like.  End of story.
--
R.Berber

Re: Sv: g++ and c++17 filesystem

[Eliot Moss]

On 11/18/2020 11:24 AM, René Berber via Cygwin wrote:

> Cygwin handles the file system with no problem, but using Posix-like notation, not Windows-like.  
> End of story.

And I'll add, this is by design: Cygwin's goal is to provide a programming (and command line) 
environment as much like Posix as reasonably possible.

It does include some tools to help interface with Windows more explicitly, such as cygpath and 
cygstart.  I have defined a bash alias ppt that refers to a bach function powerpnt, defined thusly:

powerpnt ()
{
     local ARG;
     [ -n "$1" ] && {
         ARG="$(cygpath -wa "$1")";
         shift
     };
     [ -n "$VERBOSE" ] && {
         echo powerpnt ${ARG:+"${ARG}"} "$@"
     };
     command powerpnt ${ARG:+"${ARG}"} "$@" &
}

This takes the first argument and converts it from Posix to Windows form, for passing to the 
powerpnt binary.  And then I have a file system link so that "command powerpnt" gets to the 
installed Windows binary C:\Program Files.

But this is by far the exception as opposed to the rule in how I use Cygwin.

In hope that this give you some perspective and insight as to what Cygwin is and why.

Best wishes - Eliot Moss

Re: g++ and c++17 filesystem

[Kristian Ivarsson]

> 18 nov. 2020 kl. 17:26 skrev René Berber via Cygwin <cygwin@cygwin.com>:
> 
> On 11/18/2020 3:00 AM, Kristian Ivarsson via Cygwin wrote:
> 
>>>> On 11/17/2020 9:15 AM, Kristian Ivarsson via Cygwin wrote:
>>> 
>>>> The filesystem-library as a part of C++17 seems to have some defects
>>>> and flaws in the cygwin-package and pretty much every lexical- and
>>>> canonical operation works in mysterious ways (or not at all)
>>> [snip]
>>> 
>>> https://cygwin.com/cygwin-ug-net/using.html#pathnames-win32
>> So by this you're saying that cygwin-applications cannot handle the
>> filesystem it is supposed to handle ?
> 
> I'm not saying anything, I'm pointing to the relevant documentation.
> 
>> How come std::filesystem first say "It's a valid file" followed by "It's not
>> a valid file" ? Is that part of the "circumvention" ?
> 
> The documentation states that using Windows notation is not supported and may result in unexpected results (i.e. sometimes work, sometimes doesn't).
[snip]
> Cygwin handles the file system with no problem, but using Posix-like notation, not Windows-like.  End of story.
Well, the sole reason behind this question was to either find an existing solution to this issue or perhaps invoke some work to improveme CYGWIN, because no developer, operator or user likes undefined behaviour and I think that we all can agree to that it would be kind of nifty if this worked as expected and thus I don’t consider this to be the end of the story

I’m thankful that you pointed out the documentation (that I already have read some while ago) and by “you” I didn’t mean you in person but more directed to the “community”

The only purpose CYGWIN have is to make/build posix-applications runnable on Windows and applications usually have user defined input, such as paths etc, and on Windows that input is usually Windows-native-paths unless you educate operators/users to enter paths with /cygdrive/...

... or you have to add code to kind of transform possible Windows-paths to Posix-paths (perhaps back and forth), well then you’re adding non-posix-logic (non-cross-platform-logic) to your application and that makes it one reason less to use CYGWIN 

Is there any other use cases for CYGWIN than to build applications running in Windows ? Do people use CYGWIN (shell) to operate or monitor their applications ? For all other use cases than the development (the shell) I cannot see why CYGWIN would favour posix-paths over native path’s, but maybe I’m wrong ?

So, a library can always go from undefined to defined without breaking changes, so does anyone know what the effort could be to fix this, i.e. make it work transparently/agnostic ?

As stated earlier, it seems like using mingw g++/libstdc++ (from the cygwin-package-manager) it seems like it works better, but then you can’t mix with other posix/cygwin mechanism (that uses cygstdc++) without breaking ODR (and probably some memory models etc as well) so maybe someone do have some insightful info about this ? How “special” is cygstdc++ (compared to mingw:s libstdc++) ? Could this be fixable in that library (cygstdc++) ?

Best regards,
Kristian

> --
> R.Berber
> --
> Problem reports:      https://cygwin.com/problems.html
> FAQ:                  https://cygwin.com/faq/
> Documentation:        https://cygwin.com/docs.html
> Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple

Re: g++ and c++17 filesystem

[Eliot Moss]

I would agree that if you want an executable that acts and feels more like a Windows native 
application, then mingw is probably what you want.  Cygwin is if you want something that acts and 
feels more like a Posix thing ... which means it will be oriented to Posix style paths.

EM

Re: g++ and c++17 filesystem

[Kristian Ivarsson]

> I would agree that if you want an executable that acts and feels more like a Windows native application, then mingw is probably what you want.  Cygwin is if you want something that acts and feels more like a Posix thing ... which means it will be oriented to Posix style paths.
To be able to use mingw all the code have to be ported to use Windows native mechanisms and then we might just use MSVC instead

We don’t want (either) Windows-style-paths or Posix-style-paths, we want A path and expect it to work equally regardless of what platform is used in regards to std::filesystem

As far as I see, very few applications do form their own - and/or have hard-coded absolute paths and instead they are usually input data (through UI, configuration, OS, environment or such)


> EM

Re: g++ and c++17 filesystem

[Norton Allen]

On 11/18/2020 3:46 PM, Kristian Ivarsson via Cygwin wrote:
> Is there any other use cases for CYGWIN than to build applications running in Windows ? Do people use CYGWIN (shell) to operate or monitor their applications ? For all other use cases than the development (the shell) I cannot see why CYGWIN would favour posix-paths over native path’s, but maybe I’m wrong ?

To write software that can be compiled and run under Windows, Linux and 
MacOS.

Re: g++ and c++17 filesystem

[Eliot Moss]

On 11/18/2020 4:18 PM, Kristian Ivarsson wrote:
> 
>> I would agree that if you want an executable that acts and feels more like a Windows native application, then mingw is probably what you want.  Cygwin is if you want something that acts and feels more like a Posix thing ... which means it will be oriented to Posix style paths.
> To be able to use mingw all the code have to be ported to use Windows native mechanisms and then we might just use MSVC instead
> 
> We don’t want (either) Windows-style-paths or Posix-style-paths, we want A path and expect it to work equally regardless of what platform is used in regards to std::filesystem
> 
> As far as I see, very few applications do form their own - and/or have hard-coded absolute paths and instead they are usually input data (through UI, configuration, OS, environment or such)

IN this context, I would say "Which std::filesystem?  The Cygwin Posix-like one or the mingw 
Windows-like one?"  If you want uniformity, I'd go with Cygwin; it you want platform-like behavior, 
then mingw.

Best wishes - EM

Re: g++ and c++17 filesystem

[Doug Henderson]

On Wed, 18 Nov 2020 at 13:50, Kristian Ivarsson via Cygwin
<cygwin@cygwin.com> wrote:
>
>

> The only purpose CYGWIN have is to make/build posix-applications runnable on Windows and applications usually have user defined input, such as paths etc, and on Windows that input is usually Windows-native-paths unless you educate operators/users to enter paths with /cygdrive/...

<rant>
I use CYGWIN to work around a stupid design decision made by a small
time OS developer more than 40 years ago, and inherited by an OS that
did not even have directories in its first release.
</rant>

Doug

-- 
Doug Henderson, Calgary, Alberta, Canada - from gmail.com

Re: g++ and c++17 filesystem

[Brian Inglis]

On 2020-11-18 17:08, Doug Henderson via Cygwin wrote:
> On Wed, 18 Nov 2020 at 13:50, Kristian Ivarsson wrote:
>> The only purpose CYGWIN have is to make/build posix-applications runnable 
>> on Windows and applications usually have user defined input, such as paths 
>> etc, and on Windows that input is usually Windows-native-paths unless you
>> educate operators/users to enter paths with /cygdrive/...

> <rant>
> I use CYGWIN to work around a stupid design decision made by a small
> time OS developer more than 40 years ago, and inherited by an OS that
> did not even have directories in its first release.
> </rant>

Actually, most of those design decisions were made in the 1950s-1960s by 
scientific then mainframe OS designers in multiprocessing then multiuser 
environments, and the short sighted decisions were made to get things working 
quickly by people without much design or systems background (BGIII^Wcough!) in 
some of the smaller minicomputer then microcomputer adaptations, which were 
often frankly PoS hacks, and triumphs of salesmanship over competency. ;^>

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.
[Data in binary units and prefixes, physical quantities in SI.]

Sv: g++ and c++17 filesystem

[sten.kristian.ivarsson]

> >> I would agree that if you want an executable that acts and feels more
> like a Windows native application, then mingw is probably what you want.
> Cygwin is if you want something that acts and feels more like a Posix
> thing ... which means it will be oriented to Posix style paths.
> > To be able to use mingw all the code have to be ported to use Windows
> > native mechanisms and then we might just use MSVC instead
> >
> > We don’t want (either) Windows-style-paths or Posix-style-paths, we
> > want A path and expect it to work equally regardless of what platform
> > is used in regards to std::filesystem
> >
> > As far as I see, very few applications do form their own - and/or have
> > hard-coded absolute paths and instead they are usually input data
> > (through UI, configuration, OS, environment or such)
> 
> IN this context, I would say "Which std::filesystem?  The Cygwin Posix-
> like one or the mingw Windows-like one?"  If you want uniformity, I'd go
> with Cygwin; it you want platform-like behavior, then mingw.


I'm referring to std::filesystem as a part of the C++17 standard (https://en.cppreference.com/w/cpp/filesystem) that is pretty well defined and quite agnostic to what "style" of path used as our application are and as I said, we don't care (we don't ever inspect them) what "style" of paths we're using but we expect a deterministic behaviour from that library regardless of operating system, such as and absolute path should be an absolute path regardless

That's the sole purpose of std::filesystem, i.e. to be platform independent (though all file-features is not applicable on all operating systems, but at least you can ask the library for those attributes)


GCC/MinGW support platform-INDEPENDENT-behaviour because gcc/g++ works equally regardless if Linux or Windows in regards to std::filesystem


Best regards,
Kristian


> Best wishes - EM
> --
> Problem reports:      https://cygwin.com/problems.html
> FAQ:                  https://cygwin.com/faq/
> Documentation:        https://cygwin.com/docs.html
> Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple

Sv: g++ and c++17 filesystem

[sten.kristian.ivarsson]

> > 18 nov. 2020 kl. 17:26 skrev René Berber via Cygwin <cygwin@cygwin.com>:
> >
> > On 11/18/2020 3:00 AM, Kristian Ivarsson via Cygwin wrote:
> >
> >>>> On 11/17/2020 9:15 AM, Kristian Ivarsson via Cygwin wrote:
> >>>
> >>>> The filesystem-library as a part of C++17 seems to have some
> >>>> defects and flaws in the cygwin-package and pretty much every
> >>>> lexical- and canonical operation works in mysterious ways (or not
> >>>> at all)
> >>> [snip]
> >>>
> >>> https://cygwin.com/cygwin-ug-net/using.html#pathnames-win32

[snip] 
 
> As stated earlier, it seems like using mingw g++/libstdc++ (from the
> cygwin-package-manager) it seems like it works better, but then you can’t
> mix with other posix/cygwin mechanism (that uses cygstdc++) without
> breaking ODR (and probably some memory models etc as well) so maybe
> someone do have some insightful info about this ? How “special” is
> cygstdc++ (compared to mingw:s libstdc++) ? Could this be fixable in that
> library (cygstdc++) ?

I think the problem can be viewed in /usr/lib/gcc/x86_64-pc-cygwin/10/include/c++/bits/fs_path.h and

...
 #if defined(_WIN32) && !defined(__CYGWIN__)
 # define _GLIBCXX_FILESYSTEM_IS_WINDOWS 1
 # include <algorithm>
 #endif
...

that when build in CYGWIN will make the stdc++ think it is not on Windows (and I guess cygstdc++-6.dll will be built without _GLIBCXX_FILESYSTEM_IS_WINDOWS implicitly), thus, the std::filesystem-library will act as it is on a Posix-system, but it is not and thus it makes wrong assumptions

It seems like the (ordinary) MinGW-shipping includes these directives (!defined(__CYGWIN__)) as well and my naive take on this is that it is a (the) mistake

The underlaying filesystem IS Windows and NOT Posix, but my guess is that (some) Posix-system-calls and/or assuming Posix-style-paths are invoked when _GLIBCXX_FILESYSTEM_IS_WINDOWS is not set

I guess the correct way would be to let _GLIBCXX_FILESYSTEM_IS_WINDOWS and maybe just have a #ifdef (__CYGWIN__) as an extra option ONLY when handling lexical stuff, i.e. it allows both Windows- and Posix-styles but the system calls should always be Windows calls (and I guess this would imply better performance as well to not need to walk through the whole Cygwin-posix-abstraction)

I might be totally wrong, so does anyone have any take on this ?


Best regards,
Kristian 


> Best regards,
> Kristian
[snip]

Re: Sv: g++ and c++17 filesystem

[Brian Inglis]

On 2020-11-19 03:03, Kristian Ivarsson via Cygwin wrote:
>>> 18 nov. 2020 kl. 17:26 skrev René Berber via Cygwin <cygwin@cygwin.com>:
>>> On 11/18/2020 3:00 AM, Kristian Ivarsson via Cygwin wrote:
>>>>>> On 11/17/2020 9:15 AM, Kristian Ivarsson via Cygwin wrote:
>>>>>> The filesystem-library as a part of C++17 seems to have some
>>>>>> defects and flaws in the cygwin-package and pretty much every
>>>>>> lexical- and canonical operation works in mysterious ways (or not
>>>>>> at all)

>>>>> https://cygwin.com/cygwin-ug-net/using.html#pathnames-win32

>> As stated earlier, it seems like using mingw g++/libstdc++ (from the
>> cygwin-package-manager) it seems like it works better, but then you can’t
>> mix with other posix/cygwin mechanism (that uses cygstdc++) without
>> breaking ODR (and probably some memory models etc as well) so maybe
>> someone do have some insightful info about this ? How “special” is
>> cygstdc++ (compared to mingw:s libstdc++) ? Could this be fixable in that
>> library (cygstdc++) ?

> I might be totally wrong, so does anyone have any take on this ?

Cygwin provides cross-tools like djgpp-gcc-core mingw64-i686-gcc-core, 
mingw64-x86_64-gcc-core, cygwin32-gcc-core, cygwin64-gcc-core, and 
djgpp-binutils, mingw64-i686-binutils, mingw64-x86_64-binutils, 
cygwin32-binutils, cygwin64-binutils so anyone has the freedom to choose to 
build DOS, Windows, or Cygwin apps targeting their respective APIs, under 
Cygwin, and also have the freedom to give away or sell those apps as long as 
they respect their licences.

Cygwin's goal is to have everyone and everything believe it is running in a 
POSIX environment and provide interoperability within a Windows environment 
(including Wine) based on POSIX standards, system interfaces, toolchains, 
shells, utilities:

	https://pubs.opengroup.org/onlinepubs/9699919799/

system and network servers and services, plus GUI desktops (GNOME, KDE, LXDE, 
MATE, Plasma, Xfce desktops on X Window System), and apps (task and file 
managers, web browsers, PDF viewers and editors, graphics viewers and editors 
including GIMP). This is all ported and supported by volunteers who believe 
everyone should have a choice, even when for business or family reasons they use 
Windows.

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.
[Data in binary units and prefixes, physical quantities in SI.]

Re: Sv: g++ and c++17 filesystem

[Eliot Moss]

Ok, first, I admit that I was not familiar with the details of std::filesystem.  However, after 
looking at it, I remain unsurprised that the Cygwin and Mingw versions might be different.  (I would 
also not be surprised if there is a real bug in there.)  The behavior I would _expect_ is that the 
Cygwin version will work using Posix sorts of assumptions.  While a root of C: (for example) _might_ 
work, /cygdrive/c is more normative on Cygwin.  (I put a link to that in Cygwin's / called c, so 
that, for me, /c works.)  Likewise, I would expect the normative path separator to be / not \, and 
an absolute path to start with /.  Windows offers several kinds of symlinks, with varying semantics, 
so the detailed behavior of that would be affected by the settings in the CYGWIN environment variable.

I would expect std::filesystem to present paths to construct paths to present to underlying library 
calls such as open ... and on Cygwin, open uses Posix style paths.

I "get" that you want to write portable programs that use this interface, which is analogous to the 
Java file path classes.  In terms of how this interface works, I would expect it to _claim_ that it 
is Posix, not Windows, because the paths Cygwin supports are Posix style (it _will_ recognize a few 
Windows idioms, but it is correct in not advertising itself as Windows).

So it you want to do Windows-style (but abstracted with this library), I direct you to Mingw.  Each 
has its place.  Cygwin allows one to pretend, pretty successfully though with a few small rough 
edges, that one is on Linux, not Windows.  That is its intent.  Mingw gives you the gcc/gnu 
toolchain and libraries under Windows.

I hope we're not still talking at cross purposes, though that it certainly possible!

Best wishes - EM

Sv: Sv: g++ and c++17 filesystem

[sten.kristian.ivarsson]

> Ok, first, I admit that I was not familiar with the details of
> std::filesystem.  However, after looking at it, I remain unsurprised that
> the Cygwin and Mingw versions might be different.  (I would also not be
> surprised if there is a real bug in there.)

At least semantic bugs considering the whole concept

> The behavior I would _expect_ is that the Cygwin version will work using
Posix sorts of assumptions.

I guess it will to, but then your application have to ensure that in every
place the application uses a file path and the reasons we do have
(platform-independent) libraries is to not having to do that

> While a root of C: (for example) _might_ work, /cygdrive/c is more
> normative on Cygwin.

Yes, but to what purpose ? What applications do explicitly handle any root
directory (except for possibly /tmp or /dev/null) ?

(I put a link to that in Cygwin's / called c, so
> that, for me, /c works.)  Likewise, I would expect the normative path
> separator to be / not \, and an absolute path to start with /.  Windows
> offers several kinds of symlinks, with varying semantics, so the detailed
> behavior of that would be affected by the settings in the CYGWIN
> environment variable

All the implementations of std::filesystem I've seen so far, is agnostic to
whether / or \ is used as a path separator (but the generic form is '/' and
a fun fact, the MSVC-implementation of std::filesystem handles the generic
(posix) form more close to the standard specification than GCC)

> I would expect std::filesystem to present paths to construct paths to
> present to underlying library calls such as open ... and on Cygwin, open
> uses Posix style paths.

I consider that to be a mistake in the implementation of std::filesystem,
because on Windows the preferred style would be smt like C:/ and then as an
opt-out it would consider /cygdrive/c (or such) as a valid thing as well

> I "get" that you want to write portable programs that use this interface,
> which is analogous to the Java file path classes.  In terms of how this
> interface works, I would expect it to _claim_ that it is Posix, not
> Windows, because the paths Cygwin supports are Posix style (it _will_
> recognize a few Windows idioms, but it is correct in not advertising
> itself as Windows).
> 
> So it you want to do Windows-style (but abstracted with this library), I
> direct you to Mingw.  Each has its place.  Cygwin allows one to pretend,
> pretty successfully though with a few small rough edges, that one is on
> Linux, not Windows.  That is its intent.  Mingw gives you the gcc/gnu
> toolchain and libraries under Windows.

As stated earlier, we're using Cygwin to be able to keep some kind of cross
platform code base and Cygwin offers non-cross-platform-standardized
libraries/api:s (i.e. posix) to be executable without having to #ifdef the
code base to be buildable and executable on Windows but MinGW doesn't supply
those posix libraries on Windows (maybe a few), so using GCC/MinGW is not an
option and I guess we'd go with MSVC if we wanted to port our code
completely

std::filesystem is supposed to be cross-platform (and easier to use than
various posix-library-C-functions) though and it is cross-platform per
definition but, then again, not when using Cygwin

> I hope we're not still talking at cross purposes, though that it certainly
> possible!
> 
> Best wishes - EM
> --
> Problem reports:      https://cygwin.com/problems.html
> FAQ:                  https://cygwin.com/faq/
> Documentation:        https://cygwin.com/docs.html
> Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple

Sv: Sv: g++ and c++17 filesystem

[sten.kristian.ivarsson]

[snip]
> >> As stated earlier, it seems like using mingw g++/libstdc++ (from the
> >> cygwin-package-manager) it seems like it works better, but then you
> >> can’t mix with other posix/cygwin mechanism (that uses cygstdc++)
> >> without breaking ODR (and probably some memory models etc as well) so
> >> maybe someone do have some insightful info about this ? How “special”
> >> is
> >> cygstdc++ (compared to mingw:s libstdc++) ? Could this be fixable in
> >> cygstdc++ that
> >> library (cygstdc++) ?


> > I might be totally wrong, so does anyone have any take on this ?
> 
> Cygwin provides cross-tools like djgpp-gcc-core mingw64-i686-gcc-core,
> mingw64-x86_64-gcc-core, cygwin32-gcc-core, cygwin64-gcc-core, and djgpp-
> binutils, mingw64-i686-binutils, mingw64-x86_64-binutils, cygwin32-
> binutils, cygwin64-binutils so anyone has the freedom to choose to build
> DOS, Windows, or Cygwin apps targeting their respective APIs, under
> Cygwin, and also have the freedom to give away or sell those apps as long
> as they respect their licences.
> 
> Cygwin's goal is to have everyone and everything believe it is running in
> a POSIX environment and provide interoperability within a Windows
> environment (including Wine) based on POSIX standards, system interfaces,
> toolchains, shells, utilities:
> 
> 	https://pubs.opengroup.org/onlinepubs/9699919799/
> 
> system and network servers and services, plus GUI desktops (GNOME, KDE,
> LXDE, MATE, Plasma, Xfce desktops on X Window System), and apps (task and
> file managers, web browsers, PDF viewers and editors, graphics viewers and
> editors including GIMP). This is all ported and supported by volunteers
> who believe everyone should have a choice, even when for business or
> family reasons they use Windows.

Tnx Brian

I think The Cygwin-community is doing a great job but with some caveats, such as this

We're in need of various posix-libraries and I guess they won't be available if using the mingw-g++/libstdc++ (I guess cygstdc++ is needed then due to some special memory models etc etc etc) ?

I can for sure try it out, but that would be quite cumbersome because the and I guess it'll be a whole lot of hazzle to make it working

I'd rather help out or having a dialog of how to fix std::filesystem, i.e. change the usage of the __CYGWIN__ macro in that implementation, but this seems to be a part of the "real" GCC-distribution o I guess I need to be involved in that open-source-community instead, but I guess it somehow is invoked from this project somehow so I guess some people here do have some real insights about this ?

The whole C/C++ community is striving for total cross-platform libraries (but I guess we have a few years left for that) and std::filesystem was supposed to take us all in that direction and I totally understand that std::filesystem-library in Cygwin do think it is on a posix-filesystem (though it's not) and I totally understand why the behaviour is as it is, but I don't agree that is a good thing, considering that the underlaying posix-implementation already today accepts Windows:ish-like-paths in some circumstances, I'd like the whole package to be even more agnostic because most applications don't have any wish to inspect the content of a path-object no more than the value of a socket-descriptor

Applications might wanna extract type, name, parent-folder, etc but do rarely care about what kind of separator it has (/ or \) and the style of the root directory etc and it would be very neat if the cygwin std::filesystem-library became more agnostic in these regards

Best regards,
Kristian


> --
> Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada
> 
> This email may be disturbing to some readers as it contains too much
> technical detail. Reader discretion is advised.
> [Data in binary units and prefixes, physical quantities in SI.]
> --
> Problem reports:      https://cygwin.com/problems.html
> FAQ:                  https://cygwin.com/faq/
> Documentation:        https://cygwin.com/docs.html
> Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple

Re: Sv: Sv: g++ and c++17 filesystem

[Brian Inglis]

On 2020-11-20 02:37, Kristian Ivarsson via Cygwin wrote:
> [snip]
>>>> As stated earlier, it seems like using mingw g++/libstdc++ (from the 
>>>> cygwin-package-manager) it seems like it works better, but then you 
>>>> can’t mix with other posix/cygwin mechanism (that uses cygstdc++) 
>>>> without breaking ODR (and probably some memory models etc as well) so 
>>>> maybe someone do have some insightful info about this ? How “special” 
>>>> is cygstdc++ (compared to mingw:s libstdc++) ? Could this be fixable
>>>> in cygstdc++ that library (cygstdc++)?

>>> I might be totally wrong, so does anyone have any take on this ?
>>
>> Cygwin provides cross-tools like djgpp-gcc-core mingw64-i686-gcc-core,
>> mingw64-x86_64-gcc-core, cygwin32-gcc-core, cygwin64-gcc-core, and djgpp-
>> binutils, mingw64-i686-binutils, mingw64-x86_64-binutils, cygwin32-
>> binutils, cygwin64-binutils so anyone has the freedom to choose to build
>> DOS, Windows, or Cygwin apps targeting their respective APIs, under
>> Cygwin, and also have the freedom to give away or sell those apps as long
>> as they respect their licences.
>>
>> Cygwin's goal is to have everyone and everything believe it is running in
>> a POSIX environment and provide interoperability within a Windows
>> environment (including Wine) based on POSIX standards, system interfaces,
>> toolchains, shells, utilities:
>>
>> 	https://pubs.opengroup.org/onlinepubs/9699919799/
>>
>> system and network servers and services, plus GUI desktops (GNOME, KDE,
>> LXDE, MATE, Plasma, Xfce desktops on X Window System), and apps (task and
>> file managers, web browsers, PDF viewers and editors, graphics viewers and
>> editors including GIMP). This is all ported and supported by volunteers
>> who believe everyone should have a choice, even when for business or
>> family reasons they use Windows.

> I think The Cygwin-community is doing a great job but with some caveats, such as this
> 
> We're in need of various posix-libraries and I guess they won't be available 
> if using the mingw-g++/libstdc++ (I guess cygstdc++ is needed then due to
> some special memory models etc etc etc)?

Unlikely - the (cross) tools handle the memory model differences.
There are 400 mingw64 cross-development library and tool packages available for 
*each* architecture under Cygwin?

> I can for sure try it out, but that would be quite cumbersome because the
> and I guess it'll be a whole lot of hazzle to make it working

> I'd rather help out or having a dialog of how to fix std::filesystem, i.e. 
> change the usage of the __CYGWIN__ macro in that implementation, but this
> seems to be a part of the "real" GCC-distribution o I guess I need to be
> involved in that open-source-community instead, but I guess it somehow is
> invoked from this project somehow so I guess some people here do have some
> real insights about this ?

> The whole C/C++ community is striving for total cross-platform libraries (but
> I guess we have a few years left for that) and std::filesystem was supposed
> to take us all in that direction and I totally understand that
> std::filesystem-library in Cygwin do think it is on a posix-filesystem (though
> it's not) and I totally understand why the behaviour is as it is, but I don't
> agree that is a good thing, considering that the underlaying
> posix-implementation already today accepts Windows:ish-like-paths in some
> circumstances, I'd like the whole package to be even more agnostic because most
> applications don't have any wish to inspect the content of a path-object no more
> than the value of a socket-descriptor
> 
> Applications might wanna extract type, name, parent-folder, etc but do rarely
> care about what kind of separator it has (/ or \) and the style of the root
> directory etc and it would be very neat if the cygwin std::filesystem-library
> became more agnostic in these regards
Not a goal of this project, which is to provide Unix look/feel at all levels.
Other projects have the goals of being cross-platform toolkits which you can use 
to work and/or look native and hide all differences; see:

https://en.wikipedia.org/wiki/Cross-platform_software#Cross-platform_programming_toolkits_and_environments

Which cross-development libraries/tools are you missing from the 400 mingw64 
cross-development library and tool packages available for each architecture 
under Cygwin?

You could probably use cygport to easily build any for mingw that are not yet 
available in the distro.

You could also do all this on any Linux distro that offers Wine, and natively 
under some (especially Fedora/CentOS/EPEL/RHEL) that offer mingw packages, and 
even use cygport to do the builds:

	https://repology.org/projects/?search=mingw

where one of those distros shown is Cygwin.

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.
[Data in binary units and prefixes, physical quantities in SI.]

Sv: Sv: Sv: g++ and c++17 filesystem

[sten.kristian.ivarsson]

[snip]

> > Applications might wanna extract type, name, parent-folder, etc but do
> > rarely care about what kind of separator it has (/ or \) and the style
> > of the root directory etc and it would be very neat if the cygwin
> > std::filesystem-library became more agnostic in these regards
> Not a goal of this project, which is to provide Unix look/feel at all
> levels.

Well, "this project" have already introduced some understanding of Windows-paths such as ("C:/" etc)


> Other projects have the goals of being cross-platform toolkits which you
> can use to work and/or look native and hide all differences; see:
> 
> https://en.wikipedia.org/wiki/Cross-platform_software#Cross-
> platform_programming_toolkits_and_environments
> 
> Which cross-development libraries/tools are you missing from the 400
> mingw64 cross-development library and tool packages available for each
> architecture under Cygwin?



Well, some standardized things for networking, ipc-messaging, multiplexing and such is what's missing in the C/C++ standard but we're not missing anything among libraries and tools in the Cygwin package per say

Our main target is the *nix world and thus "Unix look/feel" is exactly what we like and Cygwin seems to offer a "Unix look/feel" to run our code natively on Windows as well but this "Unix look/feel" doesn't always work in Windows and fails to assimilate that "Unix look/feel"


[1]   std::filsystem::path path{some_path_from_somewhere};
[2]   if(std::filesystem::exists(path))
[3]   {
[4]      auto canonical = std::filsystem::canonical(path);
[5]   }


The code above is very much "Unix look/feel" but the behaviour in Cygwin is not "Unix look/feel" because it might fail on line 4 (complaining that path is not a valid file despite it said so in line 2)


Best regards,
Kristian

> --
> Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada
> 
> This email may be disturbing to some readers as it contains too much
> technical detail. Reader discretion is advised.
> [Data in binary units and prefixes, physical quantities in SI.]
> 
> 
> --
> Problem reports:      https://cygwin.com/problems.html
> FAQ:                  https://cygwin.com/faq/
> Documentation:        https://cygwin.com/docs.html
> Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple

Re: Sv: Sv: g++ and c++17 filesystem

[Jonathan Yong]

[-- Attachment #1.1.1: Type: text/plain, Size: 3548 bytes --]

On 11/20/20 8:31 AM, Kristian Ivarsson via Cygwin wrote:
>> that, for me, /c works.)  Likewise, I would expect the normative path
>> separator to be / not \, and an absolute path to start with /.  Windows
>> offers several kinds of symlinks, with varying semantics, so the detailed
>> behavior of that would be affected by the settings in the CYGWIN
>> environment variable
> 
> All the implementations of std::filesystem I've seen so far, is agnostic to
> whether / or \ is used as a path separator (but the generic form is '/' and
> a fun fact, the MSVC-implementation of std::filesystem handles the generic
> (posix) form more close to the standard specification than GCC)
> 

That is not correct, \ is a valid filename under *nix, but not on 
Windows. I don't think the standards specify what filenames or character 
sets are valid. Cygwin strives for *nix compatibility, anything else is 
secondary.

>> I would expect std::filesystem to present paths to construct paths to
>> present to underlying library calls such as open ... and on Cygwin, open
>> uses Posix style paths.
> 
> I consider that to be a mistake in the implementation of std::filesystem,
> because on Windows the preferred style would be smt like C:/ and then as an
> opt-out it would consider /cygdrive/c (or such) as a valid thing as well
> 

Cygwin is not Windows, it runs on Windows, but it is not Windows. You 
should use mingw if you want Windows style paths. There isn't a magical 
tool that allows you to have it both ways.

>> I "get" that you want to write portable programs that use this interface,
>> which is analogous to the Java file path classes.  In terms of how this
>> interface works, I would expect it to _claim_ that it is Posix, not
>> Windows, because the paths Cygwin supports are Posix style (it _will_
>> recognize a few Windows idioms, but it is correct in not advertising
>> itself as Windows).
>>
>> So it you want to do Windows-style (but abstracted with this library), I
>> direct you to Mingw.  Each has its place.  Cygwin allows one to pretend,
>> pretty successfully though with a few small rough edges, that one is on
>> Linux, not Windows.  That is its intent.  Mingw gives you the gcc/gnu
>> toolchain and libraries under Windows.
> 
> As stated earlier, we're using Cygwin to be able to keep some kind of cross
> platform code base and Cygwin offers non-cross-platform-standardized
> libraries/api:s (i.e. posix) to be executable without having to #ifdef the
> code base to be buildable and executable on Windows but MinGW doesn't supply
> those posix libraries on Windows (maybe a few), so using GCC/MinGW is not an
> option and I guess we'd go with MSVC if we wanted to port our code
> completely
> 

If you use Cygwin, you are expected to use *nix paths, not Windows 
paths. Windows paths may work but are not supported overall, as Windows 
is not POSIX. If you want Windows, then use mingw or MSVC. Since you 
mentioned it is not possible to use mingw, then that will leave you with 
only MSVC, your only possible choice is now clear.

> std::filesystem is supposed to be cross-platform (and easier to use than
> various posix-library-C-functions) though and it is cross-platform per
> definition but, then again, not when using Cygwin
> 

Yes, cross platform, you can use std::filesystem on Linux, Cygwin, 
Windows, Macs, but it certainly cannot do anything that can't be handled 
for the underlying platform. For Cygwin, it means using *nix paths, not 
Windows.

[-- Attachment #1.1.2: OpenPGP_0x713B5FE29C145D45_and_old_rev.asc --]
[-- Type: application/pgp-keys, Size: 8035 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: Sv: Sv: Sv: g++ and c++17 filesystem

[Jonathan Yong]

On 11/23/20 8:35 AM, sten.kristian.ivarsson@gmail.com wrote:
>> On 11/20/20 8:31 AM, Kristian Ivarsson via Cygwin wrote:
>>>> that, for me, /c works.)  Likewise, I would expect the normative path
>>>> separator to be / not \, and an absolute path to start with /.
>>>> Windows offers several kinds of symlinks, with varying semantics, so
>>>> the detailed behavior of that would be affected by the settings in
>>>> the CYGWIN environment variable
>>>
>>> All the implementations of std::filesystem I've seen so far, is
>>> agnostic to whether / or \ is used as a path separator (but the
>>> generic form is '/' and a fun fact, the MSVC-implementation of
>>> std::filesystem handles the generic
>>> (posix) form more close to the standard specification than GCC)
>>>
>>
>> That is not correct, \ is a valid filename under *nix, but not on Windows.
>> I don't think the standards specify what filenames or character sets are
>> valid. Cygwin strives for *nix compatibility, anything else is secondary.
> 
> 
> I should have made my point clear; "All the implementations of std::filesystem for Windows ..."
> 
> 

That is an incorrect understanding, Cygwin is NOT Windows, it is its own 
platform, you should not consider it even Windows unless you are working 
on Cygwin internals.

>>>> I would expect std::filesystem to present paths to construct paths to
>>>> present to underlying library calls such as open ... and on Cygwin,
>>>> open uses Posix style paths.
>>>
>>> I consider that to be a mistake in the implementation of
>>> std::filesystem, because on Windows the preferred style would be smt
>>> like C:/ and then as an opt-out it would consider /cygdrive/c (or
>>> such) as a valid thing as well
>>>
>>
>> Cygwin is not Windows, it runs on Windows, but it is not Windows. You
>> should use mingw if you want Windows style paths. There isn't a magical
>> tool that allows you to have it both ways.
> 
> As I'm trying to say, Cygwin already accepts Windows-style-paths in some circumstances and thus it have SOME magic, so why not extend that and thus make it easier to use Cygwin on its target platform, i.e. Windows ?
> 

Because libstdc++ is not part of Cygwin, it is part of the GCC project, 
developed by completely different developers. GCC runs on Cygwin, so it 
is expected to use standard *nix API, not windows. Windows path 
conversions are entirely unreliable, see the mess that is MSYS, when 
Windows and *nix paths collide.

> As I've said, to use MinGW along with Cygwin would be very tricky without bumping into ODR/memory issues
> 
> I don't think MinGW alone give us enough support alone to keep our code base non platform specific (*nix+windows)
> 
> 

If you are on Cygwin, use *nix APIs and paths, that's the end to it. You 
should not mix Cygwin compiled code with MinGW.

>>
>> Yes, cross platform, you can use std::filesystem on Linux, Cygwin,
>> Windows, Macs, but it certainly cannot do anything that can't be handled
>> for the underlying platform. For Cygwin, it means using *nix paths, not
>> Windows.
> 
> Exactly, so why obfuscate the underlaying platform with a Posix-layer that also can do exactly just what the underlaying platform can and then back to some standard interface again (i.e. std::filesystem) which make a roundtrip that only can result in loss of information/functionality ?
> 

Because Cygwin is exactly that, to emulate POSIX layer on Windows, if 
POSIX mandates exposing Windows paths, then you would see Cygwin 
reworked to support it.

> I rather have a dialogue of how that that could be done rather than "Cygwin is *nix, deal with it" or at least it would be nice to hear if someone do have some insightful thoughts of why it must the way it is or why it would be so hard to make parts (e.g. std::filesystem) more useful ?
> 
> 

That's not what Cygwin is for, you ignore everything while conveniently 
claiming to be looking for "insightful thoughts". You still haven't 
answered where is it in the POSIX standard requires backslashes to be 
used as separator or how are you going to make other *nix platforms 
accept such a change?

Sv: Sv: Sv: Sv: g++ and c++17 filesystem

[sten.kristian.ivarsson]

> On 11/23/20 8:35 AM, sten.kristian.ivarsson@gmail.com wrote:
> >> On 11/20/20 8:31 AM, Kristian Ivarsson via Cygwin wrote:
> >>>> that, for me, /c works.)  Likewise, I would expect the normative
> >>>> path separator to be / not \, and an absolute path to start with /.
> >>>> Windows offers several kinds of symlinks, with varying semantics,
> >>>> so the detailed behavior of that would be affected by the settings
> >>>> in the CYGWIN environment variable
> >>>
> >>> All the implementations of std::filesystem I've seen so far, is
> >>> agnostic to whether / or \ is used as a path separator (but the
> >>> generic form is '/' and a fun fact, the MSVC-implementation of
> >>> std::filesystem handles the generic
> >>> (posix) form more close to the standard specification than GCC)
> >>>
> >>
> >> That is not correct, \ is a valid filename under *nix, but not on
> Windows.
> >> I don't think the standards specify what filenames or character sets
> >> are valid. Cygwin strives for *nix compatibility, anything else is
> secondary.
> >
> >
> > I should have made my point clear; "All the implementations of
> std::filesystem for Windows ..."
> >
> >
> 
> That is an incorrect understanding, Cygwin is NOT Windows, it is its own
> platform, you should not consider it even Windows unless you are working
> on Cygwin internals.

I beg to differ. My claim was that all the std::filesystem implementations I've seen for Windows, except Cygwin, handles both '\' and '/' and that was my point (Cygwin handles it in some circumstances) so that point is perfectly valid

I'm not considering either '/' or '\' in the code. I'm working with a path. I don't care what kind of separators the path handled to the application have. I want std::filesystem to work properly even if the folder-separator in current platform is ¤#¤#¤#¤

> >>>> I would expect std::filesystem to present paths to construct paths
> >>>> to present to underlying library calls such as open ... and on
> >>>> Cygwin, open uses Posix style paths.
> >>>
> >>> I consider that to be a mistake in the implementation of
> >>> std::filesystem, because on Windows the preferred style would be smt
> >>> like C:/ and then as an opt-out it would consider /cygdrive/c (or
> >>> such) as a valid thing as well
> >>>
> >>
> >> Cygwin is not Windows, it runs on Windows, but it is not Windows. You
> >> should use mingw if you want Windows style paths. There isn't a
> >> magical tool that allows you to have it both ways.
> >
> > As I'm trying to say, Cygwin already accepts Windows-style-paths in some
> circumstances and thus it have SOME magic, so why not extend that and thus
> make it easier to use Cygwin on its target platform, i.e. Windows ?
> >
> 
> Because libstdc++ is not part of Cygwin, it is part of the GCC project,
> developed by completely different developers. GCC runs on Cygwin, so it is
> expected to use standard *nix API, not windows. Windows path conversions
> are entirely unreliable, see the mess that is MSYS, when Windows and *nix
> paths collide.

I think you have misunderstood my point completely. I know that they are unreliable and that's what I'm trying to have a discussion about, because I cannot see why it has to be unreliable and how it not could be fixed

> > As I've said, to use MinGW along with Cygwin would be very tricky
> > without bumping into ODR/memory issues
> >
> > I don't think MinGW alone give us enough support alone to keep our
> > code base non platform specific (*nix+windows)
> >
> >
> 
> If you are on Cygwin, use *nix APIs and paths, that's the end to it. You
> should not mix Cygwin compiled code with MinGW.

About the code, that's exactly as I stated 

> >>
> >> Yes, cross platform, you can use std::filesystem on Linux, Cygwin,
> >> Windows, Macs, but it certainly cannot do anything that can't be
> >> handled for the underlying platform. For Cygwin, it means using *nix
> >> paths, not Windows.
> >
> > Exactly, so why obfuscate the underlaying platform with a Posix-layer
> that also can do exactly just what the underlaying platform can and then
> back to some standard interface again (i.e. std::filesystem) which make a
> roundtrip that only can result in loss of information/functionality ?
> >
> 
> Because Cygwin is exactly that, to emulate POSIX layer on Windows, if
> POSIX mandates exposing Windows paths, then you would see Cygwin reworked
> to support it.
> 
> > I rather have a dialogue of how that that could be done rather than
> "Cygwin is *nix, deal with it" or at least it would be nice to hear if
> someone do have some insightful thoughts of why it must the way it is or
> why it would be so hard to make parts (e.g. std::filesystem) more useful ?
> >
> >
> 
> That's not what Cygwin is for, you ignore everything while conveniently
> claiming to be looking for "insightful thoughts". You still haven't
> answered where is it in the POSIX standard requires backslashes to be used
> as separator or how are you going to make other *nix platforms accept such
> a change? 

Did I get a question about where I think that POSIX requires backslashes or did I make such claim ? If one of them, I have missed that question and I have certainly not made any such claim

All I'm saying is that I'd like std::filesystem in Cygwin to work properly and I still cannot see why that cannot happen ?

What would the drawbacks be if std::filesystem worked more transparent (i.e. correct) and made Cygwin more useful ? I don't think the community would shrink !

Kristian

Re: Sv: Sv: Sv: Sv: g++ and c++17 filesystem

[Jonathan Yong]

[-- Attachment #1.1.1: Type: text/plain, Size: 1846 bytes --]

On 11/24/20 9:32 AM, sten.kristian.ivarsson@gmail.com wrote:
>>
>> That's not what Cygwin is for, you ignore everything while conveniently
>> claiming to be looking for "insightful thoughts". You still haven't
>> answered where is it in the POSIX standard requires backslashes to be used
>> as separator or how are you going to make other *nix platforms accept such
>> a change?
> 
> Did I get a question about where I think that POSIX requires backslashes or did I make such claim ? If one of them, I have missed that question and I have certainly not made any such claim
> 
> All I'm saying is that I'd like std::filesystem in Cygwin to work properly and I still cannot see why that cannot happen ?
> 
> What would the drawbacks be if std::filesystem worked more transparent (i.e. correct) and made Cygwin more useful ? I don't think the community would shrink !
> 

There's that proverbial wider audience nonsense. "Sure is nice if Cygwin 
emulates NTVDM too, I don't the community would shrink!"

You wanted to make std::filesystem accept backslashes as separators, 
this is NOT POSIX compliant, it is contrary to the goal of Cygwin. 
std::filesystem POSIX mode is common to all POSIX platforms where 
backslashes are NOT directory separators. How do you make them accept 
your demands? How are you going to force POSIX platforms allow Windows 
specific code? Make it try to enter subdirectories every time 
std::filesystem is called?

You refuse to understand that Cygwin is NOT Windows, it is a POSIX 
platform. Using Cygwin means complying with POSIX expectations and 
standards.

I don't see how this conversation can continue if you still refuse to 
see Cygwin as something separate from Windows. Besides, you have already 
answered your question by ruling out MinGW, so Microsoft Visual Studio 
it is.

[-- Attachment #1.1.2: OpenPGP_0x713B5FE29C145D45_and_old_rev.asc --]
[-- Type: application/pgp-keys, Size: 8035 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Sv: Sv: Sv: Sv: Sv: g++ and c++17 filesystem

[sten.kristian.ivarsson]

[snip]

> std::filesystem POSIX mode is common to all POSIX platforms where
> backslashes are NOT directory separators. How do you make them accept your
> demands? How are you going to force POSIX platforms allow Windows specific
> code?

I've been trying to say over and over again that our code doesn't handle any Windows specific stuff and not anywhere have I claimed that anyone else need to handle Windows specific stuff either (except for the internals of Cygwin of which Windows specific logic is already present)

I repeat; I don't expect any of the Cygwin-Posix-functions to accept any Windows-style-paths (though some of them, as I repeatedly have said, already does so) and I only expect that I can operate according to the C++-standard on an instantiated std::filesystem::path 

> Make it try to enter subdirectories every time std::filesystem is
> called?
> 
> You refuse to understand that Cygwin is NOT Windows, it is a POSIX
> platform. Using Cygwin means complying with POSIX expectations and
> standards.
> 
> I don't see how this conversation can continue if you still refuse to see
> Cygwin as something separate from Windows. Besides, you have already
> answered your question by ruling out MinGW, so Microsoft Visual Studio it
> is.

I repeat (again); neither MinGW/MSVS is an option because we're trying to use Posix and C++

Just to be clear:

- Our code DOESN'T handle Windows-style-paths explicitly in any way
- We DON'T expect Cygwin-Posix-file-related-functions to accept Windows-style-paths
- We WOULD like std::filesystem to work according to the C++ ISO standard


Does that make any sense ? 


Kristian 

Re: Sv: Sv: Sv: Sv: Sv: g++ and c++17 filesystem

[Jonathan Yong]

[-- Attachment #1.1.1: Type: text/plain, Size: 2327 bytes --]

On 11/24/20 11:35 AM, sten.kristian.ivarsson@gmail.com wrote:
> [snip]
> 
>> std::filesystem POSIX mode is common to all POSIX platforms where
>> backslashes are NOT directory separators. How do you make them accept your
>> demands? How are you going to force POSIX platforms allow Windows specific
>> code?
> 
> I've been trying to say over and over again that our code doesn't handle any Windows specific stuff and not anywhere have I claimed that anyone else need to handle Windows specific stuff either (except for the internals of Cygwin of which Windows specific logic is already present)
> 
> I repeat; I don't expect any of the Cygwin-Posix-functions to accept any Windows-style-paths (though some of them, as I repeatedly have said, already does so) and I only expect that I can operate according to the C++-standard on an instantiated std::filesystem::path
> 

How do you expect std::filesystem to do that when Windows paths are not 
even accepted in API calls?

>> Make it try to enter subdirectories every time std::filesystem is
>> called?
>>
>> You refuse to understand that Cygwin is NOT Windows, it is a POSIX
>> platform. Using Cygwin means complying with POSIX expectations and
>> standards.
>>
>> I don't see how this conversation can continue if you still refuse to see
>> Cygwin as something separate from Windows. Besides, you have already
>> answered your question by ruling out MinGW, so Microsoft Visual Studio it
>> is.
> 
> I repeat (again); neither MinGW/MSVS is an option because we're trying to use Posix and C++
> 
> Just to be clear:
> 
> - Our code DOESN'T handle Windows-style-paths explicitly in any way
> - We DON'T expect Cygwin-Posix-file-related-functions to accept Windows-style-paths
> - We WOULD like std::filesystem to work according to the C++ ISO standard

Why would std::filesystem be an exception? How would it know if a 
backslash is part of a name and not a separator? How does it know when 
to apply exceptions? What about mixed paths?

The C++ standard mentions separators, not specific characters, and the 
forward slash is used under Cygwin, not the Windows backslash.

The bigger question would be how would you expect a Cygwin application 
to even accept a Windows paths. It should use *nix paths, as all Cygwin 
programs do.

[-- Attachment #1.1.2: OpenPGP_0x713B5FE29C145D45_and_old_rev.asc --]
[-- Type: application/pgp-keys, Size: 8035 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: Sv: Sv: Sv: Sv: g++ and c++17 filesystem

[Eliot Moss]

On 11/24/2020 4:32 AM, Kristian Ivarsson via Cygwin wrote:

> all the std::filesystem implementations I've seen for Windows

The implementation on top of Cygwin is not "for Windows", it's "for Cygwin", i.e., "for Posix".  And 
for Cygwin that's the right thing to do.  And that's where we keep talking at cross purposes.

Best wishes - Eliot Moss

Sv: Sv: Sv: Sv: Sv: Sv: g++ and c++17 filesystem

[sten.kristian.ivarsson]

> > [snip]
> >
> >> std::filesystem POSIX mode is common to all POSIX platforms where
> >> backslashes are NOT directory separators. How do you make them accept
> >> your demands? How are you going to force POSIX platforms allow
> >> Windows specific code?
> >
> > I've been trying to say over and over again that our code doesn't
> > handle any Windows specific stuff and not anywhere have I claimed that
> > anyone else need to handle Windows specific stuff either (except for
> > the internals of Cygwin of which Windows specific logic is already
> > present)
> >
> > I repeat; I don't expect any of the Cygwin-Posix-functions to accept
> > any Windows-style-paths (though some of them, as I repeatedly have
> > said, already does so) and I only expect that I can operate according
> > to the C++-standard on an instantiated std::filesystem::path
> >
> 
> How do you expect std::filesystem to do that when Windows paths are not
> even accepted in API calls?

What API-calls ? Cygwin-Posix-API-calls ? If so, you assume that std::filesystem in Cygwin must be implemented using the underlaying Cygwin-Posix-API and maybe it must be of some reason, but I guess it even would be possible to make it the other way around, i.e. quite a bunch of the Cygwin-Posix-file-API-functions could be implemented by using some native std::filesystem-implementation such as MinGW but I guess that not the whole Cygwin-Posix-file-API could be implemented that way and it would probably be some quite substantial rewrite of things that probably is not worth the effort, but what I'm trying to say is that std::filesystem shipped with Cygwin not necessarily have to be implemented totally with the underlaying Cygwin-Posix-API. It could be a mix to use whatever is appropriate and in some circumstances some logic is needed to understand both kind of paths


See more below



> >> Make it try to enter subdirectories every time std::filesystem is
> >> called?
> >>
> >> You refuse to understand that Cygwin is NOT Windows, it is a POSIX
> >> platform. Using Cygwin means complying with POSIX expectations and
> >> standards.
> >>
> >> I don't see how this conversation can continue if you still refuse to
> >> see Cygwin as something separate from Windows. Besides, you have
> >> already answered your question by ruling out MinGW, so Microsoft
> >> Visual Studio it is.
> >
> > I repeat (again); neither MinGW/MSVS is an option because we're trying
> > to use Posix and C++
> >
> > Just to be clear:
> >
> > - Our code DOESN'T handle Windows-style-paths explicitly in any way
> > - We DON'T expect Cygwin-Posix-file-related-functions to accept
> > Windows-style-paths
> > - We WOULD like std::filesystem to work according to the C++ ISO
> > standard
> 
> Why would std::filesystem be an exception? How would it know if a
> backslash is part of a name and not a separator? How does it know when to
> apply exceptions? What about mixed paths? 

As I've said before, there are already exceptions. Some Cygwin-Posix-mechanisms already understands both kinds of path's today

How are '\' handled as part of the name today ? I works perfectly well in many circumstances, so I guess that there are quite a few exceptions already 

> The C++ standard mentions separators, not specific characters, and the
> forward slash is used under Cygwin, not the Windows backslash.

Exactly, I haven't stated anything else

> The bigger question would be how would you expect a Cygwin application to
> even accept a Windows paths. It should use *nix paths, as all Cygwin
> programs do

Cygwin (and thus applications build within Cygwin) already do accept Windows paths here and there as well as std::filesystem

Applications don't have control of all the input that could come from UI, environment, configuration, etc

What I'm trying to make a point of is that wouldn't it be good if applications didn't have to care about that, because that is what libraries/frameworks are for, to give an abstraction ?

Otherwise every application have to do something like where ever it handles some path

#ifdef _CYGWIN_
   // fix the path-representation to something with /cygdrive or such
   ...
#endif

This is what we're hoping to avoid to do and that it could be done in the library we're using

The exact details of how to implement this and what challenges that would be is uninteresting before it could be settled as "Yeah, it would be a good feature to let std::filesystem to be path-style-agnostic"

C++ applications do mostly use std::filesystem, std::ofstream, std::ifstream and do not mix that with Posix-file-mechanisms, but for Cygwin, /cygdrive/... in std::filesystem must of course be a valid path as well


I can't stress this enough and point out that this community have already decided to here and there make Cygwin be Windows-style-paths-aware, so I cannot see that it would be that of a bad idea if it became even more agnostic to what kind of path is used, but of course I understand it comes with some challenges


p.s.

   We do have an open source product targeting the *nix-world as well and at the same time we're trying to make it runnable on Windows, so this is not just me be being stubborn

d.s

Kristian

Sv: Sv: Sv: Sv: Sv: g++ and c++17 filesystem

[sten.kristian.ivarsson]

> On 11/24/2020 4:32 AM, Kristian Ivarsson via Cygwin wrote:
> 
> > all the std::filesystem implementations I've seen for Windows
> 
> The implementation on top of Cygwin is not "for Windows", it's "for
> Cygwin", i.e., "for Posix".  And for Cygwin that's the right thing to do.
> And that's where we keep talking at cross purposes.


Maybe it is the right thing to do, but what is your take of why it must be so (all the way) ?

I also do understand that it have several advantages in the implementation of std::filesystem but it also imply an extra layer of abstraction to the underlaying platform, but to just remove the _CYGWIN_ macro when building it would make std::filesystem to not understand /cygdrive at all (and I guess that would confuse other users;-) so I guess it would require some more sophisticated implementation (or extend the number of exceptions already existing in the underlaying Cygwin-Posix-implementation)

Best regards,
Kristian

> Best wishes - Eliot Moss

Re: Sv: Sv: Sv: Sv: Sv: g++ and c++17 filesystem

[Ken Brown]

On 11/24/2020 9:31 AM, Kristian Ivarsson via Cygwin wrote:
>> On 11/24/2020 4:32 AM, Kristian Ivarsson via Cygwin wrote:
>>
>>> all the std::filesystem implementations I've seen for Windows
>>
>> The implementation on top of Cygwin is not "for Windows", it's "for
>> Cygwin", i.e., "for Posix".  And for Cygwin that's the right thing to do.
>> And that's where we keep talking at cross purposes.
> 
> 
> Maybe it is the right thing to do, but what is your take of why it must be so (all the way) ?
> 
> I also do understand that it have several advantages in the implementation of std::filesystem but it also imply an extra layer of abstraction to the underlaying platform, but to just remove the _CYGWIN_ macro when building it would make std::filesystem to not understand /cygdrive at all (and I guess that would confuse other users;-) so I guess it would require some more sophisticated implementation (or extend the number of exceptions already existing in the underlaying Cygwin-Posix-implementation)

I'd like to try to make this discussion more concrete by looking at what's 
actually going on in the test program main.cpp that you posted at the beginning 
of that thread.  (I ran it under strace and gdb to see this for myself.)

First, your program calls std::filesystem::exists, which ultimately calls 
stat(2) in the Cygwin DLL.  The work for this is done in the path_conv::check 
function and various functions that it calls.  These functions have code that 
recognizes Win32 paths like C:\Temp, and std::filesystem::exists therefore 
reports "true".  This is consistent with the statement at 
https://cygwin.com/cygwin-ug-net/using.html#pathnames-win32 about how the Cygwin 
DLL treats Win32 paths.

Next, your program calls std::filesystem::canonical.  This calls 
std::filesystem::absolute, which reports that C:\Temp is not an absolute path. 
It therefore tries to treat it as a relative path and fails with "No such file 
or directory" because <current directory>/C:\Temp does not exist.  Note that the 
Cygwin DLL never sees the original path C:\Temp in this case.  Again, this is 
consistent with the statement in the documentation that Cygwin applications do 
not necessarily recognize Win32 paths.

You say this is a bug, because first you're told that C:\Temp exists and then 
you're told it doesn't.  But I think it just illustrates the hazards of using 
Win32 paths in Cygwin, which tries hard to emulate Posix.  Sometimes you can get 
away with using Win32 paths and sometimes you can't, depending on how and when 
the Cygwin DLL is involved.

I don't see a good way to avoid this inconsistency.  We could change Cygwin so 
that it rigidly recognizes only Posix paths.  Cygwin would then be consistent, 
but we would be removing a feature that many users have become accustomed to. 
And it wouldn't help you.  Or we could ask all Cygwin package maintainers to try 
to patch their packages so that they recognize Win32 paths, but that's simply 
not feasible, nor would many package maintainers be willing to invest the 
required time.

I tried it once with emacs, which I maintain, in response to a user request.  I 
failed miserably.  Every change I made broke something else, and I finally gave up.

I don't think g++ will be any easier than emacs, and I don't think you can 
expect the g++ maintainer to work on this.

Ken

Re: Sv: Sv: Sv: Sv: Sv: g++ and c++17 filesystem

[Eliot Moss]

For the specific case C:\Temp, I found this:

cygpath -ua 'C:\Temp'

   -> /cygdrive/c/Temp

cygpath -ua /cygdrive/c/Temp

   -> /cygdrive/c/Temp

cygpath -ua '\Temp'

   -> /cygdrive/c/Temp

cygpath -ua '/Temp'

   -> /Temp

Now Cygwin is open source, so you, too, could grab the code in cygpath and
call it as a subroutine, maybe, to try to canonicalize a wider range of
paths.

Note the interesting difference between the \Temp and /Temp cases.  cygpath
is processing \Temp as a Windows-like thing, and thus interpreting it
according the the Windows root for the current drive.  However, it processes
/Temp and a Posix-like path, which will be in term of the _Cygwin_ root,
generally the CYgwin install directory.

Still, if you did this canonicalization first, maybe you would get more what
you're looking for?  I'm not sure it would be a good idea to insert this
generally into the Cygwin library stack.  Others more familiar with the fine
details can probably elaborate why.

Regards - Eliot Moss

Re: g++ and c++17 filesystem

[Brian Inglis]

On 2020-11-24 07:31, Kristian Ivarsson via Cygwin wrote:
>> On 11/24/2020 4:32 AM, Kristian Ivarsson via Cygwin wrote:
>>
>>> all the std::filesystem implementations I've seen for Windows
>>
>> The implementation on top of Cygwin is not "for Windows", it's "for
>> Cygwin", i.e., "for Posix".  And for Cygwin that's the right thing to do.
>> And that's where we keep talking at cross purposes.

> Maybe it is the right thing to do, but what is your take of why it must be
> so (all the way) ?

Nobody is interested in doing the work to submit and modify patches to get them 
accepted and change it from the way it is.

> I also do understand that it have several advantages in the implementation
> of std::filesystem but it also imply an extra layer of abstraction to the 
> underlaying platform, but to just remove the _CYGWIN_ macro when building it
>  would make std::filesystem to not understand /cygdrive at all (and I guess 
> that would confuse other users;-) so I guess it would require some more 
> sophisticated implementation (or extend the number of exceptions already 
> existing in the underlaying Cygwin-Posix-implementation)
If you only use POSIX compliant Cygwin or UNC (//) paths, all your programs 
built under Cygwin should run successfully under Windows or Wine.
Otherwise if you built under Cygwin, you can overload your directory and file 
path methods so if a path contains : or \\ you call cygwin_conv_path() with 
appropriate flags to fix your path before using it.
Or build with Mingw or MSVC++ and use whatever those allow.

Cygwin is an open source project maintained solely by volunteer contributors in 
their spare time, and its focus is on working around Windows limitations to 
provide a POSIX standard environment and experience, supporting more POSIX 
features as Windows limitations are removed.

Other projects gcc-g++, libstdc++ may have sponsored or employed contributors.
They all have their respective standards focus and are uninterested in 
non-standard-compliant changes and any non-POSIX build changes.
But they build and run just fine under Cygwin.

Feel free to make the appropriate patches to the components and submit the 
patches to the upstream projects.
But be prepared for the upstream projects to reject the patches if they break 
some standard or compliant test case or use case.
Note that it has taken Intel and then MS five years to get FSGSBASE support 
patches accepted in Linux.

If you are unprepared to do the necessary work, you still get a lot more than 
what you paid for. ;^>

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.
[Data in binary units and prefixes, physical quantities in SI.]

Re: Sv: Sv: Sv: Sv: Sv: Sv: g++ and c++17 filesystem

[Jonathan Yong]

[-- Attachment #1.1.1: Type: text/plain, Size: 6750 bytes --]

On 11/24/20 2:01 PM, sten.kristian.ivarsson@gmail.com wrote:
>>> [snip]
>>>
>>>> std::filesystem POSIX mode is common to all POSIX platforms where
>>>> backslashes are NOT directory separators. How do you make them accept
>>>> your demands? How are you going to force POSIX platforms allow
>>>> Windows specific code?
>>>
>>> I've been trying to say over and over again that our code doesn't
>>> handle any Windows specific stuff and not anywhere have I claimed that
>>> anyone else need to handle Windows specific stuff either (except for
>>> the internals of Cygwin of which Windows specific logic is already
>>> present)
>>>
>>> I repeat; I don't expect any of the Cygwin-Posix-functions to accept
>>> any Windows-style-paths (though some of them, as I repeatedly have
>>> said, already does so) and I only expect that I can operate according
>>> to the C++-standard on an instantiated std::filesystem::path
>>>
>>
>> How do you expect std::filesystem to do that when Windows paths are not
>> even accepted in API calls?
> 
> What API-calls ? Cygwin-Posix-API-calls ? If so, you assume that std::filesystem in Cygwin must be implemented using the underlaying Cygwin-Posix-API and maybe it must be of some reason, but I guess it even would be possible to make it the other way around, i.e. quite a bunch of the Cygwin-Posix-file-API-functions could be implemented by using some native std::filesystem-implementation such as MinGW but I guess that not the whole Cygwin-Posix-file-API could be implemented that way and it would probably be some quite substantial rewrite of things that probably is not worth the effort, but what I'm trying to say is that std::filesystem shipped with Cygwin not necessarily have to be implemented totally with the underlaying Cygwin-Posix-API. It could be a mix to use whatever is appropriate and in some circumstances some logic is needed to understand both kind of paths
> 
> 
> See more below
> 
> 
> 

No, std::filesystem runs on top of Cygwin, not under it, it would have 
to use POSIX APIs to get anything done. GCC is not the platform system 
library.

>>>> Make it try to enter subdirectories every time std::filesystem is
>>>> called?
>>>>
>>>> You refuse to understand that Cygwin is NOT Windows, it is a POSIX
>>>> platform. Using Cygwin means complying with POSIX expectations and
>>>> standards.
>>>>
>>>> I don't see how this conversation can continue if you still refuse to
>>>> see Cygwin as something separate from Windows. Besides, you have
>>>> already answered your question by ruling out MinGW, so Microsoft
>>>> Visual Studio it is.
>>>
>>> I repeat (again); neither MinGW/MSVS is an option because we're trying
>>> to use Posix and C++
>>>
>>> Just to be clear:
>>>
>>> - Our code DOESN'T handle Windows-style-paths explicitly in any way
>>> - We DON'T expect Cygwin-Posix-file-related-functions to accept
>>> Windows-style-paths
>>> - We WOULD like std::filesystem to work according to the C++ ISO
>>> standard
>>
>> Why would std::filesystem be an exception? How would it know if a
>> backslash is part of a name and not a separator? How does it know when to
>> apply exceptions? What about mixed paths?
> 
> As I've said before, there are already exceptions. Some Cygwin-Posix-mechanisms already understands both kinds of path's today
> 
> How are '\' handled as part of the name today ? I works perfectly well in many circumstances, so I guess that there are quite a few exceptions already
> 

Its not able to read your mind to find out your intentions, especially 
when it comes to ambiguous strings like UNC or SMB paths, none of these 
are remotely POSIX compliant.

>> The C++ standard mentions separators, not specific characters, and the
>> forward slash is used under Cygwin, not the Windows backslash.
> 
> Exactly, I haven't stated anything else
> 

So use the forward slash, not backslash.

>> The bigger question would be how would you expect a Cygwin application to
>> even accept a Windows paths. It should use *nix paths, as all Cygwin
>> programs do
> 
> Cygwin (and thus applications build within Cygwin) already do accept Windows paths here and there as well as std::filesystem
> 

It does not. There are giant warnings telling you not to rely on it.

> Applications don't have control of all the input that could come from UI, environment, configuration, etc
> 

Garbage in, garbage out. The warnings tell you to use Unix paths.

> What I'm trying to make a point of is that wouldn't it be good if applications didn't have to care about that, because that is what libraries/frameworks are for, to give an abstraction ?
> 
> Otherwise every application have to do something like where ever it handles some path
> 
> #ifdef _CYGWIN_
>     // fix the path-representation to something with /cygdrive or such
>     ...
> #endif
> 

Why? Cygwin is a POSIX system. The prefix isn't even necessarily 
/cygdrive and you're just introducing more complications that falls 
apart the moment the defaults change.

> This is what we're hoping to avoid to do and that it could be done in the library we're using
> 
> The exact details of how to implement this and what challenges that would be is uninteresting before it could be settled as "Yeah, it would be a good feature to let std::filesystem to be path-style-agnostic"
> 
> C++ applications do mostly use std::filesystem, std::ofstream, std::ifstream and do not mix that with Posix-file-mechanisms, but for Cygwin, /cygdrive/... in std::filesystem must of course be a valid path as well
> 
> 

The C++ standard library APIs are wrappers around the Cygwin APIs like 
open(3) and write(3), so a Windows path there would still fail. Just 
because it comes from the C++ standard library, doesn't mean its 
magically separated from the underlying system.

> I can't stress this enough and point out that this community have already decided to here and there make Cygwin be Windows-style-paths-aware, so I cannot see that it would be that of a bad idea if it became even more agnostic to what kind of path is used, but of course I understand it comes with some challenges
> 

No, Windows paths are not supported, merely working as a side effect. 
Are you going to demand POSIX compliant scripts to also handle Windows 
paths next?

> 
> p.s.
> 
>     We do have an open source product targeting the *nix-world as well and at the same time we're trying to make it runnable on Windows, so this is not just me be being stubborn
> 

But Cygwin is NOT Windows, you still refuse to accept this fact. Cygwin 
programs have no reason to act as Windows applications, Cygwin programs 
act like Cygwin programs.

[-- Attachment #1.1.2: OpenPGP_0x713B5FE29C145D45_and_old_rev.asc --]
[-- Type: application/pgp-keys, Size: 8035 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Sv: Sv: Sv: Sv: Sv: Sv: g++ and c++17 filesystem

[sten.kristian.ivarsson]

> For the specific case C:\Temp, I found this:
> 
> cygpath -ua 'C:\Temp'
> 
>    -> /cygdrive/c/Temp
> 
> cygpath -ua /cygdrive/c/Temp
> 
>    -> /cygdrive/c/Temp
> 
> cygpath -ua '\Temp'
> 
>    -> /cygdrive/c/Temp
> 
> cygpath -ua '/Temp'
> 
>    -> /Temp
> 
> Now Cygwin is open source, so you, too, could grab the code in cygpath and
> call it as a subroutine, maybe, to try to canonicalize a wider range of
> paths.

Yeah, thanx for the tip, but this functionality won't be available on other
*nix-platforms, so some kind of #ifdef is required

> Note the interesting difference between the \Temp and /Temp cases.
> cygpath is processing \Temp as a Windows-like thing, and thus interpreting
> it according the the Windows root for the current drive.  However, it
> processes /Temp and a Posix-like path, which will be in term of the
> _Cygwin_ root, generally the CYgwin install directory.
> 
> Still, if you did this canonicalization first, maybe you would get more
> what you're looking for?  I'm not sure it would be a good idea to insert
> this generally into the Cygwin library stack.  Others more familiar with
> the fine details can probably elaborate why.

I experimented by fiddling around with making the path canonical and as you
say, maybe we will be more successful and that could still be a part of our
code that is not platform dependant

I guess we need to go out of std::filesystem::canonical though because that
is what fails (and unfortunately I guess there's a real bug in the (real)
g++ distro of <filesystem> because the
std::filesystem::path::generic-functions doesn't work as the standard
mandates either)


Once again, thanx for the tips


Best regards,
Kristian


> Regards - Eliot Moss
> --
> Problem reports:      https://cygwin.com/problems.html
> FAQ:                  https://cygwin.com/faq/
> Documentation:        https://cygwin.com/docs.html
> Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple

Sv: Sv: Sv: Sv: Sv: Sv: g++ and c++17 filesystem

[sten.kristian.ivarsson]

Thanx for the insightful thoughts Ken

See more below

> >>> all the std::filesystem implementations I've seen for Windows
> >>
> >> The implementation on top of Cygwin is not "for Windows", it's "for
> >> Cygwin", i.e., "for Posix".  And for Cygwin that's the right thing to
> do.
> >> And that's where we keep talking at cross purposes.
> >
> >
> > Maybe it is the right thing to do, but what is your take of why it must
> be so (all the way) ?
> >
> > I also do understand that it have several advantages in the
> > implementation of std::filesystem but it also imply an extra layer of
> > abstraction to the underlaying platform, but to just remove the
> > _CYGWIN_ macro when building it would make std::filesystem to not
> > understand /cygdrive at all (and I guess that would confuse other
> > users;-) so I guess it would require some more sophisticated
> > implementation (or extend the number of exceptions already existing in
> > the underlaying Cygwin-Posix-implementation)
> 
> I'd like to try to make this discussion more concrete by looking at what's
> actually going on in the test program main.cpp that you posted at the
> beginning of that thread.  (I ran it under strace and gdb to see this for
> myself.)
> 
> First, your program calls std::filesystem::exists, which ultimately calls
> stat(2) in the Cygwin DLL.  The work for this is done in the
> path_conv::check function and various functions that it calls.  These
> functions have code that recognizes Win32 paths like C:\Temp, and
> std::filesystem::exists therefore reports "true".  This is consistent with
> the statement at
> https://cygwin.com/cygwin-ug-net/using.html#pathnames-win32 about how the
> Cygwin DLL treats Win32 paths.
> 
> Next, your program calls std::filesystem::canonical.  This calls
> std::filesystem::absolute, which reports that C:\Temp is not an absolute
> path.
> It therefore tries to treat it as a relative path and fails with "No such
> file or directory" because <current directory>/C:\Temp does not exist.
> Note that the Cygwin DLL never sees the original path C:\Temp in this
> case.  Again, this is consistent with the statement in the documentation
> that Cygwin applications do not necessarily recognize Win32 paths.
> 
> You say this is a bug, because first you're told that C:\Temp exists and
> then you're told it doesn't.  But I think it just illustrates the hazards
> of using
> Win32 paths in Cygwin, which tries hard to emulate Posix.  Sometimes you
> can get away with using Win32 paths and sometimes you can't, depending on
> how and when the Cygwin DLL is involved.

Well, to call it a bug in this context was maybe wrong, but in a
ISO-standard perspective it would be considered a bug ;-)

> I don't see a good way to avoid this inconsistency.  We could change
> Cygwin so that it rigidly recognizes only Posix paths.  Cygwin would then
> be consistent, but we would be removing a feature that many users have
> become accustomed to

I guess so too, but they are already there for some reason

I don't expect that the posix-functions should accept any non-posix-paths
but it would be nice to have the platform independent std::filesystem to
work transparently but I guess, since the cygstdc++-library uses the
underlaying posix-functions the only reasonable way to accomplish this is to
extend the list of features that already exists

The sole reason for std::filesystem was to avoid platform dependent code

So I guess in practical, it is up to the community of how well motivated
they are to extend those Posix-functions but my take in this thread is that
many rather would like it to become more ... like an emulator 

> And it wouldn't help you.  Or we could ask all Cygwin package maintainers
> to try to patch their packages so that they recognize Win32 paths, but
> that's simply not feasible, nor would many package maintainers be willing
> to invest the required time.

I'm not sure what package maintainers you're referring to here now ?
Applications ? Cygwin ? GCC ?

> I tried it once with emacs, which I maintain, in response to a user
> request.  I failed miserably.  Every change I made broke something else,
> and I finally gave up.
> 
> I don't think g++ will be any easier than emacs, and I don't think you can
> expect the g++ maintainer to work on this.

Yeah, I noticed that the _CYGWIN_ macro actually was a part of the real
libstdc++-distro

Thanx Ken and keep up the good work

Kristian

> Ken

Sv: g++ and c++17 filesystem

[sten.kristian.ivarsson]

> >>> all the std::filesystem implementations I've seen for Windows
> >>
> >> The implementation on top of Cygwin is not "for Windows", it's "for
> >> Cygwin", i.e., "for Posix".  And for Cygwin that's the right thing to
> do.
> >> And that's where we keep talking at cross purposes.
> 
> > Maybe it is the right thing to do, but what is your take of why it
> > must be so (all the way) ?
> 
> Nobody is interested in doing the work to submit and modify patches to get
> them accepted and change it from the way it is.

I totally understand that it might be a big task, but (as I said) I cannot
understand the reluctancy of even wanting the feature (as far as I
experience;-)

One might ask what the use case for Cygwin is ? We don't need to go into
this rabbit hole though ;-)

See more below


> > I also do understand that it have several advantages in the
> > implementation of std::filesystem but it also imply an extra layer of
> > abstraction to the underlaying platform, but to just remove the
> > _CYGWIN_ macro when building it  would make std::filesystem to not
> > understand /cygdrive at all (and I guess that would confuse other
> > users;-) so I guess it would require some more sophisticated
> > implementation (or extend the number of exceptions already existing in
> > the underlaying Cygwin-Posix-implementation)
> If you only use POSIX compliant Cygwin or UNC (//) paths, all your
> programs built under Cygwin should run successfully under Windows or Wine.
> Otherwise if you built under Cygwin, you can overload your directory and
> file path methods so if a path contains : or \\ you call
> cygwin_conv_path() with appropriate flags to fix your path before using
> it

Thanx for the tip, maybe I'll look into that, but I guess we need to have
our own distro of Cygwin then (or could this be set for changing the
behaviour for own built applications) ?

> Or build with Mingw or MSVC++ and use whatever those allow.
> 
> Cygwin is an open source project maintained solely by volunteer
> contributors in their spare time, and its focus is on working around
> Windows limitations to provide a POSIX standard environment and
> experience, supporting more POSIX features as Windows limitations are
> removed.

I'm aware of that and I'm asking these question because I'm working on an
open source project as well (so this is pure voluntarily as well:-),
targeting *nix-system, but we have a task to make it working on Windows as
well and we were hoping to not have the need to add platform specific code
(to "clean up" input paths that are perfectly normal for users using
Windows)

> Other projects gcc-g++, libstdc++ may have sponsored or employed
> contributors.
> They all have their respective standards focus and are uninterested in
> non-standard-compliant changes and any non-POSIX build changes.
> But they build and run just fine under Cygwin. 

Yeah, I discovered that the _CYGWIN_ defines actually were in the (real)
libstdc++-distro, so I totally understand that

> Feel free to make the appropriate patches to the components and submit the
> patches to the upstream projects.
> But be prepared for the upstream projects to reject the patches if they
> break some standard or compliant test case or use case.
> Note that it has taken Intel and then MS five years to get FSGSBASE
> support patches accepted in Linux.

Thanx for the more concrete tips and maybe I'll do that some day

> If you are unprepared to do the necessary work, you still get a lot more
> than what you paid for. ;^> 

Ha haa, you're probably right, but I can tap myself on the shoulder and feel
that I made the world a bit better

I'm totally used to have pull-requests and enhancement-issues to open source
projects laying around for ages before approved

Thanx Brian

Best regards,
Kristian

> --
> Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada
> 
> This email may be disturbing to some readers as it contains too much
> technical detail. Reader discretion is advised.
> [Data in binary units and prefixes, physical quantities in SI.]
> --
> Problem reports:      https://cygwin.com/problems.html
> FAQ:                  https://cygwin.com/faq/
> Documentation:        https://cygwin.com/docs.html
> Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple