[bug] coreutils: potentially dangerous: $(realpath //) != /

[bug] coreutils: potentially dangerous: $(realpath //) != /

[Mikhail Usenko via cygwin]

coreutils-8.26-2
cygwin-2.10.0-1

Test case 1:
-------------------
$ realpath //
//
-------------------
Expected output: /

Test case 2:
-------------------
$ MYDIR=//
$ test "$(realpath -e "$MYDIR" )" != /  &&  echo rm -rf "$MYDIR"/*
rm -rf ///bin ///cygdrive ///Cygwin.bat ///Cygwin.ico ///Cygwin-Terminal.ico ///dev ///etc ///home ///lib ///proc ///sbin ///tmp ///usr ///var
-------------------
Expected output: no output


-- 


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: [bug] coreutils: potentially dangerous: $(realpath //) != /

[Eric Blake]

On 03/12/2018 03:28 PM, Mikhail Usenko via cygwin wrote:
> coreutils-8.26-2
> cygwin-2.10.0-1
> 
> Test case 1:
> -------------------
> $ realpath //
> //

Correct.

> -------------------
> Expected output: /

Wrong.

On cygwin, '/' and '//' are two different directories, as allowed by 
POSIX. Converting // into / is a violation of the POSIX requirements on 
pathname resolution on systems where // is distinct from /.

> 
> Test case 2:
> -------------------
> $ MYDIR=//
> $ test "$(realpath -e "$MYDIR" )" != /  &&  echo rm -rf "$MYDIR"/*

Wrong.  If you want to guarantee that an arbitrary variable concatenated 
with / and then a glob expands into the expected directory, then YOU 
have to prefilter the arbitrary variable to make sure it does not 
consist solely of slashes.  Or, if you don't want to remove anything 
from the special // directory, you can guarantee that your rm attempt 
will operate on / by supplying more than one slash yourself, as in rm 
-rf "$MYDIR"///* (if MYDIR is empty, that is equivalent to '/*'; if 
MYDIR is non-empty but not //, then you are guaranteed that the * 
expands relative to MYDIR, and only when MYDIR is exactly // do you fail 
to glob what was expected).

At any rate, realpath is not broken, but only your expectations.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: [bug] coreutils: potentially dangerous: $(realpath //) != /

[Achim Gratz]

Mikhail Usenko via cygwin writes:
> coreutils-8.26-2
> cygwin-2.10.0-1
>
> Test case 1:
> -------------------
> $ realpath //
> //
> -------------------
> Expected output: /

Adjust your expectations, a double slash at the beginning signifies a
network path (as expressedly allowed for in POSIX) and can't be
simplified.

> Test case 2:
> -------------------
> $ MYDIR=//
> $ test "$(realpath -e "$MYDIR" )" != /  &&  echo rm -rf "$MYDIR"/*
> rm -rf ///bin ///cygdrive ///Cygwin.bat ///Cygwin.ico ///Cygwin-Terminal.ico ///dev ///etc ///home ///lib ///proc ///sbin ///tmp ///usr ///var
> -------------------
> Expected output: no output

This goes to show that you really, really, really want to understand the corner
cases in this script.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf microQ V2.22R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: [bug] coreutils: potentially dangerous: $(realpath //) != /

[Mikhail Usenko via cygwin]

On Mon, 12 Mar 2018 21:47:09 +0100
Achim Gratz <...> wrote:
> This goes to show that you really, really, really want to understand the corner
> cases in this script.

On Mon, 12 Mar 2018 15:41:00 -0500
Eric Blake <...> wrote:
> On cygwin, '/' and '//' are two different directories, as allowed by 
> POSIX. Converting // into / is a violation of the POSIX requirements on 
> pathname resolution on systems where // is distinct from /.
> ...
> If you want to guarantee that an arbitrary variable concatenated 
> with / and then a glob expands into the expected directory, then YOU 
> have to prefilter the arbitrary variable to make sure it does not 
> consist solely of slashes. 

Well, guys my expextations are that the program/script execution conditions
in Cygwin should be the same as in other popular POSIX systems
(namely it is Linux) and, as a consequence, I should avoid the case where
somewhere in a bash script evaluating of some variable would lead to one of
the following command to execute:
rm -rf /*
rm -rf //*
rm -rf ///*
rm -rf ////*
that all do the very same thing in Linux.

If at some time it was considered that
$ ls //Server/Folder
in Cygwin should behave like
> net view \\Server\Folder
in Windows to be able to view network shares, I belive this could be realized
with using of some special folder e.g. /cygnetwork, like /cygdrive does it for
Windows drives.
Supporting of UNC paths that are actually off the POSIX unified tree, directly
for POSIX user commands, breaks consistency with Linux user environment, which
is the feature of Cygwin that is highly valued by the most of the users.



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: [bug] coreutils: potentially dangerous: $(realpath //) != /

[Eric Blake]

On 03/12/2018 06:38 PM, Mikhail Usenko via cygwin wrote:
> Well, guys my expextations are that the program/script execution conditions
> in Cygwin should be the same as in other popular POSIX systems
> (namely it is Linux)

POSIX says that the behavior of leading // is implementation-defined 
(that is, an implementation MUST document somewhere what it means).

http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap04.html 
4.13 Pathname Resolution
"A pathname consisting of a single <slash> shall resolve to the root 
directory of the process. A null pathname shall not be successfully 
resolved. If a pathname begins with two successive <slash> characters, 
the first component following the leading <slash> characters may be 
interpreted in an implementation-defined manner, although more than two 
leading <slash> characters shall be treated as a single <slash> character."

Just because Linux has taken the stance that their documented definition 
of // is "synonym for /" does NOT mean that ALL POSIX systems have taken 
the same approach; Cygwin has taken the approach that "// is documented 
to be the root of network access points, distinct from /".

POSIX allows leeway between implementations; this is one of those 
documented places where they differ, yet are still both POSIX compliant 
with their different choices.  If your script is not robust to what 
POSIX has already warned you about, fix your script.

> and, as a consequence, I should avoid the case where
> somewhere in a bash script evaluating of some variable would lead to one of
> the following command to execute:
> rm -rf /*
> rm -rf //*
> rm -rf ///*
> rm -rf ////*
> that all do the very same thing in Linux.

3 of the 4 do the same thing on ALL POSIX platforms.  The only one that 
has implementation-defined behavior is 'rm -rf //*', which is different 
on systems where // is distinct (such as Cygwin).  All the other 
spellings (attempt) to remove all non-hidden files from the root directory.

Furthermore, you need to realize that GNU coreutils 'rm' already has 
special logic (permitted by POSIX) such that 'rm -rf /' fails unless you 
use --no-preserve-root ('rm -rf /*' unfortunately does not trigger the 
special logic, only an attempt to directly remove the root directory is 
flagged - but you're also forgetting that /* does not necessarily list 
all files under /, thanks to hidden files, and that 'rm -rf "$dir"' is 
always better than 'rm -rf "$dir"/*' if you are worried about hidden 
files in $dir and/or command-line length limits due to the glob 
expansion of *).  And coreutils, as packaged for Cygwin, has been 
patched to extend the special root-recognition logic for / to ALSO 
recognize // as a case where recursive removal should fail without 
--no-preserve-root (again, only when spelled 'rm -rf //', not 'rm -rf //*').

> 
> If at some time it was considered that
> $ ls //Server/Folder
> in Cygwin should behave like
>> net view \\Server\Folder
> in Windows to be able to view network shares,

Yes, that's what Cygwin's // is documented to behave like.

> I belive this could be realized
> with using of some special folder e.g. /cygnetwork, like /cygdrive does it for
> Windows drives.

Patches are considered, but for now, we aren't going to change behavior 
without a concrete implementation of an alternative behavior.  And 
changing it now (instead of 15 or so years ago, when Cygwin // was first 
given its current behavior) would likely break existing users that 
expect the current behavior.  So feel free to write a patch, but it may 
be an uphill battle to prove it has merit.

> Supporting of UNC paths that are actually off the POSIX unified tree, directly
> for POSIX user commands, breaks consistency with Linux user environment, which
> is the feature of Cygwin that is highly valued by the most of the users.

Use of // in Linux is already on shaky ground, given that POSIX has 
already declared it to be implementation-defined.  Any implementation is 
also free to define // as an error that never resolves to a path; such 
an implementation would still comply with POSIX, but break expectations 
of scripts that are written to cater to only the Linux and Cygwin behaviors.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: [bug] coreutils: potentially dangerous: $(realpath //) != /

[Mikhail Usenko via cygwin]

On Mon, 12 Mar 2018 20:43:13 -0500
Eric Blake <...> wrote:

> Furthermore, you need to realize that GNU coreutils 'rm' already has 
> special logic (permitted by POSIX) such that 'rm -rf /' fails unless you 
> use --no-preserve-root ('rm -rf /*' unfortunately does not trigger the 
> special logic, only an attempt to directly remove the root directory is 
> flagged - but you're also forgetting that /* does not necessarily list 
> all files under /, thanks to hidden files, and that 'rm -rf "$dir"' is 
> always better than 'rm -rf "$dir"/*' if you are worried about hidden 
> files in $dir and/or command-line length limits due to the glob 
> expansion of *).
> 
I don't know what this is for, but nevertheless,
thank you for your efforts to eliminate of illiteracy among readers...

> Just because Linux has taken the stance that their documented definition 
> of // is "synonym for /" does NOT mean that ALL POSIX systems have taken 
> the same approach; Cygwin has taken the approach that "// is documented 
> to be the root of network access points, distinct from /".
> 
> POSIX allows leeway between implementations; this is one of those 
> documented places where they differ, yet are still both POSIX compliant 
> with their different choices.  If your script is not robust to what 
> POSIX has already warned you about, fix your script.
> 
> > and, as a consequence, I should avoid the case where
> > somewhere in a bash script evaluating of some variable would lead to one of
> > the following command to execute:
> > rm -rf /*
> > rm -rf //*
> > rm -rf ///*
> > rm -rf ////*
> > that all do the very same thing in Linux.
> 
> 3 of the 4 do the same thing on ALL POSIX platforms.  The only one that 
> has implementation-defined behavior is 'rm -rf //*', which is different 
> on systems where // is distinct (such as Cygwin).  All the other 
> spellings (attempt) to remove all non-hidden files from the root directory.
> 
If you really claims that Cygwin may and should be different and distinct from all other
existing POSIX systems (the more so that it is allowed by POSIX),
then it would probably be more obvious and clear to say this at the very begining, e.g.
"Get that Linux feeling (with all those differents and distinctions) - on Windows"

> Use of // in Linux is already on shaky ground, given that POSIX has 
> already declared it to be implementation-defined.
Shaky ground? There must be some good reasons (beyond the declaration of such an abstarct
possibility) that it should to be done differently than it has been so far.


-- 

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: [bug] coreutils: potentially dangerous: $(realpath //) != /

[Brian Inglis]

On 2018-03-14 04:58, Mikhail Usenko via cygwin wrote:
> On Mon, 12 Mar 2018 20:43:13 -0500
> Eric Blake <...> wrote:
> I don't know what this is for, but nevertheless,
> thank you for your efforts to eliminate of illiteracy among readers...

>> POSIX allows leeway between implementations; this is one of those 
>> documented places where they differ, yet are still both POSIX compliant 
>> with their different choices.  If your script is not robust to what 
>> POSIX has already warned you about, fix your script.
>>
>>> and, as a consequence, I should avoid the case where
>>> somewhere in a bash script evaluating of some variable would lead to one of
>>> the following command to execute:
>>> rm -rf /*
>>> rm -rf //*
>>> rm -rf ///*
>>> rm -rf ////*
>>> that all do the very same thing in Linux.
>>
>> 3 of the 4 do the same thing on ALL POSIX platforms.  The only one that 
>> has implementation-defined behavior is 'rm -rf //*', which is different 
>> on systems where // is distinct (such as Cygwin).  All the other 
>> spellings (attempt) to remove all non-hidden files from the root directory.
>>
> If you really claims that Cygwin may and should be different and distinct from all other
> existing POSIX systems (the more so that it is allowed by POSIX),
> then it would probably be more obvious and clear to say this at the very begining, e.g.
> "Get that Linux feeling (with all those differents and distinctions) - on Windows"
> 
>> Use of // in Linux is already on shaky ground, given that POSIX has 
>> already declared it to be implementation-defined.
> Shaky ground? There must be some good reasons (beyond the declaration of such an abstarct
> possibility) that it should to be done differently than it has been so far.

POSIX Rationale for Pathname Resolution:
http://pubs.opengroup.org/onlinepubs/9699919799/xrat/V4_xbd_chap04.html#tag_21_04_13

Notes:
POSIX agrees to standardize what is common among their IEEE, OpenGroup, ISO, and
IEC members implementations.

Eric, and some of the other maintainers on Cygwin lists, are also participants
in POSIX, so they have background, knowledge, and experience there.

Background:
BSD diverged from AT&T to support TCP/IP networking and sockets instead of telco
OSI and streams, then each of the BSDs went their own ways; many vendors joined
consortiums to agree on features, took some of these approaches, and implemented
them with their own proprietary tweaks; Linux was based on one of these
implementations (Solaris, with SysV unification on SunOS BSD base); the rest was
based on GNU, designed to remove limits built into the SysV and BSD libraries
and utilities (I got into this stuff after I hit 128... line length and other
limits in various utilities and systems): the world is not only Linux or GNU;
the commercial world is heavily SysV and/or BSD based; guess who pays to
participate?
IBM, MS, HPE, and others participate to ensure that POSIX maintains conformance
for their systems, so they comply to US FIPS, and can bid them on US .gov and
.mil contracts. (WSL anyone?) ;^>

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: [bug] coreutils: potentially dangerous: $(realpath //) != /

[Thomas Wolff]

Am 14.03.2018 um 11:58 schrieb Mikhail Usenko via cygwin:
> On Mon, 12 Mar 2018 20:43:13 -0500
> Eric Blake <...> wrote:
>
> ...
>> Just because Linux has taken the stance that their documented definition
>> of // is "synonym for /" does NOT mean that ALL POSIX systems have taken
>> the same approach; Cygwin has taken the approach that "// is documented
>> to be the root of network access points, distinct from /".
>>
>> POSIX allows leeway between implementations; this is one of those
>> documented places where they differ, yet are still both POSIX compliant
>> with their different choices.  If your script is not robust to what
>> POSIX has already warned you about, fix your script.
>>
>>> ...
> If you really claims that Cygwin may and should be different and distinct from all other
> existing POSIX systems (the more so that it is allowed by POSIX),
which is not the case. There are other systems where // is the network root.
> then it would probably be more obvious and clear to say this at the very begining, e.g.
> "Get that Linux feeling (with all those differents and distinctions) - on Windows"
Considering that due to the limitations of being embedded in Windows, 
Cygwin cannot always perfectly mimic either Linux or POSIX systems, I 
personally prefer the more generic approach to define POSIX as its model.
------
Thomas

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: [bug] coreutils: potentially dangerous: $(realpath //) != /

[Brian Inglis]

On 2018-03-15 01:11, Thomas Wolff wrote:
> Am 14.03.2018 um 11:58 schrieb Mikhail Usenko via cygwin:
>> On Mon, 12 Mar 2018 20:43:13 -0500
>> Eric Blake <...> wrote:
>>> Just because Linux has taken the stance that their documented definition
>>> of // is "synonym for /" does NOT mean that ALL POSIX systems have taken
>>> the same approach; Cygwin has taken the approach that "// is documented
>>> to be the root of network access points, distinct from /".
>>>
>>> POSIX allows leeway between implementations; this is one of those
>>> documented places where they differ, yet are still both POSIX compliant
>>> with their different choices.  If your script is not robust to what
>>> POSIX has already warned you about, fix your script.

>> If you really claims that Cygwin may and should be different and distinct
>> from all other existing POSIX systems (the more so that it is allowed by
>> POSIX),
> which is not the case. There are other systems where // is the network root.
>> then it would probably be more obvious and clear to say this at the very 
>> beginning, e.g. "Get that Linux feeling (with all those differences and
>> distinctions) - on Windows"
> Considering that due to the limitations of being embedded in Windows, Cygwin
> cannot always perfectly mimic either Linux or POSIX systems, I personally prefer
> the more generic approach to define POSIX as its model.

It's comforting to know that if a script arg is supplied as / and used as the
target of rmdir $v/ or rm -rf $v/..., it won't do anything.
(Although any such args should always be canonicalized, then checked to ensure
they are not, whether deliberately or inadvertently, null, $PWD or any prefix,
/, or other significant standard paths.)

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple