wget seemingly modifies file access permissions on XP

wget seemingly modifies file access permissions on XP

[mihau]

hello,

(my setup is latest cygwin 32bit on Win XP SP3, logged in as Administrator)

I have just updated my cygwin installation after a year or so and stumbled upon
weird behavior with its wget build. it seems to screw file access permissions on
downloaded files, rendering e.g. downloaded exe files unusable (because even the
administrator doesnt have the rights to execute them).

this hasn't happened before today, but unfortunately I cannot tell a. whether wget
was updated today at all and if it was, b. which version I had installed before.

I'm aware that attaching log files is the preferred way here, but in this
case I have made screenshots because they show perfectly what's going on.
so from within cmd.exe this simple command 'C:\>wget http://mihau.de/' is executed.

screenshots of the corresponding "file properties | security tab" dialogs:
http://mihau.de/properties_dialog.png
left side: index.html downloaded with both 1.16.3 and 1.17.1 versions of GNU Wget built
on cygwin. note that "Users" is also there but out of visible range and I cannot resize
the dialog for the screenshot.
right side: the same file downloaded with 1.17.1 built on mingw32, as dowloaded from
https://eternallybored.org/misc/wget/ (wget-1.17.1-win32.zip to be exact)

seconds before hitting "send" here, I realized that the problem also exists if wget
is run from within bash/mintty; in this case, there's even more "ghost" groups added
as you can see in the screenshots above: there's one additional group "Power Users"
and sometimes also a very long, numeric entry such as "S-457547547457-3434636436-"
whcih disappears after a second or so.

I tried uninstalling, then reinstalling wget with setup*.exe, it didn't help.
the only workaround I can currently come up with is simply replacing the cygwin
build with the ming32 one; seems to work fine so far.

hope my description/problem explanation was accurate enough.

does anyone have an idea what may have happened that causes this?

many thanks
Michael

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: wget seemingly modifies file access permissions on XP

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 766 bytes --]

On Mar 30 16:39, mihau wrote:
> hello,
> 
> (my setup is latest cygwin 32bit on Win XP SP3, logged in as Administrator)
> 
> I have just updated my cygwin installation after a year or so and stumbled upon
> weird behavior with its wget build. it seems to screw file access permissions on
> downloaded files, rendering e.g. downloaded exe files unusable (because even the
> administrator doesnt have the rights to execute them).

Remove your /etc/passwd and /etc/group files.  Don't regeneate them,
they are not required anymore (https://cygwin.com/cygwin-ug-net/ntsec.html)
Then try again.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: wget seemingly modifies file access permissions on XP

[Herbert Stocker]

Hi mihau,

On 30.03.2016 16:39, mihau wrote:
 > hello,
 >
 > (my setup is latest cygwin 32bit on Win XP SP3, logged in as
 > Administrator)
 >
 > I have just updated my cygwin installation after a year or so and
 > stumbled upon weird behavior with its wget build. it seems to
 > screw file access permissions on downloaded files, rendering e.g.
 > downloaded exe files unusable (because even the administrator
 > doesnt have the rights to execute them).
 >
 > this hasn't happened before today, but unfortunately I cannot tell
 > a. whether wget was updated today at all and if it was, b. which
 > version I had installed before.
 >

And on 30.03.2016 19:55, mihau wrote:
 > any idea why wget cygwin interferes with those things though?
 > in my understanding it should just copy/save a file from a remote
 > server to a local computer, without even touching file permissions or
 > such. several windows batch files which I use now, either have to be
 > edited and "chmod +x" added in all of them - or I'll just stick with
 > wget mingw.

i just tried wget on a Linux box, and it also does not add x flags to
an .exe downloaded.

What wget does is downloading a file from an HTTP server and storing it
with default permissions. These seam to be 666 (rw-rw-rw-) with some of
the bits removed by your umask setting. (There is no x flag in (rw-rw-rw-).

We are talking about 666 or rwxrwxrwx style permissions because that's
the POSIX way of expressing permissions (untless ACLs are used).

And wget for Cygwin is a POSIX program and Cygwin translates the POSIX
system calls from wget (which are create file, write data to file,
close file, etc) to Win32 system calls that Windows can process. It
tries hard to provide a perfect POSIX environment to its applications.

On HTTP a file is just a bunch of bytes with a last modified time stamp
and a content type. HTTP does not transfer permission bits.
So if you say
 > it should just copy/save a file from a remote server
 > to a local computer, without even touching file permissions or such.
then this is what wget does. It downloads the file and stores it with 
default permissions.

i also find it a bit inconvenient to use chmod +x but that's correct
behaviour for wget.

And i just found out that a chmod +x is also necessary when extracting
ZIP files with unzip.


hth,

Herbert Stocker



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: wget seemingly modifies file access permissions on XP

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2101 bytes --]

On Mar 30 19:55, mihau wrote:
> >Btw., if you wget a file it's *supposed* to be non-executable.  Assuming
> >your umask is 0022, the file will be create with 0644 permissions.  The
> >default ACL created in this case explicitely disables execution for the
> >admin user if the admin user is part of the ACL.  Just run `chmod +x'
> >afterwards if that's the only problem.
> 
> I specifically meant exe as in windows executables, files w/ .exe suffix;
> just tried "chmod +x" and you are correct, it works, my helloworld.exe
> became executable indeed. thank you for that.
> 
> any idea why wget cygwin interferes with those things though?
> in my understanding it should just copy/save a file from a remote server
> to a local computer, without even touching file permissions or such.

That's not how it works.  Consider that the file wget downloads
doesn't exist yet on your side.  When creating the file on your
local machine, wget has to call open(2) with a set of permisssions.
The set of permissions given by wget does *not* contain an execute
bit.

The old permission implementation in Cygwin ignored this fact for
ACEs inherited from the parent directory.  This, however, leads to
incorrect POSIX permission bits if implemanted correctly.  The
recent versions of Cygwin starting with version 2.4.0 strive to
implement the POSIX permission and ACL handling as correct as possible.
And this in turn requires to reflect the correct execute permission
setting also in the ACEs inherited from the parent dir.

That's why a file downloaded with wget does not have execute permissions
at all by default.  wget doesn't request them, and for the ACEs inherited
from the parent dir Cygwin removes the execute permissions to follow suit.

Bottom line, either use rsync if possible, or when using wget try if
the --preserve-permissions option fixes your problem.  Alternatively,
fall back to chmod +x.


HTH,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: wget seemingly modifies file access permissions on XP

[Andrey Repin]

Greetings, mihau!


>> > unfortunately, this did not help, not even after a reboot.
>>
>> Did not help with what?

> sorry if this wasn't specific enough.
> "this did not help" meant the behavior described in the original message did not
> change at all; cygwin wget works like before and - in comparison to mingw wget -
> messes up the file permissions.

It doesn't "mess up", it "follow the rules".
If you want different results, you'd need to tell that to the system.

>> Don't try to second guess the created ACL, please.
>> What command do you call, and how does the ACL look afterwards?
>> Please send both, icacls and getfacl output of the created ACL.

> attached output from both, as I don't have icacls on my system - a little
> websearch revealed it was introduced with Vista - I have included cacls
> output instead if this is of any help.
> (legend:
> index1.html was downloaded with wget cygwin,
> index2.html was downloaded with wget mingw)

>> Btw., if you wget a file it's *supposed* to be non-executable.  Assuming
>> your umask is 0022, the file will be create with 0644 permissions.  The
>> default ACL created in this case explicitely disables execution for the
>> admin user if the admin user is part of the ACL.  Just run `chmod +x'
>> afterwards if that's the only problem.

> I specifically meant exe as in windows executables,

There's no such thing as "windows executables" in POSIX terminology.
And Cygwin is a POSIX compatibility layer.

> files w/ .exe suffix;
> just tried "chmod +x" and you are correct, it works, my helloworld.exe
> became executable indeed. thank you for that.

> any idea why wget cygwin interferes with those things though?

It does not "interfere" in a sense that it breaks something. It just does what
it is supposed to do.

> in my understanding it should just copy/save a file from a remote server
> to a local computer, without even touching file permissions or such.

Not quite correct. Unless you mount your /cygdrive with noacl flag, every tool
that create a file is expected to initialize default access permissions on it.
As per POSIX.
And default POSIX permissions are (0666 & umask) for file, or (0777 & umask)
for directory.
(Even if you use noacl for /cygdrive, anything you do WITHIN the Cygwin
installation root will still follow POSIX permissions. Trying to change THAT
is a sure way to break your installation. Eventually.)

> several windows batch files which I use now, either have to be edited
> and "chmod +x" added in all of them - or I'll just stick with wget mingw.

Or you start actually understanding what you are doing, and reading
documentation.

> it's just that I wonder why this didn't happen before, maybe it's my fault
> because I had replaced wget cygwin with wget mingw years ago? to be honest,
> I don't remember.

We never knew that, so it's near impossible to tell now.
The point being, this CAN be set to work the way you need it to.
If you stop blaming and start learning.


-- 
With best regards,
Andrey Repin
Wednesday, March 30, 2016 22:51:42

Sorry for my terrible english...


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: wget seemingly modifies file access permissions on XP

[mihau]

[-- Attachment #1: Type: text/plain, Size: 1862 bytes --]


> > unfortunately, this did not help, not even after a reboot.
>
> Did not help with what?

sorry if this wasn't specific enough.
"this did not help" meant the behavior described in the original message did not
change at all; cygwin wget works like before and - in comparison to mingw wget -
messes up the file permissions.

> Don't try to second guess the created ACL, please.
> What command do you call, and how does the ACL look afterwards?
> Please send both, icacls and getfacl output of the created ACL.

attached output from both, as I don't have icacls on my system - a little
websearch revealed it was introduced with Vista - I have included cacls
output instead if this is of any help.
(legend:
index1.html was downloaded with wget cygwin,
index2.html was downloaded with wget mingw)

> Btw., if you wget a file it's *supposed* to be non-executable.  Assuming
> your umask is 0022, the file will be create with 0644 permissions.  The
> default ACL created in this case explicitely disables execution for the
> admin user if the admin user is part of the ACL.  Just run `chmod +x'
> afterwards if that's the only problem.

I specifically meant exe as in windows executables, files w/ .exe suffix;
just tried "chmod +x" and you are correct, it works, my helloworld.exe
became executable indeed. thank you for that.

any idea why wget cygwin interferes with those things though?
in my understanding it should just copy/save a file from a remote server
to a local computer, without even touching file permissions or such.
several windows batch files which I use now, either have to be edited
and "chmod +x" added in all of them - or I'll just stick with wget mingw.

it's just that I wonder why this didn't happen before, maybe it's my fault
because I had replaced wget cygwin with wget mingw years ago? to be honest,
I don't remember.

greetings
Michael

[-- Attachment #2: index1_html_cacls_output.txt --]
[-- Type: text/plain, Size: 3754 bytes --]

C:\~\index1.html NULL SID:(DENY)(special access:)
                          READ_CONTROL
                          FILE_EXECUTE
                          FILE_DELETE_CHILD
 
                 DEEPBLUE\Administrator:(special access:)
                                        STANDARD_RIGHTS_ALL
                                        DELETE
                                        READ_CONTROL
                                        WRITE_DAC
                                        WRITE_OWNER
                                        SYNCHRONIZE
                                        STANDARD_RIGHTS_REQUIRED
                                        FILE_GENERIC_READ
                                        FILE_GENERIC_WRITE
                                        FILE_READ_DATA
                                        FILE_WRITE_DATA
                                        FILE_APPEND_DATA
                                        FILE_READ_EA
                                        FILE_WRITE_EA
                                        FILE_READ_ATTRIBUTES
                                        FILE_WRITE_ATTRIBUTES
 
                 NT AUTHORITY\SYSTEM:(DENY)(special access:)
                                     FILE_EXECUTE
 
                 BUILTIN\Administrators:(DENY)(special access:)
                                        FILE_EXECUTE
 
                 BUILTIN\Users:(DENY)(special access:)
                               FILE_EXECUTE
 
                 DEEPBLUE\None:(special access:)
                               READ_CONTROL
                               SYNCHRONIZE
                               FILE_GENERIC_READ
                               FILE_READ_DATA
                               FILE_READ_EA
                               FILE_READ_ATTRIBUTES
 
                 NT AUTHORITY\SYSTEM:(special access:)
                                     READ_CONTROL
                                     SYNCHRONIZE
                                     FILE_GENERIC_READ
                                     FILE_GENERIC_WRITE
                                     FILE_GENERIC_EXECUTE
                                     FILE_READ_DATA
                                     FILE_WRITE_DATA
                                     FILE_APPEND_DATA
                                     FILE_READ_EA
                                     FILE_WRITE_EA
                                     FILE_EXECUTE
                                     FILE_READ_ATTRIBUTES
                                     FILE_WRITE_ATTRIBUTES
 
                 BUILTIN\Administrators:(special access:)
                                        READ_CONTROL
                                        SYNCHRONIZE
                                        FILE_GENERIC_READ
                                        FILE_GENERIC_WRITE
                                        FILE_GENERIC_EXECUTE
                                        FILE_READ_DATA
                                        FILE_WRITE_DATA
                                        FILE_APPEND_DATA
                                        FILE_READ_EA
                                        FILE_WRITE_EA
                                        FILE_EXECUTE
                                        FILE_READ_ATTRIBUTES
                                        FILE_WRITE_ATTRIBUTES
 
                 BUILTIN\Users:R 
                 Everyone:(special access:)
                          READ_CONTROL
                          SYNCHRONIZE
                          FILE_GENERIC_READ
                          FILE_READ_DATA
                          FILE_READ_EA
                          FILE_READ_ATTRIBUTES
 


[-- Attachment #3: index1_html_getfacl_output.txt --]
[-- Type: text/plain, Size: 201 bytes --]

# file: index1.html
# owner: Administrator
# group: None
user::rw-
group::r--
group:SYSTEM:rwx	#effective:rw-
group:Administrators:rwx	#effective:rw-
group:Users:r-x	#effective:r--
mask:r--
other:r--


[-- Attachment #4: index2_html_cacls_output.txt --]
[-- Type: text/plain, Size: 166 bytes --]

C:\~\index2.html DEEPBLUE\Administrator:F 
                 BUILTIN\Administrators:F 
                 NT AUTHORITY\SYSTEM:F 
                 BUILTIN\Users:R 


[-- Attachment #5: index2_html_getfacl_output.txt --]
[-- Type: text/plain, Size: 156 bytes --]

# file: index2.html
# owner: Administrator
# group: None
user::rwx
group::---
group:SYSTEM:rwx
group:Administrators:rwx
group:Users:r-x
mask:r-x
other:---


[-- Attachment #6: Type: text/plain, Size: 218 bytes --]

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: wget seemingly modifies file access permissions on XP

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 976 bytes --]

On Mar 30 18:28, mihau wrote:
> >Remove your /etc/passwd and /etc/group files.  Don't regeneate them,
> >they are not required anymore (https://cygwin.com/cygwin-ug-net/ntsec.html)
> >Then try again.
> 
> unfortunately, this did not help, not even after a reboot.

Did not help with what?  Don't try to second guess the created ACL,
please.  What command do you call, and how does the ACL look afterwards?
Please send both, icacls and getfacl output of the created ACL.

Btw., if you wget a file it's *supposed* to be non-executable.  Assuming
your umask is 0022, the file will be create with 0644 permissions.  The
default ACL created in this case explicitely disables execution for the
admin user if the admin user is part of the ACL.  Just run `chmod +x'
afterwards if that's the only problem.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: wget seemingly modifies file access permissions on XP

[mihau]

> Remove your /etc/passwd and /etc/group files.  Don't regeneate them,
> they are not required anymore (https://cygwin.com/cygwin-ug-net/ntsec.html)
> Then try again.

unfortunately, this did not help, not even after a reboot.

Michael


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple