Exclude System entries with "ls" or "find"

Exclude System entries with "ls" or "find"

[Steven Penny]

Compare Cygwin:

    $ ls -1 -N /cygdrive/c
    $Recycle.Bin
    cygwin64
    Documents and Settings
    pagefile.sys
    PerfLogs
    Program Files
    Program Files (x86)
    ProgramData
    Recovery
    System Volume Information
    Users
    Windows

With Command Prompt:

    > dir /A:-S C:
    2018-12-08  10:14 AM    <DIR>          cygwin64
    2009-07-13  09:20 PM    <DIR>          PerfLogs
    2018-12-15  06:05 PM    <DIR>          Program Files
    2018-12-15  05:21 PM    <DIR>          Program Files (x86)
    2018-10-31  06:07 PM    <DIR>          ProgramData
    2018-11-18  01:10 AM    <DIR>          Users
    2018-11-09  08:18 AM    <DIR>          Windows

as can be seen, Command Prompt has a way to exclude System items. Does Cygwin
have some way to do this, perhaps with "ls" or "find"?


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Exclude System entries with "ls" or "find"

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1030 bytes --]

On Dec 17 16:46, Steven Penny wrote:
> Compare Cygwin:
> 
>    $ ls -1 -N /cygdrive/c
>    $Recycle.Bin
>    cygwin64
>    Documents and Settings
>    pagefile.sys
>    PerfLogs
>    Program Files
>    Program Files (x86)
>    ProgramData
>    Recovery
>    System Volume Information
>    Users
>    Windows
> 
> With Command Prompt:
> 
>    > dir /A:-S C:
>    2018-12-08  10:14 AM    <DIR>          cygwin64
>    2009-07-13  09:20 PM    <DIR>          PerfLogs
>    2018-12-15  06:05 PM    <DIR>          Program Files
>    2018-12-15  05:21 PM    <DIR>          Program Files (x86)
>    2018-10-31  06:07 PM    <DIR>          ProgramData
>    2018-11-18  01:10 AM    <DIR>          Users
>    2018-11-09  08:18 AM    <DIR>          Windows
> 
> as can be seen, Command Prompt has a way to exclude System items. Does Cygwin
> have some way to do this, perhaps with "ls" or "find"?

You're asking POSIX tools to recognize DOS attributes...


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Exclude System entries with "ls" or "find"

[Eliot Moss]

On 12/18/2018 3:13 AM, Corinna Vinschen wrote:

>> as can be seen, Command Prompt has a way to exclude System items. Does Cygwin
>> have some way to do this, perhaps with "ls" or "find"?
> 
> You're asking POSIX tools to recognize DOS attributes...

True; expanding on Corinna's response slightly.  I used "touch" to create a file.
I looked at its Windows access control information with icacls.  I then used
Windows attrib to set the SYS attribute and looked at the ACL again with icacls.
There was no difference.  The reason this is relevant is that the information
printed by (e.g., ls -l) is based on a mapping Cygwin provides from Windows
ACLs to POSIX permissions and ACLs.  (That mapping is *not* one-to-one, because
the models are substantially different, but Cygwin does what it can, and its
mapping has been refined over time to something that seems most useful, but
that can sometimes have its pitfalls if you don't use it carefully.)

The reason I mention the above was to demonstrate something that I already
knew: the Windows attribute bits are orthogonal to (distinct from) permissions /
access related information.  While access information has an analog in POSIX,
the attribute bits do not.  So Cygwin tools don't look at / display them, while
that *do* map access control, date/time stamps, etc., as well as they can.

However, you can run DOS attrib from Cygwin, just like any Windows program,
and parse its output.  So it would be possible to use a combination of Windows
and Cygwin tools to do what you're seeking, though not necessarily with high
efficiency, etc.

Regards - Eliot Moss

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Exclude System entries with "ls" or "find"

[cyg Simple]

On 12/18/2018 7:58 AM, Eliot Moss wrote:
> However, you can run DOS attrib from Cygwin, just like any Windows program,
> and parse its output.  So it would be possible to use a combination of 
> Windows
> and Cygwin tools to do what you're seeking, though not necessarily with 
> high
> efficiency, etc.
> 

That depends on the Windows program and whether or not it the data gets 
to Cygwin.

-- 
cyg Simple

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Exclude System entries with "ls" or "find"

[Eliot Moss]

On 12/18/2018 10:51 AM, cyg Simple wrote:
> On 12/18/2018 7:58 AM, Eliot Moss wrote:
>> However, you can run DOS attrib from Cygwin, just like any Windows program,
>> and parse its output.  So it would be possible to use a combination of Windows
>> and Cygwin tools to do what you're seeking, though not necessarily with high
>> efficiency, etc.
>>
> 
> That depends on the Windows program and whether or not it the data gets to Cygwin.

I was referring to processing the textual output of the DOS command program 'attrib'.
Cygwin has a wide range of POSIX style text processing tools and scripting languages.
This is clearly possible.

For example, if I run:

    attrib | grep '^...S'

I get the output from attrib (all files in the current directory), filtered by those
lines where the fourth character is S, which is the lines for the file with the SYS
attribute set.  Further parsing could extract the file name, and then (if you like)
you can apply cygpath to it to convert the name from Windows syntax to POSIX, etc.

Best - EM

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Exclude System entries with "ls" or "find"

[Brian Inglis]

On 2018-12-18 01:13, Corinna Vinschen wrote:
> On Dec 17 16:46, Steven Penny wrote:
>> Compare Cygwin:
>>    $ ls -1 -N /cygdrive/c
>>    $Recycle.Bin
>>    cygwin64
>>    Documents and Settings
>>    pagefile.sys
>>    PerfLogs
>>    Program Files
>>    Program Files (x86)
>>    ProgramData
>>    Recovery
>>    System Volume Information
>>    Users
>>    Windows
>> With Command Prompt:
>>    > dir /A:-S C:
>>    2018-12-08  10:14 AM    <DIR>          cygwin64
>>    2009-07-13  09:20 PM    <DIR>          PerfLogs
>>    2018-12-15  06:05 PM    <DIR>          Program Files
>>    2018-12-15  05:21 PM    <DIR>          Program Files (x86)
>>    2018-10-31  06:07 PM    <DIR>          ProgramData
>>    2018-11-18  01:10 AM    <DIR>          Users
>>    2018-11-09  08:18 AM    <DIR>          Windows
>> as can be seen, Command Prompt has a way to exclude System items. Does Cygwin
>> have some way to do this, perhaps with "ls" or "find"?
> You're asking POSIX tools to recognize DOS attributes...
Could perhaps be supported as an env option
	CYGWIN=hidefileswithattributes:[SH]...
with a possibility of overriding that and seeing the files (as with . and ..)
when ls -a is specified, if SWTDI? Meanwhile CyNWin (no more than CyNUX).

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Exclude System entries with "ls" or "find"

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2652 bytes --]

On Dec 18 22:00, Brian Inglis wrote:
> On 2018-12-18 01:13, Corinna Vinschen wrote:
> > On Dec 17 16:46, Steven Penny wrote:
> >> Compare Cygwin:
> >>    $ ls -1 -N /cygdrive/c
> >>    $Recycle.Bin
> >>    cygwin64
> >>    Documents and Settings
> >>    pagefile.sys
> >>    PerfLogs
> >>    Program Files
> >>    Program Files (x86)
> >>    ProgramData
> >>    Recovery
> >>    System Volume Information
> >>    Users
> >>    Windows
> >> With Command Prompt:
> >>    > dir /A:-S C:
> >>    2018-12-08  10:14 AM    <DIR>          cygwin64
> >>    2009-07-13  09:20 PM    <DIR>          PerfLogs
> >>    2018-12-15  06:05 PM    <DIR>          Program Files
> >>    2018-12-15  05:21 PM    <DIR>          Program Files (x86)
> >>    2018-10-31  06:07 PM    <DIR>          ProgramData
> >>    2018-11-18  01:10 AM    <DIR>          Users
> >>    2018-11-09  08:18 AM    <DIR>          Windows
> >> as can be seen, Command Prompt has a way to exclude System items. Does Cygwin
> >> have some way to do this, perhaps with "ls" or "find"?
> > You're asking POSIX tools to recognize DOS attributes...
> Could perhaps be supported as an env option
> 	CYGWIN=hidefileswithattributes:[SH]...
> with a possibility of overriding that and seeing the files (as with . and ..)
> when ls -a is specified, if SWTDI? Meanwhile CyNWin (no more than CyNUX).

YA logic with questionable gain in the file resolution?  I'm not
exactly looking forward to that.

Just this year, when implementing O_TMPFILE, I had this brilliant idea
to hide temporary files from directory listings, which turned out to be
not such a bright idea, after all:

https://sourceware.org/git/?p=newlib-cygwin.git;a=commitdiff;h=7ae73be14194
https://sourceware.org/git/?p=newlib-cygwin.git;a=commitdiff;h=8b6804b8a881

Cygwin symlinks have the System attribute set, too.  Do you want to see
them or not when setting this option?

Bottom line is, if you want to handle DOS attributes in a special way
not covered by our POSIX emulation, user space has to do it.  Either you
use sth. like

   w32_path = cygwin_create_path (CCP_POSIX_TO_WIN_W, posix_path);
   attrs = GetFileAttributesW (w32_path);
   free (w32_path);

Or we could implement an API for that.  As far as I can see Linux has one
already, an ioctl supported by the vfat driver:

  uint32_t attrs;
  ioctl(fd, FAT_IOCTL_GET_ATTRIBUTES, &attrs);
  ioctl(fd, FAT_IOCTL_SET_ATTRIBUTES, &attrs);

Of course, this doesn't help you a lot using standard tools like ls or
find but then WIndows' attrib command is your friend.


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Exclude System entries with "ls" or "find"

[Andrey Repin]

Greetings, Corinna Vinschen!

> Bottom line is, if you want to handle DOS attributes in a special way
> not covered by our POSIX emulation, user space has to do it.

Can something be implemented around getfattr to serve this use case in a more
POSIX'y way?


-- 
With best regards,
Andrey Repin
Wednesday, December 19, 2018 14:51:02

Sorry for my terrible english...


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Exclude System entries with "ls" or "find"

[Eliot Moss]

On 12/19/2018 7:10 AM, Andrey Repin wrote:
> Greetings, Corinna Vinschen!
> 
>> Bottom line is, if you want to handle DOS attributes in a special way
>> not covered by our POSIX emulation, user space has to do it.
> 
> Can something be implemented around getfattr to serve this use case in a more
> POSIX'y way?

I was not previously much aware of get/setfattr, but I agree that that looks
like a possible way to provide access to the Windows SYS attribute and friends.
Not that I am that familiar with details of what NTFS, etc., provide, getfattr
may be a way to provide access to additional attributes / facts about files
(paths, generally) as well.  Sounds like it would not be too hard to support
access to the basic attributes this way.

I suspect the general response will be PTC, though!  :-)

Regards - Eliot Moss

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Exclude System entries with "ls" or "find"

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1209 bytes --]

On Dec 19 08:29, Eliot Moss wrote:
> On 12/19/2018 7:10 AM, Andrey Repin wrote:
> > Greetings, Corinna Vinschen!
> > 
> > > Bottom line is, if you want to handle DOS attributes in a special way
> > > not covered by our POSIX emulation, user space has to do it.
> > 
> > Can something be implemented around getfattr to serve this use case in a more
> > POSIX'y way?
> 
> I was not previously much aware of get/setfattr, but I agree that that looks
> like a possible way to provide access to the Windows SYS attribute and friends.
> Not that I am that familiar with details of what NTFS, etc., provide, getfattr
> may be a way to provide access to additional attributes / facts about files
> (paths, generally) as well.  Sounds like it would not be too hard to support
> access to the basic attributes this way.
> 
> I suspect the general response will be PTC, though!  :-)

Not only that.  attr is an upstream package, so you would have to 
convince the upstream maintainers to take your patches, ideally.

attr is for handling of extended attributes, quite different from
DOS attributes.  I doubt that DOS attribs fit in there.


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Exclude System entries with "ls" or "find"

[Andrey Repin]

Greetings, Corinna Vinschen!

> On Dec 19 08:29, Eliot Moss wrote:
>> On 12/19/2018 7:10 AM, Andrey Repin wrote:
>> > Greetings, Corinna Vinschen!
>> > 
>> > > Bottom line is, if you want to handle DOS attributes in a special way
>> > > not covered by our POSIX emulation, user space has to do it.
>> > 
>> > Can something be implemented around getfattr to serve this use case in a more
>> > POSIX'y way?
>> 
>> I was not previously much aware of get/setfattr, but I agree that that looks
>> like a possible way to provide access to the Windows SYS attribute and friends.
>> Not that I am that familiar with details of what NTFS, etc., provide, getfattr
>> may be a way to provide access to additional attributes / facts about files
>> (paths, generally) as well.  Sounds like it would not be too hard to support
>> access to the basic attributes this way.
>> 
>> I suspect the general response will be PTC, though!  :-)

> Not only that.  attr is an upstream package, so you would have to 
> convince the upstream maintainers to take your patches, ideally.

> attr is for handling of extended attributes, quite different from
> DOS attributes.  I doubt that DOS attribs fit in there.

Well, well. We do have a living example of Samba (ab)using xattr to support
Windows ACL on *NIX.
I deduce from your response, that a DOS attributes use case would be on
different side of the table, though.


-- 
With best regards,
Andrey Repin
Wednesday, December 19, 2018 17:41:17

Sorry for my terrible english...


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Exclude System entries with "ls" or "find"

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1977 bytes --]

On Dec 19 17:43, Andrey Repin wrote:
> Greetings, Corinna Vinschen!
> 
> > On Dec 19 08:29, Eliot Moss wrote:
> >> On 12/19/2018 7:10 AM, Andrey Repin wrote:
> >> > Greetings, Corinna Vinschen!
> >> > 
> >> > > Bottom line is, if you want to handle DOS attributes in a special way
> >> > > not covered by our POSIX emulation, user space has to do it.
> >> > 
> >> > Can something be implemented around getfattr to serve this use case in a more
> >> > POSIX'y way?
> >> 
> >> I was not previously much aware of get/setfattr, but I agree that that looks
> >> like a possible way to provide access to the Windows SYS attribute and friends.
> >> Not that I am that familiar with details of what NTFS, etc., provide, getfattr
> >> may be a way to provide access to additional attributes / facts about files
> >> (paths, generally) as well.  Sounds like it would not be too hard to support
> >> access to the basic attributes this way.
> >> 
> >> I suspect the general response will be PTC, though!  :-)
> 
> > Not only that.  attr is an upstream package, so you would have to 
> > convince the upstream maintainers to take your patches, ideally.
> 
> > attr is for handling of extended attributes, quite different from
> > DOS attributes.  I doubt that DOS attribs fit in there.
> 
> Well, well. We do have a living example of Samba (ab)using xattr to support
> Windows ACL on *NIX.

What are you trying to imply?  That we should fake an user.DOSATTRIB
EA, just like Samba?  Not sure I like the idea since that would have
to be added to each and every file on the fly.  You won't see the
Samba EA from Windows, though.

> I deduce from your response, that a DOS attributes use case would be on
> different side of the table, though.

It's something different.  It's not an EA thingy on Windows but rather
a filesystem property.  So sth. like the vfat IOCTL makes more sense,
if any.


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Exclude System entries with "ls" or "find"

[Brian Inglis]

On 2018-12-19 07:43, Andrey Repin wrote:
>> On Dec 19 08:29, Eliot Moss wrote:
>>> On 12/19/2018 7:10 AM, Andrey Repin wrote:
>>>>> Bottom line is, if you want to handle DOS attributes in a special way
>>>>> not covered by our POSIX emulation, user space has to do it.
>>>> Can something be implemented around getfattr to serve this use case in a more
>>>> POSIX'y way?
>>> I was not previously much aware of get/setfattr, but I agree that that looks
>>> like a possible way to provide access to the Windows SYS attribute and friends.
>>> Not that I am that familiar with details of what NTFS, etc., provide, getfattr
>>> may be a way to provide access to additional attributes / facts about files
>>> (paths, generally) as well.  Sounds like it would not be too hard to support
>>> access to the basic attributes this way.
>>> I suspect the general response will be PTC, though!  :-)
>> Not only that.  attr is an upstream package, so you would have to 
>> convince the upstream maintainers to take your patches, ideally.
>> attr is for handling of extended attributes, quite different from
>> DOS attributes.  I doubt that DOS attribs fit in there.
> Well, well. We do have a living example of Samba (ab)using xattr to support
> Windows ACL on *NIX.
> I deduce from your response, that a DOS attributes use case would be on
> different side of the table, though.

While WSL uses Windows EAs to support POSIX stattributes[sic].

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple