A notion about saving and restoring Windows file security info

A notion about saving and restoring Windows file security info

[Eliot Moss]

Dear Cygwiners - Maybe this idea has been discussed before, and I can't say I
exactly have a specific application in mind, but I was wondering about how one
might achieve reliable backup/restore of files on a Windows system via a
backup program written to the POSIX interface and ported to Windows via
Cygwin.

The idea is this: expose the Windows file attributes (such as system, archive,
hidden, etc.) as well as security descriptors (such as managed by icacls), via
specifically named extended attributes, those read/written by get/setfattr.
The Cygwin library could be enhanced to "know" the special names of these
attributes and use the appropriate, different, underlying calls to get/set
them.

Is this a crazy idea?  A useful one?  (How useful?)  What implementation
effort would be required?

What brought it to mind is the restic backup/restore program, which AFAIK
lacks support for these things, but does know how to save/restore xattrs.  (So
I could fake the same thing there, provided I duplicate in golang in restic
what I am suggesting might be done for Cygwin in it library.)

Regards - Eliot

Re: A notion about saving and restoring Windows file security info

[Corinna Vinschen]

On Jan  4 22:45, Eliot Moss wrote:
> Dear Cygwiners - Maybe this idea has been discussed before, and I can't say I
> exactly have a specific application in mind, but I was wondering about how one
> might achieve reliable backup/restore of files on a Windows system via a
> backup program written to the POSIX interface and ported to Windows via
> Cygwin.
> 
> The idea is this: expose the Windows file attributes (such as system, archive,
> hidden, etc.) as well as security descriptors (such as managed by icacls), via
> specifically named extended attributes, those read/written by get/setfattr.
> The Cygwin library could be enhanced to "know" the special names of these
> attributes and use the appropriate, different, underlying calls to get/set
> them.
> 
> Is this a crazy idea?  A useful one?  (How useful?)  What implementation
> effort would be required?

It would be possible with not too much effort.  That would need a bit
of discussion, for instance, do we want to show up the native ACL in
listxattr?


Corinna

Re: A notion about saving and restoring Windows file security info

[Eliot Moss]

On 1/5/2022 5:34 AM, Corinna Vinschen wrote:
 > On Jan  4 22:45, Eliot Moss wrote:
 >> Dear Cygwiners - Maybe this idea has been discussed before, and I can't say I
 >> exactly have a specific application in mind, but I was wondering about how one
 >> might achieve reliable backup/restore of files on a Windows system via a
 >> backup program written to the POSIX interface and ported to Windows via
 >> Cygwin.
 >>
 >> The idea is this: expose the Windows file attributes (such as system, archive,
 >> hidden, etc.) as well as security descriptors (such as managed by icacls), via
 >> specifically named extended attributes, those read/written by get/setfattr.
 >> The Cygwin library could be enhanced to "know" the special names of these
 >> attributes and use the appropriate, different, underlying calls to get/set
 >> them.
 >>
 >> Is this a crazy idea?  A useful one?  (How useful?)  What implementation
 >> effort would be required?
 >
 > It would be possible with not too much effort.  That would need a bit
 > of discussion, for instance, do we want to show up the native ACL in
 > listxattr?

Thanks for the quick response, Corinna!

One thing I am aware of is that the intent of xattrs is to allow applications
to attach metadata to items in a file system (inodes, essentially, right?)
that are meaningful to the applications but that do not affect the basic file
system semantics.  Since Windows file attributes (A, H, S, etc., bits) and
security descriptors *do* affect semantics, this would admittedly warp things
slightly.  Therefore, my thought is to make the presentation of these
attributes by attr_list (is that the canonical call?) and probably also the
special interpretation of them for get/set, controllable in some way.  The
means of control that occurs to me is an environment variable, or a flag
within the CYGWIN env var.  Other folks more deeply involved in Cygwin library
code, etc., may have a more informed notion of the best way to control this,
but it strikes me as one of those things that you would have to ask for.

As a side point, I have discovered that xattrs are used by WSL to record uid,
god, and mod information.  (While uid and gid are obvious, I am less certain
about mod - probably chmod type mode bits, but might have to do with
modification time?)

Another question to ponder is whether an interface of the kind I am suggesting
might also present NTFS ADSs (alternate data streams) as xattrs, if
requested.  The SysInternals "streams" program can manipulate them, though it
does mention that the program uses an undocumented native function to obtain
the stream information.  Cygwin cp, unsurprisingly, does not copy such
information, while cmd's copy command *does* (maybe a little surprisingly).  I
am aware that ADSs have security implications because they are so hidden from
users.  Adding this functionality would make it easy to use ordinary Cygwin
tools to find and remove any ADS of concern, etc.

Another design question is the names to use for these "magical" xattrs.  For
generality, if the feature is turned on, it might be good to add a prefix to
the names of real xattrs when getting/listing, that would be stripped off when
setting, and would of course be different from the prefix(es) for the
"magical" attributes.  For example, we could use:

- POSIX_ as a prefix for each regular xattr name
- WindowsAttrs as an attribute that capture all attrs, for simplicity, though
   we could also represent each of them separately as an attribute with no
   value, present only when the indicated attribute is set
- WindowsSDs for the security descriptors, a single attribute since order
   matters
- WindowsADS_ as a prefix for each ADS

In principle, each of Attrs, SDs, and ADS_ could be separately controlled, if
we think that is important (might be).

Those are my thoughts for now.  Sorry this is longish ...  Eliot

Re: A notion about saving and restoring Windows file security info

[Sam Edge]

On 05/01/2022 17:41, Eliot Moss wrote:
>  Therefore, my thought is to make the presentation of these
> attributes by attr_list (is that the canonical call?) and probably also the
> special interpretation of them for get/set, controllable in some way.  The
> means of control that occurs to me is an environment variable, or a flag
> within the CYGWIN env var.  Other folks more deeply involved in Cygwin library
> code, etc., may have a more informed notion of the best way to control this,
> but it strikes me as one of those things that you would have to ask for.
>
> In principle, each of Attrs, SDs, and ADS_ could be separately controlled, if
> we think that is important (might be).
>

Hi Eliot.

I can see the appeal of mapping hidden/system/readonly to suitably
namespaced xattrs and exposing ADSes. However I would caution about having too
many selectable options as the behaviour for every combination in the running
system would have to be considered against filesystems that have been
manipulated with every other combination.

As the recent update to setup.exe has shown, even the different types of
symlink handling can cause unforeseen problems and that's a single option with
only a few possible values that only affect how they're written and not how
they're read.

Just a thought.

--
Sam Edge

Re: A notion about saving and restoring Windows file security info

[Corinna Vinschen]

On Jan  5 12:41, Eliot Moss wrote:
> On 1/5/2022 5:34 AM, Corinna Vinschen wrote:
> > On Jan  4 22:45, Eliot Moss wrote:
> >> [...]
> >> The idea is this: expose the Windows file attributes (such as system, archive,
> >> hidden, etc.) as well as security descriptors (such as managed by icacls), via
> >
> > It would be possible with not too much effort.

Reconsidered: Its a bit of effort for reasons outlined below.

> > That would need a bit
> > of discussion, for instance, do we want to show up the native ACL in
> > listxattr?
> 
> Thanks for the quick response, Corinna!
> 
> One thing I am aware of is that the intent of xattrs is to allow applications
> to attach metadata to items in a file system (inodes, essentially, right?)
> that are meaningful to the applications but that do not affect the basic file
> system semantics.  Since Windows file attributes (A, H, S, etc., bits) and
> security descriptors *do* affect semantics, this would admittedly warp things
> slightly.  Therefore, my thought is to make the presentation of these
> attributes by attr_list (is that the canonical call?) and probably also the
> special interpretation of them for get/set, controllable in some way.  The
> means of control that occurs to me is an environment variable, or a flag
> within the CYGWIN env var.  Other folks more deeply involved in Cygwin library
> code, etc., may have a more informed notion of the best way to control this,
> but it strikes me as one of those things that you would have to ask for.

No settings in that case.  *Iff* we do that, we should provide the native
ACLs in a consistent manner.

I'm a bit concerned how this is supposed to work in cases where the user
uses the tool's 'restore xattrs' flag but is missing admin rights.  There's
also a potentially confusing result if you restore ACL xattrs on another
system.  The SIDs won't match and you can easily end up with an entirely
broken permission hirarchy.

Also, to answer my own question, listxattr would have to list the xattr, of
course, otherwise backup tools wouldn't find the xattr and still not save
it.

> As a side point, I have discovered that xattrs are used by WSL to record uid,
> god, and mod information.  (While uid and gid are obvious, I am less certain
> about mod - probably chmod type mode bits, but might have to do with
> modification time?)

That's just for represent the POSIX permission bits.

> Another question to ponder is whether an interface of the kind I am suggesting
> might also present NTFS ADSs (alternate data streams) as xattrs,

See the thread starting at
https://cygwin.com/pipermail/cygwin/2022-January/250352.html

> Another design question is the names to use for these "magical" xattrs.  For
> generality, if the feature is turned on, it might be good to add a prefix to
> the names of real xattrs when getting/listing, that would be stripped off when
> setting, and would of course be different from the prefix(es) for the
> "magical" attributes.  For example, we could use:

https://man7.org/linux/man-pages/man7/xattr.7.html

Right now, all xattrs are treated by Cygwin as if they are in the "user"
namespace.  Ideally the ACL xattr would go into the "system" namespace,
but NOT use the system.posix_acl_access name.  Perhaps something like
"system.windows_acl_access"

If you want to take a stab at it, see the file winsup/cygwin/ntea.cc.
It handles reading (function "read_ea") and writing (function "write_ea")
of EAs, and it provides the external POSIXy calls {l,f}getxattr,
{l,f}listxattr, {l,f}setxattr and {l,f}removexattr.

One problem is currently that the handling of the "user" namespace
is hardcoded.  That needs a bit of mellowing.


Corinna

Re: A notion about saving and restoring Windows file security info

[Eliot Moss]

On 1/7/2022 8:02 AM, Corinna Vinschen wrote:

 > Reconsidered: Its a bit of effort for reasons outlined below.

Possibly ...

 > No settings in that case.

I didn't entirely get your meaning, but I *think* you said if this
is implemented, it should just return these "extra" things as suitably
named attributed all the time.

 >*Iff* we do that, we should provide the native ACLs in a consistent manner.

Yes, it should be consistent - but that doesn't rule out continuing the exist
get/setfacl interface, for example.

 > I'm a bit concerned how this is supposed to work in cases where the user
 > uses the tool's 'restore xattrs' flag but is missing admin rights.  There's
 > also a potentially confusing result if you restore ACL xattrs on another
 > system.  The SIDs won't match and you can easily end up with an entirely
 > broken permission hirarchy.

If you're missing the rights, setting that "attribute" will fail and a
reasonable tool will tell you.  There may also be file systems that don't
support security descriptors, and trying to restore there would also fail.
The same might be true of xatts generally - not all file systems support
them.

Restoring on a different system is not unlike extracting from a tar archive
and asking for the uid/gid/perms to be preserved - caveat utilor, though a
good tool would give some control.

 > Also, to answer my own question, listxattr would have to list the xattr, of
 > course, otherwise backup tools wouldn't find the xattr and still not save
 > it.

Right.

 >> Another question to ponder is whether an interface of the kind I am suggesting
 >> might also present NTFS ADSs (alternate data streams) as xattrs,
 >
 > See the thread starting at
 > https://cygwin.com/pipermail/cygwin/2022-January/250352.html

That does raise the interesting question of whether ADSs more appropriately
should present a file-like interface or xattr-like one.  The latter would
present an ADS as one (possibly big) blob, or else complicate the interface.
There could still be a file-like interface, separately.  An xattr-like one
might be good for transparent backup/restore.  More pondering required!

 >> Another design question is the names to use for these "magical" xattrs.  For
 >> generality, if the feature is turned on, it might be good to add a prefix to
 >> the names of real xattrs when getting/listing, that would be stripped off when
 >> setting, and would of course be different from the prefix(es) for the
 >> "magical" attributes.  For example, we could use:
 >
 > https://man7.org/linux/man-pages/man7/xattr.7.html
 >
 > Right now, all xattrs are treated by Cygwin as if they are in the "user"
 > namespace.  Ideally the ACL xattr would go into the "system" namespace,
 > but NOT use the system.posix_acl_access name.  Perhaps something like
 > "system.windows_acl_access"
 >
 > If you want to take a stab at it, see the file winsup/cygwin/ntea.cc.
 > It handles reading (function "read_ea") and writing (function "write_ea")
 > of EAs, and it provides the external POSIXy calls {l,f}getxattr,
 > {l,f}listxattr, {l,f}setxattr and {l,f}removexattr.
 >
 > One problem is currently that the handling of the "user" namespace
 > is hardcoded.  That needs a bit of mellowing.

Thanks for the pointers - I may take a look at it!  Eliot

Re: A notion about saving and restoring Windows file security info

[Corinna Vinschen]

On Jan  7 09:28, Eliot Moss wrote:
> On 1/7/2022 8:02 AM, Corinna Vinschen wrote:
> 
> > Reconsidered: Its a bit of effort for reasons outlined below.
> 
> Possibly ...
> 
> > No settings in that case.
> 
> I didn't entirely get your meaning, but I *think* you said if this
> is implemented, it should just return these "extra" things as suitably
> named attributed all the time.

What I meant is, no selectable option, as Sam pointed out in his reply.
Yes or No, not Maybe.

> >*Iff* we do that, we should provide the native ACLs in a consistent manner.
> 
> Yes, it should be consistent - but that doesn't rule out continuing the exist
> get/setfacl interface, for example.

Wait... that's an entirely different beast.

On Linux ACLs are implemented using xattrs.  The Linux (or rather,
deprecated POSIX) acl(5) API provides the means to access ACLs
independent of their actual implementation.  On Linux it uses the
getxattr/setxattr calls to access the DACL, on Cygwin it uses the native
NT and Windows APIs basically.  This API will certainly stay in place.

IIUC, you're looking for using xattrs to provide a direct means of
saving and restoring the Windows ACL.  This is different from the POSIX
ACL.  What I'm referring to in my reply is to provide a xattr with
the binary content of a Windows DACL verbatim.  That could be used
by a subsequent setxattr call to restore the Windows ACL verbatim as well.

Having said that...

> > I'm a bit concerned how this is supposed to work in cases where the user
> > uses the tool's 'restore xattrs' flag but is missing admin rights.  There's
> > also a potentially confusing result if you restore ACL xattrs on another
> > system.  The SIDs won't match and you can easily end up with an entirely
> > broken permission hirarchy.
> 
> If you're missing the rights, setting that "attribute" will fail and a
> reasonable tool will tell you.

It's not simple failing I'm concerned about.  If the file belongs to my
user and if I have WRITE_DAC access, I can restore the DACL.  However,
I'm typically not allowed to chown, and the resulting ACL should reflect
the fact that the owner didn't change.  But the verbatim Windows DACL
contains another user SID.  I didn't entirely think this through, but in
that scenario the underlying Cygwin code might have to tweak the Windows
DACL accordingly, and *that's* a complication which sounds the opposite
of funny.

> Restoring on a different system is not unlike extracting from a tar archive
> and asking for the uid/gid/perms to be preserved - caveat utilor, though a
> good tool would give some control.

If you have admin perms and ask the tool to restore xattrs, the DACL
will get written.  Windows does not check if the SIDs make sense on the
local system, because there's no notion of making sense.  On Windows,
any SID might be correct, e.  g. an account of another domain.  Maybe
it's not that much of a problem, but I remember NT4 times and how
complicated it was at times to restore useful permissions to a file with
broken ACL.

> > Also, to answer my own question, listxattr would have to list the xattr, of
> > course, otherwise backup tools wouldn't find the xattr and still not save
> > it.
> 
> Right.
> 
> >> Another question to ponder is whether an interface of the kind I am suggesting
> >> might also present NTFS ADSs (alternate data streams) as xattrs,
> >
> > See the thread starting at
> > https://cygwin.com/pipermail/cygwin/2022-January/250352.html
> 
> That does raise the interesting question of whether ADSs more appropriately
> should present a file-like interface or xattr-like one.  The latter would
> present an ADS as one (possibly big) blob, or else complicate the interface.
> There could still be a file-like interface, separately.  An xattr-like one
> might be good for transparent backup/restore.  More pondering required!

If with file-like interface you mean the file:stream expression for
filenames, than that's not an option.  As xattr interface it might be
a neat extension.


Corinna

Re: A notion about saving and restoring Windows file security info

[Andrey Repin]

Greetings, Corinna Vinschen!

> No settings in that case.  *Iff* we do that, we should provide the native
> ACLs in a consistent manner.

I have a small suggestion. Have you tried too look into existing
implementations? Samba uses xattrs to store extra information for much the
same purpose.
See f.e. https://www.samba.org/samba/docs/current/man-html/vfs_acl_xattr.8.html


-- 
With best regards,
Andrey Repin
Friday, January 7, 2022 18:14:48

Sorry for my terrible english...

Re: A notion about saving and restoring Windows file security info

[Corinna Vinschen]

On Jan  7 18:40, Andrey Repin wrote:
> Greetings, Corinna Vinschen!
> 
> > No settings in that case.  *Iff* we do that, we should provide the native
> > ACLs in a consistent manner.
> 
> I have a small suggestion. Have you tried too look into existing
> implementations? Samba uses xattrs to store extra information for much the
> same purpose.
> See f.e. https://www.samba.org/samba/docs/current/man-html/vfs_acl_xattr.8.html

Thanks!


Corinna