Retrieving per-process environment block?

Retrieving per-process environment block?

[Erik Bray]

Hi all,

For a quick bit of background, I'm working on porting the highly
useful psutil [1] Python library to Cygwin.  This has proved an
interesting exercise, as much of the functionality of psutil works on
Cygwin through existing POSIX interfaces, and a handful of
Linux-specific interfaces as well.  But there are some bits that
simply don't map at all.

The one I'm struggling with right now is retrieving Cygwin environment
variables for a process (under inspection--i.e. not listing a
process's environment from within that process which is obviously
trivial).

I've looked at every route I could conceive of but as far as I can
tell this is currently impossible.  That's fine for now--I simply
disable that functionality in psutil.  But it is unfortunate, though,
since the information is there.

There are a couple avenues I could see to this.  The most "obvious"
(to me) being to implement /proc/<pid>/environ.

I would be willing to provide a patch for this if it would be
accepted.  Is there some particular non-obvious hurdle to this that it
hasn't been implemented?  Obviously there are security
implications--the /proc/<pid>/environ should only be readable to the
process's owner, but that is already within Cygwin's capabilities, and
works for other /proc files.

Thanks,
Erik


[1] https://pythonhosted.org/psutil/

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Retrieving per-process environment block?

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1683 bytes --]

On Nov 17 14:30, Erik Bray wrote:
> Hi all,
> 
> For a quick bit of background, I'm working on porting the highly
> useful psutil [1] Python library to Cygwin.  This has proved an
> interesting exercise, as much of the functionality of psutil works on
> Cygwin through existing POSIX interfaces, and a handful of
> Linux-specific interfaces as well.  But there are some bits that
> simply don't map at all.
> 
> The one I'm struggling with right now is retrieving Cygwin environment
> variables for a process (under inspection--i.e. not listing a
> process's environment from within that process which is obviously
> trivial).
> 
> I've looked at every route I could conceive of but as far as I can
> tell this is currently impossible.  That's fine for now--I simply
> disable that functionality in psutil.  But it is unfortunate, though,
> since the information is there.
> 
> There are a couple avenues I could see to this.  The most "obvious"
> (to me) being to implement /proc/<pid>/environ.
> 
> I would be willing to provide a patch for this if it would be
> accepted.  Is there some particular non-obvious hurdle to this that it
> hasn't been implemented?  Obviously there are security
> implications--the /proc/<pid>/environ should only be readable to the
> process's owner, but that is already within Cygwin's capabilities, and
> works for other /proc files.

Patch welcome.  Implementing this should be fairly straightforward.
The only hurdle is winsup/CONTRIBUTORS ;)


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Retrieving per-process environment block?

[Erik Bray]

On Thu, Nov 17, 2016 at 3:00 PM, Corinna Vinschen
<corinna-cygwin@cygwin.com> wrote:
> On Nov 17 14:30, Erik Bray wrote:
>> Hi all,
>>
>> For a quick bit of background, I'm working on porting the highly
>> useful psutil [1] Python library to Cygwin.  This has proved an
>> interesting exercise, as much of the functionality of psutil works on
>> Cygwin through existing POSIX interfaces, and a handful of
>> Linux-specific interfaces as well.  But there are some bits that
>> simply don't map at all.
>>
>> The one I'm struggling with right now is retrieving Cygwin environment
>> variables for a process (under inspection--i.e. not listing a
>> process's environment from within that process which is obviously
>> trivial).
>>
>> I've looked at every route I could conceive of but as far as I can
>> tell this is currently impossible.  That's fine for now--I simply
>> disable that functionality in psutil.  But it is unfortunate, though,
>> since the information is there.
>>
>> There are a couple avenues I could see to this.  The most "obvious"
>> (to me) being to implement /proc/<pid>/environ.
>>
>> I would be willing to provide a patch for this if it would be
>> accepted.  Is there some particular non-obvious hurdle to this that it
>> hasn't been implemented?  Obviously there are security
>> implications--the /proc/<pid>/environ should only be readable to the
>> process's owner, but that is already within Cygwin's capabilities, and
>> works for other /proc files.
>
> Patch welcome.  Implementing this should be fairly straightforward.
> The only hurdle is winsup/CONTRIBUTORS ;)

Thanks--I went to go work on this finally but it turns out not to be
straightforward after all, as the process's environment is not shared
in any way between processes.

I could do this, if each process kept a copy of its environment block
in shared memory, which would in turn have to be updated every time
the process's environment is updated.  But I don't know what the
impact of that would be performance-wise.

Any advice?

Thanks,
Erik

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Retrieving per-process environment block?

[cyg Simple]

On 11/29/2016 8:26 AM, Erik Bray wrote:
> On Thu, Nov 17, 2016 at 3:00 PM, Corinna Vinschen
> <corinna-cygwin@cygwin.com> wrote:
>> On Nov 17 14:30, Erik Bray wrote:
>>> Hi all,
>>>
>>> For a quick bit of background, I'm working on porting the highly
>>> useful psutil [1] Python library to Cygwin.  This has proved an
>>> interesting exercise, as much of the functionality of psutil works on
>>> Cygwin through existing POSIX interfaces, and a handful of
>>> Linux-specific interfaces as well.  But there are some bits that
>>> simply don't map at all.
>>>
>>> The one I'm struggling with right now is retrieving Cygwin environment
>>> variables for a process (under inspection--i.e. not listing a
>>> process's environment from within that process which is obviously
>>> trivial).
>>>
>>> I've looked at every route I could conceive of but as far as I can
>>> tell this is currently impossible.  That's fine for now--I simply
>>> disable that functionality in psutil.  But it is unfortunate, though,
>>> since the information is there.
>>>
>>> There are a couple avenues I could see to this.  The most "obvious"
>>> (to me) being to implement /proc/<pid>/environ.
>>>
>>> I would be willing to provide a patch for this if it would be
>>> accepted.  Is there some particular non-obvious hurdle to this that it
>>> hasn't been implemented?  Obviously there are security
>>> implications--the /proc/<pid>/environ should only be readable to the
>>> process's owner, but that is already within Cygwin's capabilities, and
>>> works for other /proc files.
>>
>> Patch welcome.  Implementing this should be fairly straightforward.
>> The only hurdle is winsup/CONTRIBUTORS ;)
> 
> Thanks--I went to go work on this finally but it turns out not to be
> straightforward after all, as the process's environment is not shared
> in any way between processes.
> 
> I could do this, if each process kept a copy of its environment block
> in shared memory, which would in turn have to be updated every time
> the process's environment is updated.  But I don't know what the
> impact of that would be performance-wise.
> 
> Any advice?
> 

Sounds like a job for a thread that wakes every X time units to check
the contents of the environment.  Or is there a notification API that
could be used to wake the thread?  I know there is a disk change
notification API for this; maybe one for environment changes as well.

-- 
cyg Simple

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Retrieving per-process environment block?

[Eliot Moss]

On 11/29/2016 8:26 AM, Erik Bray wrote:
> On Thu, Nov 17, 2016 at 3:00 PM, Corinna Vinschen
> <corinna-cygwin@cygwin.com> wrote:
>> On Nov 17 14:30, Erik Bray wrote:
>>> Hi all,
>>>
>>> For a quick bit of background, I'm working on porting the highly
>>> useful psutil [1] Python library to Cygwin.  This has proved an
>>> interesting exercise, as much of the functionality of psutil works on
>>> Cygwin through existing POSIX interfaces, and a handful of
>>> Linux-specific interfaces as well.  But there are some bits that
>>> simply don't map at all.
>>>
>>> The one I'm struggling with right now is retrieving Cygwin environment
>>> variables for a process (under inspection--i.e. not listing a
>>> process's environment from within that process which is obviously
>>> trivial).
>>>
>>> I've looked at every route I could conceive of but as far as I can
>>> tell this is currently impossible.  That's fine for now--I simply
>>> disable that functionality in psutil.  But it is unfortunate, though,
>>> since the information is there.
>>>
>>> There are a couple avenues I could see to this.  The most "obvious"
>>> (to me) being to implement /proc/<pid>/environ.
>>>
>>> I would be willing to provide a patch for this if it would be
>>> accepted.  Is there some particular non-obvious hurdle to this that it
>>> hasn't been implemented?  Obviously there are security
>>> implications--the /proc/<pid>/environ should only be readable to the
>>> process's owner, but that is already within Cygwin's capabilities, and
>>> works for other /proc files.
>>
>> Patch welcome.  Implementing this should be fairly straightforward.
>> The only hurdle is winsup/CONTRIBUTORS ;)
>
> Thanks--I went to go work on this finally but it turns out not to be
> straightforward after all, as the process's environment is not shared
> in any way between processes.
>
> I could do this, if each process kept a copy of its environment block
> in shared memory, which would in turn have to be updated every time
> the process's environment is updated.  But I don't know what the
> impact of that would be performance-wise.

Sorry, no advice getting around this, but a thought about why it may have
been hard -- it can be a security hole.  Given Windows' complex access rights,
how would one determine which processes should reasonably have access to
another process's environment variables?  Maybe Windows decided to punt the
issue.  (I don't know anything about this at all - as I said, just a
thought.)

A question: Are there any Windows tools that can do this?  If so, you might
be able to trace them to find what they do.

Regards -- Eliot Moss

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Retrieving per-process environment block?

[Erik Bray]

On Tue, Nov 29, 2016 at 3:28 PM, Eliot Moss <moss@cs.umass.edu> wrote:
> On 11/29/2016 8:26 AM, Erik Bray wrote:
>>
>> On Thu, Nov 17, 2016 at 3:00 PM, Corinna Vinschen
>> <corinna-cygwin@cygwin.com> wrote:
>>>
>>> On Nov 17 14:30, Erik Bray wrote:
>>>>
>>>> Hi all,
>>>>
>>>> For a quick bit of background, I'm working on porting the highly
>>>> useful psutil [1] Python library to Cygwin.  This has proved an
>>>> interesting exercise, as much of the functionality of psutil works on
>>>> Cygwin through existing POSIX interfaces, and a handful of
>>>> Linux-specific interfaces as well.  But there are some bits that
>>>> simply don't map at all.
>>>>
>>>> The one I'm struggling with right now is retrieving Cygwin environment
>>>> variables for a process (under inspection--i.e. not listing a
>>>> process's environment from within that process which is obviously
>>>> trivial).
>>>>
>>>> I've looked at every route I could conceive of but as far as I can
>>>> tell this is currently impossible.  That's fine for now--I simply
>>>> disable that functionality in psutil.  But it is unfortunate, though,
>>>> since the information is there.
>>>>
>>>> There are a couple avenues I could see to this.  The most "obvious"
>>>> (to me) being to implement /proc/<pid>/environ.
>>>>
>>>> I would be willing to provide a patch for this if it would be
>>>> accepted.  Is there some particular non-obvious hurdle to this that it
>>>> hasn't been implemented?  Obviously there are security
>>>> implications--the /proc/<pid>/environ should only be readable to the
>>>> process's owner, but that is already within Cygwin's capabilities, and
>>>> works for other /proc files.
>>>
>>>
>>> Patch welcome.  Implementing this should be fairly straightforward.
>>> The only hurdle is winsup/CONTRIBUTORS ;)
>>
>>
>> Thanks--I went to go work on this finally but it turns out not to be
>> straightforward after all, as the process's environment is not shared
>> in any way between processes.
>>
>> I could do this, if each process kept a copy of its environment block
>> in shared memory, which would in turn have to be updated every time
>> the process's environment is updated.  But I don't know what the
>> impact of that would be performance-wise.
>
>
> Sorry, no advice getting around this, but a thought about why it may have
> been hard -- it can be a security hole.  Given Windows' complex access
> rights,
> how would one determine which processes should reasonably have access to
> another process's environment variables?  Maybe Windows decided to punt the
> issue.  (I don't know anything about this at all - as I said, just a
> thought.)
>
> A question: Are there any Windows tools that can do this?  If so, you might
> be able to trace them to find what they do.

Hi,

Thanks for the reply.  The issue here isn't getting the Windows
process environment--that can be done, albeit trickily [1].  While
it's true there are security implications, in this case that is
handled at the OpenProcess call required to obtain a handle to the
process.  It's true you can't just read the environment from a process
you don't have permission to.

The issue here though is reading the Cygwin process's environment
which is separate from its Windows environment, and the above
technique is inapplicable.

The easiest way, as I suggested, would be to keep a copy of each
process's Cygwin environment in shared memory as is the case with the
pinfo struct.  I just don't know if it's worth the effort for this
relatively narrow case (it's no so much effort to implement, I just
worry what performance impacts that could have, though I suppose I
could just try it and see...)


[1] https://theroadtodelphi.com/2012/06/09/getting-the-environment-variables-of-an-external-x86-and-x64-process/

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Retrieving per-process environment block?

[Andrey Repin]

Greetings, Erik Bray!

> Thanks for the reply.  The issue here isn't getting the Windows
> process environment--that can be done, albeit trickily [1].  While
> it's true there are security implications, in this case that is
> handled at the OpenProcess call required to obtain a handle to the
> process.  It's true you can't just read the environment from a process
> you don't have permission to.

> The issue here though is reading the Cygwin process's environment
> which is separate from its Windows environment, and the above
> technique is inapplicable.

Update Windows environment block when Cygwin environment is updated.


-- 
With best regards,
Andrey Repin
Tuesday, November 29, 2016 19:26:20

Sorry for my terrible english...


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Retrieving per-process environment block?

[Herbert Stocker]

Hi,

On 29.11.2016 17:27, Andrey Repin wrote:
> Greetings, Erik Bray!
>
>> Thanks for the reply.  The issue here isn't getting the Windows
>> process environment--that can be done, albeit trickily [1].  While
>> it's true there are security implications, in this case that is
>> handled at the OpenProcess call required to obtain a handle to the
>> process.  It's true you can't just read the environment from a process
>> you don't have permission to.
>
>> The issue here though is reading the Cygwin process's environment
>> which is separate from its Windows environment, and the above
>> technique is inapplicable.
>
> Update Windows environment block when Cygwin environment is updated.
>

How about the case where paths are translated from Windows style to
Posix style when the Windows environment is copied to Cygwin environ-
ment on process initialization? Should they be converted back?
And thus psutils has to convert them back again to Posix style?
What will happen if a Cygwin process stores a windows path in its
environment?

Maybe each such variable should have a second variable in the Windows
environment block that contains the Posix version of the variable and
psutils picks that, in order to avoid round trip conversion inaccura-
cies.

Yea it's me again advocating against round trip conversions.

Or we simply copy the variables unconverted to the Windows environment
block, as the Windows env block is not used for anything else - up to
now it is not updated at all.


Herbert



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Retrieving per-process environment block?

[Eliot Moss]

On 11/29/2016 7:28 PM, Herbert Stocker wrote:

> Or we simply copy the variables unconverted to the Windows environment
> block, as the Windows env block is not used for anything else - up to
> now it is not updated at all.

Maybe we could stash them inside one large variable?  Would that help
at all?  (It does demand a quoting convention, but since this is all
internal stuff, that may not be too hard.)

Regards -- Eliot

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Retrieving per-process environment block?

[Erik Bray]

On Wed, Nov 30, 2016 at 10:15 AM, Peter Rosin <peda@lysator.liu.se> wrote:
> On 2016-11-30 04:00, Eliot Moss wrote:
>> On 11/29/2016 7:28 PM, Herbert Stocker wrote:
>>
>>> Or we simply copy the variables unconverted to the Windows environment
>>> block, as the Windows env block is not used for anything else - up to
>>> now it is not updated at all.
>>
>> Maybe we could stash them inside one large variable?  Would that help
>> at all?  (It does demand a quoting convention, but since this is all
>> internal stuff, that may not be too hard.)
>
> I believe this is a dead end. The Windows environment block has an
> unacceptable size limit which is circumvented by the internal Cygwin
> handling. Since this is about exposing said Cygwin environment, it
> would simply not work for big environments.

I agree--although I can't off the top of my head recall where I've
read this, I believe there's a deliberate effort in Cygwin to keep the
Windows environment to the bare minimum for Cygwin processes.  Among
other things I imagine it keeps the number of conversions necessary to
a minimum.

And in any case, trying to cache the entire Cygwin environment back
into the Windows environment would be even more overhead than what I'm
proposing which is to store it in shared memory.  My goal is not to
make the Cygwin environment of a process readable from outside
Cygwin--only make it so it can be read from other Cygwin processes by
way of /proc (given the necessary permissions).

Perhaps I'll go ahead and cook up a patch and see what impact it has.
With luck it won't be much--I just don't know how much overhead there
is in open_shared.

Thanks,
Erik

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Retrieving per-process environment block?

[Peter Rosin]

On 2016-11-30 04:00, Eliot Moss wrote:
> On 11/29/2016 7:28 PM, Herbert Stocker wrote:
> 
>> Or we simply copy the variables unconverted to the Windows environment
>> block, as the Windows env block is not used for anything else - up to
>> now it is not updated at all.
> 
> Maybe we could stash them inside one large variable?  Would that help
> at all?  (It does demand a quoting convention, but since this is all
> internal stuff, that may not be too hard.)

I believe this is a dead end. The Windows environment block has an
unacceptable size limit which is circumvented by the internal Cygwin
handling. Since this is about exposing said Cygwin environment, it
would simply not work for big environments.

Cheers,
Peter

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Retrieving per-process environment block?

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 4690 bytes --]

Hi Erik,

On Nov 29 14:26, Erik Bray wrote:
> On Thu, Nov 17, 2016 at 3:00 PM, Corinna Vinschen
> <corinna-cygwin@cygwin.com> wrote:
> > On Nov 17 14:30, Erik Bray wrote:
> >> Hi all,
> >>
> >> For a quick bit of background, I'm working on porting the highly
> >> useful psutil [1] Python library to Cygwin.  This has proved an
> >> interesting exercise, as much of the functionality of psutil works on
> >> Cygwin through existing POSIX interfaces, and a handful of
> >> Linux-specific interfaces as well.  But there are some bits that
> >> simply don't map at all.
> >>
> >> The one I'm struggling with right now is retrieving Cygwin environment
> >> variables for a process (under inspection--i.e. not listing a
> >> process's environment from within that process which is obviously
> >> trivial).
> >>
> >> I've looked at every route I could conceive of but as far as I can
> >> tell this is currently impossible.  That's fine for now--I simply
> >> disable that functionality in psutil.  But it is unfortunate, though,
> >> since the information is there.
> >>
> >> There are a couple avenues I could see to this.  The most "obvious"
> >> (to me) being to implement /proc/<pid>/environ.
> >>
> >> I would be willing to provide a patch for this if it would be
> >> accepted.  Is there some particular non-obvious hurdle to this that it
> >> hasn't been implemented?  Obviously there are security
> >> implications--the /proc/<pid>/environ should only be readable to the
> >> process's owner, but that is already within Cygwin's capabilities, and
> >> works for other /proc files.
> >
> > Patch welcome.  Implementing this should be fairly straightforward.
> > The only hurdle is winsup/CONTRIBUTORS ;)
> 
> Thanks--I went to go work on this finally but it turns out not to be
> straightforward after all, as the process's environment is not shared
> in any way between processes.
> 
> I could do this, if each process kept a copy of its environment block
> in shared memory, which would in turn have to be updated every time
> the process's environment is updated.  But I don't know what the
> impact of that would be performance-wise.

You're right, it's not *that* straightforward.  I got carried away by
the idea but didn't think this through when replying to you.

So, how to implement this?

We have two types of information in /proc/$PID, one is information
readily available without having to contact the target process, the
other information is local to the target process and we have to get the
information with the target processes consent.  Argc, argv pointers
are in the second group.

So how do we contact the other process to ask for information?

We have a mechanism inside Cygwin to request info from another process.
It's part of the signal handling and consists basically of two
functions.  The applicant calls _pinfo::commune_request(), this will
send a request into the signal pipe of the target process, this in turn
will call commune_process(), a callback function, within the target
process.

Have a look into an example:

Start in fhandler_process.cc, function format_process_cmdline()
which implements /proc/$PID/cmdline.

It fetches the _pinfo pointer of the target process and calls
_pinfo::cmdline.

_pinfo::cmdline (in pinfo.cc) checks if the target process is a
native Windows process a Cygwin process, or if its itself.

In the native Windows case, it opens a handle to the target process,
fetches its RTL_USER_PROCESS_PARAMETERS block (function
open_commune_proc_parms).  and fetches the info from the target process
by directly reading from its memory.  If it's itself it just constructs
the info as desired and returns.  Boring.

In the Cygwin case, it calls _pinfo::commune_request (PICOM_CMDLINE).
PICOM_CMDLINE is defined in pinfo.h, together with other values for the
commune requests.  It send the request over the signal pipe to the
target process.

The target process receives the request and calls commune_process().  It
checks what info is requested, prepares the info and send it back over
over the signal pipe.

The (waiting) _pinfo::commune_request fetches the info from the pipe and
returns to _pinfo::cmdline, which in turn returns to format_process_cmdline().

So, ultimately, just copy the functionality of format_process_cmdline,
_pinfo::cmdline, as well as the handling of PICOM_CMDLINE in
_pinfo::commune_request and commune_process and you're done.


Does that help?


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Retrieving per-process environment block?

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 946 bytes --]

On Nov 29 09:11, cyg Simple wrote:
> On 11/29/2016 8:26 AM, Erik Bray wrote:
> > I could do this, if each process kept a copy of its environment block
> > in shared memory, which would in turn have to be updated every time
> > the process's environment is updated.  But I don't know what the
> > impact of that would be performance-wise.
> > 
> > Any advice?
> > 
> 
> Sounds like a job for a thread that wakes every X time units to check
> the contents of the environment.  Or is there a notification API that
> could be used to wake the thread?  I know there is a disk change
> notification API for this; maybe one for environment changes as well.

Cygwin env == POSIX env != Windows env.  Not only the content, but
also the internal memory handling is different.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Retrieving per-process environment block?

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 879 bytes --]

On Nov 29 09:28, Eliot Moss wrote:
> On 11/29/2016 8:26 AM, Erik Bray wrote:
> > I could do this, if each process kept a copy of its environment block
> > in shared memory, which would in turn have to be updated every time
> > the process's environment is updated.  But I don't know what the
> > impact of that would be performance-wise.
> 
> Sorry, no advice getting around this, but a thought about why it may have
> been hard -- it can be a security hole.

Using the mechanism outlined in my mail from 2 minutes ago, there's
no security problem.  Opening the signal pipe is only allowed to
same user processes or administrative processes.  Other will get a
/proc/$PID/environ content of "<defunct>"


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Retrieving per-process environment block?

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 882 bytes --]

On Nov 29 19:27, Andrey Repin wrote:
> Greetings, Erik Bray!
> 
> > Thanks for the reply.  The issue here isn't getting the Windows
> > process environment--that can be done, albeit trickily [1].  While
> > it's true there are security implications, in this case that is
> > handled at the OpenProcess call required to obtain a handle to the
> > process.  It's true you can't just read the environment from a process
> > you don't have permission to.
> 
> > The issue here though is reading the Cygwin process's environment
> > which is separate from its Windows environment, and the above
> > technique is inapplicable.
> 
> Update Windows environment block when Cygwin environment is updated.

Nope.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Retrieving per-process environment block?

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 961 bytes --]

On Nov 30 10:15, Peter Rosin wrote:
> On 2016-11-30 04:00, Eliot Moss wrote:
> > On 11/29/2016 7:28 PM, Herbert Stocker wrote:
> > 
> >> Or we simply copy the variables unconverted to the Windows environment
> >> block, as the Windows env block is not used for anything else - up to
> >> now it is not updated at all.
> > 
> > Maybe we could stash them inside one large variable?  Would that help
> > at all?  (It does demand a quoting convention, but since this is all
> > internal stuff, that may not be too hard.)
> 
> I believe this is a dead end. The Windows environment block has an
> unacceptable size limit which is circumvented by the internal Cygwin
> handling. Since this is about exposing said Cygwin environment, it
> would simply not work for big environments.

ACK


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Retrieving per-process environment block?

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1323 bytes --]

On Nov 30 11:48, Corinna Vinschen wrote:
> On Nov 30 10:15, Peter Rosin wrote:
> > On 2016-11-30 04:00, Eliot Moss wrote:
> > > On 11/29/2016 7:28 PM, Herbert Stocker wrote:
> > > 
> > >> Or we simply copy the variables unconverted to the Windows environment
> > >> block, as the Windows env block is not used for anything else - up to
> > >> now it is not updated at all.
> > > 
> > > Maybe we could stash them inside one large variable?  Would that help
> > > at all?  (It does demand a quoting convention, but since this is all
> > > internal stuff, that may not be too hard.)
> > 
> > I believe this is a dead end. The Windows environment block has an
> > unacceptable size limit which is circumvented by the internal Cygwin
> > handling. Since this is about exposing said Cygwin environment, it
> > would simply not work for big environments.
> 
> ACK

Uhm, sorry, not ACK, this is nonsense.  The Windows env is not limited
to 32K characters, only the CreateProcess call is, unless you use the
option CREATE_UNICODE_ENVIRONMENT.  In that case, you can pass bigger
envs.  Cygwin uses it, too, when calling native Windows processes.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Retrieving per-process environment block?

[Erik Bray]

On Wed, Nov 30, 2016 at 11:43 AM, Corinna Vinschen
<corinna-cygwin@cygwin.com> wrote:
> Hi Erik,
>
> On Nov 29 14:26, Erik Bray wrote:
>> On Thu, Nov 17, 2016 at 3:00 PM, Corinna Vinschen
>> <corinna-cygwin@cygwin.com> wrote:
>> > On Nov 17 14:30, Erik Bray wrote:
>> >> Hi all,
>> >>
>> >> For a quick bit of background, I'm working on porting the highly
>> >> useful psutil [1] Python library to Cygwin.  This has proved an
>> >> interesting exercise, as much of the functionality of psutil works on
>> >> Cygwin through existing POSIX interfaces, and a handful of
>> >> Linux-specific interfaces as well.  But there are some bits that
>> >> simply don't map at all.
>> >>
>> >> The one I'm struggling with right now is retrieving Cygwin environment
>> >> variables for a process (under inspection--i.e. not listing a
>> >> process's environment from within that process which is obviously
>> >> trivial).
>> >>
>> >> I've looked at every route I could conceive of but as far as I can
>> >> tell this is currently impossible.  That's fine for now--I simply
>> >> disable that functionality in psutil.  But it is unfortunate, though,
>> >> since the information is there.
>> >>
>> >> There are a couple avenues I could see to this.  The most "obvious"
>> >> (to me) being to implement /proc/<pid>/environ.
>> >>
>> >> I would be willing to provide a patch for this if it would be
>> >> accepted.  Is there some particular non-obvious hurdle to this that it
>> >> hasn't been implemented?  Obviously there are security
>> >> implications--the /proc/<pid>/environ should only be readable to the
>> >> process's owner, but that is already within Cygwin's capabilities, and
>> >> works for other /proc files.
>> >
>> > Patch welcome.  Implementing this should be fairly straightforward.
>> > The only hurdle is winsup/CONTRIBUTORS ;)
>>
>> Thanks--I went to go work on this finally but it turns out not to be
>> straightforward after all, as the process's environment is not shared
>> in any way between processes.
>>
>> I could do this, if each process kept a copy of its environment block
>> in shared memory, which would in turn have to be updated every time
>> the process's environment is updated.  But I don't know what the
>> impact of that would be performance-wise.
>
> You're right, it's not *that* straightforward.  I got carried away by
> the idea but didn't think this through when replying to you.
>
> So, how to implement this?
>
> We have two types of information in /proc/$PID, one is information
> readily available without having to contact the target process, the
> other information is local to the target process and we have to get the
> information with the target processes consent.  Argc, argv pointers
> are in the second group.
>
> So how do we contact the other process to ask for information?
>
> We have a mechanism inside Cygwin to request info from another process.
> It's part of the signal handling and consists basically of two
> functions.  The applicant calls _pinfo::commune_request(), this will
> send a request into the signal pipe of the target process, this in turn
> will call commune_process(), a callback function, within the target
> process.
>
> Have a look into an example:
>
> Start in fhandler_process.cc, function format_process_cmdline()
> which implements /proc/$PID/cmdline.
>
> It fetches the _pinfo pointer of the target process and calls
> _pinfo::cmdline.
>
> _pinfo::cmdline (in pinfo.cc) checks if the target process is a
> native Windows process a Cygwin process, or if its itself.
>
> In the native Windows case, it opens a handle to the target process,
> fetches its RTL_USER_PROCESS_PARAMETERS block (function
> open_commune_proc_parms).  and fetches the info from the target process
> by directly reading from its memory.  If it's itself it just constructs
> the info as desired and returns.  Boring.
>
> In the Cygwin case, it calls _pinfo::commune_request (PICOM_CMDLINE).
> PICOM_CMDLINE is defined in pinfo.h, together with other values for the
> commune requests.  It send the request over the signal pipe to the
> target process.
>
> The target process receives the request and calls commune_process().  It
> checks what info is requested, prepares the info and send it back over
> over the signal pipe.
>
> The (waiting) _pinfo::commune_request fetches the info from the pipe and
> returns to _pinfo::cmdline, which in turn returns to format_process_cmdline().
>
> So, ultimately, just copy the functionality of format_process_cmdline,
> _pinfo::cmdline, as well as the handling of PICOM_CMDLINE in
> _pinfo::commune_request and commune_process and you're done.

Hi Corinna,

Thank you for the detailed guidance on that--that all makes sense.  I
feel silly for missing that--I was actually poking around to see if
there was already something like that in the source but missed it.
For some reason I thought the cmdline was just stored directly in the
pinfo object itself.

But I knew, considering how signals are handled, that there had to be
some communication channel between processes; I just didn't know how
to use it for general purposes.  I just played around with it a bit
and I get the gist of how __SIGCOMMUNE works, etc.  Very clever.

I can see what to do now--thanks.

Erik

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Retrieving per-process environment block?

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 3767 bytes --]

On Nov 30 12:35, Erik Bray wrote:
> On Wed, Nov 30, 2016 at 11:43 AM, Corinna Vinschen
> <corinna-cygwin@cygwin.com> wrote:
> > So how do we contact the other process to ask for information?
> >
> > We have a mechanism inside Cygwin to request info from another process.
> > It's part of the signal handling and consists basically of two
> > functions.  The applicant calls _pinfo::commune_request(), this will
> > send a request into the signal pipe of the target process, this in turn
> > will call commune_process(), a callback function, within the target
> > process.
> >
> > Have a look into an example:
> >
> > Start in fhandler_process.cc, function format_process_cmdline()
> > which implements /proc/$PID/cmdline.
> >
> > It fetches the _pinfo pointer of the target process and calls
> > _pinfo::cmdline.
> >
> > _pinfo::cmdline (in pinfo.cc) checks if the target process is a
> > native Windows process a Cygwin process, or if its itself.
> >
> > In the native Windows case, it opens a handle to the target process,
> > fetches its RTL_USER_PROCESS_PARAMETERS block (function
> > open_commune_proc_parms).  and fetches the info from the target process
> > by directly reading from its memory.  If it's itself it just constructs
> > the info as desired and returns.  Boring.
> >
> > In the Cygwin case, it calls _pinfo::commune_request (PICOM_CMDLINE).
> > PICOM_CMDLINE is defined in pinfo.h, together with other values for the
> > commune requests.  It send the request over the signal pipe to the
> > target process.
> >
> > The target process receives the request and calls commune_process().  It
> > checks what info is requested, prepares the info and send it back over
> > over the signal pipe.
> >
> > The (waiting) _pinfo::commune_request fetches the info from the pipe and
> > returns to _pinfo::cmdline, which in turn returns to format_process_cmdline().
> >
> > So, ultimately, just copy the functionality of format_process_cmdline,
> > _pinfo::cmdline, as well as the handling of PICOM_CMDLINE in
> > _pinfo::commune_request and commune_process and you're done.
> 
> Hi Corinna,
> 
> Thank you for the detailed guidance on that--that all makes sense.  I
> feel silly for missing that--I was actually poking around to see if
> there was already something like that in the source but missed it.
> For some reason I thought the cmdline was just stored directly in the
> pinfo object itself.

There's a catch.  The shared mem regions maintained by Cygwin are
supposed to fit in one allocation page, that is, 64K, at a reproducible
slot in the processes VM (think fork).  Variable size information,
especially information which can have an almost arbitrary size, is quite
tricky to be held in shared mem.

> But I knew, considering how signals are handled, that there had to be
> some communication channel between processes; I just didn't know how
> to use it for general purposes.  I just played around with it a bit
> and I get the gist of how __SIGCOMMUNE works, etc.  Very clever.

Curtesy cgf.  The commune mechanism obviously depends on the reliability
of the signal handling.  It also has a problem in that, since it's a part
of signal handling itself, it can't be interrupted by signals.  In earlier
stages it was pretty bad, but at one point we just changed the pipe calls
to timeout after 1 second each, so either it works or it times out after
a max of 2 secs and the information is just not available on the client
side.  There's worse than that.

> I can see what to do now--thanks.

Cool.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]