* Retrieving per-process environment block? @ 2016-11-17 18:05 Erik Bray 2016-11-17 18:11 ` Corinna Vinschen 0 siblings, 1 reply; 19+ messages in thread From: Erik Bray @ 2016-11-17 18:05 UTC (permalink / raw) To: cygwin Hi all, For a quick bit of background, I'm working on porting the highly useful psutil [1] Python library to Cygwin. This has proved an interesting exercise, as much of the functionality of psutil works on Cygwin through existing POSIX interfaces, and a handful of Linux-specific interfaces as well. But there are some bits that simply don't map at all. The one I'm struggling with right now is retrieving Cygwin environment variables for a process (under inspection--i.e. not listing a process's environment from within that process which is obviously trivial). I've looked at every route I could conceive of but as far as I can tell this is currently impossible. That's fine for now--I simply disable that functionality in psutil. But it is unfortunate, though, since the information is there. There are a couple avenues I could see to this. The most "obvious" (to me) being to implement /proc/<pid>/environ. I would be willing to provide a patch for this if it would be accepted. Is there some particular non-obvious hurdle to this that it hasn't been implemented? Obviously there are security implications--the /proc/<pid>/environ should only be readable to the process's owner, but that is already within Cygwin's capabilities, and works for other /proc files. Thanks, Erik [1] https://pythonhosted.org/psutil/ -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Retrieving per-process environment block? 2016-11-17 18:05 Retrieving per-process environment block? Erik Bray @ 2016-11-17 18:11 ` Corinna Vinschen 2016-11-29 15:28 ` Erik Bray 0 siblings, 1 reply; 19+ messages in thread From: Corinna Vinschen @ 2016-11-17 18:11 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1683 bytes --] On Nov 17 14:30, Erik Bray wrote: > Hi all, > > For a quick bit of background, I'm working on porting the highly > useful psutil [1] Python library to Cygwin. This has proved an > interesting exercise, as much of the functionality of psutil works on > Cygwin through existing POSIX interfaces, and a handful of > Linux-specific interfaces as well. But there are some bits that > simply don't map at all. > > The one I'm struggling with right now is retrieving Cygwin environment > variables for a process (under inspection--i.e. not listing a > process's environment from within that process which is obviously > trivial). > > I've looked at every route I could conceive of but as far as I can > tell this is currently impossible. That's fine for now--I simply > disable that functionality in psutil. But it is unfortunate, though, > since the information is there. > > There are a couple avenues I could see to this. The most "obvious" > (to me) being to implement /proc/<pid>/environ. > > I would be willing to provide a patch for this if it would be > accepted. Is there some particular non-obvious hurdle to this that it > hasn't been implemented? Obviously there are security > implications--the /proc/<pid>/environ should only be readable to the > process's owner, but that is already within Cygwin's capabilities, and > works for other /proc files. Patch welcome. Implementing this should be fairly straightforward. The only hurdle is winsup/CONTRIBUTORS ;) Thanks, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Retrieving per-process environment block? 2016-11-17 18:11 ` Corinna Vinschen @ 2016-11-29 15:28 ` Erik Bray 2016-11-29 16:01 ` cyg Simple ` (2 more replies) 0 siblings, 3 replies; 19+ messages in thread From: Erik Bray @ 2016-11-29 15:28 UTC (permalink / raw) To: cygwin On Thu, Nov 17, 2016 at 3:00 PM, Corinna Vinschen <corinna-cygwin@cygwin.com> wrote: > On Nov 17 14:30, Erik Bray wrote: >> Hi all, >> >> For a quick bit of background, I'm working on porting the highly >> useful psutil [1] Python library to Cygwin. This has proved an >> interesting exercise, as much of the functionality of psutil works on >> Cygwin through existing POSIX interfaces, and a handful of >> Linux-specific interfaces as well. But there are some bits that >> simply don't map at all. >> >> The one I'm struggling with right now is retrieving Cygwin environment >> variables for a process (under inspection--i.e. not listing a >> process's environment from within that process which is obviously >> trivial). >> >> I've looked at every route I could conceive of but as far as I can >> tell this is currently impossible. That's fine for now--I simply >> disable that functionality in psutil. But it is unfortunate, though, >> since the information is there. >> >> There are a couple avenues I could see to this. The most "obvious" >> (to me) being to implement /proc/<pid>/environ. >> >> I would be willing to provide a patch for this if it would be >> accepted. Is there some particular non-obvious hurdle to this that it >> hasn't been implemented? Obviously there are security >> implications--the /proc/<pid>/environ should only be readable to the >> process's owner, but that is already within Cygwin's capabilities, and >> works for other /proc files. > > Patch welcome. Implementing this should be fairly straightforward. > The only hurdle is winsup/CONTRIBUTORS ;) Thanks--I went to go work on this finally but it turns out not to be straightforward after all, as the process's environment is not shared in any way between processes. I could do this, if each process kept a copy of its environment block in shared memory, which would in turn have to be updated every time the process's environment is updated. But I don't know what the impact of that would be performance-wise. Any advice? Thanks, Erik -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Retrieving per-process environment block? 2016-11-29 15:28 ` Erik Bray @ 2016-11-29 16:01 ` cyg Simple 2016-11-30 11:03 ` Corinna Vinschen 2016-11-29 16:35 ` Eliot Moss 2016-11-30 10:48 ` Corinna Vinschen 2 siblings, 1 reply; 19+ messages in thread From: cyg Simple @ 2016-11-29 16:01 UTC (permalink / raw) To: cygwin On 11/29/2016 8:26 AM, Erik Bray wrote: > On Thu, Nov 17, 2016 at 3:00 PM, Corinna Vinschen > <corinna-cygwin@cygwin.com> wrote: >> On Nov 17 14:30, Erik Bray wrote: >>> Hi all, >>> >>> For a quick bit of background, I'm working on porting the highly >>> useful psutil [1] Python library to Cygwin. This has proved an >>> interesting exercise, as much of the functionality of psutil works on >>> Cygwin through existing POSIX interfaces, and a handful of >>> Linux-specific interfaces as well. But there are some bits that >>> simply don't map at all. >>> >>> The one I'm struggling with right now is retrieving Cygwin environment >>> variables for a process (under inspection--i.e. not listing a >>> process's environment from within that process which is obviously >>> trivial). >>> >>> I've looked at every route I could conceive of but as far as I can >>> tell this is currently impossible. That's fine for now--I simply >>> disable that functionality in psutil. But it is unfortunate, though, >>> since the information is there. >>> >>> There are a couple avenues I could see to this. The most "obvious" >>> (to me) being to implement /proc/<pid>/environ. >>> >>> I would be willing to provide a patch for this if it would be >>> accepted. Is there some particular non-obvious hurdle to this that it >>> hasn't been implemented? Obviously there are security >>> implications--the /proc/<pid>/environ should only be readable to the >>> process's owner, but that is already within Cygwin's capabilities, and >>> works for other /proc files. >> >> Patch welcome. Implementing this should be fairly straightforward. >> The only hurdle is winsup/CONTRIBUTORS ;) > > Thanks--I went to go work on this finally but it turns out not to be > straightforward after all, as the process's environment is not shared > in any way between processes. > > I could do this, if each process kept a copy of its environment block > in shared memory, which would in turn have to be updated every time > the process's environment is updated. But I don't know what the > impact of that would be performance-wise. > > Any advice? > Sounds like a job for a thread that wakes every X time units to check the contents of the environment. Or is there a notification API that could be used to wake the thread? I know there is a disk change notification API for this; maybe one for environment changes as well. -- cyg Simple -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Retrieving per-process environment block? 2016-11-29 16:01 ` cyg Simple @ 2016-11-30 11:03 ` Corinna Vinschen 0 siblings, 0 replies; 19+ messages in thread From: Corinna Vinschen @ 2016-11-30 11:03 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 946 bytes --] On Nov 29 09:11, cyg Simple wrote: > On 11/29/2016 8:26 AM, Erik Bray wrote: > > I could do this, if each process kept a copy of its environment block > > in shared memory, which would in turn have to be updated every time > > the process's environment is updated. But I don't know what the > > impact of that would be performance-wise. > > > > Any advice? > > > > Sounds like a job for a thread that wakes every X time units to check > the contents of the environment. Or is there a notification API that > could be used to wake the thread? I know there is a disk change > notification API for this; maybe one for environment changes as well. Cygwin env == POSIX env != Windows env. Not only the content, but also the internal memory handling is different. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Retrieving per-process environment block? 2016-11-29 15:28 ` Erik Bray 2016-11-29 16:01 ` cyg Simple @ 2016-11-29 16:35 ` Eliot Moss 2016-11-29 17:42 ` Erik Bray 2016-11-30 11:06 ` Corinna Vinschen 2016-11-30 10:48 ` Corinna Vinschen 2 siblings, 2 replies; 19+ messages in thread From: Eliot Moss @ 2016-11-29 16:35 UTC (permalink / raw) To: cygwin On 11/29/2016 8:26 AM, Erik Bray wrote: > On Thu, Nov 17, 2016 at 3:00 PM, Corinna Vinschen > <corinna-cygwin@cygwin.com> wrote: >> On Nov 17 14:30, Erik Bray wrote: >>> Hi all, >>> >>> For a quick bit of background, I'm working on porting the highly >>> useful psutil [1] Python library to Cygwin. This has proved an >>> interesting exercise, as much of the functionality of psutil works on >>> Cygwin through existing POSIX interfaces, and a handful of >>> Linux-specific interfaces as well. But there are some bits that >>> simply don't map at all. >>> >>> The one I'm struggling with right now is retrieving Cygwin environment >>> variables for a process (under inspection--i.e. not listing a >>> process's environment from within that process which is obviously >>> trivial). >>> >>> I've looked at every route I could conceive of but as far as I can >>> tell this is currently impossible. That's fine for now--I simply >>> disable that functionality in psutil. But it is unfortunate, though, >>> since the information is there. >>> >>> There are a couple avenues I could see to this. The most "obvious" >>> (to me) being to implement /proc/<pid>/environ. >>> >>> I would be willing to provide a patch for this if it would be >>> accepted. Is there some particular non-obvious hurdle to this that it >>> hasn't been implemented? Obviously there are security >>> implications--the /proc/<pid>/environ should only be readable to the >>> process's owner, but that is already within Cygwin's capabilities, and >>> works for other /proc files. >> >> Patch welcome. Implementing this should be fairly straightforward. >> The only hurdle is winsup/CONTRIBUTORS ;) > > Thanks--I went to go work on this finally but it turns out not to be > straightforward after all, as the process's environment is not shared > in any way between processes. > > I could do this, if each process kept a copy of its environment block > in shared memory, which would in turn have to be updated every time > the process's environment is updated. But I don't know what the > impact of that would be performance-wise. Sorry, no advice getting around this, but a thought about why it may have been hard -- it can be a security hole. Given Windows' complex access rights, how would one determine which processes should reasonably have access to another process's environment variables? Maybe Windows decided to punt the issue. (I don't know anything about this at all - as I said, just a thought.) A question: Are there any Windows tools that can do this? If so, you might be able to trace them to find what they do. Regards -- Eliot Moss -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Retrieving per-process environment block? 2016-11-29 16:35 ` Eliot Moss @ 2016-11-29 17:42 ` Erik Bray 2016-11-29 18:02 ` Andrey Repin 2016-11-30 11:06 ` Corinna Vinschen 1 sibling, 1 reply; 19+ messages in thread From: Erik Bray @ 2016-11-29 17:42 UTC (permalink / raw) To: moss, cygwin On Tue, Nov 29, 2016 at 3:28 PM, Eliot Moss <moss@cs.umass.edu> wrote: > On 11/29/2016 8:26 AM, Erik Bray wrote: >> >> On Thu, Nov 17, 2016 at 3:00 PM, Corinna Vinschen >> <corinna-cygwin@cygwin.com> wrote: >>> >>> On Nov 17 14:30, Erik Bray wrote: >>>> >>>> Hi all, >>>> >>>> For a quick bit of background, I'm working on porting the highly >>>> useful psutil [1] Python library to Cygwin. This has proved an >>>> interesting exercise, as much of the functionality of psutil works on >>>> Cygwin through existing POSIX interfaces, and a handful of >>>> Linux-specific interfaces as well. But there are some bits that >>>> simply don't map at all. >>>> >>>> The one I'm struggling with right now is retrieving Cygwin environment >>>> variables for a process (under inspection--i.e. not listing a >>>> process's environment from within that process which is obviously >>>> trivial). >>>> >>>> I've looked at every route I could conceive of but as far as I can >>>> tell this is currently impossible. That's fine for now--I simply >>>> disable that functionality in psutil. But it is unfortunate, though, >>>> since the information is there. >>>> >>>> There are a couple avenues I could see to this. The most "obvious" >>>> (to me) being to implement /proc/<pid>/environ. >>>> >>>> I would be willing to provide a patch for this if it would be >>>> accepted. Is there some particular non-obvious hurdle to this that it >>>> hasn't been implemented? Obviously there are security >>>> implications--the /proc/<pid>/environ should only be readable to the >>>> process's owner, but that is already within Cygwin's capabilities, and >>>> works for other /proc files. >>> >>> >>> Patch welcome. Implementing this should be fairly straightforward. >>> The only hurdle is winsup/CONTRIBUTORS ;) >> >> >> Thanks--I went to go work on this finally but it turns out not to be >> straightforward after all, as the process's environment is not shared >> in any way between processes. >> >> I could do this, if each process kept a copy of its environment block >> in shared memory, which would in turn have to be updated every time >> the process's environment is updated. But I don't know what the >> impact of that would be performance-wise. > > > Sorry, no advice getting around this, but a thought about why it may have > been hard -- it can be a security hole. Given Windows' complex access > rights, > how would one determine which processes should reasonably have access to > another process's environment variables? Maybe Windows decided to punt the > issue. (I don't know anything about this at all - as I said, just a > thought.) > > A question: Are there any Windows tools that can do this? If so, you might > be able to trace them to find what they do. Hi, Thanks for the reply. The issue here isn't getting the Windows process environment--that can be done, albeit trickily [1]. While it's true there are security implications, in this case that is handled at the OpenProcess call required to obtain a handle to the process. It's true you can't just read the environment from a process you don't have permission to. The issue here though is reading the Cygwin process's environment which is separate from its Windows environment, and the above technique is inapplicable. The easiest way, as I suggested, would be to keep a copy of each process's Cygwin environment in shared memory as is the case with the pinfo struct. I just don't know if it's worth the effort for this relatively narrow case (it's no so much effort to implement, I just worry what performance impacts that could have, though I suppose I could just try it and see...) [1] https://theroadtodelphi.com/2012/06/09/getting-the-environment-variables-of-an-external-x86-and-x64-process/ -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Retrieving per-process environment block? 2016-11-29 17:42 ` Erik Bray @ 2016-11-29 18:02 ` Andrey Repin 2016-11-30 4:28 ` Herbert Stocker 2016-11-30 11:36 ` Corinna Vinschen 0 siblings, 2 replies; 19+ messages in thread From: Andrey Repin @ 2016-11-29 18:02 UTC (permalink / raw) To: Erik Bray, cygwin Greetings, Erik Bray! > Thanks for the reply. The issue here isn't getting the Windows > process environment--that can be done, albeit trickily [1]. While > it's true there are security implications, in this case that is > handled at the OpenProcess call required to obtain a handle to the > process. It's true you can't just read the environment from a process > you don't have permission to. > The issue here though is reading the Cygwin process's environment > which is separate from its Windows environment, and the above > technique is inapplicable. Update Windows environment block when Cygwin environment is updated. -- With best regards, Andrey Repin Tuesday, November 29, 2016 19:26:20 Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Retrieving per-process environment block? 2016-11-29 18:02 ` Andrey Repin @ 2016-11-30 4:28 ` Herbert Stocker 2016-11-30 10:43 ` Eliot Moss 2016-11-30 11:36 ` Corinna Vinschen 1 sibling, 1 reply; 19+ messages in thread From: Herbert Stocker @ 2016-11-30 4:28 UTC (permalink / raw) To: cygwin Hi, On 29.11.2016 17:27, Andrey Repin wrote: > Greetings, Erik Bray! > >> Thanks for the reply. The issue here isn't getting the Windows >> process environment--that can be done, albeit trickily [1]. While >> it's true there are security implications, in this case that is >> handled at the OpenProcess call required to obtain a handle to the >> process. It's true you can't just read the environment from a process >> you don't have permission to. > >> The issue here though is reading the Cygwin process's environment >> which is separate from its Windows environment, and the above >> technique is inapplicable. > > Update Windows environment block when Cygwin environment is updated. > How about the case where paths are translated from Windows style to Posix style when the Windows environment is copied to Cygwin environ- ment on process initialization? Should they be converted back? And thus psutils has to convert them back again to Posix style? What will happen if a Cygwin process stores a windows path in its environment? Maybe each such variable should have a second variable in the Windows environment block that contains the Posix version of the variable and psutils picks that, in order to avoid round trip conversion inaccura- cies. Yea it's me again advocating against round trip conversions. Or we simply copy the variables unconverted to the Windows environment block, as the Windows env block is not used for anything else - up to now it is not updated at all. Herbert -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Retrieving per-process environment block? 2016-11-30 4:28 ` Herbert Stocker @ 2016-11-30 10:43 ` Eliot Moss 2016-11-30 10:47 ` Peter Rosin 0 siblings, 1 reply; 19+ messages in thread From: Eliot Moss @ 2016-11-30 10:43 UTC (permalink / raw) To: cygwin On 11/29/2016 7:28 PM, Herbert Stocker wrote: > Or we simply copy the variables unconverted to the Windows environment > block, as the Windows env block is not used for anything else - up to > now it is not updated at all. Maybe we could stash them inside one large variable? Would that help at all? (It does demand a quoting convention, but since this is all internal stuff, that may not be too hard.) Regards -- Eliot -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Retrieving per-process environment block? 2016-11-30 10:43 ` Eliot Moss @ 2016-11-30 10:47 ` Peter Rosin 2016-11-30 10:47 ` Erik Bray 2016-11-30 12:29 ` Corinna Vinschen 0 siblings, 2 replies; 19+ messages in thread From: Peter Rosin @ 2016-11-30 10:47 UTC (permalink / raw) To: cygwin On 2016-11-30 04:00, Eliot Moss wrote: > On 11/29/2016 7:28 PM, Herbert Stocker wrote: > >> Or we simply copy the variables unconverted to the Windows environment >> block, as the Windows env block is not used for anything else - up to >> now it is not updated at all. > > Maybe we could stash them inside one large variable? Would that help > at all? (It does demand a quoting convention, but since this is all > internal stuff, that may not be too hard.) I believe this is a dead end. The Windows environment block has an unacceptable size limit which is circumvented by the internal Cygwin handling. Since this is about exposing said Cygwin environment, it would simply not work for big environments. Cheers, Peter -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Retrieving per-process environment block? 2016-11-30 10:47 ` Peter Rosin @ 2016-11-30 10:47 ` Erik Bray 2016-11-30 12:29 ` Corinna Vinschen 1 sibling, 0 replies; 19+ messages in thread From: Erik Bray @ 2016-11-30 10:47 UTC (permalink / raw) To: cygwin On Wed, Nov 30, 2016 at 10:15 AM, Peter Rosin <peda@lysator.liu.se> wrote: > On 2016-11-30 04:00, Eliot Moss wrote: >> On 11/29/2016 7:28 PM, Herbert Stocker wrote: >> >>> Or we simply copy the variables unconverted to the Windows environment >>> block, as the Windows env block is not used for anything else - up to >>> now it is not updated at all. >> >> Maybe we could stash them inside one large variable? Would that help >> at all? (It does demand a quoting convention, but since this is all >> internal stuff, that may not be too hard.) > > I believe this is a dead end. The Windows environment block has an > unacceptable size limit which is circumvented by the internal Cygwin > handling. Since this is about exposing said Cygwin environment, it > would simply not work for big environments. I agree--although I can't off the top of my head recall where I've read this, I believe there's a deliberate effort in Cygwin to keep the Windows environment to the bare minimum for Cygwin processes. Among other things I imagine it keeps the number of conversions necessary to a minimum. And in any case, trying to cache the entire Cygwin environment back into the Windows environment would be even more overhead than what I'm proposing which is to store it in shared memory. My goal is not to make the Cygwin environment of a process readable from outside Cygwin--only make it so it can be read from other Cygwin processes by way of /proc (given the necessary permissions). Perhaps I'll go ahead and cook up a patch and see what impact it has. With luck it won't be much--I just don't know how much overhead there is in open_shared. Thanks, Erik -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Retrieving per-process environment block? 2016-11-30 10:47 ` Peter Rosin 2016-11-30 10:47 ` Erik Bray @ 2016-11-30 12:29 ` Corinna Vinschen 2016-11-30 12:36 ` Corinna Vinschen 1 sibling, 1 reply; 19+ messages in thread From: Corinna Vinschen @ 2016-11-30 12:29 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 961 bytes --] On Nov 30 10:15, Peter Rosin wrote: > On 2016-11-30 04:00, Eliot Moss wrote: > > On 11/29/2016 7:28 PM, Herbert Stocker wrote: > > > >> Or we simply copy the variables unconverted to the Windows environment > >> block, as the Windows env block is not used for anything else - up to > >> now it is not updated at all. > > > > Maybe we could stash them inside one large variable? Would that help > > at all? (It does demand a quoting convention, but since this is all > > internal stuff, that may not be too hard.) > > I believe this is a dead end. The Windows environment block has an > unacceptable size limit which is circumvented by the internal Cygwin > handling. Since this is about exposing said Cygwin environment, it > would simply not work for big environments. ACK Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Retrieving per-process environment block? 2016-11-30 12:29 ` Corinna Vinschen @ 2016-11-30 12:36 ` Corinna Vinschen 0 siblings, 0 replies; 19+ messages in thread From: Corinna Vinschen @ 2016-11-30 12:36 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1323 bytes --] On Nov 30 11:48, Corinna Vinschen wrote: > On Nov 30 10:15, Peter Rosin wrote: > > On 2016-11-30 04:00, Eliot Moss wrote: > > > On 11/29/2016 7:28 PM, Herbert Stocker wrote: > > > > > >> Or we simply copy the variables unconverted to the Windows environment > > >> block, as the Windows env block is not used for anything else - up to > > >> now it is not updated at all. > > > > > > Maybe we could stash them inside one large variable? Would that help > > > at all? (It does demand a quoting convention, but since this is all > > > internal stuff, that may not be too hard.) > > > > I believe this is a dead end. The Windows environment block has an > > unacceptable size limit which is circumvented by the internal Cygwin > > handling. Since this is about exposing said Cygwin environment, it > > would simply not work for big environments. > > ACK Uhm, sorry, not ACK, this is nonsense. The Windows env is not limited to 32K characters, only the CreateProcess call is, unless you use the option CREATE_UNICODE_ENVIRONMENT. In that case, you can pass bigger envs. Cygwin uses it, too, when calling native Windows processes. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Retrieving per-process environment block? 2016-11-29 18:02 ` Andrey Repin 2016-11-30 4:28 ` Herbert Stocker @ 2016-11-30 11:36 ` Corinna Vinschen 1 sibling, 0 replies; 19+ messages in thread From: Corinna Vinschen @ 2016-11-30 11:36 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 882 bytes --] On Nov 29 19:27, Andrey Repin wrote: > Greetings, Erik Bray! > > > Thanks for the reply. The issue here isn't getting the Windows > > process environment--that can be done, albeit trickily [1]. While > > it's true there are security implications, in this case that is > > handled at the OpenProcess call required to obtain a handle to the > > process. It's true you can't just read the environment from a process > > you don't have permission to. > > > The issue here though is reading the Cygwin process's environment > > which is separate from its Windows environment, and the above > > technique is inapplicable. > > Update Windows environment block when Cygwin environment is updated. Nope. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Retrieving per-process environment block? 2016-11-29 16:35 ` Eliot Moss 2016-11-29 17:42 ` Erik Bray @ 2016-11-30 11:06 ` Corinna Vinschen 1 sibling, 0 replies; 19+ messages in thread From: Corinna Vinschen @ 2016-11-30 11:06 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 879 bytes --] On Nov 29 09:28, Eliot Moss wrote: > On 11/29/2016 8:26 AM, Erik Bray wrote: > > I could do this, if each process kept a copy of its environment block > > in shared memory, which would in turn have to be updated every time > > the process's environment is updated. But I don't know what the > > impact of that would be performance-wise. > > Sorry, no advice getting around this, but a thought about why it may have > been hard -- it can be a security hole. Using the mechanism outlined in my mail from 2 minutes ago, there's no security problem. Opening the signal pipe is only allowed to same user processes or administrative processes. Other will get a /proc/$PID/environ content of "<defunct>" Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Retrieving per-process environment block? 2016-11-29 15:28 ` Erik Bray 2016-11-29 16:01 ` cyg Simple 2016-11-29 16:35 ` Eliot Moss @ 2016-11-30 10:48 ` Corinna Vinschen 2016-11-30 14:49 ` Erik Bray 2 siblings, 1 reply; 19+ messages in thread From: Corinna Vinschen @ 2016-11-30 10:48 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 4690 bytes --] Hi Erik, On Nov 29 14:26, Erik Bray wrote: > On Thu, Nov 17, 2016 at 3:00 PM, Corinna Vinschen > <corinna-cygwin@cygwin.com> wrote: > > On Nov 17 14:30, Erik Bray wrote: > >> Hi all, > >> > >> For a quick bit of background, I'm working on porting the highly > >> useful psutil [1] Python library to Cygwin. This has proved an > >> interesting exercise, as much of the functionality of psutil works on > >> Cygwin through existing POSIX interfaces, and a handful of > >> Linux-specific interfaces as well. But there are some bits that > >> simply don't map at all. > >> > >> The one I'm struggling with right now is retrieving Cygwin environment > >> variables for a process (under inspection--i.e. not listing a > >> process's environment from within that process which is obviously > >> trivial). > >> > >> I've looked at every route I could conceive of but as far as I can > >> tell this is currently impossible. That's fine for now--I simply > >> disable that functionality in psutil. But it is unfortunate, though, > >> since the information is there. > >> > >> There are a couple avenues I could see to this. The most "obvious" > >> (to me) being to implement /proc/<pid>/environ. > >> > >> I would be willing to provide a patch for this if it would be > >> accepted. Is there some particular non-obvious hurdle to this that it > >> hasn't been implemented? Obviously there are security > >> implications--the /proc/<pid>/environ should only be readable to the > >> process's owner, but that is already within Cygwin's capabilities, and > >> works for other /proc files. > > > > Patch welcome. Implementing this should be fairly straightforward. > > The only hurdle is winsup/CONTRIBUTORS ;) > > Thanks--I went to go work on this finally but it turns out not to be > straightforward after all, as the process's environment is not shared > in any way between processes. > > I could do this, if each process kept a copy of its environment block > in shared memory, which would in turn have to be updated every time > the process's environment is updated. But I don't know what the > impact of that would be performance-wise. You're right, it's not *that* straightforward. I got carried away by the idea but didn't think this through when replying to you. So, how to implement this? We have two types of information in /proc/$PID, one is information readily available without having to contact the target process, the other information is local to the target process and we have to get the information with the target processes consent. Argc, argv pointers are in the second group. So how do we contact the other process to ask for information? We have a mechanism inside Cygwin to request info from another process. It's part of the signal handling and consists basically of two functions. The applicant calls _pinfo::commune_request(), this will send a request into the signal pipe of the target process, this in turn will call commune_process(), a callback function, within the target process. Have a look into an example: Start in fhandler_process.cc, function format_process_cmdline() which implements /proc/$PID/cmdline. It fetches the _pinfo pointer of the target process and calls _pinfo::cmdline. _pinfo::cmdline (in pinfo.cc) checks if the target process is a native Windows process a Cygwin process, or if its itself. In the native Windows case, it opens a handle to the target process, fetches its RTL_USER_PROCESS_PARAMETERS block (function open_commune_proc_parms). and fetches the info from the target process by directly reading from its memory. If it's itself it just constructs the info as desired and returns. Boring. In the Cygwin case, it calls _pinfo::commune_request (PICOM_CMDLINE). PICOM_CMDLINE is defined in pinfo.h, together with other values for the commune requests. It send the request over the signal pipe to the target process. The target process receives the request and calls commune_process(). It checks what info is requested, prepares the info and send it back over over the signal pipe. The (waiting) _pinfo::commune_request fetches the info from the pipe and returns to _pinfo::cmdline, which in turn returns to format_process_cmdline(). So, ultimately, just copy the functionality of format_process_cmdline, _pinfo::cmdline, as well as the handling of PICOM_CMDLINE in _pinfo::commune_request and commune_process and you're done. Does that help? Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Retrieving per-process environment block? 2016-11-30 10:48 ` Corinna Vinschen @ 2016-11-30 14:49 ` Erik Bray 2016-11-30 15:04 ` Corinna Vinschen 0 siblings, 1 reply; 19+ messages in thread From: Erik Bray @ 2016-11-30 14:49 UTC (permalink / raw) To: cygwin On Wed, Nov 30, 2016 at 11:43 AM, Corinna Vinschen <corinna-cygwin@cygwin.com> wrote: > Hi Erik, > > On Nov 29 14:26, Erik Bray wrote: >> On Thu, Nov 17, 2016 at 3:00 PM, Corinna Vinschen >> <corinna-cygwin@cygwin.com> wrote: >> > On Nov 17 14:30, Erik Bray wrote: >> >> Hi all, >> >> >> >> For a quick bit of background, I'm working on porting the highly >> >> useful psutil [1] Python library to Cygwin. This has proved an >> >> interesting exercise, as much of the functionality of psutil works on >> >> Cygwin through existing POSIX interfaces, and a handful of >> >> Linux-specific interfaces as well. But there are some bits that >> >> simply don't map at all. >> >> >> >> The one I'm struggling with right now is retrieving Cygwin environment >> >> variables for a process (under inspection--i.e. not listing a >> >> process's environment from within that process which is obviously >> >> trivial). >> >> >> >> I've looked at every route I could conceive of but as far as I can >> >> tell this is currently impossible. That's fine for now--I simply >> >> disable that functionality in psutil. But it is unfortunate, though, >> >> since the information is there. >> >> >> >> There are a couple avenues I could see to this. The most "obvious" >> >> (to me) being to implement /proc/<pid>/environ. >> >> >> >> I would be willing to provide a patch for this if it would be >> >> accepted. Is there some particular non-obvious hurdle to this that it >> >> hasn't been implemented? Obviously there are security >> >> implications--the /proc/<pid>/environ should only be readable to the >> >> process's owner, but that is already within Cygwin's capabilities, and >> >> works for other /proc files. >> > >> > Patch welcome. Implementing this should be fairly straightforward. >> > The only hurdle is winsup/CONTRIBUTORS ;) >> >> Thanks--I went to go work on this finally but it turns out not to be >> straightforward after all, as the process's environment is not shared >> in any way between processes. >> >> I could do this, if each process kept a copy of its environment block >> in shared memory, which would in turn have to be updated every time >> the process's environment is updated. But I don't know what the >> impact of that would be performance-wise. > > You're right, it's not *that* straightforward. I got carried away by > the idea but didn't think this through when replying to you. > > So, how to implement this? > > We have two types of information in /proc/$PID, one is information > readily available without having to contact the target process, the > other information is local to the target process and we have to get the > information with the target processes consent. Argc, argv pointers > are in the second group. > > So how do we contact the other process to ask for information? > > We have a mechanism inside Cygwin to request info from another process. > It's part of the signal handling and consists basically of two > functions. The applicant calls _pinfo::commune_request(), this will > send a request into the signal pipe of the target process, this in turn > will call commune_process(), a callback function, within the target > process. > > Have a look into an example: > > Start in fhandler_process.cc, function format_process_cmdline() > which implements /proc/$PID/cmdline. > > It fetches the _pinfo pointer of the target process and calls > _pinfo::cmdline. > > _pinfo::cmdline (in pinfo.cc) checks if the target process is a > native Windows process a Cygwin process, or if its itself. > > In the native Windows case, it opens a handle to the target process, > fetches its RTL_USER_PROCESS_PARAMETERS block (function > open_commune_proc_parms). and fetches the info from the target process > by directly reading from its memory. If it's itself it just constructs > the info as desired and returns. Boring. > > In the Cygwin case, it calls _pinfo::commune_request (PICOM_CMDLINE). > PICOM_CMDLINE is defined in pinfo.h, together with other values for the > commune requests. It send the request over the signal pipe to the > target process. > > The target process receives the request and calls commune_process(). It > checks what info is requested, prepares the info and send it back over > over the signal pipe. > > The (waiting) _pinfo::commune_request fetches the info from the pipe and > returns to _pinfo::cmdline, which in turn returns to format_process_cmdline(). > > So, ultimately, just copy the functionality of format_process_cmdline, > _pinfo::cmdline, as well as the handling of PICOM_CMDLINE in > _pinfo::commune_request and commune_process and you're done. Hi Corinna, Thank you for the detailed guidance on that--that all makes sense. I feel silly for missing that--I was actually poking around to see if there was already something like that in the source but missed it. For some reason I thought the cmdline was just stored directly in the pinfo object itself. But I knew, considering how signals are handled, that there had to be some communication channel between processes; I just didn't know how to use it for general purposes. I just played around with it a bit and I get the gist of how __SIGCOMMUNE works, etc. Very clever. I can see what to do now--thanks. Erik -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Retrieving per-process environment block? 2016-11-30 14:49 ` Erik Bray @ 2016-11-30 15:04 ` Corinna Vinschen 0 siblings, 0 replies; 19+ messages in thread From: Corinna Vinschen @ 2016-11-30 15:04 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 3767 bytes --] On Nov 30 12:35, Erik Bray wrote: > On Wed, Nov 30, 2016 at 11:43 AM, Corinna Vinschen > <corinna-cygwin@cygwin.com> wrote: > > So how do we contact the other process to ask for information? > > > > We have a mechanism inside Cygwin to request info from another process. > > It's part of the signal handling and consists basically of two > > functions. The applicant calls _pinfo::commune_request(), this will > > send a request into the signal pipe of the target process, this in turn > > will call commune_process(), a callback function, within the target > > process. > > > > Have a look into an example: > > > > Start in fhandler_process.cc, function format_process_cmdline() > > which implements /proc/$PID/cmdline. > > > > It fetches the _pinfo pointer of the target process and calls > > _pinfo::cmdline. > > > > _pinfo::cmdline (in pinfo.cc) checks if the target process is a > > native Windows process a Cygwin process, or if its itself. > > > > In the native Windows case, it opens a handle to the target process, > > fetches its RTL_USER_PROCESS_PARAMETERS block (function > > open_commune_proc_parms). and fetches the info from the target process > > by directly reading from its memory. If it's itself it just constructs > > the info as desired and returns. Boring. > > > > In the Cygwin case, it calls _pinfo::commune_request (PICOM_CMDLINE). > > PICOM_CMDLINE is defined in pinfo.h, together with other values for the > > commune requests. It send the request over the signal pipe to the > > target process. > > > > The target process receives the request and calls commune_process(). It > > checks what info is requested, prepares the info and send it back over > > over the signal pipe. > > > > The (waiting) _pinfo::commune_request fetches the info from the pipe and > > returns to _pinfo::cmdline, which in turn returns to format_process_cmdline(). > > > > So, ultimately, just copy the functionality of format_process_cmdline, > > _pinfo::cmdline, as well as the handling of PICOM_CMDLINE in > > _pinfo::commune_request and commune_process and you're done. > > Hi Corinna, > > Thank you for the detailed guidance on that--that all makes sense. I > feel silly for missing that--I was actually poking around to see if > there was already something like that in the source but missed it. > For some reason I thought the cmdline was just stored directly in the > pinfo object itself. There's a catch. The shared mem regions maintained by Cygwin are supposed to fit in one allocation page, that is, 64K, at a reproducible slot in the processes VM (think fork). Variable size information, especially information which can have an almost arbitrary size, is quite tricky to be held in shared mem. > But I knew, considering how signals are handled, that there had to be > some communication channel between processes; I just didn't know how > to use it for general purposes. I just played around with it a bit > and I get the gist of how __SIGCOMMUNE works, etc. Very clever. Curtesy cgf. The commune mechanism obviously depends on the reliability of the signal handling. It also has a problem in that, since it's a part of signal handling itself, it can't be interrupted by signals. In earlier stages it was pretty bad, but at one point we just changed the pipe calls to timeout after 1 second each, so either it works or it times out after a max of 2 secs and the information is just not available on the client side. There's worse than that. > I can see what to do now--thanks. Cool. Thanks, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2016-11-30 12:29 UTC | newest] Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2016-11-17 18:05 Retrieving per-process environment block? Erik Bray 2016-11-17 18:11 ` Corinna Vinschen 2016-11-29 15:28 ` Erik Bray 2016-11-29 16:01 ` cyg Simple 2016-11-30 11:03 ` Corinna Vinschen 2016-11-29 16:35 ` Eliot Moss 2016-11-29 17:42 ` Erik Bray 2016-11-29 18:02 ` Andrey Repin 2016-11-30 4:28 ` Herbert Stocker 2016-11-30 10:43 ` Eliot Moss 2016-11-30 10:47 ` Peter Rosin 2016-11-30 10:47 ` Erik Bray 2016-11-30 12:29 ` Corinna Vinschen 2016-11-30 12:36 ` Corinna Vinschen 2016-11-30 11:36 ` Corinna Vinschen 2016-11-30 11:06 ` Corinna Vinschen 2016-11-30 10:48 ` Corinna Vinschen 2016-11-30 14:49 ` Erik Bray 2016-11-30 15:04 ` Corinna Vinschen
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).