Re: (call-process ...) hangs in emacs

Re: (call-process ...) hangs in emacs

[Angelo Graziosi]

  Katsumi Yamaoka wrote:
> I'm troubled with a similar problem since Wednesday[1].
> /usr/bin/emacs that Cygwin distributed (24.3) seems ok, but
> /usr/local/bin/emacs that I built from the Emacs trunk Tuesday
> (24.4.50) got not to work conditionally.  With that version of
> Emacs:
>
> Evaluating the form `(call-process "pwd" nil t)' on normally
> running Emacs works.  It returns "/home/yamaoka" immediately.
> However, if it is done in the bacth mode,
>
> /usr/local/bin/emacs -Q -batch -eval '(call-process "pwd" nil t)'
>
> it never returns; the Emacs process eats no cpu but stays
> consistently[2].  `kill -9 PID' has no effect.
> Using the Windows Task Manager is the only means to kill it.

> [1] I did updating my Cygwin installation on Wednesday morning
>  for the first time in the last 7 days.  It must be the trigger
>  of the problem.  I also tried clean install of Cygwin, however
>  it didn't help.  Since it didn't seem to finish because of
>  texlive-collection-basic.sh (the bash process didn't eat cpu,
>  but never returned), I redid it by marking all the texlive
>  packages to be `Skip'.

Just for completeness.. or to add another story to the eternal Cygwin - 
Emacs saga..

Since I upgraded to Cygwin-1.7.31*, I see similar issue in building 
Emacs trunk (--with-w32 build)... The build always hangs in compiling 
some .el file. CTRL-C does not work and I have to search the running 
processes with "ps" and kill them with 'kill -9'. Downgrading to 1.7.30, 
things work better. Now I am using 1.7.30...

Here is on Win7 64 and cygwin64.

Ciao,
  Angelo.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Peter Hull]

On Fri, Aug 1, 2014 at 1:50 PM, Angelo Graziosi
<angelo.graziosi@alice.it> wrote:
> Since I upgraded to Cygwin-1.7.31*, I see similar issue in building Emacs
> trunk (--with-w32 build)... The build always hangs in compiling some .el
> file. CTRL-C does not work and I have to search the running processes with
> "ps" and kill them with 'kill -9'. Downgrading to 1.7.30, things work
> better. Now I am using 1.7.30...
By better, do you mean 'perfectly'? It seems like it might be a little
bit intermittent, from the reports I have seen.

It's easy enough to do a cvs rdiff between the releases if 1.7.30 is
known to be good - I am happy to help but I am unfamiliar with the
code so I don't know where to start looking...

Pete

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1104 bytes --]

On Aug  1 14:17, Peter Hull wrote:
> On Fri, Aug 1, 2014 at 1:50 PM, Angelo Graziosi
> <angelo.graziosi@alice.it> wrote:
> > Since I upgraded to Cygwin-1.7.31*, I see similar issue in building Emacs
> > trunk (--with-w32 build)... The build always hangs in compiling some .el
> > file. CTRL-C does not work and I have to search the running processes with
> > "ps" and kill them with 'kill -9'. Downgrading to 1.7.30, things work
> > better. Now I am using 1.7.30...
> By better, do you mean 'perfectly'? It seems like it might be a little
> bit intermittent, from the reports I have seen.
> 
> It's easy enough to do a cvs rdiff between the releases if 1.7.30 is
> known to be good - I am happy to help but I am unfamiliar with the
> code so I don't know where to start looking...

It could be a problem with the new default pthread mutexes being
NORMAL, rather then ERRORCHECK mutexes.  Somebody would have to
debug it...


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: (call-process ...) hangs in emacs

[Ken Brown]

On 8/1/2014 9:32 AM, Corinna Vinschen wrote:
> On Aug  1 14:17, Peter Hull wrote:
>> On Fri, Aug 1, 2014 at 1:50 PM, Angelo Graziosi
>> <angelo.graziosi@alice.it> wrote:
>>> Since I upgraded to Cygwin-1.7.31*, I see similar issue in building Emacs
>>> trunk (--with-w32 build)... The build always hangs in compiling some .el
>>> file. CTRL-C does not work and I have to search the running processes with
>>> "ps" and kill them with 'kill -9'. Downgrading to 1.7.30, things work
>>> better. Now I am using 1.7.30...
>> By better, do you mean 'perfectly'? It seems like it might be a little
>> bit intermittent, from the reports I have seen.
>>
>> It's easy enough to do a cvs rdiff between the releases if 1.7.30 is
>> known to be good - I am happy to help but I am unfamiliar with the
>> code so I don't know where to start looking...
>
> It could be a problem with the new default pthread mutexes being
> NORMAL, rather then ERRORCHECK mutexes.

That does seem to be the problem, since I can reproduce the bug starting 
with the 2014-07-14 snapshot.  More precisely, I can reproduce it using 
emacs-nox (which is what the OP was using according to his cygcheck 
output) but not using emacs-X11 or emacs-w32.

I tried running emacs under gdb with a breakpoint at call_process, but 
all I could see from that is that emacs tries to fork a subprocess, but 
the call to fork() never returns.  I also tried running it under strace, 
but again all I can see is that fork() is called and then everything 
seems to be at a standstill.

Corinna, if you want to take a look, here's the precise recipe:

1. emacs-nox -Q [This should start emacs and put you in the *scratch* 
buffer.]

2. Enter the following text into the buffer:

   (call-process "pwd" nil t)

3. Position the cursor at the end of the line and type Ctrl-j.

What should happen, and what does happen prior to the 2014-07-14 
snapshot, is that the current directory is displayed, followed by the 
exit code of 0.  What happens instead is that emacs appears to hang.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2547 bytes --]

On Aug  3 21:02, Ken Brown wrote:
> On 8/1/2014 9:32 AM, Corinna Vinschen wrote:
> >On Aug  1 14:17, Peter Hull wrote:
> >>On Fri, Aug 1, 2014 at 1:50 PM, Angelo Graziosi
> >><angelo.graziosi@alice.it> wrote:
> >>>Since I upgraded to Cygwin-1.7.31*, I see similar issue in building Emacs
> >>>trunk (--with-w32 build)... The build always hangs in compiling some .el
> >>>file. CTRL-C does not work and I have to search the running processes with
> >>>"ps" and kill them with 'kill -9'. Downgrading to 1.7.30, things work
> >>>better. Now I am using 1.7.30...
> >>By better, do you mean 'perfectly'? It seems like it might be a little
> >>bit intermittent, from the reports I have seen.
> >>
> >>It's easy enough to do a cvs rdiff between the releases if 1.7.30 is
> >>known to be good - I am happy to help but I am unfamiliar with the
> >>code so I don't know where to start looking...
> >
> >It could be a problem with the new default pthread mutexes being
> >NORMAL, rather then ERRORCHECK mutexes.
> 
> That does seem to be the problem, since I can reproduce the bug starting
> with the 2014-07-14 snapshot.  More precisely, I can reproduce it using
> emacs-nox (which is what the OP was using according to his cygcheck output)
> but not using emacs-X11 or emacs-w32.
> 
> I tried running emacs under gdb with a breakpoint at call_process, but all I
> could see from that is that emacs tries to fork a subprocess, but the call
> to fork() never returns.  I also tried running it under strace, but again
> all I can see is that fork() is called and then everything seems to be at a
> standstill.
> 
> Corinna, if you want to take a look, here's the precise recipe:
> 
> 1. emacs-nox -Q [This should start emacs and put you in the *scratch*
> buffer.]
> 
> 2. Enter the following text into the buffer:
> 
>   (call-process "pwd" nil t)
> 
> 3. Position the cursor at the end of the line and type Ctrl-j.
> 
> What should happen, and what does happen prior to the 2014-07-14 snapshot,
> is that the current directory is displayed, followed by the exit code of 0.
> What happens instead is that emacs appears to hang.

How does emacs start a process?  Does it create a thread and then
forks and execs from the thread?  Does it use its own pthread_mutex
to control the job?  Is there a chance to create an STC of this
process?


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: (call-process ...) hangs in emacs

[Ken Brown]

On 8/4/2014 4:00 AM, Corinna Vinschen wrote:
> On Aug  3 21:02, Ken Brown wrote:
>> On 8/1/2014 9:32 AM, Corinna Vinschen wrote:
>>> On Aug  1 14:17, Peter Hull wrote:
>>>> On Fri, Aug 1, 2014 at 1:50 PM, Angelo Graziosi
>>>> <angelo.graziosi@alice.it> wrote:
>>>>> Since I upgraded to Cygwin-1.7.31*, I see similar issue in building Emacs
>>>>> trunk (--with-w32 build)... The build always hangs in compiling some .el
>>>>> file. CTRL-C does not work and I have to search the running processes with
>>>>> "ps" and kill them with 'kill -9'. Downgrading to 1.7.30, things work
>>>>> better. Now I am using 1.7.30...
>>>> By better, do you mean 'perfectly'? It seems like it might be a little
>>>> bit intermittent, from the reports I have seen.
>>>>
>>>> It's easy enough to do a cvs rdiff between the releases if 1.7.30 is
>>>> known to be good - I am happy to help but I am unfamiliar with the
>>>> code so I don't know where to start looking...
>>>
>>> It could be a problem with the new default pthread mutexes being
>>> NORMAL, rather then ERRORCHECK mutexes.
>>
>> That does seem to be the problem, since I can reproduce the bug starting
>> with the 2014-07-14 snapshot.  More precisely, I can reproduce it using
>> emacs-nox (which is what the OP was using according to his cygcheck output)
>> but not using emacs-X11 or emacs-w32.
>>
>> I tried running emacs under gdb with a breakpoint at call_process, but all I
>> could see from that is that emacs tries to fork a subprocess, but the call
>> to fork() never returns.  I also tried running it under strace, but again
>> all I can see is that fork() is called and then everything seems to be at a
>> standstill.
>>
>> Corinna, if you want to take a look, here's the precise recipe:
>>
>> 1. emacs-nox -Q [This should start emacs and put you in the *scratch*
>> buffer.]
>>
>> 2. Enter the following text into the buffer:
>>
>>    (call-process "pwd" nil t)
>>
>> 3. Position the cursor at the end of the line and type Ctrl-j.
>>
>> What should happen, and what does happen prior to the 2014-07-14 snapshot,
>> is that the current directory is displayed, followed by the exit code of 0.
>> What happens instead is that emacs appears to hang.
>
> How does emacs start a process?  Does it create a thread and then
> forks and execs from the thread?  Does it use its own pthread_mutex
> to control the job?  Is there a chance to create an STC of this
> process?

emacs does some bookkeeping and then calls vfork.  It does not create a 
new thread, nor does it create a pthread_mutex.  The only 
pthread_mutexes created anywhere in the emacs source code are in its 
implementation of malloc and friends, not in anything directly related 
to controlling subprocesses.  (FWIW, this malloc implementation is used 
in the Cygwin build of emacs but not in the Linux build.)

I did think about trying to create an STC, but I'm stymied because the 
problem depends so strongly on how emacs is run:

  - If emacs is run interactively, the problem only occurs with 
emacs-nox, not with emacs-X11 or emacs-w32.

  - If emacs is run non-interactively (i.e., in batch mode), the problem 
occurs with emacs-w32 and emacs-X11 too, as Angelo and Katsumi pointed 
out earlier in the thread.

I can't think of any way to capture these peculiarities in an STC.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 3299 bytes --]

On Aug  4 09:34, Ken Brown wrote:
> On 8/4/2014 4:00 AM, Corinna Vinschen wrote:
> >On Aug  3 21:02, Ken Brown wrote:
> >>On 8/1/2014 9:32 AM, Corinna Vinschen wrote:
> >>>It could be a problem with the new default pthread mutexes being
> >>>NORMAL, rather then ERRORCHECK mutexes.
> >>
> >>That does seem to be the problem, since I can reproduce the bug starting
> >>with the 2014-07-14 snapshot.  More precisely, I can reproduce it using
> >>emacs-nox (which is what the OP was using according to his cygcheck output)
> >>but not using emacs-X11 or emacs-w32.
> >>
> >>I tried running emacs under gdb with a breakpoint at call_process, but all I
> >>could see from that is that emacs tries to fork a subprocess, but the call
> >>to fork() never returns.  I also tried running it under strace, but again
> >>all I can see is that fork() is called and then everything seems to be at a
> >>standstill.
> >>
> >>Corinna, if you want to take a look, here's the precise recipe:
> >>
> >>1. emacs-nox -Q [This should start emacs and put you in the *scratch*
> >>buffer.]
> >>
> >>2. Enter the following text into the buffer:
> >>
> >>   (call-process "pwd" nil t)
> >>
> >>3. Position the cursor at the end of the line and type Ctrl-j.
> >>
> >>What should happen, and what does happen prior to the 2014-07-14 snapshot,
> >>is that the current directory is displayed, followed by the exit code of 0.
> >>What happens instead is that emacs appears to hang.
> >
> >How does emacs start a process?  Does it create a thread and then
> >forks and execs from the thread?  Does it use its own pthread_mutex
> >to control the job?  Is there a chance to create an STC of this
> >process?
> 
> emacs does some bookkeeping and then calls vfork.  It does not create a new
> thread, nor does it create a pthread_mutex.  The only pthread_mutexes
> created anywhere in the emacs source code are in its implementation of
> malloc and friends, not in anything directly related to controlling
> subprocesses.  (FWIW, this malloc implementation is used in the Cygwin build
> of emacs but not in the Linux build.)

Can you take a close look here?  This malloc will be used by Cygwin
as well if it's implemented in the usual way and...

> I did think about trying to create an STC, but I'm stymied because the
> problem depends so strongly on how emacs is run:
> 
>  - If emacs is run interactively, the problem only occurs with emacs-nox,
> not with emacs-X11 or emacs-w32.
> 
>  - If emacs is run non-interactively (i.e., in batch mode), the problem
> occurs with emacs-w32 and emacs-X11 too, as Angelo and Katsumi pointed out
> earlier in the thread.
> 
> I can't think of any way to capture these peculiarities in an STC.

...this, and the fact that fork/exec (vfork == fork on Cygwin) still
works nicely in other scenarios points to some problem with the usage of
pthread_mutexes in the application may be the culprit.

For instance, is it possible that emacs expects the pthread_mutexes
in malloc to be ERRORCHECK mutexes?  What if you explicitely set
them to ERRORCHECK at creation time?


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: (call-process ...) hangs in emacs

[Ken Brown]

On 8/4/2014 9:45 AM, Corinna Vinschen wrote:
> On Aug  4 09:34, Ken Brown wrote:
>> On 8/4/2014 4:00 AM, Corinna Vinschen wrote:
>>> On Aug  3 21:02, Ken Brown wrote:
>>>> On 8/1/2014 9:32 AM, Corinna Vinschen wrote:
>>>>> It could be a problem with the new default pthread mutexes being
>>>>> NORMAL, rather then ERRORCHECK mutexes.
>>>>
>>>> That does seem to be the problem, since I can reproduce the bug starting
>>>> with the 2014-07-14 snapshot.  More precisely, I can reproduce it using
>>>> emacs-nox (which is what the OP was using according to his cygcheck output)
>>>> but not using emacs-X11 or emacs-w32.
>>>>
>>>> I tried running emacs under gdb with a breakpoint at call_process, but all I
>>>> could see from that is that emacs tries to fork a subprocess, but the call
>>>> to fork() never returns.  I also tried running it under strace, but again
>>>> all I can see is that fork() is called and then everything seems to be at a
>>>> standstill.
>>>>
>>>> Corinna, if you want to take a look, here's the precise recipe:
>>>>
>>>> 1. emacs-nox -Q [This should start emacs and put you in the *scratch*
>>>> buffer.]
>>>>
>>>> 2. Enter the following text into the buffer:
>>>>
>>>>    (call-process "pwd" nil t)
>>>>
>>>> 3. Position the cursor at the end of the line and type Ctrl-j.
>>>>
>>>> What should happen, and what does happen prior to the 2014-07-14 snapshot,
>>>> is that the current directory is displayed, followed by the exit code of 0.
>>>> What happens instead is that emacs appears to hang.
>>>
>>> How does emacs start a process?  Does it create a thread and then
>>> forks and execs from the thread?  Does it use its own pthread_mutex
>>> to control the job?  Is there a chance to create an STC of this
>>> process?
>>
>> emacs does some bookkeeping and then calls vfork.  It does not create a new
>> thread, nor does it create a pthread_mutex.  The only pthread_mutexes
>> created anywhere in the emacs source code are in its implementation of
>> malloc and friends, not in anything directly related to controlling
>> subprocesses.  (FWIW, this malloc implementation is used in the Cygwin build
>> of emacs but not in the Linux build.)
>
> Can you take a close look here?  This malloc will be used by Cygwin
> as well if it's implemented in the usual way and...
>
>> I did think about trying to create an STC, but I'm stymied because the
>> problem depends so strongly on how emacs is run:
>>
>>   - If emacs is run interactively, the problem only occurs with emacs-nox,
>> not with emacs-X11 or emacs-w32.
>>
>>   - If emacs is run non-interactively (i.e., in batch mode), the problem
>> occurs with emacs-w32 and emacs-X11 too, as Angelo and Katsumi pointed out
>> earlier in the thread.
>>
>> I can't think of any way to capture these peculiarities in an STC.
>
> ...this, and the fact that fork/exec (vfork == fork on Cygwin) still
> works nicely in other scenarios points to some problem with the usage of
> pthread_mutexes in the application may be the culprit.
>
> For instance, is it possible that emacs expects the pthread_mutexes
> in malloc to be ERRORCHECK mutexes?  What if you explicitely set
> them to ERRORCHECK at creation time?

That doesn't seem to be the issue, but I think I did find the problem, 
and it looks like there might be both an emacs bug and a Cygwin bug. 
Here's the relevant code from emacs's gmalloc.c:

pthread_mutex_t _malloc_mutex = PTHREAD_MUTEX_INITIALIZER;
pthread_mutex_t _aligned_blocks_mutex = PTHREAD_MUTEX_INITIALIZER;

[...]

   /* Some pthread implementations call malloc for statically
      initialized mutexes when they are used first.  To avoid such a
      situation, we initialize mutexes here while their use is
      disabled in malloc etc.  */
   pthread_mutex_init (&_malloc_mutex, NULL);
   pthread_mutex_init (&_aligned_blocks_mutex, NULL);


The pthread_mutexes are initialized twice, resulting in undefined 
behavior according to Posix.  That's the emacs bug.  But simply removing 
the static initialization doesn't fix the problem.  On the other hand, 
the following patch does seem to fix it, at least in preliminary testing:

=== modified file 'src/gmalloc.c'
--- src/gmalloc.c       2014-03-04 19:02:49 +0000
+++ src/gmalloc.c       2014-08-05 01:35:38 +0000
@@ -490,8 +490,8 @@
  }

  #ifdef USE_PTHREAD
-pthread_mutex_t _malloc_mutex = PTHREAD_MUTEX_INITIALIZER;
-pthread_mutex_t _aligned_blocks_mutex = PTHREAD_MUTEX_INITIALIZER;
+pthread_mutex_t _malloc_mutex;
+pthread_mutex_t _aligned_blocks_mutex;
  int _malloc_thread_enabled_p;

  static void
@@ -526,8 +526,11 @@
       initialized mutexes when they are used first.  To avoid such a
       situation, we initialize mutexes here while their use is
       disabled in malloc etc.  */
-  pthread_mutex_init (&_malloc_mutex, NULL);
-  pthread_mutex_init (&_aligned_blocks_mutex, NULL);
+  pthread_mutexattr_t attr1, attr2;
+  pthread_mutexattr_settype (&attr1, PTHREAD_MUTEX_NORMAL);
+  pthread_mutexattr_settype (&attr2, PTHREAD_MUTEX_NORMAL);
+  pthread_mutex_init (&_malloc_mutex, &attr1);
+  pthread_mutex_init (&_aligned_blocks_mutex, &attr2);
    pthread_atfork (malloc_atfork_handler_prepare,
                   malloc_atfork_handler_parent,
                   malloc_atfork_handler_child);


The first hunk avoids the double initialization, but I don't understand 
why the second hunk does anything.  Since PTHREAD_MUTEX_NORMAL is now 
the default, shouldn't calling pthread_mutex_init with NULL second 
argument be equivalent to my calls to pthread_mutexattr_settype?  Does 
this indicate a Cygwin bug, or am I misunderstanding something?

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Peter Hull]

On Tue, Aug 5, 2014 at 1:21 PM, Ken Brown <kbrown@cornell.edu> wrote:
> -  pthread_mutex_init (&_malloc_mutex, NULL);
> -  pthread_mutex_init (&_aligned_blocks_mutex, NULL);
> +  pthread_mutexattr_t attr1, attr2;
> +  pthread_mutexattr_settype (&attr1, PTHREAD_MUTEX_NORMAL);
> +  pthread_mutexattr_settype (&attr2, PTHREAD_MUTEX_NORMAL);
> +  pthread_mutex_init (&_malloc_mutex, &attr1);
> +  pthread_mutex_init (&_aligned_blocks_mutex, &attr2);
>    pthread_atfork (malloc_atfork_handler_prepare,
>                   malloc_atfork_handler_parent,
>                   malloc_atfork_handler_child);
Does there need to be a 'pthread_mutexattr_init' in there? I don't
think that's the problem though...
Pete

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Peter Hull]

In my experiments, not calling pthread_mutexattr_init caused errors
such that the final mutex was invalid and could not be locked.

The difference between the explicitly initialised mutex and the
statically initialised one is that the latter does get 'lazily'
initialised when the mutex is first locked (I think...?) so  maybe the
problem is something to do with the timing of that?

Pete

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 3998 bytes --]

On Aug  5 08:21, Ken Brown wrote:
> On 8/4/2014 9:45 AM, Corinna Vinschen wrote:
> >...this, and the fact that fork/exec (vfork == fork on Cygwin) still
> >works nicely in other scenarios points to some problem with the usage of
> >pthread_mutexes in the application may be the culprit.
> >
> >For instance, is it possible that emacs expects the pthread_mutexes
> >in malloc to be ERRORCHECK mutexes?  What if you explicitely set
> >them to ERRORCHECK at creation time?
> 
> That doesn't seem to be the issue, but I think I did find the problem, and
> it looks like there might be both an emacs bug and a Cygwin bug. Here's the
> relevant code from emacs's gmalloc.c:
> 
> pthread_mutex_t _malloc_mutex = PTHREAD_MUTEX_INITIALIZER;
> pthread_mutex_t _aligned_blocks_mutex = PTHREAD_MUTEX_INITIALIZER;
> 
> [...]
> 
>   /* Some pthread implementations call malloc for statically
>      initialized mutexes when they are used first.  To avoid such a
>      situation, we initialize mutexes here while their use is
>      disabled in malloc etc.  */
>   pthread_mutex_init (&_malloc_mutex, NULL);
>   pthread_mutex_init (&_aligned_blocks_mutex, NULL);
> 
> 
> The pthread_mutexes are initialized twice, resulting in undefined behavior
> according to Posix.  That's the emacs bug.

That's not the problem.  It's not necessary to call pthread_mutex_init
on statically initialized mutexes, but it doesn't hurt either.  Only
when calling pthread_mutex_init twice on the same object it goes
downhill, especially when the first incarnation of the mutex was already
locked.

> But simply removing the static
> initialization doesn't fix the problem.  On the other hand, the following
> patch does seem to fix it, at least in preliminary testing:
> 
> === modified file 'src/gmalloc.c'
> --- src/gmalloc.c       2014-03-04 19:02:49 +0000
> +++ src/gmalloc.c       2014-08-05 01:35:38 +0000
> @@ -490,8 +490,8 @@
>  }
> 
>  #ifdef USE_PTHREAD
> -pthread_mutex_t _malloc_mutex = PTHREAD_MUTEX_INITIALIZER;
> -pthread_mutex_t _aligned_blocks_mutex = PTHREAD_MUTEX_INITIALIZER;
> +pthread_mutex_t _malloc_mutex;
> +pthread_mutex_t _aligned_blocks_mutex;
>  int _malloc_thread_enabled_p;
> 
>  static void
> @@ -526,8 +526,11 @@
>       initialized mutexes when they are used first.  To avoid such a
>       situation, we initialize mutexes here while their use is
>       disabled in malloc etc.  */
> -  pthread_mutex_init (&_malloc_mutex, NULL);
> -  pthread_mutex_init (&_aligned_blocks_mutex, NULL);
> +  pthread_mutexattr_t attr1, attr2;
> +  pthread_mutexattr_settype (&attr1, PTHREAD_MUTEX_NORMAL);
> +  pthread_mutexattr_settype (&attr2, PTHREAD_MUTEX_NORMAL);
> +  pthread_mutex_init (&_malloc_mutex, &attr1);
> +  pthread_mutex_init (&_aligned_blocks_mutex, &attr2);
>    pthread_atfork (malloc_atfork_handler_prepare,
>                   malloc_atfork_handler_parent,
>                   malloc_atfork_handler_child);
> 
> 
> The first hunk avoids the double initialization, but I don't understand why
> the second hunk does anything.  Since PTHREAD_MUTEX_NORMAL is now the
> default, shouldn't calling pthread_mutex_init with NULL second argument be
> equivalent to my calls to pthread_mutexattr_settype?  Does this indicate a
> Cygwin bug, or am I misunderstanding something?

AFAICS you're missing something.  Your pthread_mutexattr_t attr1, attr2
are not initialized.  They contain some random values, thus they are not
good objects.  The calls to pthread_mutexattr_settype as well as the
calls to pthread_mutex_init will fail with EINVAL, but you won't see it
due to missing error handling, and you end up without mutexes at all.
If you call pthread_mutexattr_init before calling
pthread_mutexattr_settype the situation shoul;d be the same as before.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: (call-process ...) hangs in emacs

[Ken Brown]

[-- Attachment #1: Type: text/plain, Size: 4399 bytes --]

On 8/5/2014 9:58 AM, Corinna Vinschen wrote:
> On Aug  5 08:21, Ken Brown wrote:
>> On 8/4/2014 9:45 AM, Corinna Vinschen wrote:
>>> ...this, and the fact that fork/exec (vfork == fork on Cygwin) still
>>> works nicely in other scenarios points to some problem with the usage of
>>> pthread_mutexes in the application may be the culprit.
>>>
>>> For instance, is it possible that emacs expects the pthread_mutexes
>>> in malloc to be ERRORCHECK mutexes?  What if you explicitely set
>>> them to ERRORCHECK at creation time?
>>
>> That doesn't seem to be the issue, but I think I did find the problem, and
>> it looks like there might be both an emacs bug and a Cygwin bug. Here's the
>> relevant code from emacs's gmalloc.c:
>>
>> pthread_mutex_t _malloc_mutex = PTHREAD_MUTEX_INITIALIZER;
>> pthread_mutex_t _aligned_blocks_mutex = PTHREAD_MUTEX_INITIALIZER;
>>
>> [...]
>>
>>    /* Some pthread implementations call malloc for statically
>>       initialized mutexes when they are used first.  To avoid such a
>>       situation, we initialize mutexes here while their use is
>>       disabled in malloc etc.  */
>>    pthread_mutex_init (&_malloc_mutex, NULL);
>>    pthread_mutex_init (&_aligned_blocks_mutex, NULL);
>>
>>
>> The pthread_mutexes are initialized twice, resulting in undefined behavior
>> according to Posix.  That's the emacs bug.
>
> That's not the problem.  It's not necessary to call pthread_mutex_init
> on statically initialized mutexes, but it doesn't hurt either.  Only
> when calling pthread_mutex_init twice on the same object it goes
> downhill, especially when the first incarnation of the mutex was already
> locked.
>
>> But simply removing the static
>> initialization doesn't fix the problem.  On the other hand, the following
>> patch does seem to fix it, at least in preliminary testing:
>>
>> === modified file 'src/gmalloc.c'
>> --- src/gmalloc.c       2014-03-04 19:02:49 +0000
>> +++ src/gmalloc.c       2014-08-05 01:35:38 +0000
>> @@ -490,8 +490,8 @@
>>   }
>>
>>   #ifdef USE_PTHREAD
>> -pthread_mutex_t _malloc_mutex = PTHREAD_MUTEX_INITIALIZER;
>> -pthread_mutex_t _aligned_blocks_mutex = PTHREAD_MUTEX_INITIALIZER;
>> +pthread_mutex_t _malloc_mutex;
>> +pthread_mutex_t _aligned_blocks_mutex;
>>   int _malloc_thread_enabled_p;
>>
>>   static void
>> @@ -526,8 +526,11 @@
>>        initialized mutexes when they are used first.  To avoid such a
>>        situation, we initialize mutexes here while their use is
>>        disabled in malloc etc.  */
>> -  pthread_mutex_init (&_malloc_mutex, NULL);
>> -  pthread_mutex_init (&_aligned_blocks_mutex, NULL);
>> +  pthread_mutexattr_t attr1, attr2;
>> +  pthread_mutexattr_settype (&attr1, PTHREAD_MUTEX_NORMAL);
>> +  pthread_mutexattr_settype (&attr2, PTHREAD_MUTEX_NORMAL);
>> +  pthread_mutex_init (&_malloc_mutex, &attr1);
>> +  pthread_mutex_init (&_aligned_blocks_mutex, &attr2);
>>     pthread_atfork (malloc_atfork_handler_prepare,
>>                    malloc_atfork_handler_parent,
>>                    malloc_atfork_handler_child);
>>
>>
>> The first hunk avoids the double initialization, but I don't understand why
>> the second hunk does anything.  Since PTHREAD_MUTEX_NORMAL is now the
>> default, shouldn't calling pthread_mutex_init with NULL second argument be
>> equivalent to my calls to pthread_mutexattr_settype?  Does this indicate a
>> Cygwin bug, or am I misunderstanding something?
>
> AFAICS you're missing something.  Your pthread_mutexattr_t attr1, attr2
> are not initialized.  They contain some random values, thus they are not
> good objects.  The calls to pthread_mutexattr_settype as well as the
> calls to pthread_mutex_init will fail with EINVAL, but you won't see it
> due to missing error handling, and you end up without mutexes at all.
> If you call pthread_mutexattr_init before calling
> pthread_mutexattr_settype the situation shoul;d be the same as before.

Thanks for catching my mistake.  Your earlier suggestion about 
explicitly setting the pthread_mutexes to be ERRORCHECK mutexes seems to 
fix the problem (as long as I remember to call pthread_mutexattr_init). 
  The revised patch is attached.  I went back to using both the static 
and dynamic initializations as in the original code, since you said 
that's harmless.

Angelo and Katsumi, could you test it and see if it solves the problems 
you reported?  If so, I'll issue new emacs releases.

Ken

[-- Attachment #2: pthread_mutex.patch --]
[-- Type: text/plain, Size: 1248 bytes --]

=== modified file 'src/gmalloc.c'
--- src/gmalloc.c	2014-03-04 19:02:49 +0000
+++ src/gmalloc.c	2014-08-05 17:30:18 +0000
@@ -490,8 +490,8 @@
 }
 
 #ifdef USE_PTHREAD
-pthread_mutex_t _malloc_mutex = PTHREAD_MUTEX_INITIALIZER;
-pthread_mutex_t _aligned_blocks_mutex = PTHREAD_MUTEX_INITIALIZER;
+pthread_mutex_t _malloc_mutex = PTHREAD_ERRORCHECK_MUTEX_INITIALIZER_NP;
+pthread_mutex_t _aligned_blocks_mutex = PTHREAD_ERRORCHECK_MUTEX_INITIALIZER_NP;
 int _malloc_thread_enabled_p;
 
 static void
@@ -526,8 +526,13 @@
      initialized mutexes when they are used first.  To avoid such a
      situation, we initialize mutexes here while their use is
      disabled in malloc etc.  */
-  pthread_mutex_init (&_malloc_mutex, NULL);
-  pthread_mutex_init (&_aligned_blocks_mutex, NULL);
+  pthread_mutexattr_t attr1, attr2;
+  pthread_mutexattr_init (&attr1);
+  pthread_mutexattr_init (&attr2);
+  pthread_mutexattr_settype (&attr1, PTHREAD_MUTEX_ERRORCHECK);
+  pthread_mutexattr_settype (&attr2, PTHREAD_MUTEX_ERRORCHECK);
+  pthread_mutex_init (&_malloc_mutex, &attr1);
+  pthread_mutex_init (&_aligned_blocks_mutex, &attr2);
   pthread_atfork (malloc_atfork_handler_prepare,
 		  malloc_atfork_handler_parent,
 		  malloc_atfork_handler_child);


[-- Attachment #3: Type: text/plain, Size: 218 bytes --]

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 3319 bytes --]

Hi Ken,

On Aug  5 13:55, Ken Brown wrote:
> On 8/5/2014 9:58 AM, Corinna Vinschen wrote:
> >On Aug  5 08:21, Ken Brown wrote:
> >>=== modified file 'src/gmalloc.c'
> >>--- src/gmalloc.c       2014-03-04 19:02:49 +0000
> >>+++ src/gmalloc.c       2014-08-05 01:35:38 +0000
> >>@@ -490,8 +490,8 @@
> >>  }
> >>
> >>  #ifdef USE_PTHREAD
> >>-pthread_mutex_t _malloc_mutex = PTHREAD_MUTEX_INITIALIZER;
> >>-pthread_mutex_t _aligned_blocks_mutex = PTHREAD_MUTEX_INITIALIZER;
> >>+pthread_mutex_t _malloc_mutex;
> >>+pthread_mutex_t _aligned_blocks_mutex;
> >>  int _malloc_thread_enabled_p;
> >>
> >>  static void
> >>@@ -526,8 +526,11 @@
> >>       initialized mutexes when they are used first.  To avoid such a
> >>       situation, we initialize mutexes here while their use is
> >>       disabled in malloc etc.  */
> >>-  pthread_mutex_init (&_malloc_mutex, NULL);
> >>-  pthread_mutex_init (&_aligned_blocks_mutex, NULL);
> >>+  pthread_mutexattr_t attr1, attr2;
> >>+  pthread_mutexattr_settype (&attr1, PTHREAD_MUTEX_NORMAL);
> >>+  pthread_mutexattr_settype (&attr2, PTHREAD_MUTEX_NORMAL);
> >>+  pthread_mutex_init (&_malloc_mutex, &attr1);
> >>+  pthread_mutex_init (&_aligned_blocks_mutex, &attr2);
> >>    pthread_atfork (malloc_atfork_handler_prepare,
> >>                   malloc_atfork_handler_parent,
> >>                   malloc_atfork_handler_child);
> >>
> >>
> >>The first hunk avoids the double initialization, but I don't understand why
> >>the second hunk does anything.  Since PTHREAD_MUTEX_NORMAL is now the
> >>default, shouldn't calling pthread_mutex_init with NULL second argument be
> >>equivalent to my calls to pthread_mutexattr_settype?  Does this indicate a
> >>Cygwin bug, or am I misunderstanding something?
> >
> >AFAICS you're missing something.  Your pthread_mutexattr_t attr1, attr2
> >are not initialized.  They contain some random values, thus they are not
> >good objects.  The calls to pthread_mutexattr_settype as well as the
> >calls to pthread_mutex_init will fail with EINVAL, but you won't see it
> >due to missing error handling, and you end up without mutexes at all.
> >If you call pthread_mutexattr_init before calling
> >pthread_mutexattr_settype the situation shoul;d be the same as before.
> 
> Thanks for catching my mistake.  Your earlier suggestion about explicitly
> setting the pthread_mutexes to be ERRORCHECK mutexes seems to fix the
> problem (as long as I remember to call pthread_mutexattr_init).  The revised
> patch is attached.  I went back to using both the static and dynamic
> initializations as in the original code, since you said that's harmless.

I'm glad to read that, but I'm still a little bit concerned.  If your
code works with ERRORCHECK mutexes but hangs with NORMAL mutexes, you
*might* miss an error case.

I'd suggest to tweak the pthread_mutex_lock/unlock calls and log the
threads calling it.  It looks like the same thread calls malloc from
malloc for some reason and it might be interesting to learn how that
happens and if it's really ok in this scenario, because it seems to
be unexpected by the code.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: (call-process ...) hangs in emacs

[Ken Brown]

Hi Corinna,

On 8/5/2014 2:40 PM, Corinna Vinschen wrote:
> Hi Ken,
>
> On Aug  5 13:55, Ken Brown wrote:
>> On 8/5/2014 9:58 AM, Corinna Vinschen wrote:
>>> On Aug  5 08:21, Ken Brown wrote:
>>>> === modified file 'src/gmalloc.c'
>>>> --- src/gmalloc.c       2014-03-04 19:02:49 +0000
>>>> +++ src/gmalloc.c       2014-08-05 01:35:38 +0000
>>>> @@ -490,8 +490,8 @@
>>>>   }
>>>>
>>>>   #ifdef USE_PTHREAD
>>>> -pthread_mutex_t _malloc_mutex = PTHREAD_MUTEX_INITIALIZER;
>>>> -pthread_mutex_t _aligned_blocks_mutex = PTHREAD_MUTEX_INITIALIZER;
>>>> +pthread_mutex_t _malloc_mutex;
>>>> +pthread_mutex_t _aligned_blocks_mutex;
>>>>   int _malloc_thread_enabled_p;
>>>>
>>>>   static void
>>>> @@ -526,8 +526,11 @@
>>>>        initialized mutexes when they are used first.  To avoid such a
>>>>        situation, we initialize mutexes here while their use is
>>>>        disabled in malloc etc.  */
>>>> -  pthread_mutex_init (&_malloc_mutex, NULL);
>>>> -  pthread_mutex_init (&_aligned_blocks_mutex, NULL);
>>>> +  pthread_mutexattr_t attr1, attr2;
>>>> +  pthread_mutexattr_settype (&attr1, PTHREAD_MUTEX_NORMAL);
>>>> +  pthread_mutexattr_settype (&attr2, PTHREAD_MUTEX_NORMAL);
>>>> +  pthread_mutex_init (&_malloc_mutex, &attr1);
>>>> +  pthread_mutex_init (&_aligned_blocks_mutex, &attr2);
>>>>     pthread_atfork (malloc_atfork_handler_prepare,
>>>>                    malloc_atfork_handler_parent,
>>>>                    malloc_atfork_handler_child);
>>>>
>>>>
>>>> The first hunk avoids the double initialization, but I don't understand why
>>>> the second hunk does anything.  Since PTHREAD_MUTEX_NORMAL is now the
>>>> default, shouldn't calling pthread_mutex_init with NULL second argument be
>>>> equivalent to my calls to pthread_mutexattr_settype?  Does this indicate a
>>>> Cygwin bug, or am I misunderstanding something?
>>>
>>> AFAICS you're missing something.  Your pthread_mutexattr_t attr1, attr2
>>> are not initialized.  They contain some random values, thus they are not
>>> good objects.  The calls to pthread_mutexattr_settype as well as the
>>> calls to pthread_mutex_init will fail with EINVAL, but you won't see it
>>> due to missing error handling, and you end up without mutexes at all.
>>> If you call pthread_mutexattr_init before calling
>>> pthread_mutexattr_settype the situation shoul;d be the same as before.
>>
>> Thanks for catching my mistake.  Your earlier suggestion about explicitly
>> setting the pthread_mutexes to be ERRORCHECK mutexes seems to fix the
>> problem (as long as I remember to call pthread_mutexattr_init).  The revised
>> patch is attached.  I went back to using both the static and dynamic
>> initializations as in the original code, since you said that's harmless.
>
> I'm glad to read that, but I'm still a little bit concerned.  If your
> code works with ERRORCHECK mutexes but hangs with NORMAL mutexes, you
> *might* miss an error case.
>
> I'd suggest to tweak the pthread_mutex_lock/unlock calls and log the
> threads calling it.  It looks like the same thread calls malloc from
> malloc for some reason and it might be interesting to learn how that
> happens and if it's really ok in this scenario, because it seems to
> be unexpected by the code.

I think I found the problem with NORMAL mutexes.  emacs calls 
pthread_atfork after initializing the mutexes, and the resulting 
'prepare' handler locks the mutexes.  (The parent and child handlers 
unlock them.)  So when emacs calls fork, the mutexes are locked, and 
shortly thereafter the Cygwin DLL calls calloc, leading to a deadlock. 
Here's a gdb backtrace showing the sequence of calls:

#0  malloc (size=size@entry=40) at gmalloc.c:919
#1  0x0053fc28 in calloc (nmemb=1, size=40) at gmalloc.c:1510
#2  0x61082074 in calloc (nmemb=1, size=40)
     at /usr/src/debug/cygwin-1.7.31-3/winsup/cygwin/malloc_wrapper.cc:100
#3  0x61003177 in operator new (s=s@entry=40)
     at /usr/src/debug/cygwin-1.7.31-3/winsup/cygwin/cxx.cc:23
#4  0x610fc9d3 in pthread_mutex::init (mutex=0x61187d34 <reent_data+852>,
     attr=0x0, initializer=0x12)
     at /usr/src/debug/cygwin-1.7.31-3/winsup/cygwin/thread.cc:3118
#5  0x610fcc13 in pthread_mutex_lock (mutex=0x61187d34 <reent_data+852>)
     at /usr/src/debug/cygwin-1.7.31-3/winsup/cygwin/thread.cc:3170
#6  0x611319d8 in __fp_lock (ptr=0x61187cd0 <reent_data+752>)
     at /usr/src/debug/cygwin-1.7.31-3/newlib/libc/stdio/findfp.c:287
#7  0x61154f75 in _fwalk (ptr=0x28d544,
     function=function@entry=0x611319c0 <__fp_lock>)
     at /usr/src/debug/cygwin-1.7.31-3/newlib/libc/stdio/fwalk.c:50
#8  0x61131dea in __fp_lock_all ()
     at /usr/src/debug/cygwin-1.7.31-3/newlib/libc/stdio/findfp.c:307
#9  0x610fa45e in pthread::atforkprepare ()
     at /usr/src/debug/cygwin-1.7.31-3/winsup/cygwin/thread.cc:2031
#10 0x61076292 in lock_pthread (this=<synthetic pointer>)
     at /usr/src/debug/cygwin-1.7.31-3/winsup/cygwin/sigproc.h:137
#11 hold_everything (x=<synthetic pointer>, this=<synthetic pointer>)
     at /usr/src/debug/cygwin-1.7.31-3/winsup/cygwin/sigproc.h:169
#12 fork () at /usr/src/debug/cygwin-1.7.31-3/winsup/cygwin/fork.cc:582

Is there a better way to deal with this issue than using ERRORCHECK mutexes?

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 3425 bytes --]

Hi Ken,

On Aug  7 07:51, Ken Brown wrote:
> Hi Corinna,
> 
> On 8/5/2014 2:40 PM, Corinna Vinschen wrote:
> >I'm glad to read that, but I'm still a little bit concerned.  If your
> >code works with ERRORCHECK mutexes but hangs with NORMAL mutexes, you
> >*might* miss an error case.
> >
> >I'd suggest to tweak the pthread_mutex_lock/unlock calls and log the
> >threads calling it.  It looks like the same thread calls malloc from
> >malloc for some reason and it might be interesting to learn how that
> >happens and if it's really ok in this scenario, because it seems to
> >be unexpected by the code.
> 
> I think I found the problem with NORMAL mutexes.  emacs calls pthread_atfork
> after initializing the mutexes, and the resulting 'prepare' handler locks
> the mutexes.  (The parent and child handlers unlock them.)  So when emacs
> calls fork, the mutexes are locked, and shortly thereafter the Cygwin DLL
> calls calloc, leading to a deadlock. Here's a gdb backtrace showing the
> sequence of calls:

First question:  Why does emacs use its own malloc on Cygwin rather 
than the system-provided one?  Is that really necessary?

> #0  malloc (size=size@entry=40) at gmalloc.c:919
> #1  0x0053fc28 in calloc (nmemb=1, size=40) at gmalloc.c:1510
> #2  0x61082074 in calloc (nmemb=1, size=40)
>     at /usr/src/debug/cygwin-1.7.31-3/winsup/cygwin/malloc_wrapper.cc:100
> #3  0x61003177 in operator new (s=s@entry=40)
>     at /usr/src/debug/cygwin-1.7.31-3/winsup/cygwin/cxx.cc:23
> #4  0x610fc9d3 in pthread_mutex::init (mutex=0x61187d34 <reent_data+852>,
>     attr=0x0, initializer=0x12)
>     at /usr/src/debug/cygwin-1.7.31-3/winsup/cygwin/thread.cc:3118
> #5  0x610fcc13 in pthread_mutex_lock (mutex=0x61187d34 <reent_data+852>)
>     at /usr/src/debug/cygwin-1.7.31-3/winsup/cygwin/thread.cc:3170
> #6  0x611319d8 in __fp_lock (ptr=0x61187cd0 <reent_data+752>)

Right, __fp_lock needs a pthread lock and since this lock hasn't been
used yet, it has to create it.  The pthread_mutex creation calls the
new operator which in turn calls calloc.

>     at /usr/src/debug/cygwin-1.7.31-3/newlib/libc/stdio/findfp.c:287
> #7  0x61154f75 in _fwalk (ptr=0x28d544,
>     function=function@entry=0x611319c0 <__fp_lock>)
>     at /usr/src/debug/cygwin-1.7.31-3/newlib/libc/stdio/fwalk.c:50
> #8  0x61131dea in __fp_lock_all ()
>     at /usr/src/debug/cygwin-1.7.31-3/newlib/libc/stdio/findfp.c:307
> #9  0x610fa45e in pthread::atforkprepare ()
>     at /usr/src/debug/cygwin-1.7.31-3/winsup/cygwin/thread.cc:2031
> #10 0x61076292 in lock_pthread (this=<synthetic pointer>)
>     at /usr/src/debug/cygwin-1.7.31-3/winsup/cygwin/sigproc.h:137
> #11 hold_everything (x=<synthetic pointer>, this=<synthetic pointer>)
>     at /usr/src/debug/cygwin-1.7.31-3/winsup/cygwin/sigproc.h:169
> #12 fork () at /usr/src/debug/cygwin-1.7.31-3/winsup/cygwin/fork.cc:582
> 
> Is there a better way to deal with this issue than using ERRORCHECK mutexes?

Did you check if you get an error from pthread_mutex_lock on the
second invocation of malloc?  Is it EDEADLK?  If so, you can
ignore the error, but if you want to go ahead without adding lots
of error checking you might be better off using a RECURSIVE mutex.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: (call-process ...) hangs in emacs

[Ken Brown]

On 8/7/2014 8:51 AM, Corinna Vinschen wrote:
> Hi Ken,
>
> On Aug  7 07:51, Ken Brown wrote:
>> Hi Corinna,
>>
>> On 8/5/2014 2:40 PM, Corinna Vinschen wrote:
>>> I'm glad to read that, but I'm still a little bit concerned.  If your
>>> code works with ERRORCHECK mutexes but hangs with NORMAL mutexes, you
>>> *might* miss an error case.
>>>
>>> I'd suggest to tweak the pthread_mutex_lock/unlock calls and log the
>>> threads calling it.  It looks like the same thread calls malloc from
>>> malloc for some reason and it might be interesting to learn how that
>>> happens and if it's really ok in this scenario, because it seems to
>>> be unexpected by the code.
>>
>> I think I found the problem with NORMAL mutexes.  emacs calls pthread_atfork
>> after initializing the mutexes, and the resulting 'prepare' handler locks
>> the mutexes.  (The parent and child handlers unlock them.)  So when emacs
>> calls fork, the mutexes are locked, and shortly thereafter the Cygwin DLL
>> calls calloc, leading to a deadlock. Here's a gdb backtrace showing the
>> sequence of calls:
>
> First question:  Why does emacs use its own malloc on Cygwin rather
> than the system-provided one?  Is that really necessary?

Cygwin's malloc lacks a few features that emacs requires because of the 
unusual way emacs is built.  The most important such features (or maybe 
even the only ones) are malloc_set_state and malloc_get_state.
>
>> #0  malloc (size=size@entry=40) at gmalloc.c:919
>> #1  0x0053fc28 in calloc (nmemb=1, size=40) at gmalloc.c:1510
>> #2  0x61082074 in calloc (nmemb=1, size=40)
>>      at /usr/src/debug/cygwin-1.7.31-3/winsup/cygwin/malloc_wrapper.cc:100
>> #3  0x61003177 in operator new (s=s@entry=40)
>>      at /usr/src/debug/cygwin-1.7.31-3/winsup/cygwin/cxx.cc:23
>> #4  0x610fc9d3 in pthread_mutex::init (mutex=0x61187d34 <reent_data+852>,
>>      attr=0x0, initializer=0x12)
>>      at /usr/src/debug/cygwin-1.7.31-3/winsup/cygwin/thread.cc:3118
>> #5  0x610fcc13 in pthread_mutex_lock (mutex=0x61187d34 <reent_data+852>)
>>      at /usr/src/debug/cygwin-1.7.31-3/winsup/cygwin/thread.cc:3170
>> #6  0x611319d8 in __fp_lock (ptr=0x61187cd0 <reent_data+752>)
>
> Right, __fp_lock needs a pthread lock and since this lock hasn't been
> used yet, it has to create it.  The pthread_mutex creation calls the
> new operator which in turn calls calloc.
>
>>      at /usr/src/debug/cygwin-1.7.31-3/newlib/libc/stdio/findfp.c:287
>> #7  0x61154f75 in _fwalk (ptr=0x28d544,
>>      function=function@entry=0x611319c0 <__fp_lock>)
>>      at /usr/src/debug/cygwin-1.7.31-3/newlib/libc/stdio/fwalk.c:50
>> #8  0x61131dea in __fp_lock_all ()
>>      at /usr/src/debug/cygwin-1.7.31-3/newlib/libc/stdio/findfp.c:307
>> #9  0x610fa45e in pthread::atforkprepare ()
>>      at /usr/src/debug/cygwin-1.7.31-3/winsup/cygwin/thread.cc:2031
>> #10 0x61076292 in lock_pthread (this=<synthetic pointer>)
>>      at /usr/src/debug/cygwin-1.7.31-3/winsup/cygwin/sigproc.h:137
>> #11 hold_everything (x=<synthetic pointer>, this=<synthetic pointer>)
>>      at /usr/src/debug/cygwin-1.7.31-3/winsup/cygwin/sigproc.h:169
>> #12 fork () at /usr/src/debug/cygwin-1.7.31-3/winsup/cygwin/fork.cc:582
>>
>> Is there a better way to deal with this issue than using ERRORCHECK mutexes?
>
> Did you check if you get an error from pthread_mutex_lock on the
> second invocation of malloc?  Is it EDEADLK?  If so, you can
> ignore the error, but if you want to go ahead without adding lots
> of error checking you might be better off using a RECURSIVE mutex.

I didn't check the error, but it seemed clear from the code that that 
was what was happening.  Yes, using a RECURSIVE mutex sounds like a good 
idea.  Or maybe it would be just as good to remove the call to 
pthread_atfork.  See my reply to Eric later in the thread.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1120 bytes --]

On 08/07/2014 05:51 AM, Ken Brown wrote:
> 
> I think I found the problem with NORMAL mutexes.  emacs calls
> pthread_atfork after initializing the mutexes, and the resulting
> 'prepare' handler locks the mutexes.  (The parent and child handlers
> unlock them.)  So when emacs calls fork, the mutexes are locked, and
> shortly thereafter the Cygwin DLL calls calloc, leading to a deadlock.
> Here's a gdb backtrace showing the sequence of calls:

Arguably, that's an upstream bug in emacs.  POSIX has declared
pthread_atfork to be fundamentally useless; it is broken by design,
because you cannot use it for anything that is not async-signal-safe
without risking deadlock.  And (except for sem_post()), NONE of the
standardized locking functions are async-signal-safe.

http://austingroupbugs.net/view.php?id=858

That said, it would still be nice to support this, since even though the
theory says it is broken, there are still lots of (broken)
programs/libraries still trying to use it.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 539 bytes --]

Re: (call-process ...) hangs in emacs

[Ken Brown]

On 8/7/2014 11:30 AM, Eric Blake wrote:
> On 08/07/2014 05:51 AM, Ken Brown wrote:
>>
>> I think I found the problem with NORMAL mutexes.  emacs calls
>> pthread_atfork after initializing the mutexes, and the resulting
>> 'prepare' handler locks the mutexes.  (The parent and child handlers
>> unlock them.)  So when emacs calls fork, the mutexes are locked, and
>> shortly thereafter the Cygwin DLL calls calloc, leading to a deadlock.
>> Here's a gdb backtrace showing the sequence of calls:
>
> Arguably, that's an upstream bug in emacs.  POSIX has declared
> pthread_atfork to be fundamentally useless; it is broken by design,
> because you cannot use it for anything that is not async-signal-safe
> without risking deadlock.  And (except for sem_post()), NONE of the
> standardized locking functions are async-signal-safe.
>
> http://austingroupbugs.net/view.php?id=858
>
> That said, it would still be nice to support this, since even though the
> theory says it is broken, there are still lots of (broken)
> programs/libraries still trying to use it.

So what do you think emacs should do instead of using pthread_atfork? 
Or is it better to just remove it?  I don't know how likely it is that 
this would cause a problem.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 3329 bytes --]

On 08/07/2014 12:53 PM, Ken Brown wrote:
> On 8/7/2014 11:30 AM, Eric Blake wrote:
>> On 08/07/2014 05:51 AM, Ken Brown wrote:
>>>
>>> I think I found the problem with NORMAL mutexes.  emacs calls
>>> pthread_atfork after initializing the mutexes, and the resulting
>>> 'prepare' handler locks the mutexes.  (The parent and child handlers
>>> unlock them.)  So when emacs calls fork, the mutexes are locked, and
>>> shortly thereafter the Cygwin DLL calls calloc, leading to a deadlock.
>>> Here's a gdb backtrace showing the sequence of calls:
>>
>> Arguably, that's an upstream bug in emacs.  POSIX has declared
>> pthread_atfork to be fundamentally useless; it is broken by design,
>> because you cannot use it for anything that is not async-signal-safe
>> without risking deadlock.  And (except for sem_post()), NONE of the
>> standardized locking functions are async-signal-safe.
>>
>> http://austingroupbugs.net/view.php?id=858
>>
>> That said, it would still be nice to support this, since even though the
>> theory says it is broken, there are still lots of (broken)
>> programs/libraries still trying to use it.
> 
> So what do you think emacs should do instead of using pthread_atfork? Or
> is it better to just remove it?  I don't know how likely it is that this
> would cause a problem.

The POSIX recommendation is that multithreaded apps limit themselves
solely to async-signal-safe functions in the window between fork and
exec (or to use pthread_spawn instead of fork/exec).  I don't know what
emacs is trying to do in that window, but at this point, it's certainly
worth reporting it upstream.  If you need a pointer to the full list of
async-signal-safe functions:

http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_04
and search for "The following table defines a set of functions that
shall be async-signal-safe."

The most common deadlocks when violating async-signal-safety rules look
like this in single-threaded programs:

function calls malloc()
  malloc() grabs a non-recursive mutex
    async signal arrives
      signal handler called
        signal handler calls malloc()
          malloc() can't grab the mutex - deadlock

and this counterpart in multithreaded programs:

thread1 calls malloc()
  malloc() grabs a non-recursive mutex
thread 2 gains control and calls fork()
  because of the fork, thread1 no longer exists to release the lock
  child process calls malloc()
    malloc() tries to grab mutex, but it is locked with no thread to
release it

Switching malloc() to a recursive lock may or may not "solve" the
single-threaded deadlock (in that malloc can now obtain the mutex), but
it is probably NOT what you want to happen (unless malloc is fully
re-entrant, the inner instance will see incomplete data and either be
totally clobbered itself, or else totally clobber the outer instance
when it returns).  So it's GOOD that malloc does NOT use a recursive
mutex by default.

In the multithreaded case, you are flat out hosed. Switching to a
recursive lock does not change the picture - you are still deadlocked
waiting on thread1 to release the lock, but thread1 doesn't exist.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 539 bytes --]

Re: (call-process ...) hangs in emacs

[Ken Brown]

On 8/7/2014 5:42 PM, Eric Blake wrote:
> On 08/07/2014 12:53 PM, Ken Brown wrote:
>> On 8/7/2014 11:30 AM, Eric Blake wrote:
>>> On 08/07/2014 05:51 AM, Ken Brown wrote:
>>>>
>>>> I think I found the problem with NORMAL mutexes.  emacs calls
>>>> pthread_atfork after initializing the mutexes, and the resulting
>>>> 'prepare' handler locks the mutexes.  (The parent and child handlers
>>>> unlock them.)  So when emacs calls fork, the mutexes are locked, and
>>>> shortly thereafter the Cygwin DLL calls calloc, leading to a deadlock.
>>>> Here's a gdb backtrace showing the sequence of calls:
>>>
>>> Arguably, that's an upstream bug in emacs.  POSIX has declared
>>> pthread_atfork to be fundamentally useless; it is broken by design,
>>> because you cannot use it for anything that is not async-signal-safe
>>> without risking deadlock.  And (except for sem_post()), NONE of the
>>> standardized locking functions are async-signal-safe.
>>>
>>> http://austingroupbugs.net/view.php?id=858
>>>
>>> That said, it would still be nice to support this, since even though the
>>> theory says it is broken, there are still lots of (broken)
>>> programs/libraries still trying to use it.
>>
>> So what do you think emacs should do instead of using pthread_atfork? Or
>> is it better to just remove it?  I don't know how likely it is that this
>> would cause a problem.
>
> The POSIX recommendation is that multithreaded apps limit themselves
> solely to async-signal-safe functions in the window between fork and
> exec (or to use pthread_spawn instead of fork/exec).  I don't know what
> emacs is trying to do in that window, but at this point, it's certainly
> worth reporting it upstream.  If you need a pointer to the full list of
> async-signal-safe functions:
>
> http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_04
> and search for "The following table defines a set of functions that
> shall be async-signal-safe."
>
> The most common deadlocks when violating async-signal-safety rules look
> like this in single-threaded programs:
>
> function calls malloc()
>    malloc() grabs a non-recursive mutex
>      async signal arrives
>        signal handler called
>          signal handler calls malloc()
>            malloc() can't grab the mutex - deadlock
>
> and this counterpart in multithreaded programs:
>
> thread1 calls malloc()
>    malloc() grabs a non-recursive mutex
> thread 2 gains control and calls fork()
>    because of the fork, thread1 no longer exists to release the lock
>    child process calls malloc()
>      malloc() tries to grab mutex, but it is locked with no thread to
> release it
>
> Switching malloc() to a recursive lock may or may not "solve" the
> single-threaded deadlock (in that malloc can now obtain the mutex), but
> it is probably NOT what you want to happen (unless malloc is fully
> re-entrant, the inner instance will see incomplete data and either be
> totally clobbered itself, or else totally clobber the outer instance
> when it returns).  So it's GOOD that malloc does NOT use a recursive
> mutex by default.
>
> In the multithreaded case, you are flat out hosed. Switching to a
> recursive lock does not change the picture - you are still deadlocked
> waiting on thread1 to release the lock, but thread1 doesn't exist.

Thanks for the explanations, Eric.  I've filed an emacs bug report:

   http://debbugs.gnu.org/cgi/bugreport.cgi?bug=18222

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Peter Hull]

A bug in Emacs? Gosh I thought it was probably just me doing something silly!

Thanks for your help everyone in tracking this down.

Ken, do you know if it's possible to subscribe to the bug report - I'd
be interested in knowing how it pans out.

Pete

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Ken Brown]

On 8/8/2014 11:39 AM, Peter Hull wrote:
> A bug in Emacs? Gosh I thought it was probably just me doing something silly!
>
> Thanks for your help everyone in tracking this down.
>
> Ken, do you know if it's possible to subscribe to the bug report - I'd
> be interested in knowing how it pans out.

No, I don't think so.  (You can subscribe to the bug mailing list, but 
then you'd get all emacs bug reports.)  But I'll CC you the next time I 
write to the report, and then you'll probably stay in the CC.  And you 
can keep checking back at 
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=18222.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Ken Brown]

On 8/8/2014 9:26 AM, Ken Brown wrote:
> On 8/7/2014 5:42 PM, Eric Blake wrote:
>> On 08/07/2014 12:53 PM, Ken Brown wrote:
>>> On 8/7/2014 11:30 AM, Eric Blake wrote:
>>>> On 08/07/2014 05:51 AM, Ken Brown wrote:
>>>>>
>>>>> I think I found the problem with NORMAL mutexes.  emacs calls
>>>>> pthread_atfork after initializing the mutexes, and the resulting
>>>>> 'prepare' handler locks the mutexes.  (The parent and child handlers
>>>>> unlock them.)  So when emacs calls fork, the mutexes are locked, and
>>>>> shortly thereafter the Cygwin DLL calls calloc, leading to a deadlock.
>>>>> Here's a gdb backtrace showing the sequence of calls:
>>>>
>>>> Arguably, that's an upstream bug in emacs.  POSIX has declared
>>>> pthread_atfork to be fundamentally useless; it is broken by design,
>>>> because you cannot use it for anything that is not async-signal-safe
>>>> without risking deadlock.  And (except for sem_post()), NONE of the
>>>> standardized locking functions are async-signal-safe.
>>>>
>>>> http://austingroupbugs.net/view.php?id=858
>>>>
>>>> That said, it would still be nice to support this, since even though
>>>> the
>>>> theory says it is broken, there are still lots of (broken)
>>>> programs/libraries still trying to use it.
>>>
>>> So what do you think emacs should do instead of using pthread_atfork? Or
>>> is it better to just remove it?  I don't know how likely it is that this
>>> would cause a problem.
>>
>> The POSIX recommendation is that multithreaded apps limit themselves
>> solely to async-signal-safe functions in the window between fork and
>> exec (or to use pthread_spawn instead of fork/exec).  I don't know what
>> emacs is trying to do in that window, but at this point, it's certainly
>> worth reporting it upstream.  If you need a pointer to the full list of
>> async-signal-safe functions:
>>
>> http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_04
>>
>> and search for "The following table defines a set of functions that
>> shall be async-signal-safe."
>>
>> The most common deadlocks when violating async-signal-safety rules look
>> like this in single-threaded programs:
>>
>> function calls malloc()
>>    malloc() grabs a non-recursive mutex
>>      async signal arrives
>>        signal handler called
>>          signal handler calls malloc()
>>            malloc() can't grab the mutex - deadlock
>>
>> and this counterpart in multithreaded programs:
>>
>> thread1 calls malloc()
>>    malloc() grabs a non-recursive mutex
>> thread 2 gains control and calls fork()
>>    because of the fork, thread1 no longer exists to release the lock
>>    child process calls malloc()
>>      malloc() tries to grab mutex, but it is locked with no thread to
>> release it
>>
>> Switching malloc() to a recursive lock may or may not "solve" the
>> single-threaded deadlock (in that malloc can now obtain the mutex), but
>> it is probably NOT what you want to happen (unless malloc is fully
>> re-entrant, the inner instance will see incomplete data and either be
>> totally clobbered itself, or else totally clobber the outer instance
>> when it returns).  So it's GOOD that malloc does NOT use a recursive
>> mutex by default.
>>
>> In the multithreaded case, you are flat out hosed. Switching to a
>> recursive lock does not change the picture - you are still deadlocked
>> waiting on thread1 to release the lock, but thread1 doesn't exist.
>
> Thanks for the explanations, Eric.  I've filed an emacs bug report:
>
>    http://debbugs.gnu.org/cgi/bugreport.cgi?bug=18222

I've just made a new emacs test release that includes a workaround for 
this bug.  I think I see a way to make emacs use Cygwin's malloc; if 
this works, it will provide a better fix for the bug.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Peter Hull]

On Mon, Aug 18, 2014 at 1:28 PM, Ken Brown <kbrown@cornell.edu> wrote:
> I've just made a new emacs test release that includes a workaround for this
> bug.  I think I see a way to make emacs use Cygwin's malloc; if this works,
> it will provide a better fix for the bug.
I'd like to give this a try. I've selected Exp mode in setup.exe but I
don't see the latest version - do I need to do anything else or is it
just a matter of waiting for the mirror to catch up (I am in the UK)?
Pete

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Larry Hall (Cygwin)]

On 08/18/2014 10:58 AM, Peter Hull wrote:
> On Mon, Aug 18, 2014 at 1:28 PM, Ken Brown <kbrown@cornell.edu> wrote:
>> I've just made a new emacs test release that includes a workaround for this
>> bug.  I think I see a way to make emacs use Cygwin's malloc; if this works,
>> it will provide a better fix for the bug.
> I'd like to give this a try. I've selected Exp mode in setup.exe but I
> don't see the latest version - do I need to do anything else or is it
> just a matter of waiting for the mirror to catch up (I am in the UK)?

Probably the latter.  If you don't want to wait, check out some other
mirrors.


-- 
Larry

_____________________________________________________________________

A: Yes.
 > Q: Are you sure?
 >> A: Because it reverses the logical flow of conversation.
 >>> Q: Why is top posting annoying in email?

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Ken Brown]

On 8/18/2014 8:28 AM, Ken Brown wrote:
> On 8/8/2014 9:26 AM, Ken Brown wrote:
>> On 8/7/2014 5:42 PM, Eric Blake wrote:
>>> On 08/07/2014 12:53 PM, Ken Brown wrote:
>>>> On 8/7/2014 11:30 AM, Eric Blake wrote:
>>>>> On 08/07/2014 05:51 AM, Ken Brown wrote:
>>>>>>
>>>>>> I think I found the problem with NORMAL mutexes.  emacs calls
>>>>>> pthread_atfork after initializing the mutexes, and the resulting
>>>>>> 'prepare' handler locks the mutexes.  (The parent and child handlers
>>>>>> unlock them.)  So when emacs calls fork, the mutexes are locked, and
>>>>>> shortly thereafter the Cygwin DLL calls calloc, leading to a
>>>>>> deadlock.
>>>>>> Here's a gdb backtrace showing the sequence of calls:
>>>>>
>>>>> Arguably, that's an upstream bug in emacs.  POSIX has declared
>>>>> pthread_atfork to be fundamentally useless; it is broken by design,
>>>>> because you cannot use it for anything that is not async-signal-safe
>>>>> without risking deadlock.  And (except for sem_post()), NONE of the
>>>>> standardized locking functions are async-signal-safe.
>>>>>
>>>>> http://austingroupbugs.net/view.php?id=858
>>>>>
>>>>> That said, it would still be nice to support this, since even though
>>>>> the
>>>>> theory says it is broken, there are still lots of (broken)
>>>>> programs/libraries still trying to use it.
>>>>
>>>> So what do you think emacs should do instead of using
>>>> pthread_atfork? Or
>>>> is it better to just remove it?  I don't know how likely it is that
>>>> this
>>>> would cause a problem.
>>>
>>> The POSIX recommendation is that multithreaded apps limit themselves
>>> solely to async-signal-safe functions in the window between fork and
>>> exec (or to use pthread_spawn instead of fork/exec).  I don't know what
>>> emacs is trying to do in that window, but at this point, it's certainly
>>> worth reporting it upstream.  If you need a pointer to the full list of
>>> async-signal-safe functions:
>>>
>>> http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_04
>>>
>>>
>>> and search for "The following table defines a set of functions that
>>> shall be async-signal-safe."
>>>
>>> The most common deadlocks when violating async-signal-safety rules look
>>> like this in single-threaded programs:
>>>
>>> function calls malloc()
>>>    malloc() grabs a non-recursive mutex
>>>      async signal arrives
>>>        signal handler called
>>>          signal handler calls malloc()
>>>            malloc() can't grab the mutex - deadlock
>>>
>>> and this counterpart in multithreaded programs:
>>>
>>> thread1 calls malloc()
>>>    malloc() grabs a non-recursive mutex
>>> thread 2 gains control and calls fork()
>>>    because of the fork, thread1 no longer exists to release the lock
>>>    child process calls malloc()
>>>      malloc() tries to grab mutex, but it is locked with no thread to
>>> release it
>>>
>>> Switching malloc() to a recursive lock may or may not "solve" the
>>> single-threaded deadlock (in that malloc can now obtain the mutex), but
>>> it is probably NOT what you want to happen (unless malloc is fully
>>> re-entrant, the inner instance will see incomplete data and either be
>>> totally clobbered itself, or else totally clobber the outer instance
>>> when it returns).  So it's GOOD that malloc does NOT use a recursive
>>> mutex by default.
>>>
>>> In the multithreaded case, you are flat out hosed. Switching to a
>>> recursive lock does not change the picture - you are still deadlocked
>>> waiting on thread1 to release the lock, but thread1 doesn't exist.
>>
>> Thanks for the explanations, Eric.  I've filed an emacs bug report:
>>
>>    http://debbugs.gnu.org/cgi/bugreport.cgi?bug=18222
>
> I've just made a new emacs test release that includes a workaround for
> this bug.  I think I see a way to make emacs use Cygwin's malloc; if
> this works, it will provide a better fix for the bug.

It looks like my idea is going to work, but it needs testing to make 
sure I've implemented it correctly.  If anyone is willing to test it, 
you can download emacs-24.3.93-2 from my personal Cygwin repository:

   http://sanibeltranquility.com/cygwin/

Instructions can be found at that URL.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Peter Hull]

On Mon, Aug 25, 2014 at 8:00 PM, Ken Brown <kbrown@cornell.edu> wrote:
> It looks like my idea is going to work, but it needs testing to make sure
> I've implemented it correctly.  If anyone is willing to test it, you can
> download emacs-24.3.93-2 from my personal Cygwin repository:
I've downloaded it - no problems so far but I'll run with it from now
on and let know you if anything untoward happens.
Pete

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Achim Gratz]

Ken Brown writes:
> It looks like my idea is going to work, but it needs testing to make
> sure I've implemented it correctly.  If anyone is willing to test it,
> you can download emacs-24.3.93-2 from my personal Cygwin repository:
>
>   http://sanibeltranquility.com/cygwin/
>
> Instructions can be found at that URL.

I've switched to this version today.

I've noticed that two bugs are still present at least in the emacs-w32
version:

1) When showing the Windows desktop with Win-D and then restoring it
(including Emacs) with Win-D again, the cursor becomes a hollow
rectangle that doesn't blink.  To get the normal cursor behaviour back
you have to minimize and restore the Emacs window in the "normal" way.

2) Files that have no POSIX permissions (filemode 0000) and where access
is granted via ACL only get always opened as "read-only" and you have to
C-x C-q them before saving.  It appears that this is Cygwin specific
since on Linux the same version copes with that situation correctly
(however, the mask bits in the ACL get displayed in the group portion of
the file mode, which I've never seen happen on Cygwin, so this may be
something that Cygwin needs to do -- maybe that'd even solve the
problems that Perl has in the same situation).


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Ken Brown]

On 8/26/2014 2:55 PM, Achim Gratz wrote:
> Ken Brown writes:
>> It looks like my idea is going to work, but it needs testing to make
>> sure I've implemented it correctly.  If anyone is willing to test it,
>> you can download emacs-24.3.93-2 from my personal Cygwin repository:
>>
>>    http://sanibeltranquility.com/cygwin/
>>
>> Instructions can be found at that URL.
>
> I've switched to this version today.
>
> I've noticed that two bugs are still present at least in the emacs-w32
> version:
>
> 1) When showing the Windows desktop with Win-D and then restoring it
> (including Emacs) with Win-D again, the cursor becomes a hollow
> rectangle that doesn't blink.  To get the normal cursor behaviour back
> you have to minimize and restore the Emacs window in the "normal" way.

This one has nothing to do with emacs.  I see the same thing in mintty, 
with just a shell prompt (bash in my case).

> 2) Files that have no POSIX permissions (filemode 0000) and where access
> is granted via ACL only get always opened as "read-only" and you have to
> C-x C-q them before saving.  It appears that this is Cygwin specific
> since on Linux the same version copes with that situation correctly
> (however, the mask bits in the ACL get displayed in the group portion of
> the file mode, which I've never seen happen on Cygwin, so this may be
> something that Cygwin needs to do -- maybe that'd even solve the
> problems that Perl has in the same situation).

AFAICT, emacs decides whether the file is writable via the system call 
faccessat.  (See the function 'check_writable' in src/fileio.c.)  This 
is not Cygwin specific.  So faccessat must be returning failure in the 
scenario you described.  I don't know if that's a Cygwin bug or not.

BTW, emacs on Cygwin doesn't directly check ACLs, because the relevant 
configure test fails.  That would explain the ACL display you see on 
Linux but not on Cygwin.  But I think this is a separate matter, not 
related to the bug you're reporting.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2941 bytes --]

On Aug 26 18:12, Ken Brown wrote:
> On 8/26/2014 2:55 PM, Achim Gratz wrote:
> >Ken Brown writes:
> >>It looks like my idea is going to work, but it needs testing to make
> >>sure I've implemented it correctly.  If anyone is willing to test it,
> >>you can download emacs-24.3.93-2 from my personal Cygwin repository:
> >>
> >>   http://sanibeltranquility.com/cygwin/
> >>
> >>Instructions can be found at that URL.
> >
> >I've switched to this version today.
> >
> >I've noticed that two bugs are still present at least in the emacs-w32
> >version:
> >
> >1) When showing the Windows desktop with Win-D and then restoring it
> >(including Emacs) with Win-D again, the cursor becomes a hollow
> >rectangle that doesn't blink.  To get the normal cursor behaviour back
> >you have to minimize and restore the Emacs window in the "normal" way.
> 
> This one has nothing to do with emacs.  I see the same thing in mintty, with
> just a shell prompt (bash in my case).
> 
> >2) Files that have no POSIX permissions (filemode 0000) and where access
> >is granted via ACL only get always opened as "read-only" and you have to
> >C-x C-q them before saving.  It appears that this is Cygwin specific
> >since on Linux the same version copes with that situation correctly
> >(however, the mask bits in the ACL get displayed in the group portion of
> >the file mode, which I've never seen happen on Cygwin, so this may be
> >something that Cygwin needs to do -- maybe that'd even solve the
> >problems that Perl has in the same situation).
> 
> AFAICT, emacs decides whether the file is writable via the system call
> faccessat.  (See the function 'check_writable' in src/fileio.c.)  This is
> not Cygwin specific.  So faccessat must be returning failure in the scenario
> you described.  I don't know if that's a Cygwin bug or not.

faccessat/access/eaccess don't try to be intelligent by themselves.
Rather they just call a Windows function if the filesystem is mounted
with "acl" mount flags:

- Fetch file's security descriptor
- Create process impersonation token.
- Call NtAccessCheck
- If NtAccessCheck returns "not allowed", check for backup/restore
  privileges via NtPrivilegeCheck.

In "noacl" mode or on filesystems not supporting ACLs, access uses the
st_mode flags from stat() to figure out the permissions.

The relevant parts of the implementation are the check_file_access and
subsequently called check_access functions in security.cc.

If you see a bug there, please let me know.

> BTW, emacs on Cygwin doesn't directly check ACLs, because the relevant
> configure test fails.

Works for vim.  Does the Emacs configure test only check for POSIX
ACL functions and not for Solaris ACL functions, by any chance?


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: (call-process ...) hangs in emacs

[Ken Brown]

On 8/27/2014 4:42 AM, Corinna Vinschen wrote:
> On Aug 26 18:12, Ken Brown wrote:
>> On 8/26/2014 2:55 PM, Achim Gratz wrote:
>>> 2) Files that have no POSIX permissions (filemode 0000) and where access
>>> is granted via ACL only get always opened as "read-only" and you have to
>>> C-x C-q them before saving.  It appears that this is Cygwin specific
>>> since on Linux the same version copes with that situation correctly
>>> (however, the mask bits in the ACL get displayed in the group portion of
>>> the file mode, which I've never seen happen on Cygwin, so this may be
>>> something that Cygwin needs to do -- maybe that'd even solve the
>>> problems that Perl has in the same situation).
>>
>> AFAICT, emacs decides whether the file is writable via the system call
>> faccessat.  (See the function 'check_writable' in src/fileio.c.)  This is
>> not Cygwin specific.  So faccessat must be returning failure in the scenario
>> you described.  I don't know if that's a Cygwin bug or not.
>
> faccessat/access/eaccess don't try to be intelligent by themselves.
> Rather they just call a Windows function if the filesystem is mounted
> with "acl" mount flags:
>
> - Fetch file's security descriptor
> - Create process impersonation token.
> - Call NtAccessCheck
> - If NtAccessCheck returns "not allowed", check for backup/restore
>    privileges via NtPrivilegeCheck.
>
> In "noacl" mode or on filesystems not supporting ACLs, access uses the
> st_mode flags from stat() to figure out the permissions.
>
> The relevant parts of the implementation are the check_file_access and
> subsequently called check_access functions in security.cc.
>
> If you see a bug there, please let me know.

Achim, could you send me a recipe for reproducing the problem so that I 
can test further?  Please be very detailed; I have no experience with ACLs.

>> BTW, emacs on Cygwin doesn't directly check ACLs, because the relevant
>> configure test fails.
>
> Works for vim.  Does the Emacs configure test only check for POSIX
> ACL functions and not for Solaris ACL functions, by any chance?

I spoke too soon.  It does detect that Cygwin has certain ACL functions. 
  But the feature that Achim was asking about seems to get used only on 
systems that have acl_get_file.  I guess that's a POSIX ACL function.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 3090 bytes --]

On Aug 27 08:52, Ken Brown wrote:
> On 8/27/2014 4:42 AM, Corinna Vinschen wrote:
> >On Aug 26 18:12, Ken Brown wrote:
> >>On 8/26/2014 2:55 PM, Achim Gratz wrote:
> >>>2) Files that have no POSIX permissions (filemode 0000) and where access
> >>>is granted via ACL only get always opened as "read-only" and you have to
> >>>C-x C-q them before saving.  It appears that this is Cygwin specific
> >>>since on Linux the same version copes with that situation correctly
> >>>(however, the mask bits in the ACL get displayed in the group portion of
> >>>the file mode, which I've never seen happen on Cygwin, so this may be
> >>>something that Cygwin needs to do -- maybe that'd even solve the
> >>>problems that Perl has in the same situation).
> >>
> >>AFAICT, emacs decides whether the file is writable via the system call
> >>faccessat.  (See the function 'check_writable' in src/fileio.c.)  This is
> >>not Cygwin specific.  So faccessat must be returning failure in the scenario
> >>you described.  I don't know if that's a Cygwin bug or not.
> >
> >faccessat/access/eaccess don't try to be intelligent by themselves.
> >Rather they just call a Windows function if the filesystem is mounted
> >with "acl" mount flags:
> >
> >- Fetch file's security descriptor
> >- Create process impersonation token.
> >- Call NtAccessCheck
> >- If NtAccessCheck returns "not allowed", check for backup/restore
> >   privileges via NtPrivilegeCheck.
> >
> >In "noacl" mode or on filesystems not supporting ACLs, access uses the
> >st_mode flags from stat() to figure out the permissions.
> >
> >The relevant parts of the implementation are the check_file_access and
> >subsequently called check_access functions in security.cc.
> >
> >If you see a bug there, please let me know.
> 
> Achim, could you send me a recipe for reproducing the problem so that I can
> test further?  Please be very detailed; I have no experience with ACLs.

I'd be interested in a way to reproduce this as well.  On *real* local
or remote NTFS, if possible.

> >>BTW, emacs on Cygwin doesn't directly check ACLs, because the relevant
> >>configure test fails.
> >
> >Works for vim.  Does the Emacs configure test only check for POSIX
> >ACL functions and not for Solaris ACL functions, by any chance?
> 
> I spoke too soon.  It does detect that Cygwin has certain ACL functions.
> But the feature that Achim was asking about seems to get used only on
> systems that have acl_get_file.  I guess that's a POSIX ACL function.

Yes, it is.  It's pretty much the same as the Solaris/Cygwin function

  int acl (const char *path, int cmd, int nentries, aclent_t *aclbufp);

See http://docs.oracle.com/cd/E23823_01/html/816-5167/acl-2.html for
a description.  We're only supporting the aclent_t type (funny, isn't it?)
which is pretty much based on POSIX ACLs and which is defined in
/usr/include/cygwin/acl.h.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: (call-process ...) hangs in emacs

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1433 bytes --]

On 08/27/2014 07:47 AM, Corinna Vinschen wrote:

>>>
>>> Works for vim.  Does the Emacs configure test only check for POSIX
>>> ACL functions and not for Solaris ACL functions, by any chance?
>>
>> I spoke too soon.  It does detect that Cygwin has certain ACL functions.
>> But the feature that Achim was asking about seems to get used only on
>> systems that have acl_get_file.  I guess that's a POSIX ACL function.
> 
> Yes, it is.  It's pretty much the same as the Solaris/Cygwin function
> 
>   int acl (const char *path, int cmd, int nentries, aclent_t *aclbufp);
> 
> See http://docs.oracle.com/cd/E23823_01/html/816-5167/acl-2.html for
> a description.  We're only supporting the aclent_t type (funny, isn't it?)
> which is pretty much based on POSIX ACLs and which is defined in
> /usr/include/cygwin/acl.h.

Hmm; isn't emacs using gnulib's acl wrappers?  (Paul Eggert would know;
he's the developer that's done the most recent work on Gnulib ACL
support as well as on emacs).  In that case, shouldn't the behavior be
the same as for coreutils, which also uses gnulib?  I'm wondering if
there are any bugs in the gnulib acl wrappers which might affect more
than just emacs, and/or where a cygwin patch would make the gnulib
wrappers happier.  Sadly, I'm also not the best expert on ACLs.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 539 bytes --]

Re: (call-process ...) hangs in emacs

[Ken Brown]

On 8/27/2014 10:40 AM, Eric Blake wrote:
> On 08/27/2014 07:47 AM, Corinna Vinschen wrote:
>
>>>>
>>>> Works for vim.  Does the Emacs configure test only check for POSIX
>>>> ACL functions and not for Solaris ACL functions, by any chance?
>>>
>>> I spoke too soon.  It does detect that Cygwin has certain ACL functions.
>>> But the feature that Achim was asking about seems to get used only on
>>> systems that have acl_get_file.  I guess that's a POSIX ACL function.
>>
>> Yes, it is.  It's pretty much the same as the Solaris/Cygwin function
>>
>>    int acl (const char *path, int cmd, int nentries, aclent_t *aclbufp);
>>
>> See http://docs.oracle.com/cd/E23823_01/html/816-5167/acl-2.html for
>> a description.  We're only supporting the aclent_t type (funny, isn't it?)
>> which is pretty much based on POSIX ACLs and which is defined in
>> /usr/include/cygwin/acl.h.
>
> Hmm; isn't emacs using gnulib's acl wrappers?  (Paul Eggert would know;
> he's the developer that's done the most recent work on Gnulib ACL
> support as well as on emacs).  In that case, shouldn't the behavior be
> the same as for coreutils, which also uses gnulib?  I'm wondering if
> there are any bugs in the gnulib acl wrappers which might affect more
> than just emacs, and/or where a cygwin patch would make the gnulib
> wrappers happier.  Sadly, I'm also not the best expert on ACLs.

I'll follow up on the emacs-devel list.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Achim Gratz]

Ken Brown <kbrown <at> cornell.edu> writes:
> Achim, could you send me a recipe for reproducing the problem so that I 
> can test further?  Please be very detailed; I have no experience with ACLs.

Let's get one issue out of the way first that may be a Cygwin bug: on Linux
a file with all access removed via standard POSIX modes and then access
granted via ACL would place the mask bits of the ACL (the maximum permission
that can be granted via ACL, usually rwx) into the group portion of the
POSIX modes (ls --color would even show these in different color if you
didn't happen to notice the "+").  That doesn't happen on Cygwin and it
seems that some software optimizes based on that information to not traverse
the ACL when there's no chance to ever get a permission granted.  This
behaviour is mandated by POSIX IIUC (and it is what Linux is doing) so
unless Cygwin explicitly follows a different ACL model, I think that should
be taken care of before diving into this further.


Regards,
Achim.


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Achim Gratz]

Achim Gratz <Stromeko <at> NexGo.DE> writes:
> Let's get one issue out of the way first that may be a Cygwin bug: on Linux
> a file with all access removed via standard POSIX modes and then access
> granted via ACL would place the mask bits of the ACL (the maximum permission
> that can be granted via ACL, usually rwx) into the group portion of the
> POSIX modes (ls --color would even show these in different color if you
> didn't happen to notice the "+").  That doesn't happen on Cygwin and it
> seems that some software optimizes based on that information to not traverse
> the ACL when there's no chance to ever get a permission granted.  This
> behaviour is mandated by POSIX IIUC (and it is what Linux is doing) so
> unless Cygwin explicitly follows a different ACL model, I think that should
> be taken care of before diving into this further.

As a concrete example, in the following the directory x86 shows up on Cygwin
as follows:

> getfacl x86
# file: x86
# owner: otheruser
# group: Domain Users
user::---
group::---
group:FilerAdmins:rwx
group:ShareOwners:rwx
mask:rwx
other:---
default:user::---
default:group::---
default:group:FilerAdmins:rwx
default:group:ShareOwners:rwx
default:mask:rwx
default:other:---
> ls -ld x86
d---------+ 1 otheruser Domain Users 0 Jun 23 14:09 x86/

Under Linux in the same situation you'd get

> ls -ld x86
d---rwx---+ 1 otheruser Domain Users 0 Jun 23 14:09 x86/

instead (i.e. the mask bits shown in the group portion of the standard mode
flags).  If the file was owned by your uid, then you'd get indeed

> ls -ld x86
d---------+ 1 myself Domain Users 0 Jun 23 14:09 x86/

but you'd also really have no permissions.  On Windows you do have
permission to the file in that situation since the POSIX part of the ACL
(particularly the user::--- part that revokes all access for the file owner)
are faked by Cygwin and not taken into account when the file gets finally
accessed:

> icacls x86
x86 DOM\FilerAdmins:(I)(OI)(IO)(F)
    DOM\FilerAdmins:(I)(CI)(F)
    DOM\ShareOwners:(I)(OI)(IO)(M)
    DOM\ShareOwners:(I)(CI)(M)

If getting at the correct mask is too expensive, simply always faking an
"rwx" mask might actually be better than what we have now, since once the
ACL are fully processed you'll get the correct permissions anyway.


Regards,
Achim.


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 3067 bytes --]

On Aug 28 07:25, Achim Gratz wrote:
> Achim Gratz <Stromeko <at> NexGo.DE> writes:
> > Let's get one issue out of the way first that may be a Cygwin bug: on Linux
> > a file with all access removed via standard POSIX modes and then access
> > granted via ACL would place the mask bits of the ACL (the maximum permission
> > that can be granted via ACL, usually rwx) into the group portion of the
> > POSIX modes (ls --color would even show these in different color if you
> > didn't happen to notice the "+").  That doesn't happen on Cygwin and it
> > seems that some software optimizes based on that information to not traverse
> > the ACL when there's no chance to ever get a permission granted.  This
> > behaviour is mandated by POSIX IIUC (and it is what Linux is doing) so
> > unless Cygwin explicitly follows a different ACL model, I think that should
> > be taken care of before diving into this further.
> 
> As a concrete example, in the following the directory x86 shows up on Cygwin
> as follows:
> 
> > getfacl x86
> # file: x86
> # owner: otheruser
> # group: Domain Users
> user::---
> group::---
> group:FilerAdmins:rwx
> group:ShareOwners:rwx
> mask:rwx
> other:---
> default:user::---
> default:group::---
> default:group:FilerAdmins:rwx
> default:group:ShareOwners:rwx
> default:mask:rwx
> default:other:---
> > ls -ld x86
> d---------+ 1 otheruser Domain Users 0 Jun 23 14:09 x86/
> 
> Under Linux in the same situation you'd get
> 
> > ls -ld x86
> d---rwx---+ 1 otheruser Domain Users 0 Jun 23 14:09 x86/
> 
> instead (i.e. the mask bits shown in the group portion of the standard mode
> flags).  If the file was owned by your uid, then you'd get indeed
> 
> > ls -ld x86
> d---------+ 1 myself Domain Users 0 Jun 23 14:09 x86/
> 
> but you'd also really have no permissions.  On Windows you do have
> permission to the file in that situation since the POSIX part of the ACL
> (particularly the user::--- part that revokes all access for the file owner)
> are faked by Cygwin and not taken into account when the file gets finally
> accessed:
> 
> > icacls x86
> x86 DOM\FilerAdmins:(I)(OI)(IO)(F)
>     DOM\FilerAdmins:(I)(CI)(F)
>     DOM\ShareOwners:(I)(OI)(IO)(M)
>     DOM\ShareOwners:(I)(CI)(M)
> 
> If getting at the correct mask is too expensive, simply always faking an
> "rwx" mask might actually be better than what we have now, since once the
> ACL are fully processed you'll get the correct permissions anyway.

Handling of the CLASS object (aka "mask") has never been fully
implemented, especially because there's no such thing as a CLASS object
in a Windows ACL.

I guess it will always be some fake, but, yes, we can try to change
stat() so that the st_mode group permissions reflect the or'ed bits of
all permissions given to non-primary users and groups.  Same in acl(2).
That might be useful.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: (call-process ...) hangs in emacs

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2878 bytes --]

On Aug 28 11:55, Corinna Vinschen wrote:
> On Aug 28 07:25, Achim Gratz wrote:
> > As a concrete example, in the following the directory x86 shows up on Cygwin
> > as follows:
> > 
> > > getfacl x86
> > # file: x86
> > # owner: otheruser
> > # group: Domain Users
> > user::---
> > group::---
> > group:FilerAdmins:rwx
> > group:ShareOwners:rwx
> > mask:rwx
> > other:---
> > default:user::---
> > default:group::---
> > default:group:FilerAdmins:rwx
> > default:group:ShareOwners:rwx
> > default:mask:rwx
> > default:other:---
> > > ls -ld x86
> > d---------+ 1 otheruser Domain Users 0 Jun 23 14:09 x86/
> > 
> > Under Linux in the same situation you'd get
> > 
> > > ls -ld x86
> > d---rwx---+ 1 otheruser Domain Users 0 Jun 23 14:09 x86/
> > 
> > instead (i.e. the mask bits shown in the group portion of the standard mode
> > flags).  If the file was owned by your uid, then you'd get indeed
> > 
> > > ls -ld x86
> > d---------+ 1 myself Domain Users 0 Jun 23 14:09 x86/
> > 
> > but you'd also really have no permissions.  On Windows you do have
> > permission to the file in that situation since the POSIX part of the ACL
> > (particularly the user::--- part that revokes all access for the file owner)
> > are faked by Cygwin and not taken into account when the file gets finally
> > accessed:
> > 
> > > icacls x86
> > x86 DOM\FilerAdmins:(I)(OI)(IO)(F)
> >     DOM\FilerAdmins:(I)(CI)(F)
> >     DOM\ShareOwners:(I)(OI)(IO)(M)
> >     DOM\ShareOwners:(I)(CI)(M)
> > 
> > If getting at the correct mask is too expensive, simply always faking an
> > "rwx" mask might actually be better than what we have now, since once the
> > ACL are fully processed you'll get the correct permissions anyway.
> 
> Handling of the CLASS object (aka "mask") has never been fully
> implemented, especially because there's no such thing as a CLASS object
> in a Windows ACL.
> 
> I guess it will always be some fake, but, yes, we can try to change
> stat() so that the st_mode group permissions reflect the or'ed bits of
> all permissions given to non-primary users and groups.  Same in acl(2).
> That might be useful.

I implemented this preliminary and uploaded a snapshot to
https://cygwin.com/snapshots/

"Preliminary", because this change introduces an API change:

Since the CLASS_OBJ and DEF_CLASS_OBJ entries only exist if secondary
user and group (default) entries exist, that means the default
permission entry only consists of 3 ACEs.  This in turn means, the
constant MIN_ACL_ENTRIES changed from 4 to 3.

This might negatively affect coreutils, at least `ls', even though in my
local testing it looked all normal.

Please test.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: (call-process ...) hangs in emacs

[Achim Gratz]

Corinna Vinschen <corinna-cygwin <at> cygwin.com> writes:
> I implemented this preliminary and uploaded a snapshot to
> https://cygwin.com/snapshots/

Oh, great!  I'll bump my machine to that snapshot tomorrow.

Since I can now compile my own DLL, would that be a good time to ask what
could be done for that link count problem on NetApp volumes?  I guess the
first step would be to patch out the forced ihash mount option?


Regards,
Achim.


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1004 bytes --]

On Aug 28 15:04, Achim Gratz wrote:
> Corinna Vinschen <corinna-cygwin <at> cygwin.com> writes:
> > I implemented this preliminary and uploaded a snapshot to
> > https://cygwin.com/snapshots/
> 
> Oh, great!  I'll bump my machine to that snapshot tomorrow.
> 
> Since I can now compile my own DLL, would that be a good time to ask what
> could be done for that link count problem on NetApp volumes?  I guess the
> first step would be to patch out the forced ihash mount option?

It's a bug in Netapp which you won't be able to workaround in Cygwin.
Netapp inode numbers are unstable.  This is non-trivial.  Hardlinks
with different inode numbers will be a major PITA.

Unless, of course, this has been fixed.  But then again there's no way
for Cygwin to know whether it's acessing a fixed Netapp or an older
non-fixed one.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: (call-process ...) hangs in emacs

[Achim Gratz]

Corinna Vinschen <corinna-cygwin <at> cygwin.com> writes:
> Since the CLASS_OBJ and DEF_CLASS_OBJ entries only exist if secondary
> user and group (default) entries exist, that means the default
> permission entry only consists of 3 ACEs.  This in turn means, the
> constant MIN_ACL_ENTRIES changed from 4 to 3.
> 
> This might negatively affect coreutils, at least `ls', even though in my
> local testing it looked all normal.
> 
> Please test.

This fixes the "read-only" problem in Emacs (so that hunch was correct). 
Perl still doesn't play, but I think the 5.18 version should get it correct.
 Will need to switch a test installation over for that, though.

Thanks!


Regards,
Achim.


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Achim Gratz]

Achim Gratz <Stromeko <at> NexGo.DE> writes:
> > Please test.
> 
> This fixes the "read-only" problem in Emacs (so that hunch was correct). 
> Perl still doesn't play, but I think the 5.18 version should get it correct.
> Will need to switch a test installation over for that, though.

With that snapshot in place, ssh suddenly recognized that my private key
file was more readable than it liked it to be, so it looks that it's using
the same general strategy of dealing with ACL as Emacs.  I'm starting to
like this patch very much... :-)

Regards,
Achim.


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1650 bytes --]

On Aug 29 09:58, Achim Gratz wrote:
> Achim Gratz <Stromeko <at> NexGo.DE> writes:
> > > Please test.
> > 
> > This fixes the "read-only" problem in Emacs (so that hunch was correct). 
> > Perl still doesn't play, but I think the 5.18 version should get it correct.
> > Will need to switch a test installation over for that, though.
> 
> With that snapshot in place, ssh suddenly recognized that my private key
> file was more readable than it liked it to be, so it looks that it's using
> the same general strategy of dealing with ACL as Emacs.

...which means, they don't deal with ACLs at all.  They only see what's
given in the st_mode permission bits.  With this change, the group
permission bits now show that *somebody* has certain permissions on the
file, thus the group permissions indicate a too open access for ssh, if
somebody except you have write access to the file.

Downside: If you use inherited Windows permissions, you'll often have
the case that Administrators and/or SYSTEM have full access to your
files.  This in turn shows up as rwx group permissions now.  If you
can't change the permissions (company requirements, etc) the ssh key
file permission test will get annoying.

So it's probably a very nice change (thanks a lot for bringing this up!),
but it will probably have some negative side-effects for existing
installations.

> I'm starting to
> like this patch very much... :-)

Despite of what I'm outlining above, me too :)


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: (call-process ...) hangs in emacs

[Ken Brown]

On 8/29/2014 7:09 AM, Corinna Vinschen wrote:
> On Aug 29 09:58, Achim Gratz wrote:
>> Achim Gratz <Stromeko <at> NexGo.DE> writes:
>>>> Please test.
>>>
>>> This fixes the "read-only" problem in Emacs (so that hunch was correct).
>>> Perl still doesn't play, but I think the 5.18 version should get it correct.
>>> Will need to switch a test installation over for that, though.
>>
>> With that snapshot in place, ssh suddenly recognized that my private key
>> file was more readable than it liked it to be, so it looks that it's using
>> the same general strategy of dealing with ACL as Emacs.
>
> ...which means, they don't deal with ACLs at all.  They only see what's
> given in the st_mode permission bits.  With this change, the group
> permission bits now show that *somebody* has certain permissions on the
> file, thus the group permissions indicate a too open access for ssh, if
> somebody except you have write access to the file.
>
> Downside: If you use inherited Windows permissions, you'll often have
> the case that Administrators and/or SYSTEM have full access to your
> files.  This in turn shows up as rwx group permissions now.  If you
> can't change the permissions (company requirements, etc) the ssh key
> file permission test will get annoying.
>
> So it's probably a very nice change (thanks a lot for bringing this up!),
> but it will probably have some negative side-effects for existing
> installations.
>
>> I'm starting to
>> like this patch very much... :-)
>
> Despite of what I'm outlining above, me too :)

With the latest snapshot I can't start the sshd service.  The 
Application Log just says, "`sshd' service stopped, exit status:255". 
The problem doesn't occur with the 2014-08-27 snapshot.  I guess this 
has something to do with the new permissions on various files, but I'm 
not sure which ones.

Ken


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Achim Gratz]

Ken Brown writes:
> With the latest snapshot I can't start the sshd service.  The
> Application Log just says, "`sshd' service stopped, exit
> status:255". The problem doesn't occur with the 2014-08-27 snapshot.
> I guess this has something to do with the new permissions on various
> files, but I'm not sure which ones.

Off the top of my head for the standard installation:

/etc/ssh*
/var/empty
/var/log/sshd

When you try to debug the sshd, IIR these are the files that must be
chown'ed to the admin user that runs sshd from the terminal.  Running in
debug mode (either from the terminal or via sshd_config) should produce
messages which file or directory sshd is choking on.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf rackAttack:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Ken Brown]

On 8/29/2014 3:23 PM, Achim Gratz wrote:
> Ken Brown writes:
>> With the latest snapshot I can't start the sshd service.  The
>> Application Log just says, "`sshd' service stopped, exit
>> status:255". The problem doesn't occur with the 2014-08-27 snapshot.
>> I guess this has something to do with the new permissions on various
>> files, but I'm not sure which ones.
>
> Off the top of my head for the standard installation:
>
> /etc/ssh*
> /var/empty
> /var/log/sshd
>
> When you try to debug the sshd, IIR these are the files that must be
> chown'ed to the admin user that runs sshd from the terminal.  Running in
> debug mode (either from the terminal or via sshd_config) should produce
> messages which file or directory sshd is choking on.

I just checked /var/log/sshd.log.  (I hadn't thought to do that before.) 
  The last message in it is, "/var/empty must be owned by root and not 
group or world-writable."  So the problem seems to be that /var/empty 
appears to sshd to be group writable under the latest snapshot.  This is 
the "downside" that Corinna mentioned.  What needs to be done to 
/var/empty to fix this?

Ken


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Achim Gratz]

Ken Brown writes:
> I just checked /var/log/sshd.log.  (I hadn't thought to do that
> before.) The last message in it is, "/var/empty must be owned by root
> and not group or world-writable."  So the problem seems to be that
> /var/empty appears to sshd to be group writable under the latest
> snapshot.  This is the "downside" that Corinna mentioned.  What needs
> to be done to /var/empty to fix this?

You need to remove all ACL from the directory, either with setfacl or
(from cmd) icacls or even the security tab in Explorer.  Most likely
these are inherited from the parent directory of the Cygwin
installation.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf rackAttack:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Ken Brown]

On 8/29/2014 4:00 PM, Achim Gratz wrote:
> Ken Brown writes:
>> I just checked /var/log/sshd.log.  (I hadn't thought to do that
>> before.) The last message in it is, "/var/empty must be owned by root
>> and not group or world-writable."  So the problem seems to be that
>> /var/empty appears to sshd to be group writable under the latest
>> snapshot.  This is the "downside" that Corinna mentioned.  What needs
>> to be done to /var/empty to fix this?
> 
> You need to remove all ACL from the directory, either with setfacl or
> (from cmd) icacls or even the security tab in Explorer.  Most likely
> these are inherited from the parent directory of the Cygwin
> installation.

The ACLs aren't inherited.  They're explicitly set by ssh-host-config:

if ! /usr/bin/setfacl -m u:system:rwx "${LOCALSTATEDIR}/empty" >/dev/null 2>&1
then
  csih_warning "Can't set extended permissions on ${LOCALSTATEDIR}/empty!"
  let ++warning_cnt
fi

This must be done for a reason, but I don't know what it is.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Andrey Repin]

Greetings, Ken Brown!

>>> With the latest snapshot I can't start the sshd service.  The
>>> Application Log just says, "`sshd' service stopped, exit
>>> status:255". The problem doesn't occur with the 2014-08-27 snapshot.
>>> I guess this has something to do with the new permissions on various
>>> files, but I'm not sure which ones.
>>
>> Off the top of my head for the standard installation:
>>
>> /etc/ssh*
>> /var/empty
>> /var/log/sshd
>>
>> When you try to debug the sshd, IIR these are the files that must be
>> chown'ed to the admin user that runs sshd from the terminal.  Running in
>> debug mode (either from the terminal or via sshd_config) should produce
>> messages which file or directory sshd is choking on.

> I just checked /var/log/sshd.log.  (I hadn't thought to do that before.) 
>   The last message in it is, "/var/empty must be owned by root and not 
> group or world-writable."  So the problem seems to be that /var/empty 
> appears to sshd to be group writable under the latest snapshot.  This is 
> the "downside" that Corinna mentioned.  What needs to be done to 
> /var/empty to fix this?

I think, setting ACL that will directly translate to -rwx------ without any
"+" should help.

I'm in the middle of transfer to Win7/64, can't test anything right now.


--
WBR,
Andrey Repin (anrdaemon@yandex.ru) 29.08.2014, <23:56>

Sorry for my terrible english...


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2114 bytes --]

On Aug 29 15:36, Ken Brown wrote:
> On 8/29/2014 3:23 PM, Achim Gratz wrote:
> >Ken Brown writes:
> >>With the latest snapshot I can't start the sshd service.  The
> >>Application Log just says, "`sshd' service stopped, exit
> >>status:255". The problem doesn't occur with the 2014-08-27 snapshot.
> >>I guess this has something to do with the new permissions on various
> >>files, but I'm not sure which ones.
> >
> >Off the top of my head for the standard installation:
> >
> >/etc/ssh*
> >/var/empty
> >/var/log/sshd
> >
> >When you try to debug the sshd, IIR these are the files that must be
> >chown'ed to the admin user that runs sshd from the terminal.  Running in
> >debug mode (either from the terminal or via sshd_config) should produce
> >messages which file or directory sshd is choking on.
> 
> I just checked /var/log/sshd.log.  (I hadn't thought to do that before.)
> The last message in it is, "/var/empty must be owned by root and not group
> or world-writable."  So the problem seems to be that /var/empty appears to
> sshd to be group writable under the latest snapshot.  This is the "downside"
> that Corinna mentioned.  What needs to be done to /var/empty to fix this?

What needs to be done is to fix the ssh-host-config script.  It adds an
ACE for SYSTEM on /var/empty, /etc, and /var/log for no apparent reason.

I just sent a patch upstream which removes the code trying to generate
/etc and /var/log entirely (done by setup.exe) and which drops adding
a SYSTEM ACE to /var/empty.

A temporary workaround is either to remove the SYSTEM ACE:

  $ setfacl -d g:18: /var/empty

or to change /etc/sshd_config not to use privilege separation:

  UsePrivilegeSeparation no

However, this is obviously a problem for all existing installations.
OpenSSH 6.7p1 will be released pretty soon.  I will add a postinstall
script which removes the SYSTEM ACE from /var/empty at installation
time.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: (call-process ...) hangs in emacs

[Andrey Repin]

Greetings, Corinna Vinschen!

> What needs to be done is to fix the ssh-host-config script.  It adds an
> ACE for SYSTEM on /var/empty, /etc, and /var/log for no apparent reason.

If it's not apparent: you can actually prevent system from accessing the file
by removing access from SYSTEM user to the file.
Windows ACL's are THAT nasty.

[C:\WINDOWS\system32]$ cacls.exe cmd.exe
C:\WINDOWS\system32\cmd.exe DAEMON2\AnrDaemon:R
                            NT AUTHORITY\SYSTEM:R
                            BUILTIN\Administrators:R
                            BUILTIN\Advanced users:R
                            BUILTIN\Users:R

[C:\WINDOWS\system32]$

Ergo, Windows now unable to delete/overwrite cmd.exe... (It's not actually a
cmd.exe - it's a renamed 4nt.exe. Windows SFC always trying to "repair" it.
And always fail. So do windowsupdate.)


--
WBR,
Andrey Repin (anrdaemon@yandex.ru) 30.08.2014, <03:25>

Sorry for my terrible english...


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 765 bytes --]

On Aug 30 03:32, Andrey Repin wrote:
> Greetings, Corinna Vinschen!
> 
> > What needs to be done is to fix the ssh-host-config script.  It adds an
> > ACE for SYSTEM on /var/empty, /etc, and /var/log for no apparent reason.
> 
> If it's not apparent: you can actually prevent system from accessing the file
> by removing access from SYSTEM user to the file.

No, you can't.  You only can for accounts not having and applications
not enabling backup and restore rights.  But, anyway, this has nothing
to do with extra permissions on /var/empty which the SYSTEM account
really doesn't need.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: (call-process ...) hangs in emacs

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1207 bytes --]

On Aug 29 09:58, Achim Gratz wrote:
> Achim Gratz <Stromeko <at> NexGo.DE> writes:
> > > Please test.
> > 
> > This fixes the "read-only" problem in Emacs (so that hunch was correct). 
> > Perl still doesn't play, but I think the 5.18 version should get it correct.
> > Will need to switch a test installation over for that, though.
> 
> With that snapshot in place, ssh suddenly recognized that my private key
> file was more readable than it liked it to be, so it looks that it's using
> the same general strategy of dealing with ACL as Emacs.  I'm starting to
> like this patch very much... :-)

Over the weekend it occured to me that the acl(2) function created ACLs
which not aligned well with the ACLs created by open(2),chmod(2), etc.
Yesterday I fixed the acl(2) function to create ACLs the same way as 
the other functions, especially in terms of the owner and group entries
and the SE_DACL_PROTECTED flag.  The changes are in the latest snapshot
from https://cygwin.com/snapshots/  Please give them a try.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: (call-process ...) hangs in emacs

[Achim Gratz]

Corinna Vinschen writes:
> Over the weekend it occured to me that the acl(2) function created ACLs
> which not aligned well with the ACLs created by open(2),chmod(2), etc.
> Yesterday I fixed the acl(2) function to create ACLs the same way as 
> the other functions, especially in terms of the owner and group entries
> and the SE_DACL_PROTECTED flag.  The changes are in the latest snapshot
> from https://cygwin.com/snapshots/  Please give them a try.

Installed, everything looks fine so far.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf rackAttack:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 939 bytes --]

On Sep  1 19:38, Achim Gratz wrote:
> Corinna Vinschen writes:
> > Over the weekend it occured to me that the acl(2) function created ACLs
> > which not aligned well with the ACLs created by open(2),chmod(2), etc.
> > Yesterday I fixed the acl(2) function to create ACLs the same way as 
> > the other functions, especially in terms of the owner and group entries
> > and the SE_DACL_PROTECTED flag.  The changes are in the latest snapshot
> > from https://cygwin.com/snapshots/  Please give them a try.
> 
> Installed, everything looks fine so far.

Thanks for testing!  Maybe you'd like to have a view into the simple
as well as the more complex ACLs created by Cygwin?  Icacls output
is moderately easy to read.  If you have questions or concerns...


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: (call-process ...) hangs in emacs

[Achim Gratz]

Corinna Vinschen writes:
>> Installed, everything looks fine so far.
>
> Thanks for testing!  Maybe you'd like to have a view into the simple
> as well as the more complex ACLs created by Cygwin?  Icacls output
> is moderately easy to read.  If you have questions or concerns...

I've used icacls in the past.  Is there anything that you want me to
check specifically?


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf microQ V2.22R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1143 bytes --]

On Sep  2 19:29, Achim Gratz wrote:
> Corinna Vinschen writes:
> >> Installed, everything looks fine so far.
> >
> > Thanks for testing!  Maybe you'd like to have a view into the simple
> > as well as the more complex ACLs created by Cygwin?  Icacls output
> > is moderately easy to read.  If you have questions or concerns...
> 
> I've used icacls in the past.  Is there anything that you want me to
> check specifically?

More or less, just compare the ACLs and see if you find strange
differences.  This only works for the ACLs created or modified with
`setfacl' and the snapshot DLL.  The ACLs created or modified via
setfacl with the older DLLs always were different and, I have to admit,
kind of broke the default POSIX permissions created via open() or
chmod().  The idea of my change was to make them always in an identical
fashion.  The order may only vary in secondary permissions, but never
in the standard permissions, which also always come first.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: (call-process ...) hangs in emacs

[Achim Gratz]

Corinna Vinschen writes:
> More or less, just compare the ACLs and see if you find strange
> differences.  This only works for the ACLs created or modified with
> `setfacl' and the snapshot DLL.

I see, I'll have to make extra tests for this.  Usually I just have to
live with some inherited ACL that I can't change at all.

> The ACLs created or modified via
> setfacl with the older DLLs always were different and, I have to admit,
> kind of broke the default POSIX permissions created via open() or
> chmod().  The idea of my change was to make them always in an identical
> fashion.  The order may only vary in secondary permissions, but never
> in the standard permissions, which also always come first.

One thing I've noticed, but can't really say if it's related to the
change, is that setfacl quite often claims an "illegal ACL" when trying
to remove for instance the SYSTEM read permission.  Removing the group
owner ACL instead did the right thing in at least one instance.  But
I've mostly been removing all ACL from the whole tree via the explorer
security tab (for ~/.ssh/ and similar stuff).


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1787 bytes --]

On Sep  2 21:42, Achim Gratz wrote:
> Corinna Vinschen writes:
> > More or less, just compare the ACLs and see if you find strange
> > differences.  This only works for the ACLs created or modified with
> > `setfacl' and the snapshot DLL.
> 
> I see, I'll have to make extra tests for this.  Usually I just have to
> live with some inherited ACL that I can't change at all.
> 
> > The ACLs created or modified via
> > setfacl with the older DLLs always were different and, I have to admit,
> > kind of broke the default POSIX permissions created via open() or
> > chmod().  The idea of my change was to make them always in an identical
> > fashion.  The order may only vary in secondary permissions, but never
> > in the standard permissions, which also always come first.
> 
> One thing I've noticed, but can't really say if it's related to the
> change, is that setfacl quite often claims an "illegal ACL" when trying
> to remove for instance the SYSTEM read permission.

  $ setfacl -d g:system: filename

Note the trailing colon.  

> Removing the group
> owner ACL instead did the right thing in at least one instance.

??? It shouldn't.  Removing the standard ACL entries for the owner,
owner group, and other is not allowed:

  $ setfacl -d g:: filename
  setfacl: No error

The "No error" is a bug, related to the fact that the aclsort() function
doesn't set errno if aclcheck() failed.  I just fixed that in CVS.

> I've mostly been removing all ACL from the whole tree via the explorer
> security tab (for ~/.ssh/ and similar stuff).

*All* ACL???  That sounds wrong to me.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: (call-process ...) hangs in emacs

[Achim Gratz]

Corinna Vinschen writes:
>   $ setfacl -d g:system: filename
>
> Note the trailing colon.  

That's not what the man page specifies, however.  I'll keep it in mind.

>> Removing the group
>> owner ACL instead did the right thing in at least one instance.
>
> ??? It shouldn't.  Removing the standard ACL entries for the owner,
> owner group, and other is not allowed:
>
>   $ setfacl -d g:: filename
>   setfacl: No error
>
> The "No error" is a bug, related to the fact that the aclsort() function
> doesn't set errno if aclcheck() failed.  I just fixed that in CVS.

I'll try that again to be sure.

>> I've mostly been removing all ACL from the whole tree via the explorer
>> security tab (for ~/.ssh/ and similar stuff).
>
> *All* ACL???  That sounds wrong to me.

Take ownership (if not already owner), give full acces and nuke
everything else (general read access for user and change permission for
admin groups of various sorts mostly).  Under Cygwin this then leaves a
clean mode (600 or 700, depending) with no "+" ACL.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Q+, Q and microQ:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1183 bytes --]

On Sep  2 22:23, Achim Gratz wrote:
> Corinna Vinschen writes:
> >   $ setfacl -d g:system: filename
> >
> > Note the trailing colon.  
> 
> That's not what the man page specifies, however.  I'll keep it in mind.

I patched setfacl to not require trailing colons anymore.  This also
fixes a bug in terms of the allowed acl entries when deleting.

I now also fixed setfacl to add missing acl entries when modifying an
acl, same way as the Linux setfacl handles this.

And, this is important, given that setfacl now always creates complete
acls when modifying an acl, I could finally fix the aclcheck(2) function
in the Cygwin DLL to more thorougly test the incoming acl for all
required entries.

That means, when using the new Cygwin DLL, you also have to use the
accompanying setfacl(1).  With any older setfacl you'll suffer "setfacl:
Invalid argument" error messages.

I just created a new snapshot on https://cygwin.com/snapshots/
containing these patches.  Please give them a try.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: (call-process ...) hangs in emacs

[Achim Gratz]

Corinna Vinschen writes:
> I patched setfacl to not require trailing colons anymore.  This also
> fixes a bug in terms of the allowed acl entries when deleting.

Great, thanks!

[…]
> I just created a new snapshot on https://cygwin.com/snapshots/
> containing these patches.  Please give them a try.

I'll do.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1518 bytes --]

On 08/27/2014 09:15 AM, Achim Gratz wrote:
> Ken Brown <kbrown <at> cornell.edu> writes:
>> Achim, could you send me a recipe for reproducing the problem so that I 
>> can test further?  Please be very detailed; I have no experience with ACLs.
> 
> Let's get one issue out of the way first that may be a Cygwin bug: on Linux
> a file with all access removed via standard POSIX modes and then access
> granted via ACL would place the mask bits of the ACL (the maximum permission
> that can be granted via ACL, usually rwx) into the group portion of the
> POSIX modes (ls --color would even show these in different color if you
> didn't happen to notice the "+").  That doesn't happen on Cygwin and it
> seems that some software optimizes based on that information to not traverse
> the ACL when there's no chance to ever get a permission granted.  This
> behaviour is mandated by POSIX IIUC (and it is what Linux is doing) so
> unless Cygwin explicitly follows a different ACL model, I think that should
> be taken care of before diving into this further.

Sadly, POSIX doesn't mandate ACLs (in part because there are so many
differing implementations that it was impossible to standardize a common
ground).  So code using ACLs across multiple platforms has to deal with
quirks, and although gnulib attempts to do so, it may be a shortcoming
in the gnulib wrappers that emacs is using.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 539 bytes --]

Re: (call-process ...) hangs in emacs

[Andrey Repin]

Greetings, Corinna Vinschen!

> faccessat/access/eaccess don't try to be intelligent by themselves.
> Rather they just call a Windows function if the filesystem is mounted
> with "acl" mount flags:

> - Fetch file's security descriptor
> - Create process impersonation token.
> - Call NtAccessCheck
> - If NtAccessCheck returns "not allowed", check for backup/restore
>   privileges via NtPrivilegeCheck.

> In "noacl" mode or on filesystems not supporting ACLs, access uses the
> st_mode flags from stat() to figure out the permissions.

I'm not very much into Cygwin internals, so beg pardon if I got something
wrong here... But reading this makes my internal sanity checker go into red
alarm state.

Here's why:

When Cygwin mount a filesystem with 'acl' flag set, it mangles current ACL's
set on the files to produce something that can be understood as basic POSIX
'ugly'...erm, 'ugo' permissions. Behavior least desirable in many cases.
You say, it will then use native functions to determine access rights... No
wonder they will work, since you already mangled them to suit your needs.

When Cygwin mount a filesystem with 'noacl' flag, thus let OS use true ACL's
(a feature Windows implemented surprisingly fast, while *NIX was only
proposing it... for far too long without any result in sight), it is then
followed by some magic and guesswork on Cygwin's end to find out access
rights.

If you ask me, something isn't quite right here. Or something is missing.

> The relevant parts of the implementation are the check_file_access and
> subsequently called check_access functions in security.cc.

> If you see a bug there, please let me know.

>> BTW, emacs on Cygwin doesn't directly check ACLs, because the relevant
>> configure test fails.

> Works for vim.  Does the Emacs configure test only check for POSIX
> ACL functions and not for Solaris ACL functions, by any chance?


--
WBR,
Andrey Repin (anrdaemon@yandex.ru) 28.08.2014, <00:48>

Sorry for my terrible english...


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2123 bytes --]

On Aug 28 01:02, Andrey Repin wrote:
> Greetings, Corinna Vinschen!
> 
> > faccessat/access/eaccess don't try to be intelligent by themselves.
> > Rather they just call a Windows function if the filesystem is mounted
> > with "acl" mount flags:
> 
> > - Fetch file's security descriptor
> > - Create process impersonation token.
> > - Call NtAccessCheck
> > - If NtAccessCheck returns "not allowed", check for backup/restore
> >   privileges via NtPrivilegeCheck.
> 
> > In "noacl" mode or on filesystems not supporting ACLs, access uses the
> > st_mode flags from stat() to figure out the permissions.
> 
> I'm not very much into Cygwin internals, so beg pardon if I got something
> wrong here... But reading this makes my internal sanity checker go into red
> alarm state.
> 
> Here's why:
> 
> When Cygwin mount a filesystem with 'acl' flag set, it mangles current ACL's
> set on the files to produce something that can be understood as basic POSIX
> 'ugly'...erm, 'ugo' permissions. Behavior least desirable in many cases.
> You say, it will then use native functions to determine access rights... No
> wonder they will work, since you already mangled them to suit your needs.
> 
> When Cygwin mount a filesystem with 'noacl' flag, thus let OS use true ACL's
> (a feature Windows implemented surprisingly fast, while *NIX was only
> proposing it... for far too long without any result in sight), it is then
> followed by some magic and guesswork on Cygwin's end to find out access
> rights.
> 
> If you ask me, something isn't quite right here. Or something is missing.

It's what "acl" means on Cygwin.  "acl" means that Windowsd ACLs are used
and permissions are handled and converted to and from POSIX permissions.
"noacl" means, Cygwin ignores all ACLs and fakes ownership and POSIX
permissions only based only on filetype and DOS R/O attribute, as it has
to on filesystems not supporting ACLs, like FAT/FAT32.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: (call-process ...) hangs in emacs

[Andrey Repin]

Greetings, Corinna Vinschen!

>> > faccessat/access/eaccess don't try to be intelligent by themselves.
>> > Rather they just call a Windows function if the filesystem is mounted
>> > with "acl" mount flags:
>> 
>> > - Fetch file's security descriptor
>> > - Create process impersonation token.
>> > - Call NtAccessCheck
>> > - If NtAccessCheck returns "not allowed", check for backup/restore
>> >   privileges via NtPrivilegeCheck.
>> 
>> > In "noacl" mode or on filesystems not supporting ACLs, access uses the
>> > st_mode flags from stat() to figure out the permissions.
>> 
>> I'm not very much into Cygwin internals, so beg pardon if I got something
>> wrong here... But reading this makes my internal sanity checker go into red
>> alarm state.
>> 
>> Here's why:
>> 
>> When Cygwin mount a filesystem with 'acl' flag set, it mangles current ACL's
>> set on the files to produce something that can be understood as basic POSIX
>> 'ugly'...erm, 'ugo' permissions. Behavior least desirable in many cases.
>> You say, it will then use native functions to determine access rights... No
>> wonder they will work, since you already mangled them to suit your needs.
>> 
>> When Cygwin mount a filesystem with 'noacl' flag, thus let OS use true ACL's
>> (a feature Windows implemented surprisingly fast, while *NIX was only
>> proposing it... for far too long without any result in sight), it is then
>> followed by some magic and guesswork on Cygwin's end to find out access
>> rights.
>> 
>> If you ask me, something isn't quite right here. Or something is missing.

> It's what "acl" means on Cygwin.  "acl" means that Windowsd ACLs are used
> and permissions are handled and converted to and from POSIX permissions.
> "noacl" means, Cygwin ignores all ACLs and fakes ownership and POSIX
> permissions only based only on filetype and DOS R/O attribute, as it has
> to on filesystems not supporting ACLs, like FAT/FAT32.

Got it.
It seems, Cygwin need a middle groung between these two for cases, where FS
support access control, but don't want to be mangled.


--
WBR,
Andrey Repin (anrdaemon@yandex.ru) 28.08.2014, <17:22>

Sorry for my terrible english...


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1350 bytes --]

On Aug 28 17:23, Andrey Repin wrote:
> > It's what "acl" means on Cygwin.  "acl" means that Windowsd ACLs are used
> > and permissions are handled and converted to and from POSIX permissions.
> > "noacl" means, Cygwin ignores all ACLs and fakes ownership and POSIX
> > permissions only based only on filetype and DOS R/O attribute, as it has
> > to on filesystems not supporting ACLs, like FAT/FAT32.
> 
> Got it.
> It seems, Cygwin need a middle groung between these two for cases, where FS
> support access control, but don't want to be mangled.

I'm certainly not going to introduce another mount mode.  What Cygwin
could do is to perform ACL-based access checks independently of the
"acl"/"noacl" mount mode on FSes supporting ACLs.  However, if you want
ACLs, why not use the "acl" mount mode in the first place?

Still, it *might* makes sense in some scenarios, even if the results of
stat(2)/acl(2) may differ surprisingly from what access(2) returns.

We can also simply try it out.  A patch to enable this behaviour is
dead-simple.

Here's the prerequisite:

  Would more than one person want that *and* be willing to give this a
  *thorough* testing?


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

ACL behavior in Cygwin // Re: (call-process ...) hangs in emacs

[Andrey Repin]

Greetings, Corinna Vinschen!

>> > It's what "acl" means on Cygwin.  "acl" means that Windowsd ACLs are used
>> > and permissions are handled and converted to and from POSIX permissions.
>> > "noacl" means, Cygwin ignores all ACLs and fakes ownership and POSIX
>> > permissions only based only on filetype and DOS R/O attribute, as it has
>> > to on filesystems not supporting ACLs, like FAT/FAT32.
>> 
>> Got it.
>> It seems, Cygwin need a middle groung between these two for cases, where FS
>> support access control, but don't want to be mangled.

> I'm certainly not going to introduce another mount mode.

I didn't said it has to be mount mode... besides, it doesn't make sense to
implement YA mode to do what is already done, just a little different.

> What Cygwin could do is to perform ACL-based access checks independently of
> the "acl"/"noacl" mount mode on FSes supporting ACLs.  However, if you want
> ACLs, why not use the "acl" mount mode in the first place?

ACL inheritance, mostly. POSIX'ized permissions break inheritance on newly
created files, at times making these files inaccessible to native
applications, even though inheritance rules would allow it otherwise.

> Still, it *might* makes sense in some scenarios, even if the results of
> stat(2)/acl(2) may differ surprisingly from what access(2) returns.

> We can also simply try it out.  A patch to enable this behaviour is
> dead-simple.

> Here's the prerequisite:

>   Would more than one person want that *and* be willing to give this a
>   *thorough* testing?

I'd like to hear out expected behavior from this patch first.
I might be able to do some testing, but not in the nearest month, I'm afraid.
The "list of things to do" grew out of control, and I'm trying hard to make
it shorter.
If there's no other interested parties, let's put it on ice, until you come
back from your vacation?


--
WBR,
Andrey Repin (anrdaemon@yandex.ru) 28.08.2014, <20:37>

Sorry for my terrible english...


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: ACL behavior in Cygwin // Re: (call-process ...) hangs in emacs

[Achim Gratz]

Andrey Repin writes:
>> What Cygwin could do is to perform ACL-based access checks independently of
>> the "acl"/"noacl" mount mode on FSes supporting ACLs.  However, if you want
>> ACLs, why not use the "acl" mount mode in the first place?
>
> ACL inheritance, mostly. POSIX'ized permissions break inheritance on newly
> created files, at times making these files inaccessible to native
> applications, even though inheritance rules would allow it otherwise.

You can prevent this from happening if you forbid users to change the
ACL and enforce inheritance.  That's the reason I can't give those files
sensible POSIX permissions since they'd need to be translated into ACL
which I can't write.  All our filers are set up that way.  No I don't
think this is a good idea, but I guess there'd been one support call too
many with a share that somebody made inaccessible by fiddling with the
ACL.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

DIY Stuff:
http://Synth.Stromeko.net/DIY.html

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: ACL behavior in Cygwin // Re: (call-process ...) hangs in emacs

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2127 bytes --]

On Aug 28 21:00, Andrey Repin wrote:
> Greetings, Corinna Vinschen!
> 
> >> > It's what "acl" means on Cygwin.  "acl" means that Windowsd ACLs are used
> >> > and permissions are handled and converted to and from POSIX permissions.
> >> > "noacl" means, Cygwin ignores all ACLs and fakes ownership and POSIX
> >> > permissions only based only on filetype and DOS R/O attribute, as it has
> >> > to on filesystems not supporting ACLs, like FAT/FAT32.
> >> 
> >> Got it.
> >> It seems, Cygwin need a middle groung between these two for cases, where FS
> >> support access control, but don't want to be mangled.
> 
> > I'm certainly not going to introduce another mount mode.
> 
> I didn't said it has to be mount mode... besides, it doesn't make sense to
> implement YA mode to do what is already done, just a little different.
> 
> > What Cygwin could do is to perform ACL-based access checks independently of
> > the "acl"/"noacl" mount mode on FSes supporting ACLs.  However, if you want
> > ACLs, why not use the "acl" mount mode in the first place?
> 
> ACL inheritance, mostly. POSIX'ized permissions break inheritance on newly
> created files, at times making these files inaccessible to native
> applications, even though inheritance rules would allow it otherwise.
> 
> > Still, it *might* makes sense in some scenarios, even if the results of
> > stat(2)/acl(2) may differ surprisingly from what access(2) returns.
> 
> > We can also simply try it out.  A patch to enable this behaviour is
> > dead-simple.
> 
> > Here's the prerequisite:
> 
> >   Would more than one person want that *and* be willing to give this a
> >   *thorough* testing?
> 
> I'd like to hear out expected behavior from this patch first.

Same output from stat(2), different output from access(2).  Access(2)
would only take the actual Windows ACL into account and let Windows
functions decide about granting or denying the requested access.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: (call-process ...) hangs in emacs

[Achim Gratz]

Corinna Vinschen writes:
> Here's the prerequisite:
>
>   Would more than one person want that *and* be willing to give this a
>   *thorough* testing?

That really becomes an issue only if you have to use external shares
that are set up in peculiar ways and AD integration.  The number of
people that fall into that category is… small, let's say.  For the ACL
part it would be possible to set up a test bed on a local machine, but
my guess for the number of people doing that just for the fun of testing
is that it's likely to be an even smaller number.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Andrey Repin]

Greetings, Achim Gratz!

>> Here's the prerequisite:
>>
>>   Would more than one person want that *and* be willing to give this a
>>   *thorough* testing?

> That really becomes an issue only if you have to use external shares
> that are set up in peculiar ways and AD integration.

I've managed to do that on a local machine that is not a member of any domain.
Using noacl mount flag ever since, to avoid similar things from happening.

> The number of people that fall into that category is… small, let's say.
> For the ACL
> part it would be possible to set up a test bed on a local machine, but
> my guess for the number of people doing that just for the fun of testing
> is that it's likely to be an even smaller number.


--
WBR,
Andrey Repin (anrdaemon@yandex.ru) 28.08.2014, <23:40>

Sorry for my terrible english...

Re: (call-process ...) hangs in emacs

[Katsumi Yamaoka]

On Tue, 05 Aug 2014 13:55:31 -0400, Ken Brown wrote:
> Angelo and Katsumi, could you test it and see if it solves the
> problems you reported?  If so, I'll issue new emacs releases.

Thanks.  But currently I cannot test it since the autogen.sh
script doesn't work as the following.  I must make it work,
somehow or other...

% ./autogen.sh
[...]
Running 'autoreconf -fi -I m4' ...
      0 [main] perl 4508 child_info_fork::abort: address space needed by 'POSIX.dll' (0x2D0000) is already occupied
Can't fork, trying again in 5 seconds at /usr/bin/autoreconf-2.69 line 188.
      0 [main] perl 6264 child_info_fork::abort: address space needed by 'POSIX.dll' (0x260000) is already occupied
Can't fork, trying again in 5 seconds at /usr/lib/perl5/5.14/i686-cygwin-threads-64int/IO/File.pm line 188.
[...]

rebaseall nor reinstalling of perl and some things doesn't help.
:<

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1197 bytes --]

On Aug  6 11:30, Katsumi Yamaoka wrote:
> On Tue, 05 Aug 2014 13:55:31 -0400, Ken Brown wrote:
> > Angelo and Katsumi, could you test it and see if it solves the
> > problems you reported?  If so, I'll issue new emacs releases.
> 
> Thanks.  But currently I cannot test it since the autogen.sh
> script doesn't work as the following.  I must make it work,
> somehow or other...
> 
> % ./autogen.sh
> [...]
> Running 'autoreconf -fi -I m4' ...
>       0 [main] perl 4508 child_info_fork::abort: address space needed by 'POSIX.dll' (0x2D0000) is already occupied
> Can't fork, trying again in 5 seconds at /usr/bin/autoreconf-2.69 line 188.
>       0 [main] perl 6264 child_info_fork::abort: address space needed by 'POSIX.dll' (0x260000) is already occupied
> Can't fork, trying again in 5 seconds at /usr/lib/perl5/5.14/i686-cygwin-threads-64int/IO/File.pm line 188.
> [...]
> 
> rebaseall nor reinstalling of perl and some things doesn't help.
> :<

You definitely have a DLL collision.  What about perlrebase?


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: (call-process ...) hangs in emacs

[Katsumi Yamaoka]

On Wed, 06 Aug 2014 10:48:49 +0200, Corinna Vinschen wrote:
> On Aug  6 11:30, Katsumi Yamaoka wrote:
>> % ./autogen.sh
>> [...]
>> Running 'autoreconf -fi -I m4' ...
>>       0 [main] perl 4508 child_info_fork::abort: address space needed by 'POSIX.dll' (0x2D0000) is already occupied
[...]
> You definitely have a DLL collision.  What about perlrebase?

Oh, I did never know such a help tool existence.  Thanks so much!
But strangely enough, autogen.sh and perl work today without doing
anything in particular.  What I did since last night was only to
shutdown and to boot the PC.  Such is one of mysteries of Cygwin.

>> On Tue, 05 Aug 2014 13:55:31 -0400, Ken Brown wrote:
>>> Angelo and Katsumi, could you test it and see if it solves the
>>> problems you reported?  If so, I'll issue new emacs releases.

Great!  Now I'm running trunk Emacs that I built with the patch.
I also verified it runs with no problem for the batch jobs; I'm
using newly built ELisp packages, such as Gnus, for the first time
in these 7 days.  Thanks a lot!

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Andrey Repin]

Greetings, Katsumi Yamaoka!

>>> % ./autogen.sh
>>> [...]
>>> Running 'autoreconf -fi -I m4' ...
>>>       0 [main] perl 4508 child_info_fork::abort: address space needed by 'POSIX.dll' (0x2D0000) is already occupied
> [...]
>> You definitely have a DLL collision.  What about perlrebase?

> Oh, I did never know such a help tool existence.  Thanks so much!
> But strangely enough, autogen.sh and perl work today without doing
> anything in particular.  What I did since last night was only to
> shutdown and to boot the PC.  Such is one of mysteries of Cygwin.

That's no mystery, but looks more like BLODA. One or another DLL got loaded
into different address, and everything went up in flames.


--
WBR,
Andrey Repin (anrdaemon@yandex.ru) 07.08.2014, <04:20>

Sorry for my terrible english...


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Peter Hull]

On Mon, Aug 4, 2014 at 2:02 AM, Ken Brown <kbrown@cornell.edu> wrote:
> That does seem to be the problem, since I can reproduce the bug starting
> with the 2014-07-14 snapshot.  More precisely, I can reproduce it using
> emacs-nox (which is what the OP was using according to his cygcheck output)
> but not using emacs-X11 or emacs-w32.
Thanks for your help in resolving this. I am indeed using emacs-nox.
Do you think emacs-x11 or emacs-w32 would be a good alternative to
work round the problem in the short term?

Peter

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: (call-process ...) hangs in emacs

[Ken Brown]

On 8/4/2014 4:05 AM, Peter Hull wrote:
> On Mon, Aug 4, 2014 at 2:02 AM, Ken Brown <kbrown@cornell.edu> wrote:
>> That does seem to be the problem, since I can reproduce the bug starting
>> with the 2014-07-14 snapshot.  More precisely, I can reproduce it using
>> emacs-nox (which is what the OP was using according to his cygcheck output)
>> but not using emacs-X11 or emacs-w32.
> Thanks for your help in resolving this. I am indeed using emacs-nox.
> Do you think emacs-x11 or emacs-w32 would be a good alternative to
> work round the problem in the short term?

Yes.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple