Regression (last snapshot)

Regression (last snapshot)

[Houder]

L.S.,

Using the latest snapshot, I now get a complaint from 'ls'; this does
not happen w/ 3.0.7

Henri

  = 3.0.7:

64-@@ uname -a
CYGWIN_NT-6.1 Seven 3.0.7(0.338/5/3) 2019-04-30 18:08 x86_64 Cygwin

64-@@ ls -lL <(grep bash .bashrc)
pr-------- 1 Henri None 0 Jul 21 00:49 /dev/fd/63

  = snapshot 2019-07-12:

64-@@ uname -a
CYGWIN_NT-6.1 Seven 3.1.0s(0.339/5/3) 2019-07-12 15:28 x86_64 Cygwin

64-@@ ls -lL <(grep bash .bashrc)
ls: /dev/fd/63: No such file or directory
pr-------- 1 Henri None 0 Jul 21 00:41 /dev/fd/63

=====

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Regression (last snapshot)

[Houder]

On Sun, 21 Jul 2019 00:55:51, Houder  wrote:
> L.S.,
> 
> Using the latest snapshot, I now get a complaint from 'ls'; this does
> not happen w/ 3.0.7
> 
> Henri
> 
>   = 3.0.7:
> 
> 64-@@ uname -a
> CYGWIN_NT-6.1 Seven 3.0.7(0.338/5/3) 2019-04-30 18:08 x86_64 Cygwin
> 
> 64-@@ ls -lL <(grep bash .bashrc)
> pr-------- 1 Henri None 0 Jul 21 00:49 /dev/fd/63
> 
>   = snapshot 2019-07-12:
> 
> 64-@@ uname -a
> CYGWIN_NT-6.1 Seven 3.1.0s(0.339/5/3) 2019-07-12 15:28 x86_64 Cygwin
> 
> 64-@@ ls -lL <(grep bash .bashrc)
> ls: /dev/fd/63: No such file or directory
> pr-------- 1 Henri None 0 Jul 21 00:41 /dev/fd/63

However 32-bits Cygwin does not complain ...

Henri

@@ uname -a
CYGWIN_NT-6.1-WOW Seven 3.1.0s(0.339/5/3) 2019-07-12 15:28 i686 Cygwin
@@ ls -l <(grep bash .bashrc)
lrwxrwxrwx 1 Henri None 0 Jul 21 11:34 /dev/fd/63 -> pipe:[30064773864]

=====


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Regression (last snapshot)

[Houder]

On Sun, 21 Jul 2019 11:38:49, Houder  wrote:
> On Sun, 21 Jul 2019 00:55:51, Houder  wrote:
> > L.S.,
> > 
> > Using the latest snapshot, I now get a complaint from 'ls'; this does
> > not happen w/ 3.0.7
> > 
> > Henri
> > 
> >   = 3.0.7:
> > 
> > 64-@@ uname -a
> > CYGWIN_NT-6.1 Seven 3.0.7(0.338/5/3) 2019-04-30 18:08 x86_64 Cygwin
> > 
> > 64-@@ ls -lL <(grep bash .bashrc)
> > pr-------- 1 Henri None 0 Jul 21 00:49 /dev/fd/63
> > 
> >   = snapshot 2019-07-12:
> > 
> > 64-@@ uname -a
> > CYGWIN_NT-6.1 Seven 3.1.0s(0.339/5/3) 2019-07-12 15:28 x86_64 Cygwin
> > 
> > 64-@@ ls -lL <(grep bash .bashrc)
> > ls: /dev/fd/63: No such file or directory
> > pr-------- 1 Henri None 0 Jul 21 00:41 /dev/fd/63
> 
> However 32-bits Cygwin does not complain ...

Sorry, my mistake (32-bits also complains)

@@ uname -a
CYGWIN_NT-6.1-WOW Seven 3.1.0s(0.339/5/3) 2019-07-12 15:28 i686 Cygwin
@@ ls -lL <(grep bash .bashrc)
ls: /dev/fd/63: No such file or directory
pr-------- 1 Henri None 0 Jul 21 11:41 /dev/fd/63

=====


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Regression (last snapshot)

[Ken Brown]

On 7/20/2019 6:55 PM, Houder wrote:
> 64-@@ uname -a
> CYGWIN_NT-6.1 Seven 3.1.0s(0.339/5/3) 2019-07-12 15:28 x86_64 Cygwin
> 
> 64-@@ ls -lL <(grep bash .bashrc)
> ls: /dev/fd/63: No such file or directory
> pr-------- 1 Henri None 0 Jul 21 00:41 /dev/fd/63

Thanks for the report.  This is probably caused by my new FIFO code.  I'm 
looking into it.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Regression (last snapshot)

[Ken Brown]

On 7/22/2019 8:23 AM, Ken Brown wrote:
> On 7/20/2019 6:55 PM, Houder wrote:
>> 64-@@ uname -a
>> CYGWIN_NT-6.1 Seven 3.1.0s(0.339/5/3) 2019-07-12 15:28 x86_64 Cygwin
>>
>> 64-@@ ls -lL <(grep bash .bashrc)
>> ls: /dev/fd/63: No such file or directory
>> pr-------- 1 Henri None 0 Jul 21 00:41 /dev/fd/63
> 
> Thanks for the report.  This is probably caused by my new FIFO code.  I'm
> looking into it.

Actually, a bisection shows that the regression is due to the following commit:

commit 2607639992f6600135532831c8357c10cb248821
Author: Erik M. Bray <erik.m.bray@gmail.com>
Date:   Wed Apr 10 17:05:22 2019 +0200

     Improve error handling in /proc/[pid]/ virtual files.

     * Changes error handling to allow /proc/[pid]/ virtual files to be
       empty in some cases (in this case the file's formatter should return
       -1 upon error, not 0).

     * Better error handling of /proc/[pid]/stat for zombie processes:
       previously trying to open this file on zombie processes resulted
       in an EINVAL being returned by open().  Now the file can be read,
       and fields that can no longer be read are just zeroed.

     * Similarly for /proc/[pid]/statm for zombie processes.

     * Similarly for /proc/[pid]/maps for zombie processes (in this case the
       file can be read but is zero-length, which is consistent with observed
       behavior on Linux.


Erik, can you take a look?

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Regression (last snapshot)

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2750 bytes --]

On Jul 22 13:44, Ken Brown wrote:
> On 7/22/2019 8:23 AM, Ken Brown wrote:
> > On 7/20/2019 6:55 PM, Houder wrote:
> >> 64-@@ uname -a
> >> CYGWIN_NT-6.1 Seven 3.1.0s(0.339/5/3) 2019-07-12 15:28 x86_64 Cygwin
> >>
> >> 64-@@ ls -lL <(grep bash .bashrc)
> >> ls: /dev/fd/63: No such file or directory
> >> pr-------- 1 Henri None 0 Jul 21 00:41 /dev/fd/63
> > 
> > Thanks for the report.  This is probably caused by my new FIFO code.  I'm
> > looking into it.
> 
> Actually, a bisection shows that the regression is due to the following commit:
> 
> commit 2607639992f6600135532831c8357c10cb248821
> Author: Erik M. Bray <erik.m.bray@gmail.com>
> Date:   Wed Apr 10 17:05:22 2019 +0200
> 
>      Improve error handling in /proc/[pid]/ virtual files.
> 
>      * Changes error handling to allow /proc/[pid]/ virtual files to be
>        empty in some cases (in this case the file's formatter should return
>        -1 upon error, not 0).
> 
>      * Better error handling of /proc/[pid]/stat for zombie processes:
>        previously trying to open this file on zombie processes resulted
>        in an EINVAL being returned by open().  Now the file can be read,
>        and fields that can no longer be read are just zeroed.
> 
>      * Similarly for /proc/[pid]/statm for zombie processes.
> 
>      * Similarly for /proc/[pid]/maps for zombie processes (in this case the
>        file can be read but is zero-length, which is consistent with observed
>        behavior on Linux.
> 
> 
> Erik, can you take a look?

I have a hunch.  It's this change:

@@ -355,7 +355,7 @@ fhandler_process::fill_filebuf ()
        }
       else
        filesize = process_tab[fileid].format_func (p, filebuf);
-      return !filesize ? false : true;
+      return filesize < 0 ? false : true;
     }
   return false;
 }

The formatter for /proc/PID/fd, format_process_fd, returns *valid*
negative values.  But the above patch treats all negative values as
error now.

The fact that format_process_fd returns negative values has historical
reasons.  Negative values of type virtual_ftype_t are files, positive
values are directories.

One way to fix this is to change this to all positive values.  At a
first glance I don't see any check for an explicit negative
virtual_ftype_t value, especially not in the only consumer
path_conv::check in path.cc, and the simple numbers have long been
replaced with enum values.  At the same time, we should better drop
some outdated comments in the virtual file fhandler classes, e.g.

  /* Returns 0 if path doesn't exist, >0 if path is a directory,
     -1 if path is a file, -2 if it's a symlink.  */


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Regression (last snapshot)

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2745 bytes --]

On Jul 22 17:20, Corinna Vinschen wrote:
> On Jul 22 13:44, Ken Brown wrote:
> > On 7/22/2019 8:23 AM, Ken Brown wrote:
> > > On 7/20/2019 6:55 PM, Houder wrote:
> > >> 64-@@ uname -a
> > >> CYGWIN_NT-6.1 Seven 3.1.0s(0.339/5/3) 2019-07-12 15:28 x86_64 Cygwin
> > >>
> > >> 64-@@ ls -lL <(grep bash .bashrc)
> > >> ls: /dev/fd/63: No such file or directory
> > >> pr-------- 1 Henri None 0 Jul 21 00:41 /dev/fd/63
> > > 
> > > Thanks for the report.  This is probably caused by my new FIFO code.  I'm
> > > looking into it.
> > 
> > Actually, a bisection shows that the regression is due to the following commit:
> > 
> > commit 2607639992f6600135532831c8357c10cb248821
> > Author: Erik M. Bray <erik.m.bray@gmail.com>
> > Date:   Wed Apr 10 17:05:22 2019 +0200
> > 
> >      Improve error handling in /proc/[pid]/ virtual files.
> > 
> >      * Changes error handling to allow /proc/[pid]/ virtual files to be
> >        empty in some cases (in this case the file's formatter should return
> >        -1 upon error, not 0).
> > 
> >      * Better error handling of /proc/[pid]/stat for zombie processes:
> >        previously trying to open this file on zombie processes resulted
> >        in an EINVAL being returned by open().  Now the file can be read,
> >        and fields that can no longer be read are just zeroed.
> > 
> >      * Similarly for /proc/[pid]/statm for zombie processes.
> > 
> >      * Similarly for /proc/[pid]/maps for zombie processes (in this case the
> >        file can be read but is zero-length, which is consistent with observed
> >        behavior on Linux.
> > 
> > 
> > Erik, can you take a look?
> 
> I have a hunch.  It's this change:
> 
> @@ -355,7 +355,7 @@ fhandler_process::fill_filebuf ()
>         }
>        else
>         filesize = process_tab[fileid].format_func (p, filebuf);
> -      return !filesize ? false : true;
> +      return filesize < 0 ? false : true;
>      }
>    return false;
>  }
> 
> The formatter for /proc/PID/fd, format_process_fd, returns *valid*
> negative values.  But the above patch treats all negative values as
> error now.
> 
> The fact that format_process_fd returns negative values has historical
> reasons.  Negative values of type virtual_ftype_t are files, positive
> values are directories.
> 
> One way to fix this is to change this to all positive values.  At a
> first glance I don't see any check for an explicit negative
> virtual_ftype_t value, especially not in the only consumer
> path_conv::check in path.cc, and the simple numbers have long been
> replaced with enum values.

A second glance shows a few problems in the code...


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Regression (last snapshot)

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2208 bytes --]

On Jul 22 17:53, Corinna Vinschen wrote:
> On Jul 22 17:20, Corinna Vinschen wrote:
> > On Jul 22 13:44, Ken Brown wrote:
> > > On 7/22/2019 8:23 AM, Ken Brown wrote:
> > > > On 7/20/2019 6:55 PM, Houder wrote:
> > > >> 64-@@ uname -a
> > > >> CYGWIN_NT-6.1 Seven 3.1.0s(0.339/5/3) 2019-07-12 15:28 x86_64 Cygwin
> > > >>
> > > >> 64-@@ ls -lL <(grep bash .bashrc)
> > > >> ls: /dev/fd/63: No such file or directory
> > > >> pr-------- 1 Henri None 0 Jul 21 00:41 /dev/fd/63
> > > > 
> > > > Thanks for the report.  This is probably caused by my new FIFO code.  I'm
> > > > looking into it.
> > > 
> > > Actually, a bisection shows that the regression is due to the following commit:
> > > 
> > > commit 2607639992f6600135532831c8357c10cb248821
> > > Author: Erik M. Bray <erik.m.bray@gmail.com>
> > > Date:   Wed Apr 10 17:05:22 2019 +0200
> > > 
> > >      Improve error handling in /proc/[pid]/ virtual files.
> > > [...]
> > > Erik, can you take a look?
> > 
> > I have a hunch.  It's this change:
> > 
> > @@ -355,7 +355,7 @@ fhandler_process::fill_filebuf ()
> >         }
> >        else
> >         filesize = process_tab[fileid].format_func (p, filebuf);
> > -      return !filesize ? false : true;
> > +      return filesize < 0 ? false : true;
> >      }
> >    return false;
> >  }
> > 
> > The formatter for /proc/PID/fd, format_process_fd, returns *valid*
> > negative values.  But the above patch treats all negative values as
> > error now.
> > 
> > The fact that format_process_fd returns negative values has historical
> > reasons.  Negative values of type virtual_ftype_t are files, positive
> > values are directories.
> > 
> > One way to fix this is to change this to all positive values.  At a
> > first glance I don't see any check for an explicit negative
> > virtual_ftype_t value, especially not in the only consumer
> > path_conv::check in path.cc, and the simple numbers have long been
> > replaced with enum values.
> 
> A second glance shows a few problems in the code...

I pushed a patch and I'm building new developer snapshots right now.
Please give them a try.


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Regression (last snapshot)

[Houder]

On Mon, 22 Jul 2019 18:45:09, Corinna Vinschen  wrote:
[snip]

> I pushed a patch and I'm building new developer snapshots right now.
> Please give them a try.

Thanks Corinna.
Thanks Ken.

The specific regression as reported, has gone.

Regards,
Henri

64-@@ uname -a
CYGWIN_NT-6.1 Seven 3.1.0s(0.339/5/3) 2019-07-22 16:43 x86_64 Cygwin
64-@@ ls -lL <(grep bash .bashrc)
pr-------- 1 Henri None 0 Jul 22 20:36 /dev/fd/63

@@ uname -a
CYGWIN_NT-6.1-WOW Seven 3.1.0s(0.339/5/3) 2019-07-22 16:43 i686 Cygwin
@@ ls -lL <(grep bash .bashrc)
pr-------- 1 Henri None 0 Jul 22 20:36 /dev/fd/63

=====


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Regression (last snapshot)

[Ken Brown]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 2419 bytes --]

On 7/22/2019 2:47 PM, Houder wrote:
> The specific regression as reported, has gone.
> 
> 64-@@ uname -a
> CYGWIN_NT-6.1 Seven 3.1.0s(0.339/5/3) 2019-07-22 16:43 x86_64 Cygwin
> 64-@@ ls -lL <(grep bash .bashrc)
> pr-------- 1 Henri None 0 Jul 22 20:36 /dev/fd/63

It turns out that this isn't completely fixed, but I only see the problem when 
working under X11, and the error message is different.  Here are the complete 
reproduction steps:

1. Start the X server by using the XWin Server shortcut.

2. Use the X Applications Menu -> System Tools to start an xterm window.

3. In that window, execute the above 'ls' command.:

$ ls -lL <(grep bash .bashrc)
pr-------- 1 kbrown None 0 2019-07-26 17:24 /dev/fd/63
grep: write error: Broken pipe

This is with the 2019-07-22 snapshot.  With the 2019-07-25 snapshot, it's better 
but still broken: I have to run the ls command twice to get the error:

$ ls -lL <(grep bash .bashrc)
pr-------- 1 kbrown None 0 2019-07-26 17:39 /dev/fd/63

$ ls -lL <(grep bash .bashrc)
pr-------- 1 kbrown None 0 2019-07-26 17:39 /dev/fd/63
grep: write error: Broken pipe

Here's one more fact, which may or not provide further clues: When I exited the 
X server while testing the 2019-07-22 snapshot, there was a /bin/sh process 
still running that I couldn't kill with 'kill -9'.  I had to kill it with the 
Task Manager.

Finally, here's an excerpt from the strace output for the failing ls command:

    24   23441 [main] ls 1033 fhandler_proc::get_proc_fhandler: 
get_proc_fhandler(/proc/1033/fd/63)
    25   23466 [main] ls 1033 mount_info::conv_to_win32_path: src_path 
/proc/1033/fd/63, dst /proc/1033/fd/63, flags 0x0, rc 0
    28   23494 [main] ls 1033 build_fh_pc: fh 0x180340098, dev 000000FB
    24   23518 [main] ls 1033 fhandler_process::exists: exists (/proc/1033/fd/63)
    28   23546 [main] ls 1033 __set_errno: cygheap_fdget::cygheap_fdget(int, 
bool, bool):679 setting errno 9
    24   23570 [main] ls 1033 __set_errno: off_t format_process_fd(void*, 
char*&):394 setting errno 2

I'm in the process of building an unoptimized cygwin1.dll to see if I can get 
further information.

Ken
\0ТÒÐÐ¥\a&ö&ÆVÒ\a&W\x06÷'G3¢\x02\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒ÷\a&ö&ÆV×2æ‡FÖÀФd\x15\x13¢\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒöf\x17\x12ðФFö7VÖVçF\x17F–öã¢\x02\x02\x02\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒöFö72æ‡FÖÀÐ¥Vç7V'67&–&R\x06–æfó¢\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒöÖÂò7Vç7V'67&–&R×6–×\x06ÆPРÐ

Re: Regression (last snapshot)

[Ken Brown]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 2786 bytes --]

On 7/26/2019 6:12 PM, Ken Brown wrote:
> On 7/22/2019 2:47 PM, Houder wrote:
>> The specific regression as reported, has gone.
>>
>> 64-@@ uname -a
>> CYGWIN_NT-6.1 Seven 3.1.0s(0.339/5/3) 2019-07-22 16:43 x86_64 Cygwin
>> 64-@@ ls -lL <(grep bash .bashrc)
>> pr-------- 1 Henri None 0 Jul 22 20:36 /dev/fd/63
> 
> It turns out that this isn't completely fixed, but I only see the problem when
> working under X11, and the error message is different.  Here are the complete
> reproduction steps:
> 
> 1. Start the X server by using the XWin Server shortcut.
> 
> 2. Use the X Applications Menu -> System Tools to start an xterm window.
> 
> 3. In that window, execute the above 'ls' command.:
> 
> $ ls -lL <(grep bash .bashrc)
> pr-------- 1 kbrown None 0 2019-07-26 17:24 /dev/fd/63
> grep: write error: Broken pipe
> 
> This is with the 2019-07-22 snapshot.  With the 2019-07-25 snapshot, it's better
> but still broken: I have to run the ls command twice to get the error:
> 
> $ ls -lL <(grep bash .bashrc)
> pr-------- 1 kbrown None 0 2019-07-26 17:39 /dev/fd/63
> 
> $ ls -lL <(grep bash .bashrc)
> pr-------- 1 kbrown None 0 2019-07-26 17:39 /dev/fd/63
> grep: write error: Broken pipe
> 
> Here's one more fact, which may or not provide further clues: When I exited the
> X server while testing the 2019-07-22 snapshot, there was a /bin/sh process
> still running that I couldn't kill with 'kill -9'.  I had to kill it with the
> Task Manager.
> 
> Finally, here's an excerpt from the strace output for the failing ls command:
> 
>      24   23441 [main] ls 1033 fhandler_proc::get_proc_fhandler:
> get_proc_fhandler(/proc/1033/fd/63)
>      25   23466 [main] ls 1033 mount_info::conv_to_win32_path: src_path
> /proc/1033/fd/63, dst /proc/1033/fd/63, flags 0x0, rc 0
>      28   23494 [main] ls 1033 build_fh_pc: fh 0x180340098, dev 000000FB
>      24   23518 [main] ls 1033 fhandler_process::exists: exists (/proc/1033/fd/63)
>      28   23546 [main] ls 1033 __set_errno: cygheap_fdget::cygheap_fdget(int,
> bool, bool):679 setting errno 9
>      24   23570 [main] ls 1033 __set_errno: off_t format_process_fd(void*,
> char*&):394 setting errno 2
> 
> I'm in the process of building an unoptimized cygwin1.dll to see if I can get
> further information.

I just discovered that this problem already existed in commit f0ea836b7, which 
preceded the commit that led to the problem Houder reported.  So this must be 
something different.  I'll do a new bisection for the current problem.

Ken
\x03B‹KCB”\x1c›Ø›\x19[H\x1c™\^[ܝ\x1cΈ\b\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÜ\x1c›Ø›\x19[\Ëš\x1d^[[\x03B‘TNˆ\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÙ˜\KÃB‘^[ØÝ[Y[\x18]\x1a[ÛŽˆ\b\b\b\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÙ^[ØÜËš\x1d^[[\x03B•[œÝXœØÜšX™H\x1a[™›Îˆ\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÛ[\vÈÝ[œÝXœØÜšX™K\Ú[\^[\x19CBƒB

Re: Regression (last snapshot)

[Houder]

On Fri, 26 Jul 2019 22:12:43, Ken Brown  wrote:

> On 7/22/2019 2:47 PM, Houder wrote:

>> The specific regression as reported, has gone.
>>
>> 64-@@ uname -a
>> CYGWIN_NT-6.1 Seven 3.1.0s(0.339/5/3) 2019-07-22 16:43 x86_64 Cygwin
>> 64-@@ ls -lL <(grep bash .bashrc)
>> pr-------- 1 Henri None 0 Jul 22 20:36 /dev/fd/63
> 
> It turns out that this isn't completely fixed, but I only see the problem when
> working under X11, and the error message is different.  Here are the complete
> reproduction steps:
> 
> 1. Start the X server by using the XWin Server shortcut.
> 
> 2. Use the X Applications Menu -> System Tools to start an xterm window.
> 
> 3. In that window, execute the above 'ls' command.:
> 
> $ ls -lL <(grep bash .bashrc)
> pr-------- 1 kbrown None 0 2019-07-26 17:24 /dev/fd/63
> grep: write error: Broken pipe
> 
> This is with the 2019-07-22 snapshot.  With the 2019-07-25 snapshot, it's better
> but still broken: I have to run the ls command twice to get the error:
> 
> $ ls -lL <(grep bash .bashrc)
> pr-------- 1 kbrown None 0 2019-07-26 17:39 /dev/fd/63
> 
> $ ls -lL <(grep bash .bashrc)
> pr-------- 1 kbrown None 0 2019-07-26 17:39 /dev/fd/63
> grep: write error: Broken pipe
> 
> Here's one more fact, which may or not provide further clues: When I exited the
> X server while testing the 2019-07-22 snapshot, there was a /bin/sh process
> still running that I couldn't kill with 'kill -9'.  I had to kill it with the
> Task Manager.
> 
> Finally, here's an excerpt from the strace output for the failing ls command:

Failing ls? ... uhm, grep complains (as ls did in my case) ...

Over all the behavior has simularity w/ the error reported by David Karr:

    https://cygwin.com/ml/cygwin/2019-07/msg00150.html
    ( Piping input from subprocess loses track of temp file )

Regards,
Henri


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Regression (last snapshot)

[Ken Brown]

On 7/27/2019 6:21 AM, Houder wrote:
> On Fri, 26 Jul 2019 22:12:43, Ken Brown  wrote:
> 
>> On 7/22/2019 2:47 PM, Houder wrote:
> 
>>> The specific regression as reported, has gone.
>>>
>>> 64-@@ uname -a
>>> CYGWIN_NT-6.1 Seven 3.1.0s(0.339/5/3) 2019-07-22 16:43 x86_64 Cygwin
>>> 64-@@ ls -lL <(grep bash .bashrc)
>>> pr-------- 1 Henri None 0 Jul 22 20:36 /dev/fd/63
>>
> Over all the behavior has simularity w/ the error reported by David Karr:
> 
>      https://cygwin.com/ml/cygwin/2019-07/msg00150.html
>      ( Piping input from subprocess loses track of temp file )

Thanks, I hadn't noticed that.

The situation is more complicated than what I reported.  First, it happens even 
in cygwin-3.0.7, as David Karr's report suggests.  Second, it's true that I can 
only reproduce it under X11, but the pattern is not as regular as I thought.  I 
just ran the ls command 1000 times in an xterm window under cygwin-3.0.7, and I 
got the "Broken pipe" error 390 times, with a varying number of consecutive 
successful runs between the errors.

Repeating this under the 20190722 or 20190725 snapshots gave slightly worse 
results (close to 500 errors).  Using my own unoptimized build of cygwin1.dll, 
the error count went up to about 650.

I tried running under gdb, but I couldn't get grep to fail.  More precisely, I 
didn't see an error message from grep.  Every run looked like this:

$ gdb bash
GNU gdb (GDB) (Cygwin 8.2.1-1) 8.2.1
[...]
(gdb) r -c 'ls -lL <(grep bash .bashrc)'
Starting program: /usr/bin/bash -c 'ls -lL <(grep bash .bashrc)'
[...]
pr-------- 1 kbrown None 0 2019-07-27 11:07 /dev/fd/63
[...]
[Inferior 1 (process 21712) exited normally]

It would be better to be able to debug ls and/or grep, but I don't know how to 
get to subprocesses in gdb.  And I think I have to start with 'gdb bash' in 
order for the process substitution to happen.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Regression (last snapshot)

[Houder]

On Sat, 27 Jul 2019 15:24:49, Ken Brown  wrote:

> It would be better to be able to debug ls and/or grep, but I don't know how to 
> get to subprocesses in gdb.  And I think I have to start with 'gdb bash' in 
> order for the process substitution to happen.

Sorry Ken, I cannot help you here (gdb/subprocesses) ... I have only superficial
knowledge of using gdb.

However, as I suspect this to be related to Cygwin's implementation of "/proc",
you will need help from Corinna.

Moreover, I am using W7 (perhaps David Karr also uses W7; he did not tell).

I would advice to focus on W10 (as W7 will cease to exist soon); I remember that
Corinna wrote that some "features" related to /proc could not be realized w/ W7.

Regards,
Henri


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Regression (last snapshot)

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2409 bytes --]

On Jul 27 15:24, Ken Brown wrote:
> On 7/27/2019 6:21 AM, Houder wrote:
> > On Fri, 26 Jul 2019 22:12:43, Ken Brown  wrote:
> > 
> >> On 7/22/2019 2:47 PM, Houder wrote:
> > 
> >>> The specific regression as reported, has gone.
> >>>
> >>> 64-@@ uname -a
> >>> CYGWIN_NT-6.1 Seven 3.1.0s(0.339/5/3) 2019-07-22 16:43 x86_64 Cygwin
> >>> 64-@@ ls -lL <(grep bash .bashrc)
> >>> pr-------- 1 Henri None 0 Jul 22 20:36 /dev/fd/63
> >>
> > Over all the behavior has simularity w/ the error reported by David Karr:
> > 
> >      https://cygwin.com/ml/cygwin/2019-07/msg00150.html
> >      ( Piping input from subprocess loses track of temp file )
> 
> Thanks, I hadn't noticed that.
> 
> The situation is more complicated than what I reported.  First, it happens even 
> in cygwin-3.0.7, as David Karr's report suggests.  Second, it's true that I can 
> only reproduce it under X11, but the pattern is not as regular as I thought.  I 
> just ran the ls command 1000 times in an xterm window under cygwin-3.0.7, and I 
> got the "Broken pipe" error 390 times, with a varying number of consecutive 
> successful runs between the errors.
> 
> Repeating this under the 20190722 or 20190725 snapshots gave slightly worse 
> results (close to 500 errors).  Using my own unoptimized build of cygwin1.dll, 
> the error count went up to about 650.

I just tried this myself and I can't reproduce the problem.  1000 runs,
no error.

> I tried running under gdb, but I couldn't get grep to fail.  More precisely, I 
> didn't see an error message from grep.  Every run looked like this:
> 
> $ gdb bash
> GNU gdb (GDB) (Cygwin 8.2.1-1) 8.2.1
> [...]
> (gdb) r -c 'ls -lL <(grep bash .bashrc)'
> Starting program: /usr/bin/bash -c 'ls -lL <(grep bash .bashrc)'
> [...]
> pr-------- 1 kbrown None 0 2019-07-27 11:07 /dev/fd/63
> [...]
> [Inferior 1 (process 21712) exited normally]
> 
> It would be better to be able to debug ls and/or grep, but I don't know how to 
> get to subprocesses in gdb.  And I think I have to start with 'gdb bash' in 
> order for the process substitution to happen.

Yeah, subprocess debugging is a problem in GDB.  Given how this works,
you can at least take grep out of the picture.  Bash is doing all the
lifting, so it's just bash and ls.   Did you try to reproduce this under
strace?


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Regression (last snapshot)

[Ken Brown]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 3463 bytes --]

On 7/29/2019 4:45 AM, Corinna Vinschen wrote:
> On Jul 27 15:24, Ken Brown wrote:
>> On 7/27/2019 6:21 AM, Houder wrote:
>>> On Fri, 26 Jul 2019 22:12:43, Ken Brown  wrote:
>>>
>>>> On 7/22/2019 2:47 PM, Houder wrote:
>>>
>>>>> The specific regression as reported, has gone.
>>>>>
>>>>> 64-@@ uname -a
>>>>> CYGWIN_NT-6.1 Seven 3.1.0s(0.339/5/3) 2019-07-22 16:43 x86_64 Cygwin
>>>>> 64-@@ ls -lL <(grep bash .bashrc)
>>>>> pr-------- 1 Henri None 0 Jul 22 20:36 /dev/fd/63
>>>>
>>> Over all the behavior has simularity w/ the error reported by David Karr:
>>>
>>>       https://cygwin.com/ml/cygwin/2019-07/msg00150.html
>>>       ( Piping input from subprocess loses track of temp file )
>>
>> Thanks, I hadn't noticed that.
>>
>> The situation is more complicated than what I reported.  First, it happens even
>> in cygwin-3.0.7, as David Karr's report suggests.  Second, it's true that I can
>> only reproduce it under X11, but the pattern is not as regular as I thought.  I
>> just ran the ls command 1000 times in an xterm window under cygwin-3.0.7, and I
>> got the "Broken pipe" error 390 times, with a varying number of consecutive
>> successful runs between the errors.
>>
>> Repeating this under the 20190722 or 20190725 snapshots gave slightly worse
>> results (close to 500 errors).  Using my own unoptimized build of cygwin1.dll,
>> the error count went up to about 650.
> 
> I just tried this myself and I can't reproduce the problem.  1000 runs,
> no error.

Interesting.  And you ran this under X11 in an xterm window?

>> I tried running under gdb, but I couldn't get grep to fail.  More precisely, I
>> didn't see an error message from grep.  Every run looked like this:
>>
>> $ gdb bash
>> GNU gdb (GDB) (Cygwin 8.2.1-1) 8.2.1
>> [...]
>> (gdb) r -c 'ls -lL <(grep bash .bashrc)'
>> Starting program: /usr/bin/bash -c 'ls -lL <(grep bash .bashrc)'
>> [...]
>> pr-------- 1 kbrown None 0 2019-07-27 11:07 /dev/fd/63
>> [...]
>> [Inferior 1 (process 21712) exited normally]
>>
>> It would be better to be able to debug ls and/or grep, but I don't know how to
>> get to subprocesses in gdb.  And I think I have to start with 'gdb bash' in
>> order for the process substitution to happen.
> 
> Yeah, subprocess debugging is a problem in GDB.  Given how this works,
> you can at least take grep out of the picture.  Bash is doing all the
> lifting, so it's just bash and ls.   Did you try to reproduce this under
> strace?

Yes, but there I get an error (even under mintty) for a different reason:

$ strace -o trace.out ls -lL <(grep bash .bashrc) 
ls: cannot access '/dev/fd/63': No such file or directory

The strace output shows a call to fhandler_process::exists on /proc/45036/fd/63; 
here 45036 is the PID of 'ls'.  And then I see an EBADF error.  But I think 
what's happening here might be that bash is parsing '<(grep bash .bashrc)' too 
soon, so that '/dev/fd/63' isn't related to the 'ls' command.

By the way, I've just tried a different experiment, in which I simplify the ls 
command to 'ls <(grep bash .bashrc)'.  When I run this under xterm, I get the 
broken pipe error 98% of the time or more.  But it's fine under mintty.

Ken
\0ТÒÐÐ¥\a&ö&ÆVÒ\a&W\x06÷'G3¢\x02\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒ÷\a&ö&ÆV×2æ‡FÖÀФd\x15\x13¢\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒöf\x17\x12ðФFö7VÖVçF\x17F–öã¢\x02\x02\x02\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒöFö72æ‡FÖÀÐ¥Vç7V'67&–&R\x06–æfó¢\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒöÖÂò7Vç7V'67&–&R×6–×\x06ÆPРÐ

Re: Regression (last snapshot)

[Ken Brown]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 3988 bytes --]

On 7/29/2019 9:18 AM, Ken Brown wrote:
> On 7/29/2019 4:45 AM, Corinna Vinschen wrote:
>> On Jul 27 15:24, Ken Brown wrote:
>>> On 7/27/2019 6:21 AM, Houder wrote:
>>>> On Fri, 26 Jul 2019 22:12:43, Ken Brown  wrote:
>>>>
>>>>> On 7/22/2019 2:47 PM, Houder wrote:
>>>>
>>>>>> The specific regression as reported, has gone.
>>>>>>
>>>>>> 64-@@ uname -a
>>>>>> CYGWIN_NT-6.1 Seven 3.1.0s(0.339/5/3) 2019-07-22 16:43 x86_64 Cygwin
>>>>>> 64-@@ ls -lL <(grep bash .bashrc)
>>>>>> pr-------- 1 Henri None 0 Jul 22 20:36 /dev/fd/63
>>>>>
>>>> Over all the behavior has simularity w/ the error reported by David Karr:
>>>>
>>>>        https://cygwin.com/ml/cygwin/2019-07/msg00150.html
>>>>        ( Piping input from subprocess loses track of temp file )
>>>
>>> Thanks, I hadn't noticed that.
>>>
>>> The situation is more complicated than what I reported.  First, it happens even
>>> in cygwin-3.0.7, as David Karr's report suggests.  Second, it's true that I can
>>> only reproduce it under X11, but the pattern is not as regular as I thought.  I
>>> just ran the ls command 1000 times in an xterm window under cygwin-3.0.7, and I
>>> got the "Broken pipe" error 390 times, with a varying number of consecutive
>>> successful runs between the errors.
>>>
>>> Repeating this under the 20190722 or 20190725 snapshots gave slightly worse
>>> results (close to 500 errors).  Using my own unoptimized build of cygwin1.dll,
>>> the error count went up to about 650.
>>
>> I just tried this myself and I can't reproduce the problem.  1000 runs,
>> no error.
> 
> Interesting.  And you ran this under X11 in an xterm window?
> 
>>> I tried running under gdb, but I couldn't get grep to fail.  More precisely, I
>>> didn't see an error message from grep.  Every run looked like this:
>>>
>>> $ gdb bash
>>> GNU gdb (GDB) (Cygwin 8.2.1-1) 8.2.1
>>> [...]
>>> (gdb) r -c 'ls -lL <(grep bash .bashrc)'
>>> Starting program: /usr/bin/bash -c 'ls -lL <(grep bash .bashrc)'
>>> [...]
>>> pr-------- 1 kbrown None 0 2019-07-27 11:07 /dev/fd/63
>>> [...]
>>> [Inferior 1 (process 21712) exited normally]
>>>
>>> It would be better to be able to debug ls and/or grep, but I don't know how to
>>> get to subprocesses in gdb.  And I think I have to start with 'gdb bash' in
>>> order for the process substitution to happen.
>>
>> Yeah, subprocess debugging is a problem in GDB.  Given how this works,
>> you can at least take grep out of the picture.  Bash is doing all the
>> lifting, so it's just bash and ls.   Did you try to reproduce this under
>> strace?
> 
> Yes, but there I get an error (even under mintty) for a different reason:
> 
> $ strace -o trace.out ls -lL <(grep bash .bashrc)
> ls: cannot access '/dev/fd/63': No such file or directory
> 
> The strace output shows a call to fhandler_process::exists on /proc/45036/fd/63;
> here 45036 is the PID of 'ls'.  And then I see an EBADF error.  But I think
> what's happening here might be that bash is parsing '<(grep bash .bashrc)' too
> soon, so that '/dev/fd/63' isn't related to the 'ls' command.
> 
> By the way, I've just tried a different experiment, in which I simplify the ls
> command to 'ls <(grep bash .bashrc)'.  When I run this under xterm, I get the
> broken pipe error 98% of the time or more.  But it's fine under mintty.

I think I may have more-or-less figured out what's going on.  The "broken pipe" 
error simply means that ls has exited before grep has finished writing.  So grep 
is writing to a pipe that has no readers.  If I replace 'ls' by 'cat', I don't 
get any errors.

It remains a mystery to me why I was seeing the broken pipe only under X11, but 
I don't see any reason to think there's a problem.

Ken
\0ТÒÐÐ¥\a&ö&ÆVÒ\a&W\x06÷'G3¢\x02\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒ÷\a&ö&ÆV×2æ‡FÖÀФd\x15\x13¢\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒöf\x17\x12ðФFö7VÖVçF\x17F–öã¢\x02\x02\x02\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒöFö72æ‡FÖÀÐ¥Vç7V'67&–&R\x06–æfó¢\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒöÖÂò7Vç7V'67&–&R×6–×\x06ÆPРÐ

Re: Regression (last snapshot)

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2652 bytes --]

On Jul 29 13:18, Ken Brown wrote:
> On 7/29/2019 4:45 AM, Corinna Vinschen wrote:
> > On Jul 27 15:24, Ken Brown wrote:
> >> On 7/27/2019 6:21 AM, Houder wrote:
> >>> On Fri, 26 Jul 2019 22:12:43, Ken Brown  wrote:
> >>>
> >>>> On 7/22/2019 2:47 PM, Houder wrote:
> >>>
> >>>>> The specific regression as reported, has gone.
> >>>>>
> >>>>> 64-@@ uname -a
> >>>>> CYGWIN_NT-6.1 Seven 3.1.0s(0.339/5/3) 2019-07-22 16:43 x86_64 Cygwin
> >>>>> 64-@@ ls -lL <(grep bash .bashrc)
> >>>>> pr-------- 1 Henri None 0 Jul 22 20:36 /dev/fd/63
> >>>>
> >>> Over all the behavior has simularity w/ the error reported by David Karr:
> >>>
> >>>       https://cygwin.com/ml/cygwin/2019-07/msg00150.html
> >>>       ( Piping input from subprocess loses track of temp file )
> >> [...]
> >> Repeating this under the 20190722 or 20190725 snapshots gave slightly worse
> >> results (close to 500 errors).  Using my own unoptimized build of cygwin1.dll,
> >> the error count went up to about 650.
> > 
> > I just tried this myself and I can't reproduce the problem.  1000 runs,
> > no error.
> 
> Interesting.  And you ran this under X11 in an xterm window?

I didn't, but I just did and the result is the same, no errors.

> >> I tried running under gdb, but I couldn't get grep to fail.  More precisely, I
> >> didn't see an error message from grep.  Every run looked like this:
> >>
> >> $ gdb bash
> >> GNU gdb (GDB) (Cygwin 8.2.1-1) 8.2.1
> >> [...]
> >> (gdb) r -c 'ls -lL <(grep bash .bashrc)'
> >> Starting program: /usr/bin/bash -c 'ls -lL <(grep bash .bashrc)'
> >> [...]
> >> pr-------- 1 kbrown None 0 2019-07-27 11:07 /dev/fd/63
> >> [...]
> >> [Inferior 1 (process 21712) exited normally]
> >>
> >> It would be better to be able to debug ls and/or grep, but I don't know how to
> >> get to subprocesses in gdb.  And I think I have to start with 'gdb bash' in
> >> order for the process substitution to happen.
> > 
> > Yeah, subprocess debugging is a problem in GDB.  Given how this works,
> > you can at least take grep out of the picture.  Bash is doing all the
> > lifting, so it's just bash and ls.   Did you try to reproduce this under
> > strace?
> 
> Yes, but there I get an error (even under mintty) for a different reason:
> 
> $ strace -o trace.out ls -lL <(grep bash .bashrc) 
> ls: cannot access '/dev/fd/63': No such file or directory

No, please run bash:

  strace -o trace.out bash -c 'ls -lL <(grep bash .bashrc)'

Otherwise there's no process actually creating the pipe, given the <()
expression is a bash expression.


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Regression (last snapshot)

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1541 bytes --]

On Jul 29 13:35, Ken Brown wrote:
> On 7/29/2019 9:18 AM, Ken Brown wrote:
> > On 7/29/2019 4:45 AM, Corinna Vinschen wrote:
> >>    Did you try to reproduce this under
> >> strace?
> > 
> > Yes, but there I get an error (even under mintty) for a different reason:
> > 
> > $ strace -o trace.out ls -lL <(grep bash .bashrc)
> > ls: cannot access '/dev/fd/63': No such file or directory
> > 
> > The strace output shows a call to fhandler_process::exists on /proc/45036/fd/63;
> > here 45036 is the PID of 'ls'.  And then I see an EBADF error.  But I think
> > what's happening here might be that bash is parsing '<(grep bash .bashrc)' too
> > soon, so that '/dev/fd/63' isn't related to the 'ls' command.
> > 
> > By the way, I've just tried a different experiment, in which I simplify the ls
> > command to 'ls <(grep bash .bashrc)'.  When I run this under xterm, I get the
> > broken pipe error 98% of the time or more.  But it's fine under mintty.
> 
> I think I may have more-or-less figured out what's going on.  The "broken pipe" 
> error simply means that ls has exited before grep has finished writing.  So grep 
> is writing to a pipe that has no readers.  If I replace 'ls' by 'cat', I don't 
> get any errors.

Yeah, but it's mainly because this got started wrongly.  bash needs to run
under strace as well, otherwise you don't have the connection to the process
actually creating the fifo.  My previous reply wasn't very clear on that,
sorry.


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Regression (last snapshot)

[Ken Brown]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 3247 bytes --]

On 7/29/2019 9:47 AM, Corinna Vinschen wrote:
> On Jul 29 13:18, Ken Brown wrote:
>> On 7/29/2019 4:45 AM, Corinna Vinschen wrote:
>>> On Jul 27 15:24, Ken Brown wrote:
>>>> On 7/27/2019 6:21 AM, Houder wrote:
>>>>> On Fri, 26 Jul 2019 22:12:43, Ken Brown  wrote:
>>>>>
>>>>>> On 7/22/2019 2:47 PM, Houder wrote:
>>>>>
>>>>>>> The specific regression as reported, has gone.
>>>>>>>
>>>>>>> 64-@@ uname -a
>>>>>>> CYGWIN_NT-6.1 Seven 3.1.0s(0.339/5/3) 2019-07-22 16:43 x86_64 Cygwin
>>>>>>> 64-@@ ls -lL <(grep bash .bashrc)
>>>>>>> pr-------- 1 Henri None 0 Jul 22 20:36 /dev/fd/63
>>>>>>
>>>>> Over all the behavior has simularity w/ the error reported by David Karr:
>>>>>
>>>>>        https://cygwin.com/ml/cygwin/2019-07/msg00150.html
>>>>>        ( Piping input from subprocess loses track of temp file )
>>>> [...]
>>>> Repeating this under the 20190722 or 20190725 snapshots gave slightly worse
>>>> results (close to 500 errors).  Using my own unoptimized build of cygwin1.dll,
>>>> the error count went up to about 650.
>>>
>>> I just tried this myself and I can't reproduce the problem.  1000 runs,
>>> no error.
>>
>> Interesting.  And you ran this under X11 in an xterm window?
> 
> I didn't, but I just did and the result is the same, no errors.
> 
>>>> I tried running under gdb, but I couldn't get grep to fail.  More precisely, I
>>>> didn't see an error message from grep.  Every run looked like this:
>>>>
>>>> $ gdb bash
>>>> GNU gdb (GDB) (Cygwin 8.2.1-1) 8.2.1
>>>> [...]
>>>> (gdb) r -c 'ls -lL <(grep bash .bashrc)'
>>>> Starting program: /usr/bin/bash -c 'ls -lL <(grep bash .bashrc)'
>>>> [...]
>>>> pr-------- 1 kbrown None 0 2019-07-27 11:07 /dev/fd/63
>>>> [...]
>>>> [Inferior 1 (process 21712) exited normally]
>>>>
>>>> It would be better to be able to debug ls and/or grep, but I don't know how to
>>>> get to subprocesses in gdb.  And I think I have to start with 'gdb bash' in
>>>> order for the process substitution to happen.
>>>
>>> Yeah, subprocess debugging is a problem in GDB.  Given how this works,
>>> you can at least take grep out of the picture.  Bash is doing all the
>>> lifting, so it's just bash and ls.   Did you try to reproduce this under
>>> strace?
>>
>> Yes, but there I get an error (even under mintty) for a different reason:
>>
>> $ strace -o trace.out ls -lL <(grep bash .bashrc)
>> ls: cannot access '/dev/fd/63': No such file or directory
> 
> No, please run bash:
> 
>    strace -o trace.out bash -c 'ls -lL <(grep bash .bashrc)'
> 
> Otherwise there's no process actually creating the pipe, given the <()
> expression is a bash expression.

Yes, of course.  I should have realized this since it's exactly what I did under 
gdb.  Anyway, the result is the same as it was under gdb: If I run the command 
under strace, I don't see the broken pipe error.

Is it possible that debugging causes an fd to the read end of the pipe to stay 
open longer, thereby preventing the error?

Ken
\0ТÒÐÐ¥\a&ö&ÆVÒ\a&W\x06÷'G3¢\x02\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒ÷\a&ö&ÆV×2æ‡FÖÀФd\x15\x13¢\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒöf\x17\x12ðФFö7VÖVçF\x17F–öã¢\x02\x02\x02\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒöFö72æ‡FÖÀÐ¥Vç7V'67&–&R\x06–æfó¢\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒöÖÂò7Vç7V'67&–&R×6–×\x06ÆPРÐ

Re: Regression (last snapshot)

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1043 bytes --]

On Jul 29 14:26, Ken Brown wrote:
> On 7/29/2019 9:47 AM, Corinna Vinschen wrote:
> > On Jul 29 13:18, Ken Brown wrote:
> >> $ strace -o trace.out ls -lL <(grep bash .bashrc)
> >> ls: cannot access '/dev/fd/63': No such file or directory
> > 
> > No, please run bash:
> > 
> >    strace -o trace.out bash -c 'ls -lL <(grep bash .bashrc)'
> > 
> > Otherwise there's no process actually creating the pipe, given the <()
> > expression is a bash expression.
> 
> Yes, of course.  I should have realized this since it's exactly what I
> did under gdb.  Anyway, the result is the same as it was under gdb: If
> I run the command under strace, I don't see the broken pipe error.
> 
> Is it possible that debugging causes an fd to the read end of the pipe
> to stay open longer, thereby preventing the error?

The fact that you observe it sporadically points to a race condition.
Debugging serializes stuff usually running in parallel, potentially
eliminating the race.


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Regression (last snapshot)

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1410 bytes --]

On Jul 29 17:23, Corinna Vinschen wrote:
> On Jul 29 14:26, Ken Brown wrote:
> > On 7/29/2019 9:47 AM, Corinna Vinschen wrote:
> > > On Jul 29 13:18, Ken Brown wrote:
> > >> $ strace -o trace.out ls -lL <(grep bash .bashrc)
> > >> ls: cannot access '/dev/fd/63': No such file or directory
> > > 
> > > No, please run bash:
> > > 
> > >    strace -o trace.out bash -c 'ls -lL <(grep bash .bashrc)'
> > > 
> > > Otherwise there's no process actually creating the pipe, given the <()
> > > expression is a bash expression.
> > 
> > Yes, of course.  I should have realized this since it's exactly what I
> > did under gdb.  Anyway, the result is the same as it was under gdb: If
> > I run the command under strace, I don't see the broken pipe error.
> > 
> > Is it possible that debugging causes an fd to the read end of the pipe
> > to stay open longer, thereby preventing the error?
> 
> The fact that you observe it sporadically points to a race condition.
> Debugging serializes stuff usually running in parallel, potentially
> eliminating the race.

Is there any chance this is a BLODA problem?  If /dev/fd/63 doesn't
exist in ls, it would mean ls didn't inherit the FIFO, which sounds
very unlikely.  I ran the ls call in a loop a couple of 1000 times,
even in parallel windows including xterm and I just can't reproduce.


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Regression (last snapshot)

[Ken Brown]

On 7/29/2019 11:40 AM, Corinna Vinschen wrote:
> On Jul 29 17:23, Corinna Vinschen wrote:
>> On Jul 29 14:26, Ken Brown wrote:
>>> On 7/29/2019 9:47 AM, Corinna Vinschen wrote:
>>>> On Jul 29 13:18, Ken Brown wrote:
>>>>> $ strace -o trace.out ls -lL <(grep bash .bashrc)
>>>>> ls: cannot access '/dev/fd/63': No such file or directory
>>>>
>>>> No, please run bash:
>>>>
>>>>     strace -o trace.out bash -c 'ls -lL <(grep bash .bashrc)'
>>>>
>>>> Otherwise there's no process actually creating the pipe, given the <()
>>>> expression is a bash expression.
>>>
>>> Yes, of course.  I should have realized this since it's exactly what I
>>> did under gdb.  Anyway, the result is the same as it was under gdb: If
>>> I run the command under strace, I don't see the broken pipe error.
>>>
>>> Is it possible that debugging causes an fd to the read end of the pipe
>>> to stay open longer, thereby preventing the error?
>>
>> The fact that you observe it sporadically points to a race condition.
>> Debugging serializes stuff usually running in parallel, potentially
>> eliminating the race.
> 
> Is there any chance this is a BLODA problem?

I doubt it.  I'm seeing this on two different computers, and I haven't seen any 
other symptoms suggesting BLODA.

>  If /dev/fd/63 doesn't
> exist in ls, it would mean ls didn't inherit the FIFO, which sounds
> very unlikely.

Actually I never saw an ls error saying /dev/fd/63 doesn't exist, except in my 
incorrect run of strace.

Here's the error that I can reproduce easily in xterm:

$ ls <(grep bash .bashrc)
/dev/fd/63
grep: write error: Broken pipe

This happens 98% of the time.  Notice that I used plain 'ls' rather than the 
original 'ls -lL'.  With the latter, I get the broken pipe error 60% of the time 
rather than 98%:

$ ls -lL <(grep bash .bashrc)
pr-------- 1 kbrown None 0 2019-07-29 14:46 /dev/fd/63
grep: write error: Broken pipe

What about the explanation I tried earlier, but perhaps not clearly: ls prints 
/dev/fd/63 and then exits, thereby closing the read end of the pipe, while grep 
(running asynchronously) hasn't finished writing to the write end of the pipe.

The fact that I get the broken pipe error more often with plain 'ls' than with 
'ls -lL' is consistent with that.  And the fact that I get no errors with 'cat 
<(grep bash .bashrc)' is also consistent with it, since cat doesn't exit until 
grep has finished writing.

On the other hand, this doesn't explain why I see the error only under xterm, 
nor does it explain why you can't reproduce it at all.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Regression (last snapshot)

[Ken Brown]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 3668 bytes --]

On 7/29/2019 2:55 PM, Ken Brown wrote:
> On 7/29/2019 11:40 AM, Corinna Vinschen wrote:
>> On Jul 29 17:23, Corinna Vinschen wrote:
>>> On Jul 29 14:26, Ken Brown wrote:
>>>> On 7/29/2019 9:47 AM, Corinna Vinschen wrote:
>>>>> On Jul 29 13:18, Ken Brown wrote:
>>>>>> $ strace -o trace.out ls -lL <(grep bash .bashrc)
>>>>>> ls: cannot access '/dev/fd/63': No such file or directory
>>>>>
>>>>> No, please run bash:
>>>>>
>>>>>      strace -o trace.out bash -c 'ls -lL <(grep bash .bashrc)'
>>>>>
>>>>> Otherwise there's no process actually creating the pipe, given the <()
>>>>> expression is a bash expression.
>>>>
>>>> Yes, of course.  I should have realized this since it's exactly what I
>>>> did under gdb.  Anyway, the result is the same as it was under gdb: If
>>>> I run the command under strace, I don't see the broken pipe error.
>>>>
>>>> Is it possible that debugging causes an fd to the read end of the pipe
>>>> to stay open longer, thereby preventing the error?
>>>
>>> The fact that you observe it sporadically points to a race condition.
>>> Debugging serializes stuff usually running in parallel, potentially
>>> eliminating the race.
>>
>> Is there any chance this is a BLODA problem?
> 
> I doubt it.  I'm seeing this on two different computers, and I haven't seen any
> other symptoms suggesting BLODA.
> 
>>   If /dev/fd/63 doesn't
>> exist in ls, it would mean ls didn't inherit the FIFO, which sounds
>> very unlikely.
> 
> Actually I never saw an ls error saying /dev/fd/63 doesn't exist, except in my
> incorrect run of strace.
> 
> Here's the error that I can reproduce easily in xterm:
> 
> $ ls <(grep bash .bashrc)
> /dev/fd/63
> grep: write error: Broken pipe
> 
> This happens 98% of the time.  Notice that I used plain 'ls' rather than the
> original 'ls -lL'.  With the latter, I get the broken pipe error 60% of the time
> rather than 98%:
> 
> $ ls -lL <(grep bash .bashrc)
> pr-------- 1 kbrown None 0 2019-07-29 14:46 /dev/fd/63
> grep: write error: Broken pipe
> 
> What about the explanation I tried earlier, but perhaps not clearly: ls prints
> /dev/fd/63 and then exits, thereby closing the read end of the pipe, while grep
> (running asynchronously) hasn't finished writing to the write end of the pipe.
> 
> The fact that I get the broken pipe error more often with plain 'ls' than with
> 'ls -lL' is consistent with that.  And the fact that I get no errors with 'cat
> <(grep bash .bashrc)' is also consistent with it, since cat doesn't exit until
> grep has finished writing.
> 
> On the other hand, this doesn't explain why I see the error only under xterm,
> nor does it explain why you can't reproduce it at all.

I've made some progress.  It turns out that the problem only occurs in terminals 
launched from the xwin-xdg-menu tray icon.  I can even launch a mintty window 
from that icon (System Tools -> Cygwin Terminal) and I'll see the problem.  On 
the other hand, I can launch an xterm without using that icon (e.g., 'DISPLAY=:0 
xterm -l&' from a mintty window) and I won't see the problem.

So the issue has something to do with how xwin-xdg-menu launches applications, 
and how that interacts with bash's process substitution.  I've just downloaded 
the xwin-xdg-menu source and will see if I can figure out what's going on.  I've 
also added Jon to the CC in case he has a chance to take a look.

Ken
\x03B‹KCB”\x1c›Ø›\x19[H\x1c™\^[ܝ\x1cΈ\b\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÜ\x1c›Ø›\x19[\Ëš\x1d^[[\x03B‘TNˆ\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÙ˜\KÃB‘^[ØÝ[Y[\x18]\x1a[ÛŽˆ\b\b\b\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÙ^[ØÜËš\x1d^[[\x03B•[œÝXœØÜšX™H\x1a[™›Îˆ\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÛ[\vÈÝ[œÝXœØÜšX™K\Ú[\^[\x19CBƒB

Re: Regression (last snapshot)

[Ken Brown]

On 7/31/2019 11:53 AM, Ken Brown wrote:
> On 7/29/2019 2:55 PM, Ken Brown wrote:
>> Here's the error that I can reproduce easily in xterm:
>>
>> $ ls <(grep bash .bashrc)
>> /dev/fd/63
>> grep: write error: Broken pipe
>>
>> This happens 98% of the time.  Notice that I used plain 'ls' rather than the
>> original 'ls -lL'.  With the latter, I get the broken pipe error 60% of the time
>> rather than 98%:
>>
>> $ ls -lL <(grep bash .bashrc)
>> pr-------- 1 kbrown None 0 2019-07-29 14:46 /dev/fd/63
>> grep: write error: Broken pipe
>>
>> What about the explanation I tried earlier, but perhaps not clearly: ls prints
>> /dev/fd/63 and then exits, thereby closing the read end of the pipe, while grep
>> (running asynchronously) hasn't finished writing to the write end of the pipe.
>>
>> The fact that I get the broken pipe error more often with plain 'ls' than with
>> 'ls -lL' is consistent with that.  And the fact that I get no errors with 'cat
>> <(grep bash .bashrc)' is also consistent with it, since cat doesn't exit until
>> grep has finished writing.
>>
>> On the other hand, this doesn't explain why I see the error only under xterm,
>> nor does it explain why you can't reproduce it at all.
> 
> I've made some progress.  It turns out that the problem only occurs in terminals
> launched from the xwin-xdg-menu tray icon.  I can even launch a mintty window
> from that icon (System Tools -> Cygwin Terminal) and I'll see the problem.  On
> the other hand, I can launch an xterm without using that icon (e.g., 'DISPLAY=:0
> xterm -l&' from a mintty window) and I won't see the problem.
> 
> So the issue has something to do with how xwin-xdg-menu launches applications,
> and how that interacts with bash's process substitution.  I've just downloaded
> the xwin-xdg-menu source and will see if I can figure out what's going on.  I've
> also added Jon to the CC in case he has a chance to take a look.

OK, when xwin-xdg-menu launches an application, it creates two pipes and sets 
the application's stdout and stderr to the write ends of those pipes.  For 
example, here's what I see when I launch mintty from xwin-xdg-menu:

$ pgrep mintty
5375

$ ls -l /proc/5375/fd
total 0
lrwxrwxrwx 1 kbrown None 0 2019-07-31 13:37 0 -> /dev/null
lrwxrwxrwx 1 kbrown None 0 2019-07-31 13:37 1 -> pipe:[38654736160]
lrwxrwxrwx 1 kbrown None 0 2019-07-31 13:37 2 -> pipe:[42949703456]
lrwxrwxrwx 1 kbrown None 0 2019-07-31 13:37 3 -> /dev/ptmx
lrwxrwxrwx 1 kbrown None 0 2019-07-31 13:37 4 -> /dev/windows

These pipes are apparently used for logging.

I don't see how this would explain my observations (grep reporting a broken 
pipe), but it is at least a difference between a terminal started from 
xwin-xdg-menu and an "ordinary" terminal.

Anyway, I don't see any evidence here of a Cygwin bug.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Regression (last snapshot)

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1337 bytes --]

On Jul 31 18:00, Ken Brown wrote:
> On 7/31/2019 11:53 AM, Ken Brown wrote:
> > On 7/29/2019 2:55 PM, Ken Brown wrote:
> >> Here's the error that I can reproduce easily in xterm:
> >>
> >> $ ls <(grep bash .bashrc)
> >> /dev/fd/63
> >> grep: write error: Broken pipe
> >>[...]
> > [...]
> 
> OK, when xwin-xdg-menu launches an application, it creates two pipes and sets 
> the application's stdout and stderr to the write ends of those pipes.  For 
> example, here's what I see when I launch mintty from xwin-xdg-menu:
> 
> $ pgrep mintty
> 5375
> 
> $ ls -l /proc/5375/fd
> total 0
> lrwxrwxrwx 1 kbrown None 0 2019-07-31 13:37 0 -> /dev/null
> lrwxrwxrwx 1 kbrown None 0 2019-07-31 13:37 1 -> pipe:[38654736160]
> lrwxrwxrwx 1 kbrown None 0 2019-07-31 13:37 2 -> pipe:[42949703456]
> lrwxrwxrwx 1 kbrown None 0 2019-07-31 13:37 3 -> /dev/ptmx
> lrwxrwxrwx 1 kbrown None 0 2019-07-31 13:37 4 -> /dev/windows
> 
> These pipes are apparently used for logging.
> 
> I don't see how this would explain my observations (grep reporting a broken 
> pipe), but it is at least a difference between a terminal started from 
> xwin-xdg-menu and an "ordinary" terminal.
> 
> Anyway, I don't see any evidence here of a Cygwin bug.

I'm glad to read that :)


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Regression (last snapshot)

[Houder]

On Wed, 31 Jul 2019 15:53:27, Ken Brown  wrote:

> I've made some progress. It turns out that the problem only occurs in terminals 
> launched from the xwin-xdg-menu tray icon.  I can even launch a mintty window 
> from that icon (System Tools -> Cygwin Terminal) and I'll see the problem.  On 
> the other hand, I can launch an xterm without using that icon (e.g., 'DISPLAY=:0 
> xterm -l&' from a mintty window) and I won't see the problem.

No one confirmed your problem, as described in your initial posting. Did nobody
care, or did the problem only exist at your place?

I became curious ...

I do not use X! (on Cygwin).

Therefore I created a separate (basic) Cygwin installation, where I added xinit,
xorg-server and xterm only!

Next I started xterm as follows:

    64-++ startx /usr/bin/xterm -geometry 140x50+530+180 -- -rootless

The problem occured when using cywin1.dll 0712 (as expected), but not when using
cygwin1.dll 3.0.7 or cygwin1.dll 0722.

I concluded, that your problem does not occur w/ a "basic" X configuration.

Regards,
Henri


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Regression (last snapshot)

[Houder]

On Thu, 01 Aug 2019 12:03:34, Houder  wrote:
> On Wed, 31 Jul 2019 15:53:27, Ken Brown  wrote:
> 
> > I've made some progress. It turns out that the problem only occurs in terminals 
> > launched from the xwin-xdg-menu tray icon.  I can even launch a mintty window 
> > from that icon (System Tools -> Cygwin Terminal) and I'll see the problem.  On 
> > the other hand, I can launch an xterm without using that icon (e.g., 'DISPLAY=:0 
> > xterm -l&' from a mintty window) and I won't see the problem.
> 
> No one confirmed your problem, as described in your initial posting. Did nobody
> care, or did the problem only exist at your place?
> 
> I became curious ...
> 
> I do not use X! (on Cygwin).
> 
> Therefore I created a separate (basic) Cygwin installation, where I added xinit,
> xorg-server and xterm only!
> 
> Next I started xterm as follows:
> 
>     64-++ startx /usr/bin/xterm -geometry 140x50+530+180 -- -rootless
> 
> The problem occured when using cywin1.dll 0712 (as expected), but not when using
> cygwin1.dll 3.0.7 or cygwin1.dll 0722.
> 
> I concluded, that your problem does not occur w/ a "basic" X configuration.

Correction!

I referred to 'ls complaining', not to 'grep complaining' ...

I never noticed grep complaining!

My apologies.

Henri


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Regression (last snapshot)

[Ken Brown]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 2275 bytes --]

On 8/1/2019 6:03 AM, Houder wrote:
> On Wed, 31 Jul 2019 15:53:27, Ken Brown  wrote:
> 
>> I've made some progress. It turns out that the problem only occurs in terminals
>> launched from the xwin-xdg-menu tray icon.  I can even launch a mintty window
>> from that icon (System Tools -> Cygwin Terminal) and I'll see the problem.  On
>> the other hand, I can launch an xterm without using that icon (e.g., 'DISPLAY=:0
>> xterm -l&' from a mintty window) and I won't see the problem.
> 
> No one confirmed your problem, as described in your initial posting. Did nobody
> care, or did the problem only exist at your place?
> 
> I became curious ...
> 
> I do not use X! (on Cygwin).
> 
> Therefore I created a separate (basic) Cygwin installation, where I added xinit,
> xorg-server and xterm only!
> 
> Next I started xterm as follows:
> 
>      64-++ startx /usr/bin/xterm -geometry 140x50+530+180 -- -rootless
> 
> The problem occured when using cywin1.dll 0712 (as expected), but not when using
> cygwin1.dll 3.0.7 or cygwin1.dll 0722.
> 
> I concluded, that your problem does not occur w/ a "basic" X configuration.

As I said, the problem only occurs in terminals started from the xwin-xdg-menu. 
That isn't what you did.  Here are the reproduction instructions again:

1. Start the X server using the XWin Server shortcut in the Start Menu (look 
under Cygwin-X).

2. Locate the xwin-xdg-menu tray icon.  It looks like a black C with a green X 
inside.  If you hover over it, you'll see "X applications menu on :0".  Click on 
that icon, hover over System Tools, and select one of the terminals offered.

3. Try your 'ls' command or some variation of it in that terminal.

On my system I see the broken pipe error from grep half of the time or more if 
use your original test 'ls -lL <(grep bash .bashrc)'.  If I use the simpler 'ls 
<(grep bash .bashrc)', I see the error about 98% of the time.  And if I use the 
even simpler 'echo <(grep bash .bashrc)', I see the error 100% of the time.

Ken
\0ТÒÐÐ¥\a&ö&ÆVÒ\a&W\x06÷'G3¢\x02\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒ÷\a&ö&ÆV×2æ‡FÖÀФd\x15\x13¢\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒöf\x17\x12ðФFö7VÖVçF\x17F–öã¢\x02\x02\x02\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒöFö72æ‡FÖÀÐ¥Vç7V'67&–&R\x06–æfó¢\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒöÖÂò7Vç7V'67&–&R×6–×\x06ÆPРÐ

Re: Regression (last snapshot)

[Jon Turney]

On 31/07/2019 19:00, Ken Brown wrote:
> On 7/31/2019 11:53 AM, Ken Brown wrote:
>> On 7/29/2019 2:55 PM, Ken Brown wrote:
>>> Here's the error that I can reproduce easily in xterm:
>>>
>>> $ ls <(grep bash .bashrc)
>>> /dev/fd/63
>>> grep: write error: Broken pipe
>>>
>>> This happens 98% of the time.  Notice that I used plain 'ls' rather than the
>>> original 'ls -lL'.  With the latter, I get the broken pipe error 60% of the time
>>> rather than 98%:
>>>
>>> $ ls -lL <(grep bash .bashrc)
>>> pr-------- 1 kbrown None 0 2019-07-29 14:46 /dev/fd/63
>>> grep: write error: Broken pipe
>>>
>>> What about the explanation I tried earlier, but perhaps not clearly: ls prints
>>> /dev/fd/63 and then exits, thereby closing the read end of the pipe, while grep
>>> (running asynchronously) hasn't finished writing to the write end of the pipe.
>>>
>>> The fact that I get the broken pipe error more often with plain 'ls' than with
>>> 'ls -lL' is consistent with that.  And the fact that I get no errors with 'cat
>>> <(grep bash .bashrc)' is also consistent with it, since cat doesn't exit until
>>> grep has finished writing.
>>>
>>> On the other hand, this doesn't explain why I see the error only under xterm,
>>> nor does it explain why you can't reproduce it at all.
>>
>> I've made some progress.  It turns out that the problem only occurs in terminals
>> launched from the xwin-xdg-menu tray icon.  I can even launch a mintty window
>> from that icon (System Tools -> Cygwin Terminal) and I'll see the problem.  On
>> the other hand, I can launch an xterm without using that icon (e.g., 'DISPLAY=:0
>> xterm -l&' from a mintty window) and I won't see the problem.
>>
>> So the issue has something to do with how xwin-xdg-menu launches applications,
>> and how that interacts with bash's process substitution.  I've just downloaded
>> the xwin-xdg-menu source and will see if I can figure out what's going on.  I've
>> also added Jon to the CC in case he has a chance to take a look.
> 
> OK, when xwin-xdg-menu launches an application, it creates two pipes and sets
> the application's stdout and stderr to the write ends of those pipes.  For
> example, here's what I see when I launch mintty from xwin-xdg-menu:
> 
> $ pgrep mintty
> 5375
> 
> $ ls -l /proc/5375/fd
> total 0
> lrwxrwxrwx 1 kbrown None 0 2019-07-31 13:37 0 -> /dev/null
> lrwxrwxrwx 1 kbrown None 0 2019-07-31 13:37 1 -> pipe:[38654736160]
> lrwxrwxrwx 1 kbrown None 0 2019-07-31 13:37 2 -> pipe:[42949703456]
> lrwxrwxrwx 1 kbrown None 0 2019-07-31 13:37 3 -> /dev/ptmx
> lrwxrwxrwx 1 kbrown None 0 2019-07-31 13:37 4 -> /dev/windows
> 
> These pipes are apparently used for logging.

That's correct.  Writes to those pipes are drained into the session 
logfile (~/.xsession-errors), which you can view with the 'View logfile' 
menu item.

> I don't see how this would explain my observations (grep reporting a broken
> pipe), but it is at least a difference between a terminal started from
> xwin-xdg-menu and an "ordinary" terminal.
> 
> Anyway, I don't see any evidence here of a Cygwin bug.

I'm not so sure: these pipes for stdout/stderr of the mintty process 
shouldn't have any influence at all on the child bash process (since 
it's stdin/stdout/stderr is the slave end of a pty), but apparently they 
somehow do?

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Regression (last snapshot)

[Houder]

On Thu, 1 Aug 2019 12:19:55, Ken Brown  wrote:

> As I said, the problem only occurs in terminals started from the xwin-xdg-menu. 
> That isn't what you did.  Here are the reproduction instructions again:

Entendu!

> 1. Start the X server using the XWin Server shortcut in the Start Menu (look 
> under Cygwin-X).

I choose not to clobber the Start Menu (while installing) ...

> 2. Locate the xwin-xdg-menu tray icon.  It looks like a black C with a green X 
> inside.  If you hover over it, you'll see "X applications menu on :0".  Click on 
> that icon, hover over System Tools, and select one of the terminals offered.
> 
> 3. Try your 'ls' command or some variation of it in that terminal.

 - started login shell (either using MinTTY or a DOSbox)
 - invoked startxwin
   (which is the same as starting the server from the Start Menu)

Had to fiddle w/ the notification area (tray) before the required icons appeared.

Next proceeded as you instructed above.

Result: grep complains ("grep: Write error: Broken pipe").

Problem confirmed.

Henri


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Regression (last snapshot)

[Ken Brown]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 4388 bytes --]

On 8/1/2019 10:27 AM, Jon Turney wrote:
> On 31/07/2019 19:00, Ken Brown wrote:
>> On 7/31/2019 11:53 AM, Ken Brown wrote:
>>> On 7/29/2019 2:55 PM, Ken Brown wrote:
>>>> Here's the error that I can reproduce easily in xterm:
>>>>
>>>> $ ls <(grep bash .bashrc)
>>>> /dev/fd/63
>>>> grep: write error: Broken pipe
>>>>
>>>> This happens 98% of the time.  Notice that I used plain 'ls' rather 
>>>> than the
>>>> original 'ls -lL'.  With the latter, I get the broken pipe error 60% 
>>>> of the time
>>>> rather than 98%:
>>>>
>>>> $ ls -lL <(grep bash .bashrc)
>>>> pr-------- 1 kbrown None 0 2019-07-29 14:46 /dev/fd/63
>>>> grep: write error: Broken pipe
>>>>
>>>> What about the explanation I tried earlier, but perhaps not clearly: 
>>>> ls prints
>>>> /dev/fd/63 and then exits, thereby closing the read end of the pipe, 
>>>> while grep
>>>> (running asynchronously) hasn't finished writing to the write end of 
>>>> the pipe.
>>>>
>>>> The fact that I get the broken pipe error more often with plain 'ls' 
>>>> than with
>>>> 'ls -lL' is consistent with that.  And the fact that I get no errors 
>>>> with 'cat
>>>> <(grep bash .bashrc)' is also consistent with it, since cat doesn't 
>>>> exit until
>>>> grep has finished writing.
>>>>
>>>> On the other hand, this doesn't explain why I see the error only 
>>>> under xterm,
>>>> nor does it explain why you can't reproduce it at all.
>>>
>>> I've made some progress.  It turns out that the problem only occurs 
>>> in terminals
>>> launched from the xwin-xdg-menu tray icon.  I can even launch a 
>>> mintty window
>>> from that icon (System Tools -> Cygwin Terminal) and I'll see the 
>>> problem.  On
>>> the other hand, I can launch an xterm without using that icon (e.g., 
>>> 'DISPLAY=:0
>>> xterm -l&' from a mintty window) and I won't see the problem.
>>>
>>> So the issue has something to do with how xwin-xdg-menu launches 
>>> applications,
>>> and how that interacts with bash's process substitution.  I've just 
>>> downloaded
>>> the xwin-xdg-menu source and will see if I can figure out what's 
>>> going on.  I've
>>> also added Jon to the CC in case he has a chance to take a look.
>>
>> OK, when xwin-xdg-menu launches an application, it creates two pipes 
>> and sets
>> the application's stdout and stderr to the write ends of those pipes.  
>> For
>> example, here's what I see when I launch mintty from xwin-xdg-menu:
>>
>> $ pgrep mintty
>> 5375
>>
>> $ ls -l /proc/5375/fd
>> total 0
>> lrwxrwxrwx 1 kbrown None 0 2019-07-31 13:37 0 -> /dev/null
>> lrwxrwxrwx 1 kbrown None 0 2019-07-31 13:37 1 -> pipe:[38654736160]
>> lrwxrwxrwx 1 kbrown None 0 2019-07-31 13:37 2 -> pipe:[42949703456]
>> lrwxrwxrwx 1 kbrown None 0 2019-07-31 13:37 3 -> /dev/ptmx
>> lrwxrwxrwx 1 kbrown None 0 2019-07-31 13:37 4 -> /dev/windows
>>
>> These pipes are apparently used for logging.
> 
> That's correct.  Writes to those pipes are drained into the session 
> logfile (~/.xsession-errors), which you can view with the 'View logfile' 
> menu item.
> 
>> I don't see how this would explain my observations (grep reporting a 
>> broken
>> pipe), but it is at least a difference between a terminal started from
>> xwin-xdg-menu and an "ordinary" terminal.
>>
>> Anyway, I don't see any evidence here of a Cygwin bug.
> 
> I'm not so sure: these pipes for stdout/stderr of the mintty process 
> shouldn't have any influence at all on the child bash process (since 
> it's stdin/stdout/stderr is the slave end of a pty), but apparently they 
> somehow do?

Well, I can't be sure that the pipes are responsible.  It's just that 
the existence of the pipes is the only difference I could spot between 
an ordinary terminal and a terminal started from xwin-xdg-menu.

Is it possible that the logging somehow slows things down or changes the 
buffering, so that the grep process takes longer to complete?  This 
would be consistent with my theory that the broken pipe error doesn't 
really represent a bug, but rather it reflects the fact that ls exits 
before grep has finished writing.

Ken
\x03B‹KCB”\x1c›Ø›\x19[H\x1c™\^[ܝ\x1cΈ\b\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÜ\x1c›Ø›\x19[\Ëš\x1d^[[\x03B‘TNˆ\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÙ˜\KÃB‘^[ØÝ[Y[\x18]\x1a[ÛŽˆ\b\b\b\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÙ^[ØÜËš\x1d^[[\x03B•[œÝXœØÜšX™H\x1a[™›Îˆ\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÛ[\vÈÝ[œÝXœØÜšX™K\Ú[\^[\x19CBƒB

Re: Regression (last snapshot)

[Eric Blake]

[-- Attachment #1.1: Type: text/plain, Size: 1238 bytes --]

On 8/1/19 10:30 AM, Ken Brown wrote:

>>> OK, when xwin-xdg-menu launches an application, it creates two pipes 
>>> and sets
>>> the application's stdout and stderr to the write ends of those pipes.  

> Well, I can't be sure that the pipes are responsible.  It's just that 
> the existence of the pipes is the only difference I could spot between 
> an ordinary terminal and a terminal started from xwin-xdg-menu.
> 
> Is it possible that the logging somehow slows things down or changes the 
> buffering, so that the grep process takes longer to complete?  This 
> would be consistent with my theory that the broken pipe error doesn't 
> really represent a bug, but rather it reflects the fact that ls exits 
> before grep has finished writing.

Could it be a case of xwin-xdg-menu calling signal(SIGPIPE, SIG_IGN) or
similar, and accidentally letting grep inherit the ignored SIGPIPE?
When SIGPIPE is not ignored, grep's failure to write to a pipe causes
termination before the failed write completes; but when it is ignored,
grep sees EPIPE from the failed write and reports that.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: Regression (last snapshot)

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1116 bytes --]

On Aug  1 10:38, Eric Blake wrote:
> On 8/1/19 10:30 AM, Ken Brown wrote:
> 
> >>> OK, when xwin-xdg-menu launches an application, it creates two pipes 
> >>> and sets
> >>> the application's stdout and stderr to the write ends of those pipes.  
> 
> > Well, I can't be sure that the pipes are responsible.  It's just that 
> > the existence of the pipes is the only difference I could spot between 
> > an ordinary terminal and a terminal started from xwin-xdg-menu.
> > 
> > Is it possible that the logging somehow slows things down or changes the 
> > buffering, so that the grep process takes longer to complete?  This 
> > would be consistent with my theory that the broken pipe error doesn't 
> > really represent a bug, but rather it reflects the fact that ls exits 
> > before grep has finished writing.
> 
> Could it be a case of xwin-xdg-menu calling signal(SIGPIPE, SIG_IGN) or
> similar, and accidentally letting grep inherit the ignored SIGPIPE?

execve doesn't propagate the signal dispositions, they get reset to
default.


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Regression (last snapshot)

[Ken Brown]

On 8/1/2019 12:04 PM, Corinna Vinschen wrote:
> On Aug  1 10:38, Eric Blake wrote:
>> On 8/1/19 10:30 AM, Ken Brown wrote:
>>
>>>>> OK, when xwin-xdg-menu launches an application, it creates two pipes
>>>>> and sets
>>>>> the application's stdout and stderr to the write ends of those pipes.
>>
>>> Well, I can't be sure that the pipes are responsible.  It's just that
>>> the existence of the pipes is the only difference I could spot between
>>> an ordinary terminal and a terminal started from xwin-xdg-menu.
>>>
>>> Is it possible that the logging somehow slows things down or changes the
>>> buffering, so that the grep process takes longer to complete?  This
>>> would be consistent with my theory that the broken pipe error doesn't
>>> really represent a bug, but rather it reflects the fact that ls exits
>>> before grep has finished writing.
>>
>> Could it be a case of xwin-xdg-menu calling signal(SIGPIPE, SIG_IGN) or
>> similar, and accidentally letting grep inherit the ignored SIGPIPE?
> 
> execve doesn't propagate the signal dispositions, they get reset to
> default.

I just realized, as a result of Eric's comment, that the explanation 
I've been pushing is nonsense.

What I've been explaining is why there would be a broken pipe, and 
therefore a SIGPIPE and EPIPE.  But I now see that that's not the issue. 
  The issue is whether grep gets the SIGPIPE and terminates before it 
has a chance to see the EPIPE.

So if grep isn't ignoring SIGPIPE, the only other possibility I can 
think of is that grep isn't receiving SIGPIPE, or at least that there's 
a delay before it receives it.  Why would that happen only in terminals 
started by xwin-xdg-menu?

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Regression (last snapshot)

[Ken Brown]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 2378 bytes --]

On 8/1/2019 5:17 PM, Ken Brown wrote:
> On 8/1/2019 12:04 PM, Corinna Vinschen wrote:
>> On Aug  1 10:38, Eric Blake wrote:
>>> On 8/1/19 10:30 AM, Ken Brown wrote:
>>>
>>>>>> OK, when xwin-xdg-menu launches an application, it creates two pipes
>>>>>> and sets
>>>>>> the application's stdout and stderr to the write ends of those pipes.
>>>
>>>> Well, I can't be sure that the pipes are responsible.  It's just that
>>>> the existence of the pipes is the only difference I could spot between
>>>> an ordinary terminal and a terminal started from xwin-xdg-menu.
>>>>
>>>> Is it possible that the logging somehow slows things down or changes the
>>>> buffering, so that the grep process takes longer to complete?  This
>>>> would be consistent with my theory that the broken pipe error doesn't
>>>> really represent a bug, but rather it reflects the fact that ls exits
>>>> before grep has finished writing.
>>>
>>> Could it be a case of xwin-xdg-menu calling signal(SIGPIPE, SIG_IGN) or
>>> similar, and accidentally letting grep inherit the ignored SIGPIPE?
>>
>> execve doesn't propagate the signal dispositions, they get reset to
>> default.
> 
> I just realized, as a result of Eric's comment, that the explanation
> I've been pushing is nonsense.
> 
> What I've been explaining is why there would be a broken pipe, and
> therefore a SIGPIPE and EPIPE.  But I now see that that's not the issue.
>    The issue is whether grep gets the SIGPIPE and terminates before it
> has a chance to see the EPIPE.
> 
> So if grep isn't ignoring SIGPIPE, the only other possibility I can
> think of is that grep isn't receiving SIGPIPE, or at least that there's
> a delay before it receives it.  Why would that happen only in terminals
> started by xwin-xdg-menu?

I just built a version of grep in which I added 'signal(SIGPIPE, SIG_DFL)', and 
the error is gone.  So it looks like grep has in fact been receiving SIGPIPE, 
and for some reason it is not using the default signal handler for SIGPIPE in a 
terminal started by xwin-xdg-menu.  Could this be a gtk issue?  Does it mess 
with the signal handlers?

Ken
\x03B‹KCB”\x1c›Ø›\x19[H\x1c™\^[ܝ\x1cΈ\b\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÜ\x1c›Ø›\x19[\Ëš\x1d^[[\x03B‘TNˆ\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÙ˜\KÃB‘^[ØÝ[Y[\x18]\x1a[ÛŽˆ\b\b\b\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÙ^[ØÜËš\x1d^[[\x03B•[œÝXœØÜšX™H\x1a[™›Îˆ\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÛ[\vÈÝ[œÝXœØÜšX™K\Ú[\^[\x19CBƒB

Re: Regression (last snapshot)

[Ken Brown]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 2396 bytes --]

On 8/1/2019 10:32 PM, Ken Brown wrote:
> On 8/1/2019 5:17 PM, Ken Brown wrote:
>> On 8/1/2019 12:04 PM, Corinna Vinschen wrote:
>>> On Aug  1 10:38, Eric Blake wrote:
>>>> Could it be a case of xwin-xdg-menu calling signal(SIGPIPE, SIG_IGN) or
>>>> similar, and accidentally letting grep inherit the ignored SIGPIPE?
>>>
>>> execve doesn't propagate the signal dispositions, they get reset to
>>> default.
>>
> I just built a version of grep in which I added 'signal(SIGPIPE, SIG_DFL)', and
> the error is gone.  So it looks like grep has in fact been receiving SIGPIPE,
> and for some reason it is not using the default signal handler for SIGPIPE in a
> terminal started by xwin-xdg-menu.  Could this be a gtk issue?  Does it mess
> with the signal handlers?

I think I've finally got it.

First of all, here's what POSIX says about signal handlers after an exec:

"Signals set to the default action (SIG_DFL) in the calling process 
image shall be set to the default action in the new process image. 
Except for SIGCHLD, signals set to be ignored (SIG_IGN) by the calling 
process image shall be set to be ignored by the new process image. 
Signals set to be caught by the calling process image shall be set to 
the default action in the new process image (see <signal.h>)."

Second, here's a quote from the GTK+ documentation for gtk_init():

"Since 2.18, GTK+ calls signal (SIGPIPE, SIG_IGN) during initialization, 
to ignore SIGPIPE signals, since these are almost never wanted in 
graphical applications. If you do need to handle SIGPIPE for some 
reason, reset the handler after gtk_init(), but notice that other 
libraries (e.g. libdbus or gvfs) might do similar things."

Third, xwin-xdg-menu calls gtk_init() near the beginning of main().

Putting this all together, Eric's explanation is indeed correct.  All 
processes created by xwin-xdg-menu via fork/exec inherit the property of 
ignoring SIGPIPE.

I don't know if this is a bug, but it certainly leads to surprising 
behavior.  Jon, maybe xwin-xdg-menu needs to call signal(SIGPIPE, 
SIG_DFL) either after calling gtk_init() or before calling exec()?

Ken
\x03B‹KCB”\x1c›Ø›\x19[H\x1c™\^[ܝ\x1cΈ\b\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÜ\x1c›Ø›\x19[\Ëš\x1d^[[\x03B‘TNˆ\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÙ˜\KÃB‘^[ØÝ[Y[\x18]\x1a[ÛŽˆ\b\b\b\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÙ^[ØÜËš\x1d^[[\x03B•[œÝXœØÜšX™H\x1a[™›Îˆ\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÛ[\vÈÝ[œÝXœØÜšX™K\Ú[\^[\x19CBƒB

Re: Regression (last snapshot)

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1573 bytes --]

On Aug  2 14:34, Ken Brown wrote:
> On 8/1/2019 10:32 PM, Ken Brown wrote:
> > On 8/1/2019 5:17 PM, Ken Brown wrote:
> >> On 8/1/2019 12:04 PM, Corinna Vinschen wrote:
> >>> On Aug  1 10:38, Eric Blake wrote:
> >>>> Could it be a case of xwin-xdg-menu calling signal(SIGPIPE, SIG_IGN) or
> >>>> similar, and accidentally letting grep inherit the ignored SIGPIPE?
> >>>
> >>> execve doesn't propagate the signal dispositions, they get reset to
> >>> default.
> >>
> > I just built a version of grep in which I added 'signal(SIGPIPE, SIG_DFL)', and
> > the error is gone.  So it looks like grep has in fact been receiving SIGPIPE,
> > and for some reason it is not using the default signal handler for SIGPIPE in a
> > terminal started by xwin-xdg-menu.  Could this be a gtk issue?  Does it mess
> > with the signal handlers?
> 
> I think I've finally got it.
> 
> First of all, here's what POSIX says about signal handlers after an exec:
> 
> "Signals set to the default action (SIG_DFL) in the calling process 
> image shall be set to the default action in the new process image. 
> Except for SIGCHLD, signals set to be ignored (SIG_IGN) by the calling 
> process image shall be set to be ignored by the new process image. 
> Signals set to be caught by the calling process image shall be set to 
> the default action in the new process image (see <signal.h>)."

Oh, I see.  I misread the exceve man page.  Only signals which are caught
will be reset to SIG_DFL.  Sorry for the noise.


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Regression (last snapshot)

[Brian Inglis]

On 2019-08-02 08:34, Ken Brown wrote:
> On 8/1/2019 10:32 PM, Ken Brown wrote:
>> On 8/1/2019 5:17 PM, Ken Brown wrote:
>>> On 8/1/2019 12:04 PM, Corinna Vinschen wrote:
>>>> On Aug  1 10:38, Eric Blake wrote:
>>>>> Could it be a case of xwin-xdg-menu calling signal(SIGPIPE, SIG_IGN) or
>>>>> similar, and accidentally letting grep inherit the ignored SIGPIPE?
>>>>
>>>> execve doesn't propagate the signal dispositions, they get reset to
>>>> default.
>>>
>> I just built a version of grep in which I added 'signal(SIGPIPE, SIG_DFL)', and
>> the error is gone.  So it looks like grep has in fact been receiving SIGPIPE,
>> and for some reason it is not using the default signal handler for SIGPIPE in a
>> terminal started by xwin-xdg-menu.  Could this be a gtk issue?  Does it mess
>> with the signal handlers?
> 
> I think I've finally got it.
> 
> First of all, here's what POSIX says about signal handlers after an exec:
> 
> "Signals set to the default action (SIG_DFL) in the calling process 
> image shall be set to the default action in the new process image. 
> Except for SIGCHLD, signals set to be ignored (SIG_IGN) by the calling 
> process image shall be set to be ignored by the new process image. 
> Signals set to be caught by the calling process image shall be set to 
> the default action in the new process image (see <signal.h>)."
> 
> Second, here's a quote from the GTK+ documentation for gtk_init():
> 
> "Since 2.18, GTK+ calls signal (SIGPIPE, SIG_IGN) during initialization, 
> to ignore SIGPIPE signals, since these are almost never wanted in 
> graphical applications. If you do need to handle SIGPIPE for some 
> reason, reset the handler after gtk_init(), but notice that other 
> libraries (e.g. libdbus or gvfs) might do similar things."
> 
> Third, xwin-xdg-menu calls gtk_init() near the beginning of main().
> 
> Putting this all together, Eric's explanation is indeed correct.  All 
> processes created by xwin-xdg-menu via fork/exec inherit the property of 
> ignoring SIGPIPE.
> 
> I don't know if this is a bug, but it certainly leads to surprising 
> behavior.  Jon, maybe xwin-xdg-menu needs to call signal(SIGPIPE, 
> SIG_DFL) either after calling gtk_init() or before calling exec()?

How does that relate to this only happening in the latest snapshot, and not in
the current release, or any Linux system?
Has it been tried under Linux/xterm/bash?
I would certainly expect any shell (or any other program handling pipes) to set
or reset SIGPIPE handling, rather than accept any default.

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Regression (last snapshot)

[Ken Brown]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 3539 bytes --]

On 8/2/2019 5:26 PM, Brian Inglis wrote:
> On 2019-08-02 08:34, Ken Brown wrote:
>> On 8/1/2019 10:32 PM, Ken Brown wrote:
>>> On 8/1/2019 5:17 PM, Ken Brown wrote:
>>>> On 8/1/2019 12:04 PM, Corinna Vinschen wrote:
>>>>> On Aug  1 10:38, Eric Blake wrote:
>>>>>> Could it be a case of xwin-xdg-menu calling signal(SIGPIPE, SIG_IGN) or
>>>>>> similar, and accidentally letting grep inherit the ignored SIGPIPE?
>>>>>
>>>>> execve doesn't propagate the signal dispositions, they get reset to
>>>>> default.
>>>>
>>> I just built a version of grep in which I added 'signal(SIGPIPE, SIG_DFL)', and
>>> the error is gone.  So it looks like grep has in fact been receiving SIGPIPE,
>>> and for some reason it is not using the default signal handler for SIGPIPE in a
>>> terminal started by xwin-xdg-menu.  Could this be a gtk issue?  Does it mess
>>> with the signal handlers?
>>
>> I think I've finally got it.
>>
>> First of all, here's what POSIX says about signal handlers after an exec:
>>
>> "Signals set to the default action (SIG_DFL) in the calling process
>> image shall be set to the default action in the new process image.
>> Except for SIGCHLD, signals set to be ignored (SIG_IGN) by the calling
>> process image shall be set to be ignored by the new process image.
>> Signals set to be caught by the calling process image shall be set to
>> the default action in the new process image (see <signal.h>)."
>>
>> Second, here's a quote from the GTK+ documentation for gtk_init():
>>
>> "Since 2.18, GTK+ calls signal (SIGPIPE, SIG_IGN) during initialization,
>> to ignore SIGPIPE signals, since these are almost never wanted in
>> graphical applications. If you do need to handle SIGPIPE for some
>> reason, reset the handler after gtk_init(), but notice that other
>> libraries (e.g. libdbus or gvfs) might do similar things."
>>
>> Third, xwin-xdg-menu calls gtk_init() near the beginning of main().
>>
>> Putting this all together, Eric's explanation is indeed correct.  All
>> processes created by xwin-xdg-menu via fork/exec inherit the property of
>> ignoring SIGPIPE.
>>
>> I don't know if this is a bug, but it certainly leads to surprising
>> behavior.  Jon, maybe xwin-xdg-menu needs to call signal(SIGPIPE,
>> SIG_DFL) either after calling gtk_init() or before calling exec()?
> 
> How does that relate to this only happening in the latest snapshot, and not in
> the current release, or any Linux system?

It does happen in the current release, as I said earlier in the thread.

There's no way to test it on Linux.  xwin-xdg-menu is a Cygwin-specific 
program (written by Jon).

> I would certainly expect any shell (or any other program handling pipes) to set
> or reset SIGPIPE handling, rather than accept any default.

Take a look at the bash source code and the grep source code.  You'll 
see that neither one of them does this.  And I don't know why you would 
expect it.

Suppose I write a program that creates a pipe, sets SIGPIPE to be 
ignored, and then creates a grep subprocess with stdout set to the write 
end of that pipe.  By doing so, I've clearly indicated my intention that 
grep should ignore SIGPIPE.  Why should grep overrule my choice?  And 
how would grep even know that it's writing to a pipe?

Ken
\x03B‹KCB”\x1c›Ø›\x19[H\x1c™\^[ܝ\x1cΈ\b\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÜ\x1c›Ø›\x19[\Ëš\x1d^[[\x03B‘TNˆ\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÙ˜\KÃB‘^[ØÝ[Y[\x18]\x1a[ÛŽˆ\b\b\b\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÙ^[ØÜËš\x1d^[[\x03B•[œÝXœØÜšX™H\x1a[™›Îˆ\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÛ[\vÈÝ[œÝXœØÜšX™K\Ú[\^[\x19CBƒB

Re: Regression (last snapshot)

[Eric Blake]

[-- Attachment #1.1: Type: text/plain, Size: 1608 bytes --]

On 8/2/19 4:53 PM, Ken Brown wrote:

>>> Putting this all together, Eric's explanation is indeed correct.  All
>>> processes created by xwin-xdg-menu via fork/exec inherit the property of
>>> ignoring SIGPIPE.
>>>
>>> I don't know if this is a bug, but it certainly leads to surprising
>>> behavior.  Jon, maybe xwin-xdg-menu needs to call signal(SIGPIPE,
>>> SIG_DFL) either after calling gtk_init() or before calling exec()?
>>
>> How does that relate to this only happening in the latest snapshot, and not in
>> the current release, or any Linux system?
> 
> It does happen in the current release, as I said earlier in the thread.
> 
> There's no way to test it on Linux.  xwin-xdg-menu is a Cygwin-specific 
> program (written by Jon).

>> I would certainly expect any shell (or any other program handling pipes) to set
>> or reset SIGPIPE handling, rather than accept any default.
> 
> Take a look at the bash source code and the grep source code.  You'll 
> see that neither one of them does this.  And I don't know why you would 
> expect it.

Worse, POSIX explicitly requires that the shell is unable to reset
SIGPIPE back to SIG_DFL if it was inherited ignored (try it - you CANNOT
use the 'trap' command to undo an inherited ignored SIGPIPE, even though
it can be used to undo signals ignored locally). It is generally
considered bad practice to leak ignored SIGPIPE into a child process,
even if it makes sense in the parent process.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: Regression (last snapshot)

[Brian Inglis]

On 2019-08-02 15:58, Eric Blake wrote:
> On 8/2/19 4:53 PM, Ken Brown wrote:
> 
>>>> Putting this all together, Eric's explanation is indeed correct.  All
>>>> processes created by xwin-xdg-menu via fork/exec inherit the property of
>>>> ignoring SIGPIPE.
>>>>
>>>> I don't know if this is a bug, but it certainly leads to surprising
>>>> behavior.  Jon, maybe xwin-xdg-menu needs to call signal(SIGPIPE,
>>>> SIG_DFL) either after calling gtk_init() or before calling exec()?
>>>
>>> How does that relate to this only happening in the latest snapshot, and not in
>>> the current release, or any Linux system?
>>
>> It does happen in the current release, as I said earlier in the thread.
>>
>> There's no way to test it on Linux.  xwin-xdg-menu is a Cygwin-specific 
>> program (written by Jon).
> 
>>> I would certainly expect any shell (or any other program handling pipes) to set
>>> or reset SIGPIPE handling, rather than accept any default.
>>
>> Take a look at the bash source code and the grep source code.  You'll 
>> see that neither one of them does this.  And I don't know why you would 
>> expect it.

I would expect it in bash as bash creates and manages pipes for functions,
internal and external commands, and when I say handling pipes, I mean creating
pipes for internal child sub-shells and other external processes to use (which I
would not expect to touch SIGPIPE, unless they required specially programmed
handling).

> Worse, POSIX explicitly requires that the shell is unable to reset
> SIGPIPE back to SIG_DFL if it was inherited ignored (try it - you CANNOT
> use the 'trap' command to undo an inherited ignored SIGPIPE, even though
> it can be used to undo signals ignored locally). It is generally
> considered bad practice to leak ignored SIGPIPE into a child process,
> even if it makes sense in the parent process.

Looking at mintty wiki, it mentions pipe handling, so may reset the signal
action, and it mentions that Cygwin emulates ptys using pipes, so winpty is
needed to deal with some Windows programs that require a native console.

Could Cygwin pty emulation with pipes, under an xterm terminal that perhaps does
not reset signal actions, be causing or having problems under some
circumstances, while running external commands in sub-shells without normal
SIGPIPE signal actions?

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Regression (last snapshot)

[Ken Brown]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 887 bytes --]

On 8/2/2019 11:50 PM, Brian Inglis wrote:
> Looking at mintty wiki, it mentions pipe handling, so may reset the signal
> action,

It doesn't.  Look at the mintty source.

> and it mentions that Cygwin emulates ptys using pipes,

Cygwin pty emulation uses Windows named pipes.

> Could Cygwin pty emulation with pipes, under an xterm terminal that perhaps does
> not reset signal actions, be causing or having problems under some
> circumstances, while running external commands in sub-shells without normal
> SIGPIPE signal actions?

I don't see any connection between Cygwin's pty emulation and an application's 
handling of SIGPIPE.

Ken
\x03B‹KCB”\x1c›Ø›\x19[H\x1c™\^[ܝ\x1cΈ\b\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÜ\x1c›Ø›\x19[\Ëš\x1d^[[\x03B‘TNˆ\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÙ˜\KÃB‘^[ØÝ[Y[\x18]\x1a[ÛŽˆ\b\b\b\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÙ^[ØÜËš\x1d^[[\x03B•[œÝXœØÜšX™H\x1a[™›Îˆ\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÛ[\vÈÝ[œÝXœØÜšX™K\Ú[\^[\x19CBƒB

Re: Regression (last snapshot)

[Houder]

On Fri, 2 Aug 2019 14:34:02, Ken Brown  wrote:

> I think I've finally got it.
> 
> First of all, here's what POSIX says about signal handlers after an exec:
> 
> "Signals set to the default action (SIG_DFL) in the calling process 
> image shall be set to the default action in the new process image. 
> Except for SIGCHLD, signals set to be ignored (SIG_IGN) by the calling 
> process image shall be set to be ignored by the new process image. 
> Signals set to be caught by the calling process image shall be set to 
> the default action in the new process image (see <signal.h>)."
> 
> Second, here's a quote from the GTK+ documentation for gtk_init():
> 
> "Since 2.18, GTK+ calls signal (SIGPIPE, SIG_IGN) during initialization, 
> to ignore SIGPIPE signals, since these are almost never wanted in 
> graphical applications. If you do need to handle SIGPIPE for some 
> reason, reset the handler after gtk_init(), but notice that other 
> libraries (e.g. libdbus or gvfs) might do similar things."
> 
> Third, xwin-xdg-menu calls gtk_init() near the beginning of main().
> 
> Putting this all together, Eric's explanation is indeed correct.  All 
> processes created by xwin-xdg-menu via fork/exec inherit the property of 
> ignoring SIGPIPE.
> 
> I don't know if this is a bug, but it certainly leads to surprising 
> behavior.  Jon, maybe xwin-xdg-menu needs to call signal(SIGPIPE, 
> SIG_DFL) either after calling gtk_init() or before calling exec()?

Another option?

 - https://bugs.python.org/issue1652 - see msg115364 (Author: Mitar)

# GHC Haskell compiler is currently opting for a different solution:
#  installing an default empty handler which is cleared by exec
#  automatically and signal handler is restored back to SIG_DFL:
# 
# http://hackage.haskell.org/trac/ghc/ticket/4274

Installing the empty handler after the call to gtk_init() ...

Henri


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple