Tail not reading to end of input

Tail not reading to end of input

[smr xxxx]

Hi, I've just rebuilt my machine and done a clean install of the
latest Cygwin, and am having a problem with tail.  I'm wanting to know
whether this is a known issue before digging deeper.  For other
reasons, I've rebuilt my machine clean twice this weekend and have had
this problem both times -- I also sourced the Cygwin files from two
different mirrors.

It's been a few months since I last installed Cygwin, and all I can
think of that has changed since then is:
  1. I have a newer version of Cygwin
  2. I'm now on Vista Enterprise (was Vista Ultimate)
  3. I'm now on an SP1 of Vista (was original RTM of Vista)

When running a command such as "dir /s /a cygwin | tail -2" (from C:\)
it displays:
  03/04/2008  10:32 PM                35 7zr
  07/19/2007  02:05 PM
This last line is the 29th line of output.  Redirecting to a file
shows that there are 7 spaces at the end of this last line, and the
total output is 1028 bytes, or 1000 after running dos2unix.

Moving C:\cygwin to C:\x\cygwin (to force some extra characters into
the output of the above command, ie. the "x\") and rerunning shows
that it again stops at precisely the same place (7 spaces after the
time on the 29th line), but the output is now 1032 bytes, or 1004
after running dos2unix, due to the extra characters.

If I first redirect the output of the dir command to a file and tail
that then all is well.  Even running "type filename | tail -2" works.
Also, if I dos2unix the stream inplace (ie. "dir /s /a cygwin |
dos2unix | tail -2") it works correctly.  Note that I installed with
Unix file type (as always).

I just tried the same on \Windows, and it stops on the 26th line at a
position that seems to amount to 1124 bytes, or 1098 when dos2unix'd.

I'm guessing that this is a bigger problem that just tail (I could be
wrong), but "dir /s /a cygwin | wc -l" doesn't have any problems.

Has anyone seen this before?

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Tail not reading to end of input

[smr xxxx]

I've done some debugging of tail.exe on XP (which works) and Vista
Enterprise SP1 (which doesn't), and so far have found two differences
when tailing output piped from another program:
  1. S_ISREG (stats.st_mode) returns 0 on XP, but 1 on Vista (I
haven't yet determined if this is an Enterprise-only or an SP1-only
issue)
  2. lseek() fails on XP, but succeeds on Vista.  The SEEK_CUR below
returns 0, and the SEEK_END consistently returns 4105 in my tests (for
output that should be much longer, so presumably it's looking at a
single buffer of data).
Combines, this means that on XP the following code in tail.c results
in pipe_lines being called (as expected), but on Vista file_lines is
being called.  Does any of this mean anything to anyone?

      /* Use file_lines only if FD refers to a regular file for
	 which lseek (... SEEK_END) works.  */
      if ( ! presume_input_pipe
	   && S_ISREG (stats.st_mode)
	   && (start_pos = lseek (fd, 0, SEEK_CUR)) != -1
	   && start_pos < (end_pos = lseek (fd, 0, SEEK_END)))
	{
                          file_lines(...);
                }
                else
                {
                          pipe_line(...);
                }


On Mon, Apr 14, 2008 at 10:56 AM, smr xxxx <smrxxxx@gmail.com> wrote:
> Hi, I've just rebuilt my machine and done a clean install of the
> latest Cygwin, and am having a problem with tail.  I'm wanting to know
> whether this is a known issue before digging deeper.  For other
> reasons, I've rebuilt my machine clean twice this weekend and have had
> this problem both times -- I also sourced the Cygwin files from two
> different mirrors.
>
> It's been a few months since I last installed Cygwin, and all I can
> think of that has changed since then is:
>  1. I have a newer version of Cygwin
>  2. I'm now on Vista Enterprise (was Vista Ultimate)
>  3. I'm now on an SP1 of Vista (was original RTM of Vista)
>
> When running a command such as "dir /s /a cygwin | tail -2" (from C:\)
> it displays:
>  03/04/2008  10:32 PM                35 7zr
>  07/19/2007  02:05 PM
> This last line is the 29th line of output.  Redirecting to a file
> shows that there are 7 spaces at the end of this last line, and the
> total output is 1028 bytes, or 1000 after running dos2unix.
>
> Moving C:\cygwin to C:\x\cygwin (to force some extra characters into
> the output of the above command, ie. the "x\") and rerunning shows
> that it again stops at precisely the same place (7 spaces after the
> time on the 29th line), but the output is now 1032 bytes, or 1004
> after running dos2unix, due to the extra characters.
>
> If I first redirect the output of the dir command to a file and tail
> that then all is well.  Even running "type filename | tail -2" works.
> Also, if I dos2unix the stream inplace (ie. "dir /s /a cygwin |
> dos2unix | tail -2") it works correctly.  Note that I installed with
> Unix file type (as always).
>
> I just tried the same on \Windows, and it stops on the 26th line at a
> position that seems to amount to 1124 bytes, or 1098 when dos2unix'd.
>
> I'm guessing that this is a bigger problem that just tail (I could be
> wrong), but "dir /s /a cygwin | wc -l" doesn't have any problems.
>
> Has anyone seen this before?
>

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Tail not reading to end of input

[smr xxxx]

FWIW, a little non-Cygwin (VS2008) test app to display results from
fstat(0,...) returns the same thing on XP and Vista Ent SP1:
0x00002000 which means character special, and not regular.

On Tue, Apr 15, 2008 at 3:23 PM, smr xxxx <smrxxxx@gmail.com> wrote:
> I've done some debugging of tail.exe on XP (which works) and Vista
> Enterprise SP1 (which doesn't), and so far have found two differences
> when tailing output piped from another program:
>  1. S_ISREG (stats.st_mode) returns 0 on XP, but 1 on Vista (I
> haven't yet determined if this is an Enterprise-only or an SP1-only
> issue)
>  2. lseek() fails on XP, but succeeds on Vista.  The SEEK_CUR below
> returns 0, and the SEEK_END consistently returns 4105 in my tests (for
> output that should be much longer, so presumably it's looking at a
> single buffer of data).
> Combines, this means that on XP the following code in tail.c results
> in pipe_lines being called (as expected), but on Vista file_lines is
> being called.  Does any of this mean anything to anyone?
>
>      /* Use file_lines only if FD refers to a regular file for
>         which lseek (... SEEK_END) works.  */
>      if ( ! presume_input_pipe
>           && S_ISREG (stats.st_mode)
>           && (start_pos = lseek (fd, 0, SEEK_CUR)) != -1
>           && start_pos < (end_pos = lseek (fd, 0, SEEK_END)))
>        {
>                          file_lines(...);
>                }
>                else
>                {
>                          pipe_line(...);
>
>                }
>
>
> On Mon, Apr 14, 2008 at 10:56 AM, smr xxxx <smrxxxx@gmail.com> wrote:
> > Hi, I've just rebuilt my machine and done a clean install of the
> > latest Cygwin, and am having a problem with tail.  I'm wanting to know
> > whether this is a known issue before digging deeper.  For other
> > reasons, I've rebuilt my machine clean twice this weekend and have had
> > this problem both times -- I also sourced the Cygwin files from two
> > different mirrors.
> >
> > It's been a few months since I last installed Cygwin, and all I can
> > think of that has changed since then is:
> >  1. I have a newer version of Cygwin
> >  2. I'm now on Vista Enterprise (was Vista Ultimate)
> >  3. I'm now on an SP1 of Vista (was original RTM of Vista)
> >
> > When running a command such as "dir /s /a cygwin | tail -2" (from C:\)
> > it displays:
> >  03/04/2008  10:32 PM                35 7zr
> >  07/19/2007  02:05 PM
> > This last line is the 29th line of output.  Redirecting to a file
> > shows that there are 7 spaces at the end of this last line, and the
> > total output is 1028 bytes, or 1000 after running dos2unix.
> >
> > Moving C:\cygwin to C:\x\cygwin (to force some extra characters into
> > the output of the above command, ie. the "x\") and rerunning shows
> > that it again stops at precisely the same place (7 spaces after the
> > time on the 29th line), but the output is now 1032 bytes, or 1004
> > after running dos2unix, due to the extra characters.
> >
> > If I first redirect the output of the dir command to a file and tail
> > that then all is well.  Even running "type filename | tail -2" works.
> > Also, if I dos2unix the stream inplace (ie. "dir /s /a cygwin |
> > dos2unix | tail -2") it works correctly.  Note that I installed with
> > Unix file type (as always).
> >
> > I just tried the same on \Windows, and it stops on the 26th line at a
> > position that seems to amount to 1124 bytes, or 1098 when dos2unix'd.
> >
> > I'm guessing that this is a bigger problem that just tail (I could be
> > wrong), but "dir /s /a cygwin | wc -l" doesn't have any problems.
> >
> > Has anyone seen this before?
> >
>

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Tail not reading to end of input

[Eric Blake]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

According to smr xxxx on 4/14/2008 11:56 AM:
| Hi, I've just rebuilt my machine and done a clean install of the
| latest Cygwin, and am having a problem with tail.  I'm wanting to know
| whether this is a known issue before digging deeper.

Not known to me, but I (thankfully) don't use Vista.

|
| When running a command such as "dir /s /a cygwin | tail -2" (from C:\)

Is this in cmd.exe or a cygwin shell?  At any rate, you haven't followed
directions by posting 'cygcheck -svr' output as a text attachment, which
might provide us more insight into your setup.

| Problem reports:       http://cygwin.com/problems.html

|
| If I first redirect the output of the dir command to a file and tail
| that then all is well.  Even running "type filename | tail -2" works.
| Also, if I dos2unix the stream inplace (ie. "dir /s /a cygwin |
| dos2unix | tail -2") it works correctly.  Note that I installed with
| Unix file type (as always).

Unix file mode during cygwin's installation only affects regular files,
not pipes.  I don't know if the CR's being generated by the non-cygwin
tools are interfering with tail, but tail attempts to always operate in
binary mode, so I don't see how this could be broken.

|  1. S_ISREG (stats.st_mode) returns 0 on XP, but 1 on Vista (I
| haven't yet determined if this is an Enterprise-only or an SP1-only
| issue)
|   2. lseek() fails on XP, but succeeds on Vista.  The SEEK_CUR below

Are you sure this is on the same type of file?  You can't lseek on pipes,
and pipes are not regular files.  So maybe there is some sort of issue
where piped stdin inherited from a non-cygwin process on Vista wrongly
looks like a regular file instead of a pipe, causing confusion within
cygwin1.dll?

- --
Don't work too hard, make some time for fun as well!

Eric Blake             ebb9@byu.net
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkgF788ACgkQ84KuGfSFAYDSxQCgyo0/xK1ItHDLlXDHb/FfkNMO
aPgAn2p/aBYhh+WJdrvJzmMIfhXpagFP
=iGyp
-----END PGP SIGNATURE-----

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Tail not reading to end of input

[Corinna Vinschen]

On Apr 16 06:23, Eric Blake wrote:
> According to smr xxxx on 4/14/2008 11:56 AM:
> | Hi, I've just rebuilt my machine and done a clean install of the
> | latest Cygwin, and am having a problem with tail.  I'm wanting to know
> | whether this is a known issue before digging deeper.
>
> Not known to me, but I (thankfully) don't use Vista.

I can reproduce this problem on Windows 2008.  Pipes created by cmd on
Vista/2008 are for some reason not recognized as pipes when using Cygwin
1.5.25-11.  This problem does not occur with the upcoming Cygwin 1.7.0
so it's fixed in CVS, apparently.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Tail not reading to end of input

[Corinna Vinschen]

On Apr 16 14:49, Corinna Vinschen wrote:
> On Apr 16 06:23, Eric Blake wrote:
> > According to smr xxxx on 4/14/2008 11:56 AM:
> > | Hi, I've just rebuilt my machine and done a clean install of the
> > | latest Cygwin, and am having a problem with tail.  I'm wanting to know
> > | whether this is a known issue before digging deeper.
> >
> > Not known to me, but I (thankfully) don't use Vista.
> 
> I can reproduce this problem on Windows 2008.  Pipes created by cmd on
> Vista/2008 are for some reason not recognized as pipes when using Cygwin
> 1.5.25-11.  This problem does not occur with the upcoming Cygwin 1.7.0
> so it's fixed in CVS, apparently.

The reason is that starting with Windows Vista/2008, anonymous pipes
don't have a filename attached to them, so that the call to
NtQueryObject in handle_to_fn fails with STATUS_OBJECT_PATH_INVALID.
This is handled gracefully for pipes in CVS, but not yet in 1.5.25-11.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Tail not reading to end of input

[smr xxxx]

Thanks very much Eric and Corinna for the responses -- glad to hear
it's already fixed.

On Wed, Apr 16, 2008 at 6:13 AM, Corinna Vinschen
<corinna-cygwin@cygwin.com> wrote:
> On Apr 16 14:49, Corinna Vinschen wrote:
> > On Apr 16 06:23, Eric Blake wrote:
> > > According to smr xxxx on 4/14/2008 11:56 AM:
> > > | Hi, I've just rebuilt my machine and done a clean install of the
> > > | latest Cygwin, and am having a problem with tail.  I'm wanting to know
> > > | whether this is a known issue before digging deeper.
> > >
> > > Not known to me, but I (thankfully) don't use Vista.
> >
> > I can reproduce this problem on Windows 2008.  Pipes created by cmd on
> > Vista/2008 are for some reason not recognized as pipes when using Cygwin
> > 1.5.25-11.  This problem does not occur with the upcoming Cygwin 1.7.0
> > so it's fixed in CVS, apparently.
>
> The reason is that starting with Windows Vista/2008, anonymous pipes
> don't have a filename attached to them, so that the call to
> NtQueryObject in handle_to_fn fails with STATUS_OBJECT_PATH_INVALID.
> This is handled gracefully for pipes in CVS, but not yet in 1.5.25-11.
>
>
>
> Corinna
>
> --
> Corinna Vinschen                  Please, send mails regarding Cygwin to
> Cygwin Project Co-Leader          cygwin AT cygwin DOT com
> Red Hat
>
> --
> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
> Problem reports:       http://cygwin.com/problems.html
> Documentation:         http://cygwin.com/docs.html
> FAQ:                   http://cygwin.com/faq/
>
>

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Tail not reading to end of input

[Christopher Faylor]

On Wed, Apr 16, 2008 at 03:04:12PM -0700, smr xxxx wrote:
>Thanks very much Eric and Corinna for the responses -- glad to hear
>it's already fixed.

FYI, this will even be fixed in the 1.5.25 series eventually.

cgf

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Tail not reading to end of input

[smr xxxx]

Hi Christopher, is there currently an ETA for a release that would
include this fix (whether that's 1.5.25-xx, or 1.6/1.7)?  I've been
out of the loop for a long time, so apologies if this is a question
you've been badgered with recently.

Thanks,

Steven

On Wed, Apr 16, 2008 at 4:44 PM, Christopher Faylor
<cgf-use-the-mailinglist-please@cygwin.com> wrote:
> On Wed, Apr 16, 2008 at 03:04:12PM -0700, smr xxxx wrote:
> >Thanks very much Eric and Corinna for the responses -- glad to hear
> >it's already fixed.
>
> FYI, this will even be fixed in the 1.5.25 series eventually.
>
> cgf
>
>
> --
> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
> Problem reports:       http://cygwin.com/problems.html
> Documentation:         http://cygwin.com/docs.html
> FAQ:                   http://cygwin.com/faq/
>
>

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Tail not reading to end of input

[Eric Blake]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

According to smr xxxx on 4/16/2008 6:29 PM:

Please don't top-post: http://cygwin.com/acronyms/#TOFU

| Hi Christopher, is there currently an ETA for a release that would
| include this fix (whether that's 1.5.25-xx, or 1.6/1.7)?

In the classic open source style - when it's ready (and sooner, if you
help).  We won't promise a particular release date (that way, we won't
break the promise if issues pop up).  Meanwhile, snapshots of the future
1.7.0 are already available from cygwin.com, if you want to test the fix.

|  I've been
| out of the loop for a long time, so apologies if this is a question
| you've been badgered with recently.

Yes, searching the archives would have shown you that this question has
been asked more than enough times.

- --
Don't work too hard, make some time for fun as well!

Eric Blake             ebb9@byu.net
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkgGxOsACgkQ84KuGfSFAYBrxgCgrUQ70pxhe0OyUMkG8LygRAwm
FyIAn2y+/6PLEkUQf4p8NwSv2H7ooRLF
=8S9i
-----END PGP SIGNATURE-----

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/