stdbuf does not work

stdbuf does not work

[Donald Krieger via cygwin]

Dear list,
I require line buffered output from program1 and line buffered input to program2.The following works under Linux but not under Cygwin.   stdbuf -oL -eL program 1 |& program2

I've searched the archive without success.I have tried -o0 -e0 -i0 and have also tried
  stdbuf -oL -eL program1 |& stdbuf -i0 program2
I have a fresh installation of cygwin as of a few months ago:  uname -a              yields:CYGWIN_NT-10.0 DESKTOP-S92525R 2.10.0(0.325/5/3) 2018-02-02 15:16 x86_64 Cygwin
  stdbuf --version    yields:stdbuf (GNU coreutils) 8.26Packaged by Cygwin (8.26-2)Copyright (C) 2016 Free Software Foundation, Inc....
Thanks for your help.
Best - Don

|  | Virus-free. www.avast.com  |


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: stdbuf does not work

[Eliot Moss]

On 8/18/2018 8:36 AM, Donald Krieger via cygwin wrote:
> Dear list,
> I require line buffered output from program1 and line buffered input to program2.The following works under Linux but not under Cygwin.   stdbuf -oL -eL program 1 |& program2
> 
> I've searched the archive without success.I have tried -o0 -e0 -i0 and have also tried
>    stdbuf -oL -eL program1 |& stdbuf -i0 program2
> I have a fresh installation of cygwin as of a few months ago:  uname -a              yields:CYGWIN_NT-10.0 DESKTOP-S92525R 2.10.0(0.325/5/3) 2018-02-02 15:16 x86_64 Cygwin
>    stdbuf --version    yields:stdbuf (GNU coreutils) 8.26Packaged by Cygwin (8.26-2)Copyright (C) 2016 Free Software Foundation, Inc....

Some thoughts to consider:

- -i0 means unbuffered; did you write the wrong description above?
- As noted in the strbuf man page, some programs don't use streams and thus are
   not affected by strbuf settings.  I agree this is not something likely to
   change for the same program ported from Linux to Cygwin.
- How do you have line endings set up on Cygwin or in any files you are
   processing?  The usual Windows line ending   being CR-LF instead of Linux's
   LF can confuse some programs.

It might also help potential responders to know a little more about what program1 and
program2 are.  Are they Windows or Cygwin programs?

Regards - EM

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: stdbuf does not work

[Donald Krieger via cygwin]

Dear Eliot,
Thanks for getting back so quickly.Program1 is a windows routine, xgraph.  http://www.xgraph.org/ms/index.htmlThis is a general purpose plotting program which writes out text whenever an operation is performed in its graphic window.
That text output is piped into Program2, a home-grown tcsh script.Program2 reads the xgraph text output one line at a time and uses it to update the display list files which xgraph is polling to keep its graphic window up to date.
Here is the test harness I'm executing inside a cygwin window. For test purposes, Program2 is cat.   stdbuf -oL -eL xgraph | cat -A
Here's a typical output which is only seen once I've terminated xgraph by clicking the appropriate button in the graphic window. The only think I did while xgraph was running was click at one location in the graphics window; that's what generated the line: Clicked at (x,y) = (-0.62324, 0.255608)
XGraph v4.38^M$Window (800 x 594)^M$0 points read.^M$^M$Error:  No dimensional data was read.  Cannot determine axes ranges.^M$^M$   Clicked at (x,y) = (-0.62324, 0.255608)^M$Exiting XGraph.^M$
I tried 0 (zero) instead of L in the stdbuf switches just because.I also tried using stdbuf -i0 Program2in case that might make a difference.
I can easily eliminate the <cr>:  stdbuf -oL -eL xgraph | tr -d "\015" | cat -AThis produces the same output but with the ^M 's removed.I don't know how to alter the windows line endings but would like to.
Thanks again and for any further thoughts.
Best - Don

      From: Eliot Moss <moss@cs.umass.edu>
 To: cygwin@cygwin.com 
 Sent: Saturday, August 18, 2018 9:14 AM
 Subject: Re: stdbuf does not work
   
On 8/18/2018 8:36 AM, Donald Krieger via cygwin wrote:
> Dear list,
> I require line buffered output from program1 and line buffered input to program2.The following works under Linux but not under Cygwin.   stdbuf -oL -eL program 1 |& program2
> 
> I've searched the archive without success.I have tried -o0 -e0 -i0 and have also tried
>    stdbuf -oL -eL program1 |& stdbuf -i0 program2
> I have a fresh installation of cygwin as of a few months ago:  uname -a              yields:CYGWIN_NT-10.0 DESKTOP-S92525R 2.10.0(0.325/5/3) 2018-02-02 15:16 x86_64 Cygwin
>    stdbuf --version    yields:stdbuf (GNU coreutils) 8.26Packaged by Cygwin (8.26-2)Copyright (C) 2016 Free Software Foundation, Inc....

Some thoughts to consider:

- -i0 means unbuffered; did you write the wrong description above?
- As noted in the strbuf man page, some programs don't use streams and thus are
  not affected by strbuf settings.  I agree this is not something likely to
  change for the same program ported from Linux to Cygwin.
- How do you have line endings set up on Cygwin or in any files you are
  processing?  The usual Windows line ending  being CR-LF instead of Linux's
  LF can confuse some programs.

It might also help potential responders to know a little more about what program1 and
program2 are.  Are they Windows or Cygwin programs?

Regards - EM

--
Problem reports:      http://cygwin.com/problems.html
FAQ:                  http://cygwin.com/faq/
Documentation:        http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple



   

|  | Virus-free. www.avast.com  |


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: stdbuf does not work

[Eliot Moss]

On 8/18/2018 10:17 AM, Donald Krieger via cygwin wrote:
> Dear Eliot,
> Thanks for getting back so quickly.Program1 is a windows routine, xgraph.  http://www.xgraph.org/ms/index.htmlThis is a general purpose plotting program which writes out text whenever an operation is performed in its graphic window.
> That text output is piped into Program2, a home-grown tcsh script.Program2 reads the xgraph text output one line at a time and uses it to update the display list files which xgraph is polling to keep its graphic window up to date.
> Here is the test harness I'm executing inside a cygwin window. For test purposes, Program2 is cat.   stdbuf -oL -eL xgraph | cat -A
> Here's a typical output which is only seen once I've terminated xgraph by clicking the appropriate button in the graphic window. The only think I did while xgraph was running was click at one location in the graphics window; that's what generated the line: Clicked at (x,y) = (-0.62324, 0.255608)
> XGraph v4.38^M$Window (800 x 594)^M$0 points read.^M$^M$Error:  No dimensional data was read.  Cannot determine axes ranges.^M$^M$   Clicked at (x,y) = (-0.62324, 0.255608)^M$Exiting XGraph.^M$
> I tried 0 (zero) instead of L in the stdbuf switches just because.I also tried using stdbuf -i0 Program2in case that might make a difference.
> I can easily eliminate the <cr>:  stdbuf -oL -eL xgraph | tr -d "\015" | cat -AThis produces the same output but with the ^M 's removed.I don't know how to alter the windows line endings but would like to.
> Thanks again and for any further thoughts.

Apparently stdbuf works by interposing stuff, via a loader PRELOAD technique,
before main in the program being run.  I suspect this simply does not work
when applied to Windows programs, though I cannot say for sure.

What about this:

xgraph ... | strbuf -oL -eL tr -d '\015' | ...

That is, apply strbuf to tr, which is a Cygwin program.  You could even apply it to cat.

Alternatively, if the xgraph of which you speak is this:

https://sourceforge.net/projects/nsnam/files/xgraph/xgraph-12.1/

you might be able to re-build it under Cygwin, though doing so sometimes requires
some adjustments to a program or its build process (though a number of programs
build out-of-the-box pretty well).  Clearly this is hardr than trying the suggestion
above :-) ...

Regards - EM

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: stdbuf does not work

[Marco Atzeri]

Am 18.08.2018 um 16:28 schrieb Eliot Moss:
> On 8/18/2018 10:17 AM, Donald Krieger via cygwin wrote:
>> Dear Eliot,
>> Thanks for getting back so quickly.Program1 is a windows routine, 
>> xgraph.  http://www.xgraph.org/ms/index.htmlThis is a general purpose 
>> plotting program which writes out text whenever an operation is 
>> performed in its graphic window.

> 
> Alternatively, if the xgraph of which you speak is this:
> 
> https://sourceforge.net/projects/nsnam/files/xgraph/xgraph-12.1/
> 

> Regards - EM
> 

this version of xgraph is available as cygwin package
https://www.isi.edu/nsnam/xgraph/

---
Diese E-Mail wurde von Avast Antivirus-Software auf Viren geprüft.
https://www.avast.com/antivirus


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: stdbuf does not work

[Eliot Moss]

Donald emailed me directly that this:

xgraph ... | strbuf -oL -eL tr -d '\015' | ...
does not fix the problem.  While I don't recall his ever having described
"the problem" that he is trying to use strbuf to solve, I now gather that
it has to do with xgraph doing more buffering of output than he would
like, so the downstream programs are not responding in a timely way.

The general problem is that this kind of buffering is usual for pipes, and
thus what the Microsoft C RunTime (CRT) does (cygwin's C run-time does
the same thing, I believe, but strbuf can override it).

The fix is to be writing to a character ("console") device of some kind.
This post on stackoverflow gives a solution:

https://stackoverflow.com/questions/11516258/what-is-the-equivalent-of-unbuffer-program-on-windows

The solution proposed there is a program called winpty, and use of
(apparently undocumented) flags to winpty to achieve the desired effect.
I am posting back to the list rather than to Don directly, in hope that
this works and his confirmation to the list may be helpful in the future
to others trying to achieve that same effect.

Regards - Eliot

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: stdbuf does not work

[Donald Krieger via cygwin]

Thank you again, Eliot, for this direction.I haven't had time to get everything working but winpty does appear to work.Note that the undocumented switch,  -Xallow-non-tty must be included to use pipes.This now works, i.e. the text output from xgraph shows up immediately.
    winpty  -Xallow-non-tty xgraph | cat -A
winpty may be found here:    https://github.com/rprichard/winptyThe download is for the source. I was able to make it without a hiccup by doing the following in directory winpty-master:   ./configure   make   make install
rprichard includes he "undocumented" switches here:    https://github.com/rprichard/winpty/issues/103
Thanks again.
Best - Don

      From: Eliot Moss <moss@cs.umass.edu>
 To: cygwin@cygwin.com 
 Sent: Saturday, August 18, 2018 1:01 PM
 Subject: Re: stdbuf does not work
  
Donald emailed me directly that this:

xgraph ... | strbuf -oL -eL tr -d '\015' | ...
does not fix the problem.  While I don't recall his ever having described
"the problem" that he is trying to use strbuf to solve, I now gather that
it has to do with xgraph doing more buffering of output than he would
like, so the downstream programs are not responding in a timely way.

The general problem is that this kind of buffering is usual for pipes, and
thus what the Microsoft C RunTime (CRT) does (cygwin's C run-time does
the same thing, I believe, but strbuf can override it).

The fix is to be writing to a character ("console") device of some kind.
This post on stackoverflow gives a solution:

https://stackoverflow.com/questions/11516258/what-is-the-equivalent-of-unbuffer-program-on-windows

The solution proposed there is a program called winpty, and use of
(apparently undocumented) flags to winpty to achieve the desired effect.
I am posting back to the list rather than to Don directly, in hope that
this works and his confirmation to the list may be helpful in the future
to others trying to achieve that same effect.

Regards - Eliot

--
Problem reports:      http://cygwin.com/problems.html
FAQ:                  http://cygwin.com/faq/
Documentation:        http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple



   
--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple