RE: Buffering problem in netcat server script

RE: Buffering problem in netcat server script

[Sven Severus]

Adam Dinwoodie wrote:
> From `mad sed`:
> 
>     -u, --unbuffered
> 
>         load minimal amounts of data from the input files and flush the output
>         buffers more often
> 
> If you add the -u option, sed will buffer less and write to the pipe more
> often.  I would guess this isn't the default as it's less efficient, but
> haven't done anything to verify that.
> 
> I believe `cat` never buffers, but I base this on nothing but instinct.
> 
> A quick experiment on my handy RHEL box implies this is not Cygwin specific;
> the following command shows buffering behaviour too:
> 
>     tail -f tmpfile | sed 's/e/E/g' | tee outfile
> 
> (I'm using the pipelines so I can see what's going on without sed thinking
> stdin or stdout are a terminal.)

Thanks a lot for your reply.

Sven

-- 
Mit freundlichen GrÌßen

Dipl. Inform. Sven Severus
Softwareentwicklung
----------------------------------------------------------
HIMA Paul Hildebrandt GmbH + CO KG
Abt: Entwicklung Software
Albert-Bassermann-Strasse 28
68782 Bruehl
Germany

Tel: +49 6202 709-289
Fax: +49 6202 709-299
E-Mail: s.severus@hima.com
Internet: www.hima.de


-- 
HIMA Paul Hildebrandt GmbH + Co KG, Albert-Bassermann-Str. 28, 68782 Bruehl bei Mannheim
Kommanditgesellschaft, Sitz Bruehl, Deutschland - Registergericht Mannheim HRA 421017
Ust-ID: DE 144286400, St.Nr: 43038 00190

Persoenlich haftende Gesellschafterin Paul Hildebrandt Verwaltungsgesellschaft mbH,
Sitz Bruehl, Deutschland - Registergericht Mannheim HRB 420588

Geschaeftsfuehrer: Dipl.-Betriebswirt Steffen Philipp


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

RE: Buffering problem in netcat server script

[Adam Dinwoodie]

Sven Severus wrote:
> Question 1:
> Why do I face different behaviour with "cat -n" and "sed s/e/E/g"?
> Are there cygwin related reasons?
> Which behaviour should I expect (I know there are buffering
> mechanisms for stdout when not connected to a tty, so I tend to
> say the buffering behaviour ist the one to expect).
>
> Question 2:
> What can I do to turn off the buffering behaviour and to get the
> output lines immediately?
> Or is my server script approach inappropriate? What should work
> better?

From `mad sed`:

    -u, --unbuffered

        load minimal amounts of data from the input files and flush the output
        buffers more often

If you add the -u option, sed will buffer less and write to the pipe more
often.  I would guess this isn't the default as it's less efficient, but
haven't done anything to verify that.

I believe `cat` never buffers, but I base this on nothing but instinct.

A quick experiment on my handy RHEL box implies this is not Cygwin specific;
the following command shows buffering behaviour too:

    tail -f tmpfile | sed 's/e/E/g' | tee outfile

(I'm using the pipelines so I can see what's going on without sed thinking
stdin or stdout are a terminal.)

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Buffering problem in netcat server script

[Sven Severus]

Hallo all,

I have a very simple two-line script myserver.sh

#! /bin/sh
tee log1 | cat -n | tee log2

It writes each input line to logfile log 1, modifies it (here simply
by prepending a line number via "cat -n"), writes the modified
line to logfile log2 and outputs it to stdout.
The script is launched by netcat (in listening or server mode),
with stdin and stdout connected to port 8888:
nc -vv -lp 8888 -e myserver.sh

When a connection ist established on port 8888, myserver.sh is
started. All works properly: After sending an input line via
port 8888 I immediately receive the modified output line.
In log1 I can see each input line and in log2 each output line,
just in time. Fine!

Now I modified myserver.sh to get a different modification:
I replaced the "cat -n" part by "sed s/e/E/g" to capitalize
all "e"-characters in the input stream.
Now when a connection ist established I do not receive any output
for the given input lines. In log1 I can see any input line (in
time, as before), but log2 remains empty. Not what I wanted!
But when I continue sending input lines and the amount of input
data exceeds 64kB, then - flupp - I get all the long expected
output (and see it in log2).
It seems obvious, that this behaviour is related to buffering.

Question 1:
Why do I face different behaviour with "cat -n" and "sed s/e/E/g"?
Are there cygwin related reasons?
Which behaviour should I expect (I know there are buffering
mechanisms for stdout when not connected to a tty, so I tend to
say the buffering behaviour ist the one to expect).

Question 2:
What can I do to turn off the buffering behaviour and to get the
output lines immediately?
Or is my server script approach inappropriate? What should work
better?

Thanks a lot in advance.

Best regards
Sven Severus



-- 
Mit freundlichen GrÌßen

Dipl. Inform. Sven Severus
Softwareentwicklung
----------------------------------------------------------
HIMA Paul Hildebrandt GmbH + CO KG
Abt: Entwicklung Software
Albert-Bassermann-Strasse 28
68782 Bruehl
Germany

Tel: +49 6202 709-289
Fax: +49 6202 709-299
E-Mail: s.severus@hima.com
Internet: www.hima.de


-- 
HIMA Paul Hildebrandt GmbH + Co KG, Albert-Bassermann-Str. 28, 68782 Bruehl bei Mannheim
Kommanditgesellschaft, Sitz Bruehl, Deutschland - Registergericht Mannheim HRA 421017
Ust-ID: DE 144286400, St.Nr: 43038 00190

Persoenlich haftende Gesellschafterin Paul Hildebrandt Verwaltungsgesellschaft mbH,
Sitz Bruehl, Deutschland - Registergericht Mannheim HRB 420588

Geschaeftsfuehrer: Dipl.-Betriebswirt Steffen Philipp


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple