stpd and stdout

stpd and stdout

[Hien Nguyen]

Hi Martin,

While working on the the systemtap GUI we run into the problem of 
getting the stdout to the gui app in a timely fashion. We does get the 
output as a blob of text once in a while (looks like data were only 
flushed out when the buffer is full). To recreate the problem you can 
try this

stap -e 'probe syscall.open{ log(filename) }' | tee

Anyway, I added one line of code to flushout the data as soon as the 
stpd gets it. It seems to solve the above problem. To avoid unnecessary 
performance issues for others, perhaps we should have that as an option. 
What do you think?

Thanks, Hien.

--- librelay.c  2006-05-09 14:49:07.000000000 -0700
+++ librelay.c.new      2006-05-09 14:34:48.000000000 -0700
@@ -789,6 +789,7 @@
                }
                case STP_REALTIME_DATA:
                        fwrite_unlocked(data, nb - sizeof(int), 1, ofp);
+                       fflush_unlocked(ofp);
                        break;
                case STP_OOB_DATA:
                        fputs ((char *)data, stderr);

Re: stpd and stdout

[Martin Hunt]

On Tue, 2006-05-09 at 15:21 -0700, Hien Nguyen wrote:
> Hi Martin,
> 
> While working on the the systemtap GUI we run into the problem of 
> getting the stdout to the gui app in a timely fashion. We does get the 
> output as a blob of text once in a while (looks like data were only 
> flushed out when the buffer is full). To recreate the problem you can 
> try this
> 
> stap -e 'probe syscall.open{ log(filename) }' | tee
> 
> Anyway, I added one line of code to flushout the data as soon as the 
> stpd gets it. It seems to solve the above problem. To avoid unnecessary 
> performance issues for others, perhaps we should have that as an option. 
> What do you think?

I think that in this case (stdout from procfs) it should always be
line-buffered. No option is necessary.  In fact I tried it and couldn't
measure any significant performance advantage for full buffering.  I've
gone ahead and committed the following patch.  Let me know how it works.

Index: librelay.c
===================================================================
RCS file: /cvs/systemtap/src/runtime/stpd/librelay.c,v
retrieving revision 1.37
diff -u -r1.37 librelay.c
--- librelay.c  8 Apr 2006 21:59:36 -0000       1.37
+++ librelay.c  18 May 2006 17:51:31 -0000
@@ -722,6 +722,7 @@
        int type;
        FILE *ofp = stdout;

+       setvbuf(ofp, (char *)NULL, _IOLBF, 0);
        pthread_mutex_init(&processing_mutex, NULL);

        signal(SIGINT, sigproc);

Re: stpd and stdout

[Hien Nguyen]

Martin Hunt wrote:

>I think that in this case (stdout from procfs) it should always be
>line-buffered. No option is necessary.  In fact I tried it and couldn't
>measure any significant performance advantage for full buffering.  I've
>gone ahead and committed the following patch.  Let me know how it works.
>
>Index: librelay.c
>===================================================================
>RCS file: /cvs/systemtap/src/runtime/stpd/librelay.c,v
>retrieving revision 1.37
>diff -u -r1.37 librelay.c
>--- librelay.c  8 Apr 2006 21:59:36 -0000       1.37
>+++ librelay.c  18 May 2006 17:51:31 -0000
>@@ -722,6 +722,7 @@
>        int type;
>        FILE *ofp = stdout;
>
>+       setvbuf(ofp, (char *)NULL, _IOLBF, 0);
>        pthread_mutex_init(&processing_mutex, NULL);
>
>        signal(SIGINT, sigproc);
>
>
>  
>
Thanks Martin, it works.

RE: stpd and stdout

[Stone, Joshua I]

On Tuesday, May 09, 2006 3:21 PM, Hien Nguyen wrote:
> While working on the the systemtap GUI we run into the problem of
> getting the stdout to the gui app in a timely fashion. We does get the
> output as a blob of text once in a while (looks like data were only
> flushed out when the buffer is full). To recreate the problem you can
> try this
> 
> stap -e 'probe syscall.open{ log(filename) }' | tee
> 
> Anyway, I added one line of code to flushout the data as soon as the
> stpd gets it. It seems to solve the above problem. To avoid
> unnecessary performance issues for others, perhaps we should have
> that as an option. What do you think?

I've run into this with Perl scripts before -- Perl normally
line-buffers output to STDOUT and block-buffers otherwise (e.g. to a
pipe or socket).  Their solution for this problem is an autoflush
option:
http://perldoc.perl.org/perlvar.html#HANDLE->autoflush(EXPR)

I think it would be very reasonable to support a similar command-line
option in stap/stpd.


Josh