Language Reference, updated

Language Reference, updated

[Robb Romans]

[-- Attachment #1.1: Type: text/plain, Size: 290 bytes --]

I've updated the draft Language Reference. Here it is in gzipped LaTeX 
and PostScript. Comments welcome.

Regards,
Robb

-- 
Robb Romans                    (512) 838-0419
Linux Commando                   T/L 678-0419
robb@linux.vnet.ibm.com          ARS NA5TT
.-- - ..-. ..--..

[-- Attachment #1.2: SystemTap-LangRef-20061122.tex.gz --]
[-- Type: application/x-gzip, Size: 17892 bytes --]

[-- Attachment #1.3: SystemTap-LangRef-20061122.ps.gz --]
[-- Type: application/x-gzpostscript, Size: 85155 bytes --]

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Language Reference, updated

[Martin Hunt]

Some comments. I'm copying the list because some of these apply equally
to man pages.

1.4.2 Primes
Example is from man page. Perhaps both should change from
	if (isprime (i)) log (sprint(i))
to
	if (isprime (i)) printf("%d\n", i)
Reason: Our users are C programmers. log() in C is a math function.
printf is more powerful and more familiar. log() should be deprecated,
but we will probably keep it around forever for compatibility.

1.4.3
Same issue. Change 
	log ("11th fibonacci number: " . sprint (fibonacci (11)))
to
	printf("11th fibonacci number: %d\n", fibonacci (11))

3.2.3
printf statement has a syntax problem. Should read
	printf("reading fd=%d\n", fildes)

3.2.4
Again, I think more C-like examples are better. Instead of
	function thatfn:string(arg1:long, arg2) {
		return sprint(arg1) . arg2
	}
Use
	function thatfn:string(arg1:long, arg2) {
		return sprintf("%d%s",arg1, arg2)
	}

3.5 You start off mentioning "guru mode" without an explanation of what
it is or how you use it. Also I don't understand why the phrase "in some
sequence" is used here. 

6.0.11 foreach
Don't forget to document "limit" keyword.

7.2 It may be best to explain that
	if ([4,"hello"] in foo) {}
is the same as
	if (foo[4,"hello"] == 0) {}
And therefore one cannot test for membership of an element with value
0 or ""


8.4.3 instead of
	print(@hist_log(v))
should be
	print(@hist_log(x))

9.1.1
"stpd prepends the string "ERROR:""
We shouldn't mention "stpd" because the user doesn't really care about
the name of that program and the name is actually now "staprun". Also,
the prepending is done in the kernel. "staprun" just writes the output
to stderr,

9.1.4 SHould mention that normal printf field formatting is available
(%-20,   %o, %x, etc). Also mention %p and binary formats here.

9.1.8 There is no such thing as a "warning stream" Output goes to
stderr. Same comment as 9.1.1 re stpd

Re: Language Reference, updated

[Robb Romans]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

>>>>> "MH" == Martin Hunt <hunt@redhat.com> writes:

    MH> Some comments. I'm copying the list because some of these apply
    MH> equally to man pages.

    <snip>

Martin,

Thank you. I will incorporate your suggestions into the next release of
the Language Reference.

Regards,
Robb

- -- 
Robb Romans                     (512) 838-0419
Linux Commando                  T/L   678-0419
robb@linux.vnet.ibm.com         ARS NA5TT
.-- - ..-. ..--..
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)
Comment: Processed by Mailcrypt 3.5.8 <http://mailcrypt.sourceforge.net/>

iD8DBQFFbMyuZyOifwSFgjQRAknSAJ9SnoxxZzoMCnVlWsfsnzz7F7m+kQCfV9hW
QEaNzXySKUm+6ao82l4I8yo=
=tjXr
-----END PGP SIGNATURE-----

Re: Language Reference, updated

[Robb Romans]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

>>>>> "MH" == Martin Hunt <hunt@redhat.com> writes:

    MH> 9.1.4 SHould mention that normal printf field formatting is
    MH> available (%-20, %o, %x, etc). Also mention %p and binary
    MH> formats here.

Martin,

I've incorporated your feedback into the Language Reference, but I'd
like clarification on one section. What do you mean by "Also mention %p
and binary formats here" ?

Here is a description of %p for standard C:
"Pointer to a pointer. Inputs a memory address in the same fashion of the
%p type produced by the printf function."

Is there a difference in SystemTap's handling of %p that you want me to
call out, or is there a special usage case that you have in mind?

Is there a difference in SystemTap's handling of printing binary streams
versus C standard syntax? Thanks.

Regards,
Robb

- -- 
Robb Romans                     (512) 838-0419
Linux Commando                  T/L   678-0419
robb@linux.vnet.ibm.com         ARS NA5TT
.-- - ..-. ..--..
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)
Comment: Processed by Mailcrypt 3.5.8 <http://mailcrypt.sourceforge.net/>

iD8DBQFFdeA/ZyOifwSFgjQRAmCYAJ0a185d7j7jaZt93Ts7cYiZTrPeSwCgtzis
igXOlANUokjF8PxHV59JSPE=
=67Bh
-----END PGP SIGNATURE-----

Re: Language Reference, updated

[Martin Hunt]

On Tue, 2006-12-05 at 15:10 -0600, Robb Romans wrote:

> I've incorporated your feedback into the Language Reference, but I'd
> like clarification on one section. What do you mean by "Also mention %p
> and binary formats here" ?
> 
> Here is a description of %p for standard C:
> "Pointer to a pointer. Inputs a memory address in the same fashion of the
> %p type produced by the printf function."
> 
> Is there a difference in SystemTap's handling of %p that you want me to
> call out, or is there a special usage case that you have in mind?

You wrote that "valid types are limited to string ("%s") and integer 
("%d")"

That makes it sound like you can only use "%s" and "%d" in printf.
You can also use the standard flags and conversion specifiers from C
printf. 

> Is there a difference in SystemTap's handling of printing binary streams
> versus C standard syntax? Thanks.

Yes, C printf does not do binary output. Systemtap's printf does the
following:

%1n, %2n, %4n, %n -  These functions instruct printf to write
a binary length at this location that is the total length of
the string written by the printf.  The field width is the number
of bytes to write. The default is "2".

%1b, %2b, %4b, %8b, %b - binary write. The field width specifies the
number of bytes to write. Default is 4 (32-bits).

Examples:

stap -e 'probe begin{for(i=97;i<123;i++) printf("%3d: %1b%1b%1b\n",i,i,i-32,i-64); exit()}'

> stap -e 'probe begin{printf("%1n%b%b", 0xc0dedbad, 0x12345678);exit()}' | hexdump -C
00000000  08 ad db de c0 78 56 34  12                       |.....xV4.|

> stap -e 'probe begin{printf("%1b%1b%1blo %1b%1brld\n", 72,101,108,87,111);exit()}'
Hello World

> 
> Regards,
> Robb
> 
> - -- 
> Robb Romans                     (512) 838-0419
> Linux Commando                  T/L   678-0419
> robb@linux.vnet.ibm.com         ARS NA5TT
> .-- - ..-. ..--..
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.5 (GNU/Linux)
> Comment: Processed by Mailcrypt 3.5.8 <http://mailcrypt.sourceforge.net/>
> 
> iD8DBQFFdeA/ZyOifwSFgjQRAmCYAJ0a185d7j7jaZt93Ts7cYiZTrPeSwCgtzis
> igXOlANUokjF8PxHV59JSPE=
> =67Bh
> -----END PGP SIGNATURE-----

Re: Language Reference, updated

[Robb Romans]

[-- Attachment #1: Type: text/plain, Size: 1664 bytes --]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

>>>>> "MH" == Martin Hunt <hunt@redhat.com> writes:

    MH> You wrote that "valid types are limited to string ("%s") and
    MH> integer ("%d")"

    MH> That makes it sound like you can only use "%s" and "%d" in
    MH> printf.  You can also use the standard flags and conversion
    MH> specifiers from C printf.

Gotcha. Fixed.

    >> Is there a difference in SystemTap's handling of printing binary
    >> streams versus C standard syntax? Thanks.

    MH> Yes, C printf does not do binary output. Systemtap's printf does
    MH> the following:

    MH> %1n, %2n, %4n, %n - These functions instruct printf to write a
    MH> binary length at this location that is the total length of the
    MH> string written by the printf.  The field width is the number of
    MH> bytes to write. The default is "2".

    MH> %1b, %2b, %4b, %8b, %b - binary write. The field width specifies
    MH> the number of bytes to write. Default is 4 (32-bits).

    MH> Examples:

    MH> stap -e 'probe begin{for(i=97;i<123;i++) printf("%3d:
    MH> %1b%1b%1b\n",i,i,i-32,i-64); exit()}'

    MH>> stap -e 'probe begin{printf("%1n%b%b", 0xc0dedbad,
    MH>> 0x12345678);exit()}' | hexdump -C
    MH> 00000000 08 ad db de c0 78 56 34 12 |.....xV4.|

    MH>> stap -e 'probe begin{printf("%1b%1b%1blo %1b%1brld\n",
    MH>> 72,101,108,87,111);exit()}'
    MH> Hello World

Thank you for taking the time to make it clear to me. I've put your
reply into the Language Reference mostly verbatim.

Here's a fresh copy for review. I'll be on vacation after today, until
January 2nd, so look for updates and content development next year.


[-- Attachment #2: SystemTap-Language-Ref-2006-12-06.tex.gz --]
[-- Type: application/octet-stream, Size: 18652 bytes --]

[-- Attachment #3: SystemTap-Language-Ref-2006-12-06.ps.gz --]
[-- Type: application/octet-stream, Size: 85973 bytes --]

[-- Attachment #4: Type: text/plain, Size: 439 bytes --]


Regards,
Robb

- -- 
Robb Romans                     (512) 838-0419
Linux Commando                  T/L   678-0419
robb@linux.vnet.ibm.com         ARS NA5TT
.-- - ..-. ..--..
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)
Comment: Processed by Mailcrypt 3.5.8 <http://mailcrypt.sourceforge.net/>

iD8DBQFFdxVaZyOifwSFgjQRAhfqAKC3F2sqoRCpzWW1KFtXp77Qrjl2+QCgkA+p
llFqaccpRceuImoEo1a3Ipg=
=4Khx
-----END PGP SIGNATURE-----

Re: Language Reference, updated

[Mike Mason]

Robb Romans wrote:
> I've updated the draft Language Reference. Here it is in gzipped LaTeX 
> and PostScript. Comments welcome.
> 
> Regards,
> Robb
> 

Hi Rob,

Sorry for not getting comments to you sooner.  Here they are.  Some are actually questions which need an answer from the mailing list.

Section 1.2
This section only mentions developers as potential users.  A second major audience is system administrators. Add something like this: "SystemTap also allows system administrators to easily write scripts that gather and manipulate data from the kernel that is not otherwise available using standard Linux tools."

I also think the sentence that mentions Kprobes can be removed, though I could be convinced otherwise.  I just don't think it's relevant here, especially since it's not mentioned anywhere else in the document.

Section 1.4, 1st paragraph
Also mention "War Stories" section on SystemTap wiki.
http://sourceware.org/systemtap/wiki

Section, 1.4.3, last sentence
The built in limits (MAXACTION, MAXNESTING, etc) deserve their own section describing what they are, how they're used and how to change them.  This should be fairly early in the document.

Section 2.2
Mention that tapset scripts look just like regular systemtap scripts, but they cannot be run directly.

Section 3.2
Mention that probe aliases can also refer to one or more other aliases.

Example:

probe socket.sendmsg = kernel.function ("sock_sendmsg") { ... }
probe socket.do_write = kernel.function ("do_sock_write") { ... }
probe socket.send = socket.sendmsg, socket.do_write {}

Section 3.3
Mention that arrays must be declared as global.  No local arrays allowed.

Section 3.6
Mention use of the deref() macro in embedded C for safety.  Give an example.

Section 4.1
This section mentions probe points to catch "counters rolling over".  Is this really supported?

Sections 4.1.1 & 4.1.2
Need descriptions.

Section 4.1.4
Optionalness!? Please rewrite to get rid of this word :-) 

Sections 4.2.1 - 4.2.5, 4.2.7 & 4.3.1
Need more explanation

Section 4.4
Remove "in some sequence", both places.  Begin and end probes can now be given a sequence number to control the order they are run.  If no sequence number is given, it defaults to 0.  Sequence numbers can be both positive and negative.  This is especially useful for tapset writers who may need to do some initialization in a begin probe.

Example:

In tapset file:
probe begin(-1000) { ... }

In user script:
probe begin { ... }

The user script begin probe defaults to sequence number 0, so the tapset begin probe will run first.

Section 5.1, latter half of 1st paragraph
Needs clean up and more explanation.

Section 5.6.11
Mention that the number of index values must match the number of indexes previously specified.

Section 5.7
Change "All strings are limited in length ..." to "All strings are limited in length to MAXSTRINGLEN, which defaults to 128 bytes."  Then refer back to the new limits section requested above.

Sections 5.7.2 & 5.7.3
Could use some examples.

Section 5.8.2
Clarify the difference between kernel_vr and kernel_v.  I think the info is there, I just think it would be better to explicitly explain the difference.

Section 6, 1st paragraph
The reference to statements being limited to "some number" should instead say they're limited to MAXACTION, which defaults to 1000. Then refer back to the new limits section requested above.

Section 6.0.6
Not sure why this is here. Remove.

Section 6.0.15
I'm not sure why you'd ever use a ';' by itself instead of a newline.  Explain further why you'd want to do that if, in fact, it's valid.

Section 7.3
Add explanation.

Section 7.4
Mention that MAXMAPENTRIES defaults to 2048, then refer back to the new limits section requested above.

Sections 8.4.1 - 8.4.3
Show examples of the output that results from using these functions.

Section 9.1.2
Mentions the "common trace buffer."  Users probably don't know about or care about the buffer, so remove the reference.  Simply say that log() prints the given string.

Section 9.1.3
Mentions that it prints an integer, string, or statistics value, but doesn't show how you specify them.  Also remove reference to "common trace buffer."

Section 9.1.8
Mentions "warning stream", but says nothing about what that is or what the result of using this function will be (i.e, prints the string with "WARNING:" prepended).

Sections 9.2.4 & 9.2.5
Question for mailing list:  what's the point of having both cpu() and cpuid()?  They both appear to give the same number (current cpu #).

Section 9.2.21
Explain what is meant by the "target process."

Sections 9.4 & 9.5
Add a good example of how to use the queuing functions.  I've attempted to use them and found them confusing.

Section 9.5.4
Mention that qsq_start is also how you create a queue.

Section 9.7.1
Mention that ctime always prints GMT.

Section 9.9
I think we should remove subsystem-specific functions from this document.  There are quite a few beyond just TCP already and many more will be added as we expand the tapsets.  Keeping this document up-to-date with those functions will be difficult.  Instead I suggest we document the subsystem-specific functions in the tapset-specific man pages (stapprobes.tcp(5) in this case).

Section 10

Also mention the tapset-specific man pages.  Perhaps just say run "apropos stapprobes" for a complete list:

Example:

# apropos stapprobes
stapprobes           (5)  - systemtap probe points
stapprobes.iosched [stapprobes] (5)  - systemtap IO scheduler probe points
stapprobes.netdev [stapprobes] (5)  - systemtap network device probe points
stapprobes.nfs [stapprobes] (5)  - systemtap NFS client side probe points
stapprobes.nfsd [stapprobes] (5)  - systemtap NFS server side probe points
stapprobes.pagefault [stapprobes] (5)  - systemtap pagefault probe points
stapprobes.process [stapprobes] (5)  - systemtap process probe points
stapprobes.rpc [stapprobes] (5)  - systemtap SunRPC probe points
stapprobes.scsi [stapprobes] (5)  - systemtap scsi probe points
stapprobes.signal [stapprobes] (5)  - systemtap signal probe points
stapprobes.tcp [stapprobes] (5)  - systemtap tcp probe points
stapprobes.udp [stapprobes] (5)  - systemtap udp probe points


That's it for now.  Again, sorry for the delay.

Mike Mason

Re: Language Reference, updated

[Robb Romans]

[-- Attachment #1: Type: text/plain, Size: 4028 bytes --]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

>>>>> "MM" == Mike Mason <mmlnx@us.ibm.com> writes:

Mike, thanks for the great feedback. I have implemented most of your
suggestions, except for a remaining few where I have questions.

In section 3.2.4, "Unused alias variables", where is a good explanation
for this section?

    MM> Section 3.6 Mention use of the deref() macro in embedded C for
    MM> safety.  Give an example.

I could use a little help with this. Would anyone be able to provide a
good working example?

    MM> Section 4.1 This section mentions probe points to catch
    MM> "counters rolling over".  Is this really supported?

I've removed that verbiage. OK?

    MM> Sections 4.1.1 & 4.1.2 Need descriptions.

For prefixes and suffixes, I am not sure how to correctly document these
terms.

    MM> Sections 4.2.1 - 4.2.5, 4.2.7 & 4.3.1 Need more explanation

I found information on these sections from the man page description of
"dwarf probes." That info is pretty much contained verbatim in Section
4.2 "Built-in probe point types (Dwarf probes)." There is some
redundancy with the following subsections 4.2.1 - 4.2.4. I could use
some direction on how to sort out the overlap, and how to better
document the subsections.

    MM> Sections 5.7.2 & 5.7.3 Could use some examples.

(Now 5.7.1 and 5.7.2)
I could use help finding appropriate examples here.

    MM> Section 6.0.6 Not sure why this is here. Remove.

I left in the reference to "-" and added a link to associative
arrays. Let me know the right thing to do.

    MM> Section 6.0.15 I'm not sure why you'd ever use a ';' by itself
    MM> instead of a newline.  Explain further why you'd want to do that
    MM> if, in fact, it's valid.

(Now 6.0.16)
I read the subsequent email exchange. Should I modify the text here?
Include an example?

    MM> Section 7.3 Add explanation.

I could use some direction here.

    MM> Sections 8.4.1 - 8.4.3 Show examples of the output that results
    MM> from using these functions.

I'd like to be provided with self-contained examples that I can
document, run and capture the output.

I added a section to document dbug() in 9.1.1. If appropriate, I could
use an example and some text for this.

    MM> Section 9.1.3 Mentions that it prints an integer, string, or
    MM> statistics value, but doesn't show how you specify them.  Also
    MM> remove reference to "common trace buffer."

I am confused a bit by the logging functions, their inputs and outputs,
and syntax. Any clues here would be great.

    MM> Section 9.1.8 Mentions "warning stream", but says nothing about
    MM> what that is or what the result of using this function will be
    MM> (i.e, prints the string with "WARNING:" prepended).

I didn't find what you were referring to here. Do you mean "error
stream"? Is there a better place to find documentation on these
functions?

    MM> Sections 9.2.4 & 9.2.5 Question for mailing list: what's the
    MM> point of having both cpu() and cpuid()?  They both appear to
    MM> give the same number (current cpu #).

I didn't see an answer to this. I'll be happy to change the guide to be
more correct.

    MM> Section 9.2.21 Explain what is meant by the "target process."

Does this mean the PID of the program being probed?

    MM> Sections 9.4 & 9.5 Add a good example of how to use the queuing
    MM> functions.  I've attempted to use them and found them confusing.

Let me know if I've inserted a sufficient example.

    MM> Section 9.9 I think we should remove subsystem-specific
    MM> functions from this document.  There are quite a few beyond just
    MM> TCP already and many more will be added as we expand the
    MM> tapsets.  Keeping this document up-to-date with those functions
    MM> will be difficult.  Instead I suggest we document the
    MM> subsystem-specific functions in the tapset-specific man pages
    MM> (stapprobes.tcp(5) in this case).

I didn't see any response to your suggestion. What's the verdict on
removing these sections?

Thanks.
Robb


[-- Attachment #2: SystemTap-Language-Ref-20070130.tex.gz --]
[-- Type: application/octet-stream, Size: 22878 bytes --]

[-- Attachment #3: SystemTap-Language-Ref-20070130.ps.gz --]
[-- Type: application/octet-stream, Size: 94143 bytes --]

[-- Attachment #4: Type: text/plain, Size: 425 bytes --]



- -- 
Robb Romans                     (512) 838-0419
Linux Commando                  T/L   678-0419
robb@linux.vnet.ibm.com         ARS NA5TT
.-- - ..-. ..--..
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Processed by Mailcrypt 3.5.8 <http://mailcrypt.sourceforge.net/>

iD8DBQFFv9xyZyOifwSFgjQRApIvAJ9mjIXdLqxwMWBGCnmEx4Qmqu1CVACeNhwN
+hnLvV0yKHGFgUY41dP9G4k=
=2ZFw
-----END PGP SIGNATURE-----

Re: Language Reference, updated

[Robb Romans]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi.

I'm resending this, as I received no response. Would anyone be able
to help?

Regards,
Robb


>>>>> "MM" == Mike Mason <mmlnx@us.ibm.com> writes:

Mike, thanks for the great feedback. I have implemented most of your
suggestions, except for a remaining few where I have questions.

In section 3.2.4, "Unused alias variables", where is a good explanation
for this section?

    MM> Section 3.6 Mention use of the deref() macro in embedded C for
    MM> safety.  Give an example.

I could use a little help with this. Would anyone be able to provide a
good working example?

    MM> Section 4.1 This section mentions probe points to catch
    MM> "counters rolling over".  Is this really supported?

I've removed that verbiage. OK?

    MM> Sections 4.1.1 & 4.1.2 Need descriptions.

For prefixes and suffixes, I am not sure how to correctly document these
terms.

    MM> Sections 4.2.1 - 4.2.5, 4.2.7 & 4.3.1 Need more explanation

I found information on these sections from the man page description of
"dwarf probes." That info is pretty much contained verbatim in Section
4.2 "Built-in probe point types (Dwarf probes)." There is some
redundancy with the following subsections 4.2.1 - 4.2.4. I could use
some direction on how to sort out the overlap, and how to better
document the subsections.

    MM> Sections 5.7.2 & 5.7.3 Could use some examples.

(Now 5.7.1 and 5.7.2)
I could use help finding appropriate examples here.

    MM> Section 6.0.6 Not sure why this is here. Remove.

I left in the reference to "-" and added a link to associative
arrays. Let me know the right thing to do.

    MM> Section 6.0.15 I'm not sure why you'd ever use a ';' by itself
    MM> instead of a newline.  Explain further why you'd want to do that
    MM> if, in fact, it's valid.

(Now 6.0.16)
I read the subsequent email exchange. Should I modify the text here?
Include an example?

    MM> Section 7.3 Add explanation.

I could use some direction here.

    MM> Sections 8.4.1 - 8.4.3 Show examples of the output that results
    MM> from using these functions.

I'd like to be provided with self-contained examples that I can
document, run and capture the output.

I added a section to document dbug() in 9.1.1. If appropriate, I could
use an example and some text for this.

    MM> Section 9.1.3 Mentions that it prints an integer, string, or
    MM> statistics value, but doesn't show how you specify them.  Also
    MM> remove reference to "common trace buffer."

I am confused a bit by the logging functions, their inputs and outputs,
and syntax. Any clues here would be great.

    MM> Section 9.1.8 Mentions "warning stream", but says nothing about
    MM> what that is or what the result of using this function will be
    MM> (i.e, prints the string with "WARNING:" prepended).

I didn't find what you were referring to here. Do you mean "error
stream"? Is there a better place to find documentation on these
functions?

    MM> Sections 9.2.4 & 9.2.5 Question for mailing list: what's the
    MM> point of having both cpu() and cpuid()?  They both appear to
    MM> give the same number (current cpu #).

I didn't see an answer to this. I'll be happy to change the guide to be
more correct.

    MM> Section 9.2.21 Explain what is meant by the "target process."

Does this mean the PID of the program being probed?

    MM> Sections 9.4 & 9.5 Add a good example of how to use the queuing
    MM> functions.  I've attempted to use them and found them confusing.

Let me know if I've inserted a sufficient example.

    MM> Section 9.9 I think we should remove subsystem-specific
    MM> functions from this document.  There are quite a few beyond just
    MM> TCP already and many more will be added as we expand the
    MM> tapsets.  Keeping this document up-to-date with those functions
    MM> will be difficult.  Instead I suggest we document the
    MM> subsystem-specific functions in the tapset-specific man pages
    MM> (stapprobes.tcp(5) in this case).

I didn't see any response to your suggestion. What's the verdict on
removing these sections?

Thanks.
Robb



- -- 
Robb Romans                     (512) 838-0419
Linux Commando                  T/L   678-0419
robb@linux.vnet.ibm.com         ARS NA5TT
.-- - ..-. ..--..
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Processed by Mailcrypt 3.5.8 <http://mailcrypt.sourceforge.net/>

iD8DBQFF0L5jZyOifwSFgjQRAg6UAJ4y/ol4uDOqu16n3Q+V9ZJk4d6FSwCfa4lJ
2NR/8yA0G+29dWR1hen7y08=
=xQV+
-----END PGP SIGNATURE-----

Re: Language Reference, updated

[Frank Ch. Eigler]

Robb Romans <robb@linux.vnet.ibm.com> writes:

> In section 3.2.4, "Unused alias variables", where is a good explanation
> for this section?

Do you mean what is an unused alias variable?  It is a variable
defined in a probe alias (usually as one of a group of "var = $var"
assignments), which is not actually used by the script probe that
instantiates the alias.

>     MM> Section 3.6 Mention use of the deref() macro in embedded C for
>     MM> safety.  Give an example.
> 
> I could use a little help with this. Would anyone be able to provide a
> good working example?

There are examples in the tapset.  Josh switched us over to the
"kread" function:

function _struct_timeval:string(addr:long)
%{ /* pure */
        struct timeval *tv = (struct timeval *)(unsigned long)THIS->addr;

        if (tv == NULL)
                strlcpy (THIS->__retvalue, "NULL", MAXSTRINGLEN);
        else
                snprintf(THIS->__retvalue, MAXSTRINGLEN, "[%ld.%06ld]",
                                kread(&(tv->tv_sec)), kread(&(tv->tv_usec)));
%}

(However, it is not clear that the NULL-handling case is relevant or
appropriate.  If the timeval* coming in from the application is
permitted to be NULL, fine, but if it's meant as a crash-prevention
mechanism in this embedded-C routine, then it's not right.)


> [...]
>     MM> Sections 8.4.1 - 8.4.3 Show examples of the output that results
>     MM> from using these functions.
> 
> I'd like to be provided with self-contained examples that I can
> document, run and capture the output.

If you have a working systemtap build, it should not be hard to make
such examples yourself.  Its ease-of-use is the stuff of legend ... or
in any case well tested by newbie users.

> [...]  I am confused a bit by the logging functions, their inputs
> and outputs, and syntax. Any clues here would be great.

They are documented in the man pages, and the examples have many
instances.

>     MM> Section 9.1.8 Mentions "warning stream", but says nothing about
>     MM> what that is or what the result of using this function will be
>     MM> (i.e, prints the string with "WARNING:" prepended).
> 
> I didn't find what you were referring to here. Do you mean "error
> stream"? Is there a better place to find documentation on these
> functions?

"warning stream" was unnecessary terminology.  It should simply say
"standard error" of the stap(run) process.

>     MM> Sections 9.2.4 & 9.2.5 Question for mailing list: what's the
>     MM> point of having both cpu() and cpuid()?  They both appear to
>     MM> give the same number (current cpu #).
> 
> I didn't see an answer to this. I'll be happy to change the guide to be
> more correct.

Drop cpuid().  It is not listed in stapfuncs.5.

>     MM> Section 9.2.21 Explain what is meant by the "target process."
> 
> Does this mean the PID of the program being probed?

Sort of - see the "-c PROGRAM" and "-x PID" stap options.  The man
page for "-c" should be clarified to point out that the target()
stapfuncs.5 function is set analogously to the -x case.

>     MM> Section 9.9 I think we should remove subsystem-specific
>     MM> functions from this document.  [...]
> 
> I didn't see any response to your suggestion. What's the verdict on
> removing these sections?

FWIW, I agree.

- FChE

Re: Language Reference, updated

[Mike Mason]

Frank Ch. Eigler wrote:
>>     MM> Section 3.6 Mention use of the deref() macro in embedded C for
>>     MM> safety.  Give an example.
>>
>> I could use a little help with this. Would anyone be able to provide a
>> good working example?
> 
> There are examples in the tapset.  Josh switched us over to the
> "kread" function:
> 
> function _struct_timeval:string(addr:long)
> %{ /* pure */
>         struct timeval *tv = (struct timeval *)(unsigned long)THIS->addr;
> 
>         if (tv == NULL)
>                 strlcpy (THIS->__retvalue, "NULL", MAXSTRINGLEN);
>         else
>                 snprintf(THIS->__retvalue, MAXSTRINGLEN, "[%ld.%06ld]",
>                                 kread(&(tv->tv_sec)), kread(&(tv->tv_usec)));
> %}
> 
> (However, it is not clear that the NULL-handling case is relevant or
> appropriate.  If the timeval* coming in from the application is
> permitted to be NULL, fine, but if it's meant as a crash-prevention
> mechanism in this embedded-C routine, then it's not right.)

What's the right thing to do if NULL isn't permitted?  Should we let kread() report the error or do something else?

Thanks,
Mike

RE: Language Reference, updated

[Stone, Joshua I]

Mike Mason wrote:
> Frank Ch. Eigler wrote:
>> (However, it is not clear that the NULL-handling case is relevant or
>> appropriate.  If the timeval* coming in from the application is
>> permitted to be NULL, fine, but if it's meant as a crash-prevention
>> mechanism in this embedded-C routine, then it's not right.)
> 
> What's the right thing to do if NULL isn't permitted?  Should we let
> kread() report the error or do something else? 

Yes, let kread() trap the NULL dereference.  Using sentinal values like
"NULL" just tends to obscure the problem.  With kread() it's treated as
an error and the script user is notified.

Josh

Re: Language Reference, updated

[Robb Romans]

[-- Attachment #1: Type: text/plain, Size: 2639 bytes --]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

>>>>> "FCHE" == Frank Ch Eigler <fche@redhat.com> writes:

Hi Frank, thanks for the reply.

    MM> Section 3.6 Mention use of the deref() macro in embedded C for
    MM> safety.  Give an example.
    >>  I could use a little help with this. Would anyone be able to
    >> provide a good working example?

    FCHE> There are examples in the tapset.

Sorry, do you mean there are specific examples of how to use the deref
macro? I didn't find any. 

    FCHE> Josh switched us over to the "kread" function:

    FCHE> function _struct_timeval:string(addr:long) %{ /* pure */
    FCHE> struct timeval *tv = (struct timeval *)(unsigned
    FCHE> long)THIS->addr;

    FCHE> if (tv == NULL) strlcpy (THIS->__retvalue, "NULL",
    FCHE> MAXSTRINGLEN); else snprintf(THIS->__retvalue, MAXSTRINGLEN,
    FCHE> "[%ld.%06ld]", kread(&(tv->tv_sec)), kread(&(tv->tv_usec)));
    FCHE> %}

OK, kread replaces deref? I can certainly insert this example, but I
really don't know how to document it in the Guide.

    FCHE> (However, it is not clear that the NULL-handling case is
    FCHE> relevant or appropriate.  If the timeval* coming in from the
    FCHE> application is permitted to be NULL, fine, but if it's meant
    FCHE> as a crash-prevention mechanism in this embedded-C routine,
    FCHE> then it's not right.)

If you don't know, then I certainly don't either :)

    >> [...]
    MM> Sections 8.4.1 - 8.4.3 Show examples of the output that results
    MM> from using these functions.
    >>  I'd like to be provided with self-contained examples that I can
    >> document, run and capture the output.

    FCHE> If you have a working systemtap build, it should not be hard
    FCHE> to make such examples yourself.  Its ease-of-use is the stuff
    FCHE> of legend ... or in any case well tested by newbie users.

Right, but several runs have resulted in hard lockups of my poor
workstation and I got tired of that very fast. You will see that I have
provided output in a few more of the examples in the Guide. I get your
point, but illustrative examples provided by experts in SystemTap in the
remaining areas would be helpful, IMHO.

    MM> Section 9.2.21 Explain what is meant by the "target process."
    >>  Does this mean the PID of the program being probed?

    FCHE> Sort of - see the "-c PROGRAM" and "-x PID" stap options.  The
    FCHE> man page for "-c" should be clarified to point out that the
    FCHE> target() stapfuncs.5 function is set analogously to the -x
    FCHE> case.

Please check the Guide and see if what I included is sufficient.

Here's the latest version.


[-- Attachment #2: SystemTap-Language-Ref-20070222.tex.gz --]
[-- Type: application/octet-stream, Size: 23083 bytes --]

[-- Attachment #3: SystemTap-Language-Ref-20070222.ps.gz --]
[-- Type: application/octet-stream, Size: 103689 bytes --]

[-- Attachment #4: Type: text/plain, Size: 439 bytes --]


Regards,
Robb

- -- 
Robb Romans                     (512) 838-0419
Linux Commando                  T/L   678-0419
robb@linux.vnet.ibm.com         ARS NA5TT
.-- - ..-. ..--..
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Processed by Mailcrypt 3.5.8 <http://mailcrypt.sourceforge.net/>

iD8DBQFF3gQjZyOifwSFgjQRAqlxAKC4kiD6vsHFeA0pR6fGtgR75PspIQCfdGSr
iniW2b5Ic1nEWm5mGvFUn0c=
=g2d3
-----END PGP SIGNATURE-----

Re: Language Reference, updated

[Mike Mason]

Robb,

Sorry for the delay.  Here's what I have so far.  More to come.

Mike

Robb Romans wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
>>>>>> "MM" == Mike Mason <mmlnx@us.ibm.com> writes:
> 
> Mike, thanks for the great feedback. I have implemented most of your
> suggestions, except for a remaining few where I have questions.
> 
> In section 3.2.4, "Unused alias variables", where is a good explanation
> for this section?

Looks like your updated doc has an accurate description for this section.

> 
>     MM> Section 3.6 Mention use of the deref() macro in embedded C for
>     MM> safety.  Give an example.
> 
> I could use a little help with this. Would anyone be able to provide a
> good working example?

In the sentence that starts "There are a number of undocumented..." I 
change it as follows:

"There are a number of undocumented but complex safety constraints on 
concurrency, resource consumption and runtime limits that are applied
to code written in the SystemTap language. These constraints are not
applied to embedded C code, so use with caution." 

Here's what I would say about kread():

One of the safety mechanisms automatically used in the generated C code
is the kread() macro.  This macro protects against pointer accesses that
could crash the system.  Embedded C is used verbatim, so you must 
explicitly use kread() for any pointer accesses that are at all suspect.

For example, to access the following pointer chain in embedded C:

	name = skb->dev->name

do the following:

	struct net_device *dev;
	char *name;

	dev = kread(&(skb->dev));
	name = kread(&(dev->name));

> 
>     MM> Section 4.1 This section mentions probe points to catch
>     MM> "counters rolling over".  Is this really supported?
> 
> I've removed that verbiage. OK?

Fine with me.

> 
>     MM> Sections 4.1.1 & 4.1.2 Need descriptions.
> 
> For prefixes and suffixes, I am not sure how to correctly document these
> terms.

Leave as-is. The details of prefixes and suffixes are described in section 4.2.

> 
>     MM> Sections 4.2.1 - 4.2.5, 4.2.7 & 4.3.1 Need more explanation
> 
> I found information on these sections from the man page description of
> "dwarf probes." That info is pretty much contained verbatim in Section
> 4.2 "Built-in probe point types (Dwarf probes)." There is some
> redundancy with the following subsections 4.2.1 - 4.2.4. I could use
> some direction on how to sort out the overlap, and how to better
> document the subsections.

I'll help with this in a follow-on email.

> 
>     MM> Sections 5.7.2 & 5.7.3 Could use some examples.
> 
> (Now 5.7.1 and 5.7.2)
> I could use help finding appropriate examples here.


Add Section 5.7.3 Example:

For example, if the following script(example.stp):

probe begin { printf("%d, %s\n", $1, @2) }

is invoked as follows:

# stap example.stp 10 mystring

then 10 is substituted for $1 and "mystring" for @2.
The output will be:

10, mystring

> 
>     MM> Section 6.0.6 Not sure why this is here. Remove.
> 
> I left in the reference to "-" and added a link to associative
> arrays. Let me know the right thing to do.

I don't think this has anything to do with associative arrays.  I'm not sure what "- (delete) <name>" is telling me.  Unless we can come up with an example of it's use, I still think it should be removed.

> 
>     MM> Section 6.0.15 I'm not sure why you'd ever use a ';' by itself
>     MM> instead of a newline.  Explain further why you'd want to do that
>     MM> if, in fact, it's valid.
> 
> (Now 6.0.16)
> I read the subsequent email exchange. Should I modify the text here?
> Include an example?

Leave as is for now.

> 
>     MM> Section 7.3 Add explanation.
> 
> I could use some direction here.


> 
>     MM> Sections 8.4.1 - 8.4.3 Show examples of the output that results
>     MM> from using these functions.
> 
> I'd like to be provided with self-contained examples that I can
> document, run and capture the output.

hist_linear example script:

global reads
probe netdev.receive { reads <<< length }
probe end { print(@hist_linear(reads, 0, 10240, 200)) }

Output from script:

value |-------------------------------------------------- count
   0 |                                                     258
 200 |                                                       3
 400 |                                                       0
 600 |                                                       1
 800 |                                                       2
1000 |                                                       4
1200 |                                                       1
1400 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@  16685
1600 |                                                       0
1800 |                                                       0


hist_log example script:

global reads
probe netdev.receive { reads <<< length }
probe end { print(@hist_log(reads)) }

Output from script:

value |-------------------------------------------------- count
   8 |                                                       0
  16 |                                                       0
  32 |                                                     254
  64 |                                                       3
 128 |                                                       2
 256 |                                                       2
 512 |                                                       4
1024 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@  16689
2048 |                                                       0
4096 |                                                       0

> 
> I added a section to document dbug() in 9.1.1. If appropriate, I could
> use an example and some text for this.

dbug() is an internal function used print debug statements in 
the runtime code.  It shouldn't be included in the language reference.

> 
>     MM> Section 9.1.3 Mentions that it prints an integer, string, or
>     MM> statistics value, but doesn't show how you specify them.  Also
>     MM> remove reference to "common trace buffer."
> 
> I am confused a bit by the logging functions, their inputs and outputs,
> and syntax. Any clues here would be great.




> 
>     MM> Section 9.1.8 Mentions "warning stream", but says nothing about
>     MM> what that is or what the result of using this function will be
>     MM> (i.e, prints the string with "WARNING:" prepended).
> 
> I didn't find what you were referring to here. Do you mean "error
> stream"? Is there a better place to find documentation on these
> functions?




> 
>     MM> Sections 9.2.4 & 9.2.5 Question for mailing list: what's the
>     MM> point of having both cpu() and cpuid()?  They both appear to
>     MM> give the same number (current cpu #).
> 
> I didn't see an answer to this. I'll be happy to change the guide to be
> more correct.

Remove any mention of cpuid().

> 
>     MM> Section 9.2.21 Explain what is meant by the "target process."
> 
> Does this mean the PID of the program being probed?

The following stap options set the target pid:

-x <pid>       target() returns the pid specified by -x
-c <command>   target() returns the pid for the <command> that's
              run by specifying -c.

> 
>     MM> Sections 9.4 & 9.5 Add a good example of how to use the queuing
>     MM> functions.  I've attempted to use them and found them confusing.
> 
> Let me know if I've inserted a sufficient example.



> 
>     MM> Section 9.9 I think we should remove subsystem-specific
>     MM> functions from this document.  There are quite a few beyond just
>     MM> TCP already and many more will be added as we expand the
>     MM> tapsets.  Keeping this document up-to-date with those functions
>     MM> will be difficult.  Instead I suggest we document the
>     MM> subsystem-specific functions in the tapset-specific man pages
>     MM> (stapprobes.tcp(5) in this case).
> 
> I didn't see any response to your suggestion. What's the verdict on
> removing these sections?

The consensus from those who expressed an opinion is we should remove them.

> 
> Thanks.
> Robb
> 
> 
> 
> ------------------------------------------------------------------------
> 
> 
> 
> - -- 
> Robb Romans                     (512) 838-0419
> Linux Commando                  T/L   678-0419
> robb@linux.vnet.ibm.com         ARS NA5TT
> .-- - ..-. ..--..
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
> Comment: Processed by Mailcrypt 3.5.8 <http://mailcrypt.sourceforge.net/>
> 
> iD8DBQFFv9xyZyOifwSFgjQRApIvAJ9mjIXdLqxwMWBGCnmEx4Qmqu1CVACeNhwN
> +hnLvV0yKHGFgUY41dP9G4k=
> =2ZFw
> -----END PGP SIGNATURE-----

Re: Language Reference, updated

[Robb Romans]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

>>>>> "MM" == Mike Mason <mmlnx@us.ibm.com> writes:

    MM> Robb, Sorry for the delay.  Here's what I have so far.  More to
    MM> come.

Hi Mike. Thanks for the feedback; I'll work to get your input in the doc
ASAP. For your next round of comments, please use the latest version
that I sent you this morning, as sections have changed since your
original email.

I'd really like to see the Language Reference go into SystemTap CVS as
soon as possible. Are you willing to push for that after I have
incorporated your suggested changes?


Regards,
Robb


- -- 
Robb Romans                     (512) 838-0419
Linux Commando                  T/L   678-0419
robb@linux.vnet.ibm.com         ARS NA5TT
.-- - ..-. ..--..
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Processed by Mailcrypt 3.5.8 <http://mailcrypt.sourceforge.net/>

iD8DBQFF5HN8ZyOifwSFgjQRAvjUAJ9ZzXtIFGMV/16sRA75pEdzlmFoYQCgqUci
5biBJvC3TioBowGQTvu0uRM=
=4PnC
-----END PGP SIGNATURE-----

Re: Language Reference, updated

[Robb Romans]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

>>>>> "RR" == Robb Romans <robb@linux.vnet.ibm.com> writes:

>>>>> "MM" == Mike Mason <mmlnx@us.ibm.com> writes:
    MM> Robb, Sorry for the delay.  Here's what I have so far.  More to
    MM> come.

    RR> Hi Mike. Thanks for the feedback; I'll work to get your input in
    RR> the doc ASAP. For your next round of comments, please use the
    RR> latest version that I sent you this morning, as sections have
    RR> changed since your original email.

    RR> I'd really like to see the Language Reference go into SystemTap
    RR> CVS as soon as possible. Are you willing to push for that after
    RR> I have incorporated your suggested changes?

Obviously, that was not intended to go to the entire list. Please
accept my apologies.

Regards,
Robb

- -- 
Robb Romans                     (512) 838-0419
Linux Commando                  T/L   678-0419
robb@linux.vnet.ibm.com         ARS NA5TT
.-- - ..-. ..--..
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Processed by Mailcrypt 3.5.8 <http://mailcrypt.sourceforge.net/>

iD8DBQFF5IBsZyOifwSFgjQRAojDAKDJhOdUj4bi+7dujEiV4hjq6RY4XwCgypjr
CEhSmae4lyMJPFZ4ith9OeM=
=RGuO
-----END PGP SIGNATURE-----

RE: Language Reference, updated

[Stone, Joshua I]

On Tuesday, December 12, 2006 11:03 AM, Mike Mason wrote:
> Section 6.0.15
> I'm not sure why you'd ever use a ';' by itself instead of a newline.
> Explain further why you'd want to do that if, in fact, it's valid. 

The semicolon is necessary because it's optional.  :-/

Said with less tongue-in-cheek: There are certain grammatical
ambiguities that are due to the fact that the language has no statement
separator.  Remember that statements may also span lines, so a newline
itself is not a separator.  The semicolon then becomes necessary to make
it clear what you intend.  For example:

  probe foo {
    a = b
    ++c
  }

The spacing indicates what I want to happen, but the actual parsing will
produce this:

  probe foo {
    (a) = ((b)++)
    c
  }

Surprise!  Possible fixes here are to add a semicolon after 'b', reorder
'++c' before the assignment, or change it to a postincrement, 'c++'.

You might conclude that it's best to always include a semicolon as a
statement separator, but even that can backfire.  Another example:

  probe foo {
    if (a)
      printf("a is true\n");
    else            
      printf("a is false\n");
  }

This will give you "parse error: expected statement", pointing to the
'else' as the problem.  This is because the semicolon gets treated as a
separate statement.  Thus with two statements after an 'if' with no
curly braces, the 'else' becomes invalid syntax.

The second example might actually be a bug, depending on the intentions
of the language designers.  Frank, care to comment?


Josh

Re: Language Reference, updated

[Frank Ch. Eigler]

josh wrote:

> The semicolon is necessary because it's optional.  :-/ [...]
> 
> Said with less tongue-in-cheek: There are certain grammatical
> ambiguities that are due to the fact that the language has no statement
> separator.  

Yes. (Or, alternately, because it has both prefix and postfix operators.)

> [...] The spacing indicates what I want to happen, but the actual
> parsing will produce this:
>   probe foo {
>     (a) = ((b)++)
>     c
>   }
> Surprise!  [...]

We could have a "DWIM" post-parse pass, which would emit warnings if
suspicious line spans occur in expression statements.

> You might conclude that it's best to always include a semicolon as a
> statement separator, but even that can backfire.  [if () foo; else bar; ]
> This will give you "parse error: expected statement" [...]

I'm ambivalent about how problematic this is.  Awk and tcl, for
example are newline sensitive (";" is a separator for multiple
statements on a single line).  We all know how python feels.  IIRC
dtrace takes the C route.

- FChE