new benchmark framework

new benchmark framework

[Martin Hunt]

I've checked in a new benchmark framework that makes it easy to see how
fast a code fragment is running. I've also got some simple tests
included.

Code is in src/runtime/bench2. It is self-contained and can be moved
anywhere. It uses the installed systemtap.

To write a test, simply create a file and put the following two lines at
the top:

---
#!/usr/bin/env ruby
load './bench.rb'

#Then create a test like this:

# script test to print 5 integers
test4 = Stapbench.new("printf 5 integers")
test4.code = "printf(\"%d, %d, %d, %d, %d\\n\", 1, 0xffff, 0x8000ffff,
0xffff000011112222, 0x7000000000000000)"
test4.run
test4.print
---

Then run the above file. That's it. 

It can test C code as well as Systemtap script. See the examples.

Sample output from "run_bench":

SystemTap BENCH2        Wed Mar 15 01:00:52 PST 2006
kernel: 2.6.9-34.ELsmp x86_64
Red Hat Enterprise Linux WS release 4 (Nahant Update 3)
tiger:  01:00:52 up 51 min,  1 user,  load average: 1.46, 1.29, 0.87
processors: 4 (2 physical)                   Intel(R) Xeon(TM) CPU
2.80GHz
MemTotal:      1025428 kB       MemFree:        750220 kB
----------------------------------------------------------------
For comparison, function call overhead is 721 nsecs.
Times below are nanoseconds per probe and include kprobe overhead.
----------------------------------------------------------------
+--- S = Script, R = Runtime
|+-- * = Relayfs                Threads
|| NAME                         1       2       4
R : empty probe                 1471    737     559
S : empty probe                 1556    784     594
S : printf 100 chars            2178    1445    1148
S*: printf 100 chars            2184    1290    1108
R : printf 100 chars            2064    1333    1039
R : printf 5 integers           3793    2006    1589
S : printf 5 integers           3930    2136    1644

You can see the overhead for printing integers is high. We should be
able to improve on all the printf numbers soon.

Martin

[patch] ppc64 patch for new benchmark framework

[Gui,Jian]

[-- Attachment #1: Type: text/plain, Size: 3400 bytes --]

Hi, Martin

This patch is to fix three maybe bugs of the new benchmark framework
on ppc64.

This framework works fine on i686 but fails on my ppc64/2.6.9-34.EL
for the following reasons:
1) The function kallsyms_lookup_name("sys_getuid") always return 0
  on ppc64 and we have to use kallsysms_lookup_name(".sys_getuid")
  to get the function entry address instead;
2) The content of /proc/cpuinfo on ppc64 is different from on i686.
  It is like:
  processor       : 0
  cpu             : POWER5 (gr)
  clock           : 1502.496000MHz
  revision        : 2.2
  There is no "model name" or "physical id" information, thus current
  print_header always failed;
3) The assignment of (.addr = "sys_getuid") will trigger a compilation
  error like: "warning: initialization from incompatible pointer type"
  (I'm using gcc-3.4.5-2 + glibc-2.3.4-2.19)

This patch works on ppc64/2.6.9-34.EL and i686/2.6.15.6 and should
apply to snapshot-20060318.

Could you give some comments?  Thanks.

Gui,Jian

diff -Nur src-20060318/runtime/bench2/bench.rb 
src-modified/runtime/bench2/bench.rb
--- src-20060318/runtime/bench2/bench.rb        2006-03-24 
09:03:53.000000000 -0500
+++ src-modified/runtime/bench2/bench.rb        2006-03-24 
09:06:26.000000000 -0500
@@ -149,9 +149,14 @@
 MODULE_AUTHOR(\"automatically generated by bench2/run_bench\");\n\n"
        f << "static int inst_sys_getuid (struct kprobe *p, struct pt_regs 
*regs) {\n"
        f << @code
-       f << "\n  return 0;\n}
-static struct kprobe kp[] = {\n  {\n    .addr = \"sys_getuid\",
-    .pre_handler = inst_sys_getuid\n  }\n};\n
+       f << "\n  return 0;\n}\n"
+       f << " static struct kprobe kp[] = {\n  {\n"
+       f << "#if defined __powerpc64__ \n"
+       f << "  .addr = (void *)\".sys_getuid\",\n"
+       f << "#else \n"
+       f << "  .addr = \"sys_getuid\", \n"
+       f << "#endif\n"
+       f << ".pre_handler = inst_sys_getuid\n  }\n};\n
 #define NUM_KPROBES 1\n
 int probe_start(void)\n{\n  return _stp_register_kprobes (kp, 
NUM_KPROBES);\n}\n
 void probe_exit (void)\n{\n  _stp_unregister_kprobes (kp, NUM_KPROBES); 
\n}\n"
@@ -185,8 +190,17 @@
     else
       @@num_threads = [1]
     end
-    physical_cpus=`grep "physical id" 
/proc/cpuinfo`.split("\n").uniq.length
-    model=`grep "model name" /proc/cpuinfo`.match(/(model 
name\t: )([^\n]*)/)[2]
+    arch=`uname -m`.strip
+    if (arch.match(/ppc64/))
+       cpu=`grep "cpu" /proc/cpuinfo`.match(/(cpu\t\t: )([^\n]*)/)[2]
+       clock=`grep "clock" /proc/cpuinfo`.match(/(clock\t\t: )([^\n]*)/)[2]
+       revision=`grep "revision" 
/proc/cpuinfo`.match(/(revision\t: )([^\n]*)/)[2]
+       cpuinfo=cpu + " " + clock + " revision: " + revision
+    else
+               physical_cpus=`grep "physical id" 
/proc/cpuinfo`.split("\n").uniq.length
+               model=`grep "model name" /proc/cpuinfo`.match(/(model 
name\t: )([^\n]*)/)[2]
+       cpuinfo="(#{physical_cpus} physical) #{model}"
+    end
     puts "SystemTap BENCH2 \t" + `date`
     puts "kernel: " + `uname -r`.strip + " " + `uname -m`.strip
     begin
@@ -195,11 +209,7 @@
     end

     puts `uname -n`.strip + ": " + `uptime`
-    if nproc > 1
-      puts "processors: #{nproc} (#{physical_cpus} physical) #{model}"
-    else
-      puts "processors: #{nproc} #{model}"
-    end
+    puts "processors: #{nproc} #{cpuinfo}"

     begin
       mem=IO.read("/proc/meminfo").split("\n") 

[-- Attachment #2: bench2.ppc64.patch --]
[-- Type: application/octet-stream, Size: 2326 bytes --]

diff -Nur src-20060318/runtime/bench2/bench.rb src-modified/runtime/bench2/bench.rb
--- src-20060318/runtime/bench2/bench.rb	2006-03-24 09:03:53.000000000 -0500
+++ src-modified/runtime/bench2/bench.rb	2006-03-24 09:06:26.000000000 -0500
@@ -149,9 +149,14 @@
 MODULE_AUTHOR(\"automatically generated by bench2/run_bench\");\n\n"
 	f << "static int inst_sys_getuid (struct kprobe *p, struct pt_regs *regs) {\n"
 	f << @code
-	f << "\n  return 0;\n}
-static struct kprobe kp[] = {\n  {\n    .addr = \"sys_getuid\",
-    .pre_handler = inst_sys_getuid\n  }\n};\n
+	f << "\n  return 0;\n}\n"
+	f << " static struct kprobe kp[] = {\n  {\n"
+	f << "#if defined __powerpc64__ \n"
+	f << "  .addr = (void *)\".sys_getuid\",\n"
+	f << "#else \n"
+	f << "  .addr = \"sys_getuid\", \n"
+	f << "#endif\n"
+	f << ".pre_handler = inst_sys_getuid\n  }\n};\n
 #define NUM_KPROBES 1\n
 int probe_start(void)\n{\n  return _stp_register_kprobes (kp, NUM_KPROBES);\n}\n
 void probe_exit (void)\n{\n  _stp_unregister_kprobes (kp, NUM_KPROBES); \n}\n"
@@ -185,8 +190,17 @@
     else
       @@num_threads = [1]
     end
-    physical_cpus=`grep "physical id" /proc/cpuinfo`.split("\n").uniq.length
-    model=`grep "model name" /proc/cpuinfo`.match(/(model name\t: )([^\n]*)/)[2]
+    arch=`uname -m`.strip
+    if (arch.match(/ppc64/))
+	cpu=`grep "cpu" /proc/cpuinfo`.match(/(cpu\t\t: )([^\n]*)/)[2]
+	clock=`grep "clock" /proc/cpuinfo`.match(/(clock\t\t: )([^\n]*)/)[2]
+	revision=`grep "revision" /proc/cpuinfo`.match(/(revision\t: )([^\n]*)/)[2]
+	cpuinfo=cpu + " " + clock + " revision: " + revision
+    else
+       	physical_cpus=`grep "physical id" /proc/cpuinfo`.split("\n").uniq.length
+       	model=`grep "model name" /proc/cpuinfo`.match(/(model name\t: )([^\n]*)/)[2]
+	cpuinfo="(#{physical_cpus} physical) #{model}"
+    end	
     puts "SystemTap BENCH2 \t" + `date`
     puts "kernel: " + `uname -r`.strip + " " + `uname -m`.strip
     begin
@@ -195,11 +209,7 @@
     end
 
     puts `uname -n`.strip + ": " + `uptime`
-    if nproc > 1
-      puts "processors: #{nproc} (#{physical_cpus} physical) #{model}"
-    else
-      puts "processors: #{nproc} #{model}"
-    end
+    puts "processors: #{nproc} #{cpuinfo}"
 
     begin
       mem=IO.read("/proc/meminfo").split("\n")

Re: [patch] ppc64 patch for new benchmark framework

[Martin Hunt]

On Fri, 2006-03-24 at 18:37 +0800, Gui,Jian wrote:
> This patch is to fix three maybe bugs of the new benchmark framework
> on ppc64.

Thank you for the patch. I have applied it and checked the new code into
CVS.

Martin