* Committed version of autofdo testing&building patches
@ 2016-06-23 14:37 Andi Kleen
2016-06-23 14:38 ` [PATCH 1/3] Add gcc-auto-profile script Andi Kleen
0 siblings, 1 reply; 23+ messages in thread
From: Andi Kleen @ 2016-06-23 14:37 UTC (permalink / raw)
To: gcc-patches
I committed the autofdo bootstrap and testing patches now.
I did some small changes to the patchkit, so I'm reposting the final
committed version:
- Addressed Jeff's comments
- Fixed the grep code in the scripts
- Unsupported tooling is now reported as unsupported
- Unified tree bootstrap is tested and works
- I fixed some problems I added to the TCL code for testing, now
all the profiling test cases run again. Currently there are some new
failures with autofdo available due to differences to the instrumented bootstrap;
I'll address those separately.
-Andi
^ permalink raw reply [flat|nested] 23+ messages in thread
* [PATCH 1/3] Add gcc-auto-profile script
2016-06-23 14:37 Committed version of autofdo testing&building patches Andi Kleen
@ 2016-06-23 14:38 ` Andi Kleen
2016-06-23 14:38 ` [PATCH 2/3] Run profile feedback tests with autofdo Andi Kleen
0 siblings, 1 reply; 23+ messages in thread
From: Andi Kleen @ 2016-06-23 14:38 UTC (permalink / raw)
To: gcc-patches
From: ak <ak@138bc75d-0d04-0410-961f-82ee72b054a4>
Using autofdo is currently something difficult. It requires using the
model specific branches taken event, which differs on different CPUs.
The example shown in the manual requires a special patched version of
perf that is non standard, and also will likely not work everywhere.
This patch adds a new gcc-auto-profile script that figures out the
correct event and runs perf.
This is needed to actually make use of autofdo in a generic way
in the build system and in the test suite.
Since maintaining the script would be somewhat tedious (needs changes
every time a new CPU comes out) I auto generated it from the online
Intel event database. The script to do that is in contrib and can be
rerun.
Right now there is no test if perf works in configure. This
would vary depending on the build and target system, and since
it currently doesn't work in virtualization and needs uptodate
kernel it may often fail in common distribution build setups.
So far the script is not installed.
gcc/:
2016-06-23 Andi Kleen <ak@linux.intel.com>
* config/i386/gcc-auto-profile: New file.
contrib/:
2016-06-23 Andi Kleen <ak@linux.intel.com>
* gen_autofdo_event.py: New file to regenerate
gcc-auto-profile.
git-svn-id: svn+ssh://gcc.gnu.org/svn/gcc/trunk@237731 138bc75d-0d04-0410-961f-82ee72b054a4
---
contrib/ChangeLog | 5 ++
contrib/gen_autofdo_event.py | 155 +++++++++++++++++++++++++++++++++++++++
gcc/ChangeLog | 4 +
gcc/config/i386/gcc-auto-profile | 70 ++++++++++++++++++
4 files changed, 234 insertions(+)
create mode 100755 contrib/gen_autofdo_event.py
create mode 100755 gcc/config/i386/gcc-auto-profile
diff --git a/contrib/ChangeLog b/contrib/ChangeLog
index 8e6823d..07019c2 100644
--- a/contrib/ChangeLog
+++ b/contrib/ChangeLog
@@ -1,3 +1,8 @@
+2016-06-23 Andi Kleen <ak@linux.intel.com>
+
+ * gen_autofdo_event.py: New file to regenerate
+ gcc-auto-profile.
+
2016-06-21 Trevor Saunders <tbsaunde+gcc@tbsaunde.org>
* config-list.mk: Stop testing mep-elf.
diff --git a/contrib/gen_autofdo_event.py b/contrib/gen_autofdo_event.py
new file mode 100755
index 0000000..3865cbb
--- /dev/null
+++ b/contrib/gen_autofdo_event.py
@@ -0,0 +1,155 @@
+#!/usr/bin/python
+# Generate Intel taken branches Linux perf event script for autofdo profiling.
+
+# Copyright (C) 2016 Free Software Foundation, Inc.
+#
+# GCC is free software; you can redistribute it and/or modify it under
+# the terms of the GNU General Public License as published by the Free
+# Software Foundation; either version 3, or (at your option) any later
+# version.
+#
+# GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+# WARRANTY; without even the implied warranty of MERCHANTABILITY or
+# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+# for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING3. If not see
+# <http://www.gnu.org/licenses/>. */
+
+# Run it with perf record -b -e EVENT program ...
+# The Linux Kernel needs to support the PMU of the current CPU, and
+# It will likely not work in VMs.
+# Add --all to print for all cpus, otherwise for current cpu.
+# Add --script to generate shell script to run correct event.
+#
+# Requires internet (https) access. This may require setting up a proxy
+# with export https_proxy=...
+#
+import urllib2
+import sys
+import json
+import argparse
+import collections
+
+baseurl = "https://download.01.org/perfmon"
+
+target_events = (u'BR_INST_RETIRED.NEAR_TAKEN',
+ u'BR_INST_EXEC.TAKEN',
+ u'BR_INST_RETIRED.TAKEN_JCC',
+ u'BR_INST_TYPE_RETIRED.COND_TAKEN')
+
+ap = argparse.ArgumentParser()
+ap.add_argument('--all', '-a', help='Print for all CPUs', action='store_true')
+ap.add_argument('--script', help='Generate shell script', action='store_true')
+args = ap.parse_args()
+
+eventmap = collections.defaultdict(list)
+
+def get_cpu_str():
+ with open('/proc/cpuinfo', 'r') as c:
+ vendor, fam, model = None, None, None
+ for j in c:
+ n = j.split()
+ if n[0] == 'vendor_id':
+ vendor = n[2]
+ elif n[0] == 'model' and n[1] == ':':
+ model = int(n[2])
+ elif n[0] == 'cpu' and n[1] == 'family':
+ fam = int(n[3])
+ if vendor and fam and model:
+ return "%s-%d-%X" % (vendor, fam, model), model
+ return None, None
+
+def find_event(eventurl, model):
+ print >>sys.stderr, "Downloading", eventurl
+ u = urllib2.urlopen(eventurl)
+ events = json.loads(u.read())
+ u.close()
+
+ found = 0
+ for j in events:
+ if j[u'EventName'] in target_events:
+ event = "cpu/event=%s,umask=%s/" % (j[u'EventCode'], j[u'UMask'])
+ if u'PEBS' in j and j[u'PEBS'] > 0:
+ event += "p"
+ if args.script:
+ eventmap[event].append(model)
+ else:
+ print j[u'EventName'], "event for model", model, "is", event
+ found += 1
+ return found
+
+if not args.all:
+ cpu, model = get_cpu_str()
+ if not cpu:
+ sys.exit("Unknown CPU type")
+
+url = baseurl + "/mapfile.csv"
+print >>sys.stderr, "Downloading", url
+u = urllib2.urlopen(url)
+found = 0
+cpufound = 0
+for j in u:
+ n = j.rstrip().split(',')
+ if len(n) >= 4 and (args.all or n[0] == cpu) and n[3] == "core":
+ if args.all:
+ vendor, fam, model = n[0].split("-")
+ model = int(model, 16)
+ cpufound += 1
+ found += find_event(baseurl + n[2], model)
+u.close()
+
+if args.script:
+ print '''#!/bin/sh
+# Profile workload for gcc profile feedback (autofdo) using Linux perf.
+# Auto generated. To regenerate for new CPUs run
+# contrib/gen_autofdo_event.py --shell --all in gcc source
+
+# usages:
+# gcc-auto-profile program (profile program and children)
+# gcc-auto-profile -a sleep X (profile all for X secs, may need root)
+# gcc-auto-profile -p PID sleep X (profile PID)
+# gcc-auto-profile --kernel -a sleep X (profile kernel)
+# gcc-auto-profile --all -a sleep X (profile kernel and user space)
+
+# Identify branches taken event for CPU.
+#
+
+FLAGS=u
+
+if [ "$1" = "--kernel" ] ; then
+ FLAGS=k
+ shift
+fi
+if [ "$1" = "--all" ] ; then
+ FLAGS=uk
+ shift
+fi
+
+if ! grep -q Intel /proc/cpuinfo ; then
+ echo >&2 "Only Intel CPUs supported"
+ exit 1
+fi
+
+if grep -q hypervisor /proc/cpuinfo ; then
+ echo >&2 "Warning: branch profiling may not be functional in VMs"
+fi
+
+case `egrep -q "^cpu family\s*: 6" /proc/cpuinfo &&
+ egrep "^model\s*:" /proc/cpuinfo | head -n1` in'''
+ for event, mod in eventmap.iteritems():
+ for m in mod[:-1]:
+ print "model*:\ %s|\\" % m
+ print 'model*:\ %s) E="%s$FLAGS" ;;' % (mod[-1], event)
+ print '''*)
+echo >&2 "Unknown CPU. Run contrib/gen_autofdo_event.py --all --script to update script."
+ exit 1 ;;'''
+ print "esac"
+ print 'exec perf record -e $E -b "$@"'
+
+if cpufound == 0 and not args.all:
+ sys.exit('CPU %s not found' % cpu)
+
+if found == 0:
+ sys.exit('Branch event not found')
diff --git a/gcc/ChangeLog b/gcc/ChangeLog
index 850a3ca..84df8a7 100644
--- a/gcc/ChangeLog
+++ b/gcc/ChangeLog
@@ -1,3 +1,7 @@
+2016-06-23 Andi Kleen <ak@linux.intel.com>
+
+ * config/i386/gcc-auto-profile: New file.
+
2016-06-23 Martin Liska <mliska@suse.cz>
PR middle-end/71619
diff --git a/gcc/config/i386/gcc-auto-profile b/gcc/config/i386/gcc-auto-profile
new file mode 100755
index 0000000..5da5c63
--- /dev/null
+++ b/gcc/config/i386/gcc-auto-profile
@@ -0,0 +1,70 @@
+#!/bin/sh
+# profile workload for gcc profile feedback (autofdo) using Linux perf
+# auto generated. to regenerate for new CPUs run
+# contrib/gen_autofdo_event.py --shell --all in gcc source
+
+# usages:
+# gcc-auto-profile program (profile program and children)
+# gcc-auto-profile -a sleep X (profile all for X secs, may need root)
+# gcc-auto-profile -p PID sleep X (profile PID)
+# gcc-auto-profile --kernel -a sleep X (profile kernel)
+# gcc-auto-profile --all -a sleep X (profile kernel and user space)
+
+# identify branches taken event for CPU
+#
+
+FLAGS=u
+
+if [ "$1" = "--kernel" ] ; then
+ FLAGS=k
+ shift
+fi
+if [ "$1" = "--all" ] ; then
+ FLAGS=uk
+ shift
+fi
+
+if ! grep -q Intel /proc/cpuinfo ; then
+ echo >&2 "Only Intel CPUs supported"
+ exit 1
+fi
+
+if grep -q hypervisor /proc/cpuinfo ; then
+ echo >&2 "Warning: branch profiling may not be functional in VMs"
+fi
+
+case `egrep -q "^cpu family\s*: 6" /proc/cpuinfo &&
+ egrep "^model\s*:" /proc/cpuinfo | head -n1` in
+model*:\ 55|\
+model*:\ 77|\
+model*:\ 76) E="cpu/event=0xC4,umask=0xFE/p$FLAGS" ;;
+model*:\ 42|\
+model*:\ 45|\
+model*:\ 58|\
+model*:\ 62|\
+model*:\ 60|\
+model*:\ 69|\
+model*:\ 70|\
+model*:\ 63|\
+model*:\ 61|\
+model*:\ 71|\
+model*:\ 86|\
+model*:\ 78|\
+model*:\ 94) E="cpu/event=0xC4,umask=0x20/p$FLAGS" ;;
+model*:\ 46|\
+model*:\ 30|\
+model*:\ 31|\
+model*:\ 26|\
+model*:\ 47|\
+model*:\ 37|\
+model*:\ 44) E="cpu/event=0x88,umask=0x40/p$FLAGS" ;;
+model*:\ 28|\
+model*:\ 38|\
+model*:\ 39|\
+model*:\ 54|\
+model*:\ 53) E="cpu/event=0x88,umask=0x41/p$FLAGS" ;;
+*)
+echo >&2 "Unknown CPU. Run contrib/gen_autofdo_event.py --all --script to update script."
+ exit 1 ;;
+esac
+exec perf record -e $E -b "$@"
--
2.8.3
^ permalink raw reply [flat|nested] 23+ messages in thread
* [PATCH 2/3] Run profile feedback tests with autofdo
2016-06-23 14:38 ` [PATCH 1/3] Add gcc-auto-profile script Andi Kleen
@ 2016-06-23 14:38 ` Andi Kleen
2016-07-12 15:13 ` Bin.Cheng
0 siblings, 1 reply; 23+ messages in thread
From: Andi Kleen @ 2016-06-23 14:38 UTC (permalink / raw)
To: gcc-patches
From: ak <ak@138bc75d-0d04-0410-961f-82ee72b054a4>
Extend the existing bprob and tree-prof tests to also run with autofdo.
The test runtimes are really a bit too short for autofdo, but it's
a reasonable sanity check.
This only works natively for now.
dejagnu doesn't seem to support a wrapper for unix tests, so I had
to open code running these tests. That should be ok due to the
native run restrictions.
gcc/testsuite/:
2016-06-23 Andi Kleen <ak@linux.intel.com>
* g++.dg/bprob/bprob.exp: Support autofdo.
* g++.dg/tree-prof/tree-prof.exp: dito.
* gcc.dg/tree-prof/tree-prof.exp: dito.
* gcc.misc-tests/bprob.exp: dito.
* gfortran.dg/prof/prof.exp: dito.
* lib/profopt.exp: dito.
* lib/target-supports.exp: Check for autofdo.
git-svn-id: svn+ssh://gcc.gnu.org/svn/gcc/trunk@237732 138bc75d-0d04-0410-961f-82ee72b054a4
---
gcc/testsuite/ChangeLog | 10 ++++
gcc/testsuite/g++.dg/bprob/bprob.exp | 8 +++
gcc/testsuite/g++.dg/tree-prof/tree-prof.exp | 8 +++
gcc/testsuite/gcc.dg/tree-prof/tree-prof.exp | 8 +++
gcc/testsuite/gcc.misc-tests/bprob.exp | 7 +++
gcc/testsuite/gfortran.dg/prof/prof.exp | 7 +++
gcc/testsuite/lib/profopt.exp | 82 +++++++++++++++++++++++++++-
gcc/testsuite/lib/target-supports.exp | 37 +++++++++++++
8 files changed, 164 insertions(+), 3 deletions(-)
diff --git a/gcc/testsuite/ChangeLog b/gcc/testsuite/ChangeLog
index 287baf6..55f8dbf 100644
--- a/gcc/testsuite/ChangeLog
+++ b/gcc/testsuite/ChangeLog
@@ -1,3 +1,13 @@
+2016-06-23 Andi Kleen <ak@linux.intel.com>
+
+ * g++.dg/bprob/bprob.exp: Support autofdo.
+ * g++.dg/tree-prof/tree-prof.exp: dito.
+ * gcc.dg/tree-prof/tree-prof.exp: dito.
+ * gcc.misc-tests/bprob.exp: dito.
+ * gfortran.dg/prof/prof.exp: dito.
+ * lib/profopt.exp: dito.
+ * lib/target-supports.exp: Check for autofdo.
+
2016-06-23 Martin Liska <mliska@suse.cz>
* gcc.dg/pr71619.c: New test.
diff --git a/gcc/testsuite/g++.dg/bprob/bprob.exp b/gcc/testsuite/g++.dg/bprob/bprob.exp
index d555507..4818298 100644
--- a/gcc/testsuite/g++.dg/bprob/bprob.exp
+++ b/gcc/testsuite/g++.dg/bprob/bprob.exp
@@ -53,6 +53,7 @@ if $tracelevel then {
set profile_options "-fprofile-arcs"
set feedback_options "-fbranch-probabilities"
+set profile_wrapper ""
# Main loop.
foreach profile_option $profile_options feedback_option $feedback_options {
@@ -65,4 +66,11 @@ foreach profile_option $profile_options feedback_option $feedback_options {
}
}
+foreach src [lsort [glob -nocomplain $srcdir/$subdir/*.C]] {
+ if ![runtest_file_p $runtests $src] then {
+ continue
+ }
+ auto-profopt-execute $src
+}
+
set PROFOPT_OPTIONS $bprob_save_profopt_options
diff --git a/gcc/testsuite/g++.dg/tree-prof/tree-prof.exp b/gcc/testsuite/g++.dg/tree-prof/tree-prof.exp
index 7a4b5cb..26ee0b3 100644
--- a/gcc/testsuite/g++.dg/tree-prof/tree-prof.exp
+++ b/gcc/testsuite/g++.dg/tree-prof/tree-prof.exp
@@ -44,6 +44,7 @@ set PROFOPT_OPTIONS [list {}]
# profile data.
set profile_option "-fprofile-generate -D_PROFILE_GENERATE"
set feedback_option "-fprofile-use -D_PROFILE_USE"
+set profile_wrapper ""
foreach src [lsort [glob -nocomplain $srcdir/$subdir/*.C]] {
# If we're only testing specific files and this isn't one of them, skip it.
@@ -53,4 +54,11 @@ foreach src [lsort [glob -nocomplain $srcdir/$subdir/*.C]] {
profopt-execute $src
}
+foreach src [lsort [glob -nocomplain $srcdir/$subdir/*.C]] {
+ if ![runtest_file_p $runtests $src] then {
+ continue
+ }
+ auto-profopt-execute $src
+}
+
set PROFOPT_OPTIONS $treeprof_save_profopt_options
diff --git a/gcc/testsuite/gcc.dg/tree-prof/tree-prof.exp b/gcc/testsuite/gcc.dg/tree-prof/tree-prof.exp
index 650ad8d..aaccf19 100644
--- a/gcc/testsuite/gcc.dg/tree-prof/tree-prof.exp
+++ b/gcc/testsuite/gcc.dg/tree-prof/tree-prof.exp
@@ -44,6 +44,7 @@ set PROFOPT_OPTIONS [list {}]
# profile data.
set profile_option "-fprofile-generate -D_PROFILE_GENERATE"
set feedback_option "-fprofile-use -D_PROFILE_USE"
+set profile_wrapper ""
foreach src [lsort [glob -nocomplain $srcdir/$subdir/*.c]] {
# If we're only testing specific files and this isn't one of them, skip it.
@@ -53,4 +54,11 @@ foreach src [lsort [glob -nocomplain $srcdir/$subdir/*.c]] {
profopt-execute $src
}
+foreach src [lsort [glob -nocomplain $srcdir/$subdir/*.c]] {
+ if ![runtest_file_p $runtests $src] then {
+ continue
+ }
+ auto-profopt-execute $src
+}
+
set PROFOPT_OPTIONS $treeprof_save_profopt_options
diff --git a/gcc/testsuite/gcc.misc-tests/bprob.exp b/gcc/testsuite/gcc.misc-tests/bprob.exp
index 52dcb1f..132bfe3 100644
--- a/gcc/testsuite/gcc.misc-tests/bprob.exp
+++ b/gcc/testsuite/gcc.misc-tests/bprob.exp
@@ -41,6 +41,7 @@ load_lib profopt.exp
set bprob_save_profopt_options $PROFOPT_OPTIONS
set PROFOPT_OPTIONS [list { -O2 } { -O3 }]
+set profile_wrapper ""
set profile_options "-fprofile-arcs"
set feedback_options "-fbranch-probabilities"
@@ -54,4 +55,10 @@ foreach profile_option $profile_options feedback_option $feedback_options {
}
}
+foreach src [lsort [glob -nocomplain $srcdir/$subdir/bprob-*.c]] {
+ if ![runtest_file_p $runtests $src] then {
+ continue
+ }
+ auto-profopt-execute $src
+}
set PROFOPT_OPTIONS $bprob_save_profopt_options
diff --git a/gcc/testsuite/gfortran.dg/prof/prof.exp b/gcc/testsuite/gfortran.dg/prof/prof.exp
index 0bad01d..acc8ed0 100644
--- a/gcc/testsuite/gfortran.dg/prof/prof.exp
+++ b/gcc/testsuite/gfortran.dg/prof/prof.exp
@@ -53,4 +53,11 @@ foreach src [lsort [glob -nocomplain $srcdir/$subdir/*.f*]] {
profopt-execute $src
}
+foreach src [lsort [glob -nocomplain $srcdir/$subdir/*.f*]] {
+ if ![runtest_file_p $runtests $src] then {
+ continue
+ }
+ auto-profopt-execute $src
+}
+
set PROFOPT_OPTIONS $treeprof_save_profopt_options
diff --git a/gcc/testsuite/lib/profopt.exp b/gcc/testsuite/lib/profopt.exp
index 0aea6c4..b3e2b2e 100644
--- a/gcc/testsuite/lib/profopt.exp
+++ b/gcc/testsuite/lib/profopt.exp
@@ -36,7 +36,7 @@ load_lib gcc-dg.exp
global PROFOPT_OPTIONS perf_delta
# The including .exp file must define these.
-global tool profile_option feedback_option prof_ext
+global tool profile_option feedback_option prof_ext profile_wrapper
if ![info exists tool] {
error "Tools is not specified."
}
@@ -229,6 +229,31 @@ proc profopt-get-options { src } {
return ${dg-extra-tool-flags}
}
+# auto-profopt-execute -- Compile for auto profiling and then feedback,
+# then normal. SRC is the full path name of the testcase.
+proc auto-profopt-execute { src } {
+ global profile_wrapper
+ global profile_option
+ global feedback_option
+ global run_autofdo
+ global srcdir
+
+ if { ! [check_profiling_available "-fauto-profile"] } {
+ regsub "(?q)$srcdir/" $src "" testcase
+ unsupported "$testcase"
+ return
+ }
+ set profile_wrapper [profopt-perf-wrapper]
+ set profile_option "-g"
+ set feedback_option "-fauto-profile"
+ set run_autofdo 1
+ profopt-execute $src
+ unset profile_wrapper
+ unset profile_option
+ unset feedback_option
+ unset run_autofdo
+}
+
#
# c-prof-execute -- compile for profiling and then feedback, then normal
#
@@ -238,6 +263,7 @@ proc profopt-execute { src } {
global srcdir tmpdir
global PROFOPT_OPTIONS
global tool profile_option feedback_option prof_ext perf_ext perf_delta
+ global profile_wrapper run_autofdo ld_library_path
global generate_final_code use_final_code
global verbose
global testname_with_flags
@@ -248,6 +274,12 @@ proc profopt-execute { src } {
if ![info exists feedback_option] {
error "No feedback option specified for second compile."
}
+ if ![info exists profile_wrapper] {
+ set profile_wrapper ""
+ }
+ if ![info exists run_autofdo] {
+ set run_autofdo ""
+ }
# Use the default option list or one defined for a set of tests.
if ![info exists PROFOPT_OPTIONS] {
@@ -313,6 +345,7 @@ proc profopt-execute { src } {
# valid, by running it after dg-additional-files-options.
foreach ext $prof_ext {
profopt-target-cleanup $tmpdir $base $ext
+ profopt-target-cleanup $tmpdir perf data
}
# Tree profiling requires TLS runtime support, which may need
@@ -335,12 +368,50 @@ proc profopt-execute { src } {
}
# Run the profiled test.
+ if { $run_autofdo == 1 } {
+ if { ![info exists ld_library_path]} {
+ set ld_library_path ""
+ }
+ set orig_ld_library_path "[getenv LD_LIBRARY_PATH]"
+ setenv LD_LIBRARY_PATH "$ld_library_path:$orig_ld_library_path"
+ verbose "Running $profile_wrapper $execname1"
+ set id [remote_spawn "" "$profile_wrapper $execname1" "readonly"]
+ setenv LD_LIBRARY_PATH $orig_ld_library_path
+ if { $id < 0 } {
+ warning "Failed to run profiler"
+ set status "fail"
+ } else {
+ set result [remote_wait "" 300]
+ set status [lindex $result 0]
+ verbose "perf result $result"
+ if { $status == 0 } {
+ set status "pass"
+ } else {
+ set status "fail"
+ }
+ }
+ } else {
+ set result [${tool}_load $execname1 "" ""]
+ set status [lindex $result 0]
+ }
- set result [${tool}_load $execname1 "" ""]
- set status [lindex $result 0]
set missing_file 0
# Make sure the profile data was generated, and fail if not.
if { $status == "pass" } {
+ # convert profile
+ if { $run_autofdo == 1 } {
+ set cmd "create_gcov --binary $execname1 --profile=perf.data -gcov_version=1 --gcov=$tmpdir/$base.$ext"
+ verbose "Running $cmd"
+ set id [remote_spawn "" $cmd]
+ if { $id < 0 } {
+ unsupported "$testcase: Cannot run $cmd"
+ set status "fail"
+ return
+ }
+ set status [remote_wait "" 300]
+ set status "pass"
+ }
+
foreach ext $prof_ext {
remote_upload target $tmpdir/$base.$ext
set files [glob -nocomplain $base.$ext]
@@ -375,6 +446,10 @@ proc profopt-execute { src } {
set options "$extra_options"
lappend options "additional_flags=$option $extra_flags $feedback_option"
set optstr "$option $feedback_option"
+ if { [string first "-fauto-profile" $options] >= 0} {
+ set options [regsub -- "-fauto-profile" $options "-fauto-profile=$tmpdir/$base.$ext"]
+ }
+
set comp_output [${tool}_target_compile "$src" "$execname2" "executable" $options]
# Prune warnings we know are unwanted.
@@ -399,6 +474,7 @@ proc profopt-execute { src } {
# Remove the profiling data files.
foreach ext $prof_ext {
profopt-target-cleanup $tmpdir $base $ext
+ profopt-target-cleanup $tmpdir perf data
}
if { $status != "pass" } {
diff --git a/gcc/testsuite/lib/target-supports.exp b/gcc/testsuite/lib/target-supports.exp
index 1b1d03a..62267cf 100644
--- a/gcc/testsuite/lib/target-supports.exp
+++ b/gcc/testsuite/lib/target-supports.exp
@@ -508,6 +508,13 @@ proc check_effective_target_keeps_null_pointer_checks { } {
return 0
}
+# Return the autofdo profile wrapper
+
+proc profopt-perf-wrapper { } {
+ global srcdir
+ return "$srcdir/../config/i386/gcc-auto-profile -o perf.data "
+}
+
# Return true if profiling is supported on the target.
proc check_profiling_available { test_what } {
@@ -525,6 +532,36 @@ proc check_profiling_available { test_what } {
}
}
+ if { $test_what == "-fauto-profile" } {
+ if { ! ([istarget x86_64-*-linux*] || [istarget i?86-*-linux*]) } {
+ verbose "autofdo only supported on linux"
+ return 0
+ }
+ # not cross compiling?
+ if { ![isnative] } {
+ verbose "autofdo not supported for non native builds"
+ return 0
+ }
+ set event [profopt-perf-wrapper]
+ if {$event == "" } {
+ verbose "autofdo not supported"
+ return 0
+ }
+ global srcdir
+ set status [remote_exec host "$srcdir/../config/i386/gcc-auto-profile" "true -v >/dev/null"]
+ if { [lindex $status 0] != 0 } {
+ verbose "autofdo not supported because perf does not work"
+ return 0
+ }
+
+ # no good way to check this in advance -- check later instead.
+ #set status [remote_exec host "create_gcov" "2>/dev/null"]
+ #if { [lindex $status 0] != 255 } {
+ # verbose "autofdo not supported due to missing create_gcov"
+ # return 0
+ #}
+ }
+
# Support for -p on solaris2 relies on mcrt1.o which comes with the
# vendor compiler. We cannot reliably predict the directory where the
# vendor compiler (and thus mcrt1.o) is installed so we can't
--
2.8.3
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 2/3] Run profile feedback tests with autofdo
2016-06-23 14:38 ` [PATCH 2/3] Run profile feedback tests with autofdo Andi Kleen
@ 2016-07-12 15:13 ` Bin.Cheng
2016-07-14 10:55 ` Andi Kleen
0 siblings, 1 reply; 23+ messages in thread
From: Bin.Cheng @ 2016-07-12 15:13 UTC (permalink / raw)
To: Andi Kleen; +Cc: gcc-patches List
On Thu, Jun 23, 2016 at 3:37 PM, Andi Kleen <andi@firstfloor.org> wrote:
> From: ak <ak@138bc75d-0d04-0410-961f-82ee72b054a4>
>
> Extend the existing bprob and tree-prof tests to also run with autofdo.
> The test runtimes are really a bit too short for autofdo, but it's
> a reasonable sanity check.
>
> This only works natively for now.
>
> dejagnu doesn't seem to support a wrapper for unix tests, so I had
> to open code running these tests. That should be ok due to the
> native run restrictions.
>
> gcc/testsuite/:
>
> 2016-06-23 Andi Kleen <ak@linux.intel.com>
>
> * g++.dg/bprob/bprob.exp: Support autofdo.
> * g++.dg/tree-prof/tree-prof.exp: dito.
> * gcc.dg/tree-prof/tree-prof.exp: dito.
> * gcc.misc-tests/bprob.exp: dito.
> * gfortran.dg/prof/prof.exp: dito.
> * lib/profopt.exp: dito.
> * lib/target-supports.exp: Check for autofdo.
>
> git-svn-id: svn+ssh://gcc.gnu.org/svn/gcc/trunk@237732 138bc75d-0d04-0410-961f-82ee72b054a4
> ---
> gcc/testsuite/ChangeLog | 10 ++++
> gcc/testsuite/g++.dg/bprob/bprob.exp | 8 +++
> gcc/testsuite/g++.dg/tree-prof/tree-prof.exp | 8 +++
> gcc/testsuite/gcc.dg/tree-prof/tree-prof.exp | 8 +++
> gcc/testsuite/gcc.misc-tests/bprob.exp | 7 +++
> gcc/testsuite/gfortran.dg/prof/prof.exp | 7 +++
> gcc/testsuite/lib/profopt.exp | 82 +++++++++++++++++++++++++++-
> gcc/testsuite/lib/target-supports.exp | 37 +++++++++++++
> 8 files changed, 164 insertions(+), 3 deletions(-)
>
> diff --git a/gcc/testsuite/ChangeLog b/gcc/testsuite/ChangeLog
> index 287baf6..55f8dbf 100644
> --- a/gcc/testsuite/ChangeLog
> +++ b/gcc/testsuite/ChangeLog
> @@ -1,3 +1,13 @@
> +2016-06-23 Andi Kleen <ak@linux.intel.com>
> +
> + * g++.dg/bprob/bprob.exp: Support autofdo.
> + * g++.dg/tree-prof/tree-prof.exp: dito.
> + * gcc.dg/tree-prof/tree-prof.exp: dito.
> + * gcc.misc-tests/bprob.exp: dito.
> + * gfortran.dg/prof/prof.exp: dito.
> + * lib/profopt.exp: dito.
> + * lib/target-supports.exp: Check for autofdo.
> +
> 2016-06-23 Martin Liska <mliska@suse.cz>
>
> * gcc.dg/pr71619.c: New test.
> diff --git a/gcc/testsuite/g++.dg/bprob/bprob.exp b/gcc/testsuite/g++.dg/bprob/bprob.exp
> index d555507..4818298 100644
> --- a/gcc/testsuite/g++.dg/bprob/bprob.exp
> +++ b/gcc/testsuite/g++.dg/bprob/bprob.exp
> @@ -53,6 +53,7 @@ if $tracelevel then {
>
> set profile_options "-fprofile-arcs"
> set feedback_options "-fbranch-probabilities"
> +set profile_wrapper ""
>
> # Main loop.
> foreach profile_option $profile_options feedback_option $feedback_options {
> @@ -65,4 +66,11 @@ foreach profile_option $profile_options feedback_option $feedback_options {
> }
> }
>
> +foreach src [lsort [glob -nocomplain $srcdir/$subdir/*.C]] {
> + if ![runtest_file_p $runtests $src] then {
> + continue
> + }
> + auto-profopt-execute $src
> +}
> +
> set PROFOPT_OPTIONS $bprob_save_profopt_options
> diff --git a/gcc/testsuite/g++.dg/tree-prof/tree-prof.exp b/gcc/testsuite/g++.dg/tree-prof/tree-prof.exp
> index 7a4b5cb..26ee0b3 100644
> --- a/gcc/testsuite/g++.dg/tree-prof/tree-prof.exp
> +++ b/gcc/testsuite/g++.dg/tree-prof/tree-prof.exp
> @@ -44,6 +44,7 @@ set PROFOPT_OPTIONS [list {}]
> # profile data.
> set profile_option "-fprofile-generate -D_PROFILE_GENERATE"
> set feedback_option "-fprofile-use -D_PROFILE_USE"
> +set profile_wrapper ""
>
> foreach src [lsort [glob -nocomplain $srcdir/$subdir/*.C]] {
> # If we're only testing specific files and this isn't one of them, skip it.
> @@ -53,4 +54,11 @@ foreach src [lsort [glob -nocomplain $srcdir/$subdir/*.C]] {
> profopt-execute $src
> }
>
> +foreach src [lsort [glob -nocomplain $srcdir/$subdir/*.C]] {
> + if ![runtest_file_p $runtests $src] then {
> + continue
> + }
> + auto-profopt-execute $src
> +}
> +
> set PROFOPT_OPTIONS $treeprof_save_profopt_options
> diff --git a/gcc/testsuite/gcc.dg/tree-prof/tree-prof.exp b/gcc/testsuite/gcc.dg/tree-prof/tree-prof.exp
> index 650ad8d..aaccf19 100644
> --- a/gcc/testsuite/gcc.dg/tree-prof/tree-prof.exp
> +++ b/gcc/testsuite/gcc.dg/tree-prof/tree-prof.exp
> @@ -44,6 +44,7 @@ set PROFOPT_OPTIONS [list {}]
> # profile data.
> set profile_option "-fprofile-generate -D_PROFILE_GENERATE"
> set feedback_option "-fprofile-use -D_PROFILE_USE"
> +set profile_wrapper ""
>
> foreach src [lsort [glob -nocomplain $srcdir/$subdir/*.c]] {
> # If we're only testing specific files and this isn't one of them, skip it.
> @@ -53,4 +54,11 @@ foreach src [lsort [glob -nocomplain $srcdir/$subdir/*.c]] {
> profopt-execute $src
> }
>
> +foreach src [lsort [glob -nocomplain $srcdir/$subdir/*.c]] {
> + if ![runtest_file_p $runtests $src] then {
> + continue
> + }
> + auto-profopt-execute $src
> +}
> +
> set PROFOPT_OPTIONS $treeprof_save_profopt_options
> diff --git a/gcc/testsuite/gcc.misc-tests/bprob.exp b/gcc/testsuite/gcc.misc-tests/bprob.exp
> index 52dcb1f..132bfe3 100644
> --- a/gcc/testsuite/gcc.misc-tests/bprob.exp
> +++ b/gcc/testsuite/gcc.misc-tests/bprob.exp
> @@ -41,6 +41,7 @@ load_lib profopt.exp
> set bprob_save_profopt_options $PROFOPT_OPTIONS
> set PROFOPT_OPTIONS [list { -O2 } { -O3 }]
>
> +set profile_wrapper ""
> set profile_options "-fprofile-arcs"
> set feedback_options "-fbranch-probabilities"
>
> @@ -54,4 +55,10 @@ foreach profile_option $profile_options feedback_option $feedback_options {
> }
> }
>
> +foreach src [lsort [glob -nocomplain $srcdir/$subdir/bprob-*.c]] {
> + if ![runtest_file_p $runtests $src] then {
> + continue
> + }
> + auto-profopt-execute $src
> +}
> set PROFOPT_OPTIONS $bprob_save_profopt_options
> diff --git a/gcc/testsuite/gfortran.dg/prof/prof.exp b/gcc/testsuite/gfortran.dg/prof/prof.exp
> index 0bad01d..acc8ed0 100644
> --- a/gcc/testsuite/gfortran.dg/prof/prof.exp
> +++ b/gcc/testsuite/gfortran.dg/prof/prof.exp
> @@ -53,4 +53,11 @@ foreach src [lsort [glob -nocomplain $srcdir/$subdir/*.f*]] {
> profopt-execute $src
> }
>
> +foreach src [lsort [glob -nocomplain $srcdir/$subdir/*.f*]] {
> + if ![runtest_file_p $runtests $src] then {
> + continue
> + }
> + auto-profopt-execute $src
> +}
> +
> set PROFOPT_OPTIONS $treeprof_save_profopt_options
> diff --git a/gcc/testsuite/lib/profopt.exp b/gcc/testsuite/lib/profopt.exp
> index 0aea6c4..b3e2b2e 100644
> --- a/gcc/testsuite/lib/profopt.exp
> +++ b/gcc/testsuite/lib/profopt.exp
> @@ -36,7 +36,7 @@ load_lib gcc-dg.exp
> global PROFOPT_OPTIONS perf_delta
>
> # The including .exp file must define these.
> -global tool profile_option feedback_option prof_ext
> +global tool profile_option feedback_option prof_ext profile_wrapper
> if ![info exists tool] {
> error "Tools is not specified."
> }
> @@ -229,6 +229,31 @@ proc profopt-get-options { src } {
> return ${dg-extra-tool-flags}
> }
>
> +# auto-profopt-execute -- Compile for auto profiling and then feedback,
> +# then normal. SRC is the full path name of the testcase.
> +proc auto-profopt-execute { src } {
> + global profile_wrapper
> + global profile_option
> + global feedback_option
> + global run_autofdo
> + global srcdir
> +
> + if { ! [check_profiling_available "-fauto-profile"] } {
> + regsub "(?q)$srcdir/" $src "" testcase
> + unsupported "$testcase"
> + return
> + }
> + set profile_wrapper [profopt-perf-wrapper]
> + set profile_option "-g"
> + set feedback_option "-fauto-profile"
> + set run_autofdo 1
> + profopt-execute $src
> + unset profile_wrapper
> + unset profile_option
> + unset feedback_option
> + unset run_autofdo
> +}
> +
> #
> # c-prof-execute -- compile for profiling and then feedback, then normal
> #
> @@ -238,6 +263,7 @@ proc profopt-execute { src } {
> global srcdir tmpdir
> global PROFOPT_OPTIONS
> global tool profile_option feedback_option prof_ext perf_ext perf_delta
> + global profile_wrapper run_autofdo ld_library_path
> global generate_final_code use_final_code
> global verbose
> global testname_with_flags
> @@ -248,6 +274,12 @@ proc profopt-execute { src } {
> if ![info exists feedback_option] {
> error "No feedback option specified for second compile."
> }
> + if ![info exists profile_wrapper] {
> + set profile_wrapper ""
> + }
> + if ![info exists run_autofdo] {
> + set run_autofdo ""
> + }
>
> # Use the default option list or one defined for a set of tests.
> if ![info exists PROFOPT_OPTIONS] {
> @@ -313,6 +345,7 @@ proc profopt-execute { src } {
> # valid, by running it after dg-additional-files-options.
> foreach ext $prof_ext {
> profopt-target-cleanup $tmpdir $base $ext
> + profopt-target-cleanup $tmpdir perf data
> }
>
> # Tree profiling requires TLS runtime support, which may need
> @@ -335,12 +368,50 @@ proc profopt-execute { src } {
> }
>
> # Run the profiled test.
> + if { $run_autofdo == 1 } {
> + if { ![info exists ld_library_path]} {
> + set ld_library_path ""
> + }
> + set orig_ld_library_path "[getenv LD_LIBRARY_PATH]"
> + setenv LD_LIBRARY_PATH "$ld_library_path:$orig_ld_library_path"
> + verbose "Running $profile_wrapper $execname1"
> + set id [remote_spawn "" "$profile_wrapper $execname1" "readonly"]
> + setenv LD_LIBRARY_PATH $orig_ld_library_path
> + if { $id < 0 } {
> + warning "Failed to run profiler"
> + set status "fail"
> + } else {
> + set result [remote_wait "" 300]
> + set status [lindex $result 0]
> + verbose "perf result $result"
> + if { $status == 0 } {
> + set status "pass"
> + } else {
> + set status "fail"
> + }
> + }
> + } else {
> + set result [${tool}_load $execname1 "" ""]
> + set status [lindex $result 0]
> + }
>
> - set result [${tool}_load $execname1 "" ""]
> - set status [lindex $result 0]
> set missing_file 0
> # Make sure the profile data was generated, and fail if not.
> if { $status == "pass" } {
> + # convert profile
> + if { $run_autofdo == 1 } {
> + set cmd "create_gcov --binary $execname1 --profile=perf.data -gcov_version=1 --gcov=$tmpdir/$base.$ext"
> + verbose "Running $cmd"
> + set id [remote_spawn "" $cmd]
> + if { $id < 0 } {
> + unsupported "$testcase: Cannot run $cmd"
> + set status "fail"
> + return
> + }
> + set status [remote_wait "" 300]
> + set status "pass"
> + }
> +
> foreach ext $prof_ext {
> remote_upload target $tmpdir/$base.$ext
> set files [glob -nocomplain $base.$ext]
> @@ -375,6 +446,10 @@ proc profopt-execute { src } {
> set options "$extra_options"
> lappend options "additional_flags=$option $extra_flags $feedback_option"
> set optstr "$option $feedback_option"
> + if { [string first "-fauto-profile" $options] >= 0} {
> + set options [regsub -- "-fauto-profile" $options "-fauto-profile=$tmpdir/$base.$ext"]
> + }
> +
> set comp_output [${tool}_target_compile "$src" "$execname2" "executable" $options]
>
> # Prune warnings we know are unwanted.
> @@ -399,6 +474,7 @@ proc profopt-execute { src } {
> # Remove the profiling data files.
> foreach ext $prof_ext {
> profopt-target-cleanup $tmpdir $base $ext
> + profopt-target-cleanup $tmpdir perf data
> }
>
> if { $status != "pass" } {
> diff --git a/gcc/testsuite/lib/target-supports.exp b/gcc/testsuite/lib/target-supports.exp
> index 1b1d03a..62267cf 100644
> --- a/gcc/testsuite/lib/target-supports.exp
> +++ b/gcc/testsuite/lib/target-supports.exp
> @@ -508,6 +508,13 @@ proc check_effective_target_keeps_null_pointer_checks { } {
> return 0
> }
>
> +# Return the autofdo profile wrapper
> +
> +proc profopt-perf-wrapper { } {
> + global srcdir
> + return "$srcdir/../config/i386/gcc-auto-profile -o perf.data "
> +}
> +
> # Return true if profiling is supported on the target.
>
> proc check_profiling_available { test_what } {
> @@ -525,6 +532,36 @@ proc check_profiling_available { test_what } {
> }
> }
>
> + if { $test_what == "-fauto-profile" } {
> + if { ! ([istarget x86_64-*-linux*] || [istarget i?86-*-linux*]) } {
> + verbose "autofdo only supported on linux"
> + return 0
> + }
> + # not cross compiling?
> + if { ![isnative] } {
> + verbose "autofdo not supported for non native builds"
> + return 0
> + }
> + set event [profopt-perf-wrapper]
> + if {$event == "" } {
> + verbose "autofdo not supported"
> + return 0
> + }
> + global srcdir
> + set status [remote_exec host "$srcdir/../config/i386/gcc-auto-profile" "true -v >/dev/null"]
> + if { [lindex $status 0] != 0 } {
> + verbose "autofdo not supported because perf does not work"
> + return 0
> + }
> +
> + # no good way to check this in advance -- check later instead.
> + #set status [remote_exec host "create_gcov" "2>/dev/null"]
> + #if { [lindex $status 0] != 255 } {
> + # verbose "autofdo not supported due to missing create_gcov"
> + # return 0
> + #}
> + }
> +
> # Support for -p on solaris2 relies on mcrt1.o which comes with the
> # vendor compiler. We cannot reliably predict the directory where the
> # vendor compiler (and thus mcrt1.o) is installed so we can't
> --
> 2.8.3
>
Hi,
After this patch, I got below test results with command line: make
check-gcc RUNTESTFLAGS="tree-prof.exp" -k
PASS: gcc.dg/tree-prof/20041218-1.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/20041218-1.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/20041218-1.gcda
PASS: gcc.dg/tree-prof/20050826-2.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/20050826-2.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/20050826-2.gcda
PASS: gcc.dg/tree-prof/bb-reorg.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/bb-reorg.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/bb-reorg.gcda
PASS: gcc.dg/tree-prof/cmpsf-1.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/cmpsf-1.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/cmpsf-1.gcda
PASS: gcc.dg/tree-prof/cold_partition_label.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/cold_partition_label.c: Cannot run
create_gcov --binary
/data/work/trunk/build/gcc/testsuite/gcc/cold_partition_label.gcda
PASS: gcc.dg/tree-prof/comp-goto-1.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/comp-goto-1.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/comp-goto-1.gcda
PASS: gcc.dg/tree-prof/crossmodule-indircall-1.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/crossmodule-indircall-1.c: Cannot run
create_gcov --binary
/data/work/trunk/build/gcc/testsuite/gcc/crossmodule-indircall-1.gcda
PASS: gcc.dg/tree-prof/crossmodule-indircall-1a.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/crossmodule-indircall-1a.c: Cannot run
create_gcov --binary
/data/work/trunk/build/gcc/testsuite/gcc/crossmodule-indircall-1a.gcda
PASS: gcc.dg/tree-prof/ic-misattribution-1.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/ic-misattribution-1.c: Cannot run
create_gcov --binary
/data/work/trunk/build/gcc/testsuite/gcc/ic-misattribution-1.gcda
PASS: gcc.dg/tree-prof/ic-misattribution-1a.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/ic-misattribution-1a.c: Cannot run
create_gcov --binary
/data/work/trunk/build/gcc/testsuite/gcc/ic-misattribution-1a.gcda
PASS: gcc.dg/tree-prof/indir-call-prof.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/indir-call-prof.c: Cannot run
create_gcov --binary
/data/work/trunk/build/gcc/testsuite/gcc/indir-call-prof.gcda
PASS: gcc.dg/tree-prof/inliner-1.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/inliner-1.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/inliner-1.gcda
PASS: gcc.dg/tree-prof/merge_block.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/merge_block.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/merge_block.gcda
PASS: gcc.dg/tree-prof/peel-1.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/peel-1.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/peel-1.gcda
PASS: gcc.dg/tree-prof/pr34999.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/pr34999.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/pr34999.gcda
PASS: gcc.dg/tree-prof/pr44777.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/pr44777.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/pr44777.gcda
PASS: gcc.dg/tree-prof/pr45354.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/pr45354.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/pr45354.gcda
PASS: gcc.dg/tree-prof/pr47187.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/pr47187.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/pr47187.gcda
PASS: gcc.dg/tree-prof/pr49299-1.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/pr49299-1.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/pr49299-1.gcda
PASS: gcc.dg/tree-prof/pr49299-2.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/pr49299-2.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/pr49299-2.gcda
PASS: gcc.dg/tree-prof/pr50907.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/pr50907.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/pr50907.gcda
PASS: gcc.dg/tree-prof/pr52027.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/pr52027.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/pr52027.gcda
PASS: gcc.dg/tree-prof/pr52150.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/pr52150.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/pr52150.gcda
PASS: gcc.dg/tree-prof/pr59003.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/pr59003.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/pr59003.gcda
PASS: gcc.dg/tree-prof/prof-robust-1.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/prof-robust-1.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/prof-robust-1.gcda
PASS: gcc.dg/tree-prof/stringop-1.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/stringop-1.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/stringop-1.gcda
PASS: gcc.dg/tree-prof/stringop-2.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/stringop-2.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/stringop-2.gcda
PASS: gcc.dg/tree-prof/switch-case-1.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/switch-case-1.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/switch-case-1.gcda
PASS: gcc.dg/tree-prof/switch-case-2.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/switch-case-2.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/switch-case-2.gcda
PASS: gcc.dg/tree-prof/time-profiler-1.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/time-profiler-1.c: Cannot run
create_gcov --binary
/data/work/trunk/build/gcc/testsuite/gcc/time-profiler-1.gcda
PASS: gcc.dg/tree-prof/time-profiler-2.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/time-profiler-2.c: Cannot run
create_gcov --binary
/data/work/trunk/build/gcc/testsuite/gcc/time-profiler-2.gcda
PASS: gcc.dg/tree-prof/tracer-1.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/tracer-1.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/tracer-1.gcda
PASS: gcc.dg/tree-prof/unroll-1.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/unroll-1.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/unroll-1.gcda
PASS: gcc.dg/tree-prof/update-cunroll-2.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/update-cunroll-2.c: Cannot run
create_gcov --binary
/data/work/trunk/build/gcc/testsuite/gcc/update-cunroll-2.gcda
PASS: gcc.dg/tree-prof/update-loopch.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/update-loopch.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/update-loopch.gcda
PASS: gcc.dg/tree-prof/update-tailcall.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/update-tailcall.c: Cannot run
create_gcov --binary
/data/work/trunk/build/gcc/testsuite/gcc/update-tailcall.gcda
PASS: gcc.dg/tree-prof/va-arg-pack-1.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/va-arg-pack-1.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/va-arg-pack-1.gcda
PASS: gcc.dg/tree-prof/val-prof-1.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/val-prof-1.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/val-prof-1.gcda
PASS: gcc.dg/tree-prof/val-prof-2.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/val-prof-2.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/val-prof-2.gcda
PASS: gcc.dg/tree-prof/val-prof-3.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/val-prof-3.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/val-prof-3.gcda
PASS: gcc.dg/tree-prof/val-prof-4.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/val-prof-4.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/val-prof-4.gcda
PASS: gcc.dg/tree-prof/val-prof-5.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/val-prof-5.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/val-prof-5.gcda
PASS: gcc.dg/tree-prof/val-prof-6.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/val-prof-6.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/val-prof-6.gcda
PASS: gcc.dg/tree-prof/val-prof-7.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/val-prof-7.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/val-prof-7.gcda
PASS: gcc.dg/tree-prof/wcoverage-mismatch.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/wcoverage-mismatch.c: Cannot run
create_gcov --binary
/data/work/trunk/build/gcc/testsuite/gcc/wcoverage-mismatch.gcda
Before this patch, gcc test doesn't build/test these with "-g" option.
Also I got unstable test result in tree-prof.exp if I run
aforementioned command line with -jnum (parallelly). Does the patch
has problem in parallel testing?
Thanks,
bin
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 2/3] Run profile feedback tests with autofdo
2016-07-12 15:13 ` Bin.Cheng
@ 2016-07-14 10:55 ` Andi Kleen
2016-07-14 11:06 ` Bin.Cheng
0 siblings, 1 reply; 23+ messages in thread
From: Andi Kleen @ 2016-07-14 10:55 UTC (permalink / raw)
To: Bin.Cheng; +Cc: Andi Kleen, gcc-patches List
> After this patch, I got below test results with command line: make
> check-gcc RUNTESTFLAGS="tree-prof.exp" -k
That is expected if you don't have autofdo. You would prefer to hide it?
> Also I got unstable test result in tree-prof.exp if I run
> aforementioned command line with -jnum (parallelly). Does the patch
> has problem in parallel testing?
I haven't seen that. Unstable in what way?
-Andi
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 2/3] Run profile feedback tests with autofdo
2016-07-14 10:55 ` Andi Kleen
@ 2016-07-14 11:06 ` Bin.Cheng
2016-07-14 11:08 ` Bin.Cheng
0 siblings, 1 reply; 23+ messages in thread
From: Bin.Cheng @ 2016-07-14 11:06 UTC (permalink / raw)
To: Andi Kleen; +Cc: gcc-patches List
On Thu, Jul 14, 2016 at 11:55 AM, Andi Kleen <andi@firstfloor.org> wrote:
>> After this patch, I got below test results with command line: make
>> check-gcc RUNTESTFLAGS="tree-prof.exp" -k
>
> That is expected if you don't have autofdo. You would prefer to hide it?
>
>> Also I got unstable test result in tree-prof.exp if I run
>> aforementioned command line with -jnum (parallelly). Does the patch
>> has problem in parallel testing?
>
> I haven't seen that. Unstable in what way?
For GCC doesn't support FDO, it run below tests as you said:
PASS: gcc.dg/tree-prof/20041218-1.c compilation, -g
UNSUPPORTED: gcc.dg/tree-prof/20041218-1.c: Cannot run create_gcov
--binary /data/work/trunk/build/gcc/testsuite/gcc/20041218-1.gcda
Normally, it doesn't create gcov data file, thus the next test is
unsupported. I guess in parallel test, the second test picks up gcov
data files from other process, which results in random pass.
Is it possible to not have these when fdo is supported?
Thanks,
bin
>
> -Andi
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 2/3] Run profile feedback tests with autofdo
2016-07-14 11:06 ` Bin.Cheng
@ 2016-07-14 11:08 ` Bin.Cheng
2016-07-14 22:33 ` Andi Kleen
0 siblings, 1 reply; 23+ messages in thread
From: Bin.Cheng @ 2016-07-14 11:08 UTC (permalink / raw)
To: Andi Kleen; +Cc: gcc-patches List
On Thu, Jul 14, 2016 at 12:06 PM, Bin.Cheng <amker.cheng@gmail.com> wrote:
> On Thu, Jul 14, 2016 at 11:55 AM, Andi Kleen <andi@firstfloor.org> wrote:
>>> After this patch, I got below test results with command line: make
>>> check-gcc RUNTESTFLAGS="tree-prof.exp" -k
>>
>> That is expected if you don't have autofdo. You would prefer to hide it?
>>
>>> Also I got unstable test result in tree-prof.exp if I run
>>> aforementioned command line with -jnum (parallelly). Does the patch
>>> has problem in parallel testing?
>>
>> I haven't seen that. Unstable in what way?
> For GCC doesn't support FDO, it run below tests as you said:
>
> PASS: gcc.dg/tree-prof/20041218-1.c compilation, -g
> UNSUPPORTED: gcc.dg/tree-prof/20041218-1.c: Cannot run create_gcov
> --binary /data/work/trunk/build/gcc/testsuite/gcc/20041218-1.gcda
>
> Normally, it doesn't create gcov data file, thus the next test is
> unsupported. I guess in parallel test, the second test picks up gcov
> data files from other process, which results in random pass.
> Is it possible to not have these when fdo is supported?
Hmm, typo: s/supported/not supported/.
Thanks,
bin
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 2/3] Run profile feedback tests with autofdo
2016-07-14 11:08 ` Bin.Cheng
@ 2016-07-14 22:33 ` Andi Kleen
2016-07-15 8:37 ` Bin.Cheng
0 siblings, 1 reply; 23+ messages in thread
From: Andi Kleen @ 2016-07-14 22:33 UTC (permalink / raw)
To: Bin.Cheng; +Cc: Andi Kleen, gcc-patches List
> >> I haven't seen that. Unstable in what way?
> > For GCC doesn't support FDO, it run below tests as you said:
> >
> > PASS: gcc.dg/tree-prof/20041218-1.c compilation, -g
> > UNSUPPORTED: gcc.dg/tree-prof/20041218-1.c: Cannot run create_gcov
> > --binary /data/work/trunk/build/gcc/testsuite/gcc/20041218-1.gcda
> >
> > Normally, it doesn't create gcov data file, thus the next test is
> > unsupported. I guess in parallel test, the second test picks up gcov
> > data files from other process, which results in random pass.
> > Is it possible to not have these when fdo is supported?
> Hmm, typo: s/supported/not supported/.
I don't see how this can happen: as you can see the .gcda file name
is unique for each test case.
I think there may be problems with the perf.data, could add a postfix.
But you should only see that with real autofdo.
-Andi
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 2/3] Run profile feedback tests with autofdo
2016-07-14 22:33 ` Andi Kleen
@ 2016-07-15 8:37 ` Bin.Cheng
2016-07-15 19:44 ` Jeff Law
2016-07-25 11:08 ` Martin Liška
0 siblings, 2 replies; 23+ messages in thread
From: Bin.Cheng @ 2016-07-15 8:37 UTC (permalink / raw)
To: Andi Kleen; +Cc: gcc-patches List
On Thu, Jul 14, 2016 at 11:33 PM, Andi Kleen <andi@firstfloor.org> wrote:
>> >> I haven't seen that. Unstable in what way?
>> > For GCC doesn't support FDO, it run below tests as you said:
>> >
>> > PASS: gcc.dg/tree-prof/20041218-1.c compilation, -g
>> > UNSUPPORTED: gcc.dg/tree-prof/20041218-1.c: Cannot run create_gcov
>> > --binary /data/work/trunk/build/gcc/testsuite/gcc/20041218-1.gcda
>> >
>> > Normally, it doesn't create gcov data file, thus the next test is
>> > unsupported. I guess in parallel test, the second test picks up gcov
>> > data files from other process, which results in random pass.
>> > Is it possible to not have these when fdo is supported?
>> Hmm, typo: s/supported/not supported/.
>
> I don't see how this can happen: as you can see the .gcda file name
> is unique for each test case.
>
> I think there may be problems with the perf.data, could add a postfix.
> But you should only see that with real autofdo.
Don't know. I only configured/built GCC on x86_64 with below command lines:
$ ../gcc/configure --prefix=...
--enable-languages=c,c++,fortran,go,java,lto,objc,obj-c++
--with-build-config=bootstrap-O3 --disable-werror
$ make && make install
Not sure what the status for autofdo is in this case. "make check -k"
is stable for me, but "make check -k -j#" gives unstable result in
tree-prof.exp tests. Anything I did wrong?
Thanks,
bin
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 2/3] Run profile feedback tests with autofdo
2016-07-15 8:37 ` Bin.Cheng
@ 2016-07-15 19:44 ` Jeff Law
2016-07-15 20:53 ` Andi Kleen
2016-07-25 11:08 ` Martin Liška
1 sibling, 1 reply; 23+ messages in thread
From: Jeff Law @ 2016-07-15 19:44 UTC (permalink / raw)
To: Bin.Cheng, Andi Kleen; +Cc: gcc-patches List
On 07/15/2016 02:37 AM, Bin.Cheng wrote:
> On Thu, Jul 14, 2016 at 11:33 PM, Andi Kleen <andi@firstfloor.org> wrote:
>>>>> I haven't seen that. Unstable in what way?
>>>> For GCC doesn't support FDO, it run below tests as you said:
>>>>
>>>> PASS: gcc.dg/tree-prof/20041218-1.c compilation, -g
>>>> UNSUPPORTED: gcc.dg/tree-prof/20041218-1.c: Cannot run create_gcov
>>>> --binary /data/work/trunk/build/gcc/testsuite/gcc/20041218-1.gcda
>>>>
>>>> Normally, it doesn't create gcov data file, thus the next test is
>>>> unsupported. I guess in parallel test, the second test picks up gcov
>>>> data files from other process, which results in random pass.
>>>> Is it possible to not have these when fdo is supported?
>>> Hmm, typo: s/supported/not supported/.
>>
>> I don't see how this can happen: as you can see the .gcda file name
>> is unique for each test case.
>>
>> I think there may be problems with the perf.data, could add a postfix.
>> But you should only see that with real autofdo.
> Don't know. I only configured/built GCC on x86_64 with below command lines:
>
> $ ../gcc/configure --prefix=...
> --enable-languages=c,c++,fortran,go,java,lto,objc,obj-c++
> --with-build-config=bootstrap-O3 --disable-werror
> $ make && make install
>
> Not sure what the status for autofdo is in this case. "make check -k"
> is stable for me, but "make check -k -j#" gives unstable result in
> tree-prof.exp tests. Anything I did wrong?
I'm seeing unstable results as well, but haven't dug into it at all.
It's definitely autofdo testing though -- If I remove the
gcc-auto-profile script I get consistent results from one run to the next.
jeff
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 2/3] Run profile feedback tests with autofdo
2016-07-15 19:44 ` Jeff Law
@ 2016-07-15 20:53 ` Andi Kleen
0 siblings, 0 replies; 23+ messages in thread
From: Andi Kleen @ 2016-07-15 20:53 UTC (permalink / raw)
To: Jeff Law; +Cc: Bin.Cheng, Andi Kleen, gcc-patches List
> >Not sure what the status for autofdo is in this case. "make check -k"
> >is stable for me, but "make check -k -j#" gives unstable result in
> >tree-prof.exp tests. Anything I did wrong?
> I'm seeing unstable results as well, but haven't dug into it at all.
> It's definitely autofdo testing though -- If I remove the
> gcc-auto-profile script I get consistent results from one run to the
> next.
Ok. I'll investigate.
-Andi
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 2/3] Run profile feedback tests with autofdo
2016-07-15 8:37 ` Bin.Cheng
2016-07-15 19:44 ` Jeff Law
@ 2016-07-25 11:08 ` Martin Liška
2016-07-26 4:21 ` Andi Kleen
1 sibling, 1 reply; 23+ messages in thread
From: Martin Liška @ 2016-07-25 11:08 UTC (permalink / raw)
To: Bin.Cheng, Andi Kleen; +Cc: gcc-patches List
On 07/15/2016 10:37 AM, Bin.Cheng wrote:
> On Thu, Jul 14, 2016 at 11:33 PM, Andi Kleen <andi@firstfloor.org> wrote:
>>>>> I haven't seen that. Unstable in what way?
>>>> For GCC doesn't support FDO, it run below tests as you said:
>>>>
>>>> PASS: gcc.dg/tree-prof/20041218-1.c compilation, -g
>>>> UNSUPPORTED: gcc.dg/tree-prof/20041218-1.c: Cannot run create_gcov
>>>> --binary /data/work/trunk/build/gcc/testsuite/gcc/20041218-1.gcda
>>>>
>>>> Normally, it doesn't create gcov data file, thus the next test is
>>>> unsupported. I guess in parallel test, the second test picks up gcov
>>>> data files from other process, which results in random pass.
>>>> Is it possible to not have these when fdo is supported?
>>> Hmm, typo: s/supported/not supported/.
>>
>> I don't see how this can happen: as you can see the .gcda file name
>> is unique for each test case.
>>
>> I think there may be problems with the perf.data, could add a postfix.
>> But you should only see that with real autofdo.
> Don't know. I only configured/built GCC on x86_64 with below command lines:
>
> $ ../gcc/configure --prefix=...
> --enable-languages=c,c++,fortran,go,java,lto,objc,obj-c++
> --with-build-config=bootstrap-O3 --disable-werror
> $ make && make install
>
> Not sure what the status for autofdo is in this case. "make check -k"
> is stable for me, but "make check -k -j#" gives unstable result in
> tree-prof.exp tests. Anything I did wrong?
>
> Thanks,
> bin
>
I can confirm that I see the same problem on a Xeon machine (../configure && make && make check -k -jX).
I see many:
g++.dg/tree-prof/pr35545.C: Cannot run create_gcov --binary /home/marxin/Programming/gcc/objdir/gcc/testsuite/g++/pr35545.x01 --profile=perf.data -gcov_version=1 --gcov=/home/marxin/Programming/gcc/objdir/gcc/testsuite/g++/pr35545.gcda
Thanks,
Martin
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 2/3] Run profile feedback tests with autofdo
2016-07-25 11:08 ` Martin Liška
@ 2016-07-26 4:21 ` Andi Kleen
2016-07-26 4:25 ` Jeff Law
0 siblings, 1 reply; 23+ messages in thread
From: Andi Kleen @ 2016-07-26 4:21 UTC (permalink / raw)
To: Martin Liška; +Cc: Bin.Cheng, Andi Kleen, gcc-patches List
On Mon, Jul 25, 2016 at 01:08:43PM +0200, Martin Liška wrote:
> On 07/15/2016 10:37 AM, Bin.Cheng wrote:
> > On Thu, Jul 14, 2016 at 11:33 PM, Andi Kleen <andi@firstfloor.org> wrote:
> >>>>> I haven't seen that. Unstable in what way?
> >>>> For GCC doesn't support FDO, it run below tests as you said:
> >>>>
> >>>> PASS: gcc.dg/tree-prof/20041218-1.c compilation, -g
> >>>> UNSUPPORTED: gcc.dg/tree-prof/20041218-1.c: Cannot run create_gcov
> >>>> --binary /data/work/trunk/build/gcc/testsuite/gcc/20041218-1.gcda
> >>>>
> >>>> Normally, it doesn't create gcov data file, thus the next test is
> >>>> unsupported. I guess in parallel test, the second test picks up gcov
> >>>> data files from other process, which results in random pass.
> >>>> Is it possible to not have these when fdo is supported?
> >>> Hmm, typo: s/supported/not supported/.
> >>
> >> I don't see how this can happen: as you can see the .gcda file name
> >> is unique for each test case.
> >>
> >> I think there may be problems with the perf.data, could add a postfix.
> >> But you should only see that with real autofdo.
> > Don't know. I only configured/built GCC on x86_64 with below command lines:
> >
> > $ ../gcc/configure --prefix=...
> > --enable-languages=c,c++,fortran,go,java,lto,objc,obj-c++
> > --with-build-config=bootstrap-O3 --disable-werror
> > $ make && make install
> >
> > Not sure what the status for autofdo is in this case. "make check -k"
> > is stable for me, but "make check -k -j#" gives unstable result in
> > tree-prof.exp tests. Anything I did wrong?
> >
> > Thanks,
> > bin
> >
>
> I can confirm that I see the same problem on a Xeon machine (../configure && make && make check -k -jX).
> I see many:
>
> g++.dg/tree-prof/pr35545.C: Cannot run create_gcov --binary /home/marxin/Programming/gcc/objdir/gcc/testsuite/g++/pr35545.x01 --profile=perf.data -gcov_version=1 --gcov=/home/marxin/Programming/gcc/objdir/gcc/testsuite/g++/pr35545.gcda
Well that's expected if you don't have autofdo installed or perf with
lbrs is not working. The test case will be UNSUP in this case.
But it's not expected that it is not deterministic, so that it changes
run to run. Do you see that? Or some other problem? Please describe
it exactly.
If yes, did it start before r238320 or only afer?
-Andi
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 2/3] Run profile feedback tests with autofdo
2016-07-26 4:21 ` Andi Kleen
@ 2016-07-26 4:25 ` Jeff Law
2016-07-26 4:28 ` Andi Kleen
0 siblings, 1 reply; 23+ messages in thread
From: Jeff Law @ 2016-07-26 4:25 UTC (permalink / raw)
To: Andi Kleen, Martin Liška; +Cc: Bin.Cheng, gcc-patches List
On 07/25/2016 10:21 PM, Andi Kleen wrote:
> On Mon, Jul 25, 2016 at 01:08:43PM +0200, Martin Liška wrote:
>> On 07/15/2016 10:37 AM, Bin.Cheng wrote:
>>> On Thu, Jul 14, 2016 at 11:33 PM, Andi Kleen <andi@firstfloor.org> wrote:
>>>>>>> I haven't seen that. Unstable in what way?
>>>>>> For GCC doesn't support FDO, it run below tests as you said:
>>>>>>
>>>>>> PASS: gcc.dg/tree-prof/20041218-1.c compilation, -g
>>>>>> UNSUPPORTED: gcc.dg/tree-prof/20041218-1.c: Cannot run create_gcov
>>>>>> --binary /data/work/trunk/build/gcc/testsuite/gcc/20041218-1.gcda
>>>>>>
>>>>>> Normally, it doesn't create gcov data file, thus the next test is
>>>>>> unsupported. I guess in parallel test, the second test picks up gcov
>>>>>> data files from other process, which results in random pass.
>>>>>> Is it possible to not have these when fdo is supported?
>>>>> Hmm, typo: s/supported/not supported/.
>>>>
>>>> I don't see how this can happen: as you can see the .gcda file name
>>>> is unique for each test case.
>>>>
>>>> I think there may be problems with the perf.data, could add a postfix.
>>>> But you should only see that with real autofdo.
>>> Don't know. I only configured/built GCC on x86_64 with below command lines:
>>>
>>> $ ../gcc/configure --prefix=...
>>> --enable-languages=c,c++,fortran,go,java,lto,objc,obj-c++
>>> --with-build-config=bootstrap-O3 --disable-werror
>>> $ make && make install
>>>
>>> Not sure what the status for autofdo is in this case. "make check -k"
>>> is stable for me, but "make check -k -j#" gives unstable result in
>>> tree-prof.exp tests. Anything I did wrong?
>>>
>>> Thanks,
>>> bin
>>>
>>
>> I can confirm that I see the same problem on a Xeon machine (../configure && make && make check -k -jX).
>> I see many:
>>
>> g++.dg/tree-prof/pr35545.C: Cannot run create_gcov --binary /home/marxin/Programming/gcc/objdir/gcc/testsuite/g++/pr35545.x01 --profile=perf.data -gcov_version=1 --gcov=/home/marxin/Programming/gcc/objdir/gcc/testsuite/g++/pr35545.gcda
>
> Well that's expected if you don't have autofdo installed or perf with
> lbrs is not working. The test case will be UNSUP in this case.
>
> But it's not expected that it is not deterministic, so that it changes
> run to run. Do you see that? Or some other problem? Please describe
> it exactly.
It definitely changes run to run for me.
Jeff
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 2/3] Run profile feedback tests with autofdo
2016-07-26 4:25 ` Jeff Law
@ 2016-07-26 4:28 ` Andi Kleen
2016-07-27 8:29 ` Martin Liška
2016-07-27 20:20 ` Jeff Law
0 siblings, 2 replies; 23+ messages in thread
From: Andi Kleen @ 2016-07-26 4:28 UTC (permalink / raw)
To: Jeff Law; +Cc: Andi Kleen, Martin Liška, Bin.Cheng, gcc-patches List
> >But it's not expected that it is not deterministic, so that it changes
> >run to run. Do you see that? Or some other problem? Please describe
> >it exactly.
> It definitely changes run to run for me.
And do you have autofdo installed? (create_gcov)
-Andi
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 2/3] Run profile feedback tests with autofdo
2016-07-26 4:28 ` Andi Kleen
@ 2016-07-27 8:29 ` Martin Liška
2016-07-27 20:20 ` Jeff Law
1 sibling, 0 replies; 23+ messages in thread
From: Martin Liška @ 2016-07-27 8:29 UTC (permalink / raw)
To: Andi Kleen, Jeff Law; +Cc: Bin.Cheng, gcc-patches List
On 07/26/2016 06:28 AM, Andi Kleen wrote:
> And do you have autofdo installed? (create_gcov)
>
> -Andi
Ah, sorry for the false alarm, create_gcov is really missing on my distribution.
Martin
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 2/3] Run profile feedback tests with autofdo
2016-07-26 4:28 ` Andi Kleen
2016-07-27 8:29 ` Martin Liška
@ 2016-07-27 20:20 ` Jeff Law
1 sibling, 0 replies; 23+ messages in thread
From: Jeff Law @ 2016-07-27 20:20 UTC (permalink / raw)
To: Andi Kleen; +Cc: Martin Liška, Bin.Cheng, gcc-patches List
On 07/25/2016 10:28 PM, Andi Kleen wrote:
>>> But it's not expected that it is not deterministic, so that it changes
>>> run to run. Do you see that? Or some other problem? Please describe
>>> it exactly.
>> It definitely changes run to run for me.
>
> And do you have autofdo installed? (create_gcov)
No.
jeff
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 1/3] Add gcc-auto-profile script
2016-06-22 21:35 ` Bernhard Reutner-Fischer
@ 2016-06-22 21:43 ` Andi Kleen
0 siblings, 0 replies; 23+ messages in thread
From: Andi Kleen @ 2016-06-22 21:43 UTC (permalink / raw)
To: Bernhard Reutner-Fischer
Cc: Andi Kleen, gcc-patches, hubicka, bschmidt, bonzini, dj,
Ralf.Wildenhues, Andi Kleen
On Wed, Jun 22, 2016 at 11:34:05PM +0200, Bernhard Reutner-Fischer wrote:
> >+ for m in mod[:-1]:
> >+ print "model*:\ %s|\\" % m
> >+ print 'model*:\ %s) E="%s$FLAGS" ;;' % (mod[-1], event)
> >+ print '''*)
> >+echo >&2 "Unknown CPU. Run contrib/gen_autofdo_event.py --all --script
> >to update script."
> >+ exit 1 ;;'''
> >+ print "esac"
> >+ print 'exec perf record -e $E -b "$@"'
>
> Need to quote $E ?
There's never a space in it
> >+if ! grep -q Intel /proc/cpuinfo ] ; then
>
> I'm surprised this even runs?
It works here
$ grep -q Intel /proc/cpuinfo ] ; echo $?
0
But will fix, thanks.
-Andi
--
ak@linux.intel.com -- Speaking for myself only.
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 1/3] Add gcc-auto-profile script
2016-06-22 12:37 [PATCH 1/3] Add gcc-auto-profile script Andi Kleen
2016-06-22 17:25 ` Jeff Law
@ 2016-06-22 21:35 ` Bernhard Reutner-Fischer
2016-06-22 21:43 ` Andi Kleen
1 sibling, 1 reply; 23+ messages in thread
From: Bernhard Reutner-Fischer @ 2016-06-22 21:35 UTC (permalink / raw)
To: Andi Kleen, gcc-patches
Cc: hubicka, bschmidt, bonzini, dj, Ralf.Wildenhues, Andi Kleen
On June 22, 2016 2:37:04 PM GMT+02:00, Andi Kleen <andi@firstfloor.org> wrote:
>From: Andi Kleen <ak@linux.intel.com>
>
>Using autofdo is currently something difficult. It requires using the
>model specific branches taken event, which differs on different CPUs.
>The example shown in the manual requires a special patched version of
>perf that is non standard, and also will likely not work everywhere.
>
>This patch adds a new gcc-auto-profile script that figures out the
>correct event and runs perf.
>
>This is needed to actually make use of autofdo in a generic way
>in the build system and in the test suite.
>
>Since maintaining the script would be somewhat tedious (needs changes
>every time a new CPU comes out) I auto generated it from the online
>Intel event database. The script to do that is in contrib and can be
>rerun.
>
>Right now there is no test if perf works in configure. This
>would vary depending on the build and target system, and since
>it currently doesn't work in virtualization and needs uptodate
>kernel it may often fail in common distribution build setups.
>
>So far the script is not installed.
>
>v2: Remove documentation of gcc-auto-profile, as its not
>installed.
>
>gcc/:
>2016-06-22 Andi Kleen <ak@linux.intel.com>
>
> * doc/invoke.texi: Document gcc-auto-profile
> * config/i386/gcc-auto-profile: New file.
>
>contrib/:
>
>2016-06-22 Andi Kleen <ak@linux.intel.com>
>
> * gen_autofdo_event.py: New file to regenerate
> gcc-auto-profile.
>---
>contrib/gen_autofdo_event.py | 155
>+++++++++++++++++++++++++++++++++++++++
> gcc/config/i386/gcc-auto-profile | 70 ++++++++++++++++++
> 2 files changed, 225 insertions(+)
> create mode 100755 contrib/gen_autofdo_event.py
> create mode 100755 gcc/config/i386/gcc-auto-profile
>
>diff --git a/contrib/gen_autofdo_event.py
>b/contrib/gen_autofdo_event.py
>new file mode 100755
>index 0000000..66cd613
>--- /dev/null
>+++ b/contrib/gen_autofdo_event.py
>@@ -0,0 +1,155 @@
>+#!/usr/bin/python
>+# Generate Intel taken branches Linux perf event script for autofdo
>profiling.
>+
>+# Copyright (C) 2016 Free Software Foundation, Inc.
>+#
>+# GCC is free software; you can redistribute it and/or modify it under
>+# the terms of the GNU General Public License as published by the Free
>+# Software Foundation; either version 3, or (at your option) any later
>+# version.
>+#
>+# GCC is distributed in the hope that it will be useful, but WITHOUT
>ANY
>+# WARRANTY; without even the implied warranty of MERCHANTABILITY or
>+# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public
>License
>+# for more details.
>+#
>+# You should have received a copy of the GNU General Public License
>+# along with GCC; see the file COPYING3. If not see
>+# <http://www.gnu.org/licenses/>. */
>+
>+# Run it with perf record -b -e EVENT program ...
>+# The Linux Kernel needs to support the PMU of the current CPU, and
>+# It will likely not work in VMs.
>+# Add --all to print for all cpus, otherwise for current cpu.
>+# Add --script to generate shell script to run correct event.
>+#
>+# Requires internet (https) access. This may require setting up a
>proxy
>+# with export https_proxy=...
>+#
>+import urllib2
>+import sys
>+import json
>+import argparse
>+import collections
>+
>+baseurl = "https://download.01.org/perfmon"
>+
>+target_events = (u'BR_INST_RETIRED.NEAR_TAKEN',
>+ u'BR_INST_EXEC.TAKEN',
>+ u'BR_INST_RETIRED.TAKEN_JCC',
>+ u'BR_INST_TYPE_RETIRED.COND_TAKEN')
>+
>+ap = argparse.ArgumentParser()
>+ap.add_argument('--all', '-a', help='Print for all CPUs',
>action='store_true')
>+ap.add_argument('--script', help='Generate shell script',
>action='store_true')
>+args = ap.parse_args()
>+
>+eventmap = collections.defaultdict(list)
>+
>+def get_cpu_str():
>+ with open('/proc/cpuinfo', 'r') as c:
>+ vendor, fam, model = None, None, None
>+ for j in c:
>+ n = j.split()
>+ if n[0] == 'vendor_id':
>+ vendor = n[2]
>+ elif n[0] == 'model' and n[1] == ':':
>+ model = int(n[2])
>+ elif n[0] == 'cpu' and n[1] == 'family':
>+ fam = int(n[3])
>+ if vendor and fam and model:
>+ return "%s-%d-%X" % (vendor, fam, model), model
>+ return None, None
>+
>+def find_event(eventurl, model):
>+ print >>sys.stderr, "Downloading", eventurl
>+ u = urllib2.urlopen(eventurl)
>+ events = json.loads(u.read())
>+ u.close()
>+
>+ found = 0
>+ for j in events:
>+ if j[u'EventName'] in target_events:
>+ event = "cpu/event=%s,umask=%s/" % (j[u'EventCode'],
>j[u'UMask'])
>+ if u'PEBS' in j and j[u'PEBS'] > 0:
I'd have said
if j.get(u'PEBS, 0) > 0:
I.e. not use the default None for lets not in the but zero and test against that. I think that's more pythonic but either way.
>+ event += "p"
>+ if args.script:
>+ eventmap[event].append(model)
>+ else:
>+ print j[u'EventName'], "event for model", model, "is",
>event
>+ found += 1
>+ return found
>+
>+if not args.all:
>+ cpu, model = get_cpu_str()
>+ if not cpu:
>+ sys.exit("Unknown CPU type")
>+
>+url = baseurl + "/mapfile.csv"
>+print >>sys.stderr, "Downloading", url
>+u = urllib2.urlopen(url)
>+found = 0
>+cpufound = 0
>+for j in u:
>+ n = j.rstrip().split(',')
>+ if len(n) >= 4 and (args.all or n[0] == cpu) and n[3] == "core":
>+ if args.all:
>+ vendor, fam, model = n[0].split("-")
>+ model = int(model, 16)
>+ cpufound += 1
>+ found += find_event(baseurl + n[2], model)
>+u.close()
>+
>+if args.script:
>+ print '''#!/bin/sh
>+# Profile workload for gcc profile feedback (autofdo) using Linux
>perf.
>+# Auto generated. To regenerate for new CPUs run
>+# contrib/gen_autofdo_event.py --shell --all in gcc source
>+
>+# usages:
>+# gcc-auto-profile program (profile program and children)
>+# gcc-auto-profile -a sleep X (profile all for X secs, may
>need root)
>+# gcc-auto-profile -p PID sleep X (profile PID)
>+# gcc-auto-profile --kernel -a sleep X (profile kernel)
>+# gcc-auto-profile --all -a sleep X (profile kernel and user space)
>+
>+# Identify branches taken event for CPU.
>+#
>+
>+FLAGS=u
>+
>+if [ "$1" = "--kernel" ] ; then
>+ FLAGS=k
>+ shift
>+fi
>+if [ "$1" = "--all" ] ; then
>+ FLAGS=uk
>+ shift
>+fi
Thanks for fixing the above!
>+
>+if ! grep -q Intel /proc/cpuinfo ] ; then
But here there is a bracket too much, unless my MUA plays tricks on me..
>+ echo >&2 "Only Intel CPUs supported"
>+ exit 1
>+fi
>+
>+if grep -q hypervisor /proc/cpuinfo ; then
>+ echo >&2 "Warning: branch profiling may not be functional in VMs"
>+fi
>+
>+case `egrep -q "^cpu family\s*: 6" /proc/cpuinfo &&
>+ egrep "^model\s*:" /proc/cpuinfo | head -n1` in'''
$() please
>+ for event, mod in eventmap.iteritems():
IIRC iteritems is deprecated.
>+ for m in mod[:-1]:
>+ print "model*:\ %s|\\" % m
>+ print 'model*:\ %s) E="%s$FLAGS" ;;' % (mod[-1], event)
>+ print '''*)
>+echo >&2 "Unknown CPU. Run contrib/gen_autofdo_event.py --all --script
>to update script."
>+ exit 1 ;;'''
>+ print "esac"
>+ print 'exec perf record -e $E -b "$@"'
Need to quote $E ?
>+
>+if cpufound == 0 and not args.all:
>+ sys.exit('CPU %s not found' % cpu)
>+
>+if found == 0:
>+ sys.exit('Branch event not found')
>diff --git a/gcc/config/i386/gcc-auto-profile
>b/gcc/config/i386/gcc-auto-profile
>new file mode 100755
>index 0000000..f60cefb
>--- /dev/null
>+++ b/gcc/config/i386/gcc-auto-profile
>@@ -0,0 +1,70 @@
>+#!/bin/sh
>+# profile workload for gcc profile feedback (autofdo) using Linux perf
>+# auto generated. to regenerate for new CPUs run
>+# contrib/gen_autofdo_event.py --shell --all in gcc source
>+
>+# usages:
>+# gcc-auto-profile program (profile program and children)
>+# gcc-auto-profile -a sleep X (profile all for X secs, may
>need root)
>+# gcc-auto-profile -p PID sleep X (profile PID)
>+# gcc-auto-profile --kernel -a sleep X (profile kernel)
>+# gcc-auto-profile --all -a sleep X (profile kernel and user space)
>+
>+# identify branches taken event for CPU
>+#
>+
>+FLAGS=u
>+
>+if [ "$1" = "--kernel" ] ; then
>+ FLAGS=k
>+ shift
>+fi
>+if [ "$1" = "--all" ] ; then
>+ FLAGS=uk
>+ shift
>+fi
>+
>+if ! grep -q Intel /proc/cpuinfo ] ; then
I'm surprised this even runs?
thanks,
>+ echo >&2 "Only Intel CPUs supported"
>+ exit 1
>+fi
>+
>+if grep -q hypervisor /proc/cpuinfo ; then
>+ echo >&2 "Warning: branch profiling may not be functional in VMs"
>+fi
>+
>+case `egrep -q "^cpu family\s*: 6" /proc/cpuinfo &&
>+ egrep "^model\s*:" /proc/cpuinfo | head -n1` in
>+model*:\ 55|\
>+model*:\ 77|\
>+model*:\ 76) E="cpu/event=0xC4,umask=0xFE/p$FLAGS" ;;
>+model*:\ 42|\
>+model*:\ 45|\
>+model*:\ 58|\
>+model*:\ 62|\
>+model*:\ 60|\
>+model*:\ 69|\
>+model*:\ 70|\
>+model*:\ 63|\
>+model*:\ 61|\
>+model*:\ 71|\
>+model*:\ 86|\
>+model*:\ 78|\
>+model*:\ 94) E="cpu/event=0xC4,umask=0x20/p$FLAGS" ;;
>+model*:\ 46|\
>+model*:\ 30|\
>+model*:\ 31|\
>+model*:\ 26|\
>+model*:\ 47|\
>+model*:\ 37|\
>+model*:\ 44) E="cpu/event=0x88,umask=0x40/p$FLAGS" ;;
>+model*:\ 28|\
>+model*:\ 38|\
>+model*:\ 39|\
>+model*:\ 54|\
>+model*:\ 53) E="cpu/event=0x88,umask=0x41/p$FLAGS" ;;
>+*)
>+echo >&2 "Unknown CPU. Run contrib/gen_autofdo_event.py --all --script
>to update script."
>+ exit 1 ;;
>+esac
>+exec perf record -e $E -b "$@"
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 1/3] Add gcc-auto-profile script
2016-06-22 17:44 ` Andi Kleen
@ 2016-06-22 17:51 ` Jeff Law
0 siblings, 0 replies; 23+ messages in thread
From: Jeff Law @ 2016-06-22 17:51 UTC (permalink / raw)
To: Andi Kleen
Cc: Andi Kleen, gcc-patches, hubicka, bschmidt, bonzini, dj, Ralf.Wildenhues
On 06/22/2016 11:44 AM, Andi Kleen wrote:
>>> 2016-06-22 Andi Kleen <ak@linux.intel.com>
>>>
>>> * config/i386/gcc-auto-profile: New file.
>>>
>>> contrib/:
>>>
>>> 2016-06-22 Andi Kleen <ak@linux.intel.com>
>>>
>>> * gen_autofdo_event.py: New file to regenerate
>>> gcc-auto-profile.
>> This part looks fine to me and can probably go in independently of the other
>> bits. Right?
>
> Right. Is that an approval?
Yes.
jeff
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 1/3] Add gcc-auto-profile script
2016-06-22 17:25 ` Jeff Law
@ 2016-06-22 17:44 ` Andi Kleen
2016-06-22 17:51 ` Jeff Law
0 siblings, 1 reply; 23+ messages in thread
From: Andi Kleen @ 2016-06-22 17:44 UTC (permalink / raw)
To: Jeff Law
Cc: Andi Kleen, gcc-patches, hubicka, bschmidt, bonzini, dj, Ralf.Wildenhues
> > 2016-06-22 Andi Kleen <ak@linux.intel.com>
> >
> > * config/i386/gcc-auto-profile: New file.
> >
> > contrib/:
> >
> > 2016-06-22 Andi Kleen <ak@linux.intel.com>
> >
> > * gen_autofdo_event.py: New file to regenerate
> > gcc-auto-profile.
> This part looks fine to me and can probably go in independently of the other
> bits. Right?
Right. Is that an approval?
-Andi
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 1/3] Add gcc-auto-profile script
2016-06-22 12:37 [PATCH 1/3] Add gcc-auto-profile script Andi Kleen
@ 2016-06-22 17:25 ` Jeff Law
2016-06-22 17:44 ` Andi Kleen
2016-06-22 21:35 ` Bernhard Reutner-Fischer
1 sibling, 1 reply; 23+ messages in thread
From: Jeff Law @ 2016-06-22 17:25 UTC (permalink / raw)
To: Andi Kleen, gcc-patches
Cc: hubicka, bschmidt, bonzini, dj, Ralf.Wildenhues, Andi Kleen
On 06/22/2016 06:37 AM, Andi Kleen wrote:
> From: Andi Kleen <ak@linux.intel.com>
>
> Using autofdo is currently something difficult. It requires using the
> model specific branches taken event, which differs on different CPUs.
> The example shown in the manual requires a special patched version of
> perf that is non standard, and also will likely not work everywhere.
>
> This patch adds a new gcc-auto-profile script that figures out the
> correct event and runs perf.
>
> This is needed to actually make use of autofdo in a generic way
> in the build system and in the test suite.
>
> Since maintaining the script would be somewhat tedious (needs changes
> every time a new CPU comes out) I auto generated it from the online
> Intel event database. The script to do that is in contrib and can be
> rerun.
>
> Right now there is no test if perf works in configure. This
> would vary depending on the build and target system, and since
> it currently doesn't work in virtualization and needs uptodate
> kernel it may often fail in common distribution build setups.
>
> So far the script is not installed.
>
> v2: Remove documentation of gcc-auto-profile, as its not
> installed.
>
> gcc/:
> 2016-06-22 Andi Kleen <ak@linux.intel.com>
>
> * doc/invoke.texi: Document gcc-auto-profile
> * config/i386/gcc-auto-profile: New file.
>
> contrib/:
>
> 2016-06-22 Andi Kleen <ak@linux.intel.com>
>
> * gen_autofdo_event.py: New file to regenerate
> gcc-auto-profile.
This part looks fine to me and can probably go in independently of the
other bits. Right?
Jeff
^ permalink raw reply [flat|nested] 23+ messages in thread
* [PATCH 1/3] Add gcc-auto-profile script
@ 2016-06-22 12:37 Andi Kleen
2016-06-22 17:25 ` Jeff Law
2016-06-22 21:35 ` Bernhard Reutner-Fischer
0 siblings, 2 replies; 23+ messages in thread
From: Andi Kleen @ 2016-06-22 12:37 UTC (permalink / raw)
To: gcc-patches; +Cc: hubicka, bschmidt, bonzini, dj, Ralf.Wildenhues, Andi Kleen
From: Andi Kleen <ak@linux.intel.com>
Using autofdo is currently something difficult. It requires using the
model specific branches taken event, which differs on different CPUs.
The example shown in the manual requires a special patched version of
perf that is non standard, and also will likely not work everywhere.
This patch adds a new gcc-auto-profile script that figures out the
correct event and runs perf.
This is needed to actually make use of autofdo in a generic way
in the build system and in the test suite.
Since maintaining the script would be somewhat tedious (needs changes
every time a new CPU comes out) I auto generated it from the online
Intel event database. The script to do that is in contrib and can be
rerun.
Right now there is no test if perf works in configure. This
would vary depending on the build and target system, and since
it currently doesn't work in virtualization and needs uptodate
kernel it may often fail in common distribution build setups.
So far the script is not installed.
v2: Remove documentation of gcc-auto-profile, as its not
installed.
gcc/:
2016-06-22 Andi Kleen <ak@linux.intel.com>
* doc/invoke.texi: Document gcc-auto-profile
* config/i386/gcc-auto-profile: New file.
contrib/:
2016-06-22 Andi Kleen <ak@linux.intel.com>
* gen_autofdo_event.py: New file to regenerate
gcc-auto-profile.
---
contrib/gen_autofdo_event.py | 155 +++++++++++++++++++++++++++++++++++++++
gcc/config/i386/gcc-auto-profile | 70 ++++++++++++++++++
2 files changed, 225 insertions(+)
create mode 100755 contrib/gen_autofdo_event.py
create mode 100755 gcc/config/i386/gcc-auto-profile
diff --git a/contrib/gen_autofdo_event.py b/contrib/gen_autofdo_event.py
new file mode 100755
index 0000000..66cd613
--- /dev/null
+++ b/contrib/gen_autofdo_event.py
@@ -0,0 +1,155 @@
+#!/usr/bin/python
+# Generate Intel taken branches Linux perf event script for autofdo profiling.
+
+# Copyright (C) 2016 Free Software Foundation, Inc.
+#
+# GCC is free software; you can redistribute it and/or modify it under
+# the terms of the GNU General Public License as published by the Free
+# Software Foundation; either version 3, or (at your option) any later
+# version.
+#
+# GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+# WARRANTY; without even the implied warranty of MERCHANTABILITY or
+# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+# for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING3. If not see
+# <http://www.gnu.org/licenses/>. */
+
+# Run it with perf record -b -e EVENT program ...
+# The Linux Kernel needs to support the PMU of the current CPU, and
+# It will likely not work in VMs.
+# Add --all to print for all cpus, otherwise for current cpu.
+# Add --script to generate shell script to run correct event.
+#
+# Requires internet (https) access. This may require setting up a proxy
+# with export https_proxy=...
+#
+import urllib2
+import sys
+import json
+import argparse
+import collections
+
+baseurl = "https://download.01.org/perfmon"
+
+target_events = (u'BR_INST_RETIRED.NEAR_TAKEN',
+ u'BR_INST_EXEC.TAKEN',
+ u'BR_INST_RETIRED.TAKEN_JCC',
+ u'BR_INST_TYPE_RETIRED.COND_TAKEN')
+
+ap = argparse.ArgumentParser()
+ap.add_argument('--all', '-a', help='Print for all CPUs', action='store_true')
+ap.add_argument('--script', help='Generate shell script', action='store_true')
+args = ap.parse_args()
+
+eventmap = collections.defaultdict(list)
+
+def get_cpu_str():
+ with open('/proc/cpuinfo', 'r') as c:
+ vendor, fam, model = None, None, None
+ for j in c:
+ n = j.split()
+ if n[0] == 'vendor_id':
+ vendor = n[2]
+ elif n[0] == 'model' and n[1] == ':':
+ model = int(n[2])
+ elif n[0] == 'cpu' and n[1] == 'family':
+ fam = int(n[3])
+ if vendor and fam and model:
+ return "%s-%d-%X" % (vendor, fam, model), model
+ return None, None
+
+def find_event(eventurl, model):
+ print >>sys.stderr, "Downloading", eventurl
+ u = urllib2.urlopen(eventurl)
+ events = json.loads(u.read())
+ u.close()
+
+ found = 0
+ for j in events:
+ if j[u'EventName'] in target_events:
+ event = "cpu/event=%s,umask=%s/" % (j[u'EventCode'], j[u'UMask'])
+ if u'PEBS' in j and j[u'PEBS'] > 0:
+ event += "p"
+ if args.script:
+ eventmap[event].append(model)
+ else:
+ print j[u'EventName'], "event for model", model, "is", event
+ found += 1
+ return found
+
+if not args.all:
+ cpu, model = get_cpu_str()
+ if not cpu:
+ sys.exit("Unknown CPU type")
+
+url = baseurl + "/mapfile.csv"
+print >>sys.stderr, "Downloading", url
+u = urllib2.urlopen(url)
+found = 0
+cpufound = 0
+for j in u:
+ n = j.rstrip().split(',')
+ if len(n) >= 4 and (args.all or n[0] == cpu) and n[3] == "core":
+ if args.all:
+ vendor, fam, model = n[0].split("-")
+ model = int(model, 16)
+ cpufound += 1
+ found += find_event(baseurl + n[2], model)
+u.close()
+
+if args.script:
+ print '''#!/bin/sh
+# Profile workload for gcc profile feedback (autofdo) using Linux perf.
+# Auto generated. To regenerate for new CPUs run
+# contrib/gen_autofdo_event.py --shell --all in gcc source
+
+# usages:
+# gcc-auto-profile program (profile program and children)
+# gcc-auto-profile -a sleep X (profile all for X secs, may need root)
+# gcc-auto-profile -p PID sleep X (profile PID)
+# gcc-auto-profile --kernel -a sleep X (profile kernel)
+# gcc-auto-profile --all -a sleep X (profile kernel and user space)
+
+# Identify branches taken event for CPU.
+#
+
+FLAGS=u
+
+if [ "$1" = "--kernel" ] ; then
+ FLAGS=k
+ shift
+fi
+if [ "$1" = "--all" ] ; then
+ FLAGS=uk
+ shift
+fi
+
+if ! grep -q Intel /proc/cpuinfo ] ; then
+ echo >&2 "Only Intel CPUs supported"
+ exit 1
+fi
+
+if grep -q hypervisor /proc/cpuinfo ; then
+ echo >&2 "Warning: branch profiling may not be functional in VMs"
+fi
+
+case `egrep -q "^cpu family\s*: 6" /proc/cpuinfo &&
+ egrep "^model\s*:" /proc/cpuinfo | head -n1` in'''
+ for event, mod in eventmap.iteritems():
+ for m in mod[:-1]:
+ print "model*:\ %s|\\" % m
+ print 'model*:\ %s) E="%s$FLAGS" ;;' % (mod[-1], event)
+ print '''*)
+echo >&2 "Unknown CPU. Run contrib/gen_autofdo_event.py --all --script to update script."
+ exit 1 ;;'''
+ print "esac"
+ print 'exec perf record -e $E -b "$@"'
+
+if cpufound == 0 and not args.all:
+ sys.exit('CPU %s not found' % cpu)
+
+if found == 0:
+ sys.exit('Branch event not found')
diff --git a/gcc/config/i386/gcc-auto-profile b/gcc/config/i386/gcc-auto-profile
new file mode 100755
index 0000000..f60cefb
--- /dev/null
+++ b/gcc/config/i386/gcc-auto-profile
@@ -0,0 +1,70 @@
+#!/bin/sh
+# profile workload for gcc profile feedback (autofdo) using Linux perf
+# auto generated. to regenerate for new CPUs run
+# contrib/gen_autofdo_event.py --shell --all in gcc source
+
+# usages:
+# gcc-auto-profile program (profile program and children)
+# gcc-auto-profile -a sleep X (profile all for X secs, may need root)
+# gcc-auto-profile -p PID sleep X (profile PID)
+# gcc-auto-profile --kernel -a sleep X (profile kernel)
+# gcc-auto-profile --all -a sleep X (profile kernel and user space)
+
+# identify branches taken event for CPU
+#
+
+FLAGS=u
+
+if [ "$1" = "--kernel" ] ; then
+ FLAGS=k
+ shift
+fi
+if [ "$1" = "--all" ] ; then
+ FLAGS=uk
+ shift
+fi
+
+if ! grep -q Intel /proc/cpuinfo ] ; then
+ echo >&2 "Only Intel CPUs supported"
+ exit 1
+fi
+
+if grep -q hypervisor /proc/cpuinfo ; then
+ echo >&2 "Warning: branch profiling may not be functional in VMs"
+fi
+
+case `egrep -q "^cpu family\s*: 6" /proc/cpuinfo &&
+ egrep "^model\s*:" /proc/cpuinfo | head -n1` in
+model*:\ 55|\
+model*:\ 77|\
+model*:\ 76) E="cpu/event=0xC4,umask=0xFE/p$FLAGS" ;;
+model*:\ 42|\
+model*:\ 45|\
+model*:\ 58|\
+model*:\ 62|\
+model*:\ 60|\
+model*:\ 69|\
+model*:\ 70|\
+model*:\ 63|\
+model*:\ 61|\
+model*:\ 71|\
+model*:\ 86|\
+model*:\ 78|\
+model*:\ 94) E="cpu/event=0xC4,umask=0x20/p$FLAGS" ;;
+model*:\ 46|\
+model*:\ 30|\
+model*:\ 31|\
+model*:\ 26|\
+model*:\ 47|\
+model*:\ 37|\
+model*:\ 44) E="cpu/event=0x88,umask=0x40/p$FLAGS" ;;
+model*:\ 28|\
+model*:\ 38|\
+model*:\ 39|\
+model*:\ 54|\
+model*:\ 53) E="cpu/event=0x88,umask=0x41/p$FLAGS" ;;
+*)
+echo >&2 "Unknown CPU. Run contrib/gen_autofdo_event.py --all --script to update script."
+ exit 1 ;;
+esac
+exec perf record -e $E -b "$@"
--
2.8.3
^ permalink raw reply [flat|nested] 23+ messages in thread
end of thread, other threads:[~2016-07-27 20:20 UTC | newest]
Thread overview: 23+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-06-23 14:37 Committed version of autofdo testing&building patches Andi Kleen
2016-06-23 14:38 ` [PATCH 1/3] Add gcc-auto-profile script Andi Kleen
2016-06-23 14:38 ` [PATCH 2/3] Run profile feedback tests with autofdo Andi Kleen
2016-07-12 15:13 ` Bin.Cheng
2016-07-14 10:55 ` Andi Kleen
2016-07-14 11:06 ` Bin.Cheng
2016-07-14 11:08 ` Bin.Cheng
2016-07-14 22:33 ` Andi Kleen
2016-07-15 8:37 ` Bin.Cheng
2016-07-15 19:44 ` Jeff Law
2016-07-15 20:53 ` Andi Kleen
2016-07-25 11:08 ` Martin Liška
2016-07-26 4:21 ` Andi Kleen
2016-07-26 4:25 ` Jeff Law
2016-07-26 4:28 ` Andi Kleen
2016-07-27 8:29 ` Martin Liška
2016-07-27 20:20 ` Jeff Law
-- strict thread matches above, loose matches on Subject: below --
2016-06-22 12:37 [PATCH 1/3] Add gcc-auto-profile script Andi Kleen
2016-06-22 17:25 ` Jeff Law
2016-06-22 17:44 ` Andi Kleen
2016-06-22 17:51 ` Jeff Law
2016-06-22 21:35 ` Bernhard Reutner-Fischer
2016-06-22 21:43 ` Andi Kleen
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).