From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 1508 invoked by alias); 19 Dec 2012 20:08:51 -0000 Received: (qmail 1457 invoked by uid 22791); 19 Dec 2012 20:08:40 -0000 X-SWARE-Spam-Status: No, hits=-4.5 required=5.0 tests=AWL,BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,KHOP_RCVD_TRUST,RCVD_IN_DNSWL_LOW,RCVD_IN_HOSTKARMA_YE,TW_TM,T_RP_MATCHES_RCVD X-Spam-Check-By: sourceware.org Received: from mail-fa0-f73.google.com (HELO mail-fa0-f73.google.com) (209.85.161.73) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Wed, 19 Dec 2012 20:08:31 +0000 Received: by mail-fa0-f73.google.com with SMTP id v1so132705fav.2 for ; Wed, 19 Dec 2012 12:08:29 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:date:to:subject:user-agent:mime-version:content-type :content-transfer-encoding:message-id:from:x-gm-message-state; bh=1ZOgdX9TfLTSgms3bQrXLP3/q/2qSE9HyG2pP3IGlLc=; b=erXAfgv+DsvXB3pitP00szMb598DwKf3kS8pCrenHxSVh8vfEdyAOqO1+SCE5tLAUH XuM3UmwO7uwCbUDU3FxidhS/NhwDY4I7Hg/YB4VHoyxM92W1wViMgy7fttdAUyi7hOC5 e3FbP+1/U24jE2uiey9VPJ/HvUg7rmpSZZPMtH1C2DHC3bkfJeQEkwRzRS/ke/wgCFwb CkMg/fjabnkcBsRvyoFxfz+Z5IT9TIwlZVPR2CyUFi9i2ajwrzKF9jYBKqIWdaeMw1tZ YOfvlcWYrK/F57HNL6CnSH/CFHUjGxv08LjXQojwe5PgA4gvf5ZREs/uN2FOmjSG2TOP +ekg== X-Received: by 10.14.214.197 with SMTP id c45mr9385104eep.7.1355947709744; Wed, 19 Dec 2012 12:08:29 -0800 (PST) Received: from hpza10.eem.corp.google.com ([74.125.121.33]) by gmr-mx.google.com with ESMTPS id z44si2069222een.0.2012.12.19.12.08.29 (version=TLSv1/SSLv3 cipher=AES128-SHA); Wed, 19 Dec 2012 12:08:29 -0800 (PST) Received: from rong.mtv.corp.google.com (rong.mtv.corp.google.com [172.17.128.127]) by hpza10.eem.corp.google.com (Postfix) with ESMTP id 39425200069; Wed, 19 Dec 2012 12:08:29 -0800 (PST) Received: by rong.mtv.corp.google.com (Postfix, from userid 104659) id 73DB9106927; Wed, 19 Dec 2012 12:08:28 -0800 (PST) Date: Wed, 19 Dec 2012 20:08:00 -0000 To: gcc-patches@gcc.gnu.org, davidxl@google.com, reply@codereview.appspotmail.com Subject: [google 4.7] atomic update of profile counters (issue6965050) User-Agent: Heirloom mailx 12.5 6/20/10 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-Id: <20121219200828.73DB9106927@rong.mtv.corp.google.com> From: xur@google.com (Rong Xu) X-Gm-Message-State: ALoCoQlszLeGcj/nM0BT037cGMnkEbIFApOYBKsHSDuwmTaeeM9LAt5EtzzX6YRT8H8qUWamDjx1ThrSZoeEWpAcVt2DXXR6RaAA9U2sBmkJR94k+faM2EFFQLnm3Av5+8NZ9ciLQQFbuPBLSDpZk6YgocRSWZQ6iGe4aniGpriRg0BgURdaXPMTGeGJuTsNQdMAZPOEGQQN X-IsSubscribed: yes Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org X-SW-Source: 2012-12/txt/msg01207.txt.bz2 Hi, This patch adds the supprot of atomic update the profile counters. Tested with google internal benchmarks and fdo kernel build. Thanks, -Rong 2012-12-19 Rong Xu * gcc/common.opt: Add -fprofile-gen-atomic option. * gcc/gcov-io.h: Add profile atomic update support. * gcc/tree-profile.c (gimple_init_edge_profiler): Ditto. (gimple_gen_edge_profiler): Ditto. * libgcc/libgcov.c (__gcov_one_value_profiler_body): Ditto. (__gcov_one_value_profiler_body_atomic): Ditto. (__gcov_one_value_profiler_atomic): Ditto. (__gcov_indirect_call_profiler_atomic): Ditto. Index: gcc/common.opt =================================================================== --- gcc/common.opt (revision 194562) +++ gcc/common.opt (working copy) @@ -1754,6 +1754,15 @@ fprofile-dump Common Report Var(flag_profile_dump) Init(0) Optimization Dump CFG profile for comparison. +; fprofile-gen-atomic=0: disable aotimically update. +; fprofile-gen-atomic=1: aotimically update edge profile counters. +; fprofile-gen-atomic=2: aotimically update value profile counters. +; fprofile-gen-atomic=3: aotimically update edge and value profile counters. +; other values will be ignored (fall back to the default of 0). +fprofile-gen-atomic= +Common Joined UInteger Report Var(flag_profile_gen_atomic) Init(0) Optimization +fprofile-gen-atomic=[0..3] Atomically increments for profile counters. + fprofile-generate Common Enable common options for generating profile info for profile feedback directed optimizations Index: gcc/gcov-io.h =================================================================== --- gcc/gcov-io.h (revision 194562) +++ gcc/gcov-io.h (working copy) @@ -300,6 +300,14 @@ typedef unsigned gcov_type_unsigned __attribute__ #endif /* BITS_PER_UNIT == 8 */ +#if LONG_LONG_TYPE_SIZE > 32 +#define GCOV_TYPE_SYNC_FETCH_AND_ADD_FN __sync_fetch_and_add_8 +#define GCOV_TYPE_SYNC_FETCH_AND_ADD BUILT_IN_SYNC_FETCH_AND_ADD_8 +#else +#define GCOV_TYPE_SYNC_FETCH_AND_ADD_FN __sync_fetch_and_add_4 +#define GCOV_TYPE_SYNC_FETCH_AND_ADD BUILT_IN_SYNC_FETCH_AND_ADD_4 +#endif + #undef EXTRACT_MODULE_ID_FROM_GLOBAL_ID #undef EXTRACT_FUNC_ID_FROM_GLOBAL_ID #undef GEN_FUNC_GLOBAL_ID @@ -322,6 +330,18 @@ typedef unsigned gcov_type_unsigned __attribute__ typedef unsigned gcov_unsigned_t; typedef unsigned gcov_position_t; +#if LONG_LONG_TYPE_SIZE > 32 +#define GCOV_TYPE_SYNC_FETCH_AND_ADD_FN __sync_fetch_and_add_8 +#define GCOV_TYPE_SYNC_FETCH_AND_ADD BUILT_IN_SYNC_FETCH_AND_ADD_8 +#else +#define GCOV_TYPE_SYNC_FETCH_AND_ADD_FN __sync_fetch_and_add_4 +#define GCOV_TYPE_SYNC_FETCH_AND_ADD BUILT_IN_SYNC_FETCH_AND_ADD_4 +#endif +#define PROFILE_GEN_EDGE_ATOMIC (flag_profile_gen_atomic == 1 || \ + flag_profile_gen_atomic == 3) +#define PROFILE_GEN_VALUE_ATOMIC (flag_profile_gen_atomic == 2 || \ + flag_profile_gen_atomic == 3) + /* gcov_type is typedef'd elsewhere for the compiler */ #if IN_GCOV #define GCOV_LINKAGE static Index: gcc/tree-profile.c =================================================================== --- gcc/tree-profile.c (revision 194562) +++ gcc/tree-profile.c (working copy) @@ -471,7 +471,12 @@ gimple_init_edge_profiler (void) = build_function_type_list (void_type_node, gcov_type_ptr, gcov_type_node, NULL_TREE); - tree_one_value_profiler_fn + if (PROFILE_GEN_VALUE_ATOMIC) + tree_one_value_profiler_fn + = build_fn_decl ("__gcov_one_value_profiler_atomic", + one_value_profiler_fn_type); + else + tree_one_value_profiler_fn = build_fn_decl ("__gcov_one_value_profiler", one_value_profiler_fn_type); TREE_NOTHROW (tree_one_value_profiler_fn) = 1; @@ -487,7 +492,12 @@ gimple_init_edge_profiler (void) gcov_type_ptr, gcov_type_node, ptr_void, ptr_void, NULL_TREE); - tree_indirect_call_profiler_fn + if (PROFILE_GEN_VALUE_ATOMIC) + tree_indirect_call_profiler_fn + = build_fn_decl ("__gcov_indirect_call_profiler_atomic", + ic_profiler_fn_type); + else + tree_indirect_call_profiler_fn = build_fn_decl ("__gcov_indirect_call_profiler", ic_profiler_fn_type); TREE_NOTHROW (tree_indirect_call_profiler_fn) = 1; @@ -563,21 +573,37 @@ gimple_gen_edge_profiler (int edgeno, edge e) gets re-set in tree_profiling. */ if (gcov_type_tmp_var == NULL_TREE) gcov_type_tmp_var = create_tmp_reg (gcov_type_node, "PROF_edge_counter"); - ref = tree_coverage_counter_ref (GCOV_COUNTER_ARCS, edgeno); + + if (PROFILE_GEN_EDGE_ATOMIC) + ref = tree_coverage_counter_addr (GCOV_COUNTER_ARCS, edgeno); + else + ref = tree_coverage_counter_ref (GCOV_COUNTER_ARCS, edgeno); + one = build_int_cst (gcov_type_node, 1); - stmt1 = gimple_build_assign (gcov_type_tmp_var, ref); - gimple_assign_set_lhs (stmt1, make_ssa_name (gcov_type_tmp_var, stmt1)); - find_referenced_vars_in (stmt1); - stmt2 = gimple_build_assign_with_ops (PLUS_EXPR, gcov_type_tmp_var, - gimple_assign_lhs (stmt1), one); - gimple_assign_set_lhs (stmt2, make_ssa_name (gcov_type_tmp_var, stmt2)); - stmt3 = gimple_build_assign (unshare_expr (ref), gimple_assign_lhs (stmt2)); + if (PROFILE_GEN_EDGE_ATOMIC) + { + /* __sync_fetch_and_add_8 (&counter, 1); */ + stmt3 = gimple_build_call (builtin_decl_explicit + (GCOV_TYPE_SYNC_FETCH_AND_ADD), + 2, ref, one); + find_referenced_vars_in (stmt3); + } + else + { + stmt1 = gimple_build_assign (gcov_type_tmp_var, ref); + gimple_assign_set_lhs (stmt1, make_ssa_name (gcov_type_tmp_var, stmt1)); + find_referenced_vars_in (stmt1); + stmt2 = gimple_build_assign_with_ops (PLUS_EXPR, gcov_type_tmp_var, + gimple_assign_lhs (stmt1), one); + gimple_assign_set_lhs (stmt2, make_ssa_name (gcov_type_tmp_var, stmt2)); + stmt3 = gimple_build_assign (unshare_expr (ref), gimple_assign_lhs (stmt2)); - if (flag_profile_generate_sampling) - pointer_set_insert (instrumentation_to_be_sampled, stmt1); + if (flag_profile_generate_sampling) + pointer_set_insert (instrumentation_to_be_sampled, stmt1); - gsi_insert_on_edge (e, stmt1); - gsi_insert_on_edge (e, stmt2); + gsi_insert_on_edge (e, stmt1); + gsi_insert_on_edge (e, stmt2); + } gsi_insert_on_edge (e, stmt3); } Index: libgcc/libgcov.c =================================================================== --- libgcc/libgcov.c (revision 194562) +++ libgcc/libgcov.c (working copy) @@ -1632,6 +1632,22 @@ __gcov_one_value_profiler_body (gcov_type *counter counters[2]++; } +/* Atomic update version of __gcov_one_value_profile_body(). */ +static inline void +__gcov_one_value_profiler_body_atomic (gcov_type *counters, gcov_type value) +{ + if (value == counters[0]) + GCOV_TYPE_SYNC_FETCH_AND_ADD_FN (&counters[1], 1); + else if (counters[1] == 0) + { + counters[1] = 1; + counters[0] = value; + } + else + GCOV_TYPE_SYNC_FETCH_AND_ADD_FN (&counters[1], -1); + GCOV_TYPE_SYNC_FETCH_AND_ADD_FN (&counters[2], 1); +} + #ifdef L_gcov_indirect_call_topn_profiler /* Tries to keep track the most frequent N values in the counters where N is specified by parameter TOPN_VAL. To track top N values, 2*N counter @@ -1740,6 +1756,12 @@ __gcov_one_value_profiler (gcov_type *counters, gc { __gcov_one_value_profiler_body (counters, value); } + +void +__gcov_one_value_profiler_atomic (gcov_type *counters, gcov_type value) +{ + __gcov_one_value_profiler_body_atomic (counters, value); +} #endif #ifdef L_gcov_indirect_call_profiler @@ -1774,6 +1796,17 @@ __gcov_indirect_call_profiler (gcov_type* counter, && *(void **) cur_func == *(void **) callee_func)) __gcov_one_value_profiler_body (counter, value); } + +/* Atomic update version of __gcov_indirect_call_profiler(). */ +void +__gcov_indirect_call_profiler_atomic (gcov_type* counter, gcov_type value, + void* cur_func, void* callee_func) +{ + if (cur_func == callee_func + || (VTABLE_USES_DESCRIPTORS && callee_func + && *(void **) cur_func == *(void **) callee_func)) + __gcov_one_value_profiler_body_atomic (counter, value); +} #endif @@ -2089,9 +2122,11 @@ EXPORT_SYMBOL (__gcov_merge_reusedist); EXPORT_SYMBOL (__gcov_average_profiler); EXPORT_SYMBOL (__gcov_indirect_call_profiler); +EXPORT_SYMBOL (__gcov_indirect_call_profiler_atomic); EXPORT_SYMBOL (__gcov_interval_profiler); EXPORT_SYMBOL (__gcov_ior_profiler); EXPORT_SYMBOL (__gcov_one_value_profiler); +EXPORT_SYMBOL (__gcov_one_value_profiler_atomic); EXPORT_SYMBOL (__gcov_pow2_profiler); #endif /* __GCOV_KERNEL__ */ -- This patch is available for review at http://codereview.appspot.com/6965050