Re: [C-compiler-team] [google] ThreadSanitizer instrumentation pass (issue5303083)

[Xinliang David Li <davidxl@google.com>]

Dmitriy, I will review it next week.

Do you have some runtime overhead data ?

thanks,

David

On Fri, Oct 28, 2011 at 6:25 AM, Dmitriy Vyukov <dvyukov@google.com> wrote:
> The patch is for google/main branch.
> ThreadSanitizer is a data race detector for C/C++ programs.
> http://code.google.com/p/data-race-test/wiki/ThreadSanitizer
>
> The tool consists of two parts:
> instrumentation module (this file) and a run-time library.
> The instrumentation module mainintains shadow call stacks
> and intercepts interesting memory accesses.
> The instrumentation is enabled with -ftsan flag.
>
> Instrumentation for shadow stack maintainance is as follows:
> void somefunc ()
> {
>  __tsan_shadow_stack [-1] = __builtin_return_address (0);
>  __tsan_shadow_stack++;
>  // function body
>  __tsan_shadow_stack--;
> }
>
> Interception for memory access interception is as follows:
> *addr = 1;
> __tsan_handle_mop (addr, flags);
> where flags are (is_sblock | (is_store << 1) | ((sizeof (*addr) - 1) << 2).
> is_sblock is used merely for optimization purposes and can always
> be set to 1, see comments in instrument_mops function.
>
> Ignore files can be used to selectively non instrument some functions.
> Ignore file is specified with -ftsan-ignore=filename flag.
> There are 3 types of ignores: (1) do not instrument memory accesses
> in the function, (2) do not create sblocks in the function
> and (3) recursively ignore memory accesses in the function.
> That last ignore type requires additional instrumentation of the form:
> void somefunc ()
> {
>  __tsan_thread_ignore++;
>  // function body
>  __tsan_thread_ignore--;
> }
>
> The run-time library provides __tsan_handle_mop function,
> definitions of __tsan_shadow_stack and __tsan_thread_ignore variables,
> and intercepts synchronization related functions.
>
> 2011-10-28   Dmitriy Vyukov  <dvyukov@google.com>
>
>        * gcc/doc/invoke.texi:
>        * gcc/tree-tsan.c (enum tsan_ignore_e):
>        (enum bb_state_e):
>        (struct bb_data_t):
>        (struct mop_desc_t):
>        (struct tsan_ignore_desc_t):
>        (lookup_name):
>        (shadow_stack_def):
>        (thread_ignore_def):
>        (rtl_mop_def):
>        (ignore_append):
>        (ignore_match):
>        (ignore_load):
>        (tsan_ignore):
>        (decl_name):
>        (build_stack_op):
>        (build_rec_ignore_op):
>        (build_stack_assign):
>        (instr_mop):
>        (instr_vptr_store):
>        (instr_func):
>        (set_location):
>        (is_dtor_vptr_store):
>        (is_vtbl_read):
>        (is_load_of_const):
>        (handle_expr):
>        (handle_gimple):
>        (instrument_bblock):
>        (instrument_mops):
>        (instrument_function):
>        (tsan_pass):
>        (tsan_gate):
>        * gcc/tree-pass.h:
>        * gcc/testsuite/gcc.dg/tsan-ignore.ignore:
>        * gcc/testsuite/gcc.dg/tsan.h (__tsan_init):
>        (__tsan_expect_mop):
>        (__tsan_handle_mop):
>        * gcc/testsuite/gcc.dg/tsan-ignore.c (foo):
>        (int bar):
>        (int baz):
>        (int bla):
>        (int xxx):
>        (main):
>        * gcc/testsuite/gcc.dg/tsan-ignore.h (in_tsan_ignore_header):
>        * gcc/testsuite/gcc.dg/tsan-stack.c (foobar):
>        * gcc/testsuite/gcc.dg/tsan-mop.c:
>        * gcc/common.opt:
>        * gcc/Makefile.in:
>        * gcc/passes.c:
>
> Index: gcc/doc/invoke.texi
> ===================================================================
> --- gcc/doc/invoke.texi (revision 180522)
> +++ gcc/doc/invoke.texi (working copy)
> @@ -308,6 +308,7 @@
>  -fdump-tree-ssa@r{[}-@var{n}@r{]} -fdump-tree-pre@r{[}-@var{n}@r{]} @gol
>  -fdump-tree-ccp@r{[}-@var{n}@r{]} -fdump-tree-dce@r{[}-@var{n}@r{]} @gol
>  -fdump-tree-gimple@r{[}-raw@r{]} -fdump-tree-mudflap@r{[}-@var{n}@r{]} @gol
> +-fdump-tree-tsan@r{[}-@var{n}@r{]} @gol
>  -fdump-tree-dom@r{[}-@var{n}@r{]} @gol
>  -fdump-tree-dse@r{[}-@var{n}@r{]} @gol
>  -fdump-tree-phiprop@r{[}-@var{n}@r{]} @gol
> @@ -381,8 +382,8 @@
>  -floop-parallelize-all -flto -flto-compression-level @gol
>  -flto-partition=@var{alg} -flto-report -fmerge-all-constants @gol
>  -fmerge-constants -fmodulo-sched -fmodulo-sched-allow-regmoves @gol
> --fmove-loop-invariants fmudflap -fmudflapir -fmudflapth -fno-branch-count-reg @gol
> --fno-default-inline @gol
> +-fmove-loop-invariants -fmudflap -fmudflapir -fmudflapth -fno-branch-count-reg @gol
> +-ftsan -ftsan-ignore -fno-default-inline @gol
>  -fno-defer-pop -fno-function-cse -fno-guess-branch-probability @gol
>  -fno-inline -fno-math-errno -fno-peephole -fno-peephole2 @gol
>  -fno-sched-interblock -fno-sched-spec -fno-signed-zeros @gol
> @@ -5896,6 +5897,11 @@
>  Dump each function after adding mudflap instrumentation.  The file name is
>  made by appending @file{.mudflap} to the source file name.
>
> +@item tsan
> +@opindex fdump-tree-tsan
> +Dump each function after adding ThreadSanitizer instrumentation.  The file name is
> +made by appending @file{.tsan} to the source file name.
> +
>  @item sra
>  @opindex fdump-tree-sra
>  Dump each function after performing scalar replacement of aggregates.  The
> @@ -6674,6 +6680,12 @@
>  some protection against outright memory corrupting writes, but allows
>  erroneously read data to propagate within a program.
>
> +@item -ftsan -ftsan-ignore
> +@opindex ftsan
> +@opindex ftsan-ignore
> +Add ThreadSanitizer instrumentation. Use @option{-ftsan-ignore} to specify
> +an ignore file. Refer to http://go/tsan for details.
> +
>  @item -fthread-jumps
>  @opindex fthread-jumps
>  Perform optimizations where we check to see if a jump branches to a
> Index: gcc/tree-tsan.c
> ===================================================================
> --- gcc/tree-tsan.c     (revision 0)
> +++ gcc/tree-tsan.c     (revision 0)
> @@ -0,0 +1,1204 @@
> +/* ThreadSanitizer instrumentation pass.
> +   http://code.google.com/p/data-race-test
> +   Copyright (C) 2011
> +   Free Software Foundation, Inc.
> +   Contributed by Dmitry Vyukov <dvyukov@google.com>
> +
> +This file is part of GCC.
> +
> +GCC is free software; you can redistribute it and/or modify
> +it under the terms of the GNU General Public License as published by
> +the Free Software Foundation; either version 3, or (at your option)
> +any later version.
> +
> +GCC is distributed in the hope that it will be useful,
> +but WITHOUT ANY WARRANTY; without even the implied warranty of
> +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +GNU General Public License for more details.
> +
> +You should have received a copy of the GNU General Public License
> +along with GCC; see the file COPYING3.  If not see
> +<http://www.gnu.org/licenses/>.  */
> +
> +#include "config.h"
> +#include "system.h"
> +#include "coretypes.h"
> +#include "tree.h"
> +#include "intl.h"
> +#include "tm.h"
> +#include "basic-block.h"
> +#include "gimple.h"
> +#include "function.h"
> +#include "tree-flow.h"
> +#include "tree-pass.h"
> +#include "cfghooks.h"
> +#include "langhooks.h"
> +#include "output.h"
> +#include "options.h"
> +
> +/* The file can be compiled either as compiler pass or plugin.  */
> +#ifdef GCC_PLG
> +# include "c-common.h"
> +#else
> +# include "c-family/c-common.h"
> +#endif
> +
> +#include "diagnostic.h"
> +
> +#include <stdlib.h>
> +#include <stdio.h>
> +
> +/* ThreadSanitizer is a data race detector for C/C++ programs.
> +   http://code.google.com/p/data-race-test/wiki/ThreadSanitizer
> +
> +   The tool consists of two parts:
> +   instrumentation module (this file) and a run-time library.
> +   The instrumentation module mainintains shadow call stacks
> +   and intercepts interesting memory accesses.
> +   The instrumentation is enabled with -ftsan flag.
> +
> +   Instrumentation for shadow stack maintainance is as follows:
> +   void somefunc ()
> +   {
> +     __tsan_shadow_stack [-1] = __builtin_return_address (0);
> +     __tsan_shadow_stack++;
> +     // function body
> +     __tsan_shadow_stack--;
> +   }
> +
> +   Interception for memory access interception is as follows:
> +   *addr = 1;
> +   __tsan_handle_mop (addr, flags);
> +   where flags are (is_sblock | (is_store << 1) | ((sizeof (*addr) - 1) << 2).
> +   is_sblock is used merely for optimization purposes and can always
> +   be set to 1, see comments in instrument_mops function.
> +
> +   Ignore files can be used to selectively non instrument some functions.
> +   Ignore file is specified with -ftsan-ignore=filename flag.
> +   There are 3 types of ignores: (1) do not instrument memory accesses
> +   in the function, (2) do not create sblocks in the function
> +   and (3) recursively ignore memory accesses in the function.
> +   That last ignore type requires additional instrumentation of the form:
> +   void somefunc ()
> +   {
> +     __tsan_thread_ignore++;
> +     // function body
> +     __tsan_thread_ignore--;
> +   }
> +
> +   The run-time library provides __tsan_handle_mop function,
> +   definitions of __tsan_shadow_stack and __tsan_thread_ignore variables,
> +   and intercepts synchronization related functions.  */
> +
> +#define RTL_IGNORE "__tsan_thread_ignore"
> +#define RTL_STACK "__tsan_shadow_stack"
> +#define RTL_MOP "__tsan_handle_mop"
> +#define RTL_PERFIX "__tsan_"
> +#define MAX_MOP_BYTES 16
> +#define SBLOCK_SIZE 5
> +
> +enum tsan_ignore_e
> +{
> +  tsan_ignore_none  = 1 << 0, /* Do not ignore.  */
> +  tsan_ignore_func  = 1 << 1, /* Completely ignore the whole func.  */
> +  tsan_ignore_mop   = 1 << 2, /* Do not instrument memory accesses.  */
> +  tsan_ignore_rec   = 1 << 3, /* Do not instrument memory accesses recursively.  */
> +  tsan_ignore_hist  = 1 << 4  /* Do not create superblocks.  */
> +};
> +
> +/* Basic block state during CFG traversal.  */
> +enum bb_state_e
> +{
> +  bb_not_visited,
> +  bb_candidate,
> +  bb_visited
> +};
> +
> +/* Info associated with each basic block.
> +   Used to determine super-blocks (see instrument_mops ()).  */
> +struct bb_data_t
> +{
> +  enum bb_state_e       state;
> +  int                   has_sb;
> +  const char           *sb_file;
> +  int                   sb_line_min;
> +  int                   sb_line_max;
> +};
> +
> +/* Memory access descriptor.  */
> +struct mop_desc_t
> +{
> +  int                   is_call;
> +  gimple_stmt_iterator  gsi;
> +  tree                  expr;
> +  tree                  dtor_vptr_expr;
> +  int                   is_store;
> +};
> +
> +struct tsan_ignore_desc_t
> +{
> +  struct tsan_ignore_desc_t *next;
> +  enum tsan_ignore_e         type;
> +  char                      *name;
> +};
> +
> +/* Number of instrumented memory accesses in the current function.  */
> +static int func_mops;
> +/* Number of function calls in the current function.  */
> +static int func_calls;
> +/* Ignore status for the current function (see tsan_ignore_e).  */
> +static enum tsan_ignore_e func_ignore;
> +
> +static int ignore_init = 0;
> +static struct tsan_ignore_desc_t *ignore_head;
> +
> +typedef struct mop_desc_t mop_desc_t;
> +DEF_VEC_O (mop_desc_t);
> +DEF_VEC_ALLOC_O (mop_desc_t, heap);
> +static VEC (mop_desc_t, heap) *mop_list;
> +
> +/* The function is not available in some modules.  */
> +tree __attribute__((weak))
> +lookup_name (tree t)
> +{
> +  (void)t;
> +  return NULL_TREE;
> +}
> +
> +/* Builds the following decl
> +   extern __thread void **__tsan_shadow_stack; */
> +static tree
> +shadow_stack_def (void)
> +{
> +  static tree def;
> +
> +  if (def != NULL)
> +    return def;
> +
> +  /* Check if a user has defined it for testing */
> +  def = lookup_name (get_identifier (RTL_STACK));
> +  if (def != NULL)
> +    return def;
> +
> +  def = build_decl (UNKNOWN_LOCATION, VAR_DECL,
> +                   get_identifier (RTL_STACK),
> +                   build_pointer_type (ptr_type_node));
> +  TREE_STATIC (def) = 1;
> +  TREE_PUBLIC (def) = 1;
> +  DECL_EXTERNAL (def) = 1;
> +  DECL_TLS_MODEL (def) = decl_default_tls_model (def);
> +  TREE_USED (def) = 1;
> +  TREE_THIS_VOLATILE (def) = 1;
> +  SET_DECL_ASSEMBLER_NAME (def, get_identifier (RTL_STACK));
> +  return def;
> +}
> +
> +/* Builds the following decl
> +   extern __thread int __tsan_thread_ignore; */
> +static tree
> +thread_ignore_def (void)
> +{
> +  static tree def;
> +
> +  if (def != NULL)
> +    return def;
> +
> +  /* Check if a user has defined it for testing */
> +  def = lookup_name (get_identifier (RTL_IGNORE));
> +  if (def != NULL)
> +    return def;
> +
> +  def = build_decl (UNKNOWN_LOCATION, VAR_DECL,
> +                   get_identifier (RTL_IGNORE),
> +                   integer_type_node);
> +  TREE_STATIC (def) = 1;
> +  TREE_PUBLIC (def) = 1;
> +  DECL_EXTERNAL (def) = 1;
> +  DECL_TLS_MODEL (def) = decl_default_tls_model (def);
> +  TREE_USED (def) = 1;
> +  TREE_THIS_VOLATILE (def) = 1;
> +  SET_DECL_ASSEMBLER_NAME (def, get_identifier (RTL_IGNORE));
> +  return def;
> +}
> +
> +/* Builds the following decl
> +   void __tsan_handle_mop (void *addr, unsigned flags); */
> +static tree
> +rtl_mop_def (void)
> +{
> +  tree fn_type;
> +
> +  static tree def;
> +
> +  if (def != NULL)
> +    return def;
> +
> +  /* Check if a user has defined it for testing */
> +  def = lookup_name (get_identifier (RTL_MOP));
> +  if (def != NULL)
> +    return def;
> +
> +  fn_type = build_function_type_list (void_type_node, ptr_type_node,
> +                                      integer_type_node , NULL_TREE);
> +  def = build_fn_decl (RTL_MOP, fn_type);
> +  TREE_NOTHROW (def) = 1;
> +  DECL_ATTRIBUTES (def) = tree_cons (get_identifier ("leaf"),
> +                                     NULL, DECL_ATTRIBUTES (def));
> +  DECL_ASSEMBLER_NAME (def);
> +  return def;
> +}
> +
> +/* Adds new ignore definition to the global list */
> +static void
> +ignore_append (enum tsan_ignore_e type, char *name)
> +{
> +  struct tsan_ignore_desc_t *desc;
> +
> +  desc = (struct tsan_ignore_desc_t*)xmalloc (sizeof (*desc));
> +  desc->type = type;
> +  desc->name = xstrdup (name);
> +  desc->next = ignore_head;
> +  ignore_head = desc;
> +}
> +
> +/* Checks as to whether identifier 'str' matches template 'templ'.
> +   Templates can only contain '*', e.g. 'std*string*insert'.
> +   Templates implicitly start and end with '*'
> +   since they are matched against mangled names.  */
> +static int
> +ignore_match (char *templ, const char *str)
> +{
> +  char *tpos;
> +  const char *spos;
> +
> +  while (templ && templ [0])
> +    {
> +      if (templ [0] == '*')
> +        {
> +          templ++;
> +          continue;
> +        }
> +      if (str [0] == 0)
> +        return 0;
> +      tpos = strchr (templ, '*');
> +      if (tpos != NULL)
> +        tpos [0] = 0;
> +      spos = strstr (str, templ);
> +      str = spos + strlen (templ);
> +      templ = tpos;
> +      if (tpos != NULL)
> +        tpos [0] = '*';
> +      if (spos == NULL)
> +        return 0;
> +    }
> +  return 1;
> +}
> +
> +/* Loads ignore definitions from the file specified by -ftsan-ignore=filename.
> +   Ignore files have the following format:
> +
> +# This is a comment - ignored
> +
> +# The below line says to not instrument memory accesses
> +# in all functions that match 'std*string*insert'
> +fun:std*string*insert
> +
> +# The below line says to not instrument memory accesses
> +# in the function called 'foobar' *and* in all functions
> +# that it calls recursively
> +fun_r:foobar
> +
> +# The below line says to not create superblocks
> +# in the function called 'barbaz'
> +fun_hist:barbaz
> +
> +# Ignore all functions in the source file
> +src:atomic.c
> +
> +# Everything else is uninteresting for us (e.g. obj:)
> +*/
> +static void
> +ignore_load (void)
> +{
> +  FILE *f;
> +  char *line;
> +  size_t linesz;
> +  ssize_t sz;
> +  char buf [PATH_MAX];
> +
> +  if (flag_tsan_ignore == NULL || flag_tsan_ignore [0] == 0)
> +    return;
> +
> +  f = fopen (flag_tsan_ignore, "r");
> +  if (f == NULL)
> +    {
> +      /* Try to open it relative to main_input_filename.  */
> +      strncpy (buf, main_input_filename, sizeof (buf));
> +      buf [sizeof (buf) - 1] = 0;
> +      line = strrchr (buf, '/');
> +      if (line != NULL)
> +        {
> +          line++;
> +          strncpy (line, flag_tsan_ignore, sizeof (buf) - (line - buf));
> +          buf [sizeof (buf) - 1] = 0;
> +          f = fopen (buf, "r");
> +        }
> +    }
> +  if (f == NULL)
> +    {
> +      printf ("failed to open ignore file '%s'\n", flag_tsan_ignore);
> +      exit (1);
> +    }
> +
> +  line = 0;
> +  linesz = 0;
> +  while ((sz = getline (&line, &linesz, f)) != -1)
> +    {
> +      if (sz == 0)
> +        continue;
> +      /* strip line terminator */
> +      if (line [sz-1] == '\r' || line [sz-1] == '\n')
> +        line [sz-1] = 0;
> +      if (strncmp (line, "src:", sizeof ("src:")-1) == 0)
> +        ignore_append (tsan_ignore_func, line + sizeof ("src:")-1);
> +      else if (strncmp (line, "fun:", sizeof ("fun:")-1) == 0)
> +        ignore_append (tsan_ignore_mop, line + sizeof ("fun:")-1);
> +      else if (strncmp (line, "fun_r:", sizeof ("fun_r:")-1) == 0)
> +        ignore_append (tsan_ignore_rec, line + sizeof ("fun_r:")-1);
> +      else if (strncmp (line, "fun_hist:", sizeof ("fun_hist:")-1) == 0)
> +        ignore_append (tsan_ignore_hist, line + sizeof ("fun_hist:")-1);
> +      /* other lines are not interesting */
> +    }
> +
> +  free (line);
> +  fclose (f);
> +}
> +
> +/* Returns ignore status for the current function */
> +static enum tsan_ignore_e
> +tsan_ignore (void)
> +{
> +  const char *func_name;
> +  const char *src_name;
> +  struct tsan_ignore_desc_t *desc;
> +
> +  if (ignore_init == 0)
> +    {
> +      ignore_load ();
> +      ignore_init = 1;
> +    }
> +
> +  src_name = expand_location(cfun->function_start_locus).file;
> +  if (src_name == NULL)
> +    src_name = "";
> +
> +  func_name = IDENTIFIER_POINTER (DECL_ASSEMBLER_NAME (cfun->decl));
> +  /* Ignore all functions starting with __tsan_ - intended for testing */
> +  if (strncmp (func_name, RTL_PERFIX, sizeof (RTL_PERFIX) - 1) == 0)
> +    return tsan_ignore_func;
> +
> +  for (desc = ignore_head; desc; desc = desc->next)
> +    {
> +      if (desc->type == tsan_ignore_func)
> +        {
> +          if (ignore_match (desc->name, src_name))
> +           return desc->type;
> +        }
> +      else if (ignore_match (desc->name, func_name))
> +       return desc->type;
> +    }
> +  return tsan_ignore_none;
> +}
> +
> +static const char *
> +decl_name (tree decl)
> +{
> +  tree id;
> +  const char *name;
> +
> +  if (decl != 0 && DECL_P (decl))
> +    {
> +      id = DECL_NAME (decl);
> +      if (id != NULL)
> +        {
> +          name = IDENTIFIER_POINTER (id);
> +          if (name != NULL)
> +            return name;
> +        }
> +    }
> +  return "<unknown>";
> +}
> +
> +/* Builds either (__tsan_shadow_stack += 1) or (__tsan_shadow_stack -= 1) expression
> +   depending on 'do_dec' parameter.  Appends the result to seq.  */
> +static void
> +build_stack_op (gimple_seq *seq, bool do_dec)
> +{
> +  tree op_size;
> +  double_int op_size_cst;
> +  unsigned long long size_val;
> +  unsigned long long size_valhi;
> +  tree op_expr;
> +  tree assign;
> +  tree rtl_stack;
> +  gimple_seq s;
> +
> +  op_size = TYPE_SIZE (ptr_type_node);
> +  op_size_cst = tree_to_double_int (op_size);
> +  size_val = op_size_cst.low / BITS_PER_UNIT;
> +  size_valhi = 0;
> +  if (do_dec)
> +    {
> +      size_val = -size_val;
> +      size_valhi = -1;
> +    }
> +  op_size = build_int_cst_wide (sizetype, size_val, size_valhi);
> +  rtl_stack = shadow_stack_def ();
> +  op_expr = build2 (POINTER_PLUS_EXPR, ptr_type_node, rtl_stack, op_size);
> +  assign = build2 (MODIFY_EXPR, ptr_type_node, rtl_stack, op_expr);
> +  s = NULL;
> +  force_gimple_operand (assign, &s, true, NULL_TREE);
> +  gimple_seq_add_seq (seq, s);
> +}
> +
> +/* Builds either (__tsan_thread_ignore += 1) or (__tsan_thread_ignore -= 1) expression
> +   depending on op parameter.  Stores the result in seq.  */
> +static void
> +build_rec_ignore_op (gimple_seq *seq, enum tree_code op)
> +{
> +  tree rec_expr;
> +  gimple_seq rec_inc;
> +  gimple rec_assign;
> +  tree rtl_ignore;
> +
> +  rtl_ignore = thread_ignore_def ();
> +  rec_expr = build2 (op, integer_type_node, rtl_ignore, integer_one_node);
> +  rec_inc = NULL;
> +  rec_expr = force_gimple_operand (rec_expr, &rec_inc, true, NULL_TREE);
> +  rec_assign = gimple_build_assign (rtl_ignore, rec_expr);
> +  gimple_seq_add_seq (seq, rec_inc);
> +  gimple_seq_add_stmt (seq, rec_assign);
> +}
> +
> +/* Build the following gimple sequence:
> +   __tsan_shadow_stack [-1] = __builtin_return_address (0);
> +   Stores the result in seq.  */
> +static void
> +build_stack_assign (gimple_seq *seq)
> +{
> +  tree pc_addr;
> +  tree op_size;
> +  tree op_expr;
> +  tree stack_op;
> +  tree assign;
> +  tree rtl_retaddr;
> +
> +  rtl_retaddr = implicit_built_in_decls [BUILT_IN_RETURN_ADDRESS];
> +  pc_addr = build_call_expr (rtl_retaddr, 1, integer_zero_node);
> +  op_size = build_int_cst_wide (sizetype, -(POINTER_SIZE / BITS_PER_UNIT), -1);
> +  op_expr = build2 (POINTER_PLUS_EXPR, ptr_type_node,
> +                        shadow_stack_def (), op_size);
> +  stack_op = build1 (INDIRECT_REF, ptr_type_node, op_expr);
> +  assign = build2 (MODIFY_EXPR, ptr_type_node, stack_op, pc_addr);
> +  force_gimple_operand (assign, seq, true, NULL_TREE);
> +}
> +
> +/* Builds the following gimple sequence:
> +   __tsan_handle_mop (&expr, (is_sblock | (is_store << 1) | ((sizeof (expr)-1) << 2)
> +   The result is stored in gseq.  */
> +static void
> +instr_mop (tree expr, int is_store, int is_sblock, gimple_seq *gseq)
> +{
> +  tree addr_expr;
> +  tree expr_type;
> +  unsigned size;
> +  unsigned flags;
> +  tree flags_expr;
> +  tree call_expr;
> +
> +  gcc_assert (gseq != 0 && *gseq == 0);
> +  gcc_assert (is_gimple_addressable (expr));
> +
> +  addr_expr = build_addr (expr, current_function_decl);
> +  expr_type = TREE_TYPE (expr);
> +  while (TREE_CODE (expr_type) == ARRAY_TYPE)
> +    expr_type = TREE_TYPE (expr_type);
> +  size = TREE_INT_CST_LOW (TYPE_SIZE (expr_type));
> +  size = size / BITS_PER_UNIT;
> +  if (size > MAX_MOP_BYTES)
> +    size = MAX_MOP_BYTES;
> +  size -= 1;
> +  flags = ((!!is_sblock << 0) + (!!is_store << 1) + (size << 2));
> +  flags_expr = build_int_cst (unsigned_type_node, flags);
> +  call_expr = build_call_expr (rtl_mop_def (), 2, addr_expr, flags_expr);
> +  force_gimple_operand (call_expr, gseq, true, 0);
> +}
> +
> +/* Builds the following gimple sequence:
> +   int is_store = (expr != rhs); // the temp is not actually introduced
> +   __tsan_handle_mop (&expr, (is_sblock | (is_store << 1) | ((sizeof (expr)-1) << 2)
> +   The result is stored in gseq.  */
> +static void
> +instr_vptr_store (tree expr, tree rhs, int is_sblock, gimple_seq *gseq)
> +{
> +  tree expr_ptr;
> +  tree addr_expr;
> +  tree expr_type;
> +  tree expr_size;
> +  double_int size;
> +  unsigned flags;
> +  tree flags_expr;
> +  gimple_seq flags_seq;
> +  gimple collect;
> +  tree is_store_expr;
> +
> +  expr_ptr = build_addr (expr, current_function_decl);
> +  addr_expr = force_gimple_operand (expr_ptr, gseq, true, NULL_TREE);
> +  expr_type = TREE_TYPE (expr);
> +  while (TREE_CODE (expr_type) == ARRAY_TYPE)
> +    expr_type = TREE_TYPE (expr_type);
> +  expr_size = TYPE_SIZE (expr_type);
> +  size = tree_to_double_int (expr_size);
> +  gcc_assert (size.high == 0 && size.low != 0);
> +  if (size.low > 128)
> +    size.low = 128;
> +  size.low = (size.low / 8) - 1;
> +  flags = ((!!is_sblock << 0) + (size.low << 2));
> +  flags_expr = build_int_cst (unsigned_type_node, flags);
> +  is_store_expr = build2 (NE_EXPR, integer_type_node,
> +                              build1 (VIEW_CONVERT_EXPR, size_type_node, expr),
> +                              build1 (VIEW_CONVERT_EXPR, size_type_node, rhs));
> +  is_store_expr = build2 (LSHIFT_EXPR, integer_type_node,
> +                              is_store_expr, integer_one_node);
> +  flags_expr = build2 (BIT_IOR_EXPR, integer_type_node,
> +                              is_store_expr, flags_expr);
> +  flags_seq = 0;
> +  flags_expr = force_gimple_operand (flags_expr, &flags_seq, true, NULL_TREE);
> +  gimple_seq_add_seq (gseq, flags_seq);
> +  collect = gimple_build_call (
> +      rtl_mop_def (), 2, addr_expr, flags_expr);
> +  gimple_seq_add_stmt (gseq, collect);
> +}
> +
> +/* Builds gimple sequences that must be inserted at function entry (pre)
> +   and before function exit (post).  */
> +static void
> +instr_func (gimple_seq *pre, gimple_seq *post)
> +{
> +  /* In this case we need no instrumentation for the function */
> +  if (func_calls == 0 && func_mops == 0)
> +    return;
> +
> +  if (func_ignore != tsan_ignore_rec)
> +    {
> +      build_stack_assign (pre);
> +      build_stack_op (pre, false);
> +      build_stack_op (post, true);
> +    }
> +
> +  if (func_ignore == tsan_ignore_rec && func_calls != 0)
> +    {
> +      build_rec_ignore_op (pre, PLUS_EXPR);
> +      build_rec_ignore_op (post, MINUS_EXPR);
> +    }
> +}
> +
> +/* Sets location for all gimples in the seq.  */
> +static void
> +set_location (gimple_seq seq, location_t loc)
> +{
> +  gimple_seq_node n;
> +
> +  for (n = gimple_seq_first (seq); n != NULL; n = n->next)
> +    gimple_set_location (n->stmt, loc);
> +}
> +
> +/* Check as to whether expr refers to a store to vptr.  */
> +static tree
> +is_dtor_vptr_store (gimple stmt, tree expr, int is_store)
> +{
> +  if (is_store == 1
> +      && TREE_CODE (expr) == COMPONENT_REF
> +      && gimple_assign_single_p (stmt)
> +      && strcmp (decl_name (cfun->decl), "__base_dtor ") == 0)
> +    {
> +      tree comp = expr->exp.operands [0];
> +      while (TREE_CODE (comp) == COMPONENT_REF)
> +        comp = comp->exp.operands [0];
> +      if (TREE_CODE (comp) == INDIRECT_REF || TREE_CODE (comp) == MEM_REF)
> +        {
> +          comp = comp->exp.operands [0];
> +          if (TREE_CODE (comp) == SSA_NAME)
> +            comp = SSA_NAME_VAR (comp);
> +          if (strcmp (decl_name (comp), "this") == 0)
> +            {
> +              tree field = expr->exp.operands [1];
> +              if (TREE_CODE (field) == FIELD_DECL
> +                  && strncmp (decl_name (field),
> +                              "_vptr.", sizeof ("_vptr.") - 1) == 0)
> +                return gimple_assign_rhs1 (stmt);
> +            }
> +        }
> +    }
> +  return 0;
> +}
> +
> +/* Checks as to whether expr refers to a read from vtlb.
> +   Vtlbs are immutable, so don't bother to instrument them.  */
> +static int
> +is_vtbl_read (tree expr, int is_store)
> +{
> +  /* We may not instrument reads from vtbl, because the data is constant.
> +     vtbl read is of the form:
> +       gimple_assign <component_ref, D.2133, x->_vptr.X, NULL>
> +       gimple_assign <indirect_ref, D.2134, *D.2133, NULL>
> +     or:
> +       gimple_assign <component_ref, D.2133, x->_vptr.X, NULL>
> +       gimple_assign <pointer_plus_expr, D.2135, D.2133, 8>
> +       gimple_assign <indirect_ref, D.2136, *D.2135, NULL> */
> +
> +  if (is_store == 0
> +      && TREE_CODE (expr) == INDIRECT_REF)
> +    {
> +      tree ref_target = expr->exp.operands [0];
> +      if (TREE_CODE (ref_target) == SSA_NAME)
> +        {
> +          gimple ref_stmt = ref_target->ssa_name.def_stmt;
> +          if (gimple_code (ref_stmt) == GIMPLE_ASSIGN)
> +            {
> +              if (gimple_expr_code (ref_stmt) == POINTER_PLUS_EXPR)
> +                {
> +                  tree tmp = ref_stmt->gsmem.op [1];
> +                  if (TREE_CODE (tmp) == SSA_NAME
> +                      && gimple_code (tmp->ssa_name.def_stmt) == GIMPLE_ASSIGN)
> +                    ref_stmt = tmp->ssa_name.def_stmt;
> +                }
> +              if (gimple_expr_code (ref_stmt) == COMPONENT_REF
> +                    && gimple_assign_single_p (ref_stmt))
> +                {
> +                  tree comp_expr = ref_stmt->gsmem.op [1];
> +                  tree field_expr = comp_expr->exp.operands [1];
> +                  if (TREE_CODE (field_expr) == FIELD_DECL
> +                      && strncmp (decl_name (field_expr),
> +                                  "_vptr.", sizeof ("_vptr.") - 1) == 0)
> +                    return 1;
> +                }
> +            }
> +        }
> +    }
> +
> +  return 0;
> +}
> +
> +/* Checks as to whether expr refers to constant var/field/param.
> +   Don't bother to instrument them.  */
> +static int
> +is_load_of_const (tree expr, int is_store)
> +{
> +  if (is_store == 0)
> +    {
> +      if (TREE_CODE (expr) == COMPONENT_REF)
> +        expr = expr->exp.operands [1];
> +      if (TREE_CODE (expr) == VAR_DECL
> +          || TREE_CODE (expr) == PARM_DECL
> +          || TREE_CODE (expr) == FIELD_DECL)
> +        {
> +          if (TREE_READONLY (expr))
> +            return 1;
> +        }
> +    }
> +  return 0;
> +}
> +
> +static void
> +handle_expr (gimple stmt, gimple_stmt_iterator gsi,
> +             tree expr, int is_store, VEC (mop_desc_t, heap) **mop_list)
> +{
> +  enum tree_code tcode;
> +  struct mop_desc_t mop;
> +  unsigned fld_off;
> +  unsigned fld_size;
> +
> +  /* map SSA name to real name */
> +  if (TREE_CODE (expr) == SSA_NAME)
> +    expr = SSA_NAME_VAR (expr);
> +
> +  tcode = TREE_CODE (expr);
> +
> +  /* Below are things we do NOT want to instrument.  */
> +  if (func_ignore & (tsan_ignore_mop | tsan_ignore_rec))
> +    {
> +      return;
> +    }
> +  else if (TREE_CODE_CLASS (tcode) == tcc_constant)
> +    {
> +      /* various constant literals */
> +      return;
> +    }
> +  else if (TREE_CODE_CLASS (tcode) == tcc_declaration
> +      && DECL_ARTIFICIAL (expr))
> +    {
> +      /* compiler-emitted artificial variables */
> +      return;
> +    }
> +  if (tcode == RESULT_DECL)
> +    {
> +      /* store to function result */
> +      return;
> +    }
> +  else if (tcode == VAR_DECL
> +      && TREE_ADDRESSABLE (expr) == 0
> +      && TREE_STATIC (expr) == 0)
> +    {
> +      /* the var does not live in memory -> no possibility of races */
> +      return;
> +    }
> +  else if (TREE_CODE (TREE_TYPE (expr)) == RECORD_TYPE)
> +    {
> +      /* TODO (dvyukov): implement me */
> +      return;
> +    }
> +  else if (tcode == CONSTRUCTOR)
> +    {
> +      /* TODO (dvyukov): implement me */
> +      return;
> +    }
> +  else if (tcode == PARM_DECL)
> +    {
> +      /* TODO (dvyukov): implement me */
> +      return;
> +    }
> +  else if (is_load_of_const (expr, is_store))
> +    {
> +      /* load of a const variable/parameter/field */
> +      return;
> +    }
> +  else if (is_vtbl_read (expr, is_store))
> +    {
> +      /* vtbl read */
> +      return;
> +    }
> +  else if (tcode == COMPONENT_REF)
> +    {
> +      tree field = expr->exp.operands [1];
> +      if (TREE_CODE (field) == FIELD_DECL)
> +        {
> +          fld_off = field->field_decl.bit_offset->int_cst.int_cst.low;
> +          fld_size = field->decl_common.size->int_cst.int_cst.low;
> +          if (((fld_off % BITS_PER_UNIT) != 0)
> +              || ((fld_size % BITS_PER_UNIT) != 0))
> +            {
> +              /* As of now it crashes compilation.
> +                 TODO (dvyukov): handle bit-fields as if touching the whole field */
> +              return;
> +            }
> +        }
> +    }
> +
> +  /* TODO (dvyukov): handle other cases
> +     (FIELD_DECL, MEM_REF, ARRAY_RANGE_REF, TARGET_MEM_REF, ADDR_EXPR) */
> +  if (tcode != ARRAY_REF
> +      && tcode != VAR_DECL
> +      && tcode != COMPONENT_REF
> +      && tcode != INDIRECT_REF
> +      && tcode != MEM_REF)
> +    return;
> +
> +  mop.is_call = 0;
> +  mop.gsi = gsi;
> +  mop.expr = expr;
> +  mop.dtor_vptr_expr = is_dtor_vptr_store (stmt, expr, is_store);
> +  mop.is_store = is_store;
> +  VEC_safe_push (mop_desc_t, heap, *mop_list, &mop);
> +}
> +
> +static void
> +handle_gimple (gimple_stmt_iterator gsi, VEC (mop_desc_t, heap) **mop_list)
> +{
> +  unsigned i;
> +  struct mop_desc_t mop;
> +  gimple stmt;
> +  enum gimple_code gcode;
> +  location_t loc;
> +  tree rhs;
> +  tree lhs;
> +
> +  stmt = gsi_stmt (gsi);
> +  gcode = gimple_code (stmt);
> +  if (gcode >= LAST_AND_UNUSED_GIMPLE_CODE)
> +    return;
> +
> +  loc = gimple_location (stmt);
> +
> +  switch (gcode)
> +    {
> +      /* TODO (dvyukov): handle GIMPLE_COND (can it access memmory?) */
> +      case GIMPLE_CALL:
> +        {
> +          func_calls += 1;
> +          /* Handle call arguments as loads */
> +          for (i = 0; i < gimple_call_num_args (stmt); i++)
> +            {
> +              rhs = gimple_call_arg (stmt, i);
> +              handle_expr (stmt, gsi, rhs, 0, mop_list);
> +            }
> +
> +          memset (&mop, 0, sizeof (mop));
> +          mop.is_call = 1;
> +          VEC_safe_push (mop_desc_t, heap, *mop_list, &mop);
> +
> +          /* Handle assignment lhs as store */
> +          lhs = gimple_call_lhs (stmt);
> +          if (lhs != 0)
> +            handle_expr (stmt, gsi, lhs, 1, mop_list);
> +
> +          break;
> +        }
> +
> +      case GIMPLE_ASSIGN:
> +        {
> +          /* Handle assignment lhs as store */
> +          lhs = gimple_assign_lhs (stmt);
> +          handle_expr (stmt, gsi, lhs, 1, mop_list);
> +
> +          /* Handle operands as loads */
> +          for (i = 1; i < gimple_num_ops (stmt); i++)
> +            {
> +              rhs = gimple_op (stmt, i);
> +              handle_expr (stmt, gsi, rhs, 0, mop_list);
> +            }
> +          break;
> +        }
> +
> +      case GIMPLE_BIND:
> +        {
> +          gcc_assert (!"there should be no GIMPLE_BIND on this level");
> +          break;
> +        }
> +
> +      default:
> +        break;
> +    }
> +}
> +
> +/* Instruments single basic block.  */
> +static void
> +instrument_bblock (struct bb_data_t *bbd, basic_block bb)
> +{
> +  int ix;
> +  int is_sblock;
> +  gimple_stmt_iterator gsi;
> +  struct mop_desc_t *mop;
> +  gimple stmt;
> +  location_t loc;
> +  expanded_location eloc;
> +  gimple_seq instr_seq;
> +
> +  /* Iterate over all gimples and collect interesting mops into mop_list.  */
> +  VEC_free (mop_desc_t, heap, mop_list);
> +  for (gsi = gsi_start_bb (bb); !gsi_end_p (gsi); gsi_next (&gsi))
> +    {
> +      handle_gimple (gsi, &mop_list);
> +    }
> +
> +  mop = 0;
> +  for (ix = 0; VEC_iterate (mop_desc_t, mop_list, ix, mop); ix += 1)
> +    {
> +      if (mop->is_call != 0)
> +        {
> +          /* After a function call we must start a brand new sblock,
> +             because the function can contain synchronization.  */
> +          bbd->has_sb = 0;
> +          continue;
> +        }
> +
> +      func_mops += 1;
> +      stmt = gsi_stmt (mop->gsi);
> +      loc = gimple_location (stmt);
> +      eloc = expand_location (loc);
> +
> +      /* Check as to whether we may not set sblock flag
> +         for the access */
> +      is_sblock = (bbd->has_sb == 0
> +          || !(eloc.file != 0
> +              && bbd->sb_file != 0
> +              && strcmp (eloc.file, bbd->sb_file) == 0
> +              && eloc.line >= bbd->sb_line_min
> +              && eloc.line <= bbd->sb_line_max));
> +
> +      if (func_ignore == tsan_ignore_hist)
> +        is_sblock = 0;
> +
> +      if (is_sblock)
> +        {
> +          /* Start new sblock with new source info.  */
> +          bbd->has_sb = 1;
> +          bbd->sb_file = eloc.file;
> +          bbd->sb_line_min = eloc.line;
> +          bbd->sb_line_max = eloc.line + SBLOCK_SIZE;
> +        }
> +
> +      instr_seq = 0;
> +      if (mop->dtor_vptr_expr == 0)
> +        instr_mop (mop->expr, mop->is_store, is_sblock, &instr_seq);
> +      else
> +        instr_vptr_store (mop->expr, mop->dtor_vptr_expr, is_sblock, &instr_seq);
> +      gcc_assert (instr_seq != 0);
> +      set_location (instr_seq, loc);
> +      /* Instrumentation for assignment of a function result
> +         must be inserted after the call.  Instrumentation for
> +         reads of function arguments must be inserted before the call.
> +         That's because the call can contain synchronization.  */
> +      if (is_gimple_call (stmt) && mop->is_store == 1)
> +        gsi_insert_seq_after (&mop->gsi, instr_seq, GSI_NEW_STMT);
> +      else
> +        gsi_insert_seq_before (&mop->gsi, instr_seq, GSI_SAME_STMT);
> +    }
> +}
> +
> +/* Instruments all interesting memory accesses in the function */
> +static void
> +instrument_mops (void)
> +{
> +  int sb_line_min;
> +  int sb_line_max;
> +  int bb_cnt;
> +  int eidx;
> +  basic_block bb;
> +  basic_block entry_bb;
> +  basic_block cur_bb;
> +  basic_block any_bb;
> +  struct bb_data_t *pred;
> +  struct bb_data_t *succ;
> +  struct bb_data_t *bb_data;
> +  struct bb_data_t *bbd;
> +  edge entry_edge;
> +  edge e;
> +
> +  /* The function does breadth-first traversal of CFG.
> +     BB is visited preferably if all its predecessors are visited.
> +     Such order is required to properly mark super-blocks.
> +     The idea behind super-blocks is as follows.
> +     If several memory accesses happen within SBLOCK_SIZE source code lines
> +     from each other, then we only mark the first access as SBLOCK.
> +     This allows runtime library to memorize stack trace
> +     only for the first access and do not memorize for others.
> +     This significantly reduces memory consumption in exchange for slightly
> +     imprecise stack traces for previous accesses.  */
> +
> +  /* First, mark all blocks as not visited, and entry block as candidate.  */
> +  bb_cnt = cfun->cfg->x_n_basic_blocks;
> +  bb_data = (struct bb_data_t*) xcalloc (bb_cnt, sizeof (struct bb_data_t));
> +  entry_bb = ENTRY_BLOCK_PTR;
> +  entry_edge = single_succ_edge (entry_bb);
> +  entry_bb = entry_edge->dest;
> +  bb = 0;
> +  FOR_EACH_BB (bb)
> +    {
> +      bb_data [bb->index].state = (bb == entry_bb) ? bb_candidate : bb_not_visited;
> +    }
> +
> +  /* Until all blocks are visited.  */
> +  for (; ; )
> +    {
> +      cur_bb = 0;
> +      any_bb = 0;
> +      /* Look for a candidate with all visited predecessors.  */
> +      FOR_EACH_BB (bb)
> +        {
> +          bbd = &bb_data [bb->index];
> +          if (bbd->state == bb_candidate)
> +            {
> +              cur_bb = bb;
> +              any_bb = bb;
> +              e = 0;
> +              for (eidx = 0; VEC_iterate (edge, bb->preds, eidx, e); eidx++)
> +                {
> +                  pred = &bb_data [e->src->index];
> +                  if (pred->state != bb_visited)
> +                    {
> +                      cur_bb = 0;
> +                      break;
> +                    }
> +                }
> +            }
> +          if (cur_bb != 0)
> +            break;
> +        }
> +      /* All blocks are visited.  */
> +      if (any_bb == 0)
> +        break;
> +      /* If no blocks with all visited predecessors, choose any candidate.
> +         Must be a loop.  */
> +      cur_bb = cur_bb ? cur_bb : any_bb;
> +      bbd = &bb_data [cur_bb->index];
> +      gcc_assert (bbd->state == bb_candidate);
> +      bbd->state = bb_visited;
> +
> +      /* Iterate over all predecessors and merge their sblock info.  */
> +      e = 0;
> +      for (eidx = 0; VEC_iterate (edge, cur_bb->preds, eidx, e); eidx++)
> +        {
> +          pred = &bb_data [e->src->index];
> +          if ((pred->state != bb_visited)
> +              || (pred->has_sb == 0)
> +              || (pred == bbd))
> +            {
> +              /* If there is a not visited predecessor,
> +                 or a predecessor with no active sblock info,
> +                 or a self-loop, then we will have to start
> +                 a brand new sblock on next memory access.  */
> +              bbd->has_sb = 0;
> +              break;
> +            }
> +          else if (bbd->has_sb == 0)
> +            {
> +              /* If it's a first predecessor, just copy the info.  */
> +              bbd->has_sb = 1;
> +              bbd->sb_file = pred->sb_file;
> +              bbd->sb_line_min = pred->sb_line_min;
> +              bbd->sb_line_max = pred->sb_line_max;
> +            }
> +          else
> +            {
> +              /* Otherwise, find the interception
> +                 between two sblock descriptors.  */
> +              bbd->has_sb = 0;
> +              if (bbd->sb_file != 0 && pred->sb_file != 0
> +                  && strcmp (bbd->sb_file, pred->sb_file) == 0)
> +                {
> +                  sb_line_min = MAX (bbd->sb_line_min, pred->sb_line_min);
> +                  sb_line_max = MIN (bbd->sb_line_max, pred->sb_line_max);
> +                  if (sb_line_min <= sb_line_max)
> +                    {
> +                      bbd->has_sb = 1;
> +                      bbd->sb_line_min = sb_line_min;
> +                      bbd->sb_line_max = sb_line_max;
> +                    }
> +                }
> +              /* No interception, have to start new sblock.  */
> +              if (bbd->has_sb == 0)
> +                break;
> +            }
> +        }
> +
> +      /* Finally, instrument the block.  */
> +      instrument_bblock (bbd, cur_bb);
> +
> +      /* Mark all successors as candidates.  */
> +      for (eidx = 0; VEC_iterate (edge, cur_bb->succs, eidx, e); eidx++)
> +        {
> +          succ = &bb_data [e->dest->index];
> +          if (succ->state == bb_not_visited)
> +            succ->state = bb_candidate;
> +        }
> +    }
> +}
> +
> +/* Instruments function entry and exit, if necessary.  */
> +static void
> +instrument_function (void)
> +{
> +  location_t loc;
> +  gimple_seq pre_func_seq;
> +  gimple_seq post_func_seq;
> +  basic_block entry_bb;
> +  basic_block first_bb;
> +  basic_block bb;
> +  edge entry_edge;
> +  gimple_stmt_iterator first_gsi;
> +  gimple_stmt_iterator gsi;
> +  gimple_stmt_iterator gsi2;
> +  gimple first_stmt;
> +  gimple stmt;
> +
> +  pre_func_seq = 0;
> +  post_func_seq = 0;
> +  instr_func (&pre_func_seq, &post_func_seq);
> +
> +  if (pre_func_seq != 0)
> +    {
> +      /* Insert new BB before the first BB.  */
> +      entry_bb = ENTRY_BLOCK_PTR;
> +      entry_edge = single_succ_edge (entry_bb);
> +      first_bb = entry_edge->dest;
> +      first_gsi = gsi_start_bb (first_bb);
> +      if (!gsi_end_p (first_gsi))
> +        {
> +          first_stmt = gsi_stmt (first_gsi);
> +          loc = gimple_location (first_stmt);
> +          set_location (pre_func_seq, loc);
> +        }
> +      entry_bb = split_edge (entry_edge);
> +      gsi = gsi_start_bb (entry_bb);
> +      gsi_insert_seq_after (&gsi, pre_func_seq, GSI_NEW_STMT);
> +    }
> +
> +  if (post_func_seq != 0)
> +    {
> +      /* Find all function exits.  */
> +      FOR_EACH_BB (bb)
> +        {
> +          gsi2 = gsi_start_bb (bb);
> +          for (; ; )
> +            {
> +              gsi = gsi2;
> +              if (gsi_end_p (gsi))
> +                break;
> +              gsi_next (&gsi2);
> +
> +              stmt = gsi_stmt (gsi);
> +              loc = gimple_location (stmt);
> +
> +              if (gimple_code (stmt) == GIMPLE_RETURN)
> +                {
> +                  set_location (post_func_seq, loc);
> +                  gsi_insert_seq_before (&gsi, post_func_seq, GSI_SAME_STMT);
> +                }
> +            }
> +        }
> +    }
> +}
> +
> +static unsigned
> +tsan_pass (void)
> +{
> +  if (errorcount != 0 || sorrycount != 0)
> +    return 0;
> +
> +  func_ignore = tsan_ignore ();
> +  if (func_ignore == tsan_ignore_func)
> +    return 0;
> +
> +  func_calls = 0;
> +  func_mops = 0;
> +
> +  instrument_mops ();
> +  instrument_function ();
> +
> +  return 0;
> +}
> +
> +static bool
> +tsan_gate (void)
> +{
> +  return flag_tsan != 0;
> +}
> +
> +struct gimple_opt_pass pass_tsan = {{
> +  GIMPLE_PASS,
> +  "tsan",                               /* name */
> +  tsan_gate,                            /* gate */
> +  tsan_pass,                            /* execute */
> +  NULL,                                 /* sub */
> +  NULL,                                 /* next */
> +  0,                                    /* static_pass_number */
> +  TV_NONE,                              /* tv_id */
> +  PROP_trees | PROP_cfg,                /* properties_required */
> +  0,                                    /* properties_provided */
> +  0,                                    /* properties_destroyed */
> +  0,                                    /* todo_flags_start */
> +  TODO_dump_cgraph | TODO_dump_func | TODO_verify_all
> +    | TODO_update_ssa | TODO_update_address_taken /* todo_flags_finish */
> +}};
> +
>
> Property changes on: gcc/tree-tsan.c
> ___________________________________________________________________
> Added: svn:eol-style
>   + LF
>
> Index: gcc/tree-pass.h
> ===================================================================
> --- gcc/tree-pass.h     (revision 180522)
> +++ gcc/tree-pass.h     (working copy)
> @@ -352,6 +352,7 @@
>
>  extern struct gimple_opt_pass pass_mudflap_1;
>  extern struct gimple_opt_pass pass_mudflap_2;
> +extern struct gimple_opt_pass pass_tsan;
>  extern struct gimple_opt_pass pass_lower_cf;
>  extern struct gimple_opt_pass pass_refactor_eh;
>  extern struct gimple_opt_pass pass_lower_eh;
> Index: gcc/testsuite/gcc.dg/tsan-ignore.ignore
> ===================================================================
> --- gcc/testsuite/gcc.dg/tsan-ignore.ignore     (revision 0)
> +++ gcc/testsuite/gcc.dg/tsan-ignore.ignore     (revision 0)
> @@ -0,0 +1,7 @@
> +#comment
> +fun:foo
> +fun:*bar
> +fun:baz*
> +fun:*bla*
> +fun:x*x
> +src:tsan-ignore.h
> Index: gcc/testsuite/gcc.dg/tsan.h
> ===================================================================
> --- gcc/testsuite/gcc.dg/tsan.h (revision 0)
> +++ gcc/testsuite/gcc.dg/tsan.h (revision 0)
> @@ -0,0 +1,87 @@
> +/* Helper declarations and functions for ThreadSanitizer instrumentation (-ftsan) testing */
> +
> +int printf (char *str, ...);
> +void exit (int);
> +
> +/* Variables referenced by the instrumentation */
> +__thread void **__tsan_shadow_stack;
> +__thread int __tsan_thread_ignore;
> +
> +/* Local helper vars */
> +__thread void *shadow_stack[1024];
> +__thread int mop_expect;
> +__thread int mop_depth;
> +__thread void* mop_addr;
> +__thread unsigned long long mop_pc;
> +__thread unsigned mop_flags;
> +
> +/* Setups shadow stack var (not instrumented) */
> +void __attribute__ ((constructor))
> +__tsan_init (void)
> +{
> +  __tsan_shadow_stack = shadow_stack;
> +}
> +
> +/* Declare that we expect an instrumented memory access (not instrumented).
> +   depth - stack depth of the mop (0 - main, 1 - func called from main and so on).
> +   addr - memory access address.
> +   is_store - store/load.
> +   is_sblock - superblock flag of the access.
> +   size - memory access size in bytes. */
> +void
> +__tsan_expect_mop (int depth, void *addr, int is_store, int is_sblock, int size)
> +{
> +  if (mop_expect)
> +    {
> +      printf ("missed mop: addr=%p pc=%d\n", mop_addr, mop_pc);
> +      exit (1);
> +    }
> +
> +  mop_expect = 1;
> +  mop_depth = depth;
> +  mop_addr = addr;
> +  mop_pc = (unsigned long long)__builtin_return_address(0);
> +  mop_flags = !!is_sblock | (!!is_store << 1) | ((size - 1) << 2);
> +}
> +
> +/* Memory access function (referenced by instrumentation, not instrumented). */
> +void
> +__tsan_handle_mop (void *addr, unsigned flags)
> +{
> +  unsigned long long pc;
> +  int depth;
> +
> +  printf ("mop: addr=%p flags=%x called from %p/%p\n", addr, flags, __builtin_return_address(1), __tsan_shadow_stack[-2]);
> +  if (mop_expect == 0)
> +    return;
> +
> +  /* Verify parameters with what we expect. */
> +
> +  if (addr != mop_addr)
> +    {
> +      printf ("incorrect mop addr: %p/%p\n", addr,  mop_addr);
> +      exit (1);
> +    }
> +
> +  pc = (unsigned long long)__builtin_return_address(0);
> +  if (pc < mop_pc - 100 || pc > mop_pc + 100)
> +    {
> +      printf ("incorrect mop pc: %p/%p\n", (void*)pc, (void*)mop_pc);
> +      exit (1);
> +    }
> +
> +  depth = __tsan_shadow_stack - shadow_stack - 1;
> +  if (depth != mop_depth)
> +    {
> +      printf ("incorrect mop depth: %d/%d\n", depth, mop_depth);
> +      exit (1);
> +    }
> +
> +  if (flags != mop_flags)
> +    {
> +      printf ("incorrect mop flags: %x/%x\n", flags, mop_flags);
> +      exit (1);
> +    }
> +
> +  mop_expect = 0;
> +}
>
> Property changes on: gcc/testsuite/gcc.dg/tsan.h
> ___________________________________________________________________
> Added: svn:eol-style
>   + LF
>
> Index: gcc/testsuite/gcc.dg/tsan-ignore.c
> ===================================================================
> --- gcc/testsuite/gcc.dg/tsan-ignore.c  (revision 0)
> +++ gcc/testsuite/gcc.dg/tsan-ignore.c  (revision 0)
> @@ -0,0 +1,49 @@
> +/* { dg-do run } */
> +/* { dg-options "-ftsan -ftsan-ignore=tsan-ignore.ignore" } */
> +#include "tsan.h"
> +#include "tsan-ignore.h"
> +
> +/* Check ignore file handling. */
> +
> +int
> +foo (int *p)
> +{
> +  p [0] = 1;
> +}
> +
> +int bar (int *p)
> +{
> +  p [0] = 1;
> +}
> +
> +int baz (int *p)
> +{
> +  p [0] = 1;
> +}
> +
> +int bla (int *p)
> +{
> +  p [0] = 1;
> +}
> +
> +int xxx (int *p)
> +{
> +  p [0] = 1;
> +}
> +
> +int
> +main (void)
> +{
> +  int p, x;
> +
> +  __tsan_expect_mop(0, &p, 1, 1, sizeof(p));
> +  /* All these functions must be ignored. */
> +  foo (&x);
> +  bar (&x);
> +  baz (&x);
> +  bla (&x);
> +  xxx (&x);
> +  in_tsan_ignore_header (&x);
> +  p = 0;
> +  return 0;
> +}
>
> Property changes on: gcc/testsuite/gcc.dg/tsan-ignore.c
> ___________________________________________________________________
> Added: svn:eol-style
>   + LF
>
> Index: gcc/testsuite/gcc.dg/tsan-ignore.h
> ===================================================================
> --- gcc/testsuite/gcc.dg/tsan-ignore.h  (revision 0)
> +++ gcc/testsuite/gcc.dg/tsan-ignore.h  (revision 0)
> @@ -0,0 +1,5 @@
> +int
> +in_tsan_ignore_header (int *p)
> +{
> +  p [0] = 1;
> +}
>
> Property changes on: gcc/testsuite/gcc.dg/tsan-ignore.h
> ___________________________________________________________________
> Added: svn:eol-style
>   + LF
>
> Index: gcc/testsuite/gcc.dg/tsan-stack.c
> ===================================================================
> --- gcc/testsuite/gcc.dg/tsan-stack.c   (revision 0)
> +++ gcc/testsuite/gcc.dg/tsan-stack.c   (revision 0)
> @@ -0,0 +1,23 @@
> +/* { dg-do run } */
> +/* { dg-options "-ftsan" } */
> +#include "tsan.h"
> +
> +/* Check shadow stack maintance. */
> +
> +int
> +foobar (int *p)
> +{
> +  __tsan_expect_mop(1, p, 1, 1, sizeof(*p));
> +  p[0] = 1;
> +}
> +
> +int
> +main (void)
> +{
> +  int p;
> +
> +  __tsan_expect_mop(0, &p, 1, 1, sizeof(p));
> +  p = 0;
> +  foobar (&p);
> +  return 0;
> +}
>
> Property changes on: gcc/testsuite/gcc.dg/tsan-stack.c
> ___________________________________________________________________
> Added: svn:eol-style
>   + LF
>
> Index: gcc/testsuite/gcc.dg/tsan-mop.c
> ===================================================================
> --- gcc/testsuite/gcc.dg/tsan-mop.c     (revision 0)
> +++ gcc/testsuite/gcc.dg/tsan-mop.c     (revision 0)
> @@ -0,0 +1,29 @@
> +/* { dg-do run } */
> +/* { dg-options "-ftsan" } */
> +#include "tsan.h"
> +
> +/* Sanity check for memory accesses instrumentation. */
> +
> +int
> +foobar (int *p)
> +{
> +  __tsan_expect_mop(1, p, 1, 1, sizeof(*p));
> +  p[0] = 1;
> +
> +  __tsan_expect_mop(1, p, 1, 1, sizeof(*p));
> +  *p = 2;
> +
> +  __tsan_expect_mop(1, (char*)p+3, 1, 1, 1);
> +  *((char*)p+3) = 3;
> +}
> +
> +int
> +main (void)
> +{
> +  int p;
> +
> +  __tsan_expect_mop(0, &p, 1, 1, sizeof(p));
> +  p = 0;
> +  foobar (&p);
> +  return 0;
> +}
>
> Property changes on: gcc/testsuite/gcc.dg/tsan-mop.c
> ___________________________________________________________________
> Added: svn:eol-style
>   + LF
>
> Index: gcc/common.opt
> ===================================================================
> --- gcc/common.opt      (revision 180522)
> +++ gcc/common.opt      (working copy)
> @@ -1547,6 +1547,14 @@
>  Common RejectNegative Report Var(flag_mudflap_ignore_reads)
>  Ignore read operations when inserting mudflap instrumentation
>
> +ftsan
> +Common RejectNegative Report Var(flag_tsan)
> +Add ThreadSanitizer instrumentation
> +
> +ftsan-ignore=
> +Common RejectNegative Joined Var(flag_tsan_ignore)
> +-ftsan-ignore=filename ThreadSanitizer ignore file
> +
>  fdce
>  Common Var(flag_dce) Init(1) Optimization
>  Use the RTL dead code elimination pass
> Index: gcc/Makefile.in
> ===================================================================
> --- gcc/Makefile.in     (revision 180522)
> +++ gcc/Makefile.in     (working copy)
> @@ -1494,6 +1494,7 @@
>        tree-streamer-out.o \
>        tree-tailcall.o \
>        tree-threadsafe-analyze.o \
> +       tree-tsan.o \
>        tree-vect-generic.o \
>        tree-vect-patterns.o \
>        tree-vect-data-refs.o \
> @@ -2814,6 +2815,12 @@
>    $(C_TREE_H) $(C_COMMON_H) $(GIMPLE_H) $(DIAGNOSTIC_H) $(HASHTAB_H) \
>    output.h langhooks.h tree-mudflap.h $(TM_H) coretypes.h \
>    $(GGC_H) gt-tree-mudflap.h $(TREE_PASS_H) $(DIAGNOSTIC_CORE_H)
> +tree-tsan.o : $(CONFIG_H) $(SYSTEM_H) $(TREE_H) $(TREE_INLINE_H) \
> +   $(GIMPLE_H) $(DIAGNOSTIC_H) langhooks.h \
> +   $(TM_H) coretypes.h $(TREE_DUMP_H) $(TREE_PASS_H) $(CGRAPH_H) $(GGC_H) \
> +   $(BASIC_BLOCK_H) $(FLAGS_H) $(FUNCTION_H) \
> +   $(TM_P_H) $(TREE_FLOW_H) $(DIAGNOSTIC_CORE_H) $(GIMPLE_H) tree-iterator.h \
> +   intl.h cfghooks.h output.h options.h c-family/c-common.h
>  tree-pretty-print.o : tree-pretty-print.c $(CONFIG_H) $(SYSTEM_H) \
>    $(TREE_H) $(DIAGNOSTIC_H) $(HASHTAB_H) $(TREE_FLOW_H) \
>    $(TM_H) coretypes.h tree-iterator.h $(SCEV_H) langhooks.h \
> Index: gcc/passes.c
> ===================================================================
> --- gcc/passes.c        (revision 180522)
> +++ gcc/passes.c        (working copy)
> @@ -1420,6 +1420,7 @@
>   NEXT_PASS (pass_lower_resx);
>   NEXT_PASS (pass_nrv);
>   NEXT_PASS (pass_mudflap_2);
> +  NEXT_PASS (pass_tsan);
>   NEXT_PASS (pass_cleanup_cfg_post_optimizing);
>   NEXT_PASS (pass_warn_function_noreturn);
>
>
> --
> This patch is available for review at http://codereview.appspot.com/5303083
>
> --
> You received this message because you are subscribed to the Google Groups "C-compiler-team" group.
> To post to this group, send email to c-compiler-team@google.com.
> To unsubscribe from this group, send email to c-compiler-team+unsubscribe@google.com.
> For more options, visit this group at http://groups.google.com/a/google.com/group/c-compiler-team/?hl=en.
>
>