From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 11315 invoked by alias); 28 Oct 2011 15:54:18 -0000 Received: (qmail 11254 invoked by uid 22791); 28 Oct 2011 15:54:09 -0000 X-SWARE-Spam-Status: No, hits=-2.3 required=5.0 tests=AWL,BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,RP_MATCHES_RCVD,SPF_HELO_PASS,TW_CP,TW_TM,TW_VP,TW_VT X-Spam-Check-By: sourceware.org Received: from smtp-out.google.com (HELO smtp-out.google.com) (74.125.121.67) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Fri, 28 Oct 2011 15:53:43 +0000 Received: from hpaq13.eem.corp.google.com (hpaq13.eem.corp.google.com [172.25.149.13]) by smtp-out.google.com with ESMTP id p9SFreE0021099 for ; Fri, 28 Oct 2011 08:53:40 -0700 Received: from vws15 (vws15.prod.google.com [10.241.21.143]) by hpaq13.eem.corp.google.com with ESMTP id p9SFrSCA005159 (version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=NOT) for ; Fri, 28 Oct 2011 08:53:39 -0700 Received: by vws15 with SMTP id 15so5899968vws.10 for ; Fri, 28 Oct 2011 08:53:39 -0700 (PDT) Received: by 10.150.214.1 with SMTP id m1mr3176819ybg.89.1319817219157; Fri, 28 Oct 2011 08:53:39 -0700 (PDT) MIME-Version: 1.0 Received: by 10.150.214.1 with SMTP id m1mr3176797ybg.89.1319817218883; Fri, 28 Oct 2011 08:53:38 -0700 (PDT) Received: by 10.151.109.17 with HTTP; Fri, 28 Oct 2011 08:53:38 -0700 (PDT) In-Reply-To: <20111028132511.873EAC1B4C@1024cores.msk.corp.google.com> References: <20111028132511.873EAC1B4C@1024cores.msk.corp.google.com> Date: Fri, 28 Oct 2011 16:23:00 -0000 Message-ID: Subject: Re: [C-compiler-team] [google] ThreadSanitizer instrumentation pass (issue5303083) From: Xinliang David Li To: Dmitriy Vyukov Cc: reply@codereview.appspotmail.com, c-compiler-team@google.com, kcc@google.com, gcc-patches@gcc.gnu.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-System-Of-Record: true X-IsSubscribed: yes Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org X-SW-Source: 2011-10/txt/msg02686.txt.bz2 Dmitriy, I will review it next week. Do you have some runtime overhead data ? thanks, David On Fri, Oct 28, 2011 at 6:25 AM, Dmitriy Vyukov wrote: > The patch is for google/main branch. > ThreadSanitizer is a data race detector for C/C++ programs. > http://code.google.com/p/data-race-test/wiki/ThreadSanitizer > > The tool consists of two parts: > instrumentation module (this file) and a run-time library. > The instrumentation module mainintains shadow call stacks > and intercepts interesting memory accesses. > The instrumentation is enabled with -ftsan flag. > > Instrumentation for shadow stack maintainance is as follows: > void somefunc () > { > =A0__tsan_shadow_stack [-1] =3D __builtin_return_address (0); > =A0__tsan_shadow_stack++; > =A0// function body > =A0__tsan_shadow_stack--; > } > > Interception for memory access interception is as follows: > *addr =3D 1; > __tsan_handle_mop (addr, flags); > where flags are (is_sblock | (is_store << 1) | ((sizeof (*addr) - 1) << 2= ). > is_sblock is used merely for optimization purposes and can always > be set to 1, see comments in instrument_mops function. > > Ignore files can be used to selectively non instrument some functions. > Ignore file is specified with -ftsan-ignore=3Dfilename flag. > There are 3 types of ignores: (1) do not instrument memory accesses > in the function, (2) do not create sblocks in the function > and (3) recursively ignore memory accesses in the function. > That last ignore type requires additional instrumentation of the form: > void somefunc () > { > =A0__tsan_thread_ignore++; > =A0// function body > =A0__tsan_thread_ignore--; > } > > The run-time library provides __tsan_handle_mop function, > definitions of __tsan_shadow_stack and __tsan_thread_ignore variables, > and intercepts synchronization related functions. > > 2011-10-28 =A0 Dmitriy Vyukov =A0 > > =A0 =A0 =A0 =A0* gcc/doc/invoke.texi: > =A0 =A0 =A0 =A0* gcc/tree-tsan.c (enum tsan_ignore_e): > =A0 =A0 =A0 =A0(enum bb_state_e): > =A0 =A0 =A0 =A0(struct bb_data_t): > =A0 =A0 =A0 =A0(struct mop_desc_t): > =A0 =A0 =A0 =A0(struct tsan_ignore_desc_t): > =A0 =A0 =A0 =A0(lookup_name): > =A0 =A0 =A0 =A0(shadow_stack_def): > =A0 =A0 =A0 =A0(thread_ignore_def): > =A0 =A0 =A0 =A0(rtl_mop_def): > =A0 =A0 =A0 =A0(ignore_append): > =A0 =A0 =A0 =A0(ignore_match): > =A0 =A0 =A0 =A0(ignore_load): > =A0 =A0 =A0 =A0(tsan_ignore): > =A0 =A0 =A0 =A0(decl_name): > =A0 =A0 =A0 =A0(build_stack_op): > =A0 =A0 =A0 =A0(build_rec_ignore_op): > =A0 =A0 =A0 =A0(build_stack_assign): > =A0 =A0 =A0 =A0(instr_mop): > =A0 =A0 =A0 =A0(instr_vptr_store): > =A0 =A0 =A0 =A0(instr_func): > =A0 =A0 =A0 =A0(set_location): > =A0 =A0 =A0 =A0(is_dtor_vptr_store): > =A0 =A0 =A0 =A0(is_vtbl_read): > =A0 =A0 =A0 =A0(is_load_of_const): > =A0 =A0 =A0 =A0(handle_expr): > =A0 =A0 =A0 =A0(handle_gimple): > =A0 =A0 =A0 =A0(instrument_bblock): > =A0 =A0 =A0 =A0(instrument_mops): > =A0 =A0 =A0 =A0(instrument_function): > =A0 =A0 =A0 =A0(tsan_pass): > =A0 =A0 =A0 =A0(tsan_gate): > =A0 =A0 =A0 =A0* gcc/tree-pass.h: > =A0 =A0 =A0 =A0* gcc/testsuite/gcc.dg/tsan-ignore.ignore: > =A0 =A0 =A0 =A0* gcc/testsuite/gcc.dg/tsan.h (__tsan_init): > =A0 =A0 =A0 =A0(__tsan_expect_mop): > =A0 =A0 =A0 =A0(__tsan_handle_mop): > =A0 =A0 =A0 =A0* gcc/testsuite/gcc.dg/tsan-ignore.c (foo): > =A0 =A0 =A0 =A0(int bar): > =A0 =A0 =A0 =A0(int baz): > =A0 =A0 =A0 =A0(int bla): > =A0 =A0 =A0 =A0(int xxx): > =A0 =A0 =A0 =A0(main): > =A0 =A0 =A0 =A0* gcc/testsuite/gcc.dg/tsan-ignore.h (in_tsan_ignore_heade= r): > =A0 =A0 =A0 =A0* gcc/testsuite/gcc.dg/tsan-stack.c (foobar): > =A0 =A0 =A0 =A0* gcc/testsuite/gcc.dg/tsan-mop.c: > =A0 =A0 =A0 =A0* gcc/common.opt: > =A0 =A0 =A0 =A0* gcc/Makefile.in: > =A0 =A0 =A0 =A0* gcc/passes.c: > > Index: gcc/doc/invoke.texi > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- gcc/doc/invoke.texi (revision 180522) > +++ gcc/doc/invoke.texi (working copy) > @@ -308,6 +308,7 @@ > =A0-fdump-tree-ssa@r{[}-@var{n}@r{]} -fdump-tree-pre@r{[}-@var{n}@r{]} @g= ol > =A0-fdump-tree-ccp@r{[}-@var{n}@r{]} -fdump-tree-dce@r{[}-@var{n}@r{]} @g= ol > =A0-fdump-tree-gimple@r{[}-raw@r{]} -fdump-tree-mudflap@r{[}-@var{n}@r{]}= @gol > +-fdump-tree-tsan@r{[}-@var{n}@r{]} @gol > =A0-fdump-tree-dom@r{[}-@var{n}@r{]} @gol > =A0-fdump-tree-dse@r{[}-@var{n}@r{]} @gol > =A0-fdump-tree-phiprop@r{[}-@var{n}@r{]} @gol > @@ -381,8 +382,8 @@ > =A0-floop-parallelize-all -flto -flto-compression-level @gol > =A0-flto-partition=3D@var{alg} -flto-report -fmerge-all-constants @gol > =A0-fmerge-constants -fmodulo-sched -fmodulo-sched-allow-regmoves @gol > --fmove-loop-invariants fmudflap -fmudflapir -fmudflapth -fno-branch-coun= t-reg @gol > --fno-default-inline @gol > +-fmove-loop-invariants -fmudflap -fmudflapir -fmudflapth -fno-branch-cou= nt-reg @gol > +-ftsan -ftsan-ignore -fno-default-inline @gol > =A0-fno-defer-pop -fno-function-cse -fno-guess-branch-probability @gol > =A0-fno-inline -fno-math-errno -fno-peephole -fno-peephole2 @gol > =A0-fno-sched-interblock -fno-sched-spec -fno-signed-zeros @gol > @@ -5896,6 +5897,11 @@ > =A0Dump each function after adding mudflap instrumentation. =A0The file n= ame is > =A0made by appending @file{.mudflap} to the source file name. > > +@item tsan > +@opindex fdump-tree-tsan > +Dump each function after adding ThreadSanitizer instrumentation. =A0The = file name is > +made by appending @file{.tsan} to the source file name. > + > =A0@item sra > =A0@opindex fdump-tree-sra > =A0Dump each function after performing scalar replacement of aggregates. = =A0The > @@ -6674,6 +6680,12 @@ > =A0some protection against outright memory corrupting writes, but allows > =A0erroneously read data to propagate within a program. > > +@item -ftsan -ftsan-ignore > +@opindex ftsan > +@opindex ftsan-ignore > +Add ThreadSanitizer instrumentation. Use @option{-ftsan-ignore} to speci= fy > +an ignore file. Refer to http://go/tsan for details. > + > =A0@item -fthread-jumps > =A0@opindex fthread-jumps > =A0Perform optimizations where we check to see if a jump branches to a > Index: gcc/tree-tsan.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- gcc/tree-tsan.c =A0 =A0 (revision 0) > +++ gcc/tree-tsan.c =A0 =A0 (revision 0) > @@ -0,0 +1,1204 @@ > +/* ThreadSanitizer instrumentation pass. > + =A0 http://code.google.com/p/data-race-test > + =A0 Copyright (C) 2011 > + =A0 Free Software Foundation, Inc. > + =A0 Contributed by Dmitry Vyukov > + > +This file is part of GCC. > + > +GCC is free software; you can redistribute it and/or modify > +it under the terms of the GNU General Public License as published by > +the Free Software Foundation; either version 3, or (at your option) > +any later version. > + > +GCC is distributed in the hope that it will be useful, > +but WITHOUT ANY WARRANTY; without even the implied warranty of > +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. =A0See the > +GNU General Public License for more details. > + > +You should have received a copy of the GNU General Public License > +along with GCC; see the file COPYING3. =A0If not see > +. =A0*/ > + > +#include "config.h" > +#include "system.h" > +#include "coretypes.h" > +#include "tree.h" > +#include "intl.h" > +#include "tm.h" > +#include "basic-block.h" > +#include "gimple.h" > +#include "function.h" > +#include "tree-flow.h" > +#include "tree-pass.h" > +#include "cfghooks.h" > +#include "langhooks.h" > +#include "output.h" > +#include "options.h" > + > +/* The file can be compiled either as compiler pass or plugin. =A0*/ > +#ifdef GCC_PLG > +# include "c-common.h" > +#else > +# include "c-family/c-common.h" > +#endif > + > +#include "diagnostic.h" > + > +#include > +#include > + > +/* ThreadSanitizer is a data race detector for C/C++ programs. > + =A0 http://code.google.com/p/data-race-test/wiki/ThreadSanitizer > + > + =A0 The tool consists of two parts: > + =A0 instrumentation module (this file) and a run-time library. > + =A0 The instrumentation module mainintains shadow call stacks > + =A0 and intercepts interesting memory accesses. > + =A0 The instrumentation is enabled with -ftsan flag. > + > + =A0 Instrumentation for shadow stack maintainance is as follows: > + =A0 void somefunc () > + =A0 { > + =A0 =A0 __tsan_shadow_stack [-1] =3D __builtin_return_address (0); > + =A0 =A0 __tsan_shadow_stack++; > + =A0 =A0 // function body > + =A0 =A0 __tsan_shadow_stack--; > + =A0 } > + > + =A0 Interception for memory access interception is as follows: > + =A0 *addr =3D 1; > + =A0 __tsan_handle_mop (addr, flags); > + =A0 where flags are (is_sblock | (is_store << 1) | ((sizeof (*addr) - 1= ) << 2). > + =A0 is_sblock is used merely for optimization purposes and can always > + =A0 be set to 1, see comments in instrument_mops function. > + > + =A0 Ignore files can be used to selectively non instrument some functio= ns. > + =A0 Ignore file is specified with -ftsan-ignore=3Dfilename flag. > + =A0 There are 3 types of ignores: (1) do not instrument memory accesses > + =A0 in the function, (2) do not create sblocks in the function > + =A0 and (3) recursively ignore memory accesses in the function. > + =A0 That last ignore type requires additional instrumentation of the fo= rm: > + =A0 void somefunc () > + =A0 { > + =A0 =A0 __tsan_thread_ignore++; > + =A0 =A0 // function body > + =A0 =A0 __tsan_thread_ignore--; > + =A0 } > + > + =A0 The run-time library provides __tsan_handle_mop function, > + =A0 definitions of __tsan_shadow_stack and __tsan_thread_ignore variabl= es, > + =A0 and intercepts synchronization related functions. =A0*/ > + > +#define RTL_IGNORE "__tsan_thread_ignore" > +#define RTL_STACK "__tsan_shadow_stack" > +#define RTL_MOP "__tsan_handle_mop" > +#define RTL_PERFIX "__tsan_" > +#define MAX_MOP_BYTES 16 > +#define SBLOCK_SIZE 5 > + > +enum tsan_ignore_e > +{ > + =A0tsan_ignore_none =A0=3D 1 << 0, /* Do not ignore. =A0*/ > + =A0tsan_ignore_func =A0=3D 1 << 1, /* Completely ignore the whole func.= =A0*/ > + =A0tsan_ignore_mop =A0 =3D 1 << 2, /* Do not instrument memory accesses= . =A0*/ > + =A0tsan_ignore_rec =A0 =3D 1 << 3, /* Do not instrument memory accesses= recursively. =A0*/ > + =A0tsan_ignore_hist =A0=3D 1 << 4 =A0/* Do not create superblocks. =A0*/ > +}; > + > +/* Basic block state during CFG traversal. =A0*/ > +enum bb_state_e > +{ > + =A0bb_not_visited, > + =A0bb_candidate, > + =A0bb_visited > +}; > + > +/* Info associated with each basic block. > + =A0 Used to determine super-blocks (see instrument_mops ()). =A0*/ > +struct bb_data_t > +{ > + =A0enum bb_state_e =A0 =A0 =A0 state; > + =A0int =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 has_sb; > + =A0const char =A0 =A0 =A0 =A0 =A0 *sb_file; > + =A0int =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 sb_line_min; > + =A0int =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 sb_line_max; > +}; > + > +/* Memory access descriptor. =A0*/ > +struct mop_desc_t > +{ > + =A0int =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 is_call; > + =A0gimple_stmt_iterator =A0gsi; > + =A0tree =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0expr; > + =A0tree =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0dtor_vptr_expr; > + =A0int =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 is_store; > +}; > + > +struct tsan_ignore_desc_t > +{ > + =A0struct tsan_ignore_desc_t *next; > + =A0enum tsan_ignore_e =A0 =A0 =A0 =A0 type; > + =A0char =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0*name; > +}; > + > +/* Number of instrumented memory accesses in the current function. =A0*/ > +static int func_mops; > +/* Number of function calls in the current function. =A0*/ > +static int func_calls; > +/* Ignore status for the current function (see tsan_ignore_e). =A0*/ > +static enum tsan_ignore_e func_ignore; > + > +static int ignore_init =3D 0; > +static struct tsan_ignore_desc_t *ignore_head; > + > +typedef struct mop_desc_t mop_desc_t; > +DEF_VEC_O (mop_desc_t); > +DEF_VEC_ALLOC_O (mop_desc_t, heap); > +static VEC (mop_desc_t, heap) *mop_list; > + > +/* The function is not available in some modules. =A0*/ > +tree __attribute__((weak)) > +lookup_name (tree t) > +{ > + =A0(void)t; > + =A0return NULL_TREE; > +} > + > +/* Builds the following decl > + =A0 extern __thread void **__tsan_shadow_stack; */ > +static tree > +shadow_stack_def (void) > +{ > + =A0static tree def; > + > + =A0if (def !=3D NULL) > + =A0 =A0return def; > + > + =A0/* Check if a user has defined it for testing */ > + =A0def =3D lookup_name (get_identifier (RTL_STACK)); > + =A0if (def !=3D NULL) > + =A0 =A0return def; > + > + =A0def =3D build_decl (UNKNOWN_LOCATION, VAR_DECL, > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 get_identifier (RTL_STACK), > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 build_pointer_type (ptr_type_node)); > + =A0TREE_STATIC (def) =3D 1; > + =A0TREE_PUBLIC (def) =3D 1; > + =A0DECL_EXTERNAL (def) =3D 1; > + =A0DECL_TLS_MODEL (def) =3D decl_default_tls_model (def); > + =A0TREE_USED (def) =3D 1; > + =A0TREE_THIS_VOLATILE (def) =3D 1; > + =A0SET_DECL_ASSEMBLER_NAME (def, get_identifier (RTL_STACK)); > + =A0return def; > +} > + > +/* Builds the following decl > + =A0 extern __thread int __tsan_thread_ignore; */ > +static tree > +thread_ignore_def (void) > +{ > + =A0static tree def; > + > + =A0if (def !=3D NULL) > + =A0 =A0return def; > + > + =A0/* Check if a user has defined it for testing */ > + =A0def =3D lookup_name (get_identifier (RTL_IGNORE)); > + =A0if (def !=3D NULL) > + =A0 =A0return def; > + > + =A0def =3D build_decl (UNKNOWN_LOCATION, VAR_DECL, > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 get_identifier (RTL_IGNORE), > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 integer_type_node); > + =A0TREE_STATIC (def) =3D 1; > + =A0TREE_PUBLIC (def) =3D 1; > + =A0DECL_EXTERNAL (def) =3D 1; > + =A0DECL_TLS_MODEL (def) =3D decl_default_tls_model (def); > + =A0TREE_USED (def) =3D 1; > + =A0TREE_THIS_VOLATILE (def) =3D 1; > + =A0SET_DECL_ASSEMBLER_NAME (def, get_identifier (RTL_IGNORE)); > + =A0return def; > +} > + > +/* Builds the following decl > + =A0 void __tsan_handle_mop (void *addr, unsigned flags); */ > +static tree > +rtl_mop_def (void) > +{ > + =A0tree fn_type; > + > + =A0static tree def; > + > + =A0if (def !=3D NULL) > + =A0 =A0return def; > + > + =A0/* Check if a user has defined it for testing */ > + =A0def =3D lookup_name (get_identifier (RTL_MOP)); > + =A0if (def !=3D NULL) > + =A0 =A0return def; > + > + =A0fn_type =3D build_function_type_list (void_type_node, ptr_type_node, > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0integer_type_node , NULL_TREE); > + =A0def =3D build_fn_decl (RTL_MOP, fn_type); > + =A0TREE_NOTHROW (def) =3D 1; > + =A0DECL_ATTRIBUTES (def) =3D tree_cons (get_identifier ("leaf"), > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= NULL, DECL_ATTRIBUTES (def)); > + =A0DECL_ASSEMBLER_NAME (def); > + =A0return def; > +} > + > +/* Adds new ignore definition to the global list */ > +static void > +ignore_append (enum tsan_ignore_e type, char *name) > +{ > + =A0struct tsan_ignore_desc_t *desc; > + > + =A0desc =3D (struct tsan_ignore_desc_t*)xmalloc (sizeof (*desc)); > + =A0desc->type =3D type; > + =A0desc->name =3D xstrdup (name); > + =A0desc->next =3D ignore_head; > + =A0ignore_head =3D desc; > +} > + > +/* Checks as to whether identifier 'str' matches template 'templ'. > + =A0 Templates can only contain '*', e.g. 'std*string*insert'. > + =A0 Templates implicitly start and end with '*' > + =A0 since they are matched against mangled names. =A0*/ > +static int > +ignore_match (char *templ, const char *str) > +{ > + =A0char *tpos; > + =A0const char *spos; > + > + =A0while (templ && templ [0]) > + =A0 =A0{ > + =A0 =A0 =A0if (templ [0] =3D=3D '*') > + =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0templ++; > + =A0 =A0 =A0 =A0 =A0continue; > + =A0 =A0 =A0 =A0} > + =A0 =A0 =A0if (str [0] =3D=3D 0) > + =A0 =A0 =A0 =A0return 0; > + =A0 =A0 =A0tpos =3D strchr (templ, '*'); > + =A0 =A0 =A0if (tpos !=3D NULL) > + =A0 =A0 =A0 =A0tpos [0] =3D 0; > + =A0 =A0 =A0spos =3D strstr (str, templ); > + =A0 =A0 =A0str =3D spos + strlen (templ); > + =A0 =A0 =A0templ =3D tpos; > + =A0 =A0 =A0if (tpos !=3D NULL) > + =A0 =A0 =A0 =A0tpos [0] =3D '*'; > + =A0 =A0 =A0if (spos =3D=3D NULL) > + =A0 =A0 =A0 =A0return 0; > + =A0 =A0} > + =A0return 1; > +} > + > +/* Loads ignore definitions from the file specified by -ftsan-ignore=3Df= ilename. > + =A0 Ignore files have the following format: > + > +# This is a comment - ignored > + > +# The below line says to not instrument memory accesses > +# in all functions that match 'std*string*insert' > +fun:std*string*insert > + > +# The below line says to not instrument memory accesses > +# in the function called 'foobar' *and* in all functions > +# that it calls recursively > +fun_r:foobar > + > +# The below line says to not create superblocks > +# in the function called 'barbaz' > +fun_hist:barbaz > + > +# Ignore all functions in the source file > +src:atomic.c > + > +# Everything else is uninteresting for us (e.g. obj:) > +*/ > +static void > +ignore_load (void) > +{ > + =A0FILE *f; > + =A0char *line; > + =A0size_t linesz; > + =A0ssize_t sz; > + =A0char buf [PATH_MAX]; > + > + =A0if (flag_tsan_ignore =3D=3D NULL || flag_tsan_ignore [0] =3D=3D 0) > + =A0 =A0return; > + > + =A0f =3D fopen (flag_tsan_ignore, "r"); > + =A0if (f =3D=3D NULL) > + =A0 =A0{ > + =A0 =A0 =A0/* Try to open it relative to main_input_filename. =A0*/ > + =A0 =A0 =A0strncpy (buf, main_input_filename, sizeof (buf)); > + =A0 =A0 =A0buf [sizeof (buf) - 1] =3D 0; > + =A0 =A0 =A0line =3D strrchr (buf, '/'); > + =A0 =A0 =A0if (line !=3D NULL) > + =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0line++; > + =A0 =A0 =A0 =A0 =A0strncpy (line, flag_tsan_ignore, sizeof (buf) - (lin= e - buf)); > + =A0 =A0 =A0 =A0 =A0buf [sizeof (buf) - 1] =3D 0; > + =A0 =A0 =A0 =A0 =A0f =3D fopen (buf, "r"); > + =A0 =A0 =A0 =A0} > + =A0 =A0} > + =A0if (f =3D=3D NULL) > + =A0 =A0{ > + =A0 =A0 =A0printf ("failed to open ignore file '%s'\n", flag_tsan_ignor= e); > + =A0 =A0 =A0exit (1); > + =A0 =A0} > + > + =A0line =3D 0; > + =A0linesz =3D 0; > + =A0while ((sz =3D getline (&line, &linesz, f)) !=3D -1) > + =A0 =A0{ > + =A0 =A0 =A0if (sz =3D=3D 0) > + =A0 =A0 =A0 =A0continue; > + =A0 =A0 =A0/* strip line terminator */ > + =A0 =A0 =A0if (line [sz-1] =3D=3D '\r' || line [sz-1] =3D=3D '\n') > + =A0 =A0 =A0 =A0line [sz-1] =3D 0; > + =A0 =A0 =A0if (strncmp (line, "src:", sizeof ("src:")-1) =3D=3D 0) > + =A0 =A0 =A0 =A0ignore_append (tsan_ignore_func, line + sizeof ("src:")-= 1); > + =A0 =A0 =A0else if (strncmp (line, "fun:", sizeof ("fun:")-1) =3D=3D 0) > + =A0 =A0 =A0 =A0ignore_append (tsan_ignore_mop, line + sizeof ("fun:")-1= ); > + =A0 =A0 =A0else if (strncmp (line, "fun_r:", sizeof ("fun_r:")-1) =3D= =3D 0) > + =A0 =A0 =A0 =A0ignore_append (tsan_ignore_rec, line + sizeof ("fun_r:")= -1); > + =A0 =A0 =A0else if (strncmp (line, "fun_hist:", sizeof ("fun_hist:")-1)= =3D=3D 0) > + =A0 =A0 =A0 =A0ignore_append (tsan_ignore_hist, line + sizeof ("fun_his= t:")-1); > + =A0 =A0 =A0/* other lines are not interesting */ > + =A0 =A0} > + > + =A0free (line); > + =A0fclose (f); > +} > + > +/* Returns ignore status for the current function */ > +static enum tsan_ignore_e > +tsan_ignore (void) > +{ > + =A0const char *func_name; > + =A0const char *src_name; > + =A0struct tsan_ignore_desc_t *desc; > + > + =A0if (ignore_init =3D=3D 0) > + =A0 =A0{ > + =A0 =A0 =A0ignore_load (); > + =A0 =A0 =A0ignore_init =3D 1; > + =A0 =A0} > + > + =A0src_name =3D expand_location(cfun->function_start_locus).file; > + =A0if (src_name =3D=3D NULL) > + =A0 =A0src_name =3D ""; > + > + =A0func_name =3D IDENTIFIER_POINTER (DECL_ASSEMBLER_NAME (cfun->decl)); > + =A0/* Ignore all functions starting with __tsan_ - intended for testing= */ > + =A0if (strncmp (func_name, RTL_PERFIX, sizeof (RTL_PERFIX) - 1) =3D=3D = 0) > + =A0 =A0return tsan_ignore_func; > + > + =A0for (desc =3D ignore_head; desc; desc =3D desc->next) > + =A0 =A0{ > + =A0 =A0 =A0if (desc->type =3D=3D tsan_ignore_func) > + =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0if (ignore_match (desc->name, src_name)) > + =A0 =A0 =A0 =A0 =A0 return desc->type; > + =A0 =A0 =A0 =A0} > + =A0 =A0 =A0else if (ignore_match (desc->name, func_name)) > + =A0 =A0 =A0 return desc->type; > + =A0 =A0} > + =A0return tsan_ignore_none; > +} > + > +static const char * > +decl_name (tree decl) > +{ > + =A0tree id; > + =A0const char *name; > + > + =A0if (decl !=3D 0 && DECL_P (decl)) > + =A0 =A0{ > + =A0 =A0 =A0id =3D DECL_NAME (decl); > + =A0 =A0 =A0if (id !=3D NULL) > + =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0name =3D IDENTIFIER_POINTER (id); > + =A0 =A0 =A0 =A0 =A0if (name !=3D NULL) > + =A0 =A0 =A0 =A0 =A0 =A0return name; > + =A0 =A0 =A0 =A0} > + =A0 =A0} > + =A0return ""; > +} > + > +/* Builds either (__tsan_shadow_stack +=3D 1) or (__tsan_shadow_stack -= =3D 1) expression > + =A0 depending on 'do_dec' parameter. =A0Appends the result to seq. =A0*/ > +static void > +build_stack_op (gimple_seq *seq, bool do_dec) > +{ > + =A0tree op_size; > + =A0double_int op_size_cst; > + =A0unsigned long long size_val; > + =A0unsigned long long size_valhi; > + =A0tree op_expr; > + =A0tree assign; > + =A0tree rtl_stack; > + =A0gimple_seq s; > + > + =A0op_size =3D TYPE_SIZE (ptr_type_node); > + =A0op_size_cst =3D tree_to_double_int (op_size); > + =A0size_val =3D op_size_cst.low / BITS_PER_UNIT; > + =A0size_valhi =3D 0; > + =A0if (do_dec) > + =A0 =A0{ > + =A0 =A0 =A0size_val =3D -size_val; > + =A0 =A0 =A0size_valhi =3D -1; > + =A0 =A0} > + =A0op_size =3D build_int_cst_wide (sizetype, size_val, size_valhi); > + =A0rtl_stack =3D shadow_stack_def (); > + =A0op_expr =3D build2 (POINTER_PLUS_EXPR, ptr_type_node, rtl_stack, op_= size); > + =A0assign =3D build2 (MODIFY_EXPR, ptr_type_node, rtl_stack, op_expr); > + =A0s =3D NULL; > + =A0force_gimple_operand (assign, &s, true, NULL_TREE); > + =A0gimple_seq_add_seq (seq, s); > +} > + > +/* Builds either (__tsan_thread_ignore +=3D 1) or (__tsan_thread_ignore = -=3D 1) expression > + =A0 depending on op parameter. =A0Stores the result in seq. =A0*/ > +static void > +build_rec_ignore_op (gimple_seq *seq, enum tree_code op) > +{ > + =A0tree rec_expr; > + =A0gimple_seq rec_inc; > + =A0gimple rec_assign; > + =A0tree rtl_ignore; > + > + =A0rtl_ignore =3D thread_ignore_def (); > + =A0rec_expr =3D build2 (op, integer_type_node, rtl_ignore, integer_one_= node); > + =A0rec_inc =3D NULL; > + =A0rec_expr =3D force_gimple_operand (rec_expr, &rec_inc, true, NULL_TR= EE); > + =A0rec_assign =3D gimple_build_assign (rtl_ignore, rec_expr); > + =A0gimple_seq_add_seq (seq, rec_inc); > + =A0gimple_seq_add_stmt (seq, rec_assign); > +} > + > +/* Build the following gimple sequence: > + =A0 __tsan_shadow_stack [-1] =3D __builtin_return_address (0); > + =A0 Stores the result in seq. =A0*/ > +static void > +build_stack_assign (gimple_seq *seq) > +{ > + =A0tree pc_addr; > + =A0tree op_size; > + =A0tree op_expr; > + =A0tree stack_op; > + =A0tree assign; > + =A0tree rtl_retaddr; > + > + =A0rtl_retaddr =3D implicit_built_in_decls [BUILT_IN_RETURN_ADDRESS]; > + =A0pc_addr =3D build_call_expr (rtl_retaddr, 1, integer_zero_node); > + =A0op_size =3D build_int_cst_wide (sizetype, -(POINTER_SIZE / BITS_PER_= UNIT), -1); > + =A0op_expr =3D build2 (POINTER_PLUS_EXPR, ptr_type_node, > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0shadow_stack_def (), op_= size); > + =A0stack_op =3D build1 (INDIRECT_REF, ptr_type_node, op_expr); > + =A0assign =3D build2 (MODIFY_EXPR, ptr_type_node, stack_op, pc_addr); > + =A0force_gimple_operand (assign, seq, true, NULL_TREE); > +} > + > +/* Builds the following gimple sequence: > + =A0 __tsan_handle_mop (&expr, (is_sblock | (is_store << 1) | ((sizeof (= expr)-1) << 2) > + =A0 The result is stored in gseq. =A0*/ > +static void > +instr_mop (tree expr, int is_store, int is_sblock, gimple_seq *gseq) > +{ > + =A0tree addr_expr; > + =A0tree expr_type; > + =A0unsigned size; > + =A0unsigned flags; > + =A0tree flags_expr; > + =A0tree call_expr; > + > + =A0gcc_assert (gseq !=3D 0 && *gseq =3D=3D 0); > + =A0gcc_assert (is_gimple_addressable (expr)); > + > + =A0addr_expr =3D build_addr (expr, current_function_decl); > + =A0expr_type =3D TREE_TYPE (expr); > + =A0while (TREE_CODE (expr_type) =3D=3D ARRAY_TYPE) > + =A0 =A0expr_type =3D TREE_TYPE (expr_type); > + =A0size =3D TREE_INT_CST_LOW (TYPE_SIZE (expr_type)); > + =A0size =3D size / BITS_PER_UNIT; > + =A0if (size > MAX_MOP_BYTES) > + =A0 =A0size =3D MAX_MOP_BYTES; > + =A0size -=3D 1; > + =A0flags =3D ((!!is_sblock << 0) + (!!is_store << 1) + (size << 2)); > + =A0flags_expr =3D build_int_cst (unsigned_type_node, flags); > + =A0call_expr =3D build_call_expr (rtl_mop_def (), 2, addr_expr, flags_e= xpr); > + =A0force_gimple_operand (call_expr, gseq, true, 0); > +} > + > +/* Builds the following gimple sequence: > + =A0 int is_store =3D (expr !=3D rhs); // the temp is not actually intro= duced > + =A0 __tsan_handle_mop (&expr, (is_sblock | (is_store << 1) | ((sizeof (= expr)-1) << 2) > + =A0 The result is stored in gseq. =A0*/ > +static void > +instr_vptr_store (tree expr, tree rhs, int is_sblock, gimple_seq *gseq) > +{ > + =A0tree expr_ptr; > + =A0tree addr_expr; > + =A0tree expr_type; > + =A0tree expr_size; > + =A0double_int size; > + =A0unsigned flags; > + =A0tree flags_expr; > + =A0gimple_seq flags_seq; > + =A0gimple collect; > + =A0tree is_store_expr; > + > + =A0expr_ptr =3D build_addr (expr, current_function_decl); > + =A0addr_expr =3D force_gimple_operand (expr_ptr, gseq, true, NULL_TREE); > + =A0expr_type =3D TREE_TYPE (expr); > + =A0while (TREE_CODE (expr_type) =3D=3D ARRAY_TYPE) > + =A0 =A0expr_type =3D TREE_TYPE (expr_type); > + =A0expr_size =3D TYPE_SIZE (expr_type); > + =A0size =3D tree_to_double_int (expr_size); > + =A0gcc_assert (size.high =3D=3D 0 && size.low !=3D 0); > + =A0if (size.low > 128) > + =A0 =A0size.low =3D 128; > + =A0size.low =3D (size.low / 8) - 1; > + =A0flags =3D ((!!is_sblock << 0) + (size.low << 2)); > + =A0flags_expr =3D build_int_cst (unsigned_type_node, flags); > + =A0is_store_expr =3D build2 (NE_EXPR, integer_type_node, > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0build1 (VIEW= _CONVERT_EXPR, size_type_node, expr), > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0build1 (VIEW= _CONVERT_EXPR, size_type_node, rhs)); > + =A0is_store_expr =3D build2 (LSHIFT_EXPR, integer_type_node, > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0is_store_exp= r, integer_one_node); > + =A0flags_expr =3D build2 (BIT_IOR_EXPR, integer_type_node, > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0is_store_exp= r, flags_expr); > + =A0flags_seq =3D 0; > + =A0flags_expr =3D force_gimple_operand (flags_expr, &flags_seq, true, N= ULL_TREE); > + =A0gimple_seq_add_seq (gseq, flags_seq); > + =A0collect =3D gimple_build_call ( > + =A0 =A0 =A0rtl_mop_def (), 2, addr_expr, flags_expr); > + =A0gimple_seq_add_stmt (gseq, collect); > +} > + > +/* Builds gimple sequences that must be inserted at function entry (pre) > + =A0 and before function exit (post). =A0*/ > +static void > +instr_func (gimple_seq *pre, gimple_seq *post) > +{ > + =A0/* In this case we need no instrumentation for the function */ > + =A0if (func_calls =3D=3D 0 && func_mops =3D=3D 0) > + =A0 =A0return; > + > + =A0if (func_ignore !=3D tsan_ignore_rec) > + =A0 =A0{ > + =A0 =A0 =A0build_stack_assign (pre); > + =A0 =A0 =A0build_stack_op (pre, false); > + =A0 =A0 =A0build_stack_op (post, true); > + =A0 =A0} > + > + =A0if (func_ignore =3D=3D tsan_ignore_rec && func_calls !=3D 0) > + =A0 =A0{ > + =A0 =A0 =A0build_rec_ignore_op (pre, PLUS_EXPR); > + =A0 =A0 =A0build_rec_ignore_op (post, MINUS_EXPR); > + =A0 =A0} > +} > + > +/* Sets location for all gimples in the seq. =A0*/ > +static void > +set_location (gimple_seq seq, location_t loc) > +{ > + =A0gimple_seq_node n; > + > + =A0for (n =3D gimple_seq_first (seq); n !=3D NULL; n =3D n->next) > + =A0 =A0gimple_set_location (n->stmt, loc); > +} > + > +/* Check as to whether expr refers to a store to vptr. =A0*/ > +static tree > +is_dtor_vptr_store (gimple stmt, tree expr, int is_store) > +{ > + =A0if (is_store =3D=3D 1 > + =A0 =A0 =A0&& TREE_CODE (expr) =3D=3D COMPONENT_REF > + =A0 =A0 =A0&& gimple_assign_single_p (stmt) > + =A0 =A0 =A0&& strcmp (decl_name (cfun->decl), "__base_dtor ") =3D=3D 0) > + =A0 =A0{ > + =A0 =A0 =A0tree comp =3D expr->exp.operands [0]; > + =A0 =A0 =A0while (TREE_CODE (comp) =3D=3D COMPONENT_REF) > + =A0 =A0 =A0 =A0comp =3D comp->exp.operands [0]; > + =A0 =A0 =A0if (TREE_CODE (comp) =3D=3D INDIRECT_REF || TREE_CODE (comp)= =3D=3D MEM_REF) > + =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0comp =3D comp->exp.operands [0]; > + =A0 =A0 =A0 =A0 =A0if (TREE_CODE (comp) =3D=3D SSA_NAME) > + =A0 =A0 =A0 =A0 =A0 =A0comp =3D SSA_NAME_VAR (comp); > + =A0 =A0 =A0 =A0 =A0if (strcmp (decl_name (comp), "this") =3D=3D 0) > + =A0 =A0 =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0tree field =3D expr->exp.operands [1]; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0if (TREE_CODE (field) =3D=3D FIELD_DECL > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0&& strncmp (decl_name (field), > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0"_vptr.", si= zeof ("_vptr.") - 1) =3D=3D 0) > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0return gimple_assign_rhs1 (stmt); > + =A0 =A0 =A0 =A0 =A0 =A0} > + =A0 =A0 =A0 =A0} > + =A0 =A0} > + =A0return 0; > +} > + > +/* Checks as to whether expr refers to a read from vtlb. > + =A0 Vtlbs are immutable, so don't bother to instrument them. =A0*/ > +static int > +is_vtbl_read (tree expr, int is_store) > +{ > + =A0/* We may not instrument reads from vtbl, because the data is consta= nt. > + =A0 =A0 vtbl read is of the form: > + =A0 =A0 =A0 gimple_assign _vptr.X, NULL> > + =A0 =A0 =A0 gimple_assign > + =A0 =A0 or: > + =A0 =A0 =A0 gimple_assign _vptr.X, NULL> > + =A0 =A0 =A0 gimple_assign > + =A0 =A0 =A0 gimple_assign */ > + > + =A0if (is_store =3D=3D 0 > + =A0 =A0 =A0&& TREE_CODE (expr) =3D=3D INDIRECT_REF) > + =A0 =A0{ > + =A0 =A0 =A0tree ref_target =3D expr->exp.operands [0]; > + =A0 =A0 =A0if (TREE_CODE (ref_target) =3D=3D SSA_NAME) > + =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0gimple ref_stmt =3D ref_target->ssa_name.def_stmt; > + =A0 =A0 =A0 =A0 =A0if (gimple_code (ref_stmt) =3D=3D GIMPLE_ASSIGN) > + =A0 =A0 =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0if (gimple_expr_code (ref_stmt) =3D=3D POINT= ER_PLUS_EXPR) > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0tree tmp =3D ref_stmt->gsmem.op [1]; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if (TREE_CODE (tmp) =3D=3D SSA_NAME > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0&& gimple_code (tmp->ssa_nam= e.def_stmt) =3D=3D GIMPLE_ASSIGN) > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ref_stmt =3D tmp->ssa_name.def_s= tmt; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0} > + =A0 =A0 =A0 =A0 =A0 =A0 =A0if (gimple_expr_code (ref_stmt) =3D=3D COMPO= NENT_REF > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0&& gimple_assign_single_p (ref_s= tmt)) > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0tree comp_expr =3D ref_stmt->gsmem.o= p [1]; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0tree field_expr =3D comp_expr->exp.o= perands [1]; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if (TREE_CODE (field_expr) =3D=3D FI= ELD_DECL > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0&& strncmp (decl_name (field= _expr), > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0"_vp= tr.", sizeof ("_vptr.") - 1) =3D=3D 0) > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0return 1; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0} > + =A0 =A0 =A0 =A0 =A0 =A0} > + =A0 =A0 =A0 =A0} > + =A0 =A0} > + > + =A0return 0; > +} > + > +/* Checks as to whether expr refers to constant var/field/param. > + =A0 Don't bother to instrument them. =A0*/ > +static int > +is_load_of_const (tree expr, int is_store) > +{ > + =A0if (is_store =3D=3D 0) > + =A0 =A0{ > + =A0 =A0 =A0if (TREE_CODE (expr) =3D=3D COMPONENT_REF) > + =A0 =A0 =A0 =A0expr =3D expr->exp.operands [1]; > + =A0 =A0 =A0if (TREE_CODE (expr) =3D=3D VAR_DECL > + =A0 =A0 =A0 =A0 =A0|| TREE_CODE (expr) =3D=3D PARM_DECL > + =A0 =A0 =A0 =A0 =A0|| TREE_CODE (expr) =3D=3D FIELD_DECL) > + =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0if (TREE_READONLY (expr)) > + =A0 =A0 =A0 =A0 =A0 =A0return 1; > + =A0 =A0 =A0 =A0} > + =A0 =A0} > + =A0return 0; > +} > + > +static void > +handle_expr (gimple stmt, gimple_stmt_iterator gsi, > + =A0 =A0 =A0 =A0 =A0 =A0 tree expr, int is_store, VEC (mop_desc_t, heap)= **mop_list) > +{ > + =A0enum tree_code tcode; > + =A0struct mop_desc_t mop; > + =A0unsigned fld_off; > + =A0unsigned fld_size; > + > + =A0/* map SSA name to real name */ > + =A0if (TREE_CODE (expr) =3D=3D SSA_NAME) > + =A0 =A0expr =3D SSA_NAME_VAR (expr); > + > + =A0tcode =3D TREE_CODE (expr); > + > + =A0/* Below are things we do NOT want to instrument. =A0*/ > + =A0if (func_ignore & (tsan_ignore_mop | tsan_ignore_rec)) > + =A0 =A0{ > + =A0 =A0 =A0return; > + =A0 =A0} > + =A0else if (TREE_CODE_CLASS (tcode) =3D=3D tcc_constant) > + =A0 =A0{ > + =A0 =A0 =A0/* various constant literals */ > + =A0 =A0 =A0return; > + =A0 =A0} > + =A0else if (TREE_CODE_CLASS (tcode) =3D=3D tcc_declaration > + =A0 =A0 =A0&& DECL_ARTIFICIAL (expr)) > + =A0 =A0{ > + =A0 =A0 =A0/* compiler-emitted artificial variables */ > + =A0 =A0 =A0return; > + =A0 =A0} > + =A0if (tcode =3D=3D RESULT_DECL) > + =A0 =A0{ > + =A0 =A0 =A0/* store to function result */ > + =A0 =A0 =A0return; > + =A0 =A0} > + =A0else if (tcode =3D=3D VAR_DECL > + =A0 =A0 =A0&& TREE_ADDRESSABLE (expr) =3D=3D 0 > + =A0 =A0 =A0&& TREE_STATIC (expr) =3D=3D 0) > + =A0 =A0{ > + =A0 =A0 =A0/* the var does not live in memory -> no possibility of race= s */ > + =A0 =A0 =A0return; > + =A0 =A0} > + =A0else if (TREE_CODE (TREE_TYPE (expr)) =3D=3D RECORD_TYPE) > + =A0 =A0{ > + =A0 =A0 =A0/* TODO (dvyukov): implement me */ > + =A0 =A0 =A0return; > + =A0 =A0} > + =A0else if (tcode =3D=3D CONSTRUCTOR) > + =A0 =A0{ > + =A0 =A0 =A0/* TODO (dvyukov): implement me */ > + =A0 =A0 =A0return; > + =A0 =A0} > + =A0else if (tcode =3D=3D PARM_DECL) > + =A0 =A0{ > + =A0 =A0 =A0/* TODO (dvyukov): implement me */ > + =A0 =A0 =A0return; > + =A0 =A0} > + =A0else if (is_load_of_const (expr, is_store)) > + =A0 =A0{ > + =A0 =A0 =A0/* load of a const variable/parameter/field */ > + =A0 =A0 =A0return; > + =A0 =A0} > + =A0else if (is_vtbl_read (expr, is_store)) > + =A0 =A0{ > + =A0 =A0 =A0/* vtbl read */ > + =A0 =A0 =A0return; > + =A0 =A0} > + =A0else if (tcode =3D=3D COMPONENT_REF) > + =A0 =A0{ > + =A0 =A0 =A0tree field =3D expr->exp.operands [1]; > + =A0 =A0 =A0if (TREE_CODE (field) =3D=3D FIELD_DECL) > + =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0fld_off =3D field->field_decl.bit_offset->int_cst.in= t_cst.low; > + =A0 =A0 =A0 =A0 =A0fld_size =3D field->decl_common.size->int_cst.int_cs= t.low; > + =A0 =A0 =A0 =A0 =A0if (((fld_off % BITS_PER_UNIT) !=3D 0) > + =A0 =A0 =A0 =A0 =A0 =A0 =A0|| ((fld_size % BITS_PER_UNIT) !=3D 0)) > + =A0 =A0 =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0/* As of now it crashes compilation. > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 TODO (dvyukov): handle bit-fields as if= touching the whole field */ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0return; > + =A0 =A0 =A0 =A0 =A0 =A0} > + =A0 =A0 =A0 =A0} > + =A0 =A0} > + > + =A0/* TODO (dvyukov): handle other cases > + =A0 =A0 (FIELD_DECL, MEM_REF, ARRAY_RANGE_REF, TARGET_MEM_REF, ADDR_EXP= R) */ > + =A0if (tcode !=3D ARRAY_REF > + =A0 =A0 =A0&& tcode !=3D VAR_DECL > + =A0 =A0 =A0&& tcode !=3D COMPONENT_REF > + =A0 =A0 =A0&& tcode !=3D INDIRECT_REF > + =A0 =A0 =A0&& tcode !=3D MEM_REF) > + =A0 =A0return; > + > + =A0mop.is_call =3D 0; > + =A0mop.gsi =3D gsi; > + =A0mop.expr =3D expr; > + =A0mop.dtor_vptr_expr =3D is_dtor_vptr_store (stmt, expr, is_store); > + =A0mop.is_store =3D is_store; > + =A0VEC_safe_push (mop_desc_t, heap, *mop_list, &mop); > +} > + > +static void > +handle_gimple (gimple_stmt_iterator gsi, VEC (mop_desc_t, heap) **mop_li= st) > +{ > + =A0unsigned i; > + =A0struct mop_desc_t mop; > + =A0gimple stmt; > + =A0enum gimple_code gcode; > + =A0location_t loc; > + =A0tree rhs; > + =A0tree lhs; > + > + =A0stmt =3D gsi_stmt (gsi); > + =A0gcode =3D gimple_code (stmt); > + =A0if (gcode >=3D LAST_AND_UNUSED_GIMPLE_CODE) > + =A0 =A0return; > + > + =A0loc =3D gimple_location (stmt); > + > + =A0switch (gcode) > + =A0 =A0{ > + =A0 =A0 =A0/* TODO (dvyukov): handle GIMPLE_COND (can it access memmory= ?) */ > + =A0 =A0 =A0case GIMPLE_CALL: > + =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0func_calls +=3D 1; > + =A0 =A0 =A0 =A0 =A0/* Handle call arguments as loads */ > + =A0 =A0 =A0 =A0 =A0for (i =3D 0; i < gimple_call_num_args (stmt); i++) > + =A0 =A0 =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0rhs =3D gimple_call_arg (stmt, i); > + =A0 =A0 =A0 =A0 =A0 =A0 =A0handle_expr (stmt, gsi, rhs, 0, mop_list); > + =A0 =A0 =A0 =A0 =A0 =A0} > + > + =A0 =A0 =A0 =A0 =A0memset (&mop, 0, sizeof (mop)); > + =A0 =A0 =A0 =A0 =A0mop.is_call =3D 1; > + =A0 =A0 =A0 =A0 =A0VEC_safe_push (mop_desc_t, heap, *mop_list, &mop); > + > + =A0 =A0 =A0 =A0 =A0/* Handle assignment lhs as store */ > + =A0 =A0 =A0 =A0 =A0lhs =3D gimple_call_lhs (stmt); > + =A0 =A0 =A0 =A0 =A0if (lhs !=3D 0) > + =A0 =A0 =A0 =A0 =A0 =A0handle_expr (stmt, gsi, lhs, 1, mop_list); > + > + =A0 =A0 =A0 =A0 =A0break; > + =A0 =A0 =A0 =A0} > + > + =A0 =A0 =A0case GIMPLE_ASSIGN: > + =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0/* Handle assignment lhs as store */ > + =A0 =A0 =A0 =A0 =A0lhs =3D gimple_assign_lhs (stmt); > + =A0 =A0 =A0 =A0 =A0handle_expr (stmt, gsi, lhs, 1, mop_list); > + > + =A0 =A0 =A0 =A0 =A0/* Handle operands as loads */ > + =A0 =A0 =A0 =A0 =A0for (i =3D 1; i < gimple_num_ops (stmt); i++) > + =A0 =A0 =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0rhs =3D gimple_op (stmt, i); > + =A0 =A0 =A0 =A0 =A0 =A0 =A0handle_expr (stmt, gsi, rhs, 0, mop_list); > + =A0 =A0 =A0 =A0 =A0 =A0} > + =A0 =A0 =A0 =A0 =A0break; > + =A0 =A0 =A0 =A0} > + > + =A0 =A0 =A0case GIMPLE_BIND: > + =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0gcc_assert (!"there should be no GIMPLE_BIND on this= level"); > + =A0 =A0 =A0 =A0 =A0break; > + =A0 =A0 =A0 =A0} > + > + =A0 =A0 =A0default: > + =A0 =A0 =A0 =A0break; > + =A0 =A0} > +} > + > +/* Instruments single basic block. =A0*/ > +static void > +instrument_bblock (struct bb_data_t *bbd, basic_block bb) > +{ > + =A0int ix; > + =A0int is_sblock; > + =A0gimple_stmt_iterator gsi; > + =A0struct mop_desc_t *mop; > + =A0gimple stmt; > + =A0location_t loc; > + =A0expanded_location eloc; > + =A0gimple_seq instr_seq; > + > + =A0/* Iterate over all gimples and collect interesting mops into mop_li= st. =A0*/ > + =A0VEC_free (mop_desc_t, heap, mop_list); > + =A0for (gsi =3D gsi_start_bb (bb); !gsi_end_p (gsi); gsi_next (&gsi)) > + =A0 =A0{ > + =A0 =A0 =A0handle_gimple (gsi, &mop_list); > + =A0 =A0} > + > + =A0mop =3D 0; > + =A0for (ix =3D 0; VEC_iterate (mop_desc_t, mop_list, ix, mop); ix +=3D = 1) > + =A0 =A0{ > + =A0 =A0 =A0if (mop->is_call !=3D 0) > + =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0/* After a function call we must start a brand new s= block, > + =A0 =A0 =A0 =A0 =A0 =A0 because the function can contain synchronizatio= n. =A0*/ > + =A0 =A0 =A0 =A0 =A0bbd->has_sb =3D 0; > + =A0 =A0 =A0 =A0 =A0continue; > + =A0 =A0 =A0 =A0} > + > + =A0 =A0 =A0func_mops +=3D 1; > + =A0 =A0 =A0stmt =3D gsi_stmt (mop->gsi); > + =A0 =A0 =A0loc =3D gimple_location (stmt); > + =A0 =A0 =A0eloc =3D expand_location (loc); > + > + =A0 =A0 =A0/* Check as to whether we may not set sblock flag > + =A0 =A0 =A0 =A0 for the access */ > + =A0 =A0 =A0is_sblock =3D (bbd->has_sb =3D=3D 0 > + =A0 =A0 =A0 =A0 =A0|| !(eloc.file !=3D 0 > + =A0 =A0 =A0 =A0 =A0 =A0 =A0&& bbd->sb_file !=3D 0 > + =A0 =A0 =A0 =A0 =A0 =A0 =A0&& strcmp (eloc.file, bbd->sb_file) =3D=3D 0 > + =A0 =A0 =A0 =A0 =A0 =A0 =A0&& eloc.line >=3D bbd->sb_line_min > + =A0 =A0 =A0 =A0 =A0 =A0 =A0&& eloc.line <=3D bbd->sb_line_max)); > + > + =A0 =A0 =A0if (func_ignore =3D=3D tsan_ignore_hist) > + =A0 =A0 =A0 =A0is_sblock =3D 0; > + > + =A0 =A0 =A0if (is_sblock) > + =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0/* Start new sblock with new source info. =A0*/ > + =A0 =A0 =A0 =A0 =A0bbd->has_sb =3D 1; > + =A0 =A0 =A0 =A0 =A0bbd->sb_file =3D eloc.file; > + =A0 =A0 =A0 =A0 =A0bbd->sb_line_min =3D eloc.line; > + =A0 =A0 =A0 =A0 =A0bbd->sb_line_max =3D eloc.line + SBLOCK_SIZE; > + =A0 =A0 =A0 =A0} > + > + =A0 =A0 =A0instr_seq =3D 0; > + =A0 =A0 =A0if (mop->dtor_vptr_expr =3D=3D 0) > + =A0 =A0 =A0 =A0instr_mop (mop->expr, mop->is_store, is_sblock, &instr_s= eq); > + =A0 =A0 =A0else > + =A0 =A0 =A0 =A0instr_vptr_store (mop->expr, mop->dtor_vptr_expr, is_sbl= ock, &instr_seq); > + =A0 =A0 =A0gcc_assert (instr_seq !=3D 0); > + =A0 =A0 =A0set_location (instr_seq, loc); > + =A0 =A0 =A0/* Instrumentation for assignment of a function result > + =A0 =A0 =A0 =A0 must be inserted after the call. =A0Instrumentation for > + =A0 =A0 =A0 =A0 reads of function arguments must be inserted before the= call. > + =A0 =A0 =A0 =A0 That's because the call can contain synchronization. = =A0*/ > + =A0 =A0 =A0if (is_gimple_call (stmt) && mop->is_store =3D=3D 1) > + =A0 =A0 =A0 =A0gsi_insert_seq_after (&mop->gsi, instr_seq, GSI_NEW_STMT= ); > + =A0 =A0 =A0else > + =A0 =A0 =A0 =A0gsi_insert_seq_before (&mop->gsi, instr_seq, GSI_SAME_ST= MT); > + =A0 =A0} > +} > + > +/* Instruments all interesting memory accesses in the function */ > +static void > +instrument_mops (void) > +{ > + =A0int sb_line_min; > + =A0int sb_line_max; > + =A0int bb_cnt; > + =A0int eidx; > + =A0basic_block bb; > + =A0basic_block entry_bb; > + =A0basic_block cur_bb; > + =A0basic_block any_bb; > + =A0struct bb_data_t *pred; > + =A0struct bb_data_t *succ; > + =A0struct bb_data_t *bb_data; > + =A0struct bb_data_t *bbd; > + =A0edge entry_edge; > + =A0edge e; > + > + =A0/* The function does breadth-first traversal of CFG. > + =A0 =A0 BB is visited preferably if all its predecessors are visited. > + =A0 =A0 Such order is required to properly mark super-blocks. > + =A0 =A0 The idea behind super-blocks is as follows. > + =A0 =A0 If several memory accesses happen within SBLOCK_SIZE source cod= e lines > + =A0 =A0 from each other, then we only mark the first access as SBLOCK. > + =A0 =A0 This allows runtime library to memorize stack trace > + =A0 =A0 only for the first access and do not memorize for others. > + =A0 =A0 This significantly reduces memory consumption in exchange for s= lightly > + =A0 =A0 imprecise stack traces for previous accesses. =A0*/ > + > + =A0/* First, mark all blocks as not visited, and entry block as candida= te. =A0*/ > + =A0bb_cnt =3D cfun->cfg->x_n_basic_blocks; > + =A0bb_data =3D (struct bb_data_t*) xcalloc (bb_cnt, sizeof (struct bb_d= ata_t)); > + =A0entry_bb =3D ENTRY_BLOCK_PTR; > + =A0entry_edge =3D single_succ_edge (entry_bb); > + =A0entry_bb =3D entry_edge->dest; > + =A0bb =3D 0; > + =A0FOR_EACH_BB (bb) > + =A0 =A0{ > + =A0 =A0 =A0bb_data [bb->index].state =3D (bb =3D=3D entry_bb) ? bb_cand= idate : bb_not_visited; > + =A0 =A0} > + > + =A0/* Until all blocks are visited. =A0*/ > + =A0for (; ; ) > + =A0 =A0{ > + =A0 =A0 =A0cur_bb =3D 0; > + =A0 =A0 =A0any_bb =3D 0; > + =A0 =A0 =A0/* Look for a candidate with all visited predecessors. =A0*/ > + =A0 =A0 =A0FOR_EACH_BB (bb) > + =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0bbd =3D &bb_data [bb->index]; > + =A0 =A0 =A0 =A0 =A0if (bbd->state =3D=3D bb_candidate) > + =A0 =A0 =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0cur_bb =3D bb; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0any_bb =3D bb; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0e =3D 0; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0for (eidx =3D 0; VEC_iterate (edge, bb->pred= s, eidx, e); eidx++) > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0pred =3D &bb_data [e->src->index]; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if (pred->state !=3D bb_visited) > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0cur_bb =3D 0; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0break; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0} > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0} > + =A0 =A0 =A0 =A0 =A0 =A0} > + =A0 =A0 =A0 =A0 =A0if (cur_bb !=3D 0) > + =A0 =A0 =A0 =A0 =A0 =A0break; > + =A0 =A0 =A0 =A0} > + =A0 =A0 =A0/* All blocks are visited. =A0*/ > + =A0 =A0 =A0if (any_bb =3D=3D 0) > + =A0 =A0 =A0 =A0break; > + =A0 =A0 =A0/* If no blocks with all visited predecessors, choose any ca= ndidate. > + =A0 =A0 =A0 =A0 Must be a loop. =A0*/ > + =A0 =A0 =A0cur_bb =3D cur_bb ? cur_bb : any_bb; > + =A0 =A0 =A0bbd =3D &bb_data [cur_bb->index]; > + =A0 =A0 =A0gcc_assert (bbd->state =3D=3D bb_candidate); > + =A0 =A0 =A0bbd->state =3D bb_visited; > + > + =A0 =A0 =A0/* Iterate over all predecessors and merge their sblock info= . =A0*/ > + =A0 =A0 =A0e =3D 0; > + =A0 =A0 =A0for (eidx =3D 0; VEC_iterate (edge, cur_bb->preds, eidx, e);= eidx++) > + =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0pred =3D &bb_data [e->src->index]; > + =A0 =A0 =A0 =A0 =A0if ((pred->state !=3D bb_visited) > + =A0 =A0 =A0 =A0 =A0 =A0 =A0|| (pred->has_sb =3D=3D 0) > + =A0 =A0 =A0 =A0 =A0 =A0 =A0|| (pred =3D=3D bbd)) > + =A0 =A0 =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0/* If there is a not visited predecessor, > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 or a predecessor with no active sblock = info, > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 or a self-loop, then we will have to st= art > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 a brand new sblock on next memory acces= s. =A0*/ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0bbd->has_sb =3D 0; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0break; > + =A0 =A0 =A0 =A0 =A0 =A0} > + =A0 =A0 =A0 =A0 =A0else if (bbd->has_sb =3D=3D 0) > + =A0 =A0 =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0/* If it's a first predecessor, just copy th= e info. =A0*/ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0bbd->has_sb =3D 1; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0bbd->sb_file =3D pred->sb_file; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0bbd->sb_line_min =3D pred->sb_line_min; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0bbd->sb_line_max =3D pred->sb_line_max; > + =A0 =A0 =A0 =A0 =A0 =A0} > + =A0 =A0 =A0 =A0 =A0else > + =A0 =A0 =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0/* Otherwise, find the interception > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 between two sblock descriptors. =A0*/ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0bbd->has_sb =3D 0; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0if (bbd->sb_file !=3D 0 && pred->sb_file != =3D 0 > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0&& strcmp (bbd->sb_file, pred->sb_fi= le) =3D=3D 0) > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0sb_line_min =3D MAX (bbd->sb_line_mi= n, pred->sb_line_min); > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0sb_line_max =3D MIN (bbd->sb_line_ma= x, pred->sb_line_max); > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if (sb_line_min <=3D sb_line_max) > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0bbd->has_sb =3D 1; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0bbd->sb_line_min =3D sb_line= _min; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0bbd->sb_line_max =3D sb_line= _max; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0} > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0} > + =A0 =A0 =A0 =A0 =A0 =A0 =A0/* No interception, have to start new sblock= . =A0*/ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0if (bbd->has_sb =3D=3D 0) > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0break; > + =A0 =A0 =A0 =A0 =A0 =A0} > + =A0 =A0 =A0 =A0} > + > + =A0 =A0 =A0/* Finally, instrument the block. =A0*/ > + =A0 =A0 =A0instrument_bblock (bbd, cur_bb); > + > + =A0 =A0 =A0/* Mark all successors as candidates. =A0*/ > + =A0 =A0 =A0for (eidx =3D 0; VEC_iterate (edge, cur_bb->succs, eidx, e);= eidx++) > + =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0succ =3D &bb_data [e->dest->index]; > + =A0 =A0 =A0 =A0 =A0if (succ->state =3D=3D bb_not_visited) > + =A0 =A0 =A0 =A0 =A0 =A0succ->state =3D bb_candidate; > + =A0 =A0 =A0 =A0} > + =A0 =A0} > +} > + > +/* Instruments function entry and exit, if necessary. =A0*/ > +static void > +instrument_function (void) > +{ > + =A0location_t loc; > + =A0gimple_seq pre_func_seq; > + =A0gimple_seq post_func_seq; > + =A0basic_block entry_bb; > + =A0basic_block first_bb; > + =A0basic_block bb; > + =A0edge entry_edge; > + =A0gimple_stmt_iterator first_gsi; > + =A0gimple_stmt_iterator gsi; > + =A0gimple_stmt_iterator gsi2; > + =A0gimple first_stmt; > + =A0gimple stmt; > + > + =A0pre_func_seq =3D 0; > + =A0post_func_seq =3D 0; > + =A0instr_func (&pre_func_seq, &post_func_seq); > + > + =A0if (pre_func_seq !=3D 0) > + =A0 =A0{ > + =A0 =A0 =A0/* Insert new BB before the first BB. =A0*/ > + =A0 =A0 =A0entry_bb =3D ENTRY_BLOCK_PTR; > + =A0 =A0 =A0entry_edge =3D single_succ_edge (entry_bb); > + =A0 =A0 =A0first_bb =3D entry_edge->dest; > + =A0 =A0 =A0first_gsi =3D gsi_start_bb (first_bb); > + =A0 =A0 =A0if (!gsi_end_p (first_gsi)) > + =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0first_stmt =3D gsi_stmt (first_gsi); > + =A0 =A0 =A0 =A0 =A0loc =3D gimple_location (first_stmt); > + =A0 =A0 =A0 =A0 =A0set_location (pre_func_seq, loc); > + =A0 =A0 =A0 =A0} > + =A0 =A0 =A0entry_bb =3D split_edge (entry_edge); > + =A0 =A0 =A0gsi =3D gsi_start_bb (entry_bb); > + =A0 =A0 =A0gsi_insert_seq_after (&gsi, pre_func_seq, GSI_NEW_STMT); > + =A0 =A0} > + > + =A0if (post_func_seq !=3D 0) > + =A0 =A0{ > + =A0 =A0 =A0/* Find all function exits. =A0*/ > + =A0 =A0 =A0FOR_EACH_BB (bb) > + =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0gsi2 =3D gsi_start_bb (bb); > + =A0 =A0 =A0 =A0 =A0for (; ; ) > + =A0 =A0 =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0gsi =3D gsi2; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0if (gsi_end_p (gsi)) > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0break; > + =A0 =A0 =A0 =A0 =A0 =A0 =A0gsi_next (&gsi2); > + > + =A0 =A0 =A0 =A0 =A0 =A0 =A0stmt =3D gsi_stmt (gsi); > + =A0 =A0 =A0 =A0 =A0 =A0 =A0loc =3D gimple_location (stmt); > + > + =A0 =A0 =A0 =A0 =A0 =A0 =A0if (gimple_code (stmt) =3D=3D GIMPLE_RETURN) > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0{ > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0set_location (post_func_seq, loc); > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0gsi_insert_seq_before (&gsi, post_fu= nc_seq, GSI_SAME_STMT); > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0} > + =A0 =A0 =A0 =A0 =A0 =A0} > + =A0 =A0 =A0 =A0} > + =A0 =A0} > +} > + > +static unsigned > +tsan_pass (void) > +{ > + =A0if (errorcount !=3D 0 || sorrycount !=3D 0) > + =A0 =A0return 0; > + > + =A0func_ignore =3D tsan_ignore (); > + =A0if (func_ignore =3D=3D tsan_ignore_func) > + =A0 =A0return 0; > + > + =A0func_calls =3D 0; > + =A0func_mops =3D 0; > + > + =A0instrument_mops (); > + =A0instrument_function (); > + > + =A0return 0; > +} > + > +static bool > +tsan_gate (void) > +{ > + =A0return flag_tsan !=3D 0; > +} > + > +struct gimple_opt_pass pass_tsan =3D {{ > + =A0GIMPLE_PASS, > + =A0"tsan", =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = /* name */ > + =A0tsan_gate, =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/*= gate */ > + =A0tsan_pass, =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/*= execute */ > + =A0NULL, =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 /* sub */ > + =A0NULL, =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 /* next */ > + =A00, =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0/* static_pass_number */ > + =A0TV_NONE, =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= /* tv_id */ > + =A0PROP_trees | PROP_cfg, =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/* properties_= required */ > + =A00, =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0/* properties_provided */ > + =A00, =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0/* properties_destroyed */ > + =A00, =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0/* todo_flags_start */ > + =A0TODO_dump_cgraph | TODO_dump_func | TODO_verify_all > + =A0 =A0| TODO_update_ssa | TODO_update_address_taken /* todo_flags_fini= sh */ > +}}; > + > > Property changes on: gcc/tree-tsan.c > ___________________________________________________________________ > Added: svn:eol-style > =A0 + LF > > Index: gcc/tree-pass.h > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- gcc/tree-pass.h =A0 =A0 (revision 180522) > +++ gcc/tree-pass.h =A0 =A0 (working copy) > @@ -352,6 +352,7 @@ > > =A0extern struct gimple_opt_pass pass_mudflap_1; > =A0extern struct gimple_opt_pass pass_mudflap_2; > +extern struct gimple_opt_pass pass_tsan; > =A0extern struct gimple_opt_pass pass_lower_cf; > =A0extern struct gimple_opt_pass pass_refactor_eh; > =A0extern struct gimple_opt_pass pass_lower_eh; > Index: gcc/testsuite/gcc.dg/tsan-ignore.ignore > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- gcc/testsuite/gcc.dg/tsan-ignore.ignore =A0 =A0 (revision 0) > +++ gcc/testsuite/gcc.dg/tsan-ignore.ignore =A0 =A0 (revision 0) > @@ -0,0 +1,7 @@ > +#comment > +fun:foo > +fun:*bar > +fun:baz* > +fun:*bla* > +fun:x*x > +src:tsan-ignore.h > Index: gcc/testsuite/gcc.dg/tsan.h > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- gcc/testsuite/gcc.dg/tsan.h (revision 0) > +++ gcc/testsuite/gcc.dg/tsan.h (revision 0) > @@ -0,0 +1,87 @@ > +/* Helper declarations and functions for ThreadSanitizer instrumentation= (-ftsan) testing */ > + > +int printf (char *str, ...); > +void exit (int); > + > +/* Variables referenced by the instrumentation */ > +__thread void **__tsan_shadow_stack; > +__thread int __tsan_thread_ignore; > + > +/* Local helper vars */ > +__thread void *shadow_stack[1024]; > +__thread int mop_expect; > +__thread int mop_depth; > +__thread void* mop_addr; > +__thread unsigned long long mop_pc; > +__thread unsigned mop_flags; > + > +/* Setups shadow stack var (not instrumented) */ > +void __attribute__ ((constructor)) > +__tsan_init (void) > +{ > + =A0__tsan_shadow_stack =3D shadow_stack; > +} > + > +/* Declare that we expect an instrumented memory access (not instrumente= d). > + =A0 depth - stack depth of the mop (0 - main, 1 - func called from main= and so on). > + =A0 addr - memory access address. > + =A0 is_store - store/load. > + =A0 is_sblock - superblock flag of the access. > + =A0 size - memory access size in bytes. */ > +void > +__tsan_expect_mop (int depth, void *addr, int is_store, int is_sblock, i= nt size) > +{ > + =A0if (mop_expect) > + =A0 =A0{ > + =A0 =A0 =A0printf ("missed mop: addr=3D%p pc=3D%d\n", mop_addr, mop_pc); > + =A0 =A0 =A0exit (1); > + =A0 =A0} > + > + =A0mop_expect =3D 1; > + =A0mop_depth =3D depth; > + =A0mop_addr =3D addr; > + =A0mop_pc =3D (unsigned long long)__builtin_return_address(0); > + =A0mop_flags =3D !!is_sblock | (!!is_store << 1) | ((size - 1) << 2); > +} > + > +/* Memory access function (referenced by instrumentation, not instrument= ed). */ > +void > +__tsan_handle_mop (void *addr, unsigned flags) > +{ > + =A0unsigned long long pc; > + =A0int depth; > + > + =A0printf ("mop: addr=3D%p flags=3D%x called from %p/%p\n", addr, flags= , __builtin_return_address(1), __tsan_shadow_stack[-2]); > + =A0if (mop_expect =3D=3D 0) > + =A0 =A0return; > + > + =A0/* Verify parameters with what we expect. */ > + > + =A0if (addr !=3D mop_addr) > + =A0 =A0{ > + =A0 =A0 =A0printf ("incorrect mop addr: %p/%p\n", addr, =A0mop_addr); > + =A0 =A0 =A0exit (1); > + =A0 =A0} > + > + =A0pc =3D (unsigned long long)__builtin_return_address(0); > + =A0if (pc < mop_pc - 100 || pc > mop_pc + 100) > + =A0 =A0{ > + =A0 =A0 =A0printf ("incorrect mop pc: %p/%p\n", (void*)pc, (void*)mop_p= c); > + =A0 =A0 =A0exit (1); > + =A0 =A0} > + > + =A0depth =3D __tsan_shadow_stack - shadow_stack - 1; > + =A0if (depth !=3D mop_depth) > + =A0 =A0{ > + =A0 =A0 =A0printf ("incorrect mop depth: %d/%d\n", depth, mop_depth); > + =A0 =A0 =A0exit (1); > + =A0 =A0} > + > + =A0if (flags !=3D mop_flags) > + =A0 =A0{ > + =A0 =A0 =A0printf ("incorrect mop flags: %x/%x\n", flags, mop_flags); > + =A0 =A0 =A0exit (1); > + =A0 =A0} > + > + =A0mop_expect =3D 0; > +} > > Property changes on: gcc/testsuite/gcc.dg/tsan.h > ___________________________________________________________________ > Added: svn:eol-style > =A0 + LF > > Index: gcc/testsuite/gcc.dg/tsan-ignore.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- gcc/testsuite/gcc.dg/tsan-ignore.c =A0(revision 0) > +++ gcc/testsuite/gcc.dg/tsan-ignore.c =A0(revision 0) > @@ -0,0 +1,49 @@ > +/* { dg-do run } */ > +/* { dg-options "-ftsan -ftsan-ignore=3Dtsan-ignore.ignore" } */ > +#include "tsan.h" > +#include "tsan-ignore.h" > + > +/* Check ignore file handling. */ > + > +int > +foo (int *p) > +{ > + =A0p [0] =3D 1; > +} > + > +int bar (int *p) > +{ > + =A0p [0] =3D 1; > +} > + > +int baz (int *p) > +{ > + =A0p [0] =3D 1; > +} > + > +int bla (int *p) > +{ > + =A0p [0] =3D 1; > +} > + > +int xxx (int *p) > +{ > + =A0p [0] =3D 1; > +} > + > +int > +main (void) > +{ > + =A0int p, x; > + > + =A0__tsan_expect_mop(0, &p, 1, 1, sizeof(p)); > + =A0/* All these functions must be ignored. */ > + =A0foo (&x); > + =A0bar (&x); > + =A0baz (&x); > + =A0bla (&x); > + =A0xxx (&x); > + =A0in_tsan_ignore_header (&x); > + =A0p =3D 0; > + =A0return 0; > +} > > Property changes on: gcc/testsuite/gcc.dg/tsan-ignore.c > ___________________________________________________________________ > Added: svn:eol-style > =A0 + LF > > Index: gcc/testsuite/gcc.dg/tsan-ignore.h > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- gcc/testsuite/gcc.dg/tsan-ignore.h =A0(revision 0) > +++ gcc/testsuite/gcc.dg/tsan-ignore.h =A0(revision 0) > @@ -0,0 +1,5 @@ > +int > +in_tsan_ignore_header (int *p) > +{ > + =A0p [0] =3D 1; > +} > > Property changes on: gcc/testsuite/gcc.dg/tsan-ignore.h > ___________________________________________________________________ > Added: svn:eol-style > =A0 + LF > > Index: gcc/testsuite/gcc.dg/tsan-stack.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- gcc/testsuite/gcc.dg/tsan-stack.c =A0 (revision 0) > +++ gcc/testsuite/gcc.dg/tsan-stack.c =A0 (revision 0) > @@ -0,0 +1,23 @@ > +/* { dg-do run } */ > +/* { dg-options "-ftsan" } */ > +#include "tsan.h" > + > +/* Check shadow stack maintance. */ > + > +int > +foobar (int *p) > +{ > + =A0__tsan_expect_mop(1, p, 1, 1, sizeof(*p)); > + =A0p[0] =3D 1; > +} > + > +int > +main (void) > +{ > + =A0int p; > + > + =A0__tsan_expect_mop(0, &p, 1, 1, sizeof(p)); > + =A0p =3D 0; > + =A0foobar (&p); > + =A0return 0; > +} > > Property changes on: gcc/testsuite/gcc.dg/tsan-stack.c > ___________________________________________________________________ > Added: svn:eol-style > =A0 + LF > > Index: gcc/testsuite/gcc.dg/tsan-mop.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- gcc/testsuite/gcc.dg/tsan-mop.c =A0 =A0 (revision 0) > +++ gcc/testsuite/gcc.dg/tsan-mop.c =A0 =A0 (revision 0) > @@ -0,0 +1,29 @@ > +/* { dg-do run } */ > +/* { dg-options "-ftsan" } */ > +#include "tsan.h" > + > +/* Sanity check for memory accesses instrumentation. */ > + > +int > +foobar (int *p) > +{ > + =A0__tsan_expect_mop(1, p, 1, 1, sizeof(*p)); > + =A0p[0] =3D 1; > + > + =A0__tsan_expect_mop(1, p, 1, 1, sizeof(*p)); > + =A0*p =3D 2; > + > + =A0__tsan_expect_mop(1, (char*)p+3, 1, 1, 1); > + =A0*((char*)p+3) =3D 3; > +} > + > +int > +main (void) > +{ > + =A0int p; > + > + =A0__tsan_expect_mop(0, &p, 1, 1, sizeof(p)); > + =A0p =3D 0; > + =A0foobar (&p); > + =A0return 0; > +} > > Property changes on: gcc/testsuite/gcc.dg/tsan-mop.c > ___________________________________________________________________ > Added: svn:eol-style > =A0 + LF > > Index: gcc/common.opt > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- gcc/common.opt =A0 =A0 =A0(revision 180522) > +++ gcc/common.opt =A0 =A0 =A0(working copy) > @@ -1547,6 +1547,14 @@ > =A0Common RejectNegative Report Var(flag_mudflap_ignore_reads) > =A0Ignore read operations when inserting mudflap instrumentation > > +ftsan > +Common RejectNegative Report Var(flag_tsan) > +Add ThreadSanitizer instrumentation > + > +ftsan-ignore=3D > +Common RejectNegative Joined Var(flag_tsan_ignore) > +-ftsan-ignore=3Dfilename ThreadSanitizer ignore file > + > =A0fdce > =A0Common Var(flag_dce) Init(1) Optimization > =A0Use the RTL dead code elimination pass > Index: gcc/Makefile.in > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- gcc/Makefile.in =A0 =A0 (revision 180522) > +++ gcc/Makefile.in =A0 =A0 (working copy) > @@ -1494,6 +1494,7 @@ > =A0 =A0 =A0 =A0tree-streamer-out.o \ > =A0 =A0 =A0 =A0tree-tailcall.o \ > =A0 =A0 =A0 =A0tree-threadsafe-analyze.o \ > + =A0 =A0 =A0 tree-tsan.o \ > =A0 =A0 =A0 =A0tree-vect-generic.o \ > =A0 =A0 =A0 =A0tree-vect-patterns.o \ > =A0 =A0 =A0 =A0tree-vect-data-refs.o \ > @@ -2814,6 +2815,12 @@ > =A0 =A0$(C_TREE_H) $(C_COMMON_H) $(GIMPLE_H) $(DIAGNOSTIC_H) $(HASHTAB_H)= \ > =A0 =A0output.h langhooks.h tree-mudflap.h $(TM_H) coretypes.h \ > =A0 =A0$(GGC_H) gt-tree-mudflap.h $(TREE_PASS_H) $(DIAGNOSTIC_CORE_H) > +tree-tsan.o : $(CONFIG_H) $(SYSTEM_H) $(TREE_H) $(TREE_INLINE_H) \ > + =A0 $(GIMPLE_H) $(DIAGNOSTIC_H) langhooks.h \ > + =A0 $(TM_H) coretypes.h $(TREE_DUMP_H) $(TREE_PASS_H) $(CGRAPH_H) $(GGC= _H) \ > + =A0 $(BASIC_BLOCK_H) $(FLAGS_H) $(FUNCTION_H) \ > + =A0 $(TM_P_H) $(TREE_FLOW_H) $(DIAGNOSTIC_CORE_H) $(GIMPLE_H) tree-iter= ator.h \ > + =A0 intl.h cfghooks.h output.h options.h c-family/c-common.h > =A0tree-pretty-print.o : tree-pretty-print.c $(CONFIG_H) $(SYSTEM_H) \ > =A0 =A0$(TREE_H) $(DIAGNOSTIC_H) $(HASHTAB_H) $(TREE_FLOW_H) \ > =A0 =A0$(TM_H) coretypes.h tree-iterator.h $(SCEV_H) langhooks.h \ > Index: gcc/passes.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- gcc/passes.c =A0 =A0 =A0 =A0(revision 180522) > +++ gcc/passes.c =A0 =A0 =A0 =A0(working copy) > @@ -1420,6 +1420,7 @@ > =A0 NEXT_PASS (pass_lower_resx); > =A0 NEXT_PASS (pass_nrv); > =A0 NEXT_PASS (pass_mudflap_2); > + =A0NEXT_PASS (pass_tsan); > =A0 NEXT_PASS (pass_cleanup_cfg_post_optimizing); > =A0 NEXT_PASS (pass_warn_function_noreturn); > > > -- > This patch is available for review at http://codereview.appspot.com/53030= 83 > > -- > You received this message because you are subscribed to the Google Groups= "C-compiler-team" group. > To post to this group, send email to c-compiler-team@google.com. > To unsubscribe from this group, send email to c-compiler-team+unsubscribe= @google.com. > For more options, visit this group at http://groups.google.com/a/google.c= om/group/c-compiler-team/?hl=3Den. > >