* struct-reorg optimization
@ 2007-07-10 9:35 Olga Golovanevsky
2007-07-19 22:57 ` Daniel Berlin
2007-07-20 14:39 ` Jan Hubicka
0 siblings, 2 replies; 17+ messages in thread
From: Olga Golovanevsky @ 2007-07-10 9:35 UTC (permalink / raw)
To: Daniel Berlin, Diego Novillo; +Cc: Kenneth Zadeck, Jan Hubicka, gcc-patches
[-- Attachment #1: Type: text/plain, Size: 2781 bytes --]
This patch implements structure peeling, that is one of struct-reorg
optimizations developed on struct-reorg-branch. A structure type can be
peeled into separate fields, or groups of them, according to profile
information, or deliberately. The gain is to utilize spatial locality,
when, for example, one of the fields is sequentially accessed through
array elements. Then separating this field from others allows to make
separation in array allocation, and thus release cache from irrelevant
data fetched with other fields.
The full description of this optimization is at the beginning of
struct-reorg.c (attached below).
The optimization is global, both types of accesses - through pointers
and array refs - are handled. When allocation is made through malloc,
it is replaced by allocations of peeled structures. Only build-in malloc
is supported right now, but it can be easily extended.
In this patch, the decision how to peel is made by frequency based model,
where frequency of field accesses can be calculated through profiling or
statically predicted as with -fbranch-probabilitied flag. When there is
no such info, structure gets completely peeled. (The struct-reorg-branch
contains much more accurate model which is based on the calculation of
distance between accesses, but it's also heavier.)
The -fipa-struct-reorg flag activates this optimization.
The patch makes strong use of ipa-type-escape analysis, that provide
safety of this optimization. Also changes required for ipa-type-escape
to support POITER_PLUS_EXPR are part of this patch.
With this patch:
- two structures f1_neuron and xyz got peeled in 179. art ,
+44% w/o profiling and +33% with profiling
- the structures NEXT_MOVE got peeled, the structure CHESS_POSITION
can be peeled, but interfere with another optimization that
generated BIT_FIELD_REFs in 186.crafty; many other structures escape,
no influence on performance
- the structure MMNODE got peeled; many others escape,
no influence on performance
Bootstraped and tested (except fortran) on ppc .
:ADDPATCH ipa ssa:
Ok for mainline?
2007-07-09 Olga Golovanevsky <olga@il.ibm.com>
* struct-reorg.c, struct-reorg.h: New files.
* ipa-type-escape.h: Expose function
is_array_access_through_pointer_and_index.
* ipa-type-escape.c
(is_array_access_through_pointer_and_index):
Add three new parameters. Add support of
POINTER_PLUS_EXPR code.
* tree.h: Add pass_ipa_struct_reorg.
* common.opt: Add ipa-struct-reorg flag.
* Makefile.in: Add strcut-reorg.o compilation.
* passes.c: Add pass pass_ipa_struct_reorg.
(See attached file: struct-reorg.txt)(See attached file: struct-reorg.c)
(See attached file: struct-reorg.h)
[-- Attachment #2: struct-reorg.txt --]
[-- Type: text/plain, Size: 8217 bytes --]
Index: tree-pass.h
===================================================================
--- tree-pass.h (revision 126190)
+++ tree-pass.h (working copy)
@@ -331,6 +331,7 @@
extern struct tree_opt_pass pass_ipa_pure_const;
extern struct tree_opt_pass pass_ipa_type_escape;
extern struct tree_opt_pass pass_ipa_pta;
+extern struct tree_opt_pass pass_ipa_struct_reorg;
extern struct tree_opt_pass pass_early_local_passes;
extern struct tree_opt_pass pass_ipa_increase_alignment;
extern struct tree_opt_pass pass_ipa_function_and_variable_visibility;
Index: ipa-type-escape.c
===================================================================
--- ipa-type-escape.c (revision 126190)
+++ ipa-type-escape.c (working copy)
@@ -926,56 +926,90 @@
*/
-static bool
-is_array_access_through_pointer_and_index (tree op0, tree op1)
+bool
+is_array_access_through_pointer_and_index (enum tree_code code, tree op0, tree op1,
+ tree * base, tree * offset,
+ tree * offset_cast_stmt)
{
- tree base, offset, offset_cast_stmt;
tree before_cast, before_cast_def_stmt;
cast_t op0_cast, op1_cast;
+ *base = NULL;
+ *offset = NULL;
+ *offset_cast_stmt = NULL;
+
/* Check 1. */
- /* Init data for walk_use_def_chains function. */
- op0_cast.type = op1_cast.type = 0;
- op0_cast.stmt = op1_cast.stmt = NULL;
+ if (code == POINTER_PLUS_EXPR)
+ {
+ tree op0type = TYPE_MAIN_VARIANT (TREE_TYPE (op0));
+ tree op1type = TYPE_MAIN_VARIANT (TREE_TYPE (op1));
- visited_stmts = pointer_set_create ();
- walk_use_def_chains (op0, is_cast_from_non_pointer,(void *)(&op0_cast), false);
- pointer_set_destroy (visited_stmts);
+ /* One of op0 and op1 is of pointer type and the other is numerical. */
+ if (POINTER_TYPE_P (op0type)
+ && NUMERICAL_TYPE_CHECK (op1type))
+ {
+ *base = op0;
+ *offset = op1;
+ }
+ else if (POINTER_TYPE_P (op1type)
+ && NUMERICAL_TYPE_CHECK (op0type))
+ {
+ *base = op1;
+ *offset = op0;
+ }
+ else
+ return false;
+ }
+ else
+ {
+ /* Init data for walk_use_def_chains function. */
+ op0_cast.type = op1_cast.type = 0;
+ op0_cast.stmt = op1_cast.stmt = NULL;
- visited_stmts = pointer_set_create ();
- walk_use_def_chains (op1, is_cast_from_non_pointer,(void *)(&op1_cast), false);
- pointer_set_destroy (visited_stmts);
+ visited_stmts = pointer_set_create ();
+ walk_use_def_chains (op0, is_cast_from_non_pointer,(void *)(&op0_cast), false);
+ pointer_set_destroy (visited_stmts);
- if (op0_cast.type == 1 && op1_cast.type == 0)
- {
- base = op1;
- offset = op0;
- offset_cast_stmt = op0_cast.stmt;
+ visited_stmts = pointer_set_create ();
+ walk_use_def_chains (op1, is_cast_from_non_pointer,(void *)(&op1_cast), false);
+ pointer_set_destroy (visited_stmts);
+
+ if (op0_cast.type == 1 && op1_cast.type == 0)
+ {
+ *base = op1;
+ *offset = op0;
+ *offset_cast_stmt = op0_cast.stmt;
+ }
+ else if (op0_cast.type == 0 && op1_cast.type == 1)
+ {
+ *base = op0;
+ *offset = op1;
+ *offset_cast_stmt = op1_cast.stmt;
+ }
+ else
+ return false;
}
- else if (op0_cast.type == 0 && op1_cast.type == 1)
- {
- base = op0;
- offset = op1;
- offset_cast_stmt = op1_cast.stmt;
- }
- else
- return false;
/* Check 2.
offset_cast_stmt is of the form:
D.1606_7 = (struct str_t *) D.1605_6; */
- before_cast = SINGLE_SSA_TREE_OPERAND (offset_cast_stmt, SSA_OP_USE);
- if (!before_cast)
- return false;
+ if (*offset_cast_stmt)
+ {
+ before_cast = SINGLE_SSA_TREE_OPERAND (*offset_cast_stmt, SSA_OP_USE);
+ if (!before_cast)
+ return false;
- if (SSA_NAME_IS_DEFAULT_DEF(before_cast))
- return false;
+ if (SSA_NAME_IS_DEFAULT_DEF(before_cast))
+ return false;
- before_cast_def_stmt = SSA_NAME_DEF_STMT (before_cast);
- if (!before_cast_def_stmt)
- return false;
+ before_cast_def_stmt = SSA_NAME_DEF_STMT (before_cast);
+ if (!before_cast_def_stmt)
+ return false;
+ }
+ else
+ before_cast_def_stmt = SSA_NAME_DEF_STMT (*offset);
/* before_cast_def_stmt should be of the form:
D.1605_6 = i.1_5 * 16; */
@@ -1449,7 +1483,6 @@
okay_pointer_operation (enum tree_code code, tree op0, tree op1)
{
tree op0type = TYPE_MAIN_VARIANT (TREE_TYPE (op0));
- tree op1type = TYPE_MAIN_VARIANT (TREE_TYPE (op1));
switch (code)
{
@@ -1459,11 +1492,15 @@
break;
case MINUS_EXPR:
case PLUS_EXPR:
+ case POINTER_PLUS_EXPR:
{
- if (POINTER_TYPE_P (op1type)
+ tree base, offset, offset_cast_stmt;
+
+ if (POINTER_TYPE_P (op0type)
&& TREE_CODE (op0) == SSA_NAME
&& TREE_CODE (op1) == SSA_NAME
- && is_array_access_through_pointer_and_index (op0, op1))
+ && is_array_access_through_pointer_and_index (code, op0, op1, &base,
+ &offset, &offset_cast_stmt))
return true;
else
{
@@ -1541,7 +1578,7 @@
of circumstances and if the moon is in the correct
place could be safe, but it is hard to see how this
is worth the effort. */
-
+
if (type0 && POINTER_TYPE_P (type0)
&& !okay_pointer_operation (TREE_CODE (rhs), op0, op1))
mark_interesting_type (type0, FULL_ESCAPE);
Index: ipa-type-escape.h
===================================================================
--- ipa-type-escape.h (revision 126190)
+++ ipa-type-escape.h (working copy)
@@ -27,6 +27,7 @@
bool ipa_type_escape_field_does_not_clobber_p (tree record_type, tree field_type);
int ipa_type_escape_star_count_of_interesting_type (tree type);
int ipa_type_escape_star_count_of_interesting_or_array_type (tree type);
+bool is_array_access_through_pointer_and_index (enum tree_code, tree, tree, tree *, tree *, tree *);
#endif /* GCC_IPA_TYPE_ESCAPE_H */
Index: common.opt
===================================================================
--- common.opt (revision 126190)
+++ common.opt (working copy)
@@ -601,6 +601,11 @@
Perform matrix layout flattening and transposing based
on profiling information.
+fipa-struct-reorg
+Common Report Var(flag_ipa_struct_reorg)
+Perform structure layout optimizations based
+on profiling information.
+
fivopts
Common Report Var(flag_ivopts) Init(1) Optimization
Optimize induction variables on trees
Index: Makefile.in
===================================================================
--- Makefile.in (revision 126190)
+++ Makefile.in (working copy)
@@ -1185,6 +1185,7 @@
ipa-utils.o \
ipa.o \
matrix-reorg.o \
+ struct-reorg.o \
tree-inline.o \
tree-nomudflap.o \
varpool.o
@@ -2449,6 +2450,10 @@
pointer-set.h $(GGC_H) $(IPA_TYPE_ESCAPE_H) $(IPA_UTILS_H) $(C_COMMON_H) \
$(TREE_GIMPLE_H) $(CGRAPH_H) output.h $(FLAGS_H) tree-pass.h \
$(DIAGNOSTIC_H) $(FUNCTION_H)
+struct-reorg.o: struct-reorg.c struct-reorg.h $(CONFIG_H) $(SYSTEM_H) coretypes.h \
+ $(TM_H) $(RTL_H) $(TREE_H) $(FLAGS_H) output.h $(REGS_H) $(EXPR_H) $(FUNCTION_H) \
+ toplev.h $(GGC_H) $(TARGET_H) langhooks.h $(COVERAGE_H) libfuncs.h \
+ gt-coverage.h $(HASHTAB_H) $(IPA_TYPE_ESCAPE_H)
coverage.o : coverage.c $(GCOV_IO_H) $(CONFIG_H) $(SYSTEM_H) coretypes.h \
$(TM_H) $(RTL_H) $(TREE_H) $(FLAGS_H) output.h $(REGS_H) $(EXPR_H) \
@@ -3011,7 +3016,7 @@
$(srcdir)/reload.h \
$(srcdir)/alias.c $(srcdir)/bitmap.c $(srcdir)/cselib.c $(srcdir)/cgraph.c \
$(srcdir)/ipa-prop.c $(srcdir)/ipa-cp.c $(srcdir)/ipa-inline.c $(srcdir)/matrix-reorg.c \
- $(srcdir)/dbxout.c $(srcdir)/dwarf2out.c $(srcdir)/dwarf2asm.c \
+ $(srcdir)/dbxout.c $(srcdir)/struct-reorg.c $(srcdir)/dwarf2out.c $(srcdir)/dwarf2asm.c \
$(srcdir)/dojump.c \
$(srcdir)/emit-rtl.c $(srcdir)/except.c $(srcdir)/explow.c $(srcdir)/expr.c \
$(srcdir)/function.c $(srcdir)/except.h \
Index: passes.c
===================================================================
--- passes.c (revision 126190)
+++ passes.c (working copy)
@@ -542,6 +542,7 @@
NEXT_PASS (pass_ipa_pure_const);
NEXT_PASS (pass_ipa_type_escape);
NEXT_PASS (pass_ipa_pta);
+ NEXT_PASS (pass_ipa_struct_reorg);
*p = NULL;
/* These passes are run after IPA passes on every function that is being
[-- Attachment #3: struct-reorg.c --]
[-- Type: application/octet-stream, Size: 84522 bytes --]
/* Struct-reorg optimization.
Copyright (C) 2004, 2005, 2006, 2007 Free Software Foundation, Inc.
Contributed by Olga Golovanevsky <olga@il.ibm.com>
(initially version of this code was developed
by Caroline Tice and Mostafa Hagoge)
This file is part of GCC.
GCC is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free
Software Foundation; either version 2, or (at your option) any later
version.
GCC is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
for more details.
You should have received a copy of the GNU General Public License
along with GCC; see the file COPYING. If not, write to the Free
Software Foundation, 59 Temple Place - Suite 330, Boston, MA
02111-1307, USA. */
#include "config.h"
#include "system.h"
#include "coretypes.h"
#include "tm.h"
#include "tree.h"
#include "rtl.h"
#include "tree-gimple.h"
#include "tree-inline.h"
#include "tree-flow.h"
#include "tree-flow-inline.h"
#include "langhooks.h"
#include "pointer-set.h"
#include "hashtab.h"
#include "c-tree.h"
#include "toplev.h"
#include "flags.h"
#include "ggc.h"
#include "debug.h"
#include "target.h"
#include "cgraph.h"
#include "diagnostic.h"
#include "timevar.h"
#include "params.h"
#include "fibheap.h"
#include "intl.h"
#include "function.h"
#include "basic-block.h"
#include "tree-iterator.h"
#include "tree-pass.h"
#include "struct-reorg.h"
#include "opts.h"
#include "ipa-type-escape.h"
#include "tree-dump.h"
#include "c-common.h"
/* This optimization implements structure peeling.
For example, given a structure type:
typedef struct
{
int a;
float b;
int c;
}str_t;
it can be peeled into two structure types as follows:
typedef struct and typedef struct
{ {
int a; float b;
int c; } str_t_1;
}str_t_0;
or can be fully peeled:
typedef struct
{
int a;
}str_t_0;
typedef struct
{
float b;
}str_t_1;
typedef struct
{
int c;
}str_t_2;
When structure type is peeled all instances and their accesses
in the program are updated accordingly. For example, if there is
array of structures:
str_t A[N];
and structure type str_t was peeled into two structures str_t_0
and str_t_1 as it was shown above, then array A will be replaced
by two arrays as foolows:
str_t_0 A_0[N];
str_t_1 A_1[N];
The field access of fields a of ellement i of array A: A[i].a will be
replace by access to field a of ellement i of array A_0: A_0[i].a.
This optimization also supports dynamically allocated arrays.
If array of structures was allocated by malloc function:
str_t * p = (str_t *) malloc (sizeof (str_t) * N)
the allocation site will be replaced to reflect new structure types:
str_t_0 * p_0 = (str_t_0 *) malloc (sizeof (str_t_0) * N)
str_t_1 * p_1 = (str_t_1 *) malloc (sizeof (str_t_1) * N)
The field access through the pointer p[i].a will be changed by p_0[i].a.
The decision how to peel structure gains to utilize spatial locality.
For example, if one of fields of structure has intensive use in the loop:
for (i = 0; i < N; i++)
{
... = A[i].a;
}
the allocation of field a of str_t in the memory in contiguous manner will
increase probablity of relevant data to be fatched into cache.
The analysis part of this optimization is based on the frequency of
field accesses, which are collected all over the program. Then the simple
clustering algorithm is used for partitioning into clusters, where each cluster
define new structure type. The parameter MAX_DIST_IN_CLUSTER_RATIO control
the level of closeness of elements in cluster, i.e. if there are fields f1, f2, and
the distance between them is defined as absolute value of difference between there
frequencies
dist (f1, f2) = abs (freq (f1) - freq (f2)),
then field f will be added to the cluster, if for each field q in the cluster
the following statement keeps true:
dist (q, f) <= (max_struct_distance/100) * MAX_DIST_IN_CLUSTER_RATIO
where max_struct_distance is maximum distance between two fields in the structure.
If profiling information is provided, which can be static as with branch-probabilities
flag, or dynamic as with profile-generate/profile-use flags, its is used to
calculate the frequency of field accesses. Otherwise, structure is fully peeled.
The ipa-type-escape analysis are used to provide safety of the peeling.
The optimization is activated by flag -fipa-struct-reorg.
*/
/* Ratio of the structure count to the hottest
structure count to consider the structure cold. */
#define COLD_STRUCTURE_RATIO 10
/* Ratio of maximum distance between fields allowed in cluster.
Used only when running with profiling. */
#define MAX_DIST_IN_CLUSTER_RATIO 1
/* Data about the new program variables created (for the new peeled types).
When doing struct peeling, each variable of the original struct type will
need to be replaced with a set of new variables, one new variable for each
new type created from the original type. */
struct new_var_data {
tree orig_var; /* Var decl for original struct type */
struct struct_tree_list *new_vars; /* List of new vars to replace the
original var; one new var for each
new type peeled off the original
type. */
struct new_var_data *next;
};
typedef struct new_var_data * new_var;
struct malloc_call_new
{
tree malloc_call_stmt;
d_str str;
/* Next malloc call stmt in the same function. */
struct malloc_call_new *next;
};
/* List of data for all malloc calls in the program. */
struct malloc_struct_new
{
tree func;
struct malloc_call_new *malloc_list; /* List of call sites for each
call to malloc from the current
func. */
struct malloc_struct_new *next;
};
typedef struct malloc_struct_new *malloc_d;
typedef struct malloc_call_new *malloc_call;
struct malloc_struct_new *malloc_data_list_new = NULL;
/* List of new global variables. Generated once for whole program. */
new_var new_global_vars = NULL;
/* List of new local variables. Generated once for function. */
new_var new_local_vars = NULL;
/* Use stmts of current function. */
tree_list use_stmts = NULL;
/* List of structures to be transformed. */
struct struct_list
{
struct data_structure *struct_data;
struct struct_list *next;
};
typedef struct struct_list * list;
/* List of structs to be transformed. */
list data_struct_list = NULL;
static new_var is_in_new_vars_list (tree, new_var);
static void add_struct_to_data_struct_list (tree);
static void add_call_to_malloc_list (tree, tree, d_str);
static inline void update_fields_mapping (struct field_cluster *, tree,
struct data_field_entry *, int);
static void create_new_var (tree, new_var *);
static tree get_final_malloc_stmt (tree malloc_stmt);
static tree_list add_tree_to_tree_list (tree, tree_list);
static void free_tree_list (tree_list);
static malloc_d get_malloc_list_for_func (tree);
static struct field_access_site * add_field_acc_to_acc_sites (tree, tree, tree, tree,
tree, tree, tree, tree,
tree, struct field_access_site *);
static struct access_site * add_access_to_acc_sites (tree, tree,
struct access_site *);
static bool is_result_of_mult (tree, tree *, tree);
static void remove_struct_from_data_struct_list (list);
static tree gen_size (tree, tree, tree *);
static tree gen_cast_stmt (tree, tree, tree, tree *);
static void create_new_stmts_for_general_acc (struct access_site *, d_str);
static void create_new_stmts_for_cond_expr (tree);
static bool is_equal_types (tree, tree);
static tree * find_pos_in_stmt (tree, tree);
/* Strip type from pointers and arrays. */
static inline tree
strip_type (tree type)
{
gcc_assert (TYPE_P (type));
while (POINTER_TYPE_P (type)
|| TREE_CODE (type) == ARRAY_TYPE)
type = TREE_TYPE (type);
return type;
}
/* Returns type of VAR. */
static inline tree
get_type_of_var (tree var)
{
if (!var)
return NULL;
if (TREE_CODE (var) == PARM_DECL)
return DECL_ARG_TYPE (var);
else
return TREE_TYPE (var);
}
/* Check whether the type of VAR is suitable for peeling.
Returns true if yes, false otherwise. */
static bool
is_candidate_for_data_struct_list (tree var, tree *type_p,
bitmap non_suitable_types)
{
tree type;
bool initialized = false;
*type_p = NULL;
if (!var)
return false;
/* There is no support of initialized vars. */
if (TREE_CODE (var) == VAR_DECL
&& DECL_INITIAL (var) != NULL_TREE)
initialized = true;
type = get_type_of_var (var);
if (type)
{
type = TYPE_MAIN_VARIANT (strip_type (type));
*type_p = type;
if (initialized && non_suitable_types)
bitmap_set_bit (non_suitable_types, TYPE_UID (type));
if (TREE_CODE (type) == RECORD_TYPE)
{
return true;
}
else
return false;
}
else
return false;
}
/* Given a tree representing a type, this function returns the
name of the type, as a string. */
static char *
get_type_name (tree type_node)
{
if (! TYPE_NAME (type_node))
return NULL;
if (TREE_CODE (TYPE_NAME (type_node)) == IDENTIFIER_NODE)
return (char *) IDENTIFIER_POINTER (TYPE_NAME (type_node));
else if (TREE_CODE (TYPE_NAME (type_node)) == TYPE_DECL
&& DECL_NAME (TYPE_NAME (type_node)))
return (char *) IDENTIFIER_POINTER (DECL_NAME (TYPE_NAME (type_node)));
else
return NULL;
}
static void
print_new_vars (new_var new_vars)
{
new_var current;
tree_list cur_var;
tree var_type;
if (!dump_file)
return;
for (current = new_vars; current; current = current->next)
{
var_type = get_type_of_var (current->orig_var);
fprintf (dump_file, "\nOrig var: ");
print_generic_expr (dump_file, current->orig_var, 0);
fprintf (dump_file, " of type ");
print_generic_expr (dump_file, var_type, 0);
fprintf (dump_file, "\n");
for (cur_var = current->new_vars; cur_var; cur_var = cur_var->next)
{
tree var = cur_var->data;
var_type = get_type_of_var (var);
fprintf (dump_file, " ");
print_generic_expr (dump_file, var, 0);
fprintf (dump_file, " of type ");
print_generic_expr (dump_file, var_type, 0);
fprintf (dump_file, "\n");
}
}
}
/* Print structure type. */
static void
print_struct_type (tree new_type, char * indent)
{
char * new_indent;
char * struct_name;
tree cur_field;
int len;
int i;
if (!new_type || !dump_file)
return;
if (TREE_CODE (new_type) != RECORD_TYPE)
return;
struct_name = get_type_name (new_type);
if (struct_name)
{
fprintf (dump_file, "%s%s {\n", indent, struct_name);
len = strlen (struct_name) + strlen (indent) + 3;
}
else
{
fprintf (dump_file, "%s{\n", indent);
len = strlen (indent) + 2;
}
new_indent = (char *) xmalloc (len * sizeof (char));
memset (new_indent, ' ', len);
new_indent[len] = '\0';
for (cur_field = TYPE_FIELDS (new_type); cur_field;
cur_field = TREE_CHAIN (cur_field))
{
tree field_type;
int ptr_count = 0;
field_type = TREE_TYPE (cur_field);
while (POINTER_TYPE_P (field_type))
{
ptr_count++;
field_type = TREE_TYPE (field_type);
}
fprintf (dump_file, "%s%s ", new_indent, get_type_name (field_type));
for (i = 0; i < ptr_count; i++)
fprintf (dump_file, "*");
fprintf (dump_file, " %s;\n",
IDENTIFIER_POINTER (DECL_NAME (cur_field)));
}
fprintf (dump_file, "%s}\n", indent);
}
/* Print out the list of new types generated by this optimization. */
static void
print_new_types (void)
{
list tmp;
char indent[3] = " ";
if (!dump_file)
return;
fprintf (dump_file,
"\nThe following are the new types generated by"
" this optimization:\n");
for (tmp = data_struct_list; tmp; tmp = tmp->next)
{
d_str str = tmp->struct_data;
tree_list new_type;
for (new_type = str->new_types; new_type; new_type = new_type->next)
print_struct_type (new_type->data, indent);
}
}
static tree
find_field_in_struct_1 (tree str_type, tree field)
{
tree str_field;
for (str_field = TYPE_FIELDS (str_type); str_field;
str_field = TREE_CHAIN (str_field))
{
char * str_field_name;
char * field_name;
str_field_name = (char *) IDENTIFIER_POINTER (DECL_NAME (str_field));
field_name = (char *) IDENTIFIER_POINTER (DECL_NAME (field));
gcc_assert (str_field_name);
gcc_assert (field_name);
if (!strcmp (str_field_name, field_name))
{
/* Check their types. */
if (is_equal_types (TREE_TYPE (str_field), TREE_TYPE (field)))
return str_field;
}
}
return NULL_TREE;
}
/* Given the field declaration tree (FIELD_DECL) returns the
appropriate field entry in DS. */
static struct data_field_entry *
find_field_in_struct (d_str str, tree field_decl)
{
int i;
tree field = find_field_in_struct_1 (str->decl, field_decl);
for (i = 0; i < str->num_fields; i++)
if (str->fields[i].decl == field)
return &(str->fields[i]);
return NULL;
}
/* This function is a hack to overcome the types problem.
When number of compilation units are compiled together
under -combine flag, the TYPE_MAIN_VARIANT cannot
be structed. First we comparing names, but they do not
also appear, like for example, in stmts. Only then we
are going inside structures.
*/
static bool
is_equal_types (tree type1, tree type2)
{
char * name1,* name2;
if ((!type1 && type2)
||(!type2 && type1))
return false;
if (!type1 && !type2)
return true;
if (TREE_CODE (type1) != TREE_CODE (type2))
return false;
if (TYPE_MAIN_VARIANT (type1) == TYPE_MAIN_VARIANT (type2))
return true;
name1 = get_type_name (type1);
name2 = get_type_name (type2);
if (name1 && name2 && !strcmp (name1, name2))
return true;
if (name1 && name2 && strcmp (name1, name2))
return false;
switch (TREE_CODE (type1))
{
case POINTER_TYPE:
case REFERENCE_TYPE:
{
return is_equal_types (TREE_TYPE (type1), TREE_TYPE (type2));
}
break;
case RECORD_TYPE:
case UNION_TYPE:
case QUAL_UNION_TYPE:
case ENUMERAL_TYPE:
{
tree field1;
/* Compare fields of struture. */
for (field1 = TYPE_FIELDS (type1); field1;
field1 = TREE_CHAIN (field1))
{
tree field2 = find_field_in_struct_1 (type2, field1);
if (!field2)
return false;
}
return true;
}
break;
default:
gcc_unreachable();
}
return false;
}
/* This function returns data_structure entry
in the data_struct_list represented by TYPE, if found. */
static list
find_struct_in_list (tree type)
{
list tmp = data_struct_list;
while (tmp)
{
if (is_equal_types (tmp->struct_data->decl, TYPE_MAIN_VARIANT (type)))
return tmp;
tmp = tmp->next;
}
return NULL;
}
/* Check whether the indirect_ref is of the form we can deal with. */
static bool
is_complicated_indirect_ref (tree str_decl, tree ref, tree * num_p, tree * offset_p,
tree * base_p, tree * ref_def_stmt_p, tree * cast_stmt_p)
{
tree ref_var;
tree rhs, struct_size, op0, op1;
tree before_cast;
ref_var = TREE_OPERAND (ref, 0);
if (TREE_CODE (ref_var) != SSA_NAME)
return true;
*ref_def_stmt_p = SSA_NAME_DEF_STMT (ref_var);
if (!(*ref_def_stmt_p)
|| (TREE_CODE (*ref_def_stmt_p) != GIMPLE_MODIFY_STMT))
return true;
rhs = GIMPLE_STMT_OPERAND (*ref_def_stmt_p, 1);
if (TREE_CODE (rhs) != PLUS_EXPR
&& TREE_CODE (rhs)!= MINUS_EXPR
&& TREE_CODE (rhs) != POINTER_PLUS_EXPR)
return true;
op0 = TREE_OPERAND (rhs, 0);
op1 = TREE_OPERAND (rhs, 1);
if (!is_array_access_through_pointer_and_index (TREE_CODE (rhs), op0, op1,
base_p, offset_p,
cast_stmt_p))
return true;
if (*cast_stmt_p)
before_cast = SINGLE_SSA_TREE_OPERAND (*cast_stmt_p, SSA_OP_USE);
else
before_cast = *offset_p;
if (!before_cast)
return false;
if (SSA_NAME_IS_DEFAULT_DEF(before_cast))
return false;
struct_size = TYPE_SIZE_UNIT (str_decl);
if (!is_result_of_mult (before_cast, num_p, struct_size))
return true;
return false;
}
static bool
is_complicated_access (tree str_decl, tree ref, tree * num_p, tree * offset_p,
tree * base_p, tree * ref_def_stmt_p, tree * cast_stmt_p)
{
if (TREE_CODE (ref) == INDIRECT_REF)
return is_complicated_indirect_ref (str_decl, ref, num_p, offset_p,
base_p, ref_def_stmt_p, cast_stmt_p);
else if (TREE_CODE (ref) == ARRAY_REF)
return false;
else if (TREE_CODE (ref) == VAR_DECL)
return false;
return true;
}
/* Helper for walk_tree called from collect_accesses_in_bb function. */
static tree
get_stmt_accesses (tree *tp, int *walk_subtrees, void *data)
{
tree stmt = (tree) data;
tree t = *tp;
if (!t)
return NULL;
switch (TREE_CODE (t))
{
case BIT_FIELD_REF:
{
tree str = TREE_OPERAND(t, 0);
tree type = TYPE_MAIN_VARIANT (strip_type (get_type_of_var (str)));
list tmp = find_struct_in_list (type);
if (tmp)
remove_struct_from_data_struct_list (tmp);
}
break;
case GIMPLE_MODIFY_STMT:
{
tree lhs = GIMPLE_STMT_OPERAND (t, 0);
tree rhs = GIMPLE_STMT_OPERAND (t, 1);
*walk_subtrees=1;
walk_tree (&lhs, get_stmt_accesses, data, NULL);
walk_tree (&rhs, get_stmt_accesses, data, NULL);
*walk_subtrees=0;
}
break;
case COMPONENT_REF:
{
tree ref = TREE_OPERAND (t, 0);
tree field_decl = TREE_OPERAND (t, 1);
if ((TREE_CODE (ref) == INDIRECT_REF
|| TREE_CODE (ref) == ARRAY_REF
|| TREE_CODE (ref) == VAR_DECL)
&& TREE_CODE (field_decl) == FIELD_DECL)
{
tree type = TYPE_MAIN_VARIANT (TREE_TYPE (ref));
list tmp = find_struct_in_list (type);
if (tmp)
{
d_str str = tmp->struct_data;
struct data_field_entry * field =
find_field_in_struct (str, field_decl);
if (field)
{
tree num = NULL;
tree offset = NULL;
tree base = NULL;
tree ref_def_stmt = NULL;
tree cast_stmt = NULL;
/* Check whether the access is of the form we can deal with. */
if (is_complicated_access (str->decl, ref, &num, &offset,
&base, &ref_def_stmt, &cast_stmt))
remove_struct_from_data_struct_list (tmp);
else
{
/* Increase count of field. */
basic_block bb = bb_for_stmt (stmt);
field->count += bb->count;
/* Add stmt to the acc_sites list of field. */
field->acc_sites =
add_field_acc_to_acc_sites (stmt, ref, t, field_decl, num, offset,
base, ref_def_stmt, cast_stmt,
field->acc_sites);
}
*walk_subtrees = 0;
}
}
}
}
break;
case MINUS_EXPR:
case PLUS_EXPR:
{
tree op0 = TREE_OPERAND (t, 0);
tree op1 = TREE_OPERAND (t, 1);
*walk_subtrees=1;
walk_tree (&op0, get_stmt_accesses, data, NULL);
walk_tree (&op1, get_stmt_accesses, data, NULL);
*walk_subtrees=0;
}
break;
case COND_EXPR:
{
tree cond = COND_EXPR_COND (t);
int i;
for (i = 0; i < TREE_CODE_LENGTH (TREE_CODE (cond)); i++)
{
tree t = TREE_OPERAND (cond, i);
*walk_subtrees=1;
walk_tree (&t, get_stmt_accesses, data, NULL);
}
*walk_subtrees=0;
}
break;
case VAR_DECL:
case SSA_NAME:
{
list tmp;
if (TREE_CODE (t) == SSA_NAME)
t = SSA_NAME_VAR (t);
tmp = find_struct_in_list (strip_type (get_type_of_var (t)));
if (tmp)
{
d_str str = tmp->struct_data;
str->accs =
add_access_to_acc_sites (stmt, t, str->accs);
}
*walk_subtrees=0;
}
break;
case CALL_EXPR:
{
/* It was checked as part of stage1 that structs
to be transformed cannot be passed as parameters to function. */
*walk_subtrees=0;
}
break;
default:
return NULL;
}
return NULL;
}
/* Collect accesses to the fields of DS in BB. */
static void
collect_accesses_in_bb (basic_block bb)
{
block_stmt_iterator bsi;
/* Go over the basic block statements collecting
accesses to fields of structures. */
for (bsi = bsi_start (bb); ! bsi_end_p (bsi); bsi_next (&bsi))
{
tree stmt = bsi_stmt (bsi);
walk_tree (&stmt, get_stmt_accesses, stmt, NULL);
}
}
/* Collect accesses of the fields of the structures in function FN. */
static void
collect_accesses_in_func (struct function *fn)
{
basic_block bb;
if (! fn)
return;
/* Collect accesses in each one of the basic blocks. */
FOR_EACH_BB_FN (bb, fn)
collect_accesses_in_bb (bb);
}
/* This functions builds structure with fields FIELDS,
that should be TREE_CHAIN, named NAME, with the same
attributes as ORIG_STRUCT. */
static tree
build_basic_struct (tree fields, char * name, tree orig_struct)
{
tree attributes = NULL_TREE;
tree ref = 0;
tree x;
/* From start_struct () */
if (TYPE_ATTRIBUTES (orig_struct))
attributes = copy_node (TYPE_ATTRIBUTES (orig_struct));
ref = make_node (RECORD_TYPE);
C_TYPE_BEING_DEFINED (ref) = 1;
TYPE_PACKED (ref) = flag_pack_struct;
/* From finish_struct (ref, tree fieldlist, tree attributes); */
TYPE_SIZE (ref) = 0;
decl_attributes (&ref, attributes, (int) ATTR_FLAG_TYPE_IN_PLACE);
TYPE_PACKED (ref) = TYPE_PACKED (orig_struct);
C_TYPE_FIELDS_READONLY (ref) = C_TYPE_FIELDS_READONLY (orig_struct);
C_TYPE_FIELDS_VOLATILE (ref) = C_TYPE_FIELDS_VOLATILE (orig_struct);
C_TYPE_VARIABLE_SIZE (ref) = C_TYPE_VARIABLE_SIZE (orig_struct);
for (x = fields; x; x = TREE_CHAIN (x))
{
DECL_CONTEXT (x) = ref;
DECL_PACKED (x) |= TYPE_PACKED (ref);
}
TYPE_FIELDS (ref) = fields;
layout_type (ref);
TYPE_NAME (ref) = get_identifier (name);
return ref;
}
/* Generate new structure name. */
static inline char *
gen_cluster_name (tree decl, int clust_num, int str_num)
{
char * old_name = get_type_name (decl);
char * new_name;
if (!old_name)
{
old_name = (char *) xmalloc ((10) * sizeof (char));
sprintf (old_name, "struct_%d", str_num);
}
new_name = (char *) xmalloc ((strlen (old_name) + 11) * sizeof (char));
sprintf (new_name, "%s_sub_%d", old_name, clust_num);
/* Check that this name was not used before. */
while (maybe_get_identifier (new_name))
{
int len = strlen (new_name) + 3;
char * tmp_name = (char *) xmalloc (len * sizeof (char));
sprintf (tmp_name, "%s.0", new_name);
new_name = tmp_name;
}
return new_name;
}
/* This function copies fields from CLUSTER into TREE_CHAIN as part
of preparation for new structure building. */
static tree
create_fields (struct field_cluster * cluster,
struct data_field_entry * fields, int num_fields)
{
int i;
tree new_types = NULL_TREE;
tree last = NULL_TREE;
for (i = 0; i < num_fields; i++)
if (TEST_BIT (cluster->fields_in_cluster, i))
{
tree new_decl = copy_node (fields[i].decl);
if (!new_types)
new_types = new_decl;
else
TREE_CHAIN (last) = new_decl;
last = new_decl;
}
TREE_CHAIN (last) = NULL_TREE;
return new_types;
}
/* Update field_mapping of fields in CLUSTER with NEW_TYPE. */
static inline void
update_fields_mapping (struct field_cluster *cluster, tree new_type,
struct data_field_entry * fields, int num_fields)
{
int i;
for (i = 0; i < num_fields; i++)
if (TEST_BIT (cluster->fields_in_cluster, i))
fields[i].field_mapping = new_type;
}
/* This function creates new types according to struct clustering data. */
static void
create_new_types (void)
{
list tmp = data_struct_list;
int str_num = 0;
while (tmp)
{
int cluster_num = 0;
d_str str = tmp->struct_data;
struct field_cluster * cluster = str->struct_clustering;
while (cluster)
{
char * name = gen_cluster_name (str->decl, cluster_num, str_num);
tree fields;
tree new_type;
cluster_num++;
fields = create_fields (cluster, str->fields, str->num_fields);
new_type = build_basic_struct (fields, name, str->decl);
update_fields_mapping (cluster, new_type, str->fields, str->num_fields);
str->new_types = add_tree_to_tree_list (new_type, str->new_types);
cluster = cluster->sibling;
}
str_num++;
tmp = tmp->next;
}
}
static void
finish_var_creation (tree new_decl)
{
if (!var_ann (new_decl))
create_var_ann (new_decl);
if (is_global_var (new_decl))
mark_call_clobbered (new_decl, ESCAPE_UNKNOWN);
mark_sym_for_renaming (new_decl);
if (gimple_referenced_vars (cfun))
add_referenced_var (new_decl);
}
static void
finish_global_creation (tree var)
{
if (TREE_CODE (var) == VAR_DECL
&& is_global_var (var))
finish_var_creation (var);
}
static char *
gen_var_name (tree orig_decl, int i, tree * id_node)
{
char * new_name = NULL;
/* If the original variable decl has a name, create an
appropriate new name for the new decl. */
if (DECL_NAME (orig_decl)
&& IDENTIFIER_POINTER (DECL_NAME (orig_decl)))
{
char *old_name;
int counter = 0;
int len, new_len;
char *tmp_name;
old_name = (char *) IDENTIFIER_POINTER (DECL_NAME (orig_decl));
len = strlen (old_name) + 6;
new_name = (char *) xmalloc (len * sizeof (char));
sprintf (new_name, "%s_%d", old_name, i);
/* Make sure there isn't anything else that already has that
name. */
new_len = strlen (new_name) + 5;
tmp_name = (char *) xmalloc (new_len * sizeof (char));
sprintf (tmp_name, "%s", new_name);
while (maybe_get_identifier (tmp_name))
sprintf (tmp_name, "%s.%d", new_name, counter++);
new_name = tmp_name;
*id_node = get_identifier (new_name);
}
else
*id_node = NULL;
return new_name;
}
static struct field_access_site *
is_in_field_acc_list (tree stmt, struct field_access_site *list_p)
{
struct field_access_site *curr;
for (curr = list_p; curr; curr = curr->next)
if (curr->stmt == stmt)
return curr;
return NULL;
}
static struct field_access_site *
add_field_acc_to_acc_sites (tree stmt, tree ref, tree comp_ref, tree field_decl, tree num,
tree offset, tree base, tree ref_def_stmt, tree cast_stmt,
struct field_access_site *list_p)
{
struct field_access_site *acc;
gcc_assert (!is_in_field_acc_list (stmt, list_p));
acc = (struct field_access_site *) xmalloc (sizeof (struct field_access_site));
acc->stmt = stmt;
acc->comp_ref = comp_ref;
acc->ref = ref;
acc->field_decl = field_decl;
acc->num = num;
acc->offset = offset;
acc->base = base;
acc->ref_def_stmt = ref_def_stmt;
acc->cast_stmt = cast_stmt;
acc->next = list_p;
return acc;
}
static struct access_site *
is_in_acc_list (tree stmt, struct access_site *list_p)
{
struct access_site *curr;
for (curr = list_p; curr; curr = curr->next)
if (curr->stmt == stmt)
return curr;
return NULL;
}
static struct access_site *
add_access_to_acc_sites (tree stmt, tree var, struct access_site *list_p)
{
struct access_site *acc;
acc = is_in_acc_list (stmt, list_p);
if (acc)
{
acc->vars = add_tree_to_tree_list (var, acc->vars);
return list_p;
}
else
{
acc = (struct access_site *) xmalloc (sizeof (struct access_site));
acc->stmt = stmt;
acc->next = list_p;
acc->vars = NULL;
acc->vars = add_tree_to_tree_list (var, acc->vars);
return acc;
}
}
/* Stuff the new tree DATA into a tree_list node,
and append it to the front of the LIST_P. */
static tree_list
add_tree_to_tree_list (tree data, tree_list list_p)
{
tree_list tmp_node;
tmp_node = (tree_list) xmalloc (sizeof (struct struct_tree_list));
tmp_node->data = data;
tmp_node->next = list_p;
return tmp_node;
}
static inline void
insert_global_to_varpool (tree new_decl)
{
struct varpool_node *new_node;
new_node = varpool_node (new_decl);
notice_global_symbol (new_decl);
varpool_mark_needed_node (new_node);
varpool_finalize_decl (new_decl);
}
/* This function copy attributes form ORIG_DECL to NEW_DECL. */
static inline void
copy_decl_attributes (tree new_decl, tree orig_decl)
{
DECL_ARTIFICIAL (new_decl) = 1;
DECL_EXTERNAL (new_decl) = DECL_EXTERNAL (orig_decl);
TREE_STATIC (new_decl) = TREE_STATIC (orig_decl);
TREE_PUBLIC (new_decl) = TREE_PUBLIC (orig_decl);
TREE_USED (new_decl) = TREE_USED (orig_decl);
DECL_CONTEXT (new_decl) = DECL_CONTEXT (orig_decl);
TREE_THIS_VOLATILE (new_decl) = TREE_THIS_VOLATILE (orig_decl);
TREE_ADDRESSABLE (new_decl) = TREE_ADDRESSABLE (orig_decl);
if (TREE_CODE (orig_decl) == VAR_DECL &&
TREE_READONLY (orig_decl))
TREE_READONLY (new_decl) = 1;
}
struct type_wrapper
{
/* 0 stand for pointer wrapper,
and 1 for array wrapper. */
bool wrap;
tree domain; /* Relevant for arrays as domain or index. */
struct type_wrapper * next;
};
static struct type_wrapper *
add_wrapper (struct type_wrapper *wrapper, bool wrap, tree domain)
{
struct type_wrapper * node = (struct type_wrapper *) xmalloc
(sizeof (struct type_wrapper));
node->wrap = wrap;
node->next = wrapper;
node->domain = domain;
return node;
}
static void
free_wrapper (struct type_wrapper * wrap)
{
struct type_wrapper *node, *node1;
node = wrap;
while (node)
{
node1 = node;
node = node->next;
free (node1);
}
}
/* This function generate the same level of pointers or arrays
in the NEW_STR_TYPE as it was in type of DECL.
It returns the generated type. */
static inline tree
gen_struct_type (tree decl, tree new_str_type)
{
tree type_orig = get_type_of_var (decl);
tree new_type = new_str_type;
struct type_wrapper * wrapper = NULL;
struct type_wrapper * wr = NULL;
while (POINTER_TYPE_P (type_orig)
|| TREE_CODE (type_orig) == ARRAY_TYPE)
{
if (POINTER_TYPE_P (type_orig))
wrapper = add_wrapper (wrapper, 0, NULL_TREE);
else if (TREE_CODE (type_orig) == ARRAY_TYPE)
wrapper = add_wrapper (wrapper, 1, TYPE_DOMAIN (type_orig));
type_orig = TREE_TYPE (type_orig);
}
for (wr = wrapper; wr; wr = wr->next)
{
if (wr->wrap) /* Array. */
new_type = build_array_type (new_type, wr->domain);
else /* Pointer. */
new_type = build_pointer_type (new_type);
}
free_wrapper (wrapper);
return new_type;
}
/* Given an old variable ORIG_DECL with VAR_DECL tree code,
this function generates new variables to replace it
according to the set of types decided before. */
static struct struct_tree_list *
create_new_var_1 (tree orig_decl, d_str str)
{
tree_list new_vars = NULL;
tree_list current;
int i;
for (current = str->new_types, i = 0; current;
current = current->next, i++)
{
tree new_type = current->data;
tree new_decl = NULL;
tree id_node;
char * new_name = NULL;
new_name = gen_var_name (orig_decl, i, &id_node);
new_type = gen_struct_type (orig_decl, new_type);
if (is_global_var (orig_decl))
new_decl = build_decl (VAR_DECL, id_node, new_type);
else
new_decl = create_tmp_var (new_type, new_name);
copy_decl_attributes (new_decl, orig_decl);
new_vars = add_tree_to_tree_list (new_decl, new_vars);
}
/* Return the list of new var decls. */
return new_vars;
}
static void
finish_vars_creation (new_var list_p)
{
new_var node;
for (node = list_p; node; node = node->next)
{
tree_list curr;
for (curr = node->new_vars; curr; curr = curr->next)
finish_var_creation (curr->data);
}
}
/* For each SSA_NAME or VAR_DECL local variable of structure
type relevant for transformation in current function (cfun)
this function generate new variable(s) to replace it. */
static void
create_new_local_vars (void)
{
unsigned i;
tree var_list;
if (gimple_in_ssa_p (cfun))
{
for (i = 1; i < num_ssa_names; i++)
{
tree ssa_name = ssa_name (i) ;
if (!ssa_name)
continue;
create_new_var (SSA_NAME_VAR (ssa_name), &new_local_vars);
}
}
/* Function local variables. */
for (var_list = cfun->unexpanded_var_list; var_list;
var_list = TREE_CHAIN (var_list))
{
tree var = TREE_VALUE (var_list);
create_new_var (var, &new_local_vars);
}
finish_vars_creation (new_local_vars);
print_new_vars (new_local_vars);
}
/* This function returns true it the result variable of malloc call is
a cast to one of the structure types we are planning to transform.
STMT should contain malloc call: T.2 = malloc (T.1); */
static bool
is_malloc_of_struct (tree stmt, list * node_p)
{
tree lhs;
tree type;
tree final_stmt;
final_stmt = get_final_malloc_stmt (stmt);
if (!final_stmt)
return false;
/* final_stmt should be of the form:
T.3 = (struct_type *) T.2; */
if (TREE_CODE (final_stmt) != GIMPLE_MODIFY_STMT)
return false;
lhs = GIMPLE_STMT_OPERAND (final_stmt, 0);
type = get_type_of_var (lhs);
if (!type)
return false;
if (!POINTER_TYPE_P (type)
|| TREE_CODE (strip_type (type)) != RECORD_TYPE)
return false;
*node_p = find_struct_in_list (strip_type (type));
if (!(*node_p))
return false;
return true;
}
/* In this function we suppose that an allocation statement
var = (type_cast) malloc (size);
is converted into as follows:
T.1 = size;
T.2 = malloc (T.1);
T.3 = (type_cast) T.2;
var = T.3;
In this function we collect into malloc_data_list only allocation
function of variables of structure types presented
in data_struct_list. */
static void
collect_malloc_data (void)
{
struct cgraph_node *node;
struct cgraph_edge *cs;
for (node = cgraph_nodes; node; node = node->next)
if (node->analyzed && node->decl)
{
for (cs = node->callees; cs; cs = cs->next_callee)
{
tree stmt = cs->call_stmt;
if (stmt)
{
tree call = get_call_expr_in (stmt);
tree decl;
if (call && (decl = get_callee_fndecl (call))
&& TREE_CODE (stmt) == GIMPLE_MODIFY_STMT)
{
list str_node = NULL;
if (is_malloc_of_struct (stmt, &str_node))
{
d_str str = str_node->struct_data;
/* We support only malloc now. */
if (DECL_FUNCTION_CODE (decl) == BUILT_IN_MALLOC)
add_call_to_malloc_list (node->decl, stmt, str);
else
remove_struct_from_data_struct_list (str_node);
}
}
}
}
}
}
/* Update statements in STMT_LIST with BB info. */
static void
add_bb_info (basic_block bb, tree stmt_list)
{
if (TREE_CODE (stmt_list) == STATEMENT_LIST)
{
tree_stmt_iterator tsi;
for (tsi = tsi_start (stmt_list); !tsi_end_p (tsi); tsi_next (&tsi))
{
tree stmt = tsi_stmt (tsi);
set_bb_for_stmt (stmt, bb);
}
}
}
/* This function checks whether ARG is a result of multiplication
of some number by STRUCT_SIZE. If yes, the function returns true
and this number is substituted into NUM. */
static bool
is_result_of_mult (tree arg, tree *num, tree struct_size)
{
tree size_def_stmt = SSA_NAME_DEF_STMT (arg);
/* If malloc stmt was of the form D.2229_10 = malloc (D.2228_9);
then size_def_stmt can be D.2228_9 = num.3_8 * 8; */
if (size_def_stmt
&& TREE_CODE (size_def_stmt) == GIMPLE_MODIFY_STMT)
{
tree lhs = GIMPLE_STMT_OPERAND (size_def_stmt, 0);
tree rhs = GIMPLE_STMT_OPERAND (size_def_stmt, 1);
/* We expect temporary here. */
if (!is_gimple_reg (lhs))
return false;
if (TREE_CODE (rhs) == MULT_EXPR)
{
tree arg0 = TREE_OPERAND (rhs, 0);
tree arg1 = TREE_OPERAND (rhs, 1);
if (CONSTANT_CLASS_P (arg0)
&& simple_cst_equal (arg0,struct_size))
{
*num = arg1;
return true;
}
if (CONSTANT_CLASS_P (arg1)
&& simple_cst_equal (arg1,struct_size))
{
*num = arg0;
return true;
}
}
}
*num = NULL_TREE;
return false;
}
static inline void
finish_stmt (tree stmt)
{
update_stmt (stmt);
mark_symbols_for_renaming (stmt);
}
/* This function returns a tree represented
the number of structure instances allocated by MALLOC_STMT. */
static tree
gen_num_of_structs_in_malloc (tree stmt, tree str_decl, tree * new_stmts_p)
{
call_expr_arg_iterator iter;
tree arg;
tree call_expr;
tree struct_size;
unsigned int struct_size_int;
if (!stmt)
return NULL_TREE;
/* Get malloc agrument. */
call_expr = get_call_expr_in (stmt);
if (!call_expr)
return NULL_TREE;
arg = first_call_expr_arg (call_expr, &iter);
if (TREE_CODE (arg) != SSA_NAME
&& !TREE_CONSTANT (arg))
return NULL_TREE;
struct_size = TYPE_SIZE_UNIT (str_decl);
struct_size_int = TREE_INT_CST_LOW (struct_size);
gcc_assert (struct_size);
if (TREE_CODE (arg) == SSA_NAME)
{
tree num, div_stmt;
if (is_result_of_mult (arg, &num, struct_size))
return num;
num = create_tmp_var (integer_type_node, NULL);
if (num)
add_referenced_var (num);
if (exact_log2 (struct_size_int) == -1)
div_stmt = build2 (GIMPLE_MODIFY_STMT, integer_type_node,
num, build2 (TRUNC_DIV_EXPR, integer_type_node,
arg, struct_size));
else
div_stmt = build2 (GIMPLE_MODIFY_STMT, integer_type_node,
num,
build2 (RSHIFT_EXPR,
integer_type_node,
arg, build_int_cst (integer_type_node,
exact_log2 (struct_size_int))));
*new_stmts_p = alloc_stmt_list ();
append_to_statement_list (div_stmt,
new_stmts_p);
finish_stmt (div_stmt);
return num;
}
if (CONSTANT_CLASS_P (arg)
&& multiple_of_p (TREE_TYPE (struct_size),
arg, struct_size))
return int_const_binop (TRUNC_DIV_EXPR, arg, struct_size, 0);
return NULL_TREE;
}
static inline void
finish_stmt_and_append (tree *stmts, tree stmt)
{
append_to_statement_list (stmt,
stmts);
finish_stmt (stmt);
}
static tree
find_var_in_new_vars_list (new_var var, tree new_type)
{
tree_list curr;
for (curr = var->new_vars; curr; curr = curr->next)
{
tree type = strip_type(get_type_of_var (curr->data));
gcc_assert (type);
if (type == new_type)
return curr->data;
}
return NULL_TREE;
}
static tree
find_new_var_of_type (tree orig_var, tree new_type)
{
new_var var;
gcc_assert (orig_var && new_type);
if (TREE_CODE (orig_var) == SSA_NAME)
orig_var = SSA_NAME_VAR (orig_var);
var = is_in_new_vars_list (orig_var, new_global_vars);
if (!var)
var = is_in_new_vars_list (orig_var, new_local_vars);
gcc_assert (var);
return find_var_in_new_vars_list (var, new_type);
}
static tree
create_new_malloc (tree malloc_stmt, tree new_type, tree *new_stmts, tree num)
{
tree new_malloc_size;
tree call_expr, malloc_fn_decl;
tree new_stmt, malloc_res;
tree call_stmt, final_stmt;
tree cast_res;
gcc_assert (num && malloc_stmt && new_type);
*new_stmts = alloc_stmt_list ();
/* Generate argument to malloc as multiplication of num and size of new_type. */
new_stmt = gen_size (num, new_type, &new_malloc_size);
append_to_statement_list (new_stmt, new_stmts);
/* Generate new call for malloc. */
malloc_res = create_tmp_var (integer_type_node, NULL);
if (malloc_res)
add_referenced_var (malloc_res);
call_expr = get_call_expr_in (malloc_stmt);
malloc_fn_decl = get_callee_fndecl (call_expr);
call_expr = build_call_expr (malloc_fn_decl, 1, new_malloc_size);
call_stmt = build2 (GIMPLE_MODIFY_STMT,
TREE_TYPE (TREE_TYPE (malloc_fn_decl)),
malloc_res,
call_expr);
finish_stmt_and_append (new_stmts, call_stmt);
/* Create new cast stmt. */
final_stmt = get_final_malloc_stmt (malloc_stmt);
gcc_assert (final_stmt);
new_stmt = gen_cast_stmt (malloc_res, new_type, final_stmt, &cast_res);
append_to_statement_list (new_stmt, new_stmts);
return call_stmt;
}
/* This function build an edge between BB and E->dest and updates
phi nodes of E->dest. It returns newly created edge. */
static edge
make_edge_and_fix_phis_of_dest (basic_block bb, edge e)
{
edge new_e;
tree phi, arg;
new_e = make_edge (bb, e->dest, e->flags);
for (phi = phi_nodes (new_e->dest); phi; phi = PHI_CHAIN (phi))
{
arg = PHI_ARG_DEF_FROM_EDGE (phi, e);
add_phi_arg (phi, arg, new_e);
}
return new_e;
}
/* Update call graph with new edge generated by
new MALLOC_STMT. */
static void
update_cgraph_with_malloc_call (tree malloc_stmt, tree context)
{
tree call_expr;
struct cgraph_node *src, *dest;
tree malloc_fn_decl;
if (!malloc_stmt)
return;
call_expr = get_call_expr_in (malloc_stmt);
malloc_fn_decl = get_callee_fndecl (call_expr);
src = cgraph_node (context);
dest = cgraph_node (malloc_fn_decl);
cgraph_create_edge (src, dest, malloc_stmt,
0, 0, bb_for_stmt (malloc_stmt)->loop_depth);
}
/* If MALLOC_STMT is D.2225_7 = malloc (D.2224_6);
it should be casting stmt as for example:
p_8 = (struct str_t *) D.2225_7;
which is returned by this function. */
static tree
get_final_malloc_stmt (tree malloc_stmt)
{
tree final_stmt;
use_operand_p use_p;
tree malloc_res;
if (!malloc_stmt)
return NULL;
if (TREE_CODE (malloc_stmt) != GIMPLE_MODIFY_STMT)
return NULL;
malloc_res = GIMPLE_STMT_OPERAND (malloc_stmt, 0);
if (TREE_CODE (malloc_res) != SSA_NAME)
return NULL;
if (!single_imm_use (malloc_res, &use_p, &final_stmt))
return NULL;
else
return final_stmt;
}
/* Insert NEW_STMTS after STMT. */
static void
insert_after_stmt (tree stmt, tree new_stmts)
{
block_stmt_iterator bsi;
if (!stmt || !new_stmts)
return;
bsi = bsi_for_stmt (stmt);
bsi_insert_after (&bsi, new_stmts, BSI_SAME_STMT);
}
/* Insert NEW_STMTS before STMT. */
static void
insert_before_stmt (tree stmt, tree new_stmts)
{
block_stmt_iterator bsi;
if (!stmt || !new_stmts)
return;
bsi = bsi_for_stmt (stmt);
bsi_insert_before (&bsi, new_stmts, BSI_SAME_STMT);
}
/* This function adds allocation sites for peeled structs. */
static void
create_new_mallocs (malloc_d m_data, tree context)
{
malloc_call call;
for (call = m_data->malloc_list; call; call = call->next)
{
tree stmt = call->malloc_call_stmt;
d_str str = call->str;
tree_list n_type;
tree num;
tree new_stmts = NULL_TREE;
tree last_stmt = get_final_malloc_stmt (stmt);
num = gen_num_of_structs_in_malloc (stmt, str->decl, &new_stmts);
if (new_stmts)
{
last_stmt = tsi_stmt (tsi_last (new_stmts));
insert_after_stmt (last_stmt, new_stmts);
}
/* Generate an allocation sites for each new structure type. */
for (n_type = str->new_types; n_type; n_type = n_type->next)
{
tree type = n_type->data;
tree new_malloc_stmt = NULL_TREE;
tree last_stmt_tmp = NULL_TREE;
new_stmts = NULL_TREE;
new_malloc_stmt = create_new_malloc (stmt, type, &new_stmts, num);
last_stmt_tmp = tsi_stmt (tsi_last (new_stmts));
insert_after_stmt (last_stmt, new_stmts);
update_cgraph_with_malloc_call (new_malloc_stmt, context);
last_stmt = last_stmt_tmp;
}
}
}
/* Create new mallocs for function represented by NODE. */
static void
create_new_mallocs_for_func (struct cgraph_node *node)
{
malloc_d cur_malloc = get_malloc_list_for_func (node->decl);
if (cur_malloc)
create_new_mallocs (cur_malloc, node->decl);
}
static void
free_new_vars_list (new_var *list_p)
{
new_var node = *list_p;
new_var t_node;
while (node)
{
tree_list var, t_var;
/* Free tree_list of new vars. */
var = node->new_vars;
while (var)
{
t_var = var;
var = var->next;
free (t_var);
}
t_node = node;
node = node->next;
free (t_node);
}
*list_p = NULL;
}
static malloc_d
get_malloc_list_for_func (tree fn_decl)
{
malloc_d cur_malloc;
for (cur_malloc = malloc_data_list_new; cur_malloc;
cur_malloc = cur_malloc->next)
if (cur_malloc->func == fn_decl)
return cur_malloc;
return NULL;
}
static bool
is_part_of_malloc (tree stmt, tree fn_decl)
{
malloc_d cur_malloc = get_malloc_list_for_func (fn_decl);
if (cur_malloc)
{
malloc_call call;
for (call = cur_malloc->malloc_list; call; call = call->next)
if (call->malloc_call_stmt == stmt
|| get_final_malloc_stmt (call->malloc_call_stmt) == stmt)
return true;
}
return false;
}
static void
remove_from_acc_list (struct access_site ** list_p, tree stmt)
{
struct access_site *node = *list_p;
struct access_site *prev_node = NULL;
while (node)
{
if (node->stmt == stmt)
{
if (prev_node == NULL)
*list_p = node->next;
else
prev_node->next = node->next;
free_tree_list (node->vars);
free (node);
return;
}
prev_node = node;
node = node->next;
}
}
static bool
is_part_of_field_access (tree stmt, d_str str)
{
int i;
for (i = 0; i < str->num_fields; i++)
{
struct field_access_site *acc;
for (acc = str->fields[i].acc_sites; acc; acc = acc->next)
{
if (acc->stmt == stmt
|| acc->ref_def_stmt == stmt
|| acc->cast_stmt == stmt)
return true;
}
}
return false;
}
/* We exclude from list of general accesses of structure all stmt that
will be treated as part of new mallocs or accesses generation. */
static void
exclude_malloc_and_field_accs (struct cgraph_node *node)
{
list str_node;
for (str_node = data_struct_list; str_node; str_node = str_node->next)
{
d_str str = str_node->struct_data;
struct access_site * acc = str->accs;
while (acc)
{
tree stmt = acc->stmt;
acc = acc->next;
if (is_part_of_malloc (stmt, node->decl)
|| is_part_of_field_access (stmt, str))
remove_from_acc_list (&str->accs, stmt);
}
}
}
static inline tree
build_comp_ref (tree base, tree field_id, tree type)
{
tree field;
bool found = false;
/* Find field with the same name as field_id in type. */
for (field = TYPE_FIELDS (type); field; field = TREE_CHAIN (field))
{
if (DECL_NAME (field) == field_id)
{
found = true;
break;
}
}
gcc_assert (found);
return build3 (COMPONENT_REF, TREE_TYPE (field),
base, field, NULL_TREE);
}
struct ref_pos
{
tree *pos;
tree ref;
};
/* Helper for walk_tree called from collect_accesses_in_bb function. */
static tree
find_pos_in_stmt_1 (tree *tp, int *walk_subtrees, void * data)
{
struct ref_pos * r_pos = (struct ref_pos *) data;
tree ref = r_pos->ref;
tree t = *tp;
if (t == ref)
{
r_pos->pos = tp;
return t;
}
switch (TREE_CODE (t))
{
case GIMPLE_MODIFY_STMT:
{
tree lhs = GIMPLE_STMT_OPERAND (t, 0);
tree rhs = GIMPLE_STMT_OPERAND (t, 1);
*walk_subtrees=1;
walk_tree (&lhs, find_pos_in_stmt_1, data, NULL);
walk_tree (&rhs, find_pos_in_stmt_1, data, NULL);
*walk_subtrees=0;
}
break;
default:
*walk_subtrees=1;
}
return NULL_TREE;
}
static tree *
find_pos_in_stmt (tree stmt, tree ref)
{
struct ref_pos r_pos;
r_pos.ref = ref;
r_pos.pos = NULL;
walk_tree (&stmt, find_pos_in_stmt_1, &r_pos, NULL);
return r_pos.pos;
}
static void
replace_field_acc (struct field_access_site *acc, tree new_type)
{
tree ref_var = acc->ref;
tree new_ref;
tree lhs, rhs;
tree *pos;
tree new_acc;
tree field_id = DECL_NAME (acc->field_decl);
struct type_wrapper * wrapper = NULL;
struct type_wrapper * wr = NULL;
while (TREE_CODE (ref_var) == INDIRECT_REF
|| TREE_CODE (ref_var) == ARRAY_REF)
{
if ( TREE_CODE (ref_var) == INDIRECT_REF)
wrapper = add_wrapper (wrapper, 0, 0);
else if (TREE_CODE (ref_var) == ARRAY_REF)
wrapper = add_wrapper (wrapper, 1, TREE_OPERAND (ref_var, 1));
ref_var = TREE_OPERAND (ref_var, 0);
}
new_ref = find_new_var_of_type (ref_var, new_type);
finish_global_creation (new_ref);
for (wr = wrapper; wr; wr = wr->next)
{
tree type = TREE_TYPE (TREE_TYPE (new_ref));
if (wr->wrap) /* Array. */
new_ref = build4 (ARRAY_REF, type, new_ref,
wr->domain, NULL_TREE, NULL_TREE);
else /* Pointer. */
new_ref = build1 (INDIRECT_REF, type, new_ref);
}
new_acc = build_comp_ref (new_ref, field_id, new_type);
free_wrapper (wrapper);
if (TREE_CODE (acc->stmt) == GIMPLE_MODIFY_STMT)
{
lhs = GIMPLE_STMT_OPERAND (acc->stmt, 0);
rhs = GIMPLE_STMT_OPERAND (acc->stmt, 1);
if (lhs == acc->comp_ref)
GIMPLE_STMT_OPERAND (acc->stmt, 0) = new_acc;
else if (rhs == acc->comp_ref)
GIMPLE_STMT_OPERAND (acc->stmt, 1) = new_acc;
else
{
pos = find_pos_in_stmt (acc->stmt, acc->comp_ref);
gcc_assert (pos);
*pos = new_acc;
}
}
else
{
pos = find_pos_in_stmt (acc->stmt, acc->comp_ref);
gcc_assert (pos);
*pos = new_acc;
}
finish_stmt (acc->stmt);
}
static void
replace_field_access_stmt (struct field_access_site *acc, tree new_type)
{
if (TREE_CODE (acc->ref) == INDIRECT_REF
||TREE_CODE (acc->ref) == ARRAY_REF
||TREE_CODE (acc->ref) == VAR_DECL)
replace_field_acc (acc, new_type);
else
gcc_unreachable ();
}
/* In this function we create new stmts that has the same form as ORIG_STMT,
but of the type NET_TYPE. The stmts treated by this function are simple
assignments, like assignments: p.8_7 = p; or stmts with rhs of code PLUS_EXPR
or MINUS_EXPR. */
static tree
create_base_plus_offset (tree orig_stmt, tree new_type, tree offset)
{
tree lhs, rhs;
tree new_lhs, new_rhs;
tree new_stmt;
gcc_assert (TREE_CODE (orig_stmt) == GIMPLE_MODIFY_STMT);
lhs = GIMPLE_STMT_OPERAND (orig_stmt, 0);
rhs = GIMPLE_STMT_OPERAND (orig_stmt, 1);
gcc_assert (TREE_CODE (lhs) == VAR_DECL
|| TREE_CODE (lhs) == SSA_NAME);
new_lhs = find_new_var_of_type (lhs, new_type);
gcc_assert (new_lhs);
finish_var_creation (new_lhs);
switch (TREE_CODE (rhs))
{
case PLUS_EXPR:
case MINUS_EXPR:
case POINTER_PLUS_EXPR:
{
debug_tree (rhs);
tree op0 = TREE_OPERAND (rhs, 0);
tree op1 = TREE_OPERAND (rhs, 1);
tree new_op0 = NULL_TREE, new_op1 = NULL_TREE;
list tmp0, tmp1;
tmp0 = find_struct_in_list (strip_type (get_type_of_var (op0)));
tmp1 = find_struct_in_list (strip_type (get_type_of_var (op1)));
gcc_assert ( tmp0 || tmp1);
if (tmp0)
new_op0 = find_new_var_of_type (op0, new_type);
if (tmp1)
new_op1 = find_new_var_of_type (op1, new_type);
if (!new_op0)
new_op0 = offset;
if (!new_op1)
new_op1 = offset;
new_rhs = build2 (TREE_CODE (rhs), TREE_TYPE (new_op0),
new_op0, new_op1);
}
break;
default:
gcc_unreachable();
}
new_stmt = build2 (GIMPLE_MODIFY_STMT, TREE_TYPE (new_lhs),
new_lhs, new_rhs);
finish_stmt (new_stmt);
return new_stmt;
}
/* Generate stmt: res = NUM * sizeof(TYPE) and return it. */
static tree
gen_size (tree num, tree type, tree *res)
{
tree struct_size = TYPE_SIZE_UNIT (type);
unsigned int struct_size_int = TREE_INT_CST_LOW (struct_size);
tree new_stmt;
*res = create_tmp_var (TREE_TYPE (num), NULL);
if (*res)
add_referenced_var (*res);
if (exact_log2 (struct_size_int) == -1)
new_stmt = build2 (GIMPLE_MODIFY_STMT, TREE_TYPE (num),
*res, build2 (MULT_EXPR,
TREE_TYPE (num),
num,
struct_size));
else
new_stmt = build2 (GIMPLE_MODIFY_STMT, TREE_TYPE (num),
*res,
build2 (LSHIFT_EXPR,
TREE_TYPE (num),
num,
build_int_cst (TREE_TYPE (num),
exact_log2 (struct_size_int))));
finish_stmt (new_stmt);
return new_stmt;
}
static tree
gen_cast_stmt (tree before_cast, tree new_type, tree old_cast_stmt, tree *res_p)
{
tree lhs, new_lhs, new_stmt;
gcc_assert (TREE_CODE (old_cast_stmt) == GIMPLE_MODIFY_STMT);
lhs = GIMPLE_STMT_OPERAND (old_cast_stmt, 0);
new_lhs = find_new_var_of_type (lhs, new_type);
gcc_assert (new_lhs);
new_stmt = build2 (GIMPLE_MODIFY_STMT,
TREE_TYPE
(new_lhs),
new_lhs,
build1 (NOP_EXPR,
TREE_TYPE (new_lhs),
before_cast));
finish_stmt (new_stmt);
*res_p = new_lhs;
return new_stmt;
}
static void
create_new_field_access (struct field_access_site *acc, struct data_field_entry field)
{
tree new_type = field.field_mapping;
tree new_stmt;
tree size_res;
tree mult_stmt, cast_stmt;
tree cast_res = NULL;
if (acc->num)
{
mult_stmt = gen_size (acc->num, new_type, &size_res);
insert_before_stmt (acc->ref_def_stmt, mult_stmt);
}
if (acc->cast_stmt)
{
cast_stmt = gen_cast_stmt (size_res, new_type,
acc->cast_stmt, &cast_res);
insert_after_stmt (acc->cast_stmt, cast_stmt);
}
if (acc->ref_def_stmt)
{
tree offset;
if (cast_res)
offset = cast_res;
else
offset = size_res;
new_stmt = create_base_plus_offset (acc->ref_def_stmt, new_type, offset);
insert_after_stmt (acc->ref_def_stmt, new_stmt);
}
/* In stmt D.2163_19 = D.2162_18->b; we replace variable
D.2162_18 for the appropriate variable of new_type. */
replace_field_access_stmt (acc, new_type);
}
static void
create_new_general_access (struct access_site *acc, d_str str)
{
tree stmt = acc->stmt;
switch (TREE_CODE (stmt))
{
case COND_EXPR:
create_new_stmts_for_cond_expr (stmt);
break;
default:
create_new_stmts_for_general_acc (acc, str);
}
}
static void
create_new_accesses_in_bb (basic_block bb)
{
list tmp;
for (tmp = data_struct_list; tmp; tmp = tmp->next)
{
int i;
struct field_access_site *f_acc;
struct access_site *acc;
d_str str = tmp->struct_data;
for (i = 0; i < str->num_fields; i++)
for (f_acc = str->fields[i].acc_sites; f_acc; f_acc = f_acc->next)
if (bb_for_stmt (f_acc->stmt) == bb)
create_new_field_access (f_acc, str->fields[i]);
for (acc = str->accs; acc; acc = acc->next)
if (bb_for_stmt (acc->stmt) == bb)
create_new_general_access (acc, str);
}
}
static void
create_new_accesses_for_func (void)
{
basic_block bb;
FOR_EACH_BB_FN (bb, cfun)
create_new_accesses_in_bb (bb);
}
static tree
create_general_new_stmt (struct access_site *acc, tree new_type)
{
tree old_stmt = acc->stmt;
tree_list var_d;
tree new_stmt = copy_node (old_stmt);
for (var_d = acc->vars; var_d; var_d = var_d->next)
{
tree * pos;
tree var = var_d->data;
tree new_var = find_new_var_of_type (var, new_type);
tree lhs, rhs;
gcc_assert (new_var);
finish_var_creation (new_var);
if (TREE_CODE (new_stmt) == GIMPLE_MODIFY_STMT)
{
lhs = GIMPLE_STMT_OPERAND (new_stmt, 0);
rhs = GIMPLE_STMT_OPERAND (new_stmt, 1);
if (TREE_CODE (lhs) == SSA_NAME)
lhs = SSA_NAME_VAR (lhs);
if (TREE_CODE (rhs) == SSA_NAME)
rhs = SSA_NAME_VAR (rhs);
if (lhs == var)
GIMPLE_STMT_OPERAND (new_stmt, 0) = new_var;
else if (rhs == var)
GIMPLE_STMT_OPERAND (new_stmt, 1) = new_var;
else
{
pos = find_pos_in_stmt (new_stmt, var);
gcc_assert (pos);
*pos = new_var;
}
}
else
{
pos = find_pos_in_stmt (new_stmt, var);
gcc_assert (pos);
*pos = new_var;
}
}
finish_stmt (new_stmt);
return new_stmt;
}
static void
create_new_stmts_for_general_acc (struct access_site * acc, d_str str)
{
tree_list new_type;
tree stmt = acc->stmt;
for (new_type = str->new_types; new_type; new_type = new_type->next)
{
tree new_stmt;
new_stmt = create_general_new_stmt (acc, new_type->data);
insert_after_stmt (stmt, new_stmt);
}
}
static void
create_new_stmts_for_cond_expr_1 (tree new_var, tree cond_stmt, bool pos)
{
tree new_cond;
tree new_stmt;
edge true_e = NULL, false_e = NULL;
basic_block new_bb;
tree stmt_list;
extract_true_false_edges_from_block (bb_for_stmt (cond_stmt),
&true_e, &false_e);
new_cond = copy_node (COND_EXPR_COND (cond_stmt));
TREE_OPERAND (new_cond, pos) = new_var;
new_stmt = build3 (COND_EXPR,TREE_TYPE (cond_stmt),
new_cond, NULL_TREE, NULL_TREE);
finish_stmt (new_stmt);
/* Create a new basic block after bb. */
new_bb = create_empty_bb (bb_for_stmt (cond_stmt));
/* Add new condition stmt to the new_bb. */
stmt_list = bb_stmt_list (new_bb);
append_to_statement_list (new_stmt, &stmt_list);
add_bb_info (new_bb, stmt_list);
/* Create false and true edges from new_bb. */
make_edge_and_fix_phis_of_dest (new_bb, true_e);
make_edge_and_fix_phis_of_dest (new_bb, false_e);
/* Redirect one of original edges to point to new_bb. */
if (TREE_CODE (cond_stmt) == NE_EXPR)
redirect_edge_succ (true_e, new_bb);
else
redirect_edge_succ (false_e, new_bb);
}
static void
create_new_stmts_for_cond_expr (tree stmt)
{
tree cond = COND_EXPR_COND (stmt);
tree arg0, arg1, arg;
list tmp0, tmp1, tmp;
d_str str;
tree_list new_type;
bool pos;
gcc_assert (TREE_CODE (cond) == EQ_EXPR
|| TREE_CODE (cond) == NE_EXPR);
arg0 = TREE_OPERAND (cond, 0);
arg1 = TREE_OPERAND (cond, 1);
tmp0 = find_struct_in_list (strip_type (get_type_of_var (arg0)));
tmp1 = find_struct_in_list (strip_type (get_type_of_var (arg1)));
gcc_assert ((!tmp0 && tmp1) || (!tmp1 && tmp0));
tmp = tmp0 ? tmp0 : tmp1;
arg = tmp0 ? arg0 : arg1;
pos = tmp0 ? 0 : 1;
str = tmp->struct_data;
for (new_type = str->new_types; new_type; new_type = new_type->next)
{
tree new_arg;
new_arg = find_new_var_of_type (arg, new_type->data);
create_new_stmts_for_cond_expr_1 (new_arg, stmt, pos);
}
}
/* Do transformation for a function represented by NODE. */
static void
do_reorg_for_func (struct cgraph_node *node)
{
create_new_local_vars ();
create_new_mallocs_for_func (node);
create_new_accesses_for_func ();
update_ssa (TODO_update_ssa);
cleanup_tree_cfg ();
/* Free auxiliary data for local variables. */
free_new_vars_list (&new_local_vars);
}
static void
add_to_new_vars_list (new_var new_node, new_var *list_p)
{
new_var node;
if (!(*list_p))
*list_p = new_node;
else
{
node = *list_p;
while (node->next)
node = node->next;
node->next = new_node;
}
}
static new_var
is_in_new_vars_list (tree decl, new_var list_p)
{
new_var node;
if (!list_p)
return NULL;
else
{
node = list_p;
while (node)
{
if (node->orig_var == decl)
return node;
node = node->next;
}
}
return NULL;
}
static new_var
create_new_var_node (tree var, tree_list new_vars_list)
{
new_var node;
node = (struct new_var_data *) xmalloc (sizeof (struct new_var_data));
node->orig_var = var;
node->new_vars = new_vars_list;
node->next = NULL;
return node;
}
static void
create_new_var (tree var_decl, new_var *list_p)
{
new_var node;
tree_list new_vars;
list tmp;
tree type;
d_str str;
if (!var_decl || is_in_new_vars_list (var_decl, *list_p))
return;
if (!is_candidate_for_data_struct_list (var_decl, &type, NULL))
return;
tmp = find_struct_in_list (type);
if (!tmp)
return;
str = tmp->struct_data;
new_vars = create_new_var_1 (var_decl, str);
node = create_new_var_node (var_decl, new_vars);
add_to_new_vars_list (node, list_p);
}
static void
update_varpool_with_new_vars (void)
{
new_var old_var;
for (old_var = new_global_vars; old_var; old_var = old_var->next)
{
tree_list n_var;
for (n_var = old_var->new_vars; n_var; n_var = n_var->next)
insert_global_to_varpool (n_var->data);
}
}
/* This function create global struct variables of new
structure types in case these this variable has VAR_DECL code. */
static void
create_new_global_vars (void)
{
struct varpool_node *current_varpool;
FOR_EACH_STATIC_VARIABLE(current_varpool)
{
tree var_decl = current_varpool->decl;
if (!var_decl || TREE_CODE (var_decl) != VAR_DECL)
continue;
create_new_var (var_decl, &new_global_vars);
}
update_varpool_with_new_vars ();
}
/* This function is a helper for do_reorg. It goes over functions in call graph
and performs actual transformation on each one of them. */
static void
do_reorg_1 (void)
{
struct cgraph_node *node;
/* Initialize the default bitmap obstack. */
bitmap_obstack_initialize (NULL);
for (node = cgraph_nodes; node; node = node->next)
if (node->analyzed && node->decl && !node->next_clone)
{
push_cfun (DECL_STRUCT_FUNCTION (node->decl));
current_function_decl = node->decl;
if (dump_file)
fprintf (dump_file, "\nFunction to do reorg is %s: \n",
(char *) IDENTIFIER_POINTER (DECL_NAME (node->decl)));
do_reorg_for_func (node);
free_dominance_info (CDI_DOMINATORS);
free_dominance_info (CDI_POST_DOMINATORS);
current_function_decl = NULL;
pop_cfun ();
}
cfun = NULL;
}
static void
do_reorg (void)
{
/* Check that there is a work to do. */
if (!data_struct_list)
return;
/* Generate new types. */
create_new_types ();
print_new_types ();
/* Create new global vars. */
create_new_global_vars ();
print_new_vars (new_global_vars);
/* Do reorg for each funciton separately. */
do_reorg_1 ();
/* Free auxiliary data for global variables. */
free_new_vars_list (&new_global_vars);
}
/* This function builds an entry in struct_list for each
data structure potential for transformation. */
static void
build_data_structure_list (bitmap non_suitable_types)
{
tree var, type;
tree var_list;
struct varpool_node *current_varpool;
struct cgraph_node *c_node;
/* Check global vars. */
FOR_EACH_STATIC_VARIABLE (current_varpool)
{
var = current_varpool->decl;
if (is_candidate_for_data_struct_list (var, &type, non_suitable_types))
add_struct_to_data_struct_list (type);
}
/* Now add data structures of function parameters and local variables. */
for (c_node = cgraph_nodes; c_node; c_node = c_node->next)
{
enum availability avail = cgraph_function_body_availability (c_node);
/* We need AVAIL_AVAILABLE for main function. */
if (avail == AVAIL_LOCAL || avail == AVAIL_AVAILABLE)
{
struct function *fn = DECL_STRUCT_FUNCTION (c_node->decl);
for (var = DECL_ARGUMENTS (c_node->decl); var; var = TREE_CHAIN (var))
if (is_candidate_for_data_struct_list (var, &type, non_suitable_types))
add_struct_to_data_struct_list (type);
/* Function local variables. */
for (var_list = fn->unexpanded_var_list; var_list;
var_list = TREE_CHAIN (var_list))
{
var = TREE_VALUE (var_list);
if (is_candidate_for_data_struct_list (var, &type, non_suitable_types))
add_struct_to_data_struct_list (type);
}
}
}
}
/* This function looks for arguments of local functions
which are of pointer to or structure types. We are not handling
peeling of such structures right now. */
static void
exclude_types_passed_to_local_func (bitmap non_suitable_types)
{
struct cgraph_node *c_node;
for (c_node = cgraph_nodes; c_node; c_node = c_node->next)
{
if (cgraph_function_body_availability (c_node) == AVAIL_LOCAL)
{
tree fn = c_node->decl;
tree arg;
for (arg = DECL_ARGUMENTS (fn); arg; arg = TREE_CHAIN (arg))
{
tree type = TREE_TYPE (arg);
type = strip_type (type);
if (TREE_CODE (type) == RECORD_TYPE)
{
bitmap_set_bit (non_suitable_types, TYPE_UID (TYPE_MAIN_VARIANT (type)));
if (dump_file)
{
fprintf (dump_file, "\nPointer to type \"");
print_generic_expr (dump_file, type, 0);
fprintf (dump_file, "\" is passed to local function...Excluded.");
}
}
}
}
}
}
static void
exclude_escaping_types_1 (bitmap non_suitable_types)
{
list tmp = data_struct_list;
while (tmp)
{
tree type = tmp->struct_data->decl;
if (!ipa_type_escape_type_contained_p (type))
{
if (dump_file)
{
fprintf (dump_file, "\nEscaping type is ");
print_generic_expr (dump_file, type, 0);
}
bitmap_set_bit (non_suitable_types, TYPE_UID (type));
}
tmp = tmp->next;
}
}
static void
exclude_escaping_types (bitmap non_suitable_types)
{
exclude_types_passed_to_local_func (non_suitable_types);
exclude_escaping_types_1 (non_suitable_types);
}
/* FIXME: In this function we suppose that only fields defined
as bitfields in C program can become bitfields. If there are other ways,
this function should be extended to recognize them. */
static void
exclude_types_with_bit_fields (bitmap non_suitable_types)
{
list tmp = data_struct_list;
while (tmp)
{
int i;
d_str str = tmp->struct_data;
tree type = tmp->struct_data->decl;
for (i = 0; i < str->num_fields; i++)
if (DECL_BIT_FIELD (str->fields[i].decl))
bitmap_set_bit (non_suitable_types, TYPE_UID (type));
tmp = tmp->next;
}
}
/* This function returns true if program contains a call to user
defined allocation function. */
static bool
program_redefines_malloc_p (void)
{
bool redefines = false;
struct cgraph_node *c_node;
struct cgraph_node *c_node2;
struct cgraph_edge *c_edge;
tree fndecl;
tree fndecl2;
tree call_expr;
for (c_node = cgraph_nodes; c_node && !redefines; c_node = c_node->next)
{
fndecl = c_node->decl;
for (c_edge = c_node->callees; c_edge && !redefines;
c_edge = c_edge->next_callee)
{
call_expr = get_call_expr_in (c_edge->call_stmt);
c_node2 = c_edge->callee;
fndecl2 = c_node2->decl;
if (call_expr)
{
if ((call_expr_flags (call_expr) & ECF_MALLOC) &&
(DECL_FUNCTION_CODE (fndecl2) != BUILT_IN_MALLOC) &&
(DECL_FUNCTION_CODE (fndecl2) != BUILT_IN_CALLOC) &&
(DECL_FUNCTION_CODE (fndecl2) != BUILT_IN_ALLOCA))
redefines = true;
}
}
}
return redefines;
}
/* Auxiliary function for add_struct_to_data_struct_list. */
static struct data_field_entry *
get_fields (tree struct_decl, int num_fields)
{
struct data_field_entry *list;
tree t = TYPE_FIELDS (struct_decl);
int idx = 0;
list = (struct data_field_entry * ) xmalloc (num_fields
* sizeof
(struct data_field_entry));
for (idx = 0 ; t; t = TREE_CHAIN (t), idx++)
if (TREE_CODE (t) == FIELD_DECL)
{
list[idx].index = idx;
list[idx].decl = t;
list[idx].acc_sites = NULL;
list[idx].count = 0;
list[idx].field_mapping = NULL_TREE;
}
return list;
}
/* Auxiliary function for remove_struct_from_data_struct_list. */
static void
free_struct_cluster (struct field_cluster* cluster)
{
if (cluster)
{
if (cluster->fields_in_cluster)
sbitmap_free (cluster->fields_in_cluster);
if (cluster->sibling)
free_struct_cluster (cluster->sibling);
free (cluster);
}
}
/* The following three functions:
add_struct_to_data_struct_list
remove_struct_from_data_struct_list
free_data_structure_list
is an interface for manipulation with the list of data
structures to be transformed which is accessed through
data_struct_list. */
static void
add_struct_to_data_struct_list (tree type)
{
struct data_structure *d_node;
struct struct_list *new_node = NULL;
int num_fields;
list found = NULL;
type = TYPE_MAIN_VARIANT (type);
found = find_struct_in_list (type);
if (found)
return;
d_node = (struct data_structure *)
xcalloc (1, sizeof (struct data_structure));
num_fields = fields_length (type);
d_node->decl = type;
d_node->num_fields = num_fields;
d_node->fields = get_fields (type, num_fields);
d_node->struct_clustering = NULL;
d_node->accs = NULL;
new_node = (struct struct_list *) xmalloc (sizeof(struct struct_list));
new_node->struct_data = d_node;
new_node->next = data_struct_list;
data_struct_list = new_node;
if (dump_file)
{
fprintf (dump_file, "\nAdding data structure \"");
print_generic_expr (dump_file, type, 0);
fprintf (dump_file, "\" to data_struct_list.");
}
}
static void
free_tree_list (tree_list list_p)
{
tree_list node = list_p;
tree_list tmp_node;
while (node)
{
tmp_node = node;
node = node->next;
free (tmp_node);
}
}
static void
free_field_acc_list (struct field_access_site *accs)
{
struct field_access_site * tmp_acc = accs;
struct field_access_site * tmp_acc1;
if (!accs)
return;
while (tmp_acc)
{
tmp_acc1 = tmp_acc;
tmp_acc = tmp_acc->next;
free (tmp_acc1);
}
}
static void
free_access_list (struct access_site *accs)
{
struct access_site * tmp_acc = accs;
struct access_site * tmp_acc1;
if (!accs)
return;
while (tmp_acc)
{
tmp_acc1 = tmp_acc;
tmp_acc = tmp_acc->next;
free_tree_list (tmp_acc1->vars);
free (tmp_acc1);
}
}
static void
free_data_struct (d_str d_node)
{
int i;
if (!d_node)
return;
if (dump_file)
{
fprintf (dump_file, "\nRemoving data structure \"");
print_generic_expr (dump_file, d_node->decl, 0);
fprintf (dump_file, "\" from data_struct_list.");
}
/* Free all space under d_node. */
if (d_node->fields)
{
for (i = 0; i < d_node->num_fields; i++)
free_field_acc_list (d_node->fields[i].acc_sites);
free (d_node->fields);
}
if (d_node->accs)
free_access_list (d_node->accs);
if (d_node->struct_clustering)
free_struct_cluster (d_node->struct_clustering);
if (d_node->new_types)
free_tree_list (d_node->new_types);
}
static void
remove_struct_from_data_struct_list (struct struct_list *str_to_remove)
{
struct struct_list *curr_str, *prev;
if (data_struct_list == str_to_remove)
{
free_data_struct (data_struct_list->struct_data);
data_struct_list = data_struct_list->next;
free (str_to_remove);
return;
}
for (curr_str = data_struct_list->next, prev = data_struct_list;
curr_str; curr_str = curr_str->next, prev = prev->next)
if (curr_str == str_to_remove)
{
free_data_struct (curr_str->struct_data);
prev->next = curr_str->next;
free (curr_str);
break;
}
}
static void
remove_structs_from_data_struct_list (bitmap nested_structs)
{
list tmp = data_struct_list;
while (tmp)
{
d_str str = tmp->struct_data;
if (bitmap_bit_p (nested_structs, TYPE_UID (str->decl)))
{
list tmp1 = tmp;
tmp = tmp->next;
remove_struct_from_data_struct_list (tmp1);
}
else
tmp = tmp->next;
}
}
static void
free_data_structure_list (void)
{
list current_struct = data_struct_list;
while (current_struct)
{
list tmp = current_struct;
d_str d_node = current_struct->struct_data;
free_data_struct (d_node);
current_struct = current_struct->next;
/* Free curent_struct itself. */
if (tmp)
free (tmp);
}
}
static void
free_malloc_data (void)
{
struct malloc_struct_new *cur_malloc = malloc_data_list_new;
struct malloc_struct_new *tmp;
while (cur_malloc)
{
struct malloc_call_new * call = cur_malloc->malloc_list;
while (call)
{
struct malloc_call_new * call1 = call;
call = call->next;
free (call1);
}
tmp = cur_malloc;
cur_malloc = cur_malloc->next;
free (tmp);
}
}
/* This function calculates maximum distance between
fields in structure according to their counters
collected through profiling. */
static gcov_type
get_max_struct_dist (d_str str)
{
gcov_type max = 0, new_max = 0;
int i, j;
for (i = 0; i < str->num_fields; i++)
for (j = i+1; j < str->num_fields; j++)
{
new_max = abs (str->fields[i].count - str->fields[j].count);
if (new_max > max)
max = new_max;
}
return max;
}
static void
peel_str_type_into_seperate_fields (d_str ds)
{
struct field_cluster *crr_cluster = NULL;
int i;
gcov_type max_dist_in_cluster;
sbitmap fields_left = sbitmap_alloc (ds->num_fields);
sbitmap add_to_cluster = sbitmap_alloc (ds->num_fields);
sbitmap_ones (fields_left);
max_dist_in_cluster =
(gcov_type) (get_max_struct_dist (ds)/100)*MAX_DIST_IN_CLUSTER_RATIO;
i = sbitmap_first_set_bit (fields_left);
while (i != -1)
{
unsigned int j = 0;
sbitmap_iterator sbi;
sbitmap_zero (add_to_cluster);
ds->struct_clustering =
(struct field_cluster *) xcalloc (1, sizeof (struct field_cluster));
ds->struct_clustering->sibling = crr_cluster;
crr_cluster = ds->struct_clustering;
crr_cluster->fields_in_cluster =
sbitmap_alloc ((unsigned int) ds->num_fields);
sbitmap_zero (crr_cluster->fields_in_cluster);
SET_BIT (crr_cluster->fields_in_cluster, i);
RESET_BIT (fields_left, i);
EXECUTE_IF_SET_IN_SBITMAP (fields_left, 0, j, sbi)
{
unsigned int k;
sbitmap_iterator sbi1;
bool suitable = true;
EXECUTE_IF_SET_IN_SBITMAP (crr_cluster->fields_in_cluster, 0, k, sbi1)
{
unsigned int dist = abs (ds->fields[j].count - ds->fields[k].count);
if (dist >= max_dist_in_cluster)
suitable = false;
}
if (suitable)
SET_BIT (add_to_cluster, j);
}
j = 0;
EXECUTE_IF_SET_IN_SBITMAP (add_to_cluster, 0, j, sbi)
{
SET_BIT (crr_cluster->fields_in_cluster, j);
RESET_BIT (fields_left, j);
}
i = sbitmap_first_set_bit (fields_left);
}
sbitmap_free (add_to_cluster);
sbitmap_free (fields_left);
}
/* This function analyzes structure form of structures
in data_struct_list. If we are not capable to transform
structure of some form, we remove it from list of structs.
Right now we cannot handle nested structs, when nesting is
through any level of pointer or arrays.
TBD: release these constrains in future.
Note, that in this function we suppose that all structs
in program are members of data_struct_list right now,
without excluding escaping types.
*/
static void
analyze_struct_form (bitmap non_suitable_types)
{
list tmp = data_struct_list;
while(tmp)
{
d_str str = tmp->struct_data;
int i;
for (i = 0; i < str->num_fields; i++)
{
tree f_type = strip_type(TREE_TYPE (str->fields[i].decl));
if (TREE_CODE (f_type) == RECORD_TYPE)
{
bitmap_set_bit (non_suitable_types, TYPE_UID (TYPE_MAIN_VARIANT (f_type)));
bitmap_set_bit (non_suitable_types, TYPE_UID (str->decl));
}
}
tmp = tmp->next;
}
}
/* This function collect data structures potential
for peeling transformation and insert them into data_struct_list. */
static void
collect_data_structs (void)
{
bitmap non_suitable_types = BITMAP_ALLOC (NULL);
/* If program contains user defined mallocs, we give up. */
if (program_redefines_malloc_p ())
return;
/* Build data structures list of all data structures
in the program. */
build_data_structure_list (non_suitable_types);
/* In this function we analyze whether the form of
structure is such that we are capable to transform it.
Nested structures are checked here. */
analyze_struct_form (non_suitable_types);
/* This function excludes structure types escape compilation unit. */
exclude_escaping_types (non_suitable_types);
/* We cannot change data layout of structs with bitfields. */
exclude_types_with_bit_fields (non_suitable_types);
remove_structs_from_data_struct_list (non_suitable_types);
BITMAP_FREE (non_suitable_types);
if (!data_struct_list)
{
if (dump_file)
fprintf (dump_file, "\nNo structures to transform. Exiting...");
return;
}
}
static void
print_field_acc_sites (struct field_access_site *acc_sites)
{
struct field_access_site *acc;
if (!dump_file)
return;
for (acc = acc_sites; acc; acc = acc->next)
{
fprintf(dump_file, "\n");
if (acc->stmt)
print_generic_stmt (dump_file, acc->stmt, 0);
if (acc->ref_def_stmt)
print_generic_stmt (dump_file, acc->ref_def_stmt, 0);
if (acc->cast_stmt)
print_generic_stmt (dump_file, acc->cast_stmt, 0);
}
}
static void
print_access_sites (struct access_site *acc_sites)
{
struct access_site *acc;
if (!dump_file)
return;
for (acc = acc_sites; acc; acc = acc->next)
{
tree_list list_p;
fprintf(dump_file, "\n");
if (acc->stmt)
print_generic_stmt (dump_file, acc->stmt, 0);
fprintf(dump_file, " : ");
for (list_p = acc->vars; list_p; list_p = list_p->next)
{
print_generic_stmt (dump_file, list_p->data, 0);
fprintf(dump_file, ", ");
}
}
}
static void
print_accesses (void)
{
list tmp = data_struct_list;
if (!dump_file)
return;
for (tmp = data_struct_list; tmp; tmp = tmp->next)
{
int i;
d_str str = tmp->struct_data;
fprintf (dump_file, "\nAccess sites of struct ");
print_generic_expr (dump_file, str->decl, 0);
for (i = 0; i < str->num_fields; i++)
{
fprintf (dump_file, "\nAccess site of field ");
print_generic_expr (dump_file, str->fields[i].decl, 0);
print_field_acc_sites (str->fields[i].acc_sites);
fprintf (dump_file, ":\n");
}
fprintf (dump_file, "\nGeneral access sites\n");
print_access_sites (str->accs);
}
}
static bool
is_safe_cond_expr (tree cond_stmt)
{
tree arg0, arg1;
list tmp0, tmp1;
tree cond = COND_EXPR_COND (cond_stmt);
if (TREE_CODE (cond) != EQ_EXPR
&& TREE_CODE (cond) != NE_EXPR)
return false;
if (TREE_CODE_LENGTH (TREE_CODE (cond)) != 2)
return false;
arg0 = TREE_OPERAND (cond, 0);
arg1 = TREE_OPERAND (cond, 1);
tmp0 = find_struct_in_list (strip_type (get_type_of_var (arg0)));
tmp1 = find_struct_in_list (strip_type (get_type_of_var (arg1)));
if (!((!tmp0 && tmp1) || (!tmp1 && tmp0)))
return false;
return true;
}
static void
check_cond_exprs (void)
{
list str_node = data_struct_list;
while (str_node)
{
d_str str = str_node->struct_data;
struct access_site * acc;
list str_node1 = str_node;
str_node = str_node->next;
for (acc = str->accs; acc; acc = acc->next)
if (TREE_CODE (acc->stmt) == COND_EXPR)
{
if (!is_safe_cond_expr (acc->stmt))
remove_struct_from_data_struct_list (str_node1);
}
}
}
/* This function collects data accesses for the structures
from data_struct_list. For each structure field it updates its count
in data_field_entry. */
static void
collect_data_accesses (void)
{
struct cgraph_node *c_node;
for (c_node = cgraph_nodes; c_node; c_node = c_node->next)
{
enum availability avail = cgraph_function_body_availability (c_node);
if (avail == AVAIL_LOCAL || avail == AVAIL_AVAILABLE)
{
struct function *func = DECL_STRUCT_FUNCTION (c_node->decl);
if (!c_node->next_clone)
{
/* Collect accesses of the fields of structure
under consideretion that happen in func. */
collect_accesses_in_func (func);
}
exclude_malloc_and_field_accs (c_node);
}
}
check_cond_exprs ();
/* Print collected accesses. */
print_accesses ();
}
static void
exclude_cold_structs (void)
{
gcov_type hotest_struct_count = 0;
list tmp = data_struct_list;
/* We summarize counts of fields into structure count. */
while (tmp)
{
int i;
d_str str = tmp->struct_data;
str->count = 0;
for (i = 0; i < str->num_fields; i++)
{
if (dump_file)
{
fprintf (dump_file, "\nCounter of field \"");
print_generic_expr (dump_file, str->fields[i].decl, 0);
fprintf (dump_file, "\" is " HOST_WIDE_INT_PRINT_DEC,
str->fields[i].count);
}
str->count += str->fields[i].count;
}
if (dump_file)
{
fprintf (dump_file, "\nCounter of struct \"");
print_generic_expr (dump_file, str->decl, 0);
fprintf (dump_file, "\" is " HOST_WIDE_INT_PRINT_DEC, str->count);
}
if (str->count > hotest_struct_count)
hotest_struct_count = str->count;
tmp = tmp->next;
}
/* Compare counts of structures. */
tmp = data_struct_list;
/* Remove cold structs from data_struct_list. */
while (tmp)
{
d_str str = tmp->struct_data;
if (str->count < (hotest_struct_count / COLD_STRUCTURE_RATIO))
{
list tmp1 = tmp;
tmp = tmp->next;
remove_struct_from_data_struct_list (tmp1);
}
else
tmp = tmp->next;
}
}
static void
peel_structs (void)
{
list tmp = data_struct_list;
while (tmp)
{
d_str str = tmp->struct_data;
peel_str_type_into_seperate_fields (str);
tmp = tmp->next;
}
}
/* Collect allocation sites - mallocs. In case of arrays
we have nothing to do. */
static void
collect_allocation_sites (void)
{
collect_malloc_data ();
}
/* Free all data structures used in this optimization. */
static void
free_data_structs (void)
{
free_data_structure_list ();
free_malloc_data ();
}
/* Perform full structure decomposition (peeling). */
static void
reorg_structs (void)
{
/* Stage1. */
/* Collect data struct types. */
collect_data_structs ();
collect_allocation_sites ();
/* Collect data accesses. */
collect_data_accesses ();
/* We transform only hot structs. */
exclude_cold_structs ();
/* Stage2. */
peel_structs ();
/* Stage3. */
/* Do the actual transformation. */
do_reorg ();
/* Free all data structures used in this optimization. */
free_data_structs ();
}
static void
add_call_to_malloc_list (tree fn_decl, tree stmt, d_str str)
{
struct malloc_struct_new *cur_malloc = NULL;
struct malloc_struct_new *malloc_node = NULL;
for (cur_malloc = malloc_data_list_new; cur_malloc;
cur_malloc = cur_malloc->next)
if (cur_malloc->func == fn_decl)
break;
if (cur_malloc)
{
struct malloc_call_new * m_call = (struct malloc_call_new *)
xmalloc (sizeof (struct malloc_call_new));
m_call->next = cur_malloc->malloc_list;
m_call->malloc_call_stmt = stmt;
m_call->str = str;
cur_malloc->malloc_list = m_call;
}
else
{
malloc_node = (struct malloc_struct_new *)
xmalloc (sizeof (struct malloc_struct_new));
malloc_node->func = fn_decl;
malloc_node->malloc_list = (struct malloc_call_new *)
xmalloc (sizeof (struct malloc_call_new));
malloc_node->malloc_list->malloc_call_stmt = stmt;
malloc_node->malloc_list->str = str;
malloc_node->malloc_list->next = NULL;
malloc_node->next = malloc_data_list_new;
malloc_data_list_new = malloc_node;
}
if (dump_file)
{
fprintf (dump_file, "\nAdding stmt ");
print_generic_stmt (dump_file, stmt, 0);
fprintf (dump_file, " to list of mallocs.");
}
}
static unsigned int
reorg_structs_drive (void)
{
reorg_structs ();
return 0;
}
static bool
struct_reorg_gate (void)
{
return flag_ipa_struct_reorg && flag_whole_program;
}
struct tree_opt_pass pass_ipa_struct_reorg =
{
"ipa_struct_reorg", /* name */
struct_reorg_gate, /* gate */
reorg_structs_drive, /* execute */
NULL, /* sub */
NULL, /* next */
0, /* static_pass_number */
TV_INTEGRATION, /* tv_id */
0, /* properties_required */
0, /* properties_provided */
0, /* properties_destroyed */
TODO_verify_ssa, /* todo_flags_start */
TODO_dump_func | TODO_verify_ssa, /* todo_flags_finish */
0 /* letter */
};
[-- Attachment #4: struct-reorg.h --]
[-- Type: application/octet-stream, Size: 3720 bytes --]
/* Struct-reorg optimization.
Copyright (C) 2002, 2003-2007 Free Software Foundation, Inc.
Contributed by Olga Golovanevsky <olga@il.ibm.com>
This file is part of GCC.
GCC is free software; you can redistribute it and/or modify
under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
GCC is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with GCC; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
#ifndef IPA_REORG_STRUCTS_H
#define IPA_REORG_STRUCTS_H
/* This file describes the data structure and interface for the data
reorganization optimization. */
struct struct_tree_list {
tree data;
struct struct_tree_list *next;
};
typedef struct struct_tree_list * tree_list;
/* Represents an access site to a data field.
We consider access of the following form:
D.2166_21 = i.6_20 * 8;
D.2167_22 = (struct str_t *) D.2166_21;
D.2168_24 = D.2167_22 + p.5_23;
D.2169_25 = D.2168_24->b;
*/
struct field_access_site
{
/* The statement in which the access site occurs. */
tree stmt; /* D.2169_25 = D.2168_24->b; */
tree comp_ref; /* D.2168_24->b */
tree field_decl; /* b */
tree ref; /* D.2168_24 */
tree num; /* i.6_20 */
tree offset; /* D2167_22 */
tree base; /* p.5_23 */
tree ref_def_stmt; /* D.2168_24 = D.2167_22 + p.5_23; */
tree cast_stmt; /* D.2167_22 = (struct str_t *) D.2166_21;
This stmt is not always present. */
struct field_access_site *next;
};
struct access_site
{
/* The statement in which the access site occurs. */
tree stmt;
/* List of struct vars in stmt. */
tree_list vars;
struct access_site *next;
};
/* Represents a field within a data structure and its accesses. */
struct data_field_entry {
/* Field index, that starts with zero. */
int index;
/* The number of times the field is accessed (according to profiling). */
gcov_type count;
tree decl;
/* Type of new structure this field belongs to. */
tree field_mapping;
struct field_access_site *acc_sites;
};
/* This structure represent result of peeling. */
struct field_cluster {
/* Bitmap of field indexes, a bit is set when the field with that index
is in the cluster. */
sbitmap fields_in_cluster;
struct field_cluster *sibling;
};
/* Represents a data structure that is candidate for
clustering. */
struct data_structure {
/* Main variant of type declaration of the structure. */
tree decl;
/* Number of fields in the structure. */
int num_fields;
/* According to profiling information number of times fields of the
structure were accessed. */
gcov_type count;
/* Array of field entries one for each field of the structure,
indexed by the field ID. */
struct data_field_entry *fields;
/* Stmts with struct accesses that are not a part of field
accesses or allocation patterns. */
struct access_site *accs;
/* This is the a data structure representing result of peeling. */
struct field_cluster *struct_clustering;
/* A linked list of the newly created types, corresponding to the
reorganization of the original structure. */
tree_list new_types;
};
typedef struct data_structure * d_str;
#endif /* IPA_REORG_STRUCTS_H */
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: struct-reorg optimization
2007-07-10 9:35 struct-reorg optimization Olga Golovanevsky
@ 2007-07-19 22:57 ` Daniel Berlin
2007-08-01 12:06 ` Olga Golovanevsky
2007-07-20 14:39 ` Jan Hubicka
1 sibling, 1 reply; 17+ messages in thread
From: Daniel Berlin @ 2007-07-19 22:57 UTC (permalink / raw)
To: Olga Golovanevsky; +Cc: Diego Novillo, Kenneth Zadeck, Jan Hubicka, gcc-patches
On 7/10/07, Olga Golovanevsky <OLGA@il.ibm.com> wrote:
>
> This patch implements structure peeling, that is one of struct-reorg
> optimizations developed on struct-reorg-branch. A structure type can be
> peeled into separate fields, or groups of them, according to profile
> information, or deliberately. The gain is to utilize spatial locality,
> when, for example, one of the fields is sequentially accessed through
> array elements. Then separating this field from others allows to make
> separation in array allocation, and thus release cache from irrelevant
> data fetched with other fields.
>
> The full description of this optimization is at the beginning of
> struct-reorg.c (attached below).
>
> The optimization is global, both types of accesses - through pointers
> and array refs - are handled. When allocation is made through malloc,
> it is replaced by allocations of peeled structures. Only build-in malloc
> is supported right now, but it can be easily extended.
>
> In this patch, the decision how to peel is made by frequency based model,
> where frequency of field accesses can be calculated through profiling or
> statically predicted as with -fbranch-probabilitied flag. When there is
> no such info, structure gets completely peeled. (The struct-reorg-branch
> contains much more accurate model which is based on the calculation of
> distance between accesses, but it's also heavier.)
> The -fipa-struct-reorg flag activates this optimization.
>
> The patch makes strong use of ipa-type-escape analysis, that provide
> safety of this optimization. Also changes required for ipa-type-escape
> to support POITER_PLUS_EXPR are part of this patch.
>
> With this patch:
> - two structures f1_neuron and xyz got peeled in 179. art ,
> +44% w/o profiling and +33% with profiling
>
> - the structures NEXT_MOVE got peeled, the structure CHESS_POSITION
> can be peeled, but interfere with another optimization that
> generated BIT_FIELD_REFs in 186.crafty; many other structures escape,
> no influence on performance
>
> - the structure MMNODE got peeled; many others escape,
> no influence on performance
>
> Bootstraped and tested (except fortran) on ppc .
Comments on struct-reorg.c:
1. The comments in strip_t ype do not make clear what it does
Just based on looking at it, it should be using POINTER_TYPE_P, not
TREE_CODE (type) == POINTER_TYPE
2. why is get_type_name returning a char * and not a const char *?
3. why is get_type_var using DECL_ARG_TYPE?
4.
if (SSA_NAME_IS_DEFAULT_DEF(before_cast))
return false;
Should be
if (SSA_NAME_IS_DEFAULT_DEF (before_cast))
return false;
5. your offset/base splitting code seems to be trying to approximate
exactly what get_ref_and_base_extent will do for you.
6. In build_basic_struct, why are you touching C_* fields?
7. Don't use CONSTANT_CLASS_P and simple_cst_equal, just call
operand_equal_p on rhs
8. It seems you assume struct sizes fit in unsigned int in a lot of
places, when they should actually be HOST_WIDE_INT
I see no problems with the changes you make in the patch (ie the patch
to ipa-type-escape.c)
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: struct-reorg optimization
2007-07-10 9:35 struct-reorg optimization Olga Golovanevsky
2007-07-19 22:57 ` Daniel Berlin
@ 2007-07-20 14:39 ` Jan Hubicka
2007-08-01 11:31 ` Olga Golovanevsky
1 sibling, 1 reply; 17+ messages in thread
From: Jan Hubicka @ 2007-07-20 14:39 UTC (permalink / raw)
To: Olga Golovanevsky
Cc: Daniel Berlin, Diego Novillo, Kenneth Zadeck, Jan Hubicka, gcc-patches
Olga,
I made first pass through the patch. It seems sane overall, I will get
into more detail on the new pass itself in followup.
Can you please split ipa-type-escape changes into separate patch? I
think those can go in quite easilly independently.
Since you started on teaching ipa-type-escape about SSA world, it would
be nice to move it away from walk_tree into use of SSA operand
infrastructure like our regular IPA passes does. But that would be
independent patch anyway.
Honza
>
> This patch implements structure peeling, that is one of struct-reorg
> optimizations developed on struct-reorg-branch. A structure type can be
> peeled into separate fields, or groups of them, according to profile
> information, or deliberately. The gain is to utilize spatial locality,
> when, for example, one of the fields is sequentially accessed through
> array elements. Then separating this field from others allows to make
> separation in array allocation, and thus release cache from irrelevant
> data fetched with other fields.
>
> The full description of this optimization is at the beginning of
> struct-reorg.c (attached below).
>
> The optimization is global, both types of accesses - through pointers
> and array refs - are handled. When allocation is made through malloc,
> it is replaced by allocations of peeled structures. Only build-in malloc
> is supported right now, but it can be easily extended.
>
> In this patch, the decision how to peel is made by frequency based model,
> where frequency of field accesses can be calculated through profiling or
> statically predicted as with -fbranch-probabilitied flag. When there is
> no such info, structure gets completely peeled. (The struct-reorg-branch
> contains much more accurate model which is based on the calculation of
> distance between accesses, but it's also heavier.)
> The -fipa-struct-reorg flag activates this optimization.
>
> The patch makes strong use of ipa-type-escape analysis, that provide
> safety of this optimization. Also changes required for ipa-type-escape
> to support POITER_PLUS_EXPR are part of this patch.
>
> With this patch:
> - two structures f1_neuron and xyz got peeled in 179. art ,
> +44% w/o profiling and +33% with profiling
>
> - the structures NEXT_MOVE got peeled, the structure CHESS_POSITION
> can be peeled, but interfere with another optimization that
> generated BIT_FIELD_REFs in 186.crafty; many other structures escape,
> no influence on performance
>
> - the structure MMNODE got peeled; many others escape,
> no influence on performance
>
> Bootstraped and tested (except fortran) on ppc .
>
> :ADDPATCH ipa ssa:
>
> Ok for mainline?
>
> 2007-07-09 Olga Golovanevsky <olga@il.ibm.com>
>
> * struct-reorg.c, struct-reorg.h: New files.
> * ipa-type-escape.h: Expose function
> is_array_access_through_pointer_and_index.
> * ipa-type-escape.c
> (is_array_access_through_pointer_and_index):
> Add three new parameters. Add support of
> POINTER_PLUS_EXPR code.
> * tree.h: Add pass_ipa_struct_reorg.
> * common.opt: Add ipa-struct-reorg flag.
> * Makefile.in: Add strcut-reorg.o compilation.
> * passes.c: Add pass pass_ipa_struct_reorg.
>
> (See attached file: struct-reorg.txt)(See attached file: struct-reorg.c)
> (See attached file: struct-reorg.h)
> Index: tree-pass.h
> ===================================================================
> --- tree-pass.h (revision 126190)
> +++ tree-pass.h (working copy)
> @@ -331,6 +331,7 @@
> extern struct tree_opt_pass pass_ipa_pure_const;
> extern struct tree_opt_pass pass_ipa_type_escape;
> extern struct tree_opt_pass pass_ipa_pta;
> +extern struct tree_opt_pass pass_ipa_struct_reorg;
> extern struct tree_opt_pass pass_early_local_passes;
> extern struct tree_opt_pass pass_ipa_increase_alignment;
> extern struct tree_opt_pass pass_ipa_function_and_variable_visibility;
> Index: ipa-type-escape.c
> ===================================================================
> --- ipa-type-escape.c (revision 126190)
> +++ ipa-type-escape.c (working copy)
> @@ -926,56 +926,90 @@
>
> */
>
> -static bool
> -is_array_access_through_pointer_and_index (tree op0, tree op1)
> +bool
> +is_array_access_through_pointer_and_index (enum tree_code code, tree op0, tree op1,
> + tree * base, tree * offset,
> + tree * offset_cast_stmt)
> {
> - tree base, offset, offset_cast_stmt;
> tree before_cast, before_cast_def_stmt;
> cast_t op0_cast, op1_cast;
>
> + *base = NULL;
> + *offset = NULL;
> + *offset_cast_stmt = NULL;
> +
> /* Check 1. */
>
> - /* Init data for walk_use_def_chains function. */
> - op0_cast.type = op1_cast.type = 0;
> - op0_cast.stmt = op1_cast.stmt = NULL;
> + if (code == POINTER_PLUS_EXPR)
> + {
> + tree op0type = TYPE_MAIN_VARIANT (TREE_TYPE (op0));
> + tree op1type = TYPE_MAIN_VARIANT (TREE_TYPE (op1));
>
> - visited_stmts = pointer_set_create ();
> - walk_use_def_chains (op0, is_cast_from_non_pointer,(void *)(&op0_cast), false);
> - pointer_set_destroy (visited_stmts);
> + /* One of op0 and op1 is of pointer type and the other is numerical. */
> + if (POINTER_TYPE_P (op0type)
> + && NUMERICAL_TYPE_CHECK (op1type))
> + {
> + *base = op0;
> + *offset = op1;
> + }
> + else if (POINTER_TYPE_P (op1type)
> + && NUMERICAL_TYPE_CHECK (op0type))
> + {
> + *base = op1;
> + *offset = op0;
> + }
> + else
> + return false;
> + }
> + else
> + {
> + /* Init data for walk_use_def_chains function. */
> + op0_cast.type = op1_cast.type = 0;
> + op0_cast.stmt = op1_cast.stmt = NULL;
>
> - visited_stmts = pointer_set_create ();
> - walk_use_def_chains (op1, is_cast_from_non_pointer,(void *)(&op1_cast), false);
> - pointer_set_destroy (visited_stmts);
> + visited_stmts = pointer_set_create ();
> + walk_use_def_chains (op0, is_cast_from_non_pointer,(void *)(&op0_cast), false);
> + pointer_set_destroy (visited_stmts);
>
> - if (op0_cast.type == 1 && op1_cast.type == 0)
> - {
> - base = op1;
> - offset = op0;
> - offset_cast_stmt = op0_cast.stmt;
> + visited_stmts = pointer_set_create ();
> + walk_use_def_chains (op1, is_cast_from_non_pointer,(void *)(&op1_cast), false);
> + pointer_set_destroy (visited_stmts);
> +
> + if (op0_cast.type == 1 && op1_cast.type == 0)
> + {
> + *base = op1;
> + *offset = op0;
> + *offset_cast_stmt = op0_cast.stmt;
> + }
> + else if (op0_cast.type == 0 && op1_cast.type == 1)
> + {
> + *base = op0;
> + *offset = op1;
> + *offset_cast_stmt = op1_cast.stmt;
> + }
> + else
> + return false;
> }
> - else if (op0_cast.type == 0 && op1_cast.type == 1)
> - {
> - base = op0;
> - offset = op1;
> - offset_cast_stmt = op1_cast.stmt;
> - }
> - else
> - return false;
>
> /* Check 2.
> offset_cast_stmt is of the form:
> D.1606_7 = (struct str_t *) D.1605_6; */
>
> - before_cast = SINGLE_SSA_TREE_OPERAND (offset_cast_stmt, SSA_OP_USE);
> - if (!before_cast)
> - return false;
> + if (*offset_cast_stmt)
> + {
> + before_cast = SINGLE_SSA_TREE_OPERAND (*offset_cast_stmt, SSA_OP_USE);
> + if (!before_cast)
> + return false;
>
> - if (SSA_NAME_IS_DEFAULT_DEF(before_cast))
> - return false;
> + if (SSA_NAME_IS_DEFAULT_DEF(before_cast))
> + return false;
>
> - before_cast_def_stmt = SSA_NAME_DEF_STMT (before_cast);
> - if (!before_cast_def_stmt)
> - return false;
> + before_cast_def_stmt = SSA_NAME_DEF_STMT (before_cast);
> + if (!before_cast_def_stmt)
> + return false;
> + }
> + else
> + before_cast_def_stmt = SSA_NAME_DEF_STMT (*offset);
>
> /* before_cast_def_stmt should be of the form:
> D.1605_6 = i.1_5 * 16; */
> @@ -1449,7 +1483,6 @@
> okay_pointer_operation (enum tree_code code, tree op0, tree op1)
> {
> tree op0type = TYPE_MAIN_VARIANT (TREE_TYPE (op0));
> - tree op1type = TYPE_MAIN_VARIANT (TREE_TYPE (op1));
>
> switch (code)
> {
> @@ -1459,11 +1492,15 @@
> break;
> case MINUS_EXPR:
> case PLUS_EXPR:
> + case POINTER_PLUS_EXPR:
> {
> - if (POINTER_TYPE_P (op1type)
> + tree base, offset, offset_cast_stmt;
> +
> + if (POINTER_TYPE_P (op0type)
> && TREE_CODE (op0) == SSA_NAME
> && TREE_CODE (op1) == SSA_NAME
> - && is_array_access_through_pointer_and_index (op0, op1))
> + && is_array_access_through_pointer_and_index (code, op0, op1, &base,
> + &offset, &offset_cast_stmt))
> return true;
> else
> {
> @@ -1541,7 +1578,7 @@
> of circumstances and if the moon is in the correct
> place could be safe, but it is hard to see how this
> is worth the effort. */
> -
> +
> if (type0 && POINTER_TYPE_P (type0)
> && !okay_pointer_operation (TREE_CODE (rhs), op0, op1))
> mark_interesting_type (type0, FULL_ESCAPE);
> Index: ipa-type-escape.h
> ===================================================================
> --- ipa-type-escape.h (revision 126190)
> +++ ipa-type-escape.h (working copy)
> @@ -27,6 +27,7 @@
> bool ipa_type_escape_field_does_not_clobber_p (tree record_type, tree field_type);
> int ipa_type_escape_star_count_of_interesting_type (tree type);
> int ipa_type_escape_star_count_of_interesting_or_array_type (tree type);
> +bool is_array_access_through_pointer_and_index (enum tree_code, tree, tree, tree *, tree *, tree *);
>
>
> #endif /* GCC_IPA_TYPE_ESCAPE_H */
> Index: common.opt
> ===================================================================
> --- common.opt (revision 126190)
> +++ common.opt (working copy)
> @@ -601,6 +601,11 @@
> Perform matrix layout flattening and transposing based
> on profiling information.
>
> +fipa-struct-reorg
> +Common Report Var(flag_ipa_struct_reorg)
> +Perform structure layout optimizations based
> +on profiling information.
> +
> fivopts
> Common Report Var(flag_ivopts) Init(1) Optimization
> Optimize induction variables on trees
> Index: Makefile.in
> ===================================================================
> --- Makefile.in (revision 126190)
> +++ Makefile.in (working copy)
> @@ -1185,6 +1185,7 @@
> ipa-utils.o \
> ipa.o \
> matrix-reorg.o \
> + struct-reorg.o \
> tree-inline.o \
> tree-nomudflap.o \
> varpool.o
> @@ -2449,6 +2450,10 @@
> pointer-set.h $(GGC_H) $(IPA_TYPE_ESCAPE_H) $(IPA_UTILS_H) $(C_COMMON_H) \
> $(TREE_GIMPLE_H) $(CGRAPH_H) output.h $(FLAGS_H) tree-pass.h \
> $(DIAGNOSTIC_H) $(FUNCTION_H)
> +struct-reorg.o: struct-reorg.c struct-reorg.h $(CONFIG_H) $(SYSTEM_H) coretypes.h \
> + $(TM_H) $(RTL_H) $(TREE_H) $(FLAGS_H) output.h $(REGS_H) $(EXPR_H) $(FUNCTION_H) \
> + toplev.h $(GGC_H) $(TARGET_H) langhooks.h $(COVERAGE_H) libfuncs.h \
> + gt-coverage.h $(HASHTAB_H) $(IPA_TYPE_ESCAPE_H)
>
> coverage.o : coverage.c $(GCOV_IO_H) $(CONFIG_H) $(SYSTEM_H) coretypes.h \
> $(TM_H) $(RTL_H) $(TREE_H) $(FLAGS_H) output.h $(REGS_H) $(EXPR_H) \
> @@ -3011,7 +3016,7 @@
> $(srcdir)/reload.h \
> $(srcdir)/alias.c $(srcdir)/bitmap.c $(srcdir)/cselib.c $(srcdir)/cgraph.c \
> $(srcdir)/ipa-prop.c $(srcdir)/ipa-cp.c $(srcdir)/ipa-inline.c $(srcdir)/matrix-reorg.c \
> - $(srcdir)/dbxout.c $(srcdir)/dwarf2out.c $(srcdir)/dwarf2asm.c \
> + $(srcdir)/dbxout.c $(srcdir)/struct-reorg.c $(srcdir)/dwarf2out.c $(srcdir)/dwarf2asm.c \
> $(srcdir)/dojump.c \
> $(srcdir)/emit-rtl.c $(srcdir)/except.c $(srcdir)/explow.c $(srcdir)/expr.c \
> $(srcdir)/function.c $(srcdir)/except.h \
> Index: passes.c
> ===================================================================
> --- passes.c (revision 126190)
> +++ passes.c (working copy)
> @@ -542,6 +542,7 @@
> NEXT_PASS (pass_ipa_pure_const);
> NEXT_PASS (pass_ipa_type_escape);
> NEXT_PASS (pass_ipa_pta);
> + NEXT_PASS (pass_ipa_struct_reorg);
> *p = NULL;
>
> /* These passes are run after IPA passes on every function that is being
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: struct-reorg optimization
2007-07-20 14:39 ` Jan Hubicka
@ 2007-08-01 11:31 ` Olga Golovanevsky
0 siblings, 0 replies; 17+ messages in thread
From: Olga Golovanevsky @ 2007-08-01 11:31 UTC (permalink / raw)
To: Jan Hubicka
Cc: Andrew Pinski, Daniel Berlin, Diego Novillo, Kenneth Zadeck, gcc-patches
Jan Hubicka <hubicka@ucw.cz> wrote on 20/07/2007 16:48:01:
>
> Can you please split ipa-type-escape changes into separate patch? I
> think those can go in quite easilly independently.
>
This is a patch for ipa-type-escape. I extended it with POINTER_PLUS_EXPR
and exposed is_array_access_through_pointer_and_index function
as I am using it in struct-reorg analysis.
Ok for mainline?
Olga
2007-08-01 Olga Golovanevsky <olga@il.ibm.com>
* ipa-type-escape.h: Expose function
is_array_access_through_pointer_and_index.
* ipa-type-escape.c
(is_array_access_through_pointer_and_index):
Add three new parameters. Add support of
POINTER_PLUS_EXPR tree code.
Index: ipa-type-escape.c
===================================================================
--- ipa-type-escape.c (revision 126912)
+++ ipa-type-escape.c (working copy)
@@ -926,56 +926,93 @@ is_cast_from_non_pointer (tree var, tree
*/
-static bool
-is_array_access_through_pointer_and_index (tree op0, tree op1)
+bool
+is_array_access_through_pointer_and_index (enum tree_code code, tree op0,
+ tree op1, tree * base,
+ tree * offset,
+ tree * offset_cast_stmt)
{
- tree base, offset, offset_cast_stmt;
tree before_cast, before_cast_def_stmt;
cast_t op0_cast, op1_cast;
- /* Check 1. */
-
- /* Init data for walk_use_def_chains function. */
- op0_cast.type = op1_cast.type = 0;
- op0_cast.stmt = op1_cast.stmt = NULL;
+ *base = NULL;
+ *offset = NULL;
+ *offset_cast_stmt = NULL;
- visited_stmts = pointer_set_create ();
- walk_use_def_chains (op0, is_cast_from_non_pointer,(void *)(&op0_cast),
false);
- pointer_set_destroy (visited_stmts);
-
- visited_stmts = pointer_set_create ();
- walk_use_def_chains (op1, is_cast_from_non_pointer,(void *)(&op1_cast),
false);
- pointer_set_destroy (visited_stmts);
+ /* Check 1. */
- if (op0_cast.type == 1 && op1_cast.type == 0)
+ if (code == POINTER_PLUS_EXPR)
{
- base = op1;
- offset = op0;
- offset_cast_stmt = op0_cast.stmt;
+ tree op0type = TYPE_MAIN_VARIANT (TREE_TYPE (op0));
+ tree op1type = TYPE_MAIN_VARIANT (TREE_TYPE (op1));
+
+ /* One of op0 and op1 is of pointer type and the other is numerical.
*/
+ if (POINTER_TYPE_P (op0type)
+ && NUMERICAL_TYPE_CHECK (op1type))
+ {
+ *base = op0;
+ *offset = op1;
+ }
+ else if (POINTER_TYPE_P (op1type)
+ && NUMERICAL_TYPE_CHECK (op0type))
+ {
+ *base = op1;
+ *offset = op0;
+ }
+ else
+ return false;
}
- else if (op0_cast.type == 0 && op1_cast.type == 1)
+ else
{
- base = op0;
- offset = op1;
- offset_cast_stmt = op1_cast.stmt;
+ /* Init data for walk_use_def_chains function. */
+ op0_cast.type = op1_cast.type = 0;
+ op0_cast.stmt = op1_cast.stmt = NULL;
+
+ visited_stmts = pointer_set_create ();
+ walk_use_def_chains (op0, is_cast_from_non_pointer,
+ (void *)(&op0_cast), false);
+ pointer_set_destroy (visited_stmts);
+
+ visited_stmts = pointer_set_create ();
+ walk_use_def_chains (op1, is_cast_from_non_pointer,
+ (void *)(&op1_cast), false);
+ pointer_set_destroy (visited_stmts);
+
+ if (op0_cast.type == 1 && op1_cast.type == 0)
+ {
+ *base = op1;
+ *offset = op0;
+ *offset_cast_stmt = op0_cast.stmt;
+ }
+ else if (op0_cast.type == 0 && op1_cast.type == 1)
+ {
+ *base = op0;
+ *offset = op1;
+ *offset_cast_stmt = op1_cast.stmt;
+ }
+ else
+ return false;
}
- else
- return false;
/* Check 2.
offset_cast_stmt is of the form:
D.1606_7 = (struct str_t *) D.1605_6; */
- before_cast = SINGLE_SSA_TREE_OPERAND (offset_cast_stmt, SSA_OP_USE);
- if (!before_cast)
- return false;
+ if (*offset_cast_stmt)
+ {
+ before_cast = SINGLE_SSA_TREE_OPERAND (*offset_cast_stmt,
SSA_OP_USE);
+ if (!before_cast)
+ return false;
- if (SSA_NAME_IS_DEFAULT_DEF(before_cast))
- return false;
+ if (SSA_NAME_IS_DEFAULT_DEF(before_cast))
+ return false;
- before_cast_def_stmt = SSA_NAME_DEF_STMT (before_cast);
- if (!before_cast_def_stmt)
- return false;
+ before_cast_def_stmt = SSA_NAME_DEF_STMT (before_cast);
+ if (!before_cast_def_stmt)
+ return false;
+ }
+ else
+ before_cast_def_stmt = SSA_NAME_DEF_STMT (*offset);
/* before_cast_def_stmt should be of the form:
D.1605_6 = i.1_5 * 16; */
@@ -1449,7 +1486,6 @@ static bool
okay_pointer_operation (enum tree_code code, tree op0, tree op1)
{
tree op0type = TYPE_MAIN_VARIANT (TREE_TYPE (op0));
- tree op1type = TYPE_MAIN_VARIANT (TREE_TYPE (op1));
switch (code)
{
@@ -1459,11 +1495,17 @@ okay_pointer_operation (enum tree_code c
break;
case MINUS_EXPR:
case PLUS_EXPR:
+ case POINTER_PLUS_EXPR:
{
- if (POINTER_TYPE_P (op1type)
+ tree base, offset, offset_cast_stmt;
+
+ if (POINTER_TYPE_P (op0type)
&& TREE_CODE (op0) == SSA_NAME
&& TREE_CODE (op1) == SSA_NAME
- && is_array_access_through_pointer_and_index (op0, op1))
+ && is_array_access_through_pointer_and_index (code, op0,
+ op1, &base,
+ &offset,
+ &offset_cast_stmt))
return true;
else
{
@@ -1541,7 +1583,7 @@ scan_for_refs (tree *tp, int *walk_subtr
of circumstances and if the moon is in the correct
place could be safe, but it is hard to see how this
is worth the effort. */
-
+
if (type0 && POINTER_TYPE_P (type0)
&& !okay_pointer_operation (TREE_CODE (rhs), op0, op1))
mark_interesting_type (type0, FULL_ESCAPE);
Index: ipa-type-escape.h
===================================================================
--- ipa-type-escape.h (revision 126912)
+++ ipa-type-escape.h (working copy)
@@ -27,6 +27,8 @@ bool ipa_type_escape_type_contained_p
bool ipa_type_escape_field_does_not_clobber_p (tree record_type, tree
field_type);
int ipa_type_escape_star_count_of_interesting_type (tree type);
int ipa_type_escape_star_count_of_interesting_or_array_type (tree
type);
+bool is_array_access_through_pointer_and_index (enum tree_code, tree,
tree,
+ tree *, tree *, tree *);
#endif /* GCC_IPA_TYPE_ESCAPE_H */
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: struct-reorg optimization
2007-07-19 22:57 ` Daniel Berlin
@ 2007-08-01 12:06 ` Olga Golovanevsky
2007-08-17 22:25 ` Jan Hubicka
0 siblings, 1 reply; 17+ messages in thread
From: Olga Golovanevsky @ 2007-08-01 12:06 UTC (permalink / raw)
To: Daniel Berlin, Jan Hubicka
Cc: Diego Novillo, Kenneth Zadeck, Peter Bergner, gcc-patches
[-- Attachment #1: Type: text/plain, Size: 1937 bytes --]
"Daniel Berlin" <dberlin@dberlin.org> wrote on 20/07/2007 00:22:33:
>
> Comments on struct-reorg.c:
>
>
> 1. The comments in strip_t ype do not make clear what it does
> Just based on looking at it, it should be using POINTER_TYPE_P, not
> TREE_CODE (type) == POINTER_TYPE
done.
> 2. why is get_type_name returning a char * and not a const char *?
changed, done.
> 3. why is get_type_var using DECL_ARG_TYPE?
because get_type_var receives function parameters as arguments as well.
> 4.
> if (SSA_NAME_IS_DEFAULT_DEF(before_cast))
> return false;
> Should be
> if (SSA_NAME_IS_DEFAULT_DEF (before_cast))
> return false;
fixed.
> 5. your offset/base splitting code seems to be trying to approximate
> exactly what get_ref_and_base_extent will do for you.
>
not exactly, as I need offset/base variables there.
> 6. In build_basic_struct, why are you touching C_* fields?
fixed.
>
> 7. Don't use CONSTANT_CLASS_P and simple_cst_equal, just call
> operand_equal_p on rhs
fixed.
>
> 8. It seems you assume struct sizes fit in unsigned int in a lot of
> places, when they should actually be HOST_WIDE_INT
fixed.
This is an updated patch. In addition to changes mentioned above,
by suggestion of Peter Bergner, I have changes the model to
peel hot fields one by one instead of clustering them. It scales
better, and now we are getting 47% on art with and w/o profiling, i.e.
when by default we completely peel the structure into separate fields.
Any further comments? Ok for mainline?
:ADDPATCH ipa ssa:
Olga
2007-08-01 Olga Golovanevsky <olga@il.ibm.com>
* struct-reorg.c, struct-reorg.h: New files.
* tree.h: Add pass_ipa_struct_reorg.
* common.opt: Add ipa-struct-reorg flag.
* Makefile.in: Add strcut-reorg.o compilation.
* passes.c: Add pass pass_ipa_struct_reorg.
(See attached file: struct-reorg.h)(See attached file: struct_reorg.txt)
(See attached file: struct-reorg.c)
[-- Attachment #2: struct-reorg.h --]
[-- Type: application/octet-stream, Size: 3837 bytes --]
/* Struct-reorg optimization.
Copyright (C) 2002, 2003-2007 Free Software Foundation, Inc.
Contributed by Olga Golovanevsky <olga@il.ibm.com>
This file is part of GCC.
GCC is free software; you can redistribute it and/or modify
under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
GCC is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with GCC; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
#ifndef IPA_REORG_STRUCTS_H
#define IPA_REORG_STRUCTS_H
/* This file describes the data structure and interface for the data
reorganization optimization. */
struct struct_tree_list {
tree data;
struct struct_tree_list *next;
};
typedef struct struct_tree_list * tree_list;
/* Represents an access site to a data field.
We consider access of the following form:
D.2166_21 = i.6_20 * 8;
D.2167_22 = (struct str_t *) D.2166_21;
D.2168_24 = D.2167_22 + p.5_23;
D.2169_25 = D.2168_24->b;
*/
struct field_access_site
{
/* The statement in which the access site occurs. */
tree stmt; /* D.2169_25 = D.2168_24->b; */
tree comp_ref; /* D.2168_24->b */
tree field_decl; /* b */
tree ref; /* D.2168_24 */
tree num; /* i.6_20 */
tree offset; /* D2167_22 */
tree base; /* p.5_23 */
tree ref_def_stmt; /* D.2168_24 = D.2167_22 + p.5_23; */
tree cast_stmt; /* D.2167_22 = (struct str_t *) D.2166_21;
This stmt is not always present. */
struct field_access_site *next;
};
struct access_site
{
/* The statement in which the access site occurs. */
tree stmt;
/* List of struct vars in stmt. */
tree_list vars;
struct access_site *next;
};
/* Represents a field within a data structure and its accesses. */
struct data_field_entry {
/* Field index, that starts with zero. */
int index;
/* The number of times the field is accessed (according to profiling). */
gcov_type count;
tree decl;
/* Type of new structure this field belongs to. */
tree field_mapping;
struct field_access_site *acc_sites;
};
/* This structure represent result of peeling. */
struct field_cluster {
/* Bitmap of field indexes, a bit is set when the field with that index
is in the cluster. */
sbitmap fields_in_cluster;
struct field_cluster *sibling;
};
/* Represents a data structure that is candidate for
clustering. */
struct data_structure {
/* Main variant of type declaration of the structure. */
tree decl;
/* Number of fields in the structure. */
int num_fields;
/* According to profiling information number of times fields of the
structure were accessed. */
gcov_type count;
/* Array of field entries one for each field of the structure,
indexed by the field ID. */
struct data_field_entry *fields;
/* Stmts with struct accesses that are not a part of field
accesses or allocation patterns. */
struct access_site *accs;
/* This is the a data structure representing result of peeling. */
struct field_cluster *struct_clustering;
/* A linked list of the newly created types, corresponding to the
reorganization of the original structure. */
tree_list new_types;
};
typedef struct data_structure * d_str;
#endif /* IPA_REORG_STRUCTS_H */
[-- Attachment #3: struct_reorg.txt --]
[-- Type: text/plain, Size: 3221 bytes --]
Index: tree-pass.h
===================================================================
--- tree-pass.h (revision 126912)
+++ tree-pass.h (working copy)
@@ -331,6 +331,7 @@ extern struct tree_opt_pass pass_ipa_ref
extern struct tree_opt_pass pass_ipa_pure_const;
extern struct tree_opt_pass pass_ipa_type_escape;
extern struct tree_opt_pass pass_ipa_pta;
+extern struct tree_opt_pass pass_ipa_struct_reorg;
extern struct tree_opt_pass pass_early_local_passes;
extern struct tree_opt_pass pass_ipa_increase_alignment;
extern struct tree_opt_pass pass_ipa_function_and_variable_visibility;
Index: common.opt
===================================================================
--- common.opt (revision 126912)
+++ common.opt (working copy)
@@ -601,6 +601,11 @@ Common Report Var(flag_ipa_matrix_reorg)
Perform matrix layout flattening and transposing based
on profiling information.
+fipa-struct-reorg
+Common Report Var(flag_ipa_struct_reorg)
+Perform structure layout optimizations based
+on profiling information.
+
fivopts
Common Report Var(flag_ivopts) Init(1) Optimization
Optimize induction variables on trees
Index: Makefile.in
===================================================================
--- Makefile.in (revision 126912)
+++ Makefile.in (working copy)
@@ -1215,6 +1215,7 @@ OBJS-archive = \
ipa-utils.o \
ipa.o \
matrix-reorg.o \
+ struct-reorg.o \
tree-inline.o \
tree-nomudflap.o \
varpool.o
@@ -2480,6 +2481,10 @@ ipa-type-escape.o : ipa-type-escape.c $(
pointer-set.h $(GGC_H) $(IPA_TYPE_ESCAPE_H) $(IPA_UTILS_H) $(C_COMMON_H) \
$(TREE_GIMPLE_H) $(CGRAPH_H) output.h $(FLAGS_H) tree-pass.h \
$(DIAGNOSTIC_H) $(FUNCTION_H)
+struct-reorg.o: struct-reorg.c struct-reorg.h $(CONFIG_H) $(SYSTEM_H) coretypes.h \
+ $(TM_H) $(RTL_H) $(TREE_H) $(FLAGS_H) output.h $(REGS_H) $(EXPR_H) $(FUNCTION_H) \
+ toplev.h $(GGC_H) $(TARGET_H) langhooks.h $(COVERAGE_H) libfuncs.h \
+ gt-coverage.h $(HASHTAB_H) $(IPA_TYPE_ESCAPE_H)
coverage.o : coverage.c $(GCOV_IO_H) $(CONFIG_H) $(SYSTEM_H) coretypes.h \
$(TM_H) $(RTL_H) $(TREE_H) $(FLAGS_H) output.h $(REGS_H) $(EXPR_H) \
@@ -3044,7 +3049,7 @@ GTFILES = $(srcdir)/input.h $(srcdir)/co
$(srcdir)/reload.h $(srcdir)/caller-save.c \
$(srcdir)/alias.c $(srcdir)/bitmap.c $(srcdir)/cselib.c $(srcdir)/cgraph.c \
$(srcdir)/ipa-prop.c $(srcdir)/ipa-cp.c $(srcdir)/ipa-inline.c $(srcdir)/matrix-reorg.c \
- $(srcdir)/dbxout.c $(srcdir)/dwarf2out.c $(srcdir)/dwarf2asm.c \
+ $(srcdir)/dbxout.c $(srcdir)/struct-reorg.c $(srcdir)/dwarf2out.c $(srcdir)/dwarf2asm.c \
$(srcdir)/dojump.c \
$(srcdir)/emit-rtl.c $(srcdir)/except.c $(srcdir)/explow.c $(srcdir)/expr.c \
$(srcdir)/function.c $(srcdir)/except.h \
Index: passes.c
===================================================================
--- passes.c (revision 126912)
+++ passes.c (working copy)
@@ -542,6 +542,7 @@ init_optimization_passes (void)
NEXT_PASS (pass_ipa_pure_const);
NEXT_PASS (pass_ipa_type_escape);
NEXT_PASS (pass_ipa_pta);
+ NEXT_PASS (pass_ipa_struct_reorg);
*p = NULL;
/* These passes are run after IPA passes on every function that is being
[-- Attachment #4: struct-reorg.c --]
[-- Type: application/octet-stream, Size: 86743 bytes --]
/* Struct-reorg optimization.
Copyright (C) 2004, 2005, 2006, 2007 Free Software Foundation, Inc.
Contributed by Olga Golovanevsky <olga@il.ibm.com>
(initial version of this code was developed
by Caroline Tice and Mostafa Hagog)
This file is part of GCC.
GCC is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free
Software Foundation; either version 2, or (at your option) any later
version.
GCC is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
for more details.
You should have received a copy of the GNU General Public License
along with GCC; see the file COPYING. If not, write to the Free
Software Foundation, 59 Temple Place - Suite 330, Boston, MA
02111-1307, USA. */
#include "config.h"
#include "system.h"
#include "coretypes.h"
#include "tm.h"
#include "tree.h"
#include "rtl.h"
#include "tree-gimple.h"
#include "tree-inline.h"
#include "tree-flow.h"
#include "tree-flow-inline.h"
#include "langhooks.h"
#include "pointer-set.h"
#include "hashtab.h"
#include "c-tree.h"
#include "toplev.h"
#include "flags.h"
#include "ggc.h"
#include "debug.h"
#include "target.h"
#include "cgraph.h"
#include "diagnostic.h"
#include "timevar.h"
#include "params.h"
#include "fibheap.h"
#include "intl.h"
#include "function.h"
#include "basic-block.h"
#include "tree-iterator.h"
#include "tree-pass.h"
#include "struct-reorg.h"
#include "opts.h"
#include "ipa-type-escape.h"
#include "tree-dump.h"
#include "c-common.h"
/* This optimization implements structure peeling.
For example, given a structure type:
typedef struct
{
int a;
float b;
int c;
}str_t;
it can be peeled into two structure types as follows:
typedef struct and typedef struct
{ {
int a; float b;
int c; } str_t_1;
}str_t_0;
or can be fully peeled:
typedef struct
{
int a;
}str_t_0;
typedef struct
{
float b;
}str_t_1;
typedef struct
{
int c;
}str_t_2;
When structure type is peeled all instances and their accesses
in the program are updated accordingly. For example, if there is
array of structures:
str_t A[N];
and structure type str_t was peeled into two structures str_t_0
and str_t_1 as it was shown above, then array A will be replaced
by two arrays as foolows:
str_t_0 A_0[N];
str_t_1 A_1[N];
The field access of fields a of ellement i of array A: A[i].a will be
replace by access to field a of ellement i of array A_0: A_0[i].a.
This optimization also supports dynamically allocated arrays.
If array of structures was allocated by malloc function:
str_t * p = (str_t *) malloc (sizeof (str_t) * N)
the allocation site will be replaced to reflect new structure types:
str_t_0 * p_0 = (str_t_0 *) malloc (sizeof (str_t_0) * N)
str_t_1 * p_1 = (str_t_1 *) malloc (sizeof (str_t_1) * N)
The field access through the pointer p[i].a will be changed by p_0[i].a.
The decision how to peel structure gains to utilize spatial locality.
For example, if one of fields of structure has intensive use in the loop:
for (i = 0; i < N; i++)
{
... = A[i].a;
}
the allocation of field a of str_t in the memory in contiguous manner will
increase probablity of relevant data to be fatched into cache.
The analysis part of this optimization is based on the frequency of
field accesses, which are collected all over the program.
Then the fields with the frequencies that satisfy the following condition
got peeled out of the structure:
freq(f) > C * max_field_freq_in_struct
where max_field_freq_in_struct is maximum field frequency in structure.
The C is a constant defining which portion of max_field_freq_in_struct
the filds should have in order to be peeled.
If profiling information is provided, its is used to calculate the
frequency of field accesses. Otherwise, structure is fully peeled.
The ipa-type-escape analysis are used to provide safety of the peeling.
The optimization is activated by flag -fipa-struct-reorg.
*/
/* Ratio of the structure count to the hottest
structure count to consider the structure cold. */
#define COLD_STRUCTURE_RATIO 10
/* Ratio of maximum distance between fields allowed in cluster.
Used only when running with profiling. */
#define MAX_DIST_IN_CLUSTER_RATIO 1
/* Data about the new program variables created (for the new peeled types).
When doing struct peeling, each variable of the original struct type will
need to be replaced with a set of new variables, one new variable for each
new type created from the original type. */
struct new_var_data {
tree orig_var; /* Var decl for original struct type */
struct struct_tree_list *new_vars; /* List of new vars to replace the
original var; one new var for each
new type peeled off the original
type. */
struct new_var_data *next;
};
typedef struct new_var_data * new_var;
struct malloc_call_new
{
tree malloc_call_stmt;
d_str str;
/* Next malloc call stmt in the same function. */
struct malloc_call_new *next;
};
/* List of data for all malloc calls in the program. */
struct malloc_struct_new
{
tree func;
struct malloc_call_new *malloc_list; /* List of call sites for each
call to malloc from the current
func. */
struct malloc_struct_new *next;
};
typedef struct malloc_struct_new *malloc_d;
typedef struct malloc_call_new *malloc_call;
struct malloc_struct_new *malloc_data_list_new = NULL;
/* List of new global variables. Generated once for whole program. */
new_var new_global_vars = NULL;
/* List of new local variables. Generated once for function. */
new_var new_local_vars = NULL;
/* Use stmts of current function. */
tree_list use_stmts = NULL;
/* List of structures to be transformed. */
struct struct_list
{
struct data_structure *struct_data;
struct struct_list *next;
};
typedef struct struct_list * list;
/* List of structs to be transformed. */
list data_struct_list = NULL;
static new_var is_in_new_vars_list (tree, new_var);
static void add_struct_to_data_struct_list (tree);
static void add_call_to_malloc_list (tree, tree, d_str);
static inline void update_fields_mapping (struct field_cluster *, tree,
struct data_field_entry *, int);
static void create_new_var (tree, new_var *);
static tree get_final_malloc_stmt (tree malloc_stmt);
static tree_list add_tree_to_tree_list (tree, tree_list);
static void free_tree_list (tree_list);
static malloc_d get_malloc_list_for_func (tree);
static struct field_access_site * add_field_acc_to_acc_sites (tree, tree, tree, tree,
tree, tree, tree, tree,
tree, struct field_access_site *);
static struct access_site * add_access_to_acc_sites (tree, tree,
struct access_site *);
static bool is_result_of_mult (tree, tree *, tree);
static void remove_struct_from_data_struct_list (list);
static tree gen_size (tree, tree, tree *);
static tree gen_cast_stmt (tree, tree, tree, tree *);
static void create_new_stmts_for_general_acc (struct access_site *, d_str);
static void create_new_stmts_for_cond_expr (tree);
static bool is_equal_types (tree, tree);
static tree * find_pos_in_stmt (tree, tree);
/* Strip type from pointers and arrays. */
static inline tree
strip_type (tree type)
{
gcc_assert (TYPE_P (type));
while (POINTER_TYPE_P (type)
|| TREE_CODE (type) == ARRAY_TYPE)
type = TREE_TYPE (type);
return type;
}
/* Returns type of VAR. */
static inline tree
get_type_of_var (tree var)
{
if (!var)
return NULL;
if (TREE_CODE (var) == PARM_DECL)
return DECL_ARG_TYPE (var);
else
return TREE_TYPE (var);
}
/* Check whether the type of VAR is suitable for peeling.
Returns true if yes, false otherwise. */
static bool
is_candidate_for_data_struct_list (tree var, tree *type_p,
bitmap non_suitable_types)
{
tree type;
bool initialized = false;
*type_p = NULL;
if (!var)
return false;
/* There is no support of initialized vars. */
if (TREE_CODE (var) == VAR_DECL
&& DECL_INITIAL (var) != NULL_TREE)
initialized = true;
type = get_type_of_var (var);
if (type)
{
type = TYPE_MAIN_VARIANT (strip_type (type));
*type_p = type;
if (initialized && non_suitable_types)
bitmap_set_bit (non_suitable_types, TYPE_UID (type));
if (TREE_CODE (type) == RECORD_TYPE)
{
return true;
}
else
return false;
}
else
return false;
}
/* Given a tree representing a type, this function returns the
name of the type, as a string. */
static const char *
get_type_name (tree type_node)
{
if (! TYPE_NAME (type_node))
return NULL;
if (TREE_CODE (TYPE_NAME (type_node)) == IDENTIFIER_NODE)
return (char *) IDENTIFIER_POINTER (TYPE_NAME (type_node));
else if (TREE_CODE (TYPE_NAME (type_node)) == TYPE_DECL
&& DECL_NAME (TYPE_NAME (type_node)))
return (char *) IDENTIFIER_POINTER (DECL_NAME (TYPE_NAME (type_node)));
else
return NULL;
}
static void
print_new_vars (new_var new_vars)
{
new_var current;
tree_list cur_var;
tree var_type;
if (!dump_file)
return;
for (current = new_vars; current; current = current->next)
{
var_type = get_type_of_var (current->orig_var);
fprintf (dump_file, "\nOrig var: ");
print_generic_expr (dump_file, current->orig_var, 0);
fprintf (dump_file, " of type ");
print_generic_expr (dump_file, var_type, 0);
fprintf (dump_file, "\n");
for (cur_var = current->new_vars; cur_var; cur_var = cur_var->next)
{
tree var = cur_var->data;
var_type = get_type_of_var (var);
fprintf (dump_file, " ");
print_generic_expr (dump_file, var, 0);
fprintf (dump_file, " of type ");
print_generic_expr (dump_file, var_type, 0);
fprintf (dump_file, "\n");
}
}
}
/* Print structure type. */
static void
print_struct_type (tree new_type, char * indent)
{
char * new_indent;
const char * struct_name;
tree cur_field;
int len;
int i;
if (!new_type || !dump_file)
return;
if (TREE_CODE (new_type) != RECORD_TYPE)
return;
struct_name = get_type_name (new_type);
if (struct_name)
{
fprintf (dump_file, "%s%s {\n", indent, struct_name);
len = strlen (struct_name) + strlen (indent) + 3;
}
else
{
fprintf (dump_file, "%s{\n", indent);
len = strlen (indent) + 2;
}
new_indent = (char *) xmalloc (len * sizeof (char));
memset (new_indent, ' ', len);
new_indent[len] = '\0';
for (cur_field = TYPE_FIELDS (new_type); cur_field;
cur_field = TREE_CHAIN (cur_field))
{
tree field_type;
int ptr_count = 0;
field_type = TREE_TYPE (cur_field);
while (POINTER_TYPE_P (field_type))
{
ptr_count++;
field_type = TREE_TYPE (field_type);
}
fprintf (dump_file, "%s%s ", new_indent, get_type_name (field_type));
for (i = 0; i < ptr_count; i++)
fprintf (dump_file, "*");
fprintf (dump_file, " %s;\n",
IDENTIFIER_POINTER (DECL_NAME (cur_field)));
}
fprintf (dump_file, "%s}\n", indent);
}
/* Print out the list of new types generated by this optimization. */
static void
print_new_types (void)
{
list tmp;
char indent[3] = " ";
if (!dump_file)
return;
fprintf (dump_file,
"\nThe following are the new types generated by"
" this optimization:\n");
for (tmp = data_struct_list; tmp; tmp = tmp->next)
{
d_str str = tmp->struct_data;
tree_list new_type;
for (new_type = str->new_types; new_type; new_type = new_type->next)
print_struct_type (new_type->data, indent);
}
}
static tree
find_field_in_struct_1 (tree str_type, tree field)
{
tree str_field;
for (str_field = TYPE_FIELDS (str_type); str_field;
str_field = TREE_CHAIN (str_field))
{
char * str_field_name;
char * field_name;
str_field_name = (char *) IDENTIFIER_POINTER (DECL_NAME (str_field));
field_name = (char *) IDENTIFIER_POINTER (DECL_NAME (field));
gcc_assert (str_field_name);
gcc_assert (field_name);
if (!strcmp (str_field_name, field_name))
{
/* Check their types. */
if (is_equal_types (TREE_TYPE (str_field), TREE_TYPE (field)))
return str_field;
}
}
return NULL_TREE;
}
/* Given the field declaration tree (FIELD_DECL) returns the
appropriate field entry in DS. */
static struct data_field_entry *
find_field_in_struct (d_str str, tree field_decl)
{
int i;
tree field = find_field_in_struct_1 (str->decl, field_decl);
for (i = 0; i < str->num_fields; i++)
if (str->fields[i].decl == field)
return &(str->fields[i]);
return NULL;
}
/* This function is a hack to overcome the types problem.
When number of compilation units are compiled together
under -combine flag, the TYPE_MAIN_VARIANT cannot
be structed. First we comparing names, but they do not
also appear, like for example, in stmts. Only then we
are going inside structures.
*/
static bool
is_equal_types (tree type1, tree type2)
{
const char * name1,* name2;
if ((!type1 && type2)
||(!type2 && type1))
return false;
if (!type1 && !type2)
return true;
if (TREE_CODE (type1) != TREE_CODE (type2))
return false;
if (TYPE_MAIN_VARIANT (type1) == TYPE_MAIN_VARIANT (type2))
return true;
name1 = get_type_name (type1);
name2 = get_type_name (type2);
if (name1 && name2 && !strcmp (name1, name2))
return true;
if (name1 && name2 && strcmp (name1, name2))
return false;
switch (TREE_CODE (type1))
{
case POINTER_TYPE:
case REFERENCE_TYPE:
{
return is_equal_types (TREE_TYPE (type1), TREE_TYPE (type2));
}
break;
case RECORD_TYPE:
case UNION_TYPE:
case QUAL_UNION_TYPE:
case ENUMERAL_TYPE:
{
tree field1;
/* Compare fields of struture. */
for (field1 = TYPE_FIELDS (type1); field1;
field1 = TREE_CHAIN (field1))
{
tree field2 = find_field_in_struct_1 (type2, field1);
if (!field2)
return false;
}
return true;
}
break;
default:
gcc_unreachable();
}
return false;
}
/* This function returns data_structure entry
in the data_struct_list represented by TYPE, if found. */
static list
find_struct_in_list (tree type)
{
list tmp = data_struct_list;
while (tmp)
{
if (is_equal_types (tmp->struct_data->decl, TYPE_MAIN_VARIANT (type)))
return tmp;
tmp = tmp->next;
}
return NULL;
}
/* Check whether the indirect_ref is of the form we can deal with. */
static bool
is_complicated_indirect_ref (tree str_decl, tree ref, tree * num_p, tree * offset_p,
tree * base_p, tree * ref_def_stmt_p, tree * cast_stmt_p)
{
tree ref_var;
tree rhs, struct_size, op0, op1;
tree before_cast;
ref_var = TREE_OPERAND (ref, 0);
if (TREE_CODE (ref_var) != SSA_NAME)
return true;
*ref_def_stmt_p = SSA_NAME_DEF_STMT (ref_var);
if (!(*ref_def_stmt_p)
|| (TREE_CODE (*ref_def_stmt_p) != GIMPLE_MODIFY_STMT))
return true;
rhs = GIMPLE_STMT_OPERAND (*ref_def_stmt_p, 1);
if (TREE_CODE (rhs) != PLUS_EXPR
&& TREE_CODE (rhs)!= MINUS_EXPR
&& TREE_CODE (rhs) != POINTER_PLUS_EXPR)
return true;
op0 = TREE_OPERAND (rhs, 0);
op1 = TREE_OPERAND (rhs, 1);
if (!is_array_access_through_pointer_and_index (TREE_CODE (rhs), op0, op1,
base_p, offset_p,
cast_stmt_p))
return true;
if (*cast_stmt_p)
before_cast = SINGLE_SSA_TREE_OPERAND (*cast_stmt_p, SSA_OP_USE);
else
before_cast = *offset_p;
if (!before_cast)
return false;
if (SSA_NAME_IS_DEFAULT_DEF (before_cast))
return false;
struct_size = TYPE_SIZE_UNIT (str_decl);
if (!is_result_of_mult (before_cast, num_p, struct_size))
return true;
return false;
}
static bool
is_complicated_access (tree str_decl, tree ref, tree * num_p, tree * offset_p,
tree * base_p, tree * ref_def_stmt_p, tree * cast_stmt_p)
{
if (TREE_CODE (ref) == INDIRECT_REF)
return is_complicated_indirect_ref (str_decl, ref, num_p, offset_p,
base_p, ref_def_stmt_p, cast_stmt_p);
else if (TREE_CODE (ref) == ARRAY_REF)
return false;
else if (TREE_CODE (ref) == VAR_DECL)
return false;
return true;
}
/* Helper for walk_tree called from collect_accesses_in_bb function. */
static tree
get_stmt_accesses (tree *tp, int *walk_subtrees, void *data)
{
tree stmt = (tree) data;
tree t = *tp;
if (!t)
return NULL;
switch (TREE_CODE (t))
{
case BIT_FIELD_REF:
{
tree str = TREE_OPERAND(t, 0);
tree type = TYPE_MAIN_VARIANT (strip_type (get_type_of_var (str)));
list tmp = find_struct_in_list (type);
if (tmp)
remove_struct_from_data_struct_list (tmp);
}
break;
case GIMPLE_MODIFY_STMT:
{
tree lhs = GIMPLE_STMT_OPERAND (t, 0);
tree rhs = GIMPLE_STMT_OPERAND (t, 1);
*walk_subtrees=1;
walk_tree (&lhs, get_stmt_accesses, data, NULL);
walk_tree (&rhs, get_stmt_accesses, data, NULL);
*walk_subtrees=0;
}
break;
case COMPONENT_REF:
{
tree ref = TREE_OPERAND (t, 0);
tree field_decl = TREE_OPERAND (t, 1);
if ((TREE_CODE (ref) == INDIRECT_REF
|| TREE_CODE (ref) == ARRAY_REF
|| TREE_CODE (ref) == VAR_DECL)
&& TREE_CODE (field_decl) == FIELD_DECL)
{
tree type = TYPE_MAIN_VARIANT (TREE_TYPE (ref));
list tmp = find_struct_in_list (type);
if (tmp)
{
d_str str = tmp->struct_data;
struct data_field_entry * field =
find_field_in_struct (str, field_decl);
if (field)
{
tree num = NULL;
tree offset = NULL;
tree base = NULL;
tree ref_def_stmt = NULL;
tree cast_stmt = NULL;
/* Check whether the access is of the form we can deal with. */
if (is_complicated_access (str->decl, ref, &num, &offset,
&base, &ref_def_stmt, &cast_stmt))
remove_struct_from_data_struct_list (tmp);
else
{
/* Increase count of field. */
basic_block bb = bb_for_stmt (stmt);
field->count += bb->count;
/* Add stmt to the acc_sites list of field. */
field->acc_sites =
add_field_acc_to_acc_sites (stmt, ref, t, field_decl, num, offset,
base, ref_def_stmt, cast_stmt,
field->acc_sites);
}
*walk_subtrees = 0;
}
}
}
}
break;
case MINUS_EXPR:
case PLUS_EXPR:
{
tree op0 = TREE_OPERAND (t, 0);
tree op1 = TREE_OPERAND (t, 1);
*walk_subtrees=1;
walk_tree (&op0, get_stmt_accesses, data, NULL);
walk_tree (&op1, get_stmt_accesses, data, NULL);
*walk_subtrees=0;
}
break;
case COND_EXPR:
{
tree cond = COND_EXPR_COND (t);
int i;
for (i = 0; i < TREE_CODE_LENGTH (TREE_CODE (cond)); i++)
{
tree t = TREE_OPERAND (cond, i);
*walk_subtrees=1;
walk_tree (&t, get_stmt_accesses, data, NULL);
}
*walk_subtrees=0;
}
break;
case VAR_DECL:
case SSA_NAME:
{
list tmp;
if (TREE_CODE (t) == SSA_NAME)
t = SSA_NAME_VAR (t);
tmp = find_struct_in_list (strip_type (get_type_of_var (t)));
if (tmp)
{
d_str str = tmp->struct_data;
str->accs =
add_access_to_acc_sites (stmt, t, str->accs);
}
*walk_subtrees=0;
}
break;
case CALL_EXPR:
{
/* It was checked as part of stage1 that structs
to be transformed cannot be passed as parameters to function. */
*walk_subtrees=0;
}
break;
default:
return NULL;
}
return NULL;
}
/* Collect accesses to the fields of DS in BB. */
static void
collect_accesses_in_bb (basic_block bb)
{
block_stmt_iterator bsi;
/* Go over the basic block statements collecting
accesses to fields of structures. */
for (bsi = bsi_start (bb); ! bsi_end_p (bsi); bsi_next (&bsi))
{
tree stmt = bsi_stmt (bsi);
walk_tree (&stmt, get_stmt_accesses, stmt, NULL);
}
}
/* Collect accesses of the fields of the structures in function FN. */
static void
collect_accesses_in_func (struct function *fn)
{
basic_block bb;
if (! fn)
return;
/* Collect accesses in each one of the basic blocks. */
FOR_EACH_BB_FN (bb, fn)
collect_accesses_in_bb (bb);
}
/* This functions builds structure with fields FIELDS,
that should be TREE_CHAIN, named NAME, with the same
attributes as ORIG_STRUCT. */
static tree
build_basic_struct (tree fields, char * name, tree orig_struct)
{
tree attributes = NULL_TREE;
tree ref = 0;
tree x;
if (TYPE_ATTRIBUTES (orig_struct))
attributes = copy_node (TYPE_ATTRIBUTES (orig_struct));
ref = make_node (RECORD_TYPE);
TYPE_SIZE (ref) = 0;
decl_attributes (&ref, attributes, (int) ATTR_FLAG_TYPE_IN_PLACE);
TYPE_PACKED (ref) = TYPE_PACKED (orig_struct);
for (x = fields; x; x = TREE_CHAIN (x))
{
DECL_CONTEXT (x) = ref;
DECL_PACKED (x) |= TYPE_PACKED (ref);
}
TYPE_FIELDS (ref) = fields;
layout_type (ref);
TYPE_NAME (ref) = get_identifier (name);
return ref;
}
/* Generate new structure name. */
static inline char *
gen_cluster_name (tree decl, int clust_num, int str_num)
{
char * old_name = (char *) get_type_name (decl);
char * new_name;
if (!old_name)
{
old_name = (char *) xmalloc ((10) * sizeof (char));
sprintf (old_name, "struct_%d", str_num);
}
new_name = (char *) xmalloc ((strlen (old_name) + 11) * sizeof (char));
sprintf (new_name, "%s_sub_%d", old_name, clust_num);
/* Check that this name was not used before. */
while (maybe_get_identifier (new_name))
{
int len = strlen (new_name) + 3;
char * tmp_name = (char *) xmalloc (len * sizeof (char));
sprintf (tmp_name, "%s.0", new_name);
new_name = tmp_name;
}
return new_name;
}
/* This function copies fields from CLUSTER into TREE_CHAIN as part
of preparation for new structure building. */
static tree
create_fields (struct field_cluster * cluster,
struct data_field_entry * fields, int num_fields)
{
int i;
tree new_types = NULL_TREE;
tree last = NULL_TREE;
for (i = 0; i < num_fields; i++)
if (TEST_BIT (cluster->fields_in_cluster, i))
{
tree new_decl = copy_node (fields[i].decl);
if (!new_types)
new_types = new_decl;
else
TREE_CHAIN (last) = new_decl;
last = new_decl;
}
TREE_CHAIN (last) = NULL_TREE;
return new_types;
}
/* Update field_mapping of fields in CLUSTER with NEW_TYPE. */
static inline void
update_fields_mapping (struct field_cluster *cluster, tree new_type,
struct data_field_entry * fields, int num_fields)
{
int i;
for (i = 0; i < num_fields; i++)
if (TEST_BIT (cluster->fields_in_cluster, i))
fields[i].field_mapping = new_type;
}
/* This function creates new types according to struct clustering data. */
static void
create_new_types (void)
{
list tmp = data_struct_list;
int str_num = 0;
while (tmp)
{
int cluster_num = 0;
d_str str = tmp->struct_data;
struct field_cluster * cluster = str->struct_clustering;
while (cluster)
{
char * name = gen_cluster_name (str->decl, cluster_num, str_num);
tree fields;
tree new_type;
cluster_num++;
fields = create_fields (cluster, str->fields, str->num_fields);
new_type = build_basic_struct (fields, name, str->decl);
update_fields_mapping (cluster, new_type, str->fields, str->num_fields);
str->new_types = add_tree_to_tree_list (new_type, str->new_types);
cluster = cluster->sibling;
}
str_num++;
tmp = tmp->next;
}
}
static void
finish_var_creation (tree new_decl)
{
if (!var_ann (new_decl))
create_var_ann (new_decl);
if (is_global_var (new_decl))
mark_call_clobbered (new_decl, ESCAPE_UNKNOWN);
mark_sym_for_renaming (new_decl);
if (gimple_referenced_vars (cfun))
add_referenced_var (new_decl);
}
static void
finish_global_creation (tree var)
{
if (TREE_CODE (var) == VAR_DECL
&& is_global_var (var))
finish_var_creation (var);
}
static char *
gen_var_name (tree orig_decl, int i, tree * id_node)
{
char * new_name = NULL;
/* If the original variable decl has a name, create an
appropriate new name for the new decl. */
if (DECL_NAME (orig_decl)
&& IDENTIFIER_POINTER (DECL_NAME (orig_decl)))
{
char *old_name;
int counter = 0;
int len, new_len;
char *tmp_name;
old_name = (char *) IDENTIFIER_POINTER (DECL_NAME (orig_decl));
len = strlen (old_name) + 6;
new_name = (char *) xmalloc (len * sizeof (char));
sprintf (new_name, "%s_%d", old_name, i);
/* Make sure there isn't anything else that already has that
name. */
new_len = strlen (new_name) + 5;
tmp_name = (char *) xmalloc (new_len * sizeof (char));
sprintf (tmp_name, "%s", new_name);
while (maybe_get_identifier (tmp_name))
sprintf (tmp_name, "%s.%d", new_name, counter++);
new_name = tmp_name;
*id_node = get_identifier (new_name);
}
else
*id_node = NULL;
return new_name;
}
static struct field_access_site *
is_in_field_acc_list (tree stmt, struct field_access_site *list_p)
{
struct field_access_site *curr;
for (curr = list_p; curr; curr = curr->next)
if (curr->stmt == stmt)
return curr;
return NULL;
}
static struct field_access_site *
add_field_acc_to_acc_sites (tree stmt, tree ref, tree comp_ref, tree field_decl, tree num,
tree offset, tree base, tree ref_def_stmt, tree cast_stmt,
struct field_access_site *list_p)
{
struct field_access_site *acc;
gcc_assert (!is_in_field_acc_list (stmt, list_p));
acc = (struct field_access_site *) xmalloc (sizeof (struct field_access_site));
acc->stmt = stmt;
acc->comp_ref = comp_ref;
acc->ref = ref;
acc->field_decl = field_decl;
acc->num = num;
acc->offset = offset;
acc->base = base;
acc->ref_def_stmt = ref_def_stmt;
acc->cast_stmt = cast_stmt;
acc->next = list_p;
return acc;
}
static struct access_site *
is_in_acc_list (tree stmt, struct access_site *list_p)
{
struct access_site *curr;
for (curr = list_p; curr; curr = curr->next)
if (curr->stmt == stmt)
return curr;
return NULL;
}
static struct access_site *
add_access_to_acc_sites (tree stmt, tree var, struct access_site *list_p)
{
struct access_site *acc;
acc = is_in_acc_list (stmt, list_p);
if (acc)
{
acc->vars = add_tree_to_tree_list (var, acc->vars);
return list_p;
}
else
{
acc = (struct access_site *) xmalloc (sizeof (struct access_site));
acc->stmt = stmt;
acc->next = list_p;
acc->vars = NULL;
acc->vars = add_tree_to_tree_list (var, acc->vars);
return acc;
}
}
/* Stuff the new tree DATA into a tree_list node,
and append it to the front of the LIST_P. */
static tree_list
add_tree_to_tree_list (tree data, tree_list list_p)
{
tree_list tmp_node;
tmp_node = (tree_list) xmalloc (sizeof (struct struct_tree_list));
tmp_node->data = data;
tmp_node->next = list_p;
return tmp_node;
}
static inline void
insert_global_to_varpool (tree new_decl)
{
struct varpool_node *new_node;
new_node = varpool_node (new_decl);
notice_global_symbol (new_decl);
varpool_mark_needed_node (new_node);
varpool_finalize_decl (new_decl);
}
/* This function copy attributes form ORIG_DECL to NEW_DECL. */
static inline void
copy_decl_attributes (tree new_decl, tree orig_decl)
{
DECL_ARTIFICIAL (new_decl) = 1;
DECL_EXTERNAL (new_decl) = DECL_EXTERNAL (orig_decl);
TREE_STATIC (new_decl) = TREE_STATIC (orig_decl);
TREE_PUBLIC (new_decl) = TREE_PUBLIC (orig_decl);
TREE_USED (new_decl) = TREE_USED (orig_decl);
DECL_CONTEXT (new_decl) = DECL_CONTEXT (orig_decl);
TREE_THIS_VOLATILE (new_decl) = TREE_THIS_VOLATILE (orig_decl);
TREE_ADDRESSABLE (new_decl) = TREE_ADDRESSABLE (orig_decl);
if (TREE_CODE (orig_decl) == VAR_DECL &&
TREE_READONLY (orig_decl))
TREE_READONLY (new_decl) = 1;
}
struct type_wrapper
{
/* 0 stand for pointer wrapper,
and 1 for array wrapper. */
bool wrap;
tree domain; /* Relevant for arrays as domain or index. */
struct type_wrapper * next;
};
static struct type_wrapper *
add_wrapper (struct type_wrapper *wrapper, bool wrap, tree domain)
{
struct type_wrapper * node = (struct type_wrapper *) xmalloc
(sizeof (struct type_wrapper));
node->wrap = wrap;
node->next = wrapper;
node->domain = domain;
return node;
}
static void
free_wrapper (struct type_wrapper * wrap)
{
struct type_wrapper *node, *node1;
node = wrap;
while (node)
{
node1 = node;
node = node->next;
free (node1);
}
}
/* This function generate the same level of pointers or arrays
in the NEW_STR_TYPE as it was in type of DECL.
It returns the generated type. */
static inline tree
gen_struct_type (tree decl, tree new_str_type)
{
tree type_orig = get_type_of_var (decl);
tree new_type = new_str_type;
struct type_wrapper * wrapper = NULL;
struct type_wrapper * wr = NULL;
while (POINTER_TYPE_P (type_orig)
|| TREE_CODE (type_orig) == ARRAY_TYPE)
{
if (POINTER_TYPE_P (type_orig))
wrapper = add_wrapper (wrapper, 0, NULL_TREE);
else if (TREE_CODE (type_orig) == ARRAY_TYPE)
wrapper = add_wrapper (wrapper, 1, TYPE_DOMAIN (type_orig));
type_orig = TREE_TYPE (type_orig);
}
for (wr = wrapper; wr; wr = wr->next)
{
if (wr->wrap) /* Array. */
new_type = build_array_type (new_type, wr->domain);
else /* Pointer. */
new_type = build_pointer_type (new_type);
}
free_wrapper (wrapper);
return new_type;
}
/* Given an old variable ORIG_DECL with VAR_DECL tree code,
this function generates new variables to replace it
according to the set of types decided before. */
static struct struct_tree_list *
create_new_var_1 (tree orig_decl, d_str str)
{
tree_list new_vars = NULL;
tree_list current;
int i;
for (current = str->new_types, i = 0; current;
current = current->next, i++)
{
tree new_type = current->data;
tree new_decl = NULL;
tree id_node;
char * new_name = NULL;
new_name = gen_var_name (orig_decl, i, &id_node);
new_type = gen_struct_type (orig_decl, new_type);
if (is_global_var (orig_decl))
new_decl = build_decl (VAR_DECL, id_node, new_type);
else
new_decl = create_tmp_var (new_type, new_name);
copy_decl_attributes (new_decl, orig_decl);
new_vars = add_tree_to_tree_list (new_decl, new_vars);
}
/* Return the list of new var decls. */
return new_vars;
}
static void
finish_vars_creation (new_var list_p)
{
new_var node;
for (node = list_p; node; node = node->next)
{
tree_list curr;
for (curr = node->new_vars; curr; curr = curr->next)
finish_var_creation (curr->data);
}
}
/* For each SSA_NAME or VAR_DECL local variable of structure
type relevant for transformation in current function (cfun)
this function generate new variable(s) to replace it. */
static void
create_new_local_vars (void)
{
unsigned i;
tree var_list;
if (gimple_in_ssa_p (cfun))
{
for (i = 1; i < num_ssa_names; i++)
{
tree ssa_name = ssa_name (i) ;
if (!ssa_name)
continue;
create_new_var (SSA_NAME_VAR (ssa_name), &new_local_vars);
}
}
/* Function local variables. */
for (var_list = cfun->unexpanded_var_list; var_list;
var_list = TREE_CHAIN (var_list))
{
tree var = TREE_VALUE (var_list);
create_new_var (var, &new_local_vars);
}
finish_vars_creation (new_local_vars);
print_new_vars (new_local_vars);
}
/* This function returns true it the result variable of malloc call is
a cast to one of the structure types we are planning to transform.
STMT should contain malloc call: T.2 = malloc (T.1); */
static bool
is_malloc_of_struct (tree stmt, list * node_p)
{
tree lhs;
tree type;
tree final_stmt;
final_stmt = get_final_malloc_stmt (stmt);
if (!final_stmt)
return false;
/* final_stmt should be of the form:
T.3 = (struct_type *) T.2; */
if (TREE_CODE (final_stmt) != GIMPLE_MODIFY_STMT)
return false;
lhs = GIMPLE_STMT_OPERAND (final_stmt, 0);
type = get_type_of_var (lhs);
if (!type)
return false;
if (!POINTER_TYPE_P (type)
|| TREE_CODE (strip_type (type)) != RECORD_TYPE)
return false;
*node_p = find_struct_in_list (strip_type (type));
if (!(*node_p))
return false;
return true;
}
/* In this function we suppose that an allocation statement
var = (type_cast) malloc (size);
is converted into as follows:
T.1 = size;
T.2 = malloc (T.1);
T.3 = (type_cast) T.2;
var = T.3;
In this function we collect into malloc_data_list only allocation
function of variables of structure types presented
in data_struct_list. */
static void
collect_malloc_data (void)
{
struct cgraph_node *node;
struct cgraph_edge *cs;
for (node = cgraph_nodes; node; node = node->next)
if (node->analyzed && node->decl)
{
for (cs = node->callees; cs; cs = cs->next_callee)
{
tree stmt = cs->call_stmt;
if (stmt)
{
tree call = get_call_expr_in (stmt);
tree decl;
if (call && (decl = get_callee_fndecl (call))
&& TREE_CODE (stmt) == GIMPLE_MODIFY_STMT)
{
list str_node = NULL;
if (is_malloc_of_struct (stmt, &str_node))
{
d_str str = str_node->struct_data;
/* We support only malloc now. */
if (DECL_FUNCTION_CODE (decl) == BUILT_IN_MALLOC)
add_call_to_malloc_list (node->decl, stmt, str);
else
remove_struct_from_data_struct_list (str_node);
}
}
}
}
}
}
/* Update statements in STMT_LIST with BB info. */
static void
add_bb_info (basic_block bb, tree stmt_list)
{
if (TREE_CODE (stmt_list) == STATEMENT_LIST)
{
tree_stmt_iterator tsi;
for (tsi = tsi_start (stmt_list); !tsi_end_p (tsi); tsi_next (&tsi))
{
tree stmt = tsi_stmt (tsi);
set_bb_for_stmt (stmt, bb);
}
}
}
/* This function checks whether ARG is a result of multiplication
of some number by STRUCT_SIZE. If yes, the function returns true
and this number is substituted into NUM. */
static bool
is_result_of_mult (tree arg, tree *num, tree struct_size)
{
tree size_def_stmt = SSA_NAME_DEF_STMT (arg);
/* If malloc stmt was of the form D.2229_10 = malloc (D.2228_9);
then size_def_stmt can be D.2228_9 = num.3_8 * 8; */
if (size_def_stmt
&& TREE_CODE (size_def_stmt) == GIMPLE_MODIFY_STMT)
{
tree lhs = GIMPLE_STMT_OPERAND (size_def_stmt, 0);
tree rhs = GIMPLE_STMT_OPERAND (size_def_stmt, 1);
/* We expect temporary here. */
if (!is_gimple_reg (lhs))
return false;
if (TREE_CODE (rhs) == MULT_EXPR)
{
tree arg0 = TREE_OPERAND (rhs, 0);
tree arg1 = TREE_OPERAND (rhs, 1);
if (operand_equal_p (arg0, struct_size, OEP_ONLY_CONST))
{
*num = arg1;
return true;
}
if (operand_equal_p (arg1, struct_size, OEP_ONLY_CONST))
{
*num = arg0;
return true;
}
}
}
*num = NULL_TREE;
return false;
}
static inline void
finish_stmt (tree stmt)
{
update_stmt (stmt);
mark_symbols_for_renaming (stmt);
}
/* This function returns a tree represented
the number of structure instances allocated by MALLOC_STMT. */
static tree
gen_num_of_structs_in_malloc (tree stmt, tree str_decl, tree * new_stmts_p)
{
call_expr_arg_iterator iter;
tree arg;
tree call_expr;
tree struct_size;
HOST_WIDE_INT struct_size_int;
if (!stmt)
return NULL_TREE;
/* Get malloc agrument. */
call_expr = get_call_expr_in (stmt);
if (!call_expr)
return NULL_TREE;
arg = first_call_expr_arg (call_expr, &iter);
if (TREE_CODE (arg) != SSA_NAME
&& !TREE_CONSTANT (arg))
return NULL_TREE;
struct_size = TYPE_SIZE_UNIT (str_decl);
struct_size_int = TREE_INT_CST_LOW (struct_size);
gcc_assert (struct_size);
if (TREE_CODE (arg) == SSA_NAME)
{
tree num, div_stmt;
if (is_result_of_mult (arg, &num, struct_size))
return num;
num = create_tmp_var (integer_type_node, NULL);
if (num)
add_referenced_var (num);
if (exact_log2 (struct_size_int) == -1)
div_stmt = build2 (GIMPLE_MODIFY_STMT, integer_type_node,
num, build2 (TRUNC_DIV_EXPR, integer_type_node,
arg, struct_size));
else
div_stmt = build2 (GIMPLE_MODIFY_STMT, integer_type_node,
num,
build2 (RSHIFT_EXPR,
integer_type_node,
arg, build_int_cst (integer_type_node,
exact_log2 (struct_size_int))));
*new_stmts_p = alloc_stmt_list ();
append_to_statement_list (div_stmt,
new_stmts_p);
finish_stmt (div_stmt);
return num;
}
if (CONSTANT_CLASS_P (arg)
&& multiple_of_p (TREE_TYPE (struct_size),
arg, struct_size))
return int_const_binop (TRUNC_DIV_EXPR, arg, struct_size, 0);
return NULL_TREE;
}
static inline void
finish_stmt_and_append (tree *stmts, tree stmt)
{
append_to_statement_list (stmt,
stmts);
finish_stmt (stmt);
}
static tree
find_var_in_new_vars_list (new_var var, tree new_type)
{
tree_list curr;
for (curr = var->new_vars; curr; curr = curr->next)
{
tree type = strip_type(get_type_of_var (curr->data));
gcc_assert (type);
if (type == new_type)
return curr->data;
}
return NULL_TREE;
}
static tree
find_new_var_of_type (tree orig_var, tree new_type)
{
new_var var;
gcc_assert (orig_var && new_type);
if (TREE_CODE (orig_var) == SSA_NAME)
orig_var = SSA_NAME_VAR (orig_var);
var = is_in_new_vars_list (orig_var, new_global_vars);
if (!var)
var = is_in_new_vars_list (orig_var, new_local_vars);
gcc_assert (var);
return find_var_in_new_vars_list (var, new_type);
}
static tree
create_new_malloc (tree malloc_stmt, tree new_type, tree *new_stmts, tree num)
{
tree new_malloc_size;
tree call_expr, malloc_fn_decl;
tree new_stmt, malloc_res;
tree call_stmt, final_stmt;
tree cast_res;
gcc_assert (num && malloc_stmt && new_type);
*new_stmts = alloc_stmt_list ();
/* Generate argument to malloc as multiplication of num and size of new_type. */
new_stmt = gen_size (num, new_type, &new_malloc_size);
append_to_statement_list (new_stmt, new_stmts);
/* Generate new call for malloc. */
malloc_res = create_tmp_var (integer_type_node, NULL);
if (malloc_res)
add_referenced_var (malloc_res);
call_expr = get_call_expr_in (malloc_stmt);
malloc_fn_decl = get_callee_fndecl (call_expr);
call_expr = build_call_expr (malloc_fn_decl, 1, new_malloc_size);
call_stmt = build2 (GIMPLE_MODIFY_STMT,
TREE_TYPE (TREE_TYPE (malloc_fn_decl)),
malloc_res,
call_expr);
finish_stmt_and_append (new_stmts, call_stmt);
/* Create new cast stmt. */
final_stmt = get_final_malloc_stmt (malloc_stmt);
gcc_assert (final_stmt);
new_stmt = gen_cast_stmt (malloc_res, new_type, final_stmt, &cast_res);
append_to_statement_list (new_stmt, new_stmts);
return call_stmt;
}
/* This function build an edge between BB and E->dest and updates
phi nodes of E->dest. It returns newly created edge. */
static edge
make_edge_and_fix_phis_of_dest (basic_block bb, edge e)
{
edge new_e;
tree phi, arg;
new_e = make_edge (bb, e->dest, e->flags);
for (phi = phi_nodes (new_e->dest); phi; phi = PHI_CHAIN (phi))
{
arg = PHI_ARG_DEF_FROM_EDGE (phi, e);
add_phi_arg (phi, arg, new_e);
}
return new_e;
}
/* Update call graph with new edge generated by
new MALLOC_STMT. */
static void
update_cgraph_with_malloc_call (tree malloc_stmt, tree context)
{
tree call_expr;
struct cgraph_node *src, *dest;
tree malloc_fn_decl;
if (!malloc_stmt)
return;
call_expr = get_call_expr_in (malloc_stmt);
malloc_fn_decl = get_callee_fndecl (call_expr);
src = cgraph_node (context);
dest = cgraph_node (malloc_fn_decl);
cgraph_create_edge (src, dest, malloc_stmt,
0, 0, bb_for_stmt (malloc_stmt)->loop_depth);
}
/* If MALLOC_STMT is D.2225_7 = malloc (D.2224_6);
it should be casting stmt as for example:
p_8 = (struct str_t *) D.2225_7;
which is returned by this function. */
static tree
get_final_malloc_stmt (tree malloc_stmt)
{
tree final_stmt;
use_operand_p use_p;
tree malloc_res;
if (!malloc_stmt)
return NULL;
if (TREE_CODE (malloc_stmt) != GIMPLE_MODIFY_STMT)
return NULL;
malloc_res = GIMPLE_STMT_OPERAND (malloc_stmt, 0);
if (TREE_CODE (malloc_res) != SSA_NAME)
return NULL;
if (!single_imm_use (malloc_res, &use_p, &final_stmt))
return NULL;
else
return final_stmt;
}
/* Insert NEW_STMTS after STMT. */
static void
insert_after_stmt (tree stmt, tree new_stmts)
{
block_stmt_iterator bsi;
if (!stmt || !new_stmts)
return;
bsi = bsi_for_stmt (stmt);
bsi_insert_after (&bsi, new_stmts, BSI_SAME_STMT);
}
/* Insert NEW_STMTS before STMT. */
static void
insert_before_stmt (tree stmt, tree new_stmts)
{
block_stmt_iterator bsi;
if (!stmt || !new_stmts)
return;
bsi = bsi_for_stmt (stmt);
bsi_insert_before (&bsi, new_stmts, BSI_SAME_STMT);
}
/* This function adds allocation sites for peeled structs. */
static void
create_new_mallocs (malloc_d m_data, tree context)
{
malloc_call call;
for (call = m_data->malloc_list; call; call = call->next)
{
tree stmt = call->malloc_call_stmt;
d_str str = call->str;
tree_list n_type;
tree num;
tree new_stmts = NULL_TREE;
tree last_stmt = get_final_malloc_stmt (stmt);
num = gen_num_of_structs_in_malloc (stmt, str->decl, &new_stmts);
if (new_stmts)
{
last_stmt = tsi_stmt (tsi_last (new_stmts));
insert_after_stmt (last_stmt, new_stmts);
}
/* Generate an allocation sites for each new structure type. */
for (n_type = str->new_types; n_type; n_type = n_type->next)
{
tree type = n_type->data;
tree new_malloc_stmt = NULL_TREE;
tree last_stmt_tmp = NULL_TREE;
new_stmts = NULL_TREE;
new_malloc_stmt = create_new_malloc (stmt, type, &new_stmts, num);
last_stmt_tmp = tsi_stmt (tsi_last (new_stmts));
insert_after_stmt (last_stmt, new_stmts);
update_cgraph_with_malloc_call (new_malloc_stmt, context);
last_stmt = last_stmt_tmp;
}
}
}
/* Create new mallocs for function represented by NODE. */
static void
create_new_mallocs_for_func (struct cgraph_node *node)
{
malloc_d cur_malloc = get_malloc_list_for_func (node->decl);
if (cur_malloc)
create_new_mallocs (cur_malloc, node->decl);
}
static void
free_new_vars_list (new_var *list_p)
{
new_var node = *list_p;
new_var t_node;
while (node)
{
tree_list var, t_var;
/* Free tree_list of new vars. */
var = node->new_vars;
while (var)
{
t_var = var;
var = var->next;
free (t_var);
}
t_node = node;
node = node->next;
free (t_node);
}
*list_p = NULL;
}
static malloc_d
get_malloc_list_for_func (tree fn_decl)
{
malloc_d cur_malloc;
for (cur_malloc = malloc_data_list_new; cur_malloc;
cur_malloc = cur_malloc->next)
if (cur_malloc->func == fn_decl)
return cur_malloc;
return NULL;
}
static bool
is_part_of_malloc (tree stmt, tree fn_decl)
{
malloc_d cur_malloc = get_malloc_list_for_func (fn_decl);
if (cur_malloc)
{
malloc_call call;
for (call = cur_malloc->malloc_list; call; call = call->next)
if (call->malloc_call_stmt == stmt
|| get_final_malloc_stmt (call->malloc_call_stmt) == stmt)
return true;
}
return false;
}
static void
remove_from_acc_list (struct access_site ** list_p, tree stmt)
{
struct access_site *node = *list_p;
struct access_site *prev_node = NULL;
while (node)
{
if (node->stmt == stmt)
{
if (prev_node == NULL)
*list_p = node->next;
else
prev_node->next = node->next;
free_tree_list (node->vars);
free (node);
return;
}
prev_node = node;
node = node->next;
}
}
static bool
is_part_of_field_access (tree stmt, d_str str)
{
int i;
for (i = 0; i < str->num_fields; i++)
{
struct field_access_site *acc;
for (acc = str->fields[i].acc_sites; acc; acc = acc->next)
{
if (acc->stmt == stmt
|| acc->ref_def_stmt == stmt
|| acc->cast_stmt == stmt)
return true;
}
}
return false;
}
/* We exclude from list of general accesses of structure all stmt that
will be treated as part of new mallocs or accesses generation. */
static void
exclude_malloc_and_field_accs (struct cgraph_node *node)
{
list str_node;
for (str_node = data_struct_list; str_node; str_node = str_node->next)
{
d_str str = str_node->struct_data;
struct access_site * acc = str->accs;
while (acc)
{
tree stmt = acc->stmt;
acc = acc->next;
if (is_part_of_malloc (stmt, node->decl)
|| is_part_of_field_access (stmt, str))
remove_from_acc_list (&str->accs, stmt);
}
}
}
static inline tree
build_comp_ref (tree base, tree field_id, tree type)
{
tree field;
bool found = false;
/* Find field with the same name as field_id in type. */
for (field = TYPE_FIELDS (type); field; field = TREE_CHAIN (field))
{
if (DECL_NAME (field) == field_id)
{
found = true;
break;
}
}
gcc_assert (found);
return build3 (COMPONENT_REF, TREE_TYPE (field),
base, field, NULL_TREE);
}
struct ref_pos
{
tree *pos;
tree ref;
};
/* Helper for walk_tree called from collect_accesses_in_bb function. */
static tree
find_pos_in_stmt_1 (tree *tp, int *walk_subtrees, void * data)
{
struct ref_pos * r_pos = (struct ref_pos *) data;
tree ref = r_pos->ref;
tree t = *tp;
if (t == ref)
{
r_pos->pos = tp;
return t;
}
switch (TREE_CODE (t))
{
case GIMPLE_MODIFY_STMT:
{
tree lhs = GIMPLE_STMT_OPERAND (t, 0);
tree rhs = GIMPLE_STMT_OPERAND (t, 1);
*walk_subtrees=1;
walk_tree (&lhs, find_pos_in_stmt_1, data, NULL);
walk_tree (&rhs, find_pos_in_stmt_1, data, NULL);
*walk_subtrees=0;
}
break;
default:
*walk_subtrees=1;
}
return NULL_TREE;
}
static tree *
find_pos_in_stmt (tree stmt, tree ref)
{
struct ref_pos r_pos;
r_pos.ref = ref;
r_pos.pos = NULL;
walk_tree (&stmt, find_pos_in_stmt_1, &r_pos, NULL);
return r_pos.pos;
}
static void
replace_field_acc (struct field_access_site *acc, tree new_type)
{
tree ref_var = acc->ref;
tree new_ref;
tree lhs, rhs;
tree *pos;
tree new_acc;
tree field_id = DECL_NAME (acc->field_decl);
struct type_wrapper * wrapper = NULL;
struct type_wrapper * wr = NULL;
while (TREE_CODE (ref_var) == INDIRECT_REF
|| TREE_CODE (ref_var) == ARRAY_REF)
{
if ( TREE_CODE (ref_var) == INDIRECT_REF)
wrapper = add_wrapper (wrapper, 0, 0);
else if (TREE_CODE (ref_var) == ARRAY_REF)
wrapper = add_wrapper (wrapper, 1, TREE_OPERAND (ref_var, 1));
ref_var = TREE_OPERAND (ref_var, 0);
}
new_ref = find_new_var_of_type (ref_var, new_type);
finish_global_creation (new_ref);
for (wr = wrapper; wr; wr = wr->next)
{
tree type = TREE_TYPE (TREE_TYPE (new_ref));
if (wr->wrap) /* Array. */
new_ref = build4 (ARRAY_REF, type, new_ref,
wr->domain, NULL_TREE, NULL_TREE);
else /* Pointer. */
new_ref = build1 (INDIRECT_REF, type, new_ref);
}
new_acc = build_comp_ref (new_ref, field_id, new_type);
free_wrapper (wrapper);
if (TREE_CODE (acc->stmt) == GIMPLE_MODIFY_STMT)
{
lhs = GIMPLE_STMT_OPERAND (acc->stmt, 0);
rhs = GIMPLE_STMT_OPERAND (acc->stmt, 1);
if (lhs == acc->comp_ref)
GIMPLE_STMT_OPERAND (acc->stmt, 0) = new_acc;
else if (rhs == acc->comp_ref)
GIMPLE_STMT_OPERAND (acc->stmt, 1) = new_acc;
else
{
pos = find_pos_in_stmt (acc->stmt, acc->comp_ref);
gcc_assert (pos);
*pos = new_acc;
}
}
else
{
pos = find_pos_in_stmt (acc->stmt, acc->comp_ref);
gcc_assert (pos);
*pos = new_acc;
}
finish_stmt (acc->stmt);
}
static void
replace_field_access_stmt (struct field_access_site *acc, tree new_type)
{
if (TREE_CODE (acc->ref) == INDIRECT_REF
||TREE_CODE (acc->ref) == ARRAY_REF
||TREE_CODE (acc->ref) == VAR_DECL)
replace_field_acc (acc, new_type);
else
gcc_unreachable ();
}
/* In this function we create new stmts that has the same form as ORIG_STMT,
but of the type NET_TYPE. The stmts treated by this function are simple
assignments, like assignments: p.8_7 = p; or stmts with rhs of code PLUS_EXPR
or MINUS_EXPR. */
static tree
create_base_plus_offset (tree orig_stmt, tree new_type, tree offset)
{
tree lhs, rhs;
tree new_lhs, new_rhs;
tree new_stmt;
gcc_assert (TREE_CODE (orig_stmt) == GIMPLE_MODIFY_STMT);
lhs = GIMPLE_STMT_OPERAND (orig_stmt, 0);
rhs = GIMPLE_STMT_OPERAND (orig_stmt, 1);
gcc_assert (TREE_CODE (lhs) == VAR_DECL
|| TREE_CODE (lhs) == SSA_NAME);
new_lhs = find_new_var_of_type (lhs, new_type);
gcc_assert (new_lhs);
finish_var_creation (new_lhs);
switch (TREE_CODE (rhs))
{
case PLUS_EXPR:
case MINUS_EXPR:
case POINTER_PLUS_EXPR:
{
tree op0 = TREE_OPERAND (rhs, 0);
tree op1 = TREE_OPERAND (rhs, 1);
tree new_op0 = NULL_TREE, new_op1 = NULL_TREE;
list tmp0, tmp1;
tmp0 = find_struct_in_list (strip_type (get_type_of_var (op0)));
tmp1 = find_struct_in_list (strip_type (get_type_of_var (op1)));
gcc_assert ( tmp0 || tmp1);
if (tmp0)
new_op0 = find_new_var_of_type (op0, new_type);
if (tmp1)
new_op1 = find_new_var_of_type (op1, new_type);
if (!new_op0)
new_op0 = offset;
if (!new_op1)
new_op1 = offset;
new_rhs = build2 (TREE_CODE (rhs), TREE_TYPE (new_op0),
new_op0, new_op1);
}
break;
default:
gcc_unreachable();
}
new_stmt = build2 (GIMPLE_MODIFY_STMT, TREE_TYPE (new_lhs),
new_lhs, new_rhs);
finish_stmt (new_stmt);
return new_stmt;
}
/* Generate stmt: res = NUM * sizeof(TYPE) and return it. */
static tree
gen_size (tree num, tree type, tree *res)
{
tree struct_size = TYPE_SIZE_UNIT (type);
HOST_WIDE_INT struct_size_int = TREE_INT_CST_LOW (struct_size);
tree new_stmt;
*res = create_tmp_var (TREE_TYPE (num), NULL);
if (*res)
add_referenced_var (*res);
if (exact_log2 (struct_size_int) == -1)
new_stmt = build2 (GIMPLE_MODIFY_STMT, TREE_TYPE (num),
*res, build2 (MULT_EXPR,
TREE_TYPE (num),
num,
struct_size));
else
new_stmt = build2 (GIMPLE_MODIFY_STMT, TREE_TYPE (num),
*res,
build2 (LSHIFT_EXPR,
TREE_TYPE (num),
num,
build_int_cst (TREE_TYPE (num),
exact_log2 (struct_size_int))));
finish_stmt (new_stmt);
return new_stmt;
}
static tree
gen_cast_stmt (tree before_cast, tree new_type, tree old_cast_stmt, tree *res_p)
{
tree lhs, new_lhs, new_stmt;
gcc_assert (TREE_CODE (old_cast_stmt) == GIMPLE_MODIFY_STMT);
lhs = GIMPLE_STMT_OPERAND (old_cast_stmt, 0);
new_lhs = find_new_var_of_type (lhs, new_type);
gcc_assert (new_lhs);
new_stmt = build2 (GIMPLE_MODIFY_STMT,
TREE_TYPE
(new_lhs),
new_lhs,
build1 (NOP_EXPR,
TREE_TYPE (new_lhs),
before_cast));
finish_stmt (new_stmt);
*res_p = new_lhs;
return new_stmt;
}
static void
create_new_field_access (struct field_access_site *acc, struct data_field_entry field)
{
tree new_type = field.field_mapping;
tree new_stmt;
tree size_res;
tree mult_stmt, cast_stmt;
tree cast_res = NULL;
if (acc->num)
{
mult_stmt = gen_size (acc->num, new_type, &size_res);
insert_before_stmt (acc->ref_def_stmt, mult_stmt);
}
if (acc->cast_stmt)
{
cast_stmt = gen_cast_stmt (size_res, new_type,
acc->cast_stmt, &cast_res);
insert_after_stmt (acc->cast_stmt, cast_stmt);
}
if (acc->ref_def_stmt)
{
tree offset;
if (cast_res)
offset = cast_res;
else
offset = size_res;
new_stmt = create_base_plus_offset (acc->ref_def_stmt, new_type, offset);
insert_after_stmt (acc->ref_def_stmt, new_stmt);
}
/* In stmt D.2163_19 = D.2162_18->b; we replace variable
D.2162_18 for the appropriate variable of new_type. */
replace_field_access_stmt (acc, new_type);
}
static void
create_new_general_access (struct access_site *acc, d_str str)
{
tree stmt = acc->stmt;
switch (TREE_CODE (stmt))
{
case COND_EXPR:
create_new_stmts_for_cond_expr (stmt);
break;
default:
create_new_stmts_for_general_acc (acc, str);
}
}
static void
create_new_accesses_in_bb (basic_block bb)
{
list tmp;
for (tmp = data_struct_list; tmp; tmp = tmp->next)
{
int i;
struct field_access_site *f_acc;
struct access_site *acc;
d_str str = tmp->struct_data;
for (i = 0; i < str->num_fields; i++)
for (f_acc = str->fields[i].acc_sites; f_acc; f_acc = f_acc->next)
if (bb_for_stmt (f_acc->stmt) == bb)
create_new_field_access (f_acc, str->fields[i]);
for (acc = str->accs; acc; acc = acc->next)
if (bb_for_stmt (acc->stmt) == bb)
create_new_general_access (acc, str);
}
}
static void
create_new_accesses_for_func (void)
{
basic_block bb;
FOR_EACH_BB_FN (bb, cfun)
create_new_accesses_in_bb (bb);
}
static tree
create_general_new_stmt (struct access_site *acc, tree new_type)
{
tree old_stmt = acc->stmt;
tree_list var_d;
tree new_stmt = copy_node (old_stmt);
for (var_d = acc->vars; var_d; var_d = var_d->next)
{
tree * pos;
tree var = var_d->data;
tree new_var = find_new_var_of_type (var, new_type);
tree lhs, rhs;
gcc_assert (new_var);
finish_var_creation (new_var);
if (TREE_CODE (new_stmt) == GIMPLE_MODIFY_STMT)
{
lhs = GIMPLE_STMT_OPERAND (new_stmt, 0);
rhs = GIMPLE_STMT_OPERAND (new_stmt, 1);
if (TREE_CODE (lhs) == SSA_NAME)
lhs = SSA_NAME_VAR (lhs);
if (TREE_CODE (rhs) == SSA_NAME)
rhs = SSA_NAME_VAR (rhs);
if (lhs == var)
GIMPLE_STMT_OPERAND (new_stmt, 0) = new_var;
else if (rhs == var)
GIMPLE_STMT_OPERAND (new_stmt, 1) = new_var;
else
{
pos = find_pos_in_stmt (new_stmt, var);
gcc_assert (pos);
*pos = new_var;
}
}
else
{
pos = find_pos_in_stmt (new_stmt, var);
gcc_assert (pos);
*pos = new_var;
}
}
finish_stmt (new_stmt);
return new_stmt;
}
static void
create_new_stmts_for_general_acc (struct access_site * acc, d_str str)
{
tree_list new_type;
tree stmt = acc->stmt;
for (new_type = str->new_types; new_type; new_type = new_type->next)
{
tree new_stmt;
new_stmt = create_general_new_stmt (acc, new_type->data);
insert_after_stmt (stmt, new_stmt);
}
}
static void
create_new_stmts_for_cond_expr_1 (tree new_var, tree cond_stmt, bool pos)
{
tree new_cond;
tree new_stmt;
edge true_e = NULL, false_e = NULL;
basic_block new_bb;
tree stmt_list;
extract_true_false_edges_from_block (bb_for_stmt (cond_stmt),
&true_e, &false_e);
new_cond = copy_node (COND_EXPR_COND (cond_stmt));
TREE_OPERAND (new_cond, pos) = new_var;
new_stmt = build3 (COND_EXPR,TREE_TYPE (cond_stmt),
new_cond, NULL_TREE, NULL_TREE);
finish_stmt (new_stmt);
/* Create a new basic block after bb. */
new_bb = create_empty_bb (bb_for_stmt (cond_stmt));
/* Add new condition stmt to the new_bb. */
stmt_list = bb_stmt_list (new_bb);
append_to_statement_list (new_stmt, &stmt_list);
add_bb_info (new_bb, stmt_list);
/* Create false and true edges from new_bb. */
make_edge_and_fix_phis_of_dest (new_bb, true_e);
make_edge_and_fix_phis_of_dest (new_bb, false_e);
/* Redirect one of original edges to point to new_bb. */
if (TREE_CODE (cond_stmt) == NE_EXPR)
redirect_edge_succ (true_e, new_bb);
else
redirect_edge_succ (false_e, new_bb);
}
static void
create_new_stmts_for_cond_expr (tree stmt)
{
tree cond = COND_EXPR_COND (stmt);
tree arg0, arg1, arg;
list tmp0, tmp1, tmp;
d_str str;
tree_list new_type;
bool pos;
gcc_assert (TREE_CODE (cond) == EQ_EXPR
|| TREE_CODE (cond) == NE_EXPR);
arg0 = TREE_OPERAND (cond, 0);
arg1 = TREE_OPERAND (cond, 1);
tmp0 = find_struct_in_list (strip_type (get_type_of_var (arg0)));
tmp1 = find_struct_in_list (strip_type (get_type_of_var (arg1)));
gcc_assert ((!tmp0 && tmp1) || (!tmp1 && tmp0));
tmp = tmp0 ? tmp0 : tmp1;
arg = tmp0 ? arg0 : arg1;
pos = tmp0 ? 0 : 1;
str = tmp->struct_data;
for (new_type = str->new_types; new_type; new_type = new_type->next)
{
tree new_arg;
new_arg = find_new_var_of_type (arg, new_type->data);
create_new_stmts_for_cond_expr_1 (new_arg, stmt, pos);
}
}
/* Do transformation for a function represented by NODE. */
static void
do_reorg_for_func (struct cgraph_node *node)
{
create_new_local_vars ();
create_new_mallocs_for_func (node);
create_new_accesses_for_func ();
update_ssa (TODO_update_ssa);
cleanup_tree_cfg ();
/* Free auxiliary data for local variables. */
free_new_vars_list (&new_local_vars);
}
static void
add_to_new_vars_list (new_var new_node, new_var *list_p)
{
new_var node;
if (!(*list_p))
*list_p = new_node;
else
{
node = *list_p;
while (node->next)
node = node->next;
node->next = new_node;
}
}
static new_var
is_in_new_vars_list (tree decl, new_var list_p)
{
new_var node;
if (!list_p)
return NULL;
else
{
node = list_p;
while (node)
{
if (node->orig_var == decl)
return node;
node = node->next;
}
}
return NULL;
}
static new_var
create_new_var_node (tree var, tree_list new_vars_list)
{
new_var node;
node = (struct new_var_data *) xmalloc (sizeof (struct new_var_data));
node->orig_var = var;
node->new_vars = new_vars_list;
node->next = NULL;
return node;
}
static void
create_new_var (tree var_decl, new_var *list_p)
{
new_var node;
tree_list new_vars;
list tmp;
tree type;
d_str str;
if (!var_decl || is_in_new_vars_list (var_decl, *list_p))
return;
if (!is_candidate_for_data_struct_list (var_decl, &type, NULL))
return;
tmp = find_struct_in_list (type);
if (!tmp)
return;
str = tmp->struct_data;
new_vars = create_new_var_1 (var_decl, str);
node = create_new_var_node (var_decl, new_vars);
add_to_new_vars_list (node, list_p);
}
static void
update_varpool_with_new_vars (void)
{
new_var old_var;
for (old_var = new_global_vars; old_var; old_var = old_var->next)
{
tree_list n_var;
for (n_var = old_var->new_vars; n_var; n_var = n_var->next)
insert_global_to_varpool (n_var->data);
}
}
/* This function create global struct variables of new
structure types in case these this variable has VAR_DECL code. */
static void
create_new_global_vars (void)
{
struct varpool_node *current_varpool;
FOR_EACH_STATIC_VARIABLE(current_varpool)
{
tree var_decl = current_varpool->decl;
if (!var_decl || TREE_CODE (var_decl) != VAR_DECL)
continue;
create_new_var (var_decl, &new_global_vars);
}
update_varpool_with_new_vars ();
}
/* This function is a helper for do_reorg. It goes over functions in call graph
and performs actual transformation on each one of them. */
static void
do_reorg_1 (void)
{
struct cgraph_node *node;
/* Initialize the default bitmap obstack. */
bitmap_obstack_initialize (NULL);
for (node = cgraph_nodes; node; node = node->next)
if (node->analyzed && node->decl && !node->next_clone)
{
push_cfun (DECL_STRUCT_FUNCTION (node->decl));
current_function_decl = node->decl;
if (dump_file)
fprintf (dump_file, "\nFunction to do reorg is %s: \n",
(char *) IDENTIFIER_POINTER (DECL_NAME (node->decl)));
do_reorg_for_func (node);
free_dominance_info (CDI_DOMINATORS);
free_dominance_info (CDI_POST_DOMINATORS);
current_function_decl = NULL;
pop_cfun ();
}
cfun = NULL;
}
static void
do_reorg (void)
{
/* Check that there is a work to do. */
if (!data_struct_list)
return;
/* Generate new types. */
create_new_types ();
print_new_types ();
/* Create new global vars. */
create_new_global_vars ();
print_new_vars (new_global_vars);
/* Do reorg for each funciton separately. */
do_reorg_1 ();
/* Free auxiliary data for global variables. */
free_new_vars_list (&new_global_vars);
}
/* This function builds an entry in struct_list for each
data structure potential for transformation. */
static void
build_data_structure_list (bitmap non_suitable_types)
{
tree var, type;
tree var_list;
struct varpool_node *current_varpool;
struct cgraph_node *c_node;
/* Check global vars. */
FOR_EACH_STATIC_VARIABLE (current_varpool)
{
var = current_varpool->decl;
if (is_candidate_for_data_struct_list (var, &type, non_suitable_types))
add_struct_to_data_struct_list (type);
}
/* Now add data structures of function parameters and local variables. */
for (c_node = cgraph_nodes; c_node; c_node = c_node->next)
{
enum availability avail = cgraph_function_body_availability (c_node);
/* We need AVAIL_AVAILABLE for main function. */
if (avail == AVAIL_LOCAL || avail == AVAIL_AVAILABLE)
{
struct function *fn = DECL_STRUCT_FUNCTION (c_node->decl);
for (var = DECL_ARGUMENTS (c_node->decl); var; var = TREE_CHAIN (var))
if (is_candidate_for_data_struct_list (var, &type, non_suitable_types))
add_struct_to_data_struct_list (type);
/* Function local variables. */
for (var_list = fn->unexpanded_var_list; var_list;
var_list = TREE_CHAIN (var_list))
{
var = TREE_VALUE (var_list);
if (is_candidate_for_data_struct_list (var, &type, non_suitable_types))
add_struct_to_data_struct_list (type);
}
}
}
}
/* This function looks for arguments of local functions
which are of pointer to or structure types. We are not handling
peeling of such structures right now. */
static void
exclude_types_passed_to_local_func (bitmap non_suitable_types)
{
struct cgraph_node *c_node;
for (c_node = cgraph_nodes; c_node; c_node = c_node->next)
{
if (cgraph_function_body_availability (c_node) == AVAIL_LOCAL)
{
tree fn = c_node->decl;
tree arg;
for (arg = DECL_ARGUMENTS (fn); arg; arg = TREE_CHAIN (arg))
{
tree type = TREE_TYPE (arg);
type = strip_type (type);
if (TREE_CODE (type) == RECORD_TYPE)
{
bitmap_set_bit (non_suitable_types, TYPE_UID (TYPE_MAIN_VARIANT (type)));
if (dump_file)
{
fprintf (dump_file, "\nPointer to type \"");
print_generic_expr (dump_file, type, 0);
fprintf (dump_file, "\" is passed to local function...Excluded.");
}
}
}
}
}
}
static void
exclude_escaping_types_1 (bitmap non_suitable_types)
{
list tmp = data_struct_list;
while (tmp)
{
tree type = tmp->struct_data->decl;
if (!ipa_type_escape_type_contained_p (type))
{
if (dump_file)
{
fprintf (dump_file, "\nEscaping type is ");
print_generic_expr (dump_file, type, 0);
}
bitmap_set_bit (non_suitable_types, TYPE_UID (type));
}
tmp = tmp->next;
}
}
static void
exclude_escaping_types (bitmap non_suitable_types)
{
exclude_types_passed_to_local_func (non_suitable_types);
exclude_escaping_types_1 (non_suitable_types);
}
/* FIXME: In this function we suppose that only fields defined
as bitfields in C program can become bitfields. If there are other ways,
this function should be extended to recognize them. */
static void
exclude_types_with_bit_fields (bitmap non_suitable_types)
{
list tmp = data_struct_list;
while (tmp)
{
int i;
d_str str = tmp->struct_data;
tree type = tmp->struct_data->decl;
for (i = 0; i < str->num_fields; i++)
if (DECL_BIT_FIELD (str->fields[i].decl))
bitmap_set_bit (non_suitable_types, TYPE_UID (type));
tmp = tmp->next;
}
}
/* This function returns true if program contains a call to user
defined allocation function. */
static bool
program_redefines_malloc_p (void)
{
bool redefines = false;
struct cgraph_node *c_node;
struct cgraph_node *c_node2;
struct cgraph_edge *c_edge;
tree fndecl;
tree fndecl2;
tree call_expr;
for (c_node = cgraph_nodes; c_node && !redefines; c_node = c_node->next)
{
fndecl = c_node->decl;
for (c_edge = c_node->callees; c_edge && !redefines;
c_edge = c_edge->next_callee)
{
call_expr = get_call_expr_in (c_edge->call_stmt);
c_node2 = c_edge->callee;
fndecl2 = c_node2->decl;
if (call_expr)
{
if ((call_expr_flags (call_expr) & ECF_MALLOC) &&
(DECL_FUNCTION_CODE (fndecl2) != BUILT_IN_MALLOC) &&
(DECL_FUNCTION_CODE (fndecl2) != BUILT_IN_CALLOC) &&
(DECL_FUNCTION_CODE (fndecl2) != BUILT_IN_ALLOCA))
redefines = true;
}
}
}
return redefines;
}
/* Auxiliary function for add_struct_to_data_struct_list. */
static struct data_field_entry *
get_fields (tree struct_decl, int num_fields)
{
struct data_field_entry *list;
tree t = TYPE_FIELDS (struct_decl);
int idx = 0;
list = (struct data_field_entry * ) xmalloc (num_fields
* sizeof
(struct data_field_entry));
for (idx = 0 ; t; t = TREE_CHAIN (t), idx++)
if (TREE_CODE (t) == FIELD_DECL)
{
list[idx].index = idx;
list[idx].decl = t;
list[idx].acc_sites = NULL;
list[idx].count = 0;
list[idx].field_mapping = NULL_TREE;
}
return list;
}
/* Auxiliary function for remove_struct_from_data_struct_list. */
static void
free_struct_cluster (struct field_cluster* cluster)
{
if (cluster)
{
if (cluster->fields_in_cluster)
sbitmap_free (cluster->fields_in_cluster);
if (cluster->sibling)
free_struct_cluster (cluster->sibling);
free (cluster);
}
}
/* The following three functions:
add_struct_to_data_struct_list
remove_struct_from_data_struct_list
free_data_structure_list
is an interface for manipulation with the list of data
structures to be transformed which is accessed through
data_struct_list. */
static void
add_struct_to_data_struct_list (tree type)
{
struct data_structure *d_node;
struct struct_list *new_node = NULL;
int num_fields;
list found = NULL;
type = TYPE_MAIN_VARIANT (type);
found = find_struct_in_list (type);
if (found)
return;
d_node = (struct data_structure *)
xcalloc (1, sizeof (struct data_structure));
num_fields = fields_length (type);
d_node->decl = type;
d_node->num_fields = num_fields;
d_node->fields = get_fields (type, num_fields);
d_node->struct_clustering = NULL;
d_node->accs = NULL;
new_node = (struct struct_list *) xmalloc (sizeof(struct struct_list));
new_node->struct_data = d_node;
new_node->next = data_struct_list;
data_struct_list = new_node;
if (dump_file)
{
fprintf (dump_file, "\nAdding data structure \"");
print_generic_expr (dump_file, type, 0);
fprintf (dump_file, "\" to data_struct_list.");
}
}
static void
free_tree_list (tree_list list_p)
{
tree_list node = list_p;
tree_list tmp_node;
while (node)
{
tmp_node = node;
node = node->next;
free (tmp_node);
}
}
static void
free_field_acc_list (struct field_access_site *accs)
{
struct field_access_site * tmp_acc = accs;
struct field_access_site * tmp_acc1;
if (!accs)
return;
while (tmp_acc)
{
tmp_acc1 = tmp_acc;
tmp_acc = tmp_acc->next;
free (tmp_acc1);
}
}
static void
free_access_list (struct access_site *accs)
{
struct access_site * tmp_acc = accs;
struct access_site * tmp_acc1;
if (!accs)
return;
while (tmp_acc)
{
tmp_acc1 = tmp_acc;
tmp_acc = tmp_acc->next;
free_tree_list (tmp_acc1->vars);
free (tmp_acc1);
}
}
static void
free_data_struct (d_str d_node)
{
int i;
if (!d_node)
return;
if (dump_file)
{
fprintf (dump_file, "\nRemoving data structure \"");
print_generic_expr (dump_file, d_node->decl, 0);
fprintf (dump_file, "\" from data_struct_list.");
}
/* Free all space under d_node. */
if (d_node->fields)
{
for (i = 0; i < d_node->num_fields; i++)
free_field_acc_list (d_node->fields[i].acc_sites);
free (d_node->fields);
}
if (d_node->accs)
free_access_list (d_node->accs);
if (d_node->struct_clustering)
free_struct_cluster (d_node->struct_clustering);
if (d_node->new_types)
free_tree_list (d_node->new_types);
}
static void
remove_struct_from_data_struct_list (struct struct_list *str_to_remove)
{
struct struct_list *curr_str, *prev;
if (data_struct_list == str_to_remove)
{
free_data_struct (data_struct_list->struct_data);
data_struct_list = data_struct_list->next;
free (str_to_remove);
return;
}
for (curr_str = data_struct_list->next, prev = data_struct_list;
curr_str; curr_str = curr_str->next, prev = prev->next)
if (curr_str == str_to_remove)
{
free_data_struct (curr_str->struct_data);
prev->next = curr_str->next;
free (curr_str);
break;
}
}
static void
remove_structs_from_data_struct_list (bitmap nested_structs)
{
list tmp = data_struct_list;
while (tmp)
{
d_str str = tmp->struct_data;
if (bitmap_bit_p (nested_structs, TYPE_UID (str->decl)))
{
list tmp1 = tmp;
tmp = tmp->next;
remove_struct_from_data_struct_list (tmp1);
}
else
tmp = tmp->next;
}
}
static void
free_data_structure_list (void)
{
list current_struct = data_struct_list;
while (current_struct)
{
list tmp = current_struct;
d_str d_node = current_struct->struct_data;
free_data_struct (d_node);
current_struct = current_struct->next;
/* Free curent_struct itself. */
if (tmp)
free (tmp);
}
}
static void
free_malloc_data (void)
{
struct malloc_struct_new *cur_malloc = malloc_data_list_new;
struct malloc_struct_new *tmp;
while (cur_malloc)
{
struct malloc_call_new * call = cur_malloc->malloc_list;
while (call)
{
struct malloc_call_new * call1 = call;
call = call->next;
free (call1);
}
tmp = cur_malloc;
cur_malloc = cur_malloc->next;
free (tmp);
}
}
/* This function calculates maximum field count in
the structure STR. */
static gcov_type
get_max_field_count (d_str str)
{
gcov_type max = 0;
int i;
for (i = 0; i < str->num_fields; i++)
if (str->fields[i].count > max)
max = str->fields[i].count;
return max;
}
static void
peel_field (int i, d_str ds)
{
struct field_cluster *crr_cluster = NULL;
crr_cluster =
(struct field_cluster *) xcalloc (1, sizeof (struct field_cluster));
crr_cluster->sibling = ds->struct_clustering;
ds->struct_clustering = crr_cluster;
crr_cluster->fields_in_cluster =
sbitmap_alloc ((unsigned int) ds->num_fields);
sbitmap_zero (crr_cluster->fields_in_cluster);
SET_BIT (crr_cluster->fields_in_cluster, i);
}
static void
gen_cluster (sbitmap fields, d_str ds)
{
struct field_cluster *crr_cluster = NULL;
crr_cluster =
(struct field_cluster *) xcalloc (1, sizeof (struct field_cluster));
crr_cluster->sibling = ds->struct_clustering;
ds->struct_clustering = crr_cluster;
crr_cluster->fields_in_cluster = fields;
}
/* This function peels the field into separate structure if it's
suffitiently hot. */
static void
peel_str_type_into_seperate_fields (d_str ds)
{
gcov_type max_field_count;
sbitmap fields_left = sbitmap_alloc (ds->num_fields);
sbitmap_ones (fields_left);
int i;
max_field_count =
(gcov_type) (get_max_field_count (ds)/100)*90;
ds->struct_clustering = NULL;
for (i = 0; i < ds->num_fields; i++)
{
if (ds->count >= max_field_count)
{
RESET_BIT (fields_left, i);
peel_field (i, ds);
}
}
i = sbitmap_first_set_bit (fields_left);
if (i != -1)
gen_cluster (fields_left, ds);
else
sbitmap_free (fields_left);
}
/* This function analyzes structure form of structures
in data_struct_list. If we are not capable to transform
structure of some form, we remove it from list of structs.
Right now we cannot handle nested structs, when nesting is
through any level of pointer or arrays.
TBD: release these constrains in future.
Note, that in this function we suppose that all structs
in program are members of data_struct_list right now,
without excluding escaping types.
*/
static void
analyze_struct_form (bitmap non_suitable_types)
{
list tmp = data_struct_list;
while(tmp)
{
d_str str = tmp->struct_data;
int i;
for (i = 0; i < str->num_fields; i++)
{
tree f_type = strip_type(TREE_TYPE (str->fields[i].decl));
if (TREE_CODE (f_type) == RECORD_TYPE)
{
bitmap_set_bit (non_suitable_types, TYPE_UID (TYPE_MAIN_VARIANT (f_type)));
bitmap_set_bit (non_suitable_types, TYPE_UID (str->decl));
}
}
tmp = tmp->next;
}
}
/* This function collect data structures potential
for peeling transformation and insert them into data_struct_list. */
static void
collect_data_structs (void)
{
bitmap non_suitable_types = BITMAP_ALLOC (NULL);
/* If program contains user defined mallocs, we give up. */
if (program_redefines_malloc_p ())
return;
/* Build data structures list of all data structures
in the program. */
build_data_structure_list (non_suitable_types);
/* In this function we analyze whether the form of
structure is such that we are capable to transform it.
Nested structures are checked here. */
analyze_struct_form (non_suitable_types);
/* This function excludes structure types escape compilation unit. */
exclude_escaping_types (non_suitable_types);
/* We cannot change data layout of structs with bitfields. */
exclude_types_with_bit_fields (non_suitable_types);
remove_structs_from_data_struct_list (non_suitable_types);
BITMAP_FREE (non_suitable_types);
if (!data_struct_list)
{
if (dump_file)
fprintf (dump_file, "\nNo structures to transform. Exiting...");
return;
}
}
static void
print_field_acc_sites (struct field_access_site *acc_sites)
{
struct field_access_site *acc;
if (!dump_file)
return;
for (acc = acc_sites; acc; acc = acc->next)
{
fprintf(dump_file, "\n");
if (acc->stmt)
print_generic_stmt (dump_file, acc->stmt, 0);
if (acc->ref_def_stmt)
print_generic_stmt (dump_file, acc->ref_def_stmt, 0);
if (acc->cast_stmt)
print_generic_stmt (dump_file, acc->cast_stmt, 0);
}
}
static void
print_access_sites (struct access_site *acc_sites)
{
struct access_site *acc;
if (!dump_file)
return;
for (acc = acc_sites; acc; acc = acc->next)
{
tree_list list_p;
fprintf(dump_file, "\n");
if (acc->stmt)
print_generic_stmt (dump_file, acc->stmt, 0);
fprintf(dump_file, " : ");
for (list_p = acc->vars; list_p; list_p = list_p->next)
{
print_generic_stmt (dump_file, list_p->data, 0);
fprintf(dump_file, ", ");
}
}
}
static void
print_accesses (void)
{
list tmp = data_struct_list;
if (!dump_file)
return;
for (tmp = data_struct_list; tmp; tmp = tmp->next)
{
int i;
d_str str = tmp->struct_data;
fprintf (dump_file, "\nAccess sites of struct ");
print_generic_expr (dump_file, str->decl, 0);
for (i = 0; i < str->num_fields; i++)
{
fprintf (dump_file, "\nAccess site of field ");
print_generic_expr (dump_file, str->fields[i].decl, 0);
print_field_acc_sites (str->fields[i].acc_sites);
fprintf (dump_file, ":\n");
}
fprintf (dump_file, "\nGeneral access sites\n");
print_access_sites (str->accs);
}
}
static bool
is_safe_cond_expr (tree cond_stmt)
{
tree arg0, arg1;
list tmp0, tmp1;
tree cond = COND_EXPR_COND (cond_stmt);
if (TREE_CODE (cond) != EQ_EXPR
&& TREE_CODE (cond) != NE_EXPR)
return false;
if (TREE_CODE_LENGTH (TREE_CODE (cond)) != 2)
return false;
arg0 = TREE_OPERAND (cond, 0);
arg1 = TREE_OPERAND (cond, 1);
tmp0 = find_struct_in_list (strip_type (get_type_of_var (arg0)));
tmp1 = find_struct_in_list (strip_type (get_type_of_var (arg1)));
if (!((!tmp0 && tmp1) || (!tmp1 && tmp0)))
return false;
return true;
}
static void
check_cond_exprs (void)
{
list str_node = data_struct_list;
while (str_node)
{
d_str str = str_node->struct_data;
struct access_site * acc;
list str_node1 = str_node;
str_node = str_node->next;
for (acc = str->accs; acc; acc = acc->next)
if (TREE_CODE (acc->stmt) == COND_EXPR)
{
if (!is_safe_cond_expr (acc->stmt))
remove_struct_from_data_struct_list (str_node1);
}
}
}
/* This function collects data accesses for the structures
from data_struct_list. For each structure field it updates its count
in data_field_entry. */
static void
collect_data_accesses (void)
{
struct cgraph_node *c_node;
for (c_node = cgraph_nodes; c_node; c_node = c_node->next)
{
enum availability avail = cgraph_function_body_availability (c_node);
if (avail == AVAIL_LOCAL || avail == AVAIL_AVAILABLE)
{
struct function *func = DECL_STRUCT_FUNCTION (c_node->decl);
if (!c_node->next_clone)
{
/* Collect accesses of the fields of structure
under consideretion that happen in func. */
collect_accesses_in_func (func);
}
exclude_malloc_and_field_accs (c_node);
}
}
check_cond_exprs ();
/* Print collected accesses. */
print_accesses ();
}
static void
exclude_cold_structs (void)
{
gcov_type hotest_struct_count = 0;
list tmp = data_struct_list;
/* We summarize counts of fields into structure count. */
while (tmp)
{
int i;
d_str str = tmp->struct_data;
str->count = 0;
for (i = 0; i < str->num_fields; i++)
{
if (dump_file)
{
fprintf (dump_file, "\nCounter of field \"");
print_generic_expr (dump_file, str->fields[i].decl, 0);
fprintf (dump_file, "\" is " HOST_WIDE_INT_PRINT_DEC,
str->fields[i].count);
}
str->count += str->fields[i].count;
}
if (dump_file)
{
fprintf (dump_file, "\nCounter of struct \"");
print_generic_expr (dump_file, str->decl, 0);
fprintf (dump_file, "\" is " HOST_WIDE_INT_PRINT_DEC, str->count);
}
if (str->count > hotest_struct_count)
hotest_struct_count = str->count;
tmp = tmp->next;
}
/* Compare counts of structures. */
tmp = data_struct_list;
/* Remove cold structs from data_struct_list. */
while (tmp)
{
d_str str = tmp->struct_data;
if (str->count < (hotest_struct_count / COLD_STRUCTURE_RATIO))
{
list tmp1 = tmp;
tmp = tmp->next;
remove_struct_from_data_struct_list (tmp1);
}
else
tmp = tmp->next;
}
}
static void
peel_structs (void)
{
list tmp = data_struct_list;
while (tmp)
{
d_str str = tmp->struct_data;
peel_str_type_into_seperate_fields (str);
tmp = tmp->next;
}
}
/* Collect allocation sites - mallocs. In case of arrays
we have nothing to do. */
static void
collect_allocation_sites (void)
{
collect_malloc_data ();
}
/* Free all data structures used in this optimization. */
static void
free_data_structs (void)
{
free_data_structure_list ();
free_malloc_data ();
}
/* Perform full structure decomposition (peeling). */
static void
reorg_structs (void)
{
/* Stage1. */
/* Collect data struct types. */
collect_data_structs ();
collect_allocation_sites ();
/* Collect data accesses. */
collect_data_accesses ();
/* We transform only hot structs. */
exclude_cold_structs ();
/* Stage2. */
peel_structs ();
/* Stage3. */
/* Do the actual transformation. */
do_reorg ();
/* Free all data structures used in this optimization. */
free_data_structs ();
}
static void
add_call_to_malloc_list (tree fn_decl, tree stmt, d_str str)
{
struct malloc_struct_new *cur_malloc = NULL;
struct malloc_struct_new *malloc_node = NULL;
for (cur_malloc = malloc_data_list_new; cur_malloc;
cur_malloc = cur_malloc->next)
if (cur_malloc->func == fn_decl)
break;
if (cur_malloc)
{
struct malloc_call_new * m_call = (struct malloc_call_new *)
xmalloc (sizeof (struct malloc_call_new));
m_call->next = cur_malloc->malloc_list;
m_call->malloc_call_stmt = stmt;
m_call->str = str;
cur_malloc->malloc_list = m_call;
}
else
{
malloc_node = (struct malloc_struct_new *)
xmalloc (sizeof (struct malloc_struct_new));
malloc_node->func = fn_decl;
malloc_node->malloc_list = (struct malloc_call_new *)
xmalloc (sizeof (struct malloc_call_new));
malloc_node->malloc_list->malloc_call_stmt = stmt;
malloc_node->malloc_list->str = str;
malloc_node->malloc_list->next = NULL;
malloc_node->next = malloc_data_list_new;
malloc_data_list_new = malloc_node;
}
if (dump_file)
{
fprintf (dump_file, "\nAdding stmt ");
print_generic_stmt (dump_file, stmt, 0);
fprintf (dump_file, " to list of mallocs.");
}
}
static unsigned int
reorg_structs_drive (void)
{
reorg_structs ();
return 0;
}
static bool
struct_reorg_gate (void)
{
return flag_ipa_struct_reorg && flag_whole_program;
}
struct tree_opt_pass pass_ipa_struct_reorg =
{
"ipa_struct_reorg", /* name */
struct_reorg_gate, /* gate */
reorg_structs_drive, /* execute */
NULL, /* sub */
NULL, /* next */
0, /* static_pass_number */
TV_INTEGRATION, /* tv_id */
0, /* properties_required */
0, /* properties_provided */
0, /* properties_destroyed */
TODO_verify_ssa, /* todo_flags_start */
TODO_dump_func | TODO_verify_ssa, /* todo_flags_finish */
0 /* letter */
};
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: struct-reorg optimization
2007-08-01 12:06 ` Olga Golovanevsky
@ 2007-08-17 22:25 ` Jan Hubicka
2007-08-21 22:22 ` Olga Golovanevsky
` (2 more replies)
0 siblings, 3 replies; 17+ messages in thread
From: Jan Hubicka @ 2007-08-17 22:25 UTC (permalink / raw)
To: Olga Golovanevsky
Cc: Daniel Berlin, Jan Hubicka, Diego Novillo, Kenneth Zadeck,
Peter Bergner, gcc-patches
Hi,
I don't think I can approve the patch under the cgraph maintainer rule,
but at least couple of comments. Overall the patch looks fine to me,
the changes outside struct-reorg.c are fine.
I would perhaps call it ipa-struct-reorg.c to stay consistent with file
naming of ipa passes (we really need to move from flat directory
structure).
You are missing a number of comments in front of functions that are
required by coding standards, also documenting arguments at least in
case there are couple of those would be nice (like
is_complicated_indirect_ref)
if (TREE_CODE (type) == RECORD_TYPE)
{
return true;
}
else
return false;
is better as:
return (TREE_CODE (type) == RECORD_TYPE);
/* Print structure type. */
static void
print_struct_type (tree new_type, char * indent)
How this facility differ from what we already have in pretty-print.c?
/* This function is a hack to overcome the types problem.
When number of compilation units are compiled together
under -combine flag, the TYPE_MAIN_VARIANT cannot
be structed. First we comparing names, but they do not
also appear, like for example, in stmts. Only then we
are going inside structures.
*/
static bool
is_equal_types (tree type1, tree type2)
In longer term I definitly would like to have type merging done
separately, but I guess it is resonable short term hack (or in initial
implementation just require types to match pesimizing -combine case)
name1 = get_type_name (type1);
name2 = get_type_name (type2);
if (name1 && name2 && !strcmp (name1, name2))
return true;
if (name1 && name2 && strcmp (name1, name2))
return false;
What happens if the two units declare type with same name and completely
different layout and use it for different variables?
(ie in completely legal way). Will we every get into this matching
predicate?
/* Check whether the indirect_ref is of the form we can deal with. */
static bool
is_complicated_indirect_ref (tree str_decl, tree ref, tree * num_p, tree * offset_p,
tree * base_p, tree * ref_def_stmt_p, tree * cast_stmt_p)
Comment on operands here. Perhaps would be better if packed into some
descriptor structure a-la ifcvt.c. Also it seems bit weird for me to
have predicae returning true but setting the _p variables only when
returning false. Perhaps decompose_indirect_ref_acces would be better
name, then one can return 0 as error and 1 when _p values was computed.
/* Helper for walk_tree called from collect_accesses_in_bb function. */
static tree
get_stmt_accesses (tree *tp, int *walk_subtrees, void *data)
{
tree stmt = (tree) data;
tree t = *tp;
if (!t)
return NULL;
switch (TREE_CODE (t))
{
case BIT_FIELD_REF:
...
break;
case GIMPLE_MODIFY_STMT:
On GIMPLE we would probably benefit from organizing this on dealing with
statement codes first (ie GIMPLE_MODIFY_STMT) having separate function
looking for stmt accesses in the GIMPLE operands (ie _REFs and frineds)
walking via handled refs.
The walk_tree uses should probably mostly go away for tupleification
benefits anyway.
case COMPONENT_REF:
{
tree ref = TREE_OPERAND (t, 0);
tree field_decl = TREE_OPERAND (t, 1);
if ((TREE_CODE (ref) == INDIRECT_REF
|| TREE_CODE (ref) == ARRAY_REF
|| TREE_CODE (ref) == VAR_DECL)
&& TREE_CODE (field_decl) == FIELD_DECL)
What would happen with, for instance, COMPONENT_REF containing
VIEW_CONVERT_EXPR, REALPART_EXPR or IMAGPART_EXPR?
/* Generate new structure name. */
static inline char *
gen_cluster_name (tree decl, int clust_num, int str_num)
Can't we use here the facility for generating names of nested functions
and local statics? It has one global counter so you probably won't need
to check if the name was already taken (assuming we don't have other
sources of basename.id type of names that we probably should not).
static void
finish_var_creation (tree new_decl)
{
if (!var_ann (new_decl))
create_var_ann (new_decl);
add_referenced_var will create one for you.
I am not sure why you need gimple_referenced_vars predicate - you are
always seeing SSA functions and thus it should be always up.
if (is_global_var (new_decl))
mark_call_clobbered (new_decl, ESCAPE_UNKNOWN);
mark_sym_for_renaming (new_decl);
if (gimple_referenced_vars (cfun))
add_referenced_var (new_decl);
}
static char *
gen_var_name (tree orig_decl, int i, tree * id_node)
Similarly the facility for producing local statics should help here.
/* This function copy attributes form ORIG_DECL to NEW_DECL. */
static inline void
copy_decl_attributes (tree new_decl, tree orig_decl)
This seems like you want something similar to copy_decl_no_change in
tree-inline.c For C++ I am quite convinced it is neccesary to use the
langhook still :(
struct type_wrapper
Some explanation of what are the wrappers for?
/* For each SSA_NAME or VAR_DECL local variable of structure
type relevant for transformation in current function (cfun)
this function generate new variable(s) to replace it. */
static void
create_new_local_vars (void)
{
unsigned i;
tree var_list;
if (gimple_in_ssa_p (cfun))
You probably want just require all functions in SSA form so you don't
need to do analysis for SSA and non-SSA bodies...
if (is_malloc_of_struct (stmt, &str_node))
{
d_str str = str_node->struct_data;
/* We support only malloc now. */
C++ new operator is probably good idea to add, but you are probably
aware of that ;)
if (DECL_FUNCTION_CODE (decl) == BUILT_IN_MALLOC)
add_call_to_malloc_list (node->decl, stmt, str);
else
remove_struct_from_data_struct_list (str_node);
if (TREE_CODE (rhs) == MULT_EXPR)
{
tree arg0 = TREE_OPERAND (rhs, 0);
tree arg1 = TREE_OPERAND (rhs, 1);
if (operand_equal_p (arg0, struct_size, OEP_ONLY_CONST))
From canonicality you probably can expect the constant to be in arg1.
Don't you want to also handle constantly size allocations?
/* Do transformation for a function represented by NODE. */
static void
do_reorg_for_func (struct cgraph_node *node)
{
create_new_local_vars ();
create_new_mallocs_for_func (node);
create_new_accesses_for_func ();
update_ssa (TODO_update_ssa);
cleanup_tree_cfg ();
You should probably use function local cfun->todo lists. You care above
to update cgraph edges, however ding update_ssa/cleanup_tree_cfg is
going to disturb cgraph anyway, so you probably just want to add TODO
flag for rebuilding cgraph edges as well.
Nice work and sorry for delays with the review!
Honza
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: struct-reorg optimization
2007-08-17 22:25 ` Jan Hubicka
@ 2007-08-21 22:22 ` Olga Golovanevsky
2007-08-25 10:44 ` Jan Hubicka
2007-08-31 20:33 ` Olga Golovanevsky
2007-09-05 11:48 ` Olga Golovanevsky
2 siblings, 1 reply; 17+ messages in thread
From: Olga Golovanevsky @ 2007-08-21 22:22 UTC (permalink / raw)
To: Jan Hubicka
Cc: Peter Bergner, Daniel Berlin, Diego Novillo, gcc-patches,
Jan Hubicka, Kenneth Zadeck
Jan Hubicka <hubicka@ucw.cz> wrote on 18/08/2007 01:24:44:
> Hi,
> I don't think I can approve the patch under the cgraph maintainer rule,
> but at least couple of comments. Overall the patch looks fine to me,
> the changes outside struct-reorg.c are fine.
> I would perhaps call it ipa-struct-reorg.c to stay consistent with file
> naming of ipa passes (we really need to move from flat directory
> structure).
>
ok.
> You are missing a number of comments in front of functions that are
> required by coding standards, also documenting arguments at least in
> case there are couple of those would be nice (like
> is_complicated_indirect_ref)
sure, I'll add it.
>
> if (TREE_CODE (type) == RECORD_TYPE)
> {
> return true;
> }
> else
> return false;
> is better as:
> return (TREE_CODE (type) == RECORD_TYPE);
sure.
>
> /* Print structure type. */
> static void
> print_struct_type (tree new_type, char * indent)
>
> How this facility differ from what we already have in pretty-print.c?
In general, print_generic_expr does this work, but it prints struct name,
if
it exists and decl of struct otherwise, i.e. in tree-pretty-print.c:
case RECORD_TYPE:
...
if (TYPE_NAME (node))
dump_generic_node (buffer, TYPE_NAME (node), spc, flags, false);
else
print_struct_decl (buffer, node, spc, flags);
break;
and I would want to see both.
Is there any other function that can be used?
>
> /* This function is a hack to overcome the types problem.
> When number of compilation units are compiled together
> under -combine flag, the TYPE_MAIN_VARIANT cannot
> be structed. First we comparing names, but they do not
> also appear, like for example, in stmts. Only then we
> are going inside structures.
> */
> static bool
> is_equal_types (tree type1, tree type2)
>
> In longer term I definitly would like to have type merging done
> separately,
agree with you.
> but I guess it is resonable short term hack (or in initial
> implementation just require types to match pesimizing -combine case)
I got to this function due to number of reasons:
1. types cannot be compared as trees (type1 == type2) for multiple units,
i.e.
type1 != type2 for the same type where type1 represents type in unit1
and type2 represent type in unit2
2. TYPE_MAIN_VARIANTs cannot be compared as trees
(TYPE_MAIN_VARIAN (type1) == TYPE_MAIN_VARIANT (type2))
for multiple units, i.e. TYPE_MAIN_VARIANT of the same type are different
for different compilation units
3. TYPE_MAIN_VARIAN keeps no name of the type.
4. the name of type is not always preserved, as for example,
in array_refs of arrays of structures. (I suppose it just uses
TYPE_MAIN_VARIAN of the structure type here.)
>
> name1 = get_type_name (type1);
> name2 = get_type_name (type2);
>
> if (name1 && name2 && !strcmp (name1, name2))
> return true;
>
> if (name1 && name2 && strcmp (name1, name2))
> return false;
>
> What happens if the two units declare type with same name and completely
> different layout and use it for different variables?
> (ie in completely legal way). Will we every get into this matching
> predicate?
We have two corner cases here: structs with the same name and different
bodies,
and structs with same body and different names. So I think theoretically
structs are equal if they have the same name and the same body. But we cut
the name in many cases (like in TYPE_MAIN_VARIAN), so there is no good
solution
for now.
>
> /* Check whether the indirect_ref is of the form we can deal with. */
> static bool
> is_complicated_indirect_ref (tree str_decl, tree ref, tree * num_p,
> tree * offset_p,
> tree * base_p, tree * ref_def_stmt_p, tree * cast_stmt_p)
>
> Comment on operands here. Perhaps would be better if packed into some
> descriptor structure a-la ifcvt.c. Also it seems bit weird for me to
> have predicae returning true but setting the _p variables only when
> returning false. Perhaps decompose_indirect_ref_acces would be better
> name, then one can return 0 as error and 1 when _p values was computed.
ok with me.
>
> /* Helper for walk_tree called from collect_accesses_in_bb function. */
> static tree
> get_stmt_accesses (tree *tp, int *walk_subtrees, void *data)
> {
> tree stmt = (tree) data;
> tree t = *tp;
>
> if (!t)
> return NULL;
>
> switch (TREE_CODE (t))
> {
> case BIT_FIELD_REF:
> ...
> break;
> case GIMPLE_MODIFY_STMT:
>
> On GIMPLE we would probably benefit from organizing this on dealing with
> statement codes first (ie GIMPLE_MODIFY_STMT) having separate function
> looking for stmt accesses in the GIMPLE operands (ie _REFs and frineds)
> walking via handled refs.
yes, agree.
>
> The walk_tree uses should probably mostly go away for tupleification
> benefits anyway.
> case COMPONENT_REF:
> {
> tree ref = TREE_OPERAND (t, 0);
> tree field_decl = TREE_OPERAND (t, 1);
>
>
> if ((TREE_CODE (ref) == INDIRECT_REF
> || TREE_CODE (ref) == ARRAY_REF
> || TREE_CODE (ref) == VAR_DECL)
> && TREE_CODE (field_decl) == FIELD_DECL)
>
> What would happen with, for instance, COMPONENT_REF containing
> VIEW_CONVERT_EXPR, REALPART_EXPR or IMAGPART_EXPR?
any example of source code that can bring to these cases
would be very helpful.
>
> /* Generate new structure name. */
> static inline char *
> gen_cluster_name (tree decl, int clust_num, int str_num)
>
> Can't we use here the facility for generating names of nested functions
> and local statics? It has one global counter so you probably won't need
> to check if the name was already taken (assuming we don't have other
> sources of basename.id type of names that we probably should not).
which function do you mean?
>
> static void
> finish_var_creation (tree new_decl)
> {
> if (!var_ann (new_decl))
> create_var_ann (new_decl);
> add_referenced_var will create one for you.
ok.
> I am not sure why you need gimple_referenced_vars predicate - you are
> always seeing SSA functions and thus it should be always up.
>
> if (is_global_var (new_decl))
> mark_call_clobbered (new_decl, ESCAPE_UNKNOWN);
>
>
> mark_sym_for_renaming (new_decl);
> if (gimple_referenced_vars (cfun))
> add_referenced_var (new_decl);
> }
agree.
>
> static char *
> gen_var_name (tree orig_decl, int i, tree * id_node)
>
> Similarly the facility for producing local statics should help here.
>
> /* This function copy attributes form ORIG_DECL to NEW_DECL. */
> static inline void
> copy_decl_attributes (tree new_decl, tree orig_decl)
>
> This seems like you want something similar to copy_decl_no_change in
> tree-inline.c For C++ I am quite convinced it is neccesary to use the
> langhook still :(
>
I see. Working to test C++ patterns.
>
> struct type_wrapper
>
> Some explanation of what are the wrappers for?
it is used to generate new type when structure is wrapped
by pointers and arrays. I'll add an explanations there.
>
> /* For each SSA_NAME or VAR_DECL local variable of structure
> type relevant for transformation in current function (cfun)
> this function generate new variable(s) to replace it. */
> static void
> create_new_local_vars (void)
> {
> unsigned i;
> tree var_list;
>
> if (gimple_in_ssa_p (cfun))
>
> You probably want just require all functions in SSA form so you don't
> need to do analysis for SSA and non-SSA bodies...
yes, true.
> if (is_malloc_of_struct (stmt, &str_node))
> {
> d_str str = str_node->struct_data;
>
> /* We support only malloc now. */
> C++ new operator is probably good idea to add, but you are probably
> aware of that ;)
yes.
> if (DECL_FUNCTION_CODE (decl) == BUILT_IN_MALLOC)
> add_call_to_malloc_list (node->decl, stmt, str);
> else
> remove_struct_from_data_struct_list (str_node);
>
>
> if (TREE_CODE (rhs) == MULT_EXPR)
> {
> tree arg0 = TREE_OPERAND (rhs, 0);
> tree arg1 = TREE_OPERAND (rhs, 1);
>
> if (operand_equal_p (arg0, struct_size, OEP_ONLY_CONST))
> From canonicality you probably can expect the constant to be in arg1.
> Don't you want to also handle constantly size allocations?
yes, it requires extensions; it's partly prepared in
gen_num_of_structs_in_malloc.
>
> /* Do transformation for a function represented by NODE. */
> static void
> do_reorg_for_func (struct cgraph_node *node)
> {
> create_new_local_vars ();
>
> create_new_mallocs_for_func (node);
> create_new_accesses_for_func ();
> update_ssa (TODO_update_ssa);
> cleanup_tree_cfg ();
>
> You should probably use function local cfun->todo lists. You care above
> to update cgraph edges, however ding update_ssa/cleanup_tree_cfg is
> going to disturb cgraph anyway, so you probably just want to add TODO
> flag for rebuilding cgraph edges as well.
ok.
>
> Nice work and sorry for delays with the review!
I am working to incorporate your comments now.
Thank you a lot for reviewing this patch!
Olga
> Honza
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: struct-reorg optimization
2007-08-21 22:22 ` Olga Golovanevsky
@ 2007-08-25 10:44 ` Jan Hubicka
2007-08-26 16:21 ` Olga Golovanevsky
0 siblings, 1 reply; 17+ messages in thread
From: Jan Hubicka @ 2007-08-25 10:44 UTC (permalink / raw)
To: Olga Golovanevsky
Cc: Jan Hubicka, Peter Bergner, Daniel Berlin, Diego Novillo,
gcc-patches, Jan Hubicka, Kenneth Zadeck
Hi,
>
> In general, print_generic_expr does this work, but it prints struct name,
> if
> it exists and decl of struct otherwise, i.e. in tree-pretty-print.c:
>
> case RECORD_TYPE:
> ...
> if (TYPE_NAME (node))
> dump_generic_node (buffer, TYPE_NAME (node), spc, flags, false);
> else
> print_struct_decl (buffer, node, spc, flags);
> break;
>
> and I would want to see both.
> Is there any other function that can be used?
Hmm, i think extra TDF flag would do the job here.
> > but I guess it is resonable short term hack (or in initial
> > implementation just require types to match pesimizing -combine case)
>
> I got to this function due to number of reasons:
>
> 1. types cannot be compared as trees (type1 == type2) for multiple units,
> i.e.
> type1 != type2 for the same type where type1 represents type in unit1
> and type2 represent type in unit2
>
> 2. TYPE_MAIN_VARIANTs cannot be compared as trees
> (TYPE_MAIN_VARIAN (type1) == TYPE_MAIN_VARIANT (type2))
> for multiple units, i.e. TYPE_MAIN_VARIANT of the same type are different
> for different compilation units
>
> 3. TYPE_MAIN_VARIAN keeps no name of the type.
>
> 4. the name of type is not always preserved, as for example,
> in array_refs of arrays of structures. (I suppose it just uses
> TYPE_MAIN_VARIAN of the structure type here.)
Yep, understood. --combine is quite broken in this respect :(
>
> >
> > name1 = get_type_name (type1);
> > name2 = get_type_name (type2);
> >
> > if (name1 && name2 && !strcmp (name1, name2))
> > return true;
> >
> > if (name1 && name2 && strcmp (name1, name2))
> > return false;
> >
> > What happens if the two units declare type with same name and completely
> > different layout and use it for different variables?
> > (ie in completely legal way). Will we every get into this matching
> > predicate?
>
> We have two corner cases here: structs with the same name and different
> bodies,
> and structs with same body and different names. So I think theoretically
> structs are equal if they have the same name and the same body. But we cut
> the name in many cases (like in TYPE_MAIN_VARIAN), so there is no good
> solution
> for now.
Hmm, OK. Will tricking same name to different structures cause
miscompilations or somethign?
> > /* Generate new structure name. */
> > static inline char *
> > gen_cluster_name (tree decl, int clust_num, int str_num)
> >
> > Can't we use here the facility for generating names of nested functions
> > and local statics? It has one global counter so you probably won't need
> > to check if the name was already taken (assuming we don't have other
> > sources of basename.id type of names that we probably should not).
>
> which function do you mean?
I will have to check out. The local names are getting increasing ids
from some place I forgot. I will write later since I am bit in hurry
now.
Honza
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: struct-reorg optimization
2007-08-25 10:44 ` Jan Hubicka
@ 2007-08-26 16:21 ` Olga Golovanevsky
0 siblings, 0 replies; 17+ messages in thread
From: Olga Golovanevsky @ 2007-08-26 16:21 UTC (permalink / raw)
To: Jan Hubicka
Cc: Peter Bergner, Daniel Berlin, Diego Novillo, gcc-patches,
Jan Hubicka, Jan Hubicka, Kenneth Zadeck
Jan Hubicka <jh@suse.cz> wrote on 25/08/2007 12:57:55:
> >
> > >
> > > name1 = get_type_name (type1);
> > > name2 = get_type_name (type2);
> > >
> > > if (name1 && name2 && !strcmp (name1, name2))
> > > return true;
> > >
> > > if (name1 && name2 && strcmp (name1, name2))
> > > return false;
> > >
> > > What happens if the two units declare type with same name and
completely
> > > different layout and use it for different variables?
> > > (ie in completely legal way). Will we every get into this matching
> > > predicate?
We are not getting here with the type names in this case, because we
are comparing TYPE_MAIN_VARIANTs, which keep no names.
> >
> > We have two corner cases here: structs with the same name and different
> > bodies,
> > and structs with same body and different names. So I think
theoretically
> > structs are equal if they have the same name and the same body. But we
cut
> > the name in many cases (like in TYPE_MAIN_VARIAN), so there is no good
> > solution
> > for now.
>
> Hmm, OK. Will tricking same name to different structures cause
> miscompilations or somethign?
It looks like the error happens only if these types really conflict.
For example, if str_t is defined differently in unit1.c
and unit2.c and there is a function foo (str_t) defined in unit2.c
and called from unit1.c, then compiler detects conflicting type
for foo () when running with -combine:
unit2.c:14: error: conflicting types for ‘foo’
unit1.c:11: error: previous declaration of ‘foo’ was here
(I suppose it happens during building cgraph).
Otherwise they successfully coexist, i.e. each unit uses its own type.
Olga
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: struct-reorg optimization
2007-08-17 22:25 ` Jan Hubicka
2007-08-21 22:22 ` Olga Golovanevsky
@ 2007-08-31 20:33 ` Olga Golovanevsky
2007-09-06 21:04 ` Diego Novillo
2007-09-05 11:48 ` Olga Golovanevsky
2 siblings, 1 reply; 17+ messages in thread
From: Olga Golovanevsky @ 2007-08-31 20:33 UTC (permalink / raw)
To: Diego Novillo, Jan Hubicka
Cc: Peter Bergner, Daniel Berlin, Diego Novillo, gcc-patches,
Jan Hubicka, Kenneth Zadeck
[-- Attachment #1: Type: text/plain, Size: 1336 bytes --]
Jan Hubicka <hubicka@ucw.cz> wrote on 18/08/2007 01:24:44:
> Hi,
> I don't think I can approve the patch under the cgraph maintainer rule,
> but at least couple of comments. Overall the patch looks fine to me,
> the changes outside struct-reorg.c are fine.
Jan, Diego,
The attached is revised patch with Jan's comments inside.
I incorporated most of them, except C++ related issues and
TODO_ flag for rebuild_cgraph_edges, that I did not found
(should I define it?).
The issue of C++ is still ahead of me, and I would prefer to take
it orderly, with well developed concept, and RFC, etc..
The current patch is bootstraped on ppc. It's tested for C and fortran
with {-fwhole-program, combine} flags agains {-fwhole-program, combine,
ipa-struct-reorg} flags, and all regression has been closed.
The changes I made are part of this patch.
Any more comments?
Ok for mainline?
:ADDPATCH ipa ssa:
Olga
2007-08-31 Olga Golovanevsky <olga@il.ibm.com>
* ipa-struct-reorg.c, ipa-struct-reorg.h: New files.
* tree.h: Add pass_ipa_struct_reorg.
* common.opt: Add ipa-struct-reorg flag.
* Makefile.in: Add ipa-strcut-reorg.o compilation.
* passes.c: Add pass pass_ipa_struct_reorg.
(See attached file: ipa-struct-reorg.txt)(See attached file:
ipa-struct-reorg.c)(See attached file: ipa-struct-reorg.h)
[-- Attachment #2: ipa-struct-reorg.txt --]
[-- Type: text/plain, Size: 3249 bytes --]
Index: tree-pass.h
===================================================================
--- tree-pass.h (revision 126912)
+++ tree-pass.h (working copy)
@@ -331,6 +331,7 @@ extern struct tree_opt_pass pass_ipa_ref
extern struct tree_opt_pass pass_ipa_pure_const;
extern struct tree_opt_pass pass_ipa_type_escape;
extern struct tree_opt_pass pass_ipa_pta;
+extern struct tree_opt_pass pass_ipa_struct_reorg;
extern struct tree_opt_pass pass_early_local_passes;
extern struct tree_opt_pass pass_ipa_increase_alignment;
extern struct tree_opt_pass pass_ipa_function_and_variable_visibility;
Index: common.opt
===================================================================
--- common.opt (revision 126912)
+++ common.opt (working copy)
@@ -601,6 +601,11 @@ Common Report Var(flag_ipa_matrix_reorg)
Perform matrix layout flattening and transposing based
on profiling information.
+fipa-struct-reorg
+Common Report Var(flag_ipa_struct_reorg)
+Perform structure layout optimizations based
+on profiling information.
+
fivopts
Common Report Var(flag_ivopts) Init(1) Optimization
Optimize induction variables on trees
Index: Makefile.in
===================================================================
--- Makefile.in (revision 126912)
+++ Makefile.in (working copy)
@@ -1211,6 +1211,7 @@ OBJS-archive = \
ipa-prop.o \
ipa-pure-const.o \
ipa-reference.o \
+ ipa-struct-reorg.o \
ipa-type-escape.o \
ipa-utils.o \
ipa.o \
@@ -2480,6 +2481,10 @@ ipa-type-escape.o : ipa-type-escape.c $(
pointer-set.h $(GGC_H) $(IPA_TYPE_ESCAPE_H) $(IPA_UTILS_H) $(C_COMMON_H) \
$(TREE_GIMPLE_H) $(CGRAPH_H) output.h $(FLAGS_H) tree-pass.h \
$(DIAGNOSTIC_H) $(FUNCTION_H)
+ipa-struct-reorg.o: ipa-struct-reorg.c ipa-struct-reorg.h $(CONFIG_H) $(SYSTEM_H) \
+ coretypes.h $(TM_H) $(RTL_H) $(TREE_H) $(FLAGS_H) output.h $(REGS_H) \
+ $(EXPR_H) $(FUNCTION_H) toplev.h $(GGC_H) $(TARGET_H) langhooks.h $(COVERAGE_H) \
+ libfuncs.h gt-coverage.h $(HASHTAB_H) $(IPA_TYPE_ESCAPE_H)
coverage.o : coverage.c $(GCOV_IO_H) $(CONFIG_H) $(SYSTEM_H) coretypes.h \
$(TM_H) $(RTL_H) $(TREE_H) $(FLAGS_H) output.h $(REGS_H) $(EXPR_H) \
@@ -3044,7 +3049,7 @@ GTFILES = $(srcdir)/input.h $(srcdir)/co
$(srcdir)/reload.h $(srcdir)/caller-save.c \
$(srcdir)/alias.c $(srcdir)/bitmap.c $(srcdir)/cselib.c $(srcdir)/cgraph.c \
$(srcdir)/ipa-prop.c $(srcdir)/ipa-cp.c $(srcdir)/ipa-inline.c $(srcdir)/matrix-reorg.c \
- $(srcdir)/dbxout.c $(srcdir)/dwarf2out.c $(srcdir)/dwarf2asm.c \
+ $(srcdir)/dbxout.c $(srcdir)/ipa-struct-reorg.c $(srcdir)/dwarf2out.c $(srcdir)/dwarf2asm.c \
$(srcdir)/dojump.c \
$(srcdir)/emit-rtl.c $(srcdir)/except.c $(srcdir)/explow.c $(srcdir)/expr.c \
$(srcdir)/function.c $(srcdir)/except.h \
Index: passes.c
===================================================================
--- passes.c (revision 126912)
+++ passes.c (working copy)
@@ -542,6 +542,7 @@ init_optimization_passes (void)
NEXT_PASS (pass_ipa_pure_const);
NEXT_PASS (pass_ipa_type_escape);
NEXT_PASS (pass_ipa_pta);
+ NEXT_PASS (pass_ipa_struct_reorg);
*p = NULL;
/* These passes are run after IPA passes on every function that is being
[-- Attachment #3: ipa-struct-reorg.c --]
[-- Type: application/octet-stream, Size: 95750 bytes --]
/* Struct-reorg optimization.
Copyright (C) 2007 Free Software Foundation, Inc.
Contributed by Olga Golovanevsky <olga@il.ibm.com>
(initial version of this code was developed
by Caroline Tice and Mostafa Hagog)
This file is part of GCC.
GCC is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free
Software Foundation; either version 2, or (at your option) any later
version.
GCC is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
for more details.
You should have received a copy of the GNU General Public License
along with GCC; see the file COPYING. If not, write to the Free
Software Foundation, 59 Temple Place - Suite 330, Boston, MA
02111-1307, USA. */
#include "config.h"
#include "system.h"
#include "coretypes.h"
#include "tm.h"
#include "tree.h"
#include "rtl.h"
#include "tree-gimple.h"
#include "tree-inline.h"
#include "tree-flow.h"
#include "tree-flow-inline.h"
#include "langhooks.h"
#include "pointer-set.h"
#include "hashtab.h"
#include "c-tree.h"
#include "toplev.h"
#include "flags.h"
#include "ggc.h"
#include "debug.h"
#include "target.h"
#include "cgraph.h"
#include "diagnostic.h"
#include "timevar.h"
#include "params.h"
#include "fibheap.h"
#include "intl.h"
#include "function.h"
#include "basic-block.h"
#include "tree-iterator.h"
#include "tree-pass.h"
#include "struct-reorg.h"
#include "opts.h"
#include "ipa-type-escape.h"
#include "tree-dump.h"
#include "c-common.h"
/* This optimization implements structure peeling.
For example, given a structure type:
typedef struct
{
int a;
float b;
int c;
}str_t;
it can be peeled into two structure types as follows:
typedef struct and typedef struct
{ {
int a; float b;
int c; } str_t_1;
}str_t_0;
or can be fully peeled:
typedef struct
{
int a;
}str_t_0;
typedef struct
{
float b;
}str_t_1;
typedef struct
{
int c;
}str_t_2;
When structure type is peeled all instances and their accesses
in the program are updated accordingly. For example, if there is
array of structures:
str_t A[N];
and structure type str_t was peeled into two structures str_t_0
and str_t_1 as it was shown above, then array A will be replaced
by two arrays as foolows:
str_t_0 A_0[N];
str_t_1 A_1[N];
The field access of fields a of ellement i of array A: A[i].a will be
replace by access to field a of ellement i of array A_0: A_0[i].a.
This optimization also supports dynamically allocated arrays.
If array of structures was allocated by malloc function:
str_t * p = (str_t *) malloc (sizeof (str_t) * N)
the allocation site will be replaced to reflect new structure types:
str_t_0 * p_0 = (str_t_0 *) malloc (sizeof (str_t_0) * N)
str_t_1 * p_1 = (str_t_1 *) malloc (sizeof (str_t_1) * N)
The field access through the pointer p[i].a will be changed by p_0[i].a.
The decision how to peel structure gains to utilize spatial locality.
For example, if one of fields of structure has intensive use in the loop:
for (i = 0; i < N; i++)
{
... = A[i].a;
}
the allocation of field a of str_t in the memory in contiguous manner will
increase probablity of relevant data to be fatched into cache.
The analysis part of this optimization is based on the frequency of
field accesses, which are collected all over the program.
Then the fields with the frequencies that satisfy the following condition
got peeled out of the structure:
freq(f) > C * max_field_freq_in_struct
where max_field_freq_in_struct is maximum field frequency in structure.
The C is a constant defining which portion of max_field_freq_in_struct
the filds should have in order to be peeled.
If profiling information is provided, its is used to calculate the
frequency of field accesses. Otherwise, structure is fully peeled.
The ipa-type-escape analysis are used to provide safety of the peeling.
The optimization is activated by flag -fipa-struct-reorg.
*/
/* Ratio of the structure count to the hottest
structure count to consider the structure cold. */
#define COLD_STRUCTURE_RATIO 10
/* Data about the new program variables created (for the new peeled types).
When doing struct peeling, each variable of the original struct type will
need to be replaced with a set of new variables, one new variable for each
new type created from the original type. */
struct new_var_data {
tree orig_var; /* Var decl for original struct type */
struct struct_tree_list *new_vars; /* List of new vars to replace the
original var; one new var for each
new type peeled off the original
type. */
struct new_var_data *next;
};
typedef struct new_var_data *new_var;
/* This structure represents allocation site of structure. */
struct malloc_call
{
tree malloc_call_stmt;
d_str str;
/* Next malloc call stmt in the same function. */
struct malloc_call *next;
};
/* List of allocation sites that belong to the same function. */
struct malloc_struct
{
tree func;
/* List of allocation sites for func. */
struct malloc_call *malloc_list;
struct malloc_struct *next;
};
typedef struct malloc_struct *malloc_d;
typedef struct malloc_call *malloc_call;
/* List of all allocation sites in the program. */
malloc_d malloc_data_list = NULL;
/* List of new global variables. Generated once for whole program. */
new_var new_global_vars = NULL;
/* List of new local variables. Generated once for function. */
new_var new_local_vars = NULL;
/* Use stmts of current function. */
tree_list use_stmts = NULL;
/* List of structures to be transformed. */
struct struct_list
{
struct data_structure *struct_data;
struct struct_list *next;
};
typedef struct struct_list *list;
/* List of structs to be transformed. */
list data_struct_list = NULL;
static new_var is_in_new_vars_list (tree, new_var);
static void add_struct_to_data_struct_list (tree);
static void add_call_to_malloc_list (tree, tree, d_str);
static inline void update_fields_mapping (struct field_cluster *, tree,
struct data_field_entry *, int);
static void create_new_var (tree, new_var *);
static tree get_final_malloc_stmt (tree malloc_stmt);
static tree_list add_tree_to_tree_list (tree, tree_list);
static void free_tree_list (tree_list);
static malloc_d get_malloc_list_for_func (tree);
static struct field_access_site *
add_field_acc_to_acc_sites (struct field_access_site *,
struct field_access_site *);
static struct access_site * add_access_to_acc_sites (tree, tree,
struct access_site *);
static bool is_result_of_mult (tree, tree *, tree);
static void remove_struct_from_data_struct_list (list);
static tree gen_size (tree, tree, tree *);
static tree gen_cast_stmt (tree, tree, tree, tree *);
static void create_new_stmts_for_general_acc (struct access_site *, d_str);
static void create_new_stmts_for_cond_expr (tree);
static bool is_equal_types (tree, tree);
static tree * find_pos_in_stmt (tree, tree);
static void free_data_structure_list (void);
/* Strip structure type TYPE from pointers
and arrays. */
static inline tree
strip_type (tree type)
{
gcc_assert (TYPE_P (type));
while (POINTER_TYPE_P (type)
|| TREE_CODE (type) == ARRAY_TYPE)
type = TREE_TYPE (type);
return type;
}
/* Returns type of VAR. */
static inline tree
get_type_of_var (tree var)
{
if (!var)
return NULL;
if (TREE_CODE (var) == PARM_DECL)
return DECL_ARG_TYPE (var);
else
return TREE_TYPE (var);
}
/* Check whether the type of VAR is potential candidate for peeling.
Returns true if yes, false otherwise. If yes, TYPE_P contains
candidate type. */
static bool
is_candidate_for_data_struct_list (tree var, tree *type_p,
bitmap non_suitable_types)
{
tree type;
bool initialized = false;
*type_p = NULL;
if (!var)
return false;
/* There is no support of initialized vars. */
if (TREE_CODE (var) == VAR_DECL
&& DECL_INITIAL (var) != NULL_TREE)
initialized = true;
type = get_type_of_var (var);
if (type)
{
type = TYPE_MAIN_VARIANT (strip_type (type));
if (initialized && non_suitable_types)
bitmap_set_bit (non_suitable_types, TYPE_UID (type));
{
*type_p = type;
return (TREE_CODE (type) == RECORD_TYPE);
}
}
else
return false;
}
/* Given a tree representing a type, this function returns the
name of the type, as a string. */
static const char *
get_type_name (tree type_node)
{
if (! TYPE_NAME (type_node))
return NULL;
if (TREE_CODE (TYPE_NAME (type_node)) == IDENTIFIER_NODE)
return (char *) IDENTIFIER_POINTER (TYPE_NAME (type_node));
else if (TREE_CODE (TYPE_NAME (type_node)) == TYPE_DECL
&& DECL_NAME (TYPE_NAME (type_node)))
return (char *) IDENTIFIER_POINTER (DECL_NAME (TYPE_NAME (type_node)));
else
return NULL;
}
/* This function prints NEW_VARS into dump_file. */
static void
print_new_vars (new_var new_vars)
{
new_var current;
tree_list cur_var;
tree var_type;
if (!dump_file)
return;
for (current = new_vars; current; current = current->next)
{
var_type = get_type_of_var (current->orig_var);
fprintf (dump_file, "\nOrig var: ");
print_generic_expr (dump_file, current->orig_var, 0);
fprintf (dump_file, " of type ");
print_generic_expr (dump_file, var_type, 0);
fprintf (dump_file, "\n");
for (cur_var = current->new_vars; cur_var; cur_var = cur_var->next)
{
tree var = cur_var->data;
var_type = get_type_of_var (var);
fprintf (dump_file, " ");
print_generic_expr (dump_file, var, 0);
fprintf (dump_file, " of type ");
print_generic_expr (dump_file, var_type, 0);
fprintf (dump_file, "\n");
}
}
}
/* Print structure type , its name, if it exists, and body. */
static void
print_struct_type (tree type, char *indent)
{
char *new_indent;
const char *struct_name;
tree cur_field;
int len;
int i;
if (!type || !dump_file)
return;
if (TREE_CODE (type) != RECORD_TYPE)
return;
struct_name = get_type_name (type);
if (struct_name)
{
fprintf (dump_file, "%s%s {\n", indent, struct_name);
len = strlen (struct_name) + strlen (indent) + 3;
}
else
{
fprintf (dump_file, "%s{\n", indent);
len = strlen (indent) + 2;
}
new_indent = (char *) xmalloc (len * sizeof (char));
memset (new_indent, ' ', len);
new_indent[len] = '\0';
for (cur_field = TYPE_FIELDS (type); cur_field;
cur_field = TREE_CHAIN (cur_field))
{
tree field_type;
int ptr_count = 0;
field_type = TREE_TYPE (cur_field);
while (POINTER_TYPE_P (field_type))
{
ptr_count++;
field_type = TREE_TYPE (field_type);
}
fprintf (dump_file, "%s%s ", new_indent, get_type_name (field_type));
for (i = 0; i < ptr_count; i++)
fprintf (dump_file, "*");
fprintf (dump_file, " %s;\n",
IDENTIFIER_POINTER (DECL_NAME (cur_field)));
}
fprintf (dump_file, "%s}\n", indent);
}
/* Print out the list of new types generated by this optimization. */
static void
print_new_types (void)
{
list tmp;
char indent[3] = " ";
if (!dump_file)
return;
fprintf (dump_file,
"\nThe following are the new types generated by"
" this optimization:\n");
for (tmp = data_struct_list; tmp; tmp = tmp->next)
{
d_str str = tmp->struct_data;
tree_list new_type;
for (new_type = str->new_types; new_type;
new_type = new_type->next)
print_struct_type (new_type->data, indent);
}
}
/* Given structure type SRT_TYPE and field FIELD,
this funciton is looking for a field with the same name
and type as FIELD in STR_TYPE. It returns it if found,
or NULL_TREE otherwise. */
static tree
find_field_in_struct_1 (tree str_type, tree field)
{
tree str_field;
for (str_field = TYPE_FIELDS (str_type); str_field;
str_field = TREE_CHAIN (str_field))
{
char * str_field_name;
char * field_name;
str_field_name = (char *) IDENTIFIER_POINTER (DECL_NAME (str_field));
field_name = (char *) IDENTIFIER_POINTER (DECL_NAME (field));
gcc_assert (str_field_name);
gcc_assert (field_name);
if (!strcmp (str_field_name, field_name))
{
/* Check their types. */
if (is_equal_types (TREE_TYPE (str_field), TREE_TYPE (field)))
return str_field;
}
}
return NULL_TREE;
}
/* Given the field declaration tree FIELD_DECL, this function
returns the appropriate field entry in structure STR. */
static struct data_field_entry *
find_field_in_struct (d_str str, tree field_decl)
{
int i;
tree field = find_field_in_struct_1 (str->decl, field_decl);
for (i = 0; i < str->num_fields; i++)
if (str->fields[i].decl == field)
return &(str->fields[i]);
return NULL;
}
/* This function is a hack to overcome the types problem.
When number of compilation units are compiled together
under -combine flag, the TYPE_MAIN_VARIANT cannot
be trusted. First we comparing names, but they do not
always appear, like for example, in array_refs.
Only then we are going inside structures.
*/
static bool
is_equal_types (tree type1, tree type2)
{
const char * name1,* name2;
if ((!type1 && type2)
||(!type2 && type1))
return false;
if (!type1 && !type2)
return true;
if (TREE_CODE (type1) != TREE_CODE (type2))
return false;
if (type1 == type2)
return true;
if (TYPE_MAIN_VARIANT (type1) == TYPE_MAIN_VARIANT (type2))
return true;
name1 = get_type_name (type1);
name2 = get_type_name (type2);
if (name1 && name2 && !strcmp (name1, name2))
return true;
if (name1 && name2 && strcmp (name1, name2))
return false;
switch (TREE_CODE (type1))
{
case POINTER_TYPE:
case REFERENCE_TYPE:
{
return is_equal_types (TREE_TYPE (type1), TREE_TYPE (type2));
}
break;
case RECORD_TYPE:
case UNION_TYPE:
case QUAL_UNION_TYPE:
case ENUMERAL_TYPE:
{
tree field1;
/* Compare fields of struture. */
for (field1 = TYPE_FIELDS (type1); field1;
field1 = TREE_CHAIN (field1))
{
tree field2 = find_field_in_struct_1 (type2, field1);
if (!field2)
return false;
}
return true;
}
break;
case INTEGER_TYPE:
{
if (TYPE_UNSIGNED (type1) == TYPE_UNSIGNED (type2)
&& TYPE_PRECISION (type1) == TYPE_PRECISION (type2))
return true;
}
break;
case ARRAY_TYPE:
{
tree d1, d2;
tree max1, min1, max2, min2;
if (!is_equal_types (TREE_TYPE (type1), TREE_TYPE (type2)))
return false;
d1 = TYPE_DOMAIN (type1);
d2 = TYPE_DOMAIN (type2);
if (!d1 || !d2)
return false;
max1 = TYPE_MAX_VALUE (d1);
max2 = TYPE_MAX_VALUE (d2);
min1 = TYPE_MIN_VALUE (d1);
min2 = TYPE_MIN_VALUE (d2);
if (max1 && max2 && min1 && min2
&& TREE_CODE (max1) == TREE_CODE (max2)
&& TREE_CODE (max1) == INTEGER_CST
&& TREE_CODE (min1) == TREE_CODE (min2)
&& TREE_CODE (min1) == INTEGER_CST
&& tree_int_cst_equal (max1, max2)
&& tree_int_cst_equal (min1, min2))
return true;
}
break;
default:
gcc_unreachable();
}
return false;
}
/* This function returns data_structure entry
in the data_struct_list represented by type TYPE, if found. */
static list
find_struct_in_list (tree type)
{
list tmp = data_struct_list;
while (tmp)
{
if (is_equal_types (tmp->struct_data->decl, TYPE_MAIN_VARIANT (type)))
return tmp;
tmp = tmp->next;
}
return NULL;
}
/* This function returns true if access ACC represents pattern
generated by compiler when address of element i of array
of structures (pointed by p) is calculated (p[i]). If this
pattern is recognized correctly, this function returns true
and fills missing fields in ACC. Othewise returns false. */
static bool
decompose_indirect_ref_acc (tree str_decl, struct field_access_site *acc)
{
tree ref_var;
tree rhs, struct_size, op0, op1;
tree before_cast;
ref_var = TREE_OPERAND (acc->ref, 0);
if (TREE_CODE (ref_var) != SSA_NAME)
return false;
acc->ref_def_stmt = SSA_NAME_DEF_STMT (ref_var);
if (!(acc->ref_def_stmt)
|| (TREE_CODE (acc->ref_def_stmt) != GIMPLE_MODIFY_STMT))
return false;
rhs = GIMPLE_STMT_OPERAND (acc->ref_def_stmt, 1);
if (TREE_CODE (rhs) != PLUS_EXPR
&& TREE_CODE (rhs)!= MINUS_EXPR
&& TREE_CODE (rhs) != POINTER_PLUS_EXPR)
return false;
op0 = TREE_OPERAND (rhs, 0);
op1 = TREE_OPERAND (rhs, 1);
if (!is_array_access_through_pointer_and_index (TREE_CODE (rhs), op0, op1,
&acc->base, &acc->offset,
&acc->cast_stmt))
return false;
if (acc->cast_stmt)
before_cast = SINGLE_SSA_TREE_OPERAND (acc->cast_stmt, SSA_OP_USE);
else
before_cast = acc->offset;
if (!before_cast)
return false;
if (SSA_NAME_IS_DEFAULT_DEF (before_cast))
return false;
struct_size = TYPE_SIZE_UNIT (str_decl);
if (!is_result_of_mult (before_cast, &acc->num, struct_size))
return false;
return true;
}
/* This function checks whether the access ACC is of the form suitable
for tranformation. If yes, it returns true. False otherwise. */
static bool
decompose_access (tree str_decl, struct field_access_site *acc)
{
gcc_assert (acc->ref);
if (TREE_CODE (acc->ref) == INDIRECT_REF)
return decompose_indirect_ref_acc (str_decl, acc);
else if (TREE_CODE (acc->ref) == ARRAY_REF)
return true;
else if (TREE_CODE (acc->ref) == VAR_DECL)
return true;
return false;
}
/* Create empty field_access_site node. */
static inline struct field_access_site *
make_field_acc_node (void)
{
return (struct field_access_site *)
xcalloc (1, sizeof (struct field_access_site));
}
/* Helper for walk_tree called from collect_accesses_in_bb function. */
static tree
get_stmt_accesses (tree *tp, int *walk_subtrees, void *data)
{
tree stmt = (tree) data;
tree t = *tp;
if (!t)
return NULL;
switch (TREE_CODE (t))
{
case GIMPLE_MODIFY_STMT:
{
tree lhs = GIMPLE_STMT_OPERAND (t, 0);
tree rhs = GIMPLE_STMT_OPERAND (t, 1);
*walk_subtrees=1;
walk_tree (&lhs, get_stmt_accesses, data, NULL);
walk_tree (&rhs, get_stmt_accesses, data, NULL);
*walk_subtrees=0;
}
break;
case BIT_FIELD_REF:
{
tree str = TREE_OPERAND(t, 0);
tree type = TYPE_MAIN_VARIANT (strip_type (get_type_of_var (str)));
list tmp = find_struct_in_list (type);
if (tmp)
remove_struct_from_data_struct_list (tmp);
}
break;
case COMPONENT_REF:
{
tree ref = TREE_OPERAND (t, 0);
tree field_decl = TREE_OPERAND (t, 1);
if ((TREE_CODE (ref) == INDIRECT_REF
|| TREE_CODE (ref) == ARRAY_REF
|| TREE_CODE (ref) == VAR_DECL)
&& TREE_CODE (field_decl) == FIELD_DECL)
{
tree type = TYPE_MAIN_VARIANT (TREE_TYPE (ref));
list tmp = find_struct_in_list (type);
if (tmp)
{
d_str str = tmp->struct_data;
struct data_field_entry * field =
find_field_in_struct (str, field_decl);
if (field)
{
struct field_access_site *acc = make_field_acc_node ();
gcc_assert (acc);
acc->stmt = stmt;
acc->comp_ref = t;
acc->ref = ref;
acc->field_decl = field_decl;
/* Check whether the access is of the form we can deal with. */
if (!decompose_access (str->decl, acc))
{
remove_struct_from_data_struct_list (tmp);
free (acc);
}
else
{
/* Increase count of field. */
basic_block bb = bb_for_stmt (stmt);
field->count += bb->count;
/* Add stmt to the acc_sites list of field. */
field->acc_sites =
add_field_acc_to_acc_sites (acc, field->acc_sites);
}
*walk_subtrees = 0;
}
}
}
}
break;
case MINUS_EXPR:
case PLUS_EXPR:
{
tree op0 = TREE_OPERAND (t, 0);
tree op1 = TREE_OPERAND (t, 1);
*walk_subtrees=1;
walk_tree (&op0, get_stmt_accesses, data, NULL);
walk_tree (&op1, get_stmt_accesses, data, NULL);
*walk_subtrees=0;
}
break;
case COND_EXPR:
{
tree cond = COND_EXPR_COND (t);
int i;
for (i = 0; i < TREE_CODE_LENGTH (TREE_CODE (cond)); i++)
{
tree t = TREE_OPERAND (cond, i);
*walk_subtrees=1;
walk_tree (&t, get_stmt_accesses, data, NULL);
}
*walk_subtrees=0;
}
break;
case VAR_DECL:
case SSA_NAME:
{
list tmp;
if (TREE_CODE (t) == SSA_NAME)
t = SSA_NAME_VAR (t);
tmp = find_struct_in_list (strip_type (get_type_of_var (t)));
if (tmp)
{
d_str str = tmp->struct_data;
str->accs =
add_access_to_acc_sites (stmt, t, str->accs);
}
*walk_subtrees=0;
}
break;
case CALL_EXPR:
{
/* It was checked as part of stage1 that structs
to be transformed cannot be passed as parameters to function. */
*walk_subtrees=0;
}
break;
default:
return NULL;
}
return NULL;
}
/* Collect accesses to the struct fields in BB. */
static void
collect_accesses_in_bb (basic_block bb)
{
block_stmt_iterator bsi;
/* Go over the basic block statements collecting
accesses to fields of structures. */
for (bsi = bsi_start (bb); ! bsi_end_p (bsi); bsi_next (&bsi))
{
tree stmt = bsi_stmt (bsi);
/* In asm stmt we cannot always track the agruments,
so we just give up. */
if (TREE_CODE (stmt) == ASM_EXPR)
free_data_structure_list ();
walk_tree (&stmt, get_stmt_accesses, stmt, NULL);
}
}
/* Collect accesses of the fields of the structures
in function FN. */
static void
collect_accesses_in_func (struct function *fn)
{
basic_block bb;
if (! fn)
return;
/* Collect accesses in each one of the basic blocks. */
FOR_EACH_BB_FN (bb, fn)
collect_accesses_in_bb (bb);
}
/* This functions builds structure with fields FIELDS,
that should be TREE_CHAIN, named NAME, with the same
attributes as ORIG_STRUCT. */
static tree
build_basic_struct (tree fields, char * name, tree orig_struct)
{
tree attributes = NULL_TREE;
tree ref = 0;
tree x;
if (TYPE_ATTRIBUTES (orig_struct))
attributes = copy_node (TYPE_ATTRIBUTES (orig_struct));
ref = make_node (RECORD_TYPE);
TYPE_SIZE (ref) = 0;
decl_attributes (&ref, attributes, (int) ATTR_FLAG_TYPE_IN_PLACE);
TYPE_PACKED (ref) = TYPE_PACKED (orig_struct);
for (x = fields; x; x = TREE_CHAIN (x))
{
DECL_CONTEXT (x) = ref;
DECL_PACKED (x) |= TYPE_PACKED (ref);
}
TYPE_FIELDS (ref) = fields;
layout_type (ref);
TYPE_NAME (ref) = get_identifier (name);
return ref;
}
/* Generate new structure name. */
static inline char *
gen_cluster_name (tree decl, int clust_num, int str_num)
{
char * old_name = (char *) get_type_name (decl);
char * new_name;
if (!old_name)
{
old_name = (char *) xmalloc ((10) * sizeof (char));
sprintf (old_name, "struct_%d", str_num);
}
new_name = (char *) xmalloc ((strlen (old_name) + 11) * sizeof (char));
sprintf (new_name, "%s_sub_%d", old_name, clust_num);
/* Check that this name was not used before. */
while (maybe_get_identifier (new_name))
{
int len = strlen (new_name) + 3;
char * tmp_name = (char *) xmalloc (len * sizeof (char));
sprintf (tmp_name, "%s.0", new_name);
new_name = tmp_name;
}
return new_name;
}
/* This function copies fields from CLUSTER into TREE_CHAIN as part
of preparation for new structure building. */
static tree
create_fields (struct field_cluster * cluster,
struct data_field_entry * fields, int num_fields)
{
int i;
tree new_types = NULL_TREE;
tree last = NULL_TREE;
for (i = 0; i < num_fields; i++)
if (TEST_BIT (cluster->fields_in_cluster, i))
{
tree new_decl = copy_node (fields[i].decl);
if (!new_types)
new_types = new_decl;
else
TREE_CHAIN (last) = new_decl;
last = new_decl;
}
TREE_CHAIN (last) = NULL_TREE;
return new_types;
}
/* Update field_mapping of fields in CLUSTER with NEW_TYPE. */
static inline void
update_fields_mapping (struct field_cluster *cluster, tree new_type,
struct data_field_entry * fields, int num_fields)
{
int i;
for (i = 0; i < num_fields; i++)
if (TEST_BIT (cluster->fields_in_cluster, i))
fields[i].field_mapping = new_type;
}
/* This function creates new types according to
struct clustering data. */
static void
create_new_types (void)
{
list tmp = data_struct_list;
int str_num = 0;
while (tmp)
{
int cluster_num = 0;
d_str str = tmp->struct_data;
struct field_cluster * cluster = str->struct_clustering;
while (cluster)
{
char * name = gen_cluster_name (str->decl, cluster_num, str_num);
tree fields;
tree new_type;
cluster_num++;
fields = create_fields (cluster, str->fields, str->num_fields);
new_type = build_basic_struct (fields, name, str->decl);
update_fields_mapping (cluster, new_type, str->fields, str->num_fields);
str->new_types = add_tree_to_tree_list (new_type, str->new_types);
cluster = cluster->sibling;
}
str_num++;
tmp = tmp->next;
}
}
/* This function adds NEW_DECL to function
referenced vars list, and mark its for renaming. */
static void
finish_var_creation (tree new_decl)
{
add_referenced_var (new_decl);
if (is_global_var (new_decl))
mark_call_clobbered (new_decl, ESCAPE_UNKNOWN);
mark_sym_for_renaming (new_decl);
}
/* This function finish VAR creation in
case of global VAR_DECL. */
static void
finish_global_creation (tree var)
{
if (TREE_CODE (var) == VAR_DECL
&& is_global_var (var))
finish_var_creation (var);
}
/* This function generates new structure name as based on
ORIG_DECL name, combined with index I, and returns
generated identifier in ID_NODE. */
static char *
gen_var_name (tree orig_decl, int i, tree *id_node)
{
char * new_name = NULL;
/* If the original variable decl has a name, create an
appropriate new name for the new decl. */
if (DECL_NAME (orig_decl)
&& IDENTIFIER_POINTER (DECL_NAME (orig_decl)))
{
char *old_name;
int counter = 0;
int len, new_len;
char *tmp_name;
old_name = (char *) IDENTIFIER_POINTER (DECL_NAME (orig_decl));
len = strlen (old_name) + 6;
new_name = (char *) xmalloc (len * sizeof (char));
sprintf (new_name, "%s_%d", old_name, i);
/* Make sure there isn't anything else that already has that
name. */
new_len = strlen (new_name) + 5;
tmp_name = (char *) xmalloc (new_len * sizeof (char));
sprintf (tmp_name, "%s", new_name);
while (maybe_get_identifier (tmp_name))
sprintf (tmp_name, "%s.%d", new_name, counter++);
new_name = tmp_name;
*id_node = get_identifier (new_name);
}
else
*id_node = NULL;
return new_name;
}
/* This function returns true if struct field access
that appears in stmt STMT is field accesses list LIST_P. */
static struct field_access_site *
is_in_field_acc_list (tree stmt, struct field_access_site *list_p)
{
struct field_access_site *curr;
for (curr = list_p; curr; curr = curr->next)
if (curr->stmt == stmt)
return curr;
return NULL;
}
/* This function adds ACC to field accesses list LIST_P. */
static struct field_access_site *
add_field_acc_to_acc_sites (struct field_access_site *acc,
struct field_access_site *list_p)
{
gcc_assert (!is_in_field_acc_list (acc->stmt, list_p));
acc->next = list_p;
return acc;
}
/* This function returns true if struct non-field access
that appears in stmt STMT is non-field accesses list LIST_P. */
static struct access_site *
is_in_acc_list (tree stmt, struct access_site *list_p)
{
struct access_site *curr;
for (curr = list_p; curr; curr = curr->next)
if (curr->stmt == stmt)
return curr;
return NULL;
}
/* This function adds the VAR to access vars list.
If access entry with stmt STMT does not exist yet in LIST_P,
it creates it. */
static struct access_site *
add_access_to_acc_sites (tree stmt, tree var, struct access_site *list_p)
{
struct access_site *acc;
acc = is_in_acc_list (stmt, list_p);
if (acc)
{
acc->vars = add_tree_to_tree_list (var, acc->vars);
return list_p;
}
else
{
acc = (struct access_site *) xmalloc (sizeof (struct access_site));
acc->stmt = stmt;
acc->next = list_p;
acc->vars = NULL;
acc->vars = add_tree_to_tree_list (var, acc->vars);
return acc;
}
}
/* Stuff the new tree DATA into a tree_list node,
and append it to the front of the LIST_P. */
static tree_list
add_tree_to_tree_list (tree data, tree_list list_p)
{
tree_list tmp_node;
tmp_node = (tree_list) xmalloc (sizeof (struct struct_tree_list));
tmp_node->data = data;
tmp_node->next = list_p;
return tmp_node;
}
/* This function inserts NEW_DECL to varpool. */
static inline void
insert_global_to_varpool (tree new_decl)
{
struct varpool_node *new_node;
new_node = varpool_node (new_decl);
notice_global_symbol (new_decl);
varpool_mark_needed_node (new_node);
varpool_finalize_decl (new_decl);
}
/* This function copy attributes form ORIG_DECL to NEW_DECL. */
static inline void
copy_decl_attributes (tree new_decl, tree orig_decl)
{
DECL_ARTIFICIAL (new_decl) = 1;
DECL_EXTERNAL (new_decl) = DECL_EXTERNAL (orig_decl);
TREE_STATIC (new_decl) = TREE_STATIC (orig_decl);
TREE_PUBLIC (new_decl) = TREE_PUBLIC (orig_decl);
TREE_USED (new_decl) = TREE_USED (orig_decl);
DECL_CONTEXT (new_decl) = DECL_CONTEXT (orig_decl);
TREE_THIS_VOLATILE (new_decl) = TREE_THIS_VOLATILE (orig_decl);
TREE_ADDRESSABLE (new_decl) = TREE_ADDRESSABLE (orig_decl);
if (TREE_CODE (orig_decl) == VAR_DECL)
{
TREE_READONLY (new_decl) = TREE_READONLY (orig_decl);
DECL_TLS_MODEL (new_decl) = DECL_TLS_MODEL (orig_decl);
}
}
/* This structure is used to represent array
or pointer-to wrappers of structure type.
For example, if type1 is structure type,
then for type1 ** we generate two type_wrapper
structs with wrap=0 each one.
It's used to unwind the original type up to
struct type, replace it with the new struct type
and wrap it back in the opposite order. */
struct type_wrapper
{
/* 0 stand for pointer wrapper,
and 1 for array wrapper. */
bool wrap;
tree domain; /* Relevant for arrays as domain or index. */
struct type_wrapper * next;
};
/* This function creates new wrapper struct with WRAP and DOMAIN fields,
and places it at front of wrappers list, defined by WRAPPER.
It returns the begging of the list. */
static struct type_wrapper *
add_wrapper (struct type_wrapper *wrapper, bool wrap, tree domain)
{
struct type_wrapper * node = (struct type_wrapper *) xmalloc
(sizeof (struct type_wrapper));
node->wrap = wrap;
node->next = wrapper;
node->domain = domain;
return node;
}
/* Free list of wrappers, started from WRAP. */
static void
free_wrapper (struct type_wrapper * wrap)
{
struct type_wrapper *node, *node1;
node = wrap;
while (node)
{
node1 = node;
node = node->next;
free (node1);
}
}
/* This function generate the same level of pointers or arrays
in the NEW_STR_TYPE as it was in type of DECL.
It returns the generated type. */
static inline tree
gen_struct_type (tree decl, tree new_str_type)
{
tree type_orig = get_type_of_var (decl);
tree new_type = new_str_type;
struct type_wrapper * wrapper = NULL;
struct type_wrapper * wr = NULL;
while (POINTER_TYPE_P (type_orig)
|| TREE_CODE (type_orig) == ARRAY_TYPE)
{
if (POINTER_TYPE_P (type_orig))
wrapper = add_wrapper (wrapper, 0, NULL_TREE);
else if (TREE_CODE (type_orig) == ARRAY_TYPE)
wrapper = add_wrapper (wrapper, 1, TYPE_DOMAIN (type_orig));
type_orig = TREE_TYPE (type_orig);
}
for (wr = wrapper; wr; wr = wr->next)
{
if (wr->wrap) /* Array. */
new_type = build_array_type (new_type, wr->domain);
else /* Pointer. */
new_type = build_pointer_type (new_type);
}
free_wrapper (wrapper);
return new_type;
}
/* Given an old variable ORIG_DECL with VAR_DECL tree code,
this function generates new variables to replace it
according to the set of types decided before. */
static struct struct_tree_list *
create_new_var_1 (tree orig_decl, d_str str)
{
tree_list new_vars = NULL;
tree_list current;
int i;
for (current = str->new_types, i = 0; current;
current = current->next, i++)
{
tree new_type = current->data;
tree new_decl = NULL;
tree id_node;
char * new_name = NULL;
new_name = gen_var_name (orig_decl, i, &id_node);
new_type = gen_struct_type (orig_decl, new_type);
if (is_global_var (orig_decl))
new_decl = build_decl (VAR_DECL, id_node, new_type);
else
new_decl = create_tmp_var (new_type, new_name);
copy_decl_attributes (new_decl, orig_decl);
new_vars = add_tree_to_tree_list (new_decl, new_vars);
}
/* Return the list of new var decls. */
return new_vars;
}
/* Finish vars creation for vars in LIST_P. */
static void
finish_vars_creation (new_var list_p)
{
new_var node;
for (node = list_p; node; node = node->next)
{
tree_list curr;
for (curr = node->new_vars; curr; curr = curr->next)
finish_var_creation (curr->data);
}
}
/* For each SSA_NAME or VAR_DECL local variable of structure
type relevant for transformation in current function (cfun)
this function generate new variable(s) to replace it. */
static void
create_new_local_vars (void)
{
unsigned i;
tree var_list;
for (i = 1; i < num_ssa_names; i++)
{
tree ssa_name = ssa_name (i) ;
if (!ssa_name)
continue;
create_new_var (SSA_NAME_VAR (ssa_name), &new_local_vars);
}
/* Function local variables. */
for (var_list = cfun->unexpanded_var_list; var_list;
var_list = TREE_CHAIN (var_list))
{
tree var = TREE_VALUE (var_list);
create_new_var (var, &new_local_vars);
}
finish_vars_creation (new_local_vars);
print_new_vars (new_local_vars);
}
/* This function returns true it the result variable
of malloc call is a cast to one of the structure
types we are planning to transform.
STMT should contain malloc call: T.2 = malloc (T.1); */
static bool
is_malloc_of_struct (tree stmt, list * node_p)
{
tree lhs;
tree type;
tree final_stmt;
final_stmt = get_final_malloc_stmt (stmt);
if (!final_stmt)
return false;
/* final_stmt should be of the form:
T.3 = (struct_type *) T.2; */
if (TREE_CODE (final_stmt) != GIMPLE_MODIFY_STMT)
return false;
lhs = GIMPLE_STMT_OPERAND (final_stmt, 0);
type = get_type_of_var (lhs);
if (!type)
return false;
if (!POINTER_TYPE_P (type)
|| TREE_CODE (strip_type (type)) != RECORD_TYPE)
return false;
*node_p = find_struct_in_list (strip_type (type));
if (!(*node_p))
return false;
return true;
}
/* In this function we suppose that an allocation statement
var = (type_cast) malloc (size);
is converted into followinf set of stmts:
T.1 = size;
T.2 = malloc (T.1);
T.3 = (type_cast) T.2;
var = T.3;
In this function we collect into malloc_data_list the allocation
sites of variables of structure types that are present
in data_struct_list. */
static void
collect_malloc_data (void)
{
struct cgraph_node *node;
struct cgraph_edge *cs;
for (node = cgraph_nodes; node; node = node->next)
if (node->analyzed && node->decl)
{
for (cs = node->callees; cs; cs = cs->next_callee)
{
tree stmt = cs->call_stmt;
if (stmt)
{
tree call = get_call_expr_in (stmt);
tree decl;
if (call && (decl = get_callee_fndecl (call))
&& TREE_CODE (stmt) == GIMPLE_MODIFY_STMT)
{
list str_node = NULL;
if (is_malloc_of_struct (stmt, &str_node))
{
d_str str = str_node->struct_data;
/* We support only malloc now. */
if (DECL_FUNCTION_CODE (decl) == BUILT_IN_MALLOC)
add_call_to_malloc_list (node->decl, stmt, str);
else
remove_struct_from_data_struct_list (str_node);
}
}
}
}
}
}
/* Update statements in STMT_LIST with BB info. */
static void
add_bb_info (basic_block bb, tree stmt_list)
{
if (TREE_CODE (stmt_list) == STATEMENT_LIST)
{
tree_stmt_iterator tsi;
for (tsi = tsi_start (stmt_list); !tsi_end_p (tsi);
tsi_next (&tsi))
{
tree stmt = tsi_stmt (tsi);
set_bb_for_stmt (stmt, bb);
}
}
}
/* This function checks whether ARG is a result of multiplication
of some number by STRUCT_SIZE. If yes, the function returns true
and this number is substituted into NUM. */
static bool
is_result_of_mult (tree arg, tree *num, tree struct_size)
{
tree size_def_stmt = SSA_NAME_DEF_STMT (arg);
/* If malloc stmt was of the form D.2229_10 = malloc (D.2228_9);
then size_def_stmt can be D.2228_9 = num.3_8 * 8; */
if (size_def_stmt
&& TREE_CODE (size_def_stmt) == GIMPLE_MODIFY_STMT)
{
tree lhs = GIMPLE_STMT_OPERAND (size_def_stmt, 0);
tree rhs = GIMPLE_STMT_OPERAND (size_def_stmt, 1);
/* We expect temporary here. */
if (!is_gimple_reg (lhs))
return false;
if (TREE_CODE (rhs) == MULT_EXPR)
{
tree arg0 = TREE_OPERAND (rhs, 0);
tree arg1 = TREE_OPERAND (rhs, 1);
if (operand_equal_p (arg0, struct_size, OEP_ONLY_CONST))
{
*num = arg1;
return true;
}
if (operand_equal_p (arg1, struct_size, OEP_ONLY_CONST))
{
*num = arg0;
return true;
}
}
}
*num = NULL_TREE;
return false;
}
/* Set of actioons we do for each newly generated stmt. */
static inline void
finish_stmt (tree stmt)
{
update_stmt (stmt);
mark_symbols_for_renaming (stmt);
}
/* This function returns a tree representing
the number of structure instances allocated by MALLOC_STMT. */
static tree
gen_num_of_structs_in_malloc (tree stmt, tree str_decl,
tree *new_stmts_p)
{
call_expr_arg_iterator iter;
tree arg;
tree call_expr;
tree struct_size;
HOST_WIDE_INT struct_size_int;
if (!stmt)
return NULL_TREE;
/* Get malloc agrument. */
call_expr = get_call_expr_in (stmt);
if (!call_expr)
return NULL_TREE;
arg = first_call_expr_arg (call_expr, &iter);
if (TREE_CODE (arg) != SSA_NAME
&& !TREE_CONSTANT (arg))
return NULL_TREE;
struct_size = TYPE_SIZE_UNIT (str_decl);
struct_size_int = TREE_INT_CST_LOW (struct_size);
gcc_assert (struct_size);
if (TREE_CODE (arg) == SSA_NAME)
{
tree num, div_stmt;
if (is_result_of_mult (arg, &num, struct_size))
return num;
num = create_tmp_var (integer_type_node, NULL);
if (num)
add_referenced_var (num);
if (exact_log2 (struct_size_int) == -1)
div_stmt = build2 (GIMPLE_MODIFY_STMT, integer_type_node,
num, build2 (TRUNC_DIV_EXPR, integer_type_node,
arg, struct_size));
else
div_stmt = build2 (GIMPLE_MODIFY_STMT, integer_type_node,
num,
build2 (RSHIFT_EXPR,
integer_type_node,
arg, build_int_cst (integer_type_node,
exact_log2 (struct_size_int))));
*new_stmts_p = alloc_stmt_list ();
append_to_statement_list (div_stmt,
new_stmts_p);
finish_stmt (div_stmt);
return num;
}
if (CONSTANT_CLASS_P (arg)
&& multiple_of_p (TREE_TYPE (struct_size),
arg, struct_size))
return int_const_binop (TRUNC_DIV_EXPR, arg, struct_size, 0);
return NULL_TREE;
}
/* Append STMT to STMTS. */
static inline void
finish_stmt_and_append (tree *stmts, tree stmt)
{
append_to_statement_list (stmt, stmts);
finish_stmt (stmt);
}
/* This function is looking for a variable of
NEW_TYPE type stored in VAR. It returns it, if found,
and NULL_TREE otherwise. */
static tree
find_var_in_new_vars_list (new_var var, tree new_type)
{
tree_list curr;
for (curr = var->new_vars; curr; curr = curr->next)
{
tree type = strip_type(get_type_of_var (curr->data));
gcc_assert (type);
if (type == new_type)
return curr->data;
}
return NULL_TREE;
}
/* Given origrnal varaiable ORIG_VAR, this function returns
new variable correspondent to it of NEW_TYPE type. */
static tree
find_new_var_of_type (tree orig_var, tree new_type)
{
new_var var;
gcc_assert (orig_var && new_type);
if (TREE_CODE (orig_var) == SSA_NAME)
orig_var = SSA_NAME_VAR (orig_var);
var = is_in_new_vars_list (orig_var, new_global_vars);
if (!var)
var = is_in_new_vars_list (orig_var, new_local_vars);
gcc_assert (var);
return find_var_in_new_vars_list (var, new_type);
}
/* This function generates set of stmts required
to allocate number NUM of structures of type NEW_TYPE.
The stmts are stored in NEW_STMTS. The stmt that contain
call to malloc is returned. MALLOC_STMT is an original
call to malloc.*/
static tree
create_new_malloc (tree malloc_stmt, tree new_type,
tree *new_stmts, tree num)
{
tree new_malloc_size;
tree call_expr, malloc_fn_decl;
tree new_stmt, malloc_res;
tree call_stmt, final_stmt;
tree cast_res;
gcc_assert (num && malloc_stmt && new_type);
*new_stmts = alloc_stmt_list ();
/* Generate argument to malloc as multiplication of num and size of new_type. */
new_stmt = gen_size (num, new_type, &new_malloc_size);
append_to_statement_list (new_stmt, new_stmts);
/* Generate new call for malloc. */
malloc_res = create_tmp_var (integer_type_node, NULL);
if (malloc_res)
add_referenced_var (malloc_res);
call_expr = get_call_expr_in (malloc_stmt);
malloc_fn_decl = get_callee_fndecl (call_expr);
call_expr = build_call_expr (malloc_fn_decl, 1, new_malloc_size);
call_stmt = build2 (GIMPLE_MODIFY_STMT,
TREE_TYPE (TREE_TYPE (malloc_fn_decl)),
malloc_res,
call_expr);
finish_stmt_and_append (new_stmts, call_stmt);
/* Create new cast stmt. */
final_stmt = get_final_malloc_stmt (malloc_stmt);
gcc_assert (final_stmt);
new_stmt = gen_cast_stmt (malloc_res, new_type, final_stmt, &cast_res);
append_to_statement_list (new_stmt, new_stmts);
return call_stmt;
}
/* This function build an edge between BB and E->dest and updates
phi nodes of E->dest. It returns newly created edge. */
static edge
make_edge_and_fix_phis_of_dest (basic_block bb, edge e)
{
edge new_e;
tree phi, arg;
new_e = make_edge (bb, e->dest, e->flags);
for (phi = phi_nodes (new_e->dest); phi; phi = PHI_CHAIN (phi))
{
arg = PHI_ARG_DEF_FROM_EDGE (phi, e);
add_phi_arg (phi, arg, new_e);
}
return new_e;
}
/* Update call graph with new edge generated by
new MALLOC_STMT. */
static void
update_cgraph_with_malloc_call (tree malloc_stmt, tree context)
{
tree call_expr;
struct cgraph_node *src, *dest;
tree malloc_fn_decl;
if (!malloc_stmt)
return;
call_expr = get_call_expr_in (malloc_stmt);
malloc_fn_decl = get_callee_fndecl (call_expr);
src = cgraph_node (context);
dest = cgraph_node (malloc_fn_decl);
cgraph_create_edge (src, dest, malloc_stmt,
0, 0, bb_for_stmt (malloc_stmt)->loop_depth);
}
/* If MALLOC_STMT is D.2225_7 = malloc (D.2224_6);
it can be casting stmt as for example:
p_8 = (struct str_t *) D.2225_7;
which is returned by this function. */
static tree
get_final_malloc_stmt (tree malloc_stmt)
{
tree final_stmt;
use_operand_p use_p;
tree malloc_res;
if (!malloc_stmt)
return NULL;
if (TREE_CODE (malloc_stmt) != GIMPLE_MODIFY_STMT)
return NULL;
malloc_res = GIMPLE_STMT_OPERAND (malloc_stmt, 0);
if (TREE_CODE (malloc_res) != SSA_NAME)
return NULL;
if (!single_imm_use (malloc_res, &use_p, &final_stmt))
return NULL;
else
return final_stmt;
}
/* Insert NEW_STMTS after STMT. */
static void
insert_after_stmt (tree stmt, tree new_stmts)
{
block_stmt_iterator bsi;
if (!stmt || !new_stmts)
return;
bsi = bsi_for_stmt (stmt);
bsi_insert_after (&bsi, new_stmts, BSI_SAME_STMT);
}
/* Insert NEW_STMTS before STMT. */
static void
insert_before_stmt (tree stmt, tree new_stmts)
{
block_stmt_iterator bsi;
if (!stmt || !new_stmts)
return;
bsi = bsi_for_stmt (stmt);
bsi_insert_before (&bsi, new_stmts, BSI_SAME_STMT);
}
/* This function adds allocation sites for peeled structs.
M_DATA is list of mallocs to be replaced in function CONTEXT. */
static void
create_new_mallocs (malloc_d m_data, tree context)
{
malloc_call call;
for (call = m_data->malloc_list; call; call = call->next)
{
tree stmt = call->malloc_call_stmt;
d_str str = call->str;
tree_list n_type;
tree num;
tree new_stmts = NULL_TREE;
tree last_stmt = get_final_malloc_stmt (stmt);
num = gen_num_of_structs_in_malloc (stmt, str->decl, &new_stmts);
if (new_stmts)
{
last_stmt = tsi_stmt (tsi_last (new_stmts));
insert_after_stmt (last_stmt, new_stmts);
}
/* Generate an allocation sites for each new structure type. */
for (n_type = str->new_types; n_type; n_type = n_type->next)
{
tree type = n_type->data;
tree new_malloc_stmt = NULL_TREE;
tree last_stmt_tmp = NULL_TREE;
new_stmts = NULL_TREE;
new_malloc_stmt = create_new_malloc (stmt, type, &new_stmts, num);
last_stmt_tmp = tsi_stmt (tsi_last (new_stmts));
insert_after_stmt (last_stmt, new_stmts);
update_cgraph_with_malloc_call (new_malloc_stmt, context);
last_stmt = last_stmt_tmp;
}
}
}
/* Create new mallocs for function represented by NODE. */
static void
create_new_mallocs_for_func (struct cgraph_node *node)
{
malloc_d cur_malloc = get_malloc_list_for_func (node->decl);
if (cur_malloc)
create_new_mallocs (cur_malloc, node->decl);
}
/* Free new_vars list pointer by LIST_P. */
static void
free_new_vars_list (new_var *list_p)
{
new_var node = *list_p;
new_var t_node;
while (node)
{
tree_list var, t_var;
/* Free tree_list of new vars. */
var = node->new_vars;
while (var)
{
t_var = var;
var = var->next;
free (t_var);
}
t_node = node;
node = node->next;
free (t_node);
}
*list_p = NULL;
}
/* Collect only malloc_d nodes that belongs to function FN_DECL. */
static malloc_d
get_malloc_list_for_func (tree fn_decl)
{
malloc_d cur_malloc;
for (cur_malloc = malloc_data_list; cur_malloc;
cur_malloc = cur_malloc->next)
if (cur_malloc->func == fn_decl)
return cur_malloc;
return NULL;
}
/* This function returns true if STMT is one of
malloc data stmts in function FN_DECL, false otherwise */
static bool
is_part_of_malloc (tree stmt, tree fn_decl)
{
malloc_d cur_malloc = get_malloc_list_for_func (fn_decl);
if (cur_malloc)
{
malloc_call call;
for (call = cur_malloc->malloc_list; call; call = call->next)
if (call->malloc_call_stmt == stmt
|| get_final_malloc_stmt (call->malloc_call_stmt) == stmt)
return true;
}
return false;
}
/* This function remove remove node with stmt STMT
from list of non-field accesses LIST_P,
and updates it if it is changed. */
static void
remove_from_acc_list (struct access_site ** list_p, tree stmt)
{
struct access_site *node = *list_p;
struct access_site *prev_node = NULL;
while (node)
{
if (node->stmt == stmt)
{
if (prev_node == NULL)
*list_p = node->next;
else
prev_node->next = node->next;
free_tree_list (node->vars);
free (node);
return;
}
prev_node = node;
node = node->next;
}
}
/* This function checks whether STMT is part of field
accesses of structure STR. */
static bool
is_part_of_field_access (tree stmt, d_str str)
{
int i;
for (i = 0; i < str->num_fields; i++)
{
struct field_access_site *acc;
for (acc = str->fields[i].acc_sites; acc; acc = acc->next)
{
if (acc->stmt == stmt
|| acc->ref_def_stmt == stmt
|| acc->cast_stmt == stmt)
return true;
}
}
return false;
}
/* We exclude from list of non-field accesses of structure
all stmt that will be treated as part of new mallocs or
accesses generation. */
static void
exclude_malloc_and_field_accs (struct cgraph_node *node)
{
list str_node;
for (str_node = data_struct_list; str_node; str_node = str_node->next)
{
d_str str = str_node->struct_data;
struct access_site * acc = str->accs;
while (acc)
{
tree stmt = acc->stmt;
acc = acc->next;
if (is_part_of_malloc (stmt, node->decl)
|| is_part_of_field_access (stmt, str))
remove_from_acc_list (&str->accs, stmt);
}
}
}
/* This function returns component_ref with base BASE and
field named by FIELD_ID that appear in structure type TYPE. */
static inline tree
build_comp_ref (tree base, tree field_id, tree type)
{
tree field;
bool found = false;
/* Find field with the same name as field_id in type. */
for (field = TYPE_FIELDS (type); field; field = TREE_CHAIN (field))
{
if (DECL_NAME (field) == field_id)
{
found = true;
break;
}
}
gcc_assert (found);
return build3 (COMPONENT_REF, TREE_TYPE (field),
base, field, NULL_TREE);
}
/* This struct represent data used for walk_tree
called from function find_pos_in_stmt.
- ref is a tree to be found,
- and pos is a pointer that points to ref in stmt. */
struct ref_pos
{
tree *pos;
tree ref;
};
/* Helper for walk_tree called from collect_accesses_in_bb function. */
static tree
find_pos_in_stmt_1 (tree *tp, int *walk_subtrees, void * data)
{
struct ref_pos * r_pos = (struct ref_pos *) data;
tree ref = r_pos->ref;
tree t = *tp;
if (t == ref)
{
r_pos->pos = tp;
return t;
}
switch (TREE_CODE (t))
{
case GIMPLE_MODIFY_STMT:
{
tree lhs = GIMPLE_STMT_OPERAND (t, 0);
tree rhs = GIMPLE_STMT_OPERAND (t, 1);
*walk_subtrees=1;
walk_tree (&lhs, find_pos_in_stmt_1, data, NULL);
walk_tree (&rhs, find_pos_in_stmt_1, data, NULL);
*walk_subtrees=0;
}
break;
default:
*walk_subtrees=1;
}
return NULL_TREE;
}
/* This function finds pointer that points to REF in STMT,
and returns it. */
static tree *
find_pos_in_stmt (tree stmt, tree ref)
{
struct ref_pos r_pos;
r_pos.ref = ref;
r_pos.pos = NULL;
walk_tree (&stmt, find_pos_in_stmt_1, &r_pos, NULL);
return r_pos.pos;
}
/* This function replace field access ACC with field access of structure
of type NEW_TYPE. */
static void
replace_field_acc (struct field_access_site *acc, tree new_type)
{
tree ref_var = acc->ref;
tree new_ref;
tree lhs, rhs;
tree *pos;
tree new_acc;
tree field_id = DECL_NAME (acc->field_decl);
struct type_wrapper * wrapper = NULL;
struct type_wrapper * wr = NULL;
while (TREE_CODE (ref_var) == INDIRECT_REF
|| TREE_CODE (ref_var) == ARRAY_REF)
{
if ( TREE_CODE (ref_var) == INDIRECT_REF)
wrapper = add_wrapper (wrapper, 0, 0);
else if (TREE_CODE (ref_var) == ARRAY_REF)
wrapper = add_wrapper (wrapper, 1, TREE_OPERAND (ref_var, 1));
ref_var = TREE_OPERAND (ref_var, 0);
}
new_ref = find_new_var_of_type (ref_var, new_type);
finish_global_creation (new_ref);
for (wr = wrapper; wr; wr = wr->next)
{
tree type = TREE_TYPE (TREE_TYPE (new_ref));
if (wr->wrap) /* Array. */
new_ref = build4 (ARRAY_REF, type, new_ref,
wr->domain, NULL_TREE, NULL_TREE);
else /* Pointer. */
new_ref = build1 (INDIRECT_REF, type, new_ref);
}
new_acc = build_comp_ref (new_ref, field_id, new_type);
free_wrapper (wrapper);
if (TREE_CODE (acc->stmt) == GIMPLE_MODIFY_STMT)
{
lhs = GIMPLE_STMT_OPERAND (acc->stmt, 0);
rhs = GIMPLE_STMT_OPERAND (acc->stmt, 1);
if (lhs == acc->comp_ref)
GIMPLE_STMT_OPERAND (acc->stmt, 0) = new_acc;
else if (rhs == acc->comp_ref)
GIMPLE_STMT_OPERAND (acc->stmt, 1) = new_acc;
else
{
pos = find_pos_in_stmt (acc->stmt, acc->comp_ref);
gcc_assert (pos);
*pos = new_acc;
}
}
else
{
pos = find_pos_in_stmt (acc->stmt, acc->comp_ref);
gcc_assert (pos);
*pos = new_acc;
}
finish_stmt (acc->stmt);
}
/* This function replace field access ACC with field access of structure
of type NEW_TYPE. */
static void
replace_field_access_stmt (struct field_access_site *acc, tree new_type)
{
if (TREE_CODE (acc->ref) == INDIRECT_REF
||TREE_CODE (acc->ref) == ARRAY_REF
||TREE_CODE (acc->ref) == VAR_DECL)
replace_field_acc (acc, new_type);
else
gcc_unreachable ();
}
/* In this function we create new stmts that has the same
form as ORIG_STMT, but of the type NET_TYPE. The stmts
treated by this function are simple assignments,
like assignments: p.8_7 = p; or stmts with rhs of code
PLUS_EXPR or MINUS_EXPR. */
static tree
create_base_plus_offset (tree orig_stmt, tree new_type,
tree offset)
{
tree lhs, rhs;
tree new_lhs, new_rhs;
tree new_stmt;
gcc_assert (TREE_CODE (orig_stmt) == GIMPLE_MODIFY_STMT);
lhs = GIMPLE_STMT_OPERAND (orig_stmt, 0);
rhs = GIMPLE_STMT_OPERAND (orig_stmt, 1);
gcc_assert (TREE_CODE (lhs) == VAR_DECL
|| TREE_CODE (lhs) == SSA_NAME);
new_lhs = find_new_var_of_type (lhs, new_type);
gcc_assert (new_lhs);
finish_var_creation (new_lhs);
switch (TREE_CODE (rhs))
{
case PLUS_EXPR:
case MINUS_EXPR:
case POINTER_PLUS_EXPR:
{
tree op0 = TREE_OPERAND (rhs, 0);
tree op1 = TREE_OPERAND (rhs, 1);
tree new_op0 = NULL_TREE, new_op1 = NULL_TREE;
list tmp0, tmp1;
tmp0 = find_struct_in_list (strip_type (get_type_of_var (op0)));
tmp1 = find_struct_in_list (strip_type (get_type_of_var (op1)));
gcc_assert ( tmp0 || tmp1);
if (tmp0)
new_op0 = find_new_var_of_type (op0, new_type);
if (tmp1)
new_op1 = find_new_var_of_type (op1, new_type);
if (!new_op0)
new_op0 = offset;
if (!new_op1)
new_op1 = offset;
new_rhs = build2 (TREE_CODE (rhs), TREE_TYPE (new_op0),
new_op0, new_op1);
}
break;
default:
gcc_unreachable();
}
new_stmt = build2 (GIMPLE_MODIFY_STMT, TREE_TYPE (new_lhs),
new_lhs, new_rhs);
finish_stmt (new_stmt);
return new_stmt;
}
/* Generate stmt: res = NUM * sizeof(TYPE) and return it. */
static tree
gen_size (tree num, tree type, tree *res)
{
tree struct_size = TYPE_SIZE_UNIT (type);
HOST_WIDE_INT struct_size_int = TREE_INT_CST_LOW (struct_size);
tree new_stmt;
*res = create_tmp_var (TREE_TYPE (num), NULL);
if (*res)
add_referenced_var (*res);
if (exact_log2 (struct_size_int) == -1)
new_stmt = build2 (GIMPLE_MODIFY_STMT, TREE_TYPE (num),
*res, build2 (MULT_EXPR,
TREE_TYPE (num),
num,
struct_size));
else
new_stmt = build2 (GIMPLE_MODIFY_STMT, TREE_TYPE (num),
*res,
build2 (LSHIFT_EXPR,
TREE_TYPE (num),
num,
build_int_cst (TREE_TYPE (num),
exact_log2 (struct_size_int))));
finish_stmt (new_stmt);
return new_stmt;
}
/* This function generates and returns the stmt, that cast variable
BEFORE_CAST to NEW_TYPE. The casting result variable is stored
into RES_P. */
static tree
gen_cast_stmt (tree before_cast, tree new_type, tree old_cast_stmt, tree *res_p)
{
tree lhs, new_lhs, new_stmt;
gcc_assert (TREE_CODE (old_cast_stmt) == GIMPLE_MODIFY_STMT);
lhs = GIMPLE_STMT_OPERAND (old_cast_stmt, 0);
new_lhs = find_new_var_of_type (lhs, new_type);
gcc_assert (new_lhs);
new_stmt = build2 (GIMPLE_MODIFY_STMT,
TREE_TYPE
(new_lhs),
new_lhs,
build1 (NOP_EXPR,
TREE_TYPE (new_lhs),
before_cast));
finish_stmt (new_stmt);
*res_p = new_lhs;
return new_stmt;
}
/* Given a field access ACC, this function generates list of
stmt require for new field access of field FIELD. */
static void
create_new_field_access (struct field_access_site *acc, struct data_field_entry field)
{
tree new_type = field.field_mapping;
tree new_stmt;
tree size_res;
tree mult_stmt, cast_stmt;
tree cast_res = NULL;
if (acc->num)
{
mult_stmt = gen_size (acc->num, new_type, &size_res);
insert_before_stmt (acc->ref_def_stmt, mult_stmt);
}
if (acc->cast_stmt)
{
cast_stmt = gen_cast_stmt (size_res, new_type,
acc->cast_stmt, &cast_res);
insert_after_stmt (acc->cast_stmt, cast_stmt);
}
if (acc->ref_def_stmt)
{
tree offset;
if (cast_res)
offset = cast_res;
else
offset = size_res;
new_stmt = create_base_plus_offset (acc->ref_def_stmt, new_type, offset);
insert_after_stmt (acc->ref_def_stmt, new_stmt);
}
/* In stmt D.2163_19 = D.2162_18->b; we replace variable
D.2162_18 for the appropriate variable of new_type. */
replace_field_access_stmt (acc, new_type);
}
/* This function creates new non-field access of structure STR
correspondent to ACC data. */
static void
create_new_general_access (struct access_site *acc, d_str str)
{
tree stmt = acc->stmt;
switch (TREE_CODE (stmt))
{
case COND_EXPR:
create_new_stmts_for_cond_expr (stmt);
break;
default:
create_new_stmts_for_general_acc (acc, str);
}
}
/* This function creates new field and non-field
accesses in BB. */
static void
create_new_accesses_in_bb (basic_block bb)
{
list tmp;
for (tmp = data_struct_list; tmp; tmp = tmp->next)
{
int i;
struct field_access_site *f_acc;
struct access_site *acc;
d_str str = tmp->struct_data;
for (i = 0; i < str->num_fields; i++)
for (f_acc = str->fields[i].acc_sites; f_acc; f_acc = f_acc->next)
if (bb_for_stmt (f_acc->stmt) == bb)
create_new_field_access (f_acc, str->fields[i]);
for (acc = str->accs; acc; acc = acc->next)
if (bb_for_stmt (acc->stmt) == bb)
create_new_general_access (acc, str);
}
}
/* This function creates field and non-field
accesses for cfun. */
static void
create_new_accesses_for_func (void)
{
basic_block bb;
FOR_EACH_BB_FN (bb, cfun)
create_new_accesses_in_bb (bb);
}
/* Create general new access of structure type NEW_TYPE
according to ACC data. */
static tree
create_general_new_stmt (struct access_site *acc, tree new_type)
{
tree old_stmt = acc->stmt;
tree_list var_d;
tree new_stmt = copy_node (old_stmt);
for (var_d = acc->vars; var_d; var_d = var_d->next)
{
tree * pos;
tree var = var_d->data;
tree new_var = find_new_var_of_type (var, new_type);
tree lhs, rhs;
gcc_assert (new_var);
finish_var_creation (new_var);
if (TREE_CODE (new_stmt) == GIMPLE_MODIFY_STMT)
{
lhs = GIMPLE_STMT_OPERAND (new_stmt, 0);
rhs = GIMPLE_STMT_OPERAND (new_stmt, 1);
if (TREE_CODE (lhs) == SSA_NAME)
lhs = SSA_NAME_VAR (lhs);
if (TREE_CODE (rhs) == SSA_NAME)
rhs = SSA_NAME_VAR (rhs);
/* It can happen that rhs is a constructor.
Then we have to replace it to be of new_type. */
if (TREE_CODE (rhs) == CONSTRUCTOR)
{
/* Dealing only with empty constructors right now. */
gcc_assert (VEC_empty (constructor_elt,
CONSTRUCTOR_ELTS (rhs)));
rhs = build_constructor (new_type, 0);
GIMPLE_STMT_OPERAND (new_stmt, 1) = rhs;
}
if (lhs == var)
GIMPLE_STMT_OPERAND (new_stmt, 0) = new_var;
else if (rhs == var)
GIMPLE_STMT_OPERAND (new_stmt, 1) = new_var;
else
{
pos = find_pos_in_stmt (new_stmt, var);
gcc_assert (pos);
*pos = new_var;
}
}
else
{
pos = find_pos_in_stmt (new_stmt, var);
gcc_assert (pos);
*pos = new_var;
}
}
finish_stmt (new_stmt);
return new_stmt;
}
/* This function creates new non-field accesses for
each new type in STR. */
static void
create_new_stmts_for_general_acc (struct access_site * acc, d_str str)
{
tree_list new_type;
tree stmt = acc->stmt;
for (new_type = str->new_types; new_type; new_type = new_type->next)
{
tree new_stmt;
new_stmt = create_general_new_stmt (acc, new_type->data);
insert_after_stmt (stmt, new_stmt);
}
}
/* This function creates new condition stmt and
redirect edges to correspondent BBs.
NEW_VAR position in stmt is POS. */
static void
create_new_stmts_for_cond_expr_1 (tree new_var, tree cond_stmt, bool pos)
{
tree new_cond;
tree new_stmt;
edge true_e = NULL, false_e = NULL;
basic_block new_bb;
tree stmt_list;
extract_true_false_edges_from_block (bb_for_stmt (cond_stmt),
&true_e, &false_e);
new_cond = copy_node (COND_EXPR_COND (cond_stmt));
TREE_OPERAND (new_cond, pos) = new_var;
new_stmt = build3 (COND_EXPR,TREE_TYPE (cond_stmt),
new_cond, NULL_TREE, NULL_TREE);
finish_stmt (new_stmt);
/* Create a new basic block after bb. */
new_bb = create_empty_bb (bb_for_stmt (cond_stmt));
/* Add new condition stmt to the new_bb. */
stmt_list = bb_stmt_list (new_bb);
append_to_statement_list (new_stmt, &stmt_list);
add_bb_info (new_bb, stmt_list);
/* Create false and true edges from new_bb. */
make_edge_and_fix_phis_of_dest (new_bb, true_e);
make_edge_and_fix_phis_of_dest (new_bb, false_e);
/* Redirect one of original edges to point to new_bb. */
if (TREE_CODE (cond_stmt) == NE_EXPR)
redirect_edge_succ (true_e, new_bb);
else
redirect_edge_succ (false_e, new_bb);
}
/* This function creates new condition stmts instead of STMT,
one for each new type, and recursively redirect edges
to newly generated BBs. */
static void
create_new_stmts_for_cond_expr (tree stmt)
{
tree cond = COND_EXPR_COND (stmt);
tree arg0, arg1, arg;
list tmp0, tmp1, tmp;
d_str str;
tree_list new_type;
bool pos;
gcc_assert (TREE_CODE (cond) == EQ_EXPR
|| TREE_CODE (cond) == NE_EXPR);
arg0 = TREE_OPERAND (cond, 0);
arg1 = TREE_OPERAND (cond, 1);
tmp0 = find_struct_in_list (strip_type (get_type_of_var (arg0)));
tmp1 = find_struct_in_list (strip_type (get_type_of_var (arg1)));
gcc_assert ((!tmp0 && tmp1) || (!tmp1 && tmp0));
tmp = tmp0 ? tmp0 : tmp1;
arg = tmp0 ? arg0 : arg1;
pos = tmp0 ? 0 : 1;
str = tmp->struct_data;
for (new_type = str->new_types; new_type; new_type = new_type->next)
{
tree new_arg;
new_arg = find_new_var_of_type (arg, new_type->data);
create_new_stmts_for_cond_expr_1 (new_arg, stmt, pos);
}
}
/* Do transformation for a function represented by NODE. */
static void
do_reorg_for_func (struct cgraph_node *node)
{
create_new_local_vars ();
create_new_mallocs_for_func (node);
create_new_accesses_for_func ();
update_ssa (TODO_update_ssa);
cleanup_tree_cfg ();
/* Free auxiliary data for local variables. */
free_new_vars_list (&new_local_vars);
}
/* This function adds NEW_NODE to list of new_var's pointed by LIST_P.
It updates LIST_P if it's changed. */
static void
add_to_new_vars_list (new_var new_node, new_var *list_p)
{
new_var node;
if (!(*list_p))
*list_p = new_node;
else
{
node = *list_p;
while (node->next)
node = node->next;
node->next = new_node;
}
}
/* This function returns new_var node with DECL
as orig_var, if found in new_var's list LIST_P,
or NULL otherwise. */
static new_var
is_in_new_vars_list (tree decl, new_var list_p)
{
new_var node;
if (!list_p)
return NULL;
else
{
node = list_p;
while (node)
{
if (node->orig_var == decl)
return node;
node = node->next;
}
}
return NULL;
}
/* This function creates and returns new_var node
with NEW_VARS_LIST as new_vars and VAR as orig_var. */
static new_var
create_new_var_node (tree var, tree_list new_vars_list)
{
new_var node;
node = (struct new_var_data *) xmalloc (sizeof (struct new_var_data));
node->orig_var = var;
node->new_vars = new_vars_list;
node->next = NULL;
return node;
}
/* This function creates new variables to
substitute original variable VAR_DECL and adds
them to the new vars list pointed by LIST_P. */
static void
create_new_var (tree var_decl, new_var *list_p)
{
new_var node;
tree_list new_vars;
list tmp;
tree type;
d_str str;
if (!var_decl || is_in_new_vars_list (var_decl, *list_p))
return;
if (!is_candidate_for_data_struct_list (var_decl, &type, NULL))
return;
tmp = find_struct_in_list (type);
if (!tmp)
return;
str = tmp->struct_data;
new_vars = create_new_var_1 (var_decl, str);
node = create_new_var_node (var_decl, new_vars);
add_to_new_vars_list (node, list_p);
}
/* Update varpool with variables from new_global_vars. */
static void
update_varpool_with_new_vars (void)
{
new_var old_var;
for (old_var = new_global_vars; old_var; old_var = old_var->next)
{
tree_list n_var;
for (n_var = old_var->new_vars; n_var; n_var = n_var->next)
insert_global_to_varpool (n_var->data);
}
}
/* This function creates global struct variables of new
structure types in case these this variable has VAR_DECL code. */
static void
create_new_global_vars (void)
{
struct varpool_node *current_varpool;
FOR_EACH_STATIC_VARIABLE(current_varpool)
{
tree var_decl = current_varpool->decl;
if (!var_decl || TREE_CODE (var_decl) != VAR_DECL)
continue;
create_new_var (var_decl, &new_global_vars);
}
update_varpool_with_new_vars ();
}
/* This function is a helper for do_reorg. It goes over
functions in call graph and performs actual transformation
on each one of them. */
static void
do_reorg_1 (void)
{
struct cgraph_node *node;
/* Initialize the default bitmap obstack. */
bitmap_obstack_initialize (NULL);
for (node = cgraph_nodes; node; node = node->next)
if (node->analyzed && node->decl && !node->next_clone)
{
push_cfun (DECL_STRUCT_FUNCTION (node->decl));
current_function_decl = node->decl;
if (dump_file)
fprintf (dump_file, "\nFunction to do reorg is %s: \n",
(char *) IDENTIFIER_POINTER (DECL_NAME (node->decl)));
do_reorg_for_func (node);
free_dominance_info (CDI_DOMINATORS);
free_dominance_info (CDI_POST_DOMINATORS);
current_function_decl = NULL;
pop_cfun ();
}
cfun = NULL;
}
/* Stage 3. */
static void
do_reorg (void)
{
/* Check that there is a work to do. */
if (!data_struct_list)
return;
/* Generate new types. */
create_new_types ();
print_new_types ();
/* Create new global vars. */
create_new_global_vars ();
print_new_vars (new_global_vars);
/* Do reorg for each funciton separately. */
do_reorg_1 ();
/* Free auxiliary data for global variables. */
free_new_vars_list (&new_global_vars);
}
/* This function builds an entry in struct_list for each
data structure potential for transformation. */
static void
build_data_structure_list (bitmap non_suitable_types)
{
tree var, type;
tree var_list;
struct varpool_node *current_varpool;
struct cgraph_node *c_node;
/* Check global vars. */
FOR_EACH_STATIC_VARIABLE (current_varpool)
{
var = current_varpool->decl;
if (is_candidate_for_data_struct_list (var, &type,
non_suitable_types))
add_struct_to_data_struct_list (type);
}
/* Now add data structures of function parameters and
local variables. */
for (c_node = cgraph_nodes; c_node; c_node = c_node->next)
{
enum availability avail =
cgraph_function_body_availability (c_node);
/* We need AVAIL_AVAILABLE for main function. */
if (avail == AVAIL_LOCAL || avail == AVAIL_AVAILABLE)
{
struct function *fn = DECL_STRUCT_FUNCTION (c_node->decl);
for (var = DECL_ARGUMENTS (c_node->decl); var;
var = TREE_CHAIN (var))
if (is_candidate_for_data_struct_list (var, &type,
non_suitable_types))
add_struct_to_data_struct_list (type);
/* Function local variables. */
for (var_list = fn->unexpanded_var_list; var_list;
var_list = TREE_CHAIN (var_list))
{
var = TREE_VALUE (var_list);
if (is_candidate_for_data_struct_list (var, &type,
non_suitable_types))
add_struct_to_data_struct_list (type);
}
}
}
}
/* If struct type is a returned type of any function,
we cannot transform it. */
static void
exclude_returned_types (bitmap non_suitable_types)
{
struct cgraph_node *c_node;
for (c_node = cgraph_nodes; c_node; c_node = c_node->next)
{
tree ret_t = TREE_TYPE (TREE_TYPE (c_node->decl));
if (ret_t)
{
ret_t = strip_type (ret_t);
if (TREE_CODE (ret_t) == RECORD_TYPE)
{
bitmap_set_bit (non_suitable_types,
TYPE_UID (TYPE_MAIN_VARIANT (ret_t)));
if (dump_file)
{
fprintf (dump_file, "\nThe type \"");
print_generic_expr (dump_file, ret_t, 0);
fprintf (dump_file,
"\" is return type of function...Excluded.");
}
}
}
}
}
/* This function looks for arguments of local functions
which are of pointer to or structure types.
We are not handling peeling of such structures right now. */
static void
exclude_types_passed_to_local_func (bitmap non_suitable_types)
{
struct cgraph_node *c_node;
for (c_node = cgraph_nodes; c_node; c_node = c_node->next)
{
if (cgraph_function_body_availability (c_node) == AVAIL_LOCAL)
{
tree fn = c_node->decl;
tree arg;
for (arg = DECL_ARGUMENTS (fn); arg; arg = TREE_CHAIN (arg))
{
tree type = TREE_TYPE (arg);
type = strip_type (type);
if (TREE_CODE (type) == RECORD_TYPE)
{
bitmap_set_bit (non_suitable_types,
TYPE_UID (TYPE_MAIN_VARIANT (type)));
if (dump_file)
{
fprintf (dump_file, "\nPointer to type \"");
print_generic_expr (dump_file, type, 0);
fprintf (dump_file,
"\" is passed to local function...Excluded.");
}
}
}
}
}
}
/* This function sets the bits of NON_SUITABLE_TYPE for
those types that escape in the rules of ipa-type-escpae analysis.
See ipa-type-escpae.[c,h]. */
static void
exclude_escaping_types_1 (bitmap non_suitable_types)
{
list tmp = data_struct_list;
while (tmp)
{
tree type = tmp->struct_data->decl;
if (!ipa_type_escape_type_contained_p (type))
{
if (dump_file)
{
fprintf (dump_file, "\nEscaping type is ");
print_generic_expr (dump_file, type, 0);
}
bitmap_set_bit (non_suitable_types, TYPE_UID (type));
}
tmp = tmp->next;
}
}
/* This function sets those bits of NON_SUITABLE_TYPE that
cannot get transformed now. */
static void
exclude_escaping_types (bitmap non_suitable_types)
{
exclude_types_passed_to_local_func (non_suitable_types);
exclude_returned_types (non_suitable_types);
exclude_escaping_types_1 (non_suitable_types);
}
/* Exclude structs with bitfields.
We would not want to interfere with othe optimizations
that can be done in this case. */
static void
exclude_types_with_bit_fields (bitmap non_suitable_types)
{
list tmp = data_struct_list;
while (tmp)
{
int i;
d_str str = tmp->struct_data;
tree type = tmp->struct_data->decl;
for (i = 0; i < str->num_fields; i++)
if (DECL_BIT_FIELD (str->fields[i].decl))
bitmap_set_bit (non_suitable_types, TYPE_UID (type));
tmp = tmp->next;
}
}
/* This function returns true if program contains
a call to user defined allocation function, or other
functions that can interfere with this transformation. */
static bool
program_redefines_malloc_p (void)
{
struct cgraph_node *c_node;
struct cgraph_node *c_node2;
struct cgraph_edge *c_edge;
tree fndecl;
tree fndecl2;
tree call_expr;
for (c_node = cgraph_nodes; c_node; c_node = c_node->next)
{
fndecl = c_node->decl;
for (c_edge = c_node->callees; c_edge;
c_edge = c_edge->next_callee)
{
call_expr = get_call_expr_in (c_edge->call_stmt);
c_node2 = c_edge->callee;
fndecl2 = c_node2->decl;
if (call_expr)
{
const char * fname = get_name (fndecl2);
if ((call_expr_flags (call_expr) & ECF_MALLOC) &&
(DECL_FUNCTION_CODE (fndecl2) != BUILT_IN_MALLOC) &&
(DECL_FUNCTION_CODE (fndecl2) != BUILT_IN_CALLOC) &&
(DECL_FUNCTION_CODE (fndecl2) != BUILT_IN_ALLOCA))
return true;
/* Check check that there is no __builtin_object_size,
__builtin_offsetof, or realloc's in the program. */
if (DECL_FUNCTION_CODE (fndecl2) == BUILT_IN_OBJECT_SIZE
|| !strcmp (fname, "__builtin_offsetof")
|| !strcmp (fname, "realloc"))
return true;
}
}
}
return false;
}
/* Auxiliary function for add_struct_to_data_struct_list. */
static struct data_field_entry *
get_fields (tree struct_decl, int num_fields)
{
struct data_field_entry *list;
tree t = TYPE_FIELDS (struct_decl);
int idx = 0;
list = (struct data_field_entry * ) xmalloc (num_fields
* sizeof
(struct data_field_entry));
for (idx = 0 ; t; t = TREE_CHAIN (t), idx++)
if (TREE_CODE (t) == FIELD_DECL)
{
list[idx].index = idx;
list[idx].decl = t;
list[idx].acc_sites = NULL;
list[idx].count = 0;
list[idx].field_mapping = NULL_TREE;
}
return list;
}
/* Auxiliary function for remove_struct_from_data_struct_list. */
static void
free_struct_cluster (struct field_cluster* cluster)
{
if (cluster)
{
if (cluster->fields_in_cluster)
sbitmap_free (cluster->fields_in_cluster);
if (cluster->sibling)
free_struct_cluster (cluster->sibling);
free (cluster);
}
}
/* The following three functions:
add_struct_to_data_struct_list
remove_struct_from_data_struct_list
free_data_structure_list
is an interface for manipulation with the list of data
structures to be transformed which is accessed through
data_struct_list. */
static void
add_struct_to_data_struct_list (tree type)
{
struct data_structure *d_node;
struct struct_list *new_node = NULL;
int num_fields;
list found = NULL;
type = TYPE_MAIN_VARIANT (type);
found = find_struct_in_list (type);
if (found)
return;
d_node = (struct data_structure *)
xcalloc (1, sizeof (struct data_structure));
num_fields = fields_length (type);
d_node->decl = type;
d_node->num_fields = num_fields;
d_node->fields = get_fields (type, num_fields);
d_node->struct_clustering = NULL;
d_node->accs = NULL;
new_node = (struct struct_list *) xmalloc (sizeof(struct struct_list));
new_node->struct_data = d_node;
new_node->next = data_struct_list;
data_struct_list = new_node;
if (dump_file)
{
fprintf (dump_file, "\nAdding data structure \"");
print_generic_expr (dump_file, type, 0);
fprintf (dump_file, "\" to data_struct_list.");
}
}
/* Free list of structs pointed by LIST_P. */
static void
free_tree_list (tree_list list_p)
{
tree_list node = list_p;
tree_list tmp_node;
while (node)
{
tmp_node = node;
node = node->next;
free (tmp_node);
}
}
/* This function free field accesses list pointed by ACCS. */
static void
free_field_acc_list (struct field_access_site *accs)
{
struct field_access_site * tmp_acc = accs;
struct field_access_site * tmp_acc1;
if (!accs)
return;
while (tmp_acc)
{
tmp_acc1 = tmp_acc;
tmp_acc = tmp_acc->next;
free (tmp_acc1);
}
}
/* This function free non-field accesses list
pointed by ACCS. */
static void
free_access_list (struct access_site *accs)
{
struct access_site * tmp_acc = accs;
struct access_site * tmp_acc1;
if (!accs)
return;
while (tmp_acc)
{
tmp_acc1 = tmp_acc;
tmp_acc = tmp_acc->next;
free_tree_list (tmp_acc1->vars);
free (tmp_acc1);
}
}
/* Free d_str node D_NODE. */
static void
free_data_struct (d_str d_node)
{
int i;
if (!d_node)
return;
if (dump_file)
{
fprintf (dump_file, "\nRemoving data structure \"");
print_generic_expr (dump_file, d_node->decl, 0);
fprintf (dump_file, "\" from data_struct_list.");
}
/* Free all space under d_node. */
if (d_node->fields)
{
for (i = 0; i < d_node->num_fields; i++)
free_field_acc_list (d_node->fields[i].acc_sites);
free (d_node->fields);
}
if (d_node->accs)
free_access_list (d_node->accs);
if (d_node->struct_clustering)
free_struct_cluster (d_node->struct_clustering);
if (d_node->new_types)
free_tree_list (d_node->new_types);
}
/* This function removes struct STR_TO_REMOVE
from data_struct_list. */
static void
remove_struct_from_data_struct_list (struct struct_list *str_to_remove)
{
struct struct_list *curr_str, *prev;
if (data_struct_list == str_to_remove)
{
free_data_struct (data_struct_list->struct_data);
data_struct_list = data_struct_list->next;
free (str_to_remove);
return;
}
for (curr_str = data_struct_list->next, prev = data_struct_list;
curr_str; curr_str = curr_str->next, prev = prev->next)
if (curr_str == str_to_remove)
{
free_data_struct (curr_str->struct_data);
prev->next = curr_str->next;
free (curr_str);
break;
}
}
/* Remove structs set in STRUCTS from data_struct_list. */
static void
remove_structs_from_data_struct_list (bitmap structs)
{
list tmp = data_struct_list;
while (tmp)
{
d_str str = tmp->struct_data;
if (bitmap_bit_p (structs, TYPE_UID (str->decl)))
{
list tmp1 = tmp;
tmp = tmp->next;
remove_struct_from_data_struct_list (tmp1);
}
else
tmp = tmp->next;
}
}
/* Free data_struct_list. */
static void
free_data_structure_list (void)
{
list current_struct = data_struct_list;
while (current_struct)
{
list tmp = current_struct;
d_str d_node = current_struct->struct_data;
free_data_struct (d_node);
current_struct = current_struct->next;
/* Free curent_struct itself. */
if (tmp)
free (tmp);
}
data_struct_list = NULL;
}
/* Free malloc_data_list. */
static void
free_malloc_data (void)
{
malloc_d cur_malloc = malloc_data_list;
malloc_d tmp;
while (cur_malloc)
{
malloc_call call = cur_malloc->malloc_list;
while (call)
{
malloc_call call1 = call;
call = call->next;
free (call1);
}
tmp = cur_malloc;
cur_malloc = cur_malloc->next;
free (tmp);
}
}
/* This function calculates maximum field count in
the structure STR. */
static gcov_type
get_max_field_count (d_str str)
{
gcov_type max = 0;
int i;
for (i = 0; i < str->num_fields; i++)
if (str->fields[i].count > max)
max = str->fields[i].count;
return max;
}
/* This function peels field with index I from struct DS. */
static void
peel_field (int i, d_str ds)
{
struct field_cluster *crr_cluster = NULL;
crr_cluster =
(struct field_cluster *) xcalloc (1, sizeof (struct field_cluster));
crr_cluster->sibling = ds->struct_clustering;
ds->struct_clustering = crr_cluster;
crr_cluster->fields_in_cluster =
sbitmap_alloc ((unsigned int) ds->num_fields);
sbitmap_zero (crr_cluster->fields_in_cluster);
SET_BIT (crr_cluster->fields_in_cluster, i);
}
/* This function generates cluster with the fields FIELDS
for structure DS. */
static void
gen_cluster (sbitmap fields, d_str ds)
{
struct field_cluster *crr_cluster = NULL;
crr_cluster =
(struct field_cluster *) xcalloc (1, sizeof (struct field_cluster));
crr_cluster->sibling = ds->struct_clustering;
ds->struct_clustering = crr_cluster;
crr_cluster->fields_in_cluster = fields;
}
/* This function peels the field into separate structure if it's
suffitiently hot. */
static void
peel_hot_fields (d_str ds)
{
gcov_type max_field_count;
sbitmap fields_left = sbitmap_alloc (ds->num_fields);
int i;
sbitmap_ones (fields_left);
max_field_count =
(gcov_type) (get_max_field_count (ds)/100)*90;
ds->struct_clustering = NULL;
for (i = 0; i < ds->num_fields; i++)
{
if (ds->count >= max_field_count)
{
RESET_BIT (fields_left, i);
peel_field (i, ds);
}
}
i = sbitmap_first_set_bit (fields_left);
if (i != -1)
gen_cluster (fields_left, ds);
else
sbitmap_free (fields_left);
}
/* This function analyzes structure form of structures
in data_struct_list. If we are not capable to transform
structure of some form, we remove it from list of structs.
Right now we cannot handle nested structs, when nesting is
through any level of pointer or arrays.
TBD: release these constrains in future.
Note, that in this function we suppose that all structs
in program are members of data_struct_list right now,
without excluding escaping types.
*/
static void
analyze_struct_form (bitmap non_suitable_types)
{
list tmp = data_struct_list;
while(tmp)
{
d_str str = tmp->struct_data;
int i;
for (i = 0; i < str->num_fields; i++)
{
tree f_type = strip_type(TREE_TYPE (str->fields[i].decl));
if (TREE_CODE (f_type) == RECORD_TYPE)
{
bitmap_set_bit (non_suitable_types,
TYPE_UID (TYPE_MAIN_VARIANT (f_type)));
bitmap_set_bit (non_suitable_types,
TYPE_UID (str->decl));
}
}
tmp = tmp->next;
}
}
/* This function collect data structures potential
for peeling transformation and insert
them into data_struct_list. */
static void
collect_data_structs (void)
{
bitmap non_suitable_types = BITMAP_ALLOC (NULL);
/* If program contains user defined mallocs, we give up. */
if (program_redefines_malloc_p ())
return;
/* Build data structures list of all data structures
in the program. */
build_data_structure_list (non_suitable_types);
/* In this function we analyze whether the form of
structure is such that we are capable to transform it.
Nested structures are checked here. */
analyze_struct_form (non_suitable_types);
/* This function excludes structure types escape compilation unit. */
exclude_escaping_types (non_suitable_types);
/* We cannot change data layout of structs with bitfields. */
exclude_types_with_bit_fields (non_suitable_types);
remove_structs_from_data_struct_list (non_suitable_types);
BITMAP_FREE (non_suitable_types);
if (!data_struct_list)
{
if (dump_file)
fprintf (dump_file, "\nNo structures to transform. Exiting...");
return;
}
}
/* Print field accesses list pointed by ACC_SITES. */
static void
print_field_acc_sites (struct field_access_site *acc_sites)
{
struct field_access_site *acc;
if (!dump_file)
return;
for (acc = acc_sites; acc; acc = acc->next)
{
fprintf(dump_file, "\n");
if (acc->stmt)
print_generic_stmt (dump_file, acc->stmt, 0);
if (acc->ref_def_stmt)
print_generic_stmt (dump_file, acc->ref_def_stmt, 0);
if (acc->cast_stmt)
print_generic_stmt (dump_file, acc->cast_stmt, 0);
}
}
/* Print non-field accesses list pointed by ACC_SITES. */
static void
print_access_sites (struct access_site *acc_sites)
{
struct access_site *acc;
if (!dump_file)
return;
for (acc = acc_sites; acc; acc = acc->next)
{
tree_list list_p;
fprintf(dump_file, "\n");
if (acc->stmt)
print_generic_stmt (dump_file, acc->stmt, 0);
fprintf(dump_file, " : ");
for (list_p = acc->vars; list_p; list_p = list_p->next)
{
print_generic_stmt (dump_file, list_p->data, 0);
fprintf(dump_file, ", ");
}
}
}
/* Print collected accesses. */
static void
print_accesses (void)
{
list tmp = data_struct_list;
if (!dump_file)
return;
for (tmp = data_struct_list; tmp; tmp = tmp->next)
{
int i;
d_str str = tmp->struct_data;
fprintf (dump_file, "\nAccess sites of struct ");
print_generic_expr (dump_file, str->decl, 0);
for (i = 0; i < str->num_fields; i++)
{
fprintf (dump_file, "\nAccess site of field ");
print_generic_expr (dump_file, str->fields[i].decl, 0);
print_field_acc_sites (str->fields[i].acc_sites);
fprintf (dump_file, ":\n");
}
fprintf (dump_file, "\nGeneral access sites\n");
print_access_sites (str->accs);
}
}
/* Currently we support only EQ_EXPR or
NE_EXPR conditions. */
static bool
is_safe_cond_expr (tree cond_stmt)
{
tree arg0, arg1;
list tmp0, tmp1;
tree cond = COND_EXPR_COND (cond_stmt);
if (TREE_CODE (cond) != EQ_EXPR
&& TREE_CODE (cond) != NE_EXPR)
return false;
if (TREE_CODE_LENGTH (TREE_CODE (cond)) != 2)
return false;
arg0 = TREE_OPERAND (cond, 0);
arg1 = TREE_OPERAND (cond, 1);
tmp0 = find_struct_in_list (strip_type (get_type_of_var (arg0)));
tmp1 = find_struct_in_list (strip_type (get_type_of_var (arg1)));
if (!((!tmp0 && tmp1) || (!tmp1 && tmp0)))
return false;
return true;
}
/* Check condition expressions. */
static void
check_cond_exprs (void)
{
list str_node = data_struct_list;
while (str_node)
{
d_str str = str_node->struct_data;
struct access_site * acc;
list str_node1 = str_node;
str_node = str_node->next;
for (acc = str->accs; acc; acc = acc->next)
if (TREE_CODE (acc->stmt) == COND_EXPR)
{
if (!is_safe_cond_expr (acc->stmt))
remove_struct_from_data_struct_list (str_node1);
}
}
}
/* This function collects data accesses for the structures
from data_struct_list. For each structure
field it updates its count in data_field_entry. */
static void
collect_data_accesses (void)
{
struct cgraph_node *c_node;
for (c_node = cgraph_nodes; c_node; c_node = c_node->next)
{
enum availability avail = cgraph_function_body_availability (c_node);
if (avail == AVAIL_LOCAL || avail == AVAIL_AVAILABLE)
{
struct function *func = DECL_STRUCT_FUNCTION (c_node->decl);
if (!c_node->next_clone)
{
/* Collect accesses of the fields of structure
under consideretion that happen in func. */
collect_accesses_in_func (func);
}
exclude_malloc_and_field_accs (c_node);
}
}
check_cond_exprs ();
/* Print collected accesses. */
print_accesses ();
}
/* Exclude cold structures from transformation. */
static void
exclude_cold_structs (void)
{
gcov_type hotest_struct_count = 0;
list tmp = data_struct_list;
/* We summarize counts of fields into structure count. */
while (tmp)
{
int i;
d_str str = tmp->struct_data;
str->count = 0;
for (i = 0; i < str->num_fields; i++)
{
if (dump_file)
{
fprintf (dump_file, "\nCounter of field \"");
print_generic_expr (dump_file, str->fields[i].decl, 0);
fprintf (dump_file, "\" is " HOST_WIDE_INT_PRINT_DEC,
str->fields[i].count);
}
str->count += str->fields[i].count;
}
if (dump_file)
{
fprintf (dump_file, "\nCounter of struct \"");
print_generic_expr (dump_file, str->decl, 0);
fprintf (dump_file, "\" is " HOST_WIDE_INT_PRINT_DEC, str->count);
}
if (str->count > hotest_struct_count)
hotest_struct_count = str->count;
tmp = tmp->next;
}
/* Compare counts of structures. */
tmp = data_struct_list;
/* Remove cold structs from data_struct_list. */
while (tmp)
{
d_str str = tmp->struct_data;
if (str->count < (hotest_struct_count / COLD_STRUCTURE_RATIO))
{
list tmp1 = tmp;
tmp = tmp->next;
remove_struct_from_data_struct_list (tmp1);
}
else
tmp = tmp->next;
}
}
/* This function peels structs from data_struct_list
into clusters. */
static void
peel_structs (void)
{
list tmp = data_struct_list;
while (tmp)
{
d_str str = tmp->struct_data;
peel_hot_fields (str);
tmp = tmp->next;
}
}
/* Collect allocation sites - mallocs. In case of arrays
we have nothing to do. */
static void
collect_allocation_sites (void)
{
collect_malloc_data ();
}
/* Free all data structures used in this optimization. */
static void
free_data_structs (void)
{
free_data_structure_list ();
free_malloc_data ();
}
/* Perform structure decomposition (peeling). */
static void
reorg_structs (void)
{
/* Stage1. */
/* Collect data struct types. */
collect_data_structs ();
collect_allocation_sites ();
/* Collect data accesses. */
collect_data_accesses ();
/* We transform only hot structs. */
exclude_cold_structs ();
/* Stage2. */
peel_structs ();
/* Stage3. */
/* Do the actual transformation. */
do_reorg ();
/* Free all data structures used in this optimization. */
free_data_structs ();
}
/* This function adds call to malloc, that appears in statement
STMT in function FN_DECL and allocates structures
described in STR, to malloc_data list. */
static void
add_call_to_malloc_list (tree fn_decl, tree stmt, d_str str)
{
malloc_d cur_malloc = NULL;
malloc_d malloc_node = NULL;
for (cur_malloc = malloc_data_list; cur_malloc;
cur_malloc = cur_malloc->next)
if (cur_malloc->func == fn_decl)
break;
if (cur_malloc)
{
malloc_call m_call = (malloc_call)
xmalloc (sizeof (struct malloc_call));
m_call->next = cur_malloc->malloc_list;
m_call->malloc_call_stmt = stmt;
m_call->str = str;
cur_malloc->malloc_list = m_call;
}
else
{
malloc_node = (malloc_d)
xmalloc (sizeof (struct malloc_struct));
malloc_node->func = fn_decl;
malloc_node->malloc_list = (malloc_call)
xmalloc (sizeof (struct malloc_call));
malloc_node->malloc_list->malloc_call_stmt = stmt;
malloc_node->malloc_list->str = str;
malloc_node->malloc_list->next = NULL;
malloc_node->next = malloc_data_list;
malloc_data_list = malloc_node;
}
if (dump_file)
{
fprintf (dump_file, "\nAdding stmt ");
print_generic_stmt (dump_file, stmt, 0);
fprintf (dump_file, " to list of mallocs.");
}
}
/* Struct-reorg optimization entry point function. */
static unsigned int
reorg_structs_drive (void)
{
reorg_structs ();
return 0;
}
/* Struct-roerg optimization gate function. */
static bool
struct_reorg_gate (void)
{
return flag_ipa_struct_reorg && flag_whole_program
&& (optimize > 0);
}
struct tree_opt_pass pass_ipa_struct_reorg =
{
"ipa_struct_reorg", /* name */
struct_reorg_gate, /* gate */
reorg_structs_drive, /* execute */
NULL, /* sub */
NULL, /* next */
0, /* static_pass_number */
TV_INTEGRATION, /* tv_id */
0, /* properties_required */
0, /* properties_provided */
0, /* properties_destroyed */
TODO_verify_ssa, /* todo_flags_start */
TODO_dump_func | TODO_verify_ssa, /* todo_flags_finish */
0 /* letter */
};
[-- Attachment #4: ipa-struct-reorg.h --]
[-- Type: application/octet-stream, Size: 3837 bytes --]
/* Struct-reorg optimization.
Copyright (C) 2002, 2003-2007 Free Software Foundation, Inc.
Contributed by Olga Golovanevsky <olga@il.ibm.com>
This file is part of GCC.
GCC is free software; you can redistribute it and/or modify
under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
GCC is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with GCC; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
#ifndef IPA_REORG_STRUCTS_H
#define IPA_REORG_STRUCTS_H
/* This file describes the data structure and interface for the data
reorganization optimization. */
struct struct_tree_list {
tree data;
struct struct_tree_list *next;
};
typedef struct struct_tree_list * tree_list;
/* Represents an access site to a data field.
We consider access of the following form:
D.2166_21 = i.6_20 * 8;
D.2167_22 = (struct str_t *) D.2166_21;
D.2168_24 = D.2167_22 + p.5_23;
D.2169_25 = D.2168_24->b;
*/
struct field_access_site
{
/* The statement in which the access site occurs. */
tree stmt; /* D.2169_25 = D.2168_24->b; */
tree comp_ref; /* D.2168_24->b */
tree field_decl; /* b */
tree ref; /* D.2168_24 */
tree num; /* i.6_20 */
tree offset; /* D2167_22 */
tree base; /* p.5_23 */
tree ref_def_stmt; /* D.2168_24 = D.2167_22 + p.5_23; */
tree cast_stmt; /* D.2167_22 = (struct str_t *) D.2166_21;
This stmt is not always present. */
struct field_access_site *next;
};
struct access_site
{
/* The statement in which the access site occurs. */
tree stmt;
/* List of struct vars in stmt. */
tree_list vars;
struct access_site *next;
};
/* Represents a field within a data structure and its accesses. */
struct data_field_entry {
/* Field index, that starts with zero. */
int index;
/* The number of times the field is accessed (according to profiling). */
gcov_type count;
tree decl;
/* Type of new structure this field belongs to. */
tree field_mapping;
struct field_access_site *acc_sites;
};
/* This structure represent result of peeling. */
struct field_cluster {
/* Bitmap of field indexes, a bit is set when the field with that index
is in the cluster. */
sbitmap fields_in_cluster;
struct field_cluster *sibling;
};
/* Represents a data structure that is candidate for
clustering. */
struct data_structure {
/* Main variant of type declaration of the structure. */
tree decl;
/* Number of fields in the structure. */
int num_fields;
/* According to profiling information number of times fields of the
structure were accessed. */
gcov_type count;
/* Array of field entries one for each field of the structure,
indexed by the field ID. */
struct data_field_entry *fields;
/* Stmts with struct accesses that are not a part of field
accesses or allocation patterns. */
struct access_site *accs;
/* This is the a data structure representing result of peeling. */
struct field_cluster *struct_clustering;
/* A linked list of the newly created types, corresponding to the
reorganization of the original structure. */
tree_list new_types;
};
typedef struct data_structure * d_str;
#endif /* IPA_REORG_STRUCTS_H */
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: struct-reorg optimization
2007-08-17 22:25 ` Jan Hubicka
2007-08-21 22:22 ` Olga Golovanevsky
2007-08-31 20:33 ` Olga Golovanevsky
@ 2007-09-05 11:48 ` Olga Golovanevsky
2 siblings, 0 replies; 17+ messages in thread
From: Olga Golovanevsky @ 2007-09-05 11:48 UTC (permalink / raw)
To: Jan Hubicka
Cc: Peter Bergner, Daniel Berlin, Diego Novillo, gcc-patches,
Jan Hubicka, Kenneth Zadeck
Jan Hubicka <hubicka@ucw.cz> wrote on 18/08/2007 01:24:44:
>
>
> /* Do transformation for a function represented by NODE. */
> static void
> do_reorg_for_func (struct cgraph_node *node)
> {
> create_new_local_vars ();
>
> create_new_mallocs_for_func (node);
> create_new_accesses_for_func ();
> update_ssa (TODO_update_ssa);
> cleanup_tree_cfg ();
>
> You should probably use function local cfun->todo lists. You care above
> to update cgraph edges, however ding update_ssa/cleanup_tree_cfg is
> going to disturb cgraph anyway, so you probably just want to add TODO
> flag for rebuilding cgraph edges as well.
I have checked whether the call to cleanup_tree_cfg is obligatory or not
for the changes the struct reorg makes. It looks like it's not.
The cleanup_tree_cfg does not cause the original malloc call to be removed,
thus making call to rebuild_cgraph_edges unnecessary in this case.
Olga
>
> Nice work and sorry for delays with the review!
> Honza
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: struct-reorg optimization
2007-08-31 20:33 ` Olga Golovanevsky
@ 2007-09-06 21:04 ` Diego Novillo
2007-10-21 19:58 ` Olga Golovanevsky
0 siblings, 1 reply; 17+ messages in thread
From: Diego Novillo @ 2007-09-06 21:04 UTC (permalink / raw)
To: Olga Golovanevsky
Cc: Jan Hubicka, Peter Bergner, Daniel Berlin, gcc-patches,
Jan Hubicka, Kenneth Zadeck
On 8/31/07, Olga Golovanevsky <OLGA@il.ibm.com> wrote:
> Contributed by Olga Golovanevsky <olga@il.ibm.com>
> (initial version of this code was developed
> by Caroline Tice and Mostafa Hagog)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Spacing.
> and str_t_1 as it was shown above, then array A will be replaced
> by two arrays as foolows:
s/foolows/follows/
> The field access of fields a of ellement i of array A: A[i].a will be
> replace by access to field a of ellement i of array A_0: A_0[i].a.
s/ellement/element/
s/replace/replaced/
s/by access/by an access/
> The decision how to peel structure gains to utilize spatial locality.
"The goal of structure peeling is to improve spatial locality."
> For example, if one of fields of structure has intensive use in the loop:
s/if one of/if one of the/
s/of structure/of a structure/
s/has intensive use in the loop/is accessed frequently in the loop:/
> the allocation of field a of str_t in the memory in contiguous manner will
s/in the memory in contiguous manner/contiguously in memory/
> increase probablity of relevant data to be fatched into cache.
"increase the chances of fetching the field from cache."
> got peeled out of the structure:
s/got/get/
> freq(f) > C * max_field_freq_in_struct
>
> where max_field_freq_in_struct is maximum field frequency in structure.
s/is maximum/is the maximum/
s/in structure/in the structure/
> The C is a constant defining which portion of max_field_freq_in_struct
> the filds should have in order to be peeled.
s/The C/C/
s/the filds/the fields/
> If profiling information is provided, its is used to calculate the
s/its is/it is/
> frequency of field accesses. Otherwise, structure is fully peeled.
/structure is/the structure is/
Oh? Don't we use estimated execution frequencies to compute this?
>
> The ipa-type-escape analysis are used to provide safety of the peeling.
"IPA type-escape analysis is used to determine when it is safe to peel
a structure."
> The optimization is activated by flag -fipa-struct-reorg.
> */
Closing comment on previous line.
> /* Ratio of the structure count to the hottest
> structure count to consider the structure cold. */
> #define COLD_STRUCTURE_RATIO 10
Better convert into a --param.
> struct new_var_data {
> [ ... ]
> struct new_var_data *next;
> };
Why not use a VEC and avoid having to manage the single-linked list
yourself? Similarly with structures malloc_call, malloc_struct and
struct_list.
> static new_var is_in_new_vars_list (tree, new_var);
> static void add_struct_to_data_struct_list (tree);
> static void add_call_to_malloc_list (tree, tree, d_str);
> static inline void update_fields_mapping (struct field_cluster *, tree,
> struct data_field_entry *, int);
> static void create_new_var (tree, new_var *);
> static tree get_final_malloc_stmt (tree malloc_stmt);
> static tree_list add_tree_to_tree_list (tree, tree_list);
> static void free_tree_list (tree_list);
> static malloc_d get_malloc_list_for_func (tree);
> static struct field_access_site *
> add_field_acc_to_acc_sites (struct field_access_site *,
> struct field_access_site *);
> static struct access_site * add_access_to_acc_sites (tree, tree,
> struct access_site *);
> static bool is_result_of_mult (tree, tree *, tree);
> static void remove_struct_from_data_struct_list (list);
> static tree gen_size (tree, tree, tree *);
> static tree gen_cast_stmt (tree, tree, tree, tree *);
> static void create_new_stmts_for_general_acc (struct access_site *, d_str);
> static void create_new_stmts_for_cond_expr (tree);
> static bool is_equal_types (tree, tree);
> static tree * find_pos_in_stmt (tree, tree);
> static void free_data_structure_list (void);
I'd rather see the functions defined in such a way that you don't need
forward declarations. This forces the main functions to be grouped at
the bottom of the file, making it easier to find them later. It also
simplifies changing function signatures in the future.
Not a requirement, though. That's mostly personal preference.
> if (TREE_CODE (var) == PARM_DECL)
> return DECL_ARG_TYPE (var);
>
> else
> return TREE_TYPE (var);
> }
Spacing.
> /* Check whether the type of VAR is potential candidate for peeling.
> Returns true if yes, false otherwise. If yes, TYPE_P contains
> candidate type. */
> static bool
> is_candidate_for_data_struct_list (tree var, tree *type_p,
> bitmap non_suitable_types)
No documentation for NON_SUITABLE_TYPES.
> static const char *
> get_type_name (tree type_node)
< [ ... ]
> if (TREE_CODE (TYPE_NAME (type_node)) == IDENTIFIER_NODE)
> return (char *) IDENTIFIER_POINTER (TYPE_NAME (type_node));
s/char *//
> else if (TREE_CODE (TYPE_NAME (type_node)) == TYPE_DECL
> && DECL_NAME (TYPE_NAME (type_node)))
> return (char *) IDENTIFIER_POINTER (DECL_NAME (TYPE_NAME (type_node)));
> /* Print structure type , its name, if it exists, and body. */
> static void
> print_struct_type (tree type, char *indent)
> {
s/type , /TYPE, /
No documentation for INDENT.
> /* Given structure type SRT_TYPE and field FIELD,
> this funciton is looking for a field with the same name
s/funciton/function/
> char * str_field_name;
> char * field_name;
const char *
> if (!strcmp (str_field_name, field_name))
> {
Why? Wouldn't is_equal_types() be enough here?
> /* This function is a hack to overcome the types problem.
> When number of compilation units are compiled together
s/number of/several/
> under -combine flag, the TYPE_MAIN_VARIANT cannot
s/under -combine flag/with -combine/
> be trusted. First we comparing names, but they do not
> always appear, like for example, in array_refs.
I'm not sure what this means. Could you re-phrase?
> Only then we are going inside structures.
> */
Closing comment on previous line.
> static inline struct field_access_site *
> make_field_acc_node (void)
> {
> return (struct field_access_site *)
> xcalloc (1, sizeof (struct field_access_site));
> }
Indentation.
> *walk_subtrees=1;
> [ ... ]
> *walk_subtrees=0;
Spacing between '='. Happens elsewhere too.
> }
> break;
> case VAR_DECL:
> case SSA_NAME:
Spacing after 'break;'.
> /* In asm stmt we cannot always track the agruments,
> so we just give up. */
> if (TREE_CODE (stmt) == ASM_EXPR)
> free_data_structure_list ();
s/agruments/arguments/
If we are giving up, why does the loop keep going?
> static tree
> build_basic_struct (tree fields, char * name, tree orig_struct)
> {
>
> tree attributes = NULL_TREE;
No spacing after '{'.
> gen_cluster_name (tree decl, int clust_num, int str_num)
> {
> char * old_name = (char *) get_type_name (decl);
Better declare old_name as 'const char *'.
> /* Generate new structure name. */
> static inline char *
> gen_cluster_name (tree decl, int clust_num, int str_num)
> {
> char * old_name = (char *) get_type_name (decl);
> char * new_name;
>
> if (!old_name)
> {
> old_name = (char *) xmalloc ((10) * sizeof (char));
> sprintf (old_name, "struct_%d", str_num);
> }
>
> new_name = (char *) xmalloc ((strlen (old_name) + 11) * sizeof (char));
> sprintf (new_name, "%s_sub_%d", old_name, clust_num);
>
> /* Check that this name was not used before. */
> while (maybe_get_identifier (new_name))
> {
> int len = strlen (new_name) + 3;
> char * tmp_name = (char *) xmalloc (len * sizeof (char));
> sprintf (tmp_name, "%s.0", new_name);
> new_name = tmp_name;
> }
>
> return new_name;
> }
No. Better use alloca and make sure there are no possibility of
buffer overflow. This function leaks memory as well. See
create_omp_child_function_name and create_tmp_var_name.
Similarly, in gen_var_name. Why not use create_tmp_var_name?
> /* Update field_mapping of fields in CLUSTER with NEW_TYPE. */
> static inline void
> update_fields_mapping (struct field_cluster *cluster, tree new_type,
> struct data_field_entry * fields, int num_fields)
No documentation for FIELDS and NUM_FIELDS.
> static struct field_access_site *
> is_in_field_acc_list (tree stmt, struct field_access_site *list_p)
> {
> struct field_access_site *curr;
>
> for (curr = list_p; curr; curr = curr->next)
> if (curr->stmt == stmt)
> return curr;
Would it make sense to keep a pointer-map or a hashtable to avoid
these linear searches?
> static struct access_site *
> is_in_acc_list (tree stmt, struct access_site *list_p)
> {
> struct access_site *curr;
>
> for (curr = list_p; curr; curr = curr->next)
> if (curr->stmt == stmt)
> return curr;
Likewise here.
> struct type_wrapper
> {
> /* 0 stand for pointer wrapper,
> and 1 for array wrapper. */
> bool wrap;
> tree domain; /* Relevant for arrays as domain or index. */
> struct type_wrapper * next;
> };
Similar comment as the other structures. It seems easier to put these
inside VECs instead of having to manage the single-linked lists on
your own.
> static void
> create_new_local_vars (void)
> {
This function looks like it's doing too much work. It's going over
the same DECLs over and over again. Why not just traverse
referenced_vars?
> is converted into followinf set of stmts:
s/into followinf/into the following/
> static void
> add_bb_info (basic_block bb, tree stmt_list)
> {
> if (TREE_CODE (stmt_list) == STATEMENT_LIST)
> {
> tree_stmt_iterator tsi;
> for (tsi = tsi_start (stmt_list); !tsi_end_p (tsi);
> tsi_next (&tsi))
tsi_next() fits in the previous line.
> /* This function returns a tree representing
> the number of structure instances allocated by MALLOC_STMT. */
> gen_num_of_structs_in_malloc (tree stmt, tree str_decl,
> tree *new_stmts_p)
new_stmts_p fits in the previous line.
Missing documentation for NEW_STMTS_P and STR_DECL. I suppose that
MALLOC_STMT is STMT?
> /* Get malloc agrument. */
s/agrument/argument/
> /* Generate argument to malloc as multiplication of num and size of new_type. */
Line's too long.
> /* This function build an edge between BB and E->dest and updates
s/build/builds/
> /* If MALLOC_STMT is D.2225_7 = malloc (D.2224_6);
> it can be casting stmt as for example:
> p_8 = (struct str_t *) D.2225_7;
> which is returned by this function. */
It's not clear what this means. Could you rephrase? Perhaps
something like "Return the first statement that uses the result of the
call to MALLOC_STMT."
> return build3 (COMPONENT_REF, TREE_TYPE (field),
> base, field, NULL_TREE);
No need to wrap this line.
> struct type_wrapper * wrapper = NULL;
> struct type_wrapper * wr = NULL;
No space after '*'.
> if (TREE_CODE (acc->stmt) == GIMPLE_MODIFY_STMT)
> {
>
> lhs = GIMPLE_STMT_OPERAND (acc->stmt, 0);
No space after '{'
> /* In this function we create new stmts that has the same
> form as ORIG_STMT, but of the type NET_TYPE. The stmts
s/stmts/statements/
s/that has/that have/
s/of the type/of type/
> treated by this function are simple assignments,
> like assignments: p.8_7 = p; or stmts with rhs of code
> PLUS_EXPR or MINUS_EXPR. */
> static tree
> create_base_plus_offset (tree orig_stmt, tree new_type,
> tree offset)
> {
> [ ... ]
> gcc_assert ( tmp0 || tmp1);
No space after '('.
> if (!new_op0)
> new_op0 = offset;
> if (!new_op1)
> new_op1 = offset;
>
> new_rhs = build2 (TREE_CODE (rhs), TREE_TYPE (new_op0),
> new_op0, new_op1);
If RHS is a POINTER_PLUS_EXPR and new_op0 is OFFSET, you'll likely
create an invalid expression here.
> new_stmt = build2 (GIMPLE_MODIFY_STMT, TREE_TYPE (new_lhs),
> new_lhs, new_rhs);
Statements have no type. That should be void_type_node. Similarly in
other places where assignments are created.
> static tree
> gen_cast_stmt (tree before_cast, tree new_type, tree old_cast_stmt, tree *res_p)
This line is too long.
> static void
> create_new_field_access (struct field_access_site *acc, struct data_field_entry field)
Likewise.
> /* This function creates new non-field accesses for
> each new type in STR. */
> static void
> create_new_stmts_for_general_acc (struct access_site * acc, d_str str)
> {
Missing documentation for ACC.
> /* This function creates new condition stmt and
> redirect edges to correspondent BBs.
> NEW_VAR position in stmt is POS. */
> static void
> create_new_stmts_for_cond_expr_1 (tree new_var, tree cond_stmt, bool pos)
> {
Missing documentation for POS and COND_STMT.
s/correspondent/corresponding/
> new_cond = copy_node (COND_EXPR_COND (cond_stmt));
>
Better call unshare_expr() here. Similarly in other calls to
copy_node that copy a statement.
> new_stmt = build3 (COND_EXPR,TREE_TYPE (cond_stmt),
> new_cond, NULL_TREE, NULL_TREE);
Space after first comma.
> if (!list_p)
> return NULL;
> else
> {
> node = list_p;
> while (node)
> {
> if (node->orig_var == decl)
> return node;
> node = node->next;
> }
> }
> return NULL;
As before. Wouldn't it be better to implement this on a pointer-map
or hash table?
> /* Stage 3. */
> static void
> do_reorg (void)
Stage 3? What does that mean?
> /* Do reorg for each funciton separately. */
s/funciton/function/
Several functions have documentation missing for arguments. Also,
there should be a blank line between function comments and the start
of the function.
All in all, the patch seems fine. The major things I noticed were
the, perhaps unnecessary, use of manually-maintained single-linked
lists and the O(N) searching. Those could be solved with VECs and
pointer-maps/hashtables.
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: struct-reorg optimization
2007-09-06 21:04 ` Diego Novillo
@ 2007-10-21 19:58 ` Olga Golovanevsky
2007-10-21 22:01 ` Diego Novillo
0 siblings, 1 reply; 17+ messages in thread
From: Olga Golovanevsky @ 2007-10-21 19:58 UTC (permalink / raw)
To: Diego Novillo
Cc: Peter Bergner, Daniel Berlin, gcc-patches, Jan Hubicka,
Jan Hubicka, Kenneth Zadeck
[-- Attachment #1: Type: text/plain, Size: 16625 bytes --]
"Diego Novillo" <dnovillo@google.com> wrote on 06/09/2007 23:43:00:
> On 8/31/07, Olga Golovanevsky <OLGA@il.ibm.com> wrote:
>
> > Contributed by Olga Golovanevsky <olga@il.ibm.com>
> > (initial version of this code was developed
> > by Caroline Tice and Mostafa Hagog)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> Spacing.
done.
>
> > and str_t_1 as it was shown above, then array A will be replaced
> > by two arrays as foolows:
>
> s/foolows/follows/
done.
>
> > The field access of fields a of ellement i of array A: A[i].a will be
> > replace by access to field a of ellement i of array A_0: A_0[i].a.
>
> s/ellement/element/
> s/replace/replaced/
> s/by access/by an access/
>
done.
>
> > The decision how to peel structure gains to utilize spatial locality.
>
> "The goal of structure peeling is to improve spatial locality."
done.
>
> > For example, if one of fields of structure has intensive use in the
loop:
>
> s/if one of/if one of the/
> s/of structure/of a structure/
> s/has intensive use in the loop/is accessed frequently in the loop:/
>
done.
> > the allocation of field a of str_t in the memory in contiguous
> manner will
>
> s/in the memory in contiguous manner/contiguously in memory/
done.
>
> > increase probablity of relevant data to be fatched into cache.
>
> "increase the chances of fetching the field from cache."
done.
>
> > got peeled out of the structure:
>
> s/got/get/
done.
>
>
> > freq(f) > C * max_field_freq_in_struct
> >
> > where max_field_freq_in_struct is maximum field frequency in
structure.
>
> s/is maximum/is the maximum/
> s/in structure/in the structure/
done.
>
> > The C is a constant defining which portion of
max_field_freq_in_struct
> > the filds should have in order to be peeled.
>
> s/The C/C/
> s/the filds/the fields/
>
done.
> > If profiling information is provided, its is used to calculate the
>
> s/its is/it is/
done.
>
> > frequency of field accesses. Otherwise, structure is fully peeled.
>
> /structure is/the structure is/
done.
>
> Oh? Don't we use estimated execution frequencies to compute this?
We would want to do it. As I have learnt form Jan, the estimated execution
frequencies cannot be used at ipa level right now.
>
> >
> > The ipa-type-escape analysis are used to provide safety of the
peeling.
>
> "IPA type-escape analysis is used to determine when it is safe to peel
> a structure."
>
done.
>
> > The optimization is activated by flag -fipa-struct-reorg.
> > */
>
> Closing comment on previous line.
done.
>
>
> > /* Ratio of the structure count to the hottest
> > structure count to consider the structure cold. */
> > #define COLD_STRUCTURE_RATIO 10
>
> Better convert into a --param.
done.
>
>
> > struct new_var_data {
> > [ ... ]
> > struct new_var_data *next;
> > };
>
> Why not use a VEC and avoid having to manage the single-linked list
> yourself? Similarly with structures malloc_call, malloc_struct and
> struct_list.
done.
>
> > static new_var is_in_new_vars_list (tree, new_var);
> > static void add_struct_to_data_struct_list (tree);
> > static void add_call_to_malloc_list (tree, tree, d_str);
> > static inline void update_fields_mapping (struct field_cluster *, tree,
> > struct data_field_entry *, int);
> > static void create_new_var (tree, new_var *);
> > static tree get_final_malloc_stmt (tree malloc_stmt);
> > static tree_list add_tree_to_tree_list (tree, tree_list);
> > static void free_tree_list (tree_list);
> > static malloc_d get_malloc_list_for_func (tree);
> > static struct field_access_site *
> > add_field_acc_to_acc_sites (struct field_access_site *,
> > struct field_access_site *);
> > static struct access_site * add_access_to_acc_sites (tree, tree,
> > struct access_site *);
> > static bool is_result_of_mult (tree, tree *, tree);
> > static void remove_struct_from_data_struct_list (list);
> > static tree gen_size (tree, tree, tree *);
> > static tree gen_cast_stmt (tree, tree, tree, tree *);
> > static void create_new_stmts_for_general_acc (struct access_site *,
d_str);
> > static void create_new_stmts_for_cond_expr (tree);
> > static bool is_equal_types (tree, tree);
> > static tree * find_pos_in_stmt (tree, tree);
> > static void free_data_structure_list (void);
>
> I'd rather see the functions defined in such a way that you don't need
> forward declarations. This forces the main functions to be grouped at
> the bottom of the file, making it easier to find them later. It also
> simplifies changing function signatures in the future.
>
done, except mutually calling functions.
> Not a requirement, though. That's mostly personal preference.
>
>
>
> > if (TREE_CODE (var) == PARM_DECL)
> > return DECL_ARG_TYPE (var);
> >
> > else
> > return TREE_TYPE (var);
> > }
>
> Spacing.
done.
>
> > /* Check whether the type of VAR is potential candidate for peeling.
> > Returns true if yes, false otherwise. If yes, TYPE_P contains
> > candidate type. */
> > static bool
> > is_candidate_for_data_struct_list (tree var, tree *type_p,
> > bitmap non_suitable_types)
>
> No documentation for NON_SUITABLE_TYPES.
done.
>
> > static const char *
> > get_type_name (tree type_node)
> < [ ... ]
> > if (TREE_CODE (TYPE_NAME (type_node)) == IDENTIFIER_NODE)
> > return (char *) IDENTIFIER_POINTER (TYPE_NAME (type_node));
>
> s/char *//
done.
>
> > else if (TREE_CODE (TYPE_NAME (type_node)) == TYPE_DECL
> > && DECL_NAME (TYPE_NAME (type_node)))
> > return (char *) IDENTIFIER_POINTER (DECL_NAME (TYPE_NAME
(type_node)));
>
> > /* Print structure type , its name, if it exists, and body. */
> > static void
> > print_struct_type (tree type, char *indent)
> > {
>
> s/type , /TYPE, /
done.
>
> No documentation for INDENT.
>
done.
> > /* Given structure type SRT_TYPE and field FIELD,
> > this funciton is looking for a field with the same name
>
> s/funciton/function/
done.
>
> > char * str_field_name;
> > char * field_name;
>
> const char *
done.
>
> > if (!strcmp (str_field_name, field_name))
> > {
>
> Why? Wouldn't is_equal_types() be enough here?
It is for fields. They can be of the same type (for example, int),
but have different names ('a' and 'b').
>
> > /* This function is a hack to overcome the types problem.
> > When number of compilation units are compiled together
>
> s/number of/several/
done.
>
> > under -combine flag, the TYPE_MAIN_VARIANT cannot
>
> s/under -combine flag/with -combine/
done.
>
> > be trusted. First we comparing names, but they do not
> > always appear, like for example, in array_refs.
>
> I'm not sure what this means. Could you re-phrase?
>
done.
> > Only then we are going inside structures.
> > */
>
> Closing comment on previous line.
done.
>
> > static inline struct field_access_site *
> > make_field_acc_node (void)
> > {
> > return (struct field_access_site *)
> > xcalloc (1, sizeof (struct field_access_site));
> > }
>
> Indentation.
done.
>
> > *walk_subtrees=1;
> > [ ... ]
> > *walk_subtrees=0;
>
> Spacing between '='. Happens elsewhere too.
done.
>
> > }
> > break;
> > case VAR_DECL:
> > case SSA_NAME:
>
> Spacing after 'break;'.
done.
>
> > /* In asm stmt we cannot always track the agruments,
> > so we just give up. */
> > if (TREE_CODE (stmt) == ASM_EXPR)
> > free_data_structure_list ();
>
> s/agruments/arguments/
>
> If we are giving up, why does the loop keep going?
done.
>
> > static tree
> > build_basic_struct (tree fields, char * name, tree orig_struct)
> > {
> >
> > tree attributes = NULL_TREE;
>
> No spacing after '{'.
done.
>
> > gen_cluster_name (tree decl, int clust_num, int str_num)
> > {
> > char * old_name = (char *) get_type_name (decl);
>
> Better declare old_name as 'const char *'.
>
>
> > /* Generate new structure name. */
> > static inline char *
> > gen_cluster_name (tree decl, int clust_num, int str_num)
> > {
> > char * old_name = (char *) get_type_name (decl);
> > char * new_name;
> >
> > if (!old_name)
> > {
> > old_name = (char *) xmalloc ((10) * sizeof (char));
> > sprintf (old_name, "struct_%d", str_num);
> > }
> >
> > new_name = (char *) xmalloc ((strlen (old_name) + 11) * sizeof
(char));
> > sprintf (new_name, "%s_sub_%d", old_name, clust_num);
> >
> > /* Check that this name was not used before. */
> > while (maybe_get_identifier (new_name))
> > {
> > int len = strlen (new_name) + 3;
> > char * tmp_name = (char *) xmalloc (len * sizeof (char));
> > sprintf (tmp_name, "%s.0", new_name);
> > new_name = tmp_name;
> > }
> >
> > return new_name;
> > }
>
> No. Better use alloca and make sure there are no possibility of
> buffer overflow. This function leaks memory as well. See
> create_omp_child_function_name and create_tmp_var_name.
>
> Similarly, in gen_var_name. Why not use create_tmp_var_name?
>
Both gen_cluster_name and gen_var_name are re-written to be similar to
create_omp_child_function_name.
> > /* Update field_mapping of fields in CLUSTER with NEW_TYPE. */
> > static inline void
> > update_fields_mapping (struct field_cluster *cluster, tree new_type,
> > struct data_field_entry * fields, int num_fields)
>
> No documentation for FIELDS and NUM_FIELDS.
done.
>
> > static struct field_access_site *
> > is_in_field_acc_list (tree stmt, struct field_access_site *list_p)
> > {
> > struct field_access_site *curr;
> >
> > for (curr = list_p; curr; curr = curr->next)
> > if (curr->stmt == stmt)
> > return curr;
>
> Would it make sense to keep a pointer-map or a hashtable to avoid
> these linear searches?
done.
>
> > static struct access_site *
> > is_in_acc_list (tree stmt, struct access_site *list_p)
> > {
> > struct access_site *curr;
> >
> > for (curr = list_p; curr; curr = curr->next)
> > if (curr->stmt == stmt)
> > return curr;
>
> Likewise here.
done.
>
> > struct type_wrapper
> > {
> > /* 0 stand for pointer wrapper,
> > and 1 for array wrapper. */
> > bool wrap;
> > tree domain; /* Relevant for arrays as domain or index. */
> > struct type_wrapper * next;
> > };
>
> Similar comment as the other structures. It seems easier to put these
> inside VECs instead of having to manage the single-linked lists on
> your own.
done.
>
> > static void
> > create_new_local_vars (void)
> > {
>
> This function looks like it's doing too much work. It's going over
> the same DECLs over and over again. Why not just traverse
> referenced_vars?
>
done.
>
> > is converted into followinf set of stmts:
>
> s/into followinf/into the following/
done.
>
>
> > static void
> > add_bb_info (basic_block bb, tree stmt_list)
> > {
> > if (TREE_CODE (stmt_list) == STATEMENT_LIST)
> > {
> > tree_stmt_iterator tsi;
> > for (tsi = tsi_start (stmt_list); !tsi_end_p (tsi);
> > tsi_next (&tsi))
>
> tsi_next() fits in the previous line.
done.
>
> > /* This function returns a tree representing
> > the number of structure instances allocated by MALLOC_STMT. */
> > gen_num_of_structs_in_malloc (tree stmt, tree str_decl,
> > tree *new_stmts_p)
>
> new_stmts_p fits in the previous line.
done.
>
> Missing documentation for NEW_STMTS_P and STR_DECL. I suppose that
> MALLOC_STMT is STMT?
done.
>
>
> > /* Get malloc agrument. */
>
> s/agrument/argument/
done.
>
>
> > /* Generate argument to malloc as multiplication of num and size
> of new_type. */
>
> Line's too long.
done.
>
> > /* This function build an edge between BB and E->dest and updates
>
> s/build/builds/
done.
>
>
> > /* If MALLOC_STMT is D.2225_7 = malloc (D.2224_6);
> > it can be casting stmt as for example:
> > p_8 = (struct str_t *) D.2225_7;
> > which is returned by this function. */
>
> It's not clear what this means. Could you rephrase? Perhaps
> something like "Return the first statement that uses the result of the
> call to MALLOC_STMT."
done.
>
> > return build3 (COMPONENT_REF, TREE_TYPE (field),
> > base, field, NULL_TREE);
>
> No need to wrap this line.
done.
>
> > struct type_wrapper * wrapper = NULL;
> > struct type_wrapper * wr = NULL;
>
> No space after '*'.
done.
>
> > if (TREE_CODE (acc->stmt) == GIMPLE_MODIFY_STMT)
> > {
> >
> > lhs = GIMPLE_STMT_OPERAND (acc->stmt, 0);
>
> No space after '{'
done.
>
> > /* In this function we create new stmts that has the same
> > form as ORIG_STMT, but of the type NET_TYPE. The stmts
>
> s/stmts/statements/
> s/that has/that have/
> s/of the type/of type/
done.
>
> > treated by this function are simple assignments,
> > like assignments: p.8_7 = p; or stmts with rhs of code
> > PLUS_EXPR or MINUS_EXPR. */
> > static tree
> > create_base_plus_offset (tree orig_stmt, tree new_type,
> > tree offset)
> > {
> > [ ... ]
> > gcc_assert ( tmp0 || tmp1);
>
> No space after '('.
done.
>
> > if (!new_op0)
> > new_op0 = offset;
> > if (!new_op1)
> > new_op1 = offset;
> >
> > new_rhs = build2 (TREE_CODE (rhs), TREE_TYPE (new_op0),
> > new_op0, new_op1);
>
> If RHS is a POINTER_PLUS_EXPR and new_op0 is OFFSET, you'll likely
> create an invalid expression here.
true. If RHS is a POINTER_PLUS_EXPR, then new_op0 is found by
find_new_var_of_type, and won't be equal to OFFSET.
>
>
> > new_stmt = build2 (GIMPLE_MODIFY_STMT, TREE_TYPE (new_lhs),
> > new_lhs, new_rhs);
>
> Statements have no type. That should be void_type_node. Similarly in
> other places where assignments are created.
changed to use build_gimple_modify_stmt ().
>
> > static tree
> > gen_cast_stmt (tree before_cast, tree new_type, tree
> old_cast_stmt, tree *res_p)
>
> This line is too long.
done.
>
>
> > static void
> > create_new_field_access (struct field_access_site *acc, struct
> data_field_entry field)
>
> Likewise.
done.
>
>
>
> > /* This function creates new non-field accesses for
> > each new type in STR. */
> > static void
> > create_new_stmts_for_general_acc (struct access_site * acc, d_str str)
> > {
>
> Missing documentation for ACC.
done.
>
> > /* This function creates new condition stmt and
> > redirect edges to correspondent BBs.
> > NEW_VAR position in stmt is POS. */
> > static void
> > create_new_stmts_for_cond_expr_1 (tree new_var, tree cond_stmt, bool
pos)
> > {
>
> Missing documentation for POS and COND_STMT.
done.
>
> s/correspondent/corresponding/
done.
>
> > new_cond = copy_node (COND_EXPR_COND (cond_stmt));
> >
>
> Better call unshare_expr() here. Similarly in other calls to
> copy_node that copy a statement.
done.
>
> > new_stmt = build3 (COND_EXPR,TREE_TYPE (cond_stmt),
> > new_cond, NULL_TREE, NULL_TREE);
>
> Space after first comma.
done.
>
> > if (!list_p)
> > return NULL;
> > else
> > {
> > node = list_p;
> > while (node)
> > {
> > if (node->orig_var == decl)
> > return node;
> > node = node->next;
> > }
> > }
> > return NULL;
>
> As before. Wouldn't it be better to implement this on a pointer-map
> or hash table?
done.
>
> > /* Stage 3. */
> > static void
> > do_reorg (void)
>
> Stage 3? What does that mean?
a comment is added.
>
>
> > /* Do reorg for each funciton separately. */
>
> s/funciton/function/
done.
>
>
> Several functions have documentation missing for arguments. Also,
> there should be a blank line between function comments and the start
> of the function.
done.
>
> All in all, the patch seems fine. The major things I noticed were
> the, perhaps unnecessary, use of manually-maintained single-linked
> lists and the O(N) searching. Those could be solved with VECs and
> pointer-maps/hashtables.
no link lists any more, only vectors and hashtables.
Bootstrapped on ppc970.
Ok for mainline?
Olga
ChangeLog
2007-10-21 Olga Golovanevsky <olga@il.ibm.com>
* ipa-struct-reorg.c, ipa-struct-reorg.h: New files.
* tree-pass.h: Add pass_ipa_struct_reorg.
* common.opt: Add ipa-struct-reorg flag.
* Makefile.in: Add ipa-strcut-reorg.o compilation.
* passes.c: Add pass pass_ipa_struct_reorg.
* params.h: Add STRUCT_REORG_COLD_STRUCT_RATIO.
* params.def: Add PARAM_STRUCT_REORG_COLD_STRUCT_RATIO.
(See attached file: ipa-struct-reorg.txt)(See attached file:
ipa-struct-reorg.h)(See attached file: ipa-struct-reorg.c)
[-- Attachment #2: ipa-struct-reorg.txt --]
[-- Type: text/plain, Size: 4414 bytes --]
Index: tree-pass.h
===================================================================
--- tree-pass.h (revision 129216)
+++ tree-pass.h (working copy)
@@ -340,6 +340,7 @@
extern struct tree_opt_pass pass_ipa_pure_const;
extern struct tree_opt_pass pass_ipa_type_escape;
extern struct tree_opt_pass pass_ipa_pta;
+extern struct tree_opt_pass pass_ipa_struct_reorg;
extern struct tree_opt_pass pass_early_local_passes;
extern struct tree_opt_pass pass_ipa_increase_alignment;
extern struct tree_opt_pass pass_ipa_function_and_variable_visibility;
Index: params.h
===================================================================
--- params.h (revision 129216)
+++ params.h (working copy)
@@ -103,6 +103,8 @@
PARAM_VALUE (PARAM_SRA_MAX_STRUCTURE_COUNT)
#define SRA_FIELD_STRUCTURE_RATIO \
PARAM_VALUE (PARAM_SRA_FIELD_STRUCTURE_RATIO)
+#define STRUCT_REORG_COLD_STRUCT_RATIO \
+ PARAM_VALUE (PARAM_STRUCT_REORG_COLD_STRUCT_RATIO)
#define MAX_INLINE_INSNS_SINGLE \
PARAM_VALUE (PARAM_MAX_INLINE_INSNS_SINGLE)
#define MAX_INLINE_INSNS \
Index: common.opt
===================================================================
--- common.opt (revision 129216)
+++ common.opt (working copy)
@@ -618,6 +618,11 @@
Perform matrix layout flattening and transposing based
on profiling information.
+fipa-struct-reorg
+Common Report Var(flag_ipa_struct_reorg)
+Perform structure layout optimizations based
+on profiling information.
+
fivopts
Common Report Var(flag_ivopts) Init(1) Optimization
Optimize induction variables on trees
Index: Makefile.in
===================================================================
--- Makefile.in (revision 129216)
+++ Makefile.in (working copy)
@@ -1235,6 +1235,7 @@
ipa-prop.o \
ipa-pure-const.o \
ipa-reference.o \
+ ipa-struct-reorg.o \
ipa-type-escape.o \
ipa-utils.o \
ipa.o \
@@ -2514,6 +2515,10 @@
pointer-set.h $(GGC_H) $(IPA_TYPE_ESCAPE_H) $(IPA_UTILS_H) $(C_COMMON_H) \
$(TREE_GIMPLE_H) $(CGRAPH_H) output.h $(FLAGS_H) tree-pass.h \
$(DIAGNOSTIC_H) $(FUNCTION_H)
+ipa-struct-reorg.o: ipa-struct-reorg.c ipa-struct-reorg.h $(CONFIG_H) $(SYSTEM_H) \
+ coretypes.h $(TM_H) $(RTL_H) $(TREE_H) $(FLAGS_H) output.h $(REGS_H) \
+ $(EXPR_H) $(FUNCTION_H) toplev.h $(GGC_H) $(TARGET_H) langhooks.h $(COVERAGE_H) \
+ libfuncs.h gt-coverage.h $(HASHTAB_H) $(IPA_TYPE_ESCAPE_H)
coverage.o : coverage.c $(GCOV_IO_H) $(CONFIG_H) $(SYSTEM_H) coretypes.h \
$(TM_H) $(RTL_H) $(TREE_H) $(FLAGS_H) output.h $(REGS_H) $(EXPR_H) \
@@ -3086,7 +3091,7 @@
$(srcdir)/reload.h $(srcdir)/caller-save.c \
$(srcdir)/alias.c $(srcdir)/bitmap.c $(srcdir)/cselib.c $(srcdir)/cgraph.c \
$(srcdir)/ipa-prop.c $(srcdir)/ipa-cp.c $(srcdir)/ipa-inline.c $(srcdir)/matrix-reorg.c \
- $(srcdir)/dbxout.c $(srcdir)/dwarf2out.c $(srcdir)/dwarf2asm.c \
+ $(srcdir)/dbxout.c $(srcdir)/ipa-struct-reorg.c $(srcdir)/dwarf2out.c $(srcdir)/dwarf2asm.c \
$(srcdir)/dojump.c \
$(srcdir)/emit-rtl.c $(srcdir)/except.c $(srcdir)/explow.c $(srcdir)/expr.c \
$(srcdir)/function.c $(srcdir)/except.h \
Index: passes.c
===================================================================
--- passes.c (revision 129216)
+++ passes.c (working copy)
@@ -546,6 +546,7 @@
NEXT_PASS (pass_ipa_pure_const);
NEXT_PASS (pass_ipa_type_escape);
NEXT_PASS (pass_ipa_pta);
+ NEXT_PASS (pass_ipa_struct_reorg);
*p = NULL;
/* These passes are run after IPA passes on every function that is being
Index: params.def
===================================================================
--- params.def (revision 129216)
+++ params.def (working copy)
@@ -83,6 +83,16 @@
"The threshold ratio between instantiated fields and the total structure size",
75, 0, 100)
+/* The threshold ratio between current and hotest structure counts.
+ We say that if the ratio of the current structure count,
+ calculated by profiling, to the hotest structure count
+ in the program is less than this parameter, then structure
+ reorganization is not applied. The default is 10%. */
+DEFPARAM (PARAM_STRUCT_REORG_COLD_STRUCT_RATIO,
+ "struct-reorg-cold-struct-ratio",
+ "The threshold ratio between current and hotest structure counts",
+ 10, 0, 100)
+
/* The single function inlining limit. This is the maximum size
of a function counted in internal gcc instructions (not in
real machine instructions) that is eligible for inlining
[-- Attachment #3: ipa-struct-reorg.h --]
[-- Type: application/octet-stream, Size: 3490 bytes --]
/* Struct-reorg optimization.
Copyright (C) 2002, 2003-2007 Free Software Foundation, Inc.
Contributed by Olga Golovanevsky <olga@il.ibm.com>
This file is part of GCC.
GCC is free software; you can redistribute it and/or modify
under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
GCC is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with GCC; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
02110-1301, USA. */
#ifndef IPA_STRUCT_REORG_H
#define IPA_STRUCT_REORG_H
/* This file contains data structures and interfaces required
for struct-reorg optimizations. */
/* An access site of the structure field.
We consider an access to be of the following form:
D.2166_21 = i.6_20 * 8;
D.2167_22 = (struct str_t *) D.2166_21;
D.2168_24 = D.2167_22 + p.5_23;
D.2169_25 = D.2168_24->b;
*/
struct field_access_site
{
/* Statement in which the access site occurs. */
tree stmt; /* D.2169_25 = D.2168_24->b; */
tree comp_ref; /* D.2168_24->b */
tree field_decl; /* b */
tree ref; /* D.2168_24 */
tree num; /* i.6_20 */
tree offset; /* D2167_22 */
tree base; /* p.5_23 */
tree ref_def_stmt; /* D.2168_24 = D.2167_22 + p.5_23; */
tree cast_stmt; /* D.2167_22 = (struct str_t *) D.2166_21;
This statement is not always present. */
};
/* A non-field structure access site. */
struct access_site
{
/* A statement in which the access site occurs. */
tree stmt;
/* A list of structure variables in the access site. */
VEC (tree, heap) *vars;
};
/* A field of the structure. */
struct field_entry
{
/* A field index. */
int index;
/* Number of times the field is accessed (according to profiling). */
gcov_type count;
tree decl;
/* A type of a new structure this field belongs to. */
tree field_mapping;
htab_t acc_sites;
};
/* This structure represents a result of the structure peeling.
The original structure is decomposed into substructures, or clusters. */
struct field_cluster
{
/* A bitmap of field indices. The set bit indicates that the field
corresponding to it is a part of this cluster. */
sbitmap fields_in_cluster;
struct field_cluster *sibling;
};
/* An information about an individual structure type (RECORD_TYPE) required
by struct-reorg optimizations to perform a transformation. */
struct data_structure
{
/* A main variant of the structure type. */
tree decl;
/* Number of fields in the structure. */
int num_fields;
/* A structure access count collected through profiling. */
gcov_type count;
/* An array of the structure fields, indexed by field ID. */
struct field_entry *fields;
/* Non-field accesses of the structure. */
htab_t accs;
/* A data structure representing a reorganization decision. */
struct field_cluster *struct_clustering;
/* New types to replace an the original structure type. */
VEC(tree, heap) *new_types;
};
typedef struct data_structure * d_str;
#endif /* IPA_STRUCT_REORG_H */
[-- Attachment #4: ipa-struct-reorg.c --]
[-- Type: application/octet-stream, Size: 99069 bytes --]
/* Struct-reorg optimization.
Copyright (C) 2007 Free Software Foundation, Inc.
Contributed by Olga Golovanevsky <olga@il.ibm.com>
(Initial version of this code was developed
by Caroline Tice and Mostafa Hagog.)
This file is part of GCC.
GCC is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free
Software Foundation; either version 2, or (at your option) any later
version.
GCC is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
for more details.
You should have received a copy of the GNU General Public License
along with GCC; see the file COPYING. If not, write to the Free
Software Foundation, 59 Temple Place - Suite 330, Boston, MA
02111-1307, USA. */
#include "config.h"
#include "system.h"
#include "coretypes.h"
#include "tm.h"
#include "ggc.h"
#include "tree.h"
#include "rtl.h"
#include "tree-gimple.h"
#include "tree-inline.h"
#include "tree-flow.h"
#include "tree-flow-inline.h"
#include "langhooks.h"
#include "pointer-set.h"
#include "hashtab.h"
#include "c-tree.h"
#include "toplev.h"
#include "flags.h"
#include "debug.h"
#include "target.h"
#include "cgraph.h"
#include "diagnostic.h"
#include "timevar.h"
#include "params.h"
#include "fibheap.h"
#include "intl.h"
#include "function.h"
#include "basic-block.h"
#include "tree-iterator.h"
#include "tree-pass.h"
#include "ipa-struct-reorg.h"
#include "opts.h"
#include "ipa-type-escape.h"
#include "tree-dump.h"
#include "c-common.h"
/* This optimization implements structure peeling.
For example, given a structure type:
typedef struct
{
int a;
float b;
int c;
}str_t;
it can be peeled into two structure types as follows:
typedef struct and typedef struct
{ {
int a; float b;
int c; } str_t_1;
}str_t_0;
or can be fully peeled:
typedef struct
{
int a;
}str_t_0;
typedef struct
{
float b;
}str_t_1;
typedef struct
{
int c;
}str_t_2;
When structure type is peeled all instances and their accesses
in the program are updated accordingly. For example, if there is
array of structures:
str_t A[N];
and structure type str_t was peeled into two structures str_t_0
and str_t_1 as it was shown above, then array A will be replaced
by two arrays as follows:
str_t_0 A_0[N];
str_t_1 A_1[N];
The field access of field a of element i of array A: A[i].a will be
replaced by an access to field a of element i of array A_0: A_0[i].a.
This optimization also supports dynamically allocated arrays.
If array of structures was allocated by malloc function:
str_t * p = (str_t *) malloc (sizeof (str_t) * N)
the allocation site will be replaced to reflect new structure types:
str_t_0 * p_0 = (str_t_0 *) malloc (sizeof (str_t_0) * N)
str_t_1 * p_1 = (str_t_1 *) malloc (sizeof (str_t_1) * N)
The field access through the pointer p[i].a will be changed by p_0[i].a.
The goal of structure peeling is to improve spatial locality.
For example, if one of the fields of a structure is accessed frequently
in the loop:
for (i = 0; i < N; i++)
{
... = A[i].a;
}
the allocation of field a of str_t contiguously in memory will
increase the chances of fetching the field from cache.
The analysis part of this optimization is based on the frequency of
field accesses, which are collected all over the program.
Then the fields with the frequencies that satisfy the following condition
get peeled out of the structure:
freq(f) > C * max_field_freq_in_struct
where max_field_freq_in_struct is the maximum field frequency
in the structure. C is a constant defining which portion of
max_field_freq_in_struct the fields should have in order to be peeled.
If profiling information is provided, it is used to calculate the
frequency of field accesses. Otherwise, the structure is fully peeled.
IPA type-escape analysis is used to determine when it is safe
to peel a structure.
The optimization is activated by flag -fipa-struct-reorg. */
/* New variables created by this optimization.
When doing struct peeling, each variable of
the original struct type will be replaced by
the set of new variables corresponding to
the new structure types. */
struct new_var_data {
/* VAR_DECL for original struct type. */
tree orig_var;
/* Vector of new variables. */
VEC(tree, heap) *new_vars;
};
typedef struct new_var_data *new_var;
typedef const struct new_var_data *const_new_var;
/* This structure represents allocation site of the structure. */
typedef struct alloc_site
{
tree stmt;
d_str str;
} alloc_site_t;
DEF_VEC_O (alloc_site_t);
DEF_VEC_ALLOC_O (alloc_site_t, heap);
/* Allocation sites that belong to the same function. */
struct func_alloc_sites
{
tree func;
/* Vector of allocation sites for function. */
VEC (alloc_site_t, heap) *allocs;
};
typedef struct func_alloc_sites *fallocs_t;
typedef const struct func_alloc_sites *const_fallocs_t;
/* All allocation sites in the program. */
htab_t alloc_sites;
/* New global variables. Generated once for whole program. */
htab_t new_global_vars;
/* New local variables. Generated per-function. */
htab_t new_local_vars;
/* Vector of structures to be transformed. */
typedef struct data_structure structure;
DEF_VEC_O (structure);
DEF_VEC_ALLOC_O (structure, heap);
VEC (structure, heap) *structures;
/* Forward declarations. */
static bool is_equal_types (tree, tree);
/* Strip structure TYPE from pointers and arrays. */
static inline tree
strip_type (tree type)
{
gcc_assert (TYPE_P (type));
while (POINTER_TYPE_P (type)
|| TREE_CODE (type) == ARRAY_TYPE)
type = TREE_TYPE (type);
return type;
}
/* This function returns type of VAR. */
static inline tree
get_type_of_var (tree var)
{
if (!var)
return NULL;
if (TREE_CODE (var) == PARM_DECL)
return DECL_ARG_TYPE (var);
else
return TREE_TYPE (var);
}
/* Set of actions we do for each newly generated STMT. */
static inline void
finalize_stmt (tree stmt)
{
update_stmt (stmt);
mark_symbols_for_renaming (stmt);
}
/* This function finalizes STMT and appends it to the list STMTS. */
static inline void
finalize_stmt_and_append (tree *stmts, tree stmt)
{
append_to_statement_list (stmt, stmts);
finalize_stmt (stmt);
}
/* Given structure type SRT_TYPE and field FIELD,
this function is looking for a field with the same name
and type as FIELD in STR_TYPE. It returns it if found,
or NULL_TREE otherwise. */
static tree
find_field_in_struct_1 (tree str_type, tree field)
{
tree str_field;
for (str_field = TYPE_FIELDS (str_type); str_field;
str_field = TREE_CHAIN (str_field))
{
const char * str_field_name;
const char * field_name;
str_field_name = IDENTIFIER_POINTER (DECL_NAME (str_field));
field_name = IDENTIFIER_POINTER (DECL_NAME (field));
gcc_assert (str_field_name);
gcc_assert (field_name);
if (!strcmp (str_field_name, field_name))
{
/* Check field types. */
if (is_equal_types (TREE_TYPE (str_field), TREE_TYPE (field)))
return str_field;
}
}
return NULL_TREE;
}
/* Given a field declaration FIELD_DECL, this function
returns corresponding field entry in structure STR. */
static struct field_entry *
find_field_in_struct (d_str str, tree field_decl)
{
int i;
tree field = find_field_in_struct_1 (str->decl, field_decl);
for (i = 0; i < str->num_fields; i++)
if (str->fields[i].decl == field)
return &(str->fields[i]);
return NULL;
}
/* This function checks whether ARG is a result of multiplication
of some number by STRUCT_SIZE. If yes, the function returns true
and this number is filled into NUM. */
static bool
is_result_of_mult (tree arg, tree *num, tree struct_size)
{
tree size_def_stmt = SSA_NAME_DEF_STMT (arg);
/* If allocation statementt was of the form
D.2229_10 = <alloc_func> (D.2228_9);
then size_def_stmt can be D.2228_9 = num.3_8 * 8; */
if (size_def_stmt && TREE_CODE (size_def_stmt) == GIMPLE_MODIFY_STMT)
{
tree lhs = GIMPLE_STMT_OPERAND (size_def_stmt, 0);
tree rhs = GIMPLE_STMT_OPERAND (size_def_stmt, 1);
/* We expect temporary here. */
if (!is_gimple_reg (lhs))
return false;
if (TREE_CODE (rhs) == MULT_EXPR)
{
tree arg0 = TREE_OPERAND (rhs, 0);
tree arg1 = TREE_OPERAND (rhs, 1);
if (operand_equal_p (arg0, struct_size, OEP_ONLY_CONST))
{
*num = arg1;
return true;
}
if (operand_equal_p (arg1, struct_size, OEP_ONLY_CONST))
{
*num = arg0;
return true;
}
}
}
*num = NULL_TREE;
return false;
}
/* This function returns true if access ACC corresponds to the pattern
generated by compiler when an address of element i of an array
of structures STR_DECL (pointed by p) is calculated (p[i]). If this
pattern is recognized correctly, this function returns true
and fills missing fields in ACC. Otherwise it returns false. */
static bool
decompose_indirect_ref_acc (tree str_decl, struct field_access_site *acc)
{
tree ref_var;
tree rhs, struct_size, op0, op1;
tree before_cast;
ref_var = TREE_OPERAND (acc->ref, 0);
if (TREE_CODE (ref_var) != SSA_NAME)
return false;
acc->ref_def_stmt = SSA_NAME_DEF_STMT (ref_var);
if (!(acc->ref_def_stmt)
|| (TREE_CODE (acc->ref_def_stmt) != GIMPLE_MODIFY_STMT))
return false;
rhs = GIMPLE_STMT_OPERAND (acc->ref_def_stmt, 1);
if (TREE_CODE (rhs) != PLUS_EXPR
&& TREE_CODE (rhs)!= MINUS_EXPR
&& TREE_CODE (rhs) != POINTER_PLUS_EXPR)
return false;
op0 = TREE_OPERAND (rhs, 0);
op1 = TREE_OPERAND (rhs, 1);
if (!is_array_access_through_pointer_and_index (TREE_CODE (rhs), op0, op1,
&acc->base, &acc->offset,
&acc->cast_stmt))
return false;
if (acc->cast_stmt)
before_cast = SINGLE_SSA_TREE_OPERAND (acc->cast_stmt, SSA_OP_USE);
else
before_cast = acc->offset;
if (!before_cast)
return false;
if (SSA_NAME_IS_DEFAULT_DEF (before_cast))
return false;
struct_size = TYPE_SIZE_UNIT (str_decl);
if (!is_result_of_mult (before_cast, &acc->num, struct_size))
return false;
return true;
}
/* This function checks whether the access ACC of structure type STR
is of the form suitable for tranformation. If yes, it returns true.
False otherwise. */
static bool
decompose_access (tree str_decl, struct field_access_site *acc)
{
gcc_assert (acc->ref);
if (TREE_CODE (acc->ref) == INDIRECT_REF)
return decompose_indirect_ref_acc (str_decl, acc);
else if (TREE_CODE (acc->ref) == ARRAY_REF)
return true;
else if (TREE_CODE (acc->ref) == VAR_DECL)
return true;
return false;
}
/* This function creates empty field_access_site node. */
static inline struct field_access_site *
make_field_acc_node (void)
{
int size = sizeof (struct field_access_site);
return (struct field_access_site *) xcalloc (1, size);
}
/* This function returns the structure field access, defined by STMT,
if it is aready in hashtable of function accesses F_ACCS. */
static struct field_access_site *
is_in_field_accs (tree stmt, htab_t f_accs)
{
return (struct field_access_site *)
htab_find_with_hash (f_accs, stmt, htab_hash_pointer (stmt));
}
/* This function adds an access ACC to the hashtable
F_ACCS of field accesses. */
static void
add_field_acc_to_acc_sites (struct field_access_site *acc,
htab_t f_accs)
{
void **slot;
gcc_assert (!is_in_field_accs (acc->stmt, f_accs));
slot = htab_find_slot_with_hash (f_accs, acc->stmt,
htab_hash_pointer (acc->stmt),
INSERT);
*slot = acc;
}
/* This function adds the VAR to vector of variables of
an access site defined by statement STMT. If access entry
with statement STMT does not exist in hashtable of
accesses ACCS, this function creates it. */
static void
add_access_to_acc_sites (tree stmt, tree var, htab_t accs)
{
struct access_site *acc;
acc = (struct access_site *)
htab_find_with_hash (accs, stmt, htab_hash_pointer (stmt));
if (!acc)
{
void **slot;
acc = (struct access_site *) xmalloc (sizeof (struct access_site));
acc->stmt = stmt;
acc->vars = VEC_alloc (tree, heap, 10);
slot = htab_find_slot_with_hash (accs, stmt,
htab_hash_pointer (stmt), INSERT);
*slot = acc;
}
VEC_safe_push (tree, heap, acc->vars, var);
}
/* This function adds NEW_DECL to function
referenced vars, and marks it for renaming. */
static void
finalize_var_creation (tree new_decl)
{
add_referenced_var (new_decl);
if (is_global_var (new_decl))
mark_call_clobbered (new_decl, ESCAPE_UNKNOWN);
mark_sym_for_renaming (new_decl);
}
/* This function finalizes VAR creation if it is a global VAR_DECL. */
static void
finalize_global_creation (tree var)
{
if (TREE_CODE (var) == VAR_DECL
&& is_global_var (var))
finalize_var_creation (var);
}
/* This function inserts NEW_DECL to varpool. */
static inline void
insert_global_to_varpool (tree new_decl)
{
struct varpool_node *new_node;
new_node = varpool_node (new_decl);
notice_global_symbol (new_decl);
varpool_mark_needed_node (new_node);
varpool_finalize_decl (new_decl);
}
/* This function finalizes the creation of new variables,
defined by *SLOT->new_vars. */
static int
finalize_new_vars_creation (void **slot, void *data ATTRIBUTE_UNUSED)
{
new_var n_var = *(new_var *) slot;
unsigned i;
tree var;
for (i = 0; VEC_iterate (tree, n_var->new_vars, i, var); i++)
finalize_var_creation (var);
return 1;
}
/* This funciton updates statements in STMT_LIST with BB info. */
static void
add_bb_info (basic_block bb, tree stmt_list)
{
if (TREE_CODE (stmt_list) == STATEMENT_LIST)
{
tree_stmt_iterator tsi;
for (tsi = tsi_start (stmt_list); !tsi_end_p (tsi); tsi_next (&tsi))
{
tree stmt = tsi_stmt (tsi);
set_bb_for_stmt (stmt, bb);
}
}
}
/* This function looks for the variable of NEW_TYPE type, stored in VAR.
It returns it, if found, and NULL_TREE otherwise. */
static tree
find_var_in_new_vars_vec (new_var var, tree new_type)
{
tree n_var;
unsigned i;
for (i = 0; VEC_iterate (tree, var->new_vars, i, n_var); i++)
{
tree type = strip_type(get_type_of_var (n_var));
gcc_assert (type);
if (type == new_type)
return n_var;
}
return NULL_TREE;
}
/* This function returns new_var node, the orig_var of which is DECL.
It looks for new_var's in NEW_VARS_HTAB. If not found,
the function returns NULL. */
static new_var
is_in_new_vars_htab (tree decl, htab_t new_vars_htab)
{
return (new_var) htab_find_with_hash (new_vars_htab, decl,
htab_hash_pointer (decl));
}
/* Given original varaiable ORIG_VAR, this function returns
new variable corresponding to it of NEW_TYPE type. */
static tree
find_new_var_of_type (tree orig_var, tree new_type)
{
new_var var;
gcc_assert (orig_var && new_type);
if (TREE_CODE (orig_var) == SSA_NAME)
orig_var = SSA_NAME_VAR (orig_var);
var = is_in_new_vars_htab (orig_var, new_global_vars);
if (!var)
var = is_in_new_vars_htab (orig_var, new_local_vars);
gcc_assert (var);
return find_var_in_new_vars_vec (var, new_type);
}
/* This function generates stmt:
res = NUM * sizeof(TYPE) and returns it.
res is filled into RES. */
static tree
gen_size (tree num, tree type, tree *res)
{
tree struct_size = TYPE_SIZE_UNIT (type);
HOST_WIDE_INT struct_size_int = TREE_INT_CST_LOW (struct_size);
tree new_stmt;
*res = create_tmp_var (TREE_TYPE (num), NULL);
if (*res)
add_referenced_var (*res);
if (exact_log2 (struct_size_int) == -1)
new_stmt = build_gimple_modify_stmt (num, struct_size);
else
{
tree C = build_int_cst (TREE_TYPE (num), exact_log2 (struct_size_int));
new_stmt = build_gimple_modify_stmt (*res, build2 (LSHIFT_EXPR,
TREE_TYPE (num),
num, C));
}
finalize_stmt (new_stmt);
return new_stmt;
}
/* This function generates and returns a statement, that cast variable
BEFORE_CAST to NEW_TYPE. The cast result variable is stored
into RES_P. ORIG_CAST_STMT is the original cast statement. */
static tree
gen_cast_stmt (tree before_cast, tree new_type, tree orig_cast_stmt,
tree *res_p)
{
tree lhs, new_lhs, new_stmt;
gcc_assert (TREE_CODE (orig_cast_stmt) == GIMPLE_MODIFY_STMT);
lhs = GIMPLE_STMT_OPERAND (orig_cast_stmt, 0);
new_lhs = find_new_var_of_type (lhs, new_type);
gcc_assert (new_lhs);
new_stmt = build_gimple_modify_stmt (new_lhs,
build1 (NOP_EXPR,
TREE_TYPE (new_lhs),
before_cast));
finalize_stmt (new_stmt);
*res_p = new_lhs;
return new_stmt;
}
/* This function builds an edge between BB and E->dest and updates
phi nodes of E->dest. It returns newly created edge. */
static edge
make_edge_and_fix_phis_of_dest (basic_block bb, edge e)
{
edge new_e;
tree phi, arg;
new_e = make_edge (bb, e->dest, e->flags);
for (phi = phi_nodes (new_e->dest); phi; phi = PHI_CHAIN (phi))
{
arg = PHI_ARG_DEF_FROM_EDGE (phi, e);
add_phi_arg (phi, arg, new_e);
}
return new_e;
}
/* This function inserts NEW_STMTS before STMT. */
static void
insert_before_stmt (tree stmt, tree new_stmts)
{
block_stmt_iterator bsi;
if (!stmt || !new_stmts)
return;
bsi = bsi_for_stmt (stmt);
bsi_insert_before (&bsi, new_stmts, BSI_SAME_STMT);
}
/* Insert NEW_STMTS after STMT. */
static void
insert_after_stmt (tree stmt, tree new_stmts)
{
block_stmt_iterator bsi;
if (!stmt || !new_stmts)
return;
bsi = bsi_for_stmt (stmt);
bsi_insert_after (&bsi, new_stmts, BSI_SAME_STMT);
}
/* This function returns vector of allocation sites
that appear in function FN_DECL. */
static fallocs_t
get_fallocs (tree fn_decl)
{
return (fallocs_t) htab_find_with_hash (alloc_sites, fn_decl,
htab_hash_pointer (fn_decl));
}
/* If ALLOC_STMT is D.2225_7 = <alloc_func> (D.2224_6);
and it is a part of allocation of a structure,
then it is usually followed by a cast stmt
p_8 = (struct str_t *) D.2225_7;
which is returned by this function. */
static tree
get_final_alloc_stmt (tree alloc_stmt)
{
tree final_stmt;
use_operand_p use_p;
tree alloc_res;
if (!alloc_stmt)
return NULL;
if (TREE_CODE (alloc_stmt) != GIMPLE_MODIFY_STMT)
return NULL;
alloc_res = GIMPLE_STMT_OPERAND (alloc_stmt, 0);
if (TREE_CODE (alloc_res) != SSA_NAME)
return NULL;
if (!single_imm_use (alloc_res, &use_p, &final_stmt))
return NULL;
else
return final_stmt;
}
/* This function returns true if STMT is one of allocation
sites of function FN_DECL. It returns false otherwise. */
static bool
is_part_of_malloc (tree stmt, tree fn_decl)
{
fallocs_t fallocs = get_fallocs (fn_decl);
if (fallocs)
{
alloc_site_t *call;
unsigned i;
for (i = 0;
VEC_iterate (alloc_site_t, fallocs->allocs, i, call); i++)
if (call->stmt == stmt
|| get_final_alloc_stmt (call->stmt) == stmt)
return true;
}
return false;
}
/* Auxiliary structure for a lookup over field accesses. */
struct find_stmt_data
{
bool found;
tree stmt;
};
/* This function looks for DATA->stmt among
the statements involved in the field access,
defined by SLOT. It stops when it's found. */
static int
find_in_field_accs (void **slot, void *data)
{
struct field_access_site *f_acc =
*(struct field_access_site **) slot;
tree stmt = ((struct find_stmt_data *)data)->stmt;
if (f_acc->stmt == stmt
|| f_acc->ref_def_stmt == stmt
|| f_acc->cast_stmt == stmt)
{
((struct find_stmt_data *)data)->found = true;
return 0;
}
else
return 1;
}
/* This function checks whether STMT is part of field
accesses of structure STR. It returns true, if found,
and false otherwise. */
static bool
is_part_of_field_access (tree stmt, d_str str)
{
int i;
for (i = 0; i < str->num_fields; i++)
{
struct find_stmt_data data;
data.found = false;
data.stmt = stmt;
if (str->fields[i].acc_sites)
htab_traverse (str->fields[i].acc_sites, find_in_field_accs, &data);
if (data.found)
return true;
}
return false;
}
/* Auxiliary data for exclude_from_accs function. */
struct exclude_data
{
tree fn_decl;
d_str str;
};
/* This function returns component_ref with the BASE and
field named FIELD_ID from structure TYPE. */
static inline tree
build_comp_ref (tree base, tree field_id, tree type)
{
tree field;
bool found = false;
/* Find field of structure type with the same name as field_id. */
for (field = TYPE_FIELDS (type); field; field = TREE_CHAIN (field))
{
if (DECL_NAME (field) == field_id)
{
found = true;
break;
}
}
gcc_assert (found);
return build3 (COMPONENT_REF, TREE_TYPE (field), base, field, NULL_TREE);
}
/* This struct represent data used for walk_tree
called from function find_pos_in_stmt.
- ref is a tree to be found,
- and pos is a pointer that points to ref in stmt. */
struct ref_pos
{
tree *pos;
tree ref;
};
/* This is a callback function for walk_tree, called from
collect_accesses_in_bb function. DATA is a pointer to ref_pos structure.
When *TP is equal to DATA->ref, the walk_tree stops,
and found position, equal to TP, is assigned to DATA->pos. */
static tree
find_pos_in_stmt_1 (tree *tp, int *walk_subtrees, void * data)
{
struct ref_pos * r_pos = (struct ref_pos *) data;
tree ref = r_pos->ref;
tree t = *tp;
if (t == ref)
{
r_pos->pos = tp;
return t;
}
switch (TREE_CODE (t))
{
case GIMPLE_MODIFY_STMT:
{
tree lhs = GIMPLE_STMT_OPERAND (t, 0);
tree rhs = GIMPLE_STMT_OPERAND (t, 1);
*walk_subtrees = 1;
walk_tree (&lhs, find_pos_in_stmt_1, data, NULL);
walk_tree (&rhs, find_pos_in_stmt_1, data, NULL);
*walk_subtrees = 0;
}
break;
default:
*walk_subtrees = 1;
}
return NULL_TREE;
}
/* This function looks for the pointer of REF in STMT,
It returns it, if found, and NULL otherwise. */
static tree *
find_pos_in_stmt (tree stmt, tree ref)
{
struct ref_pos r_pos;
r_pos.ref = ref;
r_pos.pos = NULL;
walk_tree (&stmt, find_pos_in_stmt_1, &r_pos, NULL);
return r_pos.pos;
}
/* This structure is used to represent array
or pointer-to wrappers of structure type.
For example, if type1 is structure type,
then for type1 ** we generate two type_wrapper
structures with wrap = 0 each one.
It's used to unwind the original type up to
structure type, replace it with the new structure type
and wrap it back in the opposite order. */
typedef struct type_wrapper
{
/* 0 stand for pointer wrapper, and 1 for array wrapper. */
bool wrap;
/* Relevant for arrays as domain or index. */
tree domain;
}type_wrapper_t;
DEF_VEC_O (type_wrapper_t);
DEF_VEC_ALLOC_O (type_wrapper_t, heap);
/* This function replace field access ACC by the new
field access of structure type NEW_TYPE. */
static void
replace_field_acc (struct field_access_site *acc, tree new_type)
{
tree ref_var = acc->ref;
tree new_ref;
tree lhs, rhs;
tree *pos;
tree new_acc;
tree field_id = DECL_NAME (acc->field_decl);
VEC (type_wrapper_t, heap) *wrapper = VEC_alloc (type_wrapper_t, heap, 10);
type_wrapper_t wr;
type_wrapper_t *wr_p = NULL;
while (TREE_CODE (ref_var) == INDIRECT_REF
|| TREE_CODE (ref_var) == ARRAY_REF)
{
if ( TREE_CODE (ref_var) == INDIRECT_REF)
{
wr.wrap = 0;
wr.domain = 0;
}
else if (TREE_CODE (ref_var) == ARRAY_REF)
{
wr.wrap = 1;
wr.domain = TREE_OPERAND (ref_var, 1);
}
VEC_safe_push (type_wrapper_t, heap, wrapper, &wr);
ref_var = TREE_OPERAND (ref_var, 0);
}
new_ref = find_new_var_of_type (ref_var, new_type);
finalize_global_creation (new_ref);
while (VEC_length (type_wrapper_t, wrapper) != 0)
{
tree type = TREE_TYPE (TREE_TYPE (new_ref));
wr_p = VEC_last (type_wrapper_t, wrapper);
if (wr_p->wrap) /* Array. */
new_ref = build4 (ARRAY_REF, type, new_ref,
wr_p->domain, NULL_TREE, NULL_TREE);
else /* Pointer. */
new_ref = build1 (INDIRECT_REF, type, new_ref);
VEC_pop (type_wrapper_t, wrapper);
}
new_acc = build_comp_ref (new_ref, field_id, new_type);
VEC_free (type_wrapper_t, heap, wrapper);
if (TREE_CODE (acc->stmt) == GIMPLE_MODIFY_STMT)
{
lhs = GIMPLE_STMT_OPERAND (acc->stmt, 0);
rhs = GIMPLE_STMT_OPERAND (acc->stmt, 1);
if (lhs == acc->comp_ref)
GIMPLE_STMT_OPERAND (acc->stmt, 0) = new_acc;
else if (rhs == acc->comp_ref)
GIMPLE_STMT_OPERAND (acc->stmt, 1) = new_acc;
else
{
pos = find_pos_in_stmt (acc->stmt, acc->comp_ref);
gcc_assert (pos);
*pos = new_acc;
}
}
else
{
pos = find_pos_in_stmt (acc->stmt, acc->comp_ref);
gcc_assert (pos);
*pos = new_acc;
}
finalize_stmt (acc->stmt);
}
/* This function replace field access ACC by a new field access
of structure type NEW_TYPE. */
static void
replace_field_access_stmt (struct field_access_site *acc, tree new_type)
{
if (TREE_CODE (acc->ref) == INDIRECT_REF
||TREE_CODE (acc->ref) == ARRAY_REF
||TREE_CODE (acc->ref) == VAR_DECL)
replace_field_acc (acc, new_type);
else
gcc_unreachable ();
}
/* This function looks for d_str, represented by TYPE, in the structures
vector. If found, it returns an index of found structure. Otherwise
it returns a length of the structures vector. */
static unsigned
find_structure (tree type)
{
d_str str;
unsigned i;
type = TYPE_MAIN_VARIANT (type);
for (i = 0; VEC_iterate (structure, structures, i, str); i++)
if (is_equal_types (str->decl, type))
return i;
return VEC_length (structure, structures);
}
/* In this function we create new statements that have the same
form as ORIG_STMT, but of type NEW_TYPE. The statements
treated by this function are simple assignments,
like assignments: p.8_7 = p; or statements with rhs of
tree codes PLUS_EXPR and MINUS_EXPR. */
static tree
create_base_plus_offset (tree orig_stmt, tree new_type,
tree offset)
{
tree lhs, rhs;
tree new_lhs, new_rhs;
tree new_stmt;
gcc_assert (TREE_CODE (orig_stmt) == GIMPLE_MODIFY_STMT);
lhs = GIMPLE_STMT_OPERAND (orig_stmt, 0);
rhs = GIMPLE_STMT_OPERAND (orig_stmt, 1);
gcc_assert (TREE_CODE (lhs) == VAR_DECL
|| TREE_CODE (lhs) == SSA_NAME);
new_lhs = find_new_var_of_type (lhs, new_type);
gcc_assert (new_lhs);
finalize_var_creation (new_lhs);
switch (TREE_CODE (rhs))
{
case PLUS_EXPR:
case MINUS_EXPR:
case POINTER_PLUS_EXPR:
{
tree op0 = TREE_OPERAND (rhs, 0);
tree op1 = TREE_OPERAND (rhs, 1);
tree new_op0 = NULL_TREE, new_op1 = NULL_TREE;
unsigned str0, str1;
unsigned length = VEC_length (structure, structures);
str0 = find_structure (strip_type (get_type_of_var (op0)));
str1 = find_structure (strip_type (get_type_of_var (op1)));
gcc_assert ((str0 != length) || (str1 != length));
if (str0 != length)
new_op0 = find_new_var_of_type (op0, new_type);
if (str1 != length)
new_op1 = find_new_var_of_type (op1, new_type);
if (!new_op0)
new_op0 = offset;
if (!new_op1)
new_op1 = offset;
new_rhs = build2 (TREE_CODE (rhs), TREE_TYPE (new_op0),
new_op0, new_op1);
}
break;
default:
gcc_unreachable();
}
new_stmt = build_gimple_modify_stmt (new_lhs, new_rhs);
finalize_stmt (new_stmt);
return new_stmt;
}
/* Given a field access F_ACC of the FIELD, this function
replaces it by the new field access. */
static void
create_new_field_access (struct field_access_site *f_acc,
struct field_entry field)
{
tree new_type = field.field_mapping;
tree new_stmt;
tree size_res;
tree mult_stmt, cast_stmt;
tree cast_res = NULL;
if (f_acc->num)
{
mult_stmt = gen_size (f_acc->num, new_type, &size_res);
insert_before_stmt (f_acc->ref_def_stmt, mult_stmt);
}
if (f_acc->cast_stmt)
{
cast_stmt = gen_cast_stmt (size_res, new_type,
f_acc->cast_stmt, &cast_res);
insert_after_stmt (f_acc->cast_stmt, cast_stmt);
}
if (f_acc->ref_def_stmt)
{
tree offset;
if (cast_res)
offset = cast_res;
else
offset = size_res;
new_stmt = create_base_plus_offset (f_acc->ref_def_stmt,
new_type, offset);
insert_after_stmt (f_acc->ref_def_stmt, new_stmt);
}
/* In stmt D.2163_19 = D.2162_18->b; we replace variable
D.2162_18 by an appropriate variable of new_type type. */
replace_field_access_stmt (f_acc, new_type);
}
/* This function creates a new condition statement
corresponding to the original COND_STMT, adds new basic block
and redirects condition edges. NEW_VAR is a new condition
variable located in the condition statement at the position POS. */
static void
create_new_stmts_for_cond_expr_1 (tree new_var, tree cond_stmt, bool pos)
{
tree new_cond;
tree new_stmt;
edge true_e = NULL, false_e = NULL;
basic_block new_bb;
tree stmt_list;
extract_true_false_edges_from_block (bb_for_stmt (cond_stmt),
&true_e, &false_e);
new_cond = unshare_expr (COND_EXPR_COND (cond_stmt));
TREE_OPERAND (new_cond, pos) = new_var;
new_stmt = build3 (COND_EXPR, TREE_TYPE (cond_stmt),
new_cond, NULL_TREE, NULL_TREE);
finalize_stmt (new_stmt);
/* Create new basic block after bb. */
new_bb = create_empty_bb (bb_for_stmt (cond_stmt));
/* Add new condition stmt to the new_bb. */
stmt_list = bb_stmt_list (new_bb);
append_to_statement_list (new_stmt, &stmt_list);
add_bb_info (new_bb, stmt_list);
/* Create false and true edges from new_bb. */
make_edge_and_fix_phis_of_dest (new_bb, true_e);
make_edge_and_fix_phis_of_dest (new_bb, false_e);
/* Redirect one of original edges to point to new_bb. */
if (TREE_CODE (cond_stmt) == NE_EXPR)
redirect_edge_succ (true_e, new_bb);
else
redirect_edge_succ (false_e, new_bb);
}
/* This function creates new condition statements corresponding
to original condition STMT, one for each new type, and
recursively redirect edges to newly generated basic blocks. */
static void
create_new_stmts_for_cond_expr (tree stmt)
{
tree cond = COND_EXPR_COND (stmt);
tree arg0, arg1, arg;
unsigned str0, str1;
bool s0, s1;
d_str str;
tree type;
bool pos;
int i;
unsigned length = VEC_length (structure, structures);
gcc_assert (TREE_CODE (cond) == EQ_EXPR
|| TREE_CODE (cond) == NE_EXPR);
arg0 = TREE_OPERAND (cond, 0);
arg1 = TREE_OPERAND (cond, 1);
str0 = find_structure (strip_type (get_type_of_var (arg0)));
str1 = find_structure (strip_type (get_type_of_var (arg1)));
s0 = (str0 != length) ? true : false;
s1 = (str1 != length) ? true : false;
gcc_assert ((!s0 && s1) || (!s1 && s0));
str = s0 ? VEC_index (structure, structures, str0):
VEC_index (structure, structures, str1);
arg = s0 ? arg0 : arg1;
pos = s0 ? 0 : 1;
for (i = 0; VEC_iterate (tree, str->new_types, i, type); i++)
{
tree new_arg;
new_arg = find_new_var_of_type (arg, type);
create_new_stmts_for_cond_expr_1 (new_arg, stmt, pos);
}
}
/* Create a new general access to replace original access ACC
for structure type NEW_TYPE. */
static tree
create_general_new_stmt (struct access_site *acc, tree new_type)
{
tree old_stmt = acc->stmt;
tree var;
tree new_stmt = unshare_expr (old_stmt);
unsigned i;
for (i = 0; VEC_iterate (tree, acc->vars, i, var); i++)
{
tree *pos;
tree new_var = find_new_var_of_type (var, new_type);
tree lhs, rhs;
gcc_assert (new_var);
finalize_var_creation (new_var);
if (TREE_CODE (new_stmt) == GIMPLE_MODIFY_STMT)
{
lhs = GIMPLE_STMT_OPERAND (new_stmt, 0);
rhs = GIMPLE_STMT_OPERAND (new_stmt, 1);
if (TREE_CODE (lhs) == SSA_NAME)
lhs = SSA_NAME_VAR (lhs);
if (TREE_CODE (rhs) == SSA_NAME)
rhs = SSA_NAME_VAR (rhs);
/* It can happen that rhs is a constructor.
Then we have to replace it to be of new_type. */
if (TREE_CODE (rhs) == CONSTRUCTOR)
{
/* Dealing only with empty constructors right now. */
gcc_assert (VEC_empty (constructor_elt,
CONSTRUCTOR_ELTS (rhs)));
rhs = build_constructor (new_type, 0);
GIMPLE_STMT_OPERAND (new_stmt, 1) = rhs;
}
if (lhs == var)
GIMPLE_STMT_OPERAND (new_stmt, 0) = new_var;
else if (rhs == var)
GIMPLE_STMT_OPERAND (new_stmt, 1) = new_var;
else
{
pos = find_pos_in_stmt (new_stmt, var);
gcc_assert (pos);
*pos = new_var;
}
}
else
{
pos = find_pos_in_stmt (new_stmt, var);
gcc_assert (pos);
*pos = new_var;
}
}
finalize_stmt (new_stmt);
return new_stmt;
}
/* For each new type in STR this function creates new general accesses
corresponding to the original access ACC. */
static void
create_new_stmts_for_general_acc (struct access_site *acc, d_str str)
{
tree type;
tree stmt = acc->stmt;
unsigned i;
for (i = 0; VEC_iterate (tree, str->new_types, i, type); i++)
{
tree new_stmt;
new_stmt = create_general_new_stmt (acc, type);
insert_after_stmt (stmt, new_stmt);
}
}
/* This function creates a new general access of structure STR
to replace the access ACC. */
static void
create_new_general_access (struct access_site *acc, d_str str)
{
tree stmt = acc->stmt;
switch (TREE_CODE (stmt))
{
case COND_EXPR:
create_new_stmts_for_cond_expr (stmt);
break;
default:
create_new_stmts_for_general_acc (acc, str);
}
}
/* Auxiliary data for creation of accesses. */
struct create_acc_data
{
basic_block bb;
d_str str;
int field_index;
};
/* This function creates a new general access, defined by SLOT.
DATA is a pointer to create_acc_data structure. */
static int
create_new_acc (void **slot, void *data)
{
struct access_site *acc = *(struct access_site **) slot;
basic_block bb = ((struct create_acc_data *)data)->bb;
d_str str = ((struct create_acc_data *)data)->str;
if (bb_for_stmt (acc->stmt) == bb)
create_new_general_access (acc, str);
return 1;
}
/* This function creates a new field access, defined by SLOT.
DATA is a pointer to create_acc_data structure. */
static int
create_new_field_acc (void **slot, void *data)
{
struct field_access_site *f_acc = *(struct field_access_site **) slot;
basic_block bb = ((struct create_acc_data *)data)->bb;
d_str str = ((struct create_acc_data *)data)->str;
int i = ((struct create_acc_data *)data)->field_index;
if (bb_for_stmt (f_acc->stmt) == bb)
create_new_field_access (f_acc, str->fields[i]);
return 1;
}
/* This function creates new accesses for the structure
type STR in basic block BB. */
static void
create_new_accs_for_struct (d_str str, basic_block bb)
{
int i;
struct create_acc_data dt;
dt.str = str;
dt.bb = bb;
dt.field_index = -1;
for (i = 0; i < str->num_fields; i++)
{
dt.field_index = i;
if (str->fields[i].acc_sites)
htab_traverse (str->fields[i].acc_sites,
create_new_field_acc, &dt);
}
if (str->accs)
htab_traverse (str->accs, create_new_acc, &dt);
}
/* This function inserts new variables from new_var,
defined by SLOT, into varpool. */
static int
update_varpool_with_new_var (void **slot, void *data ATTRIBUTE_UNUSED)
{
new_var n_var = *(new_var *) slot;
tree var;
unsigned i;
for (i = 0; VEC_iterate (tree, n_var->new_vars, i, var); i++)
insert_global_to_varpool (var);
return 1;
}
/* This function prints a field access site, defined by SLOT. */
static int
dump_field_acc (void **slot, void *data ATTRIBUTE_UNUSED)
{
struct field_access_site *f_acc =
*(struct field_access_site **) slot;
fprintf(dump_file, "\n");
if (f_acc->stmt)
print_generic_stmt (dump_file, f_acc->stmt, 0);
if (f_acc->ref_def_stmt)
print_generic_stmt (dump_file, f_acc->ref_def_stmt, 0);
if (f_acc->cast_stmt)
print_generic_stmt (dump_file, f_acc->cast_stmt, 0);
return 1;
}
/* Print field accesses from hashtable F_ACCS. */
static void
dump_field_acc_sites (htab_t f_accs)
{
if (!dump_file)
return;
if (f_accs)
htab_traverse (f_accs, dump_field_acc, NULL);
}
/* Hash value for fallocs_t. */
static hashval_t
malloc_hash (const void *x)
{
return htab_hash_pointer (((const_fallocs_t)x)->func);
}
/* This function returns nonzero if function of func_alloc_sites' X
is equal to Y. */
static int
malloc_eq (const void *x, const void *y)
{
return ((const_fallocs_t)x)->func == (const_tree)y;
}
/* This function is a callback for traversal over a structure accesses.
It frees an access represented by SLOT. */
static int
free_accs (void **slot, void *data ATTRIBUTE_UNUSED)
{
struct access_site * acc = *(struct access_site **) slot;
VEC_free (tree, heap, acc->vars);
free (acc);
return 1;
}
/* This is a callback function for traversal over field accesses.
It frees a field access represented by SLOT. */
static int
free_field_accs (void **slot, void *data ATTRIBUTE_UNUSED)
{
struct field_access_site *f_acc = *(struct field_access_site **) slot;
free (f_acc);
return 1;
}
/* This function inserts TYPE into vector of UNSUITABLE_TYPES,
if it is not there yet. */
static void
add_unsuitable_type (VEC (tree, heap) **unsuitable_types, tree type)
{
unsigned i;
tree t;
if (!type)
return;
type = TYPE_MAIN_VARIANT (type);
for (i = 0; VEC_iterate (tree, *unsuitable_types, i, t); i++)
if (is_equal_types (t, type))
break;
if (i == VEC_length (tree, *unsuitable_types))
VEC_safe_push (tree, heap, *unsuitable_types, type);
}
/* Given a type TYPE, this function returns the name of the type. */
static const char *
get_type_name (tree type)
{
if (! TYPE_NAME (type))
return NULL;
if (TREE_CODE (TYPE_NAME (type)) == IDENTIFIER_NODE)
return IDENTIFIER_POINTER (TYPE_NAME (type));
else if (TREE_CODE (TYPE_NAME (type)) == TYPE_DECL
&& DECL_NAME (TYPE_NAME (type)))
return IDENTIFIER_POINTER (DECL_NAME (TYPE_NAME (type)));
else
return NULL;
}
/* This function is a temporary hack to overcome the types problem.
When several compilation units are compiled together
with -combine, the TYPE_MAIN_VARIANT of the same type
can appear differently in different compilation units.
Therefore this function first compares type names.
If there are no names, structure bodies are recursively
compared. */
static bool
is_equal_types (tree type1, tree type2)
{
const char * name1,* name2;
if ((!type1 && type2)
||(!type2 && type1))
return false;
if (!type1 && !type2)
return true;
if (TREE_CODE (type1) != TREE_CODE (type2))
return false;
if (type1 == type2)
return true;
if (TYPE_MAIN_VARIANT (type1) == TYPE_MAIN_VARIANT (type2))
return true;
name1 = get_type_name (type1);
name2 = get_type_name (type2);
if (name1 && name2 && !strcmp (name1, name2))
return true;
if (name1 && name2 && strcmp (name1, name2))
return false;
switch (TREE_CODE (type1))
{
case POINTER_TYPE:
case REFERENCE_TYPE:
{
return is_equal_types (TREE_TYPE (type1), TREE_TYPE (type2));
}
break;
case RECORD_TYPE:
case UNION_TYPE:
case QUAL_UNION_TYPE:
case ENUMERAL_TYPE:
{
tree field1;
/* Compare fields of struture. */
for (field1 = TYPE_FIELDS (type1); field1;
field1 = TREE_CHAIN (field1))
{
tree field2 = find_field_in_struct_1 (type2, field1);
if (!field2)
return false;
}
return true;
}
break;
case INTEGER_TYPE:
{
if (TYPE_UNSIGNED (type1) == TYPE_UNSIGNED (type2)
&& TYPE_PRECISION (type1) == TYPE_PRECISION (type2))
return true;
}
break;
case ARRAY_TYPE:
{
tree d1, d2;
tree max1, min1, max2, min2;
if (!is_equal_types (TREE_TYPE (type1), TREE_TYPE (type2)))
return false;
d1 = TYPE_DOMAIN (type1);
d2 = TYPE_DOMAIN (type2);
if (!d1 || !d2)
return false;
max1 = TYPE_MAX_VALUE (d1);
max2 = TYPE_MAX_VALUE (d2);
min1 = TYPE_MIN_VALUE (d1);
min2 = TYPE_MIN_VALUE (d2);
if (max1 && max2 && min1 && min2
&& TREE_CODE (max1) == TREE_CODE (max2)
&& TREE_CODE (max1) == INTEGER_CST
&& TREE_CODE (min1) == TREE_CODE (min2)
&& TREE_CODE (min1) == INTEGER_CST
&& tree_int_cst_equal (max1, max2)
&& tree_int_cst_equal (min1, min2))
return true;
}
break;
default:
gcc_unreachable();
}
return false;
}
/* This function free non-field accesses from hashtable ACCS. */
static void
free_accesses (htab_t accs)
{
if (accs)
htab_traverse (accs, free_accs, NULL);
htab_delete (accs);
}
/* This function free field accesses hashtable F_ACCS. */
static void
free_field_accesses (htab_t f_accs)
{
if (f_accs)
htab_traverse (f_accs, free_field_accs, NULL);
htab_delete (f_accs);
}
/* Update call graph with new edge generated by new MALLOC_STMT.
The edge origin is CONTEXT function. */
static void
update_cgraph_with_malloc_call (tree malloc_stmt, tree context)
{
tree call_expr;
struct cgraph_node *src, *dest;
tree malloc_fn_decl;
if (!malloc_stmt)
return;
call_expr = get_call_expr_in (malloc_stmt);
malloc_fn_decl = get_callee_fndecl (call_expr);
src = cgraph_node (context);
dest = cgraph_node (malloc_fn_decl);
cgraph_create_edge (src, dest, malloc_stmt,
0, 0, bb_for_stmt (malloc_stmt)->loop_depth);
}
/* This function generates set of statements required
to allocate number NUM of structures of type NEW_TYPE.
The statements are stored in NEW_STMTS. The statement that contain
call to malloc is returned. MALLOC_STMT is an original call to malloc. */
static tree
create_new_malloc (tree malloc_stmt, tree new_type, tree *new_stmts, tree num)
{
tree new_malloc_size;
tree call_expr, malloc_fn_decl;
tree new_stmt, malloc_res;
tree call_stmt, final_stmt;
tree cast_res;
gcc_assert (num && malloc_stmt && new_type);
*new_stmts = alloc_stmt_list ();
/* Generate argument to malloc as multiplication of num
and size of new_type. */
new_stmt = gen_size (num, new_type, &new_malloc_size);
append_to_statement_list (new_stmt, new_stmts);
/* Generate new call for malloc. */
malloc_res = create_tmp_var (integer_type_node, NULL);
if (malloc_res)
add_referenced_var (malloc_res);
call_expr = get_call_expr_in (malloc_stmt);
malloc_fn_decl = get_callee_fndecl (call_expr);
call_expr = build_call_expr (malloc_fn_decl, 1, new_malloc_size);
call_stmt = build_gimple_modify_stmt (malloc_res, call_expr);
finalize_stmt_and_append (new_stmts, call_stmt);
/* Create new cast statement. */
final_stmt = get_final_alloc_stmt (malloc_stmt);
gcc_assert (final_stmt);
new_stmt = gen_cast_stmt (malloc_res, new_type, final_stmt, &cast_res);
append_to_statement_list (new_stmt, new_stmts);
return call_stmt;
}
/* This function returns a tree representing
the number of instances of structure STR_DECL allocated
by allocation STMT. If new statments are generated,
they are filled into NEW_STMTS_P. */
static tree
gen_num_of_structs_in_malloc (tree stmt, tree str_decl, tree *new_stmts_p)
{
call_expr_arg_iterator iter;
tree arg;
tree call_expr;
tree struct_size;
HOST_WIDE_INT struct_size_int;
if (!stmt)
return NULL_TREE;
/* Get malloc argument. */
call_expr = get_call_expr_in (stmt);
if (!call_expr)
return NULL_TREE;
arg = first_call_expr_arg (call_expr, &iter);
if (TREE_CODE (arg) != SSA_NAME
&& !TREE_CONSTANT (arg))
return NULL_TREE;
struct_size = TYPE_SIZE_UNIT (str_decl);
struct_size_int = TREE_INT_CST_LOW (struct_size);
gcc_assert (struct_size);
if (TREE_CODE (arg) == SSA_NAME)
{
tree num, div_stmt;
if (is_result_of_mult (arg, &num, struct_size))
return num;
num = create_tmp_var (integer_type_node, NULL);
if (num)
add_referenced_var (num);
if (exact_log2 (struct_size_int) == -1)
div_stmt = build_gimple_modify_stmt (num,
build2 (TRUNC_DIV_EXPR,
integer_type_node,
arg, struct_size));
else
{
tree C = build_int_cst (integer_type_node,
exact_log2 (struct_size_int));
div_stmt =
build_gimple_modify_stmt (num, build2 (RSHIFT_EXPR,
integer_type_node,
arg, C));
}
*new_stmts_p = alloc_stmt_list ();
append_to_statement_list (div_stmt,
new_stmts_p);
finalize_stmt (div_stmt);
return num;
}
if (CONSTANT_CLASS_P (arg)
&& multiple_of_p (TREE_TYPE (struct_size), arg, struct_size))
return int_const_binop (TRUNC_DIV_EXPR, arg, struct_size, 0);
return NULL_TREE;
}
/* This function is a callback for traversal on new_var's hashtable.
SLOT is a pointer to new_var. This function prints to dump_file
an original variable and all new variables from the new_var
pointed by *SLOT. */
static int
dump_new_var (void **slot, void *data ATTRIBUTE_UNUSED)
{
new_var n_var = *(new_var *) slot;
tree var_type;
tree var;
unsigned i;
var_type = get_type_of_var (n_var->orig_var);
fprintf (dump_file, "\nOrig var: ");
print_generic_expr (dump_file, n_var->orig_var, 0);
fprintf (dump_file, " of type ");
print_generic_expr (dump_file, var_type, 0);
fprintf (dump_file, "\n");
for (i = 0;
VEC_iterate (tree, n_var->new_vars, i, var); i++)
{
var_type = get_type_of_var (var);
fprintf (dump_file, " ");
print_generic_expr (dump_file, var, 0);
fprintf (dump_file, " of type ");
print_generic_expr (dump_file, var_type, 0);
fprintf (dump_file, "\n");
}
return 1;
}
/* This function copies attributes form ORIG_DECL to NEW_DECL. */
static inline void
copy_decl_attributes (tree new_decl, tree orig_decl)
{
DECL_ARTIFICIAL (new_decl) = 1;
DECL_EXTERNAL (new_decl) = DECL_EXTERNAL (orig_decl);
TREE_STATIC (new_decl) = TREE_STATIC (orig_decl);
TREE_PUBLIC (new_decl) = TREE_PUBLIC (orig_decl);
TREE_USED (new_decl) = TREE_USED (orig_decl);
DECL_CONTEXT (new_decl) = DECL_CONTEXT (orig_decl);
TREE_THIS_VOLATILE (new_decl) = TREE_THIS_VOLATILE (orig_decl);
TREE_ADDRESSABLE (new_decl) = TREE_ADDRESSABLE (orig_decl);
if (TREE_CODE (orig_decl) == VAR_DECL)
{
TREE_READONLY (new_decl) = TREE_READONLY (orig_decl);
DECL_TLS_MODEL (new_decl) = DECL_TLS_MODEL (orig_decl);
}
}
/* This function wraps NEW_STR_TYPE in pointers or arrays wrapper
the same way as a structure type is wrapped in DECL.
It returns the generated type. */
static inline tree
gen_struct_type (tree decl, tree new_str_type)
{
tree type_orig = get_type_of_var (decl);
tree new_type = new_str_type;
VEC (type_wrapper_t, heap) *wrapper = VEC_alloc (type_wrapper_t, heap, 10);
type_wrapper_t wr;
type_wrapper_t *wr_p;
while (POINTER_TYPE_P (type_orig)
|| TREE_CODE (type_orig) == ARRAY_TYPE)
{
if (POINTER_TYPE_P (type_orig))
{
wr.wrap = 0;
wr.domain = NULL_TREE;
}
else if (TREE_CODE (type_orig) == ARRAY_TYPE)
{
wr.wrap = 1;
wr.domain = TYPE_DOMAIN (type_orig);
}
VEC_safe_push (type_wrapper_t, heap, wrapper, &wr);
type_orig = TREE_TYPE (type_orig);
}
while (VEC_length (type_wrapper_t, wrapper) != 0)
{
wr_p = VEC_last (type_wrapper_t, wrapper);
if (wr_p->wrap) /* Array. */
new_type = build_array_type (new_type, wr_p->domain);
else /* Pointer. */
new_type = build_pointer_type (new_type);
VEC_pop (type_wrapper_t, wrapper);
}
VEC_free (type_wrapper_t, heap, wrapper);
return new_type;
}
/* This function generates and returns new variable name based on
ORIG_DECL name, combined with index I.
The form of the new name is <orig_name>.<I> . */
static tree
gen_var_name (tree orig_decl, unsigned HOST_WIDE_INT i)
{
const char *old_name;
char *prefix;
char *new_name;
if (!DECL_NAME (orig_decl)
|| !IDENTIFIER_POINTER (DECL_NAME (orig_decl)))
return NULL;
/* If the original variable has a name, create an
appropriate new name for the new variable. */
old_name = IDENTIFIER_POINTER (DECL_NAME (orig_decl));
prefix = alloca (strlen (old_name) + 1);
strcpy (prefix, old_name);
ASM_FORMAT_PRIVATE_NAME (new_name, prefix, i);
return get_identifier (new_name);
}
/* This function adds NEW_NODE to hashtable of new_var's NEW_VARS_HTAB. */
static void
add_to_new_vars_htab (new_var new_node, htab_t new_vars_htab)
{
void **slot;
slot = htab_find_slot_with_hash (new_vars_htab, new_node->orig_var,
htab_hash_pointer (new_node->orig_var),
INSERT);
*slot = new_node;
}
/* This function creates and returns new_var_data node
with empty new_vars and orig_var equal to VAR. */
static new_var
create_new_var_node (tree var, d_str str)
{
new_var node;
node = (new_var) xmalloc (sizeof (struct new_var_data));
node->orig_var = var;
node->new_vars = VEC_alloc (tree, heap, VEC_length (tree, str->new_types));
return node;
}
/* Check whether the type of VAR is potential candidate for peeling.
Returns true if yes, false otherwise. If yes, TYPE_P will contain
candidate type. If VAR is initialized, the type of VAR will be added
to UNSUITABLE_TYPES. */
static bool
is_candidate (tree var, tree *type_p, VEC (tree, heap) **unsuitable_types)
{
tree type;
bool initialized = false;
*type_p = NULL;
if (!var)
return false;
/* There is no support of initialized vars. */
if (TREE_CODE (var) == VAR_DECL
&& DECL_INITIAL (var) != NULL_TREE)
initialized = true;
type = get_type_of_var (var);
if (type)
{
type = TYPE_MAIN_VARIANT (strip_type (type));
if (TREE_CODE (type) != RECORD_TYPE)
return false;
else
{
if (initialized && unsuitable_types && *unsuitable_types)
add_unsuitable_type (unsuitable_types, type);
*type_p = type;
return true;
}
}
else
return false;
}
/* Hash value for field_access_site. */
static hashval_t
field_acc_hash (const void *x)
{
return htab_hash_pointer (((const struct field_access_site *)x)->stmt);
}
/* This function returns nonzero if stmt of field_access_site X
is equal to Y. */
static int
field_acc_eq (const void *x, const void *y)
{
return ((const struct field_access_site *)x)->stmt == (const_tree)y;
}
/* This function prints an access site, defined by SLOT. */
static int
dump_acc (void **slot, void *data ATTRIBUTE_UNUSED)
{
struct access_site *acc = *(struct access_site **) slot;
tree var;
unsigned i;
fprintf(dump_file, "\n");
if (acc->stmt)
print_generic_stmt (dump_file, acc->stmt, 0);
fprintf(dump_file, " : ");
for (i = 0; VEC_iterate (tree, acc->vars, i, var); i++)
{
print_generic_expr (dump_file, var, 0);
fprintf(dump_file, ", ");
}
return 1;
}
/* This function frees memory allocated for strcuture clusters,
starting from CLUSTER. */
static void
free_struct_cluster (struct field_cluster* cluster)
{
if (cluster)
{
if (cluster->fields_in_cluster)
sbitmap_free (cluster->fields_in_cluster);
if (cluster->sibling)
free_struct_cluster (cluster->sibling);
free (cluster);
}
}
/* Free all allocated memory under the structure node pointed by D_NODE. */
static void
free_data_struct (d_str d_node)
{
int i;
if (!d_node)
return;
if (dump_file)
{
fprintf (dump_file, "\nRemoving data structure \"");
print_generic_expr (dump_file, d_node->decl, 0);
fprintf (dump_file, "\" from data_struct_list.");
}
/* Free all space under d_node. */
if (d_node->fields)
{
for (i = 0; i < d_node->num_fields; i++)
free_field_accesses (d_node->fields[i].acc_sites);
free (d_node->fields);
}
if (d_node->accs)
free_accesses (d_node->accs);
if (d_node->struct_clustering)
free_struct_cluster (d_node->struct_clustering);
if (d_node->new_types)
VEC_free (tree, heap, d_node->new_types);
}
/* This function creates new general and field accesses in BB. */
static void
create_new_accesses_in_bb (basic_block bb)
{
d_str str;
unsigned i;
for (i = 0; VEC_iterate (structure, structures, i, str); i++)
create_new_accs_for_struct (str, bb);
}
/* This function adds allocation sites for peeled structures.
M_DATA is vector of allocation sites of function CONTEXT. */
static void
create_new_alloc_sites (fallocs_t m_data, tree context)
{
alloc_site_t *call;
unsigned j;
for (j = 0;
VEC_iterate (alloc_site_t, m_data->allocs, j, call); j++)
{
tree stmt = call->stmt;
d_str str = call->str;
tree num;
tree new_stmts = NULL_TREE;
tree last_stmt = get_final_alloc_stmt (stmt);
unsigned i;
tree type;
num = gen_num_of_structs_in_malloc (stmt, str->decl, &new_stmts);
if (new_stmts)
{
last_stmt = tsi_stmt (tsi_last (new_stmts));
insert_after_stmt (last_stmt, new_stmts);
}
/* Generate an allocation sites for each new structure type. */
for (i = 0;
VEC_iterate (tree, str->new_types, i, type); i++)
{
tree new_malloc_stmt = NULL_TREE;
tree last_stmt_tmp = NULL_TREE;
new_stmts = NULL_TREE;
new_malloc_stmt = create_new_malloc (stmt, type, &new_stmts, num);
last_stmt_tmp = tsi_stmt (tsi_last (new_stmts));
insert_after_stmt (last_stmt, new_stmts);
update_cgraph_with_malloc_call (new_malloc_stmt, context);
last_stmt = last_stmt_tmp;
}
}
}
/* This function prints new variables from hashtable
NEW_VARS_HTAB to dump_file. */
static void
dump_new_vars (htab_t new_vars_htab)
{
if (!dump_file)
return;
if (new_vars_htab)
htab_traverse (new_vars_htab, dump_new_var, NULL);
}
/* Given an original variable ORIG_DECL of structure type STR,
this function generates new variables of the types defined
by STR->new_type. Generated types are saved in new_var node NODE.
ORIG_DECL should has VAR_DECL tree_code. */
static void
create_new_var_1 (tree orig_decl, d_str str, new_var node)
{
unsigned i;
tree type;
for (i = 0;
VEC_iterate (tree, str->new_types, i, type); i++)
{
tree new_decl = NULL;
tree new_name;
new_name = gen_var_name (orig_decl, i);
type = gen_struct_type (orig_decl, type);
if (is_global_var (orig_decl))
new_decl = build_decl (VAR_DECL, new_name, type);
else
{
const char *name = new_name ? IDENTIFIER_POINTER (new_name) : NULL;
new_decl = create_tmp_var (type, name);
}
copy_decl_attributes (new_decl, orig_decl);
VEC_safe_push (tree, heap, node->new_vars, new_decl);
}
}
/* This function creates new variables to
substitute the original variable VAR_DECL and adds
them to the new_var's hashtable NEW_VARS_HTAB. */
static void
create_new_var (tree var_decl, htab_t new_vars_htab)
{
new_var node;
d_str str;
tree type;
unsigned i;
if (!var_decl || is_in_new_vars_htab (var_decl, new_vars_htab))
return;
if (!is_candidate (var_decl, &type, NULL))
return;
i = find_structure (type);
if (i == VEC_length (structure, structures))
return;
str = VEC_index (structure, structures, i);
node = create_new_var_node (var_decl, str);
create_new_var_1 (var_decl, str, node);
add_to_new_vars_htab (node, new_vars_htab);
}
/* Hash value for new_var. */
static hashval_t
new_var_hash (const void *x)
{
return htab_hash_pointer (((const_new_var)x)->orig_var);
}
/* This function returns nonzero if orig_var of new_var X is equal to Y. */
static int
new_var_eq (const void *x, const void *y)
{
return ((const_new_var)x)->orig_var == (const_tree)y;
}
/* This function check whether a structure type represented by STR
escapes due to ipa-type-escape analysis. If yes, this type is added
to UNSUITABLE_TYPES vector. */
static void
check_type_escape (d_str str, VEC (tree, heap) **unsuitable_types)
{
tree type = str->decl;
if (!ipa_type_escape_type_contained_p (type))
{
if (dump_file)
{
fprintf (dump_file, "\nEscaping type is ");
print_generic_expr (dump_file, type, 0);
}
add_unsuitable_type (unsuitable_types, type);
}
}
/* Hash value for access_site. */
static hashval_t
acc_hash (const void *x)
{
return htab_hash_pointer (((const struct access_site *)x)->stmt);
}
/* Return nonzero if stmt of access_site X is equal to Y. */
static int
acc_eq (const void *x, const void *y)
{
return ((const struct access_site *)x)->stmt == (const_tree)y;
}
/* Given a structure declaration STRUCT_DECL, and number of fields
in the structure NUM_FIELDS, this function creates and returns
corresponding field_entry's. */
static struct field_entry *
get_fields (tree struct_decl, int num_fields)
{
struct field_entry *list;
tree t = TYPE_FIELDS (struct_decl);
int idx = 0;
list =
(struct field_entry *) xmalloc (num_fields * sizeof (struct field_entry));
for (idx = 0 ; t; t = TREE_CHAIN (t), idx++)
if (TREE_CODE (t) == FIELD_DECL)
{
list[idx].index = idx;
list[idx].decl = t;
list[idx].acc_sites =
htab_create (32, field_acc_hash, field_acc_eq, NULL);
list[idx].count = 0;
list[idx].field_mapping = NULL_TREE;
}
return list;
}
/* Print non-field accesses from hashtable ACCS of structure. */
static void
dump_access_sites (htab_t accs)
{
if (!dump_file)
return;
if (accs)
htab_traverse (accs, dump_acc, NULL);
}
/* This function removes the structure with index I from structures vector. */
static void
remove_structure (unsigned i)
{
d_str str;
if (i >= VEC_length (structure, structures))
return;
str = VEC_index (structure, structures, i);
free_data_struct (str);
VEC_ordered_remove (structure, structures, i);
}
/* Currently we support only EQ_EXPR or NE_EXPR conditions.
COND_STNT is a condition statement to check. */
static bool
is_safe_cond_expr (tree cond_stmt)
{
tree arg0, arg1;
unsigned str0, str1;
bool s0, s1;
unsigned length = VEC_length (structure, structures);
tree cond = COND_EXPR_COND (cond_stmt);
if (TREE_CODE (cond) != EQ_EXPR
&& TREE_CODE (cond) != NE_EXPR)
return false;
if (TREE_CODE_LENGTH (TREE_CODE (cond)) != 2)
return false;
arg0 = TREE_OPERAND (cond, 0);
arg1 = TREE_OPERAND (cond, 1);
str0 = find_structure (strip_type (get_type_of_var (arg0)));
str1 = find_structure (strip_type (get_type_of_var (arg1)));
s0 = (str0 != length) ? true : false;
s1 = (str1 != length) ? true : false;
if (!((!s0 && s1) || (!s1 && s0)))
return false;
return true;
}
/* This function excludes statements, that are
part of allocation sites or field accesses, from the
hashtable of general accesses. SLOT represents general
access that will be checked. DATA is a pointer to
exclude_data structure. */
static int
exclude_from_accs (void **slot, void *data)
{
struct access_site *acc = *(struct access_site **) slot;
tree fn_decl = ((struct exclude_data *)data)->fn_decl;
d_str str = ((struct exclude_data *)data)->str;
if (is_part_of_malloc (acc->stmt, fn_decl)
|| is_part_of_field_access (acc->stmt, str))
{
VEC_free (tree, heap, acc->vars);
free (acc);
htab_clear_slot (str->accs, slot);
}
return 1;
}
/* Callback function for walk_tree called from collect_accesses_in_bb
function. DATA is the statement which is analyzed. */
static tree
get_stmt_accesses (tree *tp, int *walk_subtrees, void *data)
{
tree stmt = (tree) data;
tree t = *tp;
if (!t)
return NULL;
switch (TREE_CODE (t))
{
case GIMPLE_MODIFY_STMT:
{
tree lhs = GIMPLE_STMT_OPERAND (t, 0);
tree rhs = GIMPLE_STMT_OPERAND (t, 1);
*walk_subtrees = 1;
walk_tree (&lhs, get_stmt_accesses, data, NULL);
walk_tree (&rhs, get_stmt_accesses, data, NULL);
*walk_subtrees = 0;
}
break;
case BIT_FIELD_REF:
{
tree var = TREE_OPERAND(t, 0);
tree type = TYPE_MAIN_VARIANT (strip_type (get_type_of_var (var)));
unsigned i = find_structure (type);
if (i != VEC_length (structure, structures))
remove_structure (i);
}
break;
case COMPONENT_REF:
{
tree ref = TREE_OPERAND (t, 0);
tree field_decl = TREE_OPERAND (t, 1);
if ((TREE_CODE (ref) == INDIRECT_REF
|| TREE_CODE (ref) == ARRAY_REF
|| TREE_CODE (ref) == VAR_DECL)
&& TREE_CODE (field_decl) == FIELD_DECL)
{
tree type = TYPE_MAIN_VARIANT (TREE_TYPE (ref));
unsigned i = find_structure (type);
if (i != VEC_length (structure, structures))
{
d_str str = VEC_index (structure, structures, i);
struct field_entry * field =
find_field_in_struct (str, field_decl);
if (field)
{
struct field_access_site *acc = make_field_acc_node ();
gcc_assert (acc);
acc->stmt = stmt;
acc->comp_ref = t;
acc->ref = ref;
acc->field_decl = field_decl;
/* Check whether the access is of the form
we can deal with. */
if (!decompose_access (str->decl, acc))
{
remove_structure (i);
free (acc);
}
else
{
/* Increase count of field. */
basic_block bb = bb_for_stmt (stmt);
field->count += bb->count;
/* Add stmt to the acc_sites of field. */
add_field_acc_to_acc_sites (acc, field->acc_sites);
}
*walk_subtrees = 0;
}
}
}
}
break;
case MINUS_EXPR:
case PLUS_EXPR:
{
tree op0 = TREE_OPERAND (t, 0);
tree op1 = TREE_OPERAND (t, 1);
*walk_subtrees = 1;
walk_tree (&op0, get_stmt_accesses, data, NULL);
walk_tree (&op1, get_stmt_accesses, data, NULL);
*walk_subtrees = 0;
}
break;
case COND_EXPR:
{
tree cond = COND_EXPR_COND (t);
int i;
for (i = 0; i < TREE_CODE_LENGTH (TREE_CODE (cond)); i++)
{
tree t = TREE_OPERAND (cond, i);
*walk_subtrees = 1;
walk_tree (&t, get_stmt_accesses, data, NULL);
}
*walk_subtrees = 0;
}
break;
case VAR_DECL:
case SSA_NAME:
{
unsigned i;
if (TREE_CODE (t) == SSA_NAME)
t = SSA_NAME_VAR (t);
i = find_structure (strip_type (get_type_of_var (t)));
if (i != VEC_length (structure, structures))
{
d_str str;
str = VEC_index (structure, structures, i);
add_access_to_acc_sites (stmt, t, str->accs);
}
*walk_subtrees = 0;
}
break;
case CALL_EXPR:
{
/* It was checked as part of stage1 that structures
to be transformed cannot be passed as parameters of functions. */
*walk_subtrees = 0;
}
break;
default:
return NULL;
}
return NULL;
}
/* Free structures hashtable. */
static void
free_structures (void)
{
d_str str;
unsigned i;
for (i = 0; VEC_iterate (structure, structures, i, str); i++)
free_data_struct (str);
VEC_free (structure, heap, structures);
structures = NULL;
}
/* This function is a callback for traversal over new_var's hashtable.
SLOT is a pointer to new_var. This function frees memory allocated
for new_var and pointed by *SLOT. */
static int
free_new_var (void **slot, void *data ATTRIBUTE_UNUSED)
{
new_var n_var = *(new_var *) slot;
/* Free vector of new_vars. */
VEC_free (tree, heap, n_var->new_vars);
free (n_var);
return 1;
}
/* Free new_vars hashtable NEW_VARS_HTAB. */
static void
free_new_vars_htab (htab_t new_vars_htab)
{
if (new_vars_htab)
htab_traverse (new_vars_htab, free_new_var, NULL);
htab_delete (new_vars_htab);
new_vars_htab = NULL;
}
/* This function creates new general and field accesses that appear in cfun. */
static void
create_new_accesses_for_func (void)
{
basic_block bb;
FOR_EACH_BB_FN (bb, cfun)
create_new_accesses_in_bb (bb);
}
/* Create new allocation sites for the function represented by NODE. */
static void
create_new_alloc_sites_for_func (struct cgraph_node *node)
{
fallocs_t fallocs = get_fallocs (node->decl);
if (fallocs)
create_new_alloc_sites (fallocs, node->decl);
}
/* For each local variable of structure type from the vector of structures
this function generates new variable(s) to replace it. */
static void
create_new_local_vars (void)
{
tree var;
referenced_var_iterator rvi;
new_local_vars = htab_create (num_referenced_vars,
new_var_hash, new_var_eq, NULL);
FOR_EACH_REFERENCED_VAR (var, rvi)
{
if (!is_global_var (var))
create_new_var (var, new_local_vars);
}
if (new_local_vars)
htab_traverse (new_local_vars, finalize_new_vars_creation, NULL);
dump_new_vars (new_local_vars);
}
/* This function prints the SHIFT number of spaces to the DUMP_FILE. */
static inline void
print_shift (unsigned HOST_WIDE_INT shift)
{
unsigned HOST_WIDE_INT sh = shift;
while (sh--)
fprintf (dump_file, " ");
}
/* This function updates field_mapping of FIELDS in CLUSTER with NEW_TYPE. */
static inline void
update_fields_mapping (struct field_cluster *cluster, tree new_type,
struct field_entry * fields, int num_fields)
{
int i;
for (i = 0; i < num_fields; i++)
if (TEST_BIT (cluster->fields_in_cluster, i))
fields[i].field_mapping = new_type;
}
/* This functions builds structure with FIELDS,
NAME and attributes similar to ORIG_STRUCT.
It returns the newly created structure. */
static tree
build_basic_struct (tree fields, tree name, tree orig_struct)
{
tree attributes = NULL_TREE;
tree ref = 0;
tree x;
if (TYPE_ATTRIBUTES (orig_struct))
attributes = unshare_expr (TYPE_ATTRIBUTES (orig_struct));
ref = make_node (RECORD_TYPE);
TYPE_SIZE (ref) = 0;
decl_attributes (&ref, attributes, (int) ATTR_FLAG_TYPE_IN_PLACE);
TYPE_PACKED (ref) = TYPE_PACKED (orig_struct);
for (x = fields; x; x = TREE_CHAIN (x))
{
DECL_CONTEXT (x) = ref;
DECL_PACKED (x) |= TYPE_PACKED (ref);
}
TYPE_FIELDS (ref) = fields;
layout_type (ref);
TYPE_NAME (ref) = name;
return ref;
}
/* This function copies FIELDS from CLUSTER into TREE_CHAIN as part
of preparation for new structure building. NUM_FIELDS is a total
number of fields in the structure. The function returns newly
generated fields. */
static tree
create_fields (struct field_cluster * cluster,
struct field_entry * fields, int num_fields)
{
int i;
tree new_types = NULL_TREE;
tree last = NULL_TREE;
for (i = 0; i < num_fields; i++)
if (TEST_BIT (cluster->fields_in_cluster, i))
{
tree new_decl = unshare_expr (fields[i].decl);
if (!new_types)
new_types = new_decl;
else
TREE_CHAIN (last) = new_decl;
last = new_decl;
}
TREE_CHAIN (last) = NULL_TREE;
return new_types;
}
/* This function creates a cluster name. The name is based on
the original structure name, if it is present. It has a form:
<original_struct_name>_sub.<CLUST_NUM>
The original structure name is taken from the type of DECL.
If an original structure name is not present, it's generated to be:
struct.<STR_NUM>
The function returns identifier of the new cluster name. */
static inline tree
gen_cluster_name (tree decl, int clust_num, int str_num)
{
const char * orig_name = get_type_name (decl);
char * tmp_name = NULL;
char * prefix;
char * new_name;
size_t len;
if (!orig_name)
ASM_FORMAT_PRIVATE_NAME(tmp_name, "struct", str_num);
len = strlen (tmp_name ? tmp_name : orig_name) + strlen ("_sub");
prefix = alloca (len + 1);
memcpy (prefix, tmp_name ? tmp_name : orig_name,
strlen (tmp_name ? tmp_name : orig_name));
strcpy (prefix + strlen (tmp_name ? tmp_name : orig_name), "_sub");
ASM_FORMAT_PRIVATE_NAME (new_name, prefix, clust_num);
return get_identifier (new_name);
}
/* This function checks whether the structure STR has bitfields.
If yes, this structure type is added to UNSUITABLE_TYPES vector. */
static void
check_bitfields (d_str str, VEC (tree, heap) **unsuitable_types)
{
tree type = str->decl;
int i;
for (i = 0; i < str->num_fields; i++)
if (DECL_BIT_FIELD (str->fields[i].decl))
{
add_unsuitable_type (unsuitable_types, type);
if (dump_file)
{
fprintf (dump_file, "\nType ");
print_generic_expr (dump_file, type, 0);
fprintf (dump_file, "\nescapes due to bitfield ");
print_generic_expr (dump_file, str->fields[i].decl, 0);
}
break;
}
}
/* This function adds to UNSUITABLE_TYPES those types that escape
due to results of ipa-type-escpae analysis. See ipa-type-escpae.[c,h]. */
static void
exclude_escaping_types_1 (VEC (tree, heap) **unsuitable_types)
{
d_str str;
unsigned i;
for (i = 0; VEC_iterate (structure, structures, i, str); i++)
check_type_escape (str, unsuitable_types);
}
/* If a structure type is a return type of any function,
we cannot transform it. Such type is added to UNSUITABLE_TYPES vector. */
static void
exclude_returned_types (VEC (tree, heap) **unsuitable_types)
{
struct cgraph_node *c_node;
for (c_node = cgraph_nodes; c_node; c_node = c_node->next)
{
tree ret_t = TREE_TYPE (TREE_TYPE (c_node->decl));
if (ret_t)
{
ret_t = strip_type (ret_t);
if (TREE_CODE (ret_t) == RECORD_TYPE)
{
add_unsuitable_type (unsuitable_types, TYPE_MAIN_VARIANT (ret_t));
if (dump_file)
{
fprintf (dump_file, "\nThe type \"");
print_generic_expr (dump_file, ret_t, 0);
fprintf (dump_file,
"\" is return type of function...Excluded.");
}
}
}
}
}
/* This function looks for parameters of local functions
which are of structure types, or derived from them (arrays
of structures, pointers to structures, or their combinations).
We are not handling peeling of such structures right now.
The found structures types are added to UNSUITABLE_TYPES vector. */
static void
exclude_types_passed_to_local_func (VEC (tree, heap) **unsuitable_types)
{
struct cgraph_node *c_node;
for (c_node = cgraph_nodes; c_node; c_node = c_node->next)
if (cgraph_function_body_availability (c_node) == AVAIL_LOCAL)
{
tree fn = c_node->decl;
tree arg;
for (arg = DECL_ARGUMENTS (fn); arg; arg = TREE_CHAIN (arg))
{
tree type = TREE_TYPE (arg);
type = strip_type (type);
if (TREE_CODE (type) == RECORD_TYPE)
{
add_unsuitable_type (unsuitable_types,
TYPE_MAIN_VARIANT (type));
if (dump_file)
{
fprintf (dump_file, "\nPointer to type \"");
print_generic_expr (dump_file, type, 0);
fprintf (dump_file,
"\" is passed to local function...Excluded.");
}
}
}
}
}
/* This function analyzes structure form of structures
potential for transformation. If we are not capable to transform
structure of some form, we remove it from the structures hashtable.
Right now we cannot handle nested structs, when nesting is
through any level of pointers or arrays.
TBD: release these constrains in future.
Note, that in this function we suppose that all structures
in the program are members of the structures hashtable right now,
without excluding escaping types. */
static void
check_struct_form (d_str str, VEC (tree, heap) **unsuitable_types)
{
int i;
for (i = 0; i < str->num_fields; i++)
{
tree f_type = strip_type(TREE_TYPE (str->fields[i].decl));
if (TREE_CODE (f_type) == RECORD_TYPE)
{
add_unsuitable_type (unsuitable_types, TYPE_MAIN_VARIANT (f_type));
add_unsuitable_type (unsuitable_types, str->decl);
if (dump_file)
{
fprintf (dump_file, "\nType ");
print_generic_expr (dump_file, f_type, 0);
fprintf (dump_file, " is a field in the structure ");
print_generic_expr (dump_file, str->decl, 0);
fprintf (dump_file, ". Escaping...");
}
}
}
}
/* This function adds a structure TYPE to the vector of structures,
if it's not already there. */
static void
add_structure (tree type)
{
struct data_structure node;
unsigned i;
int num_fields;
type = TYPE_MAIN_VARIANT (type);
i = find_structure (type);
if (i != VEC_length (structure, structures))
return;
num_fields = fields_length (type);
node.decl = type;
node.num_fields = num_fields;
node.fields = get_fields (type, num_fields);
node.struct_clustering = NULL;
node.accs = htab_create (32, acc_hash, acc_eq, NULL);
node.new_types = VEC_alloc (tree, heap, num_fields);
node.count = 0;
VEC_safe_push (structure, heap, structures, &node);
if (dump_file)
{
fprintf (dump_file, "\nAdding data structure \"");
print_generic_expr (dump_file, type, 0);
fprintf (dump_file, "\" to data_struct_list.");
}
}
/* This function adds an allocation site to alloc_sites hashtable.
The allocation site appears in STMT of function FN_DECL and
allocates the structure represented by STR. */
static void
add_alloc_site (tree fn_decl, tree stmt, d_str str)
{
fallocs_t fallocs = NULL;
alloc_site_t m_call;
m_call.stmt = stmt;
m_call.str = str;
fallocs =
(fallocs_t) htab_find_with_hash (alloc_sites,
fn_decl, htab_hash_pointer (fn_decl));
if (!fallocs)
{
void **slot;
fallocs = (fallocs_t)
xmalloc (sizeof (struct func_alloc_sites));
fallocs->func = fn_decl;
fallocs->allocs = VEC_alloc (alloc_site_t, heap, 1);
slot = htab_find_slot_with_hash (alloc_sites, fn_decl,
htab_hash_pointer (fn_decl), INSERT);
*slot = fallocs;
}
VEC_safe_push (alloc_site_t, heap,
fallocs->allocs, &m_call);
if (dump_file)
{
fprintf (dump_file, "\nAdding stmt ");
print_generic_stmt (dump_file, stmt, 0);
fprintf (dump_file, " to list of mallocs.");
}
}
/* This function returns true if the result of STMT, that contains a call
to an allocation function, is cast to one of the structure types.
STMT should be of the form: T.2 = <alloc_func> (T.1);
If true, I_P contains an index of an allocated structure.
Otherwise I_P contains the length of the vector of structures. */
static bool
is_alloc_of_struct (tree stmt, unsigned *i_p)
{
tree lhs;
tree type;
tree final_stmt;
final_stmt = get_final_alloc_stmt (stmt);
if (!final_stmt)
return false;
/* final_stmt should be of the form:
T.3 = (struct_type *) T.2; */
if (TREE_CODE (final_stmt) != GIMPLE_MODIFY_STMT)
return false;
lhs = GIMPLE_STMT_OPERAND (final_stmt, 0);
type = get_type_of_var (lhs);
if (!type)
return false;
if (!POINTER_TYPE_P (type)
|| TREE_CODE (strip_type (type)) != RECORD_TYPE)
return false;
*i_p = find_structure (strip_type (type));
if (*i_p == VEC_length (structure, structures))
return false;
return true;
}
/* This function prints non-field and field accesses
of the structure STR. */
static void
dump_accs (d_str str)
{
int i;
fprintf (dump_file, "\nAccess sites of struct ");
print_generic_expr (dump_file, str->decl, 0);
for (i = 0; i < str->num_fields; i++)
{
fprintf (dump_file, "\nAccess site of field ");
print_generic_expr (dump_file, str->fields[i].decl, 0);
dump_field_acc_sites (str->fields[i].acc_sites);
fprintf (dump_file, ":\n");
}
fprintf (dump_file, "\nGeneral access sites\n");
dump_access_sites (str->accs);
}
/* This function checks whether an access statement, pointed by SLOT,
is a condition we are capable to transform. If not, it removes
the structure with index, represented by DATA, from the vector
of structures. */
static int
safe_cond_expr_check (void **slot, void *data)
{
struct access_site *acc = *(struct access_site **) slot;
if (TREE_CODE (acc->stmt) == COND_EXPR)
{
if (!is_safe_cond_expr (acc->stmt))
remove_structure (*(unsigned *) data);
}
return 1;
}
/* This function excludes statements that are part of allocation sites and
field accesses from the hashtable of general accesses of the structure
type STR. Only accesses that belong to the function represented by
NODE are treated. */
static void
exclude_alloc_and_field_accs_1 (d_str str, struct cgraph_node *node)
{
struct exclude_data dt;
dt.str = str;
dt.fn_decl = node->decl;
if (dt.str->accs)
htab_traverse (dt.str->accs, exclude_from_accs, &dt);
}
/* Collect accesses to the structure types that apear in basic bloack BB. */
static void
collect_accesses_in_bb (basic_block bb)
{
block_stmt_iterator bsi;
for (bsi = bsi_start (bb); ! bsi_end_p (bsi); bsi_next (&bsi))
{
tree stmt = bsi_stmt (bsi);
/* In asm stmt we cannot always track the arguments,
so we just give up. */
if (TREE_CODE (stmt) == ASM_EXPR)
{
free_structures ();
break;
}
walk_tree (&stmt, get_stmt_accesses, stmt, NULL);
}
}
/* This function generates cluster substructure that cointains FIELDS.
The cluster added to the set of clusters of the structure SRT. */
static void
gen_cluster (sbitmap fields, d_str str)
{
struct field_cluster *crr_cluster = NULL;
crr_cluster =
(struct field_cluster *) xcalloc (1, sizeof (struct field_cluster));
crr_cluster->sibling = str->struct_clustering;
str->struct_clustering = crr_cluster;
crr_cluster->fields_in_cluster = fields;
}
/* This function peels a field with the index I from the structure DS. */
static void
peel_field (int i, d_str ds)
{
struct field_cluster *crr_cluster = NULL;
crr_cluster =
(struct field_cluster *) xcalloc (1, sizeof (struct field_cluster));
crr_cluster->sibling = ds->struct_clustering;
ds->struct_clustering = crr_cluster;
crr_cluster->fields_in_cluster =
sbitmap_alloc ((unsigned int) ds->num_fields);
sbitmap_zero (crr_cluster->fields_in_cluster);
SET_BIT (crr_cluster->fields_in_cluster, i);
}
/* This function calculates maximum field count in
the structure STR. */
static gcov_type
get_max_field_count (d_str str)
{
gcov_type max = 0;
int i;
for (i = 0; i < str->num_fields; i++)
if (str->fields[i].count > max)
max = str->fields[i].count;
return max;
}
/* Do struct-reorg transformation for individual function
represented by NODE. All structure types relevant
for this function are transformed. */
static void
do_reorg_for_func (struct cgraph_node *node)
{
create_new_local_vars ();
create_new_alloc_sites_for_func (node);
create_new_accesses_for_func ();
update_ssa (TODO_update_ssa);
cleanup_tree_cfg ();
/* Free auxiliary data representing local variables. */
free_new_vars_htab (new_local_vars);
}
/* Print structure TYPE, its name, if it exists, and body.
INDENT defines the level of indentation (similar
to the option -i of indent command). SHIFT parameter
defines a number of spaces by which a structure will
be shifted right. */
static void
dump_struct_type (tree type, unsigned HOST_WIDE_INT indent,
unsigned HOST_WIDE_INT shift)
{
const char *struct_name;
tree field;
if (!type || !dump_file)
return;
if (TREE_CODE (type) != RECORD_TYPE)
{
print_generic_expr (dump_file, type, 0);
return;
}
print_shift (shift);
struct_name = get_type_name (type);
fprintf (dump_file, "struct ");
if (struct_name)
fprintf (dump_file, "%s\n",struct_name);
print_shift (shift);
fprintf (dump_file, "{\n");
for (field = TYPE_FIELDS (type); field;
field = TREE_CHAIN (field))
{
unsigned HOST_WIDE_INT s = indent;
tree f_type = TREE_TYPE (field);
print_shift (shift);
while (s--)
fprintf (dump_file, " ");
dump_struct_type (f_type, indent, shift + indent);
fprintf(dump_file, " ");
print_generic_expr (dump_file, field, 0);
fprintf(dump_file, ";\n");
}
print_shift (shift);
fprintf (dump_file, "}\n");
}
/* This function creates new structure types to replace original type,
indicated by STR->decl. The names of the new structure types are
derived from the original structure type. If the original structure
type has no name, we assume that its name is 'struct.<STR_NUM>'. */
static void
create_new_type (d_str str, int *str_num)
{
int cluster_num = 0;
struct field_cluster *cluster = str->struct_clustering;
while (cluster)
{
tree name = gen_cluster_name (str->decl, cluster_num,
*str_num);
tree fields;
tree new_type;
cluster_num++;
fields = create_fields (cluster, str->fields,
str->num_fields);
new_type = build_basic_struct (fields, name, str->decl);
update_fields_mapping (cluster, new_type,
str->fields, str->num_fields);
VEC_safe_push (tree, heap, str->new_types, new_type);
cluster = cluster->sibling;
}
(*str_num)++;
}
/* This function is a callback for alloc_sites hashtable
traversal. SLOT is a pointer to fallocs_t.
This function frees memory pointed by *SLOT. */
static int
free_falloc_sites (void **slot, void *data ATTRIBUTE_UNUSED)
{
fallocs_t fallocs = *(fallocs_t *) slot;
VEC_free (alloc_site_t, heap, fallocs->allocs);
free (fallocs);
return 1;
}
/* Remove structures collected in UNSUITABLE_TYPES
from structures vector. */
static void
remove_unsuitable_types (VEC (tree, heap) *unsuitable_types)
{
d_str str;
tree type;
unsigned i, j;
for (j = 0; VEC_iterate (tree, unsuitable_types, j, type); j++)
for (i = 0; VEC_iterate (structure, structures, i, str); i++)
if (is_equal_types (str->decl, type))
{
remove_structure (i);
break;
}
}
/* Exclude structure types with bitfields.
We would not want to interfere with other optimizations
that can be done in this case. The structure types with
bitfields are added to UNSUITABLE_TYPES vector. */
static void
exclude_types_with_bit_fields (VEC (tree, heap) **unsuitable_types)
{
d_str str;
unsigned i;
for (i = 0; VEC_iterate (structure, structures, i, str); i++)
check_bitfields (str, unsuitable_types);
}
/* This function checks three types of escape. A structure type escapes:
1. if it's a type of parameter of a local function.
2. if it's a type of function return value.
3. if it escapes as a result of ipa-type-escape analysis.
The escaping structure types are added to UNSUITABLE_TYPES vector. */
static void
exclude_escaping_types (VEC (tree, heap) **unsuitable_types)
{
exclude_types_passed_to_local_func (unsuitable_types);
exclude_returned_types (unsuitable_types);
exclude_escaping_types_1 (unsuitable_types);
}
/* This function analyzes whether the form of
structure is such that we are capable to transform it.
Nested structures are checked here. Unsuitable structure
types are added to UNSUITABLE_TYPE vector. */
static void
analyze_struct_form (VEC (tree, heap) **unsuitable_types)
{
d_str str;
unsigned i;
for (i = 0; VEC_iterate (structure, structures, i, str); i++)
check_struct_form (str, unsuitable_types);
}
/* This function looks for structure types instantiated in the program.
The candidate types are added to the structures vector.
Unsuitable types are collected into UNSUITABLE_TYPES vector. */
static void
build_data_structure (VEC (tree, heap) **unsuitable_types)
{
tree var, type;
tree var_list;
struct varpool_node *current_varpool;
struct cgraph_node *c_node;
/* Check global variables. */
FOR_EACH_STATIC_VARIABLE (current_varpool)
{
var = current_varpool->decl;
if (is_candidate (var, &type, unsuitable_types))
add_structure (type);
}
/* Now add structures that are types of function parameters and
local variables. */
for (c_node = cgraph_nodes; c_node; c_node = c_node->next)
{
enum availability avail =
cgraph_function_body_availability (c_node);
/* We need AVAIL_AVAILABLE for main function. */
if (avail == AVAIL_LOCAL || avail == AVAIL_AVAILABLE)
{
struct function *fn = DECL_STRUCT_FUNCTION (c_node->decl);
for (var = DECL_ARGUMENTS (c_node->decl); var;
var = TREE_CHAIN (var))
if (is_candidate (var, &type, unsuitable_types))
add_structure (type);
/* Check function local variables. */
for (var_list = fn->unexpanded_var_list; var_list;
var_list = TREE_CHAIN (var_list))
{
var = TREE_VALUE (var_list);
if (is_candidate (var, &type, unsuitable_types))
add_structure (type);
}
}
}
}
/* This function returns true if the program contains
a call to user defined allocation function, or other
functions that can interfere with struct-reorg optimizations. */
static bool
program_redefines_malloc_p (void)
{
struct cgraph_node *c_node;
struct cgraph_node *c_node2;
struct cgraph_edge *c_edge;
tree fndecl;
tree fndecl2;
tree call_expr;
for (c_node = cgraph_nodes; c_node; c_node = c_node->next)
{
fndecl = c_node->decl;
for (c_edge = c_node->callees; c_edge; c_edge = c_edge->next_callee)
{
call_expr = get_call_expr_in (c_edge->call_stmt);
c_node2 = c_edge->callee;
fndecl2 = c_node2->decl;
if (call_expr)
{
const char * fname = get_name (fndecl2);
if ((call_expr_flags (call_expr) & ECF_MALLOC) &&
(DECL_FUNCTION_CODE (fndecl2) != BUILT_IN_MALLOC) &&
(DECL_FUNCTION_CODE (fndecl2) != BUILT_IN_CALLOC) &&
(DECL_FUNCTION_CODE (fndecl2) != BUILT_IN_ALLOCA))
return true;
/* Check that there is no __builtin_object_size,
__builtin_offsetof, or realloc's in the program. */
if (DECL_FUNCTION_CODE (fndecl2) == BUILT_IN_OBJECT_SIZE
|| !strcmp (fname, "__builtin_offsetof")
|| !strcmp (fname, "realloc"))
return true;
}
}
}
return false;
}
/* In this function we assume that an allocation statement
var = (type_cast) malloc (size);
is converted into the following set of statements:
T.1 = size;
T.2 = malloc (T.1);
T.3 = (type_cast) T.2;
var = T.3;
In this function we collect into alloc_sites the allocation
sites of variables of structure types that are present
in structures vector. */
static void
collect_alloc_sites (void)
{
struct cgraph_node *node;
struct cgraph_edge *cs;
for (node = cgraph_nodes; node; node = node->next)
if (node->analyzed && node->decl)
{
for (cs = node->callees; cs; cs = cs->next_callee)
{
tree stmt = cs->call_stmt;
if (stmt)
{
tree call = get_call_expr_in (stmt);
tree decl;
if (call && (decl = get_callee_fndecl (call))
&& TREE_CODE (stmt) == GIMPLE_MODIFY_STMT)
{
unsigned i;
if (is_alloc_of_struct (stmt, &i))
{
/* We support only malloc now. */
if (DECL_FUNCTION_CODE (decl) == BUILT_IN_MALLOC)
{
d_str str;
str = VEC_index (structure, structures, i);
add_alloc_site (node->decl, stmt, str);
}
else
remove_structure (i);
}
}
}
}
}
}
/* Print collected accesses. */
static void
dump_accesses (void)
{
d_str str;
unsigned i;
if (!dump_file)
return;
for (i = 0; VEC_iterate (structure, structures, i, str); i++)
dump_accs (str);
}
/* This function checks whether the accesses of structures in condition
expressions are of the kind we are capable to transform.
If not, such structures are removed from the vector of structures. */
static void
check_cond_exprs (void)
{
d_str str;
unsigned i;
for (i = 0; VEC_iterate (structure, structures, i, str); i++)
if (str->accs)
htab_traverse (str->accs, safe_cond_expr_check, &i);
}
/* We exclude from non-field accesses of the structure
all statements that will be treated as part of the structure
allocation sites or field accesses. */
static void
exclude_alloc_and_field_accs (struct cgraph_node *node)
{
d_str str;
unsigned i;
for (i = 0; VEC_iterate (structure, structures, i, str); i++)
exclude_alloc_and_field_accs_1 (str, node);
}
/* This function collects accesses of the fields of the structures
that appear at function FN. */
static void
collect_accesses_in_func (struct function *fn)
{
basic_block bb;
if (! fn)
return;
/* Collect accesses for each basic blocks separately. */
FOR_EACH_BB_FN (bb, fn)
collect_accesses_in_bb (bb);
}
/* This function summarizes counts of the fields into the structure count. */
static void
sum_counts (d_str str, gcov_type *hotest)
{
int i;
str->count = 0;
for (i = 0; i < str->num_fields; i++)
{
if (dump_file)
{
fprintf (dump_file, "\nCounter of field \"");
print_generic_expr (dump_file, str->fields[i].decl, 0);
fprintf (dump_file, "\" is " HOST_WIDE_INT_PRINT_DEC,
str->fields[i].count);
}
str->count += str->fields[i].count;
}
if (dump_file)
{
fprintf (dump_file, "\nCounter of struct \"");
print_generic_expr (dump_file, str->decl, 0);
fprintf (dump_file, "\" is " HOST_WIDE_INT_PRINT_DEC, str->count);
}
if (str->count > *hotest)
*hotest = str->count;
}
/* This function peels the field into separate structure if it's
sufficiently hot, i.e. if its count provides at least 90% of
the maximum field count in the structure. */
static void
peel_hot_fields (d_str str)
{
gcov_type max_field_count;
sbitmap fields_left = sbitmap_alloc (str->num_fields);
int i;
sbitmap_ones (fields_left);
max_field_count =
(gcov_type) (get_max_field_count (str)/100)*90;
str->struct_clustering = NULL;
for (i = 0; i < str->num_fields; i++)
{
if (str->count >= max_field_count)
{
RESET_BIT (fields_left, i);
peel_field (i, str);
}
}
i = sbitmap_first_set_bit (fields_left);
if (i != -1)
gen_cluster (fields_left, str);
else
sbitmap_free (fields_left);
}
/* This function is a helper for do_reorg. It goes over
functions in call graph and performs actual transformation
on them. */
static void
do_reorg_1 (void)
{
struct cgraph_node *node;
/* Initialize the default bitmap obstack. */
bitmap_obstack_initialize (NULL);
for (node = cgraph_nodes; node; node = node->next)
if (node->analyzed && node->decl && !node->next_clone)
{
push_cfun (DECL_STRUCT_FUNCTION (node->decl));
current_function_decl = node->decl;
if (dump_file)
fprintf (dump_file, "\nFunction to do reorg is %s: \n",
(const char *) IDENTIFIER_POINTER (DECL_NAME (node->decl)));
do_reorg_for_func (node);
free_dominance_info (CDI_DOMINATORS);
free_dominance_info (CDI_POST_DOMINATORS);
current_function_decl = NULL;
pop_cfun ();
}
cfun = NULL;
}
/* This function creates new global struct variables.
For each original variable, the set of new variables
is created with the new structure types corresponding
to the structure type of original variable.
Only VAR_DECL variables are treated by this function. */
static void
create_new_global_vars (void)
{
struct varpool_node *current_varpool;
unsigned HOST_WIDE_INT i;
unsigned HOST_WIDE_INT varpool_size = 0;
for (i = 0; i < 2; i++)
{
if (i)
new_global_vars = htab_create (varpool_size,
new_var_hash, new_var_eq, NULL);
FOR_EACH_STATIC_VARIABLE(current_varpool)
{
tree var_decl = current_varpool->decl;
if (!var_decl || TREE_CODE (var_decl) != VAR_DECL)
continue;
if (!i)
varpool_size++;
else
create_new_var (var_decl, new_global_vars);
}
}
if (new_global_vars)
htab_traverse (new_global_vars, update_varpool_with_new_var, NULL);
}
/* Dump all new types generated by this optimization. */
static void
dump_new_types (void)
{
d_str str;
tree type;
unsigned i, j;
if (!dump_file)
return;
fprintf (dump_file, "\nThe following are the new types generated by"
" this optimization:\n");
for (i = 0; VEC_iterate (structure, structures, i, str); i++)
for (j = 0; VEC_iterate (tree, str->new_types, j, type); j++)
dump_struct_type (type, 2, 0);
}
/* This function creates new types to replace old structure types. */
static void
create_new_types (void)
{
d_str str;
unsigned i;
int str_num = 0;
for (i = 0; VEC_iterate (structure, structures, i, str); i++)
create_new_type (str, &str_num);
}
/* Free allocation sites hashtable. */
static void
free_alloc_sites (void)
{
if (alloc_sites)
htab_traverse (alloc_sites, free_falloc_sites, NULL);
htab_delete (alloc_sites);
alloc_sites = NULL;
}
/* This function collects structures potential
for peeling transformation, and inserts
them into structures hashtable. */
static void
collect_structures (void)
{
VEC (tree, heap) *unsuitable_types = VEC_alloc (tree, heap, 32);
structures = VEC_alloc (structure, heap, 32);
/* If program contains user defined mallocs, we give up. */
if (program_redefines_malloc_p ())
return;
/* Build data structures hashtable of all data structures
in the program. */
build_data_structure (&unsuitable_types);
/* This function analyzes whether the form of
structure is such that we are capable to transform it.
Nested structures are checked here. */
analyze_struct_form (&unsuitable_types);
/* This function excludes those structure types
that escape compilation unit. */
exclude_escaping_types (&unsuitable_types);
/* We do not want to change data layout of the structures with bitfields. */
exclude_types_with_bit_fields (&unsuitable_types);
remove_unsuitable_types (unsuitable_types);
VEC_free (tree, heap, unsuitable_types);
if (!VEC_length (structure, structures))
{
if (dump_file)
fprintf (dump_file, "\nNo structures to transform. Exiting...");
return;
}
}
/* Collect structure allocation sites. In case of arrays
we have nothing to do. */
static void
collect_allocation_sites (void)
{
alloc_sites = htab_create (32, malloc_hash, malloc_eq, NULL);
collect_alloc_sites ();
}
/* This function collects data accesses for the
structures to be transformed. For each structure
field it updates the count field in field_entry. */
static void
collect_data_accesses (void)
{
struct cgraph_node *c_node;
for (c_node = cgraph_nodes; c_node; c_node = c_node->next)
{
enum availability avail = cgraph_function_body_availability (c_node);
if (avail == AVAIL_LOCAL || avail == AVAIL_AVAILABLE)
{
struct function *func = DECL_STRUCT_FUNCTION (c_node->decl);
if (!c_node->next_clone)
collect_accesses_in_func (func);
exclude_alloc_and_field_accs (c_node);
}
}
check_cond_exprs ();
/* Print collected accesses. */
dump_accesses ();
}
/* We do not bother to transform cold structures.
Coldness of the structure is defined relatively
to the highest structure count among the structures
to be transformed. It's triggered by the compiler parameter
--param struct-reorg-cold-struct-ratio=<value>
where <value> ranges from 0 to 100. Structures with count ratios
that are less than this parameter are considered to be cold. */
static void
exclude_cold_structs (void)
{
gcov_type hotest = 0;
unsigned i;
d_str str;
/* We summarize counts of fields of a structure into the structure count. */
for (i = 0; VEC_iterate (structure, structures, i, str); i++)
sum_counts (str, &hotest);
/* Remove cold structures from structures vector. */
for (i = 0; VEC_iterate (structure, structures, i, str); i++)
if (str->count * 100 < (hotest * STRUCT_REORG_COLD_STRUCT_RATIO))
remove_structure (i);
}
/* This function decomposes original structure into substructures,
i.e.clusters. */
static void
peel_structs (void)
{
d_str str;
unsigned i;
for (i = 0; VEC_iterate (structure, structures, i, str); i++)
peel_hot_fields (str);
}
/* Stage 3. */
/* Do the actual transformation for each structure
from the structures hashtable. */
static void
do_reorg (void)
{
/* Check that there is a work to do. */
if (!VEC_length (structure, structures))
return;
/* Generate new types. */
create_new_types ();
dump_new_types ();
/* Create new global variables. */
create_new_global_vars ();
dump_new_vars (new_global_vars);
/* Decompose structures for each function separately. */
do_reorg_1 ();
/* Free auxiliary data collected for global variables. */
free_new_vars_htab (new_global_vars);
}
/* Free all auxiliary data used by this optimization. */
static void
free_data_structs (void)
{
free_structures ();
free_alloc_sites ();
}
/* Perform structure decomposition (peeling). */
static void
reorg_structs (void)
{
/* Stage1. */
/* Collect structure types. */
collect_structures ();
/* Collect structure allocation sites. */
collect_allocation_sites ();
/* Collect structure accesses. */
collect_data_accesses ();
/* We transform only hot structures. */
exclude_cold_structs ();
/* Stage2. */
/* Decompose structures into substructures, i.e. clusters. */
peel_structs ();
/* Stage3. */
/* Do the actual transformation for each structure
from the structures hashtable. */
do_reorg ();
/* Free all auxiliary data used by this optimization. */
free_data_structs ();
}
/* Struct-reorg optimization entry point function. */
static unsigned int
reorg_structs_drive (void)
{
reorg_structs ();
return 0;
}
/* Struct-reorg optimization gate function. */
static bool
struct_reorg_gate (void)
{
return flag_ipa_struct_reorg && flag_whole_program
&& (optimize > 0);
}
struct tree_opt_pass pass_ipa_struct_reorg =
{
"ipa_struct_reorg", /* name */
struct_reorg_gate, /* gate */
reorg_structs_drive, /* execute */
NULL, /* sub */
NULL, /* next */
0, /* static_pass_number */
TV_INTEGRATION, /* tv_id */
0, /* properties_required */
0, /* properties_provided */
0, /* properties_destroyed */
TODO_verify_ssa, /* todo_flags_start */
TODO_dump_func | TODO_verify_ssa, /* todo_flags_finish */
0 /* letter */
};
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: struct-reorg optimization
2007-10-21 19:58 ` Olga Golovanevsky
@ 2007-10-21 22:01 ` Diego Novillo
2007-10-24 14:40 ` Richard Guenther
0 siblings, 1 reply; 17+ messages in thread
From: Diego Novillo @ 2007-10-21 22:01 UTC (permalink / raw)
To: Olga Golovanevsky
Cc: Peter Bergner, Daniel Berlin, gcc-patches, Jan Hubicka,
Jan Hubicka, Kenneth Zadeck
On 10/21/07, Olga Golovanevsky <OLGA@il.ibm.com> wrote:
> * ipa-struct-reorg.c, ipa-struct-reorg.h: New files.
> * tree-pass.h: Add pass_ipa_struct_reorg.
> * common.opt: Add ipa-struct-reorg flag.
> * Makefile.in: Add ipa-strcut-reorg.o compilation.
> * passes.c: Add pass pass_ipa_struct_reorg.
> * params.h: Add STRUCT_REORG_COLD_STRUCT_RATIO.
> * params.def: Add PARAM_STRUCT_REORG_COLD_STRUCT_RATIO.
OK.
> (See attached file: ipa-struct-reorg.txt)(See attached file:
> ipa-struct-reorg.h)(See attached file: ipa-struct-reorg.c)
For future patches, when adding files you can generate a diff by doing
'svn add' and then generating a patch. The add operation does not
modify the repository, so it's safe.
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: struct-reorg optimization
2007-10-21 22:01 ` Diego Novillo
@ 2007-10-24 14:40 ` Richard Guenther
2007-10-24 14:45 ` Olga Golovanevsky
0 siblings, 1 reply; 17+ messages in thread
From: Richard Guenther @ 2007-10-24 14:40 UTC (permalink / raw)
To: Diego Novillo
Cc: Olga Golovanevsky, Peter Bergner, Daniel Berlin, gcc-patches,
Jan Hubicka, Jan Hubicka, Kenneth Zadeck
On 10/21/07, Diego Novillo <dnovillo@google.com> wrote:
> On 10/21/07, Olga Golovanevsky <OLGA@il.ibm.com> wrote:
>
> > * ipa-struct-reorg.c, ipa-struct-reorg.h: New files.
You forgot to commit those.
Richard.
> > * tree-pass.h: Add pass_ipa_struct_reorg.
> > * common.opt: Add ipa-struct-reorg flag.
> > * Makefile.in: Add ipa-strcut-reorg.o compilation.
> > * passes.c: Add pass pass_ipa_struct_reorg.
> > * params.h: Add STRUCT_REORG_COLD_STRUCT_RATIO.
> > * params.def: Add PARAM_STRUCT_REORG_COLD_STRUCT_RATIO.
>
> OK.
>
> > (See attached file: ipa-struct-reorg.txt)(See attached file:
> > ipa-struct-reorg.h)(See attached file: ipa-struct-reorg.c)
>
> For future patches, when adding files you can generate a diff by doing
> 'svn add' and then generating a patch. The add operation does not
> modify the repository, so it's safe.
>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: struct-reorg optimization
2007-10-24 14:40 ` Richard Guenther
@ 2007-10-24 14:45 ` Olga Golovanevsky
0 siblings, 0 replies; 17+ messages in thread
From: Olga Golovanevsky @ 2007-10-24 14:45 UTC (permalink / raw)
To: Richard Guenther
Cc: Peter Bergner, Daniel Berlin, Diego Novillo, gcc-patches,
Jan Hubicka, Jan Hubicka, Kenneth Zadeck
"Richard Guenther" <richard.guenther@gmail.com> wrote on 24/10/2007
14:38:34:
> On 10/21/07, Diego Novillo <dnovillo@google.com> wrote:
> > On 10/21/07, Olga Golovanevsky <OLGA@il.ibm.com> wrote:
> >
> > > * ipa-struct-reorg.c, ipa-struct-reorg.h: New files.
>
> You forgot to commit those.
thank you, sorry, committed.
Olga
>
> Richard.
>
> > > * tree-pass.h: Add pass_ipa_struct_reorg.
> > > * common.opt: Add ipa-struct-reorg flag.
> > > * Makefile.in: Add ipa-strcut-reorg.o compilation.
> > > * passes.c: Add pass pass_ipa_struct_reorg.
> > > * params.h: Add STRUCT_REORG_COLD_STRUCT_RATIO.
> > > * params.def: Add PARAM_STRUCT_REORG_COLD_STRUCT_RATIO.
> >
> > OK.
> >
> > > (See attached file: ipa-struct-reorg.txt)(See attached file:
> > > ipa-struct-reorg.h)(See attached file: ipa-struct-reorg.c)
> >
> > For future patches, when adding files you can generate a diff by doing
> > 'svn add' and then generating a patch. The add operation does not
> > modify the repository, so it's safe.
> >
^ permalink raw reply [flat|nested] 17+ messages in thread
* struct-reorg optimization
@ 2007-07-10 9:09 Olga Golovanevsky
0 siblings, 0 replies; 17+ messages in thread
From: Olga Golovanevsky @ 2007-07-10 9:09 UTC (permalink / raw)
To: Daniel Berlin, Diego Novillo; +Cc: Kenneth Zadeck, Jan Hubicka, gcc-patches
[-- Attachment #1: Type: text/plain, Size: 2781 bytes --]
This patch implements structure peeling, that is one of struct-reorg
optimizations developed on struct-reorg-branch. A structure type can be
peeled into separate fields, or groups of them, according to profile
information, or deliberately. The gain is to utilize spatial locality,
when, for example, one of the fields is sequentially accessed through
array elements. Then separating this field from others allows to make
separation in array allocation, and thus release cache from irrelevant
data fetched with other fields.
The full description of this optimization is at the beginning of
struct-reorg.c (attached below).
The optimization is global, both types of accesses - through pointers
and array refs - are handled. When allocation is made through malloc,
it is replaced by allocations of peeled structures. Only build-in malloc
is supported right now, but it can be easily extended.
In this patch, the decision how to peel is made by frequency based model,
where frequency of field accesses can be calculated through profiling or
statically predicted as with -fbranch-probabilitied flag. When there is
no such info, structure gets completely peeled. (The struct-reorg-branch
contains much more accurate model which is based on the calculation of
distance between accesses, but it's also heavier.)
The -fipa-struct-reorg flag activates this optimization.
The patch makes strong use of ipa-type-escape analysis, that provide
safety of this optimization. Also changes required for ipa-type-escape
to support POITER_PLUS_EXPR are part of this patch.
With this patch:
- two structures f1_neuron and xyz got peeled in 179. art ,
+44% w/o profiling and +33% with profiling
- the structures NEXT_MOVE got peeled, the structure CHESS_POSITION
can be peeled, but interfere with another optimization that
generated BIT_FIELD_REFs in 186.crafty; many other structures escape,
no influence on performance
- the structure MMNODE got peeled; many others escape,
no influence on performance
Bootstraped and tested (except fortran) on ppc .
:ADDPATCH ipa ssa:
Ok for mainline?
2007-07-09 Olga Golovanevsky <olga@il.ibm.com>
* struct-reorg.c, struct-reorg.h: New files.
* ipa-type-escape.h: Expose function
is_array_access_through_pointer_and_index.
* ipa-type-escape.c
(is_array_access_through_pointer_and_index):
Add three new parameters. Add support of
POINTER_PLUS_EXPR code.
* tree.h: Add pass_ipa_struct_reorg.
* common.opt: Add ipa-struct-reorg flag.
* Makefile.in: Add strcut-reorg.o compilation.
* passes.c: Add pass pass_ipa_struct_reorg.
(See attached file: struct-reorg.txt)(See attached file: struct-reorg.c)
(See attached file: struct-reorg.h)
[-- Attachment #2: struct-reorg.txt --]
[-- Type: text/plain, Size: 8217 bytes --]
Index: tree-pass.h
===================================================================
--- tree-pass.h (revision 126190)
+++ tree-pass.h (working copy)
@@ -331,6 +331,7 @@
extern struct tree_opt_pass pass_ipa_pure_const;
extern struct tree_opt_pass pass_ipa_type_escape;
extern struct tree_opt_pass pass_ipa_pta;
+extern struct tree_opt_pass pass_ipa_struct_reorg;
extern struct tree_opt_pass pass_early_local_passes;
extern struct tree_opt_pass pass_ipa_increase_alignment;
extern struct tree_opt_pass pass_ipa_function_and_variable_visibility;
Index: ipa-type-escape.c
===================================================================
--- ipa-type-escape.c (revision 126190)
+++ ipa-type-escape.c (working copy)
@@ -926,56 +926,90 @@
*/
-static bool
-is_array_access_through_pointer_and_index (tree op0, tree op1)
+bool
+is_array_access_through_pointer_and_index (enum tree_code code, tree op0, tree op1,
+ tree * base, tree * offset,
+ tree * offset_cast_stmt)
{
- tree base, offset, offset_cast_stmt;
tree before_cast, before_cast_def_stmt;
cast_t op0_cast, op1_cast;
+ *base = NULL;
+ *offset = NULL;
+ *offset_cast_stmt = NULL;
+
/* Check 1. */
- /* Init data for walk_use_def_chains function. */
- op0_cast.type = op1_cast.type = 0;
- op0_cast.stmt = op1_cast.stmt = NULL;
+ if (code == POINTER_PLUS_EXPR)
+ {
+ tree op0type = TYPE_MAIN_VARIANT (TREE_TYPE (op0));
+ tree op1type = TYPE_MAIN_VARIANT (TREE_TYPE (op1));
- visited_stmts = pointer_set_create ();
- walk_use_def_chains (op0, is_cast_from_non_pointer,(void *)(&op0_cast), false);
- pointer_set_destroy (visited_stmts);
+ /* One of op0 and op1 is of pointer type and the other is numerical. */
+ if (POINTER_TYPE_P (op0type)
+ && NUMERICAL_TYPE_CHECK (op1type))
+ {
+ *base = op0;
+ *offset = op1;
+ }
+ else if (POINTER_TYPE_P (op1type)
+ && NUMERICAL_TYPE_CHECK (op0type))
+ {
+ *base = op1;
+ *offset = op0;
+ }
+ else
+ return false;
+ }
+ else
+ {
+ /* Init data for walk_use_def_chains function. */
+ op0_cast.type = op1_cast.type = 0;
+ op0_cast.stmt = op1_cast.stmt = NULL;
- visited_stmts = pointer_set_create ();
- walk_use_def_chains (op1, is_cast_from_non_pointer,(void *)(&op1_cast), false);
- pointer_set_destroy (visited_stmts);
+ visited_stmts = pointer_set_create ();
+ walk_use_def_chains (op0, is_cast_from_non_pointer,(void *)(&op0_cast), false);
+ pointer_set_destroy (visited_stmts);
- if (op0_cast.type == 1 && op1_cast.type == 0)
- {
- base = op1;
- offset = op0;
- offset_cast_stmt = op0_cast.stmt;
+ visited_stmts = pointer_set_create ();
+ walk_use_def_chains (op1, is_cast_from_non_pointer,(void *)(&op1_cast), false);
+ pointer_set_destroy (visited_stmts);
+
+ if (op0_cast.type == 1 && op1_cast.type == 0)
+ {
+ *base = op1;
+ *offset = op0;
+ *offset_cast_stmt = op0_cast.stmt;
+ }
+ else if (op0_cast.type == 0 && op1_cast.type == 1)
+ {
+ *base = op0;
+ *offset = op1;
+ *offset_cast_stmt = op1_cast.stmt;
+ }
+ else
+ return false;
}
- else if (op0_cast.type == 0 && op1_cast.type == 1)
- {
- base = op0;
- offset = op1;
- offset_cast_stmt = op1_cast.stmt;
- }
- else
- return false;
/* Check 2.
offset_cast_stmt is of the form:
D.1606_7 = (struct str_t *) D.1605_6; */
- before_cast = SINGLE_SSA_TREE_OPERAND (offset_cast_stmt, SSA_OP_USE);
- if (!before_cast)
- return false;
+ if (*offset_cast_stmt)
+ {
+ before_cast = SINGLE_SSA_TREE_OPERAND (*offset_cast_stmt, SSA_OP_USE);
+ if (!before_cast)
+ return false;
- if (SSA_NAME_IS_DEFAULT_DEF(before_cast))
- return false;
+ if (SSA_NAME_IS_DEFAULT_DEF(before_cast))
+ return false;
- before_cast_def_stmt = SSA_NAME_DEF_STMT (before_cast);
- if (!before_cast_def_stmt)
- return false;
+ before_cast_def_stmt = SSA_NAME_DEF_STMT (before_cast);
+ if (!before_cast_def_stmt)
+ return false;
+ }
+ else
+ before_cast_def_stmt = SSA_NAME_DEF_STMT (*offset);
/* before_cast_def_stmt should be of the form:
D.1605_6 = i.1_5 * 16; */
@@ -1449,7 +1483,6 @@
okay_pointer_operation (enum tree_code code, tree op0, tree op1)
{
tree op0type = TYPE_MAIN_VARIANT (TREE_TYPE (op0));
- tree op1type = TYPE_MAIN_VARIANT (TREE_TYPE (op1));
switch (code)
{
@@ -1459,11 +1492,15 @@
break;
case MINUS_EXPR:
case PLUS_EXPR:
+ case POINTER_PLUS_EXPR:
{
- if (POINTER_TYPE_P (op1type)
+ tree base, offset, offset_cast_stmt;
+
+ if (POINTER_TYPE_P (op0type)
&& TREE_CODE (op0) == SSA_NAME
&& TREE_CODE (op1) == SSA_NAME
- && is_array_access_through_pointer_and_index (op0, op1))
+ && is_array_access_through_pointer_and_index (code, op0, op1, &base,
+ &offset, &offset_cast_stmt))
return true;
else
{
@@ -1541,7 +1578,7 @@
of circumstances and if the moon is in the correct
place could be safe, but it is hard to see how this
is worth the effort. */
-
+
if (type0 && POINTER_TYPE_P (type0)
&& !okay_pointer_operation (TREE_CODE (rhs), op0, op1))
mark_interesting_type (type0, FULL_ESCAPE);
Index: ipa-type-escape.h
===================================================================
--- ipa-type-escape.h (revision 126190)
+++ ipa-type-escape.h (working copy)
@@ -27,6 +27,7 @@
bool ipa_type_escape_field_does_not_clobber_p (tree record_type, tree field_type);
int ipa_type_escape_star_count_of_interesting_type (tree type);
int ipa_type_escape_star_count_of_interesting_or_array_type (tree type);
+bool is_array_access_through_pointer_and_index (enum tree_code, tree, tree, tree *, tree *, tree *);
#endif /* GCC_IPA_TYPE_ESCAPE_H */
Index: common.opt
===================================================================
--- common.opt (revision 126190)
+++ common.opt (working copy)
@@ -601,6 +601,11 @@
Perform matrix layout flattening and transposing based
on profiling information.
+fipa-struct-reorg
+Common Report Var(flag_ipa_struct_reorg)
+Perform structure layout optimizations based
+on profiling information.
+
fivopts
Common Report Var(flag_ivopts) Init(1) Optimization
Optimize induction variables on trees
Index: Makefile.in
===================================================================
--- Makefile.in (revision 126190)
+++ Makefile.in (working copy)
@@ -1185,6 +1185,7 @@
ipa-utils.o \
ipa.o \
matrix-reorg.o \
+ struct-reorg.o \
tree-inline.o \
tree-nomudflap.o \
varpool.o
@@ -2449,6 +2450,10 @@
pointer-set.h $(GGC_H) $(IPA_TYPE_ESCAPE_H) $(IPA_UTILS_H) $(C_COMMON_H) \
$(TREE_GIMPLE_H) $(CGRAPH_H) output.h $(FLAGS_H) tree-pass.h \
$(DIAGNOSTIC_H) $(FUNCTION_H)
+struct-reorg.o: struct-reorg.c struct-reorg.h $(CONFIG_H) $(SYSTEM_H) coretypes.h \
+ $(TM_H) $(RTL_H) $(TREE_H) $(FLAGS_H) output.h $(REGS_H) $(EXPR_H) $(FUNCTION_H) \
+ toplev.h $(GGC_H) $(TARGET_H) langhooks.h $(COVERAGE_H) libfuncs.h \
+ gt-coverage.h $(HASHTAB_H) $(IPA_TYPE_ESCAPE_H)
coverage.o : coverage.c $(GCOV_IO_H) $(CONFIG_H) $(SYSTEM_H) coretypes.h \
$(TM_H) $(RTL_H) $(TREE_H) $(FLAGS_H) output.h $(REGS_H) $(EXPR_H) \
@@ -3011,7 +3016,7 @@
$(srcdir)/reload.h \
$(srcdir)/alias.c $(srcdir)/bitmap.c $(srcdir)/cselib.c $(srcdir)/cgraph.c \
$(srcdir)/ipa-prop.c $(srcdir)/ipa-cp.c $(srcdir)/ipa-inline.c $(srcdir)/matrix-reorg.c \
- $(srcdir)/dbxout.c $(srcdir)/dwarf2out.c $(srcdir)/dwarf2asm.c \
+ $(srcdir)/dbxout.c $(srcdir)/struct-reorg.c $(srcdir)/dwarf2out.c $(srcdir)/dwarf2asm.c \
$(srcdir)/dojump.c \
$(srcdir)/emit-rtl.c $(srcdir)/except.c $(srcdir)/explow.c $(srcdir)/expr.c \
$(srcdir)/function.c $(srcdir)/except.h \
Index: passes.c
===================================================================
--- passes.c (revision 126190)
+++ passes.c (working copy)
@@ -542,6 +542,7 @@
NEXT_PASS (pass_ipa_pure_const);
NEXT_PASS (pass_ipa_type_escape);
NEXT_PASS (pass_ipa_pta);
+ NEXT_PASS (pass_ipa_struct_reorg);
*p = NULL;
/* These passes are run after IPA passes on every function that is being
[-- Attachment #3: struct-reorg.c --]
[-- Type: application/octet-stream, Size: 84522 bytes --]
/* Struct-reorg optimization.
Copyright (C) 2004, 2005, 2006, 2007 Free Software Foundation, Inc.
Contributed by Olga Golovanevsky <olga@il.ibm.com>
(initially version of this code was developed
by Caroline Tice and Mostafa Hagoge)
This file is part of GCC.
GCC is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free
Software Foundation; either version 2, or (at your option) any later
version.
GCC is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
for more details.
You should have received a copy of the GNU General Public License
along with GCC; see the file COPYING. If not, write to the Free
Software Foundation, 59 Temple Place - Suite 330, Boston, MA
02111-1307, USA. */
#include "config.h"
#include "system.h"
#include "coretypes.h"
#include "tm.h"
#include "tree.h"
#include "rtl.h"
#include "tree-gimple.h"
#include "tree-inline.h"
#include "tree-flow.h"
#include "tree-flow-inline.h"
#include "langhooks.h"
#include "pointer-set.h"
#include "hashtab.h"
#include "c-tree.h"
#include "toplev.h"
#include "flags.h"
#include "ggc.h"
#include "debug.h"
#include "target.h"
#include "cgraph.h"
#include "diagnostic.h"
#include "timevar.h"
#include "params.h"
#include "fibheap.h"
#include "intl.h"
#include "function.h"
#include "basic-block.h"
#include "tree-iterator.h"
#include "tree-pass.h"
#include "struct-reorg.h"
#include "opts.h"
#include "ipa-type-escape.h"
#include "tree-dump.h"
#include "c-common.h"
/* This optimization implements structure peeling.
For example, given a structure type:
typedef struct
{
int a;
float b;
int c;
}str_t;
it can be peeled into two structure types as follows:
typedef struct and typedef struct
{ {
int a; float b;
int c; } str_t_1;
}str_t_0;
or can be fully peeled:
typedef struct
{
int a;
}str_t_0;
typedef struct
{
float b;
}str_t_1;
typedef struct
{
int c;
}str_t_2;
When structure type is peeled all instances and their accesses
in the program are updated accordingly. For example, if there is
array of structures:
str_t A[N];
and structure type str_t was peeled into two structures str_t_0
and str_t_1 as it was shown above, then array A will be replaced
by two arrays as foolows:
str_t_0 A_0[N];
str_t_1 A_1[N];
The field access of fields a of ellement i of array A: A[i].a will be
replace by access to field a of ellement i of array A_0: A_0[i].a.
This optimization also supports dynamically allocated arrays.
If array of structures was allocated by malloc function:
str_t * p = (str_t *) malloc (sizeof (str_t) * N)
the allocation site will be replaced to reflect new structure types:
str_t_0 * p_0 = (str_t_0 *) malloc (sizeof (str_t_0) * N)
str_t_1 * p_1 = (str_t_1 *) malloc (sizeof (str_t_1) * N)
The field access through the pointer p[i].a will be changed by p_0[i].a.
The decision how to peel structure gains to utilize spatial locality.
For example, if one of fields of structure has intensive use in the loop:
for (i = 0; i < N; i++)
{
... = A[i].a;
}
the allocation of field a of str_t in the memory in contiguous manner will
increase probablity of relevant data to be fatched into cache.
The analysis part of this optimization is based on the frequency of
field accesses, which are collected all over the program. Then the simple
clustering algorithm is used for partitioning into clusters, where each cluster
define new structure type. The parameter MAX_DIST_IN_CLUSTER_RATIO control
the level of closeness of elements in cluster, i.e. if there are fields f1, f2, and
the distance between them is defined as absolute value of difference between there
frequencies
dist (f1, f2) = abs (freq (f1) - freq (f2)),
then field f will be added to the cluster, if for each field q in the cluster
the following statement keeps true:
dist (q, f) <= (max_struct_distance/100) * MAX_DIST_IN_CLUSTER_RATIO
where max_struct_distance is maximum distance between two fields in the structure.
If profiling information is provided, which can be static as with branch-probabilities
flag, or dynamic as with profile-generate/profile-use flags, its is used to
calculate the frequency of field accesses. Otherwise, structure is fully peeled.
The ipa-type-escape analysis are used to provide safety of the peeling.
The optimization is activated by flag -fipa-struct-reorg.
*/
/* Ratio of the structure count to the hottest
structure count to consider the structure cold. */
#define COLD_STRUCTURE_RATIO 10
/* Ratio of maximum distance between fields allowed in cluster.
Used only when running with profiling. */
#define MAX_DIST_IN_CLUSTER_RATIO 1
/* Data about the new program variables created (for the new peeled types).
When doing struct peeling, each variable of the original struct type will
need to be replaced with a set of new variables, one new variable for each
new type created from the original type. */
struct new_var_data {
tree orig_var; /* Var decl for original struct type */
struct struct_tree_list *new_vars; /* List of new vars to replace the
original var; one new var for each
new type peeled off the original
type. */
struct new_var_data *next;
};
typedef struct new_var_data * new_var;
struct malloc_call_new
{
tree malloc_call_stmt;
d_str str;
/* Next malloc call stmt in the same function. */
struct malloc_call_new *next;
};
/* List of data for all malloc calls in the program. */
struct malloc_struct_new
{
tree func;
struct malloc_call_new *malloc_list; /* List of call sites for each
call to malloc from the current
func. */
struct malloc_struct_new *next;
};
typedef struct malloc_struct_new *malloc_d;
typedef struct malloc_call_new *malloc_call;
struct malloc_struct_new *malloc_data_list_new = NULL;
/* List of new global variables. Generated once for whole program. */
new_var new_global_vars = NULL;
/* List of new local variables. Generated once for function. */
new_var new_local_vars = NULL;
/* Use stmts of current function. */
tree_list use_stmts = NULL;
/* List of structures to be transformed. */
struct struct_list
{
struct data_structure *struct_data;
struct struct_list *next;
};
typedef struct struct_list * list;
/* List of structs to be transformed. */
list data_struct_list = NULL;
static new_var is_in_new_vars_list (tree, new_var);
static void add_struct_to_data_struct_list (tree);
static void add_call_to_malloc_list (tree, tree, d_str);
static inline void update_fields_mapping (struct field_cluster *, tree,
struct data_field_entry *, int);
static void create_new_var (tree, new_var *);
static tree get_final_malloc_stmt (tree malloc_stmt);
static tree_list add_tree_to_tree_list (tree, tree_list);
static void free_tree_list (tree_list);
static malloc_d get_malloc_list_for_func (tree);
static struct field_access_site * add_field_acc_to_acc_sites (tree, tree, tree, tree,
tree, tree, tree, tree,
tree, struct field_access_site *);
static struct access_site * add_access_to_acc_sites (tree, tree,
struct access_site *);
static bool is_result_of_mult (tree, tree *, tree);
static void remove_struct_from_data_struct_list (list);
static tree gen_size (tree, tree, tree *);
static tree gen_cast_stmt (tree, tree, tree, tree *);
static void create_new_stmts_for_general_acc (struct access_site *, d_str);
static void create_new_stmts_for_cond_expr (tree);
static bool is_equal_types (tree, tree);
static tree * find_pos_in_stmt (tree, tree);
/* Strip type from pointers and arrays. */
static inline tree
strip_type (tree type)
{
gcc_assert (TYPE_P (type));
while (POINTER_TYPE_P (type)
|| TREE_CODE (type) == ARRAY_TYPE)
type = TREE_TYPE (type);
return type;
}
/* Returns type of VAR. */
static inline tree
get_type_of_var (tree var)
{
if (!var)
return NULL;
if (TREE_CODE (var) == PARM_DECL)
return DECL_ARG_TYPE (var);
else
return TREE_TYPE (var);
}
/* Check whether the type of VAR is suitable for peeling.
Returns true if yes, false otherwise. */
static bool
is_candidate_for_data_struct_list (tree var, tree *type_p,
bitmap non_suitable_types)
{
tree type;
bool initialized = false;
*type_p = NULL;
if (!var)
return false;
/* There is no support of initialized vars. */
if (TREE_CODE (var) == VAR_DECL
&& DECL_INITIAL (var) != NULL_TREE)
initialized = true;
type = get_type_of_var (var);
if (type)
{
type = TYPE_MAIN_VARIANT (strip_type (type));
*type_p = type;
if (initialized && non_suitable_types)
bitmap_set_bit (non_suitable_types, TYPE_UID (type));
if (TREE_CODE (type) == RECORD_TYPE)
{
return true;
}
else
return false;
}
else
return false;
}
/* Given a tree representing a type, this function returns the
name of the type, as a string. */
static char *
get_type_name (tree type_node)
{
if (! TYPE_NAME (type_node))
return NULL;
if (TREE_CODE (TYPE_NAME (type_node)) == IDENTIFIER_NODE)
return (char *) IDENTIFIER_POINTER (TYPE_NAME (type_node));
else if (TREE_CODE (TYPE_NAME (type_node)) == TYPE_DECL
&& DECL_NAME (TYPE_NAME (type_node)))
return (char *) IDENTIFIER_POINTER (DECL_NAME (TYPE_NAME (type_node)));
else
return NULL;
}
static void
print_new_vars (new_var new_vars)
{
new_var current;
tree_list cur_var;
tree var_type;
if (!dump_file)
return;
for (current = new_vars; current; current = current->next)
{
var_type = get_type_of_var (current->orig_var);
fprintf (dump_file, "\nOrig var: ");
print_generic_expr (dump_file, current->orig_var, 0);
fprintf (dump_file, " of type ");
print_generic_expr (dump_file, var_type, 0);
fprintf (dump_file, "\n");
for (cur_var = current->new_vars; cur_var; cur_var = cur_var->next)
{
tree var = cur_var->data;
var_type = get_type_of_var (var);
fprintf (dump_file, " ");
print_generic_expr (dump_file, var, 0);
fprintf (dump_file, " of type ");
print_generic_expr (dump_file, var_type, 0);
fprintf (dump_file, "\n");
}
}
}
/* Print structure type. */
static void
print_struct_type (tree new_type, char * indent)
{
char * new_indent;
char * struct_name;
tree cur_field;
int len;
int i;
if (!new_type || !dump_file)
return;
if (TREE_CODE (new_type) != RECORD_TYPE)
return;
struct_name = get_type_name (new_type);
if (struct_name)
{
fprintf (dump_file, "%s%s {\n", indent, struct_name);
len = strlen (struct_name) + strlen (indent) + 3;
}
else
{
fprintf (dump_file, "%s{\n", indent);
len = strlen (indent) + 2;
}
new_indent = (char *) xmalloc (len * sizeof (char));
memset (new_indent, ' ', len);
new_indent[len] = '\0';
for (cur_field = TYPE_FIELDS (new_type); cur_field;
cur_field = TREE_CHAIN (cur_field))
{
tree field_type;
int ptr_count = 0;
field_type = TREE_TYPE (cur_field);
while (POINTER_TYPE_P (field_type))
{
ptr_count++;
field_type = TREE_TYPE (field_type);
}
fprintf (dump_file, "%s%s ", new_indent, get_type_name (field_type));
for (i = 0; i < ptr_count; i++)
fprintf (dump_file, "*");
fprintf (dump_file, " %s;\n",
IDENTIFIER_POINTER (DECL_NAME (cur_field)));
}
fprintf (dump_file, "%s}\n", indent);
}
/* Print out the list of new types generated by this optimization. */
static void
print_new_types (void)
{
list tmp;
char indent[3] = " ";
if (!dump_file)
return;
fprintf (dump_file,
"\nThe following are the new types generated by"
" this optimization:\n");
for (tmp = data_struct_list; tmp; tmp = tmp->next)
{
d_str str = tmp->struct_data;
tree_list new_type;
for (new_type = str->new_types; new_type; new_type = new_type->next)
print_struct_type (new_type->data, indent);
}
}
static tree
find_field_in_struct_1 (tree str_type, tree field)
{
tree str_field;
for (str_field = TYPE_FIELDS (str_type); str_field;
str_field = TREE_CHAIN (str_field))
{
char * str_field_name;
char * field_name;
str_field_name = (char *) IDENTIFIER_POINTER (DECL_NAME (str_field));
field_name = (char *) IDENTIFIER_POINTER (DECL_NAME (field));
gcc_assert (str_field_name);
gcc_assert (field_name);
if (!strcmp (str_field_name, field_name))
{
/* Check their types. */
if (is_equal_types (TREE_TYPE (str_field), TREE_TYPE (field)))
return str_field;
}
}
return NULL_TREE;
}
/* Given the field declaration tree (FIELD_DECL) returns the
appropriate field entry in DS. */
static struct data_field_entry *
find_field_in_struct (d_str str, tree field_decl)
{
int i;
tree field = find_field_in_struct_1 (str->decl, field_decl);
for (i = 0; i < str->num_fields; i++)
if (str->fields[i].decl == field)
return &(str->fields[i]);
return NULL;
}
/* This function is a hack to overcome the types problem.
When number of compilation units are compiled together
under -combine flag, the TYPE_MAIN_VARIANT cannot
be structed. First we comparing names, but they do not
also appear, like for example, in stmts. Only then we
are going inside structures.
*/
static bool
is_equal_types (tree type1, tree type2)
{
char * name1,* name2;
if ((!type1 && type2)
||(!type2 && type1))
return false;
if (!type1 && !type2)
return true;
if (TREE_CODE (type1) != TREE_CODE (type2))
return false;
if (TYPE_MAIN_VARIANT (type1) == TYPE_MAIN_VARIANT (type2))
return true;
name1 = get_type_name (type1);
name2 = get_type_name (type2);
if (name1 && name2 && !strcmp (name1, name2))
return true;
if (name1 && name2 && strcmp (name1, name2))
return false;
switch (TREE_CODE (type1))
{
case POINTER_TYPE:
case REFERENCE_TYPE:
{
return is_equal_types (TREE_TYPE (type1), TREE_TYPE (type2));
}
break;
case RECORD_TYPE:
case UNION_TYPE:
case QUAL_UNION_TYPE:
case ENUMERAL_TYPE:
{
tree field1;
/* Compare fields of struture. */
for (field1 = TYPE_FIELDS (type1); field1;
field1 = TREE_CHAIN (field1))
{
tree field2 = find_field_in_struct_1 (type2, field1);
if (!field2)
return false;
}
return true;
}
break;
default:
gcc_unreachable();
}
return false;
}
/* This function returns data_structure entry
in the data_struct_list represented by TYPE, if found. */
static list
find_struct_in_list (tree type)
{
list tmp = data_struct_list;
while (tmp)
{
if (is_equal_types (tmp->struct_data->decl, TYPE_MAIN_VARIANT (type)))
return tmp;
tmp = tmp->next;
}
return NULL;
}
/* Check whether the indirect_ref is of the form we can deal with. */
static bool
is_complicated_indirect_ref (tree str_decl, tree ref, tree * num_p, tree * offset_p,
tree * base_p, tree * ref_def_stmt_p, tree * cast_stmt_p)
{
tree ref_var;
tree rhs, struct_size, op0, op1;
tree before_cast;
ref_var = TREE_OPERAND (ref, 0);
if (TREE_CODE (ref_var) != SSA_NAME)
return true;
*ref_def_stmt_p = SSA_NAME_DEF_STMT (ref_var);
if (!(*ref_def_stmt_p)
|| (TREE_CODE (*ref_def_stmt_p) != GIMPLE_MODIFY_STMT))
return true;
rhs = GIMPLE_STMT_OPERAND (*ref_def_stmt_p, 1);
if (TREE_CODE (rhs) != PLUS_EXPR
&& TREE_CODE (rhs)!= MINUS_EXPR
&& TREE_CODE (rhs) != POINTER_PLUS_EXPR)
return true;
op0 = TREE_OPERAND (rhs, 0);
op1 = TREE_OPERAND (rhs, 1);
if (!is_array_access_through_pointer_and_index (TREE_CODE (rhs), op0, op1,
base_p, offset_p,
cast_stmt_p))
return true;
if (*cast_stmt_p)
before_cast = SINGLE_SSA_TREE_OPERAND (*cast_stmt_p, SSA_OP_USE);
else
before_cast = *offset_p;
if (!before_cast)
return false;
if (SSA_NAME_IS_DEFAULT_DEF(before_cast))
return false;
struct_size = TYPE_SIZE_UNIT (str_decl);
if (!is_result_of_mult (before_cast, num_p, struct_size))
return true;
return false;
}
static bool
is_complicated_access (tree str_decl, tree ref, tree * num_p, tree * offset_p,
tree * base_p, tree * ref_def_stmt_p, tree * cast_stmt_p)
{
if (TREE_CODE (ref) == INDIRECT_REF)
return is_complicated_indirect_ref (str_decl, ref, num_p, offset_p,
base_p, ref_def_stmt_p, cast_stmt_p);
else if (TREE_CODE (ref) == ARRAY_REF)
return false;
else if (TREE_CODE (ref) == VAR_DECL)
return false;
return true;
}
/* Helper for walk_tree called from collect_accesses_in_bb function. */
static tree
get_stmt_accesses (tree *tp, int *walk_subtrees, void *data)
{
tree stmt = (tree) data;
tree t = *tp;
if (!t)
return NULL;
switch (TREE_CODE (t))
{
case BIT_FIELD_REF:
{
tree str = TREE_OPERAND(t, 0);
tree type = TYPE_MAIN_VARIANT (strip_type (get_type_of_var (str)));
list tmp = find_struct_in_list (type);
if (tmp)
remove_struct_from_data_struct_list (tmp);
}
break;
case GIMPLE_MODIFY_STMT:
{
tree lhs = GIMPLE_STMT_OPERAND (t, 0);
tree rhs = GIMPLE_STMT_OPERAND (t, 1);
*walk_subtrees=1;
walk_tree (&lhs, get_stmt_accesses, data, NULL);
walk_tree (&rhs, get_stmt_accesses, data, NULL);
*walk_subtrees=0;
}
break;
case COMPONENT_REF:
{
tree ref = TREE_OPERAND (t, 0);
tree field_decl = TREE_OPERAND (t, 1);
if ((TREE_CODE (ref) == INDIRECT_REF
|| TREE_CODE (ref) == ARRAY_REF
|| TREE_CODE (ref) == VAR_DECL)
&& TREE_CODE (field_decl) == FIELD_DECL)
{
tree type = TYPE_MAIN_VARIANT (TREE_TYPE (ref));
list tmp = find_struct_in_list (type);
if (tmp)
{
d_str str = tmp->struct_data;
struct data_field_entry * field =
find_field_in_struct (str, field_decl);
if (field)
{
tree num = NULL;
tree offset = NULL;
tree base = NULL;
tree ref_def_stmt = NULL;
tree cast_stmt = NULL;
/* Check whether the access is of the form we can deal with. */
if (is_complicated_access (str->decl, ref, &num, &offset,
&base, &ref_def_stmt, &cast_stmt))
remove_struct_from_data_struct_list (tmp);
else
{
/* Increase count of field. */
basic_block bb = bb_for_stmt (stmt);
field->count += bb->count;
/* Add stmt to the acc_sites list of field. */
field->acc_sites =
add_field_acc_to_acc_sites (stmt, ref, t, field_decl, num, offset,
base, ref_def_stmt, cast_stmt,
field->acc_sites);
}
*walk_subtrees = 0;
}
}
}
}
break;
case MINUS_EXPR:
case PLUS_EXPR:
{
tree op0 = TREE_OPERAND (t, 0);
tree op1 = TREE_OPERAND (t, 1);
*walk_subtrees=1;
walk_tree (&op0, get_stmt_accesses, data, NULL);
walk_tree (&op1, get_stmt_accesses, data, NULL);
*walk_subtrees=0;
}
break;
case COND_EXPR:
{
tree cond = COND_EXPR_COND (t);
int i;
for (i = 0; i < TREE_CODE_LENGTH (TREE_CODE (cond)); i++)
{
tree t = TREE_OPERAND (cond, i);
*walk_subtrees=1;
walk_tree (&t, get_stmt_accesses, data, NULL);
}
*walk_subtrees=0;
}
break;
case VAR_DECL:
case SSA_NAME:
{
list tmp;
if (TREE_CODE (t) == SSA_NAME)
t = SSA_NAME_VAR (t);
tmp = find_struct_in_list (strip_type (get_type_of_var (t)));
if (tmp)
{
d_str str = tmp->struct_data;
str->accs =
add_access_to_acc_sites (stmt, t, str->accs);
}
*walk_subtrees=0;
}
break;
case CALL_EXPR:
{
/* It was checked as part of stage1 that structs
to be transformed cannot be passed as parameters to function. */
*walk_subtrees=0;
}
break;
default:
return NULL;
}
return NULL;
}
/* Collect accesses to the fields of DS in BB. */
static void
collect_accesses_in_bb (basic_block bb)
{
block_stmt_iterator bsi;
/* Go over the basic block statements collecting
accesses to fields of structures. */
for (bsi = bsi_start (bb); ! bsi_end_p (bsi); bsi_next (&bsi))
{
tree stmt = bsi_stmt (bsi);
walk_tree (&stmt, get_stmt_accesses, stmt, NULL);
}
}
/* Collect accesses of the fields of the structures in function FN. */
static void
collect_accesses_in_func (struct function *fn)
{
basic_block bb;
if (! fn)
return;
/* Collect accesses in each one of the basic blocks. */
FOR_EACH_BB_FN (bb, fn)
collect_accesses_in_bb (bb);
}
/* This functions builds structure with fields FIELDS,
that should be TREE_CHAIN, named NAME, with the same
attributes as ORIG_STRUCT. */
static tree
build_basic_struct (tree fields, char * name, tree orig_struct)
{
tree attributes = NULL_TREE;
tree ref = 0;
tree x;
/* From start_struct () */
if (TYPE_ATTRIBUTES (orig_struct))
attributes = copy_node (TYPE_ATTRIBUTES (orig_struct));
ref = make_node (RECORD_TYPE);
C_TYPE_BEING_DEFINED (ref) = 1;
TYPE_PACKED (ref) = flag_pack_struct;
/* From finish_struct (ref, tree fieldlist, tree attributes); */
TYPE_SIZE (ref) = 0;
decl_attributes (&ref, attributes, (int) ATTR_FLAG_TYPE_IN_PLACE);
TYPE_PACKED (ref) = TYPE_PACKED (orig_struct);
C_TYPE_FIELDS_READONLY (ref) = C_TYPE_FIELDS_READONLY (orig_struct);
C_TYPE_FIELDS_VOLATILE (ref) = C_TYPE_FIELDS_VOLATILE (orig_struct);
C_TYPE_VARIABLE_SIZE (ref) = C_TYPE_VARIABLE_SIZE (orig_struct);
for (x = fields; x; x = TREE_CHAIN (x))
{
DECL_CONTEXT (x) = ref;
DECL_PACKED (x) |= TYPE_PACKED (ref);
}
TYPE_FIELDS (ref) = fields;
layout_type (ref);
TYPE_NAME (ref) = get_identifier (name);
return ref;
}
/* Generate new structure name. */
static inline char *
gen_cluster_name (tree decl, int clust_num, int str_num)
{
char * old_name = get_type_name (decl);
char * new_name;
if (!old_name)
{
old_name = (char *) xmalloc ((10) * sizeof (char));
sprintf (old_name, "struct_%d", str_num);
}
new_name = (char *) xmalloc ((strlen (old_name) + 11) * sizeof (char));
sprintf (new_name, "%s_sub_%d", old_name, clust_num);
/* Check that this name was not used before. */
while (maybe_get_identifier (new_name))
{
int len = strlen (new_name) + 3;
char * tmp_name = (char *) xmalloc (len * sizeof (char));
sprintf (tmp_name, "%s.0", new_name);
new_name = tmp_name;
}
return new_name;
}
/* This function copies fields from CLUSTER into TREE_CHAIN as part
of preparation for new structure building. */
static tree
create_fields (struct field_cluster * cluster,
struct data_field_entry * fields, int num_fields)
{
int i;
tree new_types = NULL_TREE;
tree last = NULL_TREE;
for (i = 0; i < num_fields; i++)
if (TEST_BIT (cluster->fields_in_cluster, i))
{
tree new_decl = copy_node (fields[i].decl);
if (!new_types)
new_types = new_decl;
else
TREE_CHAIN (last) = new_decl;
last = new_decl;
}
TREE_CHAIN (last) = NULL_TREE;
return new_types;
}
/* Update field_mapping of fields in CLUSTER with NEW_TYPE. */
static inline void
update_fields_mapping (struct field_cluster *cluster, tree new_type,
struct data_field_entry * fields, int num_fields)
{
int i;
for (i = 0; i < num_fields; i++)
if (TEST_BIT (cluster->fields_in_cluster, i))
fields[i].field_mapping = new_type;
}
/* This function creates new types according to struct clustering data. */
static void
create_new_types (void)
{
list tmp = data_struct_list;
int str_num = 0;
while (tmp)
{
int cluster_num = 0;
d_str str = tmp->struct_data;
struct field_cluster * cluster = str->struct_clustering;
while (cluster)
{
char * name = gen_cluster_name (str->decl, cluster_num, str_num);
tree fields;
tree new_type;
cluster_num++;
fields = create_fields (cluster, str->fields, str->num_fields);
new_type = build_basic_struct (fields, name, str->decl);
update_fields_mapping (cluster, new_type, str->fields, str->num_fields);
str->new_types = add_tree_to_tree_list (new_type, str->new_types);
cluster = cluster->sibling;
}
str_num++;
tmp = tmp->next;
}
}
static void
finish_var_creation (tree new_decl)
{
if (!var_ann (new_decl))
create_var_ann (new_decl);
if (is_global_var (new_decl))
mark_call_clobbered (new_decl, ESCAPE_UNKNOWN);
mark_sym_for_renaming (new_decl);
if (gimple_referenced_vars (cfun))
add_referenced_var (new_decl);
}
static void
finish_global_creation (tree var)
{
if (TREE_CODE (var) == VAR_DECL
&& is_global_var (var))
finish_var_creation (var);
}
static char *
gen_var_name (tree orig_decl, int i, tree * id_node)
{
char * new_name = NULL;
/* If the original variable decl has a name, create an
appropriate new name for the new decl. */
if (DECL_NAME (orig_decl)
&& IDENTIFIER_POINTER (DECL_NAME (orig_decl)))
{
char *old_name;
int counter = 0;
int len, new_len;
char *tmp_name;
old_name = (char *) IDENTIFIER_POINTER (DECL_NAME (orig_decl));
len = strlen (old_name) + 6;
new_name = (char *) xmalloc (len * sizeof (char));
sprintf (new_name, "%s_%d", old_name, i);
/* Make sure there isn't anything else that already has that
name. */
new_len = strlen (new_name) + 5;
tmp_name = (char *) xmalloc (new_len * sizeof (char));
sprintf (tmp_name, "%s", new_name);
while (maybe_get_identifier (tmp_name))
sprintf (tmp_name, "%s.%d", new_name, counter++);
new_name = tmp_name;
*id_node = get_identifier (new_name);
}
else
*id_node = NULL;
return new_name;
}
static struct field_access_site *
is_in_field_acc_list (tree stmt, struct field_access_site *list_p)
{
struct field_access_site *curr;
for (curr = list_p; curr; curr = curr->next)
if (curr->stmt == stmt)
return curr;
return NULL;
}
static struct field_access_site *
add_field_acc_to_acc_sites (tree stmt, tree ref, tree comp_ref, tree field_decl, tree num,
tree offset, tree base, tree ref_def_stmt, tree cast_stmt,
struct field_access_site *list_p)
{
struct field_access_site *acc;
gcc_assert (!is_in_field_acc_list (stmt, list_p));
acc = (struct field_access_site *) xmalloc (sizeof (struct field_access_site));
acc->stmt = stmt;
acc->comp_ref = comp_ref;
acc->ref = ref;
acc->field_decl = field_decl;
acc->num = num;
acc->offset = offset;
acc->base = base;
acc->ref_def_stmt = ref_def_stmt;
acc->cast_stmt = cast_stmt;
acc->next = list_p;
return acc;
}
static struct access_site *
is_in_acc_list (tree stmt, struct access_site *list_p)
{
struct access_site *curr;
for (curr = list_p; curr; curr = curr->next)
if (curr->stmt == stmt)
return curr;
return NULL;
}
static struct access_site *
add_access_to_acc_sites (tree stmt, tree var, struct access_site *list_p)
{
struct access_site *acc;
acc = is_in_acc_list (stmt, list_p);
if (acc)
{
acc->vars = add_tree_to_tree_list (var, acc->vars);
return list_p;
}
else
{
acc = (struct access_site *) xmalloc (sizeof (struct access_site));
acc->stmt = stmt;
acc->next = list_p;
acc->vars = NULL;
acc->vars = add_tree_to_tree_list (var, acc->vars);
return acc;
}
}
/* Stuff the new tree DATA into a tree_list node,
and append it to the front of the LIST_P. */
static tree_list
add_tree_to_tree_list (tree data, tree_list list_p)
{
tree_list tmp_node;
tmp_node = (tree_list) xmalloc (sizeof (struct struct_tree_list));
tmp_node->data = data;
tmp_node->next = list_p;
return tmp_node;
}
static inline void
insert_global_to_varpool (tree new_decl)
{
struct varpool_node *new_node;
new_node = varpool_node (new_decl);
notice_global_symbol (new_decl);
varpool_mark_needed_node (new_node);
varpool_finalize_decl (new_decl);
}
/* This function copy attributes form ORIG_DECL to NEW_DECL. */
static inline void
copy_decl_attributes (tree new_decl, tree orig_decl)
{
DECL_ARTIFICIAL (new_decl) = 1;
DECL_EXTERNAL (new_decl) = DECL_EXTERNAL (orig_decl);
TREE_STATIC (new_decl) = TREE_STATIC (orig_decl);
TREE_PUBLIC (new_decl) = TREE_PUBLIC (orig_decl);
TREE_USED (new_decl) = TREE_USED (orig_decl);
DECL_CONTEXT (new_decl) = DECL_CONTEXT (orig_decl);
TREE_THIS_VOLATILE (new_decl) = TREE_THIS_VOLATILE (orig_decl);
TREE_ADDRESSABLE (new_decl) = TREE_ADDRESSABLE (orig_decl);
if (TREE_CODE (orig_decl) == VAR_DECL &&
TREE_READONLY (orig_decl))
TREE_READONLY (new_decl) = 1;
}
struct type_wrapper
{
/* 0 stand for pointer wrapper,
and 1 for array wrapper. */
bool wrap;
tree domain; /* Relevant for arrays as domain or index. */
struct type_wrapper * next;
};
static struct type_wrapper *
add_wrapper (struct type_wrapper *wrapper, bool wrap, tree domain)
{
struct type_wrapper * node = (struct type_wrapper *) xmalloc
(sizeof (struct type_wrapper));
node->wrap = wrap;
node->next = wrapper;
node->domain = domain;
return node;
}
static void
free_wrapper (struct type_wrapper * wrap)
{
struct type_wrapper *node, *node1;
node = wrap;
while (node)
{
node1 = node;
node = node->next;
free (node1);
}
}
/* This function generate the same level of pointers or arrays
in the NEW_STR_TYPE as it was in type of DECL.
It returns the generated type. */
static inline tree
gen_struct_type (tree decl, tree new_str_type)
{
tree type_orig = get_type_of_var (decl);
tree new_type = new_str_type;
struct type_wrapper * wrapper = NULL;
struct type_wrapper * wr = NULL;
while (POINTER_TYPE_P (type_orig)
|| TREE_CODE (type_orig) == ARRAY_TYPE)
{
if (POINTER_TYPE_P (type_orig))
wrapper = add_wrapper (wrapper, 0, NULL_TREE);
else if (TREE_CODE (type_orig) == ARRAY_TYPE)
wrapper = add_wrapper (wrapper, 1, TYPE_DOMAIN (type_orig));
type_orig = TREE_TYPE (type_orig);
}
for (wr = wrapper; wr; wr = wr->next)
{
if (wr->wrap) /* Array. */
new_type = build_array_type (new_type, wr->domain);
else /* Pointer. */
new_type = build_pointer_type (new_type);
}
free_wrapper (wrapper);
return new_type;
}
/* Given an old variable ORIG_DECL with VAR_DECL tree code,
this function generates new variables to replace it
according to the set of types decided before. */
static struct struct_tree_list *
create_new_var_1 (tree orig_decl, d_str str)
{
tree_list new_vars = NULL;
tree_list current;
int i;
for (current = str->new_types, i = 0; current;
current = current->next, i++)
{
tree new_type = current->data;
tree new_decl = NULL;
tree id_node;
char * new_name = NULL;
new_name = gen_var_name (orig_decl, i, &id_node);
new_type = gen_struct_type (orig_decl, new_type);
if (is_global_var (orig_decl))
new_decl = build_decl (VAR_DECL, id_node, new_type);
else
new_decl = create_tmp_var (new_type, new_name);
copy_decl_attributes (new_decl, orig_decl);
new_vars = add_tree_to_tree_list (new_decl, new_vars);
}
/* Return the list of new var decls. */
return new_vars;
}
static void
finish_vars_creation (new_var list_p)
{
new_var node;
for (node = list_p; node; node = node->next)
{
tree_list curr;
for (curr = node->new_vars; curr; curr = curr->next)
finish_var_creation (curr->data);
}
}
/* For each SSA_NAME or VAR_DECL local variable of structure
type relevant for transformation in current function (cfun)
this function generate new variable(s) to replace it. */
static void
create_new_local_vars (void)
{
unsigned i;
tree var_list;
if (gimple_in_ssa_p (cfun))
{
for (i = 1; i < num_ssa_names; i++)
{
tree ssa_name = ssa_name (i) ;
if (!ssa_name)
continue;
create_new_var (SSA_NAME_VAR (ssa_name), &new_local_vars);
}
}
/* Function local variables. */
for (var_list = cfun->unexpanded_var_list; var_list;
var_list = TREE_CHAIN (var_list))
{
tree var = TREE_VALUE (var_list);
create_new_var (var, &new_local_vars);
}
finish_vars_creation (new_local_vars);
print_new_vars (new_local_vars);
}
/* This function returns true it the result variable of malloc call is
a cast to one of the structure types we are planning to transform.
STMT should contain malloc call: T.2 = malloc (T.1); */
static bool
is_malloc_of_struct (tree stmt, list * node_p)
{
tree lhs;
tree type;
tree final_stmt;
final_stmt = get_final_malloc_stmt (stmt);
if (!final_stmt)
return false;
/* final_stmt should be of the form:
T.3 = (struct_type *) T.2; */
if (TREE_CODE (final_stmt) != GIMPLE_MODIFY_STMT)
return false;
lhs = GIMPLE_STMT_OPERAND (final_stmt, 0);
type = get_type_of_var (lhs);
if (!type)
return false;
if (!POINTER_TYPE_P (type)
|| TREE_CODE (strip_type (type)) != RECORD_TYPE)
return false;
*node_p = find_struct_in_list (strip_type (type));
if (!(*node_p))
return false;
return true;
}
/* In this function we suppose that an allocation statement
var = (type_cast) malloc (size);
is converted into as follows:
T.1 = size;
T.2 = malloc (T.1);
T.3 = (type_cast) T.2;
var = T.3;
In this function we collect into malloc_data_list only allocation
function of variables of structure types presented
in data_struct_list. */
static void
collect_malloc_data (void)
{
struct cgraph_node *node;
struct cgraph_edge *cs;
for (node = cgraph_nodes; node; node = node->next)
if (node->analyzed && node->decl)
{
for (cs = node->callees; cs; cs = cs->next_callee)
{
tree stmt = cs->call_stmt;
if (stmt)
{
tree call = get_call_expr_in (stmt);
tree decl;
if (call && (decl = get_callee_fndecl (call))
&& TREE_CODE (stmt) == GIMPLE_MODIFY_STMT)
{
list str_node = NULL;
if (is_malloc_of_struct (stmt, &str_node))
{
d_str str = str_node->struct_data;
/* We support only malloc now. */
if (DECL_FUNCTION_CODE (decl) == BUILT_IN_MALLOC)
add_call_to_malloc_list (node->decl, stmt, str);
else
remove_struct_from_data_struct_list (str_node);
}
}
}
}
}
}
/* Update statements in STMT_LIST with BB info. */
static void
add_bb_info (basic_block bb, tree stmt_list)
{
if (TREE_CODE (stmt_list) == STATEMENT_LIST)
{
tree_stmt_iterator tsi;
for (tsi = tsi_start (stmt_list); !tsi_end_p (tsi); tsi_next (&tsi))
{
tree stmt = tsi_stmt (tsi);
set_bb_for_stmt (stmt, bb);
}
}
}
/* This function checks whether ARG is a result of multiplication
of some number by STRUCT_SIZE. If yes, the function returns true
and this number is substituted into NUM. */
static bool
is_result_of_mult (tree arg, tree *num, tree struct_size)
{
tree size_def_stmt = SSA_NAME_DEF_STMT (arg);
/* If malloc stmt was of the form D.2229_10 = malloc (D.2228_9);
then size_def_stmt can be D.2228_9 = num.3_8 * 8; */
if (size_def_stmt
&& TREE_CODE (size_def_stmt) == GIMPLE_MODIFY_STMT)
{
tree lhs = GIMPLE_STMT_OPERAND (size_def_stmt, 0);
tree rhs = GIMPLE_STMT_OPERAND (size_def_stmt, 1);
/* We expect temporary here. */
if (!is_gimple_reg (lhs))
return false;
if (TREE_CODE (rhs) == MULT_EXPR)
{
tree arg0 = TREE_OPERAND (rhs, 0);
tree arg1 = TREE_OPERAND (rhs, 1);
if (CONSTANT_CLASS_P (arg0)
&& simple_cst_equal (arg0,struct_size))
{
*num = arg1;
return true;
}
if (CONSTANT_CLASS_P (arg1)
&& simple_cst_equal (arg1,struct_size))
{
*num = arg0;
return true;
}
}
}
*num = NULL_TREE;
return false;
}
static inline void
finish_stmt (tree stmt)
{
update_stmt (stmt);
mark_symbols_for_renaming (stmt);
}
/* This function returns a tree represented
the number of structure instances allocated by MALLOC_STMT. */
static tree
gen_num_of_structs_in_malloc (tree stmt, tree str_decl, tree * new_stmts_p)
{
call_expr_arg_iterator iter;
tree arg;
tree call_expr;
tree struct_size;
unsigned int struct_size_int;
if (!stmt)
return NULL_TREE;
/* Get malloc agrument. */
call_expr = get_call_expr_in (stmt);
if (!call_expr)
return NULL_TREE;
arg = first_call_expr_arg (call_expr, &iter);
if (TREE_CODE (arg) != SSA_NAME
&& !TREE_CONSTANT (arg))
return NULL_TREE;
struct_size = TYPE_SIZE_UNIT (str_decl);
struct_size_int = TREE_INT_CST_LOW (struct_size);
gcc_assert (struct_size);
if (TREE_CODE (arg) == SSA_NAME)
{
tree num, div_stmt;
if (is_result_of_mult (arg, &num, struct_size))
return num;
num = create_tmp_var (integer_type_node, NULL);
if (num)
add_referenced_var (num);
if (exact_log2 (struct_size_int) == -1)
div_stmt = build2 (GIMPLE_MODIFY_STMT, integer_type_node,
num, build2 (TRUNC_DIV_EXPR, integer_type_node,
arg, struct_size));
else
div_stmt = build2 (GIMPLE_MODIFY_STMT, integer_type_node,
num,
build2 (RSHIFT_EXPR,
integer_type_node,
arg, build_int_cst (integer_type_node,
exact_log2 (struct_size_int))));
*new_stmts_p = alloc_stmt_list ();
append_to_statement_list (div_stmt,
new_stmts_p);
finish_stmt (div_stmt);
return num;
}
if (CONSTANT_CLASS_P (arg)
&& multiple_of_p (TREE_TYPE (struct_size),
arg, struct_size))
return int_const_binop (TRUNC_DIV_EXPR, arg, struct_size, 0);
return NULL_TREE;
}
static inline void
finish_stmt_and_append (tree *stmts, tree stmt)
{
append_to_statement_list (stmt,
stmts);
finish_stmt (stmt);
}
static tree
find_var_in_new_vars_list (new_var var, tree new_type)
{
tree_list curr;
for (curr = var->new_vars; curr; curr = curr->next)
{
tree type = strip_type(get_type_of_var (curr->data));
gcc_assert (type);
if (type == new_type)
return curr->data;
}
return NULL_TREE;
}
static tree
find_new_var_of_type (tree orig_var, tree new_type)
{
new_var var;
gcc_assert (orig_var && new_type);
if (TREE_CODE (orig_var) == SSA_NAME)
orig_var = SSA_NAME_VAR (orig_var);
var = is_in_new_vars_list (orig_var, new_global_vars);
if (!var)
var = is_in_new_vars_list (orig_var, new_local_vars);
gcc_assert (var);
return find_var_in_new_vars_list (var, new_type);
}
static tree
create_new_malloc (tree malloc_stmt, tree new_type, tree *new_stmts, tree num)
{
tree new_malloc_size;
tree call_expr, malloc_fn_decl;
tree new_stmt, malloc_res;
tree call_stmt, final_stmt;
tree cast_res;
gcc_assert (num && malloc_stmt && new_type);
*new_stmts = alloc_stmt_list ();
/* Generate argument to malloc as multiplication of num and size of new_type. */
new_stmt = gen_size (num, new_type, &new_malloc_size);
append_to_statement_list (new_stmt, new_stmts);
/* Generate new call for malloc. */
malloc_res = create_tmp_var (integer_type_node, NULL);
if (malloc_res)
add_referenced_var (malloc_res);
call_expr = get_call_expr_in (malloc_stmt);
malloc_fn_decl = get_callee_fndecl (call_expr);
call_expr = build_call_expr (malloc_fn_decl, 1, new_malloc_size);
call_stmt = build2 (GIMPLE_MODIFY_STMT,
TREE_TYPE (TREE_TYPE (malloc_fn_decl)),
malloc_res,
call_expr);
finish_stmt_and_append (new_stmts, call_stmt);
/* Create new cast stmt. */
final_stmt = get_final_malloc_stmt (malloc_stmt);
gcc_assert (final_stmt);
new_stmt = gen_cast_stmt (malloc_res, new_type, final_stmt, &cast_res);
append_to_statement_list (new_stmt, new_stmts);
return call_stmt;
}
/* This function build an edge between BB and E->dest and updates
phi nodes of E->dest. It returns newly created edge. */
static edge
make_edge_and_fix_phis_of_dest (basic_block bb, edge e)
{
edge new_e;
tree phi, arg;
new_e = make_edge (bb, e->dest, e->flags);
for (phi = phi_nodes (new_e->dest); phi; phi = PHI_CHAIN (phi))
{
arg = PHI_ARG_DEF_FROM_EDGE (phi, e);
add_phi_arg (phi, arg, new_e);
}
return new_e;
}
/* Update call graph with new edge generated by
new MALLOC_STMT. */
static void
update_cgraph_with_malloc_call (tree malloc_stmt, tree context)
{
tree call_expr;
struct cgraph_node *src, *dest;
tree malloc_fn_decl;
if (!malloc_stmt)
return;
call_expr = get_call_expr_in (malloc_stmt);
malloc_fn_decl = get_callee_fndecl (call_expr);
src = cgraph_node (context);
dest = cgraph_node (malloc_fn_decl);
cgraph_create_edge (src, dest, malloc_stmt,
0, 0, bb_for_stmt (malloc_stmt)->loop_depth);
}
/* If MALLOC_STMT is D.2225_7 = malloc (D.2224_6);
it should be casting stmt as for example:
p_8 = (struct str_t *) D.2225_7;
which is returned by this function. */
static tree
get_final_malloc_stmt (tree malloc_stmt)
{
tree final_stmt;
use_operand_p use_p;
tree malloc_res;
if (!malloc_stmt)
return NULL;
if (TREE_CODE (malloc_stmt) != GIMPLE_MODIFY_STMT)
return NULL;
malloc_res = GIMPLE_STMT_OPERAND (malloc_stmt, 0);
if (TREE_CODE (malloc_res) != SSA_NAME)
return NULL;
if (!single_imm_use (malloc_res, &use_p, &final_stmt))
return NULL;
else
return final_stmt;
}
/* Insert NEW_STMTS after STMT. */
static void
insert_after_stmt (tree stmt, tree new_stmts)
{
block_stmt_iterator bsi;
if (!stmt || !new_stmts)
return;
bsi = bsi_for_stmt (stmt);
bsi_insert_after (&bsi, new_stmts, BSI_SAME_STMT);
}
/* Insert NEW_STMTS before STMT. */
static void
insert_before_stmt (tree stmt, tree new_stmts)
{
block_stmt_iterator bsi;
if (!stmt || !new_stmts)
return;
bsi = bsi_for_stmt (stmt);
bsi_insert_before (&bsi, new_stmts, BSI_SAME_STMT);
}
/* This function adds allocation sites for peeled structs. */
static void
create_new_mallocs (malloc_d m_data, tree context)
{
malloc_call call;
for (call = m_data->malloc_list; call; call = call->next)
{
tree stmt = call->malloc_call_stmt;
d_str str = call->str;
tree_list n_type;
tree num;
tree new_stmts = NULL_TREE;
tree last_stmt = get_final_malloc_stmt (stmt);
num = gen_num_of_structs_in_malloc (stmt, str->decl, &new_stmts);
if (new_stmts)
{
last_stmt = tsi_stmt (tsi_last (new_stmts));
insert_after_stmt (last_stmt, new_stmts);
}
/* Generate an allocation sites for each new structure type. */
for (n_type = str->new_types; n_type; n_type = n_type->next)
{
tree type = n_type->data;
tree new_malloc_stmt = NULL_TREE;
tree last_stmt_tmp = NULL_TREE;
new_stmts = NULL_TREE;
new_malloc_stmt = create_new_malloc (stmt, type, &new_stmts, num);
last_stmt_tmp = tsi_stmt (tsi_last (new_stmts));
insert_after_stmt (last_stmt, new_stmts);
update_cgraph_with_malloc_call (new_malloc_stmt, context);
last_stmt = last_stmt_tmp;
}
}
}
/* Create new mallocs for function represented by NODE. */
static void
create_new_mallocs_for_func (struct cgraph_node *node)
{
malloc_d cur_malloc = get_malloc_list_for_func (node->decl);
if (cur_malloc)
create_new_mallocs (cur_malloc, node->decl);
}
static void
free_new_vars_list (new_var *list_p)
{
new_var node = *list_p;
new_var t_node;
while (node)
{
tree_list var, t_var;
/* Free tree_list of new vars. */
var = node->new_vars;
while (var)
{
t_var = var;
var = var->next;
free (t_var);
}
t_node = node;
node = node->next;
free (t_node);
}
*list_p = NULL;
}
static malloc_d
get_malloc_list_for_func (tree fn_decl)
{
malloc_d cur_malloc;
for (cur_malloc = malloc_data_list_new; cur_malloc;
cur_malloc = cur_malloc->next)
if (cur_malloc->func == fn_decl)
return cur_malloc;
return NULL;
}
static bool
is_part_of_malloc (tree stmt, tree fn_decl)
{
malloc_d cur_malloc = get_malloc_list_for_func (fn_decl);
if (cur_malloc)
{
malloc_call call;
for (call = cur_malloc->malloc_list; call; call = call->next)
if (call->malloc_call_stmt == stmt
|| get_final_malloc_stmt (call->malloc_call_stmt) == stmt)
return true;
}
return false;
}
static void
remove_from_acc_list (struct access_site ** list_p, tree stmt)
{
struct access_site *node = *list_p;
struct access_site *prev_node = NULL;
while (node)
{
if (node->stmt == stmt)
{
if (prev_node == NULL)
*list_p = node->next;
else
prev_node->next = node->next;
free_tree_list (node->vars);
free (node);
return;
}
prev_node = node;
node = node->next;
}
}
static bool
is_part_of_field_access (tree stmt, d_str str)
{
int i;
for (i = 0; i < str->num_fields; i++)
{
struct field_access_site *acc;
for (acc = str->fields[i].acc_sites; acc; acc = acc->next)
{
if (acc->stmt == stmt
|| acc->ref_def_stmt == stmt
|| acc->cast_stmt == stmt)
return true;
}
}
return false;
}
/* We exclude from list of general accesses of structure all stmt that
will be treated as part of new mallocs or accesses generation. */
static void
exclude_malloc_and_field_accs (struct cgraph_node *node)
{
list str_node;
for (str_node = data_struct_list; str_node; str_node = str_node->next)
{
d_str str = str_node->struct_data;
struct access_site * acc = str->accs;
while (acc)
{
tree stmt = acc->stmt;
acc = acc->next;
if (is_part_of_malloc (stmt, node->decl)
|| is_part_of_field_access (stmt, str))
remove_from_acc_list (&str->accs, stmt);
}
}
}
static inline tree
build_comp_ref (tree base, tree field_id, tree type)
{
tree field;
bool found = false;
/* Find field with the same name as field_id in type. */
for (field = TYPE_FIELDS (type); field; field = TREE_CHAIN (field))
{
if (DECL_NAME (field) == field_id)
{
found = true;
break;
}
}
gcc_assert (found);
return build3 (COMPONENT_REF, TREE_TYPE (field),
base, field, NULL_TREE);
}
struct ref_pos
{
tree *pos;
tree ref;
};
/* Helper for walk_tree called from collect_accesses_in_bb function. */
static tree
find_pos_in_stmt_1 (tree *tp, int *walk_subtrees, void * data)
{
struct ref_pos * r_pos = (struct ref_pos *) data;
tree ref = r_pos->ref;
tree t = *tp;
if (t == ref)
{
r_pos->pos = tp;
return t;
}
switch (TREE_CODE (t))
{
case GIMPLE_MODIFY_STMT:
{
tree lhs = GIMPLE_STMT_OPERAND (t, 0);
tree rhs = GIMPLE_STMT_OPERAND (t, 1);
*walk_subtrees=1;
walk_tree (&lhs, find_pos_in_stmt_1, data, NULL);
walk_tree (&rhs, find_pos_in_stmt_1, data, NULL);
*walk_subtrees=0;
}
break;
default:
*walk_subtrees=1;
}
return NULL_TREE;
}
static tree *
find_pos_in_stmt (tree stmt, tree ref)
{
struct ref_pos r_pos;
r_pos.ref = ref;
r_pos.pos = NULL;
walk_tree (&stmt, find_pos_in_stmt_1, &r_pos, NULL);
return r_pos.pos;
}
static void
replace_field_acc (struct field_access_site *acc, tree new_type)
{
tree ref_var = acc->ref;
tree new_ref;
tree lhs, rhs;
tree *pos;
tree new_acc;
tree field_id = DECL_NAME (acc->field_decl);
struct type_wrapper * wrapper = NULL;
struct type_wrapper * wr = NULL;
while (TREE_CODE (ref_var) == INDIRECT_REF
|| TREE_CODE (ref_var) == ARRAY_REF)
{
if ( TREE_CODE (ref_var) == INDIRECT_REF)
wrapper = add_wrapper (wrapper, 0, 0);
else if (TREE_CODE (ref_var) == ARRAY_REF)
wrapper = add_wrapper (wrapper, 1, TREE_OPERAND (ref_var, 1));
ref_var = TREE_OPERAND (ref_var, 0);
}
new_ref = find_new_var_of_type (ref_var, new_type);
finish_global_creation (new_ref);
for (wr = wrapper; wr; wr = wr->next)
{
tree type = TREE_TYPE (TREE_TYPE (new_ref));
if (wr->wrap) /* Array. */
new_ref = build4 (ARRAY_REF, type, new_ref,
wr->domain, NULL_TREE, NULL_TREE);
else /* Pointer. */
new_ref = build1 (INDIRECT_REF, type, new_ref);
}
new_acc = build_comp_ref (new_ref, field_id, new_type);
free_wrapper (wrapper);
if (TREE_CODE (acc->stmt) == GIMPLE_MODIFY_STMT)
{
lhs = GIMPLE_STMT_OPERAND (acc->stmt, 0);
rhs = GIMPLE_STMT_OPERAND (acc->stmt, 1);
if (lhs == acc->comp_ref)
GIMPLE_STMT_OPERAND (acc->stmt, 0) = new_acc;
else if (rhs == acc->comp_ref)
GIMPLE_STMT_OPERAND (acc->stmt, 1) = new_acc;
else
{
pos = find_pos_in_stmt (acc->stmt, acc->comp_ref);
gcc_assert (pos);
*pos = new_acc;
}
}
else
{
pos = find_pos_in_stmt (acc->stmt, acc->comp_ref);
gcc_assert (pos);
*pos = new_acc;
}
finish_stmt (acc->stmt);
}
static void
replace_field_access_stmt (struct field_access_site *acc, tree new_type)
{
if (TREE_CODE (acc->ref) == INDIRECT_REF
||TREE_CODE (acc->ref) == ARRAY_REF
||TREE_CODE (acc->ref) == VAR_DECL)
replace_field_acc (acc, new_type);
else
gcc_unreachable ();
}
/* In this function we create new stmts that has the same form as ORIG_STMT,
but of the type NET_TYPE. The stmts treated by this function are simple
assignments, like assignments: p.8_7 = p; or stmts with rhs of code PLUS_EXPR
or MINUS_EXPR. */
static tree
create_base_plus_offset (tree orig_stmt, tree new_type, tree offset)
{
tree lhs, rhs;
tree new_lhs, new_rhs;
tree new_stmt;
gcc_assert (TREE_CODE (orig_stmt) == GIMPLE_MODIFY_STMT);
lhs = GIMPLE_STMT_OPERAND (orig_stmt, 0);
rhs = GIMPLE_STMT_OPERAND (orig_stmt, 1);
gcc_assert (TREE_CODE (lhs) == VAR_DECL
|| TREE_CODE (lhs) == SSA_NAME);
new_lhs = find_new_var_of_type (lhs, new_type);
gcc_assert (new_lhs);
finish_var_creation (new_lhs);
switch (TREE_CODE (rhs))
{
case PLUS_EXPR:
case MINUS_EXPR:
case POINTER_PLUS_EXPR:
{
debug_tree (rhs);
tree op0 = TREE_OPERAND (rhs, 0);
tree op1 = TREE_OPERAND (rhs, 1);
tree new_op0 = NULL_TREE, new_op1 = NULL_TREE;
list tmp0, tmp1;
tmp0 = find_struct_in_list (strip_type (get_type_of_var (op0)));
tmp1 = find_struct_in_list (strip_type (get_type_of_var (op1)));
gcc_assert ( tmp0 || tmp1);
if (tmp0)
new_op0 = find_new_var_of_type (op0, new_type);
if (tmp1)
new_op1 = find_new_var_of_type (op1, new_type);
if (!new_op0)
new_op0 = offset;
if (!new_op1)
new_op1 = offset;
new_rhs = build2 (TREE_CODE (rhs), TREE_TYPE (new_op0),
new_op0, new_op1);
}
break;
default:
gcc_unreachable();
}
new_stmt = build2 (GIMPLE_MODIFY_STMT, TREE_TYPE (new_lhs),
new_lhs, new_rhs);
finish_stmt (new_stmt);
return new_stmt;
}
/* Generate stmt: res = NUM * sizeof(TYPE) and return it. */
static tree
gen_size (tree num, tree type, tree *res)
{
tree struct_size = TYPE_SIZE_UNIT (type);
unsigned int struct_size_int = TREE_INT_CST_LOW (struct_size);
tree new_stmt;
*res = create_tmp_var (TREE_TYPE (num), NULL);
if (*res)
add_referenced_var (*res);
if (exact_log2 (struct_size_int) == -1)
new_stmt = build2 (GIMPLE_MODIFY_STMT, TREE_TYPE (num),
*res, build2 (MULT_EXPR,
TREE_TYPE (num),
num,
struct_size));
else
new_stmt = build2 (GIMPLE_MODIFY_STMT, TREE_TYPE (num),
*res,
build2 (LSHIFT_EXPR,
TREE_TYPE (num),
num,
build_int_cst (TREE_TYPE (num),
exact_log2 (struct_size_int))));
finish_stmt (new_stmt);
return new_stmt;
}
static tree
gen_cast_stmt (tree before_cast, tree new_type, tree old_cast_stmt, tree *res_p)
{
tree lhs, new_lhs, new_stmt;
gcc_assert (TREE_CODE (old_cast_stmt) == GIMPLE_MODIFY_STMT);
lhs = GIMPLE_STMT_OPERAND (old_cast_stmt, 0);
new_lhs = find_new_var_of_type (lhs, new_type);
gcc_assert (new_lhs);
new_stmt = build2 (GIMPLE_MODIFY_STMT,
TREE_TYPE
(new_lhs),
new_lhs,
build1 (NOP_EXPR,
TREE_TYPE (new_lhs),
before_cast));
finish_stmt (new_stmt);
*res_p = new_lhs;
return new_stmt;
}
static void
create_new_field_access (struct field_access_site *acc, struct data_field_entry field)
{
tree new_type = field.field_mapping;
tree new_stmt;
tree size_res;
tree mult_stmt, cast_stmt;
tree cast_res = NULL;
if (acc->num)
{
mult_stmt = gen_size (acc->num, new_type, &size_res);
insert_before_stmt (acc->ref_def_stmt, mult_stmt);
}
if (acc->cast_stmt)
{
cast_stmt = gen_cast_stmt (size_res, new_type,
acc->cast_stmt, &cast_res);
insert_after_stmt (acc->cast_stmt, cast_stmt);
}
if (acc->ref_def_stmt)
{
tree offset;
if (cast_res)
offset = cast_res;
else
offset = size_res;
new_stmt = create_base_plus_offset (acc->ref_def_stmt, new_type, offset);
insert_after_stmt (acc->ref_def_stmt, new_stmt);
}
/* In stmt D.2163_19 = D.2162_18->b; we replace variable
D.2162_18 for the appropriate variable of new_type. */
replace_field_access_stmt (acc, new_type);
}
static void
create_new_general_access (struct access_site *acc, d_str str)
{
tree stmt = acc->stmt;
switch (TREE_CODE (stmt))
{
case COND_EXPR:
create_new_stmts_for_cond_expr (stmt);
break;
default:
create_new_stmts_for_general_acc (acc, str);
}
}
static void
create_new_accesses_in_bb (basic_block bb)
{
list tmp;
for (tmp = data_struct_list; tmp; tmp = tmp->next)
{
int i;
struct field_access_site *f_acc;
struct access_site *acc;
d_str str = tmp->struct_data;
for (i = 0; i < str->num_fields; i++)
for (f_acc = str->fields[i].acc_sites; f_acc; f_acc = f_acc->next)
if (bb_for_stmt (f_acc->stmt) == bb)
create_new_field_access (f_acc, str->fields[i]);
for (acc = str->accs; acc; acc = acc->next)
if (bb_for_stmt (acc->stmt) == bb)
create_new_general_access (acc, str);
}
}
static void
create_new_accesses_for_func (void)
{
basic_block bb;
FOR_EACH_BB_FN (bb, cfun)
create_new_accesses_in_bb (bb);
}
static tree
create_general_new_stmt (struct access_site *acc, tree new_type)
{
tree old_stmt = acc->stmt;
tree_list var_d;
tree new_stmt = copy_node (old_stmt);
for (var_d = acc->vars; var_d; var_d = var_d->next)
{
tree * pos;
tree var = var_d->data;
tree new_var = find_new_var_of_type (var, new_type);
tree lhs, rhs;
gcc_assert (new_var);
finish_var_creation (new_var);
if (TREE_CODE (new_stmt) == GIMPLE_MODIFY_STMT)
{
lhs = GIMPLE_STMT_OPERAND (new_stmt, 0);
rhs = GIMPLE_STMT_OPERAND (new_stmt, 1);
if (TREE_CODE (lhs) == SSA_NAME)
lhs = SSA_NAME_VAR (lhs);
if (TREE_CODE (rhs) == SSA_NAME)
rhs = SSA_NAME_VAR (rhs);
if (lhs == var)
GIMPLE_STMT_OPERAND (new_stmt, 0) = new_var;
else if (rhs == var)
GIMPLE_STMT_OPERAND (new_stmt, 1) = new_var;
else
{
pos = find_pos_in_stmt (new_stmt, var);
gcc_assert (pos);
*pos = new_var;
}
}
else
{
pos = find_pos_in_stmt (new_stmt, var);
gcc_assert (pos);
*pos = new_var;
}
}
finish_stmt (new_stmt);
return new_stmt;
}
static void
create_new_stmts_for_general_acc (struct access_site * acc, d_str str)
{
tree_list new_type;
tree stmt = acc->stmt;
for (new_type = str->new_types; new_type; new_type = new_type->next)
{
tree new_stmt;
new_stmt = create_general_new_stmt (acc, new_type->data);
insert_after_stmt (stmt, new_stmt);
}
}
static void
create_new_stmts_for_cond_expr_1 (tree new_var, tree cond_stmt, bool pos)
{
tree new_cond;
tree new_stmt;
edge true_e = NULL, false_e = NULL;
basic_block new_bb;
tree stmt_list;
extract_true_false_edges_from_block (bb_for_stmt (cond_stmt),
&true_e, &false_e);
new_cond = copy_node (COND_EXPR_COND (cond_stmt));
TREE_OPERAND (new_cond, pos) = new_var;
new_stmt = build3 (COND_EXPR,TREE_TYPE (cond_stmt),
new_cond, NULL_TREE, NULL_TREE);
finish_stmt (new_stmt);
/* Create a new basic block after bb. */
new_bb = create_empty_bb (bb_for_stmt (cond_stmt));
/* Add new condition stmt to the new_bb. */
stmt_list = bb_stmt_list (new_bb);
append_to_statement_list (new_stmt, &stmt_list);
add_bb_info (new_bb, stmt_list);
/* Create false and true edges from new_bb. */
make_edge_and_fix_phis_of_dest (new_bb, true_e);
make_edge_and_fix_phis_of_dest (new_bb, false_e);
/* Redirect one of original edges to point to new_bb. */
if (TREE_CODE (cond_stmt) == NE_EXPR)
redirect_edge_succ (true_e, new_bb);
else
redirect_edge_succ (false_e, new_bb);
}
static void
create_new_stmts_for_cond_expr (tree stmt)
{
tree cond = COND_EXPR_COND (stmt);
tree arg0, arg1, arg;
list tmp0, tmp1, tmp;
d_str str;
tree_list new_type;
bool pos;
gcc_assert (TREE_CODE (cond) == EQ_EXPR
|| TREE_CODE (cond) == NE_EXPR);
arg0 = TREE_OPERAND (cond, 0);
arg1 = TREE_OPERAND (cond, 1);
tmp0 = find_struct_in_list (strip_type (get_type_of_var (arg0)));
tmp1 = find_struct_in_list (strip_type (get_type_of_var (arg1)));
gcc_assert ((!tmp0 && tmp1) || (!tmp1 && tmp0));
tmp = tmp0 ? tmp0 : tmp1;
arg = tmp0 ? arg0 : arg1;
pos = tmp0 ? 0 : 1;
str = tmp->struct_data;
for (new_type = str->new_types; new_type; new_type = new_type->next)
{
tree new_arg;
new_arg = find_new_var_of_type (arg, new_type->data);
create_new_stmts_for_cond_expr_1 (new_arg, stmt, pos);
}
}
/* Do transformation for a function represented by NODE. */
static void
do_reorg_for_func (struct cgraph_node *node)
{
create_new_local_vars ();
create_new_mallocs_for_func (node);
create_new_accesses_for_func ();
update_ssa (TODO_update_ssa);
cleanup_tree_cfg ();
/* Free auxiliary data for local variables. */
free_new_vars_list (&new_local_vars);
}
static void
add_to_new_vars_list (new_var new_node, new_var *list_p)
{
new_var node;
if (!(*list_p))
*list_p = new_node;
else
{
node = *list_p;
while (node->next)
node = node->next;
node->next = new_node;
}
}
static new_var
is_in_new_vars_list (tree decl, new_var list_p)
{
new_var node;
if (!list_p)
return NULL;
else
{
node = list_p;
while (node)
{
if (node->orig_var == decl)
return node;
node = node->next;
}
}
return NULL;
}
static new_var
create_new_var_node (tree var, tree_list new_vars_list)
{
new_var node;
node = (struct new_var_data *) xmalloc (sizeof (struct new_var_data));
node->orig_var = var;
node->new_vars = new_vars_list;
node->next = NULL;
return node;
}
static void
create_new_var (tree var_decl, new_var *list_p)
{
new_var node;
tree_list new_vars;
list tmp;
tree type;
d_str str;
if (!var_decl || is_in_new_vars_list (var_decl, *list_p))
return;
if (!is_candidate_for_data_struct_list (var_decl, &type, NULL))
return;
tmp = find_struct_in_list (type);
if (!tmp)
return;
str = tmp->struct_data;
new_vars = create_new_var_1 (var_decl, str);
node = create_new_var_node (var_decl, new_vars);
add_to_new_vars_list (node, list_p);
}
static void
update_varpool_with_new_vars (void)
{
new_var old_var;
for (old_var = new_global_vars; old_var; old_var = old_var->next)
{
tree_list n_var;
for (n_var = old_var->new_vars; n_var; n_var = n_var->next)
insert_global_to_varpool (n_var->data);
}
}
/* This function create global struct variables of new
structure types in case these this variable has VAR_DECL code. */
static void
create_new_global_vars (void)
{
struct varpool_node *current_varpool;
FOR_EACH_STATIC_VARIABLE(current_varpool)
{
tree var_decl = current_varpool->decl;
if (!var_decl || TREE_CODE (var_decl) != VAR_DECL)
continue;
create_new_var (var_decl, &new_global_vars);
}
update_varpool_with_new_vars ();
}
/* This function is a helper for do_reorg. It goes over functions in call graph
and performs actual transformation on each one of them. */
static void
do_reorg_1 (void)
{
struct cgraph_node *node;
/* Initialize the default bitmap obstack. */
bitmap_obstack_initialize (NULL);
for (node = cgraph_nodes; node; node = node->next)
if (node->analyzed && node->decl && !node->next_clone)
{
push_cfun (DECL_STRUCT_FUNCTION (node->decl));
current_function_decl = node->decl;
if (dump_file)
fprintf (dump_file, "\nFunction to do reorg is %s: \n",
(char *) IDENTIFIER_POINTER (DECL_NAME (node->decl)));
do_reorg_for_func (node);
free_dominance_info (CDI_DOMINATORS);
free_dominance_info (CDI_POST_DOMINATORS);
current_function_decl = NULL;
pop_cfun ();
}
cfun = NULL;
}
static void
do_reorg (void)
{
/* Check that there is a work to do. */
if (!data_struct_list)
return;
/* Generate new types. */
create_new_types ();
print_new_types ();
/* Create new global vars. */
create_new_global_vars ();
print_new_vars (new_global_vars);
/* Do reorg for each funciton separately. */
do_reorg_1 ();
/* Free auxiliary data for global variables. */
free_new_vars_list (&new_global_vars);
}
/* This function builds an entry in struct_list for each
data structure potential for transformation. */
static void
build_data_structure_list (bitmap non_suitable_types)
{
tree var, type;
tree var_list;
struct varpool_node *current_varpool;
struct cgraph_node *c_node;
/* Check global vars. */
FOR_EACH_STATIC_VARIABLE (current_varpool)
{
var = current_varpool->decl;
if (is_candidate_for_data_struct_list (var, &type, non_suitable_types))
add_struct_to_data_struct_list (type);
}
/* Now add data structures of function parameters and local variables. */
for (c_node = cgraph_nodes; c_node; c_node = c_node->next)
{
enum availability avail = cgraph_function_body_availability (c_node);
/* We need AVAIL_AVAILABLE for main function. */
if (avail == AVAIL_LOCAL || avail == AVAIL_AVAILABLE)
{
struct function *fn = DECL_STRUCT_FUNCTION (c_node->decl);
for (var = DECL_ARGUMENTS (c_node->decl); var; var = TREE_CHAIN (var))
if (is_candidate_for_data_struct_list (var, &type, non_suitable_types))
add_struct_to_data_struct_list (type);
/* Function local variables. */
for (var_list = fn->unexpanded_var_list; var_list;
var_list = TREE_CHAIN (var_list))
{
var = TREE_VALUE (var_list);
if (is_candidate_for_data_struct_list (var, &type, non_suitable_types))
add_struct_to_data_struct_list (type);
}
}
}
}
/* This function looks for arguments of local functions
which are of pointer to or structure types. We are not handling
peeling of such structures right now. */
static void
exclude_types_passed_to_local_func (bitmap non_suitable_types)
{
struct cgraph_node *c_node;
for (c_node = cgraph_nodes; c_node; c_node = c_node->next)
{
if (cgraph_function_body_availability (c_node) == AVAIL_LOCAL)
{
tree fn = c_node->decl;
tree arg;
for (arg = DECL_ARGUMENTS (fn); arg; arg = TREE_CHAIN (arg))
{
tree type = TREE_TYPE (arg);
type = strip_type (type);
if (TREE_CODE (type) == RECORD_TYPE)
{
bitmap_set_bit (non_suitable_types, TYPE_UID (TYPE_MAIN_VARIANT (type)));
if (dump_file)
{
fprintf (dump_file, "\nPointer to type \"");
print_generic_expr (dump_file, type, 0);
fprintf (dump_file, "\" is passed to local function...Excluded.");
}
}
}
}
}
}
static void
exclude_escaping_types_1 (bitmap non_suitable_types)
{
list tmp = data_struct_list;
while (tmp)
{
tree type = tmp->struct_data->decl;
if (!ipa_type_escape_type_contained_p (type))
{
if (dump_file)
{
fprintf (dump_file, "\nEscaping type is ");
print_generic_expr (dump_file, type, 0);
}
bitmap_set_bit (non_suitable_types, TYPE_UID (type));
}
tmp = tmp->next;
}
}
static void
exclude_escaping_types (bitmap non_suitable_types)
{
exclude_types_passed_to_local_func (non_suitable_types);
exclude_escaping_types_1 (non_suitable_types);
}
/* FIXME: In this function we suppose that only fields defined
as bitfields in C program can become bitfields. If there are other ways,
this function should be extended to recognize them. */
static void
exclude_types_with_bit_fields (bitmap non_suitable_types)
{
list tmp = data_struct_list;
while (tmp)
{
int i;
d_str str = tmp->struct_data;
tree type = tmp->struct_data->decl;
for (i = 0; i < str->num_fields; i++)
if (DECL_BIT_FIELD (str->fields[i].decl))
bitmap_set_bit (non_suitable_types, TYPE_UID (type));
tmp = tmp->next;
}
}
/* This function returns true if program contains a call to user
defined allocation function. */
static bool
program_redefines_malloc_p (void)
{
bool redefines = false;
struct cgraph_node *c_node;
struct cgraph_node *c_node2;
struct cgraph_edge *c_edge;
tree fndecl;
tree fndecl2;
tree call_expr;
for (c_node = cgraph_nodes; c_node && !redefines; c_node = c_node->next)
{
fndecl = c_node->decl;
for (c_edge = c_node->callees; c_edge && !redefines;
c_edge = c_edge->next_callee)
{
call_expr = get_call_expr_in (c_edge->call_stmt);
c_node2 = c_edge->callee;
fndecl2 = c_node2->decl;
if (call_expr)
{
if ((call_expr_flags (call_expr) & ECF_MALLOC) &&
(DECL_FUNCTION_CODE (fndecl2) != BUILT_IN_MALLOC) &&
(DECL_FUNCTION_CODE (fndecl2) != BUILT_IN_CALLOC) &&
(DECL_FUNCTION_CODE (fndecl2) != BUILT_IN_ALLOCA))
redefines = true;
}
}
}
return redefines;
}
/* Auxiliary function for add_struct_to_data_struct_list. */
static struct data_field_entry *
get_fields (tree struct_decl, int num_fields)
{
struct data_field_entry *list;
tree t = TYPE_FIELDS (struct_decl);
int idx = 0;
list = (struct data_field_entry * ) xmalloc (num_fields
* sizeof
(struct data_field_entry));
for (idx = 0 ; t; t = TREE_CHAIN (t), idx++)
if (TREE_CODE (t) == FIELD_DECL)
{
list[idx].index = idx;
list[idx].decl = t;
list[idx].acc_sites = NULL;
list[idx].count = 0;
list[idx].field_mapping = NULL_TREE;
}
return list;
}
/* Auxiliary function for remove_struct_from_data_struct_list. */
static void
free_struct_cluster (struct field_cluster* cluster)
{
if (cluster)
{
if (cluster->fields_in_cluster)
sbitmap_free (cluster->fields_in_cluster);
if (cluster->sibling)
free_struct_cluster (cluster->sibling);
free (cluster);
}
}
/* The following three functions:
add_struct_to_data_struct_list
remove_struct_from_data_struct_list
free_data_structure_list
is an interface for manipulation with the list of data
structures to be transformed which is accessed through
data_struct_list. */
static void
add_struct_to_data_struct_list (tree type)
{
struct data_structure *d_node;
struct struct_list *new_node = NULL;
int num_fields;
list found = NULL;
type = TYPE_MAIN_VARIANT (type);
found = find_struct_in_list (type);
if (found)
return;
d_node = (struct data_structure *)
xcalloc (1, sizeof (struct data_structure));
num_fields = fields_length (type);
d_node->decl = type;
d_node->num_fields = num_fields;
d_node->fields = get_fields (type, num_fields);
d_node->struct_clustering = NULL;
d_node->accs = NULL;
new_node = (struct struct_list *) xmalloc (sizeof(struct struct_list));
new_node->struct_data = d_node;
new_node->next = data_struct_list;
data_struct_list = new_node;
if (dump_file)
{
fprintf (dump_file, "\nAdding data structure \"");
print_generic_expr (dump_file, type, 0);
fprintf (dump_file, "\" to data_struct_list.");
}
}
static void
free_tree_list (tree_list list_p)
{
tree_list node = list_p;
tree_list tmp_node;
while (node)
{
tmp_node = node;
node = node->next;
free (tmp_node);
}
}
static void
free_field_acc_list (struct field_access_site *accs)
{
struct field_access_site * tmp_acc = accs;
struct field_access_site * tmp_acc1;
if (!accs)
return;
while (tmp_acc)
{
tmp_acc1 = tmp_acc;
tmp_acc = tmp_acc->next;
free (tmp_acc1);
}
}
static void
free_access_list (struct access_site *accs)
{
struct access_site * tmp_acc = accs;
struct access_site * tmp_acc1;
if (!accs)
return;
while (tmp_acc)
{
tmp_acc1 = tmp_acc;
tmp_acc = tmp_acc->next;
free_tree_list (tmp_acc1->vars);
free (tmp_acc1);
}
}
static void
free_data_struct (d_str d_node)
{
int i;
if (!d_node)
return;
if (dump_file)
{
fprintf (dump_file, "\nRemoving data structure \"");
print_generic_expr (dump_file, d_node->decl, 0);
fprintf (dump_file, "\" from data_struct_list.");
}
/* Free all space under d_node. */
if (d_node->fields)
{
for (i = 0; i < d_node->num_fields; i++)
free_field_acc_list (d_node->fields[i].acc_sites);
free (d_node->fields);
}
if (d_node->accs)
free_access_list (d_node->accs);
if (d_node->struct_clustering)
free_struct_cluster (d_node->struct_clustering);
if (d_node->new_types)
free_tree_list (d_node->new_types);
}
static void
remove_struct_from_data_struct_list (struct struct_list *str_to_remove)
{
struct struct_list *curr_str, *prev;
if (data_struct_list == str_to_remove)
{
free_data_struct (data_struct_list->struct_data);
data_struct_list = data_struct_list->next;
free (str_to_remove);
return;
}
for (curr_str = data_struct_list->next, prev = data_struct_list;
curr_str; curr_str = curr_str->next, prev = prev->next)
if (curr_str == str_to_remove)
{
free_data_struct (curr_str->struct_data);
prev->next = curr_str->next;
free (curr_str);
break;
}
}
static void
remove_structs_from_data_struct_list (bitmap nested_structs)
{
list tmp = data_struct_list;
while (tmp)
{
d_str str = tmp->struct_data;
if (bitmap_bit_p (nested_structs, TYPE_UID (str->decl)))
{
list tmp1 = tmp;
tmp = tmp->next;
remove_struct_from_data_struct_list (tmp1);
}
else
tmp = tmp->next;
}
}
static void
free_data_structure_list (void)
{
list current_struct = data_struct_list;
while (current_struct)
{
list tmp = current_struct;
d_str d_node = current_struct->struct_data;
free_data_struct (d_node);
current_struct = current_struct->next;
/* Free curent_struct itself. */
if (tmp)
free (tmp);
}
}
static void
free_malloc_data (void)
{
struct malloc_struct_new *cur_malloc = malloc_data_list_new;
struct malloc_struct_new *tmp;
while (cur_malloc)
{
struct malloc_call_new * call = cur_malloc->malloc_list;
while (call)
{
struct malloc_call_new * call1 = call;
call = call->next;
free (call1);
}
tmp = cur_malloc;
cur_malloc = cur_malloc->next;
free (tmp);
}
}
/* This function calculates maximum distance between
fields in structure according to their counters
collected through profiling. */
static gcov_type
get_max_struct_dist (d_str str)
{
gcov_type max = 0, new_max = 0;
int i, j;
for (i = 0; i < str->num_fields; i++)
for (j = i+1; j < str->num_fields; j++)
{
new_max = abs (str->fields[i].count - str->fields[j].count);
if (new_max > max)
max = new_max;
}
return max;
}
static void
peel_str_type_into_seperate_fields (d_str ds)
{
struct field_cluster *crr_cluster = NULL;
int i;
gcov_type max_dist_in_cluster;
sbitmap fields_left = sbitmap_alloc (ds->num_fields);
sbitmap add_to_cluster = sbitmap_alloc (ds->num_fields);
sbitmap_ones (fields_left);
max_dist_in_cluster =
(gcov_type) (get_max_struct_dist (ds)/100)*MAX_DIST_IN_CLUSTER_RATIO;
i = sbitmap_first_set_bit (fields_left);
while (i != -1)
{
unsigned int j = 0;
sbitmap_iterator sbi;
sbitmap_zero (add_to_cluster);
ds->struct_clustering =
(struct field_cluster *) xcalloc (1, sizeof (struct field_cluster));
ds->struct_clustering->sibling = crr_cluster;
crr_cluster = ds->struct_clustering;
crr_cluster->fields_in_cluster =
sbitmap_alloc ((unsigned int) ds->num_fields);
sbitmap_zero (crr_cluster->fields_in_cluster);
SET_BIT (crr_cluster->fields_in_cluster, i);
RESET_BIT (fields_left, i);
EXECUTE_IF_SET_IN_SBITMAP (fields_left, 0, j, sbi)
{
unsigned int k;
sbitmap_iterator sbi1;
bool suitable = true;
EXECUTE_IF_SET_IN_SBITMAP (crr_cluster->fields_in_cluster, 0, k, sbi1)
{
unsigned int dist = abs (ds->fields[j].count - ds->fields[k].count);
if (dist >= max_dist_in_cluster)
suitable = false;
}
if (suitable)
SET_BIT (add_to_cluster, j);
}
j = 0;
EXECUTE_IF_SET_IN_SBITMAP (add_to_cluster, 0, j, sbi)
{
SET_BIT (crr_cluster->fields_in_cluster, j);
RESET_BIT (fields_left, j);
}
i = sbitmap_first_set_bit (fields_left);
}
sbitmap_free (add_to_cluster);
sbitmap_free (fields_left);
}
/* This function analyzes structure form of structures
in data_struct_list. If we are not capable to transform
structure of some form, we remove it from list of structs.
Right now we cannot handle nested structs, when nesting is
through any level of pointer or arrays.
TBD: release these constrains in future.
Note, that in this function we suppose that all structs
in program are members of data_struct_list right now,
without excluding escaping types.
*/
static void
analyze_struct_form (bitmap non_suitable_types)
{
list tmp = data_struct_list;
while(tmp)
{
d_str str = tmp->struct_data;
int i;
for (i = 0; i < str->num_fields; i++)
{
tree f_type = strip_type(TREE_TYPE (str->fields[i].decl));
if (TREE_CODE (f_type) == RECORD_TYPE)
{
bitmap_set_bit (non_suitable_types, TYPE_UID (TYPE_MAIN_VARIANT (f_type)));
bitmap_set_bit (non_suitable_types, TYPE_UID (str->decl));
}
}
tmp = tmp->next;
}
}
/* This function collect data structures potential
for peeling transformation and insert them into data_struct_list. */
static void
collect_data_structs (void)
{
bitmap non_suitable_types = BITMAP_ALLOC (NULL);
/* If program contains user defined mallocs, we give up. */
if (program_redefines_malloc_p ())
return;
/* Build data structures list of all data structures
in the program. */
build_data_structure_list (non_suitable_types);
/* In this function we analyze whether the form of
structure is such that we are capable to transform it.
Nested structures are checked here. */
analyze_struct_form (non_suitable_types);
/* This function excludes structure types escape compilation unit. */
exclude_escaping_types (non_suitable_types);
/* We cannot change data layout of structs with bitfields. */
exclude_types_with_bit_fields (non_suitable_types);
remove_structs_from_data_struct_list (non_suitable_types);
BITMAP_FREE (non_suitable_types);
if (!data_struct_list)
{
if (dump_file)
fprintf (dump_file, "\nNo structures to transform. Exiting...");
return;
}
}
static void
print_field_acc_sites (struct field_access_site *acc_sites)
{
struct field_access_site *acc;
if (!dump_file)
return;
for (acc = acc_sites; acc; acc = acc->next)
{
fprintf(dump_file, "\n");
if (acc->stmt)
print_generic_stmt (dump_file, acc->stmt, 0);
if (acc->ref_def_stmt)
print_generic_stmt (dump_file, acc->ref_def_stmt, 0);
if (acc->cast_stmt)
print_generic_stmt (dump_file, acc->cast_stmt, 0);
}
}
static void
print_access_sites (struct access_site *acc_sites)
{
struct access_site *acc;
if (!dump_file)
return;
for (acc = acc_sites; acc; acc = acc->next)
{
tree_list list_p;
fprintf(dump_file, "\n");
if (acc->stmt)
print_generic_stmt (dump_file, acc->stmt, 0);
fprintf(dump_file, " : ");
for (list_p = acc->vars; list_p; list_p = list_p->next)
{
print_generic_stmt (dump_file, list_p->data, 0);
fprintf(dump_file, ", ");
}
}
}
static void
print_accesses (void)
{
list tmp = data_struct_list;
if (!dump_file)
return;
for (tmp = data_struct_list; tmp; tmp = tmp->next)
{
int i;
d_str str = tmp->struct_data;
fprintf (dump_file, "\nAccess sites of struct ");
print_generic_expr (dump_file, str->decl, 0);
for (i = 0; i < str->num_fields; i++)
{
fprintf (dump_file, "\nAccess site of field ");
print_generic_expr (dump_file, str->fields[i].decl, 0);
print_field_acc_sites (str->fields[i].acc_sites);
fprintf (dump_file, ":\n");
}
fprintf (dump_file, "\nGeneral access sites\n");
print_access_sites (str->accs);
}
}
static bool
is_safe_cond_expr (tree cond_stmt)
{
tree arg0, arg1;
list tmp0, tmp1;
tree cond = COND_EXPR_COND (cond_stmt);
if (TREE_CODE (cond) != EQ_EXPR
&& TREE_CODE (cond) != NE_EXPR)
return false;
if (TREE_CODE_LENGTH (TREE_CODE (cond)) != 2)
return false;
arg0 = TREE_OPERAND (cond, 0);
arg1 = TREE_OPERAND (cond, 1);
tmp0 = find_struct_in_list (strip_type (get_type_of_var (arg0)));
tmp1 = find_struct_in_list (strip_type (get_type_of_var (arg1)));
if (!((!tmp0 && tmp1) || (!tmp1 && tmp0)))
return false;
return true;
}
static void
check_cond_exprs (void)
{
list str_node = data_struct_list;
while (str_node)
{
d_str str = str_node->struct_data;
struct access_site * acc;
list str_node1 = str_node;
str_node = str_node->next;
for (acc = str->accs; acc; acc = acc->next)
if (TREE_CODE (acc->stmt) == COND_EXPR)
{
if (!is_safe_cond_expr (acc->stmt))
remove_struct_from_data_struct_list (str_node1);
}
}
}
/* This function collects data accesses for the structures
from data_struct_list. For each structure field it updates its count
in data_field_entry. */
static void
collect_data_accesses (void)
{
struct cgraph_node *c_node;
for (c_node = cgraph_nodes; c_node; c_node = c_node->next)
{
enum availability avail = cgraph_function_body_availability (c_node);
if (avail == AVAIL_LOCAL || avail == AVAIL_AVAILABLE)
{
struct function *func = DECL_STRUCT_FUNCTION (c_node->decl);
if (!c_node->next_clone)
{
/* Collect accesses of the fields of structure
under consideretion that happen in func. */
collect_accesses_in_func (func);
}
exclude_malloc_and_field_accs (c_node);
}
}
check_cond_exprs ();
/* Print collected accesses. */
print_accesses ();
}
static void
exclude_cold_structs (void)
{
gcov_type hotest_struct_count = 0;
list tmp = data_struct_list;
/* We summarize counts of fields into structure count. */
while (tmp)
{
int i;
d_str str = tmp->struct_data;
str->count = 0;
for (i = 0; i < str->num_fields; i++)
{
if (dump_file)
{
fprintf (dump_file, "\nCounter of field \"");
print_generic_expr (dump_file, str->fields[i].decl, 0);
fprintf (dump_file, "\" is " HOST_WIDE_INT_PRINT_DEC,
str->fields[i].count);
}
str->count += str->fields[i].count;
}
if (dump_file)
{
fprintf (dump_file, "\nCounter of struct \"");
print_generic_expr (dump_file, str->decl, 0);
fprintf (dump_file, "\" is " HOST_WIDE_INT_PRINT_DEC, str->count);
}
if (str->count > hotest_struct_count)
hotest_struct_count = str->count;
tmp = tmp->next;
}
/* Compare counts of structures. */
tmp = data_struct_list;
/* Remove cold structs from data_struct_list. */
while (tmp)
{
d_str str = tmp->struct_data;
if (str->count < (hotest_struct_count / COLD_STRUCTURE_RATIO))
{
list tmp1 = tmp;
tmp = tmp->next;
remove_struct_from_data_struct_list (tmp1);
}
else
tmp = tmp->next;
}
}
static void
peel_structs (void)
{
list tmp = data_struct_list;
while (tmp)
{
d_str str = tmp->struct_data;
peel_str_type_into_seperate_fields (str);
tmp = tmp->next;
}
}
/* Collect allocation sites - mallocs. In case of arrays
we have nothing to do. */
static void
collect_allocation_sites (void)
{
collect_malloc_data ();
}
/* Free all data structures used in this optimization. */
static void
free_data_structs (void)
{
free_data_structure_list ();
free_malloc_data ();
}
/* Perform full structure decomposition (peeling). */
static void
reorg_structs (void)
{
/* Stage1. */
/* Collect data struct types. */
collect_data_structs ();
collect_allocation_sites ();
/* Collect data accesses. */
collect_data_accesses ();
/* We transform only hot structs. */
exclude_cold_structs ();
/* Stage2. */
peel_structs ();
/* Stage3. */
/* Do the actual transformation. */
do_reorg ();
/* Free all data structures used in this optimization. */
free_data_structs ();
}
static void
add_call_to_malloc_list (tree fn_decl, tree stmt, d_str str)
{
struct malloc_struct_new *cur_malloc = NULL;
struct malloc_struct_new *malloc_node = NULL;
for (cur_malloc = malloc_data_list_new; cur_malloc;
cur_malloc = cur_malloc->next)
if (cur_malloc->func == fn_decl)
break;
if (cur_malloc)
{
struct malloc_call_new * m_call = (struct malloc_call_new *)
xmalloc (sizeof (struct malloc_call_new));
m_call->next = cur_malloc->malloc_list;
m_call->malloc_call_stmt = stmt;
m_call->str = str;
cur_malloc->malloc_list = m_call;
}
else
{
malloc_node = (struct malloc_struct_new *)
xmalloc (sizeof (struct malloc_struct_new));
malloc_node->func = fn_decl;
malloc_node->malloc_list = (struct malloc_call_new *)
xmalloc (sizeof (struct malloc_call_new));
malloc_node->malloc_list->malloc_call_stmt = stmt;
malloc_node->malloc_list->str = str;
malloc_node->malloc_list->next = NULL;
malloc_node->next = malloc_data_list_new;
malloc_data_list_new = malloc_node;
}
if (dump_file)
{
fprintf (dump_file, "\nAdding stmt ");
print_generic_stmt (dump_file, stmt, 0);
fprintf (dump_file, " to list of mallocs.");
}
}
static unsigned int
reorg_structs_drive (void)
{
reorg_structs ();
return 0;
}
static bool
struct_reorg_gate (void)
{
return flag_ipa_struct_reorg && flag_whole_program;
}
struct tree_opt_pass pass_ipa_struct_reorg =
{
"ipa_struct_reorg", /* name */
struct_reorg_gate, /* gate */
reorg_structs_drive, /* execute */
NULL, /* sub */
NULL, /* next */
0, /* static_pass_number */
TV_INTEGRATION, /* tv_id */
0, /* properties_required */
0, /* properties_provided */
0, /* properties_destroyed */
TODO_verify_ssa, /* todo_flags_start */
TODO_dump_func | TODO_verify_ssa, /* todo_flags_finish */
0 /* letter */
};
[-- Attachment #4: struct-reorg.h --]
[-- Type: application/octet-stream, Size: 3720 bytes --]
/* Struct-reorg optimization.
Copyright (C) 2002, 2003-2007 Free Software Foundation, Inc.
Contributed by Olga Golovanevsky <olga@il.ibm.com>
This file is part of GCC.
GCC is free software; you can redistribute it and/or modify
under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
GCC is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with GCC; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
#ifndef IPA_REORG_STRUCTS_H
#define IPA_REORG_STRUCTS_H
/* This file describes the data structure and interface for the data
reorganization optimization. */
struct struct_tree_list {
tree data;
struct struct_tree_list *next;
};
typedef struct struct_tree_list * tree_list;
/* Represents an access site to a data field.
We consider access of the following form:
D.2166_21 = i.6_20 * 8;
D.2167_22 = (struct str_t *) D.2166_21;
D.2168_24 = D.2167_22 + p.5_23;
D.2169_25 = D.2168_24->b;
*/
struct field_access_site
{
/* The statement in which the access site occurs. */
tree stmt; /* D.2169_25 = D.2168_24->b; */
tree comp_ref; /* D.2168_24->b */
tree field_decl; /* b */
tree ref; /* D.2168_24 */
tree num; /* i.6_20 */
tree offset; /* D2167_22 */
tree base; /* p.5_23 */
tree ref_def_stmt; /* D.2168_24 = D.2167_22 + p.5_23; */
tree cast_stmt; /* D.2167_22 = (struct str_t *) D.2166_21;
This stmt is not always present. */
struct field_access_site *next;
};
struct access_site
{
/* The statement in which the access site occurs. */
tree stmt;
/* List of struct vars in stmt. */
tree_list vars;
struct access_site *next;
};
/* Represents a field within a data structure and its accesses. */
struct data_field_entry {
/* Field index, that starts with zero. */
int index;
/* The number of times the field is accessed (according to profiling). */
gcov_type count;
tree decl;
/* Type of new structure this field belongs to. */
tree field_mapping;
struct field_access_site *acc_sites;
};
/* This structure represent result of peeling. */
struct field_cluster {
/* Bitmap of field indexes, a bit is set when the field with that index
is in the cluster. */
sbitmap fields_in_cluster;
struct field_cluster *sibling;
};
/* Represents a data structure that is candidate for
clustering. */
struct data_structure {
/* Main variant of type declaration of the structure. */
tree decl;
/* Number of fields in the structure. */
int num_fields;
/* According to profiling information number of times fields of the
structure were accessed. */
gcov_type count;
/* Array of field entries one for each field of the structure,
indexed by the field ID. */
struct data_field_entry *fields;
/* Stmts with struct accesses that are not a part of field
accesses or allocation patterns. */
struct access_site *accs;
/* This is the a data structure representing result of peeling. */
struct field_cluster *struct_clustering;
/* A linked list of the newly created types, corresponding to the
reorganization of the original structure. */
tree_list new_types;
};
typedef struct data_structure * d_str;
#endif /* IPA_REORG_STRUCTS_H */
^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2007-10-24 12:52 UTC | newest]
Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2007-07-10 9:35 struct-reorg optimization Olga Golovanevsky
2007-07-19 22:57 ` Daniel Berlin
2007-08-01 12:06 ` Olga Golovanevsky
2007-08-17 22:25 ` Jan Hubicka
2007-08-21 22:22 ` Olga Golovanevsky
2007-08-25 10:44 ` Jan Hubicka
2007-08-26 16:21 ` Olga Golovanevsky
2007-08-31 20:33 ` Olga Golovanevsky
2007-09-06 21:04 ` Diego Novillo
2007-10-21 19:58 ` Olga Golovanevsky
2007-10-21 22:01 ` Diego Novillo
2007-10-24 14:40 ` Richard Guenther
2007-10-24 14:45 ` Olga Golovanevsky
2007-09-05 11:48 ` Olga Golovanevsky
2007-07-20 14:39 ` Jan Hubicka
2007-08-01 11:31 ` Olga Golovanevsky
-- strict thread matches above, loose matches on Subject: below --
2007-07-10 9:09 Olga Golovanevsky
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).