From: Dodji Seketeli <dodji@redhat.com>
To: gcc-patches@gcc.gnu.org
Cc: jason@redhat.com, tromey@redhat.com,
gdr@integrable-solutions.net, joseph@codesourcery.com,
burnus@net-b.de, charlet@act-europe.fr
Subject: [PATCH 5/6] Add line map statistics to -fmem-report output
Date: Mon, 17 Oct 2011 10:22:00 -0000 [thread overview]
Message-ID: <1318845428-32182-6-git-send-email-dodji@redhat.com> (raw)
In-Reply-To: <1318845428-32182-1-git-send-email-dodji@redhat.com>
This patch adds statistics about line maps' memory consumption and
macro expansion to the output of -fmem-report. It has been useful in
trying to reduce the memory consumption of the macro maps support.
gcc/ChangeLog
2011-10-15 Tom Tromey <tromey@redhat.com>
Dodji Seketeli <dodji@redhat.com>
* input.c (ONE_K, ONE_M, SCALE, STAT_LABEL, FORMAT_AMOUNT): New
macros.
(num_expanded_macros_counter, num_macro_tokens_counter): Declare
new counters.
(dump_line_table_statistics): Define new function.
* input.h (dump_line_table_statistics): Declare new function.
* toplev.c (dump_memory_report): Call dump_line_table_statistics.
libcpp/ChangeLog
2011-10-15 Tom Tromey <tromey@redhat.com>
Dodji Seketeli <dodji@redhat.com>
* line-map.h (struct linemap_stats): Declare new struct.
(linemap_get_statistics): Declare ...
* line-map.c (linemap_get_statistics): ... new function.
* macro.c (num_expanded_macros_counter, num_macro_tokens_counter):
Declare new counters.
(enter_macro_context, replace_args): Update
num_macro_tokens_counter.
(cpp_get_token_1): Update num_expanded_macros_counter.
---
gcc/ChangeLog | 11 +++++
gcc/input.c | 96 +++++++++++++++++++++++++++++++++++++++++++++
gcc/input.h | 2 +
gcc/toplev.c | 1 +
libcpp/ChangeLog | 12 ++++++
libcpp/include/line-map.h | 21 ++++++++++
libcpp/line-map.c | 63 +++++++++++++++++++++++++++++
libcpp/macro.c | 29 ++++++++++++--
8 files changed, 231 insertions(+), 4 deletions(-)
diff --git a/gcc/input.c b/gcc/input.c
index 89af274..41842b7 100644
--- a/gcc/input.c
+++ b/gcc/input.c
@@ -46,3 +46,99 @@ expand_location (source_location loc)
LRK_SPELLING_LOCATION);
return xloc;
}
+
+#define ONE_K 1024
+#define ONE_M (ONE_K * ONE_K)
+
+/* Display a number as an integer multiple of either:
+ - 1024, if said integer is >= to 10 K (in base 2)
+ - 1024 * 1024, if said integer is >= 10 M in (base 2)
+ */
+#define SCALE(x) ((unsigned long) ((x) < 10 * ONE_K \
+ ? (x) \
+ : ((x) < 10 * ONE_M \
+ ? (x) / ONE_K \
+ : (x) / ONE_M)))
+
+/* For a given integer, display either:
+ - the character 'k', if the number is higher than 10 K (in base 2)
+ but strictly lower than 10 M (in base 2)
+ - the character 'M' if the number is higher than 10 M (in base2)
+ - the charcter ' ' if the number is strictly lower than 10 K */
+#define STAT_LABEL(x) ((x) < 10 * ONE_K ? ' ' : ((x) < 10 * ONE_M ? 'k' : 'M'))
+
+/* Display an integer amount as multiple of 1K or 1M (in base 2).
+ Display the correct unit (either k, M, or ' ') after the amout, as
+ well. */
+#define FORMAT_AMOUNT(size) SCALE (size), STAT_LABEL (size)
+
+/* Dump statistics to stderr about the memory usage of the line_table
+ set of line maps. This also displays some statistics about macro
+ expansion. */
+
+void
+dump_line_table_statistics (void)
+{
+ struct linemap_stats s;
+ size_t total_used_map_size,
+ macro_maps_size,
+ total_allocated_map_size;
+
+ memset (&s, 0, sizeof (s));
+
+ linemap_get_statistics (line_table, &s);
+
+ macro_maps_size = s.macro_maps_used_size
+ + s.macro_maps_locations_size;
+
+ total_allocated_map_size = s.ordinary_maps_allocated_size
+ + s.macro_maps_allocated_size
+ + s.macro_maps_locations_size;
+
+ total_used_map_size = s.ordinary_maps_used_size
+ + s.macro_maps_used_size
+ + s.macro_maps_locations_size;
+
+ fprintf (stderr, "Number of expanded macros: %5lu\n",
+ s.num_expanded_macros);
+ if (s.num_expanded_macros != 0)
+ fprintf (stderr, "Average number of tokens per macro expansion: %5lu\n",
+ s.num_macro_tokens / s.num_expanded_macros);
+ fprintf (stderr,
+ "\nLine Table allocations during the "
+ "compilation process\n");
+ fprintf (stderr, "Number of ordinary maps used: %5lu%c\n",
+ SCALE (s.num_ordinary_maps_used),
+ STAT_LABEL (s.num_ordinary_maps_used));
+ fprintf (stderr, "Ordinary map used size: %5lu%c\n",
+ SCALE (s.ordinary_maps_used_size),
+ STAT_LABEL (s.ordinary_maps_used_size));
+ fprintf (stderr, "Number of ordinary maps allocated: %5lu%c\n",
+ SCALE (s.num_ordinary_maps_allocated),
+ STAT_LABEL (s.num_ordinary_maps_allocated));
+ fprintf (stderr, "Ordinary maps allocated size: %5lu%c\n",
+ SCALE (s.ordinary_maps_allocated_size),
+ STAT_LABEL (s.ordinary_maps_allocated_size));
+ fprintf (stderr, "Number of macro maps used: %5lu%c\n",
+ SCALE (s.num_macro_maps_used),
+ STAT_LABEL (s.num_macro_maps_used));
+ fprintf (stderr, "Macro maps used size: %5lu%c\n",
+ SCALE (s.macro_maps_used_size),
+ STAT_LABEL (s.macro_maps_used_size));
+ fprintf (stderr, "Macro maps locations size: %5lu%c\n",
+ SCALE (s.macro_maps_locations_size),
+ STAT_LABEL (s.macro_maps_locations_size));
+ fprintf (stderr, "Macro maps size: %5lu%c\n",
+ SCALE (macro_maps_size),
+ STAT_LABEL (macro_maps_size));
+ fprintf (stderr, "Duplicated maps locations size: %5lu%c\n",
+ SCALE (s.duplicated_macro_maps_locations_size),
+ STAT_LABEL (s.duplicated_macro_maps_locations_size));
+ fprintf (stderr, "Total allocated maps size: %5lu%c\n",
+ SCALE (total_allocated_map_size),
+ STAT_LABEL (total_allocated_map_size));
+ fprintf (stderr, "Total used maps size: %5lu%c\n",
+ SCALE (total_used_map_size),
+ STAT_LABEL (total_used_map_size));
+ fprintf (stderr, "\n");
+}
diff --git a/gcc/input.h b/gcc/input.h
index 9fc55f3..f2f3513 100644
--- a/gcc/input.h
+++ b/gcc/input.h
@@ -55,4 +55,6 @@ extern location_t input_location;
((linemap_location_in_system_header_p (line_table, LOC)))
#define in_system_header (in_system_header_at (input_location))
+void dump_line_table_statistics (void);
+
#endif
diff --git a/gcc/toplev.c b/gcc/toplev.c
index 0188755..547ce7a 100644
--- a/gcc/toplev.c
+++ b/gcc/toplev.c
@@ -1827,6 +1827,7 @@ target_reinit (void)
void
dump_memory_report (bool final)
{
+ dump_line_table_statistics ();
ggc_print_statistics ();
stringpool_statistics ();
dump_tree_statistics ();
diff --git a/libcpp/include/line-map.h b/libcpp/include/line-map.h
index 04a523c..572e330 100644
--- a/libcpp/include/line-map.h
+++ b/libcpp/include/line-map.h
@@ -675,6 +675,27 @@ expanded_location linemap_expand_location_full (struct line_maps *,
source_location loc,
enum location_resolution_kind lrk);
+/* Statistics about maps allocation and usage as returned by
+ linemap_get_statistics. */
+struct linemap_stats
+{
+ size_t num_ordinary_maps_allocated;
+ size_t num_ordinary_maps_used;
+ size_t ordinary_maps_allocated_size;
+ size_t ordinary_maps_used_size;
+ size_t num_expanded_macros;
+ size_t num_macro_tokens;
+ size_t num_macro_maps_used;
+ size_t macro_maps_allocated_size;
+ size_t macro_maps_used_size;
+ size_t macro_maps_locations_size;
+ size_t duplicated_macro_maps_locations_size;
+};
+
+/* Compute and return statistics about the memory consumption of some
+ parts of the line table SET. */
+void linemap_get_statistics (struct line_maps *, struct linemap_stats *);
+
/* Dump debugging information about source location LOC into the file
stream STREAM. SET is the line map set LOC comes from. */
void linemap_dump_location (struct line_maps *, source_location, FILE *);
diff --git a/libcpp/line-map.c b/libcpp/line-map.c
index 3dbaeab..9086b3e 100644
--- a/libcpp/line-map.c
+++ b/libcpp/line-map.c
@@ -46,6 +46,10 @@ static source_location linemap_macro_loc_to_exp_point (struct line_maps *,
source_location,
const struct line_map **);
+/* Counters defined in macro.c. */
+extern unsigned num_expanded_macros_counter;
+extern unsigned num_macro_tokens_counter;
+
/* Initialize a line map set. */
void
@@ -1143,3 +1147,62 @@ linemap_dump_location (struct line_maps *set,
fprintf (stream, "{P:%s;F:%s;L:%d;C:%d;S:%d;M:%p;E:%d,LOC:%d}",
path, from, l, c, s, (void*)map, e, loc);
}
+
+/* Compute and return statistics about the memory consumption of some
+ parts of the line table SET. */
+
+void
+linemap_get_statistics (struct line_maps *set,
+ struct linemap_stats *s)
+{
+ size_t ordinary_maps_allocated_size, ordinary_maps_used_size,
+ macro_maps_allocated_size, macro_maps_used_size,
+ macro_maps_locations_size = 0, duplicated_macro_maps_locations_size = 0;
+
+ struct line_map *cur_map;
+
+ ordinary_maps_allocated_size =
+ LINEMAPS_ORDINARY_ALLOCATED (set) * sizeof (struct line_map);
+
+ ordinary_maps_used_size =
+ LINEMAPS_ORDINARY_USED (set) * sizeof (struct line_map);
+
+ macro_maps_allocated_size =
+ LINEMAPS_MACRO_ALLOCATED (set) * sizeof (struct line_map);
+
+ for (cur_map = LINEMAPS_MACRO_MAPS (set);
+ cur_map && cur_map <= LINEMAPS_LAST_MACRO_MAP (set);
+ ++cur_map)
+ {
+ unsigned i;
+
+ linemap_assert (linemap_macro_expansion_map_p (cur_map));
+
+ macro_maps_locations_size +=
+ 2 * MACRO_MAP_NUM_MACRO_TOKENS (cur_map) * sizeof (source_location);
+
+ for (i = 0; i < 2 * MACRO_MAP_NUM_MACRO_TOKENS (cur_map); i += 2)
+ {
+ if (MACRO_MAP_LOCATIONS (cur_map)[i] ==
+ MACRO_MAP_LOCATIONS (cur_map)[i + 1])
+ duplicated_macro_maps_locations_size +=
+ sizeof (source_location);
+ }
+ }
+
+ macro_maps_used_size =
+ LINEMAPS_MACRO_USED (set) * sizeof (struct line_map);
+
+ s->num_ordinary_maps_allocated = LINEMAPS_ORDINARY_ALLOCATED (set);
+ s->num_ordinary_maps_used = LINEMAPS_ORDINARY_USED (set);
+ s->ordinary_maps_allocated_size = ordinary_maps_allocated_size;
+ s->ordinary_maps_used_size = ordinary_maps_used_size;
+ s->num_expanded_macros = num_expanded_macros_counter;
+ s->num_macro_tokens = num_macro_tokens_counter;
+ s->num_macro_maps_used = LINEMAPS_MACRO_USED (set);
+ s->macro_maps_allocated_size = macro_maps_allocated_size;
+ s->macro_maps_locations_size = macro_maps_locations_size;
+ s->macro_maps_used_size = macro_maps_used_size;
+ s->duplicated_macro_maps_locations_size =
+ duplicated_macro_maps_locations_size;
+}
diff --git a/libcpp/macro.c b/libcpp/macro.c
index d760383..2d0eeaa 100644
--- a/libcpp/macro.c
+++ b/libcpp/macro.c
@@ -165,6 +165,13 @@ static void consume_next_token_from_context (cpp_reader *pfile,
source_location *);
static const cpp_token* cpp_get_token_1 (cpp_reader *, source_location *);
+/* Statistical counter tracking the number of macros that got
+ expanded. */
+unsigned num_expanded_macros_counter = 0;
+/* Statistical counter tracking the total number tokens resulting
+ from macro expansion. */
+unsigned num_macro_tokens_counter = 0;
+
/* Emits a warning if NODE is a macro defined in the main file that
has not been used. */
int
@@ -1082,10 +1089,15 @@ enter_macro_context (cpp_reader *pfile, cpp_hashnode *node,
(const cpp_token **)
macro_tokens->base,
count);
+ num_macro_tokens_counter += count;
}
else
- _cpp_push_token_context (pfile, node, macro->exp.tokens,
- macro_real_token_count (macro));
+ {
+ unsigned tokens_count = macro_real_token_count (macro);
+ _cpp_push_token_context (pfile, node, macro->exp.tokens,
+ tokens_count);
+ num_macro_tokens_counter += tokens_count;
+ }
}
if (pragma_buff)
@@ -1095,13 +1107,18 @@ enter_macro_context (cpp_reader *pfile, cpp_hashnode *node,
padding_token (pfile, result), 1);
do
{
+ unsigned tokens_count;
_cpp_buff *tail = pragma_buff->next;
pragma_buff->next = NULL;
+ tokens_count = ((const cpp_token **) BUFF_FRONT (pragma_buff)
+ - (const cpp_token **) pragma_buff->base);
push_ptoken_context (pfile, NULL, pragma_buff,
(const cpp_token **) pragma_buff->base,
- ((const cpp_token **) BUFF_FRONT (pragma_buff)
- - (const cpp_token **) pragma_buff->base));
+ tokens_count);
pragma_buff = tail;
+ if (!CPP_OPTION (pfile, track_macro_expansion))
+ num_macro_tokens_counter += tokens_count;
+
}
while (pragma_buff != NULL);
return 2;
@@ -1711,6 +1728,8 @@ replace_args (cpp_reader *pfile, cpp_hashnode *node, cpp_macro *macro,
else
push_ptoken_context (pfile, node, buff, first,
tokens_buff_count (buff));
+
+ num_macro_tokens_counter += tokens_buff_count (buff);
}
/* Return a special padding token, with padding inherited from SOURCE. */
@@ -2240,6 +2259,8 @@ cpp_get_token_1 (cpp_reader *pfile, source_location *location)
}
else
{
+ if (pfile->context->c.macro)
+ ++num_expanded_macros_counter;
_cpp_pop_context (pfile);
if (pfile->state.in_directive)
continue;
--
1.7.6.4
next prev parent reply other threads:[~2011-10-17 9:57 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-10-17 9:58 [PATCH 0/6] Tracking locations of tokens resulting from macro expansion Dodji Seketeli
2011-10-17 9:58 ` [PATCH 3/6] Emit macro expansion related diagnostics Dodji Seketeli
2011-10-17 10:56 ` Richard Guenther
2011-10-17 12:22 ` Dodji Seketeli
2011-10-17 14:11 ` Dodji Seketeli
2011-10-17 17:41 ` H.J. Lu
2011-10-18 0:29 ` Dodji Seketeli
2011-10-18 6:07 ` Jason Merrill
2011-10-18 9:22 ` Dodji Seketeli
2011-10-17 10:08 ` [PATCH 4/6] Support -fdebug-cpp option Dodji Seketeli
2011-10-17 10:19 ` [PATCH 6/6] Reduce memory waste due to non-power-of-2 allocs Dodji Seketeli
2011-10-17 10:22 ` Dodji Seketeli [this message]
2011-10-17 10:25 ` [PATCH 1/6] Linemap infrastructure for virtual locations Dodji Seketeli
2011-10-17 19:53 ` Gerald Pfeifer
2011-10-17 20:44 ` Dodji Seketeli
2011-10-18 7:24 ` Gerald Pfeifer
2011-10-18 9:44 ` Dodji Seketeli
2011-10-17 10:26 ` [PATCH 2/6] Generate virtual locations for tokens Dodji Seketeli
2011-10-19 14:48 ` Ulrich Weigand
2011-10-19 17:11 ` Dodji Seketeli
2011-10-19 20:42 ` Dodji Seketeli
-- strict thread matches above, loose matches on Subject: below --
2010-12-10 11:27 [PATCH 0/6] Tracking locations of tokens resulting from macro expansion Dodji Seketeli
2010-12-10 11:27 ` [PATCH 5/6] Add line map statistics to -fmem-report output Dodji Seketeli
2010-12-21 7:30 ` Gabriel Dos Reis
2011-04-13 20:08 ` Dodji Seketeli
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1318845428-32182-6-git-send-email-dodji@redhat.com \
--to=dodji@redhat.com \
--cc=burnus@net-b.de \
--cc=charlet@act-europe.fr \
--cc=gcc-patches@gcc.gnu.org \
--cc=gdr@integrable-solutions.net \
--cc=jason@redhat.com \
--cc=joseph@codesourcery.com \
--cc=tromey@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).