* [libcpp] RFA: Add support for comments retrieval @ 2008-09-26 15:23 Arnaud Charlet 2008-09-26 15:44 ` Basile STARYNKEVITCH 2008-09-26 16:07 ` Tom Tromey 0 siblings, 2 replies; 10+ messages in thread From: Arnaud Charlet @ 2008-09-26 15:23 UTC (permalink / raw) To: gcc-patches; +Cc: Matthew Gingell This is a patch that adds support for easy retrieval of C/C++ comments parsed by libcpp. We have another patch depending on this one that would then take advantage of this new "cpp_get_comments" function. This other patch will add the capability to dump GCC GENERIC trees using Ada syntax, thus providing a way to generate Ada specs automatically from C/C++ header files. In this context, being able to also dump the original comments from the header files is very useful. I'm sure this function could then be used in other contexts (e.g. other kind of source code analyzer/plug-ins based on GCC). Tested on i686-pc-linux-gnu, OK for trunk ? 2008-09-26 Matthew Gingell <gingell@adacore.com> Arnaud Charlet <charlet@adacore.com> * include/cpplib.h (cpp_comments, cpp_comment_table): New structs. (cpp_get_comments): New function. * lex.c (store_comment, cpp_get_comments): New function.s (comments): New struct. (save_comment): Store comments in comments struct. Index: include/cpplib.h =================================================================== --- include/cpplib.h (revision 140660) +++ include/cpplib.h (working copy) @@ -870,6 +870,23 @@ extern const char *cpp_type2name (enum c extern cppchar_t cpp_parse_escape (cpp_reader *, const unsigned char ** pstr, const unsigned char *limit, int wide); +typedef struct +{ + const char *comment; + source_location sloc; +} cpp_comment; + +typedef struct +{ + cpp_comment *entries; + int count; + int allocated; +} cpp_comment_table; + +/* Returns the table of comments encountered by the preprocessor. This + table is only populated when pfile->state.save_comments is true. */ +extern cpp_comment_table cpp_get_comments (void); + /* In hash.c */ /* Lookup an identifier in the hashtable. Puts the identifier in the Index: lex.c =================================================================== --- lex.c (revision 140660) +++ lex.c (working copy) @@ -55,6 +55,7 @@ static int skip_line_comment (cpp_reader static void skip_whitespace (cpp_reader *, cppchar_t); static void lex_string (cpp_reader *, cpp_token *, const uchar *); static void save_comment (cpp_reader *, cpp_token *, const uchar *, cppchar_t); +static void store_comment (cpp_token *); static void create_literal (cpp_reader *, cpp_token *, const uchar *, unsigned int, enum cpp_ttype); static bool warn_in_comment (cpp_reader *, _cpp_line_note *); @@ -670,6 +671,44 @@ lex_string (cpp_reader *pfile, cpp_token create_literal (pfile, token, base, cur - base, type); } +/* Table for storing comments in the pfile->state.save_comments case. + In other cases there is no overhead. */ + +static cpp_comment_table comments = {NULL, 0, 0}; + +/* Return the comment table. The client may not make any assumption + about the ordering of the table. */ +extern cpp_comment_table +cpp_get_comments (void) +{ + return comments; +} + +/* Append a comment to the end of the comment table. */ +static void +store_comment (cpp_token *token) +{ + if (comments.allocated == 0) + { + comments.allocated = 256; + comments.entries = + (cpp_comment *) xmalloc (comments.allocated * sizeof (cpp_comment)); + } + + if (comments.count == comments.allocated) + { + comments.allocated *= 2; + comments.entries = + (cpp_comment *) xrealloc (comments.entries, + comments.allocated * sizeof (cpp_comment)); + } + + comments.entries [comments.count].comment = + xstrdup ((char *) (token->val.str.text)); + comments.entries [comments.count].sloc = token->src_loc; + comments.count++; +} + /* The stored comment includes the comment start and any terminator. */ static void save_comment (cpp_reader *pfile, cpp_token *token, const unsigned char *from, @@ -709,6 +748,9 @@ save_comment (cpp_reader *pfile, cpp_tok buffer[clen - 2] = '*'; buffer[clen - 1] = '/'; } + + /* Finally store this comment for use by clients of libcpp. */ + store_comment (token); } /* Allocate COUNT tokens for RUN. */ ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [libcpp] RFA: Add support for comments retrieval 2008-09-26 15:23 [libcpp] RFA: Add support for comments retrieval Arnaud Charlet @ 2008-09-26 15:44 ` Basile STARYNKEVITCH 2008-09-26 16:07 ` Arnaud Charlet 2008-09-26 16:07 ` Tom Tromey 1 sibling, 1 reply; 10+ messages in thread From: Basile STARYNKEVITCH @ 2008-09-26 15:44 UTC (permalink / raw) To: Arnaud Charlet; +Cc: gcc-patches, Matthew Gingell Arnaud Charlet wrote: > This is a patch that adds support for easy retrieval of C/C++ comments > parsed by libcpp. > > We have another patch depending on this one that would then take advantage > of this new "cpp_get_comments" function. This other patch will add the > capability to dump GCC GENERIC trees using Ada syntax, thus providing > a way to generate Ada specs automatically from C/C++ header files. > In this context, being able to also dump the original comments from the > header files is very useful. > > I'm sure this function could then be used in other contexts (e.g. other > kind of source code analyzer/plug-ins based on GCC). I agree, but in that case I suggest adding some documentation somewhere, probably a small paragraph in some *.texi file. I am currently not able to figure out how could I use such a function, even if it interests me a lot. My previous thinking was that comments is the first thing a lexer wants to be removed! -- Basile STARYNKEVITCH http://starynkevitch.net/Basile/ email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359 8, rue de la Faiencerie, 92340 Bourg La Reine, France *** opinions {are only mines, sont seulement les miennes} *** ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [libcpp] RFA: Add support for comments retrieval 2008-09-26 15:44 ` Basile STARYNKEVITCH @ 2008-09-26 16:07 ` Arnaud Charlet 0 siblings, 0 replies; 10+ messages in thread From: Arnaud Charlet @ 2008-09-26 16:07 UTC (permalink / raw) To: Basile STARYNKEVITCH; +Cc: gcc-patches, Matthew Gingell > I agree, but in that case I suggest adding some documentation somewhere, > probably a small paragraph in some *.texi file. I am currently not able to It's documented in include/cpplib.h, but if people think more documentation elsewhere would be helpful, that's fine with me, lathough I do not really see why this function would be different from others that are documented in the sources directly. > figure out how could I use such a function, even if it interests me a lot. > My previous thinking was that comments is the first thing a lexer wants to > be removed! Note that the comment handling is optional in libcpp, triggered with the -C switch, so this patch only affects this mode. Arno ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [libcpp] RFA: Add support for comments retrieval 2008-09-26 15:23 [libcpp] RFA: Add support for comments retrieval Arnaud Charlet 2008-09-26 15:44 ` Basile STARYNKEVITCH @ 2008-09-26 16:07 ` Tom Tromey 2008-09-26 16:54 ` Arnaud Charlet 1 sibling, 1 reply; 10+ messages in thread From: Tom Tromey @ 2008-09-26 16:07 UTC (permalink / raw) To: Arnaud Charlet; +Cc: gcc-patches, Matthew Gingell >>>>> "Arnaud" == Arnaud Charlet <charlet@adacore.com> writes: Arnaud> This is a patch that adds support for easy retrieval of C/C++ comments Arnaud> parsed by libcpp. I think the general idea is ok. There are a few changes I would like. Some of these are nits :) Arnaud> * lex.c (store_comment, cpp_get_comments): New function.s Trailing 's'. Arnaud> +typedef struct Arnaud> +{ Arnaud> + const char *comment; Arnaud> + source_location sloc; Arnaud> +} cpp_comment; Please put a comment on the struct and the various fields. Arnaud> +typedef struct Arnaud> +{ Arnaud> + cpp_comment *entries; Arnaud> + int count; Arnaud> + int allocated; Arnaud> +} cpp_comment_table; Ditto. Arnaud> +/* Table for storing comments in the pfile->state.save_comments case. Arnaud> + In other cases there is no overhead. */ Arnaud> + Arnaud> +static cpp_comment_table comments = {NULL, 0, 0}; Instead of a global, I think this should be in struct cpp_reader. This implies that cpp_get_comments needs a cpp_reader argument. Also, destroying the cpp_reader should free the comment entries. I wonder if we should have a new option (in cpp_options) controlling this. Tom ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [libcpp] RFA: Add support for comments retrieval 2008-09-26 16:07 ` Tom Tromey @ 2008-09-26 16:54 ` Arnaud Charlet 2008-09-26 18:21 ` Paolo Bonzini 0 siblings, 1 reply; 10+ messages in thread From: Arnaud Charlet @ 2008-09-26 16:54 UTC (permalink / raw) To: Tom Tromey; +Cc: gcc-patches, Matthew Gingell > I think the general idea is ok. Thanks. > There are a few changes I would like. Some of these are nits :) Sure, thanks for the quick review. Here is a new version taking your comments into account: 2008-09-26 Matthew Gingell <gingell@adacore.com> Arnaud Charlet <charlet@adacore.com> * include/cpplib.h (cpp_comments, cpp_comment_table): New structs. (cpp_get_comments): New function. * internal.h (struct cpp_reader): Add comments field. * init.c (cpp_destroy): Free comments. * lex.c (store_comment, cpp_get_comments): New functions. (comments): New struct. (save_comment): Store comments in comments struct. Index: include/cpplib.h =================================================================== --- include/cpplib.h (revision 140660) +++ include/cpplib.h (working copy) @@ -870,6 +870,28 @@ extern const char *cpp_type2name (enum c extern cppchar_t cpp_parse_escape (cpp_reader *, const unsigned char ** pstr, const unsigned char *limit, int wide); +/* Structure used to hold a comment block at a given location in the + source code. */ + +typedef struct +{ + const char *comment; /* string containing comment at a given location. */ + source_location sloc; /* source location for the given comment. */ +} cpp_comment; + +/* Structure holding all comments for a given cpp_reader. */ + +typedef struct +{ + cpp_comment *entries; /* table of comment entries. */ + int count; /* number of actual entries entered in the table. */ + int allocated; /* number of entries allocated currently. */ +} cpp_comment_table; + +/* Returns the table of comments encountered by the preprocessor. This + table is only populated when pfile->state.save_comments is true. */ +extern cpp_comment_table *cpp_get_comments (cpp_reader *); + /* In hash.c */ /* Lookup an identifier in the hashtable. Puts the identifier in the Index: init.c =================================================================== --- init.c (revision 140660) +++ init.c (working copy) @@ -245,6 +245,7 @@ cpp_destroy (cpp_reader *pfile) { cpp_context *context, *contextn; tokenrun *run, *runn; + int i; free (pfile->op_stack); @@ -287,6 +288,14 @@ cpp_destroy (cpp_reader *pfile) free (context); } + if (pfile->comments.entries) + { + for (i = 0; i < pfile->comments.count; i++) + free (pfile->comments.entries [i].comment); + + free (pfile->comments.entries); + } + free (pfile); } Index: internal.h =================================================================== --- internal.h (revision 140660) +++ internal.h (working copy) @@ -471,6 +471,9 @@ struct cpp_reader /* Next value of __COUNTER__ macro. */ unsigned int counter; + + /* Table of comments, when state.save_comments is true. */ + cpp_comment_table comments; }; /* Character classes. Based on the more primitive macros in safe-ctype.h. Index: lex.c =================================================================== --- lex.c (revision 140660) +++ lex.c (working copy) @@ -55,6 +55,7 @@ static int skip_line_comment (cpp_reader static void skip_whitespace (cpp_reader *, cppchar_t); static void lex_string (cpp_reader *, cpp_token *, const uchar *); static void save_comment (cpp_reader *, cpp_token *, const uchar *, cppchar_t); +static void store_comment (cpp_reader *, cpp_token *); static void create_literal (cpp_reader *, cpp_token *, const uchar *, unsigned int, enum cpp_ttype); static bool warn_in_comment (cpp_reader *, _cpp_line_note *); @@ -670,6 +671,39 @@ lex_string (cpp_reader *pfile, cpp_token create_literal (pfile, token, base, cur - base, type); } +/* Return the comment table. The client may not make any assumption + about the ordering of the table. */ +cpp_comment_table * +cpp_get_comments (cpp_reader *pfile) +{ + return &pfile->comments; +} + +/* Append a comment to the end of the comment table. */ +static void +store_comment (cpp_reader *pfile, cpp_token *token) +{ + if (pfile->comments.allocated == 0) + { + pfile->comments.allocated = 256; + pfile->comments.entries = (cpp_comment *) xmalloc + (pfile->comments.allocated * sizeof (cpp_comment)); + } + + if (pfile->comments.count == pfile->comments.allocated) + { + pfile->comments.allocated *= 2; + pfile->comments.entries = (cpp_comment *) xrealloc + (pfile->comments.entries, + pfile->comments.allocated * sizeof (cpp_comment)); + } + + pfile->comments.entries [pfile->comments.count].comment = + xstrdup ((char *) (token->val.str.text)); + pfile->comments.entries [pfile->comments.count].sloc = token->src_loc; + pfile->comments.count++; +} + /* The stored comment includes the comment start and any terminator. */ static void save_comment (cpp_reader *pfile, cpp_token *token, const unsigned char *from, @@ -709,6 +743,9 @@ save_comment (cpp_reader *pfile, cpp_tok buffer[clen - 2] = '*'; buffer[clen - 1] = '/'; } + + /* Finally store this comment for use by clients of libcpp. */ + store_comment (pfile, token); } /* Allocate COUNT tokens for RUN. */ ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [libcpp] RFA: Add support for comments retrieval 2008-09-26 16:54 ` Arnaud Charlet @ 2008-09-26 18:21 ` Paolo Bonzini 2008-09-26 21:14 ` Arnaud Charlet 0 siblings, 1 reply; 10+ messages in thread From: Paolo Bonzini @ 2008-09-26 18:21 UTC (permalink / raw) To: Arnaud Charlet; +Cc: Tom Tromey, gcc-patches, Matthew Gingell > +typedef struct > +{ > + const char *comment; /* string containing comment at a given location. */ Should be /* Text of the comment including (or is it excluding?) the terminators. */ const char *comment; /* Source location for the given comment. */ source_location sloc; ... and likewise for the other struct. Thanks, Paolo ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [libcpp] RFA: Add support for comments retrieval 2008-09-26 18:21 ` Paolo Bonzini @ 2008-09-26 21:14 ` Arnaud Charlet 2008-09-26 21:17 ` Arnaud Charlet 0 siblings, 1 reply; 10+ messages in thread From: Arnaud Charlet @ 2008-09-26 21:14 UTC (permalink / raw) To: Paolo Bonzini; +Cc: Tom Tromey, gcc-patches, Matthew Gingell > Should be > > /* Text of the comment including (or is it excluding?) the > terminators. */ > const char *comment; > > /* Source location for the given comment. */ > source_location sloc; > > ... and likewise for the other struct. > > Thanks, Absolutely, will fix. Is the patch OK with these changes ? Arno ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [libcpp] RFA: Add support for comments retrieval 2008-09-26 21:14 ` Arnaud Charlet @ 2008-09-26 21:17 ` Arnaud Charlet 2008-10-03 22:06 ` Tom Tromey 0 siblings, 1 reply; 10+ messages in thread From: Arnaud Charlet @ 2008-09-26 21:17 UTC (permalink / raw) To: Paolo Bonzini; +Cc: Tom Tromey, gcc-patches, Matthew Gingell > Is the patch OK with these changes ? Appended for completeness. 2008-09-26 Matthew Gingell <gingell@adacore.com> Arnaud Charlet <charlet@adacore.com> * include/cpplib.h (cpp_comments, cpp_comment_table): New structs. (cpp_get_comments): New function. * internal.h (struct cpp_reader): Add comments field. * init.c (cpp_destroy): Free comments. * lex.c (store_comment, cpp_get_comments): New functions. (comments): New struct. (save_comment): Store comments in comments struct. Index: include/cpplib.h =================================================================== --- include/cpplib.h (revision 140660) +++ include/cpplib.h (working copy) @@ -870,6 +870,36 @@ extern const char *cpp_type2name (enum c extern cppchar_t cpp_parse_escape (cpp_reader *, const unsigned char ** pstr, const unsigned char *limit, int wide); +/* Structure used to hold a comment block at a given location in the + source code. */ + +typedef struct +{ + /* Text of the comment including the terminators. */ + char *comment; + + /* source location for the given comment. */ + source_location sloc; +} cpp_comment; + +/* Structure holding all comments for a given cpp_reader. */ + +typedef struct +{ + /* table of comment entries. */ + cpp_comment *entries; + + /* number of actual entries entered in the table. */ + int count; + + /* number of entries allocated currently. */ + int allocated; +} cpp_comment_table; + +/* Returns the table of comments encountered by the preprocessor. This + table is only populated when pfile->state.save_comments is true. */ +extern cpp_comment_table *cpp_get_comments (cpp_reader *); + /* In hash.c */ /* Lookup an identifier in the hashtable. Puts the identifier in the Index: init.c =================================================================== --- init.c (revision 140660) +++ init.c (working copy) @@ -245,6 +245,7 @@ cpp_destroy (cpp_reader *pfile) { cpp_context *context, *contextn; tokenrun *run, *runn; + int i; free (pfile->op_stack); @@ -287,6 +288,14 @@ cpp_destroy (cpp_reader *pfile) free (context); } + if (pfile->comments.entries) + { + for (i = 0; i < pfile->comments.count; i++) + free (pfile->comments.entries [i].comment); + + free (pfile->comments.entries); + } + free (pfile); } Index: internal.h =================================================================== --- internal.h (revision 140660) +++ internal.h (working copy) @@ -471,6 +471,9 @@ struct cpp_reader /* Next value of __COUNTER__ macro. */ unsigned int counter; + + /* Table of comments, when state.save_comments is true. */ + cpp_comment_table comments; }; /* Character classes. Based on the more primitive macros in safe-ctype.h. Index: lex.c =================================================================== --- lex.c (revision 140660) +++ lex.c (working copy) @@ -55,6 +55,7 @@ static int skip_line_comment (cpp_reader static void skip_whitespace (cpp_reader *, cppchar_t); static void lex_string (cpp_reader *, cpp_token *, const uchar *); static void save_comment (cpp_reader *, cpp_token *, const uchar *, cppchar_t); +static void store_comment (cpp_reader *, cpp_token *); static void create_literal (cpp_reader *, cpp_token *, const uchar *, unsigned int, enum cpp_ttype); static bool warn_in_comment (cpp_reader *, _cpp_line_note *); @@ -670,6 +671,39 @@ lex_string (cpp_reader *pfile, cpp_token create_literal (pfile, token, base, cur - base, type); } +/* Return the comment table. The client may not make any assumption + about the ordering of the table. */ +cpp_comment_table * +cpp_get_comments (cpp_reader *pfile) +{ + return &pfile->comments; +} + +/* Append a comment to the end of the comment table. */ +static void +store_comment (cpp_reader *pfile, cpp_token *token) +{ + if (pfile->comments.allocated == 0) + { + pfile->comments.allocated = 256; + pfile->comments.entries = (cpp_comment *) xmalloc + (pfile->comments.allocated * sizeof (cpp_comment)); + } + + if (pfile->comments.count == pfile->comments.allocated) + { + pfile->comments.allocated *= 2; + pfile->comments.entries = (cpp_comment *) xrealloc + (pfile->comments.entries, + pfile->comments.allocated * sizeof (cpp_comment)); + } + + pfile->comments.entries [pfile->comments.count].comment = + xstrdup ((char *) (token->val.str.text)); + pfile->comments.entries [pfile->comments.count].sloc = token->src_loc; + pfile->comments.count++; +} + /* The stored comment includes the comment start and any terminator. */ static void save_comment (cpp_reader *pfile, cpp_token *token, const unsigned char *from, @@ -709,6 +743,9 @@ save_comment (cpp_reader *pfile, cpp_tok buffer[clen - 2] = '*'; buffer[clen - 1] = '/'; } + + /* Finally store this comment for use by clients of libcpp. */ + store_comment (pfile, token); } /* Allocate COUNT tokens for RUN. */ ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [libcpp] RFA: Add support for comments retrieval 2008-09-26 21:17 ` Arnaud Charlet @ 2008-10-03 22:06 ` Tom Tromey 2008-10-05 13:19 ` Arnaud Charlet 0 siblings, 1 reply; 10+ messages in thread From: Tom Tromey @ 2008-10-03 22:06 UTC (permalink / raw) To: Arnaud Charlet; +Cc: Paolo Bonzini, gcc-patches, Matthew Gingell >>>>> "Arnaud" == Arnaud Charlet <charlet@adacore.com> writes: >> Is the patch OK with these changes ? Arnaud> Appended for completeness. I have two trivial nits. It is ok with these changes. Normally I would say that this should be delayed until Stage 1, but I think it is reasonably small and safe, and so it can go in now. Arnaud> 2008-09-26 Matthew Gingell <gingell@adacore.com> Arnaud> Arnaud Charlet <charlet@adacore.com> Arnaud> * include/cpplib.h (cpp_comments, cpp_comment_table): New structs. Arnaud> (cpp_get_comments): New function. Arnaud> * internal.h (struct cpp_reader): Add comments field. [...] For a single change which spans multiple files, I don't put blank lines between the file entries in the ChangeLog. Arnaud> + free (pfile->comments.entries [i].comment); No space before the '['. There are a couple cases of this. Tom ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [libcpp] RFA: Add support for comments retrieval 2008-10-03 22:06 ` Tom Tromey @ 2008-10-05 13:19 ` Arnaud Charlet 0 siblings, 0 replies; 10+ messages in thread From: Arnaud Charlet @ 2008-10-05 13:19 UTC (permalink / raw) To: Tom Tromey; +Cc: Paolo Bonzini, gcc-patches, Matthew Gingell > I have two trivial nits. It is ok with these changes. > > Normally I would say that this should be delayed until Stage 1, but I > think it is reasonably small and safe, and so it can go in now. Understood, thanks. Here is the final patch with nits fixed, retested and committed on trunk. 2008-10-05 Matthew Gingell <gingell@adacore.com> Arnaud Charlet <charlet@adacore.com> * include/cpplib.h (cpp_comments, cpp_comment_table): New structs. (cpp_get_comments): New function. * internal.h (struct cpp_reader): Add comments field. * init.c (cpp_destroy): Free comments. * lex.c (store_comment, cpp_get_comments): New functions. (comments): New struct. (save_comment): Store comments in comments struct. Index: include/cpplib.h =================================================================== --- include/cpplib.h (revision 140660) +++ include/cpplib.h (working copy) @@ -870,6 +870,36 @@ extern const char *cpp_type2name (enum c extern cppchar_t cpp_parse_escape (cpp_reader *, const unsigned char ** pstr, const unsigned char *limit, int wide); +/* Structure used to hold a comment block at a given location in the + source code. */ + +typedef struct +{ + /* Text of the comment including the terminators. */ + char *comment; + + /* source location for the given comment. */ + source_location sloc; +} cpp_comment; + +/* Structure holding all comments for a given cpp_reader. */ + +typedef struct +{ + /* table of comment entries. */ + cpp_comment *entries; + + /* number of actual entries entered in the table. */ + int count; + + /* number of entries allocated currently. */ + int allocated; +} cpp_comment_table; + +/* Returns the table of comments encountered by the preprocessor. This + table is only populated when pfile->state.save_comments is true. */ +extern cpp_comment_table *cpp_get_comments (cpp_reader *); + /* In hash.c */ /* Lookup an identifier in the hashtable. Puts the identifier in the Index: init.c =================================================================== --- init.c (revision 140660) +++ init.c (working copy) @@ -245,6 +245,7 @@ cpp_destroy (cpp_reader *pfile) { cpp_context *context, *contextn; tokenrun *run, *runn; + int i; free (pfile->op_stack); @@ -287,6 +288,14 @@ cpp_destroy (cpp_reader *pfile) free (context); } + if (pfile->comments.entries) + { + for (i = 0; i < pfile->comments.count; i++) + free (pfile->comments.entries[i].comment); + + free (pfile->comments.entries); + } + free (pfile); } Index: internal.h =================================================================== --- internal.h (revision 140660) +++ internal.h (working copy) @@ -471,6 +471,9 @@ struct cpp_reader /* Next value of __COUNTER__ macro. */ unsigned int counter; + + /* Table of comments, when state.save_comments is true. */ + cpp_comment_table comments; }; /* Character classes. Based on the more primitive macros in safe-ctype.h. Index: lex.c =================================================================== --- lex.c (revision 140660) +++ lex.c (working copy) @@ -55,6 +55,7 @@ static int skip_line_comment (cpp_reader static void skip_whitespace (cpp_reader *, cppchar_t); static void lex_string (cpp_reader *, cpp_token *, const uchar *); static void save_comment (cpp_reader *, cpp_token *, const uchar *, cppchar_t); +static void store_comment (cpp_reader *, cpp_token *); static void create_literal (cpp_reader *, cpp_token *, const uchar *, unsigned int, enum cpp_ttype); static bool warn_in_comment (cpp_reader *, _cpp_line_note *); @@ -670,6 +671,51 @@ lex_string (cpp_reader *pfile, cpp_token create_literal (pfile, token, base, cur - base, type); } +/* Return the comment table. The client may not make any assumption + about the ordering of the table. */ +cpp_comment_table * +cpp_get_comments (cpp_reader *pfile) +{ + return &pfile->comments; +} + +/* Append a comment to the end of the comment table. */ +static void +store_comment (cpp_reader *pfile, cpp_token *token) +{ + int len; + + if (pfile->comments.allocated == 0) + { + pfile->comments.allocated = 256; + pfile->comments.entries = (cpp_comment *) xmalloc + (pfile->comments.allocated * sizeof (cpp_comment)); + } + + if (pfile->comments.count == pfile->comments.allocated) + { + pfile->comments.allocated *= 2; + pfile->comments.entries = (cpp_comment *) xrealloc + (pfile->comments.entries, + pfile->comments.allocated * sizeof (cpp_comment)); + } + + len = token->val.str.len; + + /* Copy comment. Note, token may not be NULL terminated. */ + pfile->comments.entries[pfile->comments.count].comment = + (char *) xmalloc (sizeof (char) * (len + 1)); + memcpy (pfile->comments.entries[pfile->comments.count].comment, + token->val.str.text, len); + pfile->comments.entries[pfile->comments.count].comment[len] = '\0'; + + /* Set source location. */ + pfile->comments.entries[pfile->comments.count].sloc = token->src_loc; + + /* Increment the count of entries in the comment table. */ + pfile->comments.count++; +} + /* The stored comment includes the comment start and any terminator. */ static void save_comment (cpp_reader *pfile, cpp_token *token, const unsigned char *from, @@ -709,6 +755,9 @@ save_comment (cpp_reader *pfile, cpp_tok buffer[clen - 2] = '*'; buffer[clen - 1] = '/'; } + + /* Finally store this comment for use by clients of libcpp. */ + store_comment (pfile, token); } /* Allocate COUNT tokens for RUN. */ ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2008-10-05 12:38 UTC | newest] Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2008-09-26 15:23 [libcpp] RFA: Add support for comments retrieval Arnaud Charlet 2008-09-26 15:44 ` Basile STARYNKEVITCH 2008-09-26 16:07 ` Arnaud Charlet 2008-09-26 16:07 ` Tom Tromey 2008-09-26 16:54 ` Arnaud Charlet 2008-09-26 18:21 ` Paolo Bonzini 2008-09-26 21:14 ` Arnaud Charlet 2008-09-26 21:17 ` Arnaud Charlet 2008-10-03 22:06 ` Tom Tromey 2008-10-05 13:19 ` Arnaud Charlet
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).