* Profiling compilation time @ 2009-03-24 9:18 Yang Zhang 2009-03-24 14:18 ` Ian Lance Taylor 0 siblings, 1 reply; 10+ messages in thread From: Yang Zhang @ 2009-03-24 9:18 UTC (permalink / raw) To: GCC-help Hi, are there any tools or hooks available that would help with profiling compilation time within g++? I'd like to gain a better understanding of where the time goes, primarily on two dimensions: - Which files does the compiler spend most of its time working with? - What stages of compilation are most expensive? Thanks! -- Yang Zhang http://www.mit.edu/~y_z/ ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Profiling compilation time 2009-03-24 9:18 Profiling compilation time Yang Zhang @ 2009-03-24 14:18 ` Ian Lance Taylor 2009-03-24 17:17 ` Yang Zhang 0 siblings, 1 reply; 10+ messages in thread From: Ian Lance Taylor @ 2009-03-24 14:18 UTC (permalink / raw) To: Yang Zhang; +Cc: GCC-help Yang Zhang <yanghatespam@gmail.com> writes: > Hi, are there any tools or hooks available that would help with > profiling compilation time within g++? I'd like to gain a better > understanding of where the time goes, primarily on two dimensions: > > - Which files does the compiler spend most of its time working with? > > - What stages of compilation are most expensive? Use the -ftime-report option. Ian ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Profiling compilation time 2009-03-24 14:18 ` Ian Lance Taylor @ 2009-03-24 17:17 ` Yang Zhang 2009-03-24 17:23 ` Ian Lance Taylor 0 siblings, 1 reply; 10+ messages in thread From: Yang Zhang @ 2009-03-24 17:17 UTC (permalink / raw) To: Ian Lance Taylor; +Cc: GCC-help Ian Lance Taylor wrote: > Yang Zhang <yanghatespam@gmail.com> writes: > >> Hi, are there any tools or hooks available that would help with >> profiling compilation time within g++? I'd like to gain a better >> understanding of where the time goes, primarily on two dimensions: >> >> - Which files does the compiler spend most of its time working with? >> >> - What stages of compilation are most expensive? > > Use the -ftime-report option. > > Ian Neat! Is there something similar for breaking down the time into time spent on individual files so that I can see which #included files are the most costly? (If you treat the compiler as a template language interpreter then you can even imagine getting call-graph profiling results.) -- Yang Zhang http://www.mit.edu/~y_z/ ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Profiling compilation time 2009-03-24 17:17 ` Yang Zhang @ 2009-03-24 17:23 ` Ian Lance Taylor 2009-03-24 19:27 ` Yang Zhang 0 siblings, 1 reply; 10+ messages in thread From: Ian Lance Taylor @ 2009-03-24 17:23 UTC (permalink / raw) To: Yang Zhang; +Cc: GCC-help Yang Zhang <yanghatespam@gmail.com> writes: > Ian Lance Taylor wrote: >> Use the -ftime-report option. > > Neat! Is there something similar for breaking down the time into time > spent on individual files so that I can see which #included files are > the most costly? (If you treat the compiler as a template language > interpreter then you can even imagine getting call-graph profiling > results.) That's a good idea. Unfortunately, I don't think there is anything like that at present. It would be hard, and perhaps meaningless, to do that for the IPA passes, but it could be done for the frontends and for the general optimization passes (where I suppose the time for inlined functions would go the function into which they are inlined.) Ian ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Profiling compilation time 2009-03-24 17:23 ` Ian Lance Taylor @ 2009-03-24 19:27 ` Yang Zhang 2009-03-24 22:32 ` Ian Lance Taylor 0 siblings, 1 reply; 10+ messages in thread From: Yang Zhang @ 2009-03-24 19:27 UTC (permalink / raw) To: Ian Lance Taylor; +Cc: GCC-help Ian Lance Taylor wrote: > Yang Zhang <yanghatespam@gmail.com> writes: > >> Ian Lance Taylor wrote: >>> Use the -ftime-report option. >> Neat! Is there something similar for breaking down the time into time >> spent on individual files so that I can see which #included files are >> the most costly? (If you treat the compiler as a template language >> interpreter then you can even imagine getting call-graph profiling >> results.) > > That's a good idea. Unfortunately, I don't think there is anything like > that at present. It would be hard, and perhaps meaningless, to do that > for the IPA passes, but it could be done for the frontends and for the > general optimization passes (where I suppose the time for inlined > functions would go the function into which they are inlined.) > > Ian For me, -ftime-report shows that compilation time is by far dominated by parsing, so that's what I'm most interested in. To that end, would a valid poor man's profiling approach be to simply measure the time of #including individual files into an otherwise empty source file? Or would it be necessary to actually "trigger" the parsing somehow by using the headers' contents (particularly for templated entities)? -- Yang Zhang http://www.mit.edu/~y_z/ ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Profiling compilation time 2009-03-24 19:27 ` Yang Zhang @ 2009-03-24 22:32 ` Ian Lance Taylor 2009-03-25 6:06 ` Yang Zhang 0 siblings, 1 reply; 10+ messages in thread From: Ian Lance Taylor @ 2009-03-24 22:32 UTC (permalink / raw) To: Yang Zhang; +Cc: GCC-help Yang Zhang <yanghatespam@gmail.com> writes: > For me, -ftime-report shows that compilation time is by far dominated > by parsing, so that's what I'm most interested in. To that end, would > a valid poor man's profiling approach be to simply measure the time of > #including individual files into an otherwise empty source file? Or > would it be necessary to actually "trigger" the parsing somehow by > using the headers' contents (particularly for templated entities)? In my experience, when not optimizing, C++ compilation time is normally dominated by parsing and name lookup. As far as I know, your approach should give you a reasonable first approximation. Template code is fully parsed even if the template is not instantiated. Of course there is more work to do if the template is instantiated, and that work is also counted against parsing time. If you are really interested, I think you could separate out template expansion time by adding a new entry to gcc/timevar.def and using that entry in instantiate_decl in gcc/cp/pt.c. Ian ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Profiling compilation time 2009-03-24 22:32 ` Ian Lance Taylor @ 2009-03-25 6:06 ` Yang Zhang 2009-03-25 6:27 ` Ian Lance Taylor 0 siblings, 1 reply; 10+ messages in thread From: Yang Zhang @ 2009-03-25 6:06 UTC (permalink / raw) To: Ian Lance Taylor; +Cc: GCC-help Ian Lance Taylor wrote: > In my experience, when not optimizing, C++ compilation time is normally > dominated by parsing and name lookup. From an earlier thread "Precompiled headers and templates:" Ian Lance Taylor wrote: > However, including explicit instantiations in the precompiled header > won't make any significant difference to compilation time. The > precompiled header saves on parsing and name lookup time, it doesn't > save on code generation time. > > I don't personally find that precompiled headers help with compilation > time all that much. However, for some projects, ones with millions of > lines of header files included in every compilation, I expect that they > would help. So PCHs only help with a very small fraction of the parsing + name lookup? (Already established that they wouldn't include template instantiation component of paths.) -- Yang Zhang http://www.mit.edu/~y_z/ ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Profiling compilation time 2009-03-25 6:06 ` Yang Zhang @ 2009-03-25 6:27 ` Ian Lance Taylor 2009-03-25 7:39 ` Yang Zhang 0 siblings, 1 reply; 10+ messages in thread From: Ian Lance Taylor @ 2009-03-25 6:27 UTC (permalink / raw) To: Yang Zhang; +Cc: GCC-help Yang Zhang <yanghatespam@gmail.com> writes: > Ian Lance Taylor wrote: >> In my experience, when not optimizing, C++ compilation time is normally >> dominated by parsing and name lookup. > > From an earlier thread "Precompiled headers and templates:" > > Ian Lance Taylor wrote: >> However, including explicit instantiations in the precompiled header >> won't make any significant difference to compilation time. The >> precompiled header saves on parsing and name lookup time, it doesn't >> save on code generation time. >> >> I don't personally find that precompiled headers help with compilation >> time all that much. However, for some projects, ones with millions of >> lines of header files included in every compilation, I expect that they >> would help. > > So PCHs only help with a very small fraction of the parsing + name > lookup? (Already established that they wouldn't include template > instantiation component of paths.) PCHs help with a large fraction of parsing and name lookup for those header files which are included in the PCH. The PCH implementation is limited in that you can only include a single PCH in a compilation, and it must be the thing which include first. The more of your header files that you can put into a single PCH, the more PCH will help you. However, for the large projects which I have worked on, there is no reasonable way to create a single large PCH which works across the project, because different parts of the code use different sets of header files, and it is not desirable to expose every part of the code to every other part of the code. As I said, they will help for projects in which million of lines of header files are included in every compilation. What I didn't quite manage to say was that it must be the exact same million lines in every compilation. Ian ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Profiling compilation time 2009-03-25 6:27 ` Ian Lance Taylor @ 2009-03-25 7:39 ` Yang Zhang 2009-03-25 13:54 ` Ian Lance Taylor 0 siblings, 1 reply; 10+ messages in thread From: Yang Zhang @ 2009-03-25 7:39 UTC (permalink / raw) To: Ian Lance Taylor; +Cc: GCC-help Ian Lance Taylor wrote: > Yang Zhang <yanghatespam@gmail.com> writes: > >> Ian Lance Taylor wrote: >>> In my experience, when not optimizing, C++ compilation time is normally >>> dominated by parsing and name lookup. >> From an earlier thread "Precompiled headers and templates:" >> >> Ian Lance Taylor wrote: >>> However, including explicit instantiations in the precompiled header >>> won't make any significant difference to compilation time. The >>> precompiled header saves on parsing and name lookup time, it doesn't >>> save on code generation time. >>> >>> I don't personally find that precompiled headers help with compilation >>> time all that much. However, for some projects, ones with millions of >>> lines of header files included in every compilation, I expect that they >>> would help. >> So PCHs only help with a very small fraction of the parsing + name >> lookup? (Already established that they wouldn't include template >> instantiation component of paths.) > > PCHs help with a large fraction of parsing and name lookup for those > header files which are included in the PCH. The PCH implementation is > limited in that you can only include a single PCH in a compilation, and > it must be the thing which include first. The more of your header files > that you can put into a single PCH, the more PCH will help you. > However, for the large projects which I have worked on, there is no > reasonable way to create a single large PCH which works across the > project, because different parts of the code use different sets of > header files, and it is not desirable to expose every part of the code > to every other part of the code. Is this because, once created, the cost of including a PCH scales with the PCH's total size, and not with the just the size of the parts used/consumed by the including source? The way I have been using PCHs is precisely the "single large PCH" approach: I put every system header used anywhere in the project into mypch.h, and I use "g++ -include mypch.h ..." (so that that's the first-included header). This *does* speed up compilation time for me, though I'd still like a better understanding of what the costs are. -- Yang Zhang http://www.mit.edu/~y_z/ ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Profiling compilation time 2009-03-25 7:39 ` Yang Zhang @ 2009-03-25 13:54 ` Ian Lance Taylor 0 siblings, 0 replies; 10+ messages in thread From: Ian Lance Taylor @ 2009-03-25 13:54 UTC (permalink / raw) To: Yang Zhang; +Cc: GCC-help Yang Zhang <yanghatespam@gmail.com> writes: >> PCHs help with a large fraction of parsing and name lookup for those >> header files which are included in the PCH. The PCH implementation is >> limited in that you can only include a single PCH in a compilation, and >> it must be the thing which include first. The more of your header files >> that you can put into a single PCH, the more PCH will help you. >> However, for the large projects which I have worked on, there is no >> reasonable way to create a single large PCH which works across the >> project, because different parts of the code use different sets of >> header files, and it is not desirable to expose every part of the code >> to every other part of the code. > > Is this because, once created, the cost of including a PCH scales with > the PCH's total size, and not with the just the size of the parts > used/consumed by the including source? No, the cost of including a PCH is more or less constant. It's because in a large project (hundreds of thousands of lines of code or more) subproject A should not see the internal header files of subproject B. Certainly it's possible to use separate PCHs for separate subprojects, but as you create more PCHs the returns diminish and the complexity increases. Ian ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2009-03-25 13:54 UTC | newest] Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2009-03-24 9:18 Profiling compilation time Yang Zhang 2009-03-24 14:18 ` Ian Lance Taylor 2009-03-24 17:17 ` Yang Zhang 2009-03-24 17:23 ` Ian Lance Taylor 2009-03-24 19:27 ` Yang Zhang 2009-03-24 22:32 ` Ian Lance Taylor 2009-03-25 6:06 ` Yang Zhang 2009-03-25 6:27 ` Ian Lance Taylor 2009-03-25 7:39 ` Yang Zhang 2009-03-25 13:54 ` Ian Lance Taylor
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).