* GCC internal re-architecture proposal @ 2013-06-21 15:27 Andrew MacLeod 2013-06-21 15:31 ` Andrew MacLeod ` (2 more replies) 0 siblings, 3 replies; 10+ messages in thread From: Andrew MacLeod @ 2013-06-21 15:27 UTC (permalink / raw) To: GCC [ I foolishly sent this with the document as an attachment... hopefully it gets rejected and anyone interested can simply download the document from the wiki..] Over the past couple of months, I've slowly been putting together an action plan to help modernize GCC's source base. We've had various ideas put forth over the years, and a few of the more meritorious ones have been incorporated. My primary goal is to disentangle the front end from the middle/back ends, giving us better control over their data structures. The biggest challenge is having an executable plan which allows the source to be updated incrementally. Ie, we need a way to get from 'Here' to 'There' that does not impact ongoing normal activities, nor noticeably affect the performance of the compiler. This is a huge effort and it will require a large time commitment on my part to see it through. I've attached a link to a document which outlines my proposal. Since putting it together, I've performed a couple of actual file conversions (albeit minor ones) beyond the simple examples I used in the document from the original proof of concept. I've been refining some of the actual implementation details and logged the various design decisions that I encounter for later discussions This document is a work in progress, and should be considered a strategic guide rather than an implementation bible. There will be a BOF at Cauldron for anyone interested in discussing aspects of this. I should also have a list of the interesting issues I've encountered for discussion there. Im on vacation for about half the time between now and Cauldron, so I wanted to get this out now. As for a time line, I wont be able to get seriously started on it for a couple of months since I have the C11 _Atomic and gimple-atomic implementations to get delivered. That gives us a reasonable amount of time to address concerns and sort out the overall approach as well other nitty gritty before any serious work commences. Once underway, I expect there would be plenty to do for anyone else interesting in volunteering some time. Here's the proposal document... happy reading! Feedback is welcome. I hope this link will work :-)... http://gcc.gnu.org/wiki/AndrewMacLeod?action=AttachFile&do=view&target=gccrestructureplan2.1.odt Highlander ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: GCC internal re-architecture proposal 2013-06-21 15:27 GCC internal re-architecture proposal Andrew MacLeod @ 2013-06-21 15:31 ` Andrew MacLeod 2013-06-21 16:16 ` Joseph S. Myers 2014-03-18 14:19 ` Richard Biener 2 siblings, 0 replies; 10+ messages in thread From: Andrew MacLeod @ 2013-06-21 15:31 UTC (permalink / raw) To: gcc On 06/21/2013 11:27 AM, Andrew MacLeod wrote: > > I hope this link will work :-)... > http://gcc.gnu.org/wiki/AndrewMacLeod?action=AttachFile&do=view&target=gccrestructureplan2.1.odt > > > Highlander or you can use this PDF file... http://gcc.gnu.org/wiki/AndrewMacLeod?action=AttachFile&do=view&target=gccrestructureplan2.1.pdf ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: GCC internal re-architecture proposal 2013-06-21 15:27 GCC internal re-architecture proposal Andrew MacLeod 2013-06-21 15:31 ` Andrew MacLeod @ 2013-06-21 16:16 ` Joseph S. Myers 2013-06-21 16:34 ` Andrew MacLeod 2013-06-21 17:48 ` Eric Botcazou 2014-03-18 14:19 ` Richard Biener 2 siblings, 2 replies; 10+ messages in thread From: Joseph S. Myers @ 2013-06-21 16:16 UTC (permalink / raw) To: Andrew MacLeod; +Cc: GCC A few observations: * You don't mention anything about separating host-side and target-side configuration, which are also entangled. A slightly out-of-date list of target macros used in target-side code is at <http://gcc.gnu.org/wiki/Top-Level_Libgcc_Migration> and a clean back-end class would involve eliminating those (so libgcc only includes libgcc_tm.h and not the host-side tm.h at all). (Various of the other things listed on that page were finished by Rainer, but not the target macros.) * I don't see the advantages of separating out the host information into a different API. Any particular compiler binary can only meaningfully run on one host, whereas in principle it should be possible to have multiple instances of a target class in one process so separating out the target interface does make sense. * It's not clear to me what's supposed to be different about the target_info class compared to the existing target structure. We "just" need to convert several hundred more target macros into hooks; that's the major work rather than any change from a global object to C++ classes or similar. (The transitional conversion whereby targhooks.c contains a hook definition that uses a target macro is, to my mind, of very limited value as an end-point; such conversions need finishing by eliminating all direct users of the macro, then converting all back ends to define the hook directly, then poisoning the macro and allowing and documenting only the hook.) Target macros used in front ends are indeed a good priority for converting to hooks. See previous discussions (and in some cases old patches by Joern) of how to convert some particular cases, e.g. INT_TYPE_SIZE or SIZE_TYPE (roughly, it seems to make sense to have hooks covering groups of related macros rather than one hook per macro in all cases). (Yes, a clean separation needs some new hook structures as well, e.g. in the driver, where the approach for defining the hooks would probably be different from the present one given how they tend to depend on the target OS as much as the target architecture.) * Quite a lot of target interfaces would be used by the front ends specifically for defining macros for building target libraries only; see the "probably predefine macros if -fbuilding-libgcc in most cases" list on that wiki page. (Thus, given a clean separation of target-side and host-side configuration, c-cppbuiltin.c is inevitably going to use a lot of target macros / hooks.) * Various cases of front-end folding are to get better warnings, e.g. to detect that in a signed-to-unsigned conversion the converted value is in fact never negative. I don't see this issue mentioned in the document. But maybe the GIMPLE generated by the front end should contain some form of note "give this warning if that condition is satisfied" and then the warnings would get given after some GIMPLE optimizations. * Option handling uses a single OPT_* enumeration with options from all parts of the compiler; for a full clean separation, you can no longer just have one array of all the options generated when GCC is built, and each bit of the compiler would need to see just a subset of the options. -- Joseph S. Myers joseph@codesourcery.com ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: GCC internal re-architecture proposal 2013-06-21 16:16 ` Joseph S. Myers @ 2013-06-21 16:34 ` Andrew MacLeod 2013-06-22 12:50 ` Tobias Burnus 2013-06-21 17:48 ` Eric Botcazou 1 sibling, 1 reply; 10+ messages in thread From: Andrew MacLeod @ 2013-06-21 16:34 UTC (permalink / raw) To: Joseph S. Myers; +Cc: GCC On 06/21/2013 12:16 PM, Joseph S. Myers wrote: > A few observations: > > * You don't mention anything about separating host-side and target-side > configuration, which are also entangled. A slightly out-of-date list of > target macros used in target-side code is at > <http://gcc.gnu.org/wiki/Top-Level_Libgcc_Migration> and a clean back-end > class would involve eliminating those (so libgcc only includes libgcc_tm.h > and not the host-side tm.h at all). (Various of the other things listed > on that page were finished by Rainer, but not the target macros.) yeah, I didn't go into any configuration stuff... I don't understand it well enough (and avoid it like the plague :-) so that will require some outside assistance :-) Thats something that can be flushed out better in the future for sure. > > * I don't see the advantages of separating out the host information into a > different API. Any particular compiler binary can only meaningfully run > on one host, whereas in principle it should be possible to have multiple > instances of a target class in one process so separating out the target > interface does make sense. That has been brought up, and It may not be worth any effort at all. I added it more for completeness and consideration for getting that information "all in one place". It may well come to nothing. > > * It's not clear to me what's supposed to be different about the > target_info class compared to the existing target structure. We "just" There may not be any difference. The term target_info was just taken out of my hat. It may well be that we just extend the existing structure. It just need to be flushed out fully and completed. Im not terribly familiar with those bits, so i didn't give it much detail :-) > need to convert several hundred more target macros into hooks; that's the > major work rather than any change from a global object to C++ classes or > similar. (The transitional conversion whereby targhooks.c contains a hook > definition that uses a target macro is, to my mind, of very limited value > as an end-point; such conversions need finishing by eliminating all direct > users of the macro, then converting all back ends to define the hook > directly, then poisoning the macro and allowing and documenting only the > hook.) I think that sounds like a most reasonable way to proceed. > > Target macros used in front ends are indeed a good priority for converting > to hooks. See previous discussions (and in some cases old patches by > Joern) of how to convert some particular cases, e.g. INT_TYPE_SIZE or > SIZE_TYPE (roughly, it seems to make sense to have hooks covering groups > of related macros rather than one hook per macro in all cases). indeed, thats exactly the sort of low-hanging simplifications I was thinking about, and I'm sure there will be others > > > > * Quite a lot of target interfaces would be used by the front ends > specifically for defining macros for building target libraries only; see > the "probably predefine macros if -fbuilding-libgcc in most cases" list on > that wiki page. (Thus, given a clean separation of target-side and > host-side configuration, c-cppbuiltin.c is inevitably going to use a lot > of target macros / hooks.) If thats where most of it lies, maybe there is some way to "push" some of that activity into the backend?... I'll investigate c-cppbuiltin.c at some point when I get some time. "Heavy" consumers like that would show up after the initial conversion stages and we can consider how to improve it. > > * Various cases of front-end folding are to get better warnings, e.g. to > detect that in a signed-to-unsigned conversion the converted value is in > fact never negative. I don't see this issue mentioned in the document. > But maybe the GIMPLE generated by the front end should contain some form > of note "give this warning if that condition is satisfied" and then the > warnings would get given after some GIMPLE optimizations. Indeed, a previous incarnation of the document took that idea to further extremes, but that also sounds highly reasonable. The gimple fodler may be able to issue those sorts of warnings as it "optimizes" by folding. I'll add something back in about that. > > * Option handling uses a single OPT_* enumeration with options from all > parts of the compiler; for a full clean separation, you can no longer just > have one array of all the options generated when GCC is built, and each > bit of the compiler would need to see just a subset of the options. > Good observation, I'll add something about that too. Thanks! Andrew ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: GCC internal re-architecture proposal 2013-06-21 16:34 ` Andrew MacLeod @ 2013-06-22 12:50 ` Tobias Burnus 0 siblings, 0 replies; 10+ messages in thread From: Tobias Burnus @ 2013-06-22 12:50 UTC (permalink / raw) To: GCC Mailing List, Andrew MacLeod Andrew MacLeod wrote: >> * Quite a lot of target interfaces would be used by the front ends >> specifically for defining macros for building target libraries only; see >> the "probably predefine macros if -fbuilding-libgcc in most cases" >> list on >> that wiki page. (Thus, given a clean separation of target-side and >> host-side configuration, c-cppbuiltin.c is inevitably going to use a lot >> of target macros / hooks.) > If thats where most of it lies, maybe there is some way to "push" some > of that activity into the backend?... I'll investigate c-cppbuiltin.c > at some point when I get some time. "Heavy" consumers like that would > show up after the initial conversion stages and we can consider how to > improve it. When looking at CPP, please don't forget that also gfortran uses CPP (libcpp). (Currently, gfortran does not provide all target macro definitions as gcc/config is too C/C++ centric; the idea was to move to hooks and then to change the definitions into a generic and C/C++ only part - such that the generic part can be also used for Fortran. See PR42954.) Tobias ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: GCC internal re-architecture proposal 2013-06-21 16:16 ` Joseph S. Myers 2013-06-21 16:34 ` Andrew MacLeod @ 2013-06-21 17:48 ` Eric Botcazou 1 sibling, 0 replies; 10+ messages in thread From: Eric Botcazou @ 2013-06-21 17:48 UTC (permalink / raw) To: Joseph S. Myers; +Cc: gcc, Andrew MacLeod > * Various cases of front-end folding are to get better warnings, e.g. to > detect that in a signed-to-unsigned conversion the converted value is in > fact never negative. I don't see this issue mentioned in the document. Tree front-end folding, or more precisely folding of expressions generated for types, is required when types with variable size are first-class citizens like in Ada. In practice, something even more powerful is required, i.e. outlining (called from stor-layout.c:variable_size) and back-inlining of the expressions (tree-inline.c:maybe_inline_call_in_expr), if you want to be able to compile big codebases with a reasonable amount of resources. -- Eric Botcazou ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: GCC internal re-architecture proposal 2013-06-21 15:27 GCC internal re-architecture proposal Andrew MacLeod 2013-06-21 15:31 ` Andrew MacLeod 2013-06-21 16:16 ` Joseph S. Myers @ 2014-03-18 14:19 ` Richard Biener 2014-03-18 16:34 ` Diego Novillo 2 siblings, 1 reply; 10+ messages in thread From: Richard Biener @ 2014-03-18 14:19 UTC (permalink / raw) To: Andrew MacLeod; +Cc: GCC On Fri, Jun 21, 2013 at 5:27 PM, Andrew MacLeod <amacleod@redhat.com> wrote: > [ I foolishly sent this with the document as an attachment... hopefully it > gets rejected and anyone interested can simply download the document from > the wiki..] > > Over the past couple of months, I've slowly been putting together an action > plan to help modernize GCC's source base. We've had various ideas put forth > over the years, and a few of the more meritorious ones have been > incorporated. My primary goal is to disentangle the front end from the > middle/back ends, giving us better control over their data structures. > > The biggest challenge is having an executable plan which allows the source > to be updated incrementally. Ie, we need a way to get from 'Here' to > 'There' that does not impact ongoing normal activities, nor noticeably > affect the performance of the compiler. This is a huge effort and it will > require a large time commitment on my part to see it through. Sorry for following up on this old e-mail but it just occured to me that the (unfinished) GIMPLE frontend would be a great place to bootstrap the new and shiny GIMPLE data structures. Of course for it to work you need to translate that back to fat-and-old GIMPLE and eventually finish the textual format dumping as well - not sure what the state of the gimple frontend branch is. Richard. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: GCC internal re-architecture proposal 2014-03-18 14:19 ` Richard Biener @ 2014-03-18 16:34 ` Diego Novillo 2014-03-19 9:08 ` Richard Biener 0 siblings, 1 reply; 10+ messages in thread From: Diego Novillo @ 2014-03-18 16:34 UTC (permalink / raw) To: Richard Biener; +Cc: Andrew MacLeod, GCC On Tue, Mar 18, 2014 at 10:19 AM, Richard Biener <richard.guenther@gmail.com> wrote: > On Fri, Jun 21, 2013 at 5:27 PM, Andrew MacLeod <amacleod@redhat.com> wrote: >> [ I foolishly sent this with the document as an attachment... hopefully it >> gets rejected and anyone interested can simply download the document from >> the wiki..] >> >> Over the past couple of months, I've slowly been putting together an action >> plan to help modernize GCC's source base. We've had various ideas put forth >> over the years, and a few of the more meritorious ones have been >> incorporated. My primary goal is to disentangle the front end from the >> middle/back ends, giving us better control over their data structures. >> >> The biggest challenge is having an executable plan which allows the source >> to be updated incrementally. Ie, we need a way to get from 'Here' to >> 'There' that does not impact ongoing normal activities, nor noticeably >> affect the performance of the compiler. This is a huge effort and it will >> require a large time commitment on my part to see it through. > > Sorry for following up on this old e-mail but it just occured to me that > the (unfinished) GIMPLE frontend would be a great place to bootstrap > the new and shiny GIMPLE data structures. > > Of course for it to work you need to translate that back to fat-and-old GIMPLE > and eventually finish the textual format dumping as well - not sure what > the state of the gimple frontend branch is. Work on it stalled sometime last year. I'm not sure if I would mix the re-arch work with the GIMPLE FE work. The parser will take the most benefit out of the new structure, but I think it may be better to make the front end be a sub-branch of the re-architect work so we don't conflate the two. Some work will be independent, but it will be easier to maintain both branches if they are separate. I do like the idea of tying the GIMPLE front end to the re-arch branch, however. So, making it a sub-branch of re-arch is definitely a good idea. Diego. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: GCC internal re-architecture proposal 2014-03-18 16:34 ` Diego Novillo @ 2014-03-19 9:08 ` Richard Biener 2014-03-19 13:09 ` Andrew MacLeod 0 siblings, 1 reply; 10+ messages in thread From: Richard Biener @ 2014-03-19 9:08 UTC (permalink / raw) To: Diego Novillo; +Cc: Andrew MacLeod, GCC On Tue, Mar 18, 2014 at 5:34 PM, Diego Novillo <dnovillo@google.com> wrote: > On Tue, Mar 18, 2014 at 10:19 AM, Richard Biener > <richard.guenther@gmail.com> wrote: >> On Fri, Jun 21, 2013 at 5:27 PM, Andrew MacLeod <amacleod@redhat.com> wrote: >>> [ I foolishly sent this with the document as an attachment... hopefully it >>> gets rejected and anyone interested can simply download the document from >>> the wiki..] >>> >>> Over the past couple of months, I've slowly been putting together an action >>> plan to help modernize GCC's source base. We've had various ideas put forth >>> over the years, and a few of the more meritorious ones have been >>> incorporated. My primary goal is to disentangle the front end from the >>> middle/back ends, giving us better control over their data structures. >>> >>> The biggest challenge is having an executable plan which allows the source >>> to be updated incrementally. Ie, we need a way to get from 'Here' to >>> 'There' that does not impact ongoing normal activities, nor noticeably >>> affect the performance of the compiler. This is a huge effort and it will >>> require a large time commitment on my part to see it through. >> >> Sorry for following up on this old e-mail but it just occured to me that >> the (unfinished) GIMPLE frontend would be a great place to bootstrap >> the new and shiny GIMPLE data structures. >> >> Of course for it to work you need to translate that back to fat-and-old GIMPLE >> and eventually finish the textual format dumping as well - not sure what >> the state of the gimple frontend branch is. > > Work on it stalled sometime last year. > > I'm not sure if I would mix the re-arch work with the GIMPLE FE work. > The parser will take the most benefit out of the new structure, but I > think it may be better to make the front end be a sub-branch of the > re-architect work so we don't conflate the two. > > Some work will be independent, but it will be easier to maintain both > branches if they are separate. > > I do like the idea of tying the GIMPLE front end to the re-arch > branch, however. So, making it a sub-branch of re-arch is definitely a > good idea. Well, I thought that the GIMPLE FE itself is the easiest place where you can try out things given that it has to represent all features of GIMPLE but doesn't (necessarily) use the GIMPLE IL as its AST. Especially the no-trees part should be easy there. And we can be convinced of the beauty of the new AST before munging all these wrappers into trunk with the fear that only the wrappers will stay and the project itself will never finish ... thus we get the ugliness while with the GIMPLE FE we'd at least get the GIMPLE FE ... Richard. > > Diego. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: GCC internal re-architecture proposal 2014-03-19 9:08 ` Richard Biener @ 2014-03-19 13:09 ` Andrew MacLeod 0 siblings, 0 replies; 10+ messages in thread From: Andrew MacLeod @ 2014-03-19 13:09 UTC (permalink / raw) To: Richard Biener, Diego Novillo; +Cc: GCC On 03/19/2014 05:08 AM, Richard Biener wrote: > On Tue, Mar 18, 2014 at 5:34 PM, Diego Novillo <dnovillo@google.com> wrote: >> On Tue, Mar 18, 2014 at 10:19 AM, Richard Biener >> <richard.guenther@gmail.com> wrote: >>> On Fri, Jun 21, 2013 at 5:27 PM, Andrew MacLeod <amacleod@redhat.com> wrote: >>>> [ I foolishly sent this with the document as an attachment... hopefully it >>>> gets rejected and anyone interested can simply download the document from >>>> the wiki..] >>>> >>>> Over the past couple of months, I've slowly been putting together an action >>>> plan to help modernize GCC's source base. We've had various ideas put forth >>>> over the years, and a few of the more meritorious ones have been >>>> incorporated. My primary goal is to disentangle the front end from the >>>> middle/back ends, giving us better control over their data structures. >>>> >>>> The biggest challenge is having an executable plan which allows the source >>>> to be updated incrementally. Ie, we need a way to get from 'Here' to >>>> 'There' that does not impact ongoing normal activities, nor noticeably >>>> affect the performance of the compiler. This is a huge effort and it will >>>> require a large time commitment on my part to see it through. >>> Sorry for following up on this old e-mail but it just occured to me that >>> the (unfinished) GIMPLE frontend would be a great place to bootstrap >>> the new and shiny GIMPLE data structures. >>> >>> Of course for it to work you need to translate that back to fat-and-old GIMPLE >>> and eventually finish the textual format dumping as well - not sure what >>> the state of the gimple frontend branch is. >> Work on it stalled sometime last year. >> >> I'm not sure if I would mix the re-arch work with the GIMPLE FE work. >> The parser will take the most benefit out of the new structure, but I >> think it may be better to make the front end be a sub-branch of the >> re-architect work so we don't conflate the two. >> >> Some work will be independent, but it will be easier to maintain both >> branches if they are separate. >> >> I do like the idea of tying the GIMPLE front end to the re-arch >> branch, however. So, making it a sub-branch of re-arch is definitely a >> good idea. > Well, I thought that the GIMPLE FE itself is the easiest place where > you can try out things given that it has to represent all features > of GIMPLE but doesn't (necessarily) use the GIMPLE IL as its AST. > Especially the no-trees part should be easy there. And we can be > convinced of the beauty of the new AST before munging all these > wrappers into trunk with the fear that only the wrappers will stay > and the project itself will never finish ... thus we get the ugliness > while with the GIMPLE FE we'd at least get the GIMPLE FE ... > > Well, there are no new ASTs...The gimple IL is still the same, except we'll have an opportunity to better formalize it, and it will be statically type-safe rather than dynamic like today. Its not really practical to maintain it on a branch all the way until a completely non-tree front end can be produced. There are too many "unrelated" issues to progress from where we are today to a separate front end. That said, It has become necessary to start separating out the front end interface components shortly as part of the work. This can use the gimple FE as a litmus test. Maybe we can drum up some interest and finish development of it. In any case, no decisions have to be made yet, and I'm not planning to rush it. I'm still refining things and it'll be a couple of months before its worth discussing any sort of integration plan or options. I plan to put together some documentation during that time and we can discuss its merits and fate when that is ready. I figure we can hash things out during the month leading up to Cauldron. Andrew ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2014-03-19 13:09 UTC | newest] Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2013-06-21 15:27 GCC internal re-architecture proposal Andrew MacLeod 2013-06-21 15:31 ` Andrew MacLeod 2013-06-21 16:16 ` Joseph S. Myers 2013-06-21 16:34 ` Andrew MacLeod 2013-06-22 12:50 ` Tobias Burnus 2013-06-21 17:48 ` Eric Botcazou 2014-03-18 14:19 ` Richard Biener 2014-03-18 16:34 ` Diego Novillo 2014-03-19 9:08 ` Richard Biener 2014-03-19 13:09 ` Andrew MacLeod
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).