* Announce - Thread safety annotations no longer supported in GCC [not found] <CAFOgFcSO9NRXprgW+LxkcOCjttzGZiqyugwbLPbrLpQUqMkmHA@mail.gmail.com> @ 2012-04-19 14:15 ` Diego Novillo 2012-04-19 15:15 ` Andrew Pinski 0 siblings, 1 reply; 13+ messages in thread From: Diego Novillo @ 2012-04-19 14:15 UTC (permalink / raw) To: gcc, Delesley Hutchins, Ollie Wild, Le-Chun Wu We have decided to terminate the thread safety annotation project in GCC. The current implementation is in the branch google/main for those interested in using it. We will not be pursuing a merge into trunk. Instead, we have started implementing the same functionality in Clang. I've updated the wiki page and moved the branch out of the active development branches in svn.html. Diego. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Announce - Thread safety annotations no longer supported in GCC 2012-04-19 14:15 ` Announce - Thread safety annotations no longer supported in GCC Diego Novillo @ 2012-04-19 15:15 ` Andrew Pinski 2012-04-19 15:44 ` Delesley Hutchins 0 siblings, 1 reply; 13+ messages in thread From: Andrew Pinski @ 2012-04-19 15:15 UTC (permalink / raw) To: Diego Novillo; +Cc: gcc, Delesley Hutchins, Ollie Wild, Le-Chun Wu On Thu, Apr 19, 2012 at 7:15 AM, Diego Novillo <dnovillo@google.com> wrote: > > We have decided to terminate the thread safety annotation project in > GCC. > > The current implementation is in the branch google/main for those > interested in using it. We will not be pursuing a merge into trunk. > Instead, we have started implementing the same functionality in Clang. What went into making this decision? I know lot of folks will almost never go over to using clang since it means supporting one extra front-end. I am thinking of the embedded folks here where they cannot afford supporting something as new as clang for their customers. Thanks, Andrew Pinski > > I've updated the wiki page and moved the branch out of the active > development branches in svn.html. > > > Diego. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Announce - Thread safety annotations no longer supported in GCC 2012-04-19 15:15 ` Andrew Pinski @ 2012-04-19 15:44 ` Delesley Hutchins 2012-04-19 16:25 ` Andrew Pinski 0 siblings, 1 reply; 13+ messages in thread From: Delesley Hutchins @ 2012-04-19 15:44 UTC (permalink / raw) To: Andrew Pinski; +Cc: Diego Novillo, gcc, Ollie Wild, Le-Chun Wu The gcc version has been difficult to support and maintain, due mainly to the fact that the GIMPLE intermediate language was never designed for static analysis. The abstract syntax tree provided by Clang is an easier data structure to work with for front-end analyses of this kind. Moreover, the gcc implementation of annotalysis has some issues that make an eventual merge into trunk somewhat unlikely, and annotalysis is of little use to people outside of google as long as it stays in google/main. The clang implementation has been in trunk from the beginning. Hope that explains it a bit better, -DeLesley On Thu, Apr 19, 2012 at 8:15 AM, Andrew Pinski <pinskia@gmail.com> wrote: > On Thu, Apr 19, 2012 at 7:15 AM, Diego Novillo <dnovillo@google.com> wrote: >> >> We have decided to terminate the thread safety annotation project in >> GCC. >> >> The current implementation is in the branch google/main for those >> interested in using it. We will not be pursuing a merge into trunk. >> Instead, we have started implementing the same functionality in Clang. > > What went into making this decision? I know lot of folks will almost > never go over to using clang since it means supporting one extra > front-end. I am thinking of the embedded folks here where they cannot > afford supporting something as new as clang for their customers. > > Thanks, > Andrew Pinski > >> >> I've updated the wiki page and moved the branch out of the active >> development branches in svn.html. >> >> >> Diego. -- DeLesley Hutchins | Software Engineer | delesley@google.com | 505-206-0315 ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Announce - Thread safety annotations no longer supported in GCC 2012-04-19 15:44 ` Delesley Hutchins @ 2012-04-19 16:25 ` Andrew Pinski 2012-04-19 17:30 ` Delesley Hutchins 0 siblings, 1 reply; 13+ messages in thread From: Andrew Pinski @ 2012-04-19 16:25 UTC (permalink / raw) To: Delesley Hutchins; +Cc: Diego Novillo, gcc, Ollie Wild, Le-Chun Wu On Thu, Apr 19, 2012 at 8:44 AM, Delesley Hutchins <delesley@google.com> wrote: > The gcc version has been difficult to support and maintain, due mainly > to the fact that the GIMPLE intermediate language was never designed > for static analysis. The abstract syntax tree provided by Clang is an > easier data structure to work with for front-end analyses of this > kind. Moreover, the gcc implementation of annotalysis has some issues > that make an eventual merge into trunk somewhat unlikely, and > annotalysis is of little use to people outside of google as long as it > stays in google/main. The clang implementation has been in trunk from > the beginning. > > Hope that explains it a bit better, No, that it does not help at all. This seems like a high level issue of the problem rather than describing the reasons why GIMPLE will never work correctly for your usage. Maybe we can expand it for your usage but we need to better understand what it is lacking. Thanks, Andrew Pinski > > -DeLesley > > On Thu, Apr 19, 2012 at 8:15 AM, Andrew Pinski <pinskia@gmail.com> wrote: >> On Thu, Apr 19, 2012 at 7:15 AM, Diego Novillo <dnovillo@google.com> wrote: >>> >>> We have decided to terminate the thread safety annotation project in >>> GCC. >>> >>> The current implementation is in the branch google/main for those >>> interested in using it. We will not be pursuing a merge into trunk. >>> Instead, we have started implementing the same functionality in Clang. >> >> What went into making this decision? I know lot of folks will almost >> never go over to using clang since it means supporting one extra >> front-end. I am thinking of the embedded folks here where they cannot >> afford supporting something as new as clang for their customers. >> >> Thanks, >> Andrew Pinski >> >>> >>> I've updated the wiki page and moved the branch out of the active >>> development branches in svn.html. >>> >>> >>> Diego. > > > > -- > DeLesley Hutchins | Software Engineer | delesley@google.com | 505-206-0315 ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Announce - Thread safety annotations no longer supported in GCC 2012-04-19 16:25 ` Andrew Pinski @ 2012-04-19 17:30 ` Delesley Hutchins 2012-04-19 19:15 ` Andrew Pinski 2012-04-20 2:31 ` Miles Bader 0 siblings, 2 replies; 13+ messages in thread From: Delesley Hutchins @ 2012-04-19 17:30 UTC (permalink / raw) To: Andrew Pinski; +Cc: Diego Novillo, gcc, Ollie Wild, Le-Chun Wu I can give you detailed technical reasons why GIMPLE was not working for us if you like, but I'm not sure it would be all that constructive. We have already made the decision to switch to clang for annotalysis users within google, for reasons that are only partly technical. The only reason to support the gcc version would be if there was sufficient interest in annotalysis outside of google to warrant the effort of moving it to trunk. Given that the annotalysis branch stopped tracking trunk almost a year ago, and has been disabled in even in google/main for the past 6 months, I would be surprised to find any such users. Since the estimated number of users is currently zero, there seems little point in maintaining the software. Moreover, although I appreciate your offer to try and expand GIMPLE, I don't think it makes a lot of sense. GIMPLE works just fine for its intended use case, which is an intermediate language for compiler optimization. Changing GIMPLE would be a major effort, and it would only be warranted if there were enough users to make it worthwhile. -DeLesley On Thu, Apr 19, 2012 at 9:25 AM, Andrew Pinski <pinskia@gmail.com> wrote: > On Thu, Apr 19, 2012 at 8:44 AM, Delesley Hutchins <delesley@google.com> wrote: >> The gcc version has been difficult to support and maintain, due mainly >> to the fact that the GIMPLE intermediate language was never designed >> for static analysis. The abstract syntax tree provided by Clang is an >> easier data structure to work with for front-end analyses of this >> kind. Moreover, the gcc implementation of annotalysis has some issues >> that make an eventual merge into trunk somewhat unlikely, and >> annotalysis is of little use to people outside of google as long as it >> stays in google/main. The clang implementation has been in trunk from >> the beginning. >> >> Hope that explains it a bit better, > > No, that it does not help at all. This seems like a high level issue > of the problem rather than describing the reasons why GIMPLE will > never work correctly for your usage. Maybe we can expand it for your > usage but we need to better understand what it is lacking. > > Thanks, > Andrew Pinski > > >> >> -DeLesley >> >> On Thu, Apr 19, 2012 at 8:15 AM, Andrew Pinski <pinskia@gmail.com> wrote: >>> On Thu, Apr 19, 2012 at 7:15 AM, Diego Novillo <dnovillo@google.com> wrote: >>>> >>>> We have decided to terminate the thread safety annotation project in >>>> GCC. >>>> >>>> The current implementation is in the branch google/main for those >>>> interested in using it. We will not be pursuing a merge into trunk. >>>> Instead, we have started implementing the same functionality in Clang. >>> >>> What went into making this decision? I know lot of folks will almost >>> never go over to using clang since it means supporting one extra >>> front-end. I am thinking of the embedded folks here where they cannot >>> afford supporting something as new as clang for their customers. >>> >>> Thanks, >>> Andrew Pinski >>> >>>> >>>> I've updated the wiki page and moved the branch out of the active >>>> development branches in svn.html. >>>> >>>> >>>> Diego. >> >> >> >> -- >> DeLesley Hutchins | Software Engineer | delesley@google.com | 505-206-0315 -- DeLesley Hutchins | Software Engineer | delesley@google.com | 505-206-0315 ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Announce - Thread safety annotations no longer supported in GCC 2012-04-19 17:30 ` Delesley Hutchins @ 2012-04-19 19:15 ` Andrew Pinski 2012-04-19 20:20 ` Diego Novillo 2012-04-19 20:55 ` Delesley Hutchins 2012-04-20 2:31 ` Miles Bader 1 sibling, 2 replies; 13+ messages in thread From: Andrew Pinski @ 2012-04-19 19:15 UTC (permalink / raw) To: Delesley Hutchins; +Cc: Diego Novillo, gcc, Ollie Wild, Le-Chun Wu On Thu, Apr 19, 2012 at 10:30 AM, Delesley Hutchins <delesley@google.com> wrote: > I can give you detailed technical reasons why GIMPLE was not working > for us if you like, but I'm not sure it would be all that > constructive. We have already made the decision to switch to clang > for annotalysis users within google, for reasons that are only partly > technical. The only reason to support the gcc version would be if > there was sufficient interest in annotalysis outside of google to > warrant the effort of moving it to trunk. Given that the annotalysis > branch stopped tracking trunk almost a year ago, and has been disabled > in even in google/main for the past 6 months, I would be surprised to > find any such users. Since the estimated number of users is currently > zero, there seems little point in maintaining the software. > > Moreover, although I appreciate your offer to try and expand GIMPLE, I > don't think it makes a lot of sense. GIMPLE works just fine for its > intended use case, which is an intermediate language for compiler > optimization. Changing GIMPLE would be a major effort, and it would > only be warranted if there were enough users to make it worthwhile. How do you know it is a major effort? Has any issues related to changing Tuple/front-ends AST been raised to the mailing list and asked for help on how to implement these changes? This is why this decision is such a disappointment here because I have not seen one message about what needs to change to help support the future development of static analyzer in GCC. Yes people have said it can't be done fully currently because the AST but that does not mean asking the right question and answering how can we change GCC to allow for a better job of doing this kind of development. Being quiet about it and then saying it can't be done without any reasoning behind why moving away from GCC is seems a bit out of place. Thanks, Andrew Pinski > > -DeLesley > > On Thu, Apr 19, 2012 at 9:25 AM, Andrew Pinski <pinskia@gmail.com> wrote: >> On Thu, Apr 19, 2012 at 8:44 AM, Delesley Hutchins <delesley@google.com> wrote: >>> The gcc version has been difficult to support and maintain, due mainly >>> to the fact that the GIMPLE intermediate language was never designed >>> for static analysis. The abstract syntax tree provided by Clang is an >>> easier data structure to work with for front-end analyses of this >>> kind. Moreover, the gcc implementation of annotalysis has some issues >>> that make an eventual merge into trunk somewhat unlikely, and >>> annotalysis is of little use to people outside of google as long as it >>> stays in google/main. The clang implementation has been in trunk from >>> the beginning. >>> >>> Hope that explains it a bit better, >> >> No, that it does not help at all. This seems like a high level issue >> of the problem rather than describing the reasons why GIMPLE will >> never work correctly for your usage. Maybe we can expand it for your >> usage but we need to better understand what it is lacking. >> >> Thanks, >> Andrew Pinski >> >> >>> >>> -DeLesley >>> >>> On Thu, Apr 19, 2012 at 8:15 AM, Andrew Pinski <pinskia@gmail.com> wrote: >>>> On Thu, Apr 19, 2012 at 7:15 AM, Diego Novillo <dnovillo@google.com> wrote: >>>>> >>>>> We have decided to terminate the thread safety annotation project in >>>>> GCC. >>>>> >>>>> The current implementation is in the branch google/main for those >>>>> interested in using it. We will not be pursuing a merge into trunk. >>>>> Instead, we have started implementing the same functionality in Clang. >>>> >>>> What went into making this decision? I know lot of folks will almost >>>> never go over to using clang since it means supporting one extra >>>> front-end. I am thinking of the embedded folks here where they cannot >>>> afford supporting something as new as clang for their customers. >>>> >>>> Thanks, >>>> Andrew Pinski >>>> >>>>> >>>>> I've updated the wiki page and moved the branch out of the active >>>>> development branches in svn.html. >>>>> >>>>> >>>>> Diego. >>> >>> >>> >>> -- >>> DeLesley Hutchins | Software Engineer | delesley@google.com | 505-206-0315 > > > > -- > DeLesley Hutchins | Software Engineer | delesley@google.com | 505-206-0315 ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Announce - Thread safety annotations no longer supported in GCC 2012-04-19 19:15 ` Andrew Pinski @ 2012-04-19 20:20 ` Diego Novillo 2012-04-19 21:10 ` Delesley Hutchins 2012-04-20 8:50 ` Richard Guenther 2012-04-19 20:55 ` Delesley Hutchins 1 sibling, 2 replies; 13+ messages in thread From: Diego Novillo @ 2012-04-19 20:20 UTC (permalink / raw) To: Andrew Pinski; +Cc: Delesley Hutchins, gcc, Ollie Wild, Le-Chun Wu On 4/19/12 4:14 PM, Andrew Pinski wrote: > How do you know it is a major effort? Has any issues related to > changing Tuple/front-ends AST been raised to the mailing list and > asked for help on how to implement these changes? The kind of analysis that Annotalysis needs cannot be catered by GIMPLE, the same way that LLVM's bitcode could not cater to it. Both representations are geared towards code transformations, not source code analysis. It's not an implementation issue, but a design one. It simply does not make sense for GIMPLE or LLVM's bitcode to try to be a source code analysis framework. Annotalysis needs a high-fidelity representation of the original source code. Today, that high-fidelity representation is provided exclusively by Clang. Additionally, we are already supporting Clang as a front end to provide syntax and semantic analysis. Given that Clang provides a much more flexible framework for static analysis, the decision was a relatively simple one. This is not to say that Clang provides everything needed by Annotalysis. There is some need to use dataflow information which needs to be incorporated in Clang. However, a large fraction of the support required was already available in Clang. Diego. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Announce - Thread safety annotations no longer supported in GCC 2012-04-19 20:20 ` Diego Novillo @ 2012-04-19 21:10 ` Delesley Hutchins 2012-04-20 8:50 ` Richard Guenther 1 sibling, 0 replies; 13+ messages in thread From: Delesley Hutchins @ 2012-04-19 21:10 UTC (permalink / raw) To: Diego Novillo; +Cc: Andrew Pinski, gcc, Ollie Wild, Le-Chun Wu Diego is right. Clang maintains an accurate C++ AST, and then layers additional data structures, like the control flow graph, on top of that, so you don't get the same kind of lowering artifacts that have plagued the gcc implementation of annotalysis. That architecture makes Clang a much better platform for static analysis. However, I want to point out that Gimple is actually designed pretty well, especially considering the fact that it was never intended to be a source code analysis framework. Certain parts of our analysis were much easier to do in Gimple than in Clang, because we could leverage the work of previous transformations (e.g. lowering to SSA). However, those same transformations made some other parts of the analysis nearly impossible. There is no such thing as a perfect intermediate language that will satisfy everyone. -DeLesley considering that Gimple was never intended to be a source code analysis On Thu, Apr 19, 2012 at 1:20 PM, Diego Novillo <dnovillo@google.com> wrote: > On 4/19/12 4:14 PM, Andrew Pinski wrote: > >> How do you know it is a major effort? Has any issues related to >> changing Tuple/front-ends AST been raised to the mailing list and >> asked for help on how to implement these changes? > > > The kind of analysis that Annotalysis needs cannot be catered by GIMPLE, the > same way that LLVM's bitcode could not cater to it. Both representations > are geared towards code transformations, not source code analysis. It's not > an implementation issue, but a design one. It simply does not make sense > for GIMPLE or LLVM's bitcode to try to be a source code analysis framework. > > Annotalysis needs a high-fidelity representation of the original source > code. Today, that high-fidelity representation is provided exclusively by > Clang. > > Additionally, we are already supporting Clang as a front end to provide > syntax and semantic analysis. Given that Clang provides a much more > flexible framework for static analysis, the decision was a relatively simple > one. > > This is not to say that Clang provides everything needed by Annotalysis. > There is some need to use dataflow information which needs to be > incorporated in Clang. However, a large fraction of the support required > was already available in Clang. > > > Diego. -- DeLesley Hutchins | Software Engineer | delesley@google.com | 505-206-0315 ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Announce - Thread safety annotations no longer supported in GCC 2012-04-19 20:20 ` Diego Novillo 2012-04-19 21:10 ` Delesley Hutchins @ 2012-04-20 8:50 ` Richard Guenther 1 sibling, 0 replies; 13+ messages in thread From: Richard Guenther @ 2012-04-20 8:50 UTC (permalink / raw) To: Diego Novillo Cc: Andrew Pinski, Delesley Hutchins, gcc, Ollie Wild, Le-Chun Wu On Thu, Apr 19, 2012 at 10:20 PM, Diego Novillo <dnovillo@google.com> wrote: > On 4/19/12 4:14 PM, Andrew Pinski wrote: > >> How do you know it is a major effort? Has any issues related to >> changing Tuple/front-ends AST been raised to the mailing list and >> asked for help on how to implement these changes? > > > The kind of analysis that Annotalysis needs cannot be catered by GIMPLE, the > same way that LLVM's bitcode could not cater to it. Both representations > are geared towards code transformations, not source code analysis. It's not > an implementation issue, but a design one. It simply does not make sense > for GIMPLE or LLVM's bitcode to try to be a source code analysis framework. > > Annotalysis needs a high-fidelity representation of the original source > code. Today, that high-fidelity representation is provided exclusively by > Clang. > > Additionally, we are already supporting Clang as a front end to provide > syntax and semantic analysis. Given that Clang provides a much more > flexible framework for static analysis, the decision was a relatively simple > one. > > This is not to say that Clang provides everything needed by Annotalysis. > There is some need to use dataflow information which needs to be > incorporated in Clang. However, a large fraction of the support required > was already available in Clang. Our high-level AST is language specific. In case of C++ its GENERIC plus some C++ specific tree codes. There is no framework for building a CFG on top of that (not sure if you need that), but the cgraph is built over that representation. Of course non-optimizing ASTs will limit static analysis to TU scope, even with clang? Or does clang support a "LTO" source AST? Richard. > > Diego. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Announce - Thread safety annotations no longer supported in GCC 2012-04-19 19:15 ` Andrew Pinski 2012-04-19 20:20 ` Diego Novillo @ 2012-04-19 20:55 ` Delesley Hutchins 1 sibling, 0 replies; 13+ messages in thread From: Delesley Hutchins @ 2012-04-19 20:55 UTC (permalink / raw) To: Andrew Pinski; +Cc: Diego Novillo, gcc, Ollie Wild, Le-Chun Wu > How do you know it is a major effort? Changing the intermediate language is *always* a major effort, especially in a large, complex, and mature compiler. Moreover, it's not necessarily Gimple itself, it's the way the front-end lowers code from C++ to Gimple, it's the way in which optimizations are structured and scheduled, etc. In other words, the entire compiler architecture. The actual analysis we do is relatively simple, so re-implementing it in clang was by far the easier path. Moreover, although I can describe the problems I had doing static analysis in gcc, I can't point to any reasonable solutions. Gcc is good at what it does, and most of the things that were roadblocks for me are actually important to gcc's core mission -- which is to compile C++ programs into fast, high-quality machine code. It's not that we were deliberately keeping quiet, it's that these are not problems that can be easily fixed without breaking something far more important. Again, I would be happy to share some of our experiences with you if you are interested. But it's not going to change our decision. -DeLesley Has any issues related to > changing Tuple/front-ends AST been raised to the mailing list and > asked for help on how to implement these changes? > > This is why this decision is such a disappointment here because I have > not seen one message about what needs to change to help support the > future development of static analyzer in GCC. Yes people have said it > can't be done fully currently because the AST but that does not mean > asking the right question and answering how can we change GCC to allow > for a better job of doing this kind of development. > > Being quiet about it and then saying it can't be done without any > reasoning behind why moving away from GCC is seems a bit out of place. > > Thanks, > Andrew Pinski > >> >> -DeLesley >> >> On Thu, Apr 19, 2012 at 9:25 AM, Andrew Pinski <pinskia@gmail.com> wrote: >>> On Thu, Apr 19, 2012 at 8:44 AM, Delesley Hutchins <delesley@google.com> wrote: >>>> The gcc version has been difficult to support and maintain, due mainly >>>> to the fact that the GIMPLE intermediate language was never designed >>>> for static analysis. The abstract syntax tree provided by Clang is an >>>> easier data structure to work with for front-end analyses of this >>>> kind. Moreover, the gcc implementation of annotalysis has some issues >>>> that make an eventual merge into trunk somewhat unlikely, and >>>> annotalysis is of little use to people outside of google as long as it >>>> stays in google/main. The clang implementation has been in trunk from >>>> the beginning. >>>> >>>> Hope that explains it a bit better, >>> >>> No, that it does not help at all. This seems like a high level issue >>> of the problem rather than describing the reasons why GIMPLE will >>> never work correctly for your usage. Maybe we can expand it for your >>> usage but we need to better understand what it is lacking. >>> >>> Thanks, >>> Andrew Pinski >>> >>> >>>> >>>> -DeLesley >>>> >>>> On Thu, Apr 19, 2012 at 8:15 AM, Andrew Pinski <pinskia@gmail.com> wrote: >>>>> On Thu, Apr 19, 2012 at 7:15 AM, Diego Novillo <dnovillo@google.com> wrote: >>>>>> >>>>>> We have decided to terminate the thread safety annotation project in >>>>>> GCC. >>>>>> >>>>>> The current implementation is in the branch google/main for those >>>>>> interested in using it. We will not be pursuing a merge into trunk. >>>>>> Instead, we have started implementing the same functionality in Clang. >>>>> >>>>> What went into making this decision? I know lot of folks will almost >>>>> never go over to using clang since it means supporting one extra >>>>> front-end. I am thinking of the embedded folks here where they cannot >>>>> afford supporting something as new as clang for their customers. >>>>> >>>>> Thanks, >>>>> Andrew Pinski >>>>> >>>>>> >>>>>> I've updated the wiki page and moved the branch out of the active >>>>>> development branches in svn.html. >>>>>> >>>>>> >>>>>> Diego. >>>> >>>> >>>> >>>> -- >>>> DeLesley Hutchins | Software Engineer | delesley@google.com | 505-206-0315 >> >> >> >> -- >> DeLesley Hutchins | Software Engineer | delesley@google.com | 505-206-0315 -- DeLesley Hutchins | Software Engineer | delesley@google.com | 505-206-0315 ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Announce - Thread safety annotations no longer supported in GCC 2012-04-19 17:30 ` Delesley Hutchins 2012-04-19 19:15 ` Andrew Pinski @ 2012-04-20 2:31 ` Miles Bader 2012-04-20 17:55 ` Delesley Hutchins 1 sibling, 1 reply; 13+ messages in thread From: Miles Bader @ 2012-04-20 2:31 UTC (permalink / raw) To: Delesley Hutchins Cc: Andrew Pinski, Diego Novillo, gcc, Ollie Wild, Le-Chun Wu Delesley Hutchins <delesley@google.com> writes: > I can give you detailed technical reasons why GIMPLE was not working > for us if you like, but I'm not sure it would be all that > constructive. Why wouldn't it be constructive? Even if it's impractical for gcc to change to the degree needed to fit your particular project (especially in the short term), hearing the details of how gcc's representations fell short, and how others may have done things better, seems useful. Understanding is almost always a good thing, even if it can't always be put to immediate use... Things will evolve over time (yes, even gcc!), and such input plays a very useful part in guiding this evolution. Of course, such an explanation entails work on your part, so it's understandable if you're a bit skeptical as to whether it's worth the effort, but still, in the abstract... Thanks, -Miles -- People who are more than casually interested in computers should have at least some idea of what the underlying hardware is like. Otherwise the programs they write will be pretty weird. -- Donald Knuth ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Announce - Thread safety annotations no longer supported in GCC 2012-04-20 2:31 ` Miles Bader @ 2012-04-20 17:55 ` Delesley Hutchins 0 siblings, 0 replies; 13+ messages in thread From: Delesley Hutchins @ 2012-04-20 17:55 UTC (permalink / raw) To: Miles Bader; +Cc: Andrew Pinski, Diego Novillo, gcc, Ollie Wild, Le-Chun Wu [-- Attachment #1: Type: text/plain, Size: 777 bytes --] > Why wouldn't it be constructive? > > Even if it's impractical for gcc to change to the degree needed to fit > your particular project (especially in the short term), hearing the > details of how gcc's representations fell short, and how others may > have done things better, seems useful. My main concern is to avoid turning this into a "clang vs. gcc" debate. The two compilers make different trade-offs, so it is hardly surprising that one is better than the other for a particular use case. That does not mean that one is better than the other overall. But since it seems that there is significant interest, I have written down some of the technical problems that we ran into when working with gcc. Description and accompanying source code are enclosed. -DeLesley [-- Attachment #2: gcc-problem-description.txt --] [-- Type: text/plain, Size: 6426 bytes --] Annotalysis is a flow and path sensitive analysis, in which we track the set of locks that are known to be held at each program point, and verify that any access to guarded data, or call to a guarded method, is made while holding the appropriate lock. Annotations are used to mark data members and methods as "guarded". In the ideal world of our imagination, we would like an intermediate representation that supplies the following information, in order of priority: (1) It must provide a CFG that accurately reflects the source code. This is critical. (2) It must provide accurate source-language type information for each expression, so that we can find annotations on virtual methods. (3) We need to look at the IR before any optimizations occur. (4) It should be stable from release to release of the compiler. (5) All local variables should be in SSA form; SSA makes life much easier. (6) It should identify loads and stores. (7) It should be simple and easy to traverse. Gimple satisfies 5-7 almost perfectly, does 1-3 pretty well, and fails on 4. Unfortunately, "pretty well" on 1-3 was not sufficient for our needs, and 4 was getting to be a problem. I will provide a few examples of things that have caused major headaches. These examples are enclosed in gcc-problem-examples.cpp, which you can compile with -fdump-tree-ssa to follow along. Descriptions relate to gcc 4.6; I have not tested with gcc 4.7. ======== (1) Inaccurate CFG void foo1(Mutex* mu, bool c) { mu->Lock(); if (c) { MyClass mc; if (c) { mu->Unlock(); return; } // extra join point here to call ~MyClass() } mu->Unlock(); return; }; The lowering to GIMPLE produces an extra join point in the CFG that is not present in the source code, in order to call the destructor of MyClass. The benefit from a codegen standpoint is that different paths in the CFG can share the same destructor code, so this is ordinarily a good thing. However, our analysis algorithm merges locksets whenever it hits a join point, so we get false positives in this case. Fixing this problem would require a substantial rewrite of the algorithm to handle conditionally held locks. ======= (2) Inaccurate type information. void foo2(void* p) { ((MyClass*) p)->myMethod(); } This is a minor nit. The virtual function call uses a type cast to grab the vtable, but then passes p (with type void*) as 'this'. So when I go back and try to locate the method later, I get a method lookup failure. To be fair, this sort of thing does not happen very often. ======== (3) "local" optimizations with non-local effects. Annotalysis is scheduled as a local optimization pass that runs before all other optimizations. Unfortunately, some "local" optimizations have non-local effects that interfere with the analysis. IPA-SRA has been the main source of headaches: class DummyMutex { public: void Lock() EXCLUSIVE_LOCK_FUNCTION(this) { } void Unlock() UNLOCK_FUNCTION(this) { } }; void foo3(DummyMutex* dmu1, DummyMutex* dmu2) { dmu1->Lock(); dmu2->Lock(); dmu2->Unlock(); dmu1->Unlock(); } DummyMutex is a class that implements the interface of a mutex without doing anything. The LOCK_FUNCTION and UNLOCK_FUNCTION macros are annotations that the analysis uses to build locksets. IPA-SRA will helpfully notice that the body of these functions do not refer to 'this', and thus they can be lifted to static methods; a good optimization. It fails to notice that the *annotation* refers to 'this'. The lifting happens "locally", before the compilation of foo3(), so when checking foo3, the analysis can no longer determine which object is being locked or unlocked. The net result of this change is that we get false positives when compiling with optimizations turned on, even though the analysis is supposed to run before any optimizations. (Compare the output of -fdump-tree-ssa when compiling normally, versus compiling with -O3.) ======== (4) Unstable platform. Problems (1) and (3) were both introduced in gcc 4.6; they did not exist in earlier versions of the compiler. gcc 4.7 introduces new breakages that I have not investigated. The lowering algorithm from C++ to gimple, the gimple intermediate language, and the optimization pass framework, are all very fluid, so there's a lot of maintenance work required to keep annotalysis up-to-date. ======== Comparison with Clang. The clang AST satisfies 1-4 perfectly. The AST records all information, and follows the C++ spec as closely as possible. Clang does not peform any transformations or optimizations in the front end. It builds a CFG on top of the original AST, with pointers to the original expressions. The AST is then lowered to LLVM IR in a separate pass, which happens after the AST has been fully constructed. Because the AST is keyed to the C++ spec, it rarely changes. The clang AST fails utterly on 5-7. The AST is not simple, because C++ is not simple. For example, whereas gimple has function calls, clang has constructor, destructor, new, delete, implicit destructor, function, and method calls. So that involves more work. We also had to implement our own use-def chains that mimic SSA, and identify expressions that would cause a load or store. (Note that the clang static analyzer framework does provide some of this functionality, but our current implementation does not make use of it.) As you can see, neither platform perfectly meets our needs. However, issues 5-7 were lower priority, and we had a straightforward way to work around them that did not require major changes to the algorithm, or alterations to the architecture of the compiler. ======== Conclusion From our perspective, the ideal solution would be to have a strongly typed high-level intermediate language, somewhat like the JVM or .NET IRs, and initially lower from C++ source code to the high-level IR. Static analysis passes and high-level optimizations could be run on this IR. The high-level IR would then be lowered to a low-level IR like Gimple or LLVM, where low-level optimization and codegen passes would occur. Java uses an architecture like this, which is one reason why academic research has overwhelmingly focused on Java; Java bytecode is vastly easier to work with than C++. Neither clang nor gcc currently use such an architecture. [-- Attachment #3: gcc-problem-examples.cpp --] [-- Type: text/x-c++src, Size: 683 bytes --] class Mutex { public: void Lock(); void Unlock(); }; class MyClass { public: virtual void myMethod(); MyClass(); ~MyClass(); }; // (1) Inaccurate CFG void foo1(Mutex* mu, bool c) { mu->Lock(); if (c) { MyClass mc; if (c) { mu->Unlock(); return; } // extra join point here to call ~MyClass() } mu->Unlock(); return; }; // (2) Inaccurate type information void foo2(void* p) { ((MyClass*) p)->myMethod(); } // (3) Problem with optimizations class DummyMutex { public: void Lock() { } void Unlock() { } }; void foo3(DummyMutex* dmu1, DummyMutex* dmu2) { dmu1->Lock(); dmu2->Lock(); dmu2->Unlock(); dmu1->Unlock(); } ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Announce - Thread safety annotations no longer supported in GCC @ 2012-04-20 15:40 Manuel López-Ibáñez 0 siblings, 0 replies; 13+ messages in thread From: Manuel López-Ibáñez @ 2012-04-20 15:40 UTC (permalink / raw) To: gcc Mailing List; +Cc: Richard Guenther Since nobody answered to Richard, and I find the discussion interesting to understand what the future of GCC might be > Our high-level AST is language specific. In case of C++ its GENERIC plus > some C++ specific tree codes. There is no framework for building a CFG > on top of that (not sure if you need that), but the cgraph is built over that > representation. C/C++ GENERIC does not accurately represent the original source code, and I understand that this is on purpose (or at least, it is not a goal). This is one of the major criticisms of GCC that (supposedly) led to the development of Clang (see the first 20 minutes of http://channel9.msdn.com/Events/GoingNative/GoingNative-2012/Clang-Defending-C-from-Murphy-s-Million-Monkeys ). > Of course non-optimizing ASTs will limit static analysis to TU scope, even > with clang? Or does clang support a "LTO" source AST? It seems it does: http://clang.llvm.org/doxygen/Index_8h.html http://clang.llvm.org/doxygen/dir_e9b826b1b01168f6fc5ffb2b00be9311.html And even if it didn't, it is a clearly expressed goal of Clang to support such uses. Quoting from http://clang.llvm.org/features.html#diverseclients "The problem with this goal is that different clients have very different requirements. Consider code generation, for example: a simple front-end that parses for code generation must analyze the code for validity and emit code in some intermediate form to pass off to a optimizer or backend. Because validity analysis and code generation can largely be done on the fly, there is not hard requirement that the front-end actually build up a full AST for all the expressions and statements in the code. TCC and GCC are examples of compilers that either build no real AST (in the former case) or build a stripped down and simplified AST (in the later case) because they focus primarily on codegen. On the opposite side of the spectrum, some clients (like refactoring) want highly detailed information about the original source code and want a complete AST to describe it with. Refactoring wants to have information about macro expansions, the location of every paren expression '(((x)))' vs 'x', full position information, and much more. Further, refactoring wants to look across the whole program to ensure that it is making transformations that are safe. Making this efficient and getting this right requires a significant amount of engineering and algorithmic work that simply are unnecessary for a simple static compiler." Cheers, Manuel. ^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2012-04-20 17:55 UTC | newest] Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <CAFOgFcSO9NRXprgW+LxkcOCjttzGZiqyugwbLPbrLpQUqMkmHA@mail.gmail.com> 2012-04-19 14:15 ` Announce - Thread safety annotations no longer supported in GCC Diego Novillo 2012-04-19 15:15 ` Andrew Pinski 2012-04-19 15:44 ` Delesley Hutchins 2012-04-19 16:25 ` Andrew Pinski 2012-04-19 17:30 ` Delesley Hutchins 2012-04-19 19:15 ` Andrew Pinski 2012-04-19 20:20 ` Diego Novillo 2012-04-19 21:10 ` Delesley Hutchins 2012-04-20 8:50 ` Richard Guenther 2012-04-19 20:55 ` Delesley Hutchins 2012-04-20 2:31 ` Miles Bader 2012-04-20 17:55 ` Delesley Hutchins 2012-04-20 15:40 Manuel López-Ibáñez
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).