Static analysis updates in GCC 11

Static analysis updates in GCC 11

[David Malcolm]

I wrote a blog post covering what I've been working on in the analyzer
in this release:
 https://developers.redhat.com/blog/2021/01/28/static-analysis-updates-in-gcc-11/

Hope this is of interest
Dave

Re: Static analysis updates in GCC 11

[David Brown]

On 28/01/2021 21:23, David Malcolm via Gcc wrote:
> I wrote a blog post covering what I've been working on in the analyzer
> in this release:
>  https://developers.redhat.com/blog/2021/01/28/static-analysis-updates-in-gcc-11/
> 

As a gcc user, I am always glad to hear of more static analysis and
static warning work.  My own work is mostly on small embedded systems,
where "malloc" and friends are severely frowned upon in any case and
there is no file system, so most of the gcc 10 -fanalyzer warnings are
of no direct use to me.  (I still think they are great ideas - even if
/I/ don't write much PC code, everyone benefits if there are fewer bugs
in programs.)  I will get more use for the new warnings you've added for
gcc 11.


I wrote a feature request for gcc a while back, involving adding tag
attributes to functions in order to ensure that certain classes of
functions are only used from specific allowed functions.  The feature
request attracted only a little interest at the time.  But I suspect it
could work far better along with the kind of analysis you are doing with
-fanalyzer than with the normal syntactical analyser in gcc.

<https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88391>

David

Re: Static analysis updates in GCC 11

[David Malcolm]

On Thu, 2021-01-28 at 22:06 +0100, David Brown wrote:
> On 28/01/2021 21:23, David Malcolm via Gcc wrote:
> > I wrote a blog post covering what I've been working on in the
> > analyzer
> > in this release:
> >  https://developers.redhat.com/blog/2021/01/28/static-analysis-updates-in-gcc-11/
> > 
> 
> As a gcc user, I am always glad to hear of more static analysis and
> static warning work.  My own work is mostly on small embedded
> systems,
> where "malloc" and friends are severely frowned upon in any case and
> there is no file system, so most of the gcc 10 -fanalyzer warnings
> are
> of no direct use to me.  (I still think they are great ideas - even
> if
> /I/ don't write much PC code, everyone benefits if there are fewer
> bugs
> in programs.)  I will get more use for the new warnings you've added
> for
> gcc 11.
> 
> 
> I wrote a feature request for gcc a while back, involving adding tag
> attributes to functions in order to ensure that certain classes of
> functions are only used from specific allowed functions.  The feature
> request attracted only a little interest at the time.  But I suspect
> it
> could work far better along with the kind of analysis you are doing
> with
> -fanalyzer than with the normal syntactical analyser in gcc.
> 
> <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88391>

Interesting.  The attribute ideas seem designed to work with the
callgraph: partitioning the callgraph into families of functions for
which certain kinds of inter-partition edges are disallowed.  Can a
function change its tag internally, or is it assumed that a function
has a single tag throughout its whole body?  I see that you have a case
in example 3 where a compound statement is marked with an attribute
(which may be an extension of our syntax).

One thing I forgot to mention in the blog post is that the analyzer now
supports plugins; there's an example of a mutex-checking plugin here:
https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=66dde7bc64b75d4a338266333c9c490b12d49825
which is similar to your examples 1 and 3.  Your example 2 is also
reminiscent of the async-signal-unsafe checking that the analyzer has
(where it detects code paths that are called within a signal handler
and complains about bad calls within them).  Many of the existing
checks in the analyzer are modelled as state machines (either global
state for things like "are we in a signal handler", or per-value state
for things like "has this pointer been freed"), and your examples could
be modelled that way too (e.g. "what sections are in RAM" could be a
global state) - so maybe it could all be done as analyzer plugins, in
lieu of implementing the RFE you posted.

Hope this is constructive
Dave

Re: Static analysis updates in GCC 11

[Martin Sebor]

On 1/28/21 2:27 PM, David Malcolm via Gcc wrote:
> On Thu, 2021-01-28 at 22:06 +0100, David Brown wrote:
>> On 28/01/2021 21:23, David Malcolm via Gcc wrote:
>>> I wrote a blog post covering what I've been working on in the
>>> analyzer
>>> in this release:
>>>   https://developers.redhat.com/blog/2021/01/28/static-analysis-updates-in-gcc-11/
>>>
>>
>> As a gcc user, I am always glad to hear of more static analysis and
>> static warning work.  My own work is mostly on small embedded
>> systems,
>> where "malloc" and friends are severely frowned upon in any case and
>> there is no file system, so most of the gcc 10 -fanalyzer warnings
>> are
>> of no direct use to me.  (I still think they are great ideas - even
>> if
>> /I/ don't write much PC code, everyone benefits if there are fewer
>> bugs
>> in programs.)  I will get more use for the new warnings you've added
>> for
>> gcc 11.
>>
>>
>> I wrote a feature request for gcc a while back, involving adding tag
>> attributes to functions in order to ensure that certain classes of
>> functions are only used from specific allowed functions.  The feature
>> request attracted only a little interest at the time.  But I suspect
>> it
>> could work far better along with the kind of analysis you are doing
>> with
>> -fanalyzer than with the normal syntactical analyser in gcc.
>>
>> <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88391>
> 
> Interesting.  The attribute ideas seem designed to work with the
> callgraph: partitioning the callgraph into families of functions for
> which certain kinds of inter-partition edges are disallowed.  Can a
> function change its tag internally, or is it assumed that a function
> has a single tag throughout its whole body?  I see that you have a case
> in example 3 where a compound statement is marked with an attribute
> (which may be an extension of our syntax).

Florian suggested a similar approach (tags) as an enhancement to
the malloc attribute extension we've just added, to avoid having
to exhaustively associate every allocator with every deallocator.

Martin

> 
> One thing I forgot to mention in the blog post is that the analyzer now
> supports plugins; there's an example of a mutex-checking plugin here:
> https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=66dde7bc64b75d4a338266333c9c490b12d49825
> which is similar to your examples 1 and 3.  Your example 2 is also
> reminiscent of the async-signal-unsafe checking that the analyzer has
> (where it detects code paths that are called within a signal handler
> and complains about bad calls within them).  Many of the existing
> checks in the analyzer are modelled as state machines (either global
> state for things like "are we in a signal handler", or per-value state
> for things like "has this pointer been freed"), and your examples could
> be modelled that way too (e.g. "what sections are in RAM" could be a
> global state) - so maybe it could all be done as analyzer plugins, in
> lieu of implementing the RFE you posted.
> 
> Hope this is constructive
> Dave
> 

Re: Static analysis updates in GCC 11

[David Brown]

On 28/01/2021 22:27, David Malcolm wrote:
> On Thu, 2021-01-28 at 22:06 +0100, David Brown wrote:
>> On 28/01/2021 21:23, David Malcolm via Gcc wrote:
>>> I wrote a blog post covering what I've been working on in the
>>> analyzer
>>> in this release:
>>>   https://developers.redhat.com/blog/2021/01/28/static-analysis-updates-in-gcc-11/
>>>
>>
>> As a gcc user, I am always glad to hear of more static analysis and
>> static warning work.  My own work is mostly on small embedded
>> systems,
>> where "malloc" and friends are severely frowned upon in any case and
>> there is no file system, so most of the gcc 10 -fanalyzer warnings
>> are
>> of no direct use to me.  (I still think they are great ideas - even
>> if
>> /I/ don't write much PC code, everyone benefits if there are fewer
>> bugs
>> in programs.)  I will get more use for the new warnings you've added
>> for
>> gcc 11.
>>
>>
>> I wrote a feature request for gcc a while back, involving adding tag
>> attributes to functions in order to ensure that certain classes of
>> functions are only used from specific allowed functions.  The feature
>> request attracted only a little interest at the time.  But I suspect
>> it
>> could work far better along with the kind of analysis you are doing
>> with
>> -fanalyzer than with the normal syntactical analyser in gcc.
>>
>> <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88391>
> 
> Interesting.  The attribute ideas seem designed to work with the
> callgraph: partitioning the callgraph into families of functions for
> which certain kinds of inter-partition edges are disallowed.  Can a
> function change its tag internally, or is it assumed that a function
> has a single tag throughout its whole body?  I see that you have a case
> in example 3 where a compound statement is marked with an attribute
> (which may be an extension of our syntax).
> 
> One thing I forgot to mention in the blog post is that the analyzer now
> supports plugins; there's an example of a mutex-checking plugin here:
> https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=66dde7bc64b75d4a338266333c9c490b12d49825
> which is similar to your examples 1 and 3.  Your example 2 is also
> reminiscent of the async-signal-unsafe checking that the analyzer has
> (where it detects code paths that are called within a signal handler
> and complains about bad calls within them).  Many of the existing
> checks in the analyzer are modelled as state machines (either global
> state for things like "are we in a signal handler", or per-value state
> for things like "has this pointer been freed"), and your examples could
> be modelled that way too (e.g. "what sections are in RAM" could be a
> global state) - so maybe it could all be done as analyzer plugins, in
> lieu of implementing the RFE you posted.
> 
> Hope this is constructive
> Dave
> 

Thanks for the feedback.

Just to be clear, I am not particularly tied to the syntax I suggested 
in the bugzilla suggestion - it is merely a starting point for ideas. 
"caller_tag" and "callee_tag", for example, are probably too similar and 
easily mixed up by people using the attributes.  "needs_tag" and 
"limit_to_tag" might be a better second suggestion.  Or perhaps someone 
else will think of a completely different arrangement to get a similar 
end result.

Yes, I believe tying an attribute to a compound statement might be a new 
idea, but that could just be because the only "statement attribute" 
currently in gcc (AFAIK, from reading the manual) is "fallthrough" - and 
it is generally applied to a null statement.

I suggested the possibility of attaching attributes to statements within 
a function, so that the developer can make it clear when the tag is 
acquired and released.  But it is not essential - it would be fine to 
simply say that the whole function is tagged, if that makes an 
implementation simpler.

At least some of the potential uses here could be handled by C++ strong 
typing and tag structs (carrying no data, but with restrictions on how 
they can be created and copied around).  But tag attributes, or an 
equivalent mechanism, would let you do this in C as well, and it would 
be less strict and structured - and thus easier to add to existing code.

Re: Static analysis updates in GCC 11

[David Brown]

On 29/01/2021 01:03, Martin Sebor wrote:
> On 1/28/21 2:27 PM, David Malcolm via Gcc wrote:
>> On Thu, 2021-01-28 at 22:06 +0100, David Brown wrote:
>
>>>
>>> I wrote a feature request for gcc a while back, involving adding tag
>>> attributes to functions in order to ensure that certain classes of
>>> functions are only used from specific allowed functions.  The feature
>>> request attracted only a little interest at the time.  But I suspect
>>> it
>>> could work far better along with the kind of analysis you are doing
>>> with
>>> -fanalyzer than with the normal syntactical analyser in gcc.
>>>
>>> <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88391>
>>
>> Interesting.  The attribute ideas seem designed to work with the
>> callgraph: partitioning the callgraph into families of functions for
>> which certain kinds of inter-partition edges are disallowed.  Can a
>> function change its tag internally, or is it assumed that a function
>> has a single tag throughout its whole body?  I see that you have a case
>> in example 3 where a compound statement is marked with an attribute
>> (which may be an extension of our syntax).
> 
> Florian suggested a similar approach (tags) as an enhancement to
> the malloc attribute extension we've just added, to avoid having
> to exhaustively associate every allocator with every deallocator.
> 

That could be nice - and it could be useful for all sorts of other 
resource management, not just memory pools and allocators.

One thing that always concerns me about the "malloc" attribute and 
memory pools is that the source of the pool has to come from somewhere 
(such as an OS memory allocation, or perhaps statically memory blocks) 
and your allocator will generally have pointers to keep track of it. 
That means the pointer given out by the malloc-type function /is/ 
aliased to existing memory theoretically accessible via other methods. 
I've never felt entirely comfortable that home-made allocators are 
actually completely safe and correct for all possible alias analysis. 
(And I suspect the move towards provenance based alias tracking will not 
make this easier.)

Perhaps if there are tags for malloc-like function attributes, there 
could be attributes that use the same tags to mark data blocks or 
pointers as being the source for the allocator pools.

Re: Static analysis updates in GCC 11

[Eric Gallager]

On Sat, Jan 30, 2021 at 2:29 AM Eric Gallager <egall@gwmail.gwu.edu> wrote:
>
> On Thu, Jan 28, 2021 at 3:24 PM David Malcolm via Gcc <gcc@gcc.gnu.org> wrote:
> >
> > I wrote a blog post covering what I've been working on in the analyzer
> > in this release:
> >  https://developers.redhat.com/blog/2021/01/28/static-analysis-updates-in-gcc-11/
> >
> > Hope this is of interest
> > Dave
> >
> >
>
> Thanks for the post! One nit: the post says: "Each of these
> corresponds to a pre-existing warning implemented in the C and C++
> front ends, but with a “-Wanalyzer” prefix rather than “-W.” As an
> example, -Wanalyzer-write-to-const corresponds to -Wwrite-to-const."
> There isn't actually a warning called "-Wwrite-to-const" though. There
> is "-Wwrite-strings" which is similar, but since it isn't actually a
> warning option itself (as per bug 61579), the warning flag that the
> warnings it provokes are actually issued by is -Wdiscarded-qualifiers.
> So, while I got what you meant, it still might confuse new programmers
> who might try to go using a flag called "-Wwrite-to-const" and then
> wonder why it doesn't exist.
>
> Thanks again!
> Eric

Oops I just realized I accidentally removed the mailing list from the
recipients; I'm adding that back now