Re: Update on CPython Extension Module -fanalyzer plugin development

[David Malcolm <dmalcolm@redhat.com>]

On Wed, 2023-08-23 at 17:15 -0400, Eric Feng wrote:
> On Mon, Aug 21, 2023 at 11:04 AM David Malcolm <dmalcolm@redhat.com>
> wrote:
> > 
> > On Mon, 2023-08-21 at 10:05 -0400, Eric Feng wrote:
> > > Hi Dave,
> > > 
> > > Just wanted to give you and everyone else a short update on how
> > > reference count checking is going — we can now observe the refcnt
> > > diagnostic being emitted:
> > > 
> > > rc3.c:22:10: warning: REF COUNT PROBLEM
> > >    22 |   return list;
> > >       |          ^~~~
> > >   ‘create_py_object’: events 1-4
> > >     |
> > >     |    4 |   PyObject* item = PyLong_FromLong(3);
> > >     |      |                    ^~~~~~~~~~~~~~~~~~
> > >     |      |                    |
> > >     |      |                    (1) when ‘PyLong_FromLong’
> > > succeeds
> > >     |    5 |   PyObject* list = PyList_New(1);
> > >     |      |                    ~~~~~~~~~~~~~
> > >     |      |                    |
> > >     |      |                    (2) when ‘PyList_New’ succeeds
> > >     |......
> > >     |   14 |   PyList_Append(list, item);
> > >     |      |   ~~~~~~~~~~~~~~~~~~~~~~~~~
> > >     |      |   |
> > >     |      |   (3) when ‘PyList_Append’ fails
> > >     |......
> > >     |   22 |   return list;
> > >     |      |          ~~~~
> > >     |      |          |
> > >     |      |          (4) here
> > >     |
> > > 
> > > I will fix up and refactor the logic for counting the actual ref
> > > count
> > > before coming back and refining the diagnostic to give much more
> > > detailed information.
> > 
> > Excellent!  Thanks for the update.
> > 
> > Dave
> > 
> 
> Hi Dave,
> 
> I've since fixed up the logic to count the actual reference counts of
> the PyObject* instances. 

Sounds promising.

> Now, I'm contemplating the specific
> diagnostics we'd want to issue and the appropriate conditions for
> emitting them. With this in mind, I wanted to check in with you on
> the
> appropriate approach:
> 
> To start, I'm adopting the same assumptions as cpychecker for
> functions returning a PyObject*. That is, I'm operating under the
> premise that by default such functions return a new reference upon
> success rather than, for example, a borrowed reference (which we can
> tackle later on). Given this, it's my understanding that the
> reference
> count of the returned object should be 1 if the object is newly
> created within the function body and incremented by 1 from what it
> was
> previously if not newly created (e.g passed in as an argument).
> Furthermore, the reference count for any PyObject* instances created
> within the function should be 0, barring situations where we're
> returning a collection, like a list, that includes references to
> these
> objects.
> 
> Let me know what you think; thanks!

This sounds like a good approach for v1 of the implementation.

It's probably best to focus on getting a simple version of the patch
into trunk, and leave any polish of it to followups.

In terms of deciding what the reference count of a returned PyObject *
ought to be, cpychecker had logic to try to detect callbacks used by
PyMethodDef tables, so that e.g. in:

static PyMethodDef widget_methods[] = {
    {"display",
     (PyCFunction)widget_display,
     (METH_VARARGS | METH_KEYWORDS), /* ml_flags */
     NULL},

    {NULL, NULL, 0, NULL} /* terminator */
};

...we'd know that the callback function "widget_display" follows the
standard rules for a PyCFunction (e.g. returns a new reference).

But that's for later; don't bother trying to implement that until we
have the basics working.

Is it worth posting a work-in-progress patch of what you have so far? 
(you don't need to bother with a ChangeLog for that)

Dave