Re: Update on CPython Extension Module -fanalyzer plugin development

[Eric Feng <ef2648@columbia.edu>]

On Mon, Aug 21, 2023 at 11:04 AM David Malcolm <dmalcolm@redhat.com> wrote:
>
> On Mon, 2023-08-21 at 10:05 -0400, Eric Feng wrote:
> > Hi Dave,
> >
> > Just wanted to give you and everyone else a short update on how
> > reference count checking is going — we can now observe the refcnt
> > diagnostic being emitted:
> >
> > rc3.c:22:10: warning: REF COUNT PROBLEM
> >    22 |   return list;
> >       |          ^~~~
> >   ‘create_py_object’: events 1-4
> >     |
> >     |    4 |   PyObject* item = PyLong_FromLong(3);
> >     |      |                    ^~~~~~~~~~~~~~~~~~
> >     |      |                    |
> >     |      |                    (1) when ‘PyLong_FromLong’ succeeds
> >     |    5 |   PyObject* list = PyList_New(1);
> >     |      |                    ~~~~~~~~~~~~~
> >     |      |                    |
> >     |      |                    (2) when ‘PyList_New’ succeeds
> >     |......
> >     |   14 |   PyList_Append(list, item);
> >     |      |   ~~~~~~~~~~~~~~~~~~~~~~~~~
> >     |      |   |
> >     |      |   (3) when ‘PyList_Append’ fails
> >     |......
> >     |   22 |   return list;
> >     |      |          ~~~~
> >     |      |          |
> >     |      |          (4) here
> >     |
> >
> > I will fix up and refactor the logic for counting the actual ref
> > count
> > before coming back and refining the diagnostic to give much more
> > detailed information.
>
> Excellent!  Thanks for the update.
>
> Dave
>

Hi Dave,

I've since fixed up the logic to count the actual reference counts of
the PyObject* instances. Now, I'm contemplating the specific
diagnostics we'd want to issue and the appropriate conditions for
emitting them. With this in mind, I wanted to check in with you on the
appropriate approach:

To start, I'm adopting the same assumptions as cpychecker for
functions returning a PyObject*. That is, I'm operating under the
premise that by default such functions return a new reference upon
success rather than, for example, a borrowed reference (which we can
tackle later on). Given this, it's my understanding that the reference
count of the returned object should be 1 if the object is newly
created within the function body and incremented by 1 from what it was
previously if not newly created (e.g passed in as an argument).
Furthermore, the reference count for any PyObject* instances created
within the function should be 0, barring situations where we're
returning a collection, like a list, that includes references to these
objects.

Let me know what you think; thanks!

Best,
Eric