Query status of GSoC project: CPyChecker

Query status of GSoC project: CPyChecker

[Steven Sun]

[-- Attachment #1: Type: text/plain, Size: 1246 bytes --]

Hi Eric, I am Steven (now) from the CPython team.

How is the project going? Do you have any prototypes
or ideas that can be discussed? Which part will you start at?


I recently debugged dozens of Python bugs, some involving
C APIs. I can provide some test cases for you.


For the ref count part:

A major change (immortal objects) is introduced in Python 3.12.
Basically, immortal objects will have the ref count fixed at
a very large number (depending on `sizeof(void*)` ). But I
don't think it is necessary to implement this in the early
stages.

Some stable API steals reference conditionally (on success),
thus its behavior cannot be simply described by one attribute.


For CPython versions:

Some stable CPython API behavior varied across the minor
release. (eg. 3.10 -> 3.11) For instance, some API accepted
NULL as args for <3.8, but not >=3.8.

Considering both "GCC" and "CPython" are hard for users to
upgrade, we might want to consider how to live with these
behavioral differences in the first place.

Versions older than 3 minor releases cannot be touched. (3.13
now in active development, 3.12, 3.11 for bug fixes, 3.10, 3.9
security fixes only) So, versions <= 3.10 can be treated as frozen.

Re: Query status of GSoC project: CPyChecker

[Eric Feng]

Hi Steven,

Thanks for reaching out. The project is still in very early stages. So
far we have taught the analyzer the basic behavior for
PyLong_FromLong, PyList_New, and Py_DECREF via known function
subclassing. Additionally, Py_INCREF is supported out of the box.
Reference count checking functionality remains the priority, but it is
not yet fully implemented.

Regarding CPython versions, the goal is to just get things working on
one version first. I arbitrarily picked 3.9, but happy to consider
another version as an initial goal if it’s more helpful to the CPython
community.

Feel free to check out the repo at
https://github.com/efric/gcc-cpython-analyzer for details on the
project and to follow along and help out where you are interested.

Best,
Eric


On Tue, Jun 27, 2023 at 6:03 AM Steven Sun <StevenSun2021@hotmail.com> wrote:
>
> Hi Eric, I am Steven (now) from the CPython team.
>
> How is the project going? Do you have any prototypes
> or ideas that can be discussed? Which part will you start at?
>
>
> I recently debugged dozens of Python bugs, some involving
> C APIs. I can provide some test cases for you.
>
>
> For the ref count part:
>
> A major change (immortal objects) is introduced in Python 3.12.
> Basically, immortal objects will have the ref count fixed at
> a very large number (depending on `sizeof(void*)` ). But I
> don't think it is necessary to implement this in the early
> stages.
>
> Some stable API steals reference conditionally (on success),
> thus its behavior cannot be simply described by one attribute.
>
>
> For CPython versions:
>
> Some stable CPython API behavior varied across the minor
> release. (eg. 3.10 -> 3.11) For instance, some API accepted
> NULL as args for <3.8, but not >=3.8.
>
> Considering both "GCC" and "CPython" are hard for users to
> upgrade, we might want to consider how to live with these
> behavioral differences in the first place.
>
> Versions older than 3 minor releases cannot be touched. (3.13
> now in active development, 3.12, 3.11 for bug fixes, 3.10, 3.9
> security fixes only) So, versions <= 3.10 can be treated as frozen.

Re: Query status of GSoC project: CPyChecker

[Steven Sun]

[-- Attachment #1: Type: text/plain, Size: 1198 bytes --]

Hi Eric,

> Thanks for reaching out. The project is still in very early stages. So
> far we have taught the analyzer the basic behavior for
> PyLong_FromLong, PyList_New, and Py_DECREF via known function
> subclassing. Additionally, Py_INCREF is supported out of the box.
> Reference count checking functionality remains the priority, but it is
> not yet fully implemented.

Great!

> Regarding CPython versions, the goal is to just get things working on
> one version first. I arbitrarily picked 3.9, but happy to consider
> another version as an initial goal if it’s more helpful to the CPython
> community.

I am not sure about this.

cpychecker is more beneficial to CPython extension devs than to
CPython devs, since it is almost impossible to let the cpychecker learn
the most updated internal function definitions without handwritten
attributes or seeing the whole function definitions.

So it depends on the extension maintainer. I am observing this pattern
that popular libraries are gradually upgrading. 3.9 and 3.10 is definitely
the current mainstream.

Saying so, I think 3.9 is fine for now, but it will be outdated after 2 to 3
years.


Best,
Steven