PyIter_Check false positives

PyIter_Check false positives

[airplanemath]

[-- Attachment #1: Type: text/plain, Size: 2018 bytes --]


I found this problem running the tests for pandas, but can reproduce it
more simply.

Compile the following cython file to create a thin wrapper around the
C-API function:

> from cpython.iterator cimport PyIter_Check
> 
> def is_iterator(obj: object) -> bool:
>     return PyIter_Check(obj)

with `cythonize --build --inplace test_iterator.pyx` (probably requires
python39-cython), changing the name if needed, then try the following in
Python:

>>> from collections.abc import Iterator
>>> from fractions import Fraction
>>> from os import environ
>>> from test_iterator import is_iterator
>>> isinstance(environ, Iterator)
False
>>> is_iterator(environ)
True
>>> next(environ)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: '_Environ' object is not an iterator
>>> isinstance(Fraction(0, 1), Iterator)
False
>>> is_iterator(Fraction(0, 1))
True
>>> next(Fraction(0, 1))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'Fraction' object is not an iterator

From the Python documentation [1], it appears that `PyIter_Check` is
intended to return True only if the `PyIter_Next` will work, and
`PyIter_Next` is the C equivalent of the python `next` function.  That
is, `isinstance(thing, Iterator)` and `is_iterator(thing)` should agree
with each other, and should return True only if `next(thing)` works
(produces an element or says there aren't any).
On Linux, both checks agree with each other, returning False, and the
attempts to advance the iterator with `next` still fail.
As seen above, on (my) Cygwin, PyIter_Check disagrees with
`collections.abc.Iterator` and succeeds in cases where `next` produces a
TypeError.  I can reproduce this with python3.9 and with python2.7 on Cygwin.

Am I missing something in the documentation?  Is this a side-effect of
working on top of Windows?

In case it's relevant, I'm using an unpatched Cython with the
distribution Python packages.

[1] https://docs.python.org/3/c-api/iter.html#c.PyIter_Check


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: PyIter_Next Cython wrapper module source --]
[-- Type: text/x-python3, Size: 308 bytes --]

from cpython.iterator cimport PyIter_Check

def is_iterator(obj: object) -> bool:
    """Check whether obj is an iterator.

    Should agree with isinstance(obj, collections.abc.Iterator).

    Parameters
    ----------
    obj : object

    Returns
    -------
    bool
    """
    return PyIter_Check(obj)