python fails asyncio tests (py 3.7 & 3.8)

python fails asyncio tests (py 3.7 & 3.8)

[Jim Bell]

The standard python asyncio tests hang.

         cd /usr/lib/python3.8/test

         python3.8 test_asyncore.py -v

Passes a bunch of tests, then hangs at:

     test_connection_attributes (__main__.TestAPI_UseUnixSocketsPoll) ...

(A more focused test run: "python3.8 test_asyncore.py -c -f -k Unix")

Likewise,

         python3.8 -m unittest -v test_asyncio

Passes a bunch of tests, then hangs at:

     test_create_unix_server 
(test.test_asyncio.test_events.PollEventLoopTests) ...

I tried enabling tracing, but then unittest doesn't want to run any 
tests ("python3.8 -m trace -c test_asyncore.py" says "Ran 0 tests")

Same for python3.7. I only tested 3.7 & 3.8.

Glad to attach a run of "cygcheck -s -v -r" but it's 1.4M. Running 
CygWin-64 under Windows 10 Professional Ver 10.0 Build 19041

Re: python fails asyncio tests (py 3.7 & 3.8)

[Jim Bell]

On 2020-11-21 5:59 AM, Jim Bell wrote:
> The standard python asyncio tests hang.
>
>         cd /usr/lib/python3.8/test
>
>         python3.8 test_asyncore.py -v
>
> [...]


Using strace, stripping down this very repeatable test, and grabbing the 
source-code snapshot, it looks like winsup/cygwin/select.cc 
socket_cleanup() is waiting forever for the thread to go away.

strace:

   121 6732185 [main] python3.8 13329 select_stuff::cleanup: calling 
cleanup rout
ines
   178 6732363 [main] python3.8 13329 socket_cleanup: si 0x800290E10 
si->thread 0
x18023E758
-

But I don't see the select_printf("returning")  at select.cc:1808, so 
the si->thread->detach() call seems to be blocked forever.  I don't see 
that we make it here either:

     select.cc:1685  select_printf ("leaving thread_socket");

I see a bool stop_thread field in select_info, inherited by 
select_socket_info (select.h), used by other mechanisms. Seems that 
could be set by socket_cleanup() and used by thread_socket() to break 
out of its event loops (select.cc lines 1660  and 1667)

https://cygwin.com/git/?p=newlib-cygwin.git;a=blob;f=winsup/cygwin/select.cc;hb=HEAD#l1796

Re: python fails asyncio tests (py 3.7 & 3.8)

[Mark Geisert]

Hi Jim,

Jim Bell wrote:
> On 2020-11-21 5:59 AM, Jim Bell wrote:
>> The standard python asyncio tests hang.
>>
>>         cd /usr/lib/python3.8/test
>>
>>         python3.8 test_asyncore.py -v
>>
>> [...]
> 
> 
> Using strace, stripping down this very repeatable test, and grabbing the 
> source-code snapshot, it looks like winsup/cygwin/select.cc socket_cleanup() is 
> waiting forever for the thread to go away.
> 
> strace:
> 
>    121 6732185 [main] python3.8 13329 select_stuff::cleanup: calling cleanup rout
> ines
>    178 6732363 [main] python3.8 13329 socket_cleanup: si 0x800290E10 si->thread 0
> x18023E758
> -
> 
> But I don't see the select_printf("returning")  at select.cc:1808, so the 
> si->thread->detach() call seems to be blocked forever.  I don't see that we make 
> it here either:
> 
>      select.cc:1685  select_printf ("leaving thread_socket");
> 
> I see a bool stop_thread field in select_info, inherited by select_socket_info 
> (select.h), used by other mechanisms. Seems that could be set by socket_cleanup() 
> and used by thread_socket() to break out of its event loops (select.cc lines 1660  
> and 1667)
> 
> https://cygwin.com/git/?p=newlib-cygwin.git;a=blob;f=winsup/cygwin/select.cc;hb=HEAD#l1796 

Thanks for the report and especially for the initial debugging you've done.  I've 
reproduced the issue on my test machine.  No need to supply 'cygcheck -svr' at 
this point.  I'll investigate this further and keep you posted (on the Cygwin 
mailing list).

..mark

Re: python fails asyncio tests (py 3.7 & 3.8)

[Mark Geisert]

[Replying to myself...]

Mark Geisert wrote:
> Jim Bell wrote:
>> On 2020-11-21 5:59 AM, Jim Bell wrote:
>>> The standard python asyncio tests hang.
>>>
>>>         cd /usr/lib/python3.8/test
>>>
>>>         python3.8 test_asyncore.py -v
>>>
>>> [...]
>> Using strace, stripping down this very repeatable test, and grabbing the 
>> source-code snapshot, it looks like winsup/cygwin/select.cc socket_cleanup() is 
>> waiting forever for the thread to go away.
>>
>> strace:
>>
>>    121 6732185 [main] python3.8 13329 select_stuff::cleanup: calling cleanup rout
>> ines
>>    178 6732363 [main] python3.8 13329 socket_cleanup: si 0x800290E10 si->thread 0
>> x18023E758
>> -
[...]
> Thanks for the report and especially for the initial debugging you've done.  I've 
> reproduced the issue on my test machine.  No need to supply 'cygcheck -svr' at 
> this point.  I'll investigate this further and keep you posted (on the Cygwin 
> mailing list).

The problem is actually further down in the guts of Cygwin's Unix Domain Socket 
handling.  Specifically it has to do with the credential passing between client 
and server ends of a connection as part of its setup.  There is a workaround for 
this problem that involves turning off socket option SO_PEERCRED on both the 
connecting and listening sockets.

Unfortunately there's another problem.  With the current Cygwin DLL build you'll 
get an EINVAL error trying to do that setsockopt() operation.  I have submitted a 
patch that fixes this 2nd problem.  A future Cygwin snapshot TBA will contain this 
patch.

As for the test script errors you reported, I can submit a workaround patch that 
would make its way into the Python3.8 tests.  I am unsure at the moment which 
Cygwin package contains those tests but I can figure that out.  Otherwise, I could 
tell you what needs to be patched in test_asyncore.py and you could edit the 
script yourself to fix this locally for yourself.

Let me know how you'd like to proceed, when you have a chance.

..mark

Re: python fails asyncio tests (py 3.7 & 3.8)

[Marco Atzeri]

On 07.12.2020 08:58, Mark Geisert wrote:
> [Replying to myself...]
> 
> Mark Geisert wrote:
>> Jim Bell wrote:
>>> On 2020-11-21 5:59 AM, Jim Bell wrote:
>>>> The standard python asyncio tests hang.
>>>>
>>>>         cd /usr/lib/python3.8/test
>>>>
>>>>         python3.8 test_asyncore.py -v
>>>>

> As for the test script errors you reported, I can submit a workaround 
> patch that would make its way into the Python3.8 tests.  I am unsure at 
> the moment which Cygwin package contains those tests but I can figure 
> that out.  Otherwise, I could tell you what needs to be patched in 
> test_asyncore.py and you could edit the script yourself to fix this 
> locally for yourself.
> 
> Let me know how you'd like to proceed, when you have a chance.

test_asyncore.py is part of the test suite of python itself.
I doubt patching the test will change the functionality of the program 
itself


> ..mark

Regards
Marco

Re: python fails asyncio tests (py 3.7 & 3.8)

[Mark Geisert]

Marco Atzeri via Cygwin wrote:
> On 07.12.2020 08:58, Mark Geisert wrote:
>> [Replying to myself...]
>>
>> Mark Geisert wrote:
>>> Jim Bell wrote:
>>>> On 2020-11-21 5:59 AM, Jim Bell wrote:
>>>>> The standard python asyncio tests hang.
>>>>>
>>>>>         cd /usr/lib/python3.8/test
>>>>>
>>>>>         python3.8 test_asyncore.py -v
>>>>>
> 
>> As for the test script errors you reported, I can submit a workaround patch that 
>> would make its way into the Python3.8 tests.  I am unsure at the moment which 
>> Cygwin package contains those tests but I can figure that out.  Otherwise, I 
>> could tell you what needs to be patched in test_asyncore.py and you could edit 
>> the script yourself to fix this locally for yourself.
>>
>> Let me know how you'd like to proceed, when you have a chance.
> 
> test_asyncore.py is part of the test suite of python itself.
> I doubt patching the test will change the functionality of the program itself

I agree.  I guess I was looking for some input from OP on what direction would be 
most useful to him.  If he'd like a workaround patch directly in order to make 
progress sooner on whatever direction he's going, I can provide it.  Otherwise I 
would engage with the Python maintainer /* pause, looks up maintainer, oh Hi 
Marco! */ to get the patch into Cygwin's Python package(s) for future release in 
due course.
Regards,

..mark