python fails asyncio tests (py 3.7 & 3.8)

python fails asyncio tests (py 3.7 & 3.8)

[Mark Geisert]

Hi Marco,
Below is the patch I developed to work around the problem report in
https://cygwin.com/pipermail/cygwin/2020-November/246830.html
I called the patch file 3.8.3-peercred-cygwin.patch.

I am unable to test the patch myself because of continuing problems building a new 
Python.  I don't know if my issues are due to being on latest Cygwin code vs 
3.1.7, or gcc 10.2 vs 9.3, or what.  Could you tell me what your build environment 
is like?  I'll try to duplicate it.

Test the patch by running a Python built with it on the example from the OP. 
Without the patch, the run would hang in the middle of the test script.  With the 
patch, it should quickly complete with 4 unrelated errors mentioning MSG_OOB.
Thanks & Regards,

..mark

--- origsrc/Python-3.8.3/Modules/socketmodule.c 2020-05-13 10:31:54.000000000-0700
+++ src/Python-3.8.3/Modules/socketmodule.c     2020-12-15 21:00:15.373059900-0800
@@ -1030,6 +1030,14 @@ init_sockobject(PySocketSockObject *s,
              }
          }
      }
+#ifdef __CYGWIN__
+    /* Temporarily work around AF_UNIX credential passing issues */
+    if (s->sock_family == AF_UNIX && s->sock_fd != -1) {
+        if (setsockopt(s->sock_fd, SOL_SOCKET, SO_PEERCRED, 0, 0) == -1) {
+            return -1;
+        }
+    }
+#endif
      return 0;
  }

Re: python fails asyncio tests (py 3.7 & 3.8)

[Marco Atzeri]

On 17.12.2020 10:20, Mark Geisert wrote:
> Hi Marco,
> Below is the patch I developed to work around the problem report in
> https://cygwin.com/pipermail/cygwin/2020-November/246830.html
> I called the patch file 3.8.3-peercred-cygwin.patch.
> 
> I am unable to test the patch myself because of continuing problems 
> building a new Python.  I don't know if my issues are due to being on 
> latest Cygwin code vs 3.1.7, or gcc 10.2 vs 9.3, or what.  Could you 
> tell me what your build environment is like?  I'll try to duplicate it.
> 
> Test the patch by running a Python built with it on the example from the 
> OP. Without the patch, the run would hang in the middle of the test 
> script.  With the patch, it should quickly complete with 4 unrelated 
> errors mentioning MSG_OOB.
> Thanks & Regards,
> 
> ..mark
> 

Hi Mark,
is this the expected result ?

test_connection_attributes (__main__.TestAPI_UseUnixSocketsPoll) ... ERROR
/usr/lib/python3.8/unittest/case.py:704: ResourceWarning: unclosed 
<socket.socket fd=5, family=AddressFamily.AF_UNIX, 
type=SocketKind.SOCK_STREAM, proto=0>
   outcome.errors.clear()

It does not freeze

Re: python fails asyncio tests (py 3.7 & 3.8)

[Mark Geisert]

Hi Marco,

On Mon, 28 Dec 2020, Marco Atzeri via Cygwin-apps wrote:
> On 17.12.2020 10:20, Mark Geisert wrote:
>> Hi Marco,
>> Below is the patch I developed to work around the problem report in
>> https://cygwin.com/pipermail/cygwin/2020-November/246830.html
>> I called the patch file 3.8.3-peercred-cygwin.patch.
>> 
>> I am unable to test the patch myself because of continuing problems 
>> building a new Python.  I don't know if my issues are due to being on 
>> latest Cygwin code vs 3.1.7, or gcc 10.2 vs 9.3, or what.  Could you tell 
>> me what your build environment is like?  I'll try to duplicate it.
>> 
>> Test the patch by running a Python built with it on the example from the 
>> OP. Without the patch, the run would hang in the middle of the test 
>> script.  With the patch, it should quickly complete with 4 unrelated 
>> errors mentioning MSG_OOB.
>> Thanks & Regards,
>> 
>> ..mark
>> 
>
> Hi Mark,
> is this the expected result ?
>
> test_connection_attributes (__main__.TestAPI_UseUnixSocketsPoll) ... ERROR
> /usr/lib/python3.8/unittest/case.py:704: ResourceWarning: unclosed 
> <socket.socket fd=5, family=AddressFamily.AF_UNIX, 
> type=SocketKind.SOCK_STREAM, proto=0>
>  outcome.errors.clear()
>
> It does not freeze

That's a new error to me; I haven't run that test.  I could have been more 
specific in my test instructions.  There are apparently two test series 
(which is also news to me).  They are /usr/lib/python3.8/test and 
.../unittest.  It seems you were in the latter?  The script to run is in 
.../test.  Here's how:
     cd /usr/lib/python3.8/test
     python3.8 test_asyncore.py -v

Separately, I'm still wrestling with build issues.  Just as a known-good 
alternative, how is your test environment set up?  Is your cygwin1.dll 
from standard 3.1.7, or a snapshot, or do you build from git master?  Are 
you using the latest binutils and gcc-g++ packages or something newer?

Thanks for any info you can provide.  I seem to be having issues with 
linking programs having many object files.  Like any Python 3, or the 
Flint math library for examples.  The link fails with a SIGSEGV or an 
assertion failure in cofflink.c.  Nobody else has reported these.
Thanks & Regards,

..mark

Re: python fails asyncio tests (py 3.7 & 3.8)

[Marco Atzeri]

On 29.12.2020 01:08, Mark Geisert wrote:
> Hi Marco,
> 
> On Mon, 28 Dec 2020, Marco Atzeri via Cygwin-apps wrote:
>> On 17.12.2020 10:20, Mark Geisert wrote:
>>> Hi Marco,
>>> Below is the patch I developed to work around the problem report in
>>> https://cygwin.com/pipermail/cygwin/2020-November/246830.html
>>> I called the patch file 3.8.3-peercred-cygwin.patch.
>>>
>>> I am unable to test the patch myself because of continuing problems 
>>> building a new Python.  I don't know if my issues are due to being on 
>>> latest Cygwin code vs 3.1.7, or gcc 10.2 vs 9.3, or what.  Could you 
>>> tell me what your build environment is like?  I'll try to duplicate it.
>>>
>>> Test the patch by running a Python built with it on the example from 
>>> the OP. Without the patch, the run would hang in the middle of the 
>>> test script.  With the patch, it should quickly complete with 4 
>>> unrelated errors mentioning MSG_OOB.
>>> Thanks & Regards,
>>>
>>> ..mark
>>>
>>
>> Hi Mark,
>> is this the expected result ?
>>
>> test_connection_attributes (__main__.TestAPI_UseUnixSocketsPoll) ... 
>> ERROR
>> /usr/lib/python3.8/unittest/case.py:704: ResourceWarning: unclosed 
>> <socket.socket fd=5, family=AddressFamily.AF_UNIX, 
>> type=SocketKind.SOCK_STREAM, proto=0>
>>  outcome.errors.clear()
>>
>> It does not freeze
> 
> That's a new error to me; I haven't run that test.  I could have been 
> more specific in my test instructions.  There are apparently two test 
> series (which is also news to me).  They are /usr/lib/python3.8/test and 
> .../unittest.  It seems you were in the latter?  The script to run is in 
> .../test.  Here's how:
>      cd /usr/lib/python3.8/test
>      python3.8 test_asyncore.py -v
> <

I run this, but I am building 3.8.6
I see also other problem around so I am rebuiling without your patch
to see if this problem is due to the patch or to other

> Separately, I'm still wrestling with build issues.  Just as a known-good 
> alternative, how is your test environment set up?  Is your cygwin1.dll 
> from standard 3.1.7, or a snapshot, or do you build from git master?  
> Are you using the latest binutils and gcc-g++ packages or something newer?

standard 3.1.7, latest cygwin packages

> Thanks for any info you can provide.  I seem to be having issues with 
> linking programs having many object files.  Like any Python 3, or the 
> Flint math library for examples.  The link fails with a SIGSEGV or an 
> assertion failure in cofflink.c.  Nobody else has reported these.
> Thanks & Regards,
> 
> ..mark

Re: python fails asyncio tests (py 3.7 & 3.8)

[Marco Atzeri]

On 29.12.2020 06:41, Marco Atzeri wrote:
> On 29.12.2020 01:08, Mark Geisert wrote:
>> Hi Marco,
>>

>>> Hi Mark,
>>> is this the expected result ?
>>>
>>> test_connection_attributes (__main__.TestAPI_UseUnixSocketsPoll) ... 
>>> ERROR
>>> /usr/lib/python3.8/unittest/case.py:704: ResourceWarning: unclosed 
>>> <socket.socket fd=5, family=AddressFamily.AF_UNIX, 
>>> type=SocketKind.SOCK_STREAM, proto=0>
>>>  outcome.errors.clear()
>>>
>>> It does not freeze
>>
>> That's a new error to me; I haven't run that test.  I could have been 
>> more specific in my test instructions.  There are apparently two test 
>> series (which is also news to me).  They are /usr/lib/python3.8/test 
>> and .../unittest.  It seems you were in the latter?  The script to run 
>> is in .../test.  Here's how:
>>      cd /usr/lib/python3.8/test
>>      python3.8 test_asyncore.py -v
>> <
> 
> I run this, but I am building 3.8.6
> I see also other problem around so I am rebuiling without your patch
> to see if this problem is due to the patch or to other
> 

the other problems seem not related to your patch.
I was hoping to release all updated python packages, but the
current problem is blocking

I need to check what is causing this issue :

   File 
"/usr/lib/python3.6/site-packages/pip/_vendor/urllib3/connection.py", 
line 71, in <modul
     class HTTPConnection(_HTTPConnection, object):
   File 
"/usr/lib/python3.6/site-packages/pip/_vendor/urllib3/connection.py", 
line 99, in HTTPCo
     default_socket_options = [(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)]
AttributeError: module 'socket' has no attribute 'TCP_NODELAY'
*** ERROR: pip3.6 install failed