Python regression related to unix sockets

Python regression related to unix sockets

[Fabian Henze]

Hi Cygwin users,
I noticed a regression in the recent Python packages: Between version
3.6.10-1 and 3.6.12-2, accessing the SSH agent via the paramiko python
package broke. When a ssh agent is used, paramiko tries to connect to
it via the unix socket and just freezes. Python 3.8 is also affected,
but I don't know which was the last working one.
I was able to pinpoint that error to 3.6.12-socketmodule.patch [1].
Reverting/removing the patch fixes the ssh agent access.
A few weeks ago there was a discussion regarding "Problems with native
Unix domain sockets on Win 10/2019", but it seems unrelated as the
cygwin1.dll from [2] does not work for me.

Can you please check if you are able to reproduce that? I uploaded a
script [3] for that.

Best regards,
Fabian Henze

[1] https://bugs.python.org/file49717/3.6.12-socketmodule.patch
[2] https://cygwin.com/snapshots/x86_64/cygwin1-20210201.dll.xz
[3] https://gist.github.com/henzef/4e553fad2335227b8f6b4550cd3fa543

--
Fabian Henze | Software Engineer | Bridgestone Mobility Solutions / Webfleet Solutions | webfleet.com
Webfleet Solutions Development Germany GmbH | Inselstraße 22, D-04103 Leipzig, Germany | Amtsgericht Leipzig | HRB 22234 | Geschäftsführer: Thomas Schmidt, Michiel Wesseling

This email message and any attachments are for the sole use of the
intended recipient(s) and may contain information that is proprietary
to Bridgestone Europe NV/SA and/or its affiliates, including Webfleet
Solutions, or otherwise confidential or legally privileged. If you have
received this message in error, please notify the sender by reply, and
delete all copies of this message and any attachments. If you are the
intended recipient you may use the information contained in this
message and any files attached to this message only as authorised by
Webfleet Solutions and/or Bridgestone. Any unauthorised use,
dissemination or disclosure of this message or its attachments is
strictly prohibited. The contents of this e-mail are not to be
considered binding upon Webfleet Solutions and/or Bridgestone.

Re: Python regression related to unix sockets

[Mark Geisert]

Hi Fabian,

Fabian Henze via Cygwin wrote:
> Hi Cygwin users,
> I noticed a regression in the recent Python packages: Between version
> 3.6.10-1 and 3.6.12-2, accessing the SSH agent via the paramiko python
> package broke. When a ssh agent is used, paramiko tries to connect to
> it via the unix socket and just freezes. Python 3.8 is also affected,
> but I don't know which was the last working one.
> I was able to pinpoint that error to 3.6.12-socketmodule.patch [1].
> Reverting/removing the patch fixes the ssh agent access.
> A few weeks ago there was a discussion regarding "Problems with native
> Unix domain sockets on Win 10/2019", but it seems unrelated as the
> cygwin1.dll from [2] does not work for me.
> 
> Can you please check if you are able to reproduce that? I uploaded a
> script [3] for that.
> 
> Best regards,
> Fabian Henze
> 
> [1] https://bugs.python.org/file49717/3.6.12-socketmodule.patch
> [2] https://cygwin.com/snapshots/x86_64/cygwin1-20210201.dll.xz
> [3] https://gist.github.com/henzef/4e553fad2335227b8f6b4550cd3fa543

Thanks for the report and sorry you've run into this.  The patch is a workaround 
that's evidently too draconian.  I will debug the situation using your testcase. 
Thank you very much for the testcase.

..mark

Re: Python regression related to unix sockets

[Mark Geisert]

Following up to myself:
Mark Geisert wrote:
> Hi Fabian,
> 
> Fabian Henze via Cygwin wrote:
>> Hi Cygwin users,
>> I noticed a regression in the recent Python packages: Between version
>> 3.6.10-1 and 3.6.12-2, accessing the SSH agent via the paramiko python
>> package broke. When a ssh agent is used, paramiko tries to connect to
>> it via the unix socket and just freezes. Python 3.8 is also affected,
>> but I don't know which was the last working one.
>> I was able to pinpoint that error to 3.6.12-socketmodule.patch [1].
>> Reverting/removing the patch fixes the ssh agent access.
>> A few weeks ago there was a discussion regarding "Problems with native
>> Unix domain sockets on Win 10/2019", but it seems unrelated as the
>> cygwin1.dll from [2] does not work for me.
>>
>> Can you please check if you are able to reproduce that? I uploaded a
>> script [3] for that.
>>
>> Best regards,
>> Fabian Henze
>>
>> [1] https://bugs.python.org/file49717/3.6.12-socketmodule.patch
>> [2] https://cygwin.com/snapshots/x86_64/cygwin1-20210201.dll.xz
>> [3] https://gist.github.com/henzef/4e553fad2335227b8f6b4550cd3fa543
> 
> Thanks for the report and sorry you've run into this.  The patch is a workaround 
> that's evidently too draconian.  I will debug the situation using your testcase. 
> Thank you very much for the testcase.

It's a limitation of the patch.  It was meant to allow Python programs on the same 
computer to communicate via AF_UNIX sockets, by working around less than perfect 
support in the Cygwin DLL.  But you've got a Python program trying to communicate 
with a non-Python program, and that fails because the patch is only applied on the 
Python end of the connection.

If you can continue to run without the problematic patch, that's your best option. 
  I will learn more about the underlying code in the Cygwin DLL and see if I can 
make an improvement there so the Python patch will not be needed in the future.
Thanks again for the report!

..mark

Re: Python regression related to unix sockets

[Fabian Henze]

Hi all,

This is a follow-up on a mail thread from February 2021.

Mark Geisert via Cygwin wrote:
> Fabian Henze via Cygwin wrote:
> > Hi Cygwin users,
> > I noticed a regression in the recent Python packages: Between version
> > 3.6.10-1 and 3.6.12-2, accessing the SSH agent via the paramiko python 
> > package broke. When a ssh agent is used, paramiko tries to connect to 
> > it via the unix socket and just freezes. Python 3.8 is also affected, 
> > but I don't know which was the last working one.
> > I was able to pinpoint that error to 3.6.12-socketmodule.patch [1].
> > Reverting/removing the patch fixes the ssh agent access.
> > A few weeks ago there was a discussion regarding "Problems with native 
> > Unix domain sockets on Win 10/2019", but it seems unrelated as the 
> > cygwin1.dll from [2] does not work for me.
> > 
> > Can you please check if you are able to reproduce that? I uploaded a 
> > script [3] for that.
> > 
> > Best regards,
> > Fabian Henze
> 
> Thanks for the report and sorry you've run into this.  The patch is a workaround that's evidently too draconian.  I will debug the situation using your testcase. 
> Thank you very much for the testcase.

Did you (or anyone else) make any progress on this that you can share?
Unfortunately the issue still persists in all current python versions.
If no one has time to debug and fix this, would it be an option to remove the
3.6.12-socketmodule.patch that is causing the regression since it only fixes a
theoretical problem in unit tests, but causes real-world issues? Maybe the
upcoming (?) Python 3.10 or 3.11 bump would be a good candidate to do so?

Best regards,
Fabian Henze

Re: Python regression related to unix sockets

[Mark Geisert]

Hi Fabian,

Fabian Henze wrote:
> Hi all,
> 
> This is a follow-up on a mail thread from February 2021.
> 
> Mark Geisert via Cygwin wrote:
>> Fabian Henze via Cygwin wrote:
>>> Hi Cygwin users,
>>> I noticed a regression in the recent Python packages: Between version
>>> 3.6.10-1 and 3.6.12-2, accessing the SSH agent via the paramiko python
>>> package broke. When a ssh agent is used, paramiko tries to connect to
>>> it via the unix socket and just freezes. Python 3.8 is also affected,
>>> but I don't know which was the last working one.
>>> I was able to pinpoint that error to 3.6.12-socketmodule.patch [1].
>>> Reverting/removing the patch fixes the ssh agent access.
>>> A few weeks ago there was a discussion regarding "Problems with native
>>> Unix domain sockets on Win 10/2019", but it seems unrelated as the
>>> cygwin1.dll from [2] does not work for me.
>>>
>>> Can you please check if you are able to reproduce that? I uploaded a
>>> script [3] for that.
>>>
>>> Best regards,
>>> Fabian Henze
>>
>> Thanks for the report and sorry you've run into this.  The patch is a workaround that's evidently too draconian.  I will debug the situation using your testcase.
>> Thank you very much for the testcase.
> 
> Did you (or anyone else) make any progress on this that you can share?
> Unfortunately the issue still persists in all current python versions.
> If no one has time to debug and fix this, would it be an option to remove the
> 3.6.12-socketmodule.patch that is causing the regression since it only fixes a
> theoretical problem in unit tests, but causes real-world issues? Maybe the
> upcoming (?) Python 3.10 or 3.11 bump would be a good candidate to do so?

Sorry for delay in responding; I've just returned from vacation.

Let me find my notes for this issue and see if I can improve on the patch or find 
some other solution for now-current versions of Python and/or Cygwin DLL.

..mark

Re: Python regression related to unix sockets

[Mark Geisert]

Following up to myself...
Mark Geisert wrote:
> Hi Fabian,
> 
> Fabian Henze wrote:
>> Hi all,
>>
>> This is a follow-up on a mail thread from February 2021.
>>
>> Mark Geisert via Cygwin wrote:
>>> Fabian Henze via Cygwin wrote:
>>>> Hi Cygwin users,
>>>> I noticed a regression in the recent Python packages: Between version
>>>> 3.6.10-1 and 3.6.12-2, accessing the SSH agent via the paramiko python
>>>> package broke. When a ssh agent is used, paramiko tries to connect to
>>>> it via the unix socket and just freezes. Python 3.8 is also affected,
>>>> but I don't know which was the last working one.
>>>> I was able to pinpoint that error to 3.6.12-socketmodule.patch [1].
>>>> Reverting/removing the patch fixes the ssh agent access.
[...]
>>>> Can you please check if you are able to reproduce that? I uploaded a
>>>> script [3] for that.
>>>>
>>>> Best regards,
>>>> Fabian Henze
>>>
>>> Thanks for the report and sorry you've run into this.  The patch is a 
>>> workaround that's evidently too draconian.  I will debug the situation using 
>>> your testcase.
>>> Thank you very much for the testcase.
>>
>> Did you (or anyone else) make any progress on this that you can share?
>> Unfortunately the issue still persists in all current python versions.
>> If no one has time to debug and fix this, would it be an option to remove the
>> 3.6.12-socketmodule.patch that is causing the regression since it only fixes a
>> theoretical problem in unit tests, but causes real-world issues? Maybe the
>> upcoming (?) Python 3.10 or 3.11 bump would be a good candidate to do so?
[...]
> Let me find my notes for this issue and see if I can improve on the patch or find 
> some other solution for now-current versions of Python and/or Cygwin DLL.

Right. I determined there was nothing further I could do. The conclusion I stated 
in the thread you quoted from was this (and it still applies):
--------8<--------
It's a limitation of the patch.  It was meant to allow Python programs on the same
computer to communicate via AF_UNIX sockets, by working around less than perfect
support in the Cygwin DLL.  But you've got a Python program trying to communicate
with a non-Python program, and that fails because the patch is only applied on the
Python end of the connection.

If you can continue to run without the problematic patch, that's your best option.
-------->8--------

BTW, what you called "a theoretical problem in unit tests" was of great concern to 
the Cygwin Python maintainer at the time. We test our Python builds before 
releasing them.

The best "solution" I can think of ATM, and it's not great, is to have the 
behavior you want be the default and an environment variable such as 
PYTHON_NET_DISABLE_CREDENTIALS to choose the other behavior. I will consult the 
Cygwin Python maintainer to see if this is acceptable.

..mark