From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from aserp2130.oracle.com (aserp2130.oracle.com [141.146.126.79]) by sourceware.org (Postfix) with ESMTPS id EE3963857C7C for ; Mon, 28 Sep 2020 11:03:24 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org EE3963857C7C Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 08SAxi1q048642; Mon, 28 Sep 2020 11:03:17 GMT Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by aserp2130.oracle.com with ESMTP id 33su5ame63-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Mon, 28 Sep 2020 11:03:17 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 08SB0kPW157534; Mon, 28 Sep 2020 11:03:17 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserp3030.oracle.com with ESMTP id 33tf7k64nn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 28 Sep 2020 11:03:17 +0000 Received: from abhmp0018.oracle.com (abhmp0018.oracle.com [141.146.116.24]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id 08SB3FkQ004145; Mon, 28 Sep 2020 11:03:16 GMT Received: from dhcp-10-175-49-73.vpn.oracle.com (/10.175.49.73) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 28 Sep 2020 04:03:15 -0700 Subject: Re: Problems with native Unix domain sockets on Win 10/2019 To: Ken Brown , cygwin@cygwin.com References: <2b0aeab4-983d-e1d7-301f-edfeeb38cc85@oracle.com> <97d2b3af-224a-6873-fb4a-55a0ae9cd379@cornell.edu> <3e3cfe17-7fda-b063-4885-9114db9e748d@cornell.edu> <70b5577f-2cf1-0110-5d3b-cb2bd8ee6df2@cornell.edu> <69ad720c-8ea6-d3bb-b0a5-5556c4550091@oracle.com> From: Michael McMahon Message-ID: <2d85550f-d753-4055-8b93-35e5287a9a93@oracle.com> Date: Mon, 28 Sep 2020 12:03:13 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: <69ad720c-8ea6-d3bb-b0a5-5556c4550091@oracle.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9757 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 mlxlogscore=999 suspectscore=0 adultscore=0 malwarescore=0 phishscore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2009280088 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9757 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 suspectscore=0 lowpriorityscore=0 spamscore=0 clxscore=1015 mlxscore=0 impostorscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2009280088 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00, BODY_8BITS, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, NICE_REPLY_A, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL, SPF_HELO_PASS, SPF_PASS, TXREP, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: cygwin@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: General Cygwin discussions and problem reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Sep 2020 11:03:26 -0000 On 26/09/2020 08:30, Michael McMahon via Cygwin wrote: > > > On 25/09/2020 21:30, Ken Brown wrote: >> On 9/25/2020 2:50 PM, Ken Brown via Cygwin wrote: >>> On 9/25/2020 10:29 AM, Michael McMahon wrote: >>>> >>>> >>>> On 25/09/2020 14:19, Ken Brown wrote: >>>>> On 9/24/2020 8:01 AM, Michael McMahon wrote: >>>>>> >>>>>> >>>>>> On 24/09/2020 12:26, Ken Brown wrote: >>>>>>> On 9/23/2020 7:25 AM, Michael McMahon via Cygwin wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> I searched for related issues but haven't found anything. >>>>>>>> >>>>>>>> I am having some trouble with Windows native Unix domain sockets >>>>>>>> (a recent feature in Windows 10 and 2019 server) and Cygwin. >>>>>>>> I think I possibly know the cause since I had to investigate a >>>>>>>> similar >>>>>>>> looking issue on another platform built on Windows. >>>>>>>> >>>>>>>> The problem is that cygwin commands don't seem to recognise >>>>>>>> native Unix >>>>>>>> domain sockets correctly. For example, the socket "foo.sock" should >>>>>>>> have the same ownership and similar permissions to other files >>>>>>>> in the example below: >>>>>>>> >>>>>>>> $ ls -lrt >>>>>>>> total 2181303 >>>>>>>> >>>>>>>> -rw-r--r--  1 mimcmah      None             1259   Sep 23 10:22 >>>>>>>> test.c >>>>>>>> -rwxr-xr-x  1 mimcmah      None             3680   Sep 23 10:22 >>>>>>>> test.obj >>>>>>>> -rwxr-xr-x  1 mimcmah      None             121344 Sep 23 10:22 >>>>>>>> test.exe >>>>>>>> -rw-r-----  1 Unknown+User Unknown+Group         0 Sep 23 10:23 >>>>>>>> foo.sock >>>>>>>> -rw-r--r--  1 mimcmah      None             144356 Sep 23 10:27 >>>>>>>> check.ot >>>>>>>> >>>>>>>> A bigger problem is that foo.sock can't be deleted with the >>>>>>>> cygwin "rm" >>>>>>>> command. >>>>>>>> >>>>>>>> $ rm -f foo.sock >>>>>>>> rm: cannot remove 'foo.sock': Permission denied >>>>>>>> >>>>>>>> $ chmod 777 foo.sock >>>>>>>> chmod: changing permissions of 'foo.sock': Permission denied >>>>>>>> >>>>>>>> $ cmd /c del foo.sock >>>>>>>> >>>>>>>> But, native Windows commands are okay, as the third example shows. >>>>>>>> >>>>>>>> I think the problem may relate to the way native Unix domain >>>>>>>> sockets are >>>>>>>> implemented in Windows and the resulting special handling required. >>>>>>>> They are implemented as NTFS reparse points and when opening them >>>>>>>> with CreateFile, you need to specify the >>>>>>>> FILE_FLAG_OPEN_REPARSE_POINT >>>>>>>> flag. Otherwise, you get an ERROR_CANT_ACCESS_FILE. There are other >>>>>>>> complications unfortunately, which I'd be happy to discuss further. >>>>>>>> >>>>>>>> But, to reproduce it, you can compile the attached code snippet >>>>>>>> which creates foo.sock in the current directory. Obviously, this >>>>>>>> only works on recent versions of Windows 10 and 2019 server. >>>>>>> >>>>>>> Cygwin doesn't currently support native Windows AF_UNIX sockets, >>>>>>> as you've discovered.  See >>>>>>> >>>>>>> https://urldefense.com/v3/__https://cygwin.com/pipermail/cygwin/2020-June/245088.html__;!!GqivPVa7Brio!P7lIFI4rYAtWh8_DtCbRCxT-M_E4vwQ0qwzQ0p656T73BpJ0jbUkLI_bXdA6mmSL9lJcSQ$ >>>>>>> >>>>>>> for the current state of AF_UNIX sockets on Cygwin, including the >>>>>>> possibility of using native Windows AF_UNIX sockets on systems >>>>>>> that support them. >>>>>>> >>>>>>> If all you want is for Cygwin to recognize such sockets and allow >>>>>>> you to apply rm, chmod, etc., I don't think it would be hard to >>>>>>> add that capability.  But I doubt if that's all you want. >>>>>>> >>>>>>> Further discussion of this will have to wait until Corinna is >>>>>>> available. >>>>>>> >>>>>> >>>>>> Thanks for the info. It's mainly about recognition of sockets for >>>>>> regular commands. Since these objects can exist on Windows >>>>>> filesystems >>>>>> now, potentially created by any kind of Windows application, >>>>>> it would be great if Cygwin could handle them, irrespective of >>>>>> whether >>>>>> the Cygwin development environment does. Though that sounds like a >>>>>> good idea too. >>>>> >>>>> I think this has a simple fix (attached), but I can't easily test >>>>> it because your test program doesn't compile for me.  First, I got >>>>> >>>>> $ gcc -o native_unix_socket native_unix_socket.c >>>>> native_unix_socket.c:5:10: fatal error: WS2tcpip.h: No such file or >>>>> directory >>>>>      5 | #include >>>>>        |          ^~~~~~~~~~~~ >>>>> compilation terminated. >>>>> >>>>> I fixed this by making the include file name lower case.  (My >>>>> system is case sensitive, so it matters.) >>>>> >>>>> Next: >>>>> >>>>> $ gcc -o native_unix_socket native_unix_socket.c >>>>> native_unix_socket.c:8:10: fatal error: afunix.h: No such file or >>>>> directory >>>>>      8 | #include >>>>>        |          ^~~~~~~~~~ >>>>> compilation terminated. >>>>> >>>>> There's no file afunix.h in the Cygwin distribution, but I located >>>>> it online and pasted in the contents.  The program now compiles but >>>>> fails to link: >>>>> >>>>> $ gcc -o native_unix_socket native_unix_socket.c >>>>> /usr/lib/gcc/x86_64-pc-cygwin/10/../../../../x86_64-pc-cygwin/bin/ld: >>>>> /tmp/cc74urPr.o:native_unix_socket.c:(.text+0x3b): undefined >>>>> reference to `__imp_WSAStartup' >>>>> /tmp/cc74urPr.o:native_unix_socket.c:(.text+0x3b): relocation >>>>> truncated to fit: R_X86_64_PC32 against undefined symbol >>>>> `__imp_WSAStartup' >>>>> /usr/lib/gcc/x86_64-pc-cygwin/10/../../../../x86_64-pc-cygwin/bin/ld: >>>>> /tmp/cc74urPr.o:native_unix_socket.c:(.text+0xf2): undefined >>>>> reference to `__imp_WSAGetLastError' >>>>> /tmp/cc74urPr.o:native_unix_socket.c:(.text+0xf2): relocation >>>>> truncated to fit: R_X86_64_PC32 against undefined symbol >>>>> `__imp_WSAGetLastError' >>>>> /usr/lib/gcc/x86_64-pc-cygwin/10/../../../../x86_64-pc-cygwin/bin/ld: >>>>> /tmp/cc74urPr.o:native_unix_socket.c:(.text+0x13d): undefined >>>>> reference to `__imp_WSAGetLastError' >>>>> /tmp/cc74urPr.o:native_unix_socket.c:(.text+0x13d): relocation >>>>> truncated to fit: R_X86_64_PC32 against undefined symbol >>>>> `__imp_WSAGetLastError' >>>>> collect2: error: ld returned 1 exit status >>>>> >>>>> This is probably easy to fix too, but I don't feel like tracking it >>>>> down. Please send compilation instructions (that use Cygwin tools). >>>>> >>>>> Ken >>>> >>>> Hi >>>> >>>> Sorry, I had compiled it in a native Visual C environment. >>>> >>>> Assuming you have afunix.h in the current directory. >>>> >>>> gcc -o native_unix_socket -I. native_unix_socket.c -lws2_32 >>>> >>>> should do it. >>> >>> Thanks, that works.  But now I can't reproduce your problem.  Here's >>> what I see, using Cygwin 3.1.7 without applying my patch: >>> >>> $ ./native_unix_socket.exe >>> getsockname works >>> fam = 1, len = 11 >>> offsetof clen = 9 >>> strlen = 8 >>> name = foo.sock >>> >>> $ ls -l foo.sock >>> -rwxr-xr-x 1 kbrown None 0 2020-09-25 14:39 foo.sock* >>> >>> $ chmod 644 foo.sock >>> >>> $ ls -l foo.sock >>> -rw-r--r-- 1 kbrown None 0 2020-09-25 14:39 foo.sock >>> >>> $ rm foo.sock >>> >>> $ ls -l foo.sock >>> ls: cannot access 'foo.sock': No such file or directory >>> >>> I'm running 64-bit Cygwin on Windows 10 1909. >> >> I just ran the 'rm' command under gdb to see what's going on, and it >> seems that foo.sock is not being recognized as a reparse point.  So >> maybe your test program, when compiled and run under Cygwin, doesn't >> actually produce a native Windows AF_UNIX socket.  And when I try to >> run it in a Windows Command Prompt, I get >> >> bind failed 10050 >> getsockname failed 10022 >> >> Can you make your version of the test executable available for me to >> try?  Or tell me some other way to create a native Windows AF_UNIX >> socket? >> >> Ken > > That is all very strange. I have checked both the gcc compiled and MS > compiled executables on my system (2019 server) and they are both > definitely producing native AF_UNIX sockets. > > I can email you the two exe files. They are both quite small. But, first > I want to check the patch status of my test system. > So, it turns out that this issue only happens on some of our test systems. It does not happen on a personal copy of Windows 10 on my laptop either. I also noticed that some native Windows commands don't work properly on any affected system (eg 'attrib' or 'fsutil'). Though 'fsutil' can be used to verify that the reparse point is created correctly. Possibly, this was a Windows bug that has been fixed. It never made sense that you had to open socket files using the FILE_FLAG_OPEN_REPARSE_POINT flag, because you would have to know in advance that the file is a socket to be able to do this (or else be prepared to have to open the file twice). But, I don't fully understand yet, why some systems are affected and others not. All seem to be patched up to date. In any case, I think it's clear this isn't a Cygwin issue. So, apologies for the noise, and thanks for the assistance! Regards, Michael.