From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2102.outbound.protection.outlook.com [40.107.220.102]) by sourceware.org (Postfix) with ESMTPS id 3E3F7385800D for ; Sat, 30 Jan 2021 16:00:19 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 3E3F7385800D ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=f/XjFfPbzZh1bDuvTnEjLwEn88c/DMrgPlqbMtF45zwAlLwFWZK75ARFQVcCKRZh/SziIxLjvqiBaGd9KffkLHifohA1t/fo/7cp6Dxv1fWTGG4hXdZluzP0CgC6bCwddZdJP3DB/Wzgksr78WWPjc3/YicugLENyCB49/SLFkl4B1p5SvQA+Q/FhOss2psVELIux5dJ/xl6m0p1na95AnfS1CMTJTZ/bZudITBnLSReO0+4Ux0zGxjoCn9suLQC70kgi2F7Etgv1586lS6Pu20tpYtLwzJyayerdVjIgOfCdfYJrn3Ra1U6jF/qz8gF7KR+6Bmqv5Pl4HhomXn3bg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fRMe9FOdsMRCK0WNn8Rr5FDJKBwaNij2tBV4wcMyj/4=; b=nR1qw0Q9gk7z+TdQmU8s3YwGQuzbylzTcYeoRQ2ZkchK/GUIRWLPSSk7XT8Xb6vEm0bNj39r/O3WmHMKH9BqHSveCG2+TbrWQ8ANjplp8Un8k433qLGc1yl5aLoc3Yhcii0oNcQ6MKZ4OAK8oV17gxw8eeTI/rRRqym2A8IUyE4t55OpiWUV8FIcJoVd/U26wqDM4MG0C1hn3BgbHN6gW9y0Y5+0zjtgLtAkSi7KC8ls4KDslBTYUqdx2hcUSggjMFmPtV5ZZALDyRsfhMK6blOmRrKXs7EvNLyUAWK9gKgCWnCfxopCjY0o7TFb2PM4xceT6LnT48hySTwXW9JgmQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=cornell.edu; dmarc=pass action=none header.from=cornell.edu; dkim=pass header.d=cornell.edu; arc=none Received: from BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) by BN7PR04MB3954.namprd04.prod.outlook.com (2603:10b6:406:be::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3805.16; Sat, 30 Jan 2021 16:00:15 +0000 Received: from BN7PR04MB4388.namprd04.prod.outlook.com ([fe80::f071:e174:ef12:375c]) by BN7PR04MB4388.namprd04.prod.outlook.com ([fe80::f071:e174:ef12:375c%6]) with mapi id 15.20.3805.023; Sat, 30 Jan 2021 16:00:15 +0000 Subject: Re: Problems with native Unix domain sockets on Win 10/2019 To: Michael McMahon , cygwin@cygwin.com References: <2b0aeab4-983d-e1d7-301f-edfeeb38cc85@oracle.com> <97d2b3af-224a-6873-fb4a-55a0ae9cd379@cornell.edu> <3e3cfe17-7fda-b063-4885-9114db9e748d@cornell.edu> <70b5577f-2cf1-0110-5d3b-cb2bd8ee6df2@cornell.edu> <69ad720c-8ea6-d3bb-b0a5-5556c4550091@oracle.com> <2d85550f-d753-4055-8b93-35e5287a9a93@oracle.com> From: Ken Brown Message-ID: Date: Sat, 30 Jan 2021 11:00:13 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.7.0 In-Reply-To: <2d85550f-d753-4055-8b93-35e5287a9a93@oracle.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Originating-IP: [65.112.130.200] X-ClientProxiedBy: BN9PR03CA1000.namprd03.prod.outlook.com (2603:10b6:408:10a::15) To BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from [10.13.22.4] (65.112.130.200) by BN9PR03CA1000.namprd03.prod.outlook.com (2603:10b6:408:10a::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3805.17 via Frontend Transport; Sat, 30 Jan 2021 16:00:14 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f83264a6-f859-416e-3dad-08d8c53821db X-MS-TrafficTypeDiagnostic: BN7PR04MB3954: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: bTWMae3pJj5E4P/M0spf0Uz3sk9ntKf6SGA0AzgpKe7dE67WiNs3JzfDOqh0CKtPIDuc6+X5YwSU91WtXZkCXcRAbNN43RQBAndCCvOBj2CfZIZI3VhlPjmDLg/UEfB4eBuq0hPJ1JvHPgsX/8N1oj6vK/QNxUBiOidF6Mf8Odx3B8I5OgIf6nv3h9Gj6tdm13rNxryszuG5+wIx1ZzTLtQ+uPjUPiMWpekh1BoJCfr1vfD0GlHWoeDVvjGTCa14QN8E91RnW7k2YyT90/UZq2bT9q2Y8algYasmUGzQsIC/JW1z7k29w9rBlcKiceRXf/+b4IiN2Nwjqb/CknZMhjeBB9KBQSKM/DjO9wwUa8DvwTjA0TeGP6fVLWb0aq/kxEI0XnuG+jIkt4ilSYD+a9gaOG1ozVYKEjjxT7TP2N5kqlUumkWvr8b02mMbJ9WHPV2oUJ4xVdLscQQjzBFXvHMlLNC25JfCz5eGdKxIAHVU0NBTzHiU5Ja6mSLq5BfbKXVqsQ6qHqCZkZsXhE+i6cLqfyW1jcHsHwVArGPtvEEhTkZBN9STOKKY2ts4hj903EbRonw8ueNDgBX1raND1CLstvlY5WGG3bB9EXmpk0E8HWwhL+0RaiYn9xTgEPjZ9/1MOmhMzYD8OJiZ+stNy46rM22CmIYeeoM6fu1xx/HbX8ekfyrQV4ImV32tXoEmwNt0DTYgREKGdEI/D10BbcIISGop39aDw7HA7rgDVarjM7bdBUjtskAIlqDj3Xfi X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN7PR04MB4388.namprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(136003)(396003)(376002)(366004)(346002)(39860400002)(2616005)(5660300002)(26005)(66556008)(186003)(31696002)(31686004)(86362001)(75432002)(83380400001)(16526019)(66476007)(786003)(8676002)(966005)(53546011)(316002)(6486002)(66574015)(478600001)(2906002)(66946007)(956004)(52116002)(8936002)(16576012)(36756003)(45980500001)(43740500002)(460985005)(2480315003); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: =?Windows-1252?Q?QJJ2FYWOEOH8kmT6iSYOfwYREj8dDfcUnQAO4YeiqG2JhgQyS6tUzsj3?= =?Windows-1252?Q?b3Zn6/srQnvxqGthzA9wQGc1f1PHiAwEIjou40GxQQwJsAzpEgk/X0m/?= =?Windows-1252?Q?9nvxnzyXm/1puMVDFYI+q328kM7v2ZO8EK1Hz5cLU/qN7mn+ACN+sxLI?= =?Windows-1252?Q?ATG4iq09tjWDaFc8Faf0VE3wU6Xn513+jtCVk83nh08SM+TGG+k4fYDj?= =?Windows-1252?Q?cjzSZihw82M9WXvOPqoRDnI7s65240mSCTponTuSNQBqW9m/CawTYUX+?= =?Windows-1252?Q?YblyhwG3S89trxE+Cg/QJU9nOPWu8KtYGwGSa+LfkY8DMLc70VhCsXpj?= =?Windows-1252?Q?mZyxr6bfO0VPbCBgho4fBTcTmi1d68GF/giXLpNt3+Ya/0EzemzapjTz?= =?Windows-1252?Q?9BTbgVKJb/XVj8GBjmxN9KZbpuhHPaVOD6QisESajKgN7DvYuri94gAP?= =?Windows-1252?Q?mczIHu7yXBF8HR9SUTWzFRxEjsvpyzlluYcqUF7Xg9RQMxLLx+OUh1NZ?= =?Windows-1252?Q?BHDSqgQc2eJ0niNgx9qy9E6RCfs25DiC1qyr0sleoQFyRMlEc0crd8bF?= =?Windows-1252?Q?ryuksh4YNNzkw0KZbtEQgOZUi/PW+hrCIh5VUg1MLHsW7nQC8WdqcgPa?= =?Windows-1252?Q?iiJkLimgixH2fcIvgaTYuZ7YR7B8g4Yj+FiH0L7k1r3KSf4wrL7P5cRZ?= =?Windows-1252?Q?WBNyqrHaiWtYgKQRYnfT9I8DsaV1bn0Bng9ex9P41rS3r00pnHwA3AGU?= =?Windows-1252?Q?tdeUsJLwQbAlFilnNTC563ozXDS8HMttkhs1pk5gjI2PF09Owm35fKfA?= =?Windows-1252?Q?bo8/GevQq9vEcsmPOGn9nL9nsWDRdKEilgSe3sqRvbh5hsnEIxqWN5Ut?= =?Windows-1252?Q?CFz98F6A3TJRmGtVHlhItmrPRiprUlUKbBAlnvf3N01LoXqRdgcTecqA?= =?Windows-1252?Q?V0vU88N8Nd4mBQ1Zur+2nVDjtdmSZIobpji6+DMiC9gwyPWnp4q9aQkF?= =?Windows-1252?Q?cwIhFfQBz0Zhnkb23kUmZ55wUwjHxLa0wnWAufCRurraaalXUuNEjaYB?= =?Windows-1252?Q?j39/adLeKx0371CaPT/g3wUapylZx/gUpExbAzRnbuGzrHYcKlNAe9v6?= =?Windows-1252?Q?NDZk5recWbnbsZFLnQxkV0Ss25QatDnqpGmMBFkw3dWCHw=3D=3D?= X-OriginatorOrg: cornell.edu X-MS-Exchange-CrossTenant-Network-Message-Id: f83264a6-f859-416e-3dad-08d8c53821db X-MS-Exchange-CrossTenant-AuthSource: BN7PR04MB4388.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jan 2021 16:00:15.1271 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 5d7e4366-1b9b-45cf-8e79-b14b27df46e1 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: gz07eUddhJ4rHKA1gV0CqPC+z2PBAN2olihyj4pj57egnKri3EhJeHgSHyqOoxVLC/DmlymOkQ5Iz1kwT82bUA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR04MB3954 X-Spam-Status: No, score=-3.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, MSGID_FROM_MTA_HEADER, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: cygwin@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: General Cygwin discussions and problem reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Jan 2021 16:00:22 -0000 On 9/28/2020 7:03 AM, Michael McMahon wrote: > > > On 26/09/2020 08:30, Michael McMahon via Cygwin wrote: >> >> >> On 25/09/2020 21:30, Ken Brown wrote: >>> On 9/25/2020 2:50 PM, Ken Brown via Cygwin wrote: >>>> On 9/25/2020 10:29 AM, Michael McMahon wrote: >>>>> >>>>> >>>>> On 25/09/2020 14:19, Ken Brown wrote: >>>>>> On 9/24/2020 8:01 AM, Michael McMahon wrote: >>>>>>> >>>>>>> >>>>>>> On 24/09/2020 12:26, Ken Brown wrote: >>>>>>>> On 9/23/2020 7:25 AM, Michael McMahon via Cygwin wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I searched for related issues but haven't found anything. >>>>>>>>> >>>>>>>>> I am having some trouble with Windows native Unix domain >>>>>>>>> sockets (a recent feature in Windows 10 and 2019 server) and >>>>>>>>> Cygwin. I think I possibly know the cause since I had to >>>>>>>>> investigate a similar looking issue on another platform built >>>>>>>>> on Windows. >>>>>>>>> >>>>>>>>> The problem is that cygwin commands don't seem to recognise >>>>>>>>> native Unix domain sockets correctly. For example, the socket >>>>>>>>> "foo.sock" should have the same ownership and similar >>>>>>>>> permissions to other files in the example below: >>>>>>>>> >>>>>>>>> $ ls -lrt total 2181303 >>>>>>>>> >>>>>>>>> -rw-r--r-- 1 mimcmah None 1259 Sep 23 >>>>>>>>> 10:22 test.c -rwxr-xr-x 1 mimcmah None 3680 >>>>>>>>> Sep 23 10:22 test.obj -rwxr-xr-x 1 mimcmah None >>>>>>>>> 121344 Sep 23 10:22 test.exe -rw-r----- 1 Unknown+User >>>>>>>>> Unknown+Group 0 Sep 23 10:23 foo.sock -rw-r--r-- 1 >>>>>>>>> mimcmah None 144356 Sep 23 10:27 check.ot >>>>>>>>> >>>>>>>>> A bigger problem is that foo.sock can't be deleted with the >>>>>>>>> cygwin "rm" command. >>>>>>>>> >>>>>>>>> $ rm -f foo.sock rm: cannot remove 'foo.sock': Permission >>>>>>>>> denied >>>>>>>>> >>>>>>>>> $ chmod 777 foo.sock chmod: changing permissions of >>>>>>>>> 'foo.sock': Permission denied >>>>>>>>> >>>>>>>>> $ cmd /c del foo.sock >>>>>>>>> >>>>>>>>> But, native Windows commands are okay, as the third example >>>>>>>>> shows. >>>>>>>>> >>>>>>>>> I think the problem may relate to the way native Unix domain >>>>>>>>> sockets are implemented in Windows and the resulting special >>>>>>>>> handling required. They are implemented as NTFS reparse >>>>>>>>> points and when opening them with CreateFile, you need to >>>>>>>>> specify the FILE_FLAG_OPEN_REPARSE_POINT flag. Otherwise, you >>>>>>>>> get an ERROR_CANT_ACCESS_FILE. There are other complications >>>>>>>>> unfortunately, which I'd be happy to discuss further. >>>>>>>>> >>>>>>>>> But, to reproduce it, you can compile the attached code >>>>>>>>> snippet which creates foo.sock in the current directory. >>>>>>>>> Obviously, this only works on recent versions of Windows 10 >>>>>>>>> and 2019 server. >>>>>>>> >>>>>>>> Cygwin doesn't currently support native Windows AF_UNIX >>>>>>>> sockets, as you've discovered. See >>>>>>>> >>>>>>>> https://urldefense.com/v3/__https://cygwin.com/pipermail/cygwin/2020-June/245088.html__;!!GqivPVa7Brio!P7lIFI4rYAtWh8_DtCbRCxT-M_E4vwQ0qwzQ0p656T73BpJ0jbUkLI_bXdA6mmSL9lJcSQ$ >>>>>>>> >>>>>>>> >>>>>>>> for the current state of AF_UNIX sockets on Cygwin, including >>>>>>>> the possibility of using native Windows AF_UNIX sockets on >>>>>>>> systems that support them. >>>>>>>> >>>>>>>> If all you want is for Cygwin to recognize such sockets and >>>>>>>> allow you to apply rm, chmod, etc., I don't think it would be >>>>>>>> hard to add that capability. But I doubt if that's all you >>>>>>>> want. >>>>>>>> >>>>>>>> Further discussion of this will have to wait until Corinna is >>>>>>>> available. >>>>>>>> >>>>>>> >>>>>>> Thanks for the info. It's mainly about recognition of sockets >>>>>>> for regular commands. Since these objects can exist on Windows >>>>>>> filesystems now, potentially created by any kind of Windows >>>>>>> application, it would be great if Cygwin could handle them, >>>>>>> irrespective of whether the Cygwin development environment does. >>>>>>> Though that sounds like a good idea too. >>>>>> >>>>>> I think this has a simple fix (attached), but I can't easily test >>>>>> it because your test program doesn't compile for me. First, I got >>>>>> >>>>>> $ gcc -o native_unix_socket native_unix_socket.c >>>>>> native_unix_socket.c:5:10: fatal error: WS2tcpip.h: No such file or >>>>>> directory 5 | #include | ^~~~~~~~~~~~ >>>>>> compilation terminated. >>>>>> >>>>>> I fixed this by making the include file name lower case. (My >>>>>> system is case sensitive, so it matters.) >>>>>> >>>>>> Next: >>>>>> >>>>>> $ gcc -o native_unix_socket native_unix_socket.c >>>>>> native_unix_socket.c:8:10: fatal error: afunix.h: No such file or >>>>>> directory 8 | #include | ^~~~~~~~~~ compilation >>>>>> terminated. >>>>>> >>>>>> There's no file afunix.h in the Cygwin distribution, but I located >>>>>> it online and pasted in the contents. The program now compiles but >>>>>> fails to link: >>>>>> >>>>>> $ gcc -o native_unix_socket native_unix_socket.c >>>>>> /usr/lib/gcc/x86_64-pc-cygwin/10/../../../../x86_64-pc-cygwin/bin/ld: >>>>>> /tmp/cc74urPr.o:native_unix_socket.c:(.text+0x3b): undefined >>>>>> reference to `__imp_WSAStartup' >>>>>> /tmp/cc74urPr.o:native_unix_socket.c:(.text+0x3b): relocation >>>>>> truncated to fit: R_X86_64_PC32 against undefined symbol >>>>>> `__imp_WSAStartup' >>>>>> /usr/lib/gcc/x86_64-pc-cygwin/10/../../../../x86_64-pc-cygwin/bin/ld: >>>>>> /tmp/cc74urPr.o:native_unix_socket.c:(.text+0xf2): undefined >>>>>> reference to `__imp_WSAGetLastError' >>>>>> /tmp/cc74urPr.o:native_unix_socket.c:(.text+0xf2): relocation >>>>>> truncated to fit: R_X86_64_PC32 against undefined symbol >>>>>> `__imp_WSAGetLastError' >>>>>> /usr/lib/gcc/x86_64-pc-cygwin/10/../../../../x86_64-pc-cygwin/bin/ld: >>>>>> /tmp/cc74urPr.o:native_unix_socket.c:(.text+0x13d): undefined >>>>>> reference to `__imp_WSAGetLastError' >>>>>> /tmp/cc74urPr.o:native_unix_socket.c:(.text+0x13d): relocation >>>>>> truncated to fit: R_X86_64_PC32 against undefined symbol >>>>>> `__imp_WSAGetLastError' collect2: error: ld returned 1 exit status >>>>>> >>>>>> This is probably easy to fix too, but I don't feel like tracking it >>>>>> down. Please send compilation instructions (that use Cygwin >>>>>> tools). >>>>>> >>>>>> Ken >>>>> >>>>> Hi >>>>> >>>>> Sorry, I had compiled it in a native Visual C environment. >>>>> >>>>> Assuming you have afunix.h in the current directory. >>>>> >>>>> gcc -o native_unix_socket -I. native_unix_socket.c -lws2_32 >>>>> >>>>> should do it. >>>> >>>> Thanks, that works. But now I can't reproduce your problem. Here's >>>> what I see, using Cygwin 3.1.7 without applying my patch: >>>> >>>> $ ./native_unix_socket.exe getsockname works fam = 1, len = 11 offsetof >>>> clen = 9 strlen = 8 name = foo.sock >>>> >>>> $ ls -l foo.sock -rwxr-xr-x 1 kbrown None 0 2020-09-25 14:39 foo.sock* >>>> >>>> $ chmod 644 foo.sock >>>> >>>> $ ls -l foo.sock -rw-r--r-- 1 kbrown None 0 2020-09-25 14:39 foo.sock >>>> >>>> $ rm foo.sock >>>> >>>> $ ls -l foo.sock ls: cannot access 'foo.sock': No such file or >>>> directory >>>> >>>> I'm running 64-bit Cygwin on Windows 10 1909. >>> >>> I just ran the 'rm' command under gdb to see what's going on, and it >>> seems that foo.sock is not being recognized as a reparse point. So maybe >>> your test program, when compiled and run under Cygwin, doesn't actually >>> produce a native Windows AF_UNIX socket. And when I try to run it in a >>> Windows Command Prompt, I get >>> >>> bind failed 10050 getsockname failed 10022 >>> >>> Can you make your version of the test executable available for me to try? >>> Or tell me some other way to create a native Windows AF_UNIX socket? >>> >>> Ken >> >> That is all very strange. I have checked both the gcc compiled and MS >> compiled executables on my system (2019 server) and they are both >> definitely producing native AF_UNIX sockets. >> >> I can email you the two exe files. They are both quite small. But, first I >> want to check the patch status of my test system. >> > > So, it turns out that this issue only happens on some of our test systems. It > does not happen on a personal copy of Windows 10 on my laptop either. > > I also noticed that some native Windows commands don't work properly on any > affected system (eg 'attrib' or 'fsutil'). Though 'fsutil' can be used to > verify that the reparse point is created correctly. > > Possibly, this was a Windows bug that has been fixed. It never made sense > that you had to open socket files using the FILE_FLAG_OPEN_REPARSE_POINT > flag, because you would have to know in advance that the file is a socket to > be able to do this (or else be prepared to have to open the file twice). But, > I don't fully understand yet, why some systems are affected and others not. > All seem to be patched up to date. > > In any case, I think it's clear this isn't a Cygwin issue. It turns out that this is a Cygwin issue after all. In a private message Michael has said: > From what I can see, the only versions that are *not* affected by the problem > are 1903 and 1909 (which you tested with). Versions I have tested with since > then (2004, and 20H2) all show the problem. I can't immediately test it myself because I'm still on 1909. But I'll send a patch to cygwin-patches that I think should fix it, along with Michael's test program. Ken