From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2113.outbound.protection.outlook.com [40.107.243.113]) by sourceware.org (Postfix) with ESMTPS id A9F1F396903B for ; Thu, 29 Apr 2021 16:44:53 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org A9F1F396903B Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=cornell.edu Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=kbrown@cornell.edu ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Ruz/MLZj1ZhVqIlBNYFv53NEoqvvnlEnUL7oRJLQeFqQhaKvVa1wpLh2Ljuwt1L1ldY8f9KpEM/QrpSrDUdJduhUh+lLGamRieZ7OiOeMJJmKWekXJfVEcub+peTg42QCGebjb9IEtVjeabUISuHRpxGEzJUyu8/p3J3iv6qko6yJUBkX3ARFaF/uXtw21OgQUs+tka+87cpLBf4Kjg3O/PC2qZxJ5idDLhQljgLGvNSv1YXRwsbckTNVg7Pxr01n9hIxVCYwec8xHiMV4J5d1TYErlEOfch9TDjbl10Wui4OQTX/3iy82ELrRny8nHRShjNAIIZsTkLyNPOtDgqxQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Gx2Je+gpRngex+IXgiq06xnxdLQtzmR3J1Dn8ZNrfbo=; b=G5fQ80s3BIY7WoqLqJLLD/RKm+9ggcFbFXisA/wTTLXXIwY567GleLq7DEi5lvrDTnx8uECNL3LY9jq+DDGEFc9sUuCiDUq9OVaDQRIiMQVKMlo4uxWErSAFtMpOmfqNbgAIf7RA+H0uAduEl06GvHvZ+cF4JhPBmKG1zvZj0SKJpMPRTermiuXt2y852K9V5OFMiLvnyvJrKAjYp1kQLsAiTtrg9t/n0D5O3ygxo86cZhBBdCXE/zkljt1jrM9TgSXmvBUzbcb2PV3yy9UG6TRWYb4zrEPr2p3poeueB/H2YTXPaecZHyT5e2KVlr66/hjk31z8IQ3O1Qazg3N9wQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=cornell.edu; dmarc=pass action=none header.from=cornell.edu; dkim=pass header.d=cornell.edu; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornell.edu; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Gx2Je+gpRngex+IXgiq06xnxdLQtzmR3J1Dn8ZNrfbo=; b=QXk4s+KzzvgnZtj2SSUoKQXGq8JFZm3AVMtc4Tb7mSJ06nFVZHa7jjo5QepyvwDZEcI6uoYIrWj+SwKWct5/UWGNHtbyLIDiWpp5GOwirOxm+8HDNgkwrQjJJwXLO1b604sDsnX5T6hgOLv3vSVLWwnTZQF76Q5UAfDwMXAlMYw= Authentication-Results: cygwin.com; dkim=none (message not signed) header.d=none;cygwin.com; dmarc=none action=none header.from=cornell.edu; Received: from BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) by BN7PR04MB4307.namprd04.prod.outlook.com (2603:10b6:406:ff::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.28; Thu, 29 Apr 2021 16:44:50 +0000 Received: from BN7PR04MB4388.namprd04.prod.outlook.com ([fe80::59f8:fcc4:f07e:9a89]) by BN7PR04MB4388.namprd04.prod.outlook.com ([fe80::59f8:fcc4:f07e:9a89%4]) with mapi id 15.20.4065.027; Thu, 29 Apr 2021 16:44:50 +0000 Subject: Re: The unreliability of AF_UNIX datagram sockets To: cygwin-developers@cygwin.com References: <58da34ac-f2b6-d8b2-e872-834cfcb1ab51@cornell.edu> <6cac30e5-56fc-5bf1-b85b-fe6b91bc5e97@cornell.edu> From: Ken Brown Message-ID: <16e1d55e-15ea-6c0e-04e4-aa6cb2c0c1bd@cornell.edu> Date: Thu, 29 Apr 2021 12:44:48 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.10.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Originating-IP: [65.112.130.200] X-ClientProxiedBy: MN2PR10CA0032.namprd10.prod.outlook.com (2603:10b6:208:120::45) To BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from [10.13.22.8] (65.112.130.200) by MN2PR10CA0032.namprd10.prod.outlook.com (2603:10b6:208:120::45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4087.25 via Frontend Transport; Thu, 29 Apr 2021 16:44:49 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 527a7390-087c-4205-ee4d-08d90b2e1b20 X-MS-TrafficTypeDiagnostic: BN7PR04MB4307: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:765; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: a6j0e2B79OpV09uoj2F7volzZIuVYK03NSdXNG9pmhxKpuw07rIirn6cJaK2MPWJzZI7IAshDWU+C84Y2j8+ZBzCU6pSGUWWstuqWd1XI7XsBEYIWucpiD3+PR+3kNpMCmBCCKhXdYuts8xsGD1cSY8q7WlASMAKtZLeZS+xdpgrQ1HcWERfyor3t3P6sty4NfTR78Az8yLHFy+kAQ0vB/6KB5Nh5pN33uLOiIvOwnfbzEjR8KBAQdoivORl/DZptWV7kABqNupDxLLhkAlm4hIY4cVUNW5pmdr21lJlonCZBSRYQ9HKCGI0QPQbcHY5kiG87NMGppqdFvbPPP2K3XuROgKw9Ll0Oe1LiZeGU1MokN0TIasiZj0WyUuqu/gJrwPqv4ywinvf1CiKguhPZ3i4ISDUNZOWTXpIDDnsYuDxkd28ep/n4WIcNEc9lY/gie6GSUjml6pKn5RFLSSyDCVKsRAbNzkZinaspMQKaKfh6P+uGJZPLpesPNEOPakQQoiF8301rPuzKoMpM/dt/P7AaTBuwFmnHT/7Z0Ejgle6A3Pp7KBq/mt9ZR+nA4USNJjUwaAGcQCh2IWewMpESRMgVVI79brW2v2CXDQRd98noBbkL2lxAob7wJJGX77HTZ0X3BGyjCaUAw2352ThqMenfn1JA/qkhr044sFpN3IScZKEX0jeqOGzoS7XIPZbww8GkFYJduNY5Zr1IM9xfOszlHZZxrsAUQr9VFZA3jscjytFEfAT4GOatQRDl2AlKg20ob7KUw7U6/2UlUCs6XdSDxxLRgxNCylIby35Izk= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN7PR04MB4388.namprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39860400002)(346002)(396003)(136003)(376002)(366004)(52116002)(6486002)(2906002)(45080400002)(8936002)(83380400001)(36756003)(86362001)(6916009)(31696002)(8676002)(966005)(66476007)(31686004)(5660300002)(478600001)(75432002)(16526019)(38350700002)(66556008)(66946007)(38100700002)(16576012)(186003)(316002)(26005)(786003)(2616005)(53546011)(956004)(45980500001)(43740500002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: =?Windows-1252?Q?lg8fm+PDJZK9csg4yt7S9ow/dJgCt2U/wYGc3gSqiFIkQB9ziSS8pGQF?= =?Windows-1252?Q?PP+HsZdR8aRypEMSKcPR2WDp3jpKV+kX6SCERNYLiMzWNpmv0QzdNCZ8?= =?Windows-1252?Q?h6RBwfUi/jZgbB22e5dLeqH8zP3zThb1SHLY7kUwwRrPNU5vpnH1Ennq?= =?Windows-1252?Q?TO7K3XAL1gK7kpVRqt/n3g3iHP75dVDAnpXr0WKZkBvcUvBPoUPHQXPU?= =?Windows-1252?Q?/nsiqEv1je03bWwIcrUaPuwcq6hf6SefofcVpDNVa2+9psJQc9tNA0Mn?= =?Windows-1252?Q?0SHNI/TB0wVYedotvW9PCQeULpNyHBbV07lI7fqE50fFNhxbSz7M134Y?= =?Windows-1252?Q?K99bsajqcwl72bgJf4QoFxzBYJKUwhC5XdIpAFbUPZzpv9R2+HiFSPMs?= =?Windows-1252?Q?UxA+03yFQlV4dBVCI9dCPyMmSEl5DyhaJOFnjeJQoyl5RCA4Wd3Jn9Ty?= =?Windows-1252?Q?ZRdF9lGpNEb4vCiAK2h2q2tSvNo7xEzjQ81nw3iCEvlxZSb0I1z2Ocnx?= =?Windows-1252?Q?sXmjXTdQ6CefeCElqYW2veAjgE6yS3IamjwTCX4IWbiJodZXlw+SjR6I?= =?Windows-1252?Q?ftIZKNLxIDELkbWJBTUPj02/p2rHdrFv97iwcT4wZ0wuiE4alRsC1Aj7?= =?Windows-1252?Q?AnbJUJpSqNSwibGcJWr4CEev1tGF7dEVTE4NKxnQJPiws+Ot0H7HVgqo?= =?Windows-1252?Q?Qg445ZsRZA1qwI0q4OE7mbr/s54IoaWn/MAOheS/lmtsjXtPIdIPQfzD?= =?Windows-1252?Q?sUlk6dYc9nccRqSayHsNKHUQU506Dt1SW5uuIq87r3/ITxJL7W5W12br?= =?Windows-1252?Q?ZKuQ6ijh/GEOvplTyPRcj65CQ6WYVJT0AqeLIP1HlcMAQ3PC13NUnNwa?= =?Windows-1252?Q?b1TeSNCD8nLJCGro8GTc6ZiOT+I9KrKamBt5ja4Wj8yxJH8xX8yoStgK?= =?Windows-1252?Q?GwW9nQFhTWTi7wc+oEf7/WJJkrV1Ip4MYLD3dILFLqlNUIBPWfdaJfYb?= =?Windows-1252?Q?rk2zyPgPJnRJaZwaZjJ/fbdVPKDj3qFSm+My/IH2KRJ/oJ3QmZhcV4+b?= =?Windows-1252?Q?h3Dt75+vsfH3U5ibfdq2IbFd+uKqexSdYGK4IsURgvVtDY2AeM+caG7d?= =?Windows-1252?Q?zrmDE/O8IIG6AxbeNVHavWOYevGSm8x/xQ7vgfdfMhihqJBL4dEVmXIp?= =?Windows-1252?Q?joIt1Uzi3zr35EyCp3n8xEIVVqj6OST/C3+TKuYc7tOcLBpFE1KYiWeg?= =?Windows-1252?Q?CDCvOuALHT4nJrWpReVAWVlmLDG63dPackbzRErFnnVEvr55LLxD1Anw?= =?Windows-1252?Q?HFYDPdVBmJj6sMjK6rlx4URjSgZq5nbkWRSoN/1qcKkuGLnoysIIj4YN?= =?Windows-1252?Q?9YCRaGWMQNQeRPu8L7xBjE2ti6VHVPj50Uxc1hx4p8FjWyrafth0sJ1A?= X-OriginatorOrg: cornell.edu X-MS-Exchange-CrossTenant-Network-Message-Id: 527a7390-087c-4205-ee4d-08d90b2e1b20 X-MS-Exchange-CrossTenant-AuthSource: BN7PR04MB4388.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Apr 2021 16:44:50.2809 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 5d7e4366-1b9b-45cf-8e79-b14b27df46e1 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: TcWy5BJxhoBFwfldv4+KJhpc+OyvDD/4BjDqtqPoGI8miT/GPe/OLtir+PoE3GmOyqAjSXH+QtIj0gIL+NGapg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR04MB4307 X-Spam-Status: No, score=-3.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, MSGID_FROM_MTA_HEADER, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Apr 2021 16:44:55 -0000 On 4/29/2021 11:05 AM, Corinna Vinschen wrote: > On Apr 29 10:38, Ken Brown wrote: >> On 4/29/2021 7:05 AM, Corinna Vinschen wrote: >>> On Apr 27 11:47, Ken Brown wrote: >>>> I'm willing to start working on the switch to native AF_UNIX sockets. (I'm >>>> frankly getting bored with working on the pipe implementation, and this >>> ^^^^^^^^^^^^^ >>> I not really surprised, Windows pipe semantics are annoying. >>> >>>> doesn't really seem like it has much of a future.) But I'd like to be >>>> confident that there's a good solution to the datagram problem before I >>>> invest too much time in this. >>> >>> Summary of our short discussion on IRC: >>> >>> - Switching to SOCK_STREAM under the hood adds the necessary reliabilty >>> but breaks DGRAM message boundaries. >>> >>> - There appears to be no way in Winsock to handle send buffer overflow >>> gracefully so that user space knows that messages have been discarded. >>> Strange enoug there's a SIO_ENABLE_CIRCULAR_QUEUEING ioctl, but that >>> just makes things worse, by dropping older messages in favor of the >>> newer ones :-P >>> >>> I think it should be possible to switch to STREAM sockets to emulate >>> DGRAM semantics. Our advantage is that this is all local. For all >>> practical purposes there's no chance data gets really lost. Windows has >>> an almost indefinite send buffer. >>> >>> If you look at the STREAM as a kind of tunneling layer for getting DGRAM >>> messages over the (local) line, the DGRAM content could simply be >>> encapsulated in a tunnel packet or frame, basically the same way the >>> new, boring AF_UNIX code does it. A DGRAM message encapsulated in a >>> STREAM message always has a header which at least contains the length of >>> the actual DGRAM message. So when the peer reads from the socket, it >>> always only reads the header until it's complete. Then it knows how >>> much payload is expected and then it reads until the payload has been >>> received. >> >> This should work. We could even use MSG_PEEK to read the header and then >> MSG_WAITALL to read the whole packet. >> >> I'd be happy to try to implement this. Do you want to create a branch >> (maybe topic/dgram or something like that) for working on it? > > You can create topic branches as you see fit, don't worry about it. > >>> Ultimately this would even allow to emulate DGRAMs when using native >>> Windows AF_UNIX sockets. Then we'd just have to keep the old code for >>> backward compat. >> >> Yep. >> >>> There's just one problem with this entire switch to non-pipes: Sending >>> descriptors between peers running under different accounts requires to >>> be able to switch the user context. You need this if the sender is a >>> non-admin account to call ImpersonateNamedPipeClient in the receiver. >>> So we might need to keep the pipes even if just for the purpose of being >>> able to call ImpersonateNamedPipeClient... >>> >>> >>> Thoughts? >> >> Sounds great. Thanks. > > Don't start just yet. > > I'm still not quite sure if that's really the way to go. As I see it we > still have something to discuss here. > > For one thing, using native AF_UNIX sockets will split our user base > into two. Those who are not using a recent enough Windows will get the > old code and no descriptor passing. However, if an application has been > built with descriptor passing, it won't work for those running older > Windows versions. I don't think we want that for the distro, or, do we? Good point. Sounds like a nightmare. > Next problem... implementing actual STREAM sockets. Even using native > AF_UNIX sockets, these, too, would have to encapsulate the actual > payload because of the ancilliary data we want to send with them. > Whether or not we use native AF_UNIX sockets, they won't be compatible > with native applications... > > So maybe we should really think hard about the alternative > implementation using POSIX message queues, I guess. And *if* we do > that, this should be used likewise for STREAM as for DGRAM sockets, so > the code is easier to maintain. Obvious advantage: No problem with > older OS versions. And maybe it's even dirt easy to implement in > comparison with using other methods, because the transport mechanism > is already in place. Yes, I don't think it should be too hard. The one thing I can think of that's missing is a facility for doing a partial read of a message on the message queue. (This would be needed for a recv call on a STREAM socket, in which the buffer is smaller than the payload of the next message on the queue.) But this should be straightforward to implement. Alternatively, I guess we could read the whole message and store the excess in a readahead buffer. > What's missing is the ImpersonateNamedPipeClient stuff (but that's not > different from using native AF_UNIX) and reflections about the permission > handling. On 4/29/2021 11:18 AM, Corinna Vinschen wrote: > While searching the net I found this additional gem of information: > > Native AF_UNIX sockets don't support abstract sockets. You must bind to > a valid path, so you always have a visible file in the filesystem. > Discussed here: https://github.com/microsoft/WSL/issues/4240 > > We could workaround that with our POSIX unlink semantics, probably, > but it's YA downside Agreed. The more features that are missing from native AF_UNIX sockets, the less appealing they become. Concerning abstract sockets, would we still have an issue if we used message queues? Wouldn't there be a visible file under /dev/mqueue? Or is there a way around that? Ken