From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM02-BL2-obe.outbound.protection.outlook.com (mail-eopbgr750114.outbound.protection.outlook.com [40.107.75.114]) by sourceware.org (Postfix) with ESMTPS id A512E386FC2B for ; Thu, 29 Apr 2021 14:38:11 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org A512E386FC2B Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=cornell.edu Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=kbrown@cornell.edu ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Z1ooYJfegxRAjGkWp4hUFPiPm3oDgUd/aRg5H0dzS0cdEenar8BOK/NUmkkcEI7fx5+Mia6HzNbFJ3vHNl5tSRb+fj2/NSYxcE8C6JWoYb2RiLNPjOCZiBdhyjyIJsF/qckNG92VN1AAnt8vbBikPyg9Y54lw5gZlW8V5gxx0Wo3fs3wwyTz6eGl1x85HHHMFZhvcmPyXjGGeB7NcDFaaZ78gHl1XYwNwBANQEHoNgd5w9hz32LzuuTMhaknOCoUrkRHVOSQ3h3LxBfaGJkeA2OTrgQexCkXiG7/GEWWLOFTI9k/AjFjIjDq2vwpcp6SU6ViqC0Fs9bMLE7XG36VgA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fF1QXrXSh17k7fGN7B/wf3++2bmDK3K7Y/Z6yA9tDVc=; b=KuW6BCjyTlipJUReAa14Jo0n08ppKiueEPy0u83NHnbgLPp56wjHjH7+E27KvwYAeISbx4h5R5BXH+XVO8nNO5z5WJgNLuwAxNskNPmCN/5I17aTdMJKHc3iWP+rfY4CYOkSzJqffoIQ6eR18+VENAiA/Gs5fTwVeSC/I3GzcuJjOXwQMbPGDGThbVB8flpWqw3BozUR/9WBu8LCE72EdfW6LdXYxlBBiSkJZCPDYLjv6jQk9eDl823StRlt4786eMLBwvBm0nt8RC6n74fRR2cRDQw74pPPJz4j7vItJof4lLtcCmaH3NpPDDLsEXHeYxqqNs5w/yXuCsYpM6msMw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=cornell.edu; dmarc=pass action=none header.from=cornell.edu; dkim=pass header.d=cornell.edu; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornell.edu; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fF1QXrXSh17k7fGN7B/wf3++2bmDK3K7Y/Z6yA9tDVc=; b=C9fPXZmI4Yr8/ujmhLxeWPBUM2L1mdAPTRThzRike9vhn3mHV2++YHGsxMJainQRWiYy92pdpWcCfoObRRslwNhIycsLlFCOllKN7BnSCwrMXx7+Un86IvH8SQ17+V4J49FzoNFHvg1JTjd5uWc/XEBnISgAfKQQYpjCKfZoMvg= Authentication-Results: cygwin.com; dkim=none (message not signed) header.d=none;cygwin.com; dmarc=none action=none header.from=cornell.edu; Received: from BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) by BN6PR04MB0709.namprd04.prod.outlook.com (2603:10b6:404:d4::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.23; Thu, 29 Apr 2021 14:38:08 +0000 Received: from BN7PR04MB4388.namprd04.prod.outlook.com ([fe80::59f8:fcc4:f07e:9a89]) by BN7PR04MB4388.namprd04.prod.outlook.com ([fe80::59f8:fcc4:f07e:9a89%4]) with mapi id 15.20.4065.027; Thu, 29 Apr 2021 14:38:08 +0000 Subject: Re: The unreliability of AF_UNIX datagram sockets To: cygwin-developers@cygwin.com References: <58da34ac-f2b6-d8b2-e872-834cfcb1ab51@cornell.edu> From: Ken Brown Message-ID: <6cac30e5-56fc-5bf1-b85b-fe6b91bc5e97@cornell.edu> Date: Thu, 29 Apr 2021 10:38:05 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.10.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Originating-IP: [65.112.130.200] X-ClientProxiedBy: BLAPR03CA0047.namprd03.prod.outlook.com (2603:10b6:208:32d::22) To BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from [10.13.22.8] (65.112.130.200) by BLAPR03CA0047.namprd03.prod.outlook.com (2603:10b6:208:32d::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4087.27 via Frontend Transport; Thu, 29 Apr 2021 14:38:06 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 5a3d6154-b7ca-459e-9732-08d90b1c6754 X-MS-TrafficTypeDiagnostic: BN6PR04MB0709: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8882; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: I/mPxugJ56gqgE69wE/DIa3OKu92+lBZtfKO0brSIhly9eaIZOeE4tHwU1FndaTe2nqTpm1MD/0DxElPjl85asR8hpfhFTeupIMPYe3iTGVtRHf9Bg7u4FATfb5Dg/2rUWv5JFSjJYawyIxY7/7Ads30rWUYQIeQ4YYDVJ6ZKPAYjI2B3amyaE1HOWCYjgOYdjXsAXsz0U7haNIN1hYYggYCIL5sI3CvxypmddnTQCs32led4Ksb5hJUb6f1rFoJTPrqYQQrX4VRmStWapZIQU+ZGQgjU6VGZ2yCMkBKBiM4emCtQZmV449TnHKqni8zkF/PFOPkgmZ1GPsSOQeC7GDIkIPVFnc72HUGxakfHcyYZkhMYV8lvS/B/CqkY+gfKAkRJft3TTTAqsaRnIdbS9fUHLJnr4GvPr02zOaNpz74Rtl0+Y0eeLRZZMJtdpwzWGhMHysC7Tbj6h5hIqm+2NwC5SMyCEtNMLZwGS56G8MD3xYinSIX+UrneLg/llGFzV8sIpg9NqsidjasphShCKP48btNSrcgM1IRV/83jZlrOSFqlS0b6/Xv3tbqCuO2lp1xmbhXrEL2KYZOq3eWaNRSjN47mp4BfANnhPs8apFaU/GVZe+JYgbz2O7zNCvMyoNkMPUTMW9Uifj5G3R2lR8GDZ9TlakY3/GlM3stN+4aScFDlo49m/5ChyUt9IgH5lWvQr4T5OEDSbRhfq7NSolJMv4DCeYF9ROvnfBdGCtlP6f/XuucFkTFCLjEDkjuRI1QsTU7rru3NAWs7tbSsTrFwd0ehGS9d7qfePt996I= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN7PR04MB4388.namprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(136003)(39860400002)(396003)(366004)(376002)(346002)(53546011)(38100700002)(16526019)(186003)(26005)(38350700002)(2906002)(75432002)(36756003)(66946007)(316002)(478600001)(8676002)(16576012)(6916009)(31696002)(2616005)(5660300002)(966005)(86362001)(83380400001)(6486002)(956004)(66476007)(8936002)(786003)(52116002)(31686004)(66556008)(43740500002)(45980500001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: =?Windows-1252?Q?vHNucT7aGsamEOrtaGjekNGuCk4JUpX5uANAEDfgKK0AEv6e7ncNK7Rt?= =?Windows-1252?Q?/1AyHK7dWpn+Pa0oC2yW0SlpzI2jlYMVXzduHBBfHoBze5qxaLcZNdry?= =?Windows-1252?Q?IqfwQEeqQSeaIIRH1iu+h90vuO3PSGBjMGASareEKHuVdyo02IbnDCtz?= =?Windows-1252?Q?1JiCiQg4Hp3UaLxf1zCL4CdIK36siqPpZzz92mdBMiXcaYlYtHPJX8PS?= =?Windows-1252?Q?qxrYTZvZOqC5EYesO2B3+zMOpdLbviPascj5i1Qe6hvglSJ7Sre6t3AS?= =?Windows-1252?Q?njvYcxcsfy3SFf3Fcgrivmwxfgstx51jFeBGDEJHqjYGJN36jhPUsIvA?= =?Windows-1252?Q?UEHy9GlniZtq4I+HSrvC2vjKLTsT9A1OI2zoplyud/Urc0V1kKCHUgYC?= =?Windows-1252?Q?s4GKWlvQ6JivWS6GhpP2DUacLgEHAHFSUoPakf8ogfFq+wf79e6vjgDD?= =?Windows-1252?Q?R7t7+uRe3o3Pxwvxas+n+ekOb2D+Fc+xm0WfT27lKuTF88MSwfpOt0HL?= =?Windows-1252?Q?BpC01+nZg8KPkiM44gbG2bQInCA17zKnrbThyRinWuChxjQs+rzkeVOQ?= =?Windows-1252?Q?KsztmMzd8Vxnuo+6V+xd90yzv/FO0bxGcvg2Y+bogO17lzyqZc02dRHY?= =?Windows-1252?Q?JciN8AVTcZmTqMDGJKYp5W+lKMNa1ilM4yfM7BTbwlAfk1tHEKS9uKvE?= =?Windows-1252?Q?+W87t3rHAS6BeqXc+QhsFR0dcbeOs+DPUC0OkEV8zEIYupgG8MZS3Lmb?= =?Windows-1252?Q?Kh1NimXJ+wxiBs3fEe00m+eK62/beCfKlvz31CNd5jN3V98yB27t+Mti?= =?Windows-1252?Q?cmWrUOQXubFASP+FYCqZtOvlHaoFGHcy1BTahegYOlxp0EsFIEcjrTIj?= =?Windows-1252?Q?u1RQ/Xy3que87fb8ZWEvXC/9fOSenpaGJzf0eg1qzZeIDYuyLBuaE18l?= =?Windows-1252?Q?lI4j8qKnTgk4u9ot/P3Xp+091GaiX7Ba36fxib9BZdgsNl/0O+JxpzG2?= =?Windows-1252?Q?362t+s92ntye5LkHZSNgez7JlHp5n/TN2hsPV2GxW37+6bIwzQkJxbOG?= =?Windows-1252?Q?a7ywPYP5EZ62w5q7X8y5Vm70TW6JcLnNMIJXn64DYICeJB82XWOG4OTC?= =?Windows-1252?Q?heRAit0XDE+vIuTOPqkbFujcv0mjyNqO1orbsRXQt+UQlWcgvFGyN12K?= =?Windows-1252?Q?GKPrfcBqwkiwluACnth+PhX/r8eJRecLz2RIdHhSgY7lVON/7h5UVcZp?= =?Windows-1252?Q?pOFrhQdEJDlVT2GLnr6NictzwB2qhIcswv4z8f07v+03WdMR6Lrt/7HQ?= =?Windows-1252?Q?UZZvUboGZR9ZvPfwTHIkMnvqZ1/KO21CwDorg6H6pnXyElAljp3eb2ft?= =?Windows-1252?Q?GyNTEiTJt+HSgdRY+sIHuHG+nzuHH5ufuZ6vo54oO36iqZhrW6GFfC1T?= X-OriginatorOrg: cornell.edu X-MS-Exchange-CrossTenant-Network-Message-Id: 5a3d6154-b7ca-459e-9732-08d90b1c6754 X-MS-Exchange-CrossTenant-AuthSource: BN7PR04MB4388.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Apr 2021 14:38:08.4334 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 5d7e4366-1b9b-45cf-8e79-b14b27df46e1 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 2fpB2GXUfQwnNxm8MaJju24XpTXb7jHOU8sI/jcuOm+qArqQSINXmXSGdt8AXNi2N+m7MQmIwqqmEeQlrRlQPw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR04MB0709 X-Spam-Status: No, score=-3.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, MSGID_FROM_MTA_HEADER, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Apr 2021 14:38:13 -0000 On 4/29/2021 7:05 AM, Corinna Vinschen wrote: > On Apr 27 11:47, Ken Brown wrote: >> This is a follow-up to >> >> https://cygwin.com/pipermail/cygwin/2021-April/248383.html >> >> I'm attaching a test case slightly simpler than the one posted by the OP in >> that thread. This is a client/server scenario, with non-blocking AF_UNIX >> datagram sockets. The client writes COUNT messages while the server is >> playing with his toes. Then the server reads the messages. >> >> If COUNT is too big, the expectation is that the client's sendto call will >> eventually return EAGAIN. This is what happens on Linux. On Cygwin, >> however, there is never a sendto error; the program ends when recv fails >> with EAGAIN, indicating that some messages were dropped. >> >> I think what's happening is that WSASendTo is silently dropping messages >> without returning an error. I guess this is acceptable because of the >> documented unreliability of AF_INET datagram sockets. But AF_UNIX datagram >> sockets are supposed to be reliable. >> >> I can't think of anything that Cygwin can do about this (but I would love to >> be proven wrong). My real reason for raising the issue is that, as we >> recently discussed in a different thread, maybe it's time for Cygwin to >> start using native Windows AF_UNIX sockets. But then we would still have to >> come up with our own implementation of AF_UNIX datagram sockets, and it >> seems that we can't simply use the current implementation. AFAICT, Mark's >> suggestion of using message queues is the best idea so far. >> >> I'm willing to start working on the switch to native AF_UNIX sockets. (I'm >> frankly getting bored with working on the pipe implementation, and this > ^^^^^^^^^^^^^ > I not really surprised, Windows pipe semantics are annoying. > >> doesn't really seem like it has much of a future.) But I'd like to be >> confident that there's a good solution to the datagram problem before I >> invest too much time in this. > > Summary of our short discussion on IRC: > > - Switching to SOCK_STREAM under the hood adds the necessary reliabilty > but breaks DGRAM message boundaries. > > - There appears to be no way in Winsock to handle send buffer overflow > gracefully so that user space knows that messages have been discarded. > Strange enoug there's a SIO_ENABLE_CIRCULAR_QUEUEING ioctl, but that > just makes things worse, by dropping older messages in favor of the > newer ones :-P > > I think it should be possible to switch to STREAM sockets to emulate > DGRAM semantics. Our advantage is that this is all local. For all > practical purposes there's no chance data gets really lost. Windows has > an almost indefinite send buffer. > > If you look at the STREAM as a kind of tunneling layer for getting DGRAM > messages over the (local) line, the DGRAM content could simply be > encapsulated in a tunnel packet or frame, basically the same way the > new, boring AF_UNIX code does it. A DGRAM message encapsulated in a > STREAM message always has a header which at least contains the length of > the actual DGRAM message. So when the peer reads from the socket, it > always only reads the header until it's complete. Then it knows how > much payload is expected and then it reads until the payload has been > received. This should work. We could even use MSG_PEEK to read the header and then MSG_WAITALL to read the whole packet. I'd be happy to try to implement this. Do you want to create a branch (maybe topic/dgram or something like that) for working on it? > Ultimately this would even allow to emulate DGRAMs when using native > Windows AF_UNIX sockets. Then we'd just have to keep the old code for > backward compat. Yep. > There's just one problem with this entire switch to non-pipes: Sending > descriptors between peers running under different accounts requires to > be able to switch the user context. You need this if the sender is a > non-admin account to call ImpersonateNamedPipeClient in the receiver. > So we might need to keep the pipes even if just for the purpose of being > able to call ImpersonateNamedPipeClient... > > > Thoughts? Sounds great. Thanks. Ken