From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.kundenserver.de (mout.kundenserver.de [212.227.126.187]) by sourceware.org (Postfix) with ESMTPS id 1A0A23A19004 for ; Thu, 29 Apr 2021 11:16:29 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 1A0A23A19004 Authentication-Results: sourceware.org; dmarc=fail (p=none dis=none) header.from=cygwin.com Authentication-Results: sourceware.org; spf=fail smtp.mailfrom=corinna-cygwin@cygwin.com Received: from calimero.vinschen.de ([24.134.7.25]) by mrelayeu.kundenserver.de (mreue009 [212.227.15.167]) with ESMTPSA (Nemesis) id 1M6DSi-1leGW32KK0-006d78 for ; Thu, 29 Apr 2021 13:16:27 +0200 Received: by calimero.vinschen.de (Postfix, from userid 500) id 193BFA80F11; Thu, 29 Apr 2021 13:16:27 +0200 (CEST) Date: Thu, 29 Apr 2021 13:16:27 +0200 From: Corinna Vinschen To: cygwin-developers@cygwin.com Subject: Re: The unreliability of AF_UNIX datagram sockets Message-ID: Reply-To: cygwin-developers@cygwin.com Mail-Followup-To: cygwin-developers@cygwin.com References: <58da34ac-f2b6-d8b2-e872-834cfcb1ab51@cornell.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: X-Provags-ID: V03:K1:/JlfUtuazPvmXXDBgFkxEC52QCd69YKU+vWfjfPW7A1HsZPnsdA h7ga7VYiQtdxF3Ko0FMX/gA9YGAjlD3+9YX5cgVskoiP85Zr3fp94inrTF7fRIbkOAaSZJy xURSFmlWOHtQWF3ww+cYCJvEElG8gEcl2q2ig8iJ9Vi8xs9DrwqdA2q6NjB8DJSg5ilY5Tv SFJ0X86Mqd5E+73sStYFA== X-UI-Out-Filterresults: notjunk:1;V03:K0:W3SxlxDAgwU=:/3AY1J/3G9vV+B+JiJWUeU BnlLUUL+zNVFFb5cQDtekEaqGVigR5w08o9F95VQGgSLzFcU6qoQBB6kiQWhXfAQ9grIHKu6r 7reodDuHcaFe3nTlV2f2bnArHS0NTQpErLfNAf/66Q3Wm2y9T+PdA6FTbWfdOiYBtcgOxUlCU bX+13Spd+3BKRmLlY0RZcI9qC2HtJHPhODL5KVDa6MW6eKpaVzkGTrLz+DUGLlauPwxekFeVp rjCFPW5PHZVjm8DeiA0qSmNzru5cRhXZtrQXQjwK1DC/KUk+IWyyufzBIN4D9loh33/MTHNi3 iSUrrEUrALZ2J3bDXw6WNUeZ1oU3CNxJjnrYcr/qQlTW9wlp85L2WFF22Eayu8rmB5JcxLSWd Pe2xt1fQgTTiX/krWyKzXXCLP6HpXl7WnFXbC0b21/Hp+1fz/84U4AGamU/1bDZIbQRPJ2n6y hHfwLiHxp2SrnCTzzl6GFnSns6ErAlso2Ng4zaYKW6jjcdofk2rYUWC4k7oraADpISBIFUv2h Gfgggh6z7CQ4KPp+2hFMjQ= X-Spam-Status: No, score=-100.3 required=5.0 tests=BAYES_00, GOOD_FROM_CORINNA_CYGWIN, KAM_DMARC_NONE, KAM_DMARC_STATUS, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_NEUTRAL, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Apr 2021 11:16:31 -0000 On Apr 29 13:05, Corinna Vinschen wrote: > On Apr 27 11:47, Ken Brown wrote: > > This is a follow-up to > > > > https://cygwin.com/pipermail/cygwin/2021-April/248383.html > > > > I'm attaching a test case slightly simpler than the one posted by the OP in > > that thread. This is a client/server scenario, with non-blocking AF_UNIX > > datagram sockets. The client writes COUNT messages while the server is > > playing with his toes. Then the server reads the messages. > > > > If COUNT is too big, the expectation is that the client's sendto call will > > eventually return EAGAIN. This is what happens on Linux. On Cygwin, > > however, there is never a sendto error; the program ends when recv fails > > with EAGAIN, indicating that some messages were dropped. > > > > I think what's happening is that WSASendTo is silently dropping messages > > without returning an error. I guess this is acceptable because of the > > documented unreliability of AF_INET datagram sockets. But AF_UNIX datagram > > sockets are supposed to be reliable. > > > > I can't think of anything that Cygwin can do about this (but I would love to > > be proven wrong). My real reason for raising the issue is that, as we > > recently discussed in a different thread, maybe it's time for Cygwin to > > start using native Windows AF_UNIX sockets. But then we would still have to > > come up with our own implementation of AF_UNIX datagram sockets, and it > > seems that we can't simply use the current implementation. AFAICT, Mark's > > suggestion of using message queues is the best idea so far. > > > > I'm willing to start working on the switch to native AF_UNIX sockets. (I'm > > frankly getting bored with working on the pipe implementation, and this > ^^^^^^^^^^^^^ > I not really surprised, Windows pipe semantics are annoying. > > > doesn't really seem like it has much of a future.) But I'd like to be > > confident that there's a good solution to the datagram problem before I > > invest too much time in this. > > Summary of our short discussion on IRC: > > - Switching to SOCK_STREAM under the hood adds the necessary reliabilty > but breaks DGRAM message boundaries. > > - There appears to be no way in Winsock to handle send buffer overflow > gracefully so that user space knows that messages have been discarded. > Strange enoug there's a SIO_ENABLE_CIRCULAR_QUEUEING ioctl, but that > just makes things worse, by dropping older messages in favor of the > newer ones :-P > > I think it should be possible to switch to STREAM sockets to emulate > DGRAM semantics. Our advantage is that this is all local. For all > practical purposes there's no chance data gets really lost. Windows has > an almost indefinite send buffer. > > If you look at the STREAM as a kind of tunneling layer for getting DGRAM > messages over the (local) line, the DGRAM content could simply be > encapsulated in a tunnel packet or frame, basically the same way the > new, boring AF_UNIX code does it. A DGRAM message encapsulated in a > STREAM message always has a header which at least contains the length of > the actual DGRAM message. So when the peer reads from the socket, it > always only reads the header until it's complete. Then it knows how > much payload is expected and then it reads until the payload has been > received. Oh, btw., given this being local, and given that we always send a defined packet length, with DGRAM max reliable packet size much smaller than max STREAM packet size, there's almost no chance that the peer gets an incomplete packet. Unless, of course, user space requested a smaller packet size than the sender sent. In that case the remainder of the packet is lost, but that's business as usual (MSG_TRUNC). Obviously the recv call has to read the entire packet and just discard the rest. Corinna