From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.kundenserver.de (mout.kundenserver.de [212.227.126.133]) by sourceware.org (Postfix) with ESMTPS id 93DD53958C06 for ; Thu, 29 Apr 2021 11:05:41 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 93DD53958C06 Authentication-Results: sourceware.org; dmarc=fail (p=none dis=none) header.from=cygwin.com Authentication-Results: sourceware.org; spf=fail smtp.mailfrom=corinna-cygwin@cygwin.com Received: from calimero.vinschen.de ([24.134.7.25]) by mrelayeu.kundenserver.de (mreue011 [212.227.15.167]) with ESMTPSA (Nemesis) id 1M89XH-1lgAxb0GaR-005KCG for ; Thu, 29 Apr 2021 13:05:40 +0200 Received: by calimero.vinschen.de (Postfix, from userid 500) id 93FD4A80F11; Thu, 29 Apr 2021 13:05:39 +0200 (CEST) Date: Thu, 29 Apr 2021 13:05:39 +0200 From: Corinna Vinschen To: cygwin-developers@cygwin.com Subject: Re: The unreliability of AF_UNIX datagram sockets Message-ID: Reply-To: cygwin-developers@cygwin.com Mail-Followup-To: cygwin-developers@cygwin.com References: <58da34ac-f2b6-d8b2-e872-834cfcb1ab51@cornell.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <58da34ac-f2b6-d8b2-e872-834cfcb1ab51@cornell.edu> X-Provags-ID: V03:K1:LiC1uwRgFHHZZWZyswRqblLxBvgVrMwaQfPx5gMZHm3/94D+eBR ml8ulzkgwz3USNFfC0Kep6FjjrdVtMPjcffSuTR76gprYo3k6YzFY8oATYQCnAM8Bsdzc5b 9wuLG5STfT0q5O7tahltcVr5YAqMk1cmNhB6bWSCmlik+mblKXHqBtwkYO2PEtTdjmTsUaX gMuez6PL34O9WOup6EdQQ== X-UI-Out-Filterresults: notjunk:1;V03:K0:omBiEOC8bto=:c2AChCBkSIYyx+iBRgn2Pp v1MBvSsj1mrbqHnrK7FtO6Ebzg3EKXtXYRosdmJNwz0jn6s+v08Ik34MyIe1ujYRGiep9+e3P kcWOvCmJvLH0Wx7HLbGj+2PccaDQtjDTvUlJl5g/9zFWlQqDW/kgDV6e93DGL9HhqipatWIP+ boDxaRCTyCO3vE+HKKxi1Lg4q8CL6KK1Q14i1S/RUGvjA9O1QGyeK+AM//JigqJVTFbSN4IIe QjJdVsZYFEHS9qFImoXAdFEn81iXxERP+hxlJvYK0AGdGEEiCIfaBxFaX4G8iy6xFX97vT6JF ifyLyw3SYWeP6xyJDCV3c7prxlovAjf2W4BS7+c4Tvy1DhCPEa0qPYwEjTQwnalTc3jdqxXO9 PsaYpnXH1zR3pE0wgRQf+xmcsbYmfr5A5g0jXxzkXpYsl2pPL6lc73qDQge8MUhWu0uaO3WNn zxIRR173jeu0stDjdtJIIhgOnQOj/zwH/GZ2/LLPki8uzgxNgxPD/eq60fCkFs1Keqt1mJN2V HmHfx3NG4AR2T+jkIFUx6w= X-Spam-Status: No, score=-100.1 required=5.0 tests=BAYES_00, GOOD_FROM_CORINNA_CYGWIN, JMQ_SPF_NEUTRAL, KAM_DMARC_NONE, KAM_DMARC_STATUS, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_NEUTRAL, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Apr 2021 11:05:43 -0000 On Apr 27 11:47, Ken Brown wrote: > This is a follow-up to > > https://cygwin.com/pipermail/cygwin/2021-April/248383.html > > I'm attaching a test case slightly simpler than the one posted by the OP in > that thread. This is a client/server scenario, with non-blocking AF_UNIX > datagram sockets. The client writes COUNT messages while the server is > playing with his toes. Then the server reads the messages. > > If COUNT is too big, the expectation is that the client's sendto call will > eventually return EAGAIN. This is what happens on Linux. On Cygwin, > however, there is never a sendto error; the program ends when recv fails > with EAGAIN, indicating that some messages were dropped. > > I think what's happening is that WSASendTo is silently dropping messages > without returning an error. I guess this is acceptable because of the > documented unreliability of AF_INET datagram sockets. But AF_UNIX datagram > sockets are supposed to be reliable. > > I can't think of anything that Cygwin can do about this (but I would love to > be proven wrong). My real reason for raising the issue is that, as we > recently discussed in a different thread, maybe it's time for Cygwin to > start using native Windows AF_UNIX sockets. But then we would still have to > come up with our own implementation of AF_UNIX datagram sockets, and it > seems that we can't simply use the current implementation. AFAICT, Mark's > suggestion of using message queues is the best idea so far. > > I'm willing to start working on the switch to native AF_UNIX sockets. (I'm > frankly getting bored with working on the pipe implementation, and this ^^^^^^^^^^^^^ I not really surprised, Windows pipe semantics are annoying. > doesn't really seem like it has much of a future.) But I'd like to be > confident that there's a good solution to the datagram problem before I > invest too much time in this. Summary of our short discussion on IRC: - Switching to SOCK_STREAM under the hood adds the necessary reliabilty but breaks DGRAM message boundaries. - There appears to be no way in Winsock to handle send buffer overflow gracefully so that user space knows that messages have been discarded. Strange enoug there's a SIO_ENABLE_CIRCULAR_QUEUEING ioctl, but that just makes things worse, by dropping older messages in favor of the newer ones :-P I think it should be possible to switch to STREAM sockets to emulate DGRAM semantics. Our advantage is that this is all local. For all practical purposes there's no chance data gets really lost. Windows has an almost indefinite send buffer. If you look at the STREAM as a kind of tunneling layer for getting DGRAM messages over the (local) line, the DGRAM content could simply be encapsulated in a tunnel packet or frame, basically the same way the new, boring AF_UNIX code does it. A DGRAM message encapsulated in a STREAM message always has a header which at least contains the length of the actual DGRAM message. So when the peer reads from the socket, it always only reads the header until it's complete. Then it knows how much payload is expected and then it reads until the payload has been received. Ultimately this would even allow to emulate DGRAMs when using native Windows AF_UNIX sockets. Then we'd just have to keep the old code for backward compat. There's just one problem with this entire switch to non-pipes: Sending descriptors between peers running under different accounts requires to be able to switch the user context. You need this if the sender is a non-admin account to call ImpersonateNamedPipeClient in the receiver. So we might need to keep the pipes even if just for the purpose of being able to call ImpersonateNamedPipeClient... Thoughts? Thanks, Corinna