public inbox for cygwin-developers@cygwin.com
 help / color / mirror / Atom feed
From: Corinna Vinschen <corinna-cygwin@cygwin.com>
To: cygwin-developers@cygwin.com
Subject: Re: The unreliability of AF_UNIX datagram sockets
Date: Thu, 29 Apr 2021 13:05:39 +0200	[thread overview]
Message-ID: <YIqTA4KKd8NIuEd6@calimero.vinschen.de> (raw)
In-Reply-To: <58da34ac-f2b6-d8b2-e872-834cfcb1ab51@cornell.edu>

On Apr 27 11:47, Ken Brown wrote:
> This is a follow-up to
> 
>  https://cygwin.com/pipermail/cygwin/2021-April/248383.html
> 
> I'm attaching a test case slightly simpler than the one posted by the OP in
> that thread.  This is a client/server scenario, with non-blocking AF_UNIX
> datagram sockets.  The client writes COUNT messages while the server is
> playing with his toes.  Then the server reads the messages.
> 
> If COUNT is too big, the expectation is that the client's sendto call will
> eventually return EAGAIN.  This is what happens on Linux.  On Cygwin,
> however, there is never a sendto error; the program ends when recv fails
> with EAGAIN, indicating that some messages were dropped.
> 
> I think what's happening is that WSASendTo is silently dropping messages
> without returning an error.  I guess this is acceptable because of the
> documented unreliability of AF_INET datagram sockets.  But AF_UNIX datagram
> sockets are supposed to be reliable.
> 
> I can't think of anything that Cygwin can do about this (but I would love to
> be proven wrong).  My real reason for raising the issue is that, as we
> recently discussed in a different thread, maybe it's time for Cygwin to
> start using native Windows AF_UNIX sockets.  But then we would still have to
> come up with our own implementation of AF_UNIX datagram sockets, and it
> seems that we can't simply use the current implementation.  AFAICT, Mark's
> suggestion of using message queues is the best idea so far.
> 
> I'm willing to start working on the switch to native AF_UNIX sockets.  (I'm
> frankly getting bored with working on the pipe implementation, and this
          ^^^^^^^^^^^^^
I not really surprised, Windows pipe semantics are annoying.

> doesn't really seem like it has much of a future.)  But I'd like to be
> confident that there's a good solution to the datagram problem before I
> invest too much time in this.

Summary of our short discussion on IRC:

- Switching to SOCK_STREAM under the hood adds the necessary reliabilty
  but breaks DGRAM message boundaries.

- There appears to be no way in Winsock to handle send buffer overflow
  gracefully so that user space knows that messages have been discarded.
  Strange enoug there's a SIO_ENABLE_CIRCULAR_QUEUEING ioctl, but that
  just makes things worse, by dropping older messages in favor of the
  newer ones :-P

I think it should be possible to switch to STREAM sockets to emulate
DGRAM semantics.  Our advantage is that this is all local.  For all
practical purposes there's no chance data gets really lost.  Windows has
an almost indefinite send buffer.

If you look at the STREAM as a kind of tunneling layer for getting DGRAM
messages over the (local) line, the DGRAM content could simply be
encapsulated in a tunnel packet or frame, basically the same way the
new, boring AF_UNIX code does it.  A DGRAM message encapsulated in a
STREAM message always has a header which at least contains the length of
the actual DGRAM message.  So when the peer reads from the socket, it
always only reads the header until it's complete.  Then it knows how
much payload is expected and then it reads until the payload has been
received.

Ultimately this would even allow to emulate DGRAMs when using native
Windows AF_UNIX sockets.  Then we'd just have to keep the old code for
backward compat.

There's just one problem with this entire switch to non-pipes: Sending
descriptors between peers running under different accounts requires to
be able to switch the user context.  You need this if the sender is a
non-admin account to call ImpersonateNamedPipeClient in the receiver.
So we might need to keep the pipes even if just for the purpose of being
able to call ImpersonateNamedPipeClient...


Thoughts?


Thanks,
Corinna

  reply	other threads:[~2021-04-29 11:05 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-04-27 15:47 Ken Brown
2021-04-29 11:05 ` Corinna Vinschen [this message]
2021-04-29 11:16   ` Corinna Vinschen
2021-04-29 14:38   ` Ken Brown
2021-04-29 15:05     ` Corinna Vinschen
2021-04-29 15:18       ` Corinna Vinschen
2021-04-29 16:44       ` Ken Brown
2021-04-29 17:39         ` Corinna Vinschen
2021-05-01 21:41           ` Ken Brown
2021-05-03 10:30             ` Corinna Vinschen
2021-05-03 15:45               ` Corinna Vinschen
2021-05-03 16:56                 ` Ken Brown
2021-05-03 18:40                   ` Corinna Vinschen
2021-05-03 19:48                     ` Ken Brown
2021-05-03 20:50                       ` Ken Brown
2021-05-04 11:06                         ` Corinna Vinschen
2021-05-13 14:30                           ` Ken Brown
2021-05-17 10:26                             ` Corinna Vinschen
2021-05-17 13:02                               ` Ken Brown
2021-05-17 13:02                               ` Ken Brown
2021-05-20 13:46   ` Ken Brown
2021-05-20 19:25     ` Corinna Vinschen
2021-05-21 21:54       ` Ken Brown
2021-05-22 15:49         ` Corinna Vinschen
2021-05-22 16:50           ` Ken Brown
2021-05-22 18:21             ` Ken Brown

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=YIqTA4KKd8NIuEd6@calimero.vinschen.de \
    --to=corinna-cygwin@cygwin.com \
    --cc=cygwin-developers@cygwin.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).