From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2125.outbound.protection.outlook.com [40.107.244.125]) by sourceware.org (Postfix) with ESMTPS id 855073896C20 for ; Thu, 13 May 2021 14:30:42 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 855073896C20 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=cornell.edu Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=kbrown@cornell.edu ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=G3fre/oXdKEbn7xTGtLqI8FM/QFNCCBc6HZnUy5LB3voSk2R1+wNL93x9HTLy4oVdu1nQJJHHnT4Ftnq2c4KCqifYhm76q0qucrDaqNFxbtu2hAzLjH9B6m5GGTHlRBj3CVE/xOLLsh5gt0EMW9zjyQE3BvNqNuf9/f4zUGK/49Z/CW2VJagCTw8q6j4PYkNKLTp0VYGYc0R6E6lAwJJmHYYhtybdRY6Hb0jyPqWNCjkdoOBktKazOAraKFH1yjwVZrUAElDMAGM7B2l/esXQ+MGYXmHZUToZOWmJS1BNZZvpIe4AbAjNevfvEirraN61ryzRWCkK15ZwiMDgmZmhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=W5LfArYQ3/jbZbvOw4/5nf2cJVF/8qB8At6Q8US3ayM=; b=fm9GqEsVBl6dSCZmaxAK7pXyAFwyv79b8w/62BTzwiUuqFgvVpvu6wfamKx3MiyA9JpNc/JIH39NiB8WtFFKDvSycXXfK4ogPbz3MRaSV6+Tsz7Wr6oJvz+Fo8m+c9+BAxi0wjtYZcAhU/M3zBuk9J3gM7h3Q3Ys4LGRvRdFg1ilBS6JOs5pOHDBohzAQyGSnD8ti/OFk4nj57t+z0ZWhi7elUpfgiy7RvtpovHtP2LENPyIPJKZAlRlhxPi5CsRN/pZovII4IaOJC1m4g09gIQtZ5S5ruV54QFB81X6YzQGQlEs2WT17/0ehtn7F5Ka/laPZwI9945fQyFRsZFO7g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=cornell.edu; dmarc=pass action=none header.from=cornell.edu; dkim=pass header.d=cornell.edu; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornell.edu; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=W5LfArYQ3/jbZbvOw4/5nf2cJVF/8qB8At6Q8US3ayM=; b=dZ2GOkN9NHFosji9vNyRT8mEriglSpbMgxzToQ7qKwIXwfvqHOey+5U7mbeVDB56qoX6D3KQUUO7ZVRQu49toYStm3WaJUz68FKr1seU5tVPbvXIgQZrBfnuvjoYG+xUb6RugqoA1/F/9xy/OulehVh771F1zVaF17BdGJXpa4A= Authentication-Results: cygwin.com; dkim=none (message not signed) header.d=none;cygwin.com; dmarc=none action=none header.from=cornell.edu; Received: from BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) by BN8PR04MB5716.namprd04.prod.outlook.com (2603:10b6:408:a0::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4129.26; Thu, 13 May 2021 14:30:40 +0000 Received: from BN7PR04MB4388.namprd04.prod.outlook.com ([fe80::d452:659e:43f2:812b]) by BN7PR04MB4388.namprd04.prod.outlook.com ([fe80::d452:659e:43f2:812b%7]) with mapi id 15.20.4129.026; Thu, 13 May 2021 14:30:40 +0000 Subject: Re: The unreliability of AF_UNIX datagram sockets To: cygwin-developers@cygwin.com References: <16e1d55e-15ea-6c0e-04e4-aa6cb2c0c1bd@cornell.edu> <5564e10e-9099-fc5a-3a8d-c2ffb8ca4cff@cornell.edu> <41fde522-4d14-a957-96ad-c5eaa0e0a801@cornell.edu> <1aa91ec3-ae10-0e83-470d-a8a2dcfd83b0@cornell.edu> From: Ken Brown Message-ID: Date: Thu, 13 May 2021 10:30:38 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.10.1 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Originating-IP: [2603:7081:7e41:6a00:5498:fbd2:7594:a6ba] X-ClientProxiedBy: CH2PR05CA0016.namprd05.prod.outlook.com (2603:10b6:610::29) To BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from [IPv6:2603:7081:7e41:6a00:5498:fbd2:7594:a6ba] (2603:7081:7e41:6a00:5498:fbd2:7594:a6ba) by CH2PR05CA0016.namprd05.prod.outlook.com (2603:10b6:610::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4150.11 via Frontend Transport; Thu, 13 May 2021 14:30:40 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: be96a318-29bb-4656-f6ac-08d9161baed1 X-MS-TrafficTypeDiagnostic: BN8PR04MB5716: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:10000; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 66YOO9H8hnJwh/hPu961LDGaXvZBDpdGnlPLJ6hwJkYQ6ej79PKo6GDmLdL4ER+JW1vh24VYftNuyY8mVCg/wDIx31p6oLfwNjEaGSwg/7IUbKhiJdaU0NBcca4pjVn1FJUQ4UgB7UGE9MCjcIioUmUwEQVA0oibX2myPTImqfPq+j9pTqKCwoPRXAq7Exn0YfnUTcjQ1agjK+aEIWL3ZiXLNl1gbnY0svLS+ds06dYcJuD5UKp1AsRr9o7ys4qeUELzDc4pl+WxNGCjVCki10FkA6R07FEY7DhOZgAPu7r1FIZj2Px+OV9L85j/tsafIk6QeAHcOHZbXqABqWO0TSgoIOoM6J1/z0Y8FewcxFQBNEQ1p/VFFFxeRCes9K1yC3DhPo8K/Y7EJr/X99kPHr2aivu9VleWdBlyHq/UHEMOcw7lzx/UMD52rIDmLAOCpRP4TOyyy69py/G0lN0ydkDZs4hB5A6q8Blbj0GEGNmGrFeSUCewiokq0Rkiy3DAjK4iFy4qW+ctOzswKrTsfrjWudvomgQJwcPMsV6Br9/YtlmurEYkcOAmEcEt4aRrTMbMBp5W1KALAU24NNeMvovCKdpq9SlhkW846+tOjPXs15aLdp+W4FskzXvZdUFVa4oVmj5zMv9O5CShycw72BfOcFQk413VhlTCxplChQk9WroWrdYIOGAoL8VkeTcZ X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN7PR04MB4388.namprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39860400002)(376002)(136003)(396003)(366004)(346002)(6916009)(478600001)(66476007)(8676002)(36756003)(83380400001)(6486002)(86362001)(66556008)(75432002)(52116002)(38100700002)(5660300002)(53546011)(66946007)(31696002)(16526019)(2906002)(316002)(786003)(186003)(8936002)(31686004)(2616005)(45980500001)(43740500002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: =?Windows-1252?Q?zws+yyCZ74dZZbi8uuc2hAaohgDe++Jxl5Dw6csDuGVC7n2QzfW1rFYn?= =?Windows-1252?Q?kqPaZrbb//Bm/Nz0etN9y1X94LouckZAijTt7o6oLzKdQ5hzaLQAmmFH?= =?Windows-1252?Q?lA7K1g27bz34DDaGHWYiv5Wm5q7b1sEIjnUFdV6wQ3w0yD3nm/pp86yR?= =?Windows-1252?Q?yBvTrG8uhagnEuUzg1U8yOur41Is2nEHie/fVb9nYg54aPHs1oVris1K?= =?Windows-1252?Q?wuOno3x8e6rn6Xq4vlKbsisFQuHWecZfReVqTozxD8W2Wb+exAVVtLNt?= =?Windows-1252?Q?pKUntpWUacd0dwHXJYKHpF+ghZymZ71J3JAMf4eqOYomCo8rLFx4p7mA?= =?Windows-1252?Q?b5LWAH9fVkI+z7tAnnpAyP6CUhpE0hdpMyrVMEPcby73Al/Ddkup7nU3?= =?Windows-1252?Q?ZDAu8MNKehM/W+p701Ups0dAYqfQpl1VXBz7scpI/ajtZ3B82mQDzeaG?= =?Windows-1252?Q?1SH0c5TY+OV2R4HEa9yczyDZCZUyMZYBrXNDGTQDuii2HMxTbJGpIeAa?= =?Windows-1252?Q?OCTYVm5ToEszyHFrnE0vtXO6dQzgJ28teYjBmlusU4h5nehaXrwZ7C8C?= =?Windows-1252?Q?akD33CakQ9+Dy/ApivK5yk5My6b0UvXW6AQAxeQjhyz3QjPxrOHslA7D?= =?Windows-1252?Q?VP7QiKdR/qWBxiTc0XeS42oBxKhfeJfZ2A0M9a8EQFaEA46DgPSveb+J?= =?Windows-1252?Q?g9YbQFusZbs3WxsfGKKxGbwvTOlGuOsIILYhVezxxFyEu1QJqV4/GVq6?= =?Windows-1252?Q?eaRJ2DVmv5+elRcmmrtqya7q8l7RXx/WNQFA24I9FeZgryEo0Efh/gfN?= =?Windows-1252?Q?HfhUclFDn4C0tsqNW83EsKGmVFzgtoBTdbmsBRSlaUWN0OqSZXuuag9o?= =?Windows-1252?Q?bkxdPLRdpPpVmQaWQSa55+j5zI5eEoRvjxruJChpUj9NWWDji2YiQkzj?= =?Windows-1252?Q?SrPCSMSvk14/wKLR5yldCU8cMq7E3hy2qX6ojiToTFBCDTExY60shl8a?= =?Windows-1252?Q?534UczBhaisU0u4PCdTNSu/hbXgRuyVU/lFE+PEFq7fcltnmZdM44JWz?= =?Windows-1252?Q?G7uHzlrMyPYiQRJT/X0GlOQkOcMuFVVISLoFjB3cckQPV+CmhM35Xtm/?= =?Windows-1252?Q?kfuoT2gmPe3wYLjOdJRlxSDnvy+c9t4BWBAf9Ii3laORyf4Ji67C3G6T?= =?Windows-1252?Q?EFByS5GO0TKbELVp42H/O2hpytOLFylvM6o7F1iVFdwJWdXAkef/P8iZ?= =?Windows-1252?Q?U3jN6nQfpbzj3KNZD1QWuuytBKyrGOEQGYsB2oZSKkgP6tfIq5xa/gEj?= =?Windows-1252?Q?H/8oXH6xwUzARycSM5usxYww2Wggguv9vVUIRRqLbRZ8YjyV1npOrf0L?= =?Windows-1252?Q?A0aezqO8sGe/XxAEued6bXVoqs8WLdkyL//4mdDp1fxH44dbtJpahUOk?= =?Windows-1252?Q?7mp8cgzowWGx4hUC8gIsMXDcZ9jBbYCYABEg9gP2VkFSzzsgqdRx/iXI?= =?Windows-1252?Q?4DTlmC1o?= X-OriginatorOrg: cornell.edu X-MS-Exchange-CrossTenant-Network-Message-Id: be96a318-29bb-4656-f6ac-08d9161baed1 X-MS-Exchange-CrossTenant-AuthSource: BN7PR04MB4388.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 May 2021 14:30:40.6057 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 5d7e4366-1b9b-45cf-8e79-b14b27df46e1 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: vVi7+WUYy0pecYXDnw4o96Id0wW1NGqV1Kcw6B1R8IxakHNVER8Pr/ZqbB/eRJ1V6zlT8C9s6kwdVKnIuz+9tA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR04MB5716 X-Spam-Status: No, score=0.2 required=5.0 tests=BAYES_00, DKIM_INVALID, DKIM_SIGNED, JMQ_SPF_NEUTRAL, KAM_DMARC_STATUS, MSGID_FROM_MTA_HEADER, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 May 2021 14:30:45 -0000 On 5/4/2021 7:06 AM, Corinna Vinschen wrote: > On May 3 16:50, Ken Brown wrote: >> On 5/3/2021 3:48 PM, Ken Brown wrote: >>> On 5/3/2021 2:40 PM, Corinna Vinschen wrote: >>>> On May  3 12:56, Ken Brown wrote: >>>>> On 5/3/2021 11:45 AM, Corinna Vinschen wrote: >>>>>> 7. The idea of _mq_recv partial reads is entirely broken.  Given that >>>>>>      the information in the queue consists of header info plus payload, >>>>>>      the entire block has to be read, and then a new block with fixed >>>>>>      header and shortened payload has to be rewritten with bumped priority. >>>>>>      This in turn can only be performed by the AF_UNIX code, unless we >>>>>>      expect knowledge of the AF_UNIX packet layout in the mqueue code. >>>>> >>>>> The partial read is actually OK as is, since it's comparable to what happens >>>>> on a partial read from a pipe.  I already have AF_UNIX code (on the >>>>> topic/af_unix branch) that deals with that.  A boolean variable _unread >>>>> keeps track of whether there's unread data from a previous partial read.  If >>>>> so, the next read just reads data without expecting a header. >>>> >>>> Ok, never mind. >>>> >>>> One advantage of the mqueue when utilized as above would be that this >>>> kind of state info is not required.  The content of a packet would >>>> always be self-contained and bumping the priority would automagically >>>> move the packet content to the top of the queue.  But that's just >>>> idle musing at this point. >>> >>> I thought about that but rejected it for the following reason: Suppose >>> the receiver reads a message and tries to rewrite it with modified >>> header, shortened payload, and bumped priority.  The sender might have >>> already written more messages between the read and the write, and the >>> queue could be full. >>> >>> Now that I'm rethinking this, however, maybe we could get around that >>> problem with an internal _mq_lock function that would block senders >>> while the receiver decides whether it needs to do a partial read. >> >> Alternatively, _mq_recv could accept an _MQ_LOCK flag, which means "don't >> release the mutex", and then there could be an _mq_unlock function, which >> simply releases the mutex. > > That's an idea. However, I think this is something we can push back for > now, and ultimately we can use any of the above solutions which makes most > sense. Implemanting a defered unlock if required is not much of a problem. I've begun working on an mqueue-based implementation of AF_UNIX sockets, on a new topic/af_unix_mq branch. I've implemented accept and connect and lightly tested them, and I've implemented sendmsg (not yet tested). I'll start on recvmsg next, and then I'll be able to test sending/receiving. While working on sendmsg, I thought of another useful thing that the mqueue code could provide: an internal _mq_send function that returns EPIPE if no one has the mqueue open for reading. This plays the role of the STATUS_PIPE_IS_CLOSED macro in the pipe implementation. Ken