From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from nihcesxway5.hub.nih.gov (nihcesxway5.hub.nih.gov [128.231.90.120]) by sourceware.org (Postfix) with ESMTPS id 9D10038485B8 for ; Sat, 9 Jul 2022 15:37:03 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 9D10038485B8 X-SBRS-Extended: Low X-IronPortListener: ces-out X-IronPort-AV: E=Sophos;i="5.92,258,1650945600"; d="scan'208";a="340863691" Received: from unknown (HELO mail.nih.gov) ([156.40.79.162]) by nihcesxway5.hub.nih.gov with ESMTP/TLS/AES256-GCM-SHA384; 09 Jul 2022 11:37:02 -0400 Received: from nihexb3.nih.gov (156.40.79.163) by nihexb2.nih.gov (156.40.79.162) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.9; Sat, 9 Jul 2022 11:37:02 -0400 Received: from GCC02-BL0-obe.outbound.protection.outlook.com (156.40.79.133) by nihexb3.nih.gov (156.40.79.163) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.9 via Frontend Transport; Sat, 9 Jul 2022 11:37:02 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Rd3BDZ3ArQb3iLvyJR5XRmuMTta4YO10uXFTWpNKryscw6mu7JSapjiXlaym/wI4zCi53Mil4BiPGZ67fQeEm4BbO60dLcgc1KRpsowxKMUPTmm8LruGPKYbi3yb8zXyXO8bnCR0sa17oD22HKL4dcigHlPDtsdF008hGZnmkZkbGlCRajmjNQcgZ/9CKsKbY6zpSRZT45kWlDQtZ2qZ5aCV1qTB5XAQvHGy7rviRY61UoCOqGXnT75JWjWi5hkdUIQjDVmE2Z2yV6VEhNYYtRlklTa7dmFmivbZzLGPU/XL2Vc7YG2clldJIihmnk3FXzE+5MCxGQRG09vBlfA64A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=yGbHjyZBFNjt737/bwSctXwRwr9Pi8vuQQTM5pP+BIA=; b=MNm147Qj21yWg69amUDRLwgnguR+jxyd4z844mdDl0qzP1HoPaRxnG3+G32Ya6BoQiGHCFAmg9Lq3/2WH/U7z8Sdagrnjk8XuhaQravAsah0fdVfrIRrLMIXHY/ad0h3YrUrEus8AIFL4sR1jgmKxlIz6zIvkn5LakVjGfgC99YBZeGStg6Kvmyk/s9pCH3y/4SHY/b6MRP8HrKFbp68vaT8Xp4ooMORtdrj6lLBw6QIY5YI0us3cQOQZ/9ytoeDk4iJJtFIiu/Hw6KvIZoe3yOgJyPZBKJvaCDiX05inyV4jMgwvYWIeh1fZrnsfOi17BtzA3HlWBiXYrUZmP31Fg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ncbi.nlm.nih.gov; dmarc=pass action=none header.from=ncbi.nlm.nih.gov; dkim=pass header.d=ncbi.nlm.nih.gov; arc=none Received: from DM8PR09MB7095.namprd09.prod.outlook.com (2603:10b6:5:2e3::14) by DM6PR09MB5989.namprd09.prod.outlook.com (2603:10b6:5:262::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.15; Sat, 9 Jul 2022 15:37:00 +0000 Received: from DM8PR09MB7095.namprd09.prod.outlook.com ([fe80::282a:38f8:c3b5:8f5b]) by DM8PR09MB7095.namprd09.prod.outlook.com ([fe80::282a:38f8:c3b5:8f5b%8]) with mapi id 15.20.5417.016; Sat, 9 Jul 2022 15:37:00 +0000 From: "Lavrentiev, Anton (NIH/NLM/NCBI) [C]" To: "'cygwin@cygwin.com'" Subject: Spurious / persistent "exception" condition in half-closed sockets Thread-Topic: Spurious / persistent "exception" condition in half-closed sockets Thread-Index: AdiTqIzsqd7Y2Y4cQ8Wpylby9/6cSQ== Date: Sat, 9 Jul 2022 15:37:00 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 11a7ec06-a45c-4716-09da-08da61c0ddb0 x-ms-traffictypediagnostic: DM6PR09MB5989:EE_ x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: k+vcjE2Eh7CAWFWAPJ2Yf9yzEewK5EO4XYNwLvpM0ynHE5oJoy+gAGdoE00opeNZZkAKrBPeCUsDxc9jfdDqbmFBEipym8wuSAMOPZGkhk/8/H8KGU0Itkk+cyWCpuSU2yGOsNsQtNFSf4+Qv/wwziaRHzcsQPubSXKLXVvm5wWJcuECX1ZVpLI5Q5obxyTx7uhhPgOr1APgNkA6IqCIzps3yE6C7862FmAtoH+EFYwbp/sJe60xYWUetGzjCRowRXvtXUaG/4k7XpvFH6FEkRwFpM62trSzMwOnE78jcyuS8vNpgrtYo0qTmGsF+NmHvSBNmfDJclzggFvbLxQ2/LRVbzNt4PBhrcaCOVZJzfJO+dpW6mRldzTqbveSQxg9/5/oMd6nLAieyTajGJk0Z5np/3R2xEdBc78CmdjswlF0E4qUDXKDhEqPiwVh1yziK3QReG4gRcOcyZiTDWnvfB76XI0fLC+CUayRK0nRQB5EK1r92OEb3YY7+HanGkJpslXYRJgKN4aeX/1P8ZX/09ThR30lmob/b3MsIPkKyueKIaB2P3H69zSf40U4LD2Dd0Eeo/vRkrhVAOL2fQC2SxBmdg5wabrTG4GLG2362f+OBF5iDgC0UCcIG1GSaNS5/1Dsx9w2FZ1KDZxRNhWV7ofa47hyLdKYcbjVkCvLe4QJneAr//SfmCfrz+eCzSuRKM1OC6OlPsombg9ZF4/t0L3a6hmzf+TIgwllQB5IludqMp017vE7Mwm4/Yz7OHFkah96teYH1+105EduASagIy+KiguKHlv+pbvveL/XtrFN4SlNDqA4y7eZTIru21u+I/cpOzhGHk9mKzlONCk3DA== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM8PR09MB7095.namprd09.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(4636009)(366004)(33656002)(2906002)(8936002)(5660300002)(498600001)(38100700002)(186003)(86362001)(83380400001)(52536014)(26005)(122000001)(55016003)(6506007)(76116006)(9686003)(66556008)(71200400001)(8676002)(64756008)(66476007)(66446008)(38070700005)(6916009)(66946007)(7696005)(491001); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?2XDiKBG0X8JSRVYu2bhnqwWDp+5Qjc0n8UItDAADcR77def3T7sgTY08UCJR?= =?us-ascii?Q?uMP4p+uIU2Cm1ZQ0BJ4DReAQJqXVL8hhhFoOBr0chGBPf9qPBozSVlS49mpD?= =?us-ascii?Q?H1cp2/as/cI9UQ16ErpY8raR1k5gQZiszm4Gqme9iqBK6RdCEA/D97TZQRbh?= =?us-ascii?Q?zs5jUE84VZztPIAIZkK+aqd4r4+Hghe41daL4nPKBEMWwnpaYQ+N1j3gbh+t?= =?us-ascii?Q?cT9szG35AQNK0DOg/6UiD2gSUaoiJfzoG04o3T//DcR1Tzjs7cSXCHKtaBzz?= =?us-ascii?Q?Kbv+HAb6y/IaVQ4MMqkEZRYuz7CA3PKA/gAHuN79eMas0ddlnPsaH02LscCC?= =?us-ascii?Q?tP2z4tHe3BpwOEkZDor/hsdv43BsKJSvmvd0qubFxUi66waRr6QbuMe0OROV?= =?us-ascii?Q?JYrut8kkRe59FcAXJKD/bSZ2aE158+0XedB3CtQY4iMbbdJr/smMImyFrsyh?= =?us-ascii?Q?s22Gm8PVtQnlXzkv7c1BvVl77ezg2ijFNYcP5uyEP5ERS/YHjxyQh8WY3hqg?= =?us-ascii?Q?OOF4Ys7eCPiMKOlYwUyl+RHi7zusz7NPt4lGUmL8fHO2pXq5O0/dwBoVgj5D?= =?us-ascii?Q?LD4OLTJhTGUHLbEdQjslMuVzmpNWqbqwPn15pomEm6DydGakzX7JLJ4mGU7J?= =?us-ascii?Q?nM4JRJ3N6Iq6kds7cB7Y4Xnu65KcbSQufFjrZq5HDSZS0D0tvrou2aM7L+1w?= =?us-ascii?Q?c95UODLgCibu16Ufs42+TXBvjMnR9ukLHzdCowkqLdzZWxuUk4QFKbVHiHRl?= =?us-ascii?Q?FnY9JeIQtuaSrhmBgs5GRpNyQMxdWvmfKo0pUpkBVeTkG6m9CgF+Dq/wq5AG?= =?us-ascii?Q?uT9OEsd40Huz5Jt/apgBgNu5OcUhAIVF064+KA017muZIVrL/N5TqysWUr6N?= =?us-ascii?Q?p6h0U51cwhIa7VoRcpKUKKejzTWOHB9l4ppiGTOQqPHvnl0EFOyBYBB62BLN?= =?us-ascii?Q?VHWuFX1kZmuFp5i7M0r0aWnpIWMCedAI+RIX69wQfbiazy3UYLPH+Kr/cs9v?= =?us-ascii?Q?WtFE5aPkZBO+s3P5zKfazKl8STXOcecuBIz10Z+XZ0vekolA6i/7WcB01Dhp?= =?us-ascii?Q?9ncr9P/fgnfyCCT/XQFQbfvNW4cg8SC/Id2u+H9gMV+TWOrDcN1h17m9fpgj?= =?us-ascii?Q?LXU85RvUik7BjT9EU0VIynacy4dipPPEyYmiMJsDDTgndO1AylWcWrVBqU1v?= =?us-ascii?Q?FOL1YqehoaoHiQ5nbcNYZCiNgFQ1VduNWflU5ZloY7/x7EB3J1E7RG6NrbnX?= =?us-ascii?Q?Pw/NrNaS8RIng3ehVeye+nnjJ1vUdJGZOBYDfQ16GU2RFvFlYx7+C/HAGHe4?= =?us-ascii?Q?oClhgidAEyDip5uM/hJDL5W36itDNIL4VA4XF9HlWe3gkVfURjhGMf9QO5Ba?= =?us-ascii?Q?QModYRH2PGgHuE8D85vCu7mpyNC39IS7Pv1ZattPDPg1yVT/xM0pjt4bGpqY?= =?us-ascii?Q?VnyFF8U3qjYQmGmP8z7mdKc5BF1gOYzMkXq6pIylc2utiNTqe05OSkp4G31m?= =?us-ascii?Q?CvFj1T2TG48xutiOxpbmQfBgWPOBeZENpSbw/Bv+jyMPSr67CHgHAPzwGRGA?= =?us-ascii?Q?8zgdW5Jr/KQfWwpgNwM=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8PR09MB7095.namprd09.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 11a7ec06-a45c-4716-09da-08da61c0ddb0 X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Jul 2022 15:37:00.6950 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 14b77578-9773-42d5-8507-251ca2dc2b06 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR09MB5989 X-OriginatorOrg: ncbi.nlm.nih.gov X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE, T_SPF_HELO_PERMERROR autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: cygwin@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: General Cygwin discussions and problem reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Jul 2022 15:37:05 -0000 Hi all, It took me awhile to figure this one out, but I think I have a good test ca= se to demonstrate a (rather serious, actually) issue with Cygwin sockets and sele= ct/poll. In short, when a reading end of a socket half closes for write (basically, = signaling the other end of no more data to expect, resulting in TCP FIN and, subsequently= , EOF in the other end's read()), if that end keeps reading the still incoming remaining data,= it will face with a lot of "exception" conditions, which are just spurious. That will also b= urn CPU instead of doing a proper wait (so a read() failed with EAGAIN, and then waited for wi= th select() (or poll() -- the same issue) will be attempted again immediately (as the s= ocket would be reported as "ready" with an "exception"), and result in another EAGAIN, etc etc... Eventually, there will be successful reads squeezed in between, though... Here's the "client" code ("server" code follows): $ cat client.c #include #include #include #include #include #include #include #include #include #include static void error(const char* what) { fflush(stdout); perror(what); exit(1); } int main(int argc, const char* argv[]) { struct sockaddr_in sin; size_t total =3D 0; int c =3D socket(AF_INET, SOCK_STREAM, 0); if (c =3D=3D -1) error("socket"); memset(&sin, 0, sizeof(sin)); sin.sin_family =3D AF_INET; sin.sin_addr.s_addr =3D htonl(INADDR_LOOPBACK); sin.sin_port =3D htons(atoi(argv[1])); if (connect(c, (struct sockaddr*) &sin, (socklen_t) sizeof(sin)) !=3D 0= ) error("connect"); if (fcntl(c, F_SETFL, fcntl(c, F_GETFL, 0) | O_NONBLOCK) =3D=3D -1) error("fcntl"); #ifdef BUG if (shutdown(c, SHUT_WR) !=3D 0) error("shutdown"); #endif for(;;) { char buf[1000]; ssize_t n =3D read(c, buf, sizeof(buf)); if (n > 0) { printf("%zu byte%s received from server\n", n, &"s"[n=3D=3D1]); total +=3D n; continue; } if (n =3D=3D 0) { printf("Connection closed, %zu byte%s received\n", total, &"s"[total=3D=3D1]); break; } if (errno !=3D EAGAIN && errno !=3D EWOULDBLOCK) error("read"); fflush(stdout); perror("read"); for (;;) { fd_set rfds, efds; struct timeval tv; int m; FD_ZERO(&rfds); FD_ZERO(&efds); FD_SET(c, &rfds); FD_SET(c, &efds); memset(&tv, 0, sizeof(tv)); tv.tv_sec =3D 2; printf("Waiting...\n"); m =3D select(c + 1, &rfds, 0/*wfds*/, &efds, &tv); if (!m) continue; if (m < 0) error("select"); if (FD_ISSET(c, &efds)) { printf("Exception??\n"); break; } if (FD_ISSET(c, &rfds)) { printf("Read-ready!\n"); break; } error("select bug"); abort(); } } close(c); printf("Bye-bye\n"); return 0; } $ cat server.c #include #include #include #include #include #include #include #include #include static void error(const char* what) { fflush(stdout); perror(what); exit(1); } int main(int argc, const char* argv[]) { struct sockaddr_in sin; int s =3D socket(AF_INET, SOCK_STREAM, 0); if (s =3D=3D -1) error("socket"); memset(&sin, 0, sizeof(sin)); sin.sin_family =3D AF_INET; sin.sin_addr.s_addr =3D htonl(INADDR_LOOPBACK); sin.sin_port =3D htons(atoi(argv[1])); if (bind(s, (struct sockaddr*) &sin, (socklen_t) sizeof(sin)) !=3D 0) error("bind"); if (listen(s, 1) !=3D 0) error("listen"); for(;;) { size_t total =3D 0; socklen_t sinlen =3D sizeof(sin); int c =3D accept(s, (struct sockaddr*) &sin, &sinlen); if (c < 0) error("accept"); printf("Client accepted\n"); for (;;) { char buf[1000]; size_t len =3D rand() & 255; ssize_t n =3D write(c, buf, len ? len : 1); if (n <=3D 0) error("write"); total +=3D n; printf("%zu byte%s sent to client\n", n, &"s"[n=3D=3D1]); if (rand() & 1) usleep(100); if (!(rand() % 11)) { printf("Closing connection, %zu byte%s sent\n", total, &"s"[total=3D=3D1]); break; } } close(c); } } $ cc -Wall -o server server.c $ cc -Wall -o client client.c Start the server (which just sends random garbage to the client, once it's = accepted) in a separate Cygwin terminal: $ ./server 5555 Now run the client from another Cygwin terminal: $ ./client 5555 You should see the client connecting and receiving (maybe sometimes waiting= ) but never having a blank read (EAGAIN) after a successful select() (that wa= s read-ready). Try running the client a few times to see how it works. Now, since the client is not sending anything (or done sending, in the real= case scenario), it'd want to notify the server that it's only going to receive (by issuing a shutdown() call). Recompile the client with -DBUG enabled: $ cc -Wall -DBUG -o client client.c $ ./client 5555 When you start the client again, you'd see a ton of Exceptions, and all the= waits (select(), but poll() works exactly the same say, checked) return immediate= ly and the client keeps spinning around the read()s -- most of them are blank with= EAGAIN. In the end, the client does get everything sent to it, though, but with A L= OT of unnecessary CPU cycles. Now suppose that you have thousands of such client= s, that would create a lot of unnecessary contention. Try running the client = a few times to see how disastrous those blank reads can be in numbers! That's not a correct behavior -- you can check that same code running it on= Linux (or BSD -- Mac). I don't think there's any "exception" in the socket, to b= egin with. Also, it looks like the condition is simply stuck in there and is no= t properly re-evaluated (as the I/O still progressing). Thanks for looking! Anton Lavrentiev Contractor NIH/NLM/NCBI