From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 123752 invoked by alias); 3 Nov 2017 14:39:39 -0000 Mailing-List: contact cygwin-help@cygwin.com; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner@cygwin.com Mail-Followup-To: cygwin@cygwin.com Received: (qmail 123741 invoked by uid 89); 3 Nov 2017 14:39:39 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-2.6 required=5.0 tests=AWL,BAYES_00,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,RCVD_IN_SORBS_SPAM,SPF_PASS autolearn=no version=3.3.2 spammy=3200, opened, sk:resourc, millions X-HELO: mail-io0-f169.google.com Received: from mail-io0-f169.google.com (HELO mail-io0-f169.google.com) (209.85.223.169) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Fri, 03 Nov 2017 14:39:36 +0000 Received: by mail-io0-f169.google.com with SMTP id m81so6641670ioi.13 for ; Fri, 03 Nov 2017 07:39:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=aVtmNstVqoxiqmrT000lLTal54Vszk0H37SaWY4dhhY=; b=lyOfm5xCUgvp4/qtMYzcskMkynWsg15SMXo1Z0isTW8YXwgZUAnJHUbk383Eq/IVXd iPZwVZmGgdSd7cP9fi3G4rEGngEQVb37mEflAoBLxj90jRmX7dkhgqCC3plqWkyfKFHq 9RnWGn4uNrbSH1wAyGV4A91IHgvOSbhB1a4PcJgb4HCwazCmaEoaIzGkGfcPz2yX+cQ6 J5yH4jBbcwROPkOpb/sor1KKU6GlSPGYZZiKYKuOXPIx3aHQyps09hexzWv9robYt5iY Ktx1iYhRIca/Mgi9QvrklHdGNWn0oeG7MPoqyoH5EAd3XA3NI6nk3jTs9sgImbm6F0mA sgbA== X-Gm-Message-State: AMCzsaWxOaFsQvvJMNSTEVmg37TzveUfVrVi7ypaHGjHOojbuvYfr5bC HVJzvSrCPal+oJL2JUOErTBKZAOni4lTI8HWEYUgfvKj X-Google-Smtp-Source: ABhQp+RetAdQRi/LjwqfPivNRprbIpVuiLTVoxv0bSeX3LeR6gfkBPRlonH9PHr0ridUQkS3AODzaDB76f9vzsMuhrM= X-Received: by 10.107.33.18 with SMTP id h18mr9093603ioh.167.1509719974329; Fri, 03 Nov 2017 07:39:34 -0700 (PDT) MIME-Version: 1.0 Received: by 10.2.105.151 with HTTP; Fri, 3 Nov 2017 07:39:33 -0700 (PDT) From: Erik Bray Date: Fri, 03 Nov 2017 14:39:00 -0000 Message-ID: Subject: Bug(s) with creating large numbers of sockets To: cygwin@cygwin.com Content-Type: multipart/mixed; boundary="001a1141bbf6c92eb1055d151245" X-IsSubscribed: yes X-SW-Source: 2017-11/txt/msg00052.txt.bz2 --001a1141bbf6c92eb1055d151245 Content-Type: text/plain; charset="UTF-8" Content-length: 2498 Hi all, I found a few bugs in Cygwin w.r.t. creating large numbers of sockets. For example, Cygwin will gladly let you create up to RLIMIT_NOFILE sockets (examples in Python, where I found this problem): >>> import resource >>> import socket >>> resource.getrlimit(resource.RLIMIT_NOFILE) (256, 3200) >>> resource.setrlimit(resource.RLIMIT_NOFILE, (3200, 3200)) >>> socks = [socket.socket() for _ in range(3000)] # A bit fewer than the max but it doesn't matter However, if I try to do anything interesting with those sockets, such as poll on them, I get a rather unexpected error: >>> import select >>> poll = select.poll() >>> for sock in socks: ... poll.register(sock, select.POLLOUT) ... >>> poll.poll() Traceback (most recent call last): File "", line 1, in OSError: [Errno 14] Bad address After some playing around I found that I could make up to exactly 1365 sockets and use them without error. At 1366 I get the error. A very strange and arbitrary number. It turns out this is limited in Cygwin by the array in fhandler_socket.cc: 496 /* Maximum number of concurrently opened sockets from all Cygwin processes 497 per session. Note that shared sockets (through dup/fork/exec) are 498 counted as one socket. */ 499 #define NUM_SOCKS (32768 / sizeof (wsa_event)) ... 510 static wsa_event wsa_events[NUM_SOCKS] __attribute__((section (".cygwin_dll _common"), shared)); This choice for NUM_SOCKS is still seemingly small and pretty arbitrary, but at least it's a choice, and probably well-motivated. However, I think it's a problem that it's defined in terms of sizeof(wsa_event). On 32-bit Cygwin this is 16, so NUM_SOCKS is 2048 (a less strange number), whereas on 64-bit Cygwin sizeof(wsa_event) == 24 (due to sizeof(long) == 8, plus alignment), so we are limited to...1365 sockets. If we have to set a limit I would just hard-code it to 2048 exactly. I understand that the overhead associated with sockets in Cygwin probably limits us from having 10s of thousands (much less millions) and that's OK--I'm not trying to run some kind of C10K challenge on Cygwin :) The other problem, then, seems to be a bug in fhandler_socket::init_events(). It doesn't check the return value of search_wsa_event_slot(), which returns NULL if the wsa_events array is full (and the socket is not a shared socket). There's not a great choice here for error code, but setting ENOBUF seems like the best option. Please see attached patch. Best, Erik --001a1141bbf6c92eb1055d151245 Content-Type: application/octet-stream; name="0001-Fix-two-bugs-in-the-limit-of-large-numbers-of-socket.patch" Content-Disposition: attachment; filename="0001-Fix-two-bugs-in-the-limit-of-large-numbers-of-socket.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: f_j9k052xv0 Content-length: 2217 RnJvbSA5NjBlNDAyYzE0ZGI0MzEzMGU2MDU0ZDBmZTZhNGQwYWY3MWI0YWNm IE1vbiBTZXAgMTcgMDA6MDA6MDAgMjAwMQpGcm9tOiAiRXJpayBNLiBCcmF5 IiA8ZXJpay5tLmJyYXlAZ21haWwuY29tPgpEYXRlOiBGcmksIDMgTm92IDIw MTcgMTU6MzM6MTcgKzAxMDAKU3ViamVjdDogW1BBVENIXSBGaXggdHdvIGJ1 Z3MgaW4gdGhlIGxpbWl0IG9mIGxhcmdlIG51bWJlcnMgb2Ygc29ja2V0czoK CiogRml4IHRoZSBtYXhpbXVtIG51bWJlciBvZiBzb2NrZXRzIGFsbG93ZWQg aW4gdGhlIHNlc3Npb24gdG8gMjA0OCwKICBpbnN0ZWFkIG9mIG1ha2luZyBp dCByZWxhdGl2ZSB0byBzaXplb2Yod3NhX2V2ZW50KS4KCiogUmV0dXJuIGFu IGVycm9yIGFuZCBzZXQgZXJybm89RU5PQlVGIGlmIGEgc29ja2V0IGNhbid0 IGJlIGNyZWF0ZWQKICBkdWUgdG8gdGhpcyBsaW1pdCBiZWluZyByZWFjaGVk LgotLS0KIHdpbnN1cC9jeWd3aW4vZmhhbmRsZXJfc29ja2V0LmNjIHwgMTEg KysrKysrKysrLS0KIDEgZmlsZSBjaGFuZ2VkLCA5IGluc2VydGlvbnMoKyks IDIgZGVsZXRpb25zKC0pCgpkaWZmIC0tZ2l0IGEvd2luc3VwL2N5Z3dpbi9m aGFuZGxlcl9zb2NrZXQuY2MgYi93aW5zdXAvY3lnd2luL2ZoYW5kbGVyX3Nv Y2tldC5jYwppbmRleCA3YTZkYmRjLi5iOGVkYTU3IDEwMDY0NAotLS0gYS93 aW5zdXAvY3lnd2luL2ZoYW5kbGVyX3NvY2tldC5jYworKysgYi93aW5zdXAv Y3lnd2luL2ZoYW5kbGVyX3NvY2tldC5jYwpAQCAtNDk2LDcgKzQ5Niw3IEBA IGZoYW5kbGVyX3NvY2tldDo6YWZfbG9jYWxfc2V0X3NlY3JldCAoY2hhciAq YnVmKQogLyogTWF4aW11bSBudW1iZXIgb2YgY29uY3VycmVudGx5IG9wZW5l ZCBzb2NrZXRzIGZyb20gYWxsIEN5Z3dpbiBwcm9jZXNzZXMKICAgIHBlciBz ZXNzaW9uLiAgTm90ZSB0aGF0IHNoYXJlZCBzb2NrZXRzICh0aHJvdWdoIGR1 cC9mb3JrL2V4ZWMpIGFyZQogICAgY291bnRlZCBhcyBvbmUgc29ja2V0LiAq LwotI2RlZmluZSBOVU1fU09DS1MgICAgICAgKDMyNzY4IC8gc2l6ZW9mICh3 c2FfZXZlbnQpKQorI2RlZmluZSBOVU1fU09DS1MgICAgICAgKCh1bnNpZ25l ZCBpbnQpIDIwNDgpCiAKICNkZWZpbmUgTE9DS19FVkVOVFMJXAogICBpZiAo d3NvY2tfbXR4ICYmIFwKQEAgLTYyMyw3ICs2MjMsMTQgQEAgZmhhbmRsZXJf c29ja2V0Ojppbml0X2V2ZW50cyAoKQogICAgICAgTnRDbG9zZSAod3NvY2tf bXR4KTsKICAgICAgIHJldHVybiBmYWxzZTsKICAgICB9Ci0gIHdzb2NrX2V2 ZW50cyA9IHNlYXJjaF93c2FfZXZlbnRfc2xvdCAobmV3X3NlcmlhbF9udW1i ZXIpOworICBpZiAoISh3c29ja19ldmVudHMgPSBzZWFyY2hfd3NhX2V2ZW50 X3Nsb3QgKG5ld19zZXJpYWxfbnVtYmVyKSkpOworICAgIHsKKyAgICAgIHNl dF9lcnJubyAoRU5PQlVGUyk7CisgICAgICBOdENsb3NlICh3c29ja19ldnQp OworICAgICAgTnRDbG9zZSAod3NvY2tfbXR4KTsKKyAgICAgIHJldHVybiBm YWxzZTsKKyAgICB9CisKICAgLyogc29jayB0eXBlIG5vdCB5ZXQgc2V0IGhl cmUuICovCiAgIGlmIChwYy5kZXYgPT0gRkhfVURQIHx8IHBjLmRldiA9PSBG SF9ER1JBTSkKICAgICB3c29ja19ldmVudHMtPmV2ZW50cyA9IEZEX1dSSVRF OwotLSAKMi44LjMKCg== --001a1141bbf6c92eb1055d151245 Content-Type: text/plain; charset=us-ascii Content-length: 219 -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple --001a1141bbf6c92eb1055d151245--