From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 8729 invoked by alias); 24 Mar 2017 17:11:06 -0000 Mailing-List: contact cygwin-help@cygwin.com; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner@cygwin.com Mail-Followup-To: cygwin@cygwin.com Received: (qmail 8304 invoked by uid 89); 24 Mar 2017 17:11:06 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-101.8 required=5.0 tests=AWL,BAYES_00,GOOD_FROM_CORINNA_CYGWIN,KAM_LAZY_DOMAIN_SECURITY,RCVD_IN_DNSWL_LOW,SPF_HELO_PASS autolearn=ham version=3.3.2 spammy=hung, sixteen, Testing, 2004 X-HELO: drew.franken.de Received: from mail-n.franken.de (HELO drew.franken.de) (193.175.24.27) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Fri, 24 Mar 2017 17:11:04 +0000 Received: from aqua.hirmke.de (aquarius.franken.de [193.175.24.89]) (Authenticated sender: aquarius) by mail-n.franken.de (Postfix) with ESMTPSA id 41319721E281C; Fri, 24 Mar 2017 18:11:02 +0100 (CET) Received: from calimero.vinschen.de (calimero.vinschen.de [192.168.129.6]) by aqua.hirmke.de (Postfix) with ESMTP id 760975E01FB; Fri, 24 Mar 2017 18:11:01 +0100 (CET) Received: by calimero.vinschen.de (Postfix, from userid 500) id 5D801A806AF; Fri, 24 Mar 2017 18:11:01 +0100 (CET) Date: Fri, 24 Mar 2017 17:52:00 -0000 From: Corinna Vinschen To: cygwin@cygwin.com Cc: Noah Misch Subject: Re: cygserver - Postgres Multiple connection Load Testing - Inifinte Loop Message-ID: <20170324171101.GI29995@calimero.vinschen.de> Reply-To: cygwin@cygwin.com Mail-Followup-To: cygwin@cygwin.com, Noah Misch References: <200408030333.i733XEXn023894@mx3.redhat.com> <20040803100612.GP31522@cygbert.vinschen.de> <20170321025614.GA2100214@tornado.leadboat.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="zhXaljGHf11kAtnf" Content-Disposition: inline In-Reply-To: <20170321025614.GA2100214@tornado.leadboat.com> User-Agent: Mutt/1.8.0 (2017-02-23) X-SW-Source: 2017-03/txt/msg00307.txt.bz2 --zhXaljGHf11kAtnf Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-length: 3417 Hi Noah, thanks for the report and especially the testcase. It took me a while to debug that, but I think I fixed it now. At least your testcase is working for me now. It also got faster, albeit always slower than Linux because of the communication overhead between processes and cygserver. On Mar 21 02:56, Noah Misch wrote: > On Tue, Aug 03, 2004 at 12:06:12PM +0200, Corinna Vinschen wrote: > > On Aug 2 20:33, sarbx-cygwin6344@mailblocks.com wrote: > > > This time around, cygserver does not eat CPU. But after 5 to 6=20 > > > concurrent > > > connections nothing seem to work, looks kind of hung. There is no=20 > > > activity in the Postgres > > > log file. Opening a new database connection also hangs. There is no=20 > > > activity on the machine. >=20 > > Any chance to create a simple testcase which uncovers that behaviour > > without involving a whole database system? >=20 > Attached test program reproduces it on Cygwin 2.7.0, Cygwin 1.7.5, and a = few > intermediate versions. The program creates sixteen processes that each > perform a tight loop over the following: >=20 > - select one of four semaphores > - reduce semaphore's value from 1 to 0 ("lock" it) > - raise semaphore's value from 0 to 1 ("unlock" it) >=20 > On GNU/Linux, AIX, and Solaris, the processes keep busy and finish one mi= llion > lock/unlock cycles apiece in a few minutes. On Cygwin, they hang within = a few > seconds and under one hundred cycles apiece. At that point, cygserver is > unresponsive to other clients; for example, "strace /bin/true", opening a= new > Cygwin terminal, "cat /proc/sysvipc/sem" and "cygserver -S" all hang. In= most > tests, cygserver was not consuming CPU while unresponsive. There are two problems here. - cygserver is using a defined number of threads in a thread pool for application requests. Every request is added to a request submission queue and handled by the next free thread in the pool. The default number of threads in the pool is 10. Each wait for a semaphore is blocking one thread. If more than the number of threads in the pool are supposed to wait on a semaphore the pool starves. Raising the pool size fixes this part, but the situation is still a bit unsatisfying. You may not know the load and the number of competing processes in every scenario beforehand, but raising cygservers thread pool to some really big value doesn't always make sense either. So what I did now is to allow cygserver to raise the number of worker threads on demand. That is, if a request is in the queue and all worker threads are busy, just create a new one. There's no way yet to drop threads again, but this should be a minor problem in scenarions which really have a lot of contention. - The code emulating BSD msleep/wakeup wasn't quite up to speed. I rewrote a major part of the code to be more robust and faster. I pushed a patchset now, and uploaded new developer snapshots for testing to https://cygwin.com/snapshots/ I'm also going to create a 2.8.0-0.4 test release later today. Please give it a try, and please note that *all* patches affect cygserver itself, so you have to test the new cygserver in the first place. The Cygwin DLL is not affected by the changes. Thanks, Corinna --=20 Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat --zhXaljGHf11kAtnf Content-Type: application/pgp-signature; name="signature.asc" Content-length: 819 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJY1VMlAAoJEPU2Bp2uRE+gZ7sP/A9hoFI5ZGWBUJXMo3q05FUL yte2+/bWhpyXfXkKVZ8HMmUUQetEHsq3FSSaXdCKJIxrWhh3zb0X+HHU+OXXLszF z9uhBgrIFlzQRXiomYL9MAtoK5iQSr8bX+1Mgnn7LViwLxFozziINaXZgaJjwcXV dpQd8a0KzNq95OJiq4gSAhZTH+r4h+IZEW/r8xDMFRulfPYAToXs7X0gH9S3zoEm xVnBwDXKOxaNo8zWtXBIQLOz19AI/Zcf/kjzrNeMuPkjkse2fvb76CvCYwRTahvE XSZAMhF8wICu5gLzoMBdbKjRMQuouE4xhuYsBwiTAJKbnGI3jcbm7rWrCjmMEYoI XljXigezGMAYDwLTI5XGTr+51jwHdQC3YmNxw2LKokr8NrNDnXRx6EcwISLzEaNo 21g3so7+ZJx7OyX7NgxZqCW4ju+AS60goLL3DGh2BkncW1yOXkIGG2YrQQYPcE/+ z8YqdX0fb8Sdk+/stLws6MY2PziUmlSzSEOVmZVv2IM1JHze+UUGrfilp8T6wy+n 1mD0w/C7ZkOL+R/5RF3e8YWHyGw/OoEzOCtv64CU5ZDspxDRYop+wwXtHC1UDHkV b3kEuGy+IAZ410ozFSMElPGhMGYnspwiXDFmfZt2iVo4YClsSBVFr55WaWNHMbgV KprsiHr8S04LoZhaHfIB =O6TX -----END PGP SIGNATURE----- --zhXaljGHf11kAtnf--