From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 10212 invoked by alias); 25 Jun 2014 21:14:01 -0000 Mailing-List: contact cygwin-help@cygwin.com; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner@cygwin.com Mail-Followup-To: cygwin@cygwin.com Received: (qmail 10121 invoked by uid 89); 25 Jun 2014 21:14:00 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-5.9 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.2 X-HELO: calimero.vinschen.de Received: from aquarius.hirmke.de (HELO calimero.vinschen.de) (217.91.18.234) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Wed, 25 Jun 2014 21:13:58 +0000 Received: by calimero.vinschen.de (Postfix, from userid 500) id E5FCC8E0757; Wed, 25 Jun 2014 23:13:55 +0200 (CEST) Date: Wed, 25 Jun 2014 21:14:00 -0000 From: Corinna Vinschen To: cygwin@cygwin.com Subject: Re: timeout in LDAP access Message-ID: <20140625211355.GA25116@calimero.vinschen.de> Reply-To: cygwin@cygwin.com Mail-Followup-To: cygwin@cygwin.com References: <20140617100011.GL23700@calimero.vinschen.de> <20140618083304.GV23700@calimero.vinschen.de> <20140618180102.GA27055@calimero.vinschen.de> <20140623090959.GA1803@calimero.vinschen.de> <20140624155851.GJ1803@calimero.vinschen.de> <20140625101526.GO1803@calimero.vinschen.de> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ZGiS0Q5IWpPtfppv" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-SW-Source: 2014-06/txt/msg00404.txt.bz2 --ZGiS0Q5IWpPtfppv Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-length: 3870 On Jun 25 22:44, Denis Excoffier wrote: > On 2014-06-25 12:15, Corinna Vinschen wrote: > >> Stay tuned. I'm rewriting the LDAP access code to perform all critical > >> LDAP calls in interruptible threads. The Windows LDAP calls don't > >> provide any kind of synchronization, only timeouts. I hoped to get aw= ay > >> with short timeouts but it seems I hoped in vain. > >>=20 > >> So the next iteration of this code will not use any timeout other than > >> the default LDAP network timeout of 2 minutes, but the calls will be > >> interruptible by signals. > >>=20 > >=20 > > No more artificial timeouts, but the LDAP calls will be interruptible by > > a signal now. > >=20 > > Also, if an error occurs during ad enumeration, getpwent/getgrent will > > return NULL with errno set accordingly. > >=20 > > Please test, > I did. Again, i instrumented ldap.cc by replacing all debug_printf() calls > with system_printf() because my /usr/bin/strace does not work. Again, i > tested with =E2=80=98getent passwd > result=E2=80=99 and 'db_enum: all=E2= =80=99. >=20 > I got the following message: > [ldap_init] getent 6024 cyg_ldap::connect_non_ssl: ldap_bind(xxxxxx.zzz) = 0x51 > and getent stops after the 376000 users in my own domain. No timeout occu= rred > but the enumeration was stopped by LDAP_SERVER_DOWN (0x51) [the xxxxxx.zzz > domain name has been edited here but it was completely new to me, never s= een > before]. You asked for errors being propagated up the chain to the getpwent/getgrent calls and that's exactly what happens now. There are a lot of LDAP error codes. How is Cygwin supposed to handle every one of them? Do we need a list of ignorable and non-ignorable error codes? Alternatively this gets reverted and Cywin does *not* break the search if an error occurs, but instead skips this domain and starts enumerating the next domain, just as before? > Also, there was a large delay (more than 2 min, say at least 8 minutes) b= etween > the end of output and the end of getent. I got one single system_printf > message (see above). I can't observe this. It needs debugging in your environment so I know which part of the source is responsible for this delay under what circumstances. (and I still think it's a crazy idea to enumerate 500K users) > More than that, i added system_printf("starting open in domain %W", domai= n) > immediately at the beginning of cyg_ldap::open, and run =E2=80=98getent p= asswd=E2=80=99 now during > one minute (wait 60s, then Control-C). I got 1080 =E2=80=98starting open = in domain (null)=E2=80=99 > messages on stderr and 1016 normal passwd entries on stdout. The discrepa= ncy > 1016 vs 1080 is ok because stdout was not properly flushed out. 60 seconds for 1016 user entries? That sounds incredibly slow. > It seems that > - domain is printed as =E2=80=98(null)=E2=80=99? Strange Not at all. This indicates the primary domain. > - there are as many open() calls as passwd entries in the output? The open function is called for every account, but that doesn't mean it really needs opening. That's what the early return is for. The code starts like this: int cyg_ldap::open (PCWSTR domain) { int ret =3D 0; /* Already open? */ if (lh) return 0; if ((ret =3D connect (domain)) !=3D NO_ERROR) goto err; [...] Did you add the system_printf before the "/* Already open? */" comment, by any chance? > Also strange > - EIO (or equivalent) is produced for LDAP_SERVER_DOWN, it probably shoul= d be > better if this were not the case? See above. > I suppose it will need more testing, but i=E2=80=99m currently unavailabl= e for tests, > by the way until Friday 08:00 UTC. No worries. Thanks for pulling this through. Corinna --=20 Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat --ZGiS0Q5IWpPtfppv Content-Type: application/pgp-signature Content-length: 819 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJTqzuTAAoJEPU2Bp2uRE+gE68P/AhvxlmB5uKB3QjblwQe4n4v HaPLFC5A0xec/WI8ctLZA7kDAEyU96HQrlhWifFUaaXIlG/V53tbuM3v+0UKFvQN zJ7sUctHD/bzidd0GPReJYcLSb5KFZ+5Xkq8fZi9KY2g7SUwGpwNAimEFHUzY6RI T1LIzgtlufuVmyGbjhYNLVeiWgMggwLA+T+S49WG+tbc1x+JfbzsQfvmOQvob0ar 1zHsLP9qI5CNiqYp3+hlpr0/g/hn07yd1I9wgvtf8iFEKcLAw2RrNHW815GpH9fx LPp/l4L8yvbthSDgMRreNhcz14khxL1+Lqv0aqS9nAgiwlhuaHpFmCdtkGXJmyBJ gaZ4PYV5+k5HxlkXTeMaxlPnG0g0HznkGwlN8uYs1WZ7ux/0OBnQLVDAL93BQ1G/ fzPJvUzbiYT4WP0h7ohYHecgE5Cx1BWK1NH6u1MWELSse18v4usEeMJnZGrYK4ZC tg/kkADMrsXWb992MS37QmFBPcCNRIjCeZD5kMAN4kFqnSjDBceBpGHa198LsDfg euVnDC6Nr2lPSvqxjxpudcZ8b/QbCpAZYtZHmiPj9uMLy+TaijkOqYT7Bmc+X+lz Ni5K7LqnjQmTOTNZUfa/WYZWMZaT6f4VrY6owhZ/hOz3fs2x2PBYxvxQmOIrOAs2 VSk+3sWZrox5omT3q4bG =HSym -----END PGP SIGNATURE----- --ZGiS0Q5IWpPtfppv--