From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 124341 invoked by alias); 17 Aug 2018 18:44:13 -0000 Mailing-List: contact cygwin-help@cygwin.com; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner@cygwin.com Mail-Followup-To: cygwin@cygwin.com Received: (qmail 124290 invoked by uid 89); 17 Aug 2018 18:44:07 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-101.2 required=5.0 tests=AWL,BAYES_00,GOOD_FROM_CORINNA_CYGWIN,KAM_LAZY_DOMAIN_SECURITY,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.2 spammy= X-HELO: mout.kundenserver.de Received: from mout.kundenserver.de (HELO mout.kundenserver.de) (212.227.126.135) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Fri, 17 Aug 2018 18:44:05 +0000 Received: from calimero.vinschen.de ([24.134.7.25]) by mrelayeu.kundenserver.de (mreue005 [212.227.15.167]) with ESMTPSA (Nemesis) id 0MMrhT-1fmazd3Tyx-008YgN for ; Fri, 17 Aug 2018 20:44:03 +0200 Received: by calimero.vinschen.de (Postfix, from userid 500) id 10FE0A81F2F; Fri, 17 Aug 2018 20:44:02 +0200 (CEST) Date: Fri, 17 Aug 2018 20:36:00 -0000 From: Corinna Vinschen To: cygwin@cygwin.com Subject: Re: memory reported in /proc/pid/status is wrongly scaled Message-ID: <20180817184402.GW3747@calimero.vinschen.de> Reply-To: cygwin@cygwin.com Mail-Followup-To: cygwin@cygwin.com References: <20180817171454.GV3747@calimero.vinschen.de> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="JKvC8Wn6JllzUPdT" Content-Disposition: inline In-Reply-To: <20180817171454.GV3747@calimero.vinschen.de> User-Agent: Mutt/1.9.2 (2017-12-15) X-SW-Source: 2018-08/txt/msg00239.txt.bz2 --JKvC8Wn6JllzUPdT Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-length: 3074 On Aug 17 19:14, Corinna Vinschen wrote: > On Aug 17 16:05, Livio Bertacco wrote: > > Hi, > > While playing with reading process memory usage in Linux and cygwin, I > > found that cygwin reports too large values in /proc/self/status (in 2.1= 0.0 > > and earlier). > > Whenever I was allocating a few kB in my test program, the VmSize line = in > > /proc/self/status was growing several times faster. > >=20 > > Small bash script to show the issue: > > #!/bin/bash > > pid=3D$$ > > vmsizesplit=3D($(grep VmSize /proc/$pid/status)) > > vmsize1=3D"${vmsizesplit[1]}" > > echo Initial memory reported in status: $vmsize1 kB > > echo Allocating a 1000 kB string (bash can use more memory) > > eat=3D$(printf '%1024000s') > > vmsizesplit=3D($(grep VmSize /proc/$pid/status)) > > vmsize2=3D"${vmsizesplit[1]}" > > echo Current memory reported in status: $vmsize2 kB > > echo Difference is $[$vmsize2-$vmsize1] kB > >=20 > > Running this in cygwin on my laptop I get: > > Initial memory reported in status: 84928 kB > > Allocating a 1000 kB string (bash can use more memory) > > Current memory reported in status: 106880 kB > > Difference is 21952 kB > >=20 > > While bash may use quite more than 1000 kb in this case, 22x times larg= er > > doesn't seem right. > >=20 > > Checking source file fhandler_process.cc, the > > function format_process_status which writes the "status" proc file > > retrieves memory usage via get_mem_values. Get_mem_values obtains that = info > > from NtQueryInformationProcess/PagefileUsage which is in bytes, then it > > scales it to pages dividing by wincap.page_size: > > 1515: *vmsize =3D vmc.PagefileUsage / wincap.page_size (); > >=20 > > Then format_process_status scales it back, in theory to bytes, and shif= ts > > it by 10 bits in order to print it out in kB: > > 1219: unsigned page_size =3D wincap.allocation_granularity (); >=20 > Looks like this is the bug. get_mem_values returns all values > in multiple of OS page_size (4K), but format_process_status multiplies > with allocation_granularity (64K), leading to 16 times overallocation > value. The other caller of get_mem_values, format_process_statm, > returns number of pages. This must be expressed in multiples of > allocation_granularity since that's the virtual page_size in Cygwin. > But in case of format_process_status we're looking at KB values, so > patch 8a32c24a7bdb0, replaceing page_size with allocation_granularity, > was incorrect. >=20 > Good catch! >=20 > I'll revert patch 8a32c24a7bdb0 for 2.11.0. On second thought there's more wrong than just that. Just dividing by page_size or allocation_granularity results in too small values. I applied patches to return more correct values and I made sure the values in status and statm are consistently rounded up to Cygwin's page size of 64K. I uploaded new developer snapshots to https://cygwin.com/snapshots for testing. Please give them a try. Thanks, Corinna --=20 Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat --JKvC8Wn6JllzUPdT Content-Type: application/pgp-signature; name="signature.asc" Content-length: 833 -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEoVYPmneWZnwT6kwF9TYGna5ET6AFAlt3F3EACgkQ9TYGna5E T6C9fQ/+LGqf6j3cLdjeUq5Gih26eJaA5p0wKlaeAbQAD+yGQmNwv5sQp2eOPDjV m2uwzhUtNAnWDfnWwQ7N14suPNuymr3cn824lcJO9hH9UXbHYur18YrRxL16WFEa 9J7T5Zow9xetEjcEuOJ412l/k4q5fTE46r1waUp4RO2Q4RuAhjaU6gQfm1/IR2Mh NiPMKqv8GZdQK+z+t11Nc+AQjjNqRM0Ke+XnW2AMNC9GacHULywYst0zVIF6TJDV R+Iro69MdSxoDkuCy2NIHOh5EVGfbds21U38IDOF4uEVSQTMWMmCqrmx12mIUKNp KMZfvL7yWha/tq9p4gEWY2DftZo+qGCGGZMpeID71ionFxWZhwzDXiNo3WorjeqI 1G358TxHPgBXL8txFtS0IXDncRf7epMEnjj5qbFeFczgU/C3HBSmETx5O0w8mUU0 b/IYcONHPKpqX4ljlTjNj1vpCLiUM+O0Trxgu2ofKB/SZWn44BUB5qagVkuVC4zm r6nSv+DBCUysaujvxX4893061BNdEVn/rUn4CSwzqB+e3C5HijVfCYljE1c3SyqV r03hhV6ol8YTKqJJvXYLmxZPj69kgVVqqhdsbHacNlXoQPjbi7ekOmfE1ttWc8kl ohC3Jsq6keuKoZtn/XDuhi6P9KiigJJGdSqh5CQf3EMwnC2cW0M= =lIgF -----END PGP SIGNATURE----- --JKvC8Wn6JllzUPdT--