From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gnu.wildebeest.org (gnu.wildebeest.org [45.83.234.184]) by sourceware.org (Postfix) with ESMTPS id 5CF19383E82E for ; Mon, 30 May 2022 15:36:49 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 5CF19383E82E Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=klomp.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=klomp.org Received: from tarox.wildebeest.org (83-87-18-245.cable.dynamic.v4.ziggo.nl [83.87.18.245]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by gnu.wildebeest.org (Postfix) with ESMTPSA id 770BE30006D6; Mon, 30 May 2022 17:36:46 +0200 (CEST) Received: by tarox.wildebeest.org (Postfix, from userid 1000) id 44B70404418E; Mon, 30 May 2022 17:36:45 +0200 (CEST) Message-ID: Subject: Re: [Bug debuginfod/28577] Make run-...-fd-prefetch-caches.sh test something From: Mark Wielaard To: Noah Sanci Cc: elfutils-devel@sourceware.org Date: Mon, 30 May 2022 17:36:45 +0200 In-Reply-To: References: Content-Type: multipart/mixed; boundary="=-mITBmRTQfMfpRUPwP+3/" X-Mailer: Evolution 3.28.5 (3.28.5-10.el7) Mime-Version: 1.0 X-Spam-Status: No, score=-7.5 required=5.0 tests=BAYES_00, GIT_PATCH_0, JMQ_SPF_NEUTRAL, KAM_BADIPHTTP, KAM_DMARC_STATUS, KAM_LOTSOFHASH, NUMERIC_HTTP_ADDR, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: elfutils-devel@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Elfutils-devel mailing list List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 May 2022 15:36:52 -0000 --=-mITBmRTQfMfpRUPwP+3/ Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Noah, Hope you don't mind me adding elfutils-devel back to the CC. On Fri, 2022-05-27 at 15:32 -0400, Noah Sanci wrote: > On Fri, May 27, 2022 at 12:29 PM Mark Wielaard > wrote: >=20 > > On Wed, 2022-05-25 at 14:44 -0400, Noah Sanci via Elfutils-devel > > wrote: > > > PR28577: Make run-debuginfod-fd-prefetch-caches.sh test > > > something > > >=20 > > > Update to the run-debuginfod-fd-prefetch to make the test more > > > sound. > >=20 > > Thanks. And according to Evgeny this makes the test pass for him: > > https://sourceware.org/bugzilla/show_bug.cgi?id=3D29180#c2 >=20 > Good to hear. > > But I find it a little hard to review, it would have been helpful > > to > > get a bit more comments/documentation what exactly is being tested > > now. >=20 > Fair, I should have made what was being tested/why clearer. Thanks. I also understand now that I read the test logic the wrong way around. > > > diff --git a/tests/run-debuginfod-fd-prefetch-caches.sh > > > b/tests/run- > > > debuginfod-fd-prefetch-caches.sh > > > index 66a7a083..50ffc4ac 100755 > > > --- a/tests/run-debuginfod-fd-prefetch-caches.sh > > > +++ b/tests/run-debuginfod-fd-prefetch-caches.sh > > > @@ -21,22 +21,22 @@ > > > # for test case debugging, uncomment: > > > set -x > > > unset VALGRIND_CMD > > > +mkdir R Z > >=20 > > OK, some boilerplate. > > > -mkdir R > > > -cp -rvp ${abs_srcdir}/debuginfod-rpms R > > > -if [ "$zstd" =3D "false" ]; then # nuke the zstd fedora 31 ones > > > - rm -vrf R/debuginfod-rpms/fedora31 > > > -fi > >=20 > > This gets moved to below. >=20 > The two above changes should look better now >=20 > > > DB=3D${PWD}/.debuginfod_tmp.sqlite > > > tempfiles $DB > > > @@ -47,88 +47,68 @@ export > > > DEBUGINFOD_CACHE_PATH=3D${PWD}/.client_cache > > > ###### > > > rm -rf $DEBUGINFOD_CACHE_PATH > > > rm -rf $DB > > > + > > > # Set mb values to ensure the caches certainly have enough space > > > # To store the test files > > > env LD_LIBRARY_PATH=3D$ldpath DEBUGINFOD_URLS=3D > > > ${abs_builddir}/../debuginfod/debuginfod $VERBOSE -p $PORT1 -d > > > $DB \ > > > - --fdcache-fds=3D$FDCACHE_FDS --fdcache-prefetch- > > > fds=3D$PREFETCH_FDS --fdcache-mintmp 0 -vvvvv -g 0 -t 0 -R R \ > > > - --fdcache-mbs=3D50 --fdcache-prefetch-mbs=3D50 \ > > > - --fdcache-prefetch=3D$PREFETCH > vlog$PORT1 2>&1 & > > > + --fdcache-fds=3D$FDCACHE_FDS --fdcache-prefetch- > > > fds=3D$PREFETCH_FDS -vvvvv -g 0 -t 0 -R R \ > > > + -Z .tar.bz2=3Dbzcat Z --fdcache-mbs=3D50 --fdcache-prefetch- > > > mbs=3D100 \ > > > + --fdcache-prefetch=3D$PREFETCH -F > vlog$PORT1 2>&1 & > >=20 > > Now run with --fdcache-fds=3D1 --fdcache-mbs=3D50 and > > -fdcache-prefetch-fds=3D1 fdcache-prefetch=3D1 -fdcache-prefetch- > > mbs=3D100 > >=20 > > It is these last 3 we want to test. >=20 > This particular instance of debuginfod was intended to test > fdcache-fds and prefetch-fds. > The test is just testing prefetch-fds. fdcache-fds was tested in a > different diff, but because the > fdcache is just the basic lru the test was removed. Mb values are set > high enough so that no > files will be excluded for being too large. Maybe there should be a > test prior to this for just > fdcache-prefetch as it determines how much is prefetched. OK. I think no extra test is needed. > > > PID1=3D$! > > > tempfiles vlog$PORT1 > > > errfiles vlog$PORT1 > > > + > > > # Server must become ready > > > wait_ready $PORT1 'ready' 1 > > > ################################################################ > > > ######## > > > -cp -rvp ${abs_srcdir}/debuginfod-rpms R > > > -if [ "$zstd" =3D "false" ]; then # nuke the zstd fedora 31 ones > > > - rm -vrf R/debuginfod-rpms/fedora31 > > > -fi > > > kill -USR1 $PID1 > > > wait_ready $PORT1 'thread_work_total{role=3D"traverse"}' 1 > > > +wait_ready $PORT1 'thread_work_pending{role=3D"scan"}' 0 > > > +wait_ready $PORT1 'thread_busy{role=3D"scan"}' 0 > >=20 > > OK, boilerplate to wait for initial scan. >=20 > Yes, Frank told me that just checking traverse is insufficient to > conclude a > scan has completed. > > > -archive_hits=3D5 > > > -SHA=3Df4a1a8062be998ae93b8f1cd744a398c6de6dbb1 > > > -for i in $archive_hits > > > -do > > > - archive_test c36708a78618d597dee15d0dc989f093ca5f9120 > > > /usr/src/debug/hello2-1.0-2.x86_64/hello.c $SHA > > > -done > > > +export DEBUGINFOD_URLS=3Dhttp://127.0.0.1:$PORT1/ > > > +# load prefetch cache > > > +testrun ${abs_top_builddir}/debuginfod/debuginfod-find debuginfo > > > cee13b2ea505a7f37bd20d271c6bc7e5f8d2dfcb > >=20 > > OK, which particular file is this? >=20 > pk.pkg.tar/usr/src/debug/hello.c, now elaborated in comment Thanks. > > > metrics=3D$(curl http://127.0.0.1:$PORT1/metrics) > > > -regex=3D"fdcache_op_count\{op=3D\"enqueue\"\} ([0- > > > 9]+).*fdcache_op_count\{op=3D\"evict\"\} ([0-9]+).*" > > > -enqueue=3D0 > > > -if [[ $metrics =3D~ $regex ]] > > > -then > > > - enqueue=3D${BASH_REMATCH[1]} > > > - evict=3D${BASH_REMATCH[2]} > > > +regex=3D"fdcache_prefetch_count ([0-9])+" > > > +# Check to see if prefetch cache is loaded > > > +if [[ $metrics =3D~ $regex ]]; then > > > + if [[ ${BASH_REMATCH[1]} -ne 1 ]]; then > > > + err > > > + fi > >=20 > > Why the -ne 1 ? What number are we expecting and why? >=20 > Ensuring that prefetch is loaded, but not loaded more than it is > intended to hold. > debuginfod will try to load 2 files into prefetch, so check to ensure > that only 1 is loaded. > Explained in comment. Right. I misread the test condition. This makes sense now. > > > else > > > - err > > > -fi > > > -# This is an ad-hoc test than only works when FDCACHE_FDS < > > > "archive_test"s above (5) > > > -# otherwise there will be no eviction > > > -if [[ $(( $enqueue-$FDCACHE_FDS )) -ne $evict ]] > > > -then > > > - err > > > -fi > > > -# Test prefetch cache > > > -archive_test bc1febfd03ca05e030f0d205f7659db29f8a4b30 > > > /usr/src/debug/hello-1.0/hello.c $SHA > > > + err > > > +fi > > > +# Ensure that searching for the source prefetched finds it in > > > the prefetch cache > > > +testrun ${abs_top_builddir}/debuginfod/debuginfod-find source > > > cee13b2ea505a7f37bd20d271c6bc7e5f8d2dfcb /usr/src/debug/hello.c > > > metrics=3D$(curl http://127.0.0.1:$PORT1/metrics) > > > -regex=3D"fdcache_prefetch_count ([0-9])+" > > > -pf_count=3D0 > > > -for i in $FDCACHE_FDS > > > -do > > > - if [[ $metrics =3D~ $regex ]]; then > > > - > > > - if [[ $(( $pf_count+$PREFETCH )) -ne ${BASH_REMATCH[1]} ]]; > > > then > > > - err > > > - else > > > - pf_count=3D${BASH_REMATCH[1]} > > > - fi > > > - > > > - else > > > +regex=3D"fdcache_op_count\{op=3D\"prefetch_access\"\} ([0-9])+" > > > +if [[ $metrics =3D~ $regex ]]; then > > > + if [[ ${BASH_REMATCH[1]} -ne 1 ]]; then > > > err > > > fi > > > -done > > > +else > > > + err > > > +fi > >=20 > > Again -ne 1? Shouldn't it be different from the above now? >=20 > Now we're seeing if debuginfod is actually using the prefetch cache > by checking the access count. > We're searching for the 1 file we loaded before so we expect 1 > access. > Explained in comment Thanks. > > > metrics=3D$(curl http://127.0.0.1:$PORT1/metrics) > > > -regex=3D"fdcache_bytes ([0-9]+).*fdcache_prefetch_bytes ([0-9]+)" > > > +regex=3D"fdcache_bytes ([0-9]+)" > > > mb=3D$((1024*1024)) > > > if [[ $metrics =3D~ $regex ]]; then > > > fdbytes=3D${BASH_REMATCH[1]} > > > - pfbytes=3D${BASH_REMATCH[2]} > > > - if [ $fdbytes -gt $mb ] || [ $pfbytes -gt $mb ]; then > > > + if [ $fdbytes -gt $mb ] ; then > > > err > > > fi > >=20 > > OK, check that the prefetch cache isn't bigger than 10MB. Why that > > number? >=20 > mb should equal 1mb in bytes as this is the maximum space allocated > to both the prefetch and fd cache. and curling metrics returns memory > used in bytes Thanks. I was off by a factor 10. Sorry. > Please find the new patch attached. Thanks for adding all the comments. Looks good. Cheers, Mark --=-mITBmRTQfMfpRUPwP+3/ Content-Disposition: inline; filename*0=0001-PR28577-Make-run-debuginfod-fd-prefetch-caches.sh-te.pat; filename*1=ch Content-Type: text/x-patch; name="0001-PR28577-Make-run-debuginfod-fd-prefetch-caches.sh-te.patch"; charset="UTF-8" Content-Transfer-Encoding: base64 RnJvbSAwN2NmYTBmNjY5ODEwMmU4YmUxY2ZkNzEwMzQ0ZWY2MWM5YzE2MWYzIE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBOb2FoIFNhbmNpIDxuc2FuY2lAcmVkaGF0LmNvbT4KRGF0ZTog RnJpLCAxMyBNYXkgMjAyMiAxNDozMjoyNyAtMDQwMApTdWJqZWN0OiBbUEFUQ0hdIFBSMjg1Nzc6 IE1ha2UgcnVuLWRlYnVnaW5mb2QtZmQtcHJlZmV0Y2gtY2FjaGVzLnNoIHRlc3QgCiAgc29tZXRo aW5nCgpVcGRhdGUgdG8gdGhlIHJ1bi1kZWJ1Z2luZm9kLWZkLXByZWZldGNoIHRvIG1ha2UgdGhl IHRlc3QgbW9yZQpzb3VuZC4KLS0tCiB0ZXN0cy9ydW4tZGVidWdpbmZvZC1mZC1wcmVmZXRjaC1j YWNoZXMuc2ggfCAxMjEgKysrKysrKysrLS0tLS0tLS0tLS0tCiAxIGZpbGUgY2hhbmdlZCwgNDkg aW5zZXJ0aW9ucygrKSwgNzIgZGVsZXRpb25zKC0pCgpkaWZmIC0tZ2l0IGEvdGVzdHMvcnVuLWRl YnVnaW5mb2QtZmQtcHJlZmV0Y2gtY2FjaGVzLnNoIGIvdGVzdHMvcnVuLWRlYnVnaW5mb2QtZmQt cHJlZmV0Y2gtY2FjaGVzLnNoCmluZGV4IDY2YTdhMDgzLi44Nzc4ZDNhMiAxMDA3NTUKLS0tIGEv dGVzdHMvcnVuLWRlYnVnaW5mb2QtZmQtcHJlZmV0Y2gtY2FjaGVzLnNoCisrKyBiL3Rlc3RzL3J1 bi1kZWJ1Z2luZm9kLWZkLXByZWZldGNoLWNhY2hlcy5zaApAQCAtMjIsMTcgKzIyLDE5IEBACiBz ZXQgLXgKIHVuc2V0IFZBTEdSSU5EX0NNRAogCi1ta2RpciBSCisjIFNldCB1cCBhcmNoaXZlIGZv bGRlcnMgZm9yIHNjYW5uaW5nCitta2RpciBSIFoKK2NwIC1ydnAgJHthYnNfc3JjZGlyfS9kZWJ1 Z2luZm9kLXRhcnMgWgogY3AgLXJ2cCAke2Fic19zcmNkaXJ9L2RlYnVnaW5mb2QtcnBtcyBSCiBp ZiBbICIkenN0ZCIgPSAiZmFsc2UiIF07IHRoZW4gICMgbnVrZSB0aGUgenN0ZCBmZWRvcmEgMzEg b25lcwogICAgIHJtIC12cmYgUi9kZWJ1Z2luZm9kLXJwbXMvZmVkb3JhMzEKIGZpCiAKLUZEQ0FD SEVfTUJTPSQoKDEvMTAyNC8xMDI0KSkKKyMgU2V0IHVwIG1heGltdW1zLiAxIGlzIGp1c3QgYSBj b252ZW5pZW50IGNhcCB0byB0ZXN0IG9uLgorRkRDQUNIRV9NQlM9MQogRkRDQUNIRV9GRFM9MQot UFJFRkVUQ0hfTUJTPSQoKDEvMTAyNC8xMDI0KSkKLVBSRUZFVENIX0ZEUz0yCi1QUkVGRVRDSD0x CitQUkVGRVRDSF9GRFM9MQorUFJFRkVUQ0g9MgogIyBUaGlzIHZhcmlhYmxlIGlzIGVzc2VudGlh bCBhbmQgZW5zdXJlcyBubyB0aW1lLXJhY2UgZm9yIGNsYWltaW5nIHBvcnRzIG9jY3VycwogIyBz ZXQgYmFzZSB0byBhIHVuaXF1ZSBtdWx0aXBsZSBvZiAxMDAgbm90IHVzZWQgaW4gYW55IG90aGVy ICdydW4tZGVidWdpbmZvZC0qJyB0ZXN0CiBiYXNlPTg4MDAKQEAgLTQ3LDg4ICs0OSw2MSBAQCBl eHBvcnQgREVCVUdJTkZPRF9DQUNIRV9QQVRIPSR7UFdEfS8uY2xpZW50X2NhY2hlCiAjIyMjIyMK IHJtIC1yZiAkREVCVUdJTkZPRF9DQUNIRV9QQVRICiBybSAtcmYgJERCCi0jIFNldCBtYiB2YWx1 ZXMgdG8gZW5zdXJlIHRoZSBjYWNoZXMgY2VydGFpbmx5IGhhdmUgZW5vdWdoIHNwYWNlCi0jIFRv IHN0b3JlIHRoZSB0ZXN0IGZpbGVzCisjIFRlc3RpbmcgcHJlZmV0Y2ggZmQgbWF4aW11bSAoU2V0 IG1iIG1heGltdW1zIHRvIGJlIGJleW9uZCBjb25zaWRlcmF0aW9uKQogZW52IExEX0xJQlJBUllf UEFUSD0kbGRwYXRoIERFQlVHSU5GT0RfVVJMUz0gJHthYnNfYnVpbGRkaXJ9Ly4uL2RlYnVnaW5m b2QvZGVidWdpbmZvZCAkVkVSQk9TRSAtcCAkUE9SVDEgLWQgJERCIFwKLSAgICAtLWZkY2FjaGUt ZmRzPSRGRENBQ0hFX0ZEUyAtLWZkY2FjaGUtcHJlZmV0Y2gtZmRzPSRQUkVGRVRDSF9GRFMgLS1m ZGNhY2hlLW1pbnRtcCAwIC12dnZ2diAtZyAwIC10IDAgLVIgUiBcCi0gICAgLS1mZGNhY2hlLW1i cz01MCAgLS1mZGNhY2hlLXByZWZldGNoLW1icz01MCBcCi0gICAgLS1mZGNhY2hlLXByZWZldGNo PSRQUkVGRVRDSCAgPiB2bG9nJFBPUlQxIDI+JjEgJgorICAgIC0tZmRjYWNoZS1mZHM9JEZEQ0FD SEVfRkRTIC0tZmRjYWNoZS1wcmVmZXRjaC1mZHM9JFBSRUZFVENIX0ZEUyAgLXZ2dnZ2IC1nIDAg LXQgMCBcCisgICAgLVogLnRhci5iejI9YnpjYXQgWiAgLS1mZGNhY2hlLW1icz0xMDAgIC0tZmRj YWNoZS1wcmVmZXRjaC1tYnM9MTAwIFwKKyAgICAtLWZkY2FjaGUtcHJlZmV0Y2g9JFBSRUZFVENI ID4gdmxvZyRQT1JUMSAyPiYxICYKIFBJRDE9JCEKIHRlbXBmaWxlcyB2bG9nJFBPUlQxCiBlcnJm aWxlcyB2bG9nJFBPUlQxCiAjIFNlcnZlciBtdXN0IGJlY29tZSByZWFkeQogd2FpdF9yZWFkeSAk UE9SVDEgJ3JlYWR5JyAxCiAjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMKLWNwIC1ydnAgJHthYnNfc3JjZGlyfS9k ZWJ1Z2luZm9kLXJwbXMgUgotaWYgWyAiJHpzdGQiID0gImZhbHNlIiBdOyB0aGVuICAjIG51a2Ug dGhlIHpzdGQgZmVkb3JhIDMxIG9uZXMKLSAgICBybSAtdnJmIFIvZGVidWdpbmZvZC1ycG1zL2Zl ZG9yYTMxCi1maQoga2lsbCAtVVNSMSAkUElEMQogd2FpdF9yZWFkeSAkUE9SVDEgJ3RocmVhZF93 b3JrX3RvdGFse3JvbGU9InRyYXZlcnNlIn0nIDEKLSMgQWxsIHJwbXMgbmVlZCB0byBiZSBpbiB0 aGUgaW5kZXgsIGV4Y2VwdCB0aGUgZHVtbXkgcGVybWlzc2lvbi0wMDAgb25lCi1ycG1zPSQoZmlu ZCBSIC1uYW1lIFwqcnBtIHwgZ3JlcCAtdiBub3RoaW5nIHwgd2MgLWwpCi13YWl0X3JlYWR5ICRQ T1JUMSAnc2Nhbm5lZF9maWxlc190b3RhbHtzb3VyY2U9Ii5ycG0gYXJjaGl2ZSJ9JyAkcnBtcwot a2lsbCAtVVNSMSAkUElEMSAgIyB0d28gaGl0cyBvZiBTSUdVU1IxIG1heSBiZSBuZWVkZWQgdG8g cmVzb2x2ZSAuZGVidWctPmR3ei0+c3JlZnMKLSMgV2FpdCB0aWxsIGJvdGggZmlsZXMgYXJlIGlu IHRoZSBpbmRleCBhbmQgc2Nhbi9pbmRleCBmdWxseSBmaW5pc2hlZAotd2FpdF9yZWFkeSAkUE9S VDEgJ3RocmVhZF93b3JrX3RvdGFse3JvbGU9InRyYXZlcnNlIn0nIDIKLWV4cG9ydCBERUJVR0lO Rk9EX1VSTFM9Imh0dHA6Ly8xMjcuMC4wLjE6IiRQT1JUMQotCi1hcmNoaXZlX2hpdHM9NQotU0hB PWY0YTFhODA2MmJlOTk4YWU5M2I4ZjFjZDc0NGEzOThjNmRlNmRiYjEKLWZvciBpIGluICRhcmNo aXZlX2hpdHMKLWRvCi0gIGFyY2hpdmVfdGVzdCBjMzY3MDhhNzg2MThkNTk3ZGVlMTVkMGRjOTg5 ZjA5M2NhNWY5MTIwIC91c3Ivc3JjL2RlYnVnL2hlbGxvMi0xLjAtMi54ODZfNjQvaGVsbG8uYyAk U0hBCi1kb25lCi1tZXRyaWNzPSQoY3VybCBodHRwOi8vMTI3LjAuMC4xOiRQT1JUMS9tZXRyaWNz KQotcmVnZXg9ImZkY2FjaGVfb3BfY291bnRce29wPVwiZW5xdWV1ZVwiXH0gKFswLTldKykuKmZk Y2FjaGVfb3BfY291bnRce29wPVwiZXZpY3RcIlx9IChbMC05XSspLioiCi1lbnF1ZXVlPTAKLWlm IFtbICRtZXRyaWNzID1+ICRyZWdleCBdXQotdGhlbgotICAgZW5xdWV1ZT0ke0JBU0hfUkVNQVRD SFsxXX0KLSAgIGV2aWN0PSR7QkFTSF9SRU1BVENIWzJdfQotZWxzZQotICAgZXJyCi1maQotIyBU aGlzIGlzIGFuIGFkLWhvYyB0ZXN0IHRoYW4gb25seSB3b3JrcyB3aGVuIEZEQ0FDSEVfRkRTIDwg ImFyY2hpdmVfdGVzdCJzIGFib3ZlICg1KQotIyBvdGhlcndpc2UgdGhlcmUgd2lsbCBiZSBubyBl dmljdGlvbgotaWYgW1sgJCgoICRlbnF1ZXVlLSRGRENBQ0hFX0ZEUyApKSAtbmUgJGV2aWN0IF1d Ci10aGVuCi0gICBlcnIKLWZpCi0jIFRlc3QgcHJlZmV0Y2ggY2FjaGUKLWFyY2hpdmVfdGVzdCBi YzFmZWJmZDAzY2EwNWUwMzBmMGQyMDVmNzY1OWRiMjlmOGE0YjMwIC91c3Ivc3JjL2RlYnVnL2hl bGxvLTEuMC9oZWxsby5jICRTSEEKK3dhaXRfcmVhZHkgJFBPUlQxICd0aHJlYWRfd29ya19wZW5k aW5ne3JvbGU9InNjYW4ifScgMAord2FpdF9yZWFkeSAkUE9SVDEgJ3RocmVhZF9idXN5e3JvbGU9 InNjYW4ifScgMAorZXhwb3J0IERFQlVHSU5GT0RfVVJMUz1odHRwOi8vMTI3LjAuMC4xOiRQT1JU MS8KKyMgbG9hZCBwcmVmZXRjaCBjYWNoZSB3aXRoIGRlYnVnaW5mb2QtdGFycy91c3Ivc3JjL2Rl YnVnL2hlbGxvLmMKK3Rlc3RydW4gJHthYnNfdG9wX2J1aWxkZGlyfS9kZWJ1Z2luZm9kL2RlYnVn aW5mb2QtZmluZCBkZWJ1Z2luZm8gY2VlMTNiMmVhNTA1YTdmMzdiZDIwZDI3MWM2YmM3ZTVmOGQy ZGZjYgogbWV0cmljcz0kKGN1cmwgaHR0cDovLzEyNy4wLjAuMTokUE9SVDEvbWV0cmljcykKIHJl Z2V4PSJmZGNhY2hlX3ByZWZldGNoX2NvdW50IChbMC05XSkrIgotcGZfY291bnQ9MAotZm9yIGkg aW4gJEZEQ0FDSEVfRkRTCi1kbwotICBpZiBbWyAkbWV0cmljcyA9fiAkcmVnZXggXV07IHRoZW4K LQotICAgIGlmIFtbICQoKCAkcGZfY291bnQrJFBSRUZFVENIICkpIC1uZSAke0JBU0hfUkVNQVRD SFsxXX0gXV07IHRoZW4KLSAgICAgIGVycgotICAgIGVsc2UKLSAgICAgIHBmX2NvdW50PSR7QkFT SF9SRU1BVENIWzFdfQotICAgIGZpCisjIENoZWNrIHRvIHNlZSBpZiBwcmVmZXRjaCBjYWNoZSBp cyBtYXhpbWFsbHkgbG9hZGVkLiBOb3RlIGZkY2FjaGUtcHJlZmV0Y2ggKDIpID4gcHJlZmV0Y2gt ZmRzICgxKSwKKyMgc28gdGhlIGRlYnVnaW5mb2Qgd2lsbCB0cnkgdG8gbG9hZCB0aGUgcHJlZmV0 Y2ggY2FjaGUgd2l0aCAyIGZpbGVzLiBXZSB3YW50IHByZWZldGNoLWZkcyB0byBjYXAgdGhhdAor IyBvZmYKK2lmIFtbICRtZXRyaWNzID1+ICRyZWdleCBdXTsgdGhlbgorICBpZiBbWyAke0JBU0hf UkVNQVRDSFsxXX0gLW5lICRQUkVGRVRDSF9GRFMgXV07IHRoZW4KKyAgICBlcnIKKyAgZmkKK2Vs c2UKKyAgZXJyCitmaSAKIAotICBlbHNlCit0ZXN0cnVuICR7YWJzX3RvcF9idWlsZGRpcn0vZGVi dWdpbmZvZC9kZWJ1Z2luZm9kLWZpbmQgc291cmNlIGNlZTEzYjJlYTUwNWE3ZjM3YmQyMGQyNzFj NmJjN2U1ZjhkMmRmY2IgL3Vzci9zcmMvZGVidWcvaGVsbG8uYworbWV0cmljcz0kKGN1cmwgaHR0 cDovLzEyNy4wLjAuMTokUE9SVDEvbWV0cmljcykKK3JlZ2V4PSJmZGNhY2hlX29wX2NvdW50XHtv cD1cInByZWZldGNoX2FjY2Vzc1wiXH0gKFswLTldKSsiCitpZiBbWyAkbWV0cmljcyA9fiAkcmVn ZXggXV07IHRoZW4KKyAgIyBJbiB0aGUgdGVzdCBhYm92ZSBoZWxsby5jIHNob3VsZCd2ZSBiZWVu IGxvYWRlZCBpbnRvIHRoZSBwcmVmZXRjaCBjYWNoZS4KKyAgIyBFbnN1cmUgdGhhdCBzZWFyY2hp bmcgZm9yIGhlbGxvLmMgYSBzZWNvbmQgdGltZSBhY2Nlc3NlcyB0aGUgcHJlZmV0Y2ggY2FjaGUg b25jZQorICBpZiBbWyAgJHtCQVNIX1JFTUFUQ0hbMV19IC1uZSAxIF1dOyB0aGVuCiAgICAgZXJy CiAgIGZpCi1kb25lCitlbHNlCisgIGVycgorZmkKIAoga2lsbCAkUElEMQogd2FpdCAkUElEMQog UElEMT0wCiAKLQogIyMjIyMjIyMjCi0jIFRlc3QgbWIgbGltaXQgb24gZmQgYW5kIHByZWZldGNo IGNhY2hlCisjIFRlc3QgbWIgbGltaXQgb24gZmQgY2FjaGUKICMjIyMjIyMjIwotCiBybSAtcmYg JERFQlVHSU5GT0RfQ0FDSEVfUEFUSAogcm0gLXJmICREQgogZW52IExEX0xJQlJBUllfUEFUSD0k bGRwYXRoIERFQlVHSU5GT0RfVVJMUz0gJHthYnNfYnVpbGRkaXJ9Ly4uL2RlYnVnaW5mb2QvZGVi dWdpbmZvZCAkVkVSQk9TRSAtcCAkUE9SVDEgLWQgJERCIFwKLSAgICAtLWZkY2FjaGUtbWJzPTEg LS1mZGNhY2hlLXByZWZldGNoLW1icz0xIC0tZmRjYWNoZS1taW50bXAgMCAtdnZ2dnYgLWcgMCAt dCAwIC1SIFIgXAotICAgID4gdmxvZzIkUE9SVDEgMj4mMSAmCisgICAgLS1mZGNhY2hlLW1icz0k RkRDQUNIRV9NQlMgLXZ2dnZ2IC1nIDAgLXQgMCAtUiBSID4gdmxvZzIkUE9SVDEgMj4mMSAmCiBQ SUQxPSQhCiB0ZW1wZmlsZXMgdmxvZzIkUE9SVDEKIGVycmZpbGVzIHZsb2cyJFBPUlQxCkBAIC0x MzksMjYgKzExNCwyOCBAQCB3YWl0X3JlYWR5ICRQT1JUMSAndGhyZWFkX3dvcmtfdG90YWx7cm9s ZT0idHJhdmVyc2UifScgMQogIyBBbGwgcnBtcyBuZWVkIHRvIGJlIGluIHRoZSBpbmRleCwgZXhj ZXB0IHRoZSBkdW1teSBwZXJtaXNzaW9uLTAwMCBvbmUKIHJwbXM9JChmaW5kIFIgLW5hbWUgXCpy cG0gfCBncmVwIC12IG5vdGhpbmcgfCB3YyAtbCkKIHdhaXRfcmVhZHkgJFBPUlQxICdzY2FubmVk X2ZpbGVzX3RvdGFse3NvdXJjZT0iLnJwbSBhcmNoaXZlIn0nICRycG1zCi1raWxsIC1VU1IxICRQ SUQxICAjIHR3byBoaXRzIG9mIFNJR1VTUjEgbWF5IGJlIG5lZWRlZCB0byByZXNvbHZlIC5kZWJ1 Zy0+ZHd6LT5zcmVmcwotIyBXYWl0IHRpbGwgYm90aCBmaWxlcyBhcmUgaW4gdGhlIGluZGV4IGFu ZCBzY2FuL2luZGV4IGZ1bGx5IGZpbmlzaGVkCitraWxsIC1VU1IxICRQSUQxCiB3YWl0X3JlYWR5 ICRQT1JUMSAndGhyZWFkX3dvcmtfdG90YWx7cm9sZT0idHJhdmVyc2UifScgMgord2FpdF9yZWFk eSAkUE9SVDEgJ3RocmVhZF93b3JrX3BlbmRpbmd7cm9sZT0ic2NhbiJ9JyAwCit3YWl0X3JlYWR5 ICRQT1JUMSAndGhyZWFkX2J1c3l7cm9sZT0ic2NhbiJ9JyAwCiAKLWFyY2hpdmVfdGVzdCBjMzY3 MDhhNzg2MThkNTk3ZGVlMTVkMGRjOTg5ZjA5M2NhNWY5MTIwIC91c3Ivc3JjL2RlYnVnL2hlbGxv Mi0xLjAtMi54ODZfNjQvaGVsbG8uYyAkU0hBCisjIFRoaXMgbWFueSBhcmNoaXZlcyBjYXVzZSB0 aGUgZmQgY2FjaGUgdG8gYmUgbG9hZGVkIHdpdGgganVzdCBvdmVyIDFtYiBvZgorIyBmaWxlcy4g U28gd2UgZXhwZWN0IHRoZSAxbWIgY2FwIG9mZgorU0hBPWY0YTFhODA2MmJlOTk4YWU5M2I4ZjFj ZDc0NGEzOThjNmRlNmRiYjEKIGFyY2hpdmVfdGVzdCBiYzFmZWJmZDAzY2EwNWUwMzBmMGQyMDVm NzY1OWRiMjlmOGE0YjMwIC91c3Ivc3JjL2RlYnVnL2hlbGxvLTEuMC9oZWxsby5jICRTSEEKIGFy Y2hpdmVfdGVzdCBjMzY3MDhhNzg2MThkNTk3ZGVlMTVkMGRjOTg5ZjA5M2NhNWY5MTIwIC91c3Iv c3JjL2RlYnVnL2hlbGxvMi0xLjAtMi54ODZfNjQvaGVsbG8uYyAkU0hBCiBhcmNoaXZlX3Rlc3Qg NDFhMjM2ZWI2NjdjMzYyYTFjNDE5NjAxOGNjNDU4MWUwOTcyMmIxYiAvdXNyL3NyYy9kZWJ1Zy9o ZWxsbzItMS4wLTIueDg2XzY0L2hlbGxvLmMgJFNIQQogYXJjaGl2ZV90ZXN0IGJjMWZlYmZkMDNj YTA1ZTAzMGYwZDIwNWY3NjU5ZGIyOWY4YTRiMzAgL3Vzci9zcmMvZGVidWcvaGVsbG8tMS4wL2hl bGxvLmMgJFNIQQogYXJjaGl2ZV90ZXN0IGYwYWExNWI4YWJhNGYzYzI4Y2FjM2MyYTczODAxZmVm YTY0NGE5ZjIgL3Vzci9zcmMvZGVidWcvaGVsbG8tMS4wL2hlbGxvLmMgJFNIQQogYXJjaGl2ZV90 ZXN0IGJiYmY5MmViZWU1MjI4MzEwZTM5ODYwOWMyM2MyZDdkNTNmNmUyZjkgL3Vzci9zcmMvZGVi dWcvaGVsbG8tMS4wL2hlbGxvLmMgJFNIQQotYXJjaGl2ZV90ZXN0IGQ0NGQ0MmNiZDdkOTE1YmM5 MzhjODEzMzNhMjFlMzU1YTYwMjJmYjcgL3Vzci9zcmMvZGVidWcvaGVsbG8tMS4wL2hlbGxvLmMg JFNIQQotCiBtZXRyaWNzPSQoY3VybCBodHRwOi8vMTI3LjAuMC4xOiRQT1JUMS9tZXRyaWNzKQot cmVnZXg9ImZkY2FjaGVfYnl0ZXMgKFswLTldKykuKmZkY2FjaGVfcHJlZmV0Y2hfYnl0ZXMgKFsw LTldKykiCi1tYj0kKCgxMDI0KjEwMjQpKQorcmVnZXg9ImZkY2FjaGVfYnl0ZXMgKFswLTldKyki CisjIFNpbmNlIHRoZSBzZXJ2ZXIgbWV0cmljcyByZXBvcnQgaW4gYnl0ZXMsICRtYiBpcyBqdXN0 IHRoZSB0b3RhbCBudW1iZXIgb2YgYnl0ZXMgYWxsb2NhdGVkCisjIHRvIHRoZSBmZCBjYWNoZS4g RW5zdXJlIHRoYXQgdGhpcyBjYXAgaXNuJ3QgY3Jvc3NlZAorbWI9JCgoJEZEQ0FDSEVfTUJTKjEw MjQqMTAyNCkpCiBpZiBbWyAkbWV0cmljcyA9fiAkcmVnZXggXV07IHRoZW4KICAgZmRieXRlcz0k e0JBU0hfUkVNQVRDSFsxXX0KLSAgcGZieXRlcz0ke0JBU0hfUkVNQVRDSFsyXX0KLSAgaWYgWyAk ZmRieXRlcyAtZ3QgJG1iIF0gfHwgWyAkcGZieXRlcyAtZ3QgJG1iIF07IHRoZW4KKyAgaWYgWyAk ZmRieXRlcyAtZ3QgJG1iIF0gOyB0aGVuCiAgICAgZXJyCiAgIGZpCiBlbHNlCi0tIAoyLjM2LjEK Cg== --=-mITBmRTQfMfpRUPwP+3/--