From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gnu.wildebeest.org (gnu.wildebeest.org [45.83.234.184]) by sourceware.org (Postfix) with ESMTPS id 4A0953858D28 for ; Wed, 21 Jun 2023 16:24:53 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 4A0953858D28 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=klomp.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=klomp.org Received: from r6.localdomain (82-217-174-174.cable.dynamic.v4.ziggo.nl [82.217.174.174]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by gnu.wildebeest.org (Postfix) with ESMTPSA id E03773023089; Wed, 21 Jun 2023 18:24:49 +0200 (CEST) Received: by r6.localdomain (Postfix, from userid 1000) id 41A7034012D; Wed, 21 Jun 2023 18:24:49 +0200 (CEST) Message-ID: <3465f95ee22b1ac433f1268f113e3813430be70a.camel@klomp.org> Subject: Re: Performance issue with systemd-coredump and container process linking 2000 shared libraries. From: Mark Wielaard To: Romain GEISSLER Cc: "elfutils-devel@sourceware.org" , Francois RIGAULT Date: Wed, 21 Jun 2023 18:24:49 +0200 In-Reply-To: <754003F4-3708-4C9C-AA30-EB76DAECF059@amadeus.com> References: <20230614133920.10-1-romain.geissler@amadeus.com> <20230620213701.GN24233@gnu.wildebeest.org> <754003F4-3708-4C9C-AA30-EB76DAECF059@amadeus.com> Content-Type: multipart/mixed; boundary="=-S1YUfvqucRW43GwWqJ0v" User-Agent: Evolution 3.48.3 (3.48.3-1.fc38) MIME-Version: 1.0 X-Spam-Status: No, score=-3034.6 required=5.0 tests=BAYES_00,GIT_PATCH_0,JMQ_SPF_NEUTRAL,KAM_DMARC_STATUS,RCVD_IN_BARRACUDACENTRAL,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: --=-S1YUfvqucRW43GwWqJ0v Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Romain, On Tue, 2023-06-20 at 22:05 +0000, Romain GEISSLER wrote: >=20 > Our real use case happens on a Openshift 4.13 node, so the OS is Red Hat = Core OS 9 (which I assume shares a lot of foundations with RHEL 9). >=20 > On our side Francois also told me this afternoon that he didn=E2=80=99t r= eally reproduce the same thing with my reproducer posted here and the real = systemd-coredump issue he witnessed live, and also noticed that with DEBUGI= NFOD_URLS unset/set to the empty string my reproducer has no problem anymor= e. What he witnessed on the real case (using perf/gdb) was that apparently = lots of time was spend in elf_getdata_rawchunk and often in this kind of st= ack: >=20 > Samples: 65K of event 'cpu-clock:pppH', Event count (approx.): 1646850000= 0 = =20 > Overhead Command Shared Object Symbol = = =20 > =C2=A0=C2=A098.24% (sd-parse-elf) libelf-0.188.so [.] elf_get= data_rawchunk > =C2=A0=C2=A0=C2=A00.48% (sd-parse-elf) libelf-0.188.so [.] 0x= 00000000000048a3 > =C2=A0=C2=A0=C2=A00.27% (sd-parse-elf) libelf-0.188.so [.] ge= lf_getphdr > =C2=A0=C2=A0=C2=A00.11% (sd-parse-elf) libc.so.6 [.] _i= nt_malloc > =C2=A0=C2=A0=C2=A00.10% (sd-parse-elf) libelf-0.188.so [.] ge= lf_getnote > =C2=A0=C2=A0=C2=A00.06% (sd-parse-elf) libc.so.6 [.] __= libc_calloc > =C2=A0=C2=A0=C2=A00.05% (sd-parse-elf) [kernel.kallsyms] [k] __= softirqentry_text_start > =C2=A0=C2=A0=C2=A00.05% (sd-parse-elf) libc.so.6 [.] _i= nt_free >=20 >=20 > (gdb) bt > #0 0x00007f0ba8a88194 in elf_getdata_rawchunk () from target:/lib64/libe= lf.so.1 > #1 0x00007f0ba98e5013 in module_callback.lto_priv () from target:/usr/li= b64/systemd/libsystemd-shared-252.so > #2 0x00007f0ba8ae7291 in dwfl_getmodules () from target:/lib64/libdw.so.= 1 > #3 0x00007f0ba98e6dc0 in parse_elf_object () from target:/usr/lib64/syst= emd/libsystemd-shared-252.so > #4 0x0000562c474f2d5e in submit_coredump () > #5 0x0000562c474f57d1 in process_socket.constprop () > #6 0x0000562c474efbf8 in main () >=20 > My reproducer actually doesn=E2=80=99t fully re-implement what systemd im= plements (the parsing of the package metadata is clearly omitted), so I tho= ught I had reproduced the same problem while apparently I didn=E2=80=99t, s= orry for that. We will also have to double check if really just using 2000 = dummy libraries is enough or if this also needs to have a more complex bina= ry like we have in our real case. >=20 > Tomorrow on our side we will have to play a bit with a local build of sys= temd-coredump and try to run it manually to better understand what=E2=80=99= s going wrong. >=20 Seeing those performance results I understand why you were suspecting the linked list data structure used in elf_getdata_rawchunk. Would you be able to test a rebuild libelf with the attached patch, which replaces that datastructure with a binary search tree? It didn't really show much speedup locally (in the noise, maybe 0.01 sec faster on ~0.25 sec run). But if there are more than 2000 calls to elf_getdata_rawchunk it should make things faster. Thanks, Mark --=-S1YUfvqucRW43GwWqJ0v Content-Description: Content-Disposition: inline; filename*0=0001-libelf-Replace-list-of-elf_getdata_rawchunk-results-.pat; filename*1=ch Content-Transfer-Encoding: base64 Content-Type: text/x-patch; name="0001-libelf-Replace-list-of-elf_getdata_rawchunk-results-.patch"; charset="UTF-8" RnJvbSAzYWNhNWI1ZjFmMTYxN2RiMjIyMDAyMmQ5MDYxZGNhZjEyOWU1NGM0IE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBNYXJrIFdpZWxhYXJkIDxtYXJrQGtsb21wLm9yZz4KRGF0ZTog V2VkLCAyMSBKdW4gMjAyMyAxODowNToxMiArMDIwMApTdWJqZWN0OiBbUEFUQ0hdIGxpYmVsZjog UmVwbGFjZSBsaXN0IG9mIGVsZl9nZXRkYXRhX3Jhd2NodW5rIHJlc3VsdHMgd2l0aCBhCiB0cmVl CgplbGZfZ2V0ZGF0YV9yYXdjaHVua3MgZGlkIGEgbGluZWFyIHNlYXJjaCB0byBzZWUgaWYgYSBj aHVuayB3YXMKYWxyZWFkeSBmZXRjaGVkLiBSZXBsYWNlIHRoaXMgbGlzdCB3aXRoIGEgYmluYXJ5 IHNlYXJjaCB0cmVlIHRvIG1ha2UKbG9va3VwIGZhc3RlciB3aGVuIGEgbG90IG9mIEVsZl9EYXRh X0NodW5rIHdlcmUgY3JlYXRlZC4KCiAgICAgICAqIGxpYmVsZi9saWJlbGZQLmggKEVsZl9EYXRh X0NodW5rKTogUmVtb3ZlIG5leHQgZmllbGQuCiAgICAgICAoc3RydWN0IEVsZik6IENoYW5nZSB0 aGUgcmF3Y2h1bmtzIHR5cGUgZnJvbSBFbGZfRGF0YV9DaHVuayAqCiAgICAgICB0byB2b2lkICou CiAgICAgICAqIGVsZl9nZXRkYXRhX3Jhd2NodW5rLmMgKGNodW5rX2NvbXBhcmUpOiBOZXcgc3Rh dGljIGZ1bmN0aW9uLgogICAgICAgKGVsZl9nZXRkYXRhX3Jhd2NodW5rKTogVXNlIHRzZWFyY2gg aW5zdGVhZCBvZiBhIG1hbnVhbCBsaW5rZWQKICAgICAgIGxpc3QuCiAgICAgICAqIGVsZl9lbmQu YyAoZnJlZV9jaHVuayk6IE5ldyBzdGF0aWMgZnVuY3Rpb24uCiAgICAgICAoZWxmX2VuZCk6IENh bGwgdGRlc3Ryb3kgaW5zdGVhZCBvZiB3YWxraW5nIGxpbmtlZCBsaXN0LgoKU2lnbmVkLW9mZi1i eTogTWFyayBXaWVsYWFyZCA8bWFya0BrbG9tcC5vcmc+Ci0tLQogbGliZWxmL2VsZl9lbmQuYyAg ICAgICAgICAgICAgfCAyMiArKysrKysrKystLS0tLS0tCiBsaWJlbGYvZWxmX2dldGRhdGFfcmF3 Y2h1bmsuYyB8IDQ3ICsrKysrKysrKysrKysrKysrKysrKysrKystLS0tLS0tLS0tCiBsaWJlbGYv bGliZWxmUC5oICAgICAgICAgICAgICB8IDEzICsrKystLS0tLS0KIDMgZmlsZXMgY2hhbmdlZCwg NTIgaW5zZXJ0aW9ucygrKSwgMzAgZGVsZXRpb25zKC0pCgpkaWZmIC0tZ2l0IGEvbGliZWxmL2Vs Zl9lbmQuYyBiL2xpYmVsZi9lbGZfZW5kLmMKaW5kZXggNWM0NTFmMzYuLjNlNWQ0Yzg2IDEwMDY0 NAotLS0gYS9saWJlbGYvZWxmX2VuZC5jCisrKyBiL2xpYmVsZi9lbGZfZW5kLmMKQEAgLTEsNSAr MSw2IEBACiAvKiBGcmVlIHJlc291cmNlcyBhc3NvY2lhdGVkIHdpdGggRWxmIGRlc2NyaXB0b3Iu CiAgICBDb3B5cmlnaHQgKEMpIDE5OTgsMTk5OSwyMDAwLDIwMDEsMjAwMiwyMDA0LDIwMDUsMjAw NywyMDE1LDIwMTYgUmVkIEhhdCwgSW5jLgorICAgQ29weXJpZ2h0IChDKSAyMDIzIE1hcmsgSi4g V2llbGFhcmQgPG1hcmtAa2xvbXAub3JnPgogICAgVGhpcyBmaWxlIGlzIHBhcnQgb2YgZWxmdXRp bHMuCiAgICBXcml0dGVuIGJ5IFVscmljaCBEcmVwcGVyIDxkcmVwcGVyQHJlZGhhdC5jb20+LCAx OTk4LgogCkBAIC0zMiwxMiArMzMsMjIgQEAKICNlbmRpZgogCiAjaW5jbHVkZSA8YXNzZXJ0Lmg+ CisjaW5jbHVkZSA8c2VhcmNoLmg+CiAjaW5jbHVkZSA8c3RkZGVmLmg+CiAjaW5jbHVkZSA8c3Rk bGliLmg+CiAKICNpbmNsdWRlICJsaWJlbGZQLmgiCiAKIAorc3RhdGljIHZvaWQKK2ZyZWVfY2h1 bmsgKHZvaWQgKm4pCit7CisgIEVsZl9EYXRhX0NodW5rICpyYXdjaHVuayA9IChFbGZfRGF0YV9D aHVuayAqKW47CisgIGlmIChyYXdjaHVuay0+ZHVtbXlfc2NuLmZsYWdzICYgRUxGX0ZfTUFMTE9D RUQpCisgICAgZnJlZSAocmF3Y2h1bmstPmRhdGEuZC5kX2J1Zik7CisgIGZyZWUgKHJhd2NodW5r KTsKK30KKwogaW50CiBlbGZfZW5kIChFbGYgKmVsZikKIHsKQEAgLTExMiwyMCArMTIzLDEzIEBA IGVsZl9lbmQgKEVsZiAqZWxmKQogCiAgICAgY2FzZSBFTEZfS19FTEY6CiAgICAgICB7Ci0JRWxm X0RhdGFfQ2h1bmsgKnJhd2NodW5rcworCXZvaWQgKnJhd2NodW5rcwogCSAgPSAoZWxmLT5jbGFz cyA9PSBFTEZDTEFTUzMyCiAJICAgICB8fCAob2Zmc2V0b2YgKHN0cnVjdCBFbGYsIHN0YXRlLmVs ZjMyLnJhd2NodW5rcykKIAkJID09IG9mZnNldG9mIChzdHJ1Y3QgRWxmLCBzdGF0ZS5lbGY2NC5y YXdjaHVua3MpKQogCSAgICAgPyBlbGYtPnN0YXRlLmVsZjMyLnJhd2NodW5rcwogCSAgICAgOiBl bGYtPnN0YXRlLmVsZjY0LnJhd2NodW5rcyk7Ci0Jd2hpbGUgKHJhd2NodW5rcyAhPSBOVUxMKQot CSAgewotCSAgICBFbGZfRGF0YV9DaHVuayAqbmV4dCA9IHJhd2NodW5rcy0+bmV4dDsKLQkgICAg aWYgKHJhd2NodW5rcy0+ZHVtbXlfc2NuLmZsYWdzICYgRUxGX0ZfTUFMTE9DRUQpCi0JICAgICAg ZnJlZSAocmF3Y2h1bmtzLT5kYXRhLmQuZF9idWYpOwotCSAgICBmcmVlIChyYXdjaHVua3MpOwot CSAgICByYXdjaHVua3MgPSBuZXh0OwotCSAgfQorCXRkZXN0cm95IChyYXdjaHVua3MsIGZyZWVf Y2h1bmspOwogCiAJRWxmX1Njbkxpc3QgKmxpc3QgPSAoZWxmLT5jbGFzcyA9PSBFTEZDTEFTUzMy CiAJCQkgICAgIHx8IChvZmZzZXRvZiAoc3RydWN0IEVsZiwgc3RhdGUuZWxmMzIuc2NucykKZGlm ZiAtLWdpdCBhL2xpYmVsZi9lbGZfZ2V0ZGF0YV9yYXdjaHVuay5jIGIvbGliZWxmL2VsZl9nZXRk YXRhX3Jhd2NodW5rLmMKaW5kZXggNWEzNWNjZGMuLmNmZDQwMzk2IDEwMDY0NAotLS0gYS9saWJl bGYvZWxmX2dldGRhdGFfcmF3Y2h1bmsuYworKysgYi9saWJlbGYvZWxmX2dldGRhdGFfcmF3Y2h1 bmsuYwpAQCAtMSw2ICsxLDYgQEAKIC8qIFJldHVybiBjb252ZXJ0ZWQgZGF0YSBmcm9tIHJhdyBj aHVuayBvZiBFTEYgZmlsZS4KICAgIENvcHlyaWdodCAoQykgMjAwNywgMjAxNCwgMjAxNSBSZWQg SGF0LCBJbmMuCi0gICBDb3B5cmlnaHQgKEMpIDIwMjIgTWFyayBKLiBXaWVsYWFyZCA8bWFya0Br bG9tcC5vcmc+CisgICBDb3B5cmlnaHQgKEMpIDIwMjIsIDIwMjMgTWFyayBKLiBXaWVsYWFyZCA8 bWFya0BrbG9tcC5vcmc+CiAgICBUaGlzIGZpbGUgaXMgcGFydCBvZiBlbGZ1dGlscy4KIAogICAg VGhpcyBmaWxlIGlzIGZyZWUgc29mdHdhcmU7IHlvdSBjYW4gcmVkaXN0cmlidXRlIGl0IGFuZC9v ciBtb2RpZnkKQEAgLTMzLDEyICszMywyOCBAQAogCiAjaW5jbHVkZSA8YXNzZXJ0Lmg+CiAjaW5j bHVkZSA8ZXJybm8uaD4KKyNpbmNsdWRlIDxzZWFyY2guaD4KICNpbmNsdWRlIDxzdGRsaWIuaD4K ICNpbmNsdWRlIDxzdHJpbmcuaD4KIAogI2luY2x1ZGUgImxpYmVsZlAuaCIKICNpbmNsdWRlICJj b21tb24uaCIKIAorc3RhdGljIGludAorY2h1bmtfY29tcGFyZSAoY29uc3Qgdm9pZCAqYSwgY29u c3Qgdm9pZCAqYikKK3sKKyAgRWxmX0RhdGFfQ2h1bmsgKmRhID0gKEVsZl9EYXRhX0NodW5rICop YTsKKyAgRWxmX0RhdGFfQ2h1bmsgKmRiID0gKEVsZl9EYXRhX0NodW5rICopYjsKKworICBpZiAo ZGEtPm9mZnNldCAhPSBkYi0+b2Zmc2V0KQorICAgIHJldHVybiBkYS0+b2Zmc2V0IC0gZGItPm9m ZnNldDsKKworICBpZiAoZGEtPmRhdGEuZC5kX3NpemUgIT0gZGItPmRhdGEuZC5kX3NpemUpCisg ICAgcmV0dXJuIGRhLT5kYXRhLmQuZF9zaXplIC0gZGItPmRhdGEuZC5kX3NpemU7CisKKyAgcmV0 dXJuIGRhLT5kYXRhLmQuZF90eXBlIC0gZGItPmRhdGEuZC5kX3R5cGU7Cit9CisKIEVsZl9EYXRh ICoKIGVsZl9nZXRkYXRhX3Jhd2NodW5rIChFbGYgKmVsZiwgaW50NjRfdCBvZmZzZXQsIHNpemVf dCBzaXplLCBFbGZfVHlwZSB0eXBlKQogewpAQCAtNzUsMTkgKzkxLDI1IEBAIGVsZl9nZXRkYXRh X3Jhd2NodW5rIChFbGYgKmVsZiwgaW50NjRfdCBvZmZzZXQsIHNpemVfdCBzaXplLCBFbGZfVHlw ZSB0eXBlKQogICByd2xvY2tfcmRsb2NrIChlbGYtPmxvY2spOwogCiAgIC8qIE1heWJlIHdlIGFs cmVhZHkgZ290IHRoaXMgY2h1bms/ICAqLwotICBFbGZfRGF0YV9DaHVuayAqcmF3Y2h1bmtzID0g ZWxmLT5zdGF0ZS5lbGYucmF3Y2h1bmtzOwotICB3aGlsZSAocmF3Y2h1bmtzICE9IE5VTEwpCisg IEVsZl9EYXRhX0NodW5rIGtleTsKKyAga2V5Lm9mZnNldCA9IG9mZnNldDsKKyAga2V5LmRhdGEu ZC5kX3NpemUgPSBzaXplOworICBrZXkuZGF0YS5kLmRfdHlwZSA9IHR5cGU7CisgIEVsZl9EYXRh X0NodW5rICoqZm91bmQgPSB0c2VhcmNoICgma2V5LCAmZWxmLT5zdGF0ZS5lbGYucmF3Y2h1bmtz LAorCQkJCSAgICAmY2h1bmtfY29tcGFyZSk7CisgIGlmIChmb3VuZCA9PSBOVUxMKQorICAgIGdv dG8gbm9tZW07CisKKyAgLyogRXhpc3RpbmcgZW50cnkuICAqLworICBpZiAoKmZvdW5kICE9ICZr ZXkgJiYgKmZvdW5kICE9IE5VTEwpCiAgICAgewotICAgICAgaWYgKChyYXdjaHVua3MtPm9mZnNl dCA9PSBvZmZzZXQgfHwgc2l6ZSA9PSAwKQotCSAgJiYgcmF3Y2h1bmtzLT5kYXRhLmQuZF9zaXpl ID09IHNpemUKLQkgICYmIHJhd2NodW5rcy0+ZGF0YS5kLmRfdHlwZSA9PSB0eXBlKQotCXsKLQkg IHJlc3VsdCA9ICZyYXdjaHVua3MtPmRhdGEuZDsKLQkgIGdvdG8gb3V0OwotCX0KLSAgICAgIHJh d2NodW5rcyA9IHJhd2NodW5rcy0+bmV4dDsKKyAgICAgIHJlc3VsdCA9ICYoKmZvdW5kKS0+ZGF0 YS5kOworICAgICAgZ290byBvdXQ7CiAgICAgfQogCisgIC8qIE5ldyBlbnRyeS4gICovCisgICpm b3VuZCA9IE5VTEw7CisKICAgc2l6ZV90IGFsaWduID0gX19saWJlbGZfdHlwZV9hbGlnbiAoZWxm LT5jbGFzcywgdHlwZSk7CiAgIGlmIChlbGYtPm1hcF9hZGRyZXNzICE9IE5VTEwpCiAgICAgewpA QCAtMTg5LDggKzIxMSw3IEBAIGVsZl9nZXRkYXRhX3Jhd2NodW5rIChFbGYgKmVsZiwgaW50NjRf dCBvZmZzZXQsIHNpemVfdCBzaXplLCBFbGZfVHlwZSB0eXBlKQogICByd2xvY2tfdW5sb2NrIChl bGYtPmxvY2spOwogICByd2xvY2tfd3Jsb2NrIChlbGYtPmxvY2spOwogCi0gIGNodW5rLT5uZXh0 ID0gZWxmLT5zdGF0ZS5lbGYucmF3Y2h1bmtzOwotICBlbGYtPnN0YXRlLmVsZi5yYXdjaHVua3Mg PSBjaHVuazsKKyAgKmZvdW5kID0gY2h1bms7CiAgIHJlc3VsdCA9ICZjaHVuay0+ZGF0YS5kOwog CiAgb3V0OgpkaWZmIC0tZ2l0IGEvbGliZWxmL2xpYmVsZlAuaCBiL2xpYmVsZi9saWJlbGZQLmgK aW5kZXggNjYyNGYzOGEuLmQzYzI0MWU1IDEwMDY0NAotLS0gYS9saWJlbGYvbGliZWxmUC5oCisr KyBiL2xpYmVsZi9saWJlbGZQLmgKQEAgLTEsNSArMSw2IEBACiAvKiBJbnRlcm5hbCBpbnRlcmZh Y2VzIGZvciBsaWJlbGYuCiAgICBDb3B5cmlnaHQgKEMpIDE5OTgtMjAxMCwgMjAxNSwgMjAxNiBS ZWQgSGF0LCBJbmMuCisgICBDb3B5cmlnaHQgKEMpIDIwMjMgTWFyayBKLiBXaWVsYWFyZCA8bWFy a0BrbG9tcC5vcmc+CiAgICBUaGlzIGZpbGUgaXMgcGFydCBvZiBlbGZ1dGlscy4KICAgIENvbnRy aWJ1dGVkIGJ5IFVscmljaCBEcmVwcGVyIDxkcmVwcGVyQHJlZGhhdC5jb20+LCAxOTk4LgogCkBA IC0yNjIsMTEgKzI2Myw3IEBAIHR5cGVkZWYgc3RydWN0IEVsZl9TY25MaXN0CiB0eXBlZGVmIHN0 cnVjdCBFbGZfRGF0YV9DaHVuawogewogICBFbGZfRGF0YV9TY24gZGF0YTsKLSAgdW5pb24KLSAg ewotICAgIEVsZl9TY24gZHVtbXlfc2NuOwotICAgIHN0cnVjdCBFbGZfRGF0YV9DaHVuayAqbmV4 dDsKLSAgfTsKKyAgRWxmX1NjbiBkdW1teV9zY247CiAgIGludDY0X3Qgb2Zmc2V0OwkJLyogVGhl IG9yaWdpbmFsIHJhdyBvZmZzZXQgaW4gdGhlIEVsZiBpbWFnZS4gICovCiB9IEVsZl9EYXRhX0No dW5rOwogCkBAIC0zMjQsNyArMzIxLDcgQEAgc3RydWN0IEVsZgogICAgICAgRWxmX1Njbkxpc3Qg KnNjbnNfbGFzdDsJLyogTGFzdCBlbGVtZW50IGluIHRoZSBzZWN0aW9uIGxpc3QuCiAJCQkJICAg SWYgTlVMTCB0aGUgZGF0YSBoYXMgbm90IHlldCBiZWVuCiAJCQkJICAgcmVhZCBmcm9tIHRoZSBm aWxlLiAgKi8KLSAgICAgIEVsZl9EYXRhX0NodW5rICpyYXdjaHVua3M7IC8qIExpc3Qgb2YgZWxm X2dldGRhdGFfcmF3Y2h1bmsgcmVzdWx0cy4gICovCisgICAgICB2b2lkICpyYXdjaHVua3M7CQkv KiBUcmVlIG9mIGVsZl9nZXRkYXRhX3Jhd2NodW5rIHJlc3VsdHMuICAqLwogICAgICAgdW5zaWdu ZWQgaW50IHNjbmluY3I7CS8qIE51bWJlciBvZiBzZWN0aW9ucyBhbGxvY2F0ZSB0aGUgbGFzdAog CQkJCSAgIHRpbWUuICAqLwogICAgICAgaW50IGVoZHJfZmxhZ3M7CQkvKiBGbGFncyAoZGlydHkp IGZvciBFTEYgaGVhZGVyLiAgKi8KQEAgLTM0Myw3ICszNDAsNyBAQCBzdHJ1Y3QgRWxmCiAgICAg ICBFbGZfU2NuTGlzdCAqc2Nuc19sYXN0OwkvKiBMYXN0IGVsZW1lbnQgaW4gdGhlIHNlY3Rpb24g bGlzdC4KIAkJCQkgICBJZiBOVUxMIHRoZSBkYXRhIGhhcyBub3QgeWV0IGJlZW4KIAkJCQkgICBy ZWFkIGZyb20gdGhlIGZpbGUuICAqLwotICAgICAgRWxmX0RhdGFfQ2h1bmsgKnJhd2NodW5rczsg LyogTGlzdCBvZiBlbGZfZ2V0ZGF0YV9yYXdjaHVuayByZXN1bHRzLiAgKi8KKyAgICAgIHZvaWQg KnJhd2NodW5rczsJCS8qIFRyZWUgb2YgZWxmX2dldGRhdGFfcmF3Y2h1bmsgcmVzdWx0cy4gICov CiAgICAgICB1bnNpZ25lZCBpbnQgc2NuaW5jcjsJLyogTnVtYmVyIG9mIHNlY3Rpb25zIGFsbG9j YXRlIHRoZSBsYXN0CiAJCQkJICAgdGltZS4gICovCiAgICAgICBpbnQgZWhkcl9mbGFnczsJCS8q IEZsYWdzIChkaXJ0eSkgZm9yIEVMRiBoZWFkZXIuICAqLwpAQCAtMzY4LDcgKzM2NSw3IEBAIHN0 cnVjdCBFbGYKICAgICAgIEVsZl9TY25MaXN0ICpzY25zX2xhc3Q7CS8qIExhc3QgZWxlbWVudCBp biB0aGUgc2VjdGlvbiBsaXN0LgogCQkJCSAgIElmIE5VTEwgdGhlIGRhdGEgaGFzIG5vdCB5ZXQg YmVlbgogCQkJCSAgIHJlYWQgZnJvbSB0aGUgZmlsZS4gICovCi0gICAgICBFbGZfRGF0YV9DaHVu ayAqcmF3Y2h1bmtzOyAvKiBMaXN0IG9mIGVsZl9nZXRkYXRhX3Jhd2NodW5rIHJlc3VsdHMuICAq LworICAgICAgdm9pZCAqcmF3Y2h1bmtzOwkJLyogVHJlZSBvZiBlbGZfZ2V0ZGF0YV9yYXdjaHVu ayByZXN1bHRzLiAgKi8KICAgICAgIHVuc2lnbmVkIGludCBzY25pbmNyOwkvKiBOdW1iZXIgb2Yg c2VjdGlvbnMgYWxsb2NhdGUgdGhlIGxhc3QKIAkJCQkgICB0aW1lLiAgKi8KICAgICAgIGludCBl aGRyX2ZsYWdzOwkJLyogRmxhZ3MgKGRpcnR5KSBmb3IgRUxGIGhlYWRlci4gICovCi0tIAoyLjQw LjEKCg== --=-S1YUfvqucRW43GwWqJ0v--