From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2139.outbound.protection.outlook.com [40.107.244.139]) by sourceware.org (Postfix) with ESMTPS id 60FB63858D29 for ; Fri, 24 May 2024 09:27:38 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 60FB63858D29 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=os.amperecomputing.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=os.amperecomputing.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 60FB63858D29 Authentication-Results: server2.sourceware.org; arc=pass smtp.remote-ip=40.107.244.139 ARC-Seal: i=2; a=rsa-sha256; d=sourceware.org; s=key; t=1716542868; cv=pass; b=Af+rN/ehei6estvE3t+Ii9GMJepCj9dzlHDPZGTUa9wFiVuJoq78z0z1wZx3NwV8klz0L/PwXwJQ3WWB0rEyKgI5lpgLF9gcPXEnB4Wn9jFKQP/D7TakVc+1Md+dLYdc3l+CHwkMrFsbta1NQUShcaNSWKBE8XXoSljYylcyX7A= ARC-Message-Signature: i=2; a=rsa-sha256; d=sourceware.org; s=key; t=1716542868; c=relaxed/simple; bh=NeZ56ps1oogPjCJoQxVx4YiPHkFYGg5edKD8XjMLIfo=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=P/hpDadPAFovBOEkcf9gvumr+hSwcYUmjHR2TMHVV25JyZ/nMUzFqhKZoMsl/AnPObnaNOtYI5Wf9pK20O6AwlfihsIH/jxnOC1ulgtcbXJYNVS8MobT8gp+C5DpIykvmOWRLgjQR9VUC6iHwbdX1jH0yY9hsagrBh5AyU3LFGA= ARC-Authentication-Results: i=2; server2.sourceware.org ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WMe1ydxgJdqBoY0V6G2y6neYav24JeMjRqRVUZrAnGEKSduLPc41Ml79uW05JidF3i5IrtAGCSN/EvVQZDoJy8d//1RKICsqdR/mrd48DscXciUN3eSl4Lc03cGSLCU9MLAPjDQI9Q/iJtDXdzyX0oM5XAVD2n/tQhmhdtUQZaNfRjng/0OjBmgxPviCSzHazN4aLTzpDVg2WHUG2ra+v0g1WNK5cVO7gM2IIYxoJ07LwpYJwN/sI6CutuHFHlWyLMRq7hFE7Ogkk4L6iX4lobfvF0UB/Y6hsEJ2/zci7MCB0weQ/ql3YdrTUyKLWuWl5eXF6PL0Lr/pkMDjr0/eXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=froiYuAg5R2jisijp8hFFBN9V2Yf+X7NcxuF3qRV4iQ=; b=nNkvEtR99GnZMm2UBKOLIopLKKFunJ5Ou2pof+yzCDpdwdhJufLev88lNWGzQ3/gXOSHm9hvsBFBfRd9A+K7GsvSu8qjr5xvai8Vrh3SPJof6yO49aOpiC1Yew0U+3YrFCDcLd5c2Fwgjhp53KnrpZ2ziQDYND+0fYoumjcYuYEFWjtCWEvHdVueoXsFXticKciR9MlQHT58povolaWUel+Tm0ilfwC6jhwe+O2tzrKpMFB1mZ7JKl7v+MTh/ifW0jvuY5L7fYi8Ita3sUsFzLsWzGilfsz57pDTJv5VEzIb0jXpeaBh7yIxdNZL+jvM6xIhvGcD0VZ312viOqhGGA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=os.amperecomputing.com; dmarc=pass action=none header.from=os.amperecomputing.com; dkim=pass header.d=os.amperecomputing.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=os.amperecomputing.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=froiYuAg5R2jisijp8hFFBN9V2Yf+X7NcxuF3qRV4iQ=; b=QNB0MXTm8Nwgi2h0hBRT9ShVGVEyzPbqbLSnZ1lviQddvmflvLNZwyLLAZKJ9eBxnVpNtJlv/FttxVQdGz/QJrfmysR++yDtSoz4CU1fHRnUBTJOd3WPkTvRbBZJDcZ8DqolTU8L2/vHpWtAFETzPCAIa7kn5vXDZ8NpSAy2J8Q= Received: from LV2PR01MB7839.prod.exchangelabs.com (2603:10b6:408:14f::13) by PH0PR01MB6453.prod.exchangelabs.com (2603:10b6:510:12::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.26; Fri, 24 May 2024 09:27:33 +0000 Received: from LV2PR01MB7839.prod.exchangelabs.com ([fe80::2ac3:5a77:36fd:9c63]) by LV2PR01MB7839.prod.exchangelabs.com ([fe80::2ac3:5a77:36fd:9c63%3]) with mapi id 15.20.7611.016; Fri, 24 May 2024 09:27:33 +0000 From: Feng Xue OS To: Richard Biener CC: Tamar Christina , Richard Sandiford , "gcc-patches@gcc.gnu.org" Subject: Re: [PATCH] vect: Support multiple lane-reducing operations for loop reduction [PR114440] Thread-Topic: [PATCH] vect: Support multiple lane-reducing operations for loop reduction [PR114440] Thread-Index: AQHaiLfx2V8F9HRn8EOSIzJjtr87rbGmXvso Date: Fri, 24 May 2024 09:27:33 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: msip_labels: MSIP_Label_5b82cb1d-c2e0-4643-920a-bbe7b2d7cc47_Enabled=True;MSIP_Label_5b82cb1d-c2e0-4643-920a-bbe7b2d7cc47_SiteId=3bc2b170-fd94-476d-b0ce-4229bdc904a7;MSIP_Label_5b82cb1d-c2e0-4643-920a-bbe7b2d7cc47_SetDate=2024-05-24T09:27:32.243Z;MSIP_Label_5b82cb1d-c2e0-4643-920a-bbe7b2d7cc47_Name=Confidential;MSIP_Label_5b82cb1d-c2e0-4643-920a-bbe7b2d7cc47_ContentBits=0;MSIP_Label_5b82cb1d-c2e0-4643-920a-bbe7b2d7cc47_Method=Standard; authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=os.amperecomputing.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: LV2PR01MB7839:EE_|PH0PR01MB6453:EE_ x-ms-office365-filtering-correlation-id: c3bfc7b3-ae29-48db-b422-08dc7bd3bdfc x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0;ARA:13230031|376005|1800799015|366007|38070700009; x-microsoft-antispam-message-info: =?iso-8859-1?Q?Y/ulLAXz0EIRasdNgd1E0BzkHx6Cs8pukmtulB881avLta45Av3cBf4E0O?= =?iso-8859-1?Q?ZrBeMm551+TQGP0gO5XVMtFDJsAVPA7IRKeQpd66Odp8+vPRQFnKWPkenZ?= =?iso-8859-1?Q?PfbP6e6+b6YILebynY7G/YkQ1L2Jv1UjaFqz/HZ+VWzvikGQDpxp9i4KCj?= =?iso-8859-1?Q?zStqZGSFsj9bVDKs4N7JKEkszBcYjfiQauH5/HDlcgVLVlH+L6gP3D2+kp?= =?iso-8859-1?Q?we1H5Pt8NNKvqH3BvWfFGY3drPANzlZMiFEK4jzeDygWL4SpbYTg5Kgjt7?= =?iso-8859-1?Q?NtqAQmuQupwyueOP5QsmKgCgVMwKce211AtLACoNQJwO3cVQrY+wwFiGyN?= =?iso-8859-1?Q?oA0Q+LD16pQ9ZT4n+kdo6XhZCbodXydqzlnZ7F3iHaV+v50lB4n8+HnqMO?= =?iso-8859-1?Q?2Ze06gnWF9V2uLtRNuTVJfkl2sKVQ0aTsKhuY/ooK2qhANFcbxADlR6AqN?= =?iso-8859-1?Q?vokkLupLzdqqesyzYc2sTQK4qr5Vtj+zWOMCiJIAjokWArYQrsM11tfTCx?= =?iso-8859-1?Q?YB9xTBqu5xoSKj7FkyNZ76tkbD8xzms+CWLVR9Bx/eUyJsD9/dTbSi7A85?= =?iso-8859-1?Q?2vriyl0/hPtskP7UxrFVR2i363juA4bxLCqA+500N6ncsk/350sLMFLt46?= =?iso-8859-1?Q?r+Jul7Qd9dr9pbusOSpR2EnVfhPfa4ZWg0VPy8ijuzWs6r/hItRdx87aXZ?= =?iso-8859-1?Q?G6hXXJ63gdMj2ygMykXBx/jfI0Q/+xdDkQkqxxPGOtUG2bqTAzajxf+oyV?= =?iso-8859-1?Q?qfT+MGxk9seoliNw9kxwScFgf76N1ViIV8upLC1ZIvLYI5IPvraSb1BGnD?= =?iso-8859-1?Q?XscINGvDtZ1/ZCk7gU7vn0aRQmg7gsIVEDd+WYCnuQItdDCO0vVqJHXH8p?= =?iso-8859-1?Q?bb61U6MGunpUdPeti1kIM2w5kd2yKU1kHSLegjdRJsT7Qgye/+nizqeWf4?= =?iso-8859-1?Q?XLXAJehe085wJ82uEd0/C7NcJ5I/PG1DzjYcQbp6loQ1sCvyg+qNoe9mgL?= =?iso-8859-1?Q?+NnXqS5DAWiM8jobep86KZtCGk2eStrAYylJFn0AnHNRiZJ2PwPxkkACNO?= =?iso-8859-1?Q?N+c3Dpak2RRMK2bcfpqfUosnApImnunTElXZxzWJt6AvAp4/7WswnzlPcs?= =?iso-8859-1?Q?rzyLXABWcOt4+FbabjpRkR9vEC4YQVIqYEhrB4AbZVUaMG9sCGy9QSMBEe?= =?iso-8859-1?Q?gE+AUB+ejLYo9YUz/N3rHXYPzP+K9q+qMC8NSQmQUJKmXS9IrA2FDOJynk?= =?iso-8859-1?Q?8I+vvDeonMAfDlFHozsImTMi662+8Rdl5HQBHc9I3y6Vd1VwbVUdP7Znba?= =?iso-8859-1?Q?2LFyyqh5dJp+pUAObf3BADh+WA=3D=3D?= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV2PR01MB7839.prod.exchangelabs.com;PTR:;CAT:NONE;SFS:(13230031)(376005)(1800799015)(366007)(38070700009);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?lZfvo///957Ein0dMzkY7HANemCQUd6mEyfz09cueGaHrrsQ5kKz/WbaJp?= =?iso-8859-1?Q?xGpR+xDFBmf0XK0E50nL1zI2Sd1D4JPzj17KDIF2lnGx54WaXhSn20prec?= =?iso-8859-1?Q?4WsZypFYLPGTD//hNzu2tS1EWTXdwpXHV59LojZN7Jz4U7BXOEv6nPf7wa?= =?iso-8859-1?Q?qZWiTD9QsbaSAjF+u20c+LVThx1Hyr5tJrIxnqDe64imajnBVMYknhIRwt?= =?iso-8859-1?Q?w5ONAm+pQf1jDrK6Bo8k75ZrwlqcMVQQUAB2abeWVYQijE4H2BrwMBe/b3?= =?iso-8859-1?Q?j1raCAAiNdOj0f9yVsS2/pGYfhCJDSAidDsEgCmH/ljoQpdQ6qYN2Uw1OT?= =?iso-8859-1?Q?tkfsC++FUFliDfXcb1C8EgD9lN6oHP2ncvSLCtGNVoZCq2gHYDr5mT39Zz?= =?iso-8859-1?Q?ZIMyTiHIP7Dr1NZdT1xZ7wRpMvWpXzxhXmaSDUIG9FkngpUEpj+DTs+/I6?= =?iso-8859-1?Q?/pU26tg6pJe/BTvQcJ6rgRvd7CHYgJ5oSVqwrumALUIUZ/tcuBRJzPdolA?= =?iso-8859-1?Q?owMbEOpJMWU6Hj+fAtScP6J1IcTTUWBGvbvRFtO02QB7YvgSVdMwcbwhz/?= =?iso-8859-1?Q?a0kDKzkW9h2a8m046FPm7JGMAeXmwTFhy9NJaw23z4KVSm30LSRn1ER/iD?= =?iso-8859-1?Q?xlgGaxc80g8/aG7CYgDCRzGeghLOb4acVM3VyO5MtiDi2GkaRRWboBZ2+b?= =?iso-8859-1?Q?lOxokc3Dp3bItVNGJOUPb5GSTqugqpKDT9i4DnBStr1bdgJYxlnsJMFKo+?= =?iso-8859-1?Q?tVFeDE9Ti0ktJ2C+vDfHxbiSn6sZ2uq0UDPbKCB2tu2HBbf9B4aICMKjdO?= =?iso-8859-1?Q?MZUrFlNpSjXHA08VZklwcM/Ny63m0xaaXJLW48reKdDV2q5glYMF5qy8xz?= =?iso-8859-1?Q?FiHjoIqQ0yiVkQpEw+pdgfzuuDE0SnaceEFH/J3SE0Dap2Sws+5nC5iRm4?= =?iso-8859-1?Q?7OFJLug2RxDAdhSXHALuzdj3CWyoYc4JGv/f8o7TwlCPoQx0e0KQxgSv58?= =?iso-8859-1?Q?GXROkT7UBQSt3xeG+GLKeZ0Fmwqcff4VqoWuAlcX88bkhzRHYWeYwTRUvH?= =?iso-8859-1?Q?n92Z1QjSvN5CQCmvekhf23Dyc7WxOfGC7GrODP1UDjjIbui+Oh0s5xnaRl?= =?iso-8859-1?Q?NkkBhEbHoYEy2SM3FwnbQI1xOeeNnY0YTfrdy3vCIoL//SP9XLipPdRweO?= =?iso-8859-1?Q?5F18+oLbTHG6A/H8QGDt8hQF5IenwMkxIG9/eUvXsP0k9WJE6fUd2ChtjY?= =?iso-8859-1?Q?JU4gkmzDYMLFOwLUsn2vLfHb9OGPliInxPsruiPaGdRBloaBwZh70UvuIu?= =?iso-8859-1?Q?s6HtQs7Qe6zFPFkarAlGG1nxkREeCag1hPYCm0FAAm1tutJJRHYiYHnLkh?= =?iso-8859-1?Q?YU+8uuqR8ruFrKtdNQhq8oL/CtqgUdNW2dHr8Jru8b9GZO5w59dVOSUa7Z?= =?iso-8859-1?Q?mr+7EAs0I606qp5U51G9WmrNC+L7JkinGxS6dAXaUaO7boc67B2aepE8wt?= =?iso-8859-1?Q?S+hEOd3LXiV3eTrp8ZSaQANjoo9OEFINFhwSpPnj7bh0Vb76NZE7cu2faK?= =?iso-8859-1?Q?DnuU++zqVKMEZqq1TlL5r4tn0L2KDy+NWeDogTsz2iFXOtFTrZYfDhLKc4?= =?iso-8859-1?Q?nExmpTLY+T3GKI1g/dtHSSPCBs5114UR1N?= Content-Type: multipart/mixed; boundary="_002_LV2PR01MB78397C320D7870B56652ACC6F7F52LV2PR01MB7839prod_" MIME-Version: 1.0 X-OriginatorOrg: os.amperecomputing.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: LV2PR01MB7839.prod.exchangelabs.com X-MS-Exchange-CrossTenant-Network-Message-Id: c3bfc7b3-ae29-48db-b422-08dc7bd3bdfc X-MS-Exchange-CrossTenant-originalarrivaltime: 24 May 2024 09:27:33.5385 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: h4QG9MsFs2vjJ48/KTdihee4rxY2CaDOSISdxK66sFRCyjKVGSU9fvXz1uNk+MQ2nb17PXZgO2MzhPEG4AuF1oFG00VH8RGt4eyc1Gt2RSk= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR01MB6453 X-Spam-Status: No, score=-11.7 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,GIT_PATCH_0,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: --_002_LV2PR01MB78397C320D7870B56652ACC6F7F52LV2PR01MB7839prod_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi,=0A= =0A= The patch was updated with the newest trunk, and also contained some minor = changes.=0A= =0A= I am working on another new feature which is meant to support pattern recog= nition=0A= of lane-reducing operations in affine closure originated from loop reductio= n variable,=0A= like:=0A= =0A= sum +=3D cst1 * dot_prod_1 + cst2 * sad_2 + ... + cstN * lane_reducing_op= _N=0A= =0A= The feature WIP depends on the patch. It has been a little bit long time si= nce its post,=0A= would you please take a time to review this one? Thanks.=0A= =0A= Feng=0A= ----=0A= =0A= gcc/=0A= PR tree-optimization/114440=0A= * tree-vectorizer.h (struct _stmt_vec_info): Add a new field=0A= reduc_result_pos.=0A= (lane_reducing_op_p): New function.=0A= (vectorizable_lane_reducing): New function declaration.=0A= * tree-vect-stmts.cc (vectorizable_condition): Treat the condition=0A= statement that is pointed by stmt_vec_info of reduction PHI as the=0A= real "for_reduction" statement.=0A= (vect_analyze_stmt): Call new function vectorizable_lane_reducing=0A= to analyze lane-reducing operation.=0A= * tree-vect-loop.cc (vect_is_emulated_mixed_dot_prod): Remove parameter=0A= loop_vinfo. Get input vectype from stmt_info instead of reduction PHI.=0A= (vect_model_reduction_cost): Remove cost computation code related to=0A= emulated_mixed_dot_prod.=0A= (vect_reduction_use_partial_vector): New function.=0A= (vectorizable_lane_reducing): New function.=0A= (vectorizable_reduction): Allow multiple lane-reducing operations in=0A= loop reduction. Move some original lane-reducing related code to=0A= vectorizable_lane_reducing, and move partial vectorization checking=0A= code to vect_reduction_use_partial_vector.=0A= (vect_transform_reduction): Extend transformation to support reduction=0A= statements with mixed input vectypes.=0A= * tree-vect-slp.cc (vect_analyze_slp): Use new function=0A= lane_reducing_op_p to check statement code.=0A= =0A= gcc/testsuite/=0A= PR tree-optimization/114440=0A= * gcc.dg/vect/vect-reduc-chain-1.c=0A= * gcc.dg/vect/vect-reduc-chain-2.c=0A= * gcc.dg/vect/vect-reduc-chain-3.c=0A= * gcc.dg/vect/vect-reduc-dot-slp-1.c=0A= * gcc.dg/vect/vect-reduc-dot-slp-2.c=0A= ---=0A= .../gcc.dg/vect/vect-reduc-chain-1.c | 62 ++=0A= .../gcc.dg/vect/vect-reduc-chain-2.c | 77 ++=0A= .../gcc.dg/vect/vect-reduc-chain-3.c | 66 ++=0A= .../gcc.dg/vect/vect-reduc-dot-slp-1.c | 97 +++=0A= .../gcc.dg/vect/vect-reduc-dot-slp-2.c | 81 +++=0A= gcc/tree-vect-loop.cc | 680 ++++++++++++------=0A= gcc/tree-vect-slp.cc | 4 +-=0A= gcc/tree-vect-stmts.cc | 13 +-=0A= gcc/tree-vectorizer.h | 14 +=0A= 9 files changed, 873 insertions(+), 221 deletions(-)=0A= create mode 100644 gcc/testsuite/gcc.dg/vect/vect-reduc-chain-1.c=0A= create mode 100644 gcc/testsuite/gcc.dg/vect/vect-reduc-chain-2.c=0A= create mode 100644 gcc/testsuite/gcc.dg/vect/vect-reduc-chain-3.c=0A= create mode 100644 gcc/testsuite/gcc.dg/vect/vect-reduc-dot-slp-1.c=0A= create mode 100644 gcc/testsuite/gcc.dg/vect/vect-reduc-dot-slp-2.c=0A= =0A= diff --git a/gcc/testsuite/gcc.dg/vect/vect-reduc-chain-1.c b/gcc/testsuite= /gcc.dg/vect/vect-reduc-chain-1.c=0A= new file mode 100644=0A= index 00000000000..04bfc419dbd=0A= --- /dev/null=0A= +++ b/gcc/testsuite/gcc.dg/vect/vect-reduc-chain-1.c=0A= @@ -0,0 +1,62 @@=0A= +/* Disabling epilogues until we find a better way to deal with scans. */= =0A= +/* { dg-additional-options "--param vect-epilogues-nomask=3D0" } */=0A= +/* { dg-require-effective-target vect_int } */=0A= +/* { dg-require-effective-target arm_v8_2a_dotprod_neon_hw { target { aarc= h64*-*-* || arm*-*-* } } } */=0A= +/* { dg-add-options arm_v8_2a_dotprod_neon } */=0A= +=0A= +#include "tree-vect.h"=0A= +=0A= +#define N 50=0A= +=0A= +#ifndef SIGNEDNESS_1=0A= +#define SIGNEDNESS_1 signed=0A= +#define SIGNEDNESS_2 signed=0A= +#endif=0A= +=0A= +SIGNEDNESS_1 int __attribute__ ((noipa))=0A= +f (SIGNEDNESS_1 int res,=0A= + SIGNEDNESS_2 char *restrict a,=0A= + SIGNEDNESS_2 char *restrict b,=0A= + SIGNEDNESS_2 char *restrict c,=0A= + SIGNEDNESS_2 char *restrict d,=0A= + SIGNEDNESS_1 int *restrict e)=0A= +{=0A= + for (int i =3D 0; i < N; ++i)=0A= + {=0A= + res +=3D a[i] * b[i];=0A= + res +=3D c[i] * d[i];=0A= + res +=3D e[i];=0A= + }=0A= + return res;=0A= +}=0A= +=0A= +#define BASE ((SIGNEDNESS_2 int) -1 < 0 ? -126 : 4)=0A= +#define OFFSET 20=0A= +=0A= +int=0A= +main (void)=0A= +{=0A= + check_vect ();=0A= +=0A= + SIGNEDNESS_2 char a[N], b[N];=0A= + SIGNEDNESS_2 char c[N], d[N];=0A= + SIGNEDNESS_1 int e[N];=0A= + int expected =3D 0x12345;=0A= + for (int i =3D 0; i < N; ++i)=0A= + {=0A= + a[i] =3D BASE + i * 5;=0A= + b[i] =3D BASE + OFFSET + i * 4;=0A= + c[i] =3D BASE + i * 2;=0A= + d[i] =3D BASE + OFFSET + i * 3;=0A= + e[i] =3D i;=0A= + asm volatile ("" ::: "memory");=0A= + expected +=3D a[i] * b[i];=0A= + expected +=3D c[i] * d[i];=0A= + expected +=3D e[i];=0A= + }=0A= + if (f (0x12345, a, b, c, d, e) !=3D expected)=0A= + __builtin_abort ();=0A= +}=0A= +=0A= +/* { dg-final { scan-tree-dump "vect_recog_dot_prod_pattern: detected" "ve= ct" } } */=0A= +/* { dg-final { scan-tree-dump-times "vectorizing statement: \\S+ =3D DOT_= PROD_EXPR" 2 "vect" { target vect_sdot_qi } } } */=0A= diff --git a/gcc/testsuite/gcc.dg/vect/vect-reduc-chain-2.c b/gcc/testsuite= /gcc.dg/vect/vect-reduc-chain-2.c=0A= new file mode 100644=0A= index 00000000000..6c803b80120=0A= --- /dev/null=0A= +++ b/gcc/testsuite/gcc.dg/vect/vect-reduc-chain-2.c=0A= @@ -0,0 +1,77 @@=0A= +/* Disabling epilogues until we find a better way to deal with scans. */= =0A= +/* { dg-additional-options "--param vect-epilogues-nomask=3D0" } */=0A= +/* { dg-require-effective-target vect_int } */=0A= +/* { dg-require-effective-target arm_v8_2a_dotprod_neon_hw { target { aarc= h64*-*-* || arm*-*-* } } } */=0A= +/* { dg-add-options arm_v8_2a_dotprod_neon } */=0A= +=0A= +#include "tree-vect.h"=0A= +=0A= +#define N 50=0A= +=0A= +#ifndef SIGNEDNESS_1=0A= +#define SIGNEDNESS_1 signed=0A= +#define SIGNEDNESS_2 unsigned=0A= +#define SIGNEDNESS_3 signed=0A= +#define SIGNEDNESS_4 signed=0A= +#endif=0A= +=0A= +SIGNEDNESS_1 int __attribute__ ((noipa))=0A= +fn (SIGNEDNESS_1 int res,=0A= + SIGNEDNESS_2 char *restrict a,=0A= + SIGNEDNESS_2 char *restrict b,=0A= + SIGNEDNESS_3 char *restrict c,=0A= + SIGNEDNESS_3 char *restrict d,=0A= + SIGNEDNESS_4 short *restrict e,=0A= + SIGNEDNESS_4 short *restrict f,=0A= + SIGNEDNESS_1 int *restrict g)=0A= +{=0A= + for (int i =3D 0; i < N; ++i)=0A= + {=0A= + res +=3D a[i] * b[i];=0A= + res +=3D i + 1;=0A= + res +=3D c[i] * d[i];=0A= + res +=3D e[i] * f[i];=0A= + res +=3D g[i];=0A= + }=0A= + return res;=0A= +}=0A= +=0A= +#define BASE2 ((SIGNEDNESS_2 int) -1 < 0 ? -126 : 4)=0A= +#define BASE3 ((SIGNEDNESS_3 int) -1 < 0 ? -126 : 4)=0A= +#define BASE4 ((SIGNEDNESS_4 int) -1 < 0 ? -1026 : 373)=0A= +#define OFFSET 20=0A= +=0A= +int=0A= +main (void)=0A= +{=0A= + check_vect ();=0A= +=0A= + SIGNEDNESS_2 char a[N], b[N];=0A= + SIGNEDNESS_3 char c[N], d[N];=0A= + SIGNEDNESS_4 short e[N], f[N];=0A= + SIGNEDNESS_1 int g[N];=0A= + int expected =3D 0x12345;=0A= + for (int i =3D 0; i < N; ++i)=0A= + {=0A= + a[i] =3D BASE2 + i * 5;=0A= + b[i] =3D BASE2 + OFFSET + i * 4;=0A= + c[i] =3D BASE3 + i * 2;=0A= + d[i] =3D BASE3 + OFFSET + i * 3;=0A= + e[i] =3D BASE4 + i * 6;=0A= + f[i] =3D BASE4 + OFFSET + i * 5;=0A= + g[i] =3D i;=0A= + asm volatile ("" ::: "memory");=0A= + expected +=3D a[i] * b[i];=0A= + expected +=3D i + 1;=0A= + expected +=3D c[i] * d[i];=0A= + expected +=3D e[i] * f[i];=0A= + expected +=3D g[i];=0A= + }=0A= + if (fn (0x12345, a, b, c, d, e, f, g) !=3D expected)=0A= + __builtin_abort ();=0A= +}=0A= +=0A= +/* { dg-final { scan-tree-dump "vect_recog_dot_prod_pattern: detected" "ve= ct" } } */=0A= +/* { dg-final { scan-tree-dump "vectorizing statement: \\S+ =3D DOT_PROD_E= XPR" "vect" { target { vect_sdot_qi } } } } */=0A= +/* { dg-final { scan-tree-dump "vectorizing statement: \\S+ =3D DOT_PROD_E= XPR" "vect" { target { vect_udot_qi } } } } */=0A= +/* { dg-final { scan-tree-dump "vectorizing statement: \\S+ =3D DOT_PROD_E= XPR" "vect" { target { vect_sdot_hi } } } } */=0A= diff --git a/gcc/testsuite/gcc.dg/vect/vect-reduc-chain-3.c b/gcc/testsuite= /gcc.dg/vect/vect-reduc-chain-3.c=0A= new file mode 100644=0A= index 00000000000..a41e4b176c4=0A= --- /dev/null=0A= +++ b/gcc/testsuite/gcc.dg/vect/vect-reduc-chain-3.c=0A= @@ -0,0 +1,66 @@=0A= +/* Disabling epilogues until we find a better way to deal with scans. */= =0A= +/* { dg-additional-options "--param vect-epilogues-nomask=3D0" } */=0A= +/* { dg-require-effective-target vect_int } */=0A= +=0A= +#include "tree-vect.h"=0A= +=0A= +#define N 50=0A= +=0A= +#ifndef SIGNEDNESS_1=0A= +#define SIGNEDNESS_1 signed=0A= +#define SIGNEDNESS_2 unsigned=0A= +#define SIGNEDNESS_3 signed=0A= +#endif=0A= +=0A= +SIGNEDNESS_1 int __attribute__ ((noipa))=0A= +f (SIGNEDNESS_1 int res,=0A= + SIGNEDNESS_2 char *restrict a,=0A= + SIGNEDNESS_2 char *restrict b,=0A= + SIGNEDNESS_3 short *restrict c,=0A= + SIGNEDNESS_3 short *restrict d,=0A= + SIGNEDNESS_1 int *restrict e)=0A= +{=0A= + for (int i =3D 0; i < N; ++i)=0A= + {=0A= + short diff =3D a[i] - b[i];=0A= + SIGNEDNESS_2 short abs =3D diff < 0 ? -diff : diff;=0A= + res +=3D abs;=0A= + res +=3D c[i] * d[i];=0A= + res +=3D e[i];=0A= + }=0A= + return res;=0A= +}=0A= +=0A= +#define BASE2 ((SIGNEDNESS_2 int) -1 < 0 ? -126 : 4)=0A= +#define BASE3 ((SIGNEDNESS_3 int) -1 < 0 ? -1236 : 373)=0A= +#define OFFSET 20=0A= +=0A= +int=0A= +main (void)=0A= +{=0A= + check_vect ();=0A= +=0A= + SIGNEDNESS_2 char a[N], b[N];=0A= + SIGNEDNESS_3 short c[N], d[N];=0A= + SIGNEDNESS_1 int e[N];=0A= + int expected =3D 0x12345;=0A= + for (int i =3D 0; i < N; ++i)=0A= + {=0A= + a[i] =3D BASE2 + i * 5;=0A= + b[i] =3D BASE2 - i * 4;=0A= + c[i] =3D BASE3 + i * 2;=0A= + d[i] =3D BASE3 + OFFSET + i * 3;=0A= + e[i] =3D i;=0A= + asm volatile ("" ::: "memory");=0A= + short diff =3D a[i] - b[i];=0A= + SIGNEDNESS_2 short abs =3D diff < 0 ? -diff : diff;=0A= + expected +=3D abs;=0A= + expected +=3D c[i] * d[i];=0A= + expected +=3D e[i];=0A= + }=0A= + if (f (0x12345, a, b, c, d, e) !=3D expected)=0A= + __builtin_abort ();=0A= +}=0A= +=0A= +/* { dg-final { scan-tree-dump "vectorizing statement: \\S+ =3D SAD_EXPR" = "vect" { target vect_udot_qi } } } */=0A= +/* { dg-final { scan-tree-dump "vectorizing statement: \\S+ =3D DOT_PROD_E= XPR" "vect" { target vect_sdot_hi } } } */=0A= diff --git a/gcc/testsuite/gcc.dg/vect/vect-reduc-dot-slp-1.c b/gcc/testsui= te/gcc.dg/vect/vect-reduc-dot-slp-1.c=0A= new file mode 100644=0A= index 00000000000..51ef4eaaed8=0A= --- /dev/null=0A= +++ b/gcc/testsuite/gcc.dg/vect/vect-reduc-dot-slp-1.c=0A= @@ -0,0 +1,97 @@=0A= +/* Disabling epilogues until we find a better way to deal with scans. */= =0A= +/* { dg-additional-options "--param vect-epilogues-nomask=3D0" } */=0A= +/* { dg-require-effective-target vect_int } */=0A= +/* { dg-require-effective-target arm_v8_2a_dotprod_neon_hw { target { aarc= h64*-*-* || arm*-*-* } } } */=0A= +/* { dg-add-options arm_v8_2a_dotprod_neon } */=0A= +=0A= +#include "tree-vect.h"=0A= +=0A= +#define N 50=0A= +=0A= +#ifndef SIGNEDNESS_1=0A= +#define SIGNEDNESS_1 signed=0A= +#define SIGNEDNESS_2 signed=0A= +#endif=0A= +=0A= +SIGNEDNESS_1 int __attribute__ ((noipa))=0A= +f (SIGNEDNESS_1 int res,=0A= + SIGNEDNESS_2 char *a,=0A= + SIGNEDNESS_2 char *b,=0A= + int step, int n)=0A= +{=0A= + for (int i =3D 0; i < n; i++)=0A= + {=0A= + res +=3D a[0] * b[0];=0A= + res +=3D a[1] * b[1];=0A= + res +=3D a[2] * b[2];=0A= + res +=3D a[3] * b[3];=0A= + res +=3D a[4] * b[4];=0A= + res +=3D a[5] * b[5];=0A= + res +=3D a[6] * b[6];=0A= + res +=3D a[7] * b[7];=0A= + res +=3D a[8] * b[8];=0A= + res +=3D a[9] * b[9];=0A= + res +=3D a[10] * b[10];=0A= + res +=3D a[11] * b[11];=0A= + res +=3D a[12] * b[12];=0A= + res +=3D a[13] * b[13];=0A= + res +=3D a[14] * b[14];=0A= + res +=3D a[15] * b[15];=0A= +=0A= + a +=3D step;=0A= + b +=3D step;=0A= + }=0A= +=0A= + return res;=0A= +}=0A= +=0A= +#define BASE ((SIGNEDNESS_2 int) -1 < 0 ? -126 : 4)=0A= +#define OFFSET 20=0A= +=0A= +int=0A= +main (void)=0A= +{=0A= + check_vect ();=0A= +=0A= + SIGNEDNESS_2 char a[100], b[100];=0A= + int expected =3D 0x12345;=0A= + int step =3D 16;=0A= + int n =3D 2;=0A= + int t =3D 0;=0A= +=0A= + for (int i =3D 0; i < sizeof (a) / sizeof (a[0]); ++i)=0A= + {=0A= + a[i] =3D BASE + i * 5;=0A= + b[i] =3D BASE + OFFSET + i * 4;=0A= + asm volatile ("" ::: "memory");=0A= + }=0A= +=0A= + for (int i =3D 0; i < n; i++)=0A= + {=0A= + asm volatile ("" ::: "memory");=0A= + expected +=3D a[t + 0] * b[t + 0];=0A= + expected +=3D a[t + 1] * b[t + 1];=0A= + expected +=3D a[t + 2] * b[t + 2];=0A= + expected +=3D a[t + 3] * b[t + 3];=0A= + expected +=3D a[t + 4] * b[t + 4];=0A= + expected +=3D a[t + 5] * b[t + 5];=0A= + expected +=3D a[t + 6] * b[t + 6];=0A= + expected +=3D a[t + 7] * b[t + 7];=0A= + expected +=3D a[t + 8] * b[t + 8];=0A= + expected +=3D a[t + 9] * b[t + 9];=0A= + expected +=3D a[t + 10] * b[t + 10];=0A= + expected +=3D a[t + 11] * b[t + 11];=0A= + expected +=3D a[t + 12] * b[t + 12];=0A= + expected +=3D a[t + 13] * b[t + 13];=0A= + expected +=3D a[t + 14] * b[t + 14];=0A= + expected +=3D a[t + 15] * b[t + 15];=0A= + t +=3D step;=0A= + }=0A= +=0A= + if (f (0x12345, a, b, step, n) !=3D expected)=0A= + __builtin_abort ();=0A= +}=0A= +=0A= +/* { dg-final { scan-tree-dump "vect_recog_dot_prod_pattern: detected" "ve= ct" } } */=0A= +/* { dg-final { scan-tree-dump "vectorizing stmts using SLP" "vect" } } */= =0A= +/* { dg-final { scan-tree-dump-times "vectorizing statement: \\S+ =3D DOT_= PROD_EXPR" 16 "vect" } } */=0A= diff --git a/gcc/testsuite/gcc.dg/vect/vect-reduc-dot-slp-2.c b/gcc/testsui= te/gcc.dg/vect/vect-reduc-dot-slp-2.c=0A= new file mode 100644=0A= index 00000000000..1532833c3ae=0A= --- /dev/null=0A= +++ b/gcc/testsuite/gcc.dg/vect/vect-reduc-dot-slp-2.c=0A= @@ -0,0 +1,81 @@=0A= +/* Disabling epilogues until we find a better way to deal with scans. */= =0A= +/* { dg-additional-options "--param vect-epilogues-nomask=3D0" } */=0A= +/* { dg-require-effective-target vect_int } */=0A= +/* { dg-require-effective-target arm_v8_2a_dotprod_neon_hw { target { aarc= h64*-*-* || arm*-*-* } } } */=0A= +/* { dg-add-options arm_v8_2a_dotprod_neon } */=0A= +=0A= +#include "tree-vect.h"=0A= +=0A= +#define N 50=0A= +=0A= +#ifndef SIGNEDNESS_1=0A= +#define SIGNEDNESS_1 signed=0A= +#define SIGNEDNESS_2 signed=0A= +#endif=0A= +=0A= +SIGNEDNESS_1 int __attribute__ ((noipa))=0A= +f (SIGNEDNESS_1 int res,=0A= + SIGNEDNESS_2 short *a,=0A= + SIGNEDNESS_2 short *b,=0A= + int step, int n)=0A= +{=0A= + for (int i =3D 0; i < n; i++)=0A= + {=0A= + res +=3D a[0] * b[0];=0A= + res +=3D a[1] * b[1];=0A= + res +=3D a[2] * b[2];=0A= + res +=3D a[3] * b[3];=0A= + res +=3D a[4] * b[4];=0A= + res +=3D a[5] * b[5];=0A= + res +=3D a[6] * b[6];=0A= + res +=3D a[7] * b[7];=0A= +=0A= + a +=3D step;=0A= + b +=3D step;=0A= + }=0A= +=0A= + return res;=0A= +}=0A= +=0A= +#define BASE ((SIGNEDNESS_2 int) -1 < 0 ? -1026 : 373)=0A= +#define OFFSET 20=0A= +=0A= +int=0A= +main (void)=0A= +{=0A= + check_vect ();=0A= +=0A= + SIGNEDNESS_2 short a[100], b[100];=0A= + int expected =3D 0x12345;=0A= + int step =3D 8;=0A= + int n =3D 2;=0A= + int t =3D 0;=0A= +=0A= + for (int i =3D 0; i < sizeof (a) / sizeof (a[0]); ++i)=0A= + {=0A= + a[i] =3D BASE + i * 5;=0A= + b[i] =3D BASE + OFFSET + i * 4;=0A= + asm volatile ("" ::: "memory");=0A= + }=0A= +=0A= + for (int i =3D 0; i < n; i++)=0A= + {=0A= + asm volatile ("" ::: "memory");=0A= + expected +=3D a[t + 0] * b[t + 0];=0A= + expected +=3D a[t + 1] * b[t + 1];=0A= + expected +=3D a[t + 2] * b[t + 2];=0A= + expected +=3D a[t + 3] * b[t + 3];=0A= + expected +=3D a[t + 4] * b[t + 4];=0A= + expected +=3D a[t + 5] * b[t + 5];=0A= + expected +=3D a[t + 6] * b[t + 6];=0A= + expected +=3D a[t + 7] * b[t + 7];=0A= + t +=3D step;=0A= + }=0A= +=0A= + if (f (0x12345, a, b, step, n) !=3D expected)=0A= + __builtin_abort ();=0A= +}=0A= +=0A= +/* { dg-final { scan-tree-dump "vect_recog_dot_prod_pattern: detected" "ve= ct" } } */=0A= +/* { dg-final { scan-tree-dump "vectorizing stmts using SLP" "vect" } } */= =0A= +/* { dg-final { scan-tree-dump-times "vectorizing statement: \\S+ =3D DOT_= PROD_EXPR" 8 "vect" { target vect_sdot_hi } } } */=0A= diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc=0A= index 83c0544b6aa..92d07df2890 100644=0A= --- a/gcc/tree-vect-loop.cc=0A= +++ b/gcc/tree-vect-loop.cc=0A= @@ -5270,8 +5270,7 @@ have_whole_vector_shift (machine_mode mode)=0A= See vect_emulate_mixed_dot_prod for the actual sequence used. */=0A= =0A= static bool=0A= -vect_is_emulated_mixed_dot_prod (loop_vec_info loop_vinfo,=0A= - stmt_vec_info stmt_info)=0A= +vect_is_emulated_mixed_dot_prod (stmt_vec_info stmt_info)=0A= {=0A= gassign *assign =3D dyn_cast (stmt_info->stmt);=0A= if (!assign || gimple_assign_rhs_code (assign) !=3D DOT_PROD_EXPR)=0A= @@ -5282,10 +5281,9 @@ vect_is_emulated_mixed_dot_prod (loop_vec_info loop_= vinfo,=0A= if (TYPE_SIGN (TREE_TYPE (rhs1)) =3D=3D TYPE_SIGN (TREE_TYPE (rhs2)))=0A= return false;=0A= =0A= - stmt_vec_info reduc_info =3D info_for_reduction (loop_vinfo, stmt_info);= =0A= - gcc_assert (reduc_info->is_reduc_info);=0A= + gcc_assert (STMT_VINFO_REDUC_VECTYPE_IN (stmt_info));=0A= return !directly_supported_p (DOT_PROD_EXPR,=0A= - STMT_VINFO_REDUC_VECTYPE_IN (reduc_info),=0A= + STMT_VINFO_REDUC_VECTYPE_IN (stmt_info),=0A= optab_vector_mixed_sign);=0A= }=0A= =0A= @@ -5324,8 +5322,6 @@ vect_model_reduction_cost (loop_vec_info loop_vinfo,= =0A= if (!gimple_extract_op (orig_stmt_info->stmt, &op))=0A= gcc_unreachable ();=0A= =0A= - bool emulated_mixed_dot_prod=0A= - =3D vect_is_emulated_mixed_dot_prod (loop_vinfo, stmt_info);=0A= if (reduction_type =3D=3D EXTRACT_LAST_REDUCTION)=0A= /* No extra instructions are needed in the prologue. The loop body=0A= operations are costed in vectorizable_condition. */=0A= @@ -5360,12 +5356,8 @@ vect_model_reduction_cost (loop_vec_info loop_vinfo,= =0A= initial result of the data reduction, initial value of the index=0A= reduction. */=0A= prologue_stmts =3D 4;=0A= - else if (emulated_mixed_dot_prod)=0A= - /* We need the initial reduction value and two invariants:=0A= - one that contains the minimum signed value and one that=0A= - contains half of its negative. */=0A= - prologue_stmts =3D 3;=0A= else=0A= + /* We need the initial reduction value. */=0A= prologue_stmts =3D 1;=0A= prologue_cost +=3D record_stmt_cost (cost_vec, prologue_stmts,=0A= scalar_to_vec, stmt_info, 0,=0A= @@ -7384,6 +7376,244 @@ build_vect_cond_expr (code_helper code, tree vop[3]= , tree mask,=0A= }=0A= }=0A= =0A= +/* Given an operation with CODE in loop reduction path whose reduction PHI= is=0A= + specified by REDUC_INFO, the operation has TYPE of scalar result, and i= ts=0A= + input vectype is represented by VECTYPE_IN. The vectype of vectorized r= esult=0A= + may be different from VECTYPE_IN, either in base type or vectype lanes,= =0A= + lane-reducing operation is the case. This function check if it is poss= ible,=0A= + and how to perform partial vectorization on the operation in the contex= t=0A= + of LOOP_VINFO. */=0A= +=0A= +static void=0A= +vect_reduction_use_partial_vector (loop_vec_info loop_vinfo,=0A= + stmt_vec_info reduc_info,=0A= + slp_tree slp_node, code_helper code,=0A= + tree type, tree vectype_in)=0A= +{=0A= + if (!LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo))=0A= + return;=0A= +=0A= + enum vect_reduction_type reduc_type =3D STMT_VINFO_REDUC_TYPE (reduc_inf= o);=0A= + internal_fn reduc_fn =3D STMT_VINFO_REDUC_FN (reduc_info);=0A= + internal_fn cond_fn =3D get_conditional_internal_fn (code, type);=0A= +=0A= + if (reduc_type !=3D FOLD_LEFT_REDUCTION=0A= + && !use_mask_by_cond_expr_p (code, cond_fn, vectype_in)=0A= + && (cond_fn =3D=3D IFN_LAST=0A= + || !direct_internal_fn_supported_p (cond_fn, vectype_in,=0A= + OPTIMIZE_FOR_SPEED)))=0A= + {=0A= + if (dump_enabled_p ())=0A= + dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location,=0A= + "can't operate on partial vectors because"=0A= + " no conditional operation is available.\n");=0A= + LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) =3D false;=0A= + }=0A= + else if (reduc_type =3D=3D FOLD_LEFT_REDUCTION=0A= + && reduc_fn =3D=3D IFN_LAST=0A= + && !expand_vec_cond_expr_p (vectype_in, truth_type_for (vectype_in),= =0A= + SSA_NAME))=0A= + {=0A= + if (dump_enabled_p ())=0A= + dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location,=0A= + "can't operate on partial vectors because"=0A= + " no conditional operation is available.\n");=0A= + LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) =3D false;=0A= + }=0A= + else if (reduc_type =3D=3D FOLD_LEFT_REDUCTION=0A= + && internal_fn_mask_index (reduc_fn) =3D=3D -1=0A= + && FLOAT_TYPE_P (vectype_in)=0A= + && HONOR_SIGN_DEPENDENT_ROUNDING (vectype_in))=0A= + {=0A= + if (dump_enabled_p ())=0A= + dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location,=0A= + "can't operate on partial vectors because"=0A= + " signed zeros cannot be preserved.\n");=0A= + LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) =3D false;=0A= + }=0A= + else=0A= + {=0A= + internal_fn mask_reduc_fn=0A= + =3D get_masked_reduction_fn (reduc_fn, vectype_in);=0A= + vec_loop_masks *masks =3D &LOOP_VINFO_MASKS (loop_vinfo);=0A= + vec_loop_lens *lens =3D &LOOP_VINFO_LENS (loop_vinfo);=0A= + unsigned nvectors;=0A= +=0A= + if (slp_node)=0A= + nvectors =3D SLP_TREE_NUMBER_OF_VEC_STMTS (slp_node);=0A= + else=0A= + nvectors =3D vect_get_num_copies (loop_vinfo, vectype_in);=0A= +=0A= + if (mask_reduc_fn =3D=3D IFN_MASK_LEN_FOLD_LEFT_PLUS)=0A= + vect_record_loop_len (loop_vinfo, lens, nvectors, vectype_in, 1);=0A= + else=0A= + vect_record_loop_mask (loop_vinfo, masks, nvectors, vectype_in, NULL);=0A= + }=0A= +}=0A= +=0A= +/* Check if STMT_INFO is a lane-reducing operation that can be vectorized = in=0A= + the context of LOOP_VINFO, and vector cost will be recorded in COST_VEC= .=0A= + Now there are three such kinds of operations: dot-prod/widen-sum/sad=0A= + (sum-of-absolute-differences).=0A= +=0A= + For a lane-reducing operation, the loop reduction path that it lies in,= =0A= + may contain normal operation, or other lane-reducing operation of diffe= rent=0A= + input type size, an example as:=0A= +=0A= + int sum =3D 0;=0A= + for (i)=0A= + {=0A= + ...=0A= + sum +=3D d0[i] * d1[i]; // dot-prod =0A= + sum +=3D w[i]; // widen-sum =0A= + sum +=3D abs(s0[i] - s1[i]); // sad =0A= + sum +=3D n[i]; // normal =0A= + ...=0A= + }=0A= +=0A= + Vectorization factor is essentially determined by operation whose input= =0A= + vectype has the most lanes ("vector(16) char" in the example), while we= =0A= + need to choose input vectype with the least lanes ("vector(4) int" in t= he=0A= + example) for the reduction PHI statement. */=0A= +=0A= +bool=0A= +vectorizable_lane_reducing (loop_vec_info loop_vinfo, stmt_vec_info stmt_i= nfo,=0A= + slp_tree slp_node, stmt_vector_for_cost *cost_vec)=0A= +{=0A= + gassign *stmt =3D dyn_cast (stmt_info->stmt);=0A= + if (!stmt)=0A= + return false;=0A= +=0A= + enum tree_code code =3D gimple_assign_rhs_code (stmt);=0A= +=0A= + if (!lane_reducing_op_p (code))=0A= + return false;=0A= +=0A= + tree type =3D TREE_TYPE (gimple_assign_lhs (stmt));=0A= +=0A= + if (!INTEGRAL_TYPE_P (type) && !SCALAR_FLOAT_TYPE_P (type))=0A= + return false;=0A= +=0A= + /* Do not try to vectorize bit-precision reductions. */=0A= + if (!type_has_mode_precision_p (type))=0A= + return false;=0A= +=0A= + tree vectype_in =3D NULL_TREE;=0A= +=0A= + for (int i =3D 0; i < (int) gimple_num_ops (stmt) - 1; i++)=0A= + {=0A= + stmt_vec_info def_stmt_info;=0A= + slp_tree slp_op;=0A= + tree op;=0A= + tree vectype;=0A= + enum vect_def_type dt;=0A= +=0A= + if (!vect_is_simple_use (loop_vinfo, stmt_info, slp_node, i, &op,=0A= + &slp_op, &dt, &vectype, &def_stmt_info))=0A= + {=0A= + if (dump_enabled_p ())=0A= + dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location,=0A= + "use not simple.\n");=0A= + return false;=0A= + }=0A= +=0A= + if (!vectype)=0A= + {=0A= + vectype =3D get_vectype_for_scalar_type (loop_vinfo, TREE_TYPE (op),=0A= + slp_op);=0A= + if (!vectype)=0A= + return false;=0A= + }=0A= +=0A= + if (slp_node && !vect_maybe_update_slp_op_vectype (slp_op, vectype))= =0A= + {=0A= + if (dump_enabled_p ())=0A= + dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location,=0A= + "incompatible vector types for invariants\n");=0A= + return false;=0A= + }=0A= +=0A= + if (i =3D=3D STMT_VINFO_REDUC_IDX (stmt_info))=0A= + continue;=0A= +=0A= + /* There should be at most one cycle def in the stmt. */=0A= + if (VECTORIZABLE_CYCLE_DEF (dt))=0A= + return false;=0A= +=0A= + /* To properly compute ncopies we are interested in the widest=0A= + non-reduction input type in case we're looking at a widening=0A= + accumulation that we later handle in vect transformation. */=0A= + if (!vectype_in=0A= + || (GET_MODE_SIZE (SCALAR_TYPE_MODE (TREE_TYPE (vectype_in)))=0A= + < GET_MODE_SIZE (SCALAR_TYPE_MODE (TREE_TYPE (vectype)))))=0A= + vectype_in =3D vectype;=0A= + }=0A= +=0A= + STMT_VINFO_REDUC_VECTYPE_IN (stmt_info) =3D vectype_in;=0A= +=0A= + stmt_vec_info reduc_info =3D STMT_VINFO_REDUC_DEF (vect_orig_stmt (stmt_= info));=0A= +=0A= + /* TODO: Support lane-reducing operation that does not directly particip= ate=0A= + in loop reduction. */=0A= + if (!reduc_info || STMT_VINFO_REDUC_IDX (stmt_info) < 0)=0A= + return false;=0A= +=0A= + /* Lane-reducing pattern inside any inner loop of LOOP_VINFO is not=0A= + recoginized. */=0A= + gcc_assert (STMT_VINFO_DEF_TYPE (reduc_info) =3D=3D vect_reduction_def);= =0A= + gcc_assert (STMT_VINFO_REDUC_TYPE (reduc_info) =3D=3D TREE_CODE_REDUCTIO= N);=0A= +=0A= + tree vphi_vectype_in =3D STMT_VINFO_REDUC_VECTYPE_IN (reduc_info);=0A= +=0A= + /* To accommodate lane-reducing operations of mixed input vectypes, choo= se=0A= + input vectype with the least lanes for the reduction PHI statement, w= hich=0A= + would result in the most ncopies for vectorized reduction results. *= /=0A= + if (!vphi_vectype_in=0A= + || (GET_MODE_SIZE (SCALAR_TYPE_MODE (TREE_TYPE (vectype_in)))=0A= + > GET_MODE_SIZE (SCALAR_TYPE_MODE (TREE_TYPE (vphi_vectype_in)))))=0A= + STMT_VINFO_REDUC_VECTYPE_IN (reduc_info) =3D vectype_in;=0A= +=0A= + int ncopies_for_cost;=0A= +=0A= + if (slp_node)=0A= + {=0A= + /* Now lane-reducing operations in a slp node should only come from= =0A= + the same loop reduction path. */=0A= + gcc_assert (REDUC_GROUP_FIRST_ELEMENT (stmt_info));=0A= + ncopies_for_cost =3D 1;=0A= + }=0A= + else=0A= + {=0A= + ncopies_for_cost =3D vect_get_num_copies (loop_vinfo, vectype_in);= =0A= + gcc_assert (ncopies_for_cost >=3D 1);=0A= + }=0A= +=0A= + if (vect_is_emulated_mixed_dot_prod (stmt_info))=0A= + {=0A= + /* We need extra two invariants: one that contains the minimum signe= d=0A= + value and one that contains half of its negative. */=0A= + int prologue_stmts =3D 2;=0A= + unsigned cost =3D record_stmt_cost (cost_vec, prologue_stmts,=0A= + scalar_to_vec, stmt_info, 0,=0A= + vect_prologue);=0A= + if (dump_enabled_p ())=0A= + dump_printf (MSG_NOTE, "vectorizable_lane_reducing: "=0A= + "extra prologue_cost =3D %d .\n", cost);=0A= +=0A= + /* Three dot-products and a subtraction. */=0A= + ncopies_for_cost *=3D 4;=0A= + }=0A= +=0A= + record_stmt_cost (cost_vec, ncopies_for_cost, vector_stmt, stmt_info, 0,= =0A= + vect_body);=0A= +=0A= + vect_reduction_use_partial_vector (loop_vinfo, reduc_info, slp_node, cod= e,=0A= + type, vectype_in);=0A= +=0A= + STMT_VINFO_TYPE (stmt_info) =3D reduc_vec_info_type;=0A= + return true;=0A= +}=0A= +=0A= /* Function vectorizable_reduction.=0A= =0A= Check if STMT_INFO performs a reduction operation that can be vectorize= d.=0A= @@ -7449,7 +7679,6 @@ vectorizable_reduction (loop_vec_info loop_vinfo,=0A= bool single_defuse_cycle =3D false;=0A= bool nested_cycle =3D false;=0A= bool double_reduc =3D false;=0A= - int vec_num;=0A= tree cr_index_scalar_type =3D NULL_TREE, cr_index_vector_type =3D NULL_T= REE;=0A= tree cond_reduc_val =3D NULL_TREE;=0A= =0A= @@ -7530,6 +7759,7 @@ vectorizable_reduction (loop_vec_info loop_vinfo,=0A= (gimple_bb (reduc_def_phi)->loop_father));=0A= unsigned reduc_chain_length =3D 0;=0A= bool only_slp_reduc_chain =3D true;=0A= + bool only_lane_reducing =3D true;=0A= stmt_info =3D NULL;=0A= slp_tree slp_for_stmt_info =3D slp_node ? slp_node_instance->root : NULL= ;=0A= while (reduc_def !=3D PHI_RESULT (reduc_def_phi))=0A= @@ -7551,14 +7781,15 @@ vectorizable_reduction (loop_vec_info loop_vinfo,= =0A= all lanes here - even though we only will vectorize from=0A= the SLP node with live lane zero the other live lanes also=0A= need to be identified as part of a reduction to be able=0A= - to skip code generation for them. */=0A= + to skip code generation for them. For lane-reducing operation=0A= + vectorizable analysis needs the reduction PHI information. */=0A= if (slp_for_stmt_info)=0A= {=0A= for (auto s : SLP_TREE_SCALAR_STMTS (slp_for_stmt_info))=0A= if (STMT_VINFO_LIVE_P (s))=0A= STMT_VINFO_REDUC_DEF (vect_orig_stmt (s)) =3D phi_info;=0A= }=0A= - else if (STMT_VINFO_LIVE_P (vdef))=0A= + else=0A= STMT_VINFO_REDUC_DEF (def) =3D phi_info;=0A= gimple_match_op op;=0A= if (!gimple_extract_op (vdef->stmt, &op))=0A= @@ -7579,9 +7810,16 @@ vectorizable_reduction (loop_vec_info loop_vinfo,=0A= return false;=0A= }=0A= }=0A= - else if (!stmt_info)=0A= - /* First non-conversion stmt. */=0A= - stmt_info =3D vdef;=0A= + else=0A= + {=0A= + /* First non-conversion stmt. */=0A= + if (!stmt_info)=0A= + stmt_info =3D vdef;=0A= +=0A= + if (!lane_reducing_op_p (op.code))=0A= + only_lane_reducing =3D false;=0A= + }=0A= +=0A= reduc_def =3D op.ops[STMT_VINFO_REDUC_IDX (vdef)];=0A= reduc_chain_length++;=0A= if (!stmt_info && slp_node)=0A= @@ -7643,9 +7881,7 @@ vectorizable_reduction (loop_vec_info loop_vinfo,=0A= gimple_match_op op;=0A= if (!gimple_extract_op (stmt_info->stmt, &op))=0A= gcc_unreachable ();=0A= - bool lane_reduc_code_p =3D (op.code =3D=3D DOT_PROD_EXPR=0A= - || op.code =3D=3D WIDEN_SUM_EXPR=0A= - || op.code =3D=3D SAD_EXPR);=0A= + bool lane_reducing =3D lane_reducing_op_p (op.code);=0A= =0A= if (!POINTER_TYPE_P (op.type) && !INTEGRAL_TYPE_P (op.type)=0A= && !SCALAR_FLOAT_TYPE_P (op.type))=0A= @@ -7655,23 +7891,11 @@ vectorizable_reduction (loop_vec_info loop_vinfo,= =0A= if (!type_has_mode_precision_p (op.type))=0A= return false;=0A= =0A= - /* For lane-reducing ops we're reducing the number of reduction PHIs=0A= - which means the only use of that may be in the lane-reducing operatio= n. */=0A= - if (lane_reduc_code_p=0A= - && reduc_chain_length !=3D 1=0A= - && !only_slp_reduc_chain)=0A= - {=0A= - if (dump_enabled_p ())=0A= - dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location,=0A= - "lane-reducing reduction with extra stmts.\n");=0A= - return false;=0A= - }=0A= -=0A= /* Lane-reducing ops also never can be used in a SLP reduction group=0A= since we'll mix lanes belonging to different reductions. But it's=0A= OK to use them in a reduction chain or when the reduction group=0A= has just one element. */=0A= - if (lane_reduc_code_p=0A= + if (lane_reducing=0A= && slp_node=0A= && !REDUC_GROUP_FIRST_ELEMENT (stmt_info)=0A= && SLP_TREE_LANES (slp_node) > 1)=0A= @@ -7710,9 +7934,6 @@ vectorizable_reduction (loop_vec_info loop_vinfo,=0A= "use not simple.\n");=0A= return false;=0A= }=0A= - if (i =3D=3D STMT_VINFO_REDUC_IDX (stmt_info))=0A= - continue;=0A= -=0A= /* For an IFN_COND_OP we might hit the reduction definition operand= =0A= twice (once as definition, once as else). */=0A= if (op.ops[i] =3D=3D op.ops[STMT_VINFO_REDUC_IDX (stmt_info)])=0A= @@ -7731,7 +7952,7 @@ vectorizable_reduction (loop_vec_info loop_vinfo,=0A= /* To properly compute ncopies we are interested in the widest=0A= non-reduction input type in case we're looking at a widening=0A= accumulation that we later handle in vect_transform_reduction. */=0A= - if (lane_reduc_code_p=0A= + if (lane_reducing=0A= && vectype_op[i]=0A= && (!vectype_in=0A= || (GET_MODE_SIZE (SCALAR_TYPE_MODE (TREE_TYPE (vectype_in)))=0A= @@ -7758,12 +7979,21 @@ vectorizable_reduction (loop_vec_info loop_vinfo,= =0A= }=0A= if (!vectype_in)=0A= vectype_in =3D STMT_VINFO_VECTYPE (phi_info);=0A= - STMT_VINFO_REDUC_VECTYPE_IN (reduc_info) =3D vectype_in;=0A= =0A= - enum vect_reduction_type v_reduc_type =3D STMT_VINFO_REDUC_TYPE (phi_inf= o);=0A= - STMT_VINFO_REDUC_TYPE (reduc_info) =3D v_reduc_type;=0A= + /* If there is a normal (non-lane-reducing) operation in the loop reduct= ion=0A= + path, to ensure there will be enough copies to hold vectorized result= s of=0A= + the operation, we need set the input vectype of the reduction PHI to = be=0A= + same as the reduction output vectype somewhere, here is a suitable pl= ace.=0A= + Otherwise the input vectype is set to the one with the least lanes, w= hich=0A= + can only be determined in vectorizable analysis routine of lane-reduc= ing=0A= + operation. */=0A= + if (!only_lane_reducing)=0A= + STMT_VINFO_REDUC_VECTYPE_IN (reduc_info) =3D STMT_VINFO_VECTYPE (phi_i= nfo);=0A= +=0A= + enum vect_reduction_type reduction_type =3D STMT_VINFO_REDUC_TYPE (phi_i= nfo);=0A= + STMT_VINFO_REDUC_TYPE (reduc_info) =3D reduction_type;=0A= /* If we have a condition reduction, see if we can simplify it further. = */=0A= - if (v_reduc_type =3D=3D COND_REDUCTION)=0A= + if (reduction_type =3D=3D COND_REDUCTION)=0A= {=0A= if (slp_node)=0A= return false;=0A= @@ -7929,8 +8159,8 @@ vectorizable_reduction (loop_vec_info loop_vinfo,=0A= }=0A= =0A= STMT_VINFO_REDUC_CODE (reduc_info) =3D orig_code;=0A= + reduction_type =3D STMT_VINFO_REDUC_TYPE (reduc_info);=0A= =0A= - vect_reduction_type reduction_type =3D STMT_VINFO_REDUC_TYPE (reduc_info= );=0A= if (reduction_type =3D=3D TREE_CODE_REDUCTION)=0A= {=0A= /* Check whether it's ok to change the order of the computation.=0A= @@ -8204,14 +8434,11 @@ vectorizable_reduction (loop_vec_info loop_vinfo,= =0A= && loop_vinfo->suggested_unroll_factor =3D=3D 1)=0A= single_defuse_cycle =3D true;=0A= =0A= - if (single_defuse_cycle || lane_reduc_code_p)=0A= + if (single_defuse_cycle && !lane_reducing)=0A= {=0A= gcc_assert (op.code !=3D COND_EXPR);=0A= =0A= - /* 4. Supportable by target? */=0A= - bool ok =3D true;=0A= -=0A= - /* 4.1. check support for the operation in the loop=0A= + /* 4. check support for the operation in the loop=0A= =0A= This isn't necessary for the lane reduction codes, since they=0A= can only be produced by pattern matching, and it's up to the=0A= @@ -8220,14 +8447,13 @@ vectorizable_reduction (loop_vec_info loop_vinfo,= =0A= mixed-sign dot-products can be implemented using signed=0A= dot-products. */=0A= machine_mode vec_mode =3D TYPE_MODE (vectype_in);=0A= - if (!lane_reduc_code_p=0A= - && !directly_supported_p (op.code, vectype_in, optab_vector))=0A= + if (!directly_supported_p (op.code, vectype_in, optab_vector))=0A= {=0A= if (dump_enabled_p ())=0A= dump_printf (MSG_NOTE, "op not supported by target.\n");=0A= if (maybe_ne (GET_MODE_SIZE (vec_mode), UNITS_PER_WORD)=0A= || !vect_can_vectorize_without_simd_p (op.code))=0A= - ok =3D false;=0A= + single_defuse_cycle =3D false;=0A= else=0A= if (dump_enabled_p ())=0A= dump_printf (MSG_NOTE, "proceeding using word mode.\n");=0A= @@ -8240,35 +8466,12 @@ vectorizable_reduction (loop_vec_info loop_vinfo,= =0A= dump_printf (MSG_NOTE, "using word mode not possible.\n");=0A= return false;=0A= }=0A= -=0A= - /* lane-reducing operations have to go through vect_transform_reduct= ion.=0A= - For the other cases try without the single cycle optimization. *= /=0A= - if (!ok)=0A= - {=0A= - if (lane_reduc_code_p)=0A= - return false;=0A= - else=0A= - single_defuse_cycle =3D false;=0A= - }=0A= }=0A= STMT_VINFO_FORCE_SINGLE_CYCLE (reduc_info) =3D single_defuse_cycle;=0A= =0A= - /* If the reduction stmt is one of the patterns that have lane=0A= - reduction embedded we cannot handle the case of ! single_defuse_cycle= . */=0A= - if ((ncopies > 1 && ! single_defuse_cycle)=0A= - && lane_reduc_code_p)=0A= - {=0A= - if (dump_enabled_p ())=0A= - dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location,=0A= - "multi def-use cycle not possible for lane-reducing "=0A= - "reduction operation\n");=0A= - return false;=0A= - }=0A= -=0A= - if (slp_node=0A= - && !(!single_defuse_cycle=0A= - && !lane_reduc_code_p=0A= - && reduction_type !=3D FOLD_LEFT_REDUCTION))=0A= + /* Reduction type of lane-reducing operation is TREE_CODE_REDUCTION, the= =0A= + below processing will be done in its own vectorizable function. */= =0A= + if (slp_node && reduction_type =3D=3D FOLD_LEFT_REDUCTION)=0A= for (i =3D 0; i < (int) op.num_ops; i++)=0A= if (!vect_maybe_update_slp_op_vectype (slp_op[i], vectype_op[i]))=0A= {=0A= @@ -8278,36 +8481,24 @@ vectorizable_reduction (loop_vec_info loop_vinfo,= =0A= return false;=0A= }=0A= =0A= - if (slp_node)=0A= - vec_num =3D SLP_TREE_NUMBER_OF_VEC_STMTS (slp_node);=0A= - else=0A= - vec_num =3D 1;=0A= -=0A= vect_model_reduction_cost (loop_vinfo, stmt_info, reduc_fn,=0A= reduction_type, ncopies, cost_vec);=0A= /* Cost the reduction op inside the loop if transformed via=0A= - vect_transform_reduction. Otherwise this is costed by the=0A= - separate vectorizable_* routines. */=0A= - if (single_defuse_cycle || lane_reduc_code_p)=0A= - {=0A= - int factor =3D 1;=0A= - if (vect_is_emulated_mixed_dot_prod (loop_vinfo, stmt_info))=0A= - /* Three dot-products and a subtraction. */=0A= - factor =3D 4;=0A= - record_stmt_cost (cost_vec, ncopies * factor, vector_stmt,=0A= - stmt_info, 0, vect_body);=0A= - }=0A= + vect_transform_reduction for non-lane-reducing operation. Otherwise= =0A= + this is costed by the separate vectorizable_* routines. */=0A= + if (single_defuse_cycle && !lane_reducing)=0A= + record_stmt_cost (cost_vec, ncopies, vector_stmt, stmt_info, 0, vect_b= ody);=0A= =0A= if (dump_enabled_p ()=0A= && reduction_type =3D=3D FOLD_LEFT_REDUCTION)=0A= dump_printf_loc (MSG_NOTE, vect_location,=0A= "using an in-order (fold-left) reduction.\n");=0A= STMT_VINFO_TYPE (orig_stmt_of_analysis) =3D cycle_phi_info_type;=0A= - /* All but single defuse-cycle optimized, lane-reducing and fold-left=0A= - reductions go through their own vectorizable_* routines. */=0A= - if (!single_defuse_cycle=0A= - && !lane_reduc_code_p=0A= - && reduction_type !=3D FOLD_LEFT_REDUCTION)=0A= +=0A= + /* All but single defuse-cycle optimized and fold-left reductions go=0A= + through their own vectorizable_* routines. */=0A= + if ((!single_defuse_cycle && reduction_type !=3D FOLD_LEFT_REDUCTION)=0A= + || lane_reducing)=0A= {=0A= stmt_vec_info tem=0A= =3D vect_stmt_to_vectorize (STMT_VINFO_REDUC_DEF (phi_info));=0A= @@ -8319,60 +8510,10 @@ vectorizable_reduction (loop_vec_info loop_vinfo,= =0A= STMT_VINFO_DEF_TYPE (vect_orig_stmt (tem)) =3D vect_internal_def;=0A= STMT_VINFO_DEF_TYPE (tem) =3D vect_internal_def;=0A= }=0A= - else if (loop_vinfo && LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo)= )=0A= - {=0A= - vec_loop_masks *masks =3D &LOOP_VINFO_MASKS (loop_vinfo);=0A= - vec_loop_lens *lens =3D &LOOP_VINFO_LENS (loop_vinfo);=0A= - internal_fn cond_fn =3D get_conditional_internal_fn (op.code, op.typ= e);=0A= -=0A= - if (reduction_type !=3D FOLD_LEFT_REDUCTION=0A= - && !use_mask_by_cond_expr_p (op.code, cond_fn, vectype_in)=0A= - && (cond_fn =3D=3D IFN_LAST=0A= - || !direct_internal_fn_supported_p (cond_fn, vectype_in,=0A= - OPTIMIZE_FOR_SPEED)))=0A= - {=0A= - if (dump_enabled_p ())=0A= - dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location,=0A= - "can't operate on partial vectors because"=0A= - " no conditional operation is available.\n");=0A= - LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) =3D false;=0A= - }=0A= - else if (reduction_type =3D=3D FOLD_LEFT_REDUCTION=0A= - && reduc_fn =3D=3D IFN_LAST=0A= - && !expand_vec_cond_expr_p (vectype_in,=0A= - truth_type_for (vectype_in),=0A= - SSA_NAME))=0A= - {=0A= - if (dump_enabled_p ())=0A= - dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location,=0A= - "can't operate on partial vectors because"=0A= - " no conditional operation is available.\n");=0A= - LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) =3D false;=0A= - }=0A= - else if (reduction_type =3D=3D FOLD_LEFT_REDUCTION=0A= - && internal_fn_mask_index (reduc_fn) =3D=3D -1=0A= - && FLOAT_TYPE_P (vectype_in)=0A= - && HONOR_SIGN_DEPENDENT_ROUNDING (vectype_in))=0A= - {=0A= - if (dump_enabled_p ())=0A= - dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location,=0A= - "can't operate on partial vectors because"=0A= - " signed zeros cannot be preserved.\n");=0A= - LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) =3D false;=0A= - }=0A= - else=0A= - {=0A= - internal_fn mask_reduc_fn=0A= - =3D get_masked_reduction_fn (reduc_fn, vectype_in);=0A= + else=0A= + vect_reduction_use_partial_vector (loop_vinfo, reduc_info, slp_node,= =0A= + op.code, op.type, vectype_in);=0A= =0A= - if (mask_reduc_fn =3D=3D IFN_MASK_LEN_FOLD_LEFT_PLUS)=0A= - vect_record_loop_len (loop_vinfo, lens, ncopies * vec_num,=0A= - vectype_in, 1);=0A= - else=0A= - vect_record_loop_mask (loop_vinfo, masks, ncopies * vec_num,=0A= - vectype_in, NULL);=0A= - }=0A= - }=0A= return true;=0A= }=0A= =0A= @@ -8463,6 +8604,7 @@ vect_transform_reduction (loop_vec_info loop_vinfo,= =0A= class loop *loop =3D LOOP_VINFO_LOOP (loop_vinfo);=0A= int i;=0A= int ncopies;=0A= + int stmt_ncopies;=0A= int vec_num;=0A= =0A= stmt_vec_info reduc_info =3D info_for_reduction (loop_vinfo, stmt_info);= =0A= @@ -8486,15 +8628,28 @@ vect_transform_reduction (loop_vec_info loop_vinfo,= =0A= gphi *reduc_def_phi =3D as_a (phi_info->stmt);=0A= int reduc_index =3D STMT_VINFO_REDUC_IDX (stmt_info);=0A= tree vectype_in =3D STMT_VINFO_REDUC_VECTYPE_IN (reduc_info);=0A= + tree stmt_vectype_in =3D STMT_VINFO_REDUC_VECTYPE_IN (stmt_info);=0A= +=0A= + /* Get input vectypes from the reduction PHI and the statement to be=0A= + transformed, these two vectypes may have different lanes when=0A= + lane-reducing operation is present. */=0A= + if (!vectype_in)=0A= + vectype_in =3D STMT_VINFO_REDUC_VECTYPE (reduc_info);=0A= +=0A= + if (!stmt_vectype_in)=0A= + stmt_vectype_in =3D STMT_VINFO_VECTYPE (stmt_info);=0A= =0A= if (slp_node)=0A= {=0A= ncopies =3D 1;=0A= + stmt_ncopies =3D 1;=0A= vec_num =3D SLP_TREE_NUMBER_OF_VEC_STMTS (slp_node);=0A= }=0A= else=0A= {=0A= ncopies =3D vect_get_num_copies (loop_vinfo, vectype_in);=0A= + stmt_ncopies =3D vect_get_num_copies (loop_vinfo, stmt_vectype_in);= =0A= + gcc_assert (stmt_ncopies >=3D 1 && stmt_ncopies <=3D ncopies);=0A= vec_num =3D 1;=0A= }=0A= =0A= @@ -8503,14 +8658,10 @@ vect_transform_reduction (loop_vec_info loop_vinfo,= =0A= =0A= vec_loop_masks *masks =3D &LOOP_VINFO_MASKS (loop_vinfo);=0A= vec_loop_lens *lens =3D &LOOP_VINFO_LENS (loop_vinfo);=0A= - bool mask_by_cond_expr =3D use_mask_by_cond_expr_p (code, cond_fn, vecty= pe_in);=0A= -=0A= + bool mask_by_cond_expr =3D use_mask_by_cond_expr_p (code, cond_fn,=0A= + stmt_vectype_in);=0A= /* Transform. */=0A= - tree new_temp =3D NULL_TREE;=0A= - auto_vec vec_oprnds0;=0A= - auto_vec vec_oprnds1;=0A= - auto_vec vec_oprnds2;=0A= - tree def0;=0A= + auto_vec vec_oprnds[3];=0A= =0A= if (dump_enabled_p ())=0A= dump_printf_loc (MSG_NOTE, vect_location, "transform reduction.\n");= =0A= @@ -8534,8 +8685,6 @@ vect_transform_reduction (loop_vec_info loop_vinfo,= =0A= =3D=3D op.ops[internal_fn_else_index ((internal_fn) code)]));=0A= }=0A= =0A= - bool masked_loop_p =3D LOOP_VINFO_FULLY_MASKED_P (loop_vinfo);=0A= -=0A= vect_reduction_type reduction_type =3D STMT_VINFO_REDUC_TYPE (reduc_info= );=0A= if (reduction_type =3D=3D FOLD_LEFT_REDUCTION)=0A= {=0A= @@ -8543,69 +8692,172 @@ vect_transform_reduction (loop_vec_info loop_vinfo= ,=0A= gcc_assert (code.is_tree_code () || cond_fn_p);=0A= return vectorize_fold_left_reduction=0A= (loop_vinfo, stmt_info, gsi, vec_stmt, slp_node, reduc_def_phi,=0A= - code, reduc_fn, op.ops, op.num_ops, vectype_in,=0A= + code, reduc_fn, op.ops, op.num_ops, stmt_vectype_in,=0A= reduc_index, masks, lens);=0A= }=0A= =0A= bool single_defuse_cycle =3D STMT_VINFO_FORCE_SINGLE_CYCLE (reduc_info);= =0A= - gcc_assert (single_defuse_cycle=0A= - || code =3D=3D DOT_PROD_EXPR=0A= - || code =3D=3D WIDEN_SUM_EXPR=0A= - || code =3D=3D SAD_EXPR);=0A= + bool lane_reducing =3D lane_reducing_op_p (code);=0A= +=0A= + gcc_assert (single_defuse_cycle || lane_reducing);=0A= =0A= /* Create the destination vector */=0A= tree scalar_dest =3D gimple_get_lhs (stmt_info->stmt);=0A= tree vec_dest =3D vect_create_destination_var (scalar_dest, vectype_out)= ;=0A= =0A= - /* Get NCOPIES vector definitions for all operands except the reduction= =0A= - definition. */=0A= - if (!cond_fn_p)=0A= + gcc_assert (reduc_index < 3);=0A= +=0A= + if (slp_node)=0A= {=0A= - vect_get_vec_defs (loop_vinfo, stmt_info, slp_node, ncopies,=0A= - single_defuse_cycle && reduc_index =3D=3D 0=0A= - ? NULL_TREE : op.ops[0], &vec_oprnds0,=0A= - single_defuse_cycle && reduc_index =3D=3D 1=0A= - ? NULL_TREE : op.ops[1], &vec_oprnds1,=0A= - op.num_ops =3D=3D 3=0A= - && !(single_defuse_cycle && reduc_index =3D=3D 2)=0A= - ? op.ops[2] : NULL_TREE, &vec_oprnds2);=0A= + gcc_assert (!single_defuse_cycle && op.num_ops <=3D 3);=0A= +=0A= + for (i =3D 0; i < (int) op.num_ops; i++)=0A= + vect_get_slp_defs (SLP_TREE_CHILDREN (slp_node)[i], &vec_oprnds[i]);=0A= }=0A= else=0A= {=0A= - /* For a conditional operation pass the truth type as mask=0A= - vectype. */=0A= - gcc_assert (single_defuse_cycle=0A= - && (reduc_index =3D=3D 1 || reduc_index =3D=3D 2));=0A= - vect_get_vec_defs (loop_vinfo, stmt_info, slp_node, ncopies,=0A= - op.ops[0], truth_type_for (vectype_in), &vec_oprnds0,=0A= - reduc_index =3D=3D 1 ? NULL_TREE : op.ops[1],=0A= - NULL_TREE, &vec_oprnds1,=0A= - reduc_index =3D=3D 2 ? NULL_TREE : op.ops[2],=0A= - NULL_TREE, &vec_oprnds2);=0A= - }=0A= + int result_pos =3D 0;=0A= =0A= - /* For single def-use cycles get one copy of the vectorized reduction=0A= - definition. */=0A= - if (single_defuse_cycle)=0A= - {=0A= - gcc_assert (!slp_node);=0A= - vect_get_vec_defs_for_operand (loop_vinfo, stmt_info, 1,=0A= - op.ops[reduc_index],=0A= - reduc_index =3D=3D 0 ? &vec_oprnds0=0A= - : (reduc_index =3D=3D 1 ? &vec_oprnds1=0A= - : &vec_oprnds2));=0A= + /* The input vectype of the reduction PHI determines copies of=0A= + vectorized def-use cycles, which might be more than effective copies=0A= + of vectorized lane-reducing reduction statements. This could be=0A= + complemented by generating extra trivial pass-through copies. For=0A= + example:=0A= +=0A= + int sum =3D 0;=0A= + for (i)=0A= + {=0A= + sum +=3D d0[i] * d1[i]; // dot-prod =0A= + sum +=3D abs(s0[i] - s1[i]); // sad =0A= + sum +=3D n[i]; // normal =0A= + }=0A= +=0A= + The vector size is 128-bit,vectorization factor is 16. Reduction=0A= + statements would be transformed as:=0A= +=0A= + vector<4> int sum_v0 =3D { 0, 0, 0, 0 };=0A= + vector<4> int sum_v1 =3D { 0, 0, 0, 0 };=0A= + vector<4> int sum_v2 =3D { 0, 0, 0, 0 };=0A= + vector<4> int sum_v3 =3D { 0, 0, 0, 0 };=0A= +=0A= + for (i / 16)=0A= + {=0A= + sum_v0 =3D DOT_PROD (d0_v0[i: 0 ~ 15], d1_v0[i: 0 ~ 15], sum_v0);= =0A= + sum_v1 =3D sum_v1; // copy=0A= + sum_v2 =3D sum_v2; // copy=0A= + sum_v3 =3D sum_v3; // copy=0A= +=0A= + sum_v0 =3D sum_v0; // copy=0A= + sum_v1 =3D SAD (s0_v1[i: 0 ~ 7 ], s1_v1[i: 0 ~ 7 ], sum_v1);=0A= + sum_v2 =3D SAD (s0_v2[i: 8 ~ 15], s1_v2[i: 8 ~ 15], sum_v2);=0A= + sum_v3 =3D sum_v3; // copy=0A= +=0A= + sum_v0 +=3D n_v0[i: 0 ~ 3 ];=0A= + sum_v1 +=3D n_v1[i: 4 ~ 7 ];=0A= + sum_v2 +=3D n_v2[i: 8 ~ 11];=0A= + sum_v3 +=3D n_v3[i: 12 ~ 15];=0A= + }=0A= +=0A= + Moreover, for a higher instruction parallelism in final vectorized=0A= + loop, it is considered to make those effective vectorized=0A= + lane-reducing statements be distributed evenly among all def-use=0A= + cycles. In the above example, SADs are generated into other cycles=0A= + rather than that of DOT_PROD. */=0A= +=0A= + if (stmt_ncopies < ncopies)=0A= + {=0A= + gcc_assert (lane_reducing);=0A= + result_pos =3D reduc_info->reduc_result_pos;=0A= + reduc_info->reduc_result_pos =3D (result_pos + stmt_ncopies) % ncopies;= =0A= + gcc_assert (result_pos >=3D 0 && result_pos < ncopies);=0A= + }=0A= +=0A= + for (i =3D 0; i < MIN (3, (int) op.num_ops); i++)=0A= + {=0A= + tree vectype =3D NULL_TREE;=0A= + int used_ncopies =3D ncopies;=0A= +=0A= + if (cond_fn_p && i =3D=3D 0)=0A= + {=0A= + /* For a conditional operation pass the truth type as mask=0A= + vectype. */=0A= + gcc_assert (single_defuse_cycle && reduc_index > 0);=0A= + vectype =3D truth_type_for (vectype_in);=0A= + }=0A= +=0A= + if (i !=3D reduc_index)=0A= + {=0A= + /* For non-reduction operand, deduce effictive copies that are=0A= + involved in vectorized def-use cycles based on the input=0A= + vectype of the reduction statement. */=0A= + used_ncopies =3D stmt_ncopies;=0A= + }=0A= + else if (single_defuse_cycle)=0A= + {=0A= + /* For single def-use cycles get one copy of the vectorized=0A= + reduction definition. */=0A= + used_ncopies =3D 1;=0A= + }=0A= +=0A= + vect_get_vec_defs_for_operand (loop_vinfo, stmt_info, used_ncopies,=0A= + op.ops[i], &vec_oprnds[i], vectype);=0A= +=0A= + if (used_ncopies < ncopies)=0A= + {=0A= + vec_oprnds[i].safe_grow_cleared (ncopies);=0A= +=0A= + /* Find suitable def-use cycles to generate vectorized=0A= + statements into, and reorder operands based on the=0A= + selection. */=0A= + if (i !=3D reduc_index && result_pos)=0A= + {=0A= + int count =3D ncopies - used_ncopies;=0A= + int start =3D result_pos - count;=0A= +=0A= + if (start < 0)=0A= + {=0A= + count =3D result_pos;=0A= + start =3D 0;=0A= + }=0A= +=0A= + for (int j =3D used_ncopies - 1; j >=3D start; j--)=0A= + {=0A= + std::swap (vec_oprnds[i][j], vec_oprnds[i][j + count]);=0A= + gcc_assert (!vec_oprnds[i][j]);=0A= + }=0A= + }=0A= + }=0A= + }=0A= }=0A= =0A= - bool emulated_mixed_dot_prod=0A= - =3D vect_is_emulated_mixed_dot_prod (loop_vinfo, stmt_info);=0A= - FOR_EACH_VEC_ELT (vec_oprnds0, i, def0)=0A= + bool masked_loop_p =3D LOOP_VINFO_FULLY_MASKED_P (loop_vinfo);=0A= + bool emulated_mixed_dot_prod =3D vect_is_emulated_mixed_dot_prod (stmt_i= nfo);=0A= + tree def0;=0A= +=0A= + FOR_EACH_VEC_ELT (vec_oprnds[0], i, def0)=0A= {=0A= gimple *new_stmt;=0A= - tree vop[3] =3D { def0, vec_oprnds1[i], NULL_TREE };=0A= - if (masked_loop_p && !mask_by_cond_expr)=0A= + tree new_temp =3D NULL_TREE;=0A= + tree vop[3] =3D { def0, vec_oprnds[1][i], NULL_TREE };=0A= +=0A= + if (!vop[0] || !vop[1])=0A= {=0A= - /* No conditional ifns have been defined for dot-product yet. */=0A= - gcc_assert (code !=3D DOT_PROD_EXPR);=0A= + tree reduc_vop =3D vec_oprnds[reduc_index][i];=0A= +=0A= + /* Insert trivial copy if no need to generate vectorized=0A= + statement. */=0A= + gcc_assert (reduc_vop && stmt_ncopies < ncopies);=0A= +=0A= + new_stmt =3D gimple_build_assign (vec_dest, reduc_vop);=0A= + new_temp =3D make_ssa_name (vec_dest, new_stmt);=0A= + gimple_set_lhs (new_stmt, new_temp);=0A= + vect_finish_stmt_generation (loop_vinfo, stmt_info, new_stmt, gsi);=0A= + }=0A= + else if (masked_loop_p && !mask_by_cond_expr)=0A= + {=0A= + /* No conditional ifns have been defined for dot-product and sad=0A= + yet. */=0A= + gcc_assert (code !=3D DOT_PROD_EXPR && code !=3D SAD_EXPR);=0A= =0A= /* Make sure that the reduction accumulator is vop[0]. */=0A= if (reduc_index =3D=3D 1)=0A= @@ -8614,7 +8866,8 @@ vect_transform_reduction (loop_vec_info loop_vinfo,= =0A= std::swap (vop[0], vop[1]);=0A= }=0A= tree mask =3D vect_get_loop_mask (loop_vinfo, gsi, masks,=0A= - vec_num * ncopies, vectype_in, i);=0A= + vec_num * stmt_ncopies,=0A= + stmt_vectype_in, i);=0A= gcall *call =3D gimple_build_call_internal (cond_fn, 4, mask,=0A= vop[0], vop[1], vop[0]);=0A= new_temp =3D make_ssa_name (vec_dest, call);=0A= @@ -8626,12 +8879,13 @@ vect_transform_reduction (loop_vec_info loop_vinfo,= =0A= else=0A= {=0A= if (op.num_ops >=3D 3)=0A= - vop[2] =3D vec_oprnds2[i];=0A= + vop[2] =3D vec_oprnds[2][i];=0A= =0A= if (masked_loop_p && mask_by_cond_expr)=0A= {=0A= tree mask =3D vect_get_loop_mask (loop_vinfo, gsi, masks,=0A= - vec_num * ncopies, vectype_in, i);=0A= + vec_num * stmt_ncopies,=0A= + stmt_vectype_in, i);=0A= build_vect_cond_expr (code, vop, mask, gsi);=0A= }=0A= =0A= @@ -8658,16 +8912,8 @@ vect_transform_reduction (loop_vec_info loop_vinfo,= =0A= =0A= if (slp_node)=0A= slp_node->push_vec_def (new_stmt);=0A= - else if (single_defuse_cycle=0A= - && i < ncopies - 1)=0A= - {=0A= - if (reduc_index =3D=3D 0)=0A= - vec_oprnds0.safe_push (gimple_get_lhs (new_stmt));=0A= - else if (reduc_index =3D=3D 1)=0A= - vec_oprnds1.safe_push (gimple_get_lhs (new_stmt));=0A= - else if (reduc_index =3D=3D 2)=0A= - vec_oprnds2.safe_push (gimple_get_lhs (new_stmt));=0A= - }=0A= + else if (single_defuse_cycle && i < ncopies - 1)=0A= + vec_oprnds[reduc_index][i + 1] =3D gimple_get_lhs (new_stmt);=0A= else=0A= STMT_VINFO_VEC_STMTS (stmt_info).safe_push (new_stmt);=0A= }=0A= diff --git a/gcc/tree-vect-slp.cc b/gcc/tree-vect-slp.cc=0A= index c7ed520b629..5713e32f545 100644=0A= --- a/gcc/tree-vect-slp.cc=0A= +++ b/gcc/tree-vect-slp.cc=0A= @@ -3924,9 +3924,7 @@ vect_analyze_slp (vec_info *vinfo, unsigned max_tree_= size)=0A= /* Do not discover SLP reductions for lane-reducing ops, that=0A= will fail later. */=0A= && (!(g =3D dyn_cast (STMT_VINFO_STMT (next_info)))=0A= - || (gimple_assign_rhs_code (g) !=3D DOT_PROD_EXPR=0A= - && gimple_assign_rhs_code (g) !=3D WIDEN_SUM_EXPR=0A= - && gimple_assign_rhs_code (g) !=3D SAD_EXPR)))=0A= + || !lane_reducing_op_p (gimple_assign_rhs_code (g))))=0A= scalar_stmts.quick_push (next_info);=0A= }=0A= if (scalar_stmts.length () > 1)=0A= diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc=0A= index 4219ad832db..41ee3051756 100644=0A= --- a/gcc/tree-vect-stmts.cc=0A= +++ b/gcc/tree-vect-stmts.cc=0A= @@ -12096,11 +12096,20 @@ vectorizable_condition (vec_info *vinfo,=0A= vect_reduction_type reduction_type =3D TREE_CODE_REDUCTION;=0A= bool for_reduction=0A= =3D STMT_VINFO_REDUC_DEF (vect_orig_stmt (stmt_info)) !=3D NULL;=0A= + if (for_reduction)=0A= + {=0A= + reduc_info =3D info_for_reduction (vinfo, stmt_info);=0A= + if (STMT_VINFO_REDUC_DEF (reduc_info) !=3D vect_orig_stmt (stmt_info= ))=0A= + {=0A= + for_reduction =3D false;=0A= + reduc_info =3D NULL;=0A= + }=0A= + }=0A= +=0A= if (for_reduction)=0A= {=0A= if (slp_node)=0A= return false;=0A= - reduc_info =3D info_for_reduction (vinfo, stmt_info);=0A= reduction_type =3D STMT_VINFO_REDUC_TYPE (reduc_info);=0A= reduc_index =3D STMT_VINFO_REDUC_IDX (stmt_info);=0A= gcc_assert (reduction_type !=3D EXTRACT_LAST_REDUCTION=0A= @@ -13289,6 +13298,8 @@ vect_analyze_stmt (vec_info *vinfo,=0A= NULL, NULL, node, cost_vec)=0A= || vectorizable_load (vinfo, stmt_info, NULL, NULL, node, cost_vec)=0A= || vectorizable_store (vinfo, stmt_info, NULL, NULL, node, cost_vec)=0A= + || vectorizable_lane_reducing (as_a (vinfo),=0A= + stmt_info, node, cost_vec)=0A= || vectorizable_reduction (as_a (vinfo), stmt_info,=0A= node, node_instance, cost_vec)=0A= || vectorizable_induction (as_a (vinfo), stmt_info,=0A= diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h=0A= index 93bc30ef660..392fff5b799 100644=0A= --- a/gcc/tree-vectorizer.h=0A= +++ b/gcc/tree-vectorizer.h=0A= @@ -1399,6 +1399,12 @@ public:=0A= /* The vector type for performing the actual reduction. */=0A= tree reduc_vectype;=0A= =0A= + /* For loop reduction with multiple vectorized results (ncopies > 1), a= =0A= + lane-reducing operation participating in it may not use all of those= =0A= + results, this field specifies result index starting from which any=0A= + following land-reducing operation would be assigned to. */=0A= + int reduc_result_pos;=0A= +=0A= /* If IS_REDUC_INFO is true and if the vector code is performing=0A= N scalar reductions in parallel, this variable gives the initial=0A= scalar values of those N reductions. */=0A= @@ -2166,6 +2172,12 @@ vect_apply_runtime_profitability_check_p (loop_vec_i= nfo loop_vinfo)=0A= && th >=3D vect_vf_for_cost (loop_vinfo));=0A= }=0A= =0A= +inline bool=0A= +lane_reducing_op_p (code_helper code)=0A= +{=0A= + return code =3D=3D DOT_PROD_EXPR || code =3D=3D WIDEN_SUM_EXPR || code = =3D=3D SAD_EXPR;=0A= +}=0A= +=0A= /* Source location + hotness information. */=0A= extern dump_user_location_t vect_location;=0A= =0A= @@ -2434,6 +2446,8 @@ extern loop_vec_info vect_create_loop_vinfo (class lo= op *, vec_info_shared *,=0A= extern bool vectorizable_live_operation (vec_info *, stmt_vec_info,=0A= slp_tree, slp_instance, int,=0A= bool, stmt_vector_for_cost *);=0A= +extern bool vectorizable_lane_reducing (loop_vec_info, stmt_vec_info,=0A= + slp_tree, stmt_vector_for_cost *);=0A= extern bool vectorizable_reduction (loop_vec_info, stmt_vec_info,=0A= slp_tree, slp_instance,=0A= stmt_vector_for_cost *);=0A= =0A= ________________________________________=0A= From: Feng Xue OS =0A= Sent: Sunday, April 7, 2024 2:59 PM=0A= To: Richard Biener=0A= Cc: gcc-patches@gcc.gnu.org=0A= Subject: [PATCH] vect: Support multiple lane-reducing operations for loop r= eduction [PR114440]=0A= =0A= For lane-reducing operation(dot-prod/widen-sum/sad) in loop reduction, curr= ent=0A= vectorizer could only handle the pattern if the reduction chain does not=0A= contain other operation, no matter the other is normal or lane-reducing.=0A= =0A= Acctually, to allow multiple arbitray lane-reducing operations, we need to= =0A= support vectorization of loop reduction chain with mixed input vectypes. Si= nce=0A= lanes of vectype may vary with operation, the effective ncopies of vectoriz= ed=0A= statements for operation also may not be same to each other, this causes=0A= mismatch on vectorized def-use cycles. A simple way is to align all operati= ons=0A= with the one that has the most ncopies, the gap could be complemented by=0A= generating extra trival pass-through copies. For example:=0A= =0A= int sum =3D 0;=0A= for (i)=0A= {=0A= sum +=3D d0[i] * d1[i]; // dot-prod =0A= sum +=3D w[i]; // widen-sum =0A= sum +=3D abs(s0[i] - s1[i]); // sad =0A= sum +=3D n[i]; // normal =0A= }=0A= =0A= The vector size is 128-bit,vectorization factor is 16. Reduction statements= =0A= would be transformed as:=0A= =0A= vector<4> int sum_v0 =3D { 0, 0, 0, 0 };=0A= vector<4> int sum_v1 =3D { 0, 0, 0, 0 };=0A= vector<4> int sum_v2 =3D { 0, 0, 0, 0 };=0A= vector<4> int sum_v3 =3D { 0, 0, 0, 0 };=0A= =0A= for (i / 16)=0A= {=0A= sum_v0 =3D DOT_PROD (d0_v0[i: 0 ~ 15], d1_v0[i: 0 ~ 15], sum_v0);=0A= sum_v1 =3D sum_v1; // copy=0A= sum_v2 =3D sum_v2; // copy=0A= sum_v3 =3D sum_v3; // copy=0A= =0A= sum_v0 =3D sum_v0; // copy=0A= sum_v1 =3D WIDEN_SUM (w_v1[i: 0 ~ 15], sum_v1);=0A= sum_v2 =3D sum_v2; // copy=0A= sum_v3 =3D sum_v3; // copy=0A= =0A= sum_v0 =3D sum_v0; // copy=0A= sum_v1 =3D sum_v1; // copy=0A= sum_v2 =3D SAD (s0_v2[i: 0 ~ 7 ], s1_v2[i: 0 ~ 7 ], sum_v2);=0A= sum_v3 =3D SAD (s0_v3[i: 8 ~ 15], s1_v3[i: 8 ~ 15], sum_v3);=0A= =0A= sum_v0 +=3D n_v0[i: 0 ~ 3 ];=0A= sum_v1 +=3D n_v1[i: 4 ~ 7 ];=0A= sum_v2 +=3D n_v2[i: 8 ~ 11];=0A= sum_v3 +=3D n_v3[i: 12 ~ 15];=0A= }=0A= =0A= Moreover, for a higher instruction parallelism in final vectorized loop, it= =0A= is considered to make those effective vectorized lane-reducing statements b= e=0A= distributed evenly among all def-use cycles. In the above example, DOT_PROD= ,=0A= WIDEN_SUM and SADs are generated into disparate cycles.=0A= =0A= Bootstrapped/regtested on x86_64-linux and aarch64-linux.=0A= =0A= Feng=0A= --_002_LV2PR01MB78397C320D7870B56652ACC6F7F52LV2PR01MB7839prod_ Content-Type: text/x-patch; name="0001-vect-Support-multiple-lane-reducing-operations-for-l.patch" Content-Description: 0001-vect-Support-multiple-lane-reducing-operations-for-l.patch Content-Disposition: attachment; filename="0001-vect-Support-multiple-lane-reducing-operations-for-l.patch"; size=56024; creation-date="Fri, 24 May 2024 09:25:48 GMT"; modification-date="Fri, 24 May 2024 09:25:48 GMT" Content-Transfer-Encoding: base64 RnJvbSBjYTQwMGVlNDMyNjVjNDhjM2M3MDllYzM4YjBhYmExYmM5YzRkMGQ5IE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBGZW5nIFh1ZSA8Znh1ZUBvcy5hbXBlcmVjb21wdXRpbmcuY29t PgpEYXRlOiBGcmksIDIyIE1hciAyMDI0IDE5OjU3OjQ1ICswODAwClN1YmplY3Q6IFtQQVRDSF0g dmVjdDogU3VwcG9ydCBtdWx0aXBsZSBsYW5lLXJlZHVjaW5nIG9wZXJhdGlvbnMgZm9yIGxvb3AK IHJlZHVjdGlvbiBbUFIxMTQ0NDBdCk1JTUUtVmVyc2lvbjogMS4wCkNvbnRlbnQtVHlwZTogdGV4 dC9wbGFpbjsgY2hhcnNldD1VVEYtOApDb250ZW50LVRyYW5zZmVyLUVuY29kaW5nOiA4Yml0CgpG b3IgbGFuZS1yZWR1Y2luZyBvcGVyYXRpb24oZG90LXByb2Qvd2lkZW4tc3VtL3NhZCkgaW4gbG9v cCByZWR1Y3Rpb24sIGN1cnJlbnQKdmVjdG9yaXplciBjb3VsZCBvbmx5IGhhbmRsZSB0aGUgcGF0 dGVybiBpZiB0aGUgcmVkdWN0aW9uIGNoYWluIGRvZXMgbm90CmNvbnRhaW4gb3RoZXIgb3BlcmF0 aW9uLCBubyBtYXR0ZXIgdGhlIG90aGVyIGlzIG5vcm1hbCBvciBsYW5lLXJlZHVjaW5nLgoKQWNj dHVhbGx5LCB0byBhbGxvdyBtdWx0aXBsZSBhcmJpdHJheSBsYW5lLXJlZHVjaW5nIG9wZXJhdGlv bnMsIHdlIG5lZWQgdG8Kc3VwcG9ydCB2ZWN0b3JpemF0aW9uIG9mIGxvb3AgcmVkdWN0aW9uIGNo YWluIHdpdGggbWl4ZWQgaW5wdXQgdmVjdHlwZXMuIFNpbmNlCmxhbmVzIG9mIHZlY3R5cGUgbWF5 IHZhcnkgd2l0aCBvcGVyYXRpb24sIHRoZSBlZmZlY3RpdmUgbmNvcGllcyBvZiB2ZWN0b3JpemVk CnN0YXRlbWVudHMgZm9yIG9wZXJhdGlvbiBhbHNvIG1heSBub3QgYmUgc2FtZSB0byBlYWNoIG90 aGVyLCB0aGlzIGNhdXNlcwptaXNtYXRjaCBvbiB2ZWN0b3JpemVkIGRlZi11c2UgY3ljbGVzLiBB IHNpbXBsZSB3YXkgaXMgdG8gYWxpZ24gYWxsIG9wZXJhdGlvbnMKd2l0aCB0aGUgb25lIHRoYXQg aGFzIHRoZSBtb3N0IG5jb3BpZXMsIHRoZSBnYXAgY291bGQgYmUgY29tcGxlbWVudGVkIGJ5Cmdl bmVyYXRpbmcgZXh0cmEgdHJpdmFsIHBhc3MtdGhyb3VnaCBjb3BpZXMuIEZvciBleGFtcGxlOgoK ICAgaW50IHN1bSA9IDA7CiAgIGZvciAoaSkKICAgICB7CiAgICAgICBzdW0gKz0gZDBbaV0gKiBk MVtpXTsgICAgICAvLyBkb3QtcHJvZCA8dmVjdG9yKDE2KSBjaGFyPgogICAgICAgc3VtICs9IHdb aV07ICAgICAgICAgICAgICAgLy8gd2lkZW4tc3VtIDx2ZWN0b3IoMTYpIGNoYXI+CiAgICAgICBz dW0gKz0gYWJzKHMwW2ldIC0gczFbaV0pOyAvLyBzYWQgPHZlY3Rvcig4KSBzaG9ydD4KICAgICAg IHN1bSArPSBuW2ldOyAgICAgICAgICAgICAgIC8vIG5vcm1hbCA8dmVjdG9yKDQpIGludD4KICAg ICB9CgpUaGUgdmVjdG9yIHNpemUgaXMgMTI4LWJpdO+8jHZlY3Rvcml6YXRpb24gZmFjdG9yIGlz IDE2LiBSZWR1Y3Rpb24gc3RhdGVtZW50cwp3b3VsZCBiZSB0cmFuc2Zvcm1lZCBhczoKCiAgIHZl Y3Rvcjw0PiBpbnQgc3VtX3YwID0geyAwLCAwLCAwLCAwIH07CiAgIHZlY3Rvcjw0PiBpbnQgc3Vt X3YxID0geyAwLCAwLCAwLCAwIH07CiAgIHZlY3Rvcjw0PiBpbnQgc3VtX3YyID0geyAwLCAwLCAw LCAwIH07CiAgIHZlY3Rvcjw0PiBpbnQgc3VtX3YzID0geyAwLCAwLCAwLCAwIH07CgogICBmb3Ig KGkgLyAxNikKICAgICB7CiAgICAgICBzdW1fdjAgPSBET1RfUFJPRCAoZDBfdjBbaTogMCB+IDE1 XSwgZDFfdjBbaTogMCB+IDE1XSwgc3VtX3YwKTsKICAgICAgIHN1bV92MSA9IHN1bV92MTsgIC8v IGNvcHkKICAgICAgIHN1bV92MiA9IHN1bV92MjsgIC8vIGNvcHkKICAgICAgIHN1bV92MyA9IHN1 bV92MzsgIC8vIGNvcHkKCiAgICAgICBzdW1fdjAgPSBzdW1fdjA7ICAvLyBjb3B5CiAgICAgICBz dW1fdjEgPSBXSURFTl9TVU0gKHdfdjFbaTogMCB+IDE1XSwgc3VtX3YxKTsKICAgICAgIHN1bV92 MiA9IHN1bV92MjsgIC8vIGNvcHkKICAgICAgIHN1bV92MyA9IHN1bV92MzsgIC8vIGNvcHkKCiAg ICAgICBzdW1fdjAgPSBzdW1fdjA7ICAvLyBjb3B5CiAgICAgICBzdW1fdjEgPSBzdW1fdjE7ICAv LyBjb3B5CiAgICAgICBzdW1fdjIgPSBTQUQgKHMwX3YyW2k6IDAgfiA3IF0sIHMxX3YyW2k6IDAg fiA3IF0sIHN1bV92Mik7CiAgICAgICBzdW1fdjMgPSBTQUQgKHMwX3YzW2k6IDggfiAxNV0sIHMx X3YzW2k6IDggfiAxNV0sIHN1bV92Myk7CgogICAgICAgc3VtX3YwICs9IG5fdjBbaTogMCAgfiAz IF07CiAgICAgICBzdW1fdjEgKz0gbl92MVtpOiA0ICB+IDcgXTsKICAgICAgIHN1bV92MiArPSBu X3YyW2k6IDggIH4gMTFdOwogICAgICAgc3VtX3YzICs9IG5fdjNbaTogMTIgfiAxNV07CiAgICAg fQoKTW9yZW92ZXIsIGZvciBhIGhpZ2hlciBpbnN0cnVjdGlvbiBwYXJhbGxlbGlzbSBpbiBmaW5h bCB2ZWN0b3JpemVkIGxvb3AsIGl0CmlzIGNvbnNpZGVyZWQgdG8gbWFrZSB0aG9zZSBlZmZlY3Rp dmUgdmVjdG9yaXplZCBsYW5lLXJlZHVjaW5nIHN0YXRlbWVudHMgYmUKZGlzdHJpYnV0ZWQgZXZl bmx5IGFtb25nIGFsbCBkZWYtdXNlIGN5Y2xlcy4gSW4gdGhlIGFib3ZlIGV4YW1wbGUsIERPVF9Q Uk9ELApXSURFTl9TVU0gYW5kIFNBRHMgYXJlIGdlbmVyYXRlZCBpbnRvIGRpc3BhcmF0ZSBjeWNs ZXMuCgoyMDI0LTAzLTIyIEZlbmcgWHVlIDxmeHVlQG9zLmFtcGVyZWNvbXB1dGluZy5jb20+Cgpn Y2MvCglQUiB0cmVlLW9wdGltaXphdGlvbi8xMTQ0NDAKCSogdHJlZS12ZWN0b3JpemVyLmggKHN0 cnVjdCBfc3RtdF92ZWNfaW5mbyk6IEFkZCBhIG5ldyBmaWVsZAoJcmVkdWNfcmVzdWx0X3Bvcy4K CShsYW5lX3JlZHVjaW5nX29wX3ApOiBOZXcgZnVuY3Rpb24uCgkodmVjdG9yaXphYmxlX2xhbmVf cmVkdWNpbmcpOiBOZXcgZnVuY3Rpb24gZGVjbGFyYXRpb24uCgkqIHRyZWUtdmVjdC1zdG10cy5j YyAodmVjdG9yaXphYmxlX2NvbmRpdGlvbik6IFRyZWF0IHRoZSBjb25kaXRpb24KCXN0YXRlbWVu dCB0aGF0IGlzIHBvaW50ZWQgYnkgc3RtdF92ZWNfaW5mbyBvZiByZWR1Y3Rpb24gUEhJIGFzIHRo ZQoJcmVhbCAiZm9yX3JlZHVjdGlvbiIgc3RhdGVtZW50LgoJKHZlY3RfYW5hbHl6ZV9zdG10KTog Q2FsbCBuZXcgZnVuY3Rpb24gdmVjdG9yaXphYmxlX2xhbmVfcmVkdWNpbmcKCXRvIGFuYWx5emUg bGFuZS1yZWR1Y2luZyBvcGVyYXRpb24uCgkqIHRyZWUtdmVjdC1sb29wLmNjICh2ZWN0X2lzX2Vt dWxhdGVkX21peGVkX2RvdF9wcm9kKTogUmVtb3ZlIHBhcmFtZXRlcgoJbG9vcF92aW5mby4gR2V0 IGlucHV0IHZlY3R5cGUgZnJvbSBzdG10X2luZm8gaW5zdGVhZCBvZiByZWR1Y3Rpb24gUEhJLgoJ KHZlY3RfbW9kZWxfcmVkdWN0aW9uX2Nvc3QpOiBSZW1vdmUgY29zdCBjb21wdXRhdGlvbiBjb2Rl IHJlbGF0ZWQgdG8KCWVtdWxhdGVkX21peGVkX2RvdF9wcm9kLgoJKHZlY3RfcmVkdWN0aW9uX3Vz ZV9wYXJ0aWFsX3ZlY3Rvcik6IE5ldyBmdW5jdGlvbi4KCSh2ZWN0b3JpemFibGVfbGFuZV9yZWR1 Y2luZyk6IE5ldyBmdW5jdGlvbi4KCSh2ZWN0b3JpemFibGVfcmVkdWN0aW9uKTogQWxsb3cgbXVs dGlwbGUgbGFuZS1yZWR1Y2luZyBvcGVyYXRpb25zIGluCglsb29wIHJlZHVjdGlvbi4gTW92ZSBz b21lIG9yaWdpbmFsIGxhbmUtcmVkdWNpbmcgcmVsYXRlZCBjb2RlIHRvCgl2ZWN0b3JpemFibGVf bGFuZV9yZWR1Y2luZywgYW5kIG1vdmUgcGFydGlhbCB2ZWN0b3JpemF0aW9uIGNoZWNraW5nCglj b2RlIHRvIHZlY3RfcmVkdWN0aW9uX3VzZV9wYXJ0aWFsX3ZlY3Rvci4KCSh2ZWN0X3RyYW5zZm9y bV9yZWR1Y3Rpb24pOiBFeHRlbmQgdHJhbnNmb3JtYXRpb24gdG8gc3VwcG9ydCByZWR1Y3Rpb24K CXN0YXRlbWVudHMgd2l0aCBtaXhlZCBpbnB1dCB2ZWN0eXBlcy4KCSogdHJlZS12ZWN0LXNscC5j YyAodmVjdF9hbmFseXplX3NscCk6IFVzZSBuZXcgZnVuY3Rpb24KCWxhbmVfcmVkdWNpbmdfb3Bf cCB0byBjaGVjayBzdGF0ZW1lbnQgY29kZS4KCmdjYy90ZXN0c3VpdGUvCglQUiB0cmVlLW9wdGlt aXphdGlvbi8xMTQ0NDAKCSogZ2NjLmRnL3ZlY3QvdmVjdC1yZWR1Yy1jaGFpbi0xLmMKCSogZ2Nj LmRnL3ZlY3QvdmVjdC1yZWR1Yy1jaGFpbi0yLmMKCSogZ2NjLmRnL3ZlY3QvdmVjdC1yZWR1Yy1j aGFpbi0zLmMKCSogZ2NjLmRnL3ZlY3QvdmVjdC1yZWR1Yy1kb3Qtc2xwLTEuYwoJKiBnY2MuZGcv dmVjdC92ZWN0LXJlZHVjLWRvdC1zbHAtMi5jCi0tLQogLi4uL2djYy5kZy92ZWN0L3ZlY3QtcmVk dWMtY2hhaW4tMS5jICAgICAgICAgIHwgIDYyICsrCiAuLi4vZ2NjLmRnL3ZlY3QvdmVjdC1yZWR1 Yy1jaGFpbi0yLmMgICAgICAgICAgfCAgNzcgKysKIC4uLi9nY2MuZGcvdmVjdC92ZWN0LXJlZHVj LWNoYWluLTMuYyAgICAgICAgICB8ICA2NiArKwogLi4uL2djYy5kZy92ZWN0L3ZlY3QtcmVkdWMt ZG90LXNscC0xLmMgICAgICAgIHwgIDk3ICsrKwogLi4uL2djYy5kZy92ZWN0L3ZlY3QtcmVkdWMt ZG90LXNscC0yLmMgICAgICAgIHwgIDgxICsrKwogZ2NjL3RyZWUtdmVjdC1sb29wLmNjICAgICAg ICAgICAgICAgICAgICAgICAgIHwgNjgwICsrKysrKysrKysrKy0tLS0tLQogZ2NjL3RyZWUtdmVj dC1zbHAuY2MgICAgICAgICAgICAgICAgICAgICAgICAgIHwgICA0ICstCiBnY2MvdHJlZS12ZWN0 LXN0bXRzLmNjICAgICAgICAgICAgICAgICAgICAgICAgfCAgMTMgKy0KIGdjYy90cmVlLXZlY3Rv cml6ZXIuaCAgICAgICAgICAgICAgICAgICAgICAgICB8ICAxNCArCiA5IGZpbGVzIGNoYW5nZWQs IDg3MyBpbnNlcnRpb25zKCspLCAyMjEgZGVsZXRpb25zKC0pCiBjcmVhdGUgbW9kZSAxMDA2NDQg Z2NjL3Rlc3RzdWl0ZS9nY2MuZGcvdmVjdC92ZWN0LXJlZHVjLWNoYWluLTEuYwogY3JlYXRlIG1v ZGUgMTAwNjQ0IGdjYy90ZXN0c3VpdGUvZ2NjLmRnL3ZlY3QvdmVjdC1yZWR1Yy1jaGFpbi0yLmMK IGNyZWF0ZSBtb2RlIDEwMDY0NCBnY2MvdGVzdHN1aXRlL2djYy5kZy92ZWN0L3ZlY3QtcmVkdWMt Y2hhaW4tMy5jCiBjcmVhdGUgbW9kZSAxMDA2NDQgZ2NjL3Rlc3RzdWl0ZS9nY2MuZGcvdmVjdC92 ZWN0LXJlZHVjLWRvdC1zbHAtMS5jCiBjcmVhdGUgbW9kZSAxMDA2NDQgZ2NjL3Rlc3RzdWl0ZS9n Y2MuZGcvdmVjdC92ZWN0LXJlZHVjLWRvdC1zbHAtMi5jCgpkaWZmIC0tZ2l0IGEvZ2NjL3Rlc3Rz dWl0ZS9nY2MuZGcvdmVjdC92ZWN0LXJlZHVjLWNoYWluLTEuYyBiL2djYy90ZXN0c3VpdGUvZ2Nj LmRnL3ZlY3QvdmVjdC1yZWR1Yy1jaGFpbi0xLmMKbmV3IGZpbGUgbW9kZSAxMDA2NDQKaW5kZXgg MDAwMDAwMDAwMDAuLjA0YmZjNDE5ZGJkCi0tLSAvZGV2L251bGwKKysrIGIvZ2NjL3Rlc3RzdWl0 ZS9nY2MuZGcvdmVjdC92ZWN0LXJlZHVjLWNoYWluLTEuYwpAQCAtMCwwICsxLDYyIEBACisvKiBE aXNhYmxpbmcgZXBpbG9ndWVzIHVudGlsIHdlIGZpbmQgYSBiZXR0ZXIgd2F5IHRvIGRlYWwgd2l0 aCBzY2Fucy4gICovCisvKiB7IGRnLWFkZGl0aW9uYWwtb3B0aW9ucyAiLS1wYXJhbSB2ZWN0LWVw aWxvZ3Vlcy1ub21hc2s9MCIgfSAqLworLyogeyBkZy1yZXF1aXJlLWVmZmVjdGl2ZS10YXJnZXQg dmVjdF9pbnQgfSAqLworLyogeyBkZy1yZXF1aXJlLWVmZmVjdGl2ZS10YXJnZXQgYXJtX3Y4XzJh X2RvdHByb2RfbmVvbl9odyB7IHRhcmdldCB7IGFhcmNoNjQqLSotKiB8fCBhcm0qLSotKiB9IH0g fSAqLworLyogeyBkZy1hZGQtb3B0aW9ucyBhcm1fdjhfMmFfZG90cHJvZF9uZW9uIH0gICovCisK KyNpbmNsdWRlICJ0cmVlLXZlY3QuaCIKKworI2RlZmluZSBOIDUwCisKKyNpZm5kZWYgU0lHTkVE TkVTU18xCisjZGVmaW5lIFNJR05FRE5FU1NfMSBzaWduZWQKKyNkZWZpbmUgU0lHTkVETkVTU18y IHNpZ25lZAorI2VuZGlmCisKK1NJR05FRE5FU1NfMSBpbnQgX19hdHRyaWJ1dGVfXyAoKG5vaXBh KSkKK2YgKFNJR05FRE5FU1NfMSBpbnQgcmVzLAorICAgU0lHTkVETkVTU18yIGNoYXIgKnJlc3Ry aWN0IGEsCisgICBTSUdORURORVNTXzIgY2hhciAqcmVzdHJpY3QgYiwKKyAgIFNJR05FRE5FU1Nf MiBjaGFyICpyZXN0cmljdCBjLAorICAgU0lHTkVETkVTU18yIGNoYXIgKnJlc3RyaWN0IGQsCisg ICBTSUdORURORVNTXzEgaW50ICpyZXN0cmljdCBlKQoreworICBmb3IgKGludCBpID0gMDsgaSA8 IE47ICsraSkKKyAgICB7CisgICAgICByZXMgKz0gYVtpXSAqIGJbaV07CisgICAgICByZXMgKz0g Y1tpXSAqIGRbaV07CisgICAgICByZXMgKz0gZVtpXTsKKyAgICB9CisgIHJldHVybiByZXM7Cit9 CisKKyNkZWZpbmUgQkFTRSAoKFNJR05FRE5FU1NfMiBpbnQpIC0xIDwgMCA/IC0xMjYgOiA0KQor I2RlZmluZSBPRkZTRVQgMjAKKworaW50CittYWluICh2b2lkKQoreworICBjaGVja192ZWN0ICgp OworCisgIFNJR05FRE5FU1NfMiBjaGFyIGFbTl0sIGJbTl07CisgIFNJR05FRE5FU1NfMiBjaGFy IGNbTl0sIGRbTl07CisgIFNJR05FRE5FU1NfMSBpbnQgZVtOXTsKKyAgaW50IGV4cGVjdGVkID0g MHgxMjM0NTsKKyAgZm9yIChpbnQgaSA9IDA7IGkgPCBOOyArK2kpCisgICAgeworICAgICAgYVtp XSA9IEJBU0UgKyBpICogNTsKKyAgICAgIGJbaV0gPSBCQVNFICsgT0ZGU0VUICsgaSAqIDQ7Cisg ICAgICBjW2ldID0gQkFTRSArIGkgKiAyOworICAgICAgZFtpXSA9IEJBU0UgKyBPRkZTRVQgKyBp ICogMzsKKyAgICAgIGVbaV0gPSBpOworICAgICAgYXNtIHZvbGF0aWxlICgiIiA6OjogIm1lbW9y eSIpOworICAgICAgZXhwZWN0ZWQgKz0gYVtpXSAqIGJbaV07CisgICAgICBleHBlY3RlZCArPSBj W2ldICogZFtpXTsKKyAgICAgIGV4cGVjdGVkICs9IGVbaV07CisgICAgfQorICBpZiAoZiAoMHgx MjM0NSwgYSwgYiwgYywgZCwgZSkgIT0gZXhwZWN0ZWQpCisgICAgX19idWlsdGluX2Fib3J0ICgp OworfQorCisvKiB7IGRnLWZpbmFsIHsgc2Nhbi10cmVlLWR1bXAgInZlY3RfcmVjb2dfZG90X3By b2RfcGF0dGVybjogZGV0ZWN0ZWQiICJ2ZWN0IiB9IH0gKi8KKy8qIHsgZGctZmluYWwgeyBzY2Fu LXRyZWUtZHVtcC10aW1lcyAidmVjdG9yaXppbmcgc3RhdGVtZW50OiBcXFMrID0gRE9UX1BST0Rf RVhQUiIgMiAidmVjdCIgeyB0YXJnZXQgdmVjdF9zZG90X3FpIH0gfSB9ICovCmRpZmYgLS1naXQg YS9nY2MvdGVzdHN1aXRlL2djYy5kZy92ZWN0L3ZlY3QtcmVkdWMtY2hhaW4tMi5jIGIvZ2NjL3Rl c3RzdWl0ZS9nY2MuZGcvdmVjdC92ZWN0LXJlZHVjLWNoYWluLTIuYwpuZXcgZmlsZSBtb2RlIDEw MDY0NAppbmRleCAwMDAwMDAwMDAwMC4uNmM4MDNiODAxMjAKLS0tIC9kZXYvbnVsbAorKysgYi9n Y2MvdGVzdHN1aXRlL2djYy5kZy92ZWN0L3ZlY3QtcmVkdWMtY2hhaW4tMi5jCkBAIC0wLDAgKzEs NzcgQEAKKy8qIERpc2FibGluZyBlcGlsb2d1ZXMgdW50aWwgd2UgZmluZCBhIGJldHRlciB3YXkg dG8gZGVhbCB3aXRoIHNjYW5zLiAgKi8KKy8qIHsgZGctYWRkaXRpb25hbC1vcHRpb25zICItLXBh cmFtIHZlY3QtZXBpbG9ndWVzLW5vbWFzaz0wIiB9ICovCisvKiB7IGRnLXJlcXVpcmUtZWZmZWN0 aXZlLXRhcmdldCB2ZWN0X2ludCB9ICovCisvKiB7IGRnLXJlcXVpcmUtZWZmZWN0aXZlLXRhcmdl dCBhcm1fdjhfMmFfZG90cHJvZF9uZW9uX2h3IHsgdGFyZ2V0IHsgYWFyY2g2NCotKi0qIHx8IGFy bSotKi0qIH0gfSB9ICovCisvKiB7IGRnLWFkZC1vcHRpb25zIGFybV92OF8yYV9kb3Rwcm9kX25l b24gfSAgKi8KKworI2luY2x1ZGUgInRyZWUtdmVjdC5oIgorCisjZGVmaW5lIE4gNTAKKworI2lm bmRlZiBTSUdORURORVNTXzEKKyNkZWZpbmUgU0lHTkVETkVTU18xIHNpZ25lZAorI2RlZmluZSBT SUdORURORVNTXzIgdW5zaWduZWQKKyNkZWZpbmUgU0lHTkVETkVTU18zIHNpZ25lZAorI2RlZmlu ZSBTSUdORURORVNTXzQgc2lnbmVkCisjZW5kaWYKKworU0lHTkVETkVTU18xIGludCBfX2F0dHJp YnV0ZV9fICgobm9pcGEpKQorZm4gKFNJR05FRE5FU1NfMSBpbnQgcmVzLAorICAgU0lHTkVETkVT U18yIGNoYXIgKnJlc3RyaWN0IGEsCisgICBTSUdORURORVNTXzIgY2hhciAqcmVzdHJpY3QgYiwK KyAgIFNJR05FRE5FU1NfMyBjaGFyICpyZXN0cmljdCBjLAorICAgU0lHTkVETkVTU18zIGNoYXIg KnJlc3RyaWN0IGQsCisgICBTSUdORURORVNTXzQgc2hvcnQgKnJlc3RyaWN0IGUsCisgICBTSUdO RURORVNTXzQgc2hvcnQgKnJlc3RyaWN0IGYsCisgICBTSUdORURORVNTXzEgaW50ICpyZXN0cmlj dCBnKQoreworICBmb3IgKGludCBpID0gMDsgaSA8IE47ICsraSkKKyAgICB7CisgICAgICByZXMg Kz0gYVtpXSAqIGJbaV07CisgICAgICByZXMgKz0gaSArIDE7CisgICAgICByZXMgKz0gY1tpXSAq IGRbaV07CisgICAgICByZXMgKz0gZVtpXSAqIGZbaV07CisgICAgICByZXMgKz0gZ1tpXTsKKyAg ICB9CisgIHJldHVybiByZXM7Cit9CisKKyNkZWZpbmUgQkFTRTIgKChTSUdORURORVNTXzIgaW50 KSAtMSA8IDAgPyAtMTI2IDogNCkKKyNkZWZpbmUgQkFTRTMgKChTSUdORURORVNTXzMgaW50KSAt MSA8IDAgPyAtMTI2IDogNCkKKyNkZWZpbmUgQkFTRTQgKChTSUdORURORVNTXzQgaW50KSAtMSA8 IDAgPyAtMTAyNiA6IDM3MykKKyNkZWZpbmUgT0ZGU0VUIDIwCisKK2ludAorbWFpbiAodm9pZCkK K3sKKyAgY2hlY2tfdmVjdCAoKTsKKworICBTSUdORURORVNTXzIgY2hhciBhW05dLCBiW05dOwor ICBTSUdORURORVNTXzMgY2hhciBjW05dLCBkW05dOworICBTSUdORURORVNTXzQgc2hvcnQgZVtO XSwgZltOXTsKKyAgU0lHTkVETkVTU18xIGludCBnW05dOworICBpbnQgZXhwZWN0ZWQgPSAweDEy MzQ1OworICBmb3IgKGludCBpID0gMDsgaSA8IE47ICsraSkKKyAgICB7CisgICAgICBhW2ldID0g QkFTRTIgKyBpICogNTsKKyAgICAgIGJbaV0gPSBCQVNFMiArIE9GRlNFVCArIGkgKiA0OworICAg ICAgY1tpXSA9IEJBU0UzICsgaSAqIDI7CisgICAgICBkW2ldID0gQkFTRTMgKyBPRkZTRVQgKyBp ICogMzsKKyAgICAgIGVbaV0gPSBCQVNFNCArIGkgKiA2OworICAgICAgZltpXSA9IEJBU0U0ICsg T0ZGU0VUICsgaSAqIDU7CisgICAgICBnW2ldID0gaTsKKyAgICAgIGFzbSB2b2xhdGlsZSAoIiIg Ojo6ICJtZW1vcnkiKTsKKyAgICAgIGV4cGVjdGVkICs9IGFbaV0gKiBiW2ldOworICAgICAgZXhw ZWN0ZWQgKz0gaSArIDE7CisgICAgICBleHBlY3RlZCArPSBjW2ldICogZFtpXTsKKyAgICAgIGV4 cGVjdGVkICs9IGVbaV0gKiBmW2ldOworICAgICAgZXhwZWN0ZWQgKz0gZ1tpXTsKKyAgICB9Cisg IGlmIChmbiAoMHgxMjM0NSwgYSwgYiwgYywgZCwgZSwgZiwgZykgIT0gZXhwZWN0ZWQpCisgICAg X19idWlsdGluX2Fib3J0ICgpOworfQorCisvKiB7IGRnLWZpbmFsIHsgc2Nhbi10cmVlLWR1bXAg InZlY3RfcmVjb2dfZG90X3Byb2RfcGF0dGVybjogZGV0ZWN0ZWQiICJ2ZWN0IiB9IH0gKi8KKy8q IHsgZGctZmluYWwgeyBzY2FuLXRyZWUtZHVtcCAidmVjdG9yaXppbmcgc3RhdGVtZW50OiBcXFMr ID0gRE9UX1BST0RfRVhQUiIgInZlY3QiIHsgdGFyZ2V0IHsgdmVjdF9zZG90X3FpIH0gfSB9IH0g Ki8KKy8qIHsgZGctZmluYWwgeyBzY2FuLXRyZWUtZHVtcCAidmVjdG9yaXppbmcgc3RhdGVtZW50 OiBcXFMrID0gRE9UX1BST0RfRVhQUiIgInZlY3QiIHsgdGFyZ2V0IHsgdmVjdF91ZG90X3FpIH0g fSB9IH0gKi8KKy8qIHsgZGctZmluYWwgeyBzY2FuLXRyZWUtZHVtcCAidmVjdG9yaXppbmcgc3Rh dGVtZW50OiBcXFMrID0gRE9UX1BST0RfRVhQUiIgInZlY3QiIHsgdGFyZ2V0IHsgdmVjdF9zZG90 X2hpIH0gfSB9IH0gKi8KZGlmZiAtLWdpdCBhL2djYy90ZXN0c3VpdGUvZ2NjLmRnL3ZlY3QvdmVj dC1yZWR1Yy1jaGFpbi0zLmMgYi9nY2MvdGVzdHN1aXRlL2djYy5kZy92ZWN0L3ZlY3QtcmVkdWMt Y2hhaW4tMy5jCm5ldyBmaWxlIG1vZGUgMTAwNjQ0CmluZGV4IDAwMDAwMDAwMDAwLi5hNDFlNGIx NzZjNAotLS0gL2Rldi9udWxsCisrKyBiL2djYy90ZXN0c3VpdGUvZ2NjLmRnL3ZlY3QvdmVjdC1y ZWR1Yy1jaGFpbi0zLmMKQEAgLTAsMCArMSw2NiBAQAorLyogRGlzYWJsaW5nIGVwaWxvZ3VlcyB1 bnRpbCB3ZSBmaW5kIGEgYmV0dGVyIHdheSB0byBkZWFsIHdpdGggc2NhbnMuICAqLworLyogeyBk Zy1hZGRpdGlvbmFsLW9wdGlvbnMgIi0tcGFyYW0gdmVjdC1lcGlsb2d1ZXMtbm9tYXNrPTAiIH0g Ki8KKy8qIHsgZGctcmVxdWlyZS1lZmZlY3RpdmUtdGFyZ2V0IHZlY3RfaW50IH0gKi8KKworI2lu Y2x1ZGUgInRyZWUtdmVjdC5oIgorCisjZGVmaW5lIE4gNTAKKworI2lmbmRlZiBTSUdORURORVNT XzEKKyNkZWZpbmUgU0lHTkVETkVTU18xIHNpZ25lZAorI2RlZmluZSBTSUdORURORVNTXzIgdW5z aWduZWQKKyNkZWZpbmUgU0lHTkVETkVTU18zIHNpZ25lZAorI2VuZGlmCisKK1NJR05FRE5FU1Nf MSBpbnQgX19hdHRyaWJ1dGVfXyAoKG5vaXBhKSkKK2YgKFNJR05FRE5FU1NfMSBpbnQgcmVzLAor ICAgU0lHTkVETkVTU18yIGNoYXIgKnJlc3RyaWN0IGEsCisgICBTSUdORURORVNTXzIgY2hhciAq cmVzdHJpY3QgYiwKKyAgIFNJR05FRE5FU1NfMyBzaG9ydCAqcmVzdHJpY3QgYywKKyAgIFNJR05F RE5FU1NfMyBzaG9ydCAqcmVzdHJpY3QgZCwKKyAgIFNJR05FRE5FU1NfMSBpbnQgKnJlc3RyaWN0 IGUpCit7CisgIGZvciAoaW50IGkgPSAwOyBpIDwgTjsgKytpKQorICAgIHsKKyAgICAgIHNob3J0 IGRpZmYgPSBhW2ldIC0gYltpXTsKKyAgICAgIFNJR05FRE5FU1NfMiBzaG9ydCBhYnMgPSBkaWZm IDwgMCA/IC1kaWZmIDogZGlmZjsKKyAgICAgIHJlcyArPSBhYnM7CisgICAgICByZXMgKz0gY1tp XSAqIGRbaV07CisgICAgICByZXMgKz0gZVtpXTsKKyAgICB9CisgIHJldHVybiByZXM7Cit9CisK KyNkZWZpbmUgQkFTRTIgKChTSUdORURORVNTXzIgaW50KSAtMSA8IDAgPyAtMTI2IDogNCkKKyNk ZWZpbmUgQkFTRTMgKChTSUdORURORVNTXzMgaW50KSAtMSA8IDAgPyAtMTIzNiA6IDM3MykKKyNk ZWZpbmUgT0ZGU0VUIDIwCisKK2ludAorbWFpbiAodm9pZCkKK3sKKyAgY2hlY2tfdmVjdCAoKTsK KworICBTSUdORURORVNTXzIgY2hhciBhW05dLCBiW05dOworICBTSUdORURORVNTXzMgc2hvcnQg Y1tOXSwgZFtOXTsKKyAgU0lHTkVETkVTU18xIGludCBlW05dOworICBpbnQgZXhwZWN0ZWQgPSAw eDEyMzQ1OworICBmb3IgKGludCBpID0gMDsgaSA8IE47ICsraSkKKyAgICB7CisgICAgICBhW2ld ID0gQkFTRTIgKyBpICogNTsKKyAgICAgIGJbaV0gPSBCQVNFMiAtIGkgKiA0OworICAgICAgY1tp XSA9IEJBU0UzICsgaSAqIDI7CisgICAgICBkW2ldID0gQkFTRTMgKyBPRkZTRVQgKyBpICogMzsK KyAgICAgIGVbaV0gPSBpOworICAgICAgYXNtIHZvbGF0aWxlICgiIiA6OjogIm1lbW9yeSIpOwor ICAgICAgc2hvcnQgZGlmZiA9IGFbaV0gLSBiW2ldOworICAgICAgU0lHTkVETkVTU18yIHNob3J0 IGFicyA9IGRpZmYgPCAwID8gLWRpZmYgOiBkaWZmOworICAgICAgZXhwZWN0ZWQgKz0gYWJzOwor ICAgICAgZXhwZWN0ZWQgKz0gY1tpXSAqIGRbaV07CisgICAgICBleHBlY3RlZCArPSBlW2ldOwor ICAgIH0KKyAgaWYgKGYgKDB4MTIzNDUsIGEsIGIsIGMsIGQsIGUpICE9IGV4cGVjdGVkKQorICAg IF9fYnVpbHRpbl9hYm9ydCAoKTsKK30KKworLyogeyBkZy1maW5hbCB7IHNjYW4tdHJlZS1kdW1w ICJ2ZWN0b3JpemluZyBzdGF0ZW1lbnQ6IFxcUysgPSBTQURfRVhQUiIgInZlY3QiIHsgdGFyZ2V0 IHZlY3RfdWRvdF9xaSB9IH0gfSAqLworLyogeyBkZy1maW5hbCB7IHNjYW4tdHJlZS1kdW1wICJ2 ZWN0b3JpemluZyBzdGF0ZW1lbnQ6IFxcUysgPSBET1RfUFJPRF9FWFBSIiAidmVjdCIgeyB0YXJn ZXQgdmVjdF9zZG90X2hpIH0gfSB9ICovCmRpZmYgLS1naXQgYS9nY2MvdGVzdHN1aXRlL2djYy5k Zy92ZWN0L3ZlY3QtcmVkdWMtZG90LXNscC0xLmMgYi9nY2MvdGVzdHN1aXRlL2djYy5kZy92ZWN0 L3ZlY3QtcmVkdWMtZG90LXNscC0xLmMKbmV3IGZpbGUgbW9kZSAxMDA2NDQKaW5kZXggMDAwMDAw MDAwMDAuLjUxZWY0ZWFhZWQ4Ci0tLSAvZGV2L251bGwKKysrIGIvZ2NjL3Rlc3RzdWl0ZS9nY2Mu ZGcvdmVjdC92ZWN0LXJlZHVjLWRvdC1zbHAtMS5jCkBAIC0wLDAgKzEsOTcgQEAKKy8qIERpc2Fi bGluZyBlcGlsb2d1ZXMgdW50aWwgd2UgZmluZCBhIGJldHRlciB3YXkgdG8gZGVhbCB3aXRoIHNj YW5zLiAgKi8KKy8qIHsgZGctYWRkaXRpb25hbC1vcHRpb25zICItLXBhcmFtIHZlY3QtZXBpbG9n dWVzLW5vbWFzaz0wIiB9ICovCisvKiB7IGRnLXJlcXVpcmUtZWZmZWN0aXZlLXRhcmdldCB2ZWN0 X2ludCB9ICovCisvKiB7IGRnLXJlcXVpcmUtZWZmZWN0aXZlLXRhcmdldCBhcm1fdjhfMmFfZG90 cHJvZF9uZW9uX2h3IHsgdGFyZ2V0IHsgYWFyY2g2NCotKi0qIHx8IGFybSotKi0qIH0gfSB9ICov CisvKiB7IGRnLWFkZC1vcHRpb25zIGFybV92OF8yYV9kb3Rwcm9kX25lb24gfSAgKi8KKworI2lu Y2x1ZGUgInRyZWUtdmVjdC5oIgorCisjZGVmaW5lIE4gNTAKKworI2lmbmRlZiBTSUdORURORVNT XzEKKyNkZWZpbmUgU0lHTkVETkVTU18xIHNpZ25lZAorI2RlZmluZSBTSUdORURORVNTXzIgc2ln bmVkCisjZW5kaWYKKworU0lHTkVETkVTU18xIGludCBfX2F0dHJpYnV0ZV9fICgobm9pcGEpKQor ZiAoU0lHTkVETkVTU18xIGludCByZXMsCisgICBTSUdORURORVNTXzIgY2hhciAqYSwKKyAgIFNJ R05FRE5FU1NfMiBjaGFyICpiLAorICAgaW50IHN0ZXAsIGludCBuKQoreworICBmb3IgKGludCBp ID0gMDsgaSA8IG47IGkrKykKKyAgICB7CisgICAgICByZXMgKz0gYVswXSAqIGJbMF07CisgICAg ICByZXMgKz0gYVsxXSAqIGJbMV07CisgICAgICByZXMgKz0gYVsyXSAqIGJbMl07CisgICAgICBy ZXMgKz0gYVszXSAqIGJbM107CisgICAgICByZXMgKz0gYVs0XSAqIGJbNF07CisgICAgICByZXMg Kz0gYVs1XSAqIGJbNV07CisgICAgICByZXMgKz0gYVs2XSAqIGJbNl07CisgICAgICByZXMgKz0g YVs3XSAqIGJbN107CisgICAgICByZXMgKz0gYVs4XSAqIGJbOF07CisgICAgICByZXMgKz0gYVs5 XSAqIGJbOV07CisgICAgICByZXMgKz0gYVsxMF0gKiBiWzEwXTsKKyAgICAgIHJlcyArPSBhWzEx XSAqIGJbMTFdOworICAgICAgcmVzICs9IGFbMTJdICogYlsxMl07CisgICAgICByZXMgKz0gYVsx M10gKiBiWzEzXTsKKyAgICAgIHJlcyArPSBhWzE0XSAqIGJbMTRdOworICAgICAgcmVzICs9IGFb MTVdICogYlsxNV07CisKKyAgICAgIGEgKz0gc3RlcDsKKyAgICAgIGIgKz0gc3RlcDsKKyAgICB9 CisKKyAgcmV0dXJuIHJlczsKK30KKworI2RlZmluZSBCQVNFICgoU0lHTkVETkVTU18yIGludCkg LTEgPCAwID8gLTEyNiA6IDQpCisjZGVmaW5lIE9GRlNFVCAyMAorCitpbnQKK21haW4gKHZvaWQp Cit7CisgIGNoZWNrX3ZlY3QgKCk7CisKKyAgU0lHTkVETkVTU18yIGNoYXIgYVsxMDBdLCBiWzEw MF07CisgIGludCBleHBlY3RlZCA9IDB4MTIzNDU7CisgIGludCBzdGVwID0gMTY7CisgIGludCBu ID0gMjsKKyAgaW50IHQgPSAwOworCisgIGZvciAoaW50IGkgPSAwOyBpIDwgc2l6ZW9mIChhKSAv IHNpemVvZiAoYVswXSk7ICsraSkKKyAgICB7CisgICAgICBhW2ldID0gQkFTRSArIGkgKiA1Owor ICAgICAgYltpXSA9IEJBU0UgKyBPRkZTRVQgKyBpICogNDsKKyAgICAgIGFzbSB2b2xhdGlsZSAo IiIgOjo6ICJtZW1vcnkiKTsKKyAgICB9CisKKyAgZm9yIChpbnQgaSA9IDA7IGkgPCBuOyBpKysp CisgICAgeworICAgICAgYXNtIHZvbGF0aWxlICgiIiA6OjogIm1lbW9yeSIpOworICAgICAgZXhw ZWN0ZWQgKz0gYVt0ICsgMF0gKiBiW3QgKyAwXTsKKyAgICAgIGV4cGVjdGVkICs9IGFbdCArIDFd ICogYlt0ICsgMV07CisgICAgICBleHBlY3RlZCArPSBhW3QgKyAyXSAqIGJbdCArIDJdOworICAg ICAgZXhwZWN0ZWQgKz0gYVt0ICsgM10gKiBiW3QgKyAzXTsKKyAgICAgIGV4cGVjdGVkICs9IGFb dCArIDRdICogYlt0ICsgNF07CisgICAgICBleHBlY3RlZCArPSBhW3QgKyA1XSAqIGJbdCArIDVd OworICAgICAgZXhwZWN0ZWQgKz0gYVt0ICsgNl0gKiBiW3QgKyA2XTsKKyAgICAgIGV4cGVjdGVk ICs9IGFbdCArIDddICogYlt0ICsgN107CisgICAgICBleHBlY3RlZCArPSBhW3QgKyA4XSAqIGJb dCArIDhdOworICAgICAgZXhwZWN0ZWQgKz0gYVt0ICsgOV0gKiBiW3QgKyA5XTsKKyAgICAgIGV4 cGVjdGVkICs9IGFbdCArIDEwXSAqIGJbdCArIDEwXTsKKyAgICAgIGV4cGVjdGVkICs9IGFbdCAr IDExXSAqIGJbdCArIDExXTsKKyAgICAgIGV4cGVjdGVkICs9IGFbdCArIDEyXSAqIGJbdCArIDEy XTsKKyAgICAgIGV4cGVjdGVkICs9IGFbdCArIDEzXSAqIGJbdCArIDEzXTsKKyAgICAgIGV4cGVj dGVkICs9IGFbdCArIDE0XSAqIGJbdCArIDE0XTsKKyAgICAgIGV4cGVjdGVkICs9IGFbdCArIDE1 XSAqIGJbdCArIDE1XTsKKyAgICAgIHQgKz0gc3RlcDsKKyAgICB9CisKKyAgaWYgKGYgKDB4MTIz NDUsIGEsIGIsIHN0ZXAsIG4pICE9IGV4cGVjdGVkKQorICAgIF9fYnVpbHRpbl9hYm9ydCAoKTsK K30KKworLyogeyBkZy1maW5hbCB7IHNjYW4tdHJlZS1kdW1wICJ2ZWN0X3JlY29nX2RvdF9wcm9k X3BhdHRlcm46IGRldGVjdGVkIiAidmVjdCIgfSB9ICovCisvKiB7IGRnLWZpbmFsIHsgc2Nhbi10 cmVlLWR1bXAgInZlY3Rvcml6aW5nIHN0bXRzIHVzaW5nIFNMUCIgInZlY3QiIH0gfSAqLworLyog eyBkZy1maW5hbCB7IHNjYW4tdHJlZS1kdW1wLXRpbWVzICJ2ZWN0b3JpemluZyBzdGF0ZW1lbnQ6 IFxcUysgPSBET1RfUFJPRF9FWFBSIiAxNiAidmVjdCIgfSB9ICovCmRpZmYgLS1naXQgYS9nY2Mv dGVzdHN1aXRlL2djYy5kZy92ZWN0L3ZlY3QtcmVkdWMtZG90LXNscC0yLmMgYi9nY2MvdGVzdHN1 aXRlL2djYy5kZy92ZWN0L3ZlY3QtcmVkdWMtZG90LXNscC0yLmMKbmV3IGZpbGUgbW9kZSAxMDA2 NDQKaW5kZXggMDAwMDAwMDAwMDAuLjE1MzI4MzNjM2FlCi0tLSAvZGV2L251bGwKKysrIGIvZ2Nj L3Rlc3RzdWl0ZS9nY2MuZGcvdmVjdC92ZWN0LXJlZHVjLWRvdC1zbHAtMi5jCkBAIC0wLDAgKzEs ODEgQEAKKy8qIERpc2FibGluZyBlcGlsb2d1ZXMgdW50aWwgd2UgZmluZCBhIGJldHRlciB3YXkg dG8gZGVhbCB3aXRoIHNjYW5zLiAgKi8KKy8qIHsgZGctYWRkaXRpb25hbC1vcHRpb25zICItLXBh cmFtIHZlY3QtZXBpbG9ndWVzLW5vbWFzaz0wIiB9ICovCisvKiB7IGRnLXJlcXVpcmUtZWZmZWN0 aXZlLXRhcmdldCB2ZWN0X2ludCB9ICovCisvKiB7IGRnLXJlcXVpcmUtZWZmZWN0aXZlLXRhcmdl dCBhcm1fdjhfMmFfZG90cHJvZF9uZW9uX2h3IHsgdGFyZ2V0IHsgYWFyY2g2NCotKi0qIHx8IGFy bSotKi0qIH0gfSB9ICovCisvKiB7IGRnLWFkZC1vcHRpb25zIGFybV92OF8yYV9kb3Rwcm9kX25l b24gfSAgKi8KKworI2luY2x1ZGUgInRyZWUtdmVjdC5oIgorCisjZGVmaW5lIE4gNTAKKworI2lm bmRlZiBTSUdORURORVNTXzEKKyNkZWZpbmUgU0lHTkVETkVTU18xIHNpZ25lZAorI2RlZmluZSBT SUdORURORVNTXzIgc2lnbmVkCisjZW5kaWYKKworU0lHTkVETkVTU18xIGludCBfX2F0dHJpYnV0 ZV9fICgobm9pcGEpKQorZiAoU0lHTkVETkVTU18xIGludCByZXMsCisgICBTSUdORURORVNTXzIg c2hvcnQgKmEsCisgICBTSUdORURORVNTXzIgc2hvcnQgKmIsCisgICBpbnQgc3RlcCwgaW50IG4p Cit7CisgIGZvciAoaW50IGkgPSAwOyBpIDwgbjsgaSsrKQorICAgIHsKKyAgICAgIHJlcyArPSBh WzBdICogYlswXTsKKyAgICAgIHJlcyArPSBhWzFdICogYlsxXTsKKyAgICAgIHJlcyArPSBhWzJd ICogYlsyXTsKKyAgICAgIHJlcyArPSBhWzNdICogYlszXTsKKyAgICAgIHJlcyArPSBhWzRdICog Yls0XTsKKyAgICAgIHJlcyArPSBhWzVdICogYls1XTsKKyAgICAgIHJlcyArPSBhWzZdICogYls2 XTsKKyAgICAgIHJlcyArPSBhWzddICogYls3XTsKKworICAgICAgYSArPSBzdGVwOworICAgICAg YiArPSBzdGVwOworICAgIH0KKworICByZXR1cm4gcmVzOworfQorCisjZGVmaW5lIEJBU0UgKChT SUdORURORVNTXzIgaW50KSAtMSA8IDAgPyAtMTAyNiA6IDM3MykKKyNkZWZpbmUgT0ZGU0VUIDIw CisKK2ludAorbWFpbiAodm9pZCkKK3sKKyAgY2hlY2tfdmVjdCAoKTsKKworICBTSUdORURORVNT XzIgc2hvcnQgYVsxMDBdLCBiWzEwMF07CisgIGludCBleHBlY3RlZCA9IDB4MTIzNDU7CisgIGlu dCBzdGVwID0gODsKKyAgaW50IG4gPSAyOworICBpbnQgdCA9IDA7CisKKyAgZm9yIChpbnQgaSA9 IDA7IGkgPCBzaXplb2YgKGEpIC8gc2l6ZW9mIChhWzBdKTsgKytpKQorICAgIHsKKyAgICAgIGFb aV0gPSBCQVNFICsgaSAqIDU7CisgICAgICBiW2ldID0gQkFTRSArIE9GRlNFVCArIGkgKiA0Owor ICAgICAgYXNtIHZvbGF0aWxlICgiIiA6OjogIm1lbW9yeSIpOworICAgIH0KKworICBmb3IgKGlu dCBpID0gMDsgaSA8IG47IGkrKykKKyAgICB7CisgICAgICBhc20gdm9sYXRpbGUgKCIiIDo6OiAi bWVtb3J5Iik7CisgICAgICBleHBlY3RlZCArPSBhW3QgKyAwXSAqIGJbdCArIDBdOworICAgICAg ZXhwZWN0ZWQgKz0gYVt0ICsgMV0gKiBiW3QgKyAxXTsKKyAgICAgIGV4cGVjdGVkICs9IGFbdCAr IDJdICogYlt0ICsgMl07CisgICAgICBleHBlY3RlZCArPSBhW3QgKyAzXSAqIGJbdCArIDNdOwor ICAgICAgZXhwZWN0ZWQgKz0gYVt0ICsgNF0gKiBiW3QgKyA0XTsKKyAgICAgIGV4cGVjdGVkICs9 IGFbdCArIDVdICogYlt0ICsgNV07CisgICAgICBleHBlY3RlZCArPSBhW3QgKyA2XSAqIGJbdCAr IDZdOworICAgICAgZXhwZWN0ZWQgKz0gYVt0ICsgN10gKiBiW3QgKyA3XTsKKyAgICAgIHQgKz0g c3RlcDsKKyAgICB9CisKKyAgaWYgKGYgKDB4MTIzNDUsIGEsIGIsIHN0ZXAsIG4pICE9IGV4cGVj dGVkKQorICAgIF9fYnVpbHRpbl9hYm9ydCAoKTsKK30KKworLyogeyBkZy1maW5hbCB7IHNjYW4t dHJlZS1kdW1wICJ2ZWN0X3JlY29nX2RvdF9wcm9kX3BhdHRlcm46IGRldGVjdGVkIiAidmVjdCIg fSB9ICovCisvKiB7IGRnLWZpbmFsIHsgc2Nhbi10cmVlLWR1bXAgInZlY3Rvcml6aW5nIHN0bXRz IHVzaW5nIFNMUCIgInZlY3QiIH0gfSAqLworLyogeyBkZy1maW5hbCB7IHNjYW4tdHJlZS1kdW1w LXRpbWVzICJ2ZWN0b3JpemluZyBzdGF0ZW1lbnQ6IFxcUysgPSBET1RfUFJPRF9FWFBSIiA4ICJ2 ZWN0IiAgeyB0YXJnZXQgdmVjdF9zZG90X2hpIH0gfSB9ICovCmRpZmYgLS1naXQgYS9nY2MvdHJl ZS12ZWN0LWxvb3AuY2MgYi9nY2MvdHJlZS12ZWN0LWxvb3AuY2MKaW5kZXggODNjMDU0NGI2YWEu LjkyZDA3ZGYyODkwIDEwMDY0NAotLS0gYS9nY2MvdHJlZS12ZWN0LWxvb3AuY2MKKysrIGIvZ2Nj L3RyZWUtdmVjdC1sb29wLmNjCkBAIC01MjcwLDggKzUyNzAsNyBAQCBoYXZlX3dob2xlX3ZlY3Rv cl9zaGlmdCAobWFjaGluZV9tb2RlIG1vZGUpCiAgICBTZWUgdmVjdF9lbXVsYXRlX21peGVkX2Rv dF9wcm9kIGZvciB0aGUgYWN0dWFsIHNlcXVlbmNlIHVzZWQuICAqLwogCiBzdGF0aWMgYm9vbAot dmVjdF9pc19lbXVsYXRlZF9taXhlZF9kb3RfcHJvZCAobG9vcF92ZWNfaW5mbyBsb29wX3ZpbmZv LAotCQkJCSBzdG10X3ZlY19pbmZvIHN0bXRfaW5mbykKK3ZlY3RfaXNfZW11bGF0ZWRfbWl4ZWRf ZG90X3Byb2QgKHN0bXRfdmVjX2luZm8gc3RtdF9pbmZvKQogewogICBnYXNzaWduICphc3NpZ24g PSBkeW5fY2FzdDxnYXNzaWduICo+IChzdG10X2luZm8tPnN0bXQpOwogICBpZiAoIWFzc2lnbiB8 fCBnaW1wbGVfYXNzaWduX3Joc19jb2RlIChhc3NpZ24pICE9IERPVF9QUk9EX0VYUFIpCkBAIC01 MjgyLDEwICs1MjgxLDkgQEAgdmVjdF9pc19lbXVsYXRlZF9taXhlZF9kb3RfcHJvZCAobG9vcF92 ZWNfaW5mbyBsb29wX3ZpbmZvLAogICBpZiAoVFlQRV9TSUdOIChUUkVFX1RZUEUgKHJoczEpKSA9 PSBUWVBFX1NJR04gKFRSRUVfVFlQRSAocmhzMikpKQogICAgIHJldHVybiBmYWxzZTsKIAotICBz dG10X3ZlY19pbmZvIHJlZHVjX2luZm8gPSBpbmZvX2Zvcl9yZWR1Y3Rpb24gKGxvb3BfdmluZm8s IHN0bXRfaW5mbyk7Ci0gIGdjY19hc3NlcnQgKHJlZHVjX2luZm8tPmlzX3JlZHVjX2luZm8pOwor ICBnY2NfYXNzZXJ0IChTVE1UX1ZJTkZPX1JFRFVDX1ZFQ1RZUEVfSU4gKHN0bXRfaW5mbykpOwog ICByZXR1cm4gIWRpcmVjdGx5X3N1cHBvcnRlZF9wIChET1RfUFJPRF9FWFBSLAotCQkJCVNUTVRf VklORk9fUkVEVUNfVkVDVFlQRV9JTiAocmVkdWNfaW5mbyksCisJCQkJU1RNVF9WSU5GT19SRURV Q19WRUNUWVBFX0lOIChzdG10X2luZm8pLAogCQkJCW9wdGFiX3ZlY3Rvcl9taXhlZF9zaWduKTsK IH0KIApAQCAtNTMyNCw4ICs1MzIyLDYgQEAgdmVjdF9tb2RlbF9yZWR1Y3Rpb25fY29zdCAobG9v cF92ZWNfaW5mbyBsb29wX3ZpbmZvLAogICBpZiAoIWdpbXBsZV9leHRyYWN0X29wIChvcmlnX3N0 bXRfaW5mby0+c3RtdCwgJm9wKSkKICAgICBnY2NfdW5yZWFjaGFibGUgKCk7CiAKLSAgYm9vbCBl bXVsYXRlZF9taXhlZF9kb3RfcHJvZAotICAgID0gdmVjdF9pc19lbXVsYXRlZF9taXhlZF9kb3Rf cHJvZCAobG9vcF92aW5mbywgc3RtdF9pbmZvKTsKICAgaWYgKHJlZHVjdGlvbl90eXBlID09IEVY VFJBQ1RfTEFTVF9SRURVQ1RJT04pCiAgICAgLyogTm8gZXh0cmEgaW5zdHJ1Y3Rpb25zIGFyZSBu ZWVkZWQgaW4gdGhlIHByb2xvZ3VlLiAgVGhlIGxvb3AgYm9keQogICAgICAgIG9wZXJhdGlvbnMg YXJlIGNvc3RlZCBpbiB2ZWN0b3JpemFibGVfY29uZGl0aW9uLiAgKi8KQEAgLTUzNjAsMTIgKzUz NTYsOCBAQCB2ZWN0X21vZGVsX3JlZHVjdGlvbl9jb3N0IChsb29wX3ZlY19pbmZvIGxvb3Bfdmlu Zm8sCiAJICAgaW5pdGlhbCByZXN1bHQgb2YgdGhlIGRhdGEgcmVkdWN0aW9uLCBpbml0aWFsIHZh bHVlIG9mIHRoZSBpbmRleAogCSAgIHJlZHVjdGlvbi4gICovCiAJcHJvbG9ndWVfc3RtdHMgPSA0 OwotICAgICAgZWxzZSBpZiAoZW11bGF0ZWRfbWl4ZWRfZG90X3Byb2QpCi0JLyogV2UgbmVlZCB0 aGUgaW5pdGlhbCByZWR1Y3Rpb24gdmFsdWUgYW5kIHR3byBpbnZhcmlhbnRzOgotCSAgIG9uZSB0 aGF0IGNvbnRhaW5zIHRoZSBtaW5pbXVtIHNpZ25lZCB2YWx1ZSBhbmQgb25lIHRoYXQKLQkgICBj b250YWlucyBoYWxmIG9mIGl0cyBuZWdhdGl2ZS4gICovCi0JcHJvbG9ndWVfc3RtdHMgPSAzOwog ICAgICAgZWxzZQorCS8qIFdlIG5lZWQgdGhlIGluaXRpYWwgcmVkdWN0aW9uIHZhbHVlLiAgKi8K IAlwcm9sb2d1ZV9zdG10cyA9IDE7CiAgICAgICBwcm9sb2d1ZV9jb3N0ICs9IHJlY29yZF9zdG10 X2Nvc3QgKGNvc3RfdmVjLCBwcm9sb2d1ZV9zdG10cywKIAkJCQkJIHNjYWxhcl90b192ZWMsIHN0 bXRfaW5mbywgMCwKQEAgLTczODQsNiArNzM3NiwyNDQgQEAgYnVpbGRfdmVjdF9jb25kX2V4cHIg KGNvZGVfaGVscGVyIGNvZGUsIHRyZWUgdm9wWzNdLCB0cmVlIG1hc2ssCiAgICAgfQogfQogCisv KiBHaXZlbiBhbiBvcGVyYXRpb24gd2l0aCBDT0RFIGluIGxvb3AgcmVkdWN0aW9uIHBhdGggd2hv c2UgcmVkdWN0aW9uIFBISSBpcworICAgc3BlY2lmaWVkIGJ5IFJFRFVDX0lORk8sIHRoZSBvcGVy YXRpb24gaGFzIFRZUEUgb2Ygc2NhbGFyIHJlc3VsdCwgYW5kIGl0cworICAgaW5wdXQgdmVjdHlw ZSBpcyByZXByZXNlbnRlZCBieSBWRUNUWVBFX0lOLiBUaGUgdmVjdHlwZSBvZiB2ZWN0b3JpemVk IHJlc3VsdAorICAgbWF5IGJlIGRpZmZlcmVudCBmcm9tIFZFQ1RZUEVfSU4sIGVpdGhlciBpbiBi YXNlIHR5cGUgb3IgdmVjdHlwZSBsYW5lcywKKyAgIGxhbmUtcmVkdWNpbmcgb3BlcmF0aW9uIGlz IHRoZSBjYXNlLiAgVGhpcyBmdW5jdGlvbiBjaGVjayBpZiBpdCBpcyBwb3NzaWJsZSwKKyAgIGFu ZCBob3cgdG8gcGVyZm9ybSBwYXJ0aWFsIHZlY3Rvcml6YXRpb24gb24gdGhlIG9wZXJhdGlvbiBp biB0aGUgY29udGV4dAorICAgb2YgTE9PUF9WSU5GTy4gICovCisKK3N0YXRpYyB2b2lkCit2ZWN0 X3JlZHVjdGlvbl91c2VfcGFydGlhbF92ZWN0b3IgKGxvb3BfdmVjX2luZm8gbG9vcF92aW5mbywK KwkJCQkgICBzdG10X3ZlY19pbmZvIHJlZHVjX2luZm8sCisJCQkJICAgc2xwX3RyZWUgc2xwX25v ZGUsIGNvZGVfaGVscGVyIGNvZGUsCisJCQkJICAgdHJlZSB0eXBlLCB0cmVlIHZlY3R5cGVfaW4p Cit7CisgIGlmICghTE9PUF9WSU5GT19DQU5fVVNFX1BBUlRJQUxfVkVDVE9SU19QIChsb29wX3Zp bmZvKSkKKyAgICByZXR1cm47CisKKyAgZW51bSB2ZWN0X3JlZHVjdGlvbl90eXBlIHJlZHVjX3R5 cGUgPSBTVE1UX1ZJTkZPX1JFRFVDX1RZUEUgKHJlZHVjX2luZm8pOworICBpbnRlcm5hbF9mbiBy ZWR1Y19mbiA9IFNUTVRfVklORk9fUkVEVUNfRk4gKHJlZHVjX2luZm8pOworICBpbnRlcm5hbF9m biBjb25kX2ZuID0gZ2V0X2NvbmRpdGlvbmFsX2ludGVybmFsX2ZuIChjb2RlLCB0eXBlKTsKKwor ICBpZiAocmVkdWNfdHlwZSAhPSBGT0xEX0xFRlRfUkVEVUNUSU9OCisgICAgICAmJiAhdXNlX21h c2tfYnlfY29uZF9leHByX3AgKGNvZGUsIGNvbmRfZm4sIHZlY3R5cGVfaW4pCisgICAgICAmJiAo Y29uZF9mbiA9PSBJRk5fTEFTVAorCSAgfHwgIWRpcmVjdF9pbnRlcm5hbF9mbl9zdXBwb3J0ZWRf cCAoY29uZF9mbiwgdmVjdHlwZV9pbiwKKwkJCQkJICAgICAgT1BUSU1JWkVfRk9SX1NQRUVEKSkp CisgICAgeworICAgICAgaWYgKGR1bXBfZW5hYmxlZF9wICgpKQorCWR1bXBfcHJpbnRmX2xvYyAo TVNHX01JU1NFRF9PUFRJTUlaQVRJT04sIHZlY3RfbG9jYXRpb24sCisJCQkgImNhbid0IG9wZXJh dGUgb24gcGFydGlhbCB2ZWN0b3JzIGJlY2F1c2UiCisJCQkgIiBubyBjb25kaXRpb25hbCBvcGVy YXRpb24gaXMgYXZhaWxhYmxlLlxuIik7CisgICAgICBMT09QX1ZJTkZPX0NBTl9VU0VfUEFSVElB TF9WRUNUT1JTX1AgKGxvb3BfdmluZm8pID0gZmFsc2U7CisgICAgfQorICBlbHNlIGlmIChyZWR1 Y190eXBlID09IEZPTERfTEVGVF9SRURVQ1RJT04KKwkgICAmJiByZWR1Y19mbiA9PSBJRk5fTEFT VAorCSAgICYmICFleHBhbmRfdmVjX2NvbmRfZXhwcl9wICh2ZWN0eXBlX2luLCB0cnV0aF90eXBl X2ZvciAodmVjdHlwZV9pbiksCisJCQkJICAgICAgIFNTQV9OQU1FKSkKKyAgICB7CisgICAgICBp ZiAoZHVtcF9lbmFibGVkX3AgKCkpCisJZHVtcF9wcmludGZfbG9jIChNU0dfTUlTU0VEX09QVElN SVpBVElPTiwgdmVjdF9sb2NhdGlvbiwKKwkJCSJjYW4ndCBvcGVyYXRlIG9uIHBhcnRpYWwgdmVj dG9ycyBiZWNhdXNlIgorCQkJIiBubyBjb25kaXRpb25hbCBvcGVyYXRpb24gaXMgYXZhaWxhYmxl LlxuIik7CisgICAgICBMT09QX1ZJTkZPX0NBTl9VU0VfUEFSVElBTF9WRUNUT1JTX1AgKGxvb3Bf dmluZm8pID0gZmFsc2U7CisgICAgfQorICBlbHNlIGlmIChyZWR1Y190eXBlID09IEZPTERfTEVG VF9SRURVQ1RJT04KKwkgICAmJiBpbnRlcm5hbF9mbl9tYXNrX2luZGV4IChyZWR1Y19mbikgPT0g LTEKKwkgICAmJiBGTE9BVF9UWVBFX1AgKHZlY3R5cGVfaW4pCisJICAgJiYgSE9OT1JfU0lHTl9E RVBFTkRFTlRfUk9VTkRJTkcgKHZlY3R5cGVfaW4pKQorICAgIHsKKyAgICAgIGlmIChkdW1wX2Vu YWJsZWRfcCAoKSkKKwlkdW1wX3ByaW50Zl9sb2MgKE1TR19NSVNTRURfT1BUSU1JWkFUSU9OLCB2 ZWN0X2xvY2F0aW9uLAorCQkJICJjYW4ndCBvcGVyYXRlIG9uIHBhcnRpYWwgdmVjdG9ycyBiZWNh dXNlIgorCQkJICIgc2lnbmVkIHplcm9zIGNhbm5vdCBiZSBwcmVzZXJ2ZWQuXG4iKTsKKyAgICAg IExPT1BfVklORk9fQ0FOX1VTRV9QQVJUSUFMX1ZFQ1RPUlNfUCAobG9vcF92aW5mbykgPSBmYWxz ZTsKKyAgICB9CisgIGVsc2UKKyAgICB7CisgICAgICBpbnRlcm5hbF9mbiBtYXNrX3JlZHVjX2Zu CisJCQk9IGdldF9tYXNrZWRfcmVkdWN0aW9uX2ZuIChyZWR1Y19mbiwgdmVjdHlwZV9pbik7Cisg ICAgICB2ZWNfbG9vcF9tYXNrcyAqbWFza3MgPSAmTE9PUF9WSU5GT19NQVNLUyAobG9vcF92aW5m byk7CisgICAgICB2ZWNfbG9vcF9sZW5zICpsZW5zID0gJkxPT1BfVklORk9fTEVOUyAobG9vcF92 aW5mbyk7CisgICAgICB1bnNpZ25lZCBudmVjdG9yczsKKworICAgICAgaWYgKHNscF9ub2RlKQor CW52ZWN0b3JzID0gU0xQX1RSRUVfTlVNQkVSX09GX1ZFQ19TVE1UUyAoc2xwX25vZGUpOworICAg ICAgZWxzZQorCW52ZWN0b3JzID0gdmVjdF9nZXRfbnVtX2NvcGllcyAobG9vcF92aW5mbywgdmVj dHlwZV9pbik7CisKKyAgICAgIGlmIChtYXNrX3JlZHVjX2ZuID09IElGTl9NQVNLX0xFTl9GT0xE X0xFRlRfUExVUykKKwl2ZWN0X3JlY29yZF9sb29wX2xlbiAobG9vcF92aW5mbywgbGVucywgbnZl Y3RvcnMsIHZlY3R5cGVfaW4sIDEpOworICAgICAgZWxzZQorCXZlY3RfcmVjb3JkX2xvb3BfbWFz ayAobG9vcF92aW5mbywgbWFza3MsIG52ZWN0b3JzLCB2ZWN0eXBlX2luLCBOVUxMKTsKKyAgICB9 Cit9CisKKy8qIENoZWNrIGlmIFNUTVRfSU5GTyBpcyBhIGxhbmUtcmVkdWNpbmcgb3BlcmF0aW9u IHRoYXQgY2FuIGJlIHZlY3Rvcml6ZWQgaW4KKyAgIHRoZSBjb250ZXh0IG9mIExPT1BfVklORk8s IGFuZCB2ZWN0b3IgY29zdCB3aWxsIGJlIHJlY29yZGVkIGluIENPU1RfVkVDLgorICAgTm93IHRo ZXJlIGFyZSB0aHJlZSBzdWNoIGtpbmRzIG9mIG9wZXJhdGlvbnM6IGRvdC1wcm9kL3dpZGVuLXN1 bS9zYWQKKyAgIChzdW0tb2YtYWJzb2x1dGUtZGlmZmVyZW5jZXMpLgorCisgICBGb3IgYSBsYW5l LXJlZHVjaW5nIG9wZXJhdGlvbiwgdGhlIGxvb3AgcmVkdWN0aW9uIHBhdGggdGhhdCBpdCBsaWVz IGluLAorICAgbWF5IGNvbnRhaW4gbm9ybWFsIG9wZXJhdGlvbiwgb3Igb3RoZXIgbGFuZS1yZWR1 Y2luZyBvcGVyYXRpb24gb2YgZGlmZmVyZW50CisgICBpbnB1dCB0eXBlIHNpemUsIGFuIGV4YW1w bGUgYXM6CisKKyAgICAgaW50IHN1bSA9IDA7CisgICAgIGZvciAoaSkKKyAgICAgICB7CisgICAg ICAgICAuLi4KKyAgICAgICAgIHN1bSArPSBkMFtpXSAqIGQxW2ldOyAgICAgICAvLyBkb3QtcHJv ZCA8dmVjdG9yKDE2KSBjaGFyPgorICAgICAgICAgc3VtICs9IHdbaV07ICAgICAgICAgICAgICAg IC8vIHdpZGVuLXN1bSA8dmVjdG9yKDE2KSBjaGFyPgorICAgICAgICAgc3VtICs9IGFicyhzMFtp XSAtIHMxW2ldKTsgIC8vIHNhZCA8dmVjdG9yKDgpIHNob3J0PgorICAgICAgICAgc3VtICs9IG5b aV07ICAgICAgICAgICAgICAgIC8vIG5vcm1hbCA8dmVjdG9yKDQpIGludD4KKyAgICAgICAgIC4u LgorICAgICAgIH0KKworICAgVmVjdG9yaXphdGlvbiBmYWN0b3IgaXMgZXNzZW50aWFsbHkgZGV0 ZXJtaW5lZCBieSBvcGVyYXRpb24gd2hvc2UgaW5wdXQKKyAgIHZlY3R5cGUgaGFzIHRoZSBtb3N0 IGxhbmVzICgidmVjdG9yKDE2KSBjaGFyIiBpbiB0aGUgZXhhbXBsZSksIHdoaWxlIHdlCisgICBu ZWVkIHRvIGNob29zZSBpbnB1dCB2ZWN0eXBlIHdpdGggdGhlIGxlYXN0IGxhbmVzICgidmVjdG9y KDQpIGludCIgaW4gdGhlCisgICBleGFtcGxlKSBmb3IgdGhlIHJlZHVjdGlvbiBQSEkgc3RhdGVt ZW50LiAgKi8KKworYm9vbAordmVjdG9yaXphYmxlX2xhbmVfcmVkdWNpbmcgKGxvb3BfdmVjX2lu Zm8gbG9vcF92aW5mbywgc3RtdF92ZWNfaW5mbyBzdG10X2luZm8sCisJCQkgICAgc2xwX3RyZWUg c2xwX25vZGUsIHN0bXRfdmVjdG9yX2Zvcl9jb3N0ICpjb3N0X3ZlYykKK3sKKyAgZ2Fzc2lnbiAq c3RtdCA9IGR5bl9jYXN0IDxnYXNzaWduICo+IChzdG10X2luZm8tPnN0bXQpOworICBpZiAoIXN0 bXQpCisgICAgcmV0dXJuIGZhbHNlOworCisgIGVudW0gdHJlZV9jb2RlIGNvZGUgPSBnaW1wbGVf YXNzaWduX3Joc19jb2RlIChzdG10KTsKKworICBpZiAoIWxhbmVfcmVkdWNpbmdfb3BfcCAoY29k ZSkpCisgICAgcmV0dXJuIGZhbHNlOworCisgIHRyZWUgdHlwZSA9IFRSRUVfVFlQRSAoZ2ltcGxl X2Fzc2lnbl9saHMgKHN0bXQpKTsKKworICBpZiAoIUlOVEVHUkFMX1RZUEVfUCAodHlwZSkgJiYg IVNDQUxBUl9GTE9BVF9UWVBFX1AgKHR5cGUpKQorICAgIHJldHVybiBmYWxzZTsKKworICAvKiBE byBub3QgdHJ5IHRvIHZlY3Rvcml6ZSBiaXQtcHJlY2lzaW9uIHJlZHVjdGlvbnMuICAqLworICBp ZiAoIXR5cGVfaGFzX21vZGVfcHJlY2lzaW9uX3AgKHR5cGUpKQorICAgIHJldHVybiBmYWxzZTsK KworICB0cmVlIHZlY3R5cGVfaW4gPSBOVUxMX1RSRUU7CisKKyAgZm9yIChpbnQgaSA9IDA7IGkg PCAoaW50KSBnaW1wbGVfbnVtX29wcyAoc3RtdCkgLSAxOyBpKyspCisgICAgeworICAgICAgc3Rt dF92ZWNfaW5mbyBkZWZfc3RtdF9pbmZvOworICAgICAgc2xwX3RyZWUgc2xwX29wOworICAgICAg dHJlZSBvcDsKKyAgICAgIHRyZWUgdmVjdHlwZTsKKyAgICAgIGVudW0gdmVjdF9kZWZfdHlwZSBk dDsKKworICAgICAgaWYgKCF2ZWN0X2lzX3NpbXBsZV91c2UgKGxvb3BfdmluZm8sIHN0bXRfaW5m bywgc2xwX25vZGUsIGksICZvcCwKKwkJCSAgICAgICAmc2xwX29wLCAmZHQsICZ2ZWN0eXBlLCAm ZGVmX3N0bXRfaW5mbykpCisJeworCSAgaWYgKGR1bXBfZW5hYmxlZF9wICgpKQorCSAgICBkdW1w X3ByaW50Zl9sb2MgKE1TR19NSVNTRURfT1BUSU1JWkFUSU9OLCB2ZWN0X2xvY2F0aW9uLAorCQkJ ICAgICAidXNlIG5vdCBzaW1wbGUuXG4iKTsKKwkgIHJldHVybiBmYWxzZTsKKwl9CisKKyAgICAg IGlmICghdmVjdHlwZSkKKwl7CisJICB2ZWN0eXBlID0gZ2V0X3ZlY3R5cGVfZm9yX3NjYWxhcl90 eXBlIChsb29wX3ZpbmZvLCBUUkVFX1RZUEUgKG9wKSwKKwkJCQkJCSBzbHBfb3ApOworCSAgaWYg KCF2ZWN0eXBlKQorCSAgICByZXR1cm4gZmFsc2U7CisJfQorCisgICAgICBpZiAoc2xwX25vZGUg JiYgIXZlY3RfbWF5YmVfdXBkYXRlX3NscF9vcF92ZWN0eXBlIChzbHBfb3AsIHZlY3R5cGUpKQor CXsKKwkgIGlmIChkdW1wX2VuYWJsZWRfcCAoKSkKKwkgICAgZHVtcF9wcmludGZfbG9jIChNU0df TUlTU0VEX09QVElNSVpBVElPTiwgdmVjdF9sb2NhdGlvbiwKKwkJCSAgICAgImluY29tcGF0aWJs ZSB2ZWN0b3IgdHlwZXMgZm9yIGludmFyaWFudHNcbiIpOworCSAgcmV0dXJuIGZhbHNlOworCX0K KworICAgICAgaWYgKGkgPT0gU1RNVF9WSU5GT19SRURVQ19JRFggKHN0bXRfaW5mbykpCisJY29u dGludWU7CisKKyAgICAgIC8qIFRoZXJlIHNob3VsZCBiZSBhdCBtb3N0IG9uZSBjeWNsZSBkZWYg aW4gdGhlIHN0bXQuICAqLworICAgICAgaWYgKFZFQ1RPUklaQUJMRV9DWUNMRV9ERUYgKGR0KSkK KwlyZXR1cm4gZmFsc2U7CisKKyAgICAgIC8qIFRvIHByb3Blcmx5IGNvbXB1dGUgbmNvcGllcyB3 ZSBhcmUgaW50ZXJlc3RlZCBpbiB0aGUgd2lkZXN0CisJIG5vbi1yZWR1Y3Rpb24gaW5wdXQgdHlw ZSBpbiBjYXNlIHdlJ3JlIGxvb2tpbmcgYXQgYSB3aWRlbmluZworCSBhY2N1bXVsYXRpb24gdGhh dCB3ZSBsYXRlciBoYW5kbGUgaW4gdmVjdCB0cmFuc2Zvcm1hdGlvbi4gICovCisgICAgICBpZiAo IXZlY3R5cGVfaW4KKwkgIHx8IChHRVRfTU9ERV9TSVpFIChTQ0FMQVJfVFlQRV9NT0RFIChUUkVF X1RZUEUgKHZlY3R5cGVfaW4pKSkKKwkgICAgICA8IEdFVF9NT0RFX1NJWkUgKFNDQUxBUl9UWVBF X01PREUgKFRSRUVfVFlQRSAodmVjdHlwZSkpKSkpCisJdmVjdHlwZV9pbiA9IHZlY3R5cGU7Cisg ICAgfQorCisgIFNUTVRfVklORk9fUkVEVUNfVkVDVFlQRV9JTiAoc3RtdF9pbmZvKSA9IHZlY3R5 cGVfaW47CisKKyAgc3RtdF92ZWNfaW5mbyByZWR1Y19pbmZvID0gU1RNVF9WSU5GT19SRURVQ19E RUYgKHZlY3Rfb3JpZ19zdG10IChzdG10X2luZm8pKTsKKworICAvKiBUT0RPOiBTdXBwb3J0IGxh bmUtcmVkdWNpbmcgb3BlcmF0aW9uIHRoYXQgZG9lcyBub3QgZGlyZWN0bHkgcGFydGljaXBhdGUK KyAgICAgaW4gbG9vcCByZWR1Y3Rpb24uICovCisgIGlmICghcmVkdWNfaW5mbyB8fCBTVE1UX1ZJ TkZPX1JFRFVDX0lEWCAoc3RtdF9pbmZvKSA8IDApCisgICAgcmV0dXJuIGZhbHNlOworCisgIC8q IExhbmUtcmVkdWNpbmcgcGF0dGVybiBpbnNpZGUgYW55IGlubmVyIGxvb3Agb2YgTE9PUF9WSU5G TyBpcyBub3QKKyAgICAgcmVjb2dpbml6ZWQuICAqLworICBnY2NfYXNzZXJ0IChTVE1UX1ZJTkZP X0RFRl9UWVBFIChyZWR1Y19pbmZvKSA9PSB2ZWN0X3JlZHVjdGlvbl9kZWYpOworICBnY2NfYXNz ZXJ0IChTVE1UX1ZJTkZPX1JFRFVDX1RZUEUgKHJlZHVjX2luZm8pID09IFRSRUVfQ09ERV9SRURV Q1RJT04pOworCisgIHRyZWUgdnBoaV92ZWN0eXBlX2luID0gU1RNVF9WSU5GT19SRURVQ19WRUNU WVBFX0lOIChyZWR1Y19pbmZvKTsKKworICAvKiBUbyBhY2NvbW1vZGF0ZSBsYW5lLXJlZHVjaW5n IG9wZXJhdGlvbnMgb2YgbWl4ZWQgaW5wdXQgdmVjdHlwZXMsIGNob29zZQorICAgICBpbnB1dCB2 ZWN0eXBlIHdpdGggdGhlIGxlYXN0IGxhbmVzIGZvciB0aGUgcmVkdWN0aW9uIFBISSBzdGF0ZW1l bnQsIHdoaWNoCisgICAgIHdvdWxkIHJlc3VsdCBpbiB0aGUgbW9zdCBuY29waWVzIGZvciB2ZWN0 b3JpemVkIHJlZHVjdGlvbiByZXN1bHRzLiAgKi8KKyAgaWYgKCF2cGhpX3ZlY3R5cGVfaW4KKyAg ICAgIHx8IChHRVRfTU9ERV9TSVpFIChTQ0FMQVJfVFlQRV9NT0RFIChUUkVFX1RZUEUgKHZlY3R5 cGVfaW4pKSkKKwkgID4gR0VUX01PREVfU0laRSAoU0NBTEFSX1RZUEVfTU9ERSAoVFJFRV9UWVBF ICh2cGhpX3ZlY3R5cGVfaW4pKSkpKQorICAgIFNUTVRfVklORk9fUkVEVUNfVkVDVFlQRV9JTiAo cmVkdWNfaW5mbykgPSB2ZWN0eXBlX2luOworCisgIGludCBuY29waWVzX2Zvcl9jb3N0OworCisg IGlmIChzbHBfbm9kZSkKKyAgICB7CisgICAgICAvKiBOb3cgbGFuZS1yZWR1Y2luZyBvcGVyYXRp b25zIGluIGEgc2xwIG5vZGUgc2hvdWxkIG9ubHkgY29tZSBmcm9tCisJIHRoZSBzYW1lIGxvb3Ag cmVkdWN0aW9uIHBhdGguICAqLworICAgICAgZ2NjX2Fzc2VydCAoUkVEVUNfR1JPVVBfRklSU1Rf RUxFTUVOVCAoc3RtdF9pbmZvKSk7CisgICAgICBuY29waWVzX2Zvcl9jb3N0ID0gMTsKKyAgICB9 CisgIGVsc2UKKyAgICB7CisgICAgICBuY29waWVzX2Zvcl9jb3N0ID0gdmVjdF9nZXRfbnVtX2Nv cGllcyAobG9vcF92aW5mbywgdmVjdHlwZV9pbik7CisgICAgICBnY2NfYXNzZXJ0IChuY29waWVz X2Zvcl9jb3N0ID49IDEpOworICAgIH0KKworICBpZiAodmVjdF9pc19lbXVsYXRlZF9taXhlZF9k b3RfcHJvZCAoc3RtdF9pbmZvKSkKKyAgICB7CisgICAgICAvKiBXZSBuZWVkIGV4dHJhIHR3byBp bnZhcmlhbnRzOiBvbmUgdGhhdCBjb250YWlucyB0aGUgbWluaW11bSBzaWduZWQKKwkgdmFsdWUg YW5kIG9uZSB0aGF0IGNvbnRhaW5zIGhhbGYgb2YgaXRzIG5lZ2F0aXZlLiAgKi8KKyAgICAgIGlu dCBwcm9sb2d1ZV9zdG10cyA9IDI7CisgICAgICB1bnNpZ25lZCBjb3N0ID0gcmVjb3JkX3N0bXRf Y29zdCAoY29zdF92ZWMsIHByb2xvZ3VlX3N0bXRzLAorCQkJCQlzY2FsYXJfdG9fdmVjLCBzdG10 X2luZm8sIDAsCisJCQkJCXZlY3RfcHJvbG9ndWUpOworICAgICAgaWYgKGR1bXBfZW5hYmxlZF9w ICgpKQorCWR1bXBfcHJpbnRmIChNU0dfTk9URSwgInZlY3Rvcml6YWJsZV9sYW5lX3JlZHVjaW5n OiAiCisJCSAgICAgImV4dHJhIHByb2xvZ3VlX2Nvc3QgPSAlZCAuXG4iLCBjb3N0KTsKKworICAg ICAgLyogVGhyZWUgZG90LXByb2R1Y3RzIGFuZCBhIHN1YnRyYWN0aW9uLiAgKi8KKyAgICAgIG5j b3BpZXNfZm9yX2Nvc3QgKj0gNDsKKyAgICB9CisKKyAgcmVjb3JkX3N0bXRfY29zdCAoY29zdF92 ZWMsIG5jb3BpZXNfZm9yX2Nvc3QsIHZlY3Rvcl9zdG10LCBzdG10X2luZm8sIDAsCisJCSAgICB2 ZWN0X2JvZHkpOworCisgIHZlY3RfcmVkdWN0aW9uX3VzZV9wYXJ0aWFsX3ZlY3RvciAobG9vcF92 aW5mbywgcmVkdWNfaW5mbywgc2xwX25vZGUsIGNvZGUsCisJCQkJICAgICB0eXBlLCB2ZWN0eXBl X2luKTsKKworICBTVE1UX1ZJTkZPX1RZUEUgKHN0bXRfaW5mbykgPSByZWR1Y192ZWNfaW5mb190 eXBlOworICByZXR1cm4gdHJ1ZTsKK30KKwogLyogRnVuY3Rpb24gdmVjdG9yaXphYmxlX3JlZHVj dGlvbi4KIAogICAgQ2hlY2sgaWYgU1RNVF9JTkZPIHBlcmZvcm1zIGEgcmVkdWN0aW9uIG9wZXJh dGlvbiB0aGF0IGNhbiBiZSB2ZWN0b3JpemVkLgpAQCAtNzQ0OSw3ICs3Njc5LDYgQEAgdmVjdG9y aXphYmxlX3JlZHVjdGlvbiAobG9vcF92ZWNfaW5mbyBsb29wX3ZpbmZvLAogICBib29sIHNpbmds ZV9kZWZ1c2VfY3ljbGUgPSBmYWxzZTsKICAgYm9vbCBuZXN0ZWRfY3ljbGUgPSBmYWxzZTsKICAg Ym9vbCBkb3VibGVfcmVkdWMgPSBmYWxzZTsKLSAgaW50IHZlY19udW07CiAgIHRyZWUgY3JfaW5k ZXhfc2NhbGFyX3R5cGUgPSBOVUxMX1RSRUUsIGNyX2luZGV4X3ZlY3Rvcl90eXBlID0gTlVMTF9U UkVFOwogICB0cmVlIGNvbmRfcmVkdWNfdmFsID0gTlVMTF9UUkVFOwogCkBAIC03NTMwLDYgKzc3 NTksNyBAQCB2ZWN0b3JpemFibGVfcmVkdWN0aW9uIChsb29wX3ZlY19pbmZvIGxvb3BfdmluZm8s CiAJCQkgICAgICAgKGdpbXBsZV9iYiAocmVkdWNfZGVmX3BoaSktPmxvb3BfZmF0aGVyKSk7CiAg IHVuc2lnbmVkIHJlZHVjX2NoYWluX2xlbmd0aCA9IDA7CiAgIGJvb2wgb25seV9zbHBfcmVkdWNf Y2hhaW4gPSB0cnVlOworICBib29sIG9ubHlfbGFuZV9yZWR1Y2luZyA9IHRydWU7CiAgIHN0bXRf aW5mbyA9IE5VTEw7CiAgIHNscF90cmVlIHNscF9mb3Jfc3RtdF9pbmZvID0gc2xwX25vZGUgPyBz bHBfbm9kZV9pbnN0YW5jZS0+cm9vdCA6IE5VTEw7CiAgIHdoaWxlIChyZWR1Y19kZWYgIT0gUEhJ X1JFU1VMVCAocmVkdWNfZGVmX3BoaSkpCkBAIC03NTUxLDE0ICs3NzgxLDE1IEBAIHZlY3Rvcml6 YWJsZV9yZWR1Y3Rpb24gKGxvb3BfdmVjX2luZm8gbG9vcF92aW5mbywKIAkgYWxsIGxhbmVzIGhl cmUgLSBldmVuIHRob3VnaCB3ZSBvbmx5IHdpbGwgdmVjdG9yaXplIGZyb20KIAkgdGhlIFNMUCBu b2RlIHdpdGggbGl2ZSBsYW5lIHplcm8gdGhlIG90aGVyIGxpdmUgbGFuZXMgYWxzbwogCSBuZWVk IHRvIGJlIGlkZW50aWZpZWQgYXMgcGFydCBvZiBhIHJlZHVjdGlvbiB0byBiZSBhYmxlCi0JIHRv IHNraXAgY29kZSBnZW5lcmF0aW9uIGZvciB0aGVtLiAgKi8KKwkgdG8gc2tpcCBjb2RlIGdlbmVy YXRpb24gZm9yIHRoZW0uICBGb3IgbGFuZS1yZWR1Y2luZyBvcGVyYXRpb24KKwkgdmVjdG9yaXph YmxlIGFuYWx5c2lzIG5lZWRzIHRoZSByZWR1Y3Rpb24gUEhJIGluZm9ybWF0aW9uLiAgKi8KICAg ICAgIGlmIChzbHBfZm9yX3N0bXRfaW5mbykKIAl7CiAJICBmb3IgKGF1dG8gcyA6IFNMUF9UUkVF X1NDQUxBUl9TVE1UUyAoc2xwX2Zvcl9zdG10X2luZm8pKQogCSAgICBpZiAoU1RNVF9WSU5GT19M SVZFX1AgKHMpKQogCSAgICAgIFNUTVRfVklORk9fUkVEVUNfREVGICh2ZWN0X29yaWdfc3RtdCAo cykpID0gcGhpX2luZm87CiAJfQotICAgICAgZWxzZSBpZiAoU1RNVF9WSU5GT19MSVZFX1AgKHZk ZWYpKQorICAgICAgZWxzZQogCVNUTVRfVklORk9fUkVEVUNfREVGIChkZWYpID0gcGhpX2luZm87 CiAgICAgICBnaW1wbGVfbWF0Y2hfb3Agb3A7CiAgICAgICBpZiAoIWdpbXBsZV9leHRyYWN0X29w ICh2ZGVmLT5zdG10LCAmb3ApKQpAQCAtNzU3OSw5ICs3ODEwLDE2IEBAIHZlY3Rvcml6YWJsZV9y ZWR1Y3Rpb24gKGxvb3BfdmVjX2luZm8gbG9vcF92aW5mbywKIAkgICAgICByZXR1cm4gZmFsc2U7 CiAJICAgIH0KIAl9Ci0gICAgICBlbHNlIGlmICghc3RtdF9pbmZvKQotCS8qIEZpcnN0IG5vbi1j b252ZXJzaW9uIHN0bXQuICAqLwotCXN0bXRfaW5mbyA9IHZkZWY7CisgICAgICBlbHNlCisJewor CSAgLyogRmlyc3Qgbm9uLWNvbnZlcnNpb24gc3RtdC4gICovCisJICBpZiAoIXN0bXRfaW5mbykK KwkgICAgc3RtdF9pbmZvID0gdmRlZjsKKworCSAgaWYgKCFsYW5lX3JlZHVjaW5nX29wX3AgKG9w LmNvZGUpKQorCSAgICBvbmx5X2xhbmVfcmVkdWNpbmcgPSBmYWxzZTsKKwl9CisKICAgICAgIHJl ZHVjX2RlZiA9IG9wLm9wc1tTVE1UX1ZJTkZPX1JFRFVDX0lEWCAodmRlZildOwogICAgICAgcmVk dWNfY2hhaW5fbGVuZ3RoKys7CiAgICAgICBpZiAoIXN0bXRfaW5mbyAmJiBzbHBfbm9kZSkKQEAg LTc2NDMsOSArNzg4MSw3IEBAIHZlY3Rvcml6YWJsZV9yZWR1Y3Rpb24gKGxvb3BfdmVjX2luZm8g bG9vcF92aW5mbywKICAgZ2ltcGxlX21hdGNoX29wIG9wOwogICBpZiAoIWdpbXBsZV9leHRyYWN0 X29wIChzdG10X2luZm8tPnN0bXQsICZvcCkpCiAgICAgZ2NjX3VucmVhY2hhYmxlICgpOwotICBi b29sIGxhbmVfcmVkdWNfY29kZV9wID0gKG9wLmNvZGUgPT0gRE9UX1BST0RfRVhQUgotCQkJICAg IHx8IG9wLmNvZGUgPT0gV0lERU5fU1VNX0VYUFIKLQkJCSAgICB8fCBvcC5jb2RlID09IFNBRF9F WFBSKTsKKyAgYm9vbCBsYW5lX3JlZHVjaW5nID0gbGFuZV9yZWR1Y2luZ19vcF9wIChvcC5jb2Rl KTsKIAogICBpZiAoIVBPSU5URVJfVFlQRV9QIChvcC50eXBlKSAmJiAhSU5URUdSQUxfVFlQRV9Q IChvcC50eXBlKQogICAgICAgJiYgIVNDQUxBUl9GTE9BVF9UWVBFX1AgKG9wLnR5cGUpKQpAQCAt NzY1NSwyMyArNzg5MSwxMSBAQCB2ZWN0b3JpemFibGVfcmVkdWN0aW9uIChsb29wX3ZlY19pbmZv IGxvb3BfdmluZm8sCiAgIGlmICghdHlwZV9oYXNfbW9kZV9wcmVjaXNpb25fcCAob3AudHlwZSkp CiAgICAgcmV0dXJuIGZhbHNlOwogCi0gIC8qIEZvciBsYW5lLXJlZHVjaW5nIG9wcyB3ZSdyZSBy ZWR1Y2luZyB0aGUgbnVtYmVyIG9mIHJlZHVjdGlvbiBQSElzCi0gICAgIHdoaWNoIG1lYW5zIHRo ZSBvbmx5IHVzZSBvZiB0aGF0IG1heSBiZSBpbiB0aGUgbGFuZS1yZWR1Y2luZyBvcGVyYXRpb24u ICAqLwotICBpZiAobGFuZV9yZWR1Y19jb2RlX3AKLSAgICAgICYmIHJlZHVjX2NoYWluX2xlbmd0 aCAhPSAxCi0gICAgICAmJiAhb25seV9zbHBfcmVkdWNfY2hhaW4pCi0gICAgewotICAgICAgaWYg KGR1bXBfZW5hYmxlZF9wICgpKQotCWR1bXBfcHJpbnRmX2xvYyAoTVNHX01JU1NFRF9PUFRJTUla QVRJT04sIHZlY3RfbG9jYXRpb24sCi0JCQkgImxhbmUtcmVkdWNpbmcgcmVkdWN0aW9uIHdpdGgg ZXh0cmEgc3RtdHMuXG4iKTsKLSAgICAgIHJldHVybiBmYWxzZTsKLSAgICB9Ci0KICAgLyogTGFu ZS1yZWR1Y2luZyBvcHMgYWxzbyBuZXZlciBjYW4gYmUgdXNlZCBpbiBhIFNMUCByZWR1Y3Rpb24g Z3JvdXAKICAgICAgc2luY2Ugd2UnbGwgbWl4IGxhbmVzIGJlbG9uZ2luZyB0byBkaWZmZXJlbnQg cmVkdWN0aW9ucy4gIEJ1dCBpdCdzCiAgICAgIE9LIHRvIHVzZSB0aGVtIGluIGEgcmVkdWN0aW9u IGNoYWluIG9yIHdoZW4gdGhlIHJlZHVjdGlvbiBncm91cAogICAgICBoYXMganVzdCBvbmUgZWxl bWVudC4gICovCi0gIGlmIChsYW5lX3JlZHVjX2NvZGVfcAorICBpZiAobGFuZV9yZWR1Y2luZwog ICAgICAgJiYgc2xwX25vZGUKICAgICAgICYmICFSRURVQ19HUk9VUF9GSVJTVF9FTEVNRU5UIChz dG10X2luZm8pCiAgICAgICAmJiBTTFBfVFJFRV9MQU5FUyAoc2xwX25vZGUpID4gMSkKQEAgLTc3 MTAsOSArNzkzNCw2IEBAIHZlY3Rvcml6YWJsZV9yZWR1Y3Rpb24gKGxvb3BfdmVjX2luZm8gbG9v cF92aW5mbywKIAkJCSAgICAgInVzZSBub3Qgc2ltcGxlLlxuIik7CiAJICByZXR1cm4gZmFsc2U7 CiAJfQotICAgICAgaWYgKGkgPT0gU1RNVF9WSU5GT19SRURVQ19JRFggKHN0bXRfaW5mbykpCi0J Y29udGludWU7Ci0KICAgICAgIC8qIEZvciBhbiBJRk5fQ09ORF9PUCB3ZSBtaWdodCBoaXQgdGhl IHJlZHVjdGlvbiBkZWZpbml0aW9uIG9wZXJhbmQKIAkgdHdpY2UgKG9uY2UgYXMgZGVmaW5pdGlv biwgb25jZSBhcyBlbHNlKS4gICovCiAgICAgICBpZiAob3Aub3BzW2ldID09IG9wLm9wc1tTVE1U X1ZJTkZPX1JFRFVDX0lEWCAoc3RtdF9pbmZvKV0pCkBAIC03NzMxLDcgKzc5NTIsNyBAQCB2ZWN0 b3JpemFibGVfcmVkdWN0aW9uIChsb29wX3ZlY19pbmZvIGxvb3BfdmluZm8sCiAgICAgICAvKiBU byBwcm9wZXJseSBjb21wdXRlIG5jb3BpZXMgd2UgYXJlIGludGVyZXN0ZWQgaW4gdGhlIHdpZGVz dAogCSBub24tcmVkdWN0aW9uIGlucHV0IHR5cGUgaW4gY2FzZSB3ZSdyZSBsb29raW5nIGF0IGEg d2lkZW5pbmcKIAkgYWNjdW11bGF0aW9uIHRoYXQgd2UgbGF0ZXIgaGFuZGxlIGluIHZlY3RfdHJh bnNmb3JtX3JlZHVjdGlvbi4gICovCi0gICAgICBpZiAobGFuZV9yZWR1Y19jb2RlX3AKKyAgICAg IGlmIChsYW5lX3JlZHVjaW5nCiAJICAmJiB2ZWN0eXBlX29wW2ldCiAJICAmJiAoIXZlY3R5cGVf aW4KIAkgICAgICB8fCAoR0VUX01PREVfU0laRSAoU0NBTEFSX1RZUEVfTU9ERSAoVFJFRV9UWVBF ICh2ZWN0eXBlX2luKSkpCkBAIC03NzU4LDEyICs3OTc5LDIxIEBAIHZlY3Rvcml6YWJsZV9yZWR1 Y3Rpb24gKGxvb3BfdmVjX2luZm8gbG9vcF92aW5mbywKICAgICB9CiAgIGlmICghdmVjdHlwZV9p bikKICAgICB2ZWN0eXBlX2luID0gU1RNVF9WSU5GT19WRUNUWVBFIChwaGlfaW5mbyk7Ci0gIFNU TVRfVklORk9fUkVEVUNfVkVDVFlQRV9JTiAocmVkdWNfaW5mbykgPSB2ZWN0eXBlX2luOwogCi0g IGVudW0gdmVjdF9yZWR1Y3Rpb25fdHlwZSB2X3JlZHVjX3R5cGUgPSBTVE1UX1ZJTkZPX1JFRFVD X1RZUEUgKHBoaV9pbmZvKTsKLSAgU1RNVF9WSU5GT19SRURVQ19UWVBFIChyZWR1Y19pbmZvKSA9 IHZfcmVkdWNfdHlwZTsKKyAgLyogSWYgdGhlcmUgaXMgYSBub3JtYWwgKG5vbi1sYW5lLXJlZHVj aW5nKSBvcGVyYXRpb24gaW4gdGhlIGxvb3AgcmVkdWN0aW9uCisgICAgIHBhdGgsIHRvIGVuc3Vy ZSB0aGVyZSB3aWxsIGJlIGVub3VnaCBjb3BpZXMgdG8gaG9sZCB2ZWN0b3JpemVkIHJlc3VsdHMg b2YKKyAgICAgdGhlIG9wZXJhdGlvbiwgd2UgbmVlZCBzZXQgdGhlIGlucHV0IHZlY3R5cGUgb2Yg dGhlIHJlZHVjdGlvbiBQSEkgdG8gYmUKKyAgICAgc2FtZSBhcyB0aGUgcmVkdWN0aW9uIG91dHB1 dCB2ZWN0eXBlIHNvbWV3aGVyZSwgaGVyZSBpcyBhIHN1aXRhYmxlIHBsYWNlLgorICAgICBPdGhl cndpc2UgdGhlIGlucHV0IHZlY3R5cGUgaXMgc2V0IHRvIHRoZSBvbmUgd2l0aCB0aGUgbGVhc3Qg bGFuZXMsIHdoaWNoCisgICAgIGNhbiBvbmx5IGJlIGRldGVybWluZWQgaW4gdmVjdG9yaXphYmxl IGFuYWx5c2lzIHJvdXRpbmUgb2YgbGFuZS1yZWR1Y2luZworICAgICBvcGVyYXRpb24uICAqLwor ICBpZiAoIW9ubHlfbGFuZV9yZWR1Y2luZykKKyAgICBTVE1UX1ZJTkZPX1JFRFVDX1ZFQ1RZUEVf SU4gKHJlZHVjX2luZm8pID0gU1RNVF9WSU5GT19WRUNUWVBFIChwaGlfaW5mbyk7CisKKyAgZW51 bSB2ZWN0X3JlZHVjdGlvbl90eXBlIHJlZHVjdGlvbl90eXBlID0gU1RNVF9WSU5GT19SRURVQ19U WVBFIChwaGlfaW5mbyk7CisgIFNUTVRfVklORk9fUkVEVUNfVFlQRSAocmVkdWNfaW5mbykgPSBy ZWR1Y3Rpb25fdHlwZTsKICAgLyogSWYgd2UgaGF2ZSBhIGNvbmRpdGlvbiByZWR1Y3Rpb24sIHNl ZSBpZiB3ZSBjYW4gc2ltcGxpZnkgaXQgZnVydGhlci4gICovCi0gIGlmICh2X3JlZHVjX3R5cGUg PT0gQ09ORF9SRURVQ1RJT04pCisgIGlmIChyZWR1Y3Rpb25fdHlwZSA9PSBDT05EX1JFRFVDVElP TikKICAgICB7CiAgICAgICBpZiAoc2xwX25vZGUpCiAJcmV0dXJuIGZhbHNlOwpAQCAtNzkyOSw4 ICs4MTU5LDggQEAgdmVjdG9yaXphYmxlX3JlZHVjdGlvbiAobG9vcF92ZWNfaW5mbyBsb29wX3Zp bmZvLAogICAgIH0KIAogICBTVE1UX1ZJTkZPX1JFRFVDX0NPREUgKHJlZHVjX2luZm8pID0gb3Jp Z19jb2RlOworICByZWR1Y3Rpb25fdHlwZSA9IFNUTVRfVklORk9fUkVEVUNfVFlQRSAocmVkdWNf aW5mbyk7CiAKLSAgdmVjdF9yZWR1Y3Rpb25fdHlwZSByZWR1Y3Rpb25fdHlwZSA9IFNUTVRfVklO Rk9fUkVEVUNfVFlQRSAocmVkdWNfaW5mbyk7CiAgIGlmIChyZWR1Y3Rpb25fdHlwZSA9PSBUUkVF X0NPREVfUkVEVUNUSU9OKQogICAgIHsKICAgICAgIC8qIENoZWNrIHdoZXRoZXIgaXQncyBvayB0 byBjaGFuZ2UgdGhlIG9yZGVyIG9mIHRoZSBjb21wdXRhdGlvbi4KQEAgLTgyMDQsMTQgKzg0MzQs MTEgQEAgdmVjdG9yaXphYmxlX3JlZHVjdGlvbiAobG9vcF92ZWNfaW5mbyBsb29wX3ZpbmZvLAog ICAgICAgJiYgbG9vcF92aW5mby0+c3VnZ2VzdGVkX3Vucm9sbF9mYWN0b3IgPT0gMSkKICAgICBz aW5nbGVfZGVmdXNlX2N5Y2xlID0gdHJ1ZTsKIAotICBpZiAoc2luZ2xlX2RlZnVzZV9jeWNsZSB8 fCBsYW5lX3JlZHVjX2NvZGVfcCkKKyAgaWYgKHNpbmdsZV9kZWZ1c2VfY3ljbGUgJiYgIWxhbmVf cmVkdWNpbmcpCiAgICAgewogICAgICAgZ2NjX2Fzc2VydCAob3AuY29kZSAhPSBDT05EX0VYUFIp OwogCi0gICAgICAvKiA0LiBTdXBwb3J0YWJsZSBieSB0YXJnZXQ/ICAqLwotICAgICAgYm9vbCBv ayA9IHRydWU7Ci0KLSAgICAgIC8qIDQuMS4gY2hlY2sgc3VwcG9ydCBmb3IgdGhlIG9wZXJhdGlv biBpbiB0aGUgbG9vcAorICAgICAgLyogNC4gY2hlY2sgc3VwcG9ydCBmb3IgdGhlIG9wZXJhdGlv biBpbiB0aGUgbG9vcAogCiAJIFRoaXMgaXNuJ3QgbmVjZXNzYXJ5IGZvciB0aGUgbGFuZSByZWR1 Y3Rpb24gY29kZXMsIHNpbmNlIHRoZXkKIAkgY2FuIG9ubHkgYmUgcHJvZHVjZWQgYnkgcGF0dGVy biBtYXRjaGluZywgYW5kIGl0J3MgdXAgdG8gdGhlCkBAIC04MjIwLDE0ICs4NDQ3LDEzIEBAIHZl Y3Rvcml6YWJsZV9yZWR1Y3Rpb24gKGxvb3BfdmVjX2luZm8gbG9vcF92aW5mbywKIAkgbWl4ZWQt c2lnbiBkb3QtcHJvZHVjdHMgY2FuIGJlIGltcGxlbWVudGVkIHVzaW5nIHNpZ25lZAogCSBkb3Qt cHJvZHVjdHMuICAqLwogICAgICAgbWFjaGluZV9tb2RlIHZlY19tb2RlID0gVFlQRV9NT0RFICh2 ZWN0eXBlX2luKTsKLSAgICAgIGlmICghbGFuZV9yZWR1Y19jb2RlX3AKLQkgICYmICFkaXJlY3Rs eV9zdXBwb3J0ZWRfcCAob3AuY29kZSwgdmVjdHlwZV9pbiwgb3B0YWJfdmVjdG9yKSkKKyAgICAg IGlmICghZGlyZWN0bHlfc3VwcG9ydGVkX3AgKG9wLmNvZGUsIHZlY3R5cGVfaW4sIG9wdGFiX3Zl Y3RvcikpCiAgICAgICAgIHsKICAgICAgICAgICBpZiAoZHVtcF9lbmFibGVkX3AgKCkpCiAgICAg ICAgICAgICBkdW1wX3ByaW50ZiAoTVNHX05PVEUsICJvcCBub3Qgc3VwcG9ydGVkIGJ5IHRhcmdl dC5cbiIpOwogCSAgaWYgKG1heWJlX25lIChHRVRfTU9ERV9TSVpFICh2ZWNfbW9kZSksIFVOSVRT X1BFUl9XT1JEKQogCSAgICAgIHx8ICF2ZWN0X2Nhbl92ZWN0b3JpemVfd2l0aG91dF9zaW1kX3Ag KG9wLmNvZGUpKQotCSAgICBvayA9IGZhbHNlOworCSAgICBzaW5nbGVfZGVmdXNlX2N5Y2xlID0g ZmFsc2U7CiAJICBlbHNlCiAJICAgIGlmIChkdW1wX2VuYWJsZWRfcCAoKSkKIAkgICAgICBkdW1w X3ByaW50ZiAoTVNHX05PVEUsICJwcm9jZWVkaW5nIHVzaW5nIHdvcmQgbW9kZS5cbiIpOwpAQCAt ODI0MCwzNSArODQ2NiwxMiBAQCB2ZWN0b3JpemFibGVfcmVkdWN0aW9uIChsb29wX3ZlY19pbmZv IGxvb3BfdmluZm8sCiAJICAgIGR1bXBfcHJpbnRmIChNU0dfTk9URSwgInVzaW5nIHdvcmQgbW9k ZSBub3QgcG9zc2libGUuXG4iKTsKIAkgIHJldHVybiBmYWxzZTsKIAl9Ci0KLSAgICAgIC8qIGxh bmUtcmVkdWNpbmcgb3BlcmF0aW9ucyBoYXZlIHRvIGdvIHRocm91Z2ggdmVjdF90cmFuc2Zvcm1f cmVkdWN0aW9uLgotICAgICAgICAgRm9yIHRoZSBvdGhlciBjYXNlcyB0cnkgd2l0aG91dCB0aGUg c2luZ2xlIGN5Y2xlIG9wdGltaXphdGlvbi4gICovCi0gICAgICBpZiAoIW9rKQotCXsKLQkgIGlm IChsYW5lX3JlZHVjX2NvZGVfcCkKLQkgICAgcmV0dXJuIGZhbHNlOwotCSAgZWxzZQotCSAgICBz aW5nbGVfZGVmdXNlX2N5Y2xlID0gZmFsc2U7Ci0JfQogICAgIH0KICAgU1RNVF9WSU5GT19GT1JD RV9TSU5HTEVfQ1lDTEUgKHJlZHVjX2luZm8pID0gc2luZ2xlX2RlZnVzZV9jeWNsZTsKIAotICAv KiBJZiB0aGUgcmVkdWN0aW9uIHN0bXQgaXMgb25lIG9mIHRoZSBwYXR0ZXJucyB0aGF0IGhhdmUg bGFuZQotICAgICByZWR1Y3Rpb24gZW1iZWRkZWQgd2UgY2Fubm90IGhhbmRsZSB0aGUgY2FzZSBv ZiAhIHNpbmdsZV9kZWZ1c2VfY3ljbGUuICAqLwotICBpZiAoKG5jb3BpZXMgPiAxICYmICEgc2lu Z2xlX2RlZnVzZV9jeWNsZSkKLSAgICAgICYmIGxhbmVfcmVkdWNfY29kZV9wKQotICAgIHsKLSAg ICAgIGlmIChkdW1wX2VuYWJsZWRfcCAoKSkKLQlkdW1wX3ByaW50Zl9sb2MgKE1TR19NSVNTRURf T1BUSU1JWkFUSU9OLCB2ZWN0X2xvY2F0aW9uLAotCQkJICJtdWx0aSBkZWYtdXNlIGN5Y2xlIG5v dCBwb3NzaWJsZSBmb3IgbGFuZS1yZWR1Y2luZyAiCi0JCQkgInJlZHVjdGlvbiBvcGVyYXRpb25c biIpOwotICAgICAgcmV0dXJuIGZhbHNlOwotICAgIH0KLQotICBpZiAoc2xwX25vZGUKLSAgICAg ICYmICEoIXNpbmdsZV9kZWZ1c2VfY3ljbGUKLQkgICAmJiAhbGFuZV9yZWR1Y19jb2RlX3AKLQkg ICAmJiByZWR1Y3Rpb25fdHlwZSAhPSBGT0xEX0xFRlRfUkVEVUNUSU9OKSkKKyAgLyogUmVkdWN0 aW9uIHR5cGUgb2YgbGFuZS1yZWR1Y2luZyBvcGVyYXRpb24gaXMgVFJFRV9DT0RFX1JFRFVDVElP TiwgdGhlCisgICAgIGJlbG93IHByb2Nlc3Npbmcgd2lsbCBiZSBkb25lIGluIGl0cyBvd24gdmVj dG9yaXphYmxlIGZ1bmN0aW9uLiAgKi8KKyAgaWYgKHNscF9ub2RlICYmIHJlZHVjdGlvbl90eXBl ID09IEZPTERfTEVGVF9SRURVQ1RJT04pCiAgICAgZm9yIChpID0gMDsgaSA8IChpbnQpIG9wLm51 bV9vcHM7IGkrKykKICAgICAgIGlmICghdmVjdF9tYXliZV91cGRhdGVfc2xwX29wX3ZlY3R5cGUg KHNscF9vcFtpXSwgdmVjdHlwZV9vcFtpXSkpCiAJewpAQCAtODI3OCwzNiArODQ4MSwyNCBAQCB2 ZWN0b3JpemFibGVfcmVkdWN0aW9uIChsb29wX3ZlY19pbmZvIGxvb3BfdmluZm8sCiAJICByZXR1 cm4gZmFsc2U7CiAJfQogCi0gIGlmIChzbHBfbm9kZSkKLSAgICB2ZWNfbnVtID0gU0xQX1RSRUVf TlVNQkVSX09GX1ZFQ19TVE1UUyAoc2xwX25vZGUpOwotICBlbHNlCi0gICAgdmVjX251bSA9IDE7 Ci0KICAgdmVjdF9tb2RlbF9yZWR1Y3Rpb25fY29zdCAobG9vcF92aW5mbywgc3RtdF9pbmZvLCBy ZWR1Y19mbiwKIAkJCSAgICAgcmVkdWN0aW9uX3R5cGUsIG5jb3BpZXMsIGNvc3RfdmVjKTsKICAg LyogQ29zdCB0aGUgcmVkdWN0aW9uIG9wIGluc2lkZSB0aGUgbG9vcCBpZiB0cmFuc2Zvcm1lZCB2 aWEKLSAgICAgdmVjdF90cmFuc2Zvcm1fcmVkdWN0aW9uLiAgT3RoZXJ3aXNlIHRoaXMgaXMgY29z dGVkIGJ5IHRoZQotICAgICBzZXBhcmF0ZSB2ZWN0b3JpemFibGVfKiByb3V0aW5lcy4gICovCi0g IGlmIChzaW5nbGVfZGVmdXNlX2N5Y2xlIHx8IGxhbmVfcmVkdWNfY29kZV9wKQotICAgIHsKLSAg ICAgIGludCBmYWN0b3IgPSAxOwotICAgICAgaWYgKHZlY3RfaXNfZW11bGF0ZWRfbWl4ZWRfZG90 X3Byb2QgKGxvb3BfdmluZm8sIHN0bXRfaW5mbykpCi0JLyogVGhyZWUgZG90LXByb2R1Y3RzIGFu ZCBhIHN1YnRyYWN0aW9uLiAgKi8KLQlmYWN0b3IgPSA0OwotICAgICAgcmVjb3JkX3N0bXRfY29z dCAoY29zdF92ZWMsIG5jb3BpZXMgKiBmYWN0b3IsIHZlY3Rvcl9zdG10LAotCQkJc3RtdF9pbmZv LCAwLCB2ZWN0X2JvZHkpOwotICAgIH0KKyAgICAgdmVjdF90cmFuc2Zvcm1fcmVkdWN0aW9uIGZv ciBub24tbGFuZS1yZWR1Y2luZyBvcGVyYXRpb24uICBPdGhlcndpc2UKKyAgICAgdGhpcyBpcyBj b3N0ZWQgYnkgdGhlIHNlcGFyYXRlIHZlY3Rvcml6YWJsZV8qIHJvdXRpbmVzLiAgKi8KKyAgaWYg KHNpbmdsZV9kZWZ1c2VfY3ljbGUgJiYgIWxhbmVfcmVkdWNpbmcpCisgICAgcmVjb3JkX3N0bXRf Y29zdCAoY29zdF92ZWMsIG5jb3BpZXMsIHZlY3Rvcl9zdG10LCBzdG10X2luZm8sIDAsIHZlY3Rf Ym9keSk7CiAKICAgaWYgKGR1bXBfZW5hYmxlZF9wICgpCiAgICAgICAmJiByZWR1Y3Rpb25fdHlw ZSA9PSBGT0xEX0xFRlRfUkVEVUNUSU9OKQogICAgIGR1bXBfcHJpbnRmX2xvYyAoTVNHX05PVEUs IHZlY3RfbG9jYXRpb24sCiAJCSAgICAgInVzaW5nIGFuIGluLW9yZGVyIChmb2xkLWxlZnQpIHJl ZHVjdGlvbi5cbiIpOwogICBTVE1UX1ZJTkZPX1RZUEUgKG9yaWdfc3RtdF9vZl9hbmFseXNpcykg PSBjeWNsZV9waGlfaW5mb190eXBlOwotICAvKiBBbGwgYnV0IHNpbmdsZSBkZWZ1c2UtY3ljbGUg b3B0aW1pemVkLCBsYW5lLXJlZHVjaW5nIGFuZCBmb2xkLWxlZnQKLSAgICAgcmVkdWN0aW9ucyBn byB0aHJvdWdoIHRoZWlyIG93biB2ZWN0b3JpemFibGVfKiByb3V0aW5lcy4gICovCi0gIGlmICgh c2luZ2xlX2RlZnVzZV9jeWNsZQotICAgICAgJiYgIWxhbmVfcmVkdWNfY29kZV9wCi0gICAgICAm JiByZWR1Y3Rpb25fdHlwZSAhPSBGT0xEX0xFRlRfUkVEVUNUSU9OKQorCisgIC8qIEFsbCBidXQg c2luZ2xlIGRlZnVzZS1jeWNsZSBvcHRpbWl6ZWQgYW5kIGZvbGQtbGVmdCByZWR1Y3Rpb25zIGdv CisgICAgIHRocm91Z2ggdGhlaXIgb3duIHZlY3Rvcml6YWJsZV8qIHJvdXRpbmVzLiAgKi8KKyAg aWYgKCghc2luZ2xlX2RlZnVzZV9jeWNsZSAmJiByZWR1Y3Rpb25fdHlwZSAhPSBGT0xEX0xFRlRf UkVEVUNUSU9OKQorICAgICAgfHwgbGFuZV9yZWR1Y2luZykKICAgICB7CiAgICAgICBzdG10X3Zl Y19pbmZvIHRlbQogCT0gdmVjdF9zdG10X3RvX3ZlY3Rvcml6ZSAoU1RNVF9WSU5GT19SRURVQ19E RUYgKHBoaV9pbmZvKSk7CkBAIC04MzE5LDYwICs4NTEwLDEwIEBAIHZlY3Rvcml6YWJsZV9yZWR1 Y3Rpb24gKGxvb3BfdmVjX2luZm8gbG9vcF92aW5mbywKICAgICAgIFNUTVRfVklORk9fREVGX1RZ UEUgKHZlY3Rfb3JpZ19zdG10ICh0ZW0pKSA9IHZlY3RfaW50ZXJuYWxfZGVmOwogICAgICAgU1RN VF9WSU5GT19ERUZfVFlQRSAodGVtKSA9IHZlY3RfaW50ZXJuYWxfZGVmOwogICAgIH0KLSAgZWxz ZSBpZiAobG9vcF92aW5mbyAmJiBMT09QX1ZJTkZPX0NBTl9VU0VfUEFSVElBTF9WRUNUT1JTX1Ag KGxvb3BfdmluZm8pKQotICAgIHsKLSAgICAgIHZlY19sb29wX21hc2tzICptYXNrcyA9ICZMT09Q X1ZJTkZPX01BU0tTIChsb29wX3ZpbmZvKTsKLSAgICAgIHZlY19sb29wX2xlbnMgKmxlbnMgPSAm TE9PUF9WSU5GT19MRU5TIChsb29wX3ZpbmZvKTsKLSAgICAgIGludGVybmFsX2ZuIGNvbmRfZm4g PSBnZXRfY29uZGl0aW9uYWxfaW50ZXJuYWxfZm4gKG9wLmNvZGUsIG9wLnR5cGUpOwotCi0gICAg ICBpZiAocmVkdWN0aW9uX3R5cGUgIT0gRk9MRF9MRUZUX1JFRFVDVElPTgotCSAgJiYgIXVzZV9t YXNrX2J5X2NvbmRfZXhwcl9wIChvcC5jb2RlLCBjb25kX2ZuLCB2ZWN0eXBlX2luKQotCSAgJiYg KGNvbmRfZm4gPT0gSUZOX0xBU1QKLQkgICAgICB8fCAhZGlyZWN0X2ludGVybmFsX2ZuX3N1cHBv cnRlZF9wIChjb25kX2ZuLCB2ZWN0eXBlX2luLAotCQkJCQkJICBPUFRJTUlaRV9GT1JfU1BFRUQp KSkKLQl7Ci0JICBpZiAoZHVtcF9lbmFibGVkX3AgKCkpCi0JICAgIGR1bXBfcHJpbnRmX2xvYyAo TVNHX01JU1NFRF9PUFRJTUlaQVRJT04sIHZlY3RfbG9jYXRpb24sCi0JCQkgICAgICJjYW4ndCBv cGVyYXRlIG9uIHBhcnRpYWwgdmVjdG9ycyBiZWNhdXNlIgotCQkJICAgICAiIG5vIGNvbmRpdGlv bmFsIG9wZXJhdGlvbiBpcyBhdmFpbGFibGUuXG4iKTsKLQkgIExPT1BfVklORk9fQ0FOX1VTRV9Q QVJUSUFMX1ZFQ1RPUlNfUCAobG9vcF92aW5mbykgPSBmYWxzZTsKLQl9Ci0gICAgICBlbHNlIGlm IChyZWR1Y3Rpb25fdHlwZSA9PSBGT0xEX0xFRlRfUkVEVUNUSU9OCi0JICAgICAgICYmIHJlZHVj X2ZuID09IElGTl9MQVNUCi0JICAgICAgICYmICFleHBhbmRfdmVjX2NvbmRfZXhwcl9wICh2ZWN0 eXBlX2luLAotCQkJCQkgICB0cnV0aF90eXBlX2ZvciAodmVjdHlwZV9pbiksCi0JCQkJCSAgIFNT QV9OQU1FKSkKLQl7Ci0JICBpZiAoZHVtcF9lbmFibGVkX3AgKCkpCi0JICAgIGR1bXBfcHJpbnRm X2xvYyAoTVNHX01JU1NFRF9PUFRJTUlaQVRJT04sIHZlY3RfbG9jYXRpb24sCi0JCQkgICAgICJj YW4ndCBvcGVyYXRlIG9uIHBhcnRpYWwgdmVjdG9ycyBiZWNhdXNlIgotCQkJICAgICAiIG5vIGNv bmRpdGlvbmFsIG9wZXJhdGlvbiBpcyBhdmFpbGFibGUuXG4iKTsKLQkgIExPT1BfVklORk9fQ0FO X1VTRV9QQVJUSUFMX1ZFQ1RPUlNfUCAobG9vcF92aW5mbykgPSBmYWxzZTsKLQl9Ci0gICAgICBl bHNlIGlmIChyZWR1Y3Rpb25fdHlwZSA9PSBGT0xEX0xFRlRfUkVEVUNUSU9OCi0JICAgICAgICYm IGludGVybmFsX2ZuX21hc2tfaW5kZXggKHJlZHVjX2ZuKSA9PSAtMQotCSAgICAgICAmJiBGTE9B VF9UWVBFX1AgKHZlY3R5cGVfaW4pCi0JICAgICAgICYmIEhPTk9SX1NJR05fREVQRU5ERU5UX1JP VU5ESU5HICh2ZWN0eXBlX2luKSkKLQl7Ci0JICBpZiAoZHVtcF9lbmFibGVkX3AgKCkpCi0JICAg IGR1bXBfcHJpbnRmX2xvYyAoTVNHX01JU1NFRF9PUFRJTUlaQVRJT04sIHZlY3RfbG9jYXRpb24s Ci0JCQkgICAgICJjYW4ndCBvcGVyYXRlIG9uIHBhcnRpYWwgdmVjdG9ycyBiZWNhdXNlIgotCQkJ ICAgICAiIHNpZ25lZCB6ZXJvcyBjYW5ub3QgYmUgcHJlc2VydmVkLlxuIik7Ci0JICBMT09QX1ZJ TkZPX0NBTl9VU0VfUEFSVElBTF9WRUNUT1JTX1AgKGxvb3BfdmluZm8pID0gZmFsc2U7Ci0JfQot ICAgICAgZWxzZQotCXsKLQkgIGludGVybmFsX2ZuIG1hc2tfcmVkdWNfZm4KLQkgICAgPSBnZXRf bWFza2VkX3JlZHVjdGlvbl9mbiAocmVkdWNfZm4sIHZlY3R5cGVfaW4pOworICBlbHNlCisgICAg dmVjdF9yZWR1Y3Rpb25fdXNlX3BhcnRpYWxfdmVjdG9yIChsb29wX3ZpbmZvLCByZWR1Y19pbmZv LCBzbHBfbm9kZSwKKwkJCQkgICAgICAgb3AuY29kZSwgb3AudHlwZSwgdmVjdHlwZV9pbik7CiAK LQkgIGlmIChtYXNrX3JlZHVjX2ZuID09IElGTl9NQVNLX0xFTl9GT0xEX0xFRlRfUExVUykKLQkg ICAgdmVjdF9yZWNvcmRfbG9vcF9sZW4gKGxvb3BfdmluZm8sIGxlbnMsIG5jb3BpZXMgKiB2ZWNf bnVtLAotCQkJCSAgdmVjdHlwZV9pbiwgMSk7Ci0JICBlbHNlCi0JICAgIHZlY3RfcmVjb3JkX2xv b3BfbWFzayAobG9vcF92aW5mbywgbWFza3MsIG5jb3BpZXMgKiB2ZWNfbnVtLAotCQkJCSAgIHZl Y3R5cGVfaW4sIE5VTEwpOwotCX0KLSAgICB9CiAgIHJldHVybiB0cnVlOwogfQogCkBAIC04NDYz LDYgKzg2MDQsNyBAQCB2ZWN0X3RyYW5zZm9ybV9yZWR1Y3Rpb24gKGxvb3BfdmVjX2luZm8gbG9v cF92aW5mbywKICAgY2xhc3MgbG9vcCAqbG9vcCA9IExPT1BfVklORk9fTE9PUCAobG9vcF92aW5m byk7CiAgIGludCBpOwogICBpbnQgbmNvcGllczsKKyAgaW50IHN0bXRfbmNvcGllczsKICAgaW50 IHZlY19udW07CiAKICAgc3RtdF92ZWNfaW5mbyByZWR1Y19pbmZvID0gaW5mb19mb3JfcmVkdWN0 aW9uIChsb29wX3ZpbmZvLCBzdG10X2luZm8pOwpAQCAtODQ4NiwxNSArODYyOCwyOCBAQCB2ZWN0 X3RyYW5zZm9ybV9yZWR1Y3Rpb24gKGxvb3BfdmVjX2luZm8gbG9vcF92aW5mbywKICAgZ3BoaSAq cmVkdWNfZGVmX3BoaSA9IGFzX2EgPGdwaGkgKj4gKHBoaV9pbmZvLT5zdG10KTsKICAgaW50IHJl ZHVjX2luZGV4ID0gU1RNVF9WSU5GT19SRURVQ19JRFggKHN0bXRfaW5mbyk7CiAgIHRyZWUgdmVj dHlwZV9pbiA9IFNUTVRfVklORk9fUkVEVUNfVkVDVFlQRV9JTiAocmVkdWNfaW5mbyk7CisgIHRy ZWUgc3RtdF92ZWN0eXBlX2luID0gU1RNVF9WSU5GT19SRURVQ19WRUNUWVBFX0lOIChzdG10X2lu Zm8pOworCisgIC8qIEdldCBpbnB1dCB2ZWN0eXBlcyBmcm9tIHRoZSByZWR1Y3Rpb24gUEhJIGFu ZCB0aGUgc3RhdGVtZW50IHRvIGJlCisgICAgIHRyYW5zZm9ybWVkLCB0aGVzZSB0d28gdmVjdHlw ZXMgbWF5IGhhdmUgZGlmZmVyZW50IGxhbmVzIHdoZW4KKyAgICAgbGFuZS1yZWR1Y2luZyBvcGVy YXRpb24gaXMgcHJlc2VudC4gICovCisgIGlmICghdmVjdHlwZV9pbikKKyAgICB2ZWN0eXBlX2lu ID0gU1RNVF9WSU5GT19SRURVQ19WRUNUWVBFIChyZWR1Y19pbmZvKTsKKworICBpZiAoIXN0bXRf dmVjdHlwZV9pbikKKyAgICBzdG10X3ZlY3R5cGVfaW4gPSBTVE1UX1ZJTkZPX1ZFQ1RZUEUgKHN0 bXRfaW5mbyk7CiAKICAgaWYgKHNscF9ub2RlKQogICAgIHsKICAgICAgIG5jb3BpZXMgPSAxOwor ICAgICAgc3RtdF9uY29waWVzID0gMTsKICAgICAgIHZlY19udW0gPSBTTFBfVFJFRV9OVU1CRVJf T0ZfVkVDX1NUTVRTIChzbHBfbm9kZSk7CiAgICAgfQogICBlbHNlCiAgICAgewogICAgICAgbmNv cGllcyA9IHZlY3RfZ2V0X251bV9jb3BpZXMgKGxvb3BfdmluZm8sIHZlY3R5cGVfaW4pOworICAg ICAgc3RtdF9uY29waWVzID0gdmVjdF9nZXRfbnVtX2NvcGllcyAobG9vcF92aW5mbywgc3RtdF92 ZWN0eXBlX2luKTsKKyAgICAgIGdjY19hc3NlcnQgKHN0bXRfbmNvcGllcyA+PSAxICYmIHN0bXRf bmNvcGllcyA8PSBuY29waWVzKTsKICAgICAgIHZlY19udW0gPSAxOwogICAgIH0KIApAQCAtODUw MywxNCArODY1OCwxMCBAQCB2ZWN0X3RyYW5zZm9ybV9yZWR1Y3Rpb24gKGxvb3BfdmVjX2luZm8g bG9vcF92aW5mbywKIAogICB2ZWNfbG9vcF9tYXNrcyAqbWFza3MgPSAmTE9PUF9WSU5GT19NQVNL UyAobG9vcF92aW5mbyk7CiAgIHZlY19sb29wX2xlbnMgKmxlbnMgPSAmTE9PUF9WSU5GT19MRU5T IChsb29wX3ZpbmZvKTsKLSAgYm9vbCBtYXNrX2J5X2NvbmRfZXhwciA9IHVzZV9tYXNrX2J5X2Nv bmRfZXhwcl9wIChjb2RlLCBjb25kX2ZuLCB2ZWN0eXBlX2luKTsKLQorICBib29sIG1hc2tfYnlf Y29uZF9leHByID0gdXNlX21hc2tfYnlfY29uZF9leHByX3AgKGNvZGUsIGNvbmRfZm4sCisJCQkJ CQkgICAgc3RtdF92ZWN0eXBlX2luKTsKICAgLyogVHJhbnNmb3JtLiAgKi8KLSAgdHJlZSBuZXdf dGVtcCA9IE5VTExfVFJFRTsKLSAgYXV0b192ZWM8dHJlZT4gdmVjX29wcm5kczA7Ci0gIGF1dG9f dmVjPHRyZWU+IHZlY19vcHJuZHMxOwotICBhdXRvX3ZlYzx0cmVlPiB2ZWNfb3BybmRzMjsKLSAg dHJlZSBkZWYwOworICBhdXRvX3ZlYzx0cmVlPiB2ZWNfb3BybmRzWzNdOwogCiAgIGlmIChkdW1w X2VuYWJsZWRfcCAoKSkKICAgICBkdW1wX3ByaW50Zl9sb2MgKE1TR19OT1RFLCB2ZWN0X2xvY2F0 aW9uLCAidHJhbnNmb3JtIHJlZHVjdGlvbi5cbiIpOwpAQCAtODUzNCw4ICs4Njg1LDYgQEAgdmVj dF90cmFuc2Zvcm1fcmVkdWN0aW9uIChsb29wX3ZlY19pbmZvIGxvb3BfdmluZm8sCiAJCSAgICAg ID09IG9wLm9wc1tpbnRlcm5hbF9mbl9lbHNlX2luZGV4ICgoaW50ZXJuYWxfZm4pIGNvZGUpXSkp OwogICAgIH0KIAotICBib29sIG1hc2tlZF9sb29wX3AgPSBMT09QX1ZJTkZPX0ZVTExZX01BU0tF RF9QIChsb29wX3ZpbmZvKTsKLQogICB2ZWN0X3JlZHVjdGlvbl90eXBlIHJlZHVjdGlvbl90eXBl ID0gU1RNVF9WSU5GT19SRURVQ19UWVBFIChyZWR1Y19pbmZvKTsKICAgaWYgKHJlZHVjdGlvbl90 eXBlID09IEZPTERfTEVGVF9SRURVQ1RJT04pCiAgICAgewpAQCAtODU0Myw2OSArODY5MiwxNzIg QEAgdmVjdF90cmFuc2Zvcm1fcmVkdWN0aW9uIChsb29wX3ZlY19pbmZvIGxvb3BfdmluZm8sCiAg ICAgICBnY2NfYXNzZXJ0IChjb2RlLmlzX3RyZWVfY29kZSAoKSB8fCBjb25kX2ZuX3ApOwogICAg ICAgcmV0dXJuIHZlY3Rvcml6ZV9mb2xkX2xlZnRfcmVkdWN0aW9uCiAJICAobG9vcF92aW5mbywg c3RtdF9pbmZvLCBnc2ksIHZlY19zdG10LCBzbHBfbm9kZSwgcmVkdWNfZGVmX3BoaSwKLQkgICBj b2RlLCByZWR1Y19mbiwgb3Aub3BzLCBvcC5udW1fb3BzLCB2ZWN0eXBlX2luLAorCSAgIGNvZGUs IHJlZHVjX2ZuLCBvcC5vcHMsIG9wLm51bV9vcHMsIHN0bXRfdmVjdHlwZV9pbiwKIAkgICByZWR1 Y19pbmRleCwgbWFza3MsIGxlbnMpOwogICAgIH0KIAogICBib29sIHNpbmdsZV9kZWZ1c2VfY3lj bGUgPSBTVE1UX1ZJTkZPX0ZPUkNFX1NJTkdMRV9DWUNMRSAocmVkdWNfaW5mbyk7Ci0gIGdjY19h c3NlcnQgKHNpbmdsZV9kZWZ1c2VfY3ljbGUKLQkgICAgICB8fCBjb2RlID09IERPVF9QUk9EX0VY UFIKLQkgICAgICB8fCBjb2RlID09IFdJREVOX1NVTV9FWFBSCi0JICAgICAgfHwgY29kZSA9PSBT QURfRVhQUik7CisgIGJvb2wgbGFuZV9yZWR1Y2luZyA9IGxhbmVfcmVkdWNpbmdfb3BfcCAoY29k ZSk7CisKKyAgZ2NjX2Fzc2VydCAoc2luZ2xlX2RlZnVzZV9jeWNsZSB8fCBsYW5lX3JlZHVjaW5n KTsKIAogICAvKiBDcmVhdGUgdGhlIGRlc3RpbmF0aW9uIHZlY3RvciAgKi8KICAgdHJlZSBzY2Fs YXJfZGVzdCA9IGdpbXBsZV9nZXRfbGhzIChzdG10X2luZm8tPnN0bXQpOwogICB0cmVlIHZlY19k ZXN0ID0gdmVjdF9jcmVhdGVfZGVzdGluYXRpb25fdmFyIChzY2FsYXJfZGVzdCwgdmVjdHlwZV9v dXQpOwogCi0gIC8qIEdldCBOQ09QSUVTIHZlY3RvciBkZWZpbml0aW9ucyBmb3IgYWxsIG9wZXJh bmRzIGV4Y2VwdCB0aGUgcmVkdWN0aW9uCi0gICAgIGRlZmluaXRpb24uICAqLwotICBpZiAoIWNv bmRfZm5fcCkKKyAgZ2NjX2Fzc2VydCAocmVkdWNfaW5kZXggPCAzKTsKKworICBpZiAoc2xwX25v ZGUpCiAgICAgewotICAgICAgdmVjdF9nZXRfdmVjX2RlZnMgKGxvb3BfdmluZm8sIHN0bXRfaW5m bywgc2xwX25vZGUsIG5jb3BpZXMsCi0JCQkgc2luZ2xlX2RlZnVzZV9jeWNsZSAmJiByZWR1Y19p bmRleCA9PSAwCi0JCQkgPyBOVUxMX1RSRUUgOiBvcC5vcHNbMF0sICZ2ZWNfb3BybmRzMCwKLQkJ CSBzaW5nbGVfZGVmdXNlX2N5Y2xlICYmIHJlZHVjX2luZGV4ID09IDEKLQkJCSA/IE5VTExfVFJF RSA6IG9wLm9wc1sxXSwgJnZlY19vcHJuZHMxLAotCQkJIG9wLm51bV9vcHMgPT0gMwotCQkJICYm ICEoc2luZ2xlX2RlZnVzZV9jeWNsZSAmJiByZWR1Y19pbmRleCA9PSAyKQotCQkJID8gb3Aub3Bz WzJdIDogTlVMTF9UUkVFLCAmdmVjX29wcm5kczIpOworICAgICAgZ2NjX2Fzc2VydCAoIXNpbmds ZV9kZWZ1c2VfY3ljbGUgJiYgb3AubnVtX29wcyA8PSAzKTsKKworICAgICAgZm9yIChpID0gMDsg aSA8IChpbnQpIG9wLm51bV9vcHM7IGkrKykKKwl2ZWN0X2dldF9zbHBfZGVmcyAoU0xQX1RSRUVf Q0hJTERSRU4gKHNscF9ub2RlKVtpXSwgJnZlY19vcHJuZHNbaV0pOwogICAgIH0KICAgZWxzZQog ICAgIHsKLSAgICAgIC8qIEZvciBhIGNvbmRpdGlvbmFsIG9wZXJhdGlvbiBwYXNzIHRoZSB0cnV0 aCB0eXBlIGFzIG1hc2sKLQkgdmVjdHlwZS4gICovCi0gICAgICBnY2NfYXNzZXJ0IChzaW5nbGVf ZGVmdXNlX2N5Y2xlCi0JCSAgJiYgKHJlZHVjX2luZGV4ID09IDEgfHwgcmVkdWNfaW5kZXggPT0g MikpOwotICAgICAgdmVjdF9nZXRfdmVjX2RlZnMgKGxvb3BfdmluZm8sIHN0bXRfaW5mbywgc2xw X25vZGUsIG5jb3BpZXMsCi0JCQkgb3Aub3BzWzBdLCB0cnV0aF90eXBlX2ZvciAodmVjdHlwZV9p biksICZ2ZWNfb3BybmRzMCwKLQkJCSByZWR1Y19pbmRleCA9PSAxID8gTlVMTF9UUkVFIDogb3Au b3BzWzFdLAotCQkJIE5VTExfVFJFRSwgJnZlY19vcHJuZHMxLAotCQkJIHJlZHVjX2luZGV4ID09 IDIgPyBOVUxMX1RSRUUgOiBvcC5vcHNbMl0sCi0JCQkgTlVMTF9UUkVFLCAmdmVjX29wcm5kczIp OwotICAgIH0KKyAgICAgIGludCByZXN1bHRfcG9zID0gMDsKIAotICAvKiBGb3Igc2luZ2xlIGRl Zi11c2UgY3ljbGVzIGdldCBvbmUgY29weSBvZiB0aGUgdmVjdG9yaXplZCByZWR1Y3Rpb24KLSAg ICAgZGVmaW5pdGlvbi4gICovCi0gIGlmIChzaW5nbGVfZGVmdXNlX2N5Y2xlKQotICAgIHsKLSAg ICAgIGdjY19hc3NlcnQgKCFzbHBfbm9kZSk7Ci0gICAgICB2ZWN0X2dldF92ZWNfZGVmc19mb3Jf b3BlcmFuZCAobG9vcF92aW5mbywgc3RtdF9pbmZvLCAxLAotCQkJCSAgICAgb3Aub3BzW3JlZHVj X2luZGV4XSwKLQkJCQkgICAgIHJlZHVjX2luZGV4ID09IDAgPyAmdmVjX29wcm5kczAKLQkJCQkg ICAgIDogKHJlZHVjX2luZGV4ID09IDEgPyAmdmVjX29wcm5kczEKLQkJCQkJOiAmdmVjX29wcm5k czIpKTsKKyAgICAgIC8qIFRoZSBpbnB1dCB2ZWN0eXBlIG9mIHRoZSByZWR1Y3Rpb24gUEhJIGRl dGVybWluZXMgY29waWVzIG9mCisJIHZlY3Rvcml6ZWQgZGVmLXVzZSBjeWNsZXMsIHdoaWNoIG1p Z2h0IGJlIG1vcmUgdGhhbiBlZmZlY3RpdmUgY29waWVzCisJIG9mIHZlY3Rvcml6ZWQgbGFuZS1y ZWR1Y2luZyByZWR1Y3Rpb24gc3RhdGVtZW50cy4gIFRoaXMgY291bGQgYmUKKwkgY29tcGxlbWVu dGVkIGJ5IGdlbmVyYXRpbmcgZXh0cmEgdHJpdmlhbCBwYXNzLXRocm91Z2ggY29waWVzLiAgRm9y CisJIGV4YW1wbGU6CisKKwkgICBpbnQgc3VtID0gMDsKKwkgICBmb3IgKGkpCisJICAgICB7CisJ ICAgICAgIHN1bSArPSBkMFtpXSAqIGQxW2ldOyAgICAgIC8vIGRvdC1wcm9kIDx2ZWN0b3IoMTYp IGNoYXI+CisJICAgICAgIHN1bSArPSBhYnMoczBbaV0gLSBzMVtpXSk7IC8vIHNhZCA8dmVjdG9y KDgpIHNob3J0PgorCSAgICAgICBzdW0gKz0gbltpXTsgICAgICAgICAgICAgICAvLyBub3JtYWwg PHZlY3Rvcig0KSBpbnQ+CisJICAgICB9CisKKwkgVGhlIHZlY3RvciBzaXplIGlzIDEyOC1iaXTv vIx2ZWN0b3JpemF0aW9uIGZhY3RvciBpcyAxNi4gIFJlZHVjdGlvbgorCSBzdGF0ZW1lbnRzIHdv dWxkIGJlIHRyYW5zZm9ybWVkIGFzOgorCisJICAgdmVjdG9yPDQ+IGludCBzdW1fdjAgPSB7IDAs IDAsIDAsIDAgfTsKKwkgICB2ZWN0b3I8ND4gaW50IHN1bV92MSA9IHsgMCwgMCwgMCwgMCB9Owor CSAgIHZlY3Rvcjw0PiBpbnQgc3VtX3YyID0geyAwLCAwLCAwLCAwIH07CisJICAgdmVjdG9yPDQ+ IGludCBzdW1fdjMgPSB7IDAsIDAsIDAsIDAgfTsKKworCSAgIGZvciAoaSAvIDE2KQorCSAgICAg eworCSAgICAgICBzdW1fdjAgPSBET1RfUFJPRCAoZDBfdjBbaTogMCB+IDE1XSwgZDFfdjBbaTog MCB+IDE1XSwgc3VtX3YwKTsKKwkgICAgICAgc3VtX3YxID0gc3VtX3YxOyAgLy8gY29weQorCSAg ICAgICBzdW1fdjIgPSBzdW1fdjI7ICAvLyBjb3B5CisJICAgICAgIHN1bV92MyA9IHN1bV92Mzsg IC8vIGNvcHkKKworCSAgICAgICBzdW1fdjAgPSBzdW1fdjA7ICAvLyBjb3B5CisJICAgICAgIHN1 bV92MSA9IFNBRCAoczBfdjFbaTogMCB+IDcgXSwgczFfdjFbaTogMCB+IDcgXSwgc3VtX3YxKTsK KwkgICAgICAgc3VtX3YyID0gU0FEIChzMF92MltpOiA4IH4gMTVdLCBzMV92MltpOiA4IH4gMTVd LCBzdW1fdjIpOworCSAgICAgICBzdW1fdjMgPSBzdW1fdjM7ICAvLyBjb3B5CisKKwkgICAgICAg c3VtX3YwICs9IG5fdjBbaTogMCAgfiAzIF07CisJICAgICAgIHN1bV92MSArPSBuX3YxW2k6IDQg IH4gNyBdOworCSAgICAgICBzdW1fdjIgKz0gbl92MltpOiA4ICB+IDExXTsKKwkgICAgICAgc3Vt X3YzICs9IG5fdjNbaTogMTIgfiAxNV07CisJICAgICB9CisKKwkgTW9yZW92ZXIsIGZvciBhIGhp Z2hlciBpbnN0cnVjdGlvbiBwYXJhbGxlbGlzbSBpbiBmaW5hbCB2ZWN0b3JpemVkCisJIGxvb3As IGl0IGlzIGNvbnNpZGVyZWQgdG8gbWFrZSB0aG9zZSBlZmZlY3RpdmUgdmVjdG9yaXplZAorCSBs YW5lLXJlZHVjaW5nIHN0YXRlbWVudHMgYmUgZGlzdHJpYnV0ZWQgZXZlbmx5IGFtb25nIGFsbCBk ZWYtdXNlCisJIGN5Y2xlcy4gSW4gdGhlIGFib3ZlIGV4YW1wbGUsIFNBRHMgYXJlIGdlbmVyYXRl ZCBpbnRvIG90aGVyIGN5Y2xlcworCSByYXRoZXIgdGhhbiB0aGF0IG9mIERPVF9QUk9ELiAgKi8K KworICAgICAgaWYgKHN0bXRfbmNvcGllcyA8IG5jb3BpZXMpCisJeworCSAgZ2NjX2Fzc2VydCAo bGFuZV9yZWR1Y2luZyk7CisJICByZXN1bHRfcG9zID0gcmVkdWNfaW5mby0+cmVkdWNfcmVzdWx0 X3BvczsKKwkgIHJlZHVjX2luZm8tPnJlZHVjX3Jlc3VsdF9wb3MgPSAocmVzdWx0X3BvcyArIHN0 bXRfbmNvcGllcykgJSBuY29waWVzOworCSAgZ2NjX2Fzc2VydCAocmVzdWx0X3BvcyA+PSAwICYm IHJlc3VsdF9wb3MgPCBuY29waWVzKTsKKwl9CisKKyAgICAgIGZvciAoaSA9IDA7IGkgPCBNSU4g KDMsIChpbnQpIG9wLm51bV9vcHMpOyBpKyspCisJeworCSAgdHJlZSB2ZWN0eXBlID0gTlVMTF9U UkVFOworCSAgaW50IHVzZWRfbmNvcGllcyA9IG5jb3BpZXM7CisKKwkgIGlmIChjb25kX2ZuX3Ag JiYgaSA9PSAwKQorCSAgICB7CisJICAgICAgLyogRm9yIGEgY29uZGl0aW9uYWwgb3BlcmF0aW9u IHBhc3MgdGhlIHRydXRoIHR5cGUgYXMgbWFzaworCQkgdmVjdHlwZS4gICovCisJICAgICAgZ2Nj X2Fzc2VydCAoc2luZ2xlX2RlZnVzZV9jeWNsZSAmJiByZWR1Y19pbmRleCA+IDApOworCSAgICAg IHZlY3R5cGUgPSB0cnV0aF90eXBlX2ZvciAodmVjdHlwZV9pbik7CisJICAgIH0KKworCSAgaWYg KGkgIT0gcmVkdWNfaW5kZXgpCisJICAgIHsKKwkgICAgICAvKiBGb3Igbm9uLXJlZHVjdGlvbiBv cGVyYW5kLCBkZWR1Y2UgZWZmaWN0aXZlIGNvcGllcyB0aGF0IGFyZQorCQkgaW52b2x2ZWQgaW4g dmVjdG9yaXplZCBkZWYtdXNlIGN5Y2xlcyBiYXNlZCBvbiB0aGUgaW5wdXQKKwkJIHZlY3R5cGUg b2YgdGhlIHJlZHVjdGlvbiBzdGF0ZW1lbnQuICAqLworCSAgICAgIHVzZWRfbmNvcGllcyA9IHN0 bXRfbmNvcGllczsKKwkgICAgfQorCSAgZWxzZSBpZiAoc2luZ2xlX2RlZnVzZV9jeWNsZSkKKwkg ICAgeworCSAgICAgIC8qIEZvciBzaW5nbGUgZGVmLXVzZSBjeWNsZXMgZ2V0IG9uZSBjb3B5IG9m IHRoZSB2ZWN0b3JpemVkCisJCSByZWR1Y3Rpb24gZGVmaW5pdGlvbi4gICovCisJICAgICAgdXNl ZF9uY29waWVzID0gMTsKKwkgICAgfQorCisJICB2ZWN0X2dldF92ZWNfZGVmc19mb3Jfb3BlcmFu ZCAobG9vcF92aW5mbywgc3RtdF9pbmZvLCB1c2VkX25jb3BpZXMsCisJCQkJCSBvcC5vcHNbaV0s ICZ2ZWNfb3BybmRzW2ldLCB2ZWN0eXBlKTsKKworCSAgaWYgKHVzZWRfbmNvcGllcyA8IG5jb3Bp ZXMpCisJICAgIHsKKwkgICAgICB2ZWNfb3BybmRzW2ldLnNhZmVfZ3Jvd19jbGVhcmVkIChuY29w aWVzKTsKKworCSAgICAgIC8qIEZpbmQgc3VpdGFibGUgZGVmLXVzZSBjeWNsZXMgdG8gZ2VuZXJh dGUgdmVjdG9yaXplZAorCQkgc3RhdGVtZW50cyBpbnRvLCBhbmQgcmVvcmRlciBvcGVyYW5kcyBi YXNlZCBvbiB0aGUKKwkJIHNlbGVjdGlvbi4gICovCisJICAgICAgaWYgKGkgIT0gcmVkdWNfaW5k ZXggJiYgcmVzdWx0X3BvcykKKwkJeworCQkgIGludCBjb3VudCA9IG5jb3BpZXMgLSB1c2VkX25j b3BpZXM7CisJCSAgaW50IHN0YXJ0ID0gcmVzdWx0X3BvcyAtIGNvdW50OworCisJCSAgaWYgKHN0 YXJ0IDwgMCkKKwkJICAgIHsKKwkJICAgICAgY291bnQgPSByZXN1bHRfcG9zOworCQkgICAgICBz dGFydCA9IDA7CisJCSAgICB9CisKKwkJICBmb3IgKGludCBqID0gdXNlZF9uY29waWVzIC0gMTsg aiA+PSBzdGFydDsgai0tKQorCQkgICAgeworCQkgICAgICBzdGQ6OnN3YXAgKHZlY19vcHJuZHNb aV1bal0sIHZlY19vcHJuZHNbaV1baiArIGNvdW50XSk7CisJCSAgICAgIGdjY19hc3NlcnQgKCF2 ZWNfb3BybmRzW2ldW2pdKTsKKwkJICAgIH0KKwkJfQorCSAgICB9CisJfQogICAgIH0KIAotICBi b29sIGVtdWxhdGVkX21peGVkX2RvdF9wcm9kCi0gICAgPSB2ZWN0X2lzX2VtdWxhdGVkX21peGVk X2RvdF9wcm9kIChsb29wX3ZpbmZvLCBzdG10X2luZm8pOwotICBGT1JfRUFDSF9WRUNfRUxUICh2 ZWNfb3BybmRzMCwgaSwgZGVmMCkKKyAgYm9vbCBtYXNrZWRfbG9vcF9wID0gTE9PUF9WSU5GT19G VUxMWV9NQVNLRURfUCAobG9vcF92aW5mbyk7CisgIGJvb2wgZW11bGF0ZWRfbWl4ZWRfZG90X3By b2QgPSB2ZWN0X2lzX2VtdWxhdGVkX21peGVkX2RvdF9wcm9kIChzdG10X2luZm8pOworICB0cmVl IGRlZjA7CisKKyAgRk9SX0VBQ0hfVkVDX0VMVCAodmVjX29wcm5kc1swXSwgaSwgZGVmMCkKICAg ICB7CiAgICAgICBnaW1wbGUgKm5ld19zdG10OwotICAgICAgdHJlZSB2b3BbM10gPSB7IGRlZjAs IHZlY19vcHJuZHMxW2ldLCBOVUxMX1RSRUUgfTsKLSAgICAgIGlmIChtYXNrZWRfbG9vcF9wICYm ICFtYXNrX2J5X2NvbmRfZXhwcikKKyAgICAgIHRyZWUgbmV3X3RlbXAgPSBOVUxMX1RSRUU7Cisg ICAgICB0cmVlIHZvcFszXSA9IHsgZGVmMCwgdmVjX29wcm5kc1sxXVtpXSwgTlVMTF9UUkVFIH07 CisKKyAgICAgIGlmICghdm9wWzBdIHx8ICF2b3BbMV0pCiAJewotCSAgLyogTm8gY29uZGl0aW9u YWwgaWZucyBoYXZlIGJlZW4gZGVmaW5lZCBmb3IgZG90LXByb2R1Y3QgeWV0LiAgKi8KLQkgIGdj Y19hc3NlcnQgKGNvZGUgIT0gRE9UX1BST0RfRVhQUik7CisJICB0cmVlIHJlZHVjX3ZvcCA9IHZl Y19vcHJuZHNbcmVkdWNfaW5kZXhdW2ldOworCisJICAvKiBJbnNlcnQgdHJpdmlhbCBjb3B5IGlm IG5vIG5lZWQgdG8gZ2VuZXJhdGUgdmVjdG9yaXplZAorCSAgICAgc3RhdGVtZW50LiAgKi8KKwkg IGdjY19hc3NlcnQgKHJlZHVjX3ZvcCAmJiBzdG10X25jb3BpZXMgPCBuY29waWVzKTsKKworCSAg bmV3X3N0bXQgPSBnaW1wbGVfYnVpbGRfYXNzaWduICh2ZWNfZGVzdCwgcmVkdWNfdm9wKTsKKwkg IG5ld190ZW1wID0gbWFrZV9zc2FfbmFtZSAodmVjX2Rlc3QsIG5ld19zdG10KTsKKwkgIGdpbXBs ZV9zZXRfbGhzIChuZXdfc3RtdCwgbmV3X3RlbXApOworCSAgdmVjdF9maW5pc2hfc3RtdF9nZW5l cmF0aW9uIChsb29wX3ZpbmZvLCBzdG10X2luZm8sIG5ld19zdG10LCBnc2kpOworCX0KKyAgICAg IGVsc2UgaWYgKG1hc2tlZF9sb29wX3AgJiYgIW1hc2tfYnlfY29uZF9leHByKQorCXsKKwkgIC8q IE5vIGNvbmRpdGlvbmFsIGlmbnMgaGF2ZSBiZWVuIGRlZmluZWQgZm9yIGRvdC1wcm9kdWN0IGFu ZCBzYWQKKwkgICAgIHlldC4gICovCisJICBnY2NfYXNzZXJ0IChjb2RlICE9IERPVF9QUk9EX0VY UFIgJiYgY29kZSAhPSBTQURfRVhQUik7CiAKIAkgIC8qIE1ha2Ugc3VyZSB0aGF0IHRoZSByZWR1 Y3Rpb24gYWNjdW11bGF0b3IgaXMgdm9wWzBdLiAgKi8KIAkgIGlmIChyZWR1Y19pbmRleCA9PSAx KQpAQCAtODYxNCw3ICs4ODY2LDggQEAgdmVjdF90cmFuc2Zvcm1fcmVkdWN0aW9uIChsb29wX3Zl Y19pbmZvIGxvb3BfdmluZm8sCiAJICAgICAgc3RkOjpzd2FwICh2b3BbMF0sIHZvcFsxXSk7CiAJ ICAgIH0KIAkgIHRyZWUgbWFzayA9IHZlY3RfZ2V0X2xvb3BfbWFzayAobG9vcF92aW5mbywgZ3Np LCBtYXNrcywKLQkJCQkJICB2ZWNfbnVtICogbmNvcGllcywgdmVjdHlwZV9pbiwgaSk7CisJCQkJ CSAgdmVjX251bSAqIHN0bXRfbmNvcGllcywKKwkJCQkJICBzdG10X3ZlY3R5cGVfaW4sIGkpOwog CSAgZ2NhbGwgKmNhbGwgPSBnaW1wbGVfYnVpbGRfY2FsbF9pbnRlcm5hbCAoY29uZF9mbiwgNCwg bWFzaywKIAkJCQkJCSAgICB2b3BbMF0sIHZvcFsxXSwgdm9wWzBdKTsKIAkgIG5ld190ZW1wID0g bWFrZV9zc2FfbmFtZSAodmVjX2Rlc3QsIGNhbGwpOwpAQCAtODYyNiwxMiArODg3OSwxMyBAQCB2 ZWN0X3RyYW5zZm9ybV9yZWR1Y3Rpb24gKGxvb3BfdmVjX2luZm8gbG9vcF92aW5mbywKICAgICAg IGVsc2UKIAl7CiAJICBpZiAob3AubnVtX29wcyA+PSAzKQotCSAgICB2b3BbMl0gPSB2ZWNfb3By bmRzMltpXTsKKwkgICAgdm9wWzJdID0gdmVjX29wcm5kc1syXVtpXTsKIAogCSAgaWYgKG1hc2tl ZF9sb29wX3AgJiYgbWFza19ieV9jb25kX2V4cHIpCiAJICAgIHsKIAkgICAgICB0cmVlIG1hc2sg PSB2ZWN0X2dldF9sb29wX21hc2sgKGxvb3BfdmluZm8sIGdzaSwgbWFza3MsCi0JCQkJCSAgICAg IHZlY19udW0gKiBuY29waWVzLCB2ZWN0eXBlX2luLCBpKTsKKwkJCQkJICAgICAgdmVjX251bSAq IHN0bXRfbmNvcGllcywKKwkJCQkJICAgICAgc3RtdF92ZWN0eXBlX2luLCBpKTsKIAkgICAgICBi dWlsZF92ZWN0X2NvbmRfZXhwciAoY29kZSwgdm9wLCBtYXNrLCBnc2kpOwogCSAgICB9CiAKQEAg LTg2NTgsMTYgKzg5MTIsOCBAQCB2ZWN0X3RyYW5zZm9ybV9yZWR1Y3Rpb24gKGxvb3BfdmVjX2lu Zm8gbG9vcF92aW5mbywKIAogICAgICAgaWYgKHNscF9ub2RlKQogCXNscF9ub2RlLT5wdXNoX3Zl Y19kZWYgKG5ld19zdG10KTsKLSAgICAgIGVsc2UgaWYgKHNpbmdsZV9kZWZ1c2VfY3ljbGUKLQkg ICAgICAgJiYgaSA8IG5jb3BpZXMgLSAxKQotCXsKLQkgIGlmIChyZWR1Y19pbmRleCA9PSAwKQot CSAgICB2ZWNfb3BybmRzMC5zYWZlX3B1c2ggKGdpbXBsZV9nZXRfbGhzIChuZXdfc3RtdCkpOwot CSAgZWxzZSBpZiAocmVkdWNfaW5kZXggPT0gMSkKLQkgICAgdmVjX29wcm5kczEuc2FmZV9wdXNo IChnaW1wbGVfZ2V0X2xocyAobmV3X3N0bXQpKTsKLQkgIGVsc2UgaWYgKHJlZHVjX2luZGV4ID09 IDIpCi0JICAgIHZlY19vcHJuZHMyLnNhZmVfcHVzaCAoZ2ltcGxlX2dldF9saHMgKG5ld19zdG10 KSk7Ci0JfQorICAgICAgZWxzZSBpZiAoc2luZ2xlX2RlZnVzZV9jeWNsZSAmJiBpIDwgbmNvcGll cyAtIDEpCisJdmVjX29wcm5kc1tyZWR1Y19pbmRleF1baSArIDFdID0gZ2ltcGxlX2dldF9saHMg KG5ld19zdG10KTsKICAgICAgIGVsc2UKIAlTVE1UX1ZJTkZPX1ZFQ19TVE1UUyAoc3RtdF9pbmZv KS5zYWZlX3B1c2ggKG5ld19zdG10KTsKICAgICB9CmRpZmYgLS1naXQgYS9nY2MvdHJlZS12ZWN0 LXNscC5jYyBiL2djYy90cmVlLXZlY3Qtc2xwLmNjCmluZGV4IGM3ZWQ1MjBiNjI5Li41NzEzZTMy ZjU0NSAxMDA2NDQKLS0tIGEvZ2NjL3RyZWUtdmVjdC1zbHAuY2MKKysrIGIvZ2NjL3RyZWUtdmVj dC1zbHAuY2MKQEAgLTM5MjQsOSArMzkyNCw3IEBAIHZlY3RfYW5hbHl6ZV9zbHAgKHZlY19pbmZv ICp2aW5mbywgdW5zaWduZWQgbWF4X3RyZWVfc2l6ZSkKIAkJICAvKiBEbyBub3QgZGlzY292ZXIg U0xQIHJlZHVjdGlvbnMgZm9yIGxhbmUtcmVkdWNpbmcgb3BzLCB0aGF0CiAJCSAgICAgd2lsbCBm YWlsIGxhdGVyLiAgKi8KIAkJICAmJiAoIShnID0gZHluX2Nhc3QgPGdhc3NpZ24gKj4gKFNUTVRf VklORk9fU1RNVCAobmV4dF9pbmZvKSkpCi0JCSAgICAgIHx8IChnaW1wbGVfYXNzaWduX3Joc19j b2RlIChnKSAhPSBET1RfUFJPRF9FWFBSCi0JCQkgICYmIGdpbXBsZV9hc3NpZ25fcmhzX2NvZGUg KGcpICE9IFdJREVOX1NVTV9FWFBSCi0JCQkgICYmIGdpbXBsZV9hc3NpZ25fcmhzX2NvZGUgKGcp ICE9IFNBRF9FWFBSKSkpCisJCSAgICAgIHx8ICFsYW5lX3JlZHVjaW5nX29wX3AgKGdpbXBsZV9h c3NpZ25fcmhzX2NvZGUgKGcpKSkpCiAJCXNjYWxhcl9zdG10cy5xdWlja19wdXNoIChuZXh0X2lu Zm8pOwogCSAgICB9CiAJICBpZiAoc2NhbGFyX3N0bXRzLmxlbmd0aCAoKSA+IDEpCmRpZmYgLS1n aXQgYS9nY2MvdHJlZS12ZWN0LXN0bXRzLmNjIGIvZ2NjL3RyZWUtdmVjdC1zdG10cy5jYwppbmRl eCA0MjE5YWQ4MzJkYi4uNDFlZTMwNTE3NTYgMTAwNjQ0Ci0tLSBhL2djYy90cmVlLXZlY3Qtc3Rt dHMuY2MKKysrIGIvZ2NjL3RyZWUtdmVjdC1zdG10cy5jYwpAQCAtMTIwOTYsMTEgKzEyMDk2LDIw IEBAIHZlY3Rvcml6YWJsZV9jb25kaXRpb24gKHZlY19pbmZvICp2aW5mbywKICAgdmVjdF9yZWR1 Y3Rpb25fdHlwZSByZWR1Y3Rpb25fdHlwZSA9IFRSRUVfQ09ERV9SRURVQ1RJT047CiAgIGJvb2wg Zm9yX3JlZHVjdGlvbgogICAgID0gU1RNVF9WSU5GT19SRURVQ19ERUYgKHZlY3Rfb3JpZ19zdG10 IChzdG10X2luZm8pKSAhPSBOVUxMOworICBpZiAoZm9yX3JlZHVjdGlvbikKKyAgICB7CisgICAg ICByZWR1Y19pbmZvID0gaW5mb19mb3JfcmVkdWN0aW9uICh2aW5mbywgc3RtdF9pbmZvKTsKKyAg ICAgIGlmIChTVE1UX1ZJTkZPX1JFRFVDX0RFRiAocmVkdWNfaW5mbykgIT0gdmVjdF9vcmlnX3N0 bXQgKHN0bXRfaW5mbykpCisJeworCSAgZm9yX3JlZHVjdGlvbiA9IGZhbHNlOworCSAgcmVkdWNf aW5mbyA9IE5VTEw7CisJfQorICAgIH0KKwogICBpZiAoZm9yX3JlZHVjdGlvbikKICAgICB7CiAg ICAgICBpZiAoc2xwX25vZGUpCiAJcmV0dXJuIGZhbHNlOwotICAgICAgcmVkdWNfaW5mbyA9IGlu Zm9fZm9yX3JlZHVjdGlvbiAodmluZm8sIHN0bXRfaW5mbyk7CiAgICAgICByZWR1Y3Rpb25fdHlw ZSA9IFNUTVRfVklORk9fUkVEVUNfVFlQRSAocmVkdWNfaW5mbyk7CiAgICAgICByZWR1Y19pbmRl eCA9IFNUTVRfVklORk9fUkVEVUNfSURYIChzdG10X2luZm8pOwogICAgICAgZ2NjX2Fzc2VydCAo cmVkdWN0aW9uX3R5cGUgIT0gRVhUUkFDVF9MQVNUX1JFRFVDVElPTgpAQCAtMTMyODksNiArMTMy OTgsOCBAQCB2ZWN0X2FuYWx5emVfc3RtdCAodmVjX2luZm8gKnZpbmZvLAogCQkJCSAgICAgIE5V TEwsIE5VTEwsIG5vZGUsIGNvc3RfdmVjKQogCSAgfHwgdmVjdG9yaXphYmxlX2xvYWQgKHZpbmZv LCBzdG10X2luZm8sIE5VTEwsIE5VTEwsIG5vZGUsIGNvc3RfdmVjKQogCSAgfHwgdmVjdG9yaXph YmxlX3N0b3JlICh2aW5mbywgc3RtdF9pbmZvLCBOVUxMLCBOVUxMLCBub2RlLCBjb3N0X3ZlYykK KwkgIHx8IHZlY3Rvcml6YWJsZV9sYW5lX3JlZHVjaW5nIChhc19hIDxsb29wX3ZlY19pbmZvPiAo dmluZm8pLAorCQkJCQkgc3RtdF9pbmZvLCBub2RlLCBjb3N0X3ZlYykKIAkgIHx8IHZlY3Rvcml6 YWJsZV9yZWR1Y3Rpb24gKGFzX2EgPGxvb3BfdmVjX2luZm8+ICh2aW5mbyksIHN0bXRfaW5mbywK IAkJCQkgICAgIG5vZGUsIG5vZGVfaW5zdGFuY2UsIGNvc3RfdmVjKQogCSAgfHwgdmVjdG9yaXph YmxlX2luZHVjdGlvbiAoYXNfYSA8bG9vcF92ZWNfaW5mbz4gKHZpbmZvKSwgc3RtdF9pbmZvLApk aWZmIC0tZ2l0IGEvZ2NjL3RyZWUtdmVjdG9yaXplci5oIGIvZ2NjL3RyZWUtdmVjdG9yaXplci5o CmluZGV4IDkzYmMzMGVmNjYwLi4zOTJmZmY1Yjc5OSAxMDA2NDQKLS0tIGEvZ2NjL3RyZWUtdmVj dG9yaXplci5oCisrKyBiL2djYy90cmVlLXZlY3Rvcml6ZXIuaApAQCAtMTM5OSw2ICsxMzk5LDEy IEBAIHB1YmxpYzoKICAgLyogVGhlIHZlY3RvciB0eXBlIGZvciBwZXJmb3JtaW5nIHRoZSBhY3R1 YWwgcmVkdWN0aW9uLiAgKi8KICAgdHJlZSByZWR1Y192ZWN0eXBlOwogCisgIC8qIEZvciBsb29w IHJlZHVjdGlvbiB3aXRoIG11bHRpcGxlIHZlY3Rvcml6ZWQgcmVzdWx0cyAobmNvcGllcyA+IDEp LCBhCisgICAgIGxhbmUtcmVkdWNpbmcgb3BlcmF0aW9uIHBhcnRpY2lwYXRpbmcgaW4gaXQgbWF5 IG5vdCB1c2UgYWxsIG9mIHRob3NlCisgICAgIHJlc3VsdHMsIHRoaXMgZmllbGQgc3BlY2lmaWVz IHJlc3VsdCBpbmRleCBzdGFydGluZyBmcm9tIHdoaWNoIGFueQorICAgICBmb2xsb3dpbmcgbGFu ZC1yZWR1Y2luZyBvcGVyYXRpb24gd291bGQgYmUgYXNzaWduZWQgdG8uICAqLworICBpbnQgcmVk dWNfcmVzdWx0X3BvczsKKwogICAvKiBJZiBJU19SRURVQ19JTkZPIGlzIHRydWUgYW5kIGlmIHRo ZSB2ZWN0b3IgY29kZSBpcyBwZXJmb3JtaW5nCiAgICAgIE4gc2NhbGFyIHJlZHVjdGlvbnMgaW4g cGFyYWxsZWwsIHRoaXMgdmFyaWFibGUgZ2l2ZXMgdGhlIGluaXRpYWwKICAgICAgc2NhbGFyIHZh bHVlcyBvZiB0aG9zZSBOIHJlZHVjdGlvbnMuICAqLwpAQCAtMjE2Niw2ICsyMTcyLDEyIEBAIHZl Y3RfYXBwbHlfcnVudGltZV9wcm9maXRhYmlsaXR5X2NoZWNrX3AgKGxvb3BfdmVjX2luZm8gbG9v cF92aW5mbykKIAkgICYmIHRoID49IHZlY3RfdmZfZm9yX2Nvc3QgKGxvb3BfdmluZm8pKTsKIH0K IAoraW5saW5lIGJvb2wKK2xhbmVfcmVkdWNpbmdfb3BfcCAoY29kZV9oZWxwZXIgY29kZSkKK3sK KyAgcmV0dXJuIGNvZGUgPT0gRE9UX1BST0RfRVhQUiB8fCBjb2RlID09IFdJREVOX1NVTV9FWFBS IHx8IGNvZGUgPT0gU0FEX0VYUFI7Cit9CisKIC8qIFNvdXJjZSBsb2NhdGlvbiArIGhvdG5lc3Mg aW5mb3JtYXRpb24uICovCiBleHRlcm4gZHVtcF91c2VyX2xvY2F0aW9uX3QgdmVjdF9sb2NhdGlv bjsKIApAQCAtMjQzNCw2ICsyNDQ2LDggQEAgZXh0ZXJuIGxvb3BfdmVjX2luZm8gdmVjdF9jcmVh dGVfbG9vcF92aW5mbyAoY2xhc3MgbG9vcCAqLCB2ZWNfaW5mb19zaGFyZWQgKiwKIGV4dGVybiBi b29sIHZlY3Rvcml6YWJsZV9saXZlX29wZXJhdGlvbiAodmVjX2luZm8gKiwgc3RtdF92ZWNfaW5m bywKIAkJCQkJIHNscF90cmVlLCBzbHBfaW5zdGFuY2UsIGludCwKIAkJCQkJIGJvb2wsIHN0bXRf dmVjdG9yX2Zvcl9jb3N0ICopOworZXh0ZXJuIGJvb2wgdmVjdG9yaXphYmxlX2xhbmVfcmVkdWNp bmcgKGxvb3BfdmVjX2luZm8sIHN0bXRfdmVjX2luZm8sCisJCQkJCXNscF90cmVlLCBzdG10X3Zl Y3Rvcl9mb3JfY29zdCAqKTsKIGV4dGVybiBib29sIHZlY3Rvcml6YWJsZV9yZWR1Y3Rpb24gKGxv b3BfdmVjX2luZm8sIHN0bXRfdmVjX2luZm8sCiAJCQkJICAgIHNscF90cmVlLCBzbHBfaW5zdGFu Y2UsCiAJCQkJICAgIHN0bXRfdmVjdG9yX2Zvcl9jb3N0ICopOwotLSAKMi4xNy4xCgo= --_002_LV2PR01MB78397C320D7870B56652ACC6F7F52LV2PR01MB7839prod_--