From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 5279 invoked by alias); 15 Apr 2013 10:01:42 -0000 Mailing-List: contact libc-ports-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Post: List-Help: , Sender: libc-ports-owner@sourceware.org Received: (qmail 5268 invoked by uid 89); 15 Apr 2013 10:01:42 -0000 X-Spam-SWARE-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00,KAM_STOCKGEN,KHOP_RCVD_UNTRUST,KHOP_THREADED,RCVD_IN_DNSWL_LOW,RCVD_IN_HOSTKARMA_YE,TW_CP,TW_DR,TW_FP,TW_HW,TW_VF autolearn=no version=3.3.1 Received: from mail-ia0-f173.google.com (HELO mail-ia0-f173.google.com) (209.85.210.173) by sourceware.org (qpsmtpd/0.84/v0.84-167-ge50287c) with ESMTP; Mon, 15 Apr 2013 10:01:39 +0000 Received: by mail-ia0-f173.google.com with SMTP id j5so3648700iaf.18 for ; Mon, 15 Apr 2013 03:01:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type:x-gm-message-state; bh=l4Y3QhcXUWcxQICSsdXnYXIHnIq8XYmG7t4vSM0AXSs=; b=LtG7x4T+IZ4VTi6p2xS18/jqmfGXtmYUcJRNgHugjPZwtTW8F179FC87wZLmwEc9SI arQAIQTDcBHW3SI3yNE7Q/bA85rqdgdytAp74xSGUX/TX1XrqqVLaW9VWi+uaPgVHczG gcMF8dCW/4w4s/EJ2OHgEe/91gWvRYE/OZwjEPpr6q8NjR76tIpZs8WuBHBGJMmpWYcs VaDeU2T15UnHewzrdafwhEx3rRBM38q6/1hRKF4QC4ElSBhjb8KU2asR3oMuuKrZUIx9 7lMyBTgHpcjZvcPlHB1PQVFflkhzH6aDCXJ6ccOevUOZ3p/CS2QzW0edVc58xsDXBUg3 Y7Jg== MIME-Version: 1.0 X-Received: by 10.50.77.48 with SMTP id p16mr4794594igw.53.1366020097920; Mon, 15 Apr 2013 03:01:37 -0700 (PDT) Received: by 10.64.100.174 with HTTP; Mon, 15 Apr 2013 03:01:37 -0700 (PDT) In-Reply-To: <516BCEE5.9070809@linaro.org> References: <516BCEE5.9070809@linaro.org> Date: Mon, 15 Apr 2013 10:01:00 -0000 Message-ID: Subject: Re: [PATCH] ARM: Add Cortex-A15 optimized NEON and VFP memcpy routines, with IFUNC. From: Will Newton To: libc-ports@sourceware.org Cc: patches@linaro.org Content-Type: multipart/mixed; boundary=047d7bdcaaa4b165b604da6356c9 X-Gm-Message-State: ALoCoQlH8CKKFEhagY3F+WdFMyRMCUIFnbbRC3Ih3no7ky8pyQB0rbXXCe/4mDXEajsI4gE5nGGy X-Virus-Found: No X-SW-Source: 2013-04/txt/msg00058.txt.bz2 --047d7bdcaaa4b165b604da6356c9 Content-Type: text/plain; charset=ISO-8859-1 Content-length: 28946 Attached are a set of benchmarks of the new code versus the existing memcpy implementation on a Cortex-A15 platform. On 15 April 2013 10:56, Will Newton wrote: > > Add a high performance memcpy routine optimized for Cortex-A15 with > variants for use in the presence of NEON and VFP hardware, selected > at runtime using indirect function support. > > This was tested on armv7l-unknown-linux-gnueabihf. One new testsuite > failure was introduced (elf/ifuncmain5picstatic) which was caused by > a bug in ld. A fix for that ld issue has been submitted here: > > http://sourceware.org/ml/binutils/2013-04/msg00143.html > > ports/ChangeLog.arm: > > 2013-04-15 Will Newton > > * sysdeps/arm/armv7/multiarch/Makefile: New file. > * sysdeps/arm/armv7/multiarch/ifunc-impl-list.c: Likewise. > * sysdeps/arm/armv7/multiarch/memcpy.S: Likewise. > * sysdeps/arm/armv7/multiarch/memcpy_impl.S: Likewise. > * sysdeps/arm/armv7/multiarch/memcpy_neon.S: Likewise. > * sysdeps/arm/armv7/multiarch/memcpy_vfp.S: Likewise. > > Signed-off-by: Will Newton > --- > ports/sysdeps/arm/armv7/multiarch/Makefile | 3 + > .../sysdeps/arm/armv7/multiarch/ifunc-impl-list.c | 46 ++ > ports/sysdeps/arm/armv7/multiarch/memcpy.S | 96 ++++ > ports/sysdeps/arm/armv7/multiarch/memcpy_impl.S | 600 +++++++++++++++++++++ > ports/sysdeps/arm/armv7/multiarch/memcpy_neon.S | 3 + > ports/sysdeps/arm/armv7/multiarch/memcpy_vfp.S | 3 + > 6 files changed, 751 insertions(+) > create mode 100644 ports/sysdeps/arm/armv7/multiarch/Makefile > create mode 100644 ports/sysdeps/arm/armv7/multiarch/ifunc-impl-list.c > create mode 100644 ports/sysdeps/arm/armv7/multiarch/memcpy.S > create mode 100644 ports/sysdeps/arm/armv7/multiarch/memcpy_impl.S > create mode 100644 ports/sysdeps/arm/armv7/multiarch/memcpy_neon.S > create mode 100644 ports/sysdeps/arm/armv7/multiarch/memcpy_vfp.S > > diff --git a/ports/sysdeps/arm/armv7/multiarch/Makefile b/ports/sysdeps/arm/armv7/multiarch/Makefile > new file mode 100644 > index 0000000..e834cc9 > --- /dev/null > +++ b/ports/sysdeps/arm/armv7/multiarch/Makefile > @@ -0,0 +1,3 @@ > +ifeq ($(subdir),string) > +sysdep_routines += memcpy_neon memcpy_vfp > +endif > diff --git a/ports/sysdeps/arm/armv7/multiarch/ifunc-impl-list.c b/ports/sysdeps/arm/armv7/multiarch/ifunc-impl-list.c > new file mode 100644 > index 0000000..176288b > --- /dev/null > +++ b/ports/sysdeps/arm/armv7/multiarch/ifunc-impl-list.c > @@ -0,0 +1,46 @@ > +/* Enumerate available IFUNC implementations of a function. arm version. > + Copyright (C) 2013 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + . */ > + > +#include > +#include > +#include > +#include > +#include > +#include > + > +/* Fill ARRAY of MAX elements with IFUNC implementations for function > + NAME and return the number of valid entries. */ > + > +size_t > +__libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, > + size_t max) > +{ > + size_t i = 0; > + int hwcap; > + > + hwcap = GLRO(dl_hwcap); > + > + IFUNC_IMPL (i, name, memcpy, > + IFUNC_IMPL_ADD (array, i, memcpy, hwcap & HWCAP_ARM_NEON, > + __memcpy_neon) > + IFUNC_IMPL_ADD (array, i, memcpy, hwcap & HWCAP_ARM_VFPv3, > + __memcpy_vfp) > + IFUNC_IMPL_ADD (array, i, memcpy, 1, __memcpy_arm)); > + > + return i; > +} > diff --git a/ports/sysdeps/arm/armv7/multiarch/memcpy.S b/ports/sysdeps/arm/armv7/multiarch/memcpy.S > new file mode 100644 > index 0000000..a9e2faf > --- /dev/null > +++ b/ports/sysdeps/arm/armv7/multiarch/memcpy.S > @@ -0,0 +1,96 @@ > +/* Multiple versions of memcpy > + All versions must be listed in ifunc-impl-list.c. > + Copyright (C) 2013 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + . */ > + > +#include > +#include > +#include > + > +#if !defined NOT_IN_libc > + .text > +ENTRY(memcpy) > + .type memcpy, %gnu_indirect_function > +# ifdef PIC > + ldr a3, 1f > +0: add a3, pc, a3 > +# endif > + > + tst a1, #HWCAP_ARM_NEON > + beq .Lno_neon > +# ifdef PIC > + ldr a4, .Lmemcpy_neon > + ldr r0, [a3, a4] > +# else > + ldr r0, .Lmemcpy_neon > +# endif > + b .Lreturn > +.Lno_neon: > + > + tst a1, #HWCAP_ARM_VFP > + beq .Lno_vfp > +# ifdef PIC > + ldr a4, .Lmemcpy_vfp > + ldr r0, [a3, a4] > +# else > + ldr r0, .Lmemcpy_vfp > +# endif > + b .Lreturn > +.Lno_vfp: > +# ifdef PIC > + ldr a4, .Lmemcpy_arm > + ldr r0, [a3, a4] > +# else > + ldr r0, .Lmemcpy_arm > +# endif > + > +.Lreturn: > + DO_RET(lr) > + > +# ifdef PIC > +1: .long _GLOBAL_OFFSET_TABLE_ - 0b - PC_OFS > +.Lmemcpy_neon: > + .long C_SYMBOL_NAME(__memcpy_neon)(GOT) > +.Lmemcpy_vfp: > + .long C_SYMBOL_NAME(__memcpy_vfp)(GOT) > +.Lmemcpy_arm: > + .long C_SYMBOL_NAME(__memcpy_arm)(GOT) > +# else > +.Lmemcpy_neon: > + .long C_SYMBOL_NAME(__memcpy_neon) > +.Lmemcpy_vfp: > + .long C_SYMBOL_NAME(__memcpy_vfp) > +.Lmemcpy_arm: > + .long C_SYMBOL_NAME(__memcpy_arm) > +# endif > + > +END(memcpy) > + > +libc_hidden_builtin_def (memcpy) > + > +#undef libc_hidden_builtin_def > +#define libc_hidden_builtin_def(name) > +#undef weak_alias > +#define weak_alias(x, y) > +#undef libc_hidden_def > +#define libc_hidden_def(name) > + > +#define memcpy __memcpy_arm > + > +#endif > + > +#include "memcpy_impl.S" > diff --git a/ports/sysdeps/arm/armv7/multiarch/memcpy_impl.S b/ports/sysdeps/arm/armv7/multiarch/memcpy_impl.S > new file mode 100644 > index 0000000..2c466d25 > --- /dev/null > +++ b/ports/sysdeps/arm/armv7/multiarch/memcpy_impl.S > @@ -0,0 +1,600 @@ > +/* NEON/VFP/ARM version of memcpy optimized for Cortex-A15. > + Copyright (C) 2013 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + . > + > + This memcpy routine is optimised for Cortex-A15 cores and takes advantage > + of VFP or NEON when built with the appropriate flags. > + > + Assumptions: > + > + ARMv6 (ARMv7-a if using Neon) > + ARM state > + Unaligned accesses > + LDRD/STRD support unaligned word accesses > + > + */ > + > +#include > + > + .syntax unified > + /* This implementation requires ARM state. */ > + .arm > + > +#ifdef MEMCPY_NEON > + > + .fpu neon > + .arch armv7-a > +# define FRAME_SIZE 4 > +# define USE_VFP > +# define USE_NEON > + > +#elif defined (MEMCPY_VFP) > + > + .arch armv6 > + .fpu vfpv2 > +# define FRAME_SIZE 32 > +# define USE_VFP > + > +#else > + .arch armv6 > +# define FRAME_SIZE 32 > + > +#endif > + > +/* Old versions of GAS incorrectly implement the NEON align semantics. */ > +#ifdef BROKEN_ASM_NEON_ALIGN > +#define ALIGN(addr, align) addr,:align > +#else > +#define ALIGN(addr, align) addr:align > +#endif > + > +#define PC_OFFSET 8 /* PC pipeline compensation. */ > +#define INSN_SIZE 4 > + > +/* Call parameters. */ > +#define dstin r0 > +#define src r1 > +#define count r2 > + > +/* Locals. */ > +#define tmp1 r3 > +#define dst ip > +#define tmp2 r10 > + > +#ifndef USE_NEON > +/* For bulk copies using GP registers. */ > +#define A_l r2 /* Call-clobbered. */ > +#define A_h r3 /* Call-clobbered. */ > +#define B_l r4 > +#define B_h r5 > +#define C_l r6 > +#define C_h r7 > +#define D_l r8 > +#define D_h r9 > +#endif > + > +/* Number of lines ahead to pre-fetch data. If you change this the code > + below will need adjustment to compensate. */ > + > +#define prefetch_lines 5 > + > +#ifdef USE_VFP > + .macro cpy_line_vfp vreg, base > + vstr \vreg, [dst, #\base] > + vldr \vreg, [src, #\base] > + vstr d0, [dst, #\base + 8] > + vldr d0, [src, #\base + 8] > + vstr d1, [dst, #\base + 16] > + vldr d1, [src, #\base + 16] > + vstr d2, [dst, #\base + 24] > + vldr d2, [src, #\base + 24] > + vstr \vreg, [dst, #\base + 32] > + vldr \vreg, [src, #\base + prefetch_lines * 64 - 32] > + vstr d0, [dst, #\base + 40] > + vldr d0, [src, #\base + 40] > + vstr d1, [dst, #\base + 48] > + vldr d1, [src, #\base + 48] > + vstr d2, [dst, #\base + 56] > + vldr d2, [src, #\base + 56] > + .endm > + > + .macro cpy_tail_vfp vreg, base > + vstr \vreg, [dst, #\base] > + vldr \vreg, [src, #\base] > + vstr d0, [dst, #\base + 8] > + vldr d0, [src, #\base + 8] > + vstr d1, [dst, #\base + 16] > + vldr d1, [src, #\base + 16] > + vstr d2, [dst, #\base + 24] > + vldr d2, [src, #\base + 24] > + vstr \vreg, [dst, #\base + 32] > + vstr d0, [dst, #\base + 40] > + vldr d0, [src, #\base + 40] > + vstr d1, [dst, #\base + 48] > + vldr d1, [src, #\base + 48] > + vstr d2, [dst, #\base + 56] > + vldr d2, [src, #\base + 56] > + .endm > +#endif > + > + .p2align 6 > +ENTRY(memcpy) > + > + mov dst, dstin /* Preserve dstin, we need to return it. */ > + cmp count, #64 > + bge .Lcpy_not_short > + /* Deal with small copies quickly by dropping straight into the > + exit block. */ > + > +.Ltail63unaligned: > +#ifdef USE_NEON > + and tmp1, count, #0x38 > + rsb tmp1, tmp1, #(56 - PC_OFFSET + INSN_SIZE) > + add pc, pc, tmp1 > + vld1.8 {d0}, [src]! /* 14 words to go. */ > + vst1.8 {d0}, [dst]! > + vld1.8 {d0}, [src]! /* 12 words to go. */ > + vst1.8 {d0}, [dst]! > + vld1.8 {d0}, [src]! /* 10 words to go. */ > + vst1.8 {d0}, [dst]! > + vld1.8 {d0}, [src]! /* 8 words to go. */ > + vst1.8 {d0}, [dst]! > + vld1.8 {d0}, [src]! /* 6 words to go. */ > + vst1.8 {d0}, [dst]! > + vld1.8 {d0}, [src]! /* 4 words to go. */ > + vst1.8 {d0}, [dst]! > + vld1.8 {d0}, [src]! /* 2 words to go. */ > + vst1.8 {d0}, [dst]! > + > + tst count, #4 > + ldrne tmp1, [src], #4 > + strne tmp1, [dst], #4 > +#else > + /* Copy up to 15 full words of data. May not be aligned. */ > + /* Cannot use VFP for unaligned data. */ > + and tmp1, count, #0x3c > + add dst, dst, tmp1 > + add src, src, tmp1 > + rsb tmp1, tmp1, #(60 - PC_OFFSET/2 + INSN_SIZE/2) > + /* Jump directly into the sequence below at the correct offset. */ > + add pc, pc, tmp1, lsl #1 > + > + ldr tmp1, [src, #-60] /* 15 words to go. */ > + str tmp1, [dst, #-60] > + > + ldr tmp1, [src, #-56] /* 14 words to go. */ > + str tmp1, [dst, #-56] > + ldr tmp1, [src, #-52] > + str tmp1, [dst, #-52] > + > + ldr tmp1, [src, #-48] /* 12 words to go. */ > + str tmp1, [dst, #-48] > + ldr tmp1, [src, #-44] > + str tmp1, [dst, #-44] > + > + ldr tmp1, [src, #-40] /* 10 words to go. */ > + str tmp1, [dst, #-40] > + ldr tmp1, [src, #-36] > + str tmp1, [dst, #-36] > + > + ldr tmp1, [src, #-32] /* 8 words to go. */ > + str tmp1, [dst, #-32] > + ldr tmp1, [src, #-28] > + str tmp1, [dst, #-28] > + > + ldr tmp1, [src, #-24] /* 6 words to go. */ > + str tmp1, [dst, #-24] > + ldr tmp1, [src, #-20] > + str tmp1, [dst, #-20] > + > + ldr tmp1, [src, #-16] /* 4 words to go. */ > + str tmp1, [dst, #-16] > + ldr tmp1, [src, #-12] > + str tmp1, [dst, #-12] > + > + ldr tmp1, [src, #-8] /* 2 words to go. */ > + str tmp1, [dst, #-8] > + ldr tmp1, [src, #-4] > + str tmp1, [dst, #-4] > +#endif > + > + lsls count, count, #31 > + ldrhcs tmp1, [src], #2 > + ldrbne src, [src] /* Src is dead, use as a scratch. */ > + strhcs tmp1, [dst], #2 > + strbne src, [dst] > + bx lr > + > +.Lcpy_not_short: > + /* At least 64 bytes to copy, but don't know the alignment yet. */ > + str tmp2, [sp, #-FRAME_SIZE]! > + and tmp2, src, #3 > + and tmp1, dst, #3 > + cmp tmp1, tmp2 > + bne .Lcpy_notaligned > + > +#ifdef USE_VFP > + /* Magic dust alert! Force VFP on Cortex-A9. Experiments show > + that the FP pipeline is much better at streaming loads and > + stores. This is outside the critical loop. */ > + vmov.f32 s0, s0 > +#endif > + > + /* SRC and DST have the same mutual 32-bit alignment, but we may > + still need to pre-copy some bytes to get to natural alignment. > + We bring DST into full 64-bit alignment. */ > + lsls tmp2, dst, #29 > + beq 1f > + rsbs tmp2, tmp2, #0 > + sub count, count, tmp2, lsr #29 > + ldrmi tmp1, [src], #4 > + strmi tmp1, [dst], #4 > + lsls tmp2, tmp2, #2 > + ldrhcs tmp1, [src], #2 > + ldrbne tmp2, [src], #1 > + strhcs tmp1, [dst], #2 > + strbne tmp2, [dst], #1 > + > +1: > + subs tmp2, count, #64 /* Use tmp2 for count. */ > + blt .Ltail63aligned > + > + cmp tmp2, #512 > + bge .Lcpy_body_long > + > +.Lcpy_body_medium: /* Count in tmp2. */ > +#ifdef USE_VFP > +1: > + vldr d0, [src, #0] > + subs tmp2, tmp2, #64 > + vldr d1, [src, #8] > + vstr d0, [dst, #0] > + vldr d0, [src, #16] > + vstr d1, [dst, #8] > + vldr d1, [src, #24] > + vstr d0, [dst, #16] > + vldr d0, [src, #32] > + vstr d1, [dst, #24] > + vldr d1, [src, #40] > + vstr d0, [dst, #32] > + vldr d0, [src, #48] > + vstr d1, [dst, #40] > + vldr d1, [src, #56] > + vstr d0, [dst, #48] > + add src, src, #64 > + vstr d1, [dst, #56] > + add dst, dst, #64 > + bge 1b > + tst tmp2, #0x3f > + beq .Ldone > + > +.Ltail63aligned: /* Count in tmp2. */ > + and tmp1, tmp2, #0x38 > + add dst, dst, tmp1 > + add src, src, tmp1 > + rsb tmp1, tmp1, #(56 - PC_OFFSET + INSN_SIZE) > + add pc, pc, tmp1 > + > + vldr d0, [src, #-56] /* 14 words to go. */ > + vstr d0, [dst, #-56] > + vldr d0, [src, #-48] /* 12 words to go. */ > + vstr d0, [dst, #-48] > + vldr d0, [src, #-40] /* 10 words to go. */ > + vstr d0, [dst, #-40] > + vldr d0, [src, #-32] /* 8 words to go. */ > + vstr d0, [dst, #-32] > + vldr d0, [src, #-24] /* 6 words to go. */ > + vstr d0, [dst, #-24] > + vldr d0, [src, #-16] /* 4 words to go. */ > + vstr d0, [dst, #-16] > + vldr d0, [src, #-8] /* 2 words to go. */ > + vstr d0, [dst, #-8] > +#else > + sub src, src, #8 > + sub dst, dst, #8 > +1: > + ldrd A_l, A_h, [src, #8] > + strd A_l, A_h, [dst, #8] > + ldrd A_l, A_h, [src, #16] > + strd A_l, A_h, [dst, #16] > + ldrd A_l, A_h, [src, #24] > + strd A_l, A_h, [dst, #24] > + ldrd A_l, A_h, [src, #32] > + strd A_l, A_h, [dst, #32] > + ldrd A_l, A_h, [src, #40] > + strd A_l, A_h, [dst, #40] > + ldrd A_l, A_h, [src, #48] > + strd A_l, A_h, [dst, #48] > + ldrd A_l, A_h, [src, #56] > + strd A_l, A_h, [dst, #56] > + ldrd A_l, A_h, [src, #64]! > + strd A_l, A_h, [dst, #64]! > + subs tmp2, tmp2, #64 > + bge 1b > + tst tmp2, #0x3f > + bne 1f > + ldr tmp2,[sp], #FRAME_SIZE > + bx lr > +1: > + add src, src, #8 > + add dst, dst, #8 > + > +.Ltail63aligned: /* Count in tmp2. */ > + /* Copy up to 7 d-words of data. Similar to Ltail63unaligned, but > + we know that the src and dest are 32-bit aligned so we can use > + LDRD/STRD to improve efficiency. */ > + /* TMP2 is now negative, but we don't care about that. The bottom > + six bits still tell us how many bytes are left to copy. */ > + > + and tmp1, tmp2, #0x38 > + add dst, dst, tmp1 > + add src, src, tmp1 > + rsb tmp1, tmp1, #(56 - PC_OFFSET + INSN_SIZE) > + add pc, pc, tmp1 > + ldrd A_l, A_h, [src, #-56] /* 14 words to go. */ > + strd A_l, A_h, [dst, #-56] > + ldrd A_l, A_h, [src, #-48] /* 12 words to go. */ > + strd A_l, A_h, [dst, #-48] > + ldrd A_l, A_h, [src, #-40] /* 10 words to go. */ > + strd A_l, A_h, [dst, #-40] > + ldrd A_l, A_h, [src, #-32] /* 8 words to go. */ > + strd A_l, A_h, [dst, #-32] > + ldrd A_l, A_h, [src, #-24] /* 6 words to go. */ > + strd A_l, A_h, [dst, #-24] > + ldrd A_l, A_h, [src, #-16] /* 4 words to go. */ > + strd A_l, A_h, [dst, #-16] > + ldrd A_l, A_h, [src, #-8] /* 2 words to go. */ > + strd A_l, A_h, [dst, #-8] > + > +#endif > + tst tmp2, #4 > + ldrne tmp1, [src], #4 > + strne tmp1, [dst], #4 > + lsls tmp2, tmp2, #31 /* Count (tmp2) now dead. */ > + ldrhcs tmp1, [src], #2 > + ldrbne tmp2, [src] > + strhcs tmp1, [dst], #2 > + strbne tmp2, [dst] > + > +.Ldone: > + ldr tmp2, [sp], #FRAME_SIZE > + bx lr > + > +.Lcpy_body_long: /* Count in tmp2. */ > + > + /* Long copy. We know that there's at least (prefetch_lines * 64) > + bytes to go. */ > +#ifdef USE_VFP > + /* Don't use PLD. Instead, read some data in advance of the current > + copy position into a register. This should act like a PLD > + operation but we won't have to repeat the transfer. */ > + > + vldr d3, [src, #0] > + vldr d4, [src, #64] > + vldr d5, [src, #128] > + vldr d6, [src, #192] > + vldr d7, [src, #256] > + > + vldr d0, [src, #8] > + vldr d1, [src, #16] > + vldr d2, [src, #24] > + add src, src, #32 > + > + subs tmp2, tmp2, #prefetch_lines * 64 * 2 > + blt 2f > +1: > + cpy_line_vfp d3, 0 > + cpy_line_vfp d4, 64 > + cpy_line_vfp d5, 128 > + add dst, dst, #3 * 64 > + add src, src, #3 * 64 > + cpy_line_vfp d6, 0 > + cpy_line_vfp d7, 64 > + add dst, dst, #2 * 64 > + add src, src, #2 * 64 > + subs tmp2, tmp2, #prefetch_lines * 64 > + bge 1b > + > +2: > + cpy_tail_vfp d3, 0 > + cpy_tail_vfp d4, 64 > + cpy_tail_vfp d5, 128 > + add src, src, #3 * 64 > + add dst, dst, #3 * 64 > + cpy_tail_vfp d6, 0 > + vstr d7, [dst, #64] > + vldr d7, [src, #64] > + vstr d0, [dst, #64 + 8] > + vldr d0, [src, #64 + 8] > + vstr d1, [dst, #64 + 16] > + vldr d1, [src, #64 + 16] > + vstr d2, [dst, #64 + 24] > + vldr d2, [src, #64 + 24] > + vstr d7, [dst, #64 + 32] > + add src, src, #96 > + vstr d0, [dst, #64 + 40] > + vstr d1, [dst, #64 + 48] > + vstr d2, [dst, #64 + 56] > + add dst, dst, #128 > + add tmp2, tmp2, #prefetch_lines * 64 > + b .Lcpy_body_medium > +#else > + /* Long copy. Use an SMS style loop to maximize the I/O > + bandwidth of the core. We don't have enough spare registers > + to synthesise prefetching, so use PLD operations. */ > + /* Pre-bias src and dst. */ > + sub src, src, #8 > + sub dst, dst, #8 > + pld [src, #8] > + pld [src, #72] > + subs tmp2, tmp2, #64 > + pld [src, #136] > + ldrd A_l, A_h, [src, #8] > + strd B_l, B_h, [sp, #8] > + ldrd B_l, B_h, [src, #16] > + strd C_l, C_h, [sp, #16] > + ldrd C_l, C_h, [src, #24] > + strd D_l, D_h, [sp, #24] > + pld [src, #200] > + ldrd D_l, D_h, [src, #32]! > + b 1f > + .p2align 6 > +2: > + pld [src, #232] > + strd A_l, A_h, [dst, #40] > + ldrd A_l, A_h, [src, #40] > + strd B_l, B_h, [dst, #48] > + ldrd B_l, B_h, [src, #48] > + strd C_l, C_h, [dst, #56] > + ldrd C_l, C_h, [src, #56] > + strd D_l, D_h, [dst, #64]! > + ldrd D_l, D_h, [src, #64]! > + subs tmp2, tmp2, #64 > +1: > + strd A_l, A_h, [dst, #8] > + ldrd A_l, A_h, [src, #8] > + strd B_l, B_h, [dst, #16] > + ldrd B_l, B_h, [src, #16] > + strd C_l, C_h, [dst, #24] > + ldrd C_l, C_h, [src, #24] > + strd D_l, D_h, [dst, #32] > + ldrd D_l, D_h, [src, #32] > + bcs 2b > + /* Save the remaining bytes and restore the callee-saved regs. */ > + strd A_l, A_h, [dst, #40] > + add src, src, #40 > + strd B_l, B_h, [dst, #48] > + ldrd B_l, B_h, [sp, #8] > + strd C_l, C_h, [dst, #56] > + ldrd C_l, C_h, [sp, #16] > + strd D_l, D_h, [dst, #64] > + ldrd D_l, D_h, [sp, #24] > + add dst, dst, #72 > + tst tmp2, #0x3f > + bne .Ltail63aligned > + ldr tmp2, [sp], #FRAME_SIZE > + bx lr > +#endif > + > +.Lcpy_notaligned: > + pld [src] > + pld [src, #64] > + /* There's at least 64 bytes to copy, but there is no mutual > + alignment. */ > + /* Bring DST to 64-bit alignment. */ > + lsls tmp2, dst, #29 > + pld [src, #(2 * 64)] > + beq 1f > + rsbs tmp2, tmp2, #0 > + sub count, count, tmp2, lsr #29 > + ldrmi tmp1, [src], #4 > + strmi tmp1, [dst], #4 > + lsls tmp2, tmp2, #2 > + ldrbne tmp1, [src], #1 > + ldrhcs tmp2, [src], #2 > + strbne tmp1, [dst], #1 > + strhcs tmp2, [dst], #2 > +1: > + pld [src, #(3 * 64)] > + subs count, count, #64 > + ldrmi tmp2, [sp], #FRAME_SIZE > + bmi .Ltail63unaligned > + pld [src, #(4 * 64)] > + > +#ifdef USE_NEON > + vld1.8 {d0-d3}, [src]! > + vld1.8 {d4-d7}, [src]! > + subs count, count, #64 > + bmi 2f > +1: > + pld [src, #(4 * 64)] > + vst1.8 {d0-d3}, [ALIGN (dst, 64)]! > + vld1.8 {d0-d3}, [src]! > + vst1.8 {d4-d7}, [ALIGN (dst, 64)]! > + vld1.8 {d4-d7}, [src]! > + subs count, count, #64 > + bpl 1b > +2: > + vst1.8 {d0-d3}, [ALIGN (dst, 64)]! > + vst1.8 {d4-d7}, [ALIGN (dst, 64)]! > + ands count, count, #0x3f > +#else > + /* Use an SMS style loop to maximize the I/O bandwidth. */ > + sub src, src, #4 > + sub dst, dst, #8 > + subs tmp2, count, #64 /* Use tmp2 for count. */ > + ldr A_l, [src, #4] > + ldr A_h, [src, #8] > + strd B_l, B_h, [sp, #8] > + ldr B_l, [src, #12] > + ldr B_h, [src, #16] > + strd C_l, C_h, [sp, #16] > + ldr C_l, [src, #20] > + ldr C_h, [src, #24] > + strd D_l, D_h, [sp, #24] > + ldr D_l, [src, #28] > + ldr D_h, [src, #32]! > + b 1f > + .p2align 6 > +2: > + pld [src, #(5 * 64) - (32 - 4)] > + strd A_l, A_h, [dst, #40] > + ldr A_l, [src, #36] > + ldr A_h, [src, #40] > + strd B_l, B_h, [dst, #48] > + ldr B_l, [src, #44] > + ldr B_h, [src, #48] > + strd C_l, C_h, [dst, #56] > + ldr C_l, [src, #52] > + ldr C_h, [src, #56] > + strd D_l, D_h, [dst, #64]! > + ldr D_l, [src, #60] > + ldr D_h, [src, #64]! > + subs tmp2, tmp2, #64 > +1: > + strd A_l, A_h, [dst, #8] > + ldr A_l, [src, #4] > + ldr A_h, [src, #8] > + strd B_l, B_h, [dst, #16] > + ldr B_l, [src, #12] > + ldr B_h, [src, #16] > + strd C_l, C_h, [dst, #24] > + ldr C_l, [src, #20] > + ldr C_h, [src, #24] > + strd D_l, D_h, [dst, #32] > + ldr D_l, [src, #28] > + ldr D_h, [src, #32] > + bcs 2b > + > + /* Save the remaining bytes and restore the callee-saved regs. */ > + strd A_l, A_h, [dst, #40] > + add src, src, #36 > + strd B_l, B_h, [dst, #48] > + ldrd B_l, B_h, [sp, #8] > + strd C_l, C_h, [dst, #56] > + ldrd C_l, C_h, [sp, #16] > + strd D_l, D_h, [dst, #64] > + ldrd D_l, D_h, [sp, #24] > + add dst, dst, #72 > + ands count, tmp2, #0x3f > +#endif > + ldr tmp2, [sp], #FRAME_SIZE > + bne .Ltail63unaligned > + bx lr > + > +END(memcpy) > +libc_hidden_builtin_def (memcpy) > diff --git a/ports/sysdeps/arm/armv7/multiarch/memcpy_neon.S b/ports/sysdeps/arm/armv7/multiarch/memcpy_neon.S > new file mode 100644 > index 0000000..c0ef1f8 > --- /dev/null > +++ b/ports/sysdeps/arm/armv7/multiarch/memcpy_neon.S > @@ -0,0 +1,3 @@ > +#define MEMCPY_NEON > +#define memcpy __memcpy_neon > +#include "memcpy_impl.S" > diff --git a/ports/sysdeps/arm/armv7/multiarch/memcpy_vfp.S b/ports/sysdeps/arm/armv7/multiarch/memcpy_vfp.S > new file mode 100644 > index 0000000..d21b702 > --- /dev/null > +++ b/ports/sysdeps/arm/armv7/multiarch/memcpy_vfp.S > @@ -0,0 +1,3 @@ > +#define MEMCPY_VFP > +#define memcpy __memcpy_vfp > +#include "memcpy_impl.S" > -- > 1.8.1.4 > -- Will Newton Toolchain Working Group, Linaro --047d7bdcaaa4b165b604da6356c9 Content-Type: text/plain; charset=US-ASCII; name="memcpy_benchmark.txt" Content-Disposition: attachment; filename="memcpy_benchmark.txt" Content-Transfer-Encoding: base64 X-Attachment-Id: f_hfjh39yg0 Content-length: 11879 YmVmb3JlOjg6MTAwMDAwMDAwMDoxOjE5LjU4NDcwMTogdG9vayAxOS41ODQ3 MDEgcyBmb3IgMTAwMDAwMDAwMCBjYWxscyB0byBtZW1jcHkgb2YgOCBieXRl cy4gIH43MTkuMDY5IE1CL3MgY29ycmVjdGVkLgphZnRlcjo4OjEwMDAwMDAw MDA6MToxMC42OTI2MDY6IHRvb2sgMTAuNjkyNjA2IHMgZm9yIDEwMDAwMDAw MDAgY2FsbHMgdG8gbWVtY3B5IG9mIDggYnl0ZXMuICB+NDQ0MC44NDIgTUIv cyBjb3JyZWN0ZWQuCmJlZm9yZTo4OjEwMDAwMDAwMDA6MjoyMC42NzU0Mzg6 IHRvb2sgMjAuNjc1NDM4IHMgZm9yIDEwMDAwMDAwMDAgY2FsbHMgdG8gbWVt Y3B5IG9mIDggYnl0ZXMuICB+NjUyLjAzOCBNQi9zIGNvcnJlY3RlZC4KYWZ0 ZXI6ODoxMDAwMDAwMDAwOjI6MTQuMzk1MTg0OiB0b29rIDE0LjM5NTE4NCBz IGZvciAxMDAwMDAwMDAwIGNhbGxzIHRvIG1lbWNweSBvZiA4IGJ5dGVzLiAg fjE0MDcuNDg2IE1CL3MgY29ycmVjdGVkLgpiZWZvcmU6ODoxMDAwMDAwMDAw OjQ6MTcuMTA4MzA3OiB0b29rIDE3LjEwODMwNyBzIGZvciAxMDAwMDAwMDAw IGNhbGxzIHRvIG1lbWNweSBvZiA4IGJ5dGVzLiAgfjkzNy45OTcgTUIvcyBj b3JyZWN0ZWQuCmFmdGVyOjg6MTAwMDAwMDAwMDo0OjExLjI4MjI4ODogdG9v ayAxMS4yODIyODggcyBmb3IgMTAwMDAwMDAwMCBjYWxscyB0byBtZW1jcHkg b2YgOCBieXRlcy4gIH4zMzA2LjA3OCBNQi9zIGNvcnJlY3RlZC4KYmVmb3Jl Ojg6MTAwMDAwMDAwMDo4OjE3LjExMTQyNTogdG9vayAxNy4xMTE0MjUgcyBm b3IgMTAwMDAwMDAwMCBjYWxscyB0byBtZW1jcHkgb2YgOCBieXRlcy4gIH45 MzcuNjM4IE1CL3MgY29ycmVjdGVkLgphZnRlcjo4OjEwMDAwMDAwMDA6ODox MC43NDE0NzI6IHRvb2sgMTAuNzQxNDcyIHMgZm9yIDEwMDAwMDAwMDAgY2Fs bHMgdG8gbWVtY3B5IG9mIDggYnl0ZXMuICB+NDMxOC4wMjQgTUIvcyBjb3Jy ZWN0ZWQuCmJlZm9yZToxNjoxMDAwMDAwMDAwOjE6MjEuNTk2NjgyOiB0b29r IDIxLjU5NjY4MiBzIGZvciAxMDAwMDAwMDAwIGNhbGxzIHRvIG1lbWNweSBv ZiAxNiBieXRlcy4gIH4xMjA4Ljg5NiBNQi9zIGNvcnJlY3RlZC4KYWZ0ZXI6 MTY6MTAwMDAwMDAwMDoxOjEyLjAxMzY1NzogdG9vayAxMi4wMTM2NTcgcyBm b3IgMTAwMDAwMDAwMCBjYWxscyB0byBtZW1jcHkgb2YgMTYgYnl0ZXMuICB+ NTAyMC44OTYgTUIvcyBjb3JyZWN0ZWQuCmJlZm9yZToxNjoxMDAwMDAwMDAw OjI6MjAuNjQ3ODU5OiB0b29rIDIwLjY0Nzg1OSBzIGZvciAxMDAwMDAwMDAw IGNhbGxzIHRvIG1lbWNweSBvZiAxNiBieXRlcy4gIH4xMzA3LjE1OCBNQi9z IGNvcnJlY3RlZC4KYWZ0ZXI6MTY6MTAwMDAwMDAwMDoyOjExLjUxNzc2NDog dG9vayAxMS41MTc3NjQgcyBmb3IgMTAwMDAwMDAwMCBjYWxscyB0byBtZW1j cHkgb2YgMTYgYnl0ZXMuICB+NTk5OS45MjQgTUIvcyBjb3JyZWN0ZWQuCmJl Zm9yZToxNjoxMDAwMDAwMDAwOjQ6MTkuMjMyNTgwOiB0b29rIDE5LjIzMjU4 MCBzIGZvciAxMDAwMDAwMDAwIGNhbGxzIHRvIG1lbWNweSBvZiAxNiBieXRl cy4gIH4xNDg3LjUwNCBNQi9zIGNvcnJlY3RlZC4KYWZ0ZXI6MTY6MTAwMDAw MDAwMDo0OjE1LjkyNDU2MjogdG9vayAxNS45MjQ1NjIgcyBmb3IgMTAwMDAw MDAwMCBjYWxscyB0byBtZW1jcHkgb2YgMTYgYnl0ZXMuICB+MjE5NS41MjEg TUIvcyBjb3JyZWN0ZWQuCmJlZm9yZToxNjoxMDAwMDAwMDAwOjg6MTguODc3 ODAyOiB0b29rIDE4Ljg3NzgwMiBzIGZvciAxMDAwMDAwMDAwIGNhbGxzIHRv IG1lbWNweSBvZiAxNiBieXRlcy4gIH4xNTQwLjc5MyBNQi9zIGNvcnJlY3Rl ZC4KYWZ0ZXI6MTY6MTAwMDAwMDAwMDo4OjExLjYyNDg3NzogdG9vayAxMS42 MjQ4NzcgcyBmb3IgMTAwMDAwMDAwMCBjYWxscyB0byBtZW1jcHkgb2YgMTYg Ynl0ZXMuICB+NTc1Ny40MzEgTUIvcyBjb3JyZWN0ZWQuCmJlZm9yZToyMDox MDAwMDAwMDAwOjE6MjEuODI3MjI5OiB0b29rIDIxLjgyNzIyOSBzIGZvciAx MDAwMDAwMDAwIGNhbGxzIHRvIG1lbWNweSBvZiAyMCBieXRlcy4gIH4xNDg0 LjAxNCBNQi9zIGNvcnJlY3RlZC4KYWZ0ZXI6MjA6MTAwMDAwMDAwMDoxOjEy LjM3MDE0OTogdG9vayAxMi4zNzAxNDkgcyBmb3IgMTAwMDAwMDAwMCBjYWxs cyB0byBtZW1jcHkgb2YgMjAgYnl0ZXMuICB+NTYxNy4yMDIgTUIvcyBjb3Jy ZWN0ZWQuCmJlZm9yZToyMDoxMDAwMDAwMDAwOjI6MjEuODI3ODM4OiB0b29r IDIxLjgyNzgzOCBzIGZvciAxMDAwMDAwMDAwIGNhbGxzIHRvIG1lbWNweSBv ZiAyMCBieXRlcy4gIH4xNDgzLjk0NCBNQi9zIGNvcnJlY3RlZC4KYWZ0ZXI6 MjA6MTAwMDAwMDAwMDoyOjE1LjkyMDE5MTogdG9vayAxNS45MjAxOTEgcyBm b3IgMTAwMDAwMDAwMCBjYWxscyB0byBtZW1jcHkgb2YgMjAgYnl0ZXMuICB+ Mjc0Ni4xMjggTUIvcyBjb3JyZWN0ZWQuCmJlZm9yZToyMDoxMDAwMDAwMDAw OjQ6MTkuNDY3NzMwOiB0b29rIDE5LjQ2NzczMCBzIGZvciAxMDAwMDAwMDAw IGNhbGxzIHRvIG1lbWNweSBvZiAyMCBieXRlcy4gIH4xODE3LjcxMiBNQi9z IGNvcnJlY3RlZC4KYWZ0ZXI6MjA6MTAwMDAwMDAwMDo0OjEyLjEzNzg5Njog dG9vayAxMi4xMzc4OTYgcyBmb3IgMTAwMDAwMDAwMCBjYWxscyB0byBtZW1j cHkgb2YgMjAgYnl0ZXMuICB+NjAyOS42MjQgTUIvcyBjb3JyZWN0ZWQuCmJl Zm9yZToyMDoxMDAwMDAwMDAwOjg6MTkuNDY3OTc2OiB0b29rIDE5LjQ2Nzk3 NiBzIGZvciAxMDAwMDAwMDAwIGNhbGxzIHRvIG1lbWNweSBvZiAyMCBieXRl cy4gIH4xODE3LjY2OSBNQi9zIGNvcnJlY3RlZC4KYWZ0ZXI6MjA6MTAwMDAw MDAwMDo4OjEyLjQwNTgzNTogdG9vayAxMi40MDU4MzUgcyBmb3IgMTAwMDAw MDAwMCBjYWxscyB0byBtZW1jcHkgb2YgMjAgYnl0ZXMuICB+NTU1OC43ODIg TUIvcyBjb3JyZWN0ZWQuCmJlZm9yZTozMToxMDAwMDAwMDAwOjE6MjQuMTg3 Mjk0OiB0b29rIDI0LjE4NzI5NCBzIGZvciAxMDAwMDAwMDAwIGNhbGxzIHRv IG1lbWNweSBvZiAzMSBieXRlcy4gIH4xOTQzLjM3MSBNQi9zIGNvcnJlY3Rl ZC4KYWZ0ZXI6MzE6MTAwMDAwMDAwMDoxOjExLjgwMzYxNDogdG9vayAxMS44 MDM2MTQgcyBmb3IgMTAwMDAwMDAwMCBjYWxscyB0byBtZW1jcHkgb2YgMzEg Ynl0ZXMuICB+MTA0NTAuMjUyIE1CL3MgY29ycmVjdGVkLgpiZWZvcmU6MzE6 MTAwMDAwMDAwMDoyOjI0LjE5MTUxOTogdG9vayAyNC4xOTE1MTkgcyBmb3Ig MTAwMDAwMDAwMCBjYWxscyB0byBtZW1jcHkgb2YgMzEgYnl0ZXMuICB+MTk0 Mi44MzEgTUIvcyBjb3JyZWN0ZWQuCmFmdGVyOjMxOjEwMDAwMDAwMDA6Mjox NS44OTQ0ODM6IHRvb2sgMTUuODk0NDgzIHMgZm9yIDEwMDAwMDAwMDAgY2Fs bHMgdG8gbWVtY3B5IG9mIDMxIGJ5dGVzLiAgfjQyNzIuMzEzIE1CL3MgY29y cmVjdGVkLgpiZWZvcmU6MzE6MTAwMDAwMDAwMDo0OjIxLjIzNzI5NDogdG9v ayAyMS4yMzcyOTQgcyBmb3IgMTAwMDAwMDAwMCBjYWxscyB0byBtZW1jcHkg b2YgMzEgYnl0ZXMuICB+MjQxMC44ODIgTUIvcyBjb3JyZWN0ZWQuCmFmdGVy OjMxOjEwMDAwMDAwMDA6NDoxMy41ODkyOTM6IHRvb2sgMTMuNTg5MjkzIHMg Zm9yIDEwMDAwMDAwMDAgY2FsbHMgdG8gbWVtY3B5IG9mIDMxIGJ5dGVzLiAg fjY0MDYuNDcyIE1CL3MgY29ycmVjdGVkLgpiZWZvcmU6MzE6MTAwMDAwMDAw MDo4OjIxLjIzOTU2NDogdG9vayAyMS4yMzk1NjQgcyBmb3IgMTAwMDAwMDAw MCBjYWxscyB0byBtZW1jcHkgb2YgMzEgYnl0ZXMuICB+MjQxMC40MzUgTUIv cyBjb3JyZWN0ZWQuCmFmdGVyOjMxOjEwMDAwMDAwMDA6ODoxMy42NzE2MzM6 IHRvb2sgMTMuNjcxNjMzIHMgZm9yIDEwMDAwMDAwMDAgY2FsbHMgdG8gbWVt Y3B5IG9mIDMxIGJ5dGVzLiAgfjYyOTQuMTY2IE1CL3MgY29ycmVjdGVkLgpi ZWZvcmU6MzI6MTAwMDAwMDAwMDoxOjI0LjE4NzMwNTogdG9vayAyNC4xODcz MDUgcyBmb3IgMTAwMDAwMDAwMCBjYWxscyB0byBtZW1jcHkgb2YgMzIgYnl0 ZXMuICB+MjAwNi4wNTkgTUIvcyBjb3JyZWN0ZWQuCmFmdGVyOjMyOjEwMDAw MDAwMDA6MToxMi44NjU0NjA6IHRvb2sgMTIuODY1NDYwIHMgZm9yIDEwMDAw MDAwMDAgY2FsbHMgdG8gbWVtY3B5IG9mIDMyIGJ5dGVzLiAgfjc4NDMuNDAy IE1CL3MgY29ycmVjdGVkLgpiZWZvcmU6MzI6MTAwMDAwMDAwMDoyOjI0LjE4 NzAzODogdG9vayAyNC4xODcwMzggcyBmb3IgMTAwMDAwMDAwMCBjYWxscyB0 byBtZW1jcHkgb2YgMzIgYnl0ZXMuICB+MjAwNi4wOTQgTUIvcyBjb3JyZWN0 ZWQuCmFmdGVyOjMyOjEwMDAwMDAwMDA6MjoxMi42ODU2MTk6IHRvb2sgMTIu Njg1NjE5IHMgZm9yIDEwMDAwMDAwMDAgY2FsbHMgdG8gbWVtY3B5IG9mIDMy IGJ5dGVzLiAgfjgyMjMuNTA0IE1CL3MgY29ycmVjdGVkLgpiZWZvcmU6MzI6 MTAwMDAwMDAwMDo0OjIzLjAwNzU2MDogdG9vayAyMy4wMDc1NjAgcyBmb3Ig MTAwMDAwMDAwMCBjYWxscyB0byBtZW1jcHkgb2YgMzIgYnl0ZXMuICB+MjE3 NC43MDcgTUIvcyBjb3JyZWN0ZWQuCmFmdGVyOjMyOjEwMDAwMDAwMDA6NDox Mi43MTI0Njg6IHRvb2sgMTIuNzEyNDY4IHMgZm9yIDEwMDAwMDAwMDAgY2Fs bHMgdG8gbWVtY3B5IG9mIDMyIGJ5dGVzLiAgfjgxNjQuNDM0IE1CL3MgY29y cmVjdGVkLgpiZWZvcmU6MzI6MTAwMDAwMDAwMDo4OjIzLjAwNzY2NzogdG9v ayAyMy4wMDc2NjcgcyBmb3IgMTAwMDAwMDAwMCBjYWxscyB0byBtZW1jcHkg b2YgMzIgYnl0ZXMuICB+MjE3NC42OTEgTUIvcyBjb3JyZWN0ZWQuCmFmdGVy OjMyOjEwMDAwMDAwMDA6ODoxNi41MTcxNTI6IHRvb2sgMTYuNTE3MTUyIHMg Zm9yIDEwMDAwMDAwMDAgY2FsbHMgdG8gbWVtY3B5IG9mIDMyIGJ5dGVzLiAg fjQwNDYuMDU1IE1CL3MgY29ycmVjdGVkLgpiZWZvcmU6NjM6MTAwMDAwMDAw MDoxOjMxLjI2Njk0NDogdG9vayAzMS4yNjY5NDQgcyBmb3IgMTAwMDAwMDAw MCBjYWxscyB0byBtZW1jcHkgb2YgNjMgYnl0ZXMuICB+MjY5NS4xNjIgTUIv cyBjb3JyZWN0ZWQuCmFmdGVyOjYzOjEwMDAwMDAwMDA6MToxNS4wNDUzODM6 IHRvb2sgMTUuMDQ1MzgzIHMgZm9yIDEwMDAwMDAwMDAgY2FsbHMgdG8gbWVt Y3B5IG9mIDYzIGJ5dGVzLiAgfjk4OTYuODI2IE1CL3MgY29ycmVjdGVkLgpi ZWZvcmU6NjM6MTAwMDAwMDAwMDoyOjMxLjI2Njk0NDogdG9vayAzMS4yNjY5 NDQgcyBmb3IgMTAwMDAwMDAwMCBjYWxscyB0byBtZW1jcHkgb2YgNjMgYnl0 ZXMuICB+MjY5NS4xNjIgTUIvcyBjb3JyZWN0ZWQuCmFmdGVyOjYzOjEwMDAw MDAwMDA6MjoxNS4wNDM5MTU6IHRvb2sgMTUuMDQzOTE1IHMgZm9yIDEwMDAw MDAwMDAgY2FsbHMgdG8gbWVtY3B5IG9mIDYzIGJ5dGVzLiAgfjk4OTkuMjE5 IE1CL3MgY29ycmVjdGVkLgpiZWZvcmU6NjM6MTAwMDAwMDAwMDo0OjI4LjMx NjQ4MjogdG9vayAyOC4zMTY0ODIgcyBmb3IgMTAwMDAwMDAwMCBjYWxscyB0 byBtZW1jcHkgb2YgNjMgYnl0ZXMuICB+MzEwNi4yODkgTUIvcyBjb3JyZWN0 ZWQuCmFmdGVyOjYzOjEwMDAwMDAwMDA6NDoxNy4xMDQxMjM6IHRvb2sgMTcu MTA0MTIzIHMgZm9yIDEwMDAwMDAwMDAgY2FsbHMgdG8gbWVtY3B5IG9mIDYz IGJ5dGVzLiAgfjczOTAuNTMwIE1CL3MgY29ycmVjdGVkLgpiZWZvcmU6NjM6 MTAwMDAwMDAwMDo4OjI4LjMxNjY0NDogdG9vayAyOC4zMTY2NDQgcyBmb3Ig MTAwMDAwMDAwMCBjYWxscyB0byBtZW1jcHkgb2YgNjMgYnl0ZXMuICB+MzEw Ni4yNjMgTUIvcyBjb3JyZWN0ZWQuCmFmdGVyOjYzOjEwMDAwMDAwMDA6ODox NS4wNDM0OTU6IHRvb2sgMTUuMDQzNDk1IHMgZm9yIDEwMDAwMDAwMDAgY2Fs bHMgdG8gbWVtY3B5IG9mIDYzIGJ5dGVzLiAgfjk4OTkuOTA0IE1CL3MgY29y cmVjdGVkLgpiZWZvcmU6NjQ6MTAwMDAwMDAwMDoxOjMxLjI2NjE3NjogdG9v ayAzMS4yNjYxNzYgcyBmb3IgMTAwMDAwMDAwMCBjYWxscyB0byBtZW1jcHkg b2YgNjQgYnl0ZXMuICB+MjczOC4wMzcgTUIvcyBjb3JyZWN0ZWQuCmFmdGVy OjY0OjEwMDAwMDAwMDA6MToyMS4yMzgxOTA6IHRvb2sgMjEuMjM4MTkwIHMg Zm9yIDEwMDAwMDAwMDAgY2FsbHMgdG8gbWVtY3B5IG9mIDY0IGJ5dGVzLiAg fjQ5NzYuOTQwIE1CL3MgY29ycmVjdGVkLgpiZWZvcmU6NjQ6MTAwMDAwMDAw MDoyOjMxLjI2NjM5NDogdG9vayAzMS4yNjYzOTQgcyBmb3IgMTAwMDAwMDAw MCBjYWxscyB0byBtZW1jcHkgb2YgNjQgYnl0ZXMuICB+MjczOC4wMTAgTUIv cyBjb3JyZWN0ZWQuCmFmdGVyOjY0OjEwMDAwMDAwMDA6MjoyMS4yMzc3NjA6 IHRvb2sgMjEuMjM3NzYwIHMgZm9yIDEwMDAwMDAwMDAgY2FsbHMgdG8gbWVt Y3B5IG9mIDY0IGJ5dGVzLiAgfjQ5NzcuMTE1IE1CL3MgY29ycmVjdGVkLgpi ZWZvcmU6NjQ6MTAwMDAwMDAwMDo0OjI4LjkwNjcxNzogdG9vayAyOC45MDY3 MTcgcyBmb3IgMTAwMDAwMDAwMCBjYWxscyB0byBtZW1jcHkgb2YgNjQgYnl0 ZXMuICB+MzA2Mi4xNTEgTUIvcyBjb3JyZWN0ZWQuCmFmdGVyOjY0OjEwMDAw MDAwMDA6NDoyMS4yMzg1ODk6IHRvb2sgMjEuMjM4NTg5IHMgZm9yIDEwMDAw MDAwMDAgY2FsbHMgdG8gbWVtY3B5IG9mIDY0IGJ5dGVzLiAgfjQ5NzYuNzc4 IE1CL3MgY29ycmVjdGVkLgpiZWZvcmU6NjQ6MTAwMDAwMDAwMDo4OjI4Ljkw NjU1MjogdG9vayAyOC45MDY1NTIgcyBmb3IgMTAwMDAwMDAwMCBjYWxscyB0 byBtZW1jcHkgb2YgNjQgYnl0ZXMuICB+MzA2Mi4xNzcgTUIvcyBjb3JyZWN0 ZWQuCmFmdGVyOjY0OjEwMDAwMDAwMDA6ODoxNS45MzA2MTk6IHRvb2sgMTUu OTMwNjE5IHMgZm9yIDEwMDAwMDAwMDAgY2FsbHMgdG8gbWVtY3B5IG9mIDY0 IGJ5dGVzLiAgfjg3NzQuNDM3IE1CL3MgY29ycmVjdGVkLgpiZWZvcmU6MTAw OjEwMDAwMDAwMDA6MTozNy43NTU1MDU6IHRvb2sgMzcuNzU1NTA1IHMgZm9y IDEwMDAwMDAwMDAgY2FsbHMgdG8gbWVtY3B5IG9mIDEwMCBieXRlcy4gIH4z MzEzLjU2NiBNQi9zIGNvcnJlY3RlZC4KYWZ0ZXI6MTAwOjEwMDAwMDAwMDA6 MToyNS42NjI0OTU6IHRvb2sgMjUuNjYyNDk1IHMgZm9yIDEwMDAwMDAwMDAg Y2FsbHMgdG8gbWVtY3B5IG9mIDEwMCBieXRlcy4gIH41NzE0Ljc2NyBNQi9z IGNvcnJlY3RlZC4KYmVmb3JlOjEwMDoxMDAwMDAwMDAwOjI6MzcuNzU1NzIx OiB0b29rIDM3Ljc1NTcyMSBzIGZvciAxMDAwMDAwMDAwIGNhbGxzIHRvIG1l bWNweSBvZiAxMDAgYnl0ZXMuICB+MzMxMy41NDEgTUIvcyBjb3JyZWN0ZWQu CmFmdGVyOjEwMDoxMDAwMDAwMDAwOjI6MjUuNjYyNzIxOiB0b29rIDI1LjY2 MjcyMSBzIGZvciAxMDAwMDAwMDAwIGNhbGxzIHRvIG1lbWNweSBvZiAxMDAg Ynl0ZXMuICB+NTcxNC42OTAgTUIvcyBjb3JyZWN0ZWQuCmJlZm9yZToxMDA6 MTAwMDAwMDAwMDo0OjM1LjM5NjIyMzogdG9vayAzNS4zOTYyMjMgcyBmb3Ig MTAwMDAwMDAwMCBjYWxscyB0byBtZW1jcHkgb2YgMTAwIGJ5dGVzLiAgfjM2 MDkuNDQ2IE1CL3MgY29ycmVjdGVkLgphZnRlcjoxMDA6MTAwMDAwMDAwMDo0 OjI2LjU0NzQzODogdG9vayAyNi41NDc0MzggcyBmb3IgMTAwMDAwMDAwMCBj YWxscyB0byBtZW1jcHkgb2YgMTAwIGJ5dGVzLiAgfjU0MjYuOTgwIE1CL3Mg Y29ycmVjdGVkLgpiZWZvcmU6MTAwOjEwMDAwMDAwMDA6ODozNS4zOTYwMTQ6 IHRvb2sgMzUuMzk2MDE0IHMgZm9yIDEwMDAwMDAwMDAgY2FsbHMgdG8gbWVt Y3B5IG9mIDEwMCBieXRlcy4gIH4zNjA5LjQ3NSBNQi9zIGNvcnJlY3RlZC4K YWZ0ZXI6MTAwOjEwMDAwMDAwMDA6ODoyMS4yMzc3NzI6IHRvb2sgMjEuMjM3 NzcyIHMgZm9yIDEwMDAwMDAwMDAgY2FsbHMgdG8gbWVtY3B5IG9mIDEwMCBi eXRlcy4gIH43Nzc2LjczNCBNQi9zIGNvcnJlY3RlZC4KYmVmb3JlOjIwMDox MDAwMDAwMDAwOjE6NTYuNjMzMzk4OiB0b29rIDU2LjYzMzM5OCBzIGZvciAx MDAwMDAwMDAwIGNhbGxzIHRvIG1lbWNweSBvZiAyMDAgYnl0ZXMuICB+NDAw Mi4wOTEgTUIvcyBjb3JyZWN0ZWQuCmFmdGVyOjIwMDoxMDAwMDAwMDAwOjE6 MzMuNjI2NzIyOiB0b29rIDMzLjYyNjcyMiBzIGZvciAxMDAwMDAwMDAwIGNh bGxzIHRvIG1lbWNweSBvZiAyMDAgYnl0ZXMuICB+NzczNy4wNTcgTUIvcyBj b3JyZWN0ZWQuCmJlZm9yZToyMDA6MTAwMDAwMDAwMDoyOjU2LjYzMzY4NDog dG9vayA1Ni42MzM2ODQgcyBmb3IgMTAwMDAwMDAwMCBjYWxscyB0byBtZW1j cHkgb2YgMjAwIGJ5dGVzLiAgfjQwMDIuMDY3IE1CL3MgY29ycmVjdGVkLgph ZnRlcjoyMDA6MTAwMDAwMDAwMDoyOjMzLjYyNjY5MjogdG9vayAzMy42MjY2 OTIgcyBmb3IgMTAwMDAwMDAwMCBjYWxscyB0byBtZW1jcHkgb2YgMjAwIGJ5 dGVzLiAgfjc3MzcuMDY2IE1CL3MgY29ycmVjdGVkLgpiZWZvcmU6MjAwOjEw MDAwMDAwMDA6NDo1NC4yNzMxMDg6IHRvb2sgNTQuMjczMTA4IHMgZm9yIDEw MDAwMDAwMDAgY2FsbHMgdG8gbWVtY3B5IG9mIDIwMCBieXRlcy4gIH40MjEw LjYyMSBNQi9zIGNvcnJlY3RlZC4KYWZ0ZXI6MjAwOjEwMDAwMDAwMDA6NDoz My42MjYzOTE6IHRvb2sgMzMuNjI2MzkxIHMgZm9yIDEwMDAwMDAwMDAgY2Fs bHMgdG8gbWVtY3B5IG9mIDIwMCBieXRlcy4gIH43NzM3LjE2MCBNQi9zIGNv cnJlY3RlZC4KYmVmb3JlOjIwMDoxMDAwMDAwMDAwOjg6NTQuMjczOTY5OiB0 b29rIDU0LjI3Mzk2OSBzIGZvciAxMDAwMDAwMDAwIGNhbGxzIHRvIG1lbWNw eSBvZiAyMDAgYnl0ZXMuICB+NDIxMC41NDEgTUIvcyBjb3JyZWN0ZWQuCmFm dGVyOjIwMDoxMDAwMDAwMDAwOjg6MjkuNDk3NDA4OiB0b29rIDI5LjQ5NzQw OCBzIGZvciAxMDAwMDAwMDAwIGNhbGxzIHRvIG1lbWNweSBvZiAyMDAgYnl0 ZXMuICB+OTI5My43OTkgTUIvcyBjb3JyZWN0ZWQuCg== --047d7bdcaaa4b165b604da6356c9--