[Bug sanitizer/64435] New: [5.0.0 Regression] Bootstrap failure in libsanitizer on AArch64 with Linux kernel <= 3.15

[Bug sanitizer/64435] New: [5.0.0 Regression] Bootstrap failure in libsanitizer on AArch64 with Linux kernel <= 3.15

[david.abdurachmanov at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64435

            Bug ID: 64435
           Summary: [5.0.0 Regression] Bootstrap failure in libsanitizer
                    on AArch64 with Linux kernel <= 3.15
           Product: gcc
           Version: 5.0
            Status: UNCONFIRMED
          Severity: major
          Priority: P3
         Component: sanitizer
          Assignee: unassigned at gcc dot gnu.org
          Reporter: david.abdurachmanov at gmail dot com
                CC: dodji at gcc dot gnu.org, dvyukov at gcc dot gnu.org,
                    jakub at gcc dot gnu.org, kcc at gcc dot gnu.org

I finally started testing GCC trunk (5.0.0) on AArch64 before release is made.

I quickly find out that I couldn't bootstrap GCC on APM Mustang board with APM
Linux images (F19 + 3.12 kernel), but it worked fine with QEMU + F21 + 3.17
kernel). Failure details are below.

The culprit is 34c65c43f1518bf85f93526ad373adc6a683b4c5 from Linux repository.
Commit details are provided below.

Simply put __sanitizer___kernel_{uid,gid}_t depends on kernel version. <=3.15
it's unsigned int and >=3.16 it's unsigned short.

##### GCC trunk failure #####

In file included from
../../../../libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.cc:176:0:
../../../../libsanitizer/sanitizer_common/sanitizer_internal_defs.h:272:72:
error: size of array 'assertion_failed__1006' is negative
     typedef char IMPL_PASTE(assertion_failed_##_, line)[2*(int)(pred)-1]
                                                                        ^
../../../../libsanitizer/sanitizer_common/sanitizer_internal_defs.h:266:30:
note: in expansion of macro 'IMPL_COMPILER_ASSERT'
 #define COMPILER_CHECK(pred) IMPL_COMPILER_ASSERT(pred, __LINE__)
                              ^
../../../../libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.h:1285:3:
note: in expansion of macro 'COMPILER_CHECK'
   COMPILER_CHECK(sizeof(__sanitizer_##TYPE) == sizeof(TYPE))
   ^
../../../../libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.cc:1006:1:
note: in expansion of macro 'CHECK_TYPE_SIZE'
 CHECK_TYPE_SIZE(__kernel_old_uid_t);
 ^
../../../../libsanitizer/sanitizer_common/sanitizer_internal_defs.h:272:72:
error: size of array 'assertion_failed__1007' is negative
     typedef char IMPL_PASTE(assertion_failed_##_, line)[2*(int)(pred)-1]
                                                                        ^
../../../../libsanitizer/sanitizer_common/sanitizer_internal_defs.h:266:30:
note: in expansion of macro 'IMPL_COMPILER_ASSERT'
 #define COMPILER_CHECK(pred) IMPL_COMPILER_ASSERT(pred, __LINE__)
                              ^
../../../../libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.h:1285:3:
note: in expansion of macro 'COMPILER_CHECK'
   COMPILER_CHECK(sizeof(__sanitizer_##TYPE) == sizeof(TYPE))
   ^
../../../../libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.cc:1007:1:
note: in expansion of macro 'CHECK_TYPE_SIZE'
 CHECK_TYPE_SIZE(__kernel_old_gid_t);
 ^
make[4]: *** [sanitizer_platform_limits_posix.lo] Error 1
make[4]: *** Waiting for unfinished jobs....

##### Linux Commit #####

commit 34c65c43f1518bf85f93526ad373adc6a683b4c5
Author: Will Deacon <will.deacon@arm.com>
Date:   Mon Jun 2 11:47:29 2014 +0100

    arm64: uid16: fix __kernel_old_{gid,uid}_t definitions

    Whilst native arm64 applications don't have the 16-bit UID/GID syscalls
    wired up, compat tasks can still access them. The 16-bit wrappers for
    these syscalls use __kernel_old_uid_t and __kernel_old_gid_t, which must
    be 16-bit data types to maintain compatibility with the 16-bit UIDs used
    by compat applications.

    This patch defines 16-bit __kernel_old_{gid,uid}_t types for arm64
    instead of using the 32-bit types provided by asm-generic.

    Signed-off-by: Will Deacon <will.deacon@arm.com>
    Acked-by: Arnd Bergmann <arnd@arndb.de>
    Cc: <stable@vger.kernel.org>
    Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>

diff --git a/arch/arm64/include/asm/Kbuild b/arch/arm64/include/asm/Kbuild
index 42c7eec..0b3fcf8 100644
--- a/arch/arm64/include/asm/Kbuild
+++ b/arch/arm64/include/asm/Kbuild
@@ -30,7 +30,6 @@ generic-y += msgbuf.h
 generic-y += mutex.h
 generic-y += pci.h
 generic-y += poll.h
-generic-y += posix_types.h
 generic-y += preempt.h
 generic-y += resource.h
 generic-y += rwsem.h
diff --git a/arch/arm64/include/uapi/asm/posix_types.h
b/arch/arm64/include/uapi/asm/posix_types.h
new file mode 100644
index 0000000..7985ff6
--- /dev/null
+++ b/arch/arm64/include/uapi/asm/posix_types.h
@@ -0,0 +1,10 @@
+#ifndef __ASM_POSIX_TYPES_H
+#define __ASM_POSIX_TYPES_H
+
+typedef unsigned short __kernel_old_uid_t;
+typedef unsigned short __kernel_old_gid_t;
+#define __kernel_old_uid_t __kernel_old_uid_t
+
+#include <asm-generic/posix_types.h>
+
+#endif /*  __ASM_POSIX_TYPES_H */

[Bug sanitizer/64435] [5.0.0 Regression] Bootstrap failure in libsanitizer on AArch64 with Linux kernel <= 3.15

[david.abdurachmanov at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64435

--- Comment #1 from David Abdurachmanov <david.abdurachmanov at gmail dot com> ---
Created attachment 34356
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=34356&action=edit
RFC patch (tested)

Bootstrapped on aarch64-linux-gnu machine with F19 + 3.12 and on QEMU with F21
+ 3.17.

[Bug sanitizer/64435] [5.0.0 Regression] Bootstrap failure in libsanitizer on AArch64 with Linux kernel <= 3.15

[david.abdurachmanov at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64435

--- Comment #2 from David Abdurachmanov <david.abdurachmanov at gmail dot com> ---
linux/version.h does not bring any additional kernel headers, its fully
standalone and seems fine to include.

There might be a problem is someone builds a distribution with GCC 5 and kernel
<=3.15 and then decides to update the kernel creating mismatch with
libsanitizer.

[Bug sanitizer/64435] [5.0.0 Regression] Bootstrap failure in libsanitizer on AArch64 with Linux kernel <= 3.15

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64435

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |5.0

[Bug sanitizer/64435] [5.0.0 Regression] Bootstrap failure in libsanitizer on AArch64 with Linux kernel <= 3.15

[david.abdurachmanov at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64435

--- Comment #3 from David Abdurachmanov <david.abdurachmanov at gmail dot com> ---
The patch has been submitted to LLVM/compiler-rt and approved. Not yet
committed, pending testing with Clang.

http://reviews.llvm.org/D6854

[Bug sanitizer/64435] [5 Regression] Bootstrap failure in libsanitizer on AArch64 with Linux kernel <= 3.15

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64435

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |build
             Target|                            |aarch64
           Priority|P3                          |P1
            Summary|[5.0.0 Regression]          |[5 Regression] Bootstrap
                   |Bootstrap failure in        |failure in libsanitizer on
                   |libsanitizer on AArch64     |AArch64 with Linux kernel
                   |with Linux kernel <= 3.15   |<= 3.15

[Bug sanitizer/64435] [5 Regression] Bootstrap failure in libsanitizer on AArch64 with Linux kernel <= 3.15

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64435

--- Comment #4 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
So, no progress upstream yet?

[Bug sanitizer/64435] [5 Regression] Bootstrap failure in libsanitizer on AArch64 with Linux kernel <= 3.15

[clyon at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64435

clyon at gcc dot gnu.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |clyon at gcc dot gnu.org

--- Comment #5 from clyon at gcc dot gnu.org ---
Not yet. My AArch64 board has just been brought back online, and I managed to
execute the asan tests in clang/llvm. I can see failure, I'll look at them in
more details.

[Bug sanitizer/64435] [5 Regression] Bootstrap failure in libsanitizer on AArch64 with Linux kernel <= 3.15

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64435

--- Comment #6 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Well, for the old uid/git we can temporarily also just cherry pick upstream
r223925.  Even with that patch I'm running into:
../../../../libsanitizer/sanitizer_common/sanitizer_internal_defs.h:272:72:
error: size of array 'assertion_failed__1062' is negative
     typedef char IMPL_PASTE(assertion_failed_##_, line)[2*(int)(pred)-1]
                                                                        ^
../../../../libsanitizer/sanitizer_common/sanitizer_internal_defs.h:266:30:
note: in expansion of macro 'IMPL_COMPILER_ASSERT'
 #define COMPILER_CHECK(pred) IMPL_COMPILER_ASSERT(pred, __LINE__)
                              ^
../../../../libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.h:1288:3:
note: in expansion of macro 'COMPILER_CHECK'
   COMPILER_CHECK(sizeof(((__sanitizer_##CLASS *) NULL)->MEMBER) == \
   ^
../../../../libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.cc:1062:1:
note: in expansion of macro 'CHECK_SIZE_AND_OFFSET'
 CHECK_SIZE_AND_OFFSET(ipc_perm, mode);

[Bug sanitizer/64435] [5 Regression] Bootstrap failure in libsanitizer on AArch64 with Linux kernel <= 3.15

[david.abdurachmanov at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64435

--- Comment #8 from David Abdurachmanov <david.abdurachmanov at gmail dot com> ---
I will finish testing my patch for upstream next week. I was busy with other
tasks.

AArch64 is young, this kind of things are bound to happen :/

[Bug sanitizer/64435] [5 Regression] Bootstrap failure in libsanitizer on AArch64 with Linux kernel <= 3.15

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64435

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
   Last reconfirmed|                            |2015-01-18
     Ever confirmed|0                           |1

--- Comment #9 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
I've used:
--- libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.h   
2014-11-14 00:10:33.735060963 +0100
+++ libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.h   
2015-01-18 14:43:05.812763769 +0100
@@ -167,7 +167,7 @@ namespace __sanitizer {
     unsigned __seq;
     u64 __unused1;
     u64 __unused2;
-#elif defined(__mips__)
+#elif defined(__mips__) || defined(__aarch64__)
     unsigned int mode;
     unsigned short __seq;
     unsigned short __pad1;
--- libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.cc   
2014-11-14 00:10:33.735060963 +0100
+++ libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.cc   
2015-01-18 14:47:18.901482608 +0100
@@ -1055,7 +1055,10 @@ CHECK_SIZE_AND_OFFSET(ipc_perm, uid);
 CHECK_SIZE_AND_OFFSET(ipc_perm, gid);
 CHECK_SIZE_AND_OFFSET(ipc_perm, cuid);
 CHECK_SIZE_AND_OFFSET(ipc_perm, cgid);
+#if !defined(__aarch64__) || !SANITIZER_LINUX || __GNUC_PREREQ (2, 21)
+/* On aarch64 glibc 2.20 and earlier provided incorrect mode field.  */
 CHECK_SIZE_AND_OFFSET(ipc_perm, mode);
+#endif

 CHECK_TYPE_SIZE(shmid_ds);
 CHECK_SIZE_AND_OFFSET(shmid_ds, shm_perm);

to fix the #c6 issue.  That said, seems asan is totally broken at least in
Fedora 22/aarch64, but at least it builds with this
and r223925 cherry-pick.
Pretty much every test dies with
AddressSanitizer CHECK failed:
../../../../libsanitizer/asan/asan_poisoning.cc:24 "((AddrIsInMem(addr))) !=
(0)

[Bug sanitizer/64435] [5 Regression] Bootstrap failure in libsanitizer on AArch64 with Linux kernel <= 3.15

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64435

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Depends on|                            |64131

--- Comment #10 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
(In reply to Jakub Jelinek from comment #7)
> And the reason for that is that aarch64 again changed ABI, what a stable
> port :(.
> 
> https://sourceware.org/git/?p=glibc.git;a=commit;
> h=5c40c3bab2fddaca8cfe12d75944d1fef8adf1a4

This was recorded as bug 64131.  I filed it when I updated my glibc sources and
found Asan was broken.

Note ILP32 is also broken with Asan.

[Bug sanitizer/64435] [5 Regression] Bootstrap failure in libsanitizer on AArch64 with Linux kernel <= 3.15

[clyon at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64435

--- Comment #11 from clyon at gcc dot gnu.org ---
(In reply to David Abdurachmanov from comment #8)
> I will finish testing my patch for upstream next week. I was busy with other
> tasks.
> 
How are you going to test it?

FYI, I am now able to run the LLVM/Asan tests on a AArch64 system, and I have
noticed numerous failures. I'm currently debugging the Asan instrumentation
code which is generating memory accesses to invalid memory areas.

[Bug sanitizer/64435] [5 Regression] Bootstrap failure in libsanitizer on AArch64 with Linux kernel <= 3.15

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64435

--- Comment #12 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Author: jakub
Date: Mon Jan 19 08:39:27 2015
New Revision: 219833

URL: https://gcc.gnu.org/viewcvs?rev=219833&root=gcc&view=rev
Log:
    PR sanitizer/64435
    * sanitizer_common/sanitizer_platform_limits_posix.cc: Cherry pick
    upstream r223925.

Modified:
    trunk/libsanitizer/ChangeLog
    trunk/libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.cc

[Bug sanitizer/64435] [5 Regression] Bootstrap failure in libsanitizer on AArch64 with Linux kernel <= 3.15

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64435

--- Comment #13 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
In the #c9 patch obviously I meant to write __GLIBC_PREREQ (2, 21) instead of
__GNUC_PREREQ (2, 21), finger memory apparently doesn't work always well.
And there probably needs to be
#ifndef __GLIBC_PREREQ
#define __GLIBC_PREREQ(major, minor) 0
#endif
somewhere before that, as otherwise it will fail on non-Linux.

[Bug sanitizer/64435] [5 Regression] Bootstrap failure in libsanitizer on AArch64 with Linux kernel <= 3.15

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64435

--- Comment #14 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
And the reason why at least in RHEL and Fedora asan doesn't work at all is:
cat /proc/self/maps; ASAN_OPTIONS=verbosity=1 ./null-deref-1.exe
00400000-00410000 r-xp 00000000 fd:00 134631897                         
/usr/bin/cat
00410000-00420000 r--p 00000000 fd:00 134631897                         
/usr/bin/cat
00420000-00430000 rw-p 00010000 fd:00 134631897                         
/usr/bin/cat
142c0000-142f0000 rw-p 00000000 00:00 0                                  [heap]
3ff929d0000-3ff99270000 r--p 00000000 fd:00 4086                        
/usr/lib/locale/locale-archive
3ff99270000-3ff993d0000 r-xp 00000000 fd:00 203072141                   
/usr/lib64/libc-2.17.so
3ff993d0000-3ff993e0000 r--p 00150000 fd:00 203072141                   
/usr/lib64/libc-2.17.so
3ff993e0000-3ff993f0000 rw-p 00160000 fd:00 203072141                   
/usr/lib64/libc-2.17.so
3ff99410000-3ff99420000 r--p 00000000 00:00 0                            [vvar]
3ff99420000-3ff99430000 r-xp 00000000 00:00 0                            [vdso]
3ff99430000-3ff99450000 r-xp 00000000 fd:00 201341988                   
/usr/lib64/ld-2.17.so
3ff99450000-3ff99460000 r--p 00010000 fd:00 201341988                   
/usr/lib64/ld-2.17.so
3ff99460000-3ff99470000 rw-p 00020000 fd:00 201341988                   
/usr/lib64/ld-2.17.so
3fff0680000-3fff06b0000 rw-p 00000000 00:00 0                           
[stack]
==2862==Parsed ASAN_OPTIONS: verbosity=1
==2862==AddressSanitizer: libc interceptors initialized
|| `[0x002000000000, 0x007fffffffff]` || HighMem    ||
|| `[0x001400000000, 0x001fffffffff]` || HighShadow ||
|| `[0x001200000000, 0x0013ffffffff]` || ShadowGap  ||
|| `[0x001000000000, 0x0011ffffffff]` || LowShadow  ||
|| `[0x000000000000, 0x000fffffffff]` || LowMem     ||
MemToShadow(shadow): 0x001200000000 0x00123fffffff 0x001280000000
0x0013ffffffff
redzone=16
max_redzone=2048
quarantine_size=256M
malloc_context_size=30
SHADOW_SCALE: 3
SHADOW_GRANULARITY: 8
SHADOW_OFFSET: 1000000000
==2862==Installed the sigaction for signal 11
==2862==AddressSanitizer CHECK failed:
../../../../libsanitizer/asan/asan_poisoning.cc:24 "((AddrIsInMem(addr))) !=
(0)" (0x0, 0x0)

https://github.com/torvalds/linux/blob/master/arch/arm64/include/asm/memory.h
https://github.com/torvalds/linux/blob/master/arch/arm64/Kconfig
reveals that aarch64 can be on Linux configured to support 39, 42 or 48 bits
virtual address space.  The current libsanitizer/asan/ and
gcc/config/aarch64/aarch64.c (aarch64_asan_shadow_offset) seems to be okay only
for the 39 bits virtual address space, while Fedora/RHEL apparently use 42 bits
VA.  Wonder if aarch64 couldn't use a layout closer to what x86_64 uses for
asan, with shadow offset low 0x7fff8000, which is flexible to different sizes
of the virtual address space.

[Bug sanitizer/64435] [5 Regression] Bootstrap failure in libsanitizer on AArch64 with Linux kernel <= 3.15

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64435

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |pinskia at gcc dot gnu.org

--- Comment #15 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
(In reply to Jakub Jelinek from comment #14)

> https://github.com/torvalds/linux/blob/master/arch/arm64/include/asm/memory.h
> https://github.com/torvalds/linux/blob/master/arch/arm64/Kconfig
> reveals that aarch64 can be on Linux configured to support 39, 42 or 48 bits
> virtual address space.  The current libsanitizer/asan/ and
> gcc/config/aarch64/aarch64.c (aarch64_asan_shadow_offset) seems to be okay
> only for the 39 bits virtual address space, while Fedora/RHEL apparently use
> 42 bits VA.  Wonder if aarch64 couldn't use a layout closer to what x86_64
> uses for asan, with shadow offset low 0x7fff8000, which is flexible to
> different sizes of the virtual address space.

Right selecting 64k pages causes this.  Asan is known to be broken with the
different virtual address space due page size differences.  PowerPC has a
similar issue IIRC.  x86_64 does not have this issue as there is only one page
size and not different virtual address spaces there.

I think ASAN for AARCH64 needs testing on multiple virtual address sizes before
it can be turned on by default.  Note most distros uses 64k pages (Fedora for
an example) but some don't MontaVista CG7 for X-gene does not due to needing to
allow ARMv7 programs to run.

[Bug sanitizer/64435] [5 Regression] Bootstrap failure in libsanitizer on AArch64 with Linux kernel <= 3.15

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64435

--- Comment #16 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Trying a PIE on aarch64 reveals:
2aaca300000-2aaca310000 r-xp 00000000 fd:00 17567415                     /tmp/m
2aaca310000-2aaca320000 rw-p 00000000 fd:00 17567415                     /tmp/m
mappings alongside of 3ffXXXXXXXX, thus we need low mem from 0 to 2GB or 4GB?,
then maybe some middle memory, and then high memory.

[Bug sanitizer/64435] [5 Regression] Bootstrap failure in libsanitizer on AArch64 with Linux kernel <= 3.15

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64435

--- Comment #17 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
PPC64 actually supports both 44 and 46 bit address space:

uptr GetMaxVirtualAddress() {
#if SANITIZER_WORDSIZE == 64
# if defined(__powerpc64__)
  // On PowerPC64 we have two different address space layouts: 44- and 46-bit.
  // We somehow need to figure out which one we are using now and choose
  // one of 0x00000fffffffffffUL and 0x00003fffffffffffUL.
  // Note that with 'ulimit -s unlimited' the stack is moved away from the top
  // of the address space, so simply checking the stack address is not enough.
  // This should (does) work for both PowerPC64 Endian modes.
  return (1ULL << (MostSignificantSetBitIndex(GET_CURRENT_FRAME()) + 1)) - 1;
# elif defined(__aarch64__)
  return (1ULL << 39) - 1;
# elif defined(__mips64)
  return (1ULL << 40) - 1;
# else
  return (1ULL << 47) - 1;  // 0x00007fffffffffffUL;
# endif
#else  // SANITIZER_WORDSIZE == 32
  uptr res = (1ULL << 32) - 1;  // 0xffffffff;
  if (!common_flags()->full_address_space)
    res -= GetKernelAreaSize();
  CHECK_LT(reinterpret_cast<uptr>(&res), res);
  return res;
#endif  // SANITIZER_WORDSIZE
}

it is just that aarch64 hardcodes this right now.  Suppose doing something like
ppc64 does could work.

[Bug sanitizer/64435] [5 Regression] Bootstrap failure in libsanitizer on AArch64 with Linux kernel <= 3.15

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64435

--- Comment #19 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
So, with:
--- libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.h.jj   
2014-11-14 00:10:33.000000000 +0100
+++ libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.h   
2015-01-19 19:22:43.777780408 +0100
@@ -167,7 +167,7 @@ namespace __sanitizer {
     unsigned __seq;
     u64 __unused1;
     u64 __unused2;
-#elif defined(__mips__)
+#elif defined(__mips__) || defined(__aarch64__)
     unsigned int mode;
     unsigned short __seq;
     unsigned short __pad1;
--- libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.cc.jj   
2015-01-19 09:39:09.000000000 +0100
+++ libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.cc   
2015-01-19 19:21:57.608564325 +0100
@@ -1059,7 +1059,13 @@ CHECK_SIZE_AND_OFFSET(ipc_perm, uid);
 CHECK_SIZE_AND_OFFSET(ipc_perm, gid);
 CHECK_SIZE_AND_OFFSET(ipc_perm, cuid);
 CHECK_SIZE_AND_OFFSET(ipc_perm, cgid);
+#ifndef __GLIBC_PREREQ
+#define __GLIBC_PREREQ(x, y) 0
+#endif
+#if !defined(__aarch64__) || !SANITIZER_LINUX || __GLIBC_PREREQ (2, 21)
+/* On aarch64 glibc 2.20 and earlier provided incorrect mode field.  */
 CHECK_SIZE_AND_OFFSET(ipc_perm, mode);
+#endif

 CHECK_TYPE_SIZE(shmid_ds);
 CHECK_SIZE_AND_OFFSET(shmid_ds, shm_perm);
--- libsanitizer/sanitizer_common/sanitizer_posix.cc.jj    2014-11-14
00:10:33.000000000 +0100
+++ libsanitizer/sanitizer_common/sanitizer_posix.cc    2015-01-19
19:24:14.636237703 +0100
@@ -76,16 +76,15 @@ static uptr GetKernelAreaSize() {

 uptr GetMaxVirtualAddress() {
 #if SANITIZER_WORDSIZE == 64
-# if defined(__powerpc64__)
+# if defined(__powerpc64__) || defined(__aarch64__)
   // On PowerPC64 we have two different address space layouts: 44- and 46-bit.
   // We somehow need to figure out which one we are using now and choose
   // one of 0x00000fffffffffffUL and 0x00003fffffffffffUL.
   // Note that with 'ulimit -s unlimited' the stack is moved away from the top
   // of the address space, so simply checking the stack address is not enough.
   // This should (does) work for both PowerPC64 Endian modes.
+  // Similarly, aarch64 has multiple address space layouts: 39, 42 and 48-bit.
   return (1ULL << (MostSignificantSetBitIndex(GET_CURRENT_FRAME()) + 1)) - 1;
-# elif defined(__aarch64__)
-  return (1ULL << 39) - 1;
 # elif defined(__mips64)
   return (1ULL << 40) - 1;
 # else

the layout looks reasonable:
|| `[0x009000000000, 0x03ffffffffff]` || HighMem    ||
|| `[0x002200000000, 0x008fffffffff]` || HighShadow ||
|| `[0x001200000000, 0x0021ffffffff]` || ShadowGap  ||
|| `[0x001000000000, 0x0011ffffffff]` || LowShadow  ||
|| `[0x000000000000, 0x000fffffffff]` || LowMem     ||
MemToShadow(shadow): 0x001200000000 0x00123fffffff 0x001440000000
0x0021ffffffff
and should accomodate the different addresses I saw for 42-bit address space so
far: <4GB, 0x10000000000 (place where libraries are mapped with ulimit -s
unlimited), 0x2aaXXXXXXXX (PIEs), 0x3ffXXXXXXXX (stack, normal mmap area for
limited ulimit -s).
But it still doesn't work:
==4746==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000028 (pc
0x0000004008f8 bp 0x03fff8078fc0 sp 0x03fff8078fc0 T0)
==4746==AddressSanitizer CHECK failed:
../../../../libsanitizer/sanitizer_common/sanitizer_allocator.h:835 "((res)) <
((kNumPossibleRegions))" (0x3ffa33, 0x80000)

I've tried to change
#if SANITIZER_CAN_USE_ALLOCATOR64
# if defined(__powerpc64__)
const uptr kAllocatorSpace =  0xa0000000000ULL;
const uptr kAllocatorSize  =  0x20000000000ULL;  // 2T.
# else
const uptr kAllocatorSpace = 0x600000000000ULL;
const uptr kAllocatorSize  =  0x40000000000ULL;  // 4T.
# endif
from the non-ppc64 values to the ppc64 values (i.e. added " ||
defined(__aarch64__)"), but strangely that didn't change anything.  So I'm out
of ideas on what else needs to be tweaked.  Kostya?

[Bug sanitizer/64435] [5 Regression] Bootstrap failure in libsanitizer on AArch64 with Linux kernel <= 3.15

[clyon at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64435

--- Comment #20 from clyon at gcc dot gnu.org ---
(In reply to Jakub Jelinek from comment #14)
> 
> https://github.com/torvalds/linux/blob/master/arch/arm64/include/asm/memory.h
> https://github.com/torvalds/linux/blob/master/arch/arm64/Kconfig
> reveals that aarch64 can be on Linux configured to support 39, 42 or 48 bits
> virtual address space.  The current libsanitizer/asan/ and
> gcc/config/aarch64/aarch64.c (aarch64_asan_shadow_offset) seems to be okay
> only for the 39 bits virtual address space, while Fedora/RHEL apparently use
> 42 bits VA.  Wonder if aarch64 couldn't use a layout closer to what x86_64
> uses for asan, with shadow offset low 0x7fff8000, which is flexible to
> different sizes of the virtual address space.

Thanks for pointing that, I wasn't aware of it when I worked on the initial
port. My platform had 39 bits virtual address space, and I didn't notice there
were other possibilities.

[Bug sanitizer/64435] [5 Regression] Bootstrap failure in libsanitizer on AArch64 with Linux kernel <= 3.15

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64435

--- Comment #21 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Ugh, there is also this junk:
// By default we allow to use SizeClassAllocator64 on 64-bit platform.
// But in some cases (e.g. AArch64's 39-bit address space) SizeClassAllocator64
// does not work well and we need to fallback to SizeClassAllocator32.
// For such platforms build this code with -DSANITIZER_CAN_USE_ALLOCATOR64=0 or
// change the definition of SANITIZER_CAN_USE_ALLOCATOR64 here.
#ifndef SANITIZER_CAN_USE_ALLOCATOR64
# if defined(__aarch64__) || defined(__mips64)
#  define SANITIZER_CAN_USE_ALLOCATOR64 0
# else
#  define SANITIZER_CAN_USE_ALLOCATOR64 (SANITIZER_WORDSIZE == 64)
# endif
#endif

It would be nice to know why that has been added.
Was that just because
#if SANITIZER_CAN_USE_ALLOCATOR64
# if defined(__powerpc64__)
const uptr kAllocatorSpace =  0xa0000000000ULL;
const uptr kAllocatorSize  =  0x20000000000ULL;  // 2T.
# else
const uptr kAllocatorSpace = 0x600000000000ULL;
const uptr kAllocatorSize  =  0x40000000000ULL;  // 4T.
# endif
has not been adjusted for the port?
What is the minimum kAllocatorSize that would work?  Given the very small
39-bit VA option on aarch64, I think if the allocator64 memory has to be normal
memory with shadow, we could only use say something like
kAllocatorSpace 0x800000000 and kAllocatorSize the same value, i.e. 32GB. 
Would that be enough?

There is also the:
// The range of addresses which can be returned my mmap.
// FIXME: this value should be different on different platforms,
// e.g. on AArch64 it is most likely (1ULL << 39). Larger values will still
work
// but will consume more memory for TwoLevelByteMap.
#if defined(__aarch64__)
# define SANITIZER_MMAP_RANGE_SIZE FIRST_32_SECOND_64(1ULL << 32, 1ULL << 39)
#else
# define SANITIZER_MMAP_RANGE_SIZE FIRST_32_SECOND_64(1ULL << 32, 1ULL << 47)
#endif
which is again wrong for aarch64.  Does it really has to be compile time
constant?  The 1ULL << 47 is also wrong ppc64, mips64, isn't it?

[Bug sanitizer/64435] [5 Regression] Bootstrap failure in libsanitizer on AArch64 with Linux kernel <= 3.15

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64435

--- Comment #22 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Or perhaps we could change SizeClassAllocator64 so that on some architectures
it could use variable bounds.  The fact that they are template parameters makes
this harder, but perhaps we could say that if some template
parameters are zero then some variable is used instead.
Use kSpaceBeg ? kSpaceBeg : kSpaceBegVar instead of kSpaceBeg and
kSpaceSize ? kSpaceSize : kSpaceSizeVar instead of kSpace (perhaps put into
some
method).  For architectures where there is no such big variability of address
space sizes it could stay constant, while for aarch64 we could decide during
asan initialization, after finding out how large address space we have.
So for say the 39-bit VA we could use 0x2000000000 to 0x3fffffffff (i.e. 128GB
for the allocator), which is not possible for 42-bit VA, because that is shadow
gap and high shadow.  And for 42-bit VA we could use say 0x10000000000 to
0x2ffffffffff (i.e. 2TB).

[Bug sanitizer/64435] [5 Regression] Bootstrap failure in libsanitizer on AArch64 with Linux kernel <= 3.15

[kcc at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64435

--- Comment #23 from Kostya Serebryany <kcc at gcc dot gnu.org> ---
Can we move the discussion to the asan issue tracker? 
https://code.google.com/p/address-sanitizer/issues/list

There is already quite a bit of aarch64 discussion at
https://code.google.com/p/address-sanitizer/issues/detail?id=246

Not all of the asan developers are looking at the gcc buganizer and the fix
will need to go to the upstream anyway.

[Bug sanitizer/64435] [5 Regression] Bootstrap failure in libsanitizer on AArch64 with Linux kernel <= 3.15

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64435

--- Comment #24 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Author: jakub
Date: Wed Jan 21 21:21:27 2015
New Revision: 219968

URL: https://gcc.gnu.org/viewcvs?rev=219968&root=gcc&view=rev
Log:
    PR sanitizer/64435
    * sanitizer_common/sanitizer_platform_limits_posix.h: Cherry pick
    upstream r226637.
    * sanitizer_common/sanitizer_platform_limits_posix.cc: Likewise.
    * sanitizer_common/sanitizer_posix.cc: Cherry pick upstream r226639.

Modified:
    trunk/libsanitizer/ChangeLog
    trunk/libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.cc
    trunk/libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.h
    trunk/libsanitizer/sanitizer_common/sanitizer_posix.cc

[Bug sanitizer/64435] [5 Regression] Bootstrap failure in libsanitizer on AArch64 with Linux kernel <= 3.15

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64435
Bug 64435 depends on bug 64131, which changed state.

Bug 64131 Summary: [5 Regression] libsanitizer fails to build for AARCH64 with the glibc from the trunk
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64131

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|---                         |FIXED

[Bug sanitizer/64435] [5 Regression] Bootstrap failure in libsanitizer on AArch64 with Linux kernel <= 3.15

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64435

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED

--- Comment #25 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
The bootstrap failure should be fixed now.  ASAN doesn't work at all with != 39
bits of virtual address space though.