libsanitizer merge from upstream r208536

libsanitizer merge from upstream r208536

[Konstantin Serebryany]

[-- Attachment #1: Type: text/plain, Size: 1430 bytes --]

This is the first libsanitizer merge in 4.10 (the last merge was in
December 2013).

Tested on Ubuntu 12.04 like this:
rm -rf */{*/,}libsanitizer && make -j 50
make -j 40 -C gcc check-g{cc,++}
RUNTESTFLAGS='--target_board=unix\{-m32,-m64\} asan.exp' && \
make -j 40 -C gcc check-g{cc,++}
RUNTESTFLAGS='--target_board=unix\{-m32,-m64\} tsan.exp' && \
make -j 40 -C gcc check
RUNTESTFLAGS='--target_board=unix\{-m32,-m64\} ubsan.exp' && \
echo PASS

5 months' worth of changes may break any platform we are not testing ourselves
(that includes Ubuntu 12.04, 13.10, 14.04, Mac 10.9, Windows 7, Android ARM),
please help us test this patch on your favorite platform.

Expected ChangeLog entries:
=============== gcc/testsuite/ChangeLog
2014-05-XX  Kostya Serebryany  <kcc@google.com>

        * c-c++-common/tsan/mutexset1.c: Update the test to match
        upstream r208536.
        * g++.dg/asan/symbolize-callback-1.C: Delete the deprecated test.

=============== libsanitizer/ChangeLog
2014-05-XX  Kostya Serebryany  <kcc@google.com>

        * All source files: Merge from upstream r208536.
        * asan/Makefile.am (asan_files): Added new files.
        * asan/Makefile.in: Regenerate.
        * tsan/Makefile.am (tsan_files): Added new files.
        * tsan/Makefile.in: Regenerate.
        * sanitizer_common/Makefile.am (sanitizer_common_files): Added
new files.
        * sanitizer_common/Makefile.in: Regenerate.

--kcc

[-- Attachment #2: libsanitizer-208536.patch.gz --]
[-- Type: application/x-gzip, Size: 197801 bytes --]

Re: libsanitizer merge from upstream r208536

[Maxim Ostapenko]

[-- Attachment #1: Type: text/plain, Size: 1670 bytes --]

Hi,

I see a couple of errors when building for arm-linux-gnueabi (host is 
x86_64 Ubuntu 12.04 LTS, host compiler is gcc version 4.6.3 
(Ubuntu/Linaro 4.6.3-1ubuntu5)):

1)   In file included from 
/home/max/workspace/downloads/gcc/libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.cc:164:0:
/home/max/workspace/downloads/gcc/libsanitizer/sanitizer_common/sanitizer_internal_defs.h:253:72: 
error: size of array 'assertion_failed__1128' is negative
      typedef char IMPL_PASTE(assertion_failed_##_, line)[2*(int)(pred)-1]
^
/home/max/workspace/downloads/gcc/libsanitizer/sanitizer_common/sanitizer_internal_defs.h:247:30: 
note: in expansion of macro 'IMPL_COMPILER_ASSERT'
  #define COMPILER_CHECK(pred) IMPL_COMPILER_ASSERT(pred, __LINE__)
                               ^
/home/max/workspace/downloads/gcc/libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.h:1249:3: 
note: in expansion of macro 'COMPILER_CHECK'
    COMPILER_CHECK(sizeof(__sanitizer_##TYPE) == sizeof(TYPE))
    ^
/home/max/workspace/downloads/gcc/libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.cc:1128:1: 
note: in expansion of macro 'CHECK_TYPE_SIZE'
  CHECK_TYPE_SIZE(XDR::xdr_ops);
  ^
make[4]: *** [sanitizer_platform_limits_posix.lo] Error 1


2) 
/home/max/workspace/downloads/gcc/libsanitizer/sanitizer_common/sanitizer_atomic_clang_other.h:54:77: 
error: cast from type 'const volatile Type* {aka const volatile unsigned 
char*}' to type 'volatile Type* {aka volatile unsigned char*}' casts 
away qualifiers [-Werror=cast-qual]
      v = __sync_fetch_and_add((typename T::Type 
volatile*)&a->val_dont_use, 0);

Attached patch seems to help.

-Maxim

[-- Attachment #2: libsanitizer-merge.diff --]
[-- Type: text/x-patch, Size: 1416 bytes --]

diff --git a/libsanitizer/sanitizer_common/sanitizer_atomic_clang_other.h b/libsanitizer/sanitizer_common/sanitizer_atomic_clang_other.h
index 75aa2c8..f2f05a8 100644
--- a/libsanitizer/sanitizer_common/sanitizer_atomic_clang_other.h
+++ b/libsanitizer/sanitizer_common/sanitizer_atomic_clang_other.h
@@ -51,7 +51,7 @@ INLINE typename T::Type atomic_load(
     // 64-bit load on 32-bit platform.
     // Gross, but simple and reliable.
     // Assume that it is not in read-only memory.
-    v = __sync_fetch_and_add((typename T::Type volatile*)&a->val_dont_use, 0);  
+    v = __sync_fetch_and_add(const_cast<typename T::Type volatile*>(&a->val_dont_use), 0);  
   }
   return v;
 }
diff --git a/libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.h b/libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.h
index 97fda51..3ee8e33 100644
--- a/libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.h
+++ b/libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.h
@@ -273,10 +273,17 @@ namespace __sanitizer {
 #endif
 
 #if SANITIZER_LINUX && !SANITIZER_ANDROID
+
+#if defined(__arm__)
+  const unsigned struct_xdr_ops_num_funs = 9;
+#else
+  const unsigned struct_xdr_ops_num_funs = 10;
+#endif
+
   struct __sanitizer_XDR {
     int x_op;
     struct xdr_ops {
-      uptr fns[10];
+      uptr fns[struct_xdr_ops_num_funs];
     } *x_ops;
     uptr x_public;
     uptr x_private;

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

Thanks! May I ask you to contribute the patch upstream?
https://code.google.com/p/address-sanitizer/wiki/HowToContribute

On Mon, May 12, 2014 at 7:19 PM, Maxim Ostapenko
<m.ostapenko@partner.samsung.com> wrote:
> Hi,
>
> I see a couple of errors when building for arm-linux-gnueabi (host is x86_64
> Ubuntu 12.04 LTS, host compiler is gcc version 4.6.3 (Ubuntu/Linaro
> 4.6.3-1ubuntu5)):
>
> 1)   In file included from
> /home/max/workspace/downloads/gcc/libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.cc:164:0:
> /home/max/workspace/downloads/gcc/libsanitizer/sanitizer_common/sanitizer_internal_defs.h:253:72:
> error: size of array 'assertion_failed__1128' is negative
>      typedef char IMPL_PASTE(assertion_failed_##_, line)[2*(int)(pred)-1]
> ^
> /home/max/workspace/downloads/gcc/libsanitizer/sanitizer_common/sanitizer_internal_defs.h:247:30:
> note: in expansion of macro 'IMPL_COMPILER_ASSERT'
>  #define COMPILER_CHECK(pred) IMPL_COMPILER_ASSERT(pred, __LINE__)
>                               ^
> /home/max/workspace/downloads/gcc/libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.h:1249:3:
> note: in expansion of macro 'COMPILER_CHECK'
>    COMPILER_CHECK(sizeof(__sanitizer_##TYPE) == sizeof(TYPE))
>    ^
> /home/max/workspace/downloads/gcc/libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.cc:1128:1:
> note: in expansion of macro 'CHECK_TYPE_SIZE'
>  CHECK_TYPE_SIZE(XDR::xdr_ops);
>  ^
> make[4]: *** [sanitizer_platform_limits_posix.lo] Error 1
>
>
> 2)
> /home/max/workspace/downloads/gcc/libsanitizer/sanitizer_common/sanitizer_atomic_clang_other.h:54:77:
> error: cast from type 'const volatile Type* {aka const volatile unsigned
> char*}' to type 'volatile Type* {aka volatile unsigned char*}' casts away
> qualifiers [-Werror=cast-qual]
>      v = __sync_fetch_and_add((typename T::Type volatile*)&a->val_dont_use,
> 0);
>
> Attached patch seems to help.
>
> -Maxim

Re: libsanitizer merge from upstream r208536

[Yury Gribov]

On 05/12/2014 08:16 PM, Konstantin Serebryany wrote:
> Thanks! May I ask you to contribute the patch upstream?
> https://code.google.com/p/address-sanitizer/wiki/HowToContribute

Note that we wouldn't be able to repro in upstream because
* ARM isn't supported
* LLVM won't build with gcc 4.6 anyway

-Y

Re: libsanitizer merge from upstream r208536

[Yury Gribov]

> * ARM isn't supported

I meant ARM-Linux.

-Y

Re: libsanitizer merge from upstream r208536

[Yury Gribov]

On 05/12/2014 03:20 PM, Konstantin Serebryany wrote:
> This is the first libsanitizer merge in 4.10

Thanks, Kostya.

I see that ASAN_DYNAMIC is not fully supported in libsanitizer 
Makefiles, I'll work on this once your patch goes in. I also have 
pending patch for asan-instrumentation-with-call-threshold parameter 
support in GCC.

-Y

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

On Mon, May 12, 2014 at 7:36 PM, Yury Gribov <y.gribov@samsung.com> wrote:
> On 05/12/2014 03:20 PM, Konstantin Serebryany wrote:
>>
>> This is the first libsanitizer merge in 4.10
>
>
> Thanks, Kostya.
>
> I see that ASAN_DYNAMIC is not fully supported in libsanitizer Makefiles,
Generally, I prefer to minimize the changes in the build files and put
as much logic into the source as possible.
Didn't  http://llvm.org/viewvc/llvm-project?view=revision&revision=208530 help?

> I'll work on this once your patch goes in.
Good!
> I also have pending patch for
> asan-instrumentation-with-call-threshold parameter support in GCC.
>
> -Y

Re: libsanitizer merge from upstream r208536

[Yury Gribov]

On 05/12/2014 08:18 PM, Konstantin Serebryany wrote:
> Didn'thttp://llvm.org/viewvc/llvm-project?view=revision&revision=208530  help?

Ah true, I missed the fact that you define PIC in libsanitizer/configure.

-Y

Re: libsanitizer merge from upstream r208536

[H.J. Lu]

On Mon, May 12, 2014 at 4:20 AM, Konstantin Serebryany
<konstantin.s.serebryany@gmail.com> wrote:
> This is the first libsanitizer merge in 4.10 (the last merge was in
> December 2013).
>
> Tested on Ubuntu 12.04 like this:
> rm -rf */{*/,}libsanitizer && make -j 50
> make -j 40 -C gcc check-g{cc,++}
> RUNTESTFLAGS='--target_board=unix\{-m32,-m64\} asan.exp' && \
> make -j 40 -C gcc check-g{cc,++}
> RUNTESTFLAGS='--target_board=unix\{-m32,-m64\} tsan.exp' && \
> make -j 40 -C gcc check
> RUNTESTFLAGS='--target_board=unix\{-m32,-m64\} ubsan.exp' && \
> echo PASS
>
> 5 months' worth of changes may break any platform we are not testing ourselves
> (that includes Ubuntu 12.04, 13.10, 14.04, Mac 10.9, Windows 7, Android ARM),
> please help us test this patch on your favorite platform.
>
> Expected ChangeLog entries:
> =============== gcc/testsuite/ChangeLog
> 2014-05-XX  Kostya Serebryany  <kcc@google.com>
>
>         * c-c++-common/tsan/mutexset1.c: Update the test to match
>         upstream r208536.
>         * g++.dg/asan/symbolize-callback-1.C: Delete the deprecated test.
>
> =============== libsanitizer/ChangeLog
> 2014-05-XX  Kostya Serebryany  <kcc@google.com>
>
>         * All source files: Merge from upstream r208536.
>         * asan/Makefile.am (asan_files): Added new files.
>         * asan/Makefile.in: Regenerate.
>         * tsan/Makefile.am (tsan_files): Added new files.
>         * tsan/Makefile.in: Regenerate.
>         * sanitizer_common/Makefile.am (sanitizer_common_files): Added
> new files.
>         * sanitizer_common/Makefile.in: Regenerate.
>
> --kcc

sanitizer_common/sanitizer_platform_limits_posix.h has

  struct __sanitizer_timeb {
    long time;
    unsigned short millitm;
    short timezone;
    short dstflag;
  };

On Linux, timeb is

struct timeb
  {
    time_t time; /* Seconds since epoch, as from `time'.  */
    unsigned short int millitm; /* Additional milliseconds.  */
    short int timezone; /* Minutes west of GMT.  */
    short int dstflag; /* Nonzero if Daylight Savings Time used.  */
  };

For x32, long is 32-bit and time_t is 64-bit.  We need something
like

 struct __sanitizer_timeb {
#if defined(__x86_64__) && !defined(_LP64)
    long long time;
#else
    long time;
#endif

similar to other places.


-- 
H.J.

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

I've committed this upstream and will include it into my next updated patch:

+#if defined(__x86_64__) && !defined(_LP64)
+  typedef long long __sanitizer_time_t;
+#else
+  typedef long __sanitizer_time_t;
+#endif
+
   struct __sanitizer_timeb {
-    long time;
+    __sanitizer_time_t time;
     unsigned short millitm;


If this is not enough, please contribute patches directly upstream --
I can not accept them from here.

Also, what's the story with x32 in LLVM?
Is there any chance you can set up a public build bot for x32 asan (upstream)?

On Mon, May 12, 2014 at 9:53 PM, H.J. Lu <hjl.tools@gmail.com> wrote:
> On Mon, May 12, 2014 at 4:20 AM, Konstantin Serebryany
> <konstantin.s.serebryany@gmail.com> wrote:
>> This is the first libsanitizer merge in 4.10 (the last merge was in
>> December 2013).
>>
>> Tested on Ubuntu 12.04 like this:
>> rm -rf */{*/,}libsanitizer && make -j 50
>> make -j 40 -C gcc check-g{cc,++}
>> RUNTESTFLAGS='--target_board=unix\{-m32,-m64\} asan.exp' && \
>> make -j 40 -C gcc check-g{cc,++}
>> RUNTESTFLAGS='--target_board=unix\{-m32,-m64\} tsan.exp' && \
>> make -j 40 -C gcc check
>> RUNTESTFLAGS='--target_board=unix\{-m32,-m64\} ubsan.exp' && \
>> echo PASS
>>
>> 5 months' worth of changes may break any platform we are not testing ourselves
>> (that includes Ubuntu 12.04, 13.10, 14.04, Mac 10.9, Windows 7, Android ARM),
>> please help us test this patch on your favorite platform.
>>
>> Expected ChangeLog entries:
>> =============== gcc/testsuite/ChangeLog
>> 2014-05-XX  Kostya Serebryany  <kcc@google.com>
>>
>>         * c-c++-common/tsan/mutexset1.c: Update the test to match
>>         upstream r208536.
>>         * g++.dg/asan/symbolize-callback-1.C: Delete the deprecated test.
>>
>> =============== libsanitizer/ChangeLog
>> 2014-05-XX  Kostya Serebryany  <kcc@google.com>
>>
>>         * All source files: Merge from upstream r208536.
>>         * asan/Makefile.am (asan_files): Added new files.
>>         * asan/Makefile.in: Regenerate.
>>         * tsan/Makefile.am (tsan_files): Added new files.
>>         * tsan/Makefile.in: Regenerate.
>>         * sanitizer_common/Makefile.am (sanitizer_common_files): Added
>> new files.
>>         * sanitizer_common/Makefile.in: Regenerate.
>>
>> --kcc
>
> sanitizer_common/sanitizer_platform_limits_posix.h has
>
>   struct __sanitizer_timeb {
>     long time;
>     unsigned short millitm;
>     short timezone;
>     short dstflag;
>   };
>
> On Linux, timeb is
>
> struct timeb
>   {
>     time_t time; /* Seconds since epoch, as from `time'.  */
>     unsigned short int millitm; /* Additional milliseconds.  */
>     short int timezone; /* Minutes west of GMT.  */
>     short int dstflag; /* Nonzero if Daylight Savings Time used.  */
>   };
>
> For x32, long is 32-bit and time_t is 64-bit.  We need something
> like
>
>  struct __sanitizer_timeb {
> #if defined(__x86_64__) && !defined(_LP64)
>     long long time;
> #else
>     long time;
> #endif
>
> similar to other places.
>
>
> --
> H.J.

Re: libsanitizer merge from upstream r208536

[pinskia]

> On May 13, 2014, at 12:59 AM, Konstantin Serebryany <konstantin.s.serebryany@gmail.com> wrote:
> 
> I've committed this upstream and will include it into my next updated patch:
> 
> +#if defined(__x86_64__) && !defined(_LP64)
> +  typedef long long __sanitizer_time_t;
> +#else
> +  typedef long __sanitizer_time_t;
> +#endif
> +
>   struct __sanitizer_timeb {
> -    long time;
> +    __sanitizer_time_t time;
>     unsigned short millitm;


This will also be needed for ilp32 aarch64 too. 
> 
> 
> If this is not enough, please contribute patches directly upstream --
> I can not accept them from here.


I am still against the way Libsanitizer is being maintained. I really think the maintainers of the source in gcc should submit the patches upstream and not burden the target maintainers with it.  Or maybe having libsantizer as another project separate from gcc and LLVM would be even better. 

Thanks,
Andrew


> 
> Also, what's the story with x32 in LLVM?
> Is there any chance you can set up a public build bot for x32 asan (upstream)?
> 
>> On Mon, May 12, 2014 at 9:53 PM, H.J. Lu <hjl.tools@gmail.com> wrote:
>> On Mon, May 12, 2014 at 4:20 AM, Konstantin Serebryany
>> <konstantin.s.serebryany@gmail.com> wrote:
>>> This is the first libsanitizer merge in 4.10 (the last merge was in
>>> December 2013).
>>> 
>>> Tested on Ubuntu 12.04 like this:
>>> rm -rf */{*/,}libsanitizer && make -j 50
>>> make -j 40 -C gcc check-g{cc,++}
>>> RUNTESTFLAGS='--target_board=unix\{-m32,-m64\} asan.exp' && \
>>> make -j 40 -C gcc check-g{cc,++}
>>> RUNTESTFLAGS='--target_board=unix\{-m32,-m64\} tsan.exp' && \
>>> make -j 40 -C gcc check
>>> RUNTESTFLAGS='--target_board=unix\{-m32,-m64\} ubsan.exp' && \
>>> echo PASS
>>> 
>>> 5 months' worth of changes may break any platform we are not testing ourselves
>>> (that includes Ubuntu 12.04, 13.10, 14.04, Mac 10.9, Windows 7, Android ARM),
>>> please help us test this patch on your favorite platform.
>>> 
>>> Expected ChangeLog entries:
>>> =============== gcc/testsuite/ChangeLog
>>> 2014-05-XX  Kostya Serebryany  <kcc@google.com>
>>> 
>>>        * c-c++-common/tsan/mutexset1.c: Update the test to match
>>>        upstream r208536.
>>>        * g++.dg/asan/symbolize-callback-1.C: Delete the deprecated test.
>>> 
>>> =============== libsanitizer/ChangeLog
>>> 2014-05-XX  Kostya Serebryany  <kcc@google.com>
>>> 
>>>        * All source files: Merge from upstream r208536.
>>>        * asan/Makefile.am (asan_files): Added new files.
>>>        * asan/Makefile.in: Regenerate.
>>>        * tsan/Makefile.am (tsan_files): Added new files.
>>>        * tsan/Makefile.in: Regenerate.
>>>        * sanitizer_common/Makefile.am (sanitizer_common_files): Added
>>> new files.
>>>        * sanitizer_common/Makefile.in: Regenerate.
>>> 
>>> --kcc
>> 
>> sanitizer_common/sanitizer_platform_limits_posix.h has
>> 
>>  struct __sanitizer_timeb {
>>    long time;
>>    unsigned short millitm;
>>    short timezone;
>>    short dstflag;
>>  };
>> 
>> On Linux, timeb is
>> 
>> struct timeb
>>  {
>>    time_t time; /* Seconds since epoch, as from `time'.  */
>>    unsigned short int millitm; /* Additional milliseconds.  */
>>    short int timezone; /* Minutes west of GMT.  */
>>    short int dstflag; /* Nonzero if Daylight Savings Time used.  */
>>  };
>> 
>> For x32, long is 32-bit and time_t is 64-bit.  We need something
>> like
>> 
>> struct __sanitizer_timeb {
>> #if defined(__x86_64__) && !defined(_LP64)
>>    long long time;
>> #else
>>    long time;
>> #endif
>> 
>> similar to other places.
>> 
>> 
>> --
>> H.J.

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

On Tue, May 13, 2014 at 12:05 PM,  <pinskia@gmail.com> wrote:
>
>
>> On May 13, 2014, at 12:59 AM, Konstantin Serebryany <konstantin.s.serebryany@gmail.com> wrote:
>>
>> I've committed this upstream and will include it into my next updated patch:
>>
>> +#if defined(__x86_64__) && !defined(_LP64)
>> +  typedef long long __sanitizer_time_t;
>> +#else
>> +  typedef long __sanitizer_time_t;
>> +#endif
>> +
>>   struct __sanitizer_timeb {
>> -    long time;
>> +    __sanitizer_time_t time;
>>     unsigned short millitm;
>
>
> This will also be needed for ilp32 aarch64 too.
>>
>>
>> If this is not enough, please contribute patches directly upstream --
>> I can not accept them from here.
>
>
> I am still against the way Libsanitizer is being maintained.

I do not enjoy it either. Suggestions are welcome, and see below!

> I really think the maintainers of the source in gcc should submit the patches upstream and not burden the target maintainers with it.

This is impossible, sorry.
We do not know how to fix a platform we do not have access to
ourselves and we can not accept patches for upstream
other than by following the standard process because this will violate
the LLVM developer policy.


>  Or maybe having libsantizer as another project separate from gcc and LLVM would be even better.

libsanitizer is essentially a separate project.
It lives in a separate svn tree (compiler-rt).
We've recently made a set of changes to allow the libsanitizer tests
to be run with an arbitrary compiler
(i.e. this could be GCC, or some other build of Clang, or any other
compiler that implements appropriate instrumentation).

Ideally for me, GCC would pull the sanitizer sources as "svn external"
and the merge will be simply a change of revision.

--kcc


>
> Thanks,
> Andrew
>
>
>>
>> Also, what's the story with x32 in LLVM?
>> Is there any chance you can set up a public build bot for x32 asan (upstream)?
>>
>>> On Mon, May 12, 2014 at 9:53 PM, H.J. Lu <hjl.tools@gmail.com> wrote:
>>> On Mon, May 12, 2014 at 4:20 AM, Konstantin Serebryany
>>> <konstantin.s.serebryany@gmail.com> wrote:
>>>> This is the first libsanitizer merge in 4.10 (the last merge was in
>>>> December 2013).
>>>>
>>>> Tested on Ubuntu 12.04 like this:
>>>> rm -rf */{*/,}libsanitizer && make -j 50
>>>> make -j 40 -C gcc check-g{cc,++}
>>>> RUNTESTFLAGS='--target_board=unix\{-m32,-m64\} asan.exp' && \
>>>> make -j 40 -C gcc check-g{cc,++}
>>>> RUNTESTFLAGS='--target_board=unix\{-m32,-m64\} tsan.exp' && \
>>>> make -j 40 -C gcc check
>>>> RUNTESTFLAGS='--target_board=unix\{-m32,-m64\} ubsan.exp' && \
>>>> echo PASS
>>>>
>>>> 5 months' worth of changes may break any platform we are not testing ourselves
>>>> (that includes Ubuntu 12.04, 13.10, 14.04, Mac 10.9, Windows 7, Android ARM),
>>>> please help us test this patch on your favorite platform.
>>>>
>>>> Expected ChangeLog entries:
>>>> =============== gcc/testsuite/ChangeLog
>>>> 2014-05-XX  Kostya Serebryany  <kcc@google.com>
>>>>
>>>>        * c-c++-common/tsan/mutexset1.c: Update the test to match
>>>>        upstream r208536.
>>>>        * g++.dg/asan/symbolize-callback-1.C: Delete the deprecated test.
>>>>
>>>> =============== libsanitizer/ChangeLog
>>>> 2014-05-XX  Kostya Serebryany  <kcc@google.com>
>>>>
>>>>        * All source files: Merge from upstream r208536.
>>>>        * asan/Makefile.am (asan_files): Added new files.
>>>>        * asan/Makefile.in: Regenerate.
>>>>        * tsan/Makefile.am (tsan_files): Added new files.
>>>>        * tsan/Makefile.in: Regenerate.
>>>>        * sanitizer_common/Makefile.am (sanitizer_common_files): Added
>>>> new files.
>>>>        * sanitizer_common/Makefile.in: Regenerate.
>>>>
>>>> --kcc
>>>
>>> sanitizer_common/sanitizer_platform_limits_posix.h has
>>>
>>>  struct __sanitizer_timeb {
>>>    long time;
>>>    unsigned short millitm;
>>>    short timezone;
>>>    short dstflag;
>>>  };
>>>
>>> On Linux, timeb is
>>>
>>> struct timeb
>>>  {
>>>    time_t time; /* Seconds since epoch, as from `time'.  */
>>>    unsigned short int millitm; /* Additional milliseconds.  */
>>>    short int timezone; /* Minutes west of GMT.  */
>>>    short int dstflag; /* Nonzero if Daylight Savings Time used.  */
>>>  };
>>>
>>> For x32, long is 32-bit and time_t is 64-bit.  We need something
>>> like
>>>
>>> struct __sanitizer_timeb {
>>> #if defined(__x86_64__) && !defined(_LP64)
>>>    long long time;
>>> #else
>>>    long time;
>>> #endif
>>>
>>> similar to other places.
>>>
>>>
>>> --
>>> H.J.

Re: libsanitizer merge from upstream r208536

[Rainer Orth]

Konstantin Serebryany <konstantin.s.serebryany@gmail.com> writes:

>> I am still against the way Libsanitizer is being maintained.
>
> I do not enjoy it either. Suggestions are welcome, and see below!
>
>> I really think the maintainers of the source in gcc should submit the
>> patches upstream and not burden the target maintainers with it.
>
> This is impossible, sorry.
> We do not know how to fix a platform we do not have access to
> ourselves and we can not accept patches for upstream

Nobody expects you to: if things break for Solaris in Go/libgo by a
merge from upstream, I try to get things fixed in the gcc tree and the
changes are pushed back later (or rather: first integrated upstream and
than, often simulaneously, merged back into gcc).

> other than by following the standard process because this will violate
> the LLVM developer policy.

Which says what?  Your current policy seems to massively impede
contributions.

	Rainer

-- 
-----------------------------------------------------------------------------
Rainer Orth, Center for Biotechnology, Bielefeld University

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

>> other than by following the standard process because this will violate
>> the LLVM developer policy.
>
> Which says what?

http://llvm.org/docs/DeveloperPolicy.html#making-and-submitting-a-patch

"Once your patch is ready, submit it by emailing it to the appropriate
project’s commit mailing list (or commit it directly if applicable).
Alternatively, some patches get sent to the project’s development list
or component of the LLVM bug tracker, but the commit list is the
primary place for reviews and should generally be preferred."

> Your current policy seems to massively impede contributions.
How? (BTW, I am not the one who sets these policies and discussing
them with me makes little sense)

Or do you refer to our own (non-LLVM) requirement to use arc for
non-trivial libsanitizer patches?
The tool requires 3 minutes to set up for a patch author and saves us
(reviewers) much more time, so we will continue to require it.

--kcc

Re: libsanitizer merge from upstream r208536

[Rainer Orth]

Konstantin Serebryany <konstantin.s.serebryany@gmail.com> writes:

>>> other than by following the standard process because this will violate
>>> the LLVM developer policy.
>>
>> Which says what?
>
> http://llvm.org/docs/DeveloperPolicy.html#making-and-submitting-a-patch
>
> "Once your patch is ready, submit it by emailing it to the appropriate
> project’s commit mailing list (or commit it directly if applicable).
> Alternatively, some patches get sent to the project’s development list
> or component of the LLVM bug tracker, but the commit list is the
> primary place for reviews and should generally be preferred."
>
>> Your current policy seems to massively impede contributions.
> How? (BTW, I am not the one who sets these policies and discussing
> them with me makes little sense)

Requiring target maintainers to interact with several different
upstream projects to keep their port in gcc in shape just doesn't
scale.  There's no such requirement for boehm-gc, libffi, libgo,
classpath, but instead people directly involved in those projects (or
the upstream maintainers themselves) act as liaisons between gcc and
their project, removing that burden.  Just libsanitizer is the odd man
out here.

	Rainer

-- 
-----------------------------------------------------------------------------
Rainer Orth, Center for Biotechnology, Bielefeld University

Re: libsanitizer merge from upstream r208536

[Yury Gribov]

>> Your current policy seems to massively impede contributions.
> How? (BTW, I am not the one who sets these policies and discussing
> them with me makes little sense)

I think people are mainly worried with checking out Clang, making it to 
build (install Cmake, C++11-friendly compiler, etc.), then making 
testsuites to run (install Python, etc.) on their particular platform, etc.

(BTW I totally agree that forcing all patches to go through compiler-rt 
greatly simplifies maintenance of libsanitizer.)

-Y

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

[-- Attachment #1: Type: text/plain, Size: 161 bytes --]

New patch attached.
It is based on r208674 which enables LeakSanitizer
(https://code.google.com/p/address-sanitizer/wiki/LeakSanitizer) by
default in asan mode.

[-- Attachment #2: libsanitizer-208674.patch.gz --]
[-- Type: application/x-gzip, Size: 198284 bytes --]

Re: libsanitizer merge from upstream r208536

[H.J. Lu]

[-- Attachment #1: Type: text/plain, Size: 3447 bytes --]

On Tue, May 13, 2014 at 2:02 AM, Konstantin Serebryany
<konstantin.s.serebryany@gmail.com> wrote:
> New patch attached.
> It is based on r208674 which enables LeakSanitizer
> (https://code.google.com/p/address-sanitizer/wiki/LeakSanitizer) by
> default in asan mode.

There are a couple issues for x32:

1.  This change

@@ -56,13 +55,6 @@ typedef signed   long long sptr;  // NOLINT
 typedef unsigned long uptr;  // NOLINT
 typedef signed   long sptr;  // NOLINT
 #endif  // defined(_WIN64)
-#if defined(__x86_64__)
-// Since x32 uses ILP32 data model in 64-bit hardware mode,  we must use
-// 64-bit pointer to unwind stack frame.
-typedef unsigned long long uhwptr;  // NOLINT
-#else
-typedef uptr uhwptr;   // NOLINT
-#endif
 typedef unsigned char u8;
 typedef unsigned short u16;  // NOLINT
 typedef unsigned int u32;

@@ -120,46 +34,43 @@ uptr StackTrace::GetCurrentPc() {
 void StackTrace::FastUnwindStack(uptr pc, uptr bp,
                                  uptr stack_top, uptr stack_bottom,
                                  uptr max_depth) {
-  if (max_depth == 0) {
-    size = 0;
-    return;
-  }
+  CHECK_GE(max_depth, 2);
   trace[0] = pc;
   size = 1;
-  uhwptr *frame = (uhwptr *)bp;
-  uhwptr *prev_frame = frame - 1;
+  uptr *frame = (uptr *)bp;
+  uptr *prev_frame = frame - 1;
   if (stack_top < 4096) return;  // Sanity check for stack top.
   // Avoid infinite loop when frame == frame[0] by using frame > prev_frame.
   while (frame > prev_frame &&
-         frame < (uhwptr *)stack_top - 2 &&
-         frame > (uhwptr *)stack_bottom &&
+         frame < (uptr *)stack_top - 2 &&
+         frame > (uptr *)stack_bottom &&
          IsAligned((uptr)frame, sizeof(*frame)) &&
          size < max_depth) {
-    uhwptr pc1 = frame[1];
+    uptr pc1 = frame[1];
     if (pc1 != pc) {
-      trace[size++] = (uptr) pc1;
+      trace[size++] = pc1;
     }
     prev_frame = frame;
-    frame = (uhwptr *)frame[0];
+    frame = (uptr*)frame[0];
   }
 }

reverted:

2012-11-16  H.J. Lu  <hongjiu.lu@intel.com>

        PR other/55333
        * include/sanitizer/common_interface_defs.h (uhwptr): New type
        for hardware pointer.
        * sanitizer_common/sanitizer_stacktrace.cc
(StackTrace::FastUnwindStack):
        Replace uptr with uhwptr for stack unwind.

2. struct linux_dirent is incorrect for x32.  We need something like

struct linux_dirent {
#if defined(__x86_64__) && !defined(_LP64)
  unsigned long long d_ino;
  unsigned long long d_off;
#else
  unsigned long      d_ino;
  unsigned long      d_off;
#endif
  unsigned short     d_reclen;
  char               d_name[256];
};

3. Pointers passed to internal_syscall should be casted to (uptr).
Otherwise, they won't be properly extended to 64-bit.  We need
to use (uptr) in internal_sigprocmask, like

uptr internal_sigprocmask(int how, __sanitizer_sigset_t *set,
    __sanitizer_sigset_t *oldset) {
#if SANITIZER_FREEBSD
  return internal_syscall(SYSCALL(sigprocmask), how, set, oldset);
#else
  __sanitizer_kernel_sigset_t *k_set = (__sanitizer_kernel_sigset_t *)set;
  __sanitizer_kernel_sigset_t *k_oldset = (__sanitizer_kernel_sigset_t *)oldset;
  return internal_syscall(SYSCALL(rt_sigprocmask), (uptr)how,
(uptr)&k_set->sig[0],
      (uptr)&k_oldset->sig[0], sizeof(__sanitizer_kernel_sigset_t));
#endif
}

4. ThreadDescriptorSize returns wrong value for x32. Size of struct
pthread should be 1728.

I am enclosing patches for those issues.


-- 
H.J.

[-- Attachment #2: p1.patch --]
[-- Type: text/x-patch, Size: 1767 bytes --]

--- ./sanitizer_common/sanitizer_internal_defs.h.x32	2014-05-13 08:03:33.531933986 -0700
+++ ./sanitizer_common/sanitizer_internal_defs.h	2014-05-13 13:02:17.552659366 -0700
@@ -55,6 +55,13 @@ typedef signed   long long sptr;  // NOL
 typedef unsigned long uptr;  // NOLINT
 typedef signed   long sptr;  // NOLINT
 #endif  // defined(_WIN64)
+#if defined(__x86_64__)
+// Since x32 uses ILP32 data model in 64-bit hardware mode,  we must use
+// 64-bit pointer to unwind stack frame.
+typedef unsigned long long uhwptr;  // NOLINT
+#else
+typedef uptr uhwptr;   // NOLINT
+#endif
 typedef unsigned char u8;
 typedef unsigned short u16;  // NOLINT
 typedef unsigned int u32;
--- ./sanitizer_common/sanitizer_stacktrace.cc.x32	2014-05-13 08:03:33.532933986 -0700
+++ ./sanitizer_common/sanitizer_stacktrace.cc	2014-05-13 13:04:59.109780837 -0700
@@ -37,21 +37,21 @@ void StackTrace::FastUnwindStack(uptr pc
   CHECK_GE(max_depth, 2);
   trace[0] = pc;
   size = 1;
-  uptr *frame = (uptr *)bp;
-  uptr *prev_frame = frame - 1;
+  uhwptr *frame = (uhwptr *)bp;
+  uhwptr *prev_frame = frame - 1;
   if (stack_top < 4096) return;  // Sanity check for stack top.
   // Avoid infinite loop when frame == frame[0] by using frame > prev_frame.
   while (frame > prev_frame &&
-         frame < (uptr *)stack_top - 2 &&
-         frame > (uptr *)stack_bottom &&
+         frame < (uhwptr *)stack_top - 2 &&
+         frame > (uhwptr *)stack_bottom &&
          IsAligned((uptr)frame, sizeof(*frame)) &&
          size < max_depth) {
-    uptr pc1 = frame[1];
+    uhwptr pc1 = frame[1];
     if (pc1 != pc) {
-      trace[size++] = pc1;
+      trace[size++] = (uptr) pc1;
     }
     prev_frame = frame;
-    frame = (uptr*)frame[0];
+    frame = (uhwptr *)frame[0];
   }
 }
 

[-- Attachment #3: p2.patch --]
[-- Type: text/x-patch, Size: 591 bytes --]

--- ./sanitizer_common/sanitizer_linux.cc.x32	2014-05-13 08:03:33.528933987 -0700
+++ ./sanitizer_common/sanitizer_linux.cc	2014-05-13 11:26:03.033181658 -0700
@@ -451,8 +451,13 @@ void BlockingMutex::CheckLocked() {
 // Note that getdents64 uses a different structure format. We only provide the
 // 32-bit syscall here.
 struct linux_dirent {
+#if defined(__x86_64__) && !defined(_LP64)
+  unsigned long long d_ino;
+  unsigned long long d_off;
+#else
   unsigned long      d_ino;
   unsigned long      d_off;
+#endif
   unsigned short     d_reclen;
   char               d_name[256];
 };

[-- Attachment #4: p3.patch --]
[-- Type: text/x-patch, Size: 697 bytes --]

--- ./sanitizer_common/sanitizer_linux.cc.x32	2014-05-13 08:03:33.528933987 -0700
+++ ./sanitizer_common/sanitizer_linux.cc	2014-05-13 11:26:03.033181658 -0700
@@ -548,8 +553,8 @@ uptr internal_sigprocmask(int how, __san
 #else
   __sanitizer_kernel_sigset_t *k_set = (__sanitizer_kernel_sigset_t *)set;
   __sanitizer_kernel_sigset_t *k_oldset = (__sanitizer_kernel_sigset_t *)oldset;
-  return internal_syscall(SYSCALL(rt_sigprocmask), (uptr)how, &k_set->sig[0],
-      &k_oldset->sig[0], sizeof(__sanitizer_kernel_sigset_t));
+  return internal_syscall(SYSCALL(rt_sigprocmask), (uptr)how, (uptr)&k_set->sig[0],
+      (uptr)&k_oldset->sig[0], sizeof(__sanitizer_kernel_sigset_t));
 #endif
 }
 

[-- Attachment #5: p4.patch --]
[-- Type: text/x-patch, Size: 840 bytes --]

--- sanitizer_common/sanitizer_linux_libcdep.cc.x32	2014-05-13 08:03:33.524933988 -0700
+++ sanitizer_common/sanitizer_linux_libcdep.cc	2014-05-13 14:58:22.483884958 -0700
@@ -310,6 +310,9 @@ uptr ThreadDescriptorSize() {
     int minor = internal_simple_strtoll(buf + 8, &end, 10);
     if (end != buf + 8 && (*end == '\0' || *end == '.')) {
       /* sizeof(struct thread) values from various glibc versions.  */
+#if defined(__x86_64__) && !defined(_LP64)
+      val = 1728;
+#else
       if (minor <= 3)
         val = FIRST_32_SECOND_64(1104, 1696);
       else if (minor == 4)
@@ -324,6 +327,7 @@ uptr ThreadDescriptorSize() {
         val = FIRST_32_SECOND_64(1168, 2288);
       else
         val = FIRST_32_SECOND_64(1216, 2304);
+#endif
     }
     if (val)
       atomic_store(&kThreadDescriptorSize, val, memory_order_relaxed);

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

H.J.,
Thanks for the patches. Please (re)send them to llvm-commits,
otherwise I can not accept them.

--kcc

On Wed, May 14, 2014 at 2:31 AM, H.J. Lu <hjl.tools@gmail.com> wrote:
> On Tue, May 13, 2014 at 2:02 AM, Konstantin Serebryany
> <konstantin.s.serebryany@gmail.com> wrote:
>> New patch attached.
>> It is based on r208674 which enables LeakSanitizer
>> (https://code.google.com/p/address-sanitizer/wiki/LeakSanitizer) by
>> default in asan mode.
>
> There are a couple issues for x32:
>
> 1.  This change
>
> @@ -56,13 +55,6 @@ typedef signed   long long sptr;  // NOLINT
>  typedef unsigned long uptr;  // NOLINT
>  typedef signed   long sptr;  // NOLINT
>  #endif  // defined(_WIN64)
> -#if defined(__x86_64__)
> -// Since x32 uses ILP32 data model in 64-bit hardware mode,  we must use
> -// 64-bit pointer to unwind stack frame.
> -typedef unsigned long long uhwptr;  // NOLINT
> -#else
> -typedef uptr uhwptr;   // NOLINT
> -#endif
>  typedef unsigned char u8;
>  typedef unsigned short u16;  // NOLINT
>  typedef unsigned int u32;
>
> @@ -120,46 +34,43 @@ uptr StackTrace::GetCurrentPc() {
>  void StackTrace::FastUnwindStack(uptr pc, uptr bp,
>                                   uptr stack_top, uptr stack_bottom,
>                                   uptr max_depth) {
> -  if (max_depth == 0) {
> -    size = 0;
> -    return;
> -  }
> +  CHECK_GE(max_depth, 2);
>    trace[0] = pc;
>    size = 1;
> -  uhwptr *frame = (uhwptr *)bp;
> -  uhwptr *prev_frame = frame - 1;
> +  uptr *frame = (uptr *)bp;
> +  uptr *prev_frame = frame - 1;
>    if (stack_top < 4096) return;  // Sanity check for stack top.
>    // Avoid infinite loop when frame == frame[0] by using frame > prev_frame.
>    while (frame > prev_frame &&
> -         frame < (uhwptr *)stack_top - 2 &&
> -         frame > (uhwptr *)stack_bottom &&
> +         frame < (uptr *)stack_top - 2 &&
> +         frame > (uptr *)stack_bottom &&
>           IsAligned((uptr)frame, sizeof(*frame)) &&
>           size < max_depth) {
> -    uhwptr pc1 = frame[1];
> +    uptr pc1 = frame[1];
>      if (pc1 != pc) {
> -      trace[size++] = (uptr) pc1;
> +      trace[size++] = pc1;
>      }
>      prev_frame = frame;
> -    frame = (uhwptr *)frame[0];
> +    frame = (uptr*)frame[0];
>    }
>  }
>
> reverted:
>
> 2012-11-16  H.J. Lu  <hongjiu.lu@intel.com>
>
>         PR other/55333
>         * include/sanitizer/common_interface_defs.h (uhwptr): New type
>         for hardware pointer.
>         * sanitizer_common/sanitizer_stacktrace.cc
> (StackTrace::FastUnwindStack):
>         Replace uptr with uhwptr for stack unwind.
>
> 2. struct linux_dirent is incorrect for x32.  We need something like
>
> struct linux_dirent {
> #if defined(__x86_64__) && !defined(_LP64)
>   unsigned long long d_ino;
>   unsigned long long d_off;
> #else
>   unsigned long      d_ino;
>   unsigned long      d_off;
> #endif
>   unsigned short     d_reclen;
>   char               d_name[256];
> };
>
> 3. Pointers passed to internal_syscall should be casted to (uptr).
> Otherwise, they won't be properly extended to 64-bit.  We need
> to use (uptr) in internal_sigprocmask, like
>
> uptr internal_sigprocmask(int how, __sanitizer_sigset_t *set,
>     __sanitizer_sigset_t *oldset) {
> #if SANITIZER_FREEBSD
>   return internal_syscall(SYSCALL(sigprocmask), how, set, oldset);
> #else
>   __sanitizer_kernel_sigset_t *k_set = (__sanitizer_kernel_sigset_t *)set;
>   __sanitizer_kernel_sigset_t *k_oldset = (__sanitizer_kernel_sigset_t *)oldset;
>   return internal_syscall(SYSCALL(rt_sigprocmask), (uptr)how,
> (uptr)&k_set->sig[0],
>       (uptr)&k_oldset->sig[0], sizeof(__sanitizer_kernel_sigset_t));
> #endif
> }
>
> 4. ThreadDescriptorSize returns wrong value for x32. Size of struct
> pthread should be 1728.
>
> I am enclosing patches for those issues.
>
>
> --
> H.J.

Re: libsanitizer merge from upstream r208536

[Andrew Pinski]

On Thu, May 15, 2014 at 1:05 AM, Konstantin Serebryany
<konstantin.s.serebryany@gmail.com> wrote:
> H.J.,
> Thanks for the patches. Please (re)send them to llvm-commits,
> otherwise I can not accept them.


I think this is bogus reasoning.  You should be able to take and post
them yourself.  Those patches.

Thanks,
Andrew Pinski

>
> --kcc
>
> On Wed, May 14, 2014 at 2:31 AM, H.J. Lu <hjl.tools@gmail.com> wrote:
>> On Tue, May 13, 2014 at 2:02 AM, Konstantin Serebryany
>> <konstantin.s.serebryany@gmail.com> wrote:
>>> New patch attached.
>>> It is based on r208674 which enables LeakSanitizer
>>> (https://code.google.com/p/address-sanitizer/wiki/LeakSanitizer) by
>>> default in asan mode.
>>
>> There are a couple issues for x32:
>>
>> 1.  This change
>>
>> @@ -56,13 +55,6 @@ typedef signed   long long sptr;  // NOLINT
>>  typedef unsigned long uptr;  // NOLINT
>>  typedef signed   long sptr;  // NOLINT
>>  #endif  // defined(_WIN64)
>> -#if defined(__x86_64__)
>> -// Since x32 uses ILP32 data model in 64-bit hardware mode,  we must use
>> -// 64-bit pointer to unwind stack frame.
>> -typedef unsigned long long uhwptr;  // NOLINT
>> -#else
>> -typedef uptr uhwptr;   // NOLINT
>> -#endif
>>  typedef unsigned char u8;
>>  typedef unsigned short u16;  // NOLINT
>>  typedef unsigned int u32;
>>
>> @@ -120,46 +34,43 @@ uptr StackTrace::GetCurrentPc() {
>>  void StackTrace::FastUnwindStack(uptr pc, uptr bp,
>>                                   uptr stack_top, uptr stack_bottom,
>>                                   uptr max_depth) {
>> -  if (max_depth == 0) {
>> -    size = 0;
>> -    return;
>> -  }
>> +  CHECK_GE(max_depth, 2);
>>    trace[0] = pc;
>>    size = 1;
>> -  uhwptr *frame = (uhwptr *)bp;
>> -  uhwptr *prev_frame = frame - 1;
>> +  uptr *frame = (uptr *)bp;
>> +  uptr *prev_frame = frame - 1;
>>    if (stack_top < 4096) return;  // Sanity check for stack top.
>>    // Avoid infinite loop when frame == frame[0] by using frame > prev_frame.
>>    while (frame > prev_frame &&
>> -         frame < (uhwptr *)stack_top - 2 &&
>> -         frame > (uhwptr *)stack_bottom &&
>> +         frame < (uptr *)stack_top - 2 &&
>> +         frame > (uptr *)stack_bottom &&
>>           IsAligned((uptr)frame, sizeof(*frame)) &&
>>           size < max_depth) {
>> -    uhwptr pc1 = frame[1];
>> +    uptr pc1 = frame[1];
>>      if (pc1 != pc) {
>> -      trace[size++] = (uptr) pc1;
>> +      trace[size++] = pc1;
>>      }
>>      prev_frame = frame;
>> -    frame = (uhwptr *)frame[0];
>> +    frame = (uptr*)frame[0];
>>    }
>>  }
>>
>> reverted:
>>
>> 2012-11-16  H.J. Lu  <hongjiu.lu@intel.com>
>>
>>         PR other/55333
>>         * include/sanitizer/common_interface_defs.h (uhwptr): New type
>>         for hardware pointer.
>>         * sanitizer_common/sanitizer_stacktrace.cc
>> (StackTrace::FastUnwindStack):
>>         Replace uptr with uhwptr for stack unwind.
>>
>> 2. struct linux_dirent is incorrect for x32.  We need something like
>>
>> struct linux_dirent {
>> #if defined(__x86_64__) && !defined(_LP64)
>>   unsigned long long d_ino;
>>   unsigned long long d_off;
>> #else
>>   unsigned long      d_ino;
>>   unsigned long      d_off;
>> #endif
>>   unsigned short     d_reclen;
>>   char               d_name[256];
>> };
>>
>> 3. Pointers passed to internal_syscall should be casted to (uptr).
>> Otherwise, they won't be properly extended to 64-bit.  We need
>> to use (uptr) in internal_sigprocmask, like
>>
>> uptr internal_sigprocmask(int how, __sanitizer_sigset_t *set,
>>     __sanitizer_sigset_t *oldset) {
>> #if SANITIZER_FREEBSD
>>   return internal_syscall(SYSCALL(sigprocmask), how, set, oldset);
>> #else
>>   __sanitizer_kernel_sigset_t *k_set = (__sanitizer_kernel_sigset_t *)set;
>>   __sanitizer_kernel_sigset_t *k_oldset = (__sanitizer_kernel_sigset_t *)oldset;
>>   return internal_syscall(SYSCALL(rt_sigprocmask), (uptr)how,
>> (uptr)&k_set->sig[0],
>>       (uptr)&k_oldset->sig[0], sizeof(__sanitizer_kernel_sigset_t));
>> #endif
>> }
>>
>> 4. ThreadDescriptorSize returns wrong value for x32. Size of struct
>> pthread should be 1728.
>>
>> I am enclosing patches for those issues.
>>
>>
>> --
>> H.J.

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

On Thu, May 15, 2014 at 12:07 PM, Andrew Pinski <pinskia@gmail.com> wrote:
> On Thu, May 15, 2014 at 1:05 AM, Konstantin Serebryany
> <konstantin.s.serebryany@gmail.com> wrote:
>> H.J.,
>> Thanks for the patches. Please (re)send them to llvm-commits,
>> otherwise I can not accept them.
>
>
> I think this is bogus reasoning.  You should be able to take and post
> them yourself.  Those patches.

I can't, sorry.

Re: libsanitizer merge from upstream r208536

[Rainer Orth]

pinskia@gmail.com writes:

>> If this is not enough, please contribute patches directly upstream --
>> I can not accept them from here.
>
>
> I am still against the way Libsanitizer is being maintained. I really think
> the maintainers of the source in gcc should submit the patches upstream and
> not burden the target maintainers with it.  Or maybe having libsantizer as

Agreed: this works just fine for Go/libgo (and used to work similarly
for the copy of classpath in libjava, where the Java maintainers handled
this).  If I contribute patches on the GCC side, Ian takes care of
getting them in upstream.  I do need a Google contributor agreement, but
that's ok.

	Rainer

-- 
-----------------------------------------------------------------------------
Rainer Orth, Center for Biotechnology, Bielefeld University

Re: libsanitizer merge from upstream r208536

[Jakub Jelinek]

On Mon, May 12, 2014 at 03:20:37PM +0400, Konstantin Serebryany wrote:
> This is the first libsanitizer merge in 4.10 (the last merge was in
> December 2013).

Worked on x86_64-linux and i686-linux (Fedora 20 and 17), bootstrap/regtest
showed no regressions.

	Jakub

Re: libsanitizer merge from upstream r208536

[Andrew Pinski]

On Mon, May 12, 2014 at 4:20 AM, Konstantin Serebryany
<konstantin.s.serebryany@gmail.com> wrote:
> This is the first libsanitizer merge in 4.10 (the last merge was in
> December 2013).
>
> Tested on Ubuntu 12.04 like this:
> rm -rf */{*/,}libsanitizer && make -j 50
> make -j 40 -C gcc check-g{cc,++}
> RUNTESTFLAGS='--target_board=unix\{-m32,-m64\} asan.exp' && \
> make -j 40 -C gcc check-g{cc,++}
> RUNTESTFLAGS='--target_board=unix\{-m32,-m64\} tsan.exp' && \
> make -j 40 -C gcc check
> RUNTESTFLAGS='--target_board=unix\{-m32,-m64\} ubsan.exp' && \
> echo PASS
>
> 5 months' worth of changes may break any platform we are not testing ourselves
> (that includes Ubuntu 12.04, 13.10, 14.04, Mac 10.9, Windows 7, Android ARM),
> please help us test this patch on your favorite platform.

One theme I have been noticing in the libsanitizer code is that it has
all of the knowledge of glibc when it comes to syscalls but makes some
bad assumptions dealing with some structures.  For an example on MIPS
n64: stat and stat64 are different layout between user and kernel.
Don't ask, it is a historical mistake.

Also it really forces target maintainers to maintain both glibc and
libsanitizer code.  ILP32 on aarch64 has some similar issues as x32
and MIPS n32.
Even Linus is pushing new 32bit targets to go the route of 64bit time
which is means libsanitizer code becomes even more complex.

Is there a way to reduce this maintenance overhead?  Can we please
split the target specific structures out to a header file which can
then be included.  This will reduce the overall messy #if's all of the
code and help to add new targets.

Thanks,
Andrew Pinski


>
> Expected ChangeLog entries:
> =============== gcc/testsuite/ChangeLog
> 2014-05-XX  Kostya Serebryany  <kcc@google.com>
>
>         * c-c++-common/tsan/mutexset1.c: Update the test to match
>         upstream r208536.
>         * g++.dg/asan/symbolize-callback-1.C: Delete the deprecated test.
>
> =============== libsanitizer/ChangeLog
> 2014-05-XX  Kostya Serebryany  <kcc@google.com>
>
>         * All source files: Merge from upstream r208536.
>         * asan/Makefile.am (asan_files): Added new files.
>         * asan/Makefile.in: Regenerate.
>         * tsan/Makefile.am (tsan_files): Added new files.
>         * tsan/Makefile.in: Regenerate.
>         * sanitizer_common/Makefile.am (sanitizer_common_files): Added
> new files.
>         * sanitizer_common/Makefile.in: Regenerate.
>
> --kcc

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

[plain text]

On Wed, May 14, 2014 at 2:42 AM, Andrew Pinski <pinskia@gmail.com> wrote:
> On Mon, May 12, 2014 at 4:20 AM, Konstantin Serebryany
> <konstantin.s.serebryany@gmail.com> wrote:
>> This is the first libsanitizer merge in 4.10 (the last merge was in
>> December 2013).
>>
>> Tested on Ubuntu 12.04 like this:
>> rm -rf */{*/,}libsanitizer && make -j 50
>> make -j 40 -C gcc check-g{cc,++}
>> RUNTESTFLAGS='--target_board=unix\{-m32,-m64\} asan.exp' && \
>> make -j 40 -C gcc check-g{cc,++}
>> RUNTESTFLAGS='--target_board=unix\{-m32,-m64\} tsan.exp' && \
>> make -j 40 -C gcc check
>> RUNTESTFLAGS='--target_board=unix\{-m32,-m64\} ubsan.exp' && \
>> echo PASS
>>
>> 5 months' worth of changes may break any platform we are not testing ourselves
>> (that includes Ubuntu 12.04, 13.10, 14.04, Mac 10.9, Windows 7, Android ARM),
>> please help us test this patch on your favorite platform.
>
> One theme I have been noticing in the libsanitizer code is that it has
> all of the knowledge of glibc when it comes to syscalls but makes some
> bad assumptions dealing with some structures.  For an example on MIPS
> n64: stat and stat64 are different layout between user and kernel.
> Don't ask, it is a historical mistake.
>
> Also it really forces target maintainers to maintain both glibc and
> libsanitizer code.  ILP32 on aarch64 has some similar issues as x32
> and MIPS n32.
> Even Linus is pushing new 32bit targets to go the route of 64bit time
> which is means libsanitizer code becomes even more complex.
This is indeed a PITA.
It is caused by our need to avoid even greater PITA of including
system headers into the sanitizer files
(we allow system #includes only in a handful of files).
>
> Is there a way to reduce this maintenance overhead?  Can we please
> split the target specific structures out to a header file which can
> then be included.  This will reduce the overall messy #if's all of the
> code and help to add new targets.

The problem is far from trivial.
I will be more than happy to receive help in form of patches.

--kcc

>
> Thanks,
> Andrew Pinski
>
>
>>
>> Expected ChangeLog entries:
>> =============== gcc/testsuite/ChangeLog
>> 2014-05-XX  Kostya Serebryany  <kcc@google.com>
>>
>>         * c-c++-common/tsan/mutexset1.c: Update the test to match
>>         upstream r208536.
>>         * g++.dg/asan/symbolize-callback-1.C: Delete the deprecated test.
>>
>> =============== libsanitizer/ChangeLog
>> 2014-05-XX  Kostya Serebryany  <kcc@google.com>
>>
>>         * All source files: Merge from upstream r208536.
>>         * asan/Makefile.am (asan_files): Added new files.
>>         * asan/Makefile.in: Regenerate.
>>         * tsan/Makefile.am (tsan_files): Added new files.
>>         * tsan/Makefile.in: Regenerate.
>>         * sanitizer_common/Makefile.am (sanitizer_common_files): Added
>> new files.
>>         * sanitizer_common/Makefile.in: Regenerate.
>>
>> --kcc

Re: libsanitizer merge from upstream r208536

[Yury Gribov]

On 05/14/2014 08:51 AM, Konstantin Serebryany wrote:
>> One theme I have been noticing in the libsanitizer code is that it has
>> all of the knowledge of glibc when it comes to syscalls but makes some
>> bad assumptions dealing with some structures.  For an example on MIPS
>> n64: stat and stat64 are different layout between user and kernel.
>> Don't ask, it is a historical mistake.
>>
>> Also it really forces target maintainers to maintain both glibc and
>> libsanitizer code.  ILP32 on aarch64 has some similar issues as x32
>> and MIPS n32.
>> Even Linus is pushing new 32bit targets to go the route of 64bit time
>> which is means libsanitizer code becomes even more complex.
>
>This is indeed a PITA.
> It is caused by our need to avoid even greater PITA of including
> system headers into the sanitizer files
> (we allow system #includes only in a handful of files).

Have you considered adding a configure-like step to your build process 
to determine necessary sizes and then provide them to sanitizer files 
via some config.h?

-Y

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

On Thu, May 15, 2014 at 9:28 AM, Yury Gribov <y.gribov@samsung.com> wrote:
> On 05/14/2014 08:51 AM, Konstantin Serebryany wrote:
>>>
>>> One theme I have been noticing in the libsanitizer code is that it has
>>> all of the knowledge of glibc when it comes to syscalls but makes some
>>> bad assumptions dealing with some structures.  For an example on MIPS
>>> n64: stat and stat64 are different layout between user and kernel.
>>> Don't ask, it is a historical mistake.
>>>
>>> Also it really forces target maintainers to maintain both glibc and
>>> libsanitizer code.  ILP32 on aarch64 has some similar issues as x32
>>> and MIPS n32.
>>> Even Linus is pushing new 32bit targets to go the route of 64bit time
>>> which is means libsanitizer code becomes even more complex.
>>
>>
>> This is indeed a PITA.
>> It is caused by our need to avoid even greater PITA of including
>> system headers into the sanitizer files
>> (we allow system #includes only in a handful of files).
>
>
> Have you considered adding a configure-like step to your build process to
> determine necessary sizes and then provide them to sanitizer files via some
> config.h?

No. We have to support too many build systems and hence do not want
any configure step.
All configuration has to be done in the sources.

Re: libsanitizer merge from upstream r208536

[Andrew Pinski]

On Thu, May 15, 2014 at 1:05 AM, Konstantin Serebryany
<konstantin.s.serebryany@gmail.com> wrote:
> On Thu, May 15, 2014 at 9:28 AM, Yury Gribov <y.gribov@samsung.com> wrote:
>> On 05/14/2014 08:51 AM, Konstantin Serebryany wrote:
>>>>
>>>> One theme I have been noticing in the libsanitizer code is that it has
>>>> all of the knowledge of glibc when it comes to syscalls but makes some
>>>> bad assumptions dealing with some structures.  For an example on MIPS
>>>> n64: stat and stat64 are different layout between user and kernel.
>>>> Don't ask, it is a historical mistake.
>>>>
>>>> Also it really forces target maintainers to maintain both glibc and
>>>> libsanitizer code.  ILP32 on aarch64 has some similar issues as x32
>>>> and MIPS n32.
>>>> Even Linus is pushing new 32bit targets to go the route of 64bit time
>>>> which is means libsanitizer code becomes even more complex.
>>>
>>>
>>> This is indeed a PITA.
>>> It is caused by our need to avoid even greater PITA of including
>>> system headers into the sanitizer files
>>> (we allow system #includes only in a handful of files).
>>
>>
>> Have you considered adding a configure-like step to your build process to
>> determine necessary sizes and then provide them to sanitizer files via some
>> config.h?
>
> No. We have to support too many build systems and hence do not want
> any configure step.
> All configuration has to be done in the sources.


I think  this argument is bogus.  Please do one build system and
support it.  Simple makefile and some scripts seems simple to support
so you don't need anything extra.

Thanks,
Andrew Pinski

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

>
> I think  this argument is bogus.  Please do one build system and
> support it.  Simple makefile and some scripts seems simple to support
> so you don't need anything extra.

Would be glad to. One (but not the only) prerequisite for that GCC
exports libsanitizer
as "svn external" and uses our cmake build for libsanitizer.

Re: libsanitizer merge from upstream r208536

[Yury Gribov]

On 05/15/2014 12:05 PM, Konstantin Serebryany wrote:
> No. We have to support too many build systems and hence do not want
> any configure step.
> All configuration has to be done in the sources.

Yeah, I see your point. But filling code with magic constants isn't very 
nice either.

We could make a separate CMake check like GetTargetStructSize or 
something (somewhere in projects/compiler-rt/make/platform/*.mk).

-Y

Re: libsanitizer merge from upstream r208536

[Jakub Jelinek]

On Thu, May 15, 2014 at 12:20:06PM +0400, Yury Gribov wrote:
> On 05/15/2014 12:05 PM, Konstantin Serebryany wrote:
> >No. We have to support too many build systems and hence do not want
> >any configure step.
> >All configuration has to be done in the sources.
> 
> Yeah, I see your point. But filling code with magic constants isn't
> very nice either.
> 
> We could make a separate CMake check like GetTargetStructSize or
> something (somewhere in projects/compiler-rt/make/platform/*.mk).

Also, I'd suggest to have a look at glibc or libgomp how a directory system
with easily overridable target specific headers is way better than a maze of
#ifdefs all around the source.
I mean glibc sysdeps/ stuff or libgomp config/ stuff (the latter is far more
limited of course).
So, individual structures/constants/defines (or related groups thereof)
could be split into separate headers, and have target specific set of
-I arguments added by the Makefile/CMake, so that you can have some
defaults, some per-CPU defaults, some OS defaults and some per-CPU OS
defaults.  Say, you could have a header file with layout of kernel stat
structure, one for Linux, another one for Darwin, another one for Windows,
and then if say on ppc64-linux it is different, it could supply it's own
file that would override the generic Linux one.

	Jakub

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

[-- Attachment #1: Type: text/plain, Size: 1524 bytes --]

A new patch based on r209283.
This one has the H.J.'s patches for x32.

On Thu, May 15, 2014 at 12:39 PM, Jakub Jelinek <jakub@redhat.com> wrote:
> On Thu, May 15, 2014 at 12:20:06PM +0400, Yury Gribov wrote:
>> On 05/15/2014 12:05 PM, Konstantin Serebryany wrote:
>> >No. We have to support too many build systems and hence do not want
>> >any configure step.
>> >All configuration has to be done in the sources.
>>
>> Yeah, I see your point. But filling code with magic constants isn't
>> very nice either.
>>
>> We could make a separate CMake check like GetTargetStructSize or
>> something (somewhere in projects/compiler-rt/make/platform/*.mk).
>
> Also, I'd suggest to have a look at glibc or libgomp how a directory system
> with easily overridable target specific headers is way better than a maze of
> #ifdefs all around the source.
> I mean glibc sysdeps/ stuff or libgomp config/ stuff (the latter is far more
> limited of course).
> So, individual structures/constants/defines (or related groups thereof)
> could be split into separate headers, and have target specific set of
> -I arguments added by the Makefile/CMake, so that you can have some
> defaults, some per-CPU defaults, some OS defaults and some per-CPU OS
> defaults.  Say, you could have a header file with layout of kernel stat
> structure, one for Linux, another one for Darwin, another one for Windows,
> and then if say on ppc64-linux it is different, it could supply it's own
> file that would override the generic Linux one.
>
>         Jakub

[-- Attachment #2: libsanitizer-209283.patch.gz --]
[-- Type: application/x-gzip, Size: 207535 bytes --]

Re: libsanitizer merge from upstream r208536

[H.J. Lu]

On Wed, May 21, 2014 at 5:09 AM, Konstantin Serebryany
<konstantin.s.serebryany@gmail.com> wrote:
> A new patch based on r209283.
> This one has the H.J.'s patches for x32.
>

This one is good on x32.

Thanks.

-- 
H.J.

Re: libsanitizer merge from upstream r208536

[Jakub Jelinek]

On Wed, May 21, 2014 at 04:09:19PM +0400, Konstantin Serebryany wrote:
> A new patch based on r209283.
> This one has the H.J.'s patches for x32.

Ok for trunk then.  But please help the ppc*/arm*/sparc* maintainers if issues on
those targets are reported.
Thanks.

	Jakub

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

On Wed, May 21, 2014 at 11:43 PM, Jakub Jelinek <jakub@redhat.com> wrote:
> On Wed, May 21, 2014 at 04:09:19PM +0400, Konstantin Serebryany wrote:
>> A new patch based on r209283.
>> This one has the H.J.'s patches for x32.
>
> Ok for trunk then.  But please help the ppc*/arm*/sparc* maintainers if issues on
> those targets are reported.
Of course. arm should be in a good shape since there are arm users
upstream, including ourselves.
ppc may get broken -- I haven't heard from anyone there for a while.
Same for sparc.
I'll commit later today.


> Thanks.
>
>         Jakub

Re: libsanitizer merge from upstream r208536

[Paolo Carlini]

.. sorry, I didn't follow the whole thread, but today I'm seeing loads 
of failures when testing C++ on x86_64-linux, beginning with:

FAIL: c-c++-common/asan/asan-interface-1.c  -O0  execution test
FAIL: c-c++-common/asan/asan-interface-1.c  -O1  execution test
FAIL: c-c++-common/asan/asan-interface-1.c  -O2  execution test
FAIL: c-c++-common/asan/asan-interface-1.c  -O3 -fomit-frame-pointer  
execution test
FAIL: c-c++-common/asan/asan-interface-1.c  -O3 -g  execution test
FAIL: c-c++-common/asan/asan-interface-1.c  -Os  execution test
FAIL: c-c++-common/asan/asan-interface-1.c  -O2 -flto 
-fno-use-linker-plugin -flto-partition=none  execution test
FAIL: c-c++-common/asan/asan-interface-1.c  -O2 -flto 
-fuse-linker-plugin -fno-fat-lto-objects  execution test
...

this is already being addressed, right? Because I cannot really work.

Thanks,
Paolo.

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

On Thu, May 22, 2014 at 12:54 PM, Paolo Carlini
<paolo.carlini@oracle.com> wrote:
> .. sorry, I didn't follow the whole thread, but today I'm seeing loads of
> failures when testing C++ on x86_64-linux, beginning with:
>
> FAIL: c-c++-common/asan/asan-interface-1.c  -O0  execution test
Is that before or after r210743?
> FAIL: c-c++-common/asan/asan-interface-1.c  -O1  execution test
> FAIL: c-c++-common/asan/asan-interface-1.c  -O2  execution test
> FAIL: c-c++-common/asan/asan-interface-1.c  -O3 -fomit-frame-pointer
> execution test
> FAIL: c-c++-common/asan/asan-interface-1.c  -O3 -g  execution test
> FAIL: c-c++-common/asan/asan-interface-1.c  -Os  execution test
> FAIL: c-c++-common/asan/asan-interface-1.c  -O2 -flto -fno-use-linker-plugin
> -flto-partition=none  execution test
> FAIL: c-c++-common/asan/asan-interface-1.c  -O2 -flto -fuse-linker-plugin
> -fno-fat-lto-objects  execution test
> ...
>
> this is already being addressed, right? Because I cannot really work.
>
> Thanks,
> Paolo.
>
>

Re: libsanitizer merge from upstream r208536

[Paolo Carlini]

Hi,

On 22 maggio 2014 11:05:31 CEST, Konstantin Serebryany <konstantin.s.serebryany@gmail.com> wrote:
>On Thu, May 22, 2014 at 12:54 PM, Paolo Carlini
><paolo.carlini@oracle.com> wrote:
>> .. sorry, I didn't follow the whole thread, but today I'm seeing
>loads of
>> failures when testing C++ on x86_64-linux, beginning with:
>>
>> FAIL: c-c++-common/asan/asan-interface-1.c  -O0  execution test
>Is that before or after r210743?

This is after the merge, up to date tree 30 mins ago. Yesterday everything was ok, as usual.

Paolo

Re: libsanitizer merge from upstream r208536

[Jakub Jelinek]

On Thu, May 22, 2014 at 11:44:06AM +0200, Paolo Carlini wrote:
> On 22 maggio 2014 11:05:31 CEST, Konstantin Serebryany <konstantin.s.serebryany@gmail.com> wrote:
> >On Thu, May 22, 2014 at 12:54 PM, Paolo Carlini
> ><paolo.carlini@oracle.com> wrote:
> >> .. sorry, I didn't follow the whole thread, but today I'm seeing
> >loads of
> >> failures when testing C++ on x86_64-linux, beginning with:
> >>
> >> FAIL: c-c++-common/asan/asan-interface-1.c  -O0  execution test
> >Is that before or after r210743?
> 
> This is after the merge, up to date tree 30 mins ago. Yesterday everything was ok, as usual.

Yesterday

FAIL: c-c++-common/asan/asan-interface-1.c  -O0  (test for excess errors)
UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -O0  compilation failed to produce executable
FAIL: c-c++-common/asan/asan-interface-1.c  -O1  (test for excess errors)
UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -O1  compilation failed to produce executable
FAIL: c-c++-common/asan/asan-interface-1.c  -O2  (test for excess errors)
UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -O2  compilation failed to produce executable
FAIL: c-c++-common/asan/asan-interface-1.c  -O3 -fomit-frame-pointer  (test for excess errors)
UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -O3 -fomit-frame-pointer  compilation failed to produce executable
FAIL: c-c++-common/asan/asan-interface-1.c  -O3 -g  (test for excess errors)
UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -O3 -g  compilation failed to produce executable
FAIL: c-c++-common/asan/asan-interface-1.c  -Os  (test for excess errors)
UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -Os  compilation failed to produce executable
FAIL: c-c++-common/asan/asan-interface-1.c  -O2 -flto -fno-use-linker-plugin -flto-partition=none  (test for excess errors)
UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -O2 -flto -fno-use-linker-plugin -flto-partition=none  compilation failed to produce executable
FAIL: c-c++-common/asan/asan-interface-1.c  -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects  (test for excess errors)
UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects  compilation failed to produce executable

failed for C, but not C++.

	Jakub

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

What is the exact command are you running?

On Thu, May 22, 2014 at 1:47 PM, Jakub Jelinek <jakub@redhat.com> wrote:
> On Thu, May 22, 2014 at 11:44:06AM +0200, Paolo Carlini wrote:
>> On 22 maggio 2014 11:05:31 CEST, Konstantin Serebryany <konstantin.s.serebryany@gmail.com> wrote:
>> >On Thu, May 22, 2014 at 12:54 PM, Paolo Carlini
>> ><paolo.carlini@oracle.com> wrote:
>> >> .. sorry, I didn't follow the whole thread, but today I'm seeing
>> >loads of
>> >> failures when testing C++ on x86_64-linux, beginning with:
>> >>
>> >> FAIL: c-c++-common/asan/asan-interface-1.c  -O0  execution test
>> >Is that before or after r210743?
>>
>> This is after the merge, up to date tree 30 mins ago. Yesterday everything was ok, as usual.
>
> Yesterday
>
> FAIL: c-c++-common/asan/asan-interface-1.c  -O0  (test for excess errors)
> UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -O0  compilation failed to produce executable
> FAIL: c-c++-common/asan/asan-interface-1.c  -O1  (test for excess errors)
> UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -O1  compilation failed to produce executable
> FAIL: c-c++-common/asan/asan-interface-1.c  -O2  (test for excess errors)
> UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -O2  compilation failed to produce executable
> FAIL: c-c++-common/asan/asan-interface-1.c  -O3 -fomit-frame-pointer  (test for excess errors)
> UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -O3 -fomit-frame-pointer  compilation failed to produce executable
> FAIL: c-c++-common/asan/asan-interface-1.c  -O3 -g  (test for excess errors)
> UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -O3 -g  compilation failed to produce executable
> FAIL: c-c++-common/asan/asan-interface-1.c  -Os  (test for excess errors)
> UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -Os  compilation failed to produce executable
> FAIL: c-c++-common/asan/asan-interface-1.c  -O2 -flto -fno-use-linker-plugin -flto-partition=none  (test for excess errors)
> UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -O2 -flto -fno-use-linker-plugin -flto-partition=none  compilation failed to produce executable
> FAIL: c-c++-common/asan/asan-interface-1.c  -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects  (test for excess errors)
> UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects  compilation failed to produce executable
>
> failed for C, but not C++.
>
>         Jakub

Re: libsanitizer merge from upstream r208536

[Paolo Carlini]

Hi

On 22 maggio 2014 12:26:19 CEST, Konstantin Serebryany <konstantin.s.serebryany@gmail.com> wrote:
>What is the exact command are you running?

Sorry, now I'm traveling. But nothing special, a very simple c and c++ build on x86_64-linux. No special flags, all defaults. If nobody can reproduce I'll try to fetch again the tree from scratch. Weird, anyway.

Paolo

Re: libsanitizer merge from upstream r208536

[Jakub Jelinek]

On Thu, May 22, 2014 at 02:26:19PM +0400, Konstantin Serebryany wrote:
> >> >> FAIL: c-c++-common/asan/asan-interface-1.c  -O0  execution test
> >> >Is that before or after r210743?

Can't reproduce the above (note, not bootstrapped compiler, just
--disable-bootstrap), check-gcc RUNTESTFLAGS=asan.exp is clean, but
check-g++ RUNTESTFLAGS=asan.exp I got:
Running /usr/src/gcc/gcc/testsuite/g++.dg/asan/asan.exp ...
FAIL: g++.dg/asan/asan_test.C  -O2  AddressSanitizer_MallocUsableSizeTest execution test
:
Setting LD_LIBRARY_PATH to .:/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libstdc++-v3/src/.libs:/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libst
dc++-v3/src/.libs:/usr/src/gcc/obj2/gcc:/usr/src/gcc/obj2/gcc/32:/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libsanitizer/asan/.libs:.:/usr/src/gc
c/obj2/x86_64-unknown-linux-gnu/./libstdc++-v3/src/.libs:/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libstdc++-v3/src/.libs:/usr/src/gcc/obj2/gcc:
/usr/src/gcc/obj2/gcc/32:/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libsanitizer/asan/.libs
spawn [open ...]
DEJAGNU_GTEST_EXPECT_DEATH1 malloc_usable_size((void*)0x123) DEJAGNU_GTEST_EXPECT_DEATH1 AddressSanitizer: attempting to call malloc_usable_size() 
DEJAGNU_GTEST_EXPECT_DEATH1
DEJAGNU_GTEST_EXPECT_DEATH2 malloc_usable_size(array + kArraySize / 2) DEJAGNU_GTEST_EXPECT_DEATH2 AddressSanitizer: attempting to call malloc_usab
le_size() DEJAGNU_GTEST_EXPECT_DEATH2
DEJAGNU_GTEST_EXPECT_DEATH3 malloc_usable_size(array) DEJAGNU_GTEST_EXPECT_DEATH3 AddressSanitizer: attempting to call malloc_usable_size() DEJAGNU
_GTEST_EXPECT_DEATH3

=================================================================
==28396==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 4 byte(s) in 1 object(s) allocated from:
    #0 0x7f5c18048df9 in operator new(unsigned long) ../../../../libsanitizer/asan/asan_new_delete.cc:53
    #1 0x409134 in AddressSanitizer_MallocUsableSizeTest_fn /usr/src/gcc/gcc/testsuite/g++.dg/asan/asan_test.cc:407

SUMMARY: AddressSanitizer: 4 byte(s) leaked in 1 allocation(s).
FAIL: g++.dg/asan/asan_test.C  -O2  AddressSanitizer_MallocUsableSizeTest execution test

If I manually compile it:
/usr/src/gcc/obj2/gcc/testsuite/g++/../../xg++ -B/usr/src/gcc/obj2/gcc/testsuite/g++/../../ /usr/src/gcc/gcc/testsuite/g++.dg/asan/asan_test.C -fsanitize=address -g -I/usr/src/gcc/gcc/testsuite/../../libsanitizer/include -fno-diagnostics-show-caret -fdiagnostics-color=never -nostdinc++ -I/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/libstdc++-v3/include/x86_64-unknown-linux-gnu -I/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/libstdc++-v3/include -I/usr/src/gcc/libstdc++-v3/libsupc++ -I/usr/src/gcc/libstdc++-v3/include/backward -I/usr/src/gcc/libstdc++-v3/testsuite/util -fmessage-length=0 -O2 -fsanitize=address -fno-builtin -Wall -Wno-format -Werror -g -DASAN_UAR=0 -DASAN_HAS_EXCEPTIONS=1 -DASAN_HAS_BLACKLIST=0 -DASAN_USE_DEJAGNU_GTEST=1 -lasan -lpthread -ldl -DASAN_NEEDS_SEGV=1 -DASAN_AVOID_EXPENSIVE_TESTS=1 -msse2 -D__NO_INLINE__ /usr/src/gcc/gcc/testsuite/g++.dg/asan/asan_globals_test-wrapper.cc -B/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libsanitizer/ -B/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libsanitizer/asan/ -L/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libsanitizer/asan/.libs -L/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libstdc++-v3/src/.libs -B/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libstdc++-v3/src/.libs -L/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libstdc++-v3/src/.libs -lm -o ./asan_test.exe -Wl,-rpath,/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libsanitizer/asan/.libs/
and run:
./asan_test.exe AddressSanitizer_MallocUsableSizeTest
I get the same error.

Seems compiler-rt has there:

--- gcc/testsuite/g++.dg/asan/asan_test.cc	2013-11-12 11:31:20.000000000 +0100
+++ gcc/testsuite/g++.dg/asan/asan_test.cc	2014-05-22 13:00:23.391901708 +0200
@@ -413,6 +413,7 @@ TEST(AddressSanitizer, MallocUsableSizeT
                kMallocUsableSizeErrorMsg);
   free(array);
   EXPECT_DEATH(malloc_usable_size(array), kMallocUsableSizeErrorMsg);
+  delete int_ptr;
 }
 #endif

There are various other changes to asan_test.cc, so guess some work is needed on that. 

	Jakub

Re: libsanitizer merge from upstream r208536

[Jakub Jelinek]

On Thu, May 22, 2014 at 01:03:48PM +0200, Jakub Jelinek wrote:
> There are various other changes to asan_test.cc, so guess some work is needed on that. 

Ok, tried to merge also g++.dg/asan from the same revision as you've merged
libsanitizer.

On x86_64-linux, both -m32 and -m64 I see
FAIL: g++.dg/asan/asan_test.C  -O2  AddressSanitizer_UAF_Packed5 p[0] = p[3] execution test
FAIL: g++.dg/asan/asan_test.C  -O2  AddressSanitizer_UAF_Packed5 p[3] = p[0] execution test
and with -m32 additionally:
FAIL: g++.dg/asan/asan_test.C  -O2  AddressSanitizer_UAF_long_double Ident(p)[12] = 0 execution test
FAIL: g++.dg/asan/asan_test.C  -O2  AddressSanitizer_UAF_long_double Ident(p)[0] = Ident(p)[12] execution test
The former boils down into:
struct Packed5 {
  int x;
  char c;
} __attribute__((packed));

int
main ()
{
  Packed5 *p = new Packed5[2];
  asm volatile ("" : "+r" (p));
  p[0] = p[3];
  p[3] = p[0];
  return 0;
}

Right now we instrument only accesses which have sizes power of two
<= 16:

  size_in_bytes = int_size_in_bytes (type);
  if ((size_in_bytes & (size_in_bytes - 1)) != 0
      || (unsigned HOST_WIDE_INT) size_in_bytes - 1 >= 16)
    return;

I thought so did LLVM, has that changed recently?

2014-05-22  Jakub Jelinek  <jakub@redhat.com>

	* g++.dg/asan/asan_test.C: Add -std=c++11 and
	-DSANITIZER_USE_DEJAGNU_GTEST=1 to dg-options, remove
	-DASAN_USE_DEJAGNU_GTEST=1.
	* g++.dg/asan/asan_mem_test.cc: Updated from upstream
	r209283.
	* g++.dg/asan/asan_oob_test.cc: Likewise.
	* g++.dg/asan/sanitizer_test_utils.h: Likewise.
	* g++.dg/asan/asan_str_test.cc: Likewise.
	* g++.dg/asan/asan_test_utils.h: Likewise.
	* g++.dg/asan/sanitizer_test_config.h: Likewise.
	* g++.dg/asan/asan_test.cc: Likewise.
	* g++.dg/asan/sanitizer_pthread_wrappers.h: New file.
	Imported from upstream r209283.
	* g++.dg/asan/asan_test_config.h: Likewise.

--- gcc/testsuite/g++.dg/asan/asan_test.C.jj	2013-02-14 09:38:05.000000000 +0100
+++ gcc/testsuite/g++.dg/asan/asan_test.C	2014-05-22 13:22:33.747459933 +0200
@@ -2,7 +2,7 @@
 // { dg-skip-if "" { *-*-* } { "*" } { "-O2" } }
 // { dg-skip-if "" { *-*-* } { "-flto" } { "" } }
 // { dg-additional-sources "asan_globals_test-wrapper.cc" }
-// { dg-options "-fsanitize=address -fno-builtin -Wall -Wno-format -Werror -g -DASAN_UAR=0 -DASAN_HAS_EXCEPTIONS=1 -DASAN_HAS_BLACKLIST=0 -DASAN_USE_DEJAGNU_GTEST=1 -lasan -lpthread -ldl" }
+// { dg-options "-std=c++11 -fsanitize=address -fno-builtin -Wall -Wno-format -Werror -g -DASAN_UAR=0 -DASAN_HAS_EXCEPTIONS=1 -DASAN_HAS_BLACKLIST=0 -DSANITIZER_USE_DEJAGNU_GTEST=1 -lasan -lpthread -ldl" }
 // { dg-additional-options "-DASAN_NEEDS_SEGV=1" { target { ! arm*-*-* } } }
 // { dg-additional-options "-DASAN_LOW_MEMORY=1 -DASAN_NEEDS_SEGV=0" { target arm*-*-* } }
 // { dg-additional-options "-DASAN_AVOID_EXPENSIVE_TESTS=1" { target { ! run_expensive_tests } } }
--- gcc/testsuite/g++.dg/asan/asan_mem_test.cc.jj	2013-02-14 09:38:05.697773648 +0100
+++ gcc/testsuite/g++.dg/asan/asan_mem_test.cc	2014-05-22 13:13:48.158014955 +0200
@@ -74,17 +74,17 @@ TEST(AddressSanitizer, MemSetOOBTest) {
 // Strictly speaking we are not guaranteed to find such two pointers,
 // but given the structure of asan's allocator we will.
 static bool AllocateTwoAdjacentArrays(char **x1, char **x2, size_t size) {
-  vector<char *> v;
+  vector<uintptr_t> v;
   bool res = false;
   for (size_t i = 0; i < 1000U && !res; i++) {
-    v.push_back(new char[size]);
+    v.push_back(reinterpret_cast<uintptr_t>(new char[size]));
     if (i == 0) continue;
     sort(v.begin(), v.end());
     for (size_t j = 1; j < v.size(); j++) {
       assert(v[j] > v[j-1]);
       if ((size_t)(v[j] - v[j-1]) < size * 2) {
-        *x2 = v[j];
-        *x1 = v[j-1];
+        *x2 = reinterpret_cast<char*>(v[j]);
+        *x1 = reinterpret_cast<char*>(v[j-1]);
         res = true;
         break;
       }
@@ -92,9 +92,10 @@ static bool AllocateTwoAdjacentArrays(ch
   }
 
   for (size_t i = 0; i < v.size(); i++) {
-    if (res && v[i] == *x1) continue;
-    if (res && v[i] == *x2) continue;
-    delete [] v[i];
+    char *p = reinterpret_cast<char *>(v[i]);
+    if (res && p == *x1) continue;
+    if (res && p == *x2) continue;
+    delete [] p;
   }
   return res;
 }
@@ -223,6 +224,13 @@ TEST(AddressSanitizer, MemCmpOOBTest) {
   s1[size - 1] = '\0';
   s2[size - 1] = '\0';
   EXPECT_DEATH(Ident(memcmp)(s1, s2, size + 1), RightOOBReadMessage(0));
+
+  // Even if the buffers differ in the first byte, we still assume that
+  // memcmp may access the whole buffer and thus reporting the overflow here:
+  s1[0] = 1;
+  s2[0] = 123;
+  EXPECT_DEATH(Ident(memcmp)(s1, s2, size + 1), RightOOBReadMessage(0));
+
   free(s1);
   free(s2);
 }
--- gcc/testsuite/g++.dg/asan/asan_oob_test.cc.jj	2013-02-14 09:38:05.697773648 +0100
+++ gcc/testsuite/g++.dg/asan/asan_oob_test.cc	2014-05-22 13:13:49.976009670 +0200
@@ -97,7 +97,6 @@ TEST(AddressSanitizer, OOBRightTest) {
   }
 }
 
-#if ASAN_ALLOCATOR_VERSION == 2  // Broken with the asan_allocator1
 TEST(AddressSanitizer, LargeOOBRightTest) {
   size_t large_power_of_two = 1 << 19;
   for (size_t i = 16; i <= 256; i *= 2) {
@@ -107,7 +106,6 @@ TEST(AddressSanitizer, LargeOOBRightTest
     delete [] p;
   }
 }
-#endif  // ASAN_ALLOCATOR_VERSION == 2
 
 TEST(AddressSanitizer, DISABLED_DemoOOBLeftLow) {
   oob_test<U1>(10, -1);
--- gcc/testsuite/g++.dg/asan/sanitizer_test_utils.h.jj	2013-02-28 22:22:03.000000000 +0100
+++ gcc/testsuite/g++.dg/asan/sanitizer_test_utils.h	2014-05-22 13:10:27.781920740 +0200
@@ -14,32 +14,47 @@
 #define SANITIZER_TEST_UTILS_H
 
 #if defined(_WIN32)
-typedef unsigned __int8  uint8_t;
-typedef unsigned __int16 uint16_t;
-typedef unsigned __int32 uint32_t;
-typedef unsigned __int64 uint64_t;
-typedef __int8           int8_t;
-typedef __int16          int16_t;
-typedef __int32          int32_t;
-typedef __int64          int64_t;
+// <windows.h> should always be the first include on Windows.
+# include <windows.h>
+// MSVS headers define max/min as macros, so std::max/min gets crazy.
+# undef max
+# undef min
+#endif
+
+#if !defined(SANITIZER_EXTERNAL_TEST_CONFIG)
+# define INCLUDED_FROM_SANITIZER_TEST_UTILS_H
+# include "sanitizer_test_config.h"
+# undef INCLUDED_FROM_SANITIZER_TEST_UTILS_H
+#endif
+
+#include <stdint.h>
+
+#if defined(_MSC_VER)
 # define NOINLINE __declspec(noinline)
-# define USED
-#else  // defined(_WIN32)
+#else  // defined(_MSC_VER)
 # define NOINLINE __attribute__((noinline))
+#endif  // defined(_MSC_VER)
+
+#if !defined(_MSC_VER) || defined(__clang__)
+# define UNUSED __attribute__((unused))
 # define USED __attribute__((used))
-#include <stdint.h>
-#endif  // defined(_WIN32)
+#else
+# define UNUSED
+# define USED
+#endif
 
 #if !defined(__has_feature)
 #define __has_feature(x) 0
 #endif
 
-#if __has_feature(address_sanitizer) || defined(__SANITIZE_ADDRESS__)
-# define ATTRIBUTE_NO_ADDRESS_SAFETY_ANALYSIS \
+#ifndef ATTRIBUTE_NO_SANITIZE_ADDRESS
+# if __has_feature(address_sanitizer) || defined(__SANITIZE_ADDRESS__)
+#  define ATTRIBUTE_NO_SANITIZE_ADDRESS \
     __attribute__((no_sanitize_address))
-#else
-# define ATTRIBUTE_NO_ADDRESS_SAFETY_ANALYSIS
-#endif
+# else
+#  define ATTRIBUTE_NO_SANITIZE_ADDRESS
+# endif
+#endif  // ATTRIBUTE_NO_SANITIZE_ADDRESS
 
 #if __LP64__ || defined(_WIN64)
 #  define SANITIZER_WORDSIZE 64
@@ -49,7 +64,9 @@ typedef __int64          int64_t;
 
 // Make the compiler thinks that something is going on there.
 inline void break_optimization(void *arg) {
+#if !defined(_WIN32) || defined(__clang__)
   __asm__ __volatile__("" : : "r" (arg) : "memory");
+#endif
 }
 
 // This function returns its parameter but in such a way that compiler
@@ -74,5 +91,28 @@ static inline uint32_t my_rand() {
   return my_rand_r(&global_seed);
 }
 
+// Set availability of platform-specific functions.
+
+#if !defined(__APPLE__) && !defined(ANDROID) && !defined(__ANDROID__) && !defined(_WIN32)
+# define SANITIZER_TEST_HAS_POSIX_MEMALIGN 1
+#else
+# define SANITIZER_TEST_HAS_POSIX_MEMALIGN 0
+#endif
+
+#if !defined(__APPLE__) && !defined(__FreeBSD__) && !defined(_WIN32)
+# define SANITIZER_TEST_HAS_MEMALIGN 1
+# define SANITIZER_TEST_HAS_PVALLOC 1
+# define SANITIZER_TEST_HAS_MALLOC_USABLE_SIZE 1
+#else
+# define SANITIZER_TEST_HAS_MEMALIGN 0
+# define SANITIZER_TEST_HAS_PVALLOC 0
+# define SANITIZER_TEST_HAS_MALLOC_USABLE_SIZE 0
+#endif
+
+#if !defined(__APPLE__)
+# define SANITIZER_TEST_HAS_STRNLEN 1
+#else
+# define SANITIZER_TEST_HAS_STRNLEN 0
+#endif
 
 #endif  // SANITIZER_TEST_UTILS_H
--- gcc/testsuite/g++.dg/asan/asan_test_config.h.jj	2012-12-03 12:44:28.983724406 +0100
+++ gcc/testsuite/g++.dg/asan/asan_test_config.h	2014-05-22 13:13:53.392991725 +0200
@@ -19,12 +19,6 @@
 #include <string>
 #include <map>
 
-#if ASAN_USE_DEJAGNU_GTEST
-# include "dejagnu-gtest.h"
-#else
-# include "gtest/gtest.h"
-#endif
-
 using std::string;
 using std::vector;
 using std::map;
@@ -42,11 +36,11 @@ using std::map;
 #endif
 
 #ifndef ASAN_NEEDS_SEGV
-# error "please define ASAN_NEEDS_SEGV"
-#endif
-
-#ifndef ASAN_LOW_MEMORY
-# define ASAN_LOW_MEMORY 0
+# if defined(_WIN32)
+#  define ASAN_NEEDS_SEGV 0
+# else
+#  define ASAN_NEEDS_SEGV 1
+# endif
 #endif
 
 #ifndef ASAN_AVOID_EXPENSIVE_TESTS
--- gcc/testsuite/g++.dg/asan/asan_str_test.cc.jj	2013-02-14 09:38:05.697773648 +0100
+++ gcc/testsuite/g++.dg/asan/asan_str_test.cc	2014-05-22 13:13:51.694002758 +0200
@@ -10,6 +10,10 @@
 //===----------------------------------------------------------------------===//
 #include "asan_test_utils.h"
 
+#if defined(__APPLE__)
+#include <AvailabilityMacros.h>  // For MAC_OS_X_VERSION_*
+#endif
+
 // Used for string functions tests
 static char global_string[] = "global";
 static size_t global_string_length = 6;
@@ -59,7 +63,19 @@ TEST(AddressSanitizer, StrLenOOBTest) {
   free(heap_string);
 }
 
-#ifndef __APPLE__
+TEST(AddressSanitizer, WcsLenTest) {
+  EXPECT_EQ(0U, wcslen(Ident(L"")));
+  size_t hello_len = 13;
+  size_t hello_size = (hello_len + 1) * sizeof(wchar_t);
+  EXPECT_EQ(hello_len, wcslen(Ident(L"Hello, World!")));
+  wchar_t *heap_string = Ident((wchar_t*)malloc(hello_size));
+  memcpy(heap_string, L"Hello, World!", hello_size);
+  EXPECT_EQ(hello_len, Ident(wcslen(heap_string)));
+  EXPECT_DEATH(Ident(wcslen(heap_string + 14)), RightOOBReadMessage(0));
+  free(heap_string);
+}
+
+#if SANITIZER_TEST_HAS_STRNLEN
 TEST(AddressSanitizer, StrNLenOOBTest) {
   size_t size = Ident(123);
   char *str = MallocAndMemsetString(size);
@@ -77,7 +93,7 @@ TEST(AddressSanitizer, StrNLenOOBTest) {
   EXPECT_DEATH(Ident(strnlen(str, size + 1)), RightOOBReadMessage(0));
   free(str);
 }
-#endif
+#endif  // SANITIZER_TEST_HAS_STRNLEN
 
 TEST(AddressSanitizer, StrDupOOBTest) {
   size_t size = Ident(42);
@@ -168,7 +184,7 @@ TEST(AddressSanitizer, StrNCpyOOBTest) {
 typedef char*(*PointerToStrChr1)(const char*, int);
 typedef char*(*PointerToStrChr2)(char*, int);
 
-USED static void RunStrChrTest(PointerToStrChr1 StrChr) {
+UNUSED static void RunStrChrTest(PointerToStrChr1 StrChr) {
   size_t size = Ident(100);
   char *str = MallocAndMemsetString(size);
   str[10] = 'q';
@@ -184,7 +200,7 @@ USED static void RunStrChrTest(PointerTo
   EXPECT_DEATH(Ident(StrChr(str, 'a')), RightOOBReadMessage(0));
   free(str);
 }
-USED static void RunStrChrTest(PointerToStrChr2 StrChr) {
+UNUSED static void RunStrChrTest(PointerToStrChr2 StrChr) {
   size_t size = Ident(100);
   char *str = MallocAndMemsetString(size);
   str[10] = 'q';
@@ -203,7 +219,9 @@ USED static void RunStrChrTest(PointerTo
 
 TEST(AddressSanitizer, StrChrAndIndexOOBTest) {
   RunStrChrTest(&strchr);
+#if !defined(_WIN32)  // no index() on Windows.
   RunStrChrTest(&index);
+#endif
 }
 
 TEST(AddressSanitizer, StrCmpAndFriendsLogicTest) {
@@ -226,6 +244,7 @@ TEST(AddressSanitizer, StrCmpAndFriendsL
   EXPECT_LT(0, strncmp("baa", "aaa", 1));
   EXPECT_LT(0, strncmp("zyx", "", 2));
 
+#if !defined(_WIN32)  // no str[n]casecmp on Windows.
   // strcasecmp
   EXPECT_EQ(0, strcasecmp("", ""));
   EXPECT_EQ(0, strcasecmp("zzz", "zzz"));
@@ -245,6 +264,7 @@ TEST(AddressSanitizer, StrCmpAndFriendsL
   EXPECT_LT(0, strncasecmp("xyz", "xyy", 10));
   EXPECT_LT(0, strncasecmp("Baa", "aaa", 1));
   EXPECT_LT(0, strncasecmp("zyx", "", 2));
+#endif
 
   // memcmp
   EXPECT_EQ(0, memcmp("a", "b", 0));
@@ -287,9 +307,11 @@ TEST(AddressSanitizer, StrCmpOOBTest) {
   RunStrCmpTest(&strcmp);
 }
 
+#if !defined(_WIN32)  // no str[n]casecmp on Windows.
 TEST(AddressSanitizer, StrCaseCmpOOBTest) {
   RunStrCmpTest(&strcasecmp);
 }
+#endif
 
 typedef int(*PointerToStrNCmp)(const char*, const char*, size_t);
 void RunStrNCmpTest(PointerToStrNCmp StrNCmp) {
@@ -322,9 +344,12 @@ TEST(AddressSanitizer, StrNCmpOOBTest) {
   RunStrNCmpTest(&strncmp);
 }
 
+#if !defined(_WIN32)  // no str[n]casecmp on Windows.
 TEST(AddressSanitizer, StrNCaseCmpOOBTest) {
   RunStrNCmpTest(&strncasecmp);
 }
+#endif
+
 TEST(AddressSanitizer, StrCatOOBTest) {
   // strcat() reads strlen(to) bytes from |to| before concatenating.
   size_t to_size = Ident(100);
@@ -506,11 +531,13 @@ void RunAtoiOOBTest(PointerToCallAtoi At
   free(array);
 }
 
+#if !defined(_WIN32)  // FIXME: Fix and enable on Windows.
 TEST(AddressSanitizer, AtoiAndFriendsOOBTest) {
   RunAtoiOOBTest(&CallAtoi);
   RunAtoiOOBTest(&CallAtol);
   RunAtoiOOBTest(&CallAtoll);
 }
+#endif
 
 void CallStrtol(const char *nptr, char **endptr, int base) {
   Ident(strtol(nptr, endptr, base));
@@ -560,11 +587,13 @@ void RunStrtolOOBTest(PointerToCallStrto
   free(array);
 }
 
+#if !defined(_WIN32)  // FIXME: Fix and enable on Windows.
 TEST(AddressSanitizer, StrtollOOBTest) {
   RunStrtolOOBTest(&CallStrtoll);
 }
 TEST(AddressSanitizer, StrtolOOBTest) {
   RunStrtolOOBTest(&CallStrtol);
 }
+#endif
 
 
--- gcc/testsuite/g++.dg/asan/asan_test_utils.h.jj	2013-02-14 09:38:05.688773689 +0100
+++ gcc/testsuite/g++.dg/asan/asan_test_utils.h	2014-05-22 13:13:55.087985295 +0200
@@ -12,24 +12,28 @@
 #ifndef ASAN_TEST_UTILS_H
 #define ASAN_TEST_UTILS_H
 
-#if !defined(ASAN_EXTERNAL_TEST_CONFIG)
+#if !defined(SANITIZER_EXTERNAL_TEST_CONFIG)
 # define INCLUDED_FROM_ASAN_TEST_UTILS_H
 # include "asan_test_config.h"
 # undef INCLUDED_FROM_ASAN_TEST_UTILS_H
 #endif
 
 #include "sanitizer_test_utils.h"
+#include "sanitizer_pthread_wrappers.h"
+
 #include <stdio.h>
 #include <signal.h>
 #include <stdlib.h>
 #include <string.h>
-#include <strings.h>
-#include <pthread.h>
 #include <stdint.h>
-#include <setjmp.h>
 #include <assert.h>
 #include <algorithm>
-#include <sys/mman.h>
+
+#if !defined(_WIN32)
+# include <strings.h>
+# include <sys/mman.h>
+# include <setjmp.h>
+#endif
 
 #ifdef __linux__
 # include <sys/prctl.h>
@@ -39,18 +43,10 @@
 #include <unistd.h>
 #endif
 
-#if defined(__i386__) || defined(__x86_64__)
-#include <emmintrin.h>
-#endif
-
-#ifndef __APPLE__
+#if !defined(__APPLE__) && !defined(__FreeBSD__)
 #include <malloc.h>
 #endif
 
-// Check that pthread_create/pthread_join return success.
-#define PTHREAD_CREATE(a, b, c, d) ASSERT_EQ(0, pthread_create(a, b, c, d))
-#define PTHREAD_JOIN(a, b) ASSERT_EQ(0, pthread_join(a, b))
-
 #if ASAN_HAS_EXCEPTIONS
 # define ASAN_THROW(x) throw (x)
 #else
--- gcc/testsuite/g++.dg/asan/asan_test.cc.jj	2013-11-12 11:31:20.000000000 +0100
+++ gcc/testsuite/g++.dg/asan/asan_test.cc	2014-05-22 13:08:39.223454867 +0200
@@ -23,27 +23,10 @@ NOINLINE void *malloc_bbb(size_t size) {
 NOINLINE void *malloc_aaa(size_t size) {
   void *res = malloc_bbb(size); break_optimization(0); return res;}
 
-#ifndef __APPLE__
-NOINLINE void *memalign_fff(size_t alignment, size_t size) {
-  void *res = memalign/**/(alignment, size); break_optimization(0); return res;}
-NOINLINE void *memalign_eee(size_t alignment, size_t size) {
-  void *res = memalign_fff(alignment, size); break_optimization(0); return res;}
-NOINLINE void *memalign_ddd(size_t alignment, size_t size) {
-  void *res = memalign_eee(alignment, size); break_optimization(0); return res;}
-NOINLINE void *memalign_ccc(size_t alignment, size_t size) {
-  void *res = memalign_ddd(alignment, size); break_optimization(0); return res;}
-NOINLINE void *memalign_bbb(size_t alignment, size_t size) {
-  void *res = memalign_ccc(alignment, size); break_optimization(0); return res;}
-NOINLINE void *memalign_aaa(size_t alignment, size_t size) {
-  void *res = memalign_bbb(alignment, size); break_optimization(0); return res;}
-#endif  // __APPLE__
-
-
 NOINLINE void free_ccc(void *p) { free(p); break_optimization(0);}
 NOINLINE void free_bbb(void *p) { free_ccc(p); break_optimization(0);}
 NOINLINE void free_aaa(void *p) { free_bbb(p); break_optimization(0);}
 
-
 template<typename T>
 NOINLINE void uaf_test(int size, int off) {
   char *p = (char *)malloc_aaa(size);
@@ -88,19 +71,19 @@ TEST(AddressSanitizer, VariousMallocsTes
   *c = 0;
   delete c;
 
-#if !defined(__APPLE__) && !defined(ANDROID) && !defined(__ANDROID__)
+#if SANITIZER_TEST_HAS_POSIX_MEMALIGN
   int *pm;
   int pm_res = posix_memalign((void**)&pm, kPageSize, kPageSize);
   EXPECT_EQ(0, pm_res);
   free(pm);
-#endif
+#endif  // SANITIZER_TEST_HAS_POSIX_MEMALIGN
 
-#if !defined(__APPLE__)
+#if SANITIZER_TEST_HAS_MEMALIGN
   int *ma = (int*)memalign(kPageSize, kPageSize);
   EXPECT_EQ(0U, (uintptr_t)ma % kPageSize);
   ma[123] = 0;
   free(ma);
-#endif  // __APPLE__
+#endif  // SANITIZER_TEST_HAS_MEMALIGN
 }
 
 TEST(AddressSanitizer, CallocTest) {
@@ -109,13 +92,37 @@ TEST(AddressSanitizer, CallocTest) {
   free(a);
 }
 
+TEST(AddressSanitizer, CallocReturnsZeroMem) {
+  size_t sizes[] = {16, 1000, 10000, 100000, 2100000};
+  for (size_t s = 0; s < sizeof(sizes)/sizeof(sizes[0]); s++) {
+    size_t size = sizes[s];
+    for (size_t iter = 0; iter < 5; iter++) {
+      char *x = Ident((char*)calloc(1, size));
+      EXPECT_EQ(x[0], 0);
+      EXPECT_EQ(x[size - 1], 0);
+      EXPECT_EQ(x[size / 2], 0);
+      EXPECT_EQ(x[size / 3], 0);
+      EXPECT_EQ(x[size / 4], 0);
+      memset(x, 0x42, size);
+      free(Ident(x));
+#if !defined(_WIN32)
+      // FIXME: OOM on Windows. We should just make this a lit test
+      // with quarantine size set to 1.
+      free(Ident(malloc(Ident(1 << 27))));  // Try to drain the quarantine.
+#endif
+    }
+  }
+}
+
+#if !defined(_WIN32)  // No valloc on Windows.
 TEST(AddressSanitizer, VallocTest) {
   void *a = valloc(100);
   EXPECT_EQ(0U, (uintptr_t)a % kPageSize);
   free(a);
 }
+#endif
 
-#ifndef __APPLE__
+#if SANITIZER_TEST_HAS_PVALLOC
 TEST(AddressSanitizer, PvallocTest) {
   char *a = (char*)pvalloc(kPageSize + 100);
   EXPECT_EQ(0U, (uintptr_t)a % kPageSize);
@@ -127,8 +134,10 @@ TEST(AddressSanitizer, PvallocTest) {
   a[101] = 1;  // we should not report an error here.
   free(a);
 }
-#endif  // __APPLE__
+#endif  // SANITIZER_TEST_HAS_PVALLOC
 
+#if !defined(_WIN32)
+// FIXME: Use an equivalent of pthread_setspecific on Windows.
 void *TSDWorker(void *test_key) {
   if (test_key) {
     pthread_setspecific(*(pthread_key_t*)test_key, (void*)0xfeedface);
@@ -158,6 +167,7 @@ TEST(AddressSanitizer, DISABLED_TSDTest)
   PTHREAD_JOIN(th, NULL);
   pthread_key_delete(test_key);
 }
+#endif
 
 TEST(AddressSanitizer, UAF_char) {
   const char *uaf_string = "AddressSanitizer:.*heap-use-after-free";
@@ -168,6 +178,36 @@ TEST(AddressSanitizer, UAF_char) {
   EXPECT_DEATH(uaf_test<U1>(kLargeMalloc, kLargeMalloc / 2), uaf_string);
 }
 
+TEST(AddressSanitizer, UAF_long_double) {
+  if (sizeof(long double) == sizeof(double)) return;
+  long double *p = Ident(new long double[10]);
+  EXPECT_DEATH(Ident(p)[12] = 0, "WRITE of size 1[06]");
+  EXPECT_DEATH(Ident(p)[0] = Ident(p)[12], "READ of size 1[06]");
+  delete [] Ident(p);
+}
+
+#if !defined(_WIN32)
+struct Packed5 {
+  int x;
+  char c;
+} __attribute__((packed));
+#else
+# pragma pack(push, 1)
+struct Packed5 {
+  int x;
+  char c;
+};
+# pragma pack(pop)
+#endif
+
+TEST(AddressSanitizer, UAF_Packed5) {
+  static_assert(sizeof(Packed5) == 5, "Please check the keywords used");
+  Packed5 *p = Ident(new Packed5[2]);
+  EXPECT_DEATH(p[0] = p[3], "READ of size 5");
+  EXPECT_DEATH(p[3] = p[0], "WRITE of size 5");
+  delete [] Ident(p);
+}
+
 #if ASAN_HAS_BLACKLIST
 TEST(AddressSanitizer, IgnoreTest) {
   int *x = Ident(new int);
@@ -244,41 +284,6 @@ TEST(AddressSanitizer, SignalTest) {
 }  // namespace
 #endif
 
-static void MallocStress(size_t n) {
-  uint32_t seed = my_rand();
-  for (size_t iter = 0; iter < 10; iter++) {
-    vector<void *> vec;
-    for (size_t i = 0; i < n; i++) {
-      if ((i % 3) == 0) {
-        if (vec.empty()) continue;
-        size_t idx = my_rand_r(&seed) % vec.size();
-        void *ptr = vec[idx];
-        vec[idx] = vec.back();
-        vec.pop_back();
-        free_aaa(ptr);
-      } else {
-        size_t size = my_rand_r(&seed) % 1000 + 1;
-#ifndef __APPLE__
-        size_t alignment = 1 << (my_rand_r(&seed) % 7 + 3);
-        char *ptr = (char*)memalign_aaa(alignment, size);
-#else
-        char *ptr = (char*) malloc_aaa(size);
-#endif
-        vec.push_back(ptr);
-        ptr[0] = 0;
-        ptr[size-1] = 0;
-        ptr[size/2] = 0;
-      }
-    }
-    for (size_t i = 0; i < vec.size(); i++)
-      free_aaa(vec[i]);
-  }
-}
-
-TEST(AddressSanitizer, MallocStressTest) {
-  MallocStress((ASAN_LOW_MEMORY) ? 20000 : 200000);
-}
-
 static void TestLargeMalloc(size_t size) {
   char buff[1024];
   sprintf(buff, "is located 1 bytes to the left of %lu-byte", (long)size);
@@ -286,28 +291,21 @@ static void TestLargeMalloc(size_t size)
 }
 
 TEST(AddressSanitizer, LargeMallocTest) {
-  for (int i = 113; i < (1 << 28); i = i * 2 + 13) {
+  const int max_size = (SANITIZER_WORDSIZE == 32) ? 1 << 26 : 1 << 28;
+  for (int i = 113; i < max_size; i = i * 2 + 13) {
     TestLargeMalloc(i);
   }
 }
 
-#if ASAN_LOW_MEMORY != 1
 TEST(AddressSanitizer, HugeMallocTest) {
-#ifdef __APPLE__
-  // It was empirically found out that 1215 megabytes is the maximum amount of
-  // memory available to the process under AddressSanitizer on 32-bit Mac 10.6.
-  // 32-bit Mac 10.7 gives even less (< 1G).
-  // (the libSystem malloc() allows allocating up to 2300 megabytes without
-  // ASan).
-  size_t n_megs = SANITIZER_WORDSIZE == 32 ? 500 : 4100;
-#else
-  size_t n_megs = SANITIZER_WORDSIZE == 32 ? 2600 : 4100;
-#endif
-  TestLargeMalloc(n_megs << 20);
+  if (SANITIZER_WORDSIZE != 64 || ASAN_AVOID_EXPENSIVE_TESTS) return;
+  size_t n_megs = 4100;
+  EXPECT_DEATH(Ident((char*)malloc(n_megs << 20))[-1] = 0,
+               "is located 1 bytes to the left|"
+               "AddressSanitizer failed to allocate");
 }
-#endif
 
-#ifndef __APPLE__
+#if SANITIZER_TEST_HAS_MEMALIGN
 void MemalignRun(size_t align, size_t size, int idx) {
   char *p = (char *)memalign(align, size);
   Ident(p)[idx] = 0;
@@ -323,20 +321,7 @@ TEST(AddressSanitizer, memalign) {
                  "is located 1 bytes to the right");
   }
 }
-#endif
-
-TEST(AddressSanitizer, ThreadedMallocStressTest) {
-  const int kNumThreads = 4;
-  const int kNumIterations = (ASAN_LOW_MEMORY) ? 10000 : 100000;
-  pthread_t t[kNumThreads];
-  for (int i = 0; i < kNumThreads; i++) {
-    PTHREAD_CREATE(&t[i], 0, (void* (*)(void *x))MallocStress,
-        (void*)kNumIterations);
-  }
-  for (int i = 0; i < kNumThreads; i++) {
-    PTHREAD_JOIN(t[i], 0);
-  }
-}
+#endif  // SANITIZER_TEST_HAS_MEMALIGN
 
 void *ManyThreadsWorker(void *a) {
   for (int iter = 0; iter < 100; iter++) {
@@ -377,17 +362,30 @@ TEST(AddressSanitizer, ReallocTest) {
   free(ptr2);
 }
 
+TEST(AddressSanitizer, ReallocFreedPointerTest) {
+  void *ptr = Ident(malloc(42));
+  ASSERT_TRUE(NULL != ptr);
+  free(ptr);
+  EXPECT_DEATH(ptr = realloc(ptr, 77), "attempting double-free");
+}
+
+TEST(AddressSanitizer, ReallocInvalidPointerTest) {
+  void *ptr = Ident(malloc(42));
+  EXPECT_DEATH(ptr = realloc((int*)ptr + 1, 77), "attempting free.*not malloc");
+  free(ptr);
+}
+
 TEST(AddressSanitizer, ZeroSizeMallocTest) {
   // Test that malloc(0) and similar functions don't return NULL.
   void *ptr = Ident(malloc(0));
   EXPECT_TRUE(NULL != ptr);
   free(ptr);
-#if !defined(__APPLE__) && !defined(ANDROID) && !defined(__ANDROID__)
+#if SANITIZER_TEST_HAS_POSIX_MEMALIGN
   int pm_res = posix_memalign(&ptr, 1<<20, 0);
   EXPECT_EQ(0, pm_res);
   EXPECT_TRUE(NULL != ptr);
   free(ptr);
-#endif
+#endif  // SANITIZER_TEST_HAS_POSIX_MEMALIGN
   int *int_ptr = new int[0];
   int *int_ptr2 = new int[0];
   EXPECT_TRUE(NULL != int_ptr);
@@ -397,7 +395,7 @@ TEST(AddressSanitizer, ZeroSizeMallocTes
   delete[] int_ptr2;
 }
 
-#ifndef __APPLE__
+#if SANITIZER_TEST_HAS_MALLOC_USABLE_SIZE
 static const char *kMallocUsableSizeErrorMsg =
   "AddressSanitizer: attempting to call malloc_usable_size()";
 
@@ -413,8 +411,9 @@ TEST(AddressSanitizer, MallocUsableSizeT
                kMallocUsableSizeErrorMsg);
   free(array);
   EXPECT_DEATH(malloc_usable_size(array), kMallocUsableSizeErrorMsg);
+  delete int_ptr;
 }
-#endif
+#endif  // SANITIZER_TEST_HAS_MALLOC_USABLE_SIZE
 
 void WrongFree() {
   int *x = (int*)malloc(100 * sizeof(int));
@@ -423,10 +422,14 @@ void WrongFree() {
   free(x + 1);
 }
 
+#if !defined(_WIN32)  // FIXME: This should be a lit test.
 TEST(AddressSanitizer, WrongFreeTest) {
-  EXPECT_DEATH(WrongFree(),
-               "ERROR: AddressSanitizer: attempting free.*not malloc");
+  EXPECT_DEATH(WrongFree(), ASAN_PCRE_DOTALL
+               "ERROR: AddressSanitizer: attempting free.*not malloc"
+               ".*is located 4 bytes inside of 400-byte region"
+               ".*allocated by thread");
 }
+#endif
 
 void DoubleFree() {
   int *x = (int*)malloc(100 * sizeof(int));
@@ -437,6 +440,7 @@ void DoubleFree() {
   abort();
 }
 
+#if !defined(_WIN32)  // FIXME: This should be a lit test.
 TEST(AddressSanitizer, DoubleFreeTest) {
   EXPECT_DEATH(DoubleFree(), ASAN_PCRE_DOTALL
                "ERROR: AddressSanitizer: attempting double-free"
@@ -444,20 +448,22 @@ TEST(AddressSanitizer, DoubleFreeTest) {
                ".*freed by thread T0 here"
                ".*previously allocated by thread T0 here");
 }
+#endif
 
 template<int kSize>
 NOINLINE void SizedStackTest() {
   char a[kSize];
   char  *A = Ident((char*)&a);
+  const char *expected_death = "AddressSanitizer: stack-buffer-";
   for (size_t i = 0; i < kSize; i++)
     A[i] = i;
-  EXPECT_DEATH(A[-1] = 0, "");
-  EXPECT_DEATH(A[-20] = 0, "");
-  EXPECT_DEATH(A[-31] = 0, "");
-  EXPECT_DEATH(A[kSize] = 0, "");
-  EXPECT_DEATH(A[kSize + 1] = 0, "");
-  EXPECT_DEATH(A[kSize + 10] = 0, "");
-  EXPECT_DEATH(A[kSize + 31] = 0, "");
+  EXPECT_DEATH(A[-1] = 0, expected_death);
+  EXPECT_DEATH(A[-5] = 0, expected_death);
+  EXPECT_DEATH(A[kSize] = 0, expected_death);
+  EXPECT_DEATH(A[kSize + 1] = 0, expected_death);
+  EXPECT_DEATH(A[kSize + 5] = 0, expected_death);
+  if (kSize > 16)
+    EXPECT_DEATH(A[kSize + 31] = 0, expected_death);
 }
 
 TEST(AddressSanitizer, SimpleStackTest) {
@@ -478,6 +484,9 @@ TEST(AddressSanitizer, SimpleStackTest)
   SizedStackTest<128>();
 }
 
+#if !defined(_WIN32)
+// FIXME: It's a bit hard to write multi-line death test expectations
+// in a portable way.  Anyways, this should just be turned into a lit test.
 TEST(AddressSanitizer, ManyStackObjectsTest) {
   char XXX[10];
   char YYY[20];
@@ -486,6 +495,47 @@ TEST(AddressSanitizer, ManyStackObjectsT
   Ident(YYY);
   EXPECT_DEATH(Ident(ZZZ)[-1] = 0, ASAN_PCRE_DOTALL "XXX.*YYY.*ZZZ");
 }
+#endif
+
+#if 0  // This test requires online symbolizer.
+// Moved to lit_tests/stack-oob-frames.cc.
+// Reenable here once we have online symbolizer by default.
+NOINLINE static void Frame0(int frame, char *a, char *b, char *c) {
+  char d[4] = {0};
+  char *D = Ident(d);
+  switch (frame) {
+    case 3: a[5]++; break;
+    case 2: b[5]++; break;
+    case 1: c[5]++; break;
+    case 0: D[5]++; break;
+  }
+}
+NOINLINE static void Frame1(int frame, char *a, char *b) {
+  char c[4] = {0}; Frame0(frame, a, b, c);
+  break_optimization(0);
+}
+NOINLINE static void Frame2(int frame, char *a) {
+  char b[4] = {0}; Frame1(frame, a, b);
+  break_optimization(0);
+}
+NOINLINE static void Frame3(int frame) {
+  char a[4] = {0}; Frame2(frame, a);
+  break_optimization(0);
+}
+
+TEST(AddressSanitizer, GuiltyStackFrame0Test) {
+  EXPECT_DEATH(Frame3(0), "located .*in frame <.*Frame0");
+}
+TEST(AddressSanitizer, GuiltyStackFrame1Test) {
+  EXPECT_DEATH(Frame3(1), "located .*in frame <.*Frame1");
+}
+TEST(AddressSanitizer, GuiltyStackFrame2Test) {
+  EXPECT_DEATH(Frame3(2), "located .*in frame <.*Frame2");
+}
+TEST(AddressSanitizer, GuiltyStackFrame3Test) {
+  EXPECT_DEATH(Frame3(3), "located .*in frame <.*Frame3");
+}
+#endif
 
 NOINLINE void LongJmpFunc1(jmp_buf buf) {
   // create three red zones for these two stack objects.
@@ -498,6 +548,24 @@ NOINLINE void LongJmpFunc1(jmp_buf buf)
   longjmp(buf, 1);
 }
 
+NOINLINE void TouchStackFunc() {
+  int a[100];  // long array will intersect with redzones from LongJmpFunc1.
+  int *A = Ident(a);
+  for (int i = 0; i < 100; i++)
+    A[i] = i*i;
+}
+
+// Test that we handle longjmp and do not report false positives on stack.
+TEST(AddressSanitizer, LongJmpTest) {
+  static jmp_buf buf;
+  if (!setjmp(buf)) {
+    LongJmpFunc1(buf);
+  } else {
+    TouchStackFunc();
+  }
+}
+
+#if !defined(_WIN32)  // Only basic longjmp is available on Windows.
 NOINLINE void BuiltinLongJmpFunc1(jmp_buf buf) {
   // create three red zones for these two stack objects.
   int a;
@@ -531,25 +599,10 @@ NOINLINE void SigLongJmpFunc1(sigjmp_buf
   siglongjmp(buf, 1);
 }
 
-
-NOINLINE void TouchStackFunc() {
-  int a[100];  // long array will intersect with redzones from LongJmpFunc1.
-  int *A = Ident(a);
-  for (int i = 0; i < 100; i++)
-    A[i] = i*i;
-}
-
-// Test that we handle longjmp and do not report fals positives on stack.
-TEST(AddressSanitizer, LongJmpTest) {
-  static jmp_buf buf;
-  if (!setjmp(buf)) {
-    LongJmpFunc1(buf);
-  } else {
-    TouchStackFunc();
-  }
-}
-
-#if not defined(__ANDROID__)
+#if !defined(__ANDROID__) && \
+    !defined(__powerpc64__) && !defined(__powerpc__)
+// Does not work on Power:
+// https://code.google.com/p/address-sanitizer/issues/detail?id=185
 TEST(AddressSanitizer, BuiltinLongJmpTest) {
   static jmp_buf buf;
   if (!__builtin_setjmp((void**)buf)) {
@@ -558,7 +611,8 @@ TEST(AddressSanitizer, BuiltinLongJmpTes
     TouchStackFunc();
   }
 }
-#endif  // not defined(__ANDROID__)
+#endif  // !defined(__ANDROID__) && !defined(__powerpc64__) &&
+        // !defined(__powerpc__)
 
 TEST(AddressSanitizer, UnderscopeLongJmpTest) {
   static jmp_buf buf;
@@ -577,8 +631,10 @@ TEST(AddressSanitizer, SigLongJmpTest) {
     TouchStackFunc();
   }
 }
+#endif
 
-#ifdef __EXCEPTIONS
+// FIXME: Why does clang-cl define __EXCEPTIONS?
+#if defined(__EXCEPTIONS) && !defined(_WIN32)
 NOINLINE void ThrowFunc() {
   // create three red zones for these two stack objects.
   int a;
@@ -626,7 +682,8 @@ TEST(AddressSanitizer, ThreadStackReuseT
   PTHREAD_JOIN(t, 0);
 }
 
-#if defined(__i386__) || defined(__x86_64__)
+#if defined(__i686__) || defined(__x86_64__)
+#include <emmintrin.h>
 TEST(AddressSanitizer, Store128Test) {
   char *a = Ident((char*)malloc(Ident(12)));
   char *p = a;
@@ -644,11 +701,19 @@ TEST(AddressSanitizer, Store128Test) {
 }
 #endif
 
+// FIXME: All tests that use this function should be turned into lit tests.
 string RightOOBErrorMessage(int oob_distance, bool is_write) {
   assert(oob_distance >= 0);
   char expected_str[100];
-  sprintf(expected_str, ASAN_PCRE_DOTALL "%s.*located %d bytes to the right",
-          is_write ? "WRITE" : "READ", oob_distance);
+  sprintf(expected_str, ASAN_PCRE_DOTALL
+#if !GTEST_USES_SIMPLE_RE
+          "buffer-overflow.*%s.*"
+#endif
+          "located %d bytes to the right",
+#if !GTEST_USES_SIMPLE_RE
+          is_write ? "WRITE" : "READ",
+#endif
+          oob_distance);
   return string(expected_str);
 }
 
@@ -660,11 +725,19 @@ string RightOOBReadMessage(int oob_dista
   return RightOOBErrorMessage(oob_distance, /*is_write*/false);
 }
 
+// FIXME: All tests that use this function should be turned into lit tests.
 string LeftOOBErrorMessage(int oob_distance, bool is_write) {
   assert(oob_distance > 0);
   char expected_str[100];
-  sprintf(expected_str, ASAN_PCRE_DOTALL "%s.*located %d bytes to the left",
-          is_write ? "WRITE" : "READ", oob_distance);
+  sprintf(expected_str,
+#if !GTEST_USES_SIMPLE_RE
+          ASAN_PCRE_DOTALL "%s.*"
+#endif
+          "located %d bytes to the left",
+#if !GTEST_USES_SIMPLE_RE
+          is_write ? "WRITE" : "READ",
+#endif
+          oob_distance);
   return string(expected_str);
 }
 
@@ -818,6 +891,7 @@ void ThreadedTestSpawn() {
   PTHREAD_JOIN(t, 0);
 }
 
+#if !defined(_WIN32)  // FIXME: This should be a lit test.
 TEST(AddressSanitizer, ThreadedTest) {
   EXPECT_DEATH(ThreadedTestSpawn(),
                ASAN_PCRE_DOTALL
@@ -825,6 +899,7 @@ TEST(AddressSanitizer, ThreadedTest) {
                ".*Thread T.*created"
                ".*Thread T.*created");
 }
+#endif
 
 void *ThreadedTestFunc(void *unused) {
   // Check if prctl(PR_SET_NAME) is supported. Return if not.
@@ -855,7 +930,11 @@ TEST(AddressSanitizer, ShadowGapTest) {
 #if SANITIZER_WORDSIZE == 32
   char *addr = (char*)0x22000000;
 #else
+# if defined(__powerpc64__)
+  char *addr = (char*)0x024000800000;
+# else
   char *addr = (char*)0x0000100000080000;
+# endif
 #endif
   EXPECT_DEATH(*addr = 1, "AddressSanitizer: SEGV on unknown");
 }
@@ -1015,7 +1094,8 @@ TEST(AddressSanitizer, PthreadExitTest)
   }
 }
 
-#ifdef __EXCEPTIONS
+// FIXME: Why does clang-cl define __EXCEPTIONS?
+#if defined(__EXCEPTIONS) && !defined(_WIN32)
 NOINLINE static void StackReuseAndException() {
   int large_stack[1000];
   Ident(large_stack);
@@ -1033,12 +1113,14 @@ TEST(AddressSanitizer, DISABLED_StressSt
 }
 #endif
 
+#if !defined(_WIN32)
 TEST(AddressSanitizer, MlockTest) {
   EXPECT_EQ(0, mlockall(MCL_CURRENT));
   EXPECT_EQ(0, mlock((void*)0x12345, 0x5678));
   EXPECT_EQ(0, munlockall());
   EXPECT_EQ(0, munlock((void*)0x987, 0x654));
 }
+#endif
 
 struct LargeStruct {
   int foo[100];
@@ -1051,19 +1133,26 @@ TEST(AddressSanitizer, LargeStructCopyTe
   *Ident(&a) = *Ident(&a);
 }
 
-ATTRIBUTE_NO_ADDRESS_SAFETY_ANALYSIS
-static void NoAddressSafety() {
+ATTRIBUTE_NO_SANITIZE_ADDRESS
+static void NoSanitizeAddress() {
   char *foo = new char[10];
   Ident(foo)[10] = 0;
   delete [] foo;
 }
 
-TEST(AddressSanitizer, AttributeNoAddressSafetyTest) {
-  Ident(NoAddressSafety)();
+TEST(AddressSanitizer, AttributeNoSanitizeAddressTest) {
+  Ident(NoSanitizeAddress)();
 }
 
-// It doesn't work on Android, as calls to new/delete go through malloc/free.
-#if !defined(ANDROID) && !defined(__ANDROID__)
+// The new/delete/etc mismatch checks don't work on Android,
+//   as calls to new/delete go through malloc/free.
+// OS X support is tracked here:
+//   https://code.google.com/p/address-sanitizer/issues/detail?id=131
+// Windows support is tracked here:
+//   https://code.google.com/p/address-sanitizer/issues/detail?id=309
+#if !defined(ANDROID) && !defined(__ANDROID__) && \
+    !defined(__APPLE__) && \
+    !defined(_WIN32)
 static string MismatchStr(const string &str) {
   return string("AddressSanitizer: alloc-dealloc-mismatch \\(") + str;
 }
@@ -1177,3 +1266,18 @@ TEST(AddressSanitizer, LongDoubleNegativ
   memcpy(Ident(&a), Ident(&b), sizeof(long double));
   memcpy(Ident(&c), Ident(&b), sizeof(long double));
 }
+
+#if !defined(_WIN32)
+TEST(AddressSanitizer, pthread_getschedparam) {
+  int policy;
+  struct sched_param param;
+  EXPECT_DEATH(
+      pthread_getschedparam(pthread_self(), &policy, Ident(&param) + 2),
+      "AddressSanitizer: stack-buffer-.*flow");
+  EXPECT_DEATH(
+      pthread_getschedparam(pthread_self(), Ident(&policy) - 1, &param),
+      "AddressSanitizer: stack-buffer-.*flow");
+  int res = pthread_getschedparam(pthread_self(), &policy, &param);
+  ASSERT_EQ(0, res);
+}
+#endif
--- gcc/testsuite/g++.dg/asan/sanitizer_pthread_wrappers.h.jj	2014-05-22 13:20:42.496012107 +0200
+++ gcc/testsuite/g++.dg/asan/sanitizer_pthread_wrappers.h	2014-05-22 13:20:45.523996247 +0200
@@ -0,0 +1,64 @@
+//===-- sanitizer_pthread_wrappers.h ----------------------------*- C++ -*-===//
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file is a part of *Sanitizer runtime.
+// It provides handy wrappers for thread manipulation, that:
+//  a) assert on any failure rather than returning an error code
+//  b) defines pthread-like interface on platforms where where <pthread.h>
+//     is not supplied by default.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef SANITIZER_PTHREAD_WRAPPERS_H
+#define SANITIZER_PTHREAD_WRAPPERS_H
+
+#include "sanitizer_test_utils.h"
+
+#if !defined(_WIN32)
+# include <pthread.h>
+// Simply forward the arguments and check that the pthread functions succeed.
+# define PTHREAD_CREATE(a, b, c, d) ASSERT_EQ(0, pthread_create(a, b, c, d))
+# define PTHREAD_JOIN(a, b) ASSERT_EQ(0, pthread_join(a, b))
+#else
+typedef HANDLE pthread_t;
+
+struct PthreadHelperCreateThreadInfo {
+  void *(*start_routine)(void *);
+  void *arg;
+};
+
+inline DWORD WINAPI PthreadHelperThreadProc(void *arg) {
+  PthreadHelperCreateThreadInfo *start_data =
+      reinterpret_cast<PthreadHelperCreateThreadInfo*>(arg);
+  void *ret = (start_data->start_routine)(start_data->arg);
+  delete start_data;
+  return (DWORD)ret;
+}
+
+inline void PTHREAD_CREATE(pthread_t *thread, void *attr,
+                           void *(*start_routine)(void *), void *arg) {
+  ASSERT_EQ(0, attr) << "Thread attributes are not supported yet.";
+  PthreadHelperCreateThreadInfo *data = new PthreadHelperCreateThreadInfo;
+  data->start_routine = start_routine;
+  data->arg = arg;
+  *thread = CreateThread(0, 0, PthreadHelperThreadProc, data, 0, 0);
+  ASSERT_NE(nullptr, *thread) << "Failed to create a thread.";
+}
+
+inline void PTHREAD_JOIN(pthread_t thread, void **value_ptr) {
+  ASSERT_EQ(0, value_ptr) << "Nonzero value_ptr is not supported yet.";
+  ASSERT_EQ(WAIT_OBJECT_0, WaitForSingleObject(thread, INFINITE));
+  ASSERT_NE(0, CloseHandle(thread));
+}
+
+inline void pthread_exit(void *retval) {
+  ASSERT_EQ(0, retval) << "Nonzero retval is not supported yet.";
+  ExitThread((DWORD)retval);
+}
+#endif  // _WIN32
+
+#endif  // SANITIZER_PTHREAD_WRAPPERS_H
--- gcc/testsuite/g++.dg/asan/sanitizer_test_config.h.jj	2014-05-22 13:11:24.552661120 +0200
+++ gcc/testsuite/g++.dg/asan/sanitizer_test_config.h	2014-05-22 13:11:28.153650267 +0200
@@ -0,0 +1,28 @@
+//===-- sanitizer_test_config.h ---------------------------------*- C++ -*-===//
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file is a part of *Sanitizer runtime.
+//
+//===----------------------------------------------------------------------===//
+#if !defined(INCLUDED_FROM_SANITIZER_TEST_UTILS_H)
+# error "This file should be included into sanitizer_test_utils.h only"
+#endif
+
+#ifndef SANITIZER_TEST_CONFIG_H
+#define SANITIZER_TEST_CONFIG_H
+
+#include <vector>
+#include <string>
+#include <map>
+
+#if SANITIZER_USE_DEJAGNU_GTEST
+# include "dejagnu-gtest.h"
+#else
+# include "gtest/gtest.h"
+#endif
+
+#endif  // SANITIZER_TEST_CONFIG_H


	Jakub

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

On Thu, May 22, 2014 at 3:43 PM, Jakub Jelinek <jakub@redhat.com> wrote:
> On Thu, May 22, 2014 at 01:03:48PM +0200, Jakub Jelinek wrote:
>> There are various other changes to asan_test.cc, so guess some work is needed on that.
>
> Ok, tried to merge also g++.dg/asan from the same revision as you've merged
> libsanitizer.
Oh, cool.

>
> On x86_64-linux, both -m32 and -m64 I see
> FAIL: g++.dg/asan/asan_test.C  -O2  AddressSanitizer_UAF_Packed5 p[0] = p[3] execution test
> FAIL: g++.dg/asan/asan_test.C  -O2  AddressSanitizer_UAF_Packed5 p[3] = p[0] execution test
> and with -m32 additionally:
> FAIL: g++.dg/asan/asan_test.C  -O2  AddressSanitizer_UAF_long_double Ident(p)[12] = 0 execution test
> FAIL: g++.dg/asan/asan_test.C  -O2  AddressSanitizer_UAF_long_double Ident(p)[0] = Ident(p)[12] execution test
> The former boils down into:
> struct Packed5 {
>   int x;
>   char c;
> } __attribute__((packed));
>
> int
> main ()
> {
>   Packed5 *p = new Packed5[2];
>   asm volatile ("" : "+r" (p));
>   p[0] = p[3];
>   p[3] = p[0];
>   return 0;
> }
>
> Right now we instrument only accesses which have sizes power of two
> <= 16:
>
>   size_in_bytes = int_size_in_bytes (type);
>   if ((size_in_bytes & (size_in_bytes - 1)) != 0
>       || (unsigned HOST_WIDE_INT) size_in_bytes - 1 >= 16)
>     return;
>
> I thought so did LLVM, has that changed recently?

Not really recently... (Feb 2013)
http://llvm.org/viewvc/llvm-project?rev=175507&view=rev

--kcc

>
> 2014-05-22  Jakub Jelinek  <jakub@redhat.com>
>
>         * g++.dg/asan/asan_test.C: Add -std=c++11 and
>         -DSANITIZER_USE_DEJAGNU_GTEST=1 to dg-options, remove
>         -DASAN_USE_DEJAGNU_GTEST=1.
>         * g++.dg/asan/asan_mem_test.cc: Updated from upstream
>         r209283.
>         * g++.dg/asan/asan_oob_test.cc: Likewise.
>         * g++.dg/asan/sanitizer_test_utils.h: Likewise.
>         * g++.dg/asan/asan_str_test.cc: Likewise.
>         * g++.dg/asan/asan_test_utils.h: Likewise.
>         * g++.dg/asan/sanitizer_test_config.h: Likewise.
>         * g++.dg/asan/asan_test.cc: Likewise.
>         * g++.dg/asan/sanitizer_pthread_wrappers.h: New file.
>         Imported from upstream r209283.
>         * g++.dg/asan/asan_test_config.h: Likewise.
>
> --- gcc/testsuite/g++.dg/asan/asan_test.C.jj    2013-02-14 09:38:05.000000000 +0100
> +++ gcc/testsuite/g++.dg/asan/asan_test.C       2014-05-22 13:22:33.747459933 +0200
> @@ -2,7 +2,7 @@
>  // { dg-skip-if "" { *-*-* } { "*" } { "-O2" } }
>  // { dg-skip-if "" { *-*-* } { "-flto" } { "" } }
>  // { dg-additional-sources "asan_globals_test-wrapper.cc" }
> -// { dg-options "-fsanitize=address -fno-builtin -Wall -Wno-format -Werror -g -DASAN_UAR=0 -DASAN_HAS_EXCEPTIONS=1 -DASAN_HAS_BLACKLIST=0 -DASAN_USE_DEJAGNU_GTEST=1 -lasan -lpthread -ldl" }
> +// { dg-options "-std=c++11 -fsanitize=address -fno-builtin -Wall -Wno-format -Werror -g -DASAN_UAR=0 -DASAN_HAS_EXCEPTIONS=1 -DASAN_HAS_BLACKLIST=0 -DSANITIZER_USE_DEJAGNU_GTEST=1 -lasan -lpthread -ldl" }
>  // { dg-additional-options "-DASAN_NEEDS_SEGV=1" { target { ! arm*-*-* } } }
>  // { dg-additional-options "-DASAN_LOW_MEMORY=1 -DASAN_NEEDS_SEGV=0" { target arm*-*-* } }
>  // { dg-additional-options "-DASAN_AVOID_EXPENSIVE_TESTS=1" { target { ! run_expensive_tests } } }
> --- gcc/testsuite/g++.dg/asan/asan_mem_test.cc.jj       2013-02-14 09:38:05.697773648 +0100
> +++ gcc/testsuite/g++.dg/asan/asan_mem_test.cc  2014-05-22 13:13:48.158014955 +0200
> @@ -74,17 +74,17 @@ TEST(AddressSanitizer, MemSetOOBTest) {
>  // Strictly speaking we are not guaranteed to find such two pointers,
>  // but given the structure of asan's allocator we will.
>  static bool AllocateTwoAdjacentArrays(char **x1, char **x2, size_t size) {
> -  vector<char *> v;
> +  vector<uintptr_t> v;
>    bool res = false;
>    for (size_t i = 0; i < 1000U && !res; i++) {
> -    v.push_back(new char[size]);
> +    v.push_back(reinterpret_cast<uintptr_t>(new char[size]));
>      if (i == 0) continue;
>      sort(v.begin(), v.end());
>      for (size_t j = 1; j < v.size(); j++) {
>        assert(v[j] > v[j-1]);
>        if ((size_t)(v[j] - v[j-1]) < size * 2) {
> -        *x2 = v[j];
> -        *x1 = v[j-1];
> +        *x2 = reinterpret_cast<char*>(v[j]);
> +        *x1 = reinterpret_cast<char*>(v[j-1]);
>          res = true;
>          break;
>        }
> @@ -92,9 +92,10 @@ static bool AllocateTwoAdjacentArrays(ch
>    }
>
>    for (size_t i = 0; i < v.size(); i++) {
> -    if (res && v[i] == *x1) continue;
> -    if (res && v[i] == *x2) continue;
> -    delete [] v[i];
> +    char *p = reinterpret_cast<char *>(v[i]);
> +    if (res && p == *x1) continue;
> +    if (res && p == *x2) continue;
> +    delete [] p;
>    }
>    return res;
>  }
> @@ -223,6 +224,13 @@ TEST(AddressSanitizer, MemCmpOOBTest) {
>    s1[size - 1] = '\0';
>    s2[size - 1] = '\0';
>    EXPECT_DEATH(Ident(memcmp)(s1, s2, size + 1), RightOOBReadMessage(0));
> +
> +  // Even if the buffers differ in the first byte, we still assume that
> +  // memcmp may access the whole buffer and thus reporting the overflow here:
> +  s1[0] = 1;
> +  s2[0] = 123;
> +  EXPECT_DEATH(Ident(memcmp)(s1, s2, size + 1), RightOOBReadMessage(0));
> +
>    free(s1);
>    free(s2);
>  }
> --- gcc/testsuite/g++.dg/asan/asan_oob_test.cc.jj       2013-02-14 09:38:05.697773648 +0100
> +++ gcc/testsuite/g++.dg/asan/asan_oob_test.cc  2014-05-22 13:13:49.976009670 +0200
> @@ -97,7 +97,6 @@ TEST(AddressSanitizer, OOBRightTest) {
>    }
>  }
>
> -#if ASAN_ALLOCATOR_VERSION == 2  // Broken with the asan_allocator1
>  TEST(AddressSanitizer, LargeOOBRightTest) {
>    size_t large_power_of_two = 1 << 19;
>    for (size_t i = 16; i <= 256; i *= 2) {
> @@ -107,7 +106,6 @@ TEST(AddressSanitizer, LargeOOBRightTest
>      delete [] p;
>    }
>  }
> -#endif  // ASAN_ALLOCATOR_VERSION == 2
>
>  TEST(AddressSanitizer, DISABLED_DemoOOBLeftLow) {
>    oob_test<U1>(10, -1);
> --- gcc/testsuite/g++.dg/asan/sanitizer_test_utils.h.jj 2013-02-28 22:22:03.000000000 +0100
> +++ gcc/testsuite/g++.dg/asan/sanitizer_test_utils.h    2014-05-22 13:10:27.781920740 +0200
> @@ -14,32 +14,47 @@
>  #define SANITIZER_TEST_UTILS_H
>
>  #if defined(_WIN32)
> -typedef unsigned __int8  uint8_t;
> -typedef unsigned __int16 uint16_t;
> -typedef unsigned __int32 uint32_t;
> -typedef unsigned __int64 uint64_t;
> -typedef __int8           int8_t;
> -typedef __int16          int16_t;
> -typedef __int32          int32_t;
> -typedef __int64          int64_t;
> +// <windows.h> should always be the first include on Windows.
> +# include <windows.h>
> +// MSVS headers define max/min as macros, so std::max/min gets crazy.
> +# undef max
> +# undef min
> +#endif
> +
> +#if !defined(SANITIZER_EXTERNAL_TEST_CONFIG)
> +# define INCLUDED_FROM_SANITIZER_TEST_UTILS_H
> +# include "sanitizer_test_config.h"
> +# undef INCLUDED_FROM_SANITIZER_TEST_UTILS_H
> +#endif
> +
> +#include <stdint.h>
> +
> +#if defined(_MSC_VER)
>  # define NOINLINE __declspec(noinline)
> -# define USED
> -#else  // defined(_WIN32)
> +#else  // defined(_MSC_VER)
>  # define NOINLINE __attribute__((noinline))
> +#endif  // defined(_MSC_VER)
> +
> +#if !defined(_MSC_VER) || defined(__clang__)
> +# define UNUSED __attribute__((unused))
>  # define USED __attribute__((used))
> -#include <stdint.h>
> -#endif  // defined(_WIN32)
> +#else
> +# define UNUSED
> +# define USED
> +#endif
>
>  #if !defined(__has_feature)
>  #define __has_feature(x) 0
>  #endif
>
> -#if __has_feature(address_sanitizer) || defined(__SANITIZE_ADDRESS__)
> -# define ATTRIBUTE_NO_ADDRESS_SAFETY_ANALYSIS \
> +#ifndef ATTRIBUTE_NO_SANITIZE_ADDRESS
> +# if __has_feature(address_sanitizer) || defined(__SANITIZE_ADDRESS__)
> +#  define ATTRIBUTE_NO_SANITIZE_ADDRESS \
>      __attribute__((no_sanitize_address))
> -#else
> -# define ATTRIBUTE_NO_ADDRESS_SAFETY_ANALYSIS
> -#endif
> +# else
> +#  define ATTRIBUTE_NO_SANITIZE_ADDRESS
> +# endif
> +#endif  // ATTRIBUTE_NO_SANITIZE_ADDRESS
>
>  #if __LP64__ || defined(_WIN64)
>  #  define SANITIZER_WORDSIZE 64
> @@ -49,7 +64,9 @@ typedef __int64          int64_t;
>
>  // Make the compiler thinks that something is going on there.
>  inline void break_optimization(void *arg) {
> +#if !defined(_WIN32) || defined(__clang__)
>    __asm__ __volatile__("" : : "r" (arg) : "memory");
> +#endif
>  }
>
>  // This function returns its parameter but in such a way that compiler
> @@ -74,5 +91,28 @@ static inline uint32_t my_rand() {
>    return my_rand_r(&global_seed);
>  }
>
> +// Set availability of platform-specific functions.
> +
> +#if !defined(__APPLE__) && !defined(ANDROID) && !defined(__ANDROID__) && !defined(_WIN32)
> +# define SANITIZER_TEST_HAS_POSIX_MEMALIGN 1
> +#else
> +# define SANITIZER_TEST_HAS_POSIX_MEMALIGN 0
> +#endif
> +
> +#if !defined(__APPLE__) && !defined(__FreeBSD__) && !defined(_WIN32)
> +# define SANITIZER_TEST_HAS_MEMALIGN 1
> +# define SANITIZER_TEST_HAS_PVALLOC 1
> +# define SANITIZER_TEST_HAS_MALLOC_USABLE_SIZE 1
> +#else
> +# define SANITIZER_TEST_HAS_MEMALIGN 0
> +# define SANITIZER_TEST_HAS_PVALLOC 0
> +# define SANITIZER_TEST_HAS_MALLOC_USABLE_SIZE 0
> +#endif
> +
> +#if !defined(__APPLE__)
> +# define SANITIZER_TEST_HAS_STRNLEN 1
> +#else
> +# define SANITIZER_TEST_HAS_STRNLEN 0
> +#endif
>
>  #endif  // SANITIZER_TEST_UTILS_H
> --- gcc/testsuite/g++.dg/asan/asan_test_config.h.jj     2012-12-03 12:44:28.983724406 +0100
> +++ gcc/testsuite/g++.dg/asan/asan_test_config.h        2014-05-22 13:13:53.392991725 +0200
> @@ -19,12 +19,6 @@
>  #include <string>
>  #include <map>
>
> -#if ASAN_USE_DEJAGNU_GTEST
> -# include "dejagnu-gtest.h"
> -#else
> -# include "gtest/gtest.h"
> -#endif
> -
>  using std::string;
>  using std::vector;
>  using std::map;
> @@ -42,11 +36,11 @@ using std::map;
>  #endif
>
>  #ifndef ASAN_NEEDS_SEGV
> -# error "please define ASAN_NEEDS_SEGV"
> -#endif
> -
> -#ifndef ASAN_LOW_MEMORY
> -# define ASAN_LOW_MEMORY 0
> +# if defined(_WIN32)
> +#  define ASAN_NEEDS_SEGV 0
> +# else
> +#  define ASAN_NEEDS_SEGV 1
> +# endif
>  #endif
>
>  #ifndef ASAN_AVOID_EXPENSIVE_TESTS
> --- gcc/testsuite/g++.dg/asan/asan_str_test.cc.jj       2013-02-14 09:38:05.697773648 +0100
> +++ gcc/testsuite/g++.dg/asan/asan_str_test.cc  2014-05-22 13:13:51.694002758 +0200
> @@ -10,6 +10,10 @@
>  //===----------------------------------------------------------------------===//
>  #include "asan_test_utils.h"
>
> +#if defined(__APPLE__)
> +#include <AvailabilityMacros.h>  // For MAC_OS_X_VERSION_*
> +#endif
> +
>  // Used for string functions tests
>  static char global_string[] = "global";
>  static size_t global_string_length = 6;
> @@ -59,7 +63,19 @@ TEST(AddressSanitizer, StrLenOOBTest) {
>    free(heap_string);
>  }
>
> -#ifndef __APPLE__
> +TEST(AddressSanitizer, WcsLenTest) {
> +  EXPECT_EQ(0U, wcslen(Ident(L"")));
> +  size_t hello_len = 13;
> +  size_t hello_size = (hello_len + 1) * sizeof(wchar_t);
> +  EXPECT_EQ(hello_len, wcslen(Ident(L"Hello, World!")));
> +  wchar_t *heap_string = Ident((wchar_t*)malloc(hello_size));
> +  memcpy(heap_string, L"Hello, World!", hello_size);
> +  EXPECT_EQ(hello_len, Ident(wcslen(heap_string)));
> +  EXPECT_DEATH(Ident(wcslen(heap_string + 14)), RightOOBReadMessage(0));
> +  free(heap_string);
> +}
> +
> +#if SANITIZER_TEST_HAS_STRNLEN
>  TEST(AddressSanitizer, StrNLenOOBTest) {
>    size_t size = Ident(123);
>    char *str = MallocAndMemsetString(size);
> @@ -77,7 +93,7 @@ TEST(AddressSanitizer, StrNLenOOBTest) {
>    EXPECT_DEATH(Ident(strnlen(str, size + 1)), RightOOBReadMessage(0));
>    free(str);
>  }
> -#endif
> +#endif  // SANITIZER_TEST_HAS_STRNLEN
>
>  TEST(AddressSanitizer, StrDupOOBTest) {
>    size_t size = Ident(42);
> @@ -168,7 +184,7 @@ TEST(AddressSanitizer, StrNCpyOOBTest) {
>  typedef char*(*PointerToStrChr1)(const char*, int);
>  typedef char*(*PointerToStrChr2)(char*, int);
>
> -USED static void RunStrChrTest(PointerToStrChr1 StrChr) {
> +UNUSED static void RunStrChrTest(PointerToStrChr1 StrChr) {
>    size_t size = Ident(100);
>    char *str = MallocAndMemsetString(size);
>    str[10] = 'q';
> @@ -184,7 +200,7 @@ USED static void RunStrChrTest(PointerTo
>    EXPECT_DEATH(Ident(StrChr(str, 'a')), RightOOBReadMessage(0));
>    free(str);
>  }
> -USED static void RunStrChrTest(PointerToStrChr2 StrChr) {
> +UNUSED static void RunStrChrTest(PointerToStrChr2 StrChr) {
>    size_t size = Ident(100);
>    char *str = MallocAndMemsetString(size);
>    str[10] = 'q';
> @@ -203,7 +219,9 @@ USED static void RunStrChrTest(PointerTo
>
>  TEST(AddressSanitizer, StrChrAndIndexOOBTest) {
>    RunStrChrTest(&strchr);
> +#if !defined(_WIN32)  // no index() on Windows.
>    RunStrChrTest(&index);
> +#endif
>  }
>
>  TEST(AddressSanitizer, StrCmpAndFriendsLogicTest) {
> @@ -226,6 +244,7 @@ TEST(AddressSanitizer, StrCmpAndFriendsL
>    EXPECT_LT(0, strncmp("baa", "aaa", 1));
>    EXPECT_LT(0, strncmp("zyx", "", 2));
>
> +#if !defined(_WIN32)  // no str[n]casecmp on Windows.
>    // strcasecmp
>    EXPECT_EQ(0, strcasecmp("", ""));
>    EXPECT_EQ(0, strcasecmp("zzz", "zzz"));
> @@ -245,6 +264,7 @@ TEST(AddressSanitizer, StrCmpAndFriendsL
>    EXPECT_LT(0, strncasecmp("xyz", "xyy", 10));
>    EXPECT_LT(0, strncasecmp("Baa", "aaa", 1));
>    EXPECT_LT(0, strncasecmp("zyx", "", 2));
> +#endif
>
>    // memcmp
>    EXPECT_EQ(0, memcmp("a", "b", 0));
> @@ -287,9 +307,11 @@ TEST(AddressSanitizer, StrCmpOOBTest) {
>    RunStrCmpTest(&strcmp);
>  }
>
> +#if !defined(_WIN32)  // no str[n]casecmp on Windows.
>  TEST(AddressSanitizer, StrCaseCmpOOBTest) {
>    RunStrCmpTest(&strcasecmp);
>  }
> +#endif
>
>  typedef int(*PointerToStrNCmp)(const char*, const char*, size_t);
>  void RunStrNCmpTest(PointerToStrNCmp StrNCmp) {
> @@ -322,9 +344,12 @@ TEST(AddressSanitizer, StrNCmpOOBTest) {
>    RunStrNCmpTest(&strncmp);
>  }
>
> +#if !defined(_WIN32)  // no str[n]casecmp on Windows.
>  TEST(AddressSanitizer, StrNCaseCmpOOBTest) {
>    RunStrNCmpTest(&strncasecmp);
>  }
> +#endif
> +
>  TEST(AddressSanitizer, StrCatOOBTest) {
>    // strcat() reads strlen(to) bytes from |to| before concatenating.
>    size_t to_size = Ident(100);
> @@ -506,11 +531,13 @@ void RunAtoiOOBTest(PointerToCallAtoi At
>    free(array);
>  }
>
> +#if !defined(_WIN32)  // FIXME: Fix and enable on Windows.
>  TEST(AddressSanitizer, AtoiAndFriendsOOBTest) {
>    RunAtoiOOBTest(&CallAtoi);
>    RunAtoiOOBTest(&CallAtol);
>    RunAtoiOOBTest(&CallAtoll);
>  }
> +#endif
>
>  void CallStrtol(const char *nptr, char **endptr, int base) {
>    Ident(strtol(nptr, endptr, base));
> @@ -560,11 +587,13 @@ void RunStrtolOOBTest(PointerToCallStrto
>    free(array);
>  }
>
> +#if !defined(_WIN32)  // FIXME: Fix and enable on Windows.
>  TEST(AddressSanitizer, StrtollOOBTest) {
>    RunStrtolOOBTest(&CallStrtoll);
>  }
>  TEST(AddressSanitizer, StrtolOOBTest) {
>    RunStrtolOOBTest(&CallStrtol);
>  }
> +#endif
>
>
> --- gcc/testsuite/g++.dg/asan/asan_test_utils.h.jj      2013-02-14 09:38:05.688773689 +0100
> +++ gcc/testsuite/g++.dg/asan/asan_test_utils.h 2014-05-22 13:13:55.087985295 +0200
> @@ -12,24 +12,28 @@
>  #ifndef ASAN_TEST_UTILS_H
>  #define ASAN_TEST_UTILS_H
>
> -#if !defined(ASAN_EXTERNAL_TEST_CONFIG)
> +#if !defined(SANITIZER_EXTERNAL_TEST_CONFIG)
>  # define INCLUDED_FROM_ASAN_TEST_UTILS_H
>  # include "asan_test_config.h"
>  # undef INCLUDED_FROM_ASAN_TEST_UTILS_H
>  #endif
>
>  #include "sanitizer_test_utils.h"
> +#include "sanitizer_pthread_wrappers.h"
> +
>  #include <stdio.h>
>  #include <signal.h>
>  #include <stdlib.h>
>  #include <string.h>
> -#include <strings.h>
> -#include <pthread.h>
>  #include <stdint.h>
> -#include <setjmp.h>
>  #include <assert.h>
>  #include <algorithm>
> -#include <sys/mman.h>
> +
> +#if !defined(_WIN32)
> +# include <strings.h>
> +# include <sys/mman.h>
> +# include <setjmp.h>
> +#endif
>
>  #ifdef __linux__
>  # include <sys/prctl.h>
> @@ -39,18 +43,10 @@
>  #include <unistd.h>
>  #endif
>
> -#if defined(__i386__) || defined(__x86_64__)
> -#include <emmintrin.h>
> -#endif
> -
> -#ifndef __APPLE__
> +#if !defined(__APPLE__) && !defined(__FreeBSD__)
>  #include <malloc.h>
>  #endif
>
> -// Check that pthread_create/pthread_join return success.
> -#define PTHREAD_CREATE(a, b, c, d) ASSERT_EQ(0, pthread_create(a, b, c, d))
> -#define PTHREAD_JOIN(a, b) ASSERT_EQ(0, pthread_join(a, b))
> -
>  #if ASAN_HAS_EXCEPTIONS
>  # define ASAN_THROW(x) throw (x)
>  #else
> --- gcc/testsuite/g++.dg/asan/asan_test.cc.jj   2013-11-12 11:31:20.000000000 +0100
> +++ gcc/testsuite/g++.dg/asan/asan_test.cc      2014-05-22 13:08:39.223454867 +0200
> @@ -23,27 +23,10 @@ NOINLINE void *malloc_bbb(size_t size) {
>  NOINLINE void *malloc_aaa(size_t size) {
>    void *res = malloc_bbb(size); break_optimization(0); return res;}
>
> -#ifndef __APPLE__
> -NOINLINE void *memalign_fff(size_t alignment, size_t size) {
> -  void *res = memalign/**/(alignment, size); break_optimization(0); return res;}
> -NOINLINE void *memalign_eee(size_t alignment, size_t size) {
> -  void *res = memalign_fff(alignment, size); break_optimization(0); return res;}
> -NOINLINE void *memalign_ddd(size_t alignment, size_t size) {
> -  void *res = memalign_eee(alignment, size); break_optimization(0); return res;}
> -NOINLINE void *memalign_ccc(size_t alignment, size_t size) {
> -  void *res = memalign_ddd(alignment, size); break_optimization(0); return res;}
> -NOINLINE void *memalign_bbb(size_t alignment, size_t size) {
> -  void *res = memalign_ccc(alignment, size); break_optimization(0); return res;}
> -NOINLINE void *memalign_aaa(size_t alignment, size_t size) {
> -  void *res = memalign_bbb(alignment, size); break_optimization(0); return res;}
> -#endif  // __APPLE__
> -
> -
>  NOINLINE void free_ccc(void *p) { free(p); break_optimization(0);}
>  NOINLINE void free_bbb(void *p) { free_ccc(p); break_optimization(0);}
>  NOINLINE void free_aaa(void *p) { free_bbb(p); break_optimization(0);}
>
> -
>  template<typename T>
>  NOINLINE void uaf_test(int size, int off) {
>    char *p = (char *)malloc_aaa(size);
> @@ -88,19 +71,19 @@ TEST(AddressSanitizer, VariousMallocsTes
>    *c = 0;
>    delete c;
>
> -#if !defined(__APPLE__) && !defined(ANDROID) && !defined(__ANDROID__)
> +#if SANITIZER_TEST_HAS_POSIX_MEMALIGN
>    int *pm;
>    int pm_res = posix_memalign((void**)&pm, kPageSize, kPageSize);
>    EXPECT_EQ(0, pm_res);
>    free(pm);
> -#endif
> +#endif  // SANITIZER_TEST_HAS_POSIX_MEMALIGN
>
> -#if !defined(__APPLE__)
> +#if SANITIZER_TEST_HAS_MEMALIGN
>    int *ma = (int*)memalign(kPageSize, kPageSize);
>    EXPECT_EQ(0U, (uintptr_t)ma % kPageSize);
>    ma[123] = 0;
>    free(ma);
> -#endif  // __APPLE__
> +#endif  // SANITIZER_TEST_HAS_MEMALIGN
>  }
>
>  TEST(AddressSanitizer, CallocTest) {
> @@ -109,13 +92,37 @@ TEST(AddressSanitizer, CallocTest) {
>    free(a);
>  }
>
> +TEST(AddressSanitizer, CallocReturnsZeroMem) {
> +  size_t sizes[] = {16, 1000, 10000, 100000, 2100000};
> +  for (size_t s = 0; s < sizeof(sizes)/sizeof(sizes[0]); s++) {
> +    size_t size = sizes[s];
> +    for (size_t iter = 0; iter < 5; iter++) {
> +      char *x = Ident((char*)calloc(1, size));
> +      EXPECT_EQ(x[0], 0);
> +      EXPECT_EQ(x[size - 1], 0);
> +      EXPECT_EQ(x[size / 2], 0);
> +      EXPECT_EQ(x[size / 3], 0);
> +      EXPECT_EQ(x[size / 4], 0);
> +      memset(x, 0x42, size);
> +      free(Ident(x));
> +#if !defined(_WIN32)
> +      // FIXME: OOM on Windows. We should just make this a lit test
> +      // with quarantine size set to 1.
> +      free(Ident(malloc(Ident(1 << 27))));  // Try to drain the quarantine.
> +#endif
> +    }
> +  }
> +}
> +
> +#if !defined(_WIN32)  // No valloc on Windows.
>  TEST(AddressSanitizer, VallocTest) {
>    void *a = valloc(100);
>    EXPECT_EQ(0U, (uintptr_t)a % kPageSize);
>    free(a);
>  }
> +#endif
>
> -#ifndef __APPLE__
> +#if SANITIZER_TEST_HAS_PVALLOC
>  TEST(AddressSanitizer, PvallocTest) {
>    char *a = (char*)pvalloc(kPageSize + 100);
>    EXPECT_EQ(0U, (uintptr_t)a % kPageSize);
> @@ -127,8 +134,10 @@ TEST(AddressSanitizer, PvallocTest) {
>    a[101] = 1;  // we should not report an error here.
>    free(a);
>  }
> -#endif  // __APPLE__
> +#endif  // SANITIZER_TEST_HAS_PVALLOC
>
> +#if !defined(_WIN32)
> +// FIXME: Use an equivalent of pthread_setspecific on Windows.
>  void *TSDWorker(void *test_key) {
>    if (test_key) {
>      pthread_setspecific(*(pthread_key_t*)test_key, (void*)0xfeedface);
> @@ -158,6 +167,7 @@ TEST(AddressSanitizer, DISABLED_TSDTest)
>    PTHREAD_JOIN(th, NULL);
>    pthread_key_delete(test_key);
>  }
> +#endif
>
>  TEST(AddressSanitizer, UAF_char) {
>    const char *uaf_string = "AddressSanitizer:.*heap-use-after-free";
> @@ -168,6 +178,36 @@ TEST(AddressSanitizer, UAF_char) {
>    EXPECT_DEATH(uaf_test<U1>(kLargeMalloc, kLargeMalloc / 2), uaf_string);
>  }
>
> +TEST(AddressSanitizer, UAF_long_double) {
> +  if (sizeof(long double) == sizeof(double)) return;
> +  long double *p = Ident(new long double[10]);
> +  EXPECT_DEATH(Ident(p)[12] = 0, "WRITE of size 1[06]");
> +  EXPECT_DEATH(Ident(p)[0] = Ident(p)[12], "READ of size 1[06]");
> +  delete [] Ident(p);
> +}
> +
> +#if !defined(_WIN32)
> +struct Packed5 {
> +  int x;
> +  char c;
> +} __attribute__((packed));
> +#else
> +# pragma pack(push, 1)
> +struct Packed5 {
> +  int x;
> +  char c;
> +};
> +# pragma pack(pop)
> +#endif
> +
> +TEST(AddressSanitizer, UAF_Packed5) {
> +  static_assert(sizeof(Packed5) == 5, "Please check the keywords used");
> +  Packed5 *p = Ident(new Packed5[2]);
> +  EXPECT_DEATH(p[0] = p[3], "READ of size 5");
> +  EXPECT_DEATH(p[3] = p[0], "WRITE of size 5");
> +  delete [] Ident(p);
> +}
> +
>  #if ASAN_HAS_BLACKLIST
>  TEST(AddressSanitizer, IgnoreTest) {
>    int *x = Ident(new int);
> @@ -244,41 +284,6 @@ TEST(AddressSanitizer, SignalTest) {
>  }  // namespace
>  #endif
>
> -static void MallocStress(size_t n) {
> -  uint32_t seed = my_rand();
> -  for (size_t iter = 0; iter < 10; iter++) {
> -    vector<void *> vec;
> -    for (size_t i = 0; i < n; i++) {
> -      if ((i % 3) == 0) {
> -        if (vec.empty()) continue;
> -        size_t idx = my_rand_r(&seed) % vec.size();
> -        void *ptr = vec[idx];
> -        vec[idx] = vec.back();
> -        vec.pop_back();
> -        free_aaa(ptr);
> -      } else {
> -        size_t size = my_rand_r(&seed) % 1000 + 1;
> -#ifndef __APPLE__
> -        size_t alignment = 1 << (my_rand_r(&seed) % 7 + 3);
> -        char *ptr = (char*)memalign_aaa(alignment, size);
> -#else
> -        char *ptr = (char*) malloc_aaa(size);
> -#endif
> -        vec.push_back(ptr);
> -        ptr[0] = 0;
> -        ptr[size-1] = 0;
> -        ptr[size/2] = 0;
> -      }
> -    }
> -    for (size_t i = 0; i < vec.size(); i++)
> -      free_aaa(vec[i]);
> -  }
> -}
> -
> -TEST(AddressSanitizer, MallocStressTest) {
> -  MallocStress((ASAN_LOW_MEMORY) ? 20000 : 200000);
> -}
> -
>  static void TestLargeMalloc(size_t size) {
>    char buff[1024];
>    sprintf(buff, "is located 1 bytes to the left of %lu-byte", (long)size);
> @@ -286,28 +291,21 @@ static void TestLargeMalloc(size_t size)
>  }
>
>  TEST(AddressSanitizer, LargeMallocTest) {
> -  for (int i = 113; i < (1 << 28); i = i * 2 + 13) {
> +  const int max_size = (SANITIZER_WORDSIZE == 32) ? 1 << 26 : 1 << 28;
> +  for (int i = 113; i < max_size; i = i * 2 + 13) {
>      TestLargeMalloc(i);
>    }
>  }
>
> -#if ASAN_LOW_MEMORY != 1
>  TEST(AddressSanitizer, HugeMallocTest) {
> -#ifdef __APPLE__
> -  // It was empirically found out that 1215 megabytes is the maximum amount of
> -  // memory available to the process under AddressSanitizer on 32-bit Mac 10.6.
> -  // 32-bit Mac 10.7 gives even less (< 1G).
> -  // (the libSystem malloc() allows allocating up to 2300 megabytes without
> -  // ASan).
> -  size_t n_megs = SANITIZER_WORDSIZE == 32 ? 500 : 4100;
> -#else
> -  size_t n_megs = SANITIZER_WORDSIZE == 32 ? 2600 : 4100;
> -#endif
> -  TestLargeMalloc(n_megs << 20);
> +  if (SANITIZER_WORDSIZE != 64 || ASAN_AVOID_EXPENSIVE_TESTS) return;
> +  size_t n_megs = 4100;
> +  EXPECT_DEATH(Ident((char*)malloc(n_megs << 20))[-1] = 0,
> +               "is located 1 bytes to the left|"
> +               "AddressSanitizer failed to allocate");
>  }
> -#endif
>
> -#ifndef __APPLE__
> +#if SANITIZER_TEST_HAS_MEMALIGN
>  void MemalignRun(size_t align, size_t size, int idx) {
>    char *p = (char *)memalign(align, size);
>    Ident(p)[idx] = 0;
> @@ -323,20 +321,7 @@ TEST(AddressSanitizer, memalign) {
>                   "is located 1 bytes to the right");
>    }
>  }
> -#endif
> -
> -TEST(AddressSanitizer, ThreadedMallocStressTest) {
> -  const int kNumThreads = 4;
> -  const int kNumIterations = (ASAN_LOW_MEMORY) ? 10000 : 100000;
> -  pthread_t t[kNumThreads];
> -  for (int i = 0; i < kNumThreads; i++) {
> -    PTHREAD_CREATE(&t[i], 0, (void* (*)(void *x))MallocStress,
> -        (void*)kNumIterations);
> -  }
> -  for (int i = 0; i < kNumThreads; i++) {
> -    PTHREAD_JOIN(t[i], 0);
> -  }
> -}
> +#endif  // SANITIZER_TEST_HAS_MEMALIGN
>
>  void *ManyThreadsWorker(void *a) {
>    for (int iter = 0; iter < 100; iter++) {
> @@ -377,17 +362,30 @@ TEST(AddressSanitizer, ReallocTest) {
>    free(ptr2);
>  }
>
> +TEST(AddressSanitizer, ReallocFreedPointerTest) {
> +  void *ptr = Ident(malloc(42));
> +  ASSERT_TRUE(NULL != ptr);
> +  free(ptr);
> +  EXPECT_DEATH(ptr = realloc(ptr, 77), "attempting double-free");
> +}
> +
> +TEST(AddressSanitizer, ReallocInvalidPointerTest) {
> +  void *ptr = Ident(malloc(42));
> +  EXPECT_DEATH(ptr = realloc((int*)ptr + 1, 77), "attempting free.*not malloc");
> +  free(ptr);
> +}
> +
>  TEST(AddressSanitizer, ZeroSizeMallocTest) {
>    // Test that malloc(0) and similar functions don't return NULL.
>    void *ptr = Ident(malloc(0));
>    EXPECT_TRUE(NULL != ptr);
>    free(ptr);
> -#if !defined(__APPLE__) && !defined(ANDROID) && !defined(__ANDROID__)
> +#if SANITIZER_TEST_HAS_POSIX_MEMALIGN
>    int pm_res = posix_memalign(&ptr, 1<<20, 0);
>    EXPECT_EQ(0, pm_res);
>    EXPECT_TRUE(NULL != ptr);
>    free(ptr);
> -#endif
> +#endif  // SANITIZER_TEST_HAS_POSIX_MEMALIGN
>    int *int_ptr = new int[0];
>    int *int_ptr2 = new int[0];
>    EXPECT_TRUE(NULL != int_ptr);
> @@ -397,7 +395,7 @@ TEST(AddressSanitizer, ZeroSizeMallocTes
>    delete[] int_ptr2;
>  }
>
> -#ifndef __APPLE__
> +#if SANITIZER_TEST_HAS_MALLOC_USABLE_SIZE
>  static const char *kMallocUsableSizeErrorMsg =
>    "AddressSanitizer: attempting to call malloc_usable_size()";
>
> @@ -413,8 +411,9 @@ TEST(AddressSanitizer, MallocUsableSizeT
>                 kMallocUsableSizeErrorMsg);
>    free(array);
>    EXPECT_DEATH(malloc_usable_size(array), kMallocUsableSizeErrorMsg);
> +  delete int_ptr;
>  }
> -#endif
> +#endif  // SANITIZER_TEST_HAS_MALLOC_USABLE_SIZE
>
>  void WrongFree() {
>    int *x = (int*)malloc(100 * sizeof(int));
> @@ -423,10 +422,14 @@ void WrongFree() {
>    free(x + 1);
>  }
>
> +#if !defined(_WIN32)  // FIXME: This should be a lit test.
>  TEST(AddressSanitizer, WrongFreeTest) {
> -  EXPECT_DEATH(WrongFree(),
> -               "ERROR: AddressSanitizer: attempting free.*not malloc");
> +  EXPECT_DEATH(WrongFree(), ASAN_PCRE_DOTALL
> +               "ERROR: AddressSanitizer: attempting free.*not malloc"
> +               ".*is located 4 bytes inside of 400-byte region"
> +               ".*allocated by thread");
>  }
> +#endif
>
>  void DoubleFree() {
>    int *x = (int*)malloc(100 * sizeof(int));
> @@ -437,6 +440,7 @@ void DoubleFree() {
>    abort();
>  }
>
> +#if !defined(_WIN32)  // FIXME: This should be a lit test.
>  TEST(AddressSanitizer, DoubleFreeTest) {
>    EXPECT_DEATH(DoubleFree(), ASAN_PCRE_DOTALL
>                 "ERROR: AddressSanitizer: attempting double-free"
> @@ -444,20 +448,22 @@ TEST(AddressSanitizer, DoubleFreeTest) {
>                 ".*freed by thread T0 here"
>                 ".*previously allocated by thread T0 here");
>  }
> +#endif
>
>  template<int kSize>
>  NOINLINE void SizedStackTest() {
>    char a[kSize];
>    char  *A = Ident((char*)&a);
> +  const char *expected_death = "AddressSanitizer: stack-buffer-";
>    for (size_t i = 0; i < kSize; i++)
>      A[i] = i;
> -  EXPECT_DEATH(A[-1] = 0, "");
> -  EXPECT_DEATH(A[-20] = 0, "");
> -  EXPECT_DEATH(A[-31] = 0, "");
> -  EXPECT_DEATH(A[kSize] = 0, "");
> -  EXPECT_DEATH(A[kSize + 1] = 0, "");
> -  EXPECT_DEATH(A[kSize + 10] = 0, "");
> -  EXPECT_DEATH(A[kSize + 31] = 0, "");
> +  EXPECT_DEATH(A[-1] = 0, expected_death);
> +  EXPECT_DEATH(A[-5] = 0, expected_death);
> +  EXPECT_DEATH(A[kSize] = 0, expected_death);
> +  EXPECT_DEATH(A[kSize + 1] = 0, expected_death);
> +  EXPECT_DEATH(A[kSize + 5] = 0, expected_death);
> +  if (kSize > 16)
> +    EXPECT_DEATH(A[kSize + 31] = 0, expected_death);
>  }
>
>  TEST(AddressSanitizer, SimpleStackTest) {
> @@ -478,6 +484,9 @@ TEST(AddressSanitizer, SimpleStackTest)
>    SizedStackTest<128>();
>  }
>
> +#if !defined(_WIN32)
> +// FIXME: It's a bit hard to write multi-line death test expectations
> +// in a portable way.  Anyways, this should just be turned into a lit test.
>  TEST(AddressSanitizer, ManyStackObjectsTest) {
>    char XXX[10];
>    char YYY[20];
> @@ -486,6 +495,47 @@ TEST(AddressSanitizer, ManyStackObjectsT
>    Ident(YYY);
>    EXPECT_DEATH(Ident(ZZZ)[-1] = 0, ASAN_PCRE_DOTALL "XXX.*YYY.*ZZZ");
>  }
> +#endif
> +
> +#if 0  // This test requires online symbolizer.
> +// Moved to lit_tests/stack-oob-frames.cc.
> +// Reenable here once we have online symbolizer by default.
> +NOINLINE static void Frame0(int frame, char *a, char *b, char *c) {
> +  char d[4] = {0};
> +  char *D = Ident(d);
> +  switch (frame) {
> +    case 3: a[5]++; break;
> +    case 2: b[5]++; break;
> +    case 1: c[5]++; break;
> +    case 0: D[5]++; break;
> +  }
> +}
> +NOINLINE static void Frame1(int frame, char *a, char *b) {
> +  char c[4] = {0}; Frame0(frame, a, b, c);
> +  break_optimization(0);
> +}
> +NOINLINE static void Frame2(int frame, char *a) {
> +  char b[4] = {0}; Frame1(frame, a, b);
> +  break_optimization(0);
> +}
> +NOINLINE static void Frame3(int frame) {
> +  char a[4] = {0}; Frame2(frame, a);
> +  break_optimization(0);
> +}
> +
> +TEST(AddressSanitizer, GuiltyStackFrame0Test) {
> +  EXPECT_DEATH(Frame3(0), "located .*in frame <.*Frame0");
> +}
> +TEST(AddressSanitizer, GuiltyStackFrame1Test) {
> +  EXPECT_DEATH(Frame3(1), "located .*in frame <.*Frame1");
> +}
> +TEST(AddressSanitizer, GuiltyStackFrame2Test) {
> +  EXPECT_DEATH(Frame3(2), "located .*in frame <.*Frame2");
> +}
> +TEST(AddressSanitizer, GuiltyStackFrame3Test) {
> +  EXPECT_DEATH(Frame3(3), "located .*in frame <.*Frame3");
> +}
> +#endif
>
>  NOINLINE void LongJmpFunc1(jmp_buf buf) {
>    // create three red zones for these two stack objects.
> @@ -498,6 +548,24 @@ NOINLINE void LongJmpFunc1(jmp_buf buf)
>    longjmp(buf, 1);
>  }
>
> +NOINLINE void TouchStackFunc() {
> +  int a[100];  // long array will intersect with redzones from LongJmpFunc1.
> +  int *A = Ident(a);
> +  for (int i = 0; i < 100; i++)
> +    A[i] = i*i;
> +}
> +
> +// Test that we handle longjmp and do not report false positives on stack.
> +TEST(AddressSanitizer, LongJmpTest) {
> +  static jmp_buf buf;
> +  if (!setjmp(buf)) {
> +    LongJmpFunc1(buf);
> +  } else {
> +    TouchStackFunc();
> +  }
> +}
> +
> +#if !defined(_WIN32)  // Only basic longjmp is available on Windows.
>  NOINLINE void BuiltinLongJmpFunc1(jmp_buf buf) {
>    // create three red zones for these two stack objects.
>    int a;
> @@ -531,25 +599,10 @@ NOINLINE void SigLongJmpFunc1(sigjmp_buf
>    siglongjmp(buf, 1);
>  }
>
> -
> -NOINLINE void TouchStackFunc() {
> -  int a[100];  // long array will intersect with redzones from LongJmpFunc1.
> -  int *A = Ident(a);
> -  for (int i = 0; i < 100; i++)
> -    A[i] = i*i;
> -}
> -
> -// Test that we handle longjmp and do not report fals positives on stack.
> -TEST(AddressSanitizer, LongJmpTest) {
> -  static jmp_buf buf;
> -  if (!setjmp(buf)) {
> -    LongJmpFunc1(buf);
> -  } else {
> -    TouchStackFunc();
> -  }
> -}
> -
> -#if not defined(__ANDROID__)
> +#if !defined(__ANDROID__) && \
> +    !defined(__powerpc64__) && !defined(__powerpc__)
> +// Does not work on Power:
> +// https://code.google.com/p/address-sanitizer/issues/detail?id=185
>  TEST(AddressSanitizer, BuiltinLongJmpTest) {
>    static jmp_buf buf;
>    if (!__builtin_setjmp((void**)buf)) {
> @@ -558,7 +611,8 @@ TEST(AddressSanitizer, BuiltinLongJmpTes
>      TouchStackFunc();
>    }
>  }
> -#endif  // not defined(__ANDROID__)
> +#endif  // !defined(__ANDROID__) && !defined(__powerpc64__) &&
> +        // !defined(__powerpc__)
>
>  TEST(AddressSanitizer, UnderscopeLongJmpTest) {
>    static jmp_buf buf;
> @@ -577,8 +631,10 @@ TEST(AddressSanitizer, SigLongJmpTest) {
>      TouchStackFunc();
>    }
>  }
> +#endif
>
> -#ifdef __EXCEPTIONS
> +// FIXME: Why does clang-cl define __EXCEPTIONS?
> +#if defined(__EXCEPTIONS) && !defined(_WIN32)
>  NOINLINE void ThrowFunc() {
>    // create three red zones for these two stack objects.
>    int a;
> @@ -626,7 +682,8 @@ TEST(AddressSanitizer, ThreadStackReuseT
>    PTHREAD_JOIN(t, 0);
>  }
>
> -#if defined(__i386__) || defined(__x86_64__)
> +#if defined(__i686__) || defined(__x86_64__)
> +#include <emmintrin.h>
>  TEST(AddressSanitizer, Store128Test) {
>    char *a = Ident((char*)malloc(Ident(12)));
>    char *p = a;
> @@ -644,11 +701,19 @@ TEST(AddressSanitizer, Store128Test) {
>  }
>  #endif
>
> +// FIXME: All tests that use this function should be turned into lit tests.
>  string RightOOBErrorMessage(int oob_distance, bool is_write) {
>    assert(oob_distance >= 0);
>    char expected_str[100];
> -  sprintf(expected_str, ASAN_PCRE_DOTALL "%s.*located %d bytes to the right",
> -          is_write ? "WRITE" : "READ", oob_distance);
> +  sprintf(expected_str, ASAN_PCRE_DOTALL
> +#if !GTEST_USES_SIMPLE_RE
> +          "buffer-overflow.*%s.*"
> +#endif
> +          "located %d bytes to the right",
> +#if !GTEST_USES_SIMPLE_RE
> +          is_write ? "WRITE" : "READ",
> +#endif
> +          oob_distance);
>    return string(expected_str);
>  }
>
> @@ -660,11 +725,19 @@ string RightOOBReadMessage(int oob_dista
>    return RightOOBErrorMessage(oob_distance, /*is_write*/false);
>  }
>
> +// FIXME: All tests that use this function should be turned into lit tests.
>  string LeftOOBErrorMessage(int oob_distance, bool is_write) {
>    assert(oob_distance > 0);
>    char expected_str[100];
> -  sprintf(expected_str, ASAN_PCRE_DOTALL "%s.*located %d bytes to the left",
> -          is_write ? "WRITE" : "READ", oob_distance);
> +  sprintf(expected_str,
> +#if !GTEST_USES_SIMPLE_RE
> +          ASAN_PCRE_DOTALL "%s.*"
> +#endif
> +          "located %d bytes to the left",
> +#if !GTEST_USES_SIMPLE_RE
> +          is_write ? "WRITE" : "READ",
> +#endif
> +          oob_distance);
>    return string(expected_str);
>  }
>
> @@ -818,6 +891,7 @@ void ThreadedTestSpawn() {
>    PTHREAD_JOIN(t, 0);
>  }
>
> +#if !defined(_WIN32)  // FIXME: This should be a lit test.
>  TEST(AddressSanitizer, ThreadedTest) {
>    EXPECT_DEATH(ThreadedTestSpawn(),
>                 ASAN_PCRE_DOTALL
> @@ -825,6 +899,7 @@ TEST(AddressSanitizer, ThreadedTest) {
>                 ".*Thread T.*created"
>                 ".*Thread T.*created");
>  }
> +#endif
>
>  void *ThreadedTestFunc(void *unused) {
>    // Check if prctl(PR_SET_NAME) is supported. Return if not.
> @@ -855,7 +930,11 @@ TEST(AddressSanitizer, ShadowGapTest) {
>  #if SANITIZER_WORDSIZE == 32
>    char *addr = (char*)0x22000000;
>  #else
> +# if defined(__powerpc64__)
> +  char *addr = (char*)0x024000800000;
> +# else
>    char *addr = (char*)0x0000100000080000;
> +# endif
>  #endif
>    EXPECT_DEATH(*addr = 1, "AddressSanitizer: SEGV on unknown");
>  }
> @@ -1015,7 +1094,8 @@ TEST(AddressSanitizer, PthreadExitTest)
>    }
>  }
>
> -#ifdef __EXCEPTIONS
> +// FIXME: Why does clang-cl define __EXCEPTIONS?
> +#if defined(__EXCEPTIONS) && !defined(_WIN32)
>  NOINLINE static void StackReuseAndException() {
>    int large_stack[1000];
>    Ident(large_stack);
> @@ -1033,12 +1113,14 @@ TEST(AddressSanitizer, DISABLED_StressSt
>  }
>  #endif
>
> +#if !defined(_WIN32)
>  TEST(AddressSanitizer, MlockTest) {
>    EXPECT_EQ(0, mlockall(MCL_CURRENT));
>    EXPECT_EQ(0, mlock((void*)0x12345, 0x5678));
>    EXPECT_EQ(0, munlockall());
>    EXPECT_EQ(0, munlock((void*)0x987, 0x654));
>  }
> +#endif
>
>  struct LargeStruct {
>    int foo[100];
> @@ -1051,19 +1133,26 @@ TEST(AddressSanitizer, LargeStructCopyTe
>    *Ident(&a) = *Ident(&a);
>  }
>
> -ATTRIBUTE_NO_ADDRESS_SAFETY_ANALYSIS
> -static void NoAddressSafety() {
> +ATTRIBUTE_NO_SANITIZE_ADDRESS
> +static void NoSanitizeAddress() {
>    char *foo = new char[10];
>    Ident(foo)[10] = 0;
>    delete [] foo;
>  }
>
> -TEST(AddressSanitizer, AttributeNoAddressSafetyTest) {
> -  Ident(NoAddressSafety)();
> +TEST(AddressSanitizer, AttributeNoSanitizeAddressTest) {
> +  Ident(NoSanitizeAddress)();
>  }
>
> -// It doesn't work on Android, as calls to new/delete go through malloc/free.
> -#if !defined(ANDROID) && !defined(__ANDROID__)
> +// The new/delete/etc mismatch checks don't work on Android,
> +//   as calls to new/delete go through malloc/free.
> +// OS X support is tracked here:
> +//   https://code.google.com/p/address-sanitizer/issues/detail?id=131
> +// Windows support is tracked here:
> +//   https://code.google.com/p/address-sanitizer/issues/detail?id=309
> +#if !defined(ANDROID) && !defined(__ANDROID__) && \
> +    !defined(__APPLE__) && \
> +    !defined(_WIN32)
>  static string MismatchStr(const string &str) {
>    return string("AddressSanitizer: alloc-dealloc-mismatch \\(") + str;
>  }
> @@ -1177,3 +1266,18 @@ TEST(AddressSanitizer, LongDoubleNegativ
>    memcpy(Ident(&a), Ident(&b), sizeof(long double));
>    memcpy(Ident(&c), Ident(&b), sizeof(long double));
>  }
> +
> +#if !defined(_WIN32)
> +TEST(AddressSanitizer, pthread_getschedparam) {
> +  int policy;
> +  struct sched_param param;
> +  EXPECT_DEATH(
> +      pthread_getschedparam(pthread_self(), &policy, Ident(&param) + 2),
> +      "AddressSanitizer: stack-buffer-.*flow");
> +  EXPECT_DEATH(
> +      pthread_getschedparam(pthread_self(), Ident(&policy) - 1, &param),
> +      "AddressSanitizer: stack-buffer-.*flow");
> +  int res = pthread_getschedparam(pthread_self(), &policy, &param);
> +  ASSERT_EQ(0, res);
> +}
> +#endif
> --- gcc/testsuite/g++.dg/asan/sanitizer_pthread_wrappers.h.jj   2014-05-22 13:20:42.496012107 +0200
> +++ gcc/testsuite/g++.dg/asan/sanitizer_pthread_wrappers.h      2014-05-22 13:20:45.523996247 +0200
> @@ -0,0 +1,64 @@
> +//===-- sanitizer_pthread_wrappers.h ----------------------------*- C++ -*-===//
> +//
> +// This file is distributed under the University of Illinois Open Source
> +// License. See LICENSE.TXT for details.
> +//
> +//===----------------------------------------------------------------------===//
> +//
> +// This file is a part of *Sanitizer runtime.
> +// It provides handy wrappers for thread manipulation, that:
> +//  a) assert on any failure rather than returning an error code
> +//  b) defines pthread-like interface on platforms where where <pthread.h>
> +//     is not supplied by default.
> +//
> +//===----------------------------------------------------------------------===//
> +
> +#ifndef SANITIZER_PTHREAD_WRAPPERS_H
> +#define SANITIZER_PTHREAD_WRAPPERS_H
> +
> +#include "sanitizer_test_utils.h"
> +
> +#if !defined(_WIN32)
> +# include <pthread.h>
> +// Simply forward the arguments and check that the pthread functions succeed.
> +# define PTHREAD_CREATE(a, b, c, d) ASSERT_EQ(0, pthread_create(a, b, c, d))
> +# define PTHREAD_JOIN(a, b) ASSERT_EQ(0, pthread_join(a, b))
> +#else
> +typedef HANDLE pthread_t;
> +
> +struct PthreadHelperCreateThreadInfo {
> +  void *(*start_routine)(void *);
> +  void *arg;
> +};
> +
> +inline DWORD WINAPI PthreadHelperThreadProc(void *arg) {
> +  PthreadHelperCreateThreadInfo *start_data =
> +      reinterpret_cast<PthreadHelperCreateThreadInfo*>(arg);
> +  void *ret = (start_data->start_routine)(start_data->arg);
> +  delete start_data;
> +  return (DWORD)ret;
> +}
> +
> +inline void PTHREAD_CREATE(pthread_t *thread, void *attr,
> +                           void *(*start_routine)(void *), void *arg) {
> +  ASSERT_EQ(0, attr) << "Thread attributes are not supported yet.";
> +  PthreadHelperCreateThreadInfo *data = new PthreadHelperCreateThreadInfo;
> +  data->start_routine = start_routine;
> +  data->arg = arg;
> +  *thread = CreateThread(0, 0, PthreadHelperThreadProc, data, 0, 0);
> +  ASSERT_NE(nullptr, *thread) << "Failed to create a thread.";
> +}
> +
> +inline void PTHREAD_JOIN(pthread_t thread, void **value_ptr) {
> +  ASSERT_EQ(0, value_ptr) << "Nonzero value_ptr is not supported yet.";
> +  ASSERT_EQ(WAIT_OBJECT_0, WaitForSingleObject(thread, INFINITE));
> +  ASSERT_NE(0, CloseHandle(thread));
> +}
> +
> +inline void pthread_exit(void *retval) {
> +  ASSERT_EQ(0, retval) << "Nonzero retval is not supported yet.";
> +  ExitThread((DWORD)retval);
> +}
> +#endif  // _WIN32
> +
> +#endif  // SANITIZER_PTHREAD_WRAPPERS_H
> --- gcc/testsuite/g++.dg/asan/sanitizer_test_config.h.jj        2014-05-22 13:11:24.552661120 +0200
> +++ gcc/testsuite/g++.dg/asan/sanitizer_test_config.h   2014-05-22 13:11:28.153650267 +0200
> @@ -0,0 +1,28 @@
> +//===-- sanitizer_test_config.h ---------------------------------*- C++ -*-===//
> +//
> +// This file is distributed under the University of Illinois Open Source
> +// License. See LICENSE.TXT for details.
> +//
> +//===----------------------------------------------------------------------===//
> +//
> +// This file is a part of *Sanitizer runtime.
> +//
> +//===----------------------------------------------------------------------===//
> +#if !defined(INCLUDED_FROM_SANITIZER_TEST_UTILS_H)
> +# error "This file should be included into sanitizer_test_utils.h only"
> +#endif
> +
> +#ifndef SANITIZER_TEST_CONFIG_H
> +#define SANITIZER_TEST_CONFIG_H
> +
> +#include <vector>
> +#include <string>
> +#include <map>
> +
> +#if SANITIZER_USE_DEJAGNU_GTEST
> +# include "dejagnu-gtest.h"
> +#else
> +# include "gtest/gtest.h"
> +#endif
> +
> +#endif  // SANITIZER_TEST_CONFIG_H
>
>
>         Jakub

Re: libsanitizer merge from upstream r208536

[Jakub Jelinek]

On Thu, May 22, 2014 at 04:06:21PM +0400, Konstantin Serebryany wrote:
> Not really recently... (Feb 2013)
> http://llvm.org/viewvc/llvm-project?rev=175507&view=rev

Ah, wasn't aware of that.

Here is (so far not bootstrapped/regtested) patch for the GCC side.

Notes:
1) the cases where we e.g. for various stringops perform first and
   last address check and use separate __asan_report_*1 for reporting
   that should probably be converted to use this __asan_report_*_n too
2) it doesn't still deal with unaligned power of two accesses properly,
   but neither does llvm (at least not 3.4).  Am not talking about
   undefined behavior cases where the compiler isn't told the access
   is misaligned, but e.g. when accessing struct S { int x; }
   __attribute__((packed)) and similar (or aligned(1)).  Supposedly
   we could force __asan_report_*_n for that case too, because
   normal wider check assumes it is aligned
3) there is still a failure for -m32:
FAIL: g++.dg/asan/asan_test.C  -O2  AddressSanitizer_UAF_long_double Ident(p)[12] = 0 output pattern test
Output should match: WRITE of size 1[06]
FAIL: g++.dg/asan/asan_test.C  -O2  AddressSanitizer_UAF_long_double Ident(p)[0] = Ident(p)[12] output pattern test
Output should match: READ of size 1[06]
   That sounds like something to fix in upstream, it should allow also size
   12 which is the size of long double on ia32 (16 bytes on x86_64),
   thus 1[026].  Kostya, can you please change it, I'll then apply it
   to the testsuite patch too?  As mentioned earlier, ubsan has similar
   problem where it doesn't recognize float bitsize 96 (but unlike this
   case where clang+llvm pass in 10 bytes, which is what is actually
   accessed by hw if using i387 stack, but not if using other means of
   copying it, in ubsan case clang also passes in bitsize 96 that ubsan
   doesn't handle).

I'll bootstrap/regtest this later today.

2014-05-22  Jakub Jelinek  <jakub@redhat.com>

	* sanitizer.def (BUILT_IN_ASAN_REPORT_LOAD_N,
	BUILT_IN_ASAN_REPORT_STORE_N): New.
	* asan.c (struct asan_mem_ref): Change access_size type to
	HOST_WIDE_INT.
	(asan_mem_ref_init, asan_mem_ref_new, get_mem_refs_of_builtin_call,
	update_mem_ref_hash_table): Likewise.
	(asan_mem_ref_hasher::hash): Hash in a HWI.
	(report_error_func): Change size_in_bytes argument to HWI.
	Use *_N builtins if size_in_bytes is larger than 16 or not power of
	two.
	(build_shadow_mem_access): New function.
	(build_check_stmt): Use it.  Change size_in_bytes argument to HWI.
	Handle size_in_bytes not power of two or larger than 16.
	(instrument_derefs): Don't give up if size_in_bytes is not
	power of two or is larger than 16.

--- gcc/sanitizer.def.jj	2014-05-22 10:18:01.000000000 +0200
+++ gcc/sanitizer.def	2014-05-22 14:13:27.859513839 +0200
@@ -41,6 +41,9 @@ DEF_SANITIZER_BUILTIN(BUILT_IN_ASAN_REPO
 		      BT_FN_VOID_PTR, ATTR_TMPURE_NORETURN_NOTHROW_LEAF_LIST)
 DEF_SANITIZER_BUILTIN(BUILT_IN_ASAN_REPORT_LOAD16, "__asan_report_load16",
 		      BT_FN_VOID_PTR, ATTR_TMPURE_NORETURN_NOTHROW_LEAF_LIST)
+DEF_SANITIZER_BUILTIN(BUILT_IN_ASAN_REPORT_LOAD_N, "__asan_report_load_n",
+		      BT_FN_VOID_PTR_PTRMODE,
+		      ATTR_TMPURE_NORETURN_NOTHROW_LEAF_LIST)
 DEF_SANITIZER_BUILTIN(BUILT_IN_ASAN_REPORT_STORE1, "__asan_report_store1",
 		      BT_FN_VOID_PTR, ATTR_TMPURE_NORETURN_NOTHROW_LEAF_LIST)
 DEF_SANITIZER_BUILTIN(BUILT_IN_ASAN_REPORT_STORE2, "__asan_report_store2",
@@ -51,6 +54,9 @@ DEF_SANITIZER_BUILTIN(BUILT_IN_ASAN_REPO
 		      BT_FN_VOID_PTR, ATTR_TMPURE_NORETURN_NOTHROW_LEAF_LIST)
 DEF_SANITIZER_BUILTIN(BUILT_IN_ASAN_REPORT_STORE16, "__asan_report_store16",
 		      BT_FN_VOID_PTR, ATTR_TMPURE_NORETURN_NOTHROW_LEAF_LIST)
+DEF_SANITIZER_BUILTIN(BUILT_IN_ASAN_REPORT_STORE_N, "__asan_report_store_n",
+		      BT_FN_VOID_PTR_PTRMODE,
+		      ATTR_TMPURE_NORETURN_NOTHROW_LEAF_LIST)
 DEF_SANITIZER_BUILTIN(BUILT_IN_ASAN_REGISTER_GLOBALS,
 		      "__asan_register_globals",
 		      BT_FN_VOID_PTR_PTRMODE, ATTR_NOTHROW_LEAF_LIST)
--- gcc/asan.c.jj	2014-05-11 22:21:23.000000000 +0200
+++ gcc/asan.c	2014-05-22 15:28:30.125998730 +0200
@@ -251,8 +251,8 @@ struct asan_mem_ref
   /* The expression of the beginning of the memory region.  */
   tree start;
 
-  /* The size of the access (can be 1, 2, 4, 8, 16 for now).  */
-  char access_size;
+  /* The size of the access.  */
+  HOST_WIDE_INT access_size;
 };
 
 static alloc_pool asan_mem_ref_alloc_pool;
@@ -274,7 +274,7 @@ asan_mem_ref_get_alloc_pool ()
 /* Initializes an instance of asan_mem_ref.  */
 
 static void
-asan_mem_ref_init (asan_mem_ref *ref, tree start, char access_size)
+asan_mem_ref_init (asan_mem_ref *ref, tree start, HOST_WIDE_INT access_size)
 {
   ref->start = start;
   ref->access_size = access_size;
@@ -287,7 +287,7 @@ asan_mem_ref_init (asan_mem_ref *ref, tr
    access to the referenced memory.  */
 
 static asan_mem_ref*
-asan_mem_ref_new (tree start, char access_size)
+asan_mem_ref_new (tree start, HOST_WIDE_INT access_size)
 {
   asan_mem_ref *ref =
     (asan_mem_ref *) pool_alloc (asan_mem_ref_get_alloc_pool ());
@@ -334,7 +334,7 @@ inline hashval_t
 asan_mem_ref_hasher::hash (const asan_mem_ref *mem_ref)
 {
   hashval_t h = iterative_hash_expr (mem_ref->start, 0);
-  h = iterative_hash_hashval_t (h, mem_ref->access_size);
+  h = iterative_hash_host_wide_int (mem_ref->access_size, h);
   return h;
 }
 
@@ -392,7 +392,7 @@ free_mem_ref_resources ()
 /* Return true iff the memory reference REF has been instrumented.  */
 
 static bool
-has_mem_ref_been_instrumented (tree ref, char access_size)
+has_mem_ref_been_instrumented (tree ref, HOST_WIDE_INT access_size)
 {
   asan_mem_ref r;
   asan_mem_ref_init (&r, ref, access_size);
@@ -480,7 +480,7 @@ get_mem_refs_of_builtin_call (const gimp
   tree source0 = NULL_TREE, source1 = NULL_TREE,
     dest = NULL_TREE, len = NULL_TREE;
   bool is_store = true, got_reference_p = false;
-  char access_size = 1;
+  HOST_WIDE_INT access_size = 1;
 
   switch (DECL_FUNCTION_CODE (callee))
     {
@@ -842,7 +842,7 @@ has_stmt_been_instrumented_p (gimple stm
 /*  Insert a memory reference into the hash table.  */
 
 static void
-update_mem_ref_hash_table (tree ref, char access_size)
+update_mem_ref_hash_table (tree ref, HOST_WIDE_INT access_size)
 {
   hash_table <asan_mem_ref_hasher> ht = get_mem_ref_hash_table ();
 
@@ -1315,20 +1315,22 @@ asan_protect_global (tree decl)
   return true;
 }
 
-/* Construct a function tree for __asan_report_{load,store}{1,2,4,8,16}.
-   IS_STORE is either 1 (for a store) or 0 (for a load).
-   SIZE_IN_BYTES is one of 1, 2, 4, 8, 16.  */
+/* Construct a function tree for __asan_report_{load,store}{1,2,4,8,16,_n}.
+   IS_STORE is either 1 (for a store) or 0 (for a load).  */
 
 static tree
-report_error_func (bool is_store, int size_in_bytes)
+report_error_func (bool is_store, HOST_WIDE_INT size_in_bytes)
 {
-  static enum built_in_function report[2][5]
+  static enum built_in_function report[2][6]
     = { { BUILT_IN_ASAN_REPORT_LOAD1, BUILT_IN_ASAN_REPORT_LOAD2,
 	  BUILT_IN_ASAN_REPORT_LOAD4, BUILT_IN_ASAN_REPORT_LOAD8,
-	  BUILT_IN_ASAN_REPORT_LOAD16 },
+	  BUILT_IN_ASAN_REPORT_LOAD16, BUILT_IN_ASAN_REPORT_LOAD_N },
 	{ BUILT_IN_ASAN_REPORT_STORE1, BUILT_IN_ASAN_REPORT_STORE2,
 	  BUILT_IN_ASAN_REPORT_STORE4, BUILT_IN_ASAN_REPORT_STORE8,
-	  BUILT_IN_ASAN_REPORT_STORE16 } };
+	  BUILT_IN_ASAN_REPORT_STORE16, BUILT_IN_ASAN_REPORT_STORE_N } };
+  if ((size_in_bytes & (size_in_bytes - 1)) != 0
+      || size_in_bytes > 16)
+    return builtin_decl_implicit (report[is_store][5]);
   return builtin_decl_implicit (report[is_store][exact_log2 (size_in_bytes)]);
 }
 
@@ -1450,6 +1452,47 @@ insert_if_then_before_iter (gimple cond,
   gsi_insert_after (&cond_insert_point, cond, GSI_NEW_STMT);
 }
 
+/* Build
+   (base_addr >> ASAN_SHADOW_SHIFT) + targetm.asan_shadow_offset ().  */
+
+static tree
+build_shadow_mem_access (gimple_stmt_iterator *gsi, location_t location,
+			 tree base_addr, tree shadow_ptr_type)
+{
+  tree t, uintptr_type = TREE_TYPE (base_addr);
+  tree shadow_type = TREE_TYPE (shadow_ptr_type);
+  gimple g;
+
+  t = build_int_cst (uintptr_type, ASAN_SHADOW_SHIFT);
+  g = gimple_build_assign_with_ops (RSHIFT_EXPR,
+				    make_ssa_name (uintptr_type, NULL),
+				    base_addr, t);
+  gimple_set_location (g, location);
+  gsi_insert_after (gsi, g, GSI_NEW_STMT);
+
+  t = build_int_cst (uintptr_type, targetm.asan_shadow_offset ());
+  g = gimple_build_assign_with_ops (PLUS_EXPR,
+				    make_ssa_name (uintptr_type, NULL),
+				    gimple_assign_lhs (g), t);
+  gimple_set_location (g, location);
+  gsi_insert_after (gsi, g, GSI_NEW_STMT);
+
+  g = gimple_build_assign_with_ops (NOP_EXPR,
+				    make_ssa_name (shadow_ptr_type, NULL),
+				    gimple_assign_lhs (g), NULL_TREE);
+  gimple_set_location (g, location);
+  gsi_insert_after (gsi, g, GSI_NEW_STMT);
+
+  t = build2 (MEM_REF, shadow_type, gimple_assign_lhs (g),
+	      build_int_cst (shadow_ptr_type, 0));
+  g = gimple_build_assign_with_ops (MEM_REF,
+				    make_ssa_name (shadow_type, NULL),
+				    t, NULL_TREE);
+  gimple_set_location (g, location);
+  gsi_insert_after (gsi, g, GSI_NEW_STMT);
+  return gimple_assign_lhs (g);
+}
+
 /* Instrument the memory access instruction BASE.  Insert new
    statements before or after ITER.
 
@@ -1457,8 +1500,7 @@ insert_if_then_before_iter (gimple cond,
    SSA_NAME, or a non-SSA expression.  LOCATION is the source code
    location.  IS_STORE is TRUE for a store, FALSE for a load.
    BEFORE_P is TRUE for inserting the instrumentation code before
-   ITER, FALSE for inserting it after ITER.  SIZE_IN_BYTES is one of
-   1, 2, 4, 8, 16.
+   ITER, FALSE for inserting it after ITER.
 
    If BEFORE_P is TRUE, *ITER is arranged to still point to the
    statement it was pointing to prior to calling this function,
@@ -1466,7 +1508,7 @@ insert_if_then_before_iter (gimple cond,
 
 static void
 build_check_stmt (location_t location, tree base, gimple_stmt_iterator *iter,
-		  bool before_p, bool is_store, int size_in_bytes)
+		  bool before_p, bool is_store, HOST_WIDE_INT size_in_bytes)
 {
   gimple_stmt_iterator gsi;
   basic_block then_bb, else_bb;
@@ -1477,6 +1519,12 @@ build_check_stmt (location_t location, t
   tree uintptr_type
     = build_nonstandard_integer_type (TYPE_PRECISION (TREE_TYPE (base)), 1);
   tree base_ssa = base;
+  HOST_WIDE_INT real_size_in_bytes = size_in_bytes;
+  tree sz_arg = NULL_TREE;
+
+  if ((size_in_bytes & (size_in_bytes - 1)) != 0
+      || size_in_bytes > 16)
+    real_size_in_bytes = 1;
 
   /* Get an iterator on the point where we can add the condition
      statement for the instrumentation.  */
@@ -1509,51 +1557,24 @@ build_check_stmt (location_t location, t
 
   /* Build
      (base_addr >> ASAN_SHADOW_SHIFT) + targetm.asan_shadow_offset ().  */
+  shadow = build_shadow_mem_access (&gsi, location, base_addr,
+				    shadow_ptr_type);
 
-  t = build_int_cst (uintptr_type, ASAN_SHADOW_SHIFT);
-  g = gimple_build_assign_with_ops (RSHIFT_EXPR,
-				    make_ssa_name (uintptr_type, NULL),
-				    base_addr, t);
-  gimple_set_location (g, location);
-  gsi_insert_after (&gsi, g, GSI_NEW_STMT);
-
-  t = build_int_cst (uintptr_type, targetm.asan_shadow_offset ());
-  g = gimple_build_assign_with_ops (PLUS_EXPR,
-				    make_ssa_name (uintptr_type, NULL),
-				    gimple_assign_lhs (g), t);
-  gimple_set_location (g, location);
-  gsi_insert_after (&gsi, g, GSI_NEW_STMT);
-
-  g = gimple_build_assign_with_ops (NOP_EXPR,
-				    make_ssa_name (shadow_ptr_type, NULL),
-				    gimple_assign_lhs (g), NULL_TREE);
-  gimple_set_location (g, location);
-  gsi_insert_after (&gsi, g, GSI_NEW_STMT);
-
-  t = build2 (MEM_REF, shadow_type, gimple_assign_lhs (g),
-	      build_int_cst (shadow_ptr_type, 0));
-  g = gimple_build_assign_with_ops (MEM_REF,
-				    make_ssa_name (shadow_type, NULL),
-				    t, NULL_TREE);
-  gimple_set_location (g, location);
-  gsi_insert_after (&gsi, g, GSI_NEW_STMT);
-  shadow = gimple_assign_lhs (g);
-
-  if (size_in_bytes < 8)
+  if (real_size_in_bytes < 8)
     {
       /* Slow path for 1, 2 and 4 byte accesses.
 	 Test (shadow != 0)
-	      & ((base_addr & 7) + (size_in_bytes - 1)) >= shadow).  */
+	      & ((base_addr & 7) + (real_size_in_bytes - 1)) >= shadow).  */
       gimple_seq seq = NULL;
       gimple shadow_test = build_assign (NE_EXPR, shadow, 0);
       gimple_seq_add_stmt (&seq, shadow_test);
       gimple_seq_add_stmt (&seq, build_assign (BIT_AND_EXPR, base_addr, 7));
       gimple_seq_add_stmt (&seq, build_type_cast (shadow_type,
                                                   gimple_seq_last (seq)));
-      if (size_in_bytes > 1)
+      if (real_size_in_bytes > 1)
         gimple_seq_add_stmt (&seq,
                              build_assign (PLUS_EXPR, gimple_seq_last (seq),
-                                           size_in_bytes - 1));
+					   real_size_in_bytes - 1));
       gimple_seq_add_stmt (&seq, build_assign (GE_EXPR, gimple_seq_last (seq),
                                                shadow));
       gimple_seq_add_stmt (&seq, build_assign (BIT_AND_EXPR, shadow_test,
@@ -1561,6 +1582,39 @@ build_check_stmt (location_t location, t
       t = gimple_assign_lhs (gimple_seq_last (seq));
       gimple_seq_set_location (seq, location);
       gsi_insert_seq_after (&gsi, seq, GSI_CONTINUE_LINKING);
+      /* For weird access sizes, check first and last byte.  */
+      if (real_size_in_bytes != size_in_bytes)
+	{
+	  g = gimple_build_assign_with_ops (PLUS_EXPR,
+					    make_ssa_name (uintptr_type, NULL),
+					    base_addr,
+					    build_int_cst (uintptr_type,
+							   size_in_bytes - 1));
+	  gimple_set_location (g, location);
+	  gsi_insert_after (&gsi, g, GSI_NEW_STMT);
+	  tree base_end_addr = gimple_assign_lhs (g);
+
+	  shadow = build_shadow_mem_access (&gsi, location, base_end_addr,
+					    shadow_ptr_type);
+	  seq = NULL;
+	  shadow_test = build_assign (NE_EXPR, shadow, 0);
+	  gimple_seq_add_stmt (&seq, shadow_test);
+	  gimple_seq_add_stmt (&seq, build_assign (BIT_AND_EXPR,
+						   base_end_addr, 7));
+	  gimple_seq_add_stmt (&seq, build_type_cast (shadow_type,
+						      gimple_seq_last (seq)));
+	  gimple_seq_add_stmt (&seq, build_assign (GE_EXPR,
+						   gimple_seq_last (seq),
+						   shadow));
+	  gimple_seq_add_stmt (&seq, build_assign (BIT_AND_EXPR, shadow_test,
+						   gimple_seq_last (seq)));
+	  gimple_seq_add_stmt (&seq, build_assign (BIT_IOR_EXPR, t,
+						   gimple_seq_last (seq)));
+	  t = gimple_assign_lhs (gimple_seq_last (seq));
+	  gimple_seq_set_location (seq, location);
+	  gsi_insert_seq_after (&gsi, seq, GSI_CONTINUE_LINKING);
+	  sz_arg = build_int_cst (pointer_sized_int_node, size_in_bytes);
+	}
     }
   else
     t = shadow;
@@ -1573,7 +1627,7 @@ build_check_stmt (location_t location, t
   /* Generate call to the run-time library (e.g. __asan_report_load8).  */
   gsi = gsi_start_bb (then_bb);
   g = gimple_build_call (report_error_func (is_store, size_in_bytes),
-			 1, base_addr);
+			 sz_arg ? 2 : 1, base_addr, sz_arg);
   gimple_set_location (g, location);
   gsi_insert_after (&gsi, g, GSI_NEW_STMT);
 
@@ -1611,8 +1665,7 @@ instrument_derefs (gimple_stmt_iterator
     }
 
   size_in_bytes = int_size_in_bytes (type);
-  if ((size_in_bytes & (size_in_bytes - 1)) != 0
-      || (unsigned HOST_WIDE_INT) size_in_bytes - 1 >= 16)
+  if (size_in_bytes <= 0)
     return;
 
   HOST_WIDE_INT bitsize, bitpos;
@@ -1621,7 +1674,8 @@ instrument_derefs (gimple_stmt_iterator
   int volatilep = 0, unsignedp = 0;
   tree inner = get_inner_reference (t, &bitsize, &bitpos, &offset,
 				    &mode, &unsignedp, &volatilep, false);
-  if (bitpos % (size_in_bytes * BITS_PER_UNIT)
+  if (((size_in_bytes & (size_in_bytes - 1)) == 0
+       && (bitpos % (size_in_bytes * BITS_PER_UNIT)))
       || bitsize != size_in_bytes * BITS_PER_UNIT)
     {
       if (TREE_CODE (t) == COMPONENT_REF
@@ -1634,6 +1688,8 @@ instrument_derefs (gimple_stmt_iterator
 	}
       return;
     }
+  if (bitpos % BITS_PER_UNIT)
+    return;
 
   if (TREE_CODE (inner) == VAR_DECL
       && offset == NULL_TREE


	Jakub

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

On Thu, May 22, 2014 at 6:07 PM, Jakub Jelinek <jakub@redhat.com> wrote:
> On Thu, May 22, 2014 at 04:06:21PM +0400, Konstantin Serebryany wrote:
>> Not really recently... (Feb 2013)
>> http://llvm.org/viewvc/llvm-project?rev=175507&view=rev
>
> Ah, wasn't aware of that.
>
> Here is (so far not bootstrapped/regtested) patch for the GCC side.
>
> Notes:
> 1) the cases where we e.g. for various stringops perform first and
>    last address check and use separate __asan_report_*1 for reporting
>    that should probably be converted to use this __asan_report_*_n too

Note that in clang we completely removed the code that instruments
srtingops (we had memset/memcpy/memmove).
Instead we replace memset with __asan_memset (ditto for
memcpy/memmove) so that it does not get inlined later.
This simplifies the code and allows to properly analyze all memset/etc
calls, not just check the first and the last bytes.


> 2) it doesn't still deal with unaligned power of two accesses properly,
>    but neither does llvm (at least not 3.4).  Am not talking about
>    undefined behavior cases where the compiler isn't told the access
>    is misaligned, but e.g. when accessing struct S { int x; }
>    __attribute__((packed)) and similar (or aligned(1)).  Supposedly
>    we could force __asan_report_*_n for that case too, because
>    normal wider check assumes it is aligned

Yep, we don't do it.

> 3) there is still a failure for -m32:
> FAIL: g++.dg/asan/asan_test.C  -O2  AddressSanitizer_UAF_long_double Ident(p)[12] = 0 output pattern test
> Output should match: WRITE of size 1[06]
> FAIL: g++.dg/asan/asan_test.C  -O2  AddressSanitizer_UAF_long_double Ident(p)[0] = Ident(p)[12] output pattern test
> Output should match: READ of size 1[06]
>    That sounds like something to fix in upstream, it should allow also size
>    12 which is the size of long double on ia32 (16 bytes on x86_64),
>    thus 1[026].  Kostya, can you please change it, I'll then apply it
>    to the testsuite patch too?
Like this?

--- lib/asan/tests/asan_test.cc (revision 209430)
+++ lib/asan/tests/asan_test.cc (working copy)
@@ -183,8 +183,8 @@
 TEST(AddressSanitizer, UAF_long_double) {
   if (sizeof(long double) == sizeof(double)) return;
   long double *p = Ident(new long double[10]);
-  EXPECT_DEATH(Ident(p)[12] = 0, "WRITE of size 1[06]");
-  EXPECT_DEATH(Ident(p)[0] = Ident(p)[12], "READ of size 1[06]");
+  EXPECT_DEATH(Ident(p)[12] = 0, "WRITE of size 1[026]");
+  EXPECT_DEATH(Ident(p)[0] = Ident(p)[12], "READ of size 1[026]");
   delete [] Ident(p);
 }


> As mentioned earlier, ubsan has similar
>    problem where it doesn't recognize float bitsize 96 (but unlike this
>    case where clang+llvm pass in 10 bytes, which is what is actually
>    accessed by hw if using i387 stack, but not if using other means of
>    copying it, in ubsan case clang also passes in bitsize 96 that ubsan
>    doesn't handle).

Yea, this long double business is rather confusing to me...

--kcc

>
> I'll bootstrap/regtest this later today.
>
> 2014-05-22  Jakub Jelinek  <jakub@redhat.com>
>
>         * sanitizer.def (BUILT_IN_ASAN_REPORT_LOAD_N,
>         BUILT_IN_ASAN_REPORT_STORE_N): New.
>         * asan.c (struct asan_mem_ref): Change access_size type to
>         HOST_WIDE_INT.
>         (asan_mem_ref_init, asan_mem_ref_new, get_mem_refs_of_builtin_call,
>         update_mem_ref_hash_table): Likewise.
>         (asan_mem_ref_hasher::hash): Hash in a HWI.
>         (report_error_func): Change size_in_bytes argument to HWI.
>         Use *_N builtins if size_in_bytes is larger than 16 or not power of
>         two.
>         (build_shadow_mem_access): New function.
>         (build_check_stmt): Use it.  Change size_in_bytes argument to HWI.
>         Handle size_in_bytes not power of two or larger than 16.
>         (instrument_derefs): Don't give up if size_in_bytes is not
>         power of two or is larger than 16.
>
> --- gcc/sanitizer.def.jj        2014-05-22 10:18:01.000000000 +0200
> +++ gcc/sanitizer.def   2014-05-22 14:13:27.859513839 +0200
> @@ -41,6 +41,9 @@ DEF_SANITIZER_BUILTIN(BUILT_IN_ASAN_REPO
>                       BT_FN_VOID_PTR, ATTR_TMPURE_NORETURN_NOTHROW_LEAF_LIST)
>  DEF_SANITIZER_BUILTIN(BUILT_IN_ASAN_REPORT_LOAD16, "__asan_report_load16",
>                       BT_FN_VOID_PTR, ATTR_TMPURE_NORETURN_NOTHROW_LEAF_LIST)
> +DEF_SANITIZER_BUILTIN(BUILT_IN_ASAN_REPORT_LOAD_N, "__asan_report_load_n",
> +                     BT_FN_VOID_PTR_PTRMODE,
> +                     ATTR_TMPURE_NORETURN_NOTHROW_LEAF_LIST)
>  DEF_SANITIZER_BUILTIN(BUILT_IN_ASAN_REPORT_STORE1, "__asan_report_store1",
>                       BT_FN_VOID_PTR, ATTR_TMPURE_NORETURN_NOTHROW_LEAF_LIST)
>  DEF_SANITIZER_BUILTIN(BUILT_IN_ASAN_REPORT_STORE2, "__asan_report_store2",
> @@ -51,6 +54,9 @@ DEF_SANITIZER_BUILTIN(BUILT_IN_ASAN_REPO
>                       BT_FN_VOID_PTR, ATTR_TMPURE_NORETURN_NOTHROW_LEAF_LIST)
>  DEF_SANITIZER_BUILTIN(BUILT_IN_ASAN_REPORT_STORE16, "__asan_report_store16",
>                       BT_FN_VOID_PTR, ATTR_TMPURE_NORETURN_NOTHROW_LEAF_LIST)
> +DEF_SANITIZER_BUILTIN(BUILT_IN_ASAN_REPORT_STORE_N, "__asan_report_store_n",
> +                     BT_FN_VOID_PTR_PTRMODE,
> +                     ATTR_TMPURE_NORETURN_NOTHROW_LEAF_LIST)
>  DEF_SANITIZER_BUILTIN(BUILT_IN_ASAN_REGISTER_GLOBALS,
>                       "__asan_register_globals",
>                       BT_FN_VOID_PTR_PTRMODE, ATTR_NOTHROW_LEAF_LIST)
> --- gcc/asan.c.jj       2014-05-11 22:21:23.000000000 +0200
> +++ gcc/asan.c  2014-05-22 15:28:30.125998730 +0200
> @@ -251,8 +251,8 @@ struct asan_mem_ref
>    /* The expression of the beginning of the memory region.  */
>    tree start;
>
> -  /* The size of the access (can be 1, 2, 4, 8, 16 for now).  */
> -  char access_size;
> +  /* The size of the access.  */
> +  HOST_WIDE_INT access_size;
>  };
>
>  static alloc_pool asan_mem_ref_alloc_pool;
> @@ -274,7 +274,7 @@ asan_mem_ref_get_alloc_pool ()
>  /* Initializes an instance of asan_mem_ref.  */
>
>  static void
> -asan_mem_ref_init (asan_mem_ref *ref, tree start, char access_size)
> +asan_mem_ref_init (asan_mem_ref *ref, tree start, HOST_WIDE_INT access_size)
>  {
>    ref->start = start;
>    ref->access_size = access_size;
> @@ -287,7 +287,7 @@ asan_mem_ref_init (asan_mem_ref *ref, tr
>     access to the referenced memory.  */
>
>  static asan_mem_ref*
> -asan_mem_ref_new (tree start, char access_size)
> +asan_mem_ref_new (tree start, HOST_WIDE_INT access_size)
>  {
>    asan_mem_ref *ref =
>      (asan_mem_ref *) pool_alloc (asan_mem_ref_get_alloc_pool ());
> @@ -334,7 +334,7 @@ inline hashval_t
>  asan_mem_ref_hasher::hash (const asan_mem_ref *mem_ref)
>  {
>    hashval_t h = iterative_hash_expr (mem_ref->start, 0);
> -  h = iterative_hash_hashval_t (h, mem_ref->access_size);
> +  h = iterative_hash_host_wide_int (mem_ref->access_size, h);
>    return h;
>  }
>
> @@ -392,7 +392,7 @@ free_mem_ref_resources ()
>  /* Return true iff the memory reference REF has been instrumented.  */
>
>  static bool
> -has_mem_ref_been_instrumented (tree ref, char access_size)
> +has_mem_ref_been_instrumented (tree ref, HOST_WIDE_INT access_size)
>  {
>    asan_mem_ref r;
>    asan_mem_ref_init (&r, ref, access_size);
> @@ -480,7 +480,7 @@ get_mem_refs_of_builtin_call (const gimp
>    tree source0 = NULL_TREE, source1 = NULL_TREE,
>      dest = NULL_TREE, len = NULL_TREE;
>    bool is_store = true, got_reference_p = false;
> -  char access_size = 1;
> +  HOST_WIDE_INT access_size = 1;
>
>    switch (DECL_FUNCTION_CODE (callee))
>      {
> @@ -842,7 +842,7 @@ has_stmt_been_instrumented_p (gimple stm
>  /*  Insert a memory reference into the hash table.  */
>
>  static void
> -update_mem_ref_hash_table (tree ref, char access_size)
> +update_mem_ref_hash_table (tree ref, HOST_WIDE_INT access_size)
>  {
>    hash_table <asan_mem_ref_hasher> ht = get_mem_ref_hash_table ();
>
> @@ -1315,20 +1315,22 @@ asan_protect_global (tree decl)
>    return true;
>  }
>
> -/* Construct a function tree for __asan_report_{load,store}{1,2,4,8,16}.
> -   IS_STORE is either 1 (for a store) or 0 (for a load).
> -   SIZE_IN_BYTES is one of 1, 2, 4, 8, 16.  */
> +/* Construct a function tree for __asan_report_{load,store}{1,2,4,8,16,_n}.
> +   IS_STORE is either 1 (for a store) or 0 (for a load).  */
>
>  static tree
> -report_error_func (bool is_store, int size_in_bytes)
> +report_error_func (bool is_store, HOST_WIDE_INT size_in_bytes)
>  {
> -  static enum built_in_function report[2][5]
> +  static enum built_in_function report[2][6]
>      = { { BUILT_IN_ASAN_REPORT_LOAD1, BUILT_IN_ASAN_REPORT_LOAD2,
>           BUILT_IN_ASAN_REPORT_LOAD4, BUILT_IN_ASAN_REPORT_LOAD8,
> -         BUILT_IN_ASAN_REPORT_LOAD16 },
> +         BUILT_IN_ASAN_REPORT_LOAD16, BUILT_IN_ASAN_REPORT_LOAD_N },
>         { BUILT_IN_ASAN_REPORT_STORE1, BUILT_IN_ASAN_REPORT_STORE2,
>           BUILT_IN_ASAN_REPORT_STORE4, BUILT_IN_ASAN_REPORT_STORE8,
> -         BUILT_IN_ASAN_REPORT_STORE16 } };
> +         BUILT_IN_ASAN_REPORT_STORE16, BUILT_IN_ASAN_REPORT_STORE_N } };
> +  if ((size_in_bytes & (size_in_bytes - 1)) != 0
> +      || size_in_bytes > 16)
> +    return builtin_decl_implicit (report[is_store][5]);
>    return builtin_decl_implicit (report[is_store][exact_log2 (size_in_bytes)]);
>  }
>
> @@ -1450,6 +1452,47 @@ insert_if_then_before_iter (gimple cond,
>    gsi_insert_after (&cond_insert_point, cond, GSI_NEW_STMT);
>  }
>
> +/* Build
> +   (base_addr >> ASAN_SHADOW_SHIFT) + targetm.asan_shadow_offset ().  */
> +
> +static tree
> +build_shadow_mem_access (gimple_stmt_iterator *gsi, location_t location,
> +                        tree base_addr, tree shadow_ptr_type)
> +{
> +  tree t, uintptr_type = TREE_TYPE (base_addr);
> +  tree shadow_type = TREE_TYPE (shadow_ptr_type);
> +  gimple g;
> +
> +  t = build_int_cst (uintptr_type, ASAN_SHADOW_SHIFT);
> +  g = gimple_build_assign_with_ops (RSHIFT_EXPR,
> +                                   make_ssa_name (uintptr_type, NULL),
> +                                   base_addr, t);
> +  gimple_set_location (g, location);
> +  gsi_insert_after (gsi, g, GSI_NEW_STMT);
> +
> +  t = build_int_cst (uintptr_type, targetm.asan_shadow_offset ());
> +  g = gimple_build_assign_with_ops (PLUS_EXPR,
> +                                   make_ssa_name (uintptr_type, NULL),
> +                                   gimple_assign_lhs (g), t);
> +  gimple_set_location (g, location);
> +  gsi_insert_after (gsi, g, GSI_NEW_STMT);
> +
> +  g = gimple_build_assign_with_ops (NOP_EXPR,
> +                                   make_ssa_name (shadow_ptr_type, NULL),
> +                                   gimple_assign_lhs (g), NULL_TREE);
> +  gimple_set_location (g, location);
> +  gsi_insert_after (gsi, g, GSI_NEW_STMT);
> +
> +  t = build2 (MEM_REF, shadow_type, gimple_assign_lhs (g),
> +             build_int_cst (shadow_ptr_type, 0));
> +  g = gimple_build_assign_with_ops (MEM_REF,
> +                                   make_ssa_name (shadow_type, NULL),
> +                                   t, NULL_TREE);
> +  gimple_set_location (g, location);
> +  gsi_insert_after (gsi, g, GSI_NEW_STMT);
> +  return gimple_assign_lhs (g);
> +}
> +
>  /* Instrument the memory access instruction BASE.  Insert new
>     statements before or after ITER.
>
> @@ -1457,8 +1500,7 @@ insert_if_then_before_iter (gimple cond,
>     SSA_NAME, or a non-SSA expression.  LOCATION is the source code
>     location.  IS_STORE is TRUE for a store, FALSE for a load.
>     BEFORE_P is TRUE for inserting the instrumentation code before
> -   ITER, FALSE for inserting it after ITER.  SIZE_IN_BYTES is one of
> -   1, 2, 4, 8, 16.
> +   ITER, FALSE for inserting it after ITER.
>
>     If BEFORE_P is TRUE, *ITER is arranged to still point to the
>     statement it was pointing to prior to calling this function,
> @@ -1466,7 +1508,7 @@ insert_if_then_before_iter (gimple cond,
>
>  static void
>  build_check_stmt (location_t location, tree base, gimple_stmt_iterator *iter,
> -                 bool before_p, bool is_store, int size_in_bytes)
> +                 bool before_p, bool is_store, HOST_WIDE_INT size_in_bytes)
>  {
>    gimple_stmt_iterator gsi;
>    basic_block then_bb, else_bb;
> @@ -1477,6 +1519,12 @@ build_check_stmt (location_t location, t
>    tree uintptr_type
>      = build_nonstandard_integer_type (TYPE_PRECISION (TREE_TYPE (base)), 1);
>    tree base_ssa = base;
> +  HOST_WIDE_INT real_size_in_bytes = size_in_bytes;
> +  tree sz_arg = NULL_TREE;
> +
> +  if ((size_in_bytes & (size_in_bytes - 1)) != 0
> +      || size_in_bytes > 16)
> +    real_size_in_bytes = 1;
>
>    /* Get an iterator on the point where we can add the condition
>       statement for the instrumentation.  */
> @@ -1509,51 +1557,24 @@ build_check_stmt (location_t location, t
>
>    /* Build
>       (base_addr >> ASAN_SHADOW_SHIFT) + targetm.asan_shadow_offset ().  */
> +  shadow = build_shadow_mem_access (&gsi, location, base_addr,
> +                                   shadow_ptr_type);
>
> -  t = build_int_cst (uintptr_type, ASAN_SHADOW_SHIFT);
> -  g = gimple_build_assign_with_ops (RSHIFT_EXPR,
> -                                   make_ssa_name (uintptr_type, NULL),
> -                                   base_addr, t);
> -  gimple_set_location (g, location);
> -  gsi_insert_after (&gsi, g, GSI_NEW_STMT);
> -
> -  t = build_int_cst (uintptr_type, targetm.asan_shadow_offset ());
> -  g = gimple_build_assign_with_ops (PLUS_EXPR,
> -                                   make_ssa_name (uintptr_type, NULL),
> -                                   gimple_assign_lhs (g), t);
> -  gimple_set_location (g, location);
> -  gsi_insert_after (&gsi, g, GSI_NEW_STMT);
> -
> -  g = gimple_build_assign_with_ops (NOP_EXPR,
> -                                   make_ssa_name (shadow_ptr_type, NULL),
> -                                   gimple_assign_lhs (g), NULL_TREE);
> -  gimple_set_location (g, location);
> -  gsi_insert_after (&gsi, g, GSI_NEW_STMT);
> -
> -  t = build2 (MEM_REF, shadow_type, gimple_assign_lhs (g),
> -             build_int_cst (shadow_ptr_type, 0));
> -  g = gimple_build_assign_with_ops (MEM_REF,
> -                                   make_ssa_name (shadow_type, NULL),
> -                                   t, NULL_TREE);
> -  gimple_set_location (g, location);
> -  gsi_insert_after (&gsi, g, GSI_NEW_STMT);
> -  shadow = gimple_assign_lhs (g);
> -
> -  if (size_in_bytes < 8)
> +  if (real_size_in_bytes < 8)
>      {
>        /* Slow path for 1, 2 and 4 byte accesses.
>          Test (shadow != 0)
> -             & ((base_addr & 7) + (size_in_bytes - 1)) >= shadow).  */
> +             & ((base_addr & 7) + (real_size_in_bytes - 1)) >= shadow).  */
>        gimple_seq seq = NULL;
>        gimple shadow_test = build_assign (NE_EXPR, shadow, 0);
>        gimple_seq_add_stmt (&seq, shadow_test);
>        gimple_seq_add_stmt (&seq, build_assign (BIT_AND_EXPR, base_addr, 7));
>        gimple_seq_add_stmt (&seq, build_type_cast (shadow_type,
>                                                    gimple_seq_last (seq)));
> -      if (size_in_bytes > 1)
> +      if (real_size_in_bytes > 1)
>          gimple_seq_add_stmt (&seq,
>                               build_assign (PLUS_EXPR, gimple_seq_last (seq),
> -                                           size_in_bytes - 1));
> +                                          real_size_in_bytes - 1));
>        gimple_seq_add_stmt (&seq, build_assign (GE_EXPR, gimple_seq_last (seq),
>                                                 shadow));
>        gimple_seq_add_stmt (&seq, build_assign (BIT_AND_EXPR, shadow_test,
> @@ -1561,6 +1582,39 @@ build_check_stmt (location_t location, t
>        t = gimple_assign_lhs (gimple_seq_last (seq));
>        gimple_seq_set_location (seq, location);
>        gsi_insert_seq_after (&gsi, seq, GSI_CONTINUE_LINKING);
> +      /* For weird access sizes, check first and last byte.  */
> +      if (real_size_in_bytes != size_in_bytes)
> +       {
> +         g = gimple_build_assign_with_ops (PLUS_EXPR,
> +                                           make_ssa_name (uintptr_type, NULL),
> +                                           base_addr,
> +                                           build_int_cst (uintptr_type,
> +                                                          size_in_bytes - 1));
> +         gimple_set_location (g, location);
> +         gsi_insert_after (&gsi, g, GSI_NEW_STMT);
> +         tree base_end_addr = gimple_assign_lhs (g);
> +
> +         shadow = build_shadow_mem_access (&gsi, location, base_end_addr,
> +                                           shadow_ptr_type);
> +         seq = NULL;
> +         shadow_test = build_assign (NE_EXPR, shadow, 0);
> +         gimple_seq_add_stmt (&seq, shadow_test);
> +         gimple_seq_add_stmt (&seq, build_assign (BIT_AND_EXPR,
> +                                                  base_end_addr, 7));
> +         gimple_seq_add_stmt (&seq, build_type_cast (shadow_type,
> +                                                     gimple_seq_last (seq)));
> +         gimple_seq_add_stmt (&seq, build_assign (GE_EXPR,
> +                                                  gimple_seq_last (seq),
> +                                                  shadow));
> +         gimple_seq_add_stmt (&seq, build_assign (BIT_AND_EXPR, shadow_test,
> +                                                  gimple_seq_last (seq)));
> +         gimple_seq_add_stmt (&seq, build_assign (BIT_IOR_EXPR, t,
> +                                                  gimple_seq_last (seq)));
> +         t = gimple_assign_lhs (gimple_seq_last (seq));
> +         gimple_seq_set_location (seq, location);
> +         gsi_insert_seq_after (&gsi, seq, GSI_CONTINUE_LINKING);
> +         sz_arg = build_int_cst (pointer_sized_int_node, size_in_bytes);
> +       }
>      }
>    else
>      t = shadow;
> @@ -1573,7 +1627,7 @@ build_check_stmt (location_t location, t
>    /* Generate call to the run-time library (e.g. __asan_report_load8).  */
>    gsi = gsi_start_bb (then_bb);
>    g = gimple_build_call (report_error_func (is_store, size_in_bytes),
> -                        1, base_addr);
> +                        sz_arg ? 2 : 1, base_addr, sz_arg);
>    gimple_set_location (g, location);
>    gsi_insert_after (&gsi, g, GSI_NEW_STMT);
>
> @@ -1611,8 +1665,7 @@ instrument_derefs (gimple_stmt_iterator
>      }
>
>    size_in_bytes = int_size_in_bytes (type);
> -  if ((size_in_bytes & (size_in_bytes - 1)) != 0
> -      || (unsigned HOST_WIDE_INT) size_in_bytes - 1 >= 16)
> +  if (size_in_bytes <= 0)
>      return;
>
>    HOST_WIDE_INT bitsize, bitpos;
> @@ -1621,7 +1674,8 @@ instrument_derefs (gimple_stmt_iterator
>    int volatilep = 0, unsignedp = 0;
>    tree inner = get_inner_reference (t, &bitsize, &bitpos, &offset,
>                                     &mode, &unsignedp, &volatilep, false);
> -  if (bitpos % (size_in_bytes * BITS_PER_UNIT)
> +  if (((size_in_bytes & (size_in_bytes - 1)) == 0
> +       && (bitpos % (size_in_bytes * BITS_PER_UNIT)))
>        || bitsize != size_in_bytes * BITS_PER_UNIT)
>      {
>        if (TREE_CODE (t) == COMPONENT_REF
> @@ -1634,6 +1688,8 @@ instrument_derefs (gimple_stmt_iterator
>         }
>        return;
>      }
> +  if (bitpos % BITS_PER_UNIT)
> +    return;
>
>    if (TREE_CODE (inner) == VAR_DECL
>        && offset == NULL_TREE
>
>
>         Jakub

Re: libsanitizer merge from upstream r208536

[Jakub Jelinek]

On Thu, May 22, 2014 at 06:34:22PM +0400, Konstantin Serebryany wrote:
> > Notes:
> > 1) the cases where we e.g. for various stringops perform first and
> >    last address check and use separate __asan_report_*1 for reporting
> >    that should probably be converted to use this __asan_report_*_n too
> 
> Note that in clang we completely removed the code that instruments
> srtingops (we had memset/memcpy/memmove).
> Instead we replace memset with __asan_memset (ditto for
> memcpy/memmove) so that it does not get inlined later.
> This simplifies the code and allows to properly analyze all memset/etc
> calls, not just check the first and the last bytes.

Food for thought, yes.

> > 3) there is still a failure for -m32:
> > FAIL: g++.dg/asan/asan_test.C  -O2  AddressSanitizer_UAF_long_double Ident(p)[12] = 0 output pattern test
> > Output should match: WRITE of size 1[06]
> > FAIL: g++.dg/asan/asan_test.C  -O2  AddressSanitizer_UAF_long_double Ident(p)[0] = Ident(p)[12] output pattern test
> > Output should match: READ of size 1[06]
> >    That sounds like something to fix in upstream, it should allow also size
> >    12 which is the size of long double on ia32 (16 bytes on x86_64),
> >    thus 1[026].  Kostya, can you please change it, I'll then apply it
> >    to the testsuite patch too?
> Like this?
> 
> --- lib/asan/tests/asan_test.cc (revision 209430)
> +++ lib/asan/tests/asan_test.cc (working copy)
> @@ -183,8 +183,8 @@
>  TEST(AddressSanitizer, UAF_long_double) {
>    if (sizeof(long double) == sizeof(double)) return;
>    long double *p = Ident(new long double[10]);
> -  EXPECT_DEATH(Ident(p)[12] = 0, "WRITE of size 1[06]");
> -  EXPECT_DEATH(Ident(p)[0] = Ident(p)[12], "READ of size 1[06]");
> +  EXPECT_DEATH(Ident(p)[12] = 0, "WRITE of size 1[026]");
> +  EXPECT_DEATH(Ident(p)[0] = Ident(p)[12], "READ of size 1[026]");
>    delete [] Ident(p);
>  }
> 

Yep, exactly.

> > As mentioned earlier, ubsan has similar
> >    problem where it doesn't recognize float bitsize 96 (but unlike this
> >    case where clang+llvm pass in 10 bytes, which is what is actually
> >    accessed by hw if using i387 stack, but not if using other means of
> >    copying it, in ubsan case clang also passes in bitsize 96 that ubsan
> >    doesn't handle).
> 
> Yea, this long double business is rather confusing to me...

It is actually even more complicated than that, on x86_64 GCC supports
long double (== __float80) which has 128 bits (64-bit) or 96 bits (32-bit),
and is the 80-bit extended double format, plus also __float128, which
is a software implemented IEEE 754 quad long double (also 128 bit).
Now, ubsan handles all of 80 and 128 bit (and should 96 bit) floats
as long double, so it is unclear how to tell libubsan about __float128
type (because bitsize 128 is already used for something else),
plus in order to actually print __float128 one would need to link in
libquadmath, which is probably not desirable.  So, ideally there should
be some way to tell the library we have IEEE 754 quad __float128 and
for now the library should just print that the number is not printable.
Ditto for _Decimal{32,64,128}, which should probably just be a different
kind from float, and again would require additional library to support
printing into strings, so perhaps it could be for now just printed
as something unprintable.

	Jakub

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

>
>> > 3) there is still a failure for -m32:
>> > FAIL: g++.dg/asan/asan_test.C  -O2  AddressSanitizer_UAF_long_double Ident(p)[12] = 0 output pattern test
>> > Output should match: WRITE of size 1[06]
>> > FAIL: g++.dg/asan/asan_test.C  -O2  AddressSanitizer_UAF_long_double Ident(p)[0] = Ident(p)[12] output pattern test
>> > Output should match: READ of size 1[06]
>> >    That sounds like something to fix in upstream, it should allow also size
>> >    12 which is the size of long double on ia32 (16 bytes on x86_64),
>> >    thus 1[026].  Kostya, can you please change it, I'll then apply it
>> >    to the testsuite patch too?
>> Like this?
>>
>> --- lib/asan/tests/asan_test.cc (revision 209430)
>> +++ lib/asan/tests/asan_test.cc (working copy)
>> @@ -183,8 +183,8 @@
>>  TEST(AddressSanitizer, UAF_long_double) {
>>    if (sizeof(long double) == sizeof(double)) return;
>>    long double *p = Ident(new long double[10]);
>> -  EXPECT_DEATH(Ident(p)[12] = 0, "WRITE of size 1[06]");
>> -  EXPECT_DEATH(Ident(p)[0] = Ident(p)[12], "READ of size 1[06]");
>> +  EXPECT_DEATH(Ident(p)[12] = 0, "WRITE of size 1[026]");
>> +  EXPECT_DEATH(Ident(p)[0] = Ident(p)[12], "READ of size 1[026]");
>>    delete [] Ident(p);
>>  }
>>
>
> Yep, exactly.

done, r209445.

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

>> 2) it doesn't still deal with unaligned power of two accesses properly,
>>    but neither does llvm (at least not 3.4).  Am not talking about
>>    undefined behavior cases where the compiler isn't told the access
>>    is misaligned, but e.g. when accessing struct S { int x; }
>>    __attribute__((packed)) and similar (or aligned(1)).  Supposedly
>>    we could force __asan_report_*_n for that case too, because
>>    normal wider check assumes it is aligned
>
> Yep, we don't do it.
Now we do: http://llvm.org/viewvc/llvm-project?rev=209508&view=rev

Re: libsanitizer merge from upstream r208536

[Jakub Jelinek]

On Fri, May 23, 2014 at 04:11:48PM +0400, Konstantin Serebryany wrote:
> >> 2) it doesn't still deal with unaligned power of two accesses properly,
> >>    but neither does llvm (at least not 3.4).  Am not talking about
> >>    undefined behavior cases where the compiler isn't told the access
> >>    is misaligned, but e.g. when accessing struct S { int x; }
> >>    __attribute__((packed)) and similar (or aligned(1)).  Supposedly
> >>    we could force __asan_report_*_n for that case too, because
> >>    normal wider check assumes it is aligned
> >
> > Yep, we don't do it.
> Now we do: http://llvm.org/viewvc/llvm-project?rev=209508&view=rev

Here is the GCC side of the thing, ok for trunk if it bootstraps/regtests?
(on top of the earlier posted two patches).

Note, I think some work is needed on the library side, 
ERROR: AddressSanitizer: unknown-crash on address 0xffc439cf at pc 0x804898e bp 0xffc438d8 sp 0xffc438cc
READ of size 4 at 0xffc439cf thread T0
    #0 0x804898d in foo /usr/src/gcc/gcc/testsuite/c-c++-common/asan/misalign-1.c:10
    #1 0x8048746 in main /usr/src/gcc/gcc/testsuite/c-c++-common/asan/misalign-1.c:34
    #2 0x49e39b72 in __libc_start_main (/lib/libc.so.6+0x49e39b72)
    #3 0x8048848 (/usr/src/gcc/obj2/gcc/testsuite/gcc/misalign-1.exe+0x8048848)

Address 0xffc439cf is located in stack of thread T0 at offset 175 in frame
    #0 0x804868f in main /usr/src/gcc/gcc/testsuite/c-c++-common/asan/misalign-1.c:27

  This frame has 3 object(s):
    [32, 36) 'v'
    [96, 100) 'w'
    [160, 176) 't' <== Memory access at offset 175 partially overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: unknown-crash /usr/src/gcc/gcc/testsuite/c-c++-common/asan/misalign-1.c:10 foo
Shadow bytes around the buggy address:
  0x3ff886e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x3ff886f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x3ff88700: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x3ff88710: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x3ff88720: 00 00 00 00 f1 f1 f1 f1 04 f4 f4 f4 f2 f2 f2 f2
=>0x3ff88730: 04 f4 f4 f4 f2 f2 f2 f2 00[00]f4 f4 f3 f3 f3 f3
  0x3ff88740: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x3ff88750: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x3ff88760: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x3ff88770: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x3ff88780: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  ASan internal:           fe

is just too ugly (I mean, it shouldn't print unknown-crash).
It is true that the first byte of the __asan_report_load_n range
corresponds to shadow byte 0, but for _n you should either check
it for all bytes in that range, or at least the first and last byte
(which would correspond to what the caller of __asan_report_*_n
actually does right now).

2014-05-23  Jakub Jelinek  <jakub@redhat.com>

	* asan.c (report_error_func): Add SLOW_P argument, use
	BUILT_IN_ASAN_*_N if set.
	(build_check_stmt): Likewise.
	(instrument_derefs): If T has insufficient alignment,
	force same handling as for odd sizes.

	* c-c++-common/asan/misalign-1.c: New test.
	* c-c++-common/asan/misalign-2.c: New test.

--- gcc/asan.c.jj	2014-05-23 17:17:46.000000000 +0200
+++ gcc/asan.c	2014-05-23 18:14:08.630807014 +0200
@@ -1319,7 +1319,7 @@ asan_protect_global (tree decl)
    IS_STORE is either 1 (for a store) or 0 (for a load).  */
 
 static tree
-report_error_func (bool is_store, HOST_WIDE_INT size_in_bytes)
+report_error_func (bool is_store, HOST_WIDE_INT size_in_bytes, bool slow_p)
 {
   static enum built_in_function report[2][6]
     = { { BUILT_IN_ASAN_REPORT_LOAD1, BUILT_IN_ASAN_REPORT_LOAD2,
@@ -1329,7 +1329,8 @@ report_error_func (bool is_store, HOST_W
 	  BUILT_IN_ASAN_REPORT_STORE4, BUILT_IN_ASAN_REPORT_STORE8,
 	  BUILT_IN_ASAN_REPORT_STORE16, BUILT_IN_ASAN_REPORT_STORE_N } };
   if ((size_in_bytes & (size_in_bytes - 1)) != 0
-      || size_in_bytes > 16)
+      || size_in_bytes > 16
+      || slow_p)
     return builtin_decl_implicit (report[is_store][5]);
   return builtin_decl_implicit (report[is_store][exact_log2 (size_in_bytes)]);
 }
@@ -1508,7 +1509,8 @@ build_shadow_mem_access (gimple_stmt_ite
 
 static void
 build_check_stmt (location_t location, tree base, gimple_stmt_iterator *iter,
-		  bool before_p, bool is_store, HOST_WIDE_INT size_in_bytes)
+		  bool before_p, bool is_store, HOST_WIDE_INT size_in_bytes,
+		  bool slow_p = false)
 {
   gimple_stmt_iterator gsi;
   basic_block then_bb, else_bb;
@@ -1522,9 +1524,15 @@ build_check_stmt (location_t location, t
   HOST_WIDE_INT real_size_in_bytes = size_in_bytes;
   tree sz_arg = NULL_TREE;
 
-  if ((size_in_bytes & (size_in_bytes - 1)) != 0
-      || size_in_bytes > 16)
-    real_size_in_bytes = 1;
+  if (size_in_bytes == 1)
+    slow_p = false;
+  else if ((size_in_bytes & (size_in_bytes - 1)) != 0
+	   || size_in_bytes > 16
+	   || slow_p)
+    {
+      real_size_in_bytes = 1;
+      slow_p = true;
+    }
 
   /* Get an iterator on the point where we can add the condition
      statement for the instrumentation.  */
@@ -1582,8 +1590,8 @@ build_check_stmt (location_t location, t
       t = gimple_assign_lhs (gimple_seq_last (seq));
       gimple_seq_set_location (seq, location);
       gsi_insert_seq_after (&gsi, seq, GSI_CONTINUE_LINKING);
-      /* For weird access sizes, check first and last byte.  */
-      if (real_size_in_bytes != size_in_bytes)
+      /* For weird access sizes or misaligned, check first and last byte.  */
+      if (slow_p)
 	{
 	  g = gimple_build_assign_with_ops (PLUS_EXPR,
 					    make_ssa_name (uintptr_type, NULL),
@@ -1626,7 +1634,7 @@ build_check_stmt (location_t location, t
 
   /* Generate call to the run-time library (e.g. __asan_report_load8).  */
   gsi = gsi_start_bb (then_bb);
-  g = gimple_build_call (report_error_func (is_store, size_in_bytes),
+  g = gimple_build_call (report_error_func (is_store, size_in_bytes, slow_p),
 			 sz_arg ? 2 : 1, base_addr, sz_arg);
   gimple_set_location (g, location);
   gsi_insert_after (&gsi, g, GSI_NEW_STMT);
@@ -1722,8 +1730,31 @@ instrument_derefs (gimple_stmt_iterator
   base = build_fold_addr_expr (t);
   if (!has_mem_ref_been_instrumented (base, size_in_bytes))
     {
+      bool slow_p = false;
+      if (size_in_bytes > 1)
+	{
+	  if ((size_in_bytes & (size_in_bytes - 1)) != 0
+	      || size_in_bytes > 16)
+	    slow_p = true;
+	  else
+	    {
+	      unsigned int align = get_object_alignment (t);
+	      if (align < size_in_bytes * BITS_PER_UNIT)
+		{
+		  /* On non-strict alignment targets, if
+		     16-byte access is just 8-byte aligned,
+		     this will result in misaligned shadow
+		     memory 2 byte load, but otherwise can
+		     be handled using one read.  */
+		  if (size_in_bytes != 16
+		      || STRICT_ALIGNMENT
+		      || align < 8 * BITS_PER_UNIT)
+		    slow_p = true;
+		}
+	    }
+	}
       build_check_stmt (location, base, iter, /*before_p=*/true,
-			is_store, size_in_bytes);
+			is_store, size_in_bytes, slow_p);
       update_mem_ref_hash_table (base, size_in_bytes);
       update_mem_ref_hash_table (t, size_in_bytes);
     }
--- gcc/testsuite/c-c++-common/asan/misalign-1.c.jj	2014-05-23 18:03:40.400842565 +0200
+++ gcc/testsuite/c-c++-common/asan/misalign-1.c	2014-05-23 18:29:42.899264340 +0200
@@ -0,0 +1,42 @@
+/* { dg-do run { target { ilp32 || lp64 } } } */
+/* { dg-options "-O2" } */
+/* { dg-shouldfail "asan" } */
+
+struct S { int i; } __attribute__ ((packed));
+
+__attribute__((noinline, noclone)) int
+foo (struct S *s)
+{
+  return s->i;
+}
+
+__attribute__((noinline, noclone)) int
+bar (int *s)
+{
+  return *s;
+}
+
+__attribute__((noinline, noclone)) struct S
+baz (struct S *s)
+{
+  return *s;
+}
+
+int
+main ()
+{
+  struct T { char a[3]; struct S b[3]; char c; } t;
+  int v = 5;
+  struct S *p = t.b;
+  asm volatile ("" : "+rm" (p));
+  p += 3;
+  if (bar (&v) != 5) __builtin_abort ();
+  volatile int w = foo (p);
+  return 0;
+}
+
+/* { dg-output "ERROR: AddressSanitizer:\[^\n\r]*on address\[^\n\r]*" } */
+/* { dg-output "0x\[0-9a-f\]+ at pc 0x\[0-9a-f\]+ bp 0x\[0-9a-f\]+ sp 0x\[0-9a-f\]+\[^\n\r]*(\n|\r\n|\r)" } */
+/* { dg-output "\[^\n\r]*READ of size 4 at 0x\[0-9a-f\]+ thread T0\[^\n\r]*(\n|\r\n|\r)" } */
+/* { dg-output "    #0 0x\[0-9a-f\]+ (in _*foo(\[^\n\r]*misalign-1.c:10|\[^\n\r]*:0)|\[(\])\[^\n\r]*(\n|\r\n|\r)" } */
+/* { dg-output "    #1 0x\[0-9a-f\]+ (in _*main (\[^\n\r]*misalign-1.c:34|\[^\n\r]*:0)|\[(\]).*(\n|\r\n|\r)" } */
--- gcc/testsuite/c-c++-common/asan/misalign-2.c.jj	2014-05-23 18:07:59.820594400 +0200
+++ gcc/testsuite/c-c++-common/asan/misalign-2.c	2014-05-23 18:29:51.808220759 +0200
@@ -0,0 +1,42 @@
+/* { dg-do run { target { ilp32 || lp64 } } } */
+/* { dg-options "-O2" } */
+/* { dg-shouldfail "asan" } */
+
+struct S { int i; } __attribute__ ((packed));
+
+__attribute__((noinline, noclone)) int
+foo (struct S *s)
+{
+  return s->i;
+}
+
+__attribute__((noinline, noclone)) int
+bar (int *s)
+{
+  return *s;
+}
+
+__attribute__((noinline, noclone)) struct S
+baz (struct S *s)
+{
+  return *s;
+}
+
+int
+main ()
+{
+  struct T { char a[3]; struct S b[3]; char c; } t;
+  int v = 5;
+  struct S *p = t.b;
+  asm volatile ("" : "+rm" (p));
+  p += 3;
+  if (bar (&v) != 5) __builtin_abort ();
+  volatile struct S w = baz (p);
+  return 0;
+}
+
+/* { dg-output "ERROR: AddressSanitizer:\[^\n\r]*on address\[^\n\r]*" } */
+/* { dg-output "0x\[0-9a-f\]+ at pc 0x\[0-9a-f\]+ bp 0x\[0-9a-f\]+ sp 0x\[0-9a-f\]+\[^\n\r]*(\n|\r\n|\r)" } */
+/* { dg-output "\[^\n\r]*READ of size 4 at 0x\[0-9a-f\]+ thread T0\[^\n\r]*(\n|\r\n|\r)" } */
+/* { dg-output "    #0 0x\[0-9a-f\]+ (in _*baz(\[^\n\r]*misalign-2.c:22|\[^\n\r]*:0)|\[(\])\[^\n\r]*(\n|\r\n|\r)" } */
+/* { dg-output "    #1 0x\[0-9a-f\]+ (in _*main (\[^\n\r]*misalign-2.c:34|\[^\n\r]*:0)|\[(\]).*(\n|\r\n|\r)" } */


	Jakub

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

On Fri, May 23, 2014 at 8:45 PM, Jakub Jelinek <jakub@redhat.com> wrote:
> On Fri, May 23, 2014 at 04:11:48PM +0400, Konstantin Serebryany wrote:
>> >> 2) it doesn't still deal with unaligned power of two accesses properly,
>> >>    but neither does llvm (at least not 3.4).  Am not talking about
>> >>    undefined behavior cases where the compiler isn't told the access
>> >>    is misaligned, but e.g. when accessing struct S { int x; }
>> >>    __attribute__((packed)) and similar (or aligned(1)).  Supposedly
>> >>    we could force __asan_report_*_n for that case too, because
>> >>    normal wider check assumes it is aligned
>> >
>> > Yep, we don't do it.
>> Now we do: http://llvm.org/viewvc/llvm-project?rev=209508&view=rev
>
> Here is the GCC side of the thing, ok for trunk if it bootstraps/regtests?
> (on top of the earlier posted two patches).
>
> Note, I think some work is needed on the library side,
> ERROR: AddressSanitizer: unknown-crash on address 0xffc439cf at pc 0x804898e bp 0xffc438d8 sp 0xffc438cc


With clang I get this nice report:

==20989==ERROR: AddressSanitizer: stack-buffer-overflow on address
0x7fffffffdf02 at pc 0x4b20db bp 0x7fffffffdd70 sp 0x7fffffffdd68
READ of size 4 at 0x7fffffffdf02 thread T0
    #0 0x4b20da in foo(S*) /home/kcc/tmp/jakub-unaligned.c:6
    #1 0x4b2744 in main /home/kcc/tmp/jakub-unaligned.c:30
    #2 0x7ffff6bfd76c in __libc_start_main
/build/buildd/eglibc-2.15/csu/libc-start.c:226
    #3 0x4b1e6c in _start (/usr/local/google/kcc/llvm_cmake/a.out+0x4b1e6c)

Address 0x7fffffffdf02 is located in stack of thread T0 at offset 66 in frame
    #0 0x4b24ef in main /home/kcc/tmp/jakub-unaligned.c:23

  This frame has 5 object(s):
    [32, 36) 'retval'
    [48, 64) 't' <== Memory access at offset 66 overflows this variable
    [80, 84) 'v'
    [96, 104) 'p'
    [128, 132) 'w'
HINT: this may be a false positive if your program uses some custom
stack unwind mechanism or swapcontext
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow
/home/kcc/tmp/jakub-unaligned.c:6 foo(S*)
Shadow bytes around the buggy address:
  0x10007fff7b90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10007fff7ba0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10007fff7bb0: 00 00 00 00 00 00 00 00 00 00 00 00 f1 f1 f1 f1
  0x10007fff7bc0: 00 f3 f3 f3 00 00 00 00 00 00 00 00 00 00 00 00
  0x10007fff7bd0: 00 00 00 00 00 00 00 00 f1 f1 f1 f1 04 f2 00 00
=>0x10007fff7be0:[f2]f2 04 f2 00 f2 f2 f2 04 f3 f3 f3 00 00 00 00
  0x10007fff7bf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10007fff7c00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10007fff7c10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10007fff7c20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00



> READ of size 4 at 0xffc439cf thread T0
>     #0 0x804898d in foo /usr/src/gcc/gcc/testsuite/c-c++-common/asan/misalign-1.c:10
>     #1 0x8048746 in main /usr/src/gcc/gcc/testsuite/c-c++-common/asan/misalign-1.c:34
>     #2 0x49e39b72 in __libc_start_main (/lib/libc.so.6+0x49e39b72)
>     #3 0x8048848 (/usr/src/gcc/obj2/gcc/testsuite/gcc/misalign-1.exe+0x8048848)
>
> Address 0xffc439cf is located in stack of thread T0 at offset 175 in frame
>     #0 0x804868f in main /usr/src/gcc/gcc/testsuite/c-c++-common/asan/misalign-1.c:27
>
>   This frame has 3 object(s):
>     [32, 36) 'v'
>     [96, 100) 'w'
>     [160, 176) 't' <== Memory access at offset 175 partially overflows this variable
> HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
>       (longjmp and C++ exceptions *are* supported)
> SUMMARY: AddressSanitizer: unknown-crash /usr/src/gcc/gcc/testsuite/c-c++-common/asan/misalign-1.c:10 foo
> Shadow bytes around the buggy address:
>   0x3ff886e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>   0x3ff886f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>   0x3ff88700: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>   0x3ff88710: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>   0x3ff88720: 00 00 00 00 f1 f1 f1 f1 04 f4 f4 f4 f2 f2 f2 f2
> =>0x3ff88730: 04 f4 f4 f4 f2 f2 f2 f2 00[00]f4 f4 f3 f3 f3 f3
>   0x3ff88740: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>   0x3ff88750: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>   0x3ff88760: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>   0x3ff88770: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>   0x3ff88780: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> Shadow byte legend (one shadow byte represents 8 application bytes):
>   Addressable:           00
>   Partially addressable: 01 02 03 04 05 06 07
>   Heap left redzone:       fa
>   Heap right redzone:      fb
>   Freed heap region:       fd
>   Stack left redzone:      f1
>   Stack mid redzone:       f2
>   Stack right redzone:     f3
>   Stack partial redzone:   f4
>   Stack after return:      f5
>   Stack use after scope:   f8
>   Global redzone:          f9
>   Global init order:       f6
>   Poisoned by user:        f7
>   Container overflow:      fc
>   ASan internal:           fe
>
> is just too ugly (I mean, it shouldn't print unknown-crash).
> It is true that the first byte of the __asan_report_load_n range
> corresponds to shadow byte 0, but for _n you should either check
> it for all bytes in that range, or at least the first and last byte
> (which would correspond to what the caller of __asan_report_*_n
> actually does right now).
>
> 2014-05-23  Jakub Jelinek  <jakub@redhat.com>
>
>         * asan.c (report_error_func): Add SLOW_P argument, use
>         BUILT_IN_ASAN_*_N if set.
>         (build_check_stmt): Likewise.
>         (instrument_derefs): If T has insufficient alignment,
>         force same handling as for odd sizes.
>
>         * c-c++-common/asan/misalign-1.c: New test.
>         * c-c++-common/asan/misalign-2.c: New test.
>
> --- gcc/asan.c.jj       2014-05-23 17:17:46.000000000 +0200
> +++ gcc/asan.c  2014-05-23 18:14:08.630807014 +0200
> @@ -1319,7 +1319,7 @@ asan_protect_global (tree decl)
>     IS_STORE is either 1 (for a store) or 0 (for a load).  */
>
>  static tree
> -report_error_func (bool is_store, HOST_WIDE_INT size_in_bytes)
> +report_error_func (bool is_store, HOST_WIDE_INT size_in_bytes, bool slow_p)
>  {
>    static enum built_in_function report[2][6]
>      = { { BUILT_IN_ASAN_REPORT_LOAD1, BUILT_IN_ASAN_REPORT_LOAD2,
> @@ -1329,7 +1329,8 @@ report_error_func (bool is_store, HOST_W
>           BUILT_IN_ASAN_REPORT_STORE4, BUILT_IN_ASAN_REPORT_STORE8,
>           BUILT_IN_ASAN_REPORT_STORE16, BUILT_IN_ASAN_REPORT_STORE_N } };
>    if ((size_in_bytes & (size_in_bytes - 1)) != 0
> -      || size_in_bytes > 16)
> +      || size_in_bytes > 16
> +      || slow_p)
>      return builtin_decl_implicit (report[is_store][5]);
>    return builtin_decl_implicit (report[is_store][exact_log2 (size_in_bytes)]);
>  }
> @@ -1508,7 +1509,8 @@ build_shadow_mem_access (gimple_stmt_ite
>
>  static void
>  build_check_stmt (location_t location, tree base, gimple_stmt_iterator *iter,
> -                 bool before_p, bool is_store, HOST_WIDE_INT size_in_bytes)
> +                 bool before_p, bool is_store, HOST_WIDE_INT size_in_bytes,
> +                 bool slow_p = false)
>  {
>    gimple_stmt_iterator gsi;
>    basic_block then_bb, else_bb;
> @@ -1522,9 +1524,15 @@ build_check_stmt (location_t location, t
>    HOST_WIDE_INT real_size_in_bytes = size_in_bytes;
>    tree sz_arg = NULL_TREE;
>
> -  if ((size_in_bytes & (size_in_bytes - 1)) != 0
> -      || size_in_bytes > 16)
> -    real_size_in_bytes = 1;
> +  if (size_in_bytes == 1)
> +    slow_p = false;
> +  else if ((size_in_bytes & (size_in_bytes - 1)) != 0
> +          || size_in_bytes > 16
> +          || slow_p)
> +    {
> +      real_size_in_bytes = 1;
> +      slow_p = true;
> +    }
>
>    /* Get an iterator on the point where we can add the condition
>       statement for the instrumentation.  */
> @@ -1582,8 +1590,8 @@ build_check_stmt (location_t location, t
>        t = gimple_assign_lhs (gimple_seq_last (seq));
>        gimple_seq_set_location (seq, location);
>        gsi_insert_seq_after (&gsi, seq, GSI_CONTINUE_LINKING);
> -      /* For weird access sizes, check first and last byte.  */
> -      if (real_size_in_bytes != size_in_bytes)
> +      /* For weird access sizes or misaligned, check first and last byte.  */
> +      if (slow_p)
>         {
>           g = gimple_build_assign_with_ops (PLUS_EXPR,
>                                             make_ssa_name (uintptr_type, NULL),
> @@ -1626,7 +1634,7 @@ build_check_stmt (location_t location, t
>
>    /* Generate call to the run-time library (e.g. __asan_report_load8).  */
>    gsi = gsi_start_bb (then_bb);
> -  g = gimple_build_call (report_error_func (is_store, size_in_bytes),
> +  g = gimple_build_call (report_error_func (is_store, size_in_bytes, slow_p),
>                          sz_arg ? 2 : 1, base_addr, sz_arg);
>    gimple_set_location (g, location);
>    gsi_insert_after (&gsi, g, GSI_NEW_STMT);
> @@ -1722,8 +1730,31 @@ instrument_derefs (gimple_stmt_iterator
>    base = build_fold_addr_expr (t);
>    if (!has_mem_ref_been_instrumented (base, size_in_bytes))
>      {
> +      bool slow_p = false;
> +      if (size_in_bytes > 1)
> +       {
> +         if ((size_in_bytes & (size_in_bytes - 1)) != 0
> +             || size_in_bytes > 16)
> +           slow_p = true;
> +         else
> +           {
> +             unsigned int align = get_object_alignment (t);
> +             if (align < size_in_bytes * BITS_PER_UNIT)
> +               {
> +                 /* On non-strict alignment targets, if
> +                    16-byte access is just 8-byte aligned,
> +                    this will result in misaligned shadow
> +                    memory 2 byte load, but otherwise can
> +                    be handled using one read.  */
> +                 if (size_in_bytes != 16
> +                     || STRICT_ALIGNMENT
> +                     || align < 8 * BITS_PER_UNIT)
> +                   slow_p = true;
> +               }
> +           }
> +       }
>        build_check_stmt (location, base, iter, /*before_p=*/true,
> -                       is_store, size_in_bytes);
> +                       is_store, size_in_bytes, slow_p);
>        update_mem_ref_hash_table (base, size_in_bytes);
>        update_mem_ref_hash_table (t, size_in_bytes);
>      }
> --- gcc/testsuite/c-c++-common/asan/misalign-1.c.jj     2014-05-23 18:03:40.400842565 +0200
> +++ gcc/testsuite/c-c++-common/asan/misalign-1.c        2014-05-23 18:29:42.899264340 +0200
> @@ -0,0 +1,42 @@
> +/* { dg-do run { target { ilp32 || lp64 } } } */
> +/* { dg-options "-O2" } */
> +/* { dg-shouldfail "asan" } */
> +
> +struct S { int i; } __attribute__ ((packed));
> +
> +__attribute__((noinline, noclone)) int
> +foo (struct S *s)
> +{
> +  return s->i;
> +}
> +
> +__attribute__((noinline, noclone)) int
> +bar (int *s)
> +{
> +  return *s;
> +}
> +
> +__attribute__((noinline, noclone)) struct S
> +baz (struct S *s)
> +{
> +  return *s;
> +}
> +
> +int
> +main ()
> +{
> +  struct T { char a[3]; struct S b[3]; char c; } t;
> +  int v = 5;
> +  struct S *p = t.b;
> +  asm volatile ("" : "+rm" (p));
> +  p += 3;
> +  if (bar (&v) != 5) __builtin_abort ();
> +  volatile int w = foo (p);
> +  return 0;
> +}
> +
> +/* { dg-output "ERROR: AddressSanitizer:\[^\n\r]*on address\[^\n\r]*" } */
> +/* { dg-output "0x\[0-9a-f\]+ at pc 0x\[0-9a-f\]+ bp 0x\[0-9a-f\]+ sp 0x\[0-9a-f\]+\[^\n\r]*(\n|\r\n|\r)" } */
> +/* { dg-output "\[^\n\r]*READ of size 4 at 0x\[0-9a-f\]+ thread T0\[^\n\r]*(\n|\r\n|\r)" } */
> +/* { dg-output "    #0 0x\[0-9a-f\]+ (in _*foo(\[^\n\r]*misalign-1.c:10|\[^\n\r]*:0)|\[(\])\[^\n\r]*(\n|\r\n|\r)" } */
> +/* { dg-output "    #1 0x\[0-9a-f\]+ (in _*main (\[^\n\r]*misalign-1.c:34|\[^\n\r]*:0)|\[(\]).*(\n|\r\n|\r)" } */
> --- gcc/testsuite/c-c++-common/asan/misalign-2.c.jj     2014-05-23 18:07:59.820594400 +0200
> +++ gcc/testsuite/c-c++-common/asan/misalign-2.c        2014-05-23 18:29:51.808220759 +0200
> @@ -0,0 +1,42 @@
> +/* { dg-do run { target { ilp32 || lp64 } } } */
> +/* { dg-options "-O2" } */
> +/* { dg-shouldfail "asan" } */
> +
> +struct S { int i; } __attribute__ ((packed));
> +
> +__attribute__((noinline, noclone)) int
> +foo (struct S *s)
> +{
> +  return s->i;
> +}
> +
> +__attribute__((noinline, noclone)) int
> +bar (int *s)
> +{
> +  return *s;
> +}
> +
> +__attribute__((noinline, noclone)) struct S
> +baz (struct S *s)
> +{
> +  return *s;
> +}
> +
> +int
> +main ()
> +{
> +  struct T { char a[3]; struct S b[3]; char c; } t;
> +  int v = 5;
> +  struct S *p = t.b;
> +  asm volatile ("" : "+rm" (p));
> +  p += 3;
> +  if (bar (&v) != 5) __builtin_abort ();
> +  volatile struct S w = baz (p);
> +  return 0;
> +}
> +
> +/* { dg-output "ERROR: AddressSanitizer:\[^\n\r]*on address\[^\n\r]*" } */
> +/* { dg-output "0x\[0-9a-f\]+ at pc 0x\[0-9a-f\]+ bp 0x\[0-9a-f\]+ sp 0x\[0-9a-f\]+\[^\n\r]*(\n|\r\n|\r)" } */
> +/* { dg-output "\[^\n\r]*READ of size 4 at 0x\[0-9a-f\]+ thread T0\[^\n\r]*(\n|\r\n|\r)" } */
> +/* { dg-output "    #0 0x\[0-9a-f\]+ (in _*baz(\[^\n\r]*misalign-2.c:22|\[^\n\r]*:0)|\[(\])\[^\n\r]*(\n|\r\n|\r)" } */
> +/* { dg-output "    #1 0x\[0-9a-f\]+ (in _*main (\[^\n\r]*misalign-2.c:34|\[^\n\r]*:0)|\[(\]).*(\n|\r\n|\r)" } */
>
>
>         Jakub

Re: libsanitizer merge from upstream r208536

[Jakub Jelinek]

On Mon, May 26, 2014 at 11:29:28AM +0400, Konstantin Serebryany wrote:
> > Note, I think some work is needed on the library side,
> > ERROR: AddressSanitizer: unknown-crash on address 0xffc439cf at pc 0x804898e bp 0xffc438d8 sp 0xffc438cc
> 
> 
> With clang I get this nice report:
> 
> ==20989==ERROR: AddressSanitizer: stack-buffer-overflow on address
> 0x7fffffffdf02 at pc 0x4b20db bp 0x7fffffffdd70 sp 0x7fffffffdd68
> READ of size 4 at 0x7fffffffdf02 thread T0
>     #0 0x4b20da in foo(S*) /home/kcc/tmp/jakub-unaligned.c:6
>     #1 0x4b2744 in main /home/kcc/tmp/jakub-unaligned.c:30
>     #2 0x7ffff6bfd76c in __libc_start_main
> /build/buildd/eglibc-2.15/csu/libc-start.c:226
>     #3 0x4b1e6c in _start (/usr/local/google/kcc/llvm_cmake/a.out+0x4b1e6c)
> 
> Address 0x7fffffffdf02 is located in stack of thread T0 at offset 66 in frame
>     #0 0x4b24ef in main /home/kcc/tmp/jakub-unaligned.c:23

So you are passing two different base-addr values to the
__asan_report_load_n ?  If yes, that doesn't sound like a good idea to me,
because it will almost always generate larger code, plus the diagnostic
is incorrect, there is no 4 byte read at address 0x7fffffffdf02,
there is a 4 byte read at address 0x7fffffffdeff, where 1 byte read at
at 0x7fffffffdeff has been checked and found valid, and one byte read
at 0x7fffffffdf02 has been checked and found invalid.

So in that case IMNSHO you need to fix both asan library and llvm.

	Jakub

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

Agree.
I was going to change instrumentation of unaligned and unusual-sized
accesses to simple callbacks.
00000000004b1f30 <_Z3fooP1S>:
  4b1f30:       53                      push   %rbx
  4b1f31:       48 89 fb                mov    %rdi,%rbx
  4b1f34:       be 04 00 00 00          mov    $0x4,%esi
  4b1f39:       e8 62 9d fe ff          callq  49bca0 <__asan_loadN>
  4b1f3e:       8b 03                   mov    (%rbx),%eax
  4b1f40:       5b                      pop    %rbx
  4b1f41:       c3                      retq

This is implemented in llvm, just need a flag flip. It also needs a
performance improvement in the run-time.
This is in my TODO, just didn't have time.

clang++ -g -O2  -fsanitize=address  ~/tmp/jakub-unaligned.c -mllvm
-asan-instrumentation-with-call-threshold=0  && ./a.out
==6925==ERROR: AddressSanitizer: unknown-crash on address
0x7fff9968a1af at pc 0x4b1f3e bp 0x7fff9968a160 sp 0x7fff9968a148
READ of size 4 at 0x7fff9968a1af thread T0
    #0 0x4b1f3d in foo(S*) /home/kcc/tmp/jakub-unaligned.c:6
    #1 0x4b205a in main /home/kcc/tmp/jakub-unaligned.c:30
    #2 0x7fec5540f76c in __libc_start_main
/build/buildd/eglibc-2.15/csu/libc-start.c:226
    #3 0x4b1e6c in _start (/usr/local/google/kcc/llvm_cmake/a.out+0x4b1e6c)

Address 0x7fff9968a1af is located in stack of thread T0 at offset 47 in frame
    #0 0x4b1f8f in main /home/kcc/tmp/jakub-unaligned.c:23

  This frame has 4 object(s):
    [32, 48) 't' <== Memory access at offset 47 partially overflows
this variable
    [64, 68) 'v'
    [80, 88) 'p'
    [112, 116) 'w'

Apparently, this "unknown-crash" needs to be fixed.
Give me some time (may not have it this week though).

--kcc


On Mon, May 26, 2014 at 12:57 PM, Jakub Jelinek <jakub@redhat.com> wrote:
> On Mon, May 26, 2014 at 11:29:28AM +0400, Konstantin Serebryany wrote:
>> > Note, I think some work is needed on the library side,
>> > ERROR: AddressSanitizer: unknown-crash on address 0xffc439cf at pc 0x804898e bp 0xffc438d8 sp 0xffc438cc
>>
>>
>> With clang I get this nice report:
>>
>> ==20989==ERROR: AddressSanitizer: stack-buffer-overflow on address
>> 0x7fffffffdf02 at pc 0x4b20db bp 0x7fffffffdd70 sp 0x7fffffffdd68
>> READ of size 4 at 0x7fffffffdf02 thread T0
>>     #0 0x4b20da in foo(S*) /home/kcc/tmp/jakub-unaligned.c:6
>>     #1 0x4b2744 in main /home/kcc/tmp/jakub-unaligned.c:30
>>     #2 0x7ffff6bfd76c in __libc_start_main
>> /build/buildd/eglibc-2.15/csu/libc-start.c:226
>>     #3 0x4b1e6c in _start (/usr/local/google/kcc/llvm_cmake/a.out+0x4b1e6c)
>>
>> Address 0x7fffffffdf02 is located in stack of thread T0 at offset 66 in frame
>>     #0 0x4b24ef in main /home/kcc/tmp/jakub-unaligned.c:23
>
> So you are passing two different base-addr values to the
> __asan_report_load_n ?  If yes, that doesn't sound like a good idea to me,
> because it will almost always generate larger code, plus the diagnostic
> is incorrect, there is no 4 byte read at address 0x7fffffffdf02,
> there is a 4 byte read at address 0x7fffffffdeff, where 1 byte read at
> at 0x7fffffffdeff has been checked and found valid, and one byte read
> at 0x7fffffffdf02 has been checked and found invalid.
>
> So in that case IMNSHO you need to fix both asan library and llvm.
>
>         Jakub

Re: libsanitizer merge from upstream r208536

[Yury Gribov]

On 05/26/2014 01:19 PM, Konstantin Serebryany wrote:
> This is implemented in llvm, just need a flag flip. It also needs a
> performance improvement in the run-time.
> This is in my TODO, just didn't have time.

FYI I have a patch that adds -asan-instrumentation-with-call-threshold 
for GCC (will send in couple of days).

-Y

Re: libsanitizer merge from upstream r208536

[Dodji Seketeli]

Jakub Jelinek <jakub@redhat.com> writes:


[...]

> Here is the GCC side of the thing, 

[...]

> 2014-05-23  Jakub Jelinek  <jakub@redhat.com>
>
> 	* asan.c (report_error_func): Add SLOW_P argument, use
> 	BUILT_IN_ASAN_*_N if set.
> 	(build_check_stmt): Likewise.
> 	(instrument_derefs): If T has insufficient alignment,
> 	force same handling as for odd sizes.
>
> 	* c-c++-common/asan/misalign-1.c: New test.
> 	* c-c++-common/asan/misalign-2.c: New test.
>

> ok for trunk if it bootstraps/regtests?

Yes, this is OK for me.

Thanks.

-- 
		Dodji

Re: libsanitizer merge from upstream r208536

[Jakub Jelinek]

On Thu, May 22, 2014 at 04:07:38PM +0200, Jakub Jelinek wrote:
> 2014-05-22  Jakub Jelinek  <jakub@redhat.com>
> 
> 	* sanitizer.def (BUILT_IN_ASAN_REPORT_LOAD_N,
> 	BUILT_IN_ASAN_REPORT_STORE_N): New.
> 	* asan.c (struct asan_mem_ref): Change access_size type to
> 	HOST_WIDE_INT.
> 	(asan_mem_ref_init, asan_mem_ref_new, get_mem_refs_of_builtin_call,
> 	update_mem_ref_hash_table): Likewise.
> 	(asan_mem_ref_hasher::hash): Hash in a HWI.
> 	(report_error_func): Change size_in_bytes argument to HWI.
> 	Use *_N builtins if size_in_bytes is larger than 16 or not power of
> 	two.
> 	(build_shadow_mem_access): New function.
> 	(build_check_stmt): Use it.  Change size_in_bytes argument to HWI.
> 	Handle size_in_bytes not power of two or larger than 16.
> 	(instrument_derefs): Don't give up if size_in_bytes is not
> 	power of two or is larger than 16.

Both patches bootstrapped/regtested on x86_64-linux and i686-linux
successfully, ok for trunk for both?

	Jakub

Re: libsanitizer merge from upstream r208536

[Dodji Seketeli]

Jakub Jelinek <jakub@redhat.com> writes:

> On Thu, May 22, 2014 at 04:07:38PM +0200, Jakub Jelinek wrote:
>> 2014-05-22  Jakub Jelinek  <jakub@redhat.com>
>> 
>> 	* sanitizer.def (BUILT_IN_ASAN_REPORT_LOAD_N,
>> 	BUILT_IN_ASAN_REPORT_STORE_N): New.
>> 	* asan.c (struct asan_mem_ref): Change access_size type to
>> 	HOST_WIDE_INT.
>> 	(asan_mem_ref_init, asan_mem_ref_new, get_mem_refs_of_builtin_call,
>> 	update_mem_ref_hash_table): Likewise.
>> 	(asan_mem_ref_hasher::hash): Hash in a HWI.
>> 	(report_error_func): Change size_in_bytes argument to HWI.
>> 	Use *_N builtins if size_in_bytes is larger than 16 or not power of
>> 	two.
>> 	(build_shadow_mem_access): New function.
>> 	(build_check_stmt): Use it.  Change size_in_bytes argument to HWI.
>> 	Handle size_in_bytes not power of two or larger than 16.
>> 	(instrument_derefs): Don't give up if size_in_bytes is not
>> 	power of two or is larger than 16.
>
> Both patches bootstrapped/regtested on x86_64-linux and i686-linux
> successfully, ok for trunk for both?

Yes, this is OK.  Thanks.

-- 
		Dodji

Re: libsanitizer merge from upstream r208536

[Dodji Seketeli]

Jakub Jelinek <jakub@redhat.com> writes:

[...]

> Ok, tried to merge also g++.dg/asan from the same revision as you've merged
> libsanitizer.

[...]

>
> 2014-05-22  Jakub Jelinek  <jakub@redhat.com>
>
> 	* g++.dg/asan/asan_test.C: Add -std=c++11 and
> 	-DSANITIZER_USE_DEJAGNU_GTEST=1 to dg-options, remove
> 	-DASAN_USE_DEJAGNU_GTEST=1.
> 	* g++.dg/asan/asan_mem_test.cc: Updated from upstream
> 	r209283.
> 	* g++.dg/asan/asan_oob_test.cc: Likewise.
> 	* g++.dg/asan/sanitizer_test_utils.h: Likewise.
> 	* g++.dg/asan/asan_str_test.cc: Likewise.
> 	* g++.dg/asan/asan_test_utils.h: Likewise.
> 	* g++.dg/asan/sanitizer_test_config.h: Likewise.
> 	* g++.dg/asan/asan_test.cc: Likewise.
> 	* g++.dg/asan/sanitizer_pthread_wrappers.h: New file.
> 	Imported from upstream r209283.
> 	* g++.dg/asan/asan_test_config.h: Likewise.

This is OK.

Thanks.

-- 
		Dodji

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

On Thu, May 22, 2014 at 3:03 PM, Jakub Jelinek <jakub@redhat.com> wrote:
> On Thu, May 22, 2014 at 02:26:19PM +0400, Konstantin Serebryany wrote:
>> >> >> FAIL: c-c++-common/asan/asan-interface-1.c  -O0  execution test
>> >> >Is that before or after r210743?
>
> Can't reproduce the above (note, not bootstrapped compiler, just
> --disable-bootstrap), check-gcc RUNTESTFLAGS=asan.exp is clean, but
> check-g++ RUNTESTFLAGS=asan.exp I got:

How exactly do you run this check-g++?

For me:
% make check-gcc RUNTESTFLAGS=asan.exp
make[1]: Entering directory ...  <works>

make check-g++ RUNTESTFLAGS=asan.exp
make: *** No rule to make target `check-g++'.  Stop.






> Running /usr/src/gcc/gcc/testsuite/g++.dg/asan/asan.exp ...
> FAIL: g++.dg/asan/asan_test.C  -O2  AddressSanitizer_MallocUsableSizeTest execution test
> :
> Setting LD_LIBRARY_PATH to .:/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libstdc++-v3/src/.libs:/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libst
> dc++-v3/src/.libs:/usr/src/gcc/obj2/gcc:/usr/src/gcc/obj2/gcc/32:/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libsanitizer/asan/.libs:.:/usr/src/gc
> c/obj2/x86_64-unknown-linux-gnu/./libstdc++-v3/src/.libs:/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libstdc++-v3/src/.libs:/usr/src/gcc/obj2/gcc:
> /usr/src/gcc/obj2/gcc/32:/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libsanitizer/asan/.libs
> spawn [open ...]
> DEJAGNU_GTEST_EXPECT_DEATH1 malloc_usable_size((void*)0x123) DEJAGNU_GTEST_EXPECT_DEATH1 AddressSanitizer: attempting to call malloc_usable_size()
> DEJAGNU_GTEST_EXPECT_DEATH1
> DEJAGNU_GTEST_EXPECT_DEATH2 malloc_usable_size(array + kArraySize / 2) DEJAGNU_GTEST_EXPECT_DEATH2 AddressSanitizer: attempting to call malloc_usab
> le_size() DEJAGNU_GTEST_EXPECT_DEATH2
> DEJAGNU_GTEST_EXPECT_DEATH3 malloc_usable_size(array) DEJAGNU_GTEST_EXPECT_DEATH3 AddressSanitizer: attempting to call malloc_usable_size() DEJAGNU
> _GTEST_EXPECT_DEATH3
>
> =================================================================
> ==28396==ERROR: LeakSanitizer: detected memory leaks
>
> Direct leak of 4 byte(s) in 1 object(s) allocated from:
>     #0 0x7f5c18048df9 in operator new(unsigned long) ../../../../libsanitizer/asan/asan_new_delete.cc:53
>     #1 0x409134 in AddressSanitizer_MallocUsableSizeTest_fn /usr/src/gcc/gcc/testsuite/g++.dg/asan/asan_test.cc:407
>
> SUMMARY: AddressSanitizer: 4 byte(s) leaked in 1 allocation(s).
> FAIL: g++.dg/asan/asan_test.C  -O2  AddressSanitizer_MallocUsableSizeTest execution test
>
> If I manually compile it:
> /usr/src/gcc/obj2/gcc/testsuite/g++/../../xg++ -B/usr/src/gcc/obj2/gcc/testsuite/g++/../../ /usr/src/gcc/gcc/testsuite/g++.dg/asan/asan_test.C -fsanitize=address -g -I/usr/src/gcc/gcc/testsuite/../../libsanitizer/include -fno-diagnostics-show-caret -fdiagnostics-color=never -nostdinc++ -I/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/libstdc++-v3/include/x86_64-unknown-linux-gnu -I/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/libstdc++-v3/include -I/usr/src/gcc/libstdc++-v3/libsupc++ -I/usr/src/gcc/libstdc++-v3/include/backward -I/usr/src/gcc/libstdc++-v3/testsuite/util -fmessage-length=0 -O2 -fsanitize=address -fno-builtin -Wall -Wno-format -Werror -g -DASAN_UAR=0 -DASAN_HAS_EXCEPTIONS=1 -DASAN_HAS_BLACKLIST=0 -DASAN_USE_DEJAGNU_GTEST=1 -lasan -lpthread -ldl -DASAN_NEEDS_SEGV=1 -DASAN_AVOID_EXPENSIVE_TESTS=1 -msse2 -D__NO_INLINE__ /usr/src/gcc/gcc/testsuite/g++.dg/asan/asan_globals_test-wrapper.cc -B/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libsanitizer/ -B/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libsanitizer/asan/ -L/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libsanitizer/asan/.libs -L/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libstdc++-v3/src/.libs -B/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libstdc++-v3/src/.libs -L/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libstdc++-v3/src/.libs -lm -o ./asan_test.exe -Wl,-rpath,/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libsanitizer/asan/.libs/
> and run:
> ./asan_test.exe AddressSanitizer_MallocUsableSizeTest
> I get the same error.
>
> Seems compiler-rt has there:
>
> --- gcc/testsuite/g++.dg/asan/asan_test.cc      2013-11-12 11:31:20.000000000 +0100
> +++ gcc/testsuite/g++.dg/asan/asan_test.cc      2014-05-22 13:00:23.391901708 +0200
> @@ -413,6 +413,7 @@ TEST(AddressSanitizer, MallocUsableSizeT
>                 kMallocUsableSizeErrorMsg);
>    free(array);
>    EXPECT_DEATH(malloc_usable_size(array), kMallocUsableSizeErrorMsg);
> +  delete int_ptr;
>  }
>  #endif
>
> There are various other changes to asan_test.cc, so guess some work is needed on that.
>
>         Jakub

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

Oops, ignore that. Forgot -C gcc

On Thu, May 22, 2014 at 4:49 PM, Konstantin Serebryany
<konstantin.s.serebryany@gmail.com> wrote:
> On Thu, May 22, 2014 at 3:03 PM, Jakub Jelinek <jakub@redhat.com> wrote:
>> On Thu, May 22, 2014 at 02:26:19PM +0400, Konstantin Serebryany wrote:
>>> >> >> FAIL: c-c++-common/asan/asan-interface-1.c  -O0  execution test
>>> >> >Is that before or after r210743?
>>
>> Can't reproduce the above (note, not bootstrapped compiler, just
>> --disable-bootstrap), check-gcc RUNTESTFLAGS=asan.exp is clean, but
>> check-g++ RUNTESTFLAGS=asan.exp I got:
>
> How exactly do you run this check-g++?
>
> For me:
> % make check-gcc RUNTESTFLAGS=asan.exp
> make[1]: Entering directory ...  <works>
>
> make check-g++ RUNTESTFLAGS=asan.exp
> make: *** No rule to make target `check-g++'.  Stop.
>
>
>
>
>
>
>> Running /usr/src/gcc/gcc/testsuite/g++.dg/asan/asan.exp ...
>> FAIL: g++.dg/asan/asan_test.C  -O2  AddressSanitizer_MallocUsableSizeTest execution test
>> :
>> Setting LD_LIBRARY_PATH to .:/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libstdc++-v3/src/.libs:/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libst
>> dc++-v3/src/.libs:/usr/src/gcc/obj2/gcc:/usr/src/gcc/obj2/gcc/32:/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libsanitizer/asan/.libs:.:/usr/src/gc
>> c/obj2/x86_64-unknown-linux-gnu/./libstdc++-v3/src/.libs:/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libstdc++-v3/src/.libs:/usr/src/gcc/obj2/gcc:
>> /usr/src/gcc/obj2/gcc/32:/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libsanitizer/asan/.libs
>> spawn [open ...]
>> DEJAGNU_GTEST_EXPECT_DEATH1 malloc_usable_size((void*)0x123) DEJAGNU_GTEST_EXPECT_DEATH1 AddressSanitizer: attempting to call malloc_usable_size()
>> DEJAGNU_GTEST_EXPECT_DEATH1
>> DEJAGNU_GTEST_EXPECT_DEATH2 malloc_usable_size(array + kArraySize / 2) DEJAGNU_GTEST_EXPECT_DEATH2 AddressSanitizer: attempting to call malloc_usab
>> le_size() DEJAGNU_GTEST_EXPECT_DEATH2
>> DEJAGNU_GTEST_EXPECT_DEATH3 malloc_usable_size(array) DEJAGNU_GTEST_EXPECT_DEATH3 AddressSanitizer: attempting to call malloc_usable_size() DEJAGNU
>> _GTEST_EXPECT_DEATH3
>>
>> =================================================================
>> ==28396==ERROR: LeakSanitizer: detected memory leaks
>>
>> Direct leak of 4 byte(s) in 1 object(s) allocated from:
>>     #0 0x7f5c18048df9 in operator new(unsigned long) ../../../../libsanitizer/asan/asan_new_delete.cc:53
>>     #1 0x409134 in AddressSanitizer_MallocUsableSizeTest_fn /usr/src/gcc/gcc/testsuite/g++.dg/asan/asan_test.cc:407
>>
>> SUMMARY: AddressSanitizer: 4 byte(s) leaked in 1 allocation(s).
>> FAIL: g++.dg/asan/asan_test.C  -O2  AddressSanitizer_MallocUsableSizeTest execution test
>>
>> If I manually compile it:
>> /usr/src/gcc/obj2/gcc/testsuite/g++/../../xg++ -B/usr/src/gcc/obj2/gcc/testsuite/g++/../../ /usr/src/gcc/gcc/testsuite/g++.dg/asan/asan_test.C -fsanitize=address -g -I/usr/src/gcc/gcc/testsuite/../../libsanitizer/include -fno-diagnostics-show-caret -fdiagnostics-color=never -nostdinc++ -I/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/libstdc++-v3/include/x86_64-unknown-linux-gnu -I/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/libstdc++-v3/include -I/usr/src/gcc/libstdc++-v3/libsupc++ -I/usr/src/gcc/libstdc++-v3/include/backward -I/usr/src/gcc/libstdc++-v3/testsuite/util -fmessage-length=0 -O2 -fsanitize=address -fno-builtin -Wall -Wno-format -Werror -g -DASAN_UAR=0 -DASAN_HAS_EXCEPTIONS=1 -DASAN_HAS_BLACKLIST=0 -DASAN_USE_DEJAGNU_GTEST=1 -lasan -lpthread -ldl -DASAN_NEEDS_SEGV=1 -DASAN_AVOID_EXPENSIVE_TESTS=1 -msse2 -D__NO_INLINE__ /usr/src/gcc/gcc/testsuite/g++.dg/asan/asan_globals_test-wrapper.cc -B/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libsanitizer/ -B/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libsanitizer/asan/ -L/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libsanitizer/asan/.libs -L/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libstdc++-v3/src/.libs -B/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libstdc++-v3/src/.libs -L/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libstdc++-v3/src/.libs -lm -o ./asan_test.exe -Wl,-rpath,/usr/src/gcc/obj2/x86_64-unknown-linux-gnu/./libsanitizer/asan/.libs/
>> and run:
>> ./asan_test.exe AddressSanitizer_MallocUsableSizeTest
>> I get the same error.
>>
>> Seems compiler-rt has there:
>>
>> --- gcc/testsuite/g++.dg/asan/asan_test.cc      2013-11-12 11:31:20.000000000 +0100
>> +++ gcc/testsuite/g++.dg/asan/asan_test.cc      2014-05-22 13:00:23.391901708 +0200
>> @@ -413,6 +413,7 @@ TEST(AddressSanitizer, MallocUsableSizeT
>>                 kMallocUsableSizeErrorMsg);
>>    free(array);
>>    EXPECT_DEATH(malloc_usable_size(array), kMallocUsableSizeErrorMsg);
>> +  delete int_ptr;
>>  }
>>  #endif
>>
>> There are various other changes to asan_test.cc, so guess some work is needed on that.
>>
>>         Jakub

Re: libsanitizer merge from upstream r208536

[Paolo Carlini]

Hi,

On 05/22/2014 01:03 PM, Jakub Jelinek wrote:
> On Thu, May 22, 2014 at 02:26:19PM +0400, Konstantin Serebryany wrote:
>>>>>> FAIL: c-c++-common/asan/asan-interface-1.c  -O0  execution test
>>>>> Is that before or after r210743?
> Can't reproduce the above (note, not bootstrapped compiler, just
> --disable-bootstrap), check-gcc RUNTESTFLAGS=asan.exp is clean,
Thanks Jakub. I'm not bootstrapping either, but all the fails are still 
there (fetched a fresh tree, built again). I have just noticed the below 
during the build, are expected?

Paolo.

////////////////

In file included from 
../../../../trunk/libsanitizer/asan/asan_interceptors.cc:147:0:
../../../../trunk/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc: 
In function ‘int __interceptor_accept4(int, void*, unsigned int*, int)’:
../../../../trunk/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:1821:12: 
warning: ‘addrlen0’ may be used uninitialized in this function 
[-Wmaybe-uninitialized]
unsigned addrlen0;
^
../../../../trunk/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc: 
In function ‘int __interceptor_accept(int, void*, unsigned int*)’:
../../../../trunk/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:1799:12: 
warning: ‘addrlen0’ may be used uninitialized in this function 
[-Wmaybe-uninitialized]
unsigned addrlen0;
^

Re: libsanitizer merge from upstream r208536

[Jakub Jelinek]

On Thu, May 22, 2014 at 08:38:31PM +0200, Paolo Carlini wrote:
> Hi,
> 
> On 05/22/2014 01:03 PM, Jakub Jelinek wrote:
> >On Thu, May 22, 2014 at 02:26:19PM +0400, Konstantin Serebryany wrote:
> >>>>>>FAIL: c-c++-common/asan/asan-interface-1.c  -O0  execution test
> >>>>>Is that before or after r210743?
> >Can't reproduce the above (note, not bootstrapped compiler, just
> >--disable-bootstrap), check-gcc RUNTESTFLAGS=asan.exp is clean,
> Thanks Jakub. I'm not bootstrapping either, but all the fails are
> still there (fetched a fresh tree, built again). I have just noticed
> the below during the build, are expected?

In any case, I couldn't reproduce that even during fresh bootstrap/regtest
I did.

> In file included from
> ../../../../trunk/libsanitizer/asan/asan_interceptors.cc:147:0:
> ../../../../trunk/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:
> In function ‘int __interceptor_accept4(int, void*, unsigned int*,
> int)’:
> ../../../../trunk/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:1821:12:
> warning: ‘addrlen0’ may be used uninitialized in this function
> [-Wmaybe-uninitialized]
> unsigned addrlen0;
> ^
> ../../../../trunk/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:
> In function ‘int __interceptor_accept(int, void*, unsigned int*)’:
> ../../../../trunk/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:1799:12:
> warning: ‘addrlen0’ may be used uninitialized in this function
> [-Wmaybe-uninitialized]
> unsigned addrlen0;
> ^

That sounds like a false positive warning:
  unsigned addrlen0;
  if (addrlen) {
    COMMON_INTERCEPTOR_READ_RANGE(ctx, addrlen, sizeof(*addrlen));
    addrlen0 = *addrlen;
  }
  int fd2 = REAL(accept4)(fd, addr, addrlen, f);
  if (fd2 >= 0) {
    if (fd >= 0) COMMON_INTERCEPTOR_FD_SOCKET_ACCEPT(ctx, fd, fd2);
    if (addr && addrlen)
      COMMON_INTERCEPTOR_WRITE_RANGE(ctx, addr, Min(*addrlen, addrlen0));
  }
(unless the COMMON_INTERCEPTOR* macros do too weird stuff), wonder why the
predicate aware uninit doesn't handle this.

	Jakub

Re: libsanitizer merge from upstream r208536

[Paolo Carlini]

Hi,

On 05/22/2014 09:02 PM, Jakub Jelinek wrote:
> On Thu, May 22, 2014 at 08:38:31PM +0200, Paolo Carlini wrote:
>> Hi,
>>
>> On 05/22/2014 01:03 PM, Jakub Jelinek wrote:
>>> On Thu, May 22, 2014 at 02:26:19PM +0400, Konstantin Serebryany wrote:
>>>>>>>> FAIL: c-c++-common/asan/asan-interface-1.c  -O0  execution test
>>>>>>> Is that before or after r210743?
>>> Can't reproduce the above (note, not bootstrapped compiler, just
>>> --disable-bootstrap), check-gcc RUNTESTFLAGS=asan.exp is clean,
>> Thanks Jakub. I'm not bootstrapping either, but all the fails are
>> still there (fetched a fresh tree, built again). I have just noticed
>> the below during the build, are expected?
> In any case, I couldn't reproduce that even during fresh bootstrap/regtest
> I did.
I see. Some of the fails I'm seeing spill this kind of diagnostic, does 
that tell you something?

FAIL: g++.dg/asan/large-func-test-1.C  -O1  output pattern test, is 
==18037==ASan runtime does not come first in initial library list; you 
should either link runtime to your application or manually preload it 
with LD_PRELOAD.
, should match ERROR: AddressSanitizer:? heap-buffer-overflow on address[^

....

Paolo.

Re: libsanitizer merge from upstream r208536

[Jakub Jelinek]

On Thu, May 22, 2014 at 09:03:44PM +0200, Paolo Carlini wrote:
> Hi,
> 
> On 05/22/2014 09:02 PM, Jakub Jelinek wrote:
> >On Thu, May 22, 2014 at 08:38:31PM +0200, Paolo Carlini wrote:
> >>Hi,
> >>
> >>On 05/22/2014 01:03 PM, Jakub Jelinek wrote:
> >>>On Thu, May 22, 2014 at 02:26:19PM +0400, Konstantin Serebryany wrote:
> >>>>>>>>FAIL: c-c++-common/asan/asan-interface-1.c  -O0  execution test
> >>>>>>>Is that before or after r210743?
> >>>Can't reproduce the above (note, not bootstrapped compiler, just
> >>>--disable-bootstrap), check-gcc RUNTESTFLAGS=asan.exp is clean,
> >>Thanks Jakub. I'm not bootstrapping either, but all the fails are
> >>still there (fetched a fresh tree, built again). I have just noticed
> >>the below during the build, are expected?
> >In any case, I couldn't reproduce that even during fresh bootstrap/regtest
> >I did.
> I see. Some of the fails I'm seeing spill this kind of diagnostic,
> does that tell you something?
> 
> FAIL: g++.dg/asan/large-func-test-1.C  -O1  output pattern test, is
> ==18037==ASan runtime does not come first in initial library list;
> you should either link runtime to your application or manually
> preload it with LD_PRELOAD.
> , should match ERROR: AddressSanitizer:? heap-buffer-overflow on address[^
> 
> ....

Do you have LD_PRELOAD set in the environment?
If not, can you cut'n'paste the large-func-test-1.exe command line
and run it with the given LD_LIBRARY_PATH under ldd?

	Jakub

Re: libsanitizer merge from upstream r208536

[Paolo Carlini]

Hi,

On 05/22/2014 09:15 PM, Jakub Jelinek wrote:
> Do you have LD_PRELOAD set in the environment?
I don't.
> If not, can you cut'n'paste the large-func-test-1.exe command line and 
> run it with the given LD_LIBRARY_PATH under ldd? Jakub 
Sure. This is what I get:

         linux-vdso.so.1 (0x00007fffee578000)
         libasan.so.1 => 
/home/paolo/Gcc/svn-dirs/trunk-build/x86_64-unknown-linux-gnu/./libsanitizer/asan/.libs/libasan.so.1 
(0x00007f41610c7000)
         libstdc++.so.6 => 
/home/paolo/Gcc/svn-dirs/trunk-build/x86_64-unknown-linux-gnu/./libstdc++-v3/src/.libs/libstdc++.so.6 
(0x00007f4160db6000)
         libm.so.6 => /lib64/libm.so.6 (0x00007f4160ab3000)
         libgcc_s.so.1 => 
/home/paolo/Gcc/svn-dirs/trunk-build/gcc/libgcc_s.so.1 (0x00007f416089c000)
         libc.so.6 => /lib64/libc.so.6 (0x00007f41604ed000)
         libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f41602cf000)
         libdl.so.2 => /lib64/libdl.so.2 (0x00007f41600cb000)
         /lib64/ld-linux-x86-64.so.2 (0x00007f4161ff3000)

Paolo.

Re: libsanitizer merge from upstream r208536

[Jakub Jelinek]

On Thu, May 22, 2014 at 09:43:01PM +0200, Paolo Carlini wrote:
> Hi,
> 
> On 05/22/2014 09:15 PM, Jakub Jelinek wrote:
> >Do you have LD_PRELOAD set in the environment?
> I don't.
> >If not, can you cut'n'paste the large-func-test-1.exe command line
> >and run it with the given LD_LIBRARY_PATH under ldd? Jakub
> Sure. This is what I get:
> 
>         linux-vdso.so.1 (0x00007fffee578000)
>         libasan.so.1 => /home/paolo/Gcc/svn-dirs/trunk-build/x86_64-unknown-linux-gnu/./libsanitizer/asan/.libs/libasan.so.1
> (0x00007f41610c7000)
>         libstdc++.so.6 => /home/paolo/Gcc/svn-dirs/trunk-build/x86_64-unknown-linux-gnu/./libstdc++-v3/src/.libs/libstdc++.so.6
> (0x00007f4160db6000)
>         libm.so.6 => /lib64/libm.so.6 (0x00007f4160ab3000)
>         libgcc_s.so.1 =>
> /home/paolo/Gcc/svn-dirs/trunk-build/gcc/libgcc_s.so.1
> (0x00007f416089c000)
>         libc.so.6 => /lib64/libc.so.6 (0x00007f41604ed000)
>         libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f41602cf000)
>         libdl.so.2 => /lib64/libdl.so.2 (0x00007f41600cb000)
>         /lib64/ld-linux-x86-64.so.2 (0x00007f4161ff3000)

Kostya, guess you should ignore the vDSO.

I'd say the test is useless anyway, if you really want to do some
quick sanity check, much better would be just dlsym a couple of
interesting symbols to verify that libasan.so.1 is ahead
of libc.so.6, libstdc++.so.6, libpthread.so.0 (whatever else
you want to override).  Otherwise libasan apps will simply stop
working altogether if LD_PRELOAD is set, to whatever library,
even if it doesn't define any symbols you care about.
I'd verify that malloc, memcpy, pthread_create, operator new
all resolve to the libasan.so.1 copy (if all those have local aliases,
just compare what dlsym returns with those local aliases).
Typically people in LD_PRELOAD override malloc (which we want to diagnose),
or far more rarely stringops (e.g. memstomp, also undesirable).

	Jakub

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

Yuri, this comes from yours
http://llvm.org/viewvc/llvm-project?view=revision&revision=205308
Could you please take a look?

On Thu, May 22, 2014 at 11:54 PM, Jakub Jelinek <jakub@redhat.com> wrote:
> On Thu, May 22, 2014 at 09:43:01PM +0200, Paolo Carlini wrote:
>> Hi,
>>
>> On 05/22/2014 09:15 PM, Jakub Jelinek wrote:
>> >Do you have LD_PRELOAD set in the environment?
>> I don't.
>> >If not, can you cut'n'paste the large-func-test-1.exe command line
>> >and run it with the given LD_LIBRARY_PATH under ldd? Jakub
>> Sure. This is what I get:
>>
>>         linux-vdso.so.1 (0x00007fffee578000)
>>         libasan.so.1 => /home/paolo/Gcc/svn-dirs/trunk-build/x86_64-unknown-linux-gnu/./libsanitizer/asan/.libs/libasan.so.1
>> (0x00007f41610c7000)
>>         libstdc++.so.6 => /home/paolo/Gcc/svn-dirs/trunk-build/x86_64-unknown-linux-gnu/./libstdc++-v3/src/.libs/libstdc++.so.6
>> (0x00007f4160db6000)
>>         libm.so.6 => /lib64/libm.so.6 (0x00007f4160ab3000)
>>         libgcc_s.so.1 =>
>> /home/paolo/Gcc/svn-dirs/trunk-build/gcc/libgcc_s.so.1
>> (0x00007f416089c000)
>>         libc.so.6 => /lib64/libc.so.6 (0x00007f41604ed000)
>>         libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f41602cf000)
>>         libdl.so.2 => /lib64/libdl.so.2 (0x00007f41600cb000)
>>         /lib64/ld-linux-x86-64.so.2 (0x00007f4161ff3000)
>
> Kostya, guess you should ignore the vDSO.
>
> I'd say the test is useless anyway, if you really want to do some
> quick sanity check, much better would be just dlsym a couple of
> interesting symbols to verify that libasan.so.1 is ahead
> of libc.so.6, libstdc++.so.6, libpthread.so.0 (whatever else
> you want to override).  Otherwise libasan apps will simply stop
> working altogether if LD_PRELOAD is set, to whatever library,
> even if it doesn't define any symbols you care about.
> I'd verify that malloc, memcpy, pthread_create, operator new
> all resolve to the libasan.so.1 copy (if all those have local aliases,
> just compare what dlsym returns with those local aliases).
> Typically people in LD_PRELOAD override malloc (which we want to diagnose),
> or far more rarely stringops (e.g. memstomp, also undesirable).
>
>         Jakub

Re: libsanitizer merge from upstream r208536

[Yury Gribov]

On 05/22/2014 11:54 PM, Jakub Jelinek wrote:
 > Kostya, guess you should ignore the vDSO.

Right, we didn't encounter this problem when testing on our kernels. 
I'll cook a patch for this.

 > much better would be just dlsym a couple of
 > interesting symbols to verify that libasan.so.1 is ahead
 > of libc.so.6, libstdc++.so.6, libpthread.so.0 (whatever else
 > you want to override).

One problem with dlsym() that I've seen is that it causes a call to 
malloc on failure (to allocate buffer for dlerror()) which forces 
allocator initialization and breaks Asan's delayed initialization feature.

Also on Linux dlsym() tends to return address of executable's PLT entry 
which is useless.

 > Otherwise libasan apps will simply stop
 > working altogether if LD_PRELOAD is set, to whatever library,
 > even if it doesn't define any symbols you care about.

Right but I'm not sure whether failing fast here is necessarily bad.
Imagine preloaded library has an initializer which calls intercepted 
APIs. Asan didn't get a chance to initialize at the point of call and if 
interceptor doesn't contain a sanity call to asan_init,
we are risking hard-to-debug runtime error (call to NULL, etc.).
I've seen numerous bugs like this (both locally and on mailing lists) 
and they were main motivation to add this check.

Note that user can just prepend libasan to LD_PRELOAD list
to overcome this error.

We could remove the Die() but still leave the warning there
so that user is aware of possible consequences of using unconstrained 
LD_PRELOAD with Asan.

 > Typically people in LD_PRELOAD override malloc (which we want to 
diagnose),
 > or far more rarely stringops (e.g. memstomp, also undesirable).

I wonder whether overriding Asan's malloc, etc. is expected to work at 
all? Perhaps banning it altogether is just the safest thing to do?

-Y

Re: libsanitizer merge from upstream r208536

[Jakub Jelinek]

On Fri, May 23, 2014 at 09:30:12AM +0400, Yury Gribov wrote:
> > much better would be just dlsym a couple of
> > interesting symbols to verify that libasan.so.1 is ahead
> > of libc.so.6, libstdc++.so.6, libpthread.so.0 (whatever else
> > you want to override).
> 
> One problem with dlsym() that I've seen is that it causes a call to
> malloc on failure (to allocate buffer for dlerror()) which forces
> allocator initialization and breaks Asan's delayed initialization
> feature.
> 
> Also on Linux dlsym() tends to return address of executable's PLT
> entry which is useless.

You don't need to use dlsym actually, just comparing
if (malloc != __hidden_malloc_alias) would do it.
You're right that will return PLT slots in the executable though.

> > Otherwise libasan apps will simply stop
> > working altogether if LD_PRELOAD is set, to whatever library,
> > even if it doesn't define any symbols you care about.
> 
> Right but I'm not sure whether failing fast here is necessarily bad.

I think it is very bad.  In fact, if you really want such a check,
I'd say it shouldn't be at least enabled by default, unless some
env var requests it; and document that if you are having troubles with
asan sanitized programs, try this magic env var to get better
troubleshooting.
Even before this exaggerated check asan imposes far more restrictions than
good, and this just makes asan less usable just for fear that it wouldn't
work right.  Most preloaded libs will just provide symbols asan never cares
about, or even if say they override malloc, it could be just some malloc
wrappers that add some bookkeeping and call the original malloc through
dlsym RTLD_NEXT, or even if you say override malloc completely without
calling the original implementation, the world doesn't end, the shadow
mem of those allocations just won't be surrounded by protected paddings,
so what, you don't detect out of bounds for malloc, but can still detect
out of bounds in your program's stack etc.
Ditto for string ops etc.
What really matters is that to avoid crashes, libasan unfortunately has
to be constructed very early, but this check doesn't help with that,
furthermore this code is run during the libasan construction and thus if it
is not early, the library has already crashed by then.

> Imagine preloaded library has an initializer which calls intercepted
> APIs. Asan didn't get a chance to initialize at the point of call
> and if interceptor doesn't contain a sanity call to asan_init,
> we are risking hard-to-debug runtime error (call to NULL, etc.).
> I've seen numerous bugs like this (both locally and on mailing
> lists) and they were main motivation to add this check.

That is nonsense.  Early in the symbol search scope is the opposite of being
initialized early, on the contrary, such libraries are initialized last.
That is the reason why LD_PRELOAD=libasan.so.1 still doesn't help with
the __asan_init_v3 being performed early, you need either .preinit_array
in the executable, or the ctor called by some library late in the symbol
search scope (== early constructed).

> > Typically people in LD_PRELOAD override malloc (which we want to
> diagnose),
> > or far more rarely stringops (e.g. memstomp, also undesirable).
> 
> I wonder whether overriding Asan's malloc, etc. is expected to work
> at all? Perhaps banning it altogether is just the safest thing to
> do?

Don't know why you want to ban everything.

	Jakub

Re: libsanitizer merge from upstream r208536

[Yury Gribov]

On 05/23/2014 10:34 AM, Jakub Jelinek wrote:
>>> Otherwise libasan apps will simply stop
>>> working altogether if LD_PRELOAD is set, to whatever library,
>>> even if it doesn't define any symbols you care about.
>>
>> Right but I'm not sure whether failing fast here is necessarily bad.
>
> I think it is very bad.  In fact, if you really want such a check,
> I'd say it shouldn't be at least enabled by default, unless some
> env var requests it; and document that if you are having troubles with
> asan sanitized programs, try this magic env var to get better
> troubleshooting.

As I said, we can remove that Die() there. Warning can also be hidden 
under ASAN_OPTIONS=verbosity=1.

> Even before this exaggerated check asan imposes far more restrictions than
> good, and this just makes asan less usable just for fear that it wouldn't
> work right.

Ok, we seem to approach this from two different angles.
I usually try to prohibit functionality that's not proven to work
It could be later enabled if users need it.
	
> Most preloaded libs will just provide symbols asan never cares
> about

Maybe. In my experience it's all libc decorations.

> even if you say override malloc completely without
> calling the original implementation, the world doesn't end, the shadow
> mem of those allocations just won't be surrounded by protected paddings,
> so what, you don't detect out of bounds for malloc, but can still detect
> out of bounds in your program's stack etc.
> Ditto for string ops etc.

Let's wait for Konstantin to comment on this.
I don't know runtime well enough to guarantee that arbitrary symbol 
overloads are expected to work.

>> Imagine preloaded library has an initializer which calls intercepted
>> APIs. Asan didn't get a chance to initialize at the point of call
>> and if interceptor doesn't contain a sanity call to asan_init,
>> we are risking hard-to-debug runtime error (call to NULL, etc.).
>> I've seen numerous bugs like this (both locally and on mailing
>> lists) and they were main motivation to add this check.
>
> That is nonsense.  Early in the symbol search scope is the opposite of being
> initialized early, on the contrary, such libraries are initialized last.

I may be wrong but my understanding was that ld.so performs a reverse 
topological sort of dependencies and initializes them in that order.
Given that libasan depends on standard libs (librt, lipthread, libdl, 
etc.) it'll be initialized after them but before user libs.
Initializers of std libs may indeed cause problems but we at least make 
sure to initialize before arbitrary user libraries.

>> I wonder whether overriding Asan's malloc, etc. is expected to work
>> at all? Perhaps banning it altogether is just the safest thing to
>> do?
>
> Don't know why you want to ban everything.

To guarantee predictable and consistent behavior.

-Y

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

On Fri, May 23, 2014 at 11:20 AM, Yury Gribov <y.gribov@samsung.com> wrote:
> On 05/23/2014 10:34 AM, Jakub Jelinek wrote:
>>>>
>>>> Otherwise libasan apps will simply stop
>>>> working altogether if LD_PRELOAD is set, to whatever library,
>>>> even if it doesn't define any symbols you care about.
>>>
>>>
>>> Right but I'm not sure whether failing fast here is necessarily bad.
>>
>>
>> I think it is very bad.  In fact, if you really want such a check,
>> I'd say it shouldn't be at least enabled by default, unless some
>> env var requests it; and document that if you are having troubles with
>> asan sanitized programs, try this magic env var to get better
>> troubleshooting.
>
>
> As I said, we can remove that Die() there. Warning can also be hidden under
> ASAN_OPTIONS=verbosity=1.
>
>
>> Even before this exaggerated check asan imposes far more restrictions than
>> good, and this just makes asan less usable just for fear that it wouldn't
>> work right.
>
>
> Ok, we seem to approach this from two different angles.
> I usually try to prohibit functionality that's not proven to work
> It could be later enabled if users need it.
>
>
>>
>> Most preloaded libs will just provide symbols asan never cares
>> about
>
>
> Maybe. In my experience it's all libc decorations.
>
>
>> even if you say override malloc completely without
>> calling the original implementation, the world doesn't end, the shadow
>> mem of those allocations just won't be surrounded by protected paddings,
>> so what, you don't detect out of bounds for malloc, but can still detect
>> out of bounds in your program's stack etc.
>> Ditto for string ops etc.
>
>
> Let's wait for Konstantin to comment on this.
> I don't know runtime well enough to guarantee that arbitrary symbol
> overloads are expected to work.

[These things are really better discussed at
address-sanitizer@googlegroups.com, where more asan people will read.
maybe we should start a separate topic there. This conversation is
already too long to comprehend. ]

Failing to intercept something may cause not just false negatives, but
also false positives.
These cases are often exceptionally hard to debug, so any checking that
the interception machinery works as intended is good. Of course if
these checks are wrong we should fix them.

>
>
>>> Imagine preloaded library has an initializer which calls intercepted
>>> APIs. Asan didn't get a chance to initialize at the point of call
>>> and if interceptor doesn't contain a sanity call to asan_init,
>>> we are risking hard-to-debug runtime error (call to NULL, etc.).
>>> I've seen numerous bugs like this (both locally and on mailing
>>> lists) and they were main motivation to add this check.
>>
>>
>> That is nonsense.  Early in the symbol search scope is the opposite of
>> being
>> initialized early, on the contrary, such libraries are initialized last.
>
>
> I may be wrong but my understanding was that ld.so performs a reverse
> topological sort of dependencies and initializes them in that order.
> Given that libasan depends on standard libs (librt, lipthread, libdl, etc.)
> it'll be initialized after them but before user libs.
> Initializers of std libs may indeed cause problems but we at least make sure
> to initialize before arbitrary user libraries.
>
>
>>> I wonder whether overriding Asan's malloc, etc. is expected to work
>>> at all? Perhaps banning it altogether is just the safest thing to
>>> do?
>>
>>
>> Don't know why you want to ban everything.
>
>
> To guarantee predictable and consistent behavior.
>
> -Y

Re: libsanitizer merge from upstream r208536

[Jakub Jelinek]

On Fri, May 23, 2014 at 11:34:38AM +0400, Konstantin Serebryany wrote:
> Failing to intercept something may cause not just false negatives, but
> also false positives.
> These cases are often exceptionally hard to debug, so any checking that
> the interception machinery works as intended is good. Of course if
> these checks are wrong we should fix them.

But at least don't report anything when the program starts, but at the
end of the ASAN failure report (if anything has been detected).
Like:
WARNING: libasan.so.1 isn't early in the symbol search scope, some symbols
in it might be interposed.  Try LD_PRELOAD=libasan.so.1.

or so.  Of course, LD_PRELOAD=libasan.so.1 wouldn't help with this check,
because the vDSO still can come up earlier.

	Jakub

Re: libsanitizer merge from upstream r208536

[Paolo Carlini]

Hi,

On 05/23/2014 09:47 AM, Jakub Jelinek wrote:
> On Fri, May 23, 2014 at 11:34:38AM +0400, Konstantin Serebryany wrote:
>> Failing to intercept something may cause not just false negatives, but
>> also false positives.
>> These cases are often exceptionally hard to debug, so any checking that
>> the interception machinery works as intended is good. Of course if
>> these checks are wrong we should fix them.
> But at least don't report anything when the program starts, but at the
> end of the ASAN failure report (if anything has been detected).
> Like:
> WARNING: libasan.so.1 isn't early in the symbol search scope, some symbols
> in it might be interposed.  Try LD_PRELOAD=libasan.so.1.
>
> or so.
Yes please. For me, and I don't think my setup is such uncommon, the 
noise is quite annoying.

Paolo.

Re: libsanitizer merge from upstream r208536

[Jakub Jelinek]

On Fri, May 23, 2014 at 11:20:01AM +0400, Yury Gribov wrote:
> >Even before this exaggerated check asan imposes far more restrictions than
> >good, and this just makes asan less usable just for fear that it wouldn't
> >work right.
> 
> Ok, we seem to approach this from two different angles.
> I usually try to prohibit functionality that's not proven to work
> It could be later enabled if users need it.

No other shared library does anything close to that, for each such library
you can interpose any of its public symbols, either you know what you are
doing when interposing it, or it breaks.

> >That is nonsense.  Early in the symbol search scope is the opposite of being
> >initialized early, on the contrary, such libraries are initialized last.
> 
> I may be wrong but my understanding was that ld.so performs a
> reverse topological sort of dependencies and initializes them in
> that order.
> Given that libasan depends on standard libs (librt, lipthread,
> libdl, etc.) it'll be initialized after them but before user libs.
> Initializers of std libs may indeed cause problems but we at least
> make sure to initialize before arbitrary user libraries.

Just try say
LD_DEBUG=all LD_PRELOAD=libasan.so.1 /bin/bash
to see (non-instrumented bash or any of its shared libraries).
...
     30218:     object=/bin/bash [0]
     30218:      scope 0: /bin/bash ./libasan.so.1.0.0 /lib64/libtinfo.so.5 /lib64/libdl.so.2 /lib64/libc.so.6 /lib64/libpthread.so.0 /usr/src/gcc/obj2/x86_64-unknown-linux-gnu/libstdc++-v3/src/.libs/libstdc++.so.6 /lib64/libm.so.6 /lib64/ld-linux-x86-64.so.2 /lib64/libgcc_s.so.1
...
     30218:	calling init: /lib64/libpthread.so.0
     30218:	calling init: /lib64/libc.so.6
     30218:	calling init: /lib64/libgcc_s.so.1
     30218:	calling init: /lib64/libm.so.6
     30218:	calling init: /usr/src/gcc/obj2/x86_64-unknown-linux-gnu/libstdc++-v3/src/.libs/libstdc++.so.6
     30218:	calling init: /lib64/libdl.so.2
     30218:	calling init: /lib64/libtinfo.so.5
     30218:	calling init: ./libasan.so.1.0.0
libasan.so.1 doesn't depend on libtinfo.so.5, only bash itself does,
yet libtinfo.so.5 is constructed before libasan.so.1.

	Jakub

Re: libsanitizer merge from upstream r208536

[Yuri Gribov]

On Fri, May 23, 2014 at 11:40 AM, Jakub Jelinek <jakub@redhat.com> wrote:
> No other shared library does anything close to that, for each such library
> you can interpose any of its public symbols, either you know what you are
> doing when interposing it, or it breaks.

I think I have finally recalled why we added this check. They main
usecase was sanitized dynamic library linked into unsanitized
executable. This of course requires that libasan comes earlier in
loaded DSO list than libc libraries so that malloc, etc. can be
successfully intercepted with dlsym(..., RTLD_NEXT) in libasan (for
more details grep for LD_PRELOAD in
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=56393).

Jakub is probably right that forcing libasan to be the first library
in DSO list (that's what AsanCheckDynamicRTPrereqs currently does) is
an overkill but it felt a lot easier to do back in the day.

As already said, removing abort and hiding warning under verbosity>0
runtime option is probably enough to support usecases mentioned by
Jakub (preloading some other DSO before libasan). As Konstantin
mentioned, this should be discussed in address-sanitizer ML which I'll
probably do on Monday if noone takes the lead.

-Y

Re: libsanitizer merge from upstream r208536

[Yury Gribov]

Paolo,

I've checked all available systems and wasn't able to repro this.
Every time vDSO was already filtered with

   if (!info->dlpi_name || info->dlpi_name[0] == 0)
     return 0;

in FindFirstDSOCallback.

Could you provide additional details of your setup? Or perhaps print 
dlpi_name of offending library?

-Y

Re: libsanitizer merge from upstream r208536

[Paolo Carlini]

Hi,

On 05/23/2014 10:50 AM, Yury Gribov wrote:
> Paolo,
>
> I've checked all available systems and wasn't able to repro this.
Given your exchanges with Jakub I thought that at this point it was 
clear that the issue is real.
> Every time vDSO was already filtered with
>
>   if (!info->dlpi_name || info->dlpi_name[0] == 0)
>     return 0;
>
> in FindFirstDSOCallback.
>
> Could you provide additional details of your setup? Or perhaps print 
> dlpi_name of offending library?
How do I print dlpi_name? And which detail do you want? It's a very, 
very, standard Linux machine, running a 3.11.10 kernel and a 2.18 glibc, 
two days ago everything was fine, 4_9-branch is fine.

Paolo.

Re: libsanitizer merge from upstream r208536

[Yury Gribov]

>> I've checked all available systems and wasn't able to repro this.
>
> Given your exchanges with Jakub I thought that at this point it was
> clear that the issue is real.

There are three issues here:
1) whether warning should cause termination
2) whether warning should be displayed by default
3) why warning occurs on your machine

We mainly discussed 1 and 2 with Jakub but even if we hide this warning 
it still should have come up on your machine in the first place.
It's either false positive which needs to be fixed or symptom of some 
real problem. I believe it still makes sense to investigate the original 
problem.

>> Every time vDSO was already filtered with
>>
>>   if (!info->dlpi_name || info->dlpi_name[0] == 0)
>>     return 0;
>>
>> in FindFirstDSOCallback.
>>
>> Could you provide additional details of your setup? Or perhaps print
>> dlpi_name of offending library?
>
> How do I print dlpi_name?

Could you add something like
   Report("'%s'\n", info->dlpi_name);
after
   if (!info->dlpi_name || info->dlpi_name[0] == 0)
check in FindFirstDSOCallback? This should give us the name of library 
which causes problems.

> And which detail do you want?

Just the name of the library.

>It's a very,
> very, standard Linux machine, running a 3.11.10 kernel and a 2.18 glibc,
> two days ago everything was fine, 4_9-branch is fine.

True but being unable to repro this, I'd need some additional help to 
diagnose the problem.

-Y

Re: libsanitizer merge from upstream r208536

[Yury Gribov]

 > still should have come up on your machine in the first place.

should not have

Re: libsanitizer merge from upstream r208536

[Paolo Carlini]

Hi,

On 05/23/2014 11:21 AM, Yury Gribov wrote:
>>> I've checked all available systems and wasn't able to repro this.
>>
>> Given your exchanges with Jakub I thought that at this point it was
>> clear that the issue is real.
>
> There are three issues here:
> 1) whether warning should cause termination
> 2) whether warning should be displayed by default
> 3) why warning occurs on your machine
>
> We mainly discussed 1 and 2 with Jakub but even if we hide this 
> warning it still should have come up on your machine in the first place.
> It's either false positive which needs to be fixed or symptom of some 
> real problem. I believe it still makes sense to investigate the 
> original problem.
>
>>> Every time vDSO was already filtered with
>>>
>>>   if (!info->dlpi_name || info->dlpi_name[0] == 0)
>>>     return 0;
>>>
>>> in FindFirstDSOCallback.
>>>
>>> Could you provide additional details of your setup? Or perhaps print
>>> dlpi_name of offending library?
>>
>> How do I print dlpi_name?
>
> Could you add something like
>   Report("'%s'\n", info->dlpi_name);
> after
>   if (!info->dlpi_name || info->dlpi_name[0] == 0)
> check in FindFirstDSOCallback? This should give us the name of library 
> which causes problems.
It's always linux-vdso.so.1, but wasn't that already known, given the 
ldd requested by Jakub?!?

Paolo.

Re: libsanitizer merge from upstream r208536

[Jakub Jelinek]

On Fri, May 23, 2014 at 11:25:02AM +0200, Paolo Carlini wrote:
> >>How do I print dlpi_name?
> >
> >Could you add something like
> >  Report("'%s'\n", info->dlpi_name);
> >after
> >  if (!info->dlpi_name || info->dlpi_name[0] == 0)
> >check in FindFirstDSOCallback? This should give us the name of
> >library which causes problems.
> It's always linux-vdso.so.1, but wasn't that already known, given
> the ldd requested by Jakub?!?

Bet it depends on the versions of glibc and kernel you have,
where exactly in the search scope glibc inserts the vDSO and
whether the kernel provides DT_SONAME for it.

	Jakub

Re: libsanitizer merge from upstream r208536

[Yury Gribov]

[-- Attachment #1: Type: text/plain, Size: 479 bytes --]

 >> Could you add something like
 >
 > It's always linux-vdso.so.1, but wasn't that already known, given the
 > ldd requested by Jakub?!?

Well, for me dlpi_name for vdso was empty string hence I kept asking. I 
also thought that ldd and dl_iterate_phdr might have used slightly 
different code paths when quering information from dynamic linker.

Could you check if the attached patch fixes the problem for you?
Note that I only did limited testing (RUNTESTFLAGS=asan.exp).

-Y

[-- Attachment #2: vdso_fix_1.diff --]
[-- Type: text/x-diff, Size: 514 bytes --]

diff --git a/libsanitizer/asan/asan_linux.cc b/libsanitizer/asan/asan_linux.cc
index d893b23..11137d9 100644
--- a/libsanitizer/asan/asan_linux.cc
+++ b/libsanitizer/asan/asan_linux.cc
@@ -89,6 +89,10 @@ static int FindFirstDSOCallback(struct dl_phdr_info *info, size_t size,
   if (!info->dlpi_name || info->dlpi_name[0] == 0)
     return 0;
 
+  // Ignore vDSO
+  if (internal_strncmp(info->dlpi_name, "linux-", sizeof("linux-")) == 0)
+    return 0;
+
   *(const char **)data = info->dlpi_name;
   return 1;
 }

Re: libsanitizer merge from upstream r208536

[Paolo Carlini]

Hi,

On 05/23/2014 12:26 PM, Yury Gribov wrote:
> >> Could you add something like
> >
> > It's always linux-vdso.so.1, but wasn't that already known, given the
> > ldd requested by Jakub?!?
>
> Well, for me dlpi_name for vdso was empty string hence I kept asking. 
> I also thought that ldd and dl_iterate_phdr might have used slightly 
> different code paths when quering information from dynamic linker.
>
> Could you check if the attached patch fixes the problem for you?
> Note that I only did limited testing (RUNTESTFLAGS=asan.exp).
Thanks. It appears to work great for me modulo a trivial off-by-one (you 
want sizeof(...) - 1) and the asan_test.C issue already discussed by Jakub.

Paolo.

Re: libsanitizer merge from upstream r208536

[Paolo Carlini]

Hi,

On 05/22/2014 09:02 PM, Jakub Jelinek wrote:
> In file included from
> ../../../../trunk/libsanitizer/asan/asan_interceptors.cc:147:0:
> ../../../../trunk/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:
> In function ‘int __interceptor_accept4(int, void*, unsigned int*,
> int)’:
> ../../../../trunk/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:1821:12:
> warning: ‘addrlen0’ may be used uninitialized in this function
> [-Wmaybe-uninitialized]
> unsigned addrlen0;
> ^
> ../../../../trunk/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:
> In function ‘int __interceptor_accept(int, void*, unsigned int*)’:
> ../../../../trunk/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:1799:12:
> warning: ‘addrlen0’ may be used uninitialized in this function
> [-Wmaybe-uninitialized]
> unsigned addrlen0;
> ^
> That sounds like a false positive warning:
>    unsigned addrlen0;
>    if (addrlen) {
>      COMMON_INTERCEPTOR_READ_RANGE(ctx, addrlen, sizeof(*addrlen));
>      addrlen0 = *addrlen;
>    }
>    int fd2 = REAL(accept4)(fd, addr, addrlen, f);
>    if (fd2 >= 0) {
>      if (fd >= 0) COMMON_INTERCEPTOR_FD_SOCKET_ACCEPT(ctx, fd, fd2);
>      if (addr && addrlen)
>        COMMON_INTERCEPTOR_WRITE_RANGE(ctx, addr, Min(*addrlen, addrlen0));
>    }
> (unless the COMMON_INTERCEPTOR* macros do too weird stuff), wonder why the
> predicate aware uninit doesn't handle this.
By the way, I'm still seeing the above. Maybe a maintainer can have a 
look, double check it's just a false positive a shut it up somehow?

Thanks!
Paolo.

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

On Wed, Jun 11, 2014 at 2:28 PM, Paolo Carlini <paolo.carlini@oracle.com> wrote:
> Hi,
>
> On 05/22/2014 09:02 PM, Jakub Jelinek wrote:
>>
>> In file included from
>> ../../../../trunk/libsanitizer/asan/asan_interceptors.cc:147:0:
>>
>> ../../../../trunk/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:
>> In function ‘int __interceptor_accept4(int, void*, unsigned int*,
>> int)’:
>>
>> ../../../../trunk/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:1821:12:
>> warning: ‘addrlen0’ may be used uninitialized in this function
>> [-Wmaybe-uninitialized]
>> unsigned addrlen0;
>> ^
>>
>> ../../../../trunk/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:
>> In function ‘int __interceptor_accept(int, void*, unsigned int*)’:
>>
>> ../../../../trunk/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:1799:12:
>> warning: ‘addrlen0’ may be used uninitialized in this function
>> [-Wmaybe-uninitialized]
>> unsigned addrlen0;
>> ^
>> That sounds like a false positive warning:
>>    unsigned addrlen0;
>>    if (addrlen) {
>>      COMMON_INTERCEPTOR_READ_RANGE(ctx, addrlen, sizeof(*addrlen));
>>      addrlen0 = *addrlen;
>>    }
>>    int fd2 = REAL(accept4)(fd, addr, addrlen, f);
>>    if (fd2 >= 0) {
>>      if (fd >= 0) COMMON_INTERCEPTOR_FD_SOCKET_ACCEPT(ctx, fd, fd2);
>>      if (addr && addrlen)
>>        COMMON_INTERCEPTOR_WRITE_RANGE(ctx, addr, Min(*addrlen, addrlen0));
>>    }
>> (unless the COMMON_INTERCEPTOR* macros do too weird stuff), wonder why the
>> predicate aware uninit doesn't handle this.
>
> By the way, I'm still seeing the above. Maybe a maintainer can have a look,
> double check it's just a false positive a shut it up somehow?

I've "fixed" this in upstream trunk:
http://llvm.org/viewvc/llvm-project?view=revision&revision=211008
This will get into GCC with the next merge; or feel free to cherry pick.

Meanwhile, this still smells like a bug in GCC's -Wmaybe-uninitialized
-- I'd encourage you to at least file a bug on it.

--kcc

>
> Thanks!
> Paolo.

Re: libsanitizer merge from upstream r208536

[Paolo Carlini]

Hi,

On 16/06/14 10:42, Konstantin Serebryany wrote:
> I've "fixed" this in upstream trunk: 
> http://llvm.org/viewvc/llvm-project?view=revision&revision=211008 This 
> will get into GCC with the next merge; or feel free to cherry pick.
Excellent thanks a lot.
> Meanwhile, this still smells like a bug in GCC's -Wmaybe-uninitialized 
> -- I'd encourage you to at least file a bug on it.
Yes, but we should first double check the issue isn't already known and, 
otherwise, figure out a minimized testcase.

Thanks,
Paolo.

[PATCH][libsanitizer] Cherry-pick r211008 [Was: Re: libsanitizer merge from upstream r208536]

[Paolo Carlini]

[-- Attachment #1: Type: text/plain, Size: 2119 bytes --]

Hi,

On 06/16/2014 10:42 AM, Konstantin Serebryany wrote:
> On Wed, Jun 11, 2014 at 2:28 PM, Paolo Carlini <paolo.carlini@oracle.com> wrote:
>> Hi,
>>
>> On 05/22/2014 09:02 PM, Jakub Jelinek wrote:
>>> In file included from
>>> ../../../../trunk/libsanitizer/asan/asan_interceptors.cc:147:0:
>>>
>>> ../../../../trunk/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:
>>> In function ‘int __interceptor_accept4(int, void*, unsigned int*,
>>> int)’:
>>>
>>> ../../../../trunk/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:1821:12:
>>> warning: ‘addrlen0’ may be used uninitialized in this function
>>> [-Wmaybe-uninitialized]
>>> unsigned addrlen0;
>>> ^
>>>
>>> ../../../../trunk/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:
>>> In function ‘int __interceptor_accept(int, void*, unsigned int*)’:
>>>
>>> ../../../../trunk/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:1799:12:
>>> warning: ‘addrlen0’ may be used uninitialized in this function
>>> [-Wmaybe-uninitialized]
>>> unsigned addrlen0;
>>> ^
>>> That sounds like a false positive warning:
>>>     unsigned addrlen0;
>>>     if (addrlen) {
>>>       COMMON_INTERCEPTOR_READ_RANGE(ctx, addrlen, sizeof(*addrlen));
>>>       addrlen0 = *addrlen;
>>>     }
>>>     int fd2 = REAL(accept4)(fd, addr, addrlen, f);
>>>     if (fd2 >= 0) {
>>>       if (fd >= 0) COMMON_INTERCEPTOR_FD_SOCKET_ACCEPT(ctx, fd, fd2);
>>>       if (addr && addrlen)
>>>         COMMON_INTERCEPTOR_WRITE_RANGE(ctx, addr, Min(*addrlen, addrlen0));
>>>     }
>>> (unless the COMMON_INTERCEPTOR* macros do too weird stuff), wonder why the
>>> predicate aware uninit doesn't handle this.
>> By the way, I'm still seeing the above. Maybe a maintainer can have a look,
>> double check it's just a false positive a shut it up somehow?
> I've "fixed" this in upstream trunk:
> http://llvm.org/viewvc/llvm-project?view=revision&revision=211008
> This will get into GCC with the next merge; or feel free to cherry pick.
Thus, can I apply the below?

Thanks!
Paolo.

/////////////////////

[-- Attachment #2: CL --]
[-- Type: text/plain, Size: 141 bytes --]

2014-06-23  Paolo Carlini  <paolo.carlini@oracle.com>

	* sanitizer_common/sanitizer_common_interceptors.inc:
	Cherry pick upstream r211008.

[-- Attachment #3: p --]
[-- Type: text/plain, Size: 1031 bytes --]

Index: sanitizer_common/sanitizer_common_interceptors.inc
===================================================================
--- sanitizer_common/sanitizer_common_interceptors.inc	(revision 211905)
+++ sanitizer_common/sanitizer_common_interceptors.inc	(working copy)
@@ -1796,7 +1796,7 @@ INTERCEPTOR(int, getsockopt, int sockfd, int level
 INTERCEPTOR(int, accept, int fd, void *addr, unsigned *addrlen) {
   void *ctx;
   COMMON_INTERCEPTOR_ENTER(ctx, accept, fd, addr, addrlen);
-  unsigned addrlen0;
+  unsigned addrlen0 = 0;
   if (addrlen) {
     COMMON_INTERCEPTOR_READ_RANGE(ctx, addrlen, sizeof(*addrlen));
     addrlen0 = *addrlen;
@@ -1818,7 +1818,7 @@ INTERCEPTOR(int, accept, int fd, void *addr, unsig
 INTERCEPTOR(int, accept4, int fd, void *addr, unsigned *addrlen, int f) {
   void *ctx;
   COMMON_INTERCEPTOR_ENTER(ctx, accept4, fd, addr, addrlen, f);
-  unsigned addrlen0;
+  unsigned addrlen0 = 0;
   if (addrlen) {
     COMMON_INTERCEPTOR_READ_RANGE(ctx, addrlen, sizeof(*addrlen));
     addrlen0 = *addrlen;

Re: [PATCH][libsanitizer] Cherry-pick r211008 [Was: Re: libsanitizer merge from upstream r208536]

[Jakub Jelinek]

On Mon, Jun 23, 2014 at 07:23:09PM +0200, Paolo Carlini wrote:
> /////////////////////

> 2014-06-23  Paolo Carlini  <paolo.carlini@oracle.com>
> 
> 	* sanitizer_common/sanitizer_common_interceptors.inc:
> 	Cherry pick upstream r211008.

Sure, thanks.

> Index: sanitizer_common/sanitizer_common_interceptors.inc
> ===================================================================
> --- sanitizer_common/sanitizer_common_interceptors.inc	(revision 211905)
> +++ sanitizer_common/sanitizer_common_interceptors.inc	(working copy)
> @@ -1796,7 +1796,7 @@ INTERCEPTOR(int, getsockopt, int sockfd, int level
>  INTERCEPTOR(int, accept, int fd, void *addr, unsigned *addrlen) {
>    void *ctx;
>    COMMON_INTERCEPTOR_ENTER(ctx, accept, fd, addr, addrlen);
> -  unsigned addrlen0;
> +  unsigned addrlen0 = 0;
>    if (addrlen) {
>      COMMON_INTERCEPTOR_READ_RANGE(ctx, addrlen, sizeof(*addrlen));
>      addrlen0 = *addrlen;
> @@ -1818,7 +1818,7 @@ INTERCEPTOR(int, accept, int fd, void *addr, unsig
>  INTERCEPTOR(int, accept4, int fd, void *addr, unsigned *addrlen, int f) {
>    void *ctx;
>    COMMON_INTERCEPTOR_ENTER(ctx, accept4, fd, addr, addrlen, f);
> -  unsigned addrlen0;
> +  unsigned addrlen0 = 0;
>    if (addrlen) {
>      COMMON_INTERCEPTOR_READ_RANGE(ctx, addrlen, sizeof(*addrlen));
>      addrlen0 = *addrlen;


	Jakub

Re: libsanitizer merge from upstream r208536

[Ramana Radhakrishnan]

On Thu, May 22, 2014 at 7:31 AM, Konstantin Serebryany
<konstantin.s.serebryany@gmail.com> wrote:
> On Wed, May 21, 2014 at 11:43 PM, Jakub Jelinek <jakub@redhat.com> wrote:
>> On Wed, May 21, 2014 at 04:09:19PM +0400, Konstantin Serebryany wrote:
>>> A new patch based on r209283.
>>> This one has the H.J.'s patches for x32.
>>
>> Ok for trunk then.  But please help the ppc*/arm*/sparc* maintainers if issues on
>> those targets are reported.
> Of course. arm should be in a good shape since there are arm users
> upstream, including ourselves.

On ARM the asan tests have always been a random generator of PASS /
FAIL on qemu despite efforts to "nobble" qemu for /proc/self/maps
outputs.

On a board where this appears to work well ( my A15 / A7 Odroid XU at home)

https://gcc.gnu.org/ml/gcc-testresults/2014-05/msg01902.html

the set of results from before the merge. indicates the following failures.

Running target unix
FAIL: c-c++-common/asan/asan-interface-1.c  -O0  (test for excess errors)
UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -O0  compilation
failed to produce executable
FAIL: c-c++-common/asan/asan-interface-1.c  -O1  (test for excess errors)
UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -O1  compilation
failed to produce executable
FAIL: c-c++-common/asan/asan-interface-1.c  -O2  (test for excess errors)
UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -O2  compilation
failed to produce executable
FAIL: c-c++-common/asan/asan-interface-1.c  -O3 -fomit-frame-pointer
(test for excess errors)
UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -O3
-fomit-frame-pointer  compilation failed to produce executable
FAIL: c-c++-common/asan/asan-interface-1.c  -O3 -g  (test for excess errors)
UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -O3 -g  compilation
failed to produce executable
FAIL: c-c++-common/asan/asan-interface-1.c  -Os  (test for excess errors)
UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -Os  compilation
failed to produce executable
FAIL: c-c++-common/asan/asan-interface-1.c  -O2 -flto
-fno-use-linker-plugin -flto-partition=none  (test for excess errors)
UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -O2 -flto
-fno-use-linker-plugin -flto-partition=none  compilation failed to
produce executable
FAIL: c-c++-common/asan/asan-interface-1.c  -O2 -flto
-fuse-linker-plugin -fno-fat-lto-objects  (test for excess errors)
UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -O2 -flto
-fuse-linker-plugin -fno-fat-lto-objects  compilation failed to
produce executable

After the merge I see these new failures instead

https://gcc.gnu.org/ml/gcc-testresults/2014-05/msg02018.html

FAIL: c-c++-common/asan/heap-overflow-1.c  -O0  output pattern test,
is =================================================================
FAIL: c-c++-common/asan/heap-overflow-1.c  -O1  output pattern test,
is =================================================================
FAIL: c-c++-common/asan/heap-overflow-1.c  -O2  output pattern test,
is =================================================================
FAIL: c-c++-common/asan/heap-overflow-1.c  -O3 -fomit-frame-pointer
output pattern test, is
=================================================================
FAIL: c-c++-common/asan/heap-overflow-1.c  -O3 -g  output pattern
test, is =================================================================
FAIL: c-c++-common/asan/heap-overflow-1.c  -Os  output pattern test,
is =================================================================
FAIL: c-c++-common/asan/heap-overflow-1.c  -O2 -flto
-fno-use-linker-plugin -flto-partition=none  output pattern test, is
=================================================================
FAIL: c-c++-common/asan/heap-overflow-1.c  -O2 -flto
-fuse-linker-plugin -fno-fat-lto-objects  output pattern test, is
=================================================================
FAIL: c-c++-common/asan/sanity-check-pure-c-1.c  -O0  output pattern
test, is =================================================================
FAIL: c-c++-common/asan/sanity-check-pure-c-1.c  -O1  output pattern
test, is =================================================================
FAIL: c-c++-common/asan/sanity-check-pure-c-1.c  -O2  output pattern
test, is =================================================================
FAIL: c-c++-common/asan/sanity-check-pure-c-1.c  -O3
-fomit-frame-pointer  output pattern test, is
=================================================================
FAIL: c-c++-common/asan/sanity-check-pure-c-1.c  -O3 -g  output
pattern test, is
=================================================================
FAIL: c-c++-common/asan/sanity-check-pure-c-1.c  -Os  output pattern
test, is =================================================================
FAIL: c-c++-common/asan/sanity-check-pure-c-1.c  -O2 -flto
-fno-use-linker-plugin -flto-partition=none  output pattern test, is
=================================================================
FAIL: c-c++-common/asan/sanity-check-pure-c-1.c  -O2 -flto
-fuse-linker-plugin -fno-fat-lto-objects  output pattern test, is
=================================================================
FAIL: c-c++-common/asan/strncpy-overflow-1.c  -O0  output pattern
test, is =================================================================
FAIL: c-c++-common/asan/strncpy-overflow-1.c  -O1  output pattern
test, is =================================================================
FAIL: c-c++-common/asan/strncpy-overflow-1.c  -O2  output pattern
test, is =================================================================
FAIL: c-c++-common/asan/strncpy-overflow-1.c  -O3 -fomit-frame-pointer
 output pattern test, is
=================================================================
FAIL: c-c++-common/asan/strncpy-overflow-1.c  -O3 -g  output pattern
test, is =================================================================
FAIL: c-c++-common/asan/strncpy-overflow-1.c  -Os  output pattern
test, is =================================================================
FAIL: c-c++-common/asan/strncpy-overflow-1.c  -O2 -flto
-fno-use-linker-plugin -flto-partition=none  output pattern test, is
=================================================================
FAIL: c-c++-common/asan/strncpy-overflow-1.c  -O2 -flto
-fuse-linker-plugin -fno-fat-lto-objects  output pattern test, is
=================================================================
FAIL: c-c++-common/asan/use-after-free-1.c  -O0  output pattern test,
is =================================================================
FAIL: c-c++-common/asan/use-after-free-1.c  -O1  output pattern test,
is =================================================================
FAIL: c-c++-common/asan/use-after-free-1.c  -O2  output pattern test,
is =================================================================
FAIL: c-c++-common/asan/use-after-free-1.c  -O3 -fomit-frame-pointer
output pattern test, is
=================================================================
FAIL: c-c++-common/asan/use-after-free-1.c  -O3 -g  output pattern
test, is =================================================================
FAIL: c-c++-common/asan/use-after-free-1.c  -Os  output pattern test,
is =================================================================
FAIL: c-c++-common/asan/use-after-free-1.c  -O2 -flto
-fno-use-linker-plugin -flto-partition=none  output pattern test, is
=================================================================
FAIL: c-c++-common/asan/use-after-free-1.c  -O2 -flto
-fuse-linker-plugin -fno-fat-lto-objects  output pattern test, is
================



regards
Ramana

> ppc may get broken -- I haven't heard from anyone there for a while.
> Same for sparc.
> I'll commit later today.
>
>
>> Thanks.
>>
>>         Jakub

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

On Fri, May 23, 2014 at 11:32 AM, Ramana Radhakrishnan
<ramana.gcc@googlemail.com> wrote:
> On Thu, May 22, 2014 at 7:31 AM, Konstantin Serebryany
> <konstantin.s.serebryany@gmail.com> wrote:
>> On Wed, May 21, 2014 at 11:43 PM, Jakub Jelinek <jakub@redhat.com> wrote:
>>> On Wed, May 21, 2014 at 04:09:19PM +0400, Konstantin Serebryany wrote:
>>>> A new patch based on r209283.
>>>> This one has the H.J.'s patches for x32.
>>>
>>> Ok for trunk then.  But please help the ppc*/arm*/sparc* maintainers if issues on
>>> those targets are reported.
>> Of course. arm should be in a good shape since there are arm users
>> upstream, including ourselves.
>
> On ARM the asan tests have always been a random generator of PASS /
> FAIL on qemu despite efforts to "nobble" qemu for /proc/self/maps
> outputs.

We ourselves test only Android ARM on a real box. There the tests work.
On ARM Linux there are quite a few known test failures (mostly due to
unwinding), and no public regular testing.
See e.g. http://reviews.llvm.org/D3857
We don't have any experience with running asan on quemu and I remember
some complaints regarding quemu itself,
e.g. https://code.google.com/p/address-sanitizer/issues/detail?id=160

As usual: if you are interested in supporting asan on any given
platform, please work with us upstream.


>
> On a board where this appears to work well ( my A15 / A7 Odroid XU at home)
>
> https://gcc.gnu.org/ml/gcc-testresults/2014-05/msg01902.html
>
> the set of results from before the merge. indicates the following failures.
>
> Running target unix
> FAIL: c-c++-common/asan/asan-interface-1.c  -O0  (test for excess errors)
> UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -O0  compilation
> failed to produce executable
> FAIL: c-c++-common/asan/asan-interface-1.c  -O1  (test for excess errors)
> UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -O1  compilation
> failed to produce executable
> FAIL: c-c++-common/asan/asan-interface-1.c  -O2  (test for excess errors)
> UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -O2  compilation
> failed to produce executable
> FAIL: c-c++-common/asan/asan-interface-1.c  -O3 -fomit-frame-pointer
> (test for excess errors)
> UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -O3
> -fomit-frame-pointer  compilation failed to produce executable
> FAIL: c-c++-common/asan/asan-interface-1.c  -O3 -g  (test for excess errors)
> UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -O3 -g  compilation
> failed to produce executable
> FAIL: c-c++-common/asan/asan-interface-1.c  -Os  (test for excess errors)
> UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -Os  compilation
> failed to produce executable
> FAIL: c-c++-common/asan/asan-interface-1.c  -O2 -flto
> -fno-use-linker-plugin -flto-partition=none  (test for excess errors)
> UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -O2 -flto
> -fno-use-linker-plugin -flto-partition=none  compilation failed to
> produce executable
> FAIL: c-c++-common/asan/asan-interface-1.c  -O2 -flto
> -fuse-linker-plugin -fno-fat-lto-objects  (test for excess errors)
> UNRESOLVED: c-c++-common/asan/asan-interface-1.c  -O2 -flto
> -fuse-linker-plugin -fno-fat-lto-objects  compilation failed to
> produce executable
>
> After the merge I see these new failures instead
>
> https://gcc.gnu.org/ml/gcc-testresults/2014-05/msg02018.html
>
> FAIL: c-c++-common/asan/heap-overflow-1.c  -O0  output pattern test,
> is =================================================================
> FAIL: c-c++-common/asan/heap-overflow-1.c  -O1  output pattern test,
> is =================================================================
> FAIL: c-c++-common/asan/heap-overflow-1.c  -O2  output pattern test,
> is =================================================================
> FAIL: c-c++-common/asan/heap-overflow-1.c  -O3 -fomit-frame-pointer
> output pattern test, is
> =================================================================
> FAIL: c-c++-common/asan/heap-overflow-1.c  -O3 -g  output pattern
> test, is =================================================================
> FAIL: c-c++-common/asan/heap-overflow-1.c  -Os  output pattern test,
> is =================================================================
> FAIL: c-c++-common/asan/heap-overflow-1.c  -O2 -flto
> -fno-use-linker-plugin -flto-partition=none  output pattern test, is
> =================================================================
> FAIL: c-c++-common/asan/heap-overflow-1.c  -O2 -flto
> -fuse-linker-plugin -fno-fat-lto-objects  output pattern test, is
> =================================================================
> FAIL: c-c++-common/asan/sanity-check-pure-c-1.c  -O0  output pattern
> test, is =================================================================
> FAIL: c-c++-common/asan/sanity-check-pure-c-1.c  -O1  output pattern
> test, is =================================================================
> FAIL: c-c++-common/asan/sanity-check-pure-c-1.c  -O2  output pattern
> test, is =================================================================
> FAIL: c-c++-common/asan/sanity-check-pure-c-1.c  -O3
> -fomit-frame-pointer  output pattern test, is
> =================================================================
> FAIL: c-c++-common/asan/sanity-check-pure-c-1.c  -O3 -g  output
> pattern test, is
> =================================================================
> FAIL: c-c++-common/asan/sanity-check-pure-c-1.c  -Os  output pattern
> test, is =================================================================
> FAIL: c-c++-common/asan/sanity-check-pure-c-1.c  -O2 -flto
> -fno-use-linker-plugin -flto-partition=none  output pattern test, is
> =================================================================
> FAIL: c-c++-common/asan/sanity-check-pure-c-1.c  -O2 -flto
> -fuse-linker-plugin -fno-fat-lto-objects  output pattern test, is
> =================================================================
> FAIL: c-c++-common/asan/strncpy-overflow-1.c  -O0  output pattern
> test, is =================================================================
> FAIL: c-c++-common/asan/strncpy-overflow-1.c  -O1  output pattern
> test, is =================================================================
> FAIL: c-c++-common/asan/strncpy-overflow-1.c  -O2  output pattern
> test, is =================================================================
> FAIL: c-c++-common/asan/strncpy-overflow-1.c  -O3 -fomit-frame-pointer
>  output pattern test, is
> =================================================================
> FAIL: c-c++-common/asan/strncpy-overflow-1.c  -O3 -g  output pattern
> test, is =================================================================
> FAIL: c-c++-common/asan/strncpy-overflow-1.c  -Os  output pattern
> test, is =================================================================
> FAIL: c-c++-common/asan/strncpy-overflow-1.c  -O2 -flto
> -fno-use-linker-plugin -flto-partition=none  output pattern test, is
> =================================================================
> FAIL: c-c++-common/asan/strncpy-overflow-1.c  -O2 -flto
> -fuse-linker-plugin -fno-fat-lto-objects  output pattern test, is
> =================================================================
> FAIL: c-c++-common/asan/use-after-free-1.c  -O0  output pattern test,
> is =================================================================
> FAIL: c-c++-common/asan/use-after-free-1.c  -O1  output pattern test,
> is =================================================================
> FAIL: c-c++-common/asan/use-after-free-1.c  -O2  output pattern test,
> is =================================================================
> FAIL: c-c++-common/asan/use-after-free-1.c  -O3 -fomit-frame-pointer
> output pattern test, is
> =================================================================
> FAIL: c-c++-common/asan/use-after-free-1.c  -O3 -g  output pattern
> test, is =================================================================
> FAIL: c-c++-common/asan/use-after-free-1.c  -Os  output pattern test,
> is =================================================================
> FAIL: c-c++-common/asan/use-after-free-1.c  -O2 -flto
> -fno-use-linker-plugin -flto-partition=none  output pattern test, is
> =================================================================
> FAIL: c-c++-common/asan/use-after-free-1.c  -O2 -flto
> -fuse-linker-plugin -fno-fat-lto-objects  output pattern test, is
> ================
>
>
>
> regards
> Ramana
>
>> ppc may get broken -- I haven't heard from anyone there for a while.
>> Same for sparc.
>> I'll commit later today.
>>
>>
>>> Thanks.
>>>
>>>         Jakub

Re: libsanitizer merge from upstream r208536

[Yury Gribov]

 > On ARM the asan tests have always been a random generator of PASS /
 > FAIL on qemu despite efforts to "nobble" qemu for /proc/self/maps
 > outputs.

This should improve once upstream Asan sets up an ARM build bot. This 
has been discussed recently but noone has yet volunteered to do the 
server installation and setup.

 > After the merge I see these new failures instead

A pity that gcc-testresults does not report mismatched lines.

My guess is this is caused by some unwinding failures when reporting 
heap allocations:

$ grep -R 'allocated by thread' ./gcc/testsuite/c-c++-common/asan -l
./gcc/testsuite/c-c++-common/asan/strncpy-overflow-1.c
./gcc/testsuite/c-c++-common/asan/heap-overflow-1.c
./gcc/testsuite/c-c++-common/asan/use-after-free-1.c

-Y

Re: libsanitizer merge from upstream r208536

[Ramana Radhakrishnan]

On 05/23/14 08:50, Yury Gribov wrote:
>   > On ARM the asan tests have always been a random generator of PASS /
>   > FAIL on qemu despite efforts to "nobble" qemu for /proc/self/maps
>   > outputs.
>
> This should improve once upstream Asan sets up an ARM build bot. This
> has been discussed recently but noone has yet volunteered to do the
> server installation and setup.
>
>   > After the merge I see these new failures instead
>
> A pity that gcc-testresults does not report mismatched lines.
>
> My guess is this is caused by some unwinding failures when reporting
> heap allocations:
>

Ok, so we need to fix that if there is a bug.

> $ grep -R 'allocated by thread' ./gcc/testsuite/c-c++-common/asan -l
> ./gcc/testsuite/c-c++-common/asan/strncpy-overflow-1.c
> ./gcc/testsuite/c-c++-common/asan/heap-overflow-1.c
> ./gcc/testsuite/c-c++-common/asan/use-after-free-1.c
>

This is what I see from the log and all failures look identical.

==14627==ERROR: AddressSanitizer: heap-buffer-overflow on address 
0x41a007fa at pc 0x88c4 bp 0xbebd0084 sp 0xbebd007c
READ of size 1 at 0x41a007fa thread T0
     #0 0x88c3 in main 
/work/gcc/gcc/testsuite/c-c++-common/asan/heap-overflow-1.c:21
     #1 0x40626631 in __libc_start_main 
(/lib/arm-linux-gnueabihf/libc.so.6+0x17631)

0x41a007fa is located 0 bytes to the right of 10-byte region 
[0x41a007f0,0x41a007fa)
allocated by thread T0 here:
     #0 0x400cd587 in __interceptor_malloc 
/work/gcc/libsanitizer/asan/asan_malloc_linux.cc:73

SUMMARY: AddressSanitizer: heap-buffer-overflow 
/work/gcc/gcc/testsuite/c-c++-common/asan/heap-overflow-1.c:21 main
Shadow bytes around the buggy address:
   0x283400a0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
   0x283400b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
   0x283400c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
   0x283400d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
   0x283400e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x283400f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa 00[02]
   0x28340100: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
   0x28340110: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
   0x28340120: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
   0x28340130: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
   0x28340140: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
   Addressable:           00
   Partially addressable: 01 02 03 04 05 06 07
   Heap left redzone:       fa
   Heap right redzone:      fb
   Freed heap region:       fd
   Stack left redzone:      f1
   Stack mid redzone:       f2
   Stack right redzone:     f3
   Stack partial redzone:   f4
   Stack after return:      f5
   Stack use after scope:   f8
   Global redzone:          f9
   Global init order:       f6
   Poisoned by user:        f7
   Container overflow:      fc
   ASan internal:           fe
==14627==ABORTING




> -Y
>

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

On Fri, May 23, 2014 at 11:56 AM, Ramana Radhakrishnan
<ramana.radhakrishnan@arm.com> wrote:
> On 05/23/14 08:50, Yury Gribov wrote:
>>
>>   > On ARM the asan tests have always been a random generator of PASS /
>>   > FAIL on qemu despite efforts to "nobble" qemu for /proc/self/maps
>>   > outputs.
>>
>> This should improve once upstream Asan sets up an ARM build bot. This
>> has been discussed recently but noone has yet volunteered to do the
>> server installation and setup.
>>
>>   > After the merge I see these new failures instead
>>
>> A pity that gcc-testresults does not report mismatched lines.
>>
>> My guess is this is caused by some unwinding failures when reporting
>> heap allocations:
>>
>
> Ok, so we need to fix that if there is a bug.

Yep.

>
>
>> $ grep -R 'allocated by thread' ./gcc/testsuite/c-c++-common/asan -l
>> ./gcc/testsuite/c-c++-common/asan/strncpy-overflow-1.c
>> ./gcc/testsuite/c-c++-common/asan/heap-overflow-1.c
>> ./gcc/testsuite/c-c++-common/asan/use-after-free-1.c
>>
>
> This is what I see from the log and all failures look identical.
>
> ==14627==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x41a007fa
> at pc 0x88c4 bp 0xbebd0084 sp 0xbebd007c
> READ of size 1 at 0x41a007fa thread T0
>     #0 0x88c3 in main
> /work/gcc/gcc/testsuite/c-c++-common/asan/heap-overflow-1.c:21
>     #1 0x40626631 in __libc_start_main
> (/lib/arm-linux-gnueabihf/libc.so.6+0x17631)
>
> 0x41a007fa is located 0 bytes to the right of 10-byte region
> [0x41a007f0,0x41a007fa)
> allocated by thread T0 here:
>     #0 0x400cd587 in __interceptor_malloc
> /work/gcc/libsanitizer/asan/asan_malloc_linux.cc:73

Looks indeed like wrong unwind, similar to what has been recently
discussed here:
http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20140519/218239.html

>
> SUMMARY: AddressSanitizer: heap-buffer-overflow
> /work/gcc/gcc/testsuite/c-c++-common/asan/heap-overflow-1.c:21 main
> Shadow bytes around the buggy address:
>   0x283400a0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
>   0x283400b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
>   0x283400c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
>   0x283400d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
>   0x283400e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
> =>0x283400f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa 00[02]
>   0x28340100: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
>   0x28340110: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
>   0x28340120: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
>   0x28340130: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
>   0x28340140: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
> Shadow byte legend (one shadow byte represents 8 application bytes):
>   Addressable:           00
>   Partially addressable: 01 02 03 04 05 06 07
>   Heap left redzone:       fa
>   Heap right redzone:      fb
>   Freed heap region:       fd
>   Stack left redzone:      f1
>   Stack mid redzone:       f2
>   Stack right redzone:     f3
>   Stack partial redzone:   f4
>   Stack after return:      f5
>   Stack use after scope:   f8
>   Global redzone:          f9
>   Global init order:       f6
>   Poisoned by user:        f7
>   Container overflow:      fc
>   ASan internal:           fe
> ==14627==ABORTING
>
>
>
>
>> -Y
>>
>

Re: libsanitizer merge from upstream r208536

[Marek Polacek]

On Mon, May 12, 2014 at 03:20:37PM +0400, Konstantin Serebryany wrote:
> 5 months' worth of changes may break any platform we are not testing ourselves
> (that includes Ubuntu 12.04, 13.10, 14.04, Mac 10.9, Windows 7, Android ARM),
> please help us test this patch on your favorite platform.

On powerpc64 I hit
/home/polacek/gcc/libsanitizer/asan/asan_linux.cc:209:3: error: #error "Unsupported arch"
 # error "Unsupported arch"

because the merge (aka clang's r196802) removed ppc64 hunk of code:

-# elif defined(__powerpc__) || defined(__powerpc64__)
-  ucontext_t *ucontext = (ucontext_t*)context;
-  *pc = ucontext->uc_mcontext.regs->nip;
-  *sp = ucontext->uc_mcontext.regs->gpr[PT_R1];
-  // The powerpc{,64}-linux ABIs do not specify r31 as the frame
-  // pointer, but GCC always uses r31 when we need a frame pointer.
-  *bp = ucontext->uc_mcontext.regs->gpr[PT_R31];
-# elif defined(__sparc__)

	Marek

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

On Fri, May 23, 2014 at 5:41 PM, Marek Polacek <polacek@redhat.com> wrote:
> On Mon, May 12, 2014 at 03:20:37PM +0400, Konstantin Serebryany wrote:
>> 5 months' worth of changes may break any platform we are not testing ourselves
>> (that includes Ubuntu 12.04, 13.10, 14.04, Mac 10.9, Windows 7, Android ARM),
>> please help us test this patch on your favorite platform.
>
> On powerpc64 I hit
> /home/polacek/gcc/libsanitizer/asan/asan_linux.cc:209:3: error: #error "Unsupported arch"
>  # error "Unsupported arch"
>
> because the merge (aka clang's r196802) removed ppc64 hunk of code:
>
> -# elif defined(__powerpc__) || defined(__powerpc64__)
> -  ucontext_t *ucontext = (ucontext_t*)context;
> -  *pc = ucontext->uc_mcontext.regs->nip;
> -  *sp = ucontext->uc_mcontext.regs->gpr[PT_R1];
> -  // The powerpc{,64}-linux ABIs do not specify r31 as the frame
> -  // pointer, but GCC always uses r31 when we need a frame pointer.
> -  *bp = ucontext->uc_mcontext.regs->gpr[PT_R31];
> -# elif defined(__sparc__)

Someone will have to send this patch via llvm-commits  :(
(I've pinged Peter Bergner once with no luck).

>
>         Marek

Re: libsanitizer merge from upstream r208536

[Christophe Lyon]

Hi,
Since merge from upstream r209283 (210743 in GCC), my build fails on
ARM, because rpc/xdr.h is not found.
Is this expected?

Thanks,

Christophe.


On 23 May 2014 15:45, Konstantin Serebryany
<konstantin.s.serebryany@gmail.com> wrote:
> On Fri, May 23, 2014 at 5:41 PM, Marek Polacek <polacek@redhat.com> wrote:
>> On Mon, May 12, 2014 at 03:20:37PM +0400, Konstantin Serebryany wrote:
>>> 5 months' worth of changes may break any platform we are not testing ourselves
>>> (that includes Ubuntu 12.04, 13.10, 14.04, Mac 10.9, Windows 7, Android ARM),
>>> please help us test this patch on your favorite platform.
>>
>> On powerpc64 I hit
>> /home/polacek/gcc/libsanitizer/asan/asan_linux.cc:209:3: error: #error "Unsupported arch"
>>  # error "Unsupported arch"
>>
>> because the merge (aka clang's r196802) removed ppc64 hunk of code:
>>
>> -# elif defined(__powerpc__) || defined(__powerpc64__)
>> -  ucontext_t *ucontext = (ucontext_t*)context;
>> -  *pc = ucontext->uc_mcontext.regs->nip;
>> -  *sp = ucontext->uc_mcontext.regs->gpr[PT_R1];
>> -  // The powerpc{,64}-linux ABIs do not specify r31 as the frame
>> -  // pointer, but GCC always uses r31 when we need a frame pointer.
>> -  *bp = ucontext->uc_mcontext.regs->gpr[PT_R31];
>> -# elif defined(__sparc__)
>
> Someone will have to send this patch via llvm-commits  :(
> (I've pinged Peter Bergner once with no luck).
>
>>
>>         Marek

Re: libsanitizer merge from upstream r208536

[Kugan]

On 24/05/14 00:06, Christophe Lyon wrote:
> Hi,
> Since merge from upstream r209283 (210743 in GCC), my build fails on
> ARM, because rpc/xdr.h is not found.
> Is this expected?
I also have the same issue. I had to build glibc with
--enable-obsolete-rpc to bootstrap now.

Thanks,
Kugan

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

On Fri, May 23, 2014 at 6:11 PM, Kugan
<kugan.vivekanandarajah@linaro.org> wrote:
> On 24/05/14 00:06, Christophe Lyon wrote:
>> Hi,
>> Since merge from upstream r209283 (210743 in GCC), my build fails on
>> ARM, because rpc/xdr.h is not found.
>> Is this expected?
> I also have the same issue. I had to build glibc with
> --enable-obsolete-rpc to bootstrap now.


+eugenis

This comes from
http://llvm.org/viewvc/llvm-project?view=revision&revision=205627 :

>
> Thanks,
> Kugan

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

On Fri, May 23, 2014 at 6:12 PM, Konstantin Serebryany
<konstantin.s.serebryany@gmail.com> wrote:
> On Fri, May 23, 2014 at 6:11 PM, Kugan
> <kugan.vivekanandarajah@linaro.org> wrote:
>> On 24/05/14 00:06, Christophe Lyon wrote:
>>> Hi,
>>> Since merge from upstream r209283 (210743 in GCC), my build fails on
>>> ARM, because rpc/xdr.h is not found.
>>> Is this expected?
>> I also have the same issue. I had to build glibc with
>> --enable-obsolete-rpc to bootstrap now.
>
>
> +eugenis
>
> This comes from
> http://llvm.org/viewvc/llvm-project?view=revision&revision=205627 :
>
>>
>> Thanks,
>> Kugan

Re: libsanitizer merge from upstream r208536

[Evgeniy Stepanov]

Hi Christophe,

is there anything special about your setup? We've seen it build on
arm/linux and arm/android correctly.


On Fri, May 23, 2014 at 6:06 PM, Christophe Lyon
<christophe.lyon@linaro.org> wrote:
> Hi,
> Since merge from upstream r209283 (210743 in GCC), my build fails on
> ARM, because rpc/xdr.h is not found.
> Is this expected?
>
> Thanks,
>
> Christophe.
>
>
> On 23 May 2014 15:45, Konstantin Serebryany
> <konstantin.s.serebryany@gmail.com> wrote:
>> On Fri, May 23, 2014 at 5:41 PM, Marek Polacek <polacek@redhat.com> wrote:
>>> On Mon, May 12, 2014 at 03:20:37PM +0400, Konstantin Serebryany wrote:
>>>> 5 months' worth of changes may break any platform we are not testing ourselves
>>>> (that includes Ubuntu 12.04, 13.10, 14.04, Mac 10.9, Windows 7, Android ARM),
>>>> please help us test this patch on your favorite platform.
>>>
>>> On powerpc64 I hit
>>> /home/polacek/gcc/libsanitizer/asan/asan_linux.cc:209:3: error: #error "Unsupported arch"
>>>  # error "Unsupported arch"
>>>
>>> because the merge (aka clang's r196802) removed ppc64 hunk of code:
>>>
>>> -# elif defined(__powerpc__) || defined(__powerpc64__)
>>> -  ucontext_t *ucontext = (ucontext_t*)context;
>>> -  *pc = ucontext->uc_mcontext.regs->nip;
>>> -  *sp = ucontext->uc_mcontext.regs->gpr[PT_R1];
>>> -  // The powerpc{,64}-linux ABIs do not specify r31 as the frame
>>> -  // pointer, but GCC always uses r31 when we need a frame pointer.
>>> -  *bp = ucontext->uc_mcontext.regs->gpr[PT_R31];
>>> -# elif defined(__sparc__)
>>
>> Someone will have to send this patch via llvm-commits  :(
>> (I've pinged Peter Bergner once with no luck).
>>
>>>
>>>         Marek

Re: libsanitizer merge from upstream r208536

[Christophe Lyon]

On 23 May 2014 17:01, Evgeniy Stepanov <eugeni.stepanov@gmail.com> wrote:
> Hi Christophe,
>
> is there anything special about your setup? We've seen it build on
> arm/linux and arm/android correctly.
>
>
Hi,

As Kugan said, I needed to add --enable-obsolete-rpc when configuring glibc.

Thanks,

Christophe.

> On Fri, May 23, 2014 at 6:06 PM, Christophe Lyon
> <christophe.lyon@linaro.org> wrote:
>> Hi,
>> Since merge from upstream r209283 (210743 in GCC), my build fails on
>> ARM, because rpc/xdr.h is not found.
>> Is this expected?
>>
>> Thanks,
>>
>> Christophe.
>>
>>
>> On 23 May 2014 15:45, Konstantin Serebryany
>> <konstantin.s.serebryany@gmail.com> wrote:
>>> On Fri, May 23, 2014 at 5:41 PM, Marek Polacek <polacek@redhat.com> wrote:
>>>> On Mon, May 12, 2014 at 03:20:37PM +0400, Konstantin Serebryany wrote:
>>>>> 5 months' worth of changes may break any platform we are not testing ourselves
>>>>> (that includes Ubuntu 12.04, 13.10, 14.04, Mac 10.9, Windows 7, Android ARM),
>>>>> please help us test this patch on your favorite platform.
>>>>
>>>> On powerpc64 I hit
>>>> /home/polacek/gcc/libsanitizer/asan/asan_linux.cc:209:3: error: #error "Unsupported arch"
>>>>  # error "Unsupported arch"
>>>>
>>>> because the merge (aka clang's r196802) removed ppc64 hunk of code:
>>>>
>>>> -# elif defined(__powerpc__) || defined(__powerpc64__)
>>>> -  ucontext_t *ucontext = (ucontext_t*)context;
>>>> -  *pc = ucontext->uc_mcontext.regs->nip;
>>>> -  *sp = ucontext->uc_mcontext.regs->gpr[PT_R1];
>>>> -  // The powerpc{,64}-linux ABIs do not specify r31 as the frame
>>>> -  // pointer, but GCC always uses r31 when we need a frame pointer.
>>>> -  *bp = ucontext->uc_mcontext.regs->gpr[PT_R31];
>>>> -# elif defined(__sparc__)
>>>
>>> Someone will have to send this patch via llvm-commits  :(
>>> (I've pinged Peter Bergner once with no luck).
>>>
>>>>
>>>>         Marek

Re: libsanitizer merge from upstream r208536

[Peter Bergner]

On Fri, 2014-05-23 at 17:45 +0400, Konstantin Serebryany wrote:
> On Fri, May 23, 2014 at 5:41 PM, Marek Polacek <polacek@redhat.com> wrote:
> > On Mon, May 12, 2014 at 03:20:37PM +0400, Konstantin Serebryany wrote:
> >> 5 months' worth of changes may break any platform we are not testing ourselves
> >> (that includes Ubuntu 12.04, 13.10, 14.04, Mac 10.9, Windows 7, Android ARM),
> >> please help us test this patch on your favorite platform.
> >
> > On powerpc64 I hit
> > /home/polacek/gcc/libsanitizer/asan/asan_linux.cc:209:3: error: #error "Unsupported arch"
> >  # error "Unsupported arch"
> >
> > because the merge (aka clang's r196802) removed ppc64 hunk of code:
> >
> > -# elif defined(__powerpc__) || defined(__powerpc64__)
> > -  ucontext_t *ucontext = (ucontext_t*)context;
> > -  *pc = ucontext->uc_mcontext.regs->nip;
> > -  *sp = ucontext->uc_mcontext.regs->gpr[PT_R1];
> > -  // The powerpc{,64}-linux ABIs do not specify r31 as the frame
> > -  // pointer, but GCC always uses r31 when we need a frame pointer.
> > -  *bp = ucontext->uc_mcontext.regs->gpr[PT_R31];
> > -# elif defined(__sparc__)
> 
> Someone will have to send this patch via llvm-commits  :(
> (I've pinged Peter Bergner once with no luck).

Sorry, I'm not purposely ignoring you.  Just had a lot of other higher
priority items on my TODO list when you asked before.  I'll try and
reapply the code that was removed and resubmit to the correct
mailing list.

Sorry for being so tardy and non-responsive.

Peter

Re: libsanitizer merge from upstream r208536

[Peter Bergner]

On Fri, 2014-05-23 at 09:25 -0500, Peter Bergner wrote:
> xagsmtp4.20140523142452.1860@vmsdvm6.vnet.ibm.com
> X-Xagent-Gateway: vmsdvm6.vnet.ibm.com (XAGSMTP4 at VMSDVM6)
> 
> On Fri, 2014-05-23 at 17:45 +0400, Konstantin Serebryany wrote:
> > On Fri, May 23, 2014 at 5:41 PM, Marek Polacek <polacek@redhat.com> wrote:
> > > On Mon, May 12, 2014 at 03:20:37PM +0400, Konstantin Serebryany wrote:
> > >> 5 months' worth of changes may break any platform we are not testing ourselves
> > >> (that includes Ubuntu 12.04, 13.10, 14.04, Mac 10.9, Windows 7, Android ARM),
> > >> please help us test this patch on your favorite platform.
> > >
> > > On powerpc64 I hit
> > > /home/polacek/gcc/libsanitizer/asan/asan_linux.cc:209:3: error: #error "Unsupported arch"
> > >  # error "Unsupported arch"
> > >
> > > because the merge (aka clang's r196802) removed ppc64 hunk of code:
> > >
> > > -# elif defined(__powerpc__) || defined(__powerpc64__)
> > > -  ucontext_t *ucontext = (ucontext_t*)context;
> > > -  *pc = ucontext->uc_mcontext.regs->nip;
> > > -  *sp = ucontext->uc_mcontext.regs->gpr[PT_R1];
> > > -  // The powerpc{,64}-linux ABIs do not specify r31 as the frame
> > > -  // pointer, but GCC always uses r31 when we need a frame pointer.
> > > -  *bp = ucontext->uc_mcontext.regs->gpr[PT_R31];
> > > -# elif defined(__sparc__)
> > 
> > Someone will have to send this patch via llvm-commits  :(
> > (I've pinged Peter Bergner once with no luck).

The following patch gets bootstrap working again, but there seems to
be a large number of testsuite failures I will look into before
submitting the patch to the LLVM mailing list.

Peter


Index: libsanitizer/asan/asan_linux.cc
===================================================================
--- libsanitizer/asan/asan_linux.cc	(revision 210861)
+++ libsanitizer/asan/asan_linux.cc	(working copy)
@@ -186,6 +186,13 @@ void GetPcSpBp(void *context, uptr *pc,
   *bp = ucontext->uc_mcontext.gregs[REG_EBP];
   *sp = ucontext->uc_mcontext.gregs[REG_ESP];
 # endif
+#elif defined(__powerpc__) || defined(__powerpc64__)
+  ucontext_t *ucontext = (ucontext_t*)context;
+  *pc = ucontext->uc_mcontext.regs->nip;
+  *sp = ucontext->uc_mcontext.regs->gpr[PT_R1];
+  // The powerpc{,64}-linux ABIs do not specify r31 as the frame
+  // pointer, but GCC always uses r31 when we need a frame pointer.
+  *bp = ucontext->uc_mcontext.regs->gpr[PT_R31];
 #elif defined(__sparc__)
   ucontext_t *ucontext = (ucontext_t*)context;
   uptr *stk_ptr;
Index: libsanitizer/asan/asan_mapping.h
===================================================================
--- libsanitizer/asan/asan_mapping.h	(revision 210861)
+++ libsanitizer/asan/asan_mapping.h	(working copy)
@@ -85,6 +85,7 @@ static const u64 kDefaultShadowOffset64
 static const u64 kDefaultShort64bitShadowOffset = 0x7FFF8000;  // < 2G.
 static const u64 kAArch64_ShadowOffset64 = 1ULL << 36;
 static const u64 kMIPS32_ShadowOffset32 = 0x0aaa8000;
+static const u64 kPPC64_ShadowOffset64 = 1ULL << 41;
 static const u64 kFreeBSD_ShadowOffset32 = 1ULL << 30;  // 0x40000000
 static const u64 kFreeBSD_ShadowOffset64 = 1ULL << 46;  // 0x400000000000
 
@@ -107,6 +108,8 @@ static const u64 kFreeBSD_ShadowOffset64
 # else
 #  if defined(__aarch64__)
 #    define SHADOW_OFFSET kAArch64_ShadowOffset64
+#  elif defined(__powerpc64__)
+#    define SHADOW_OFFSET kPPC64_ShadowOffset64
 #  elif SANITIZER_FREEBSD
 #    define SHADOW_OFFSET kFreeBSD_ShadowOffset64
 #  elif SANITIZER_MAC
Index: libsanitizer/sanitizer_common/sanitizer_common.h
===================================================================
--- libsanitizer/sanitizer_common/sanitizer_common.h	(revision 210861)
+++ libsanitizer/sanitizer_common/sanitizer_common.h	(working copy)
@@ -26,7 +26,11 @@ struct StackTrace;
 const uptr kWordSize = SANITIZER_WORDSIZE / 8;
 const uptr kWordSizeInBits = 8 * kWordSize;
 
-const uptr kCacheLineSize = 64;
+#if defined(__powerpc__) || defined(__powerpc64__)
+  const uptr kCacheLineSize = 128;
+#else
+  const uptr kCacheLineSize = 64;
+#endif
 
 const uptr kMaxPathLength = 512;
 
Index: libsanitizer/sanitizer_common/sanitizer_stacktrace.cc
===================================================================
--- libsanitizer/sanitizer_common/sanitizer_stacktrace.cc	(revision 210861)
+++ libsanitizer/sanitizer_common/sanitizer_stacktrace.cc	(working copy)
@@ -16,11 +16,13 @@
 namespace __sanitizer {
 
 uptr StackTrace::GetPreviousInstructionPc(uptr pc) {
-#ifdef __arm__
+#if defined(__arm__)
   // Cancel Thumb bit.
   pc = pc & (~1);
-#endif
-#if defined(__sparc__)
+#elif defined(__powerpc__) || defined(__powerpc64__)
+  // PCs are always 4 byte aligned.
+  return pc - 4;
+#elif defined(__sparc__)
   return pc - 8;
 #else
   return pc - 1;

Re: libsanitizer merge from upstream r208536

[Peter Bergner]

On Fri, 2014-05-23 at 11:47 -0500, Peter Bergner wrote:
> The following patch gets bootstrap working again, but there seems to
> be a large number of testsuite failures I will look into before
> submitting the patch to the LLVM mailing list.

FYI, it looks like a huge number of the failures are from the following
in 32-bit mode:

/home/bergner/gcc/build/gcc-fsf-mainline-asan-debug/powerpc64-linux/32/libsanitizer/asan/.libs/libasan.so: undefined reference to `__sync_fetch_and_add_8'^M
collect2: error: ld returned 1 exit status^M

Kosta, we don't have a __sync_fetch_and_add_8 in 32-bit, since
we don't have the necessary hardware instructions to do a 64-bit
fetch and add.  How is this supposed to work?  There are some
libatomic routines we could call (eg, __atomic_fetch_add_8).
Is that an option?  ...or maybe we shouldn't even be attempting
call a 64-bit fetch/add for things larger than a register size?


Peter

Re: libsanitizer merge from upstream r208536

[Jakub Jelinek]

On Fri, May 23, 2014 at 01:54:47PM -0500, Peter Bergner wrote:
> On Fri, 2014-05-23 at 11:47 -0500, Peter Bergner wrote:
> > The following patch gets bootstrap working again, but there seems to
> > be a large number of testsuite failures I will look into before
> > submitting the patch to the LLVM mailing list.
> 
> FYI, it looks like a huge number of the failures are from the following
> in 32-bit mode:
> 
> /home/bergner/gcc/build/gcc-fsf-mainline-asan-debug/powerpc64-linux/32/libsanitizer/asan/.libs/libasan.so: undefined reference to `__sync_fetch_and_add_8'^M
> collect2: error: ld returned 1 exit status^M
> 
> Kosta, we don't have a __sync_fetch_and_add_8 in 32-bit, since
> we don't have the necessary hardware instructions to do a 64-bit
> fetch and add.  How is this supposed to work?  There are some
> libatomic routines we could call (eg, __atomic_fetch_add_8).

But libatomic in the end for that cases uses locking and non-atomic access
anyway.  On i?86 64-bit atomics are available starting with i586,
on sparc starting with v9, but I believe on most architectures
if you have atomics at all you don't have atomics larger than word size.

For GCC, if you want in the preprocessor to test if a CPU has 64-bit
atomics, you'd check #ifdef __GCC_HAVE_SYNC_COMPARE_AND_SWAP_8
Can you use locking instead otherwise?

	Jakub

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

Hi Peter,

Last time I tried, asan did not work on ppc32 for a large number of
different reasons.
In upstream build system ppc32 is simply disabled, so imho it should
be also disabled in the GCC build.
If there is enough interest in ppc32, please work with up on fixing
upstream and enabling the build bots --
then we can talk about supporting ppc32 in GCC (it will actually come
for free then).
(I can't say whether ppc64 works today, but at least it worked for me
at some point).

--kcc

On Fri, May 23, 2014 at 10:54 PM, Peter Bergner <bergner@vnet.ibm.com> wrote:
> On Fri, 2014-05-23 at 11:47 -0500, Peter Bergner wrote:
>> The following patch gets bootstrap working again, but there seems to
>> be a large number of testsuite failures I will look into before
>> submitting the patch to the LLVM mailing list.
>
> FYI, it looks like a huge number of the failures are from the following
> in 32-bit mode:
>
> /home/bergner/gcc/build/gcc-fsf-mainline-asan-debug/powerpc64-linux/32/libsanitizer/asan/.libs/libasan.so: undefined reference to `__sync_fetch_and_add_8'^M
> collect2: error: ld returned 1 exit status^M
>
> Kosta, we don't have a __sync_fetch_and_add_8 in 32-bit, since
> we don't have the necessary hardware instructions to do a 64-bit
> fetch and add.  How is this supposed to work?  There are some
> libatomic routines we could call (eg, __atomic_fetch_add_8).
> Is that an option?  ...or maybe we shouldn't even be attempting
> call a 64-bit fetch/add for things larger than a register size?
>
>
> Peter
>
>

Re: libsanitizer merge from upstream r208536

[Jakub Jelinek]

On Mon, May 26, 2014 at 08:57:11AM +0400, Konstantin Serebryany wrote:
> Last time I tried, asan did not work on ppc32 for a large number of
> different reasons.

???
Comparing my 4.9.0 ppc/ppc64 testresults, for 32-bit I see:

FAIL: c-c++-common/asan/null-deref-1.c  -O0  output pattern test, is ASAN:SIGSEGV
FAIL: c-c++-common/asan/null-deref-1.c  -O1  output pattern test, is ASAN:SIGSEGV
FAIL: c-c++-common/asan/null-deref-1.c  -O2  output pattern test, is ASAN:SIGSEGV
FAIL: c-c++-common/asan/null-deref-1.c  -O3 -fomit-frame-pointer  output pattern test, is ASAN:SIGSEGV
FAIL: c-c++-common/asan/null-deref-1.c  -O3 -g  output pattern test, is ASAN:SIGSEGV
FAIL: c-c++-common/asan/null-deref-1.c  -Os  output pattern test, is ASAN:SIGSEGV
FAIL: c-c++-common/asan/null-deref-1.c  -O2 -flto -fno-use-linker-plugin -flto-partition=none  output pattern test, is ASAN:SIGSEGV
FAIL: c-c++-common/asan/null-deref-1.c  -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects  output pattern test, is ASAN:SIGSEGV
FAIL: c-c++-common/asan/pr59063-2.c  -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects  (test for excess errors)
UNRESOLVED: c-c++-common/asan/pr59063-2.c  -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects  compilation failed to produce executable

and for 64-bit:

FAIL: c-c++-common/asan/null-deref-1.c  -O2  output pattern test, is ASAN:SIGSEGV
FAIL: c-c++-common/asan/null-deref-1.c  -O3 -fomit-frame-pointer  output pattern test, is ASAN:SIGSEGV
FAIL: c-c++-common/asan/null-deref-1.c  -O3 -g  output pattern test, is ASAN:SIGSEGV
FAIL: c-c++-common/asan/null-deref-1.c  -O2 -flto -fno-use-linker-plugin -flto-partition=none  output pattern test, is ASAN:SIGSEGV
FAIL: c-c++-common/asan/null-deref-1.c  -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects  output pattern test, is ASAN:SIGSEGV
FAIL: c-c++-common/asan/pr59063-2.c  -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects  (test for excess errors)
UNRESOLVED: c-c++-common/asan/pr59063-2.c  -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects  compilation failed to produce executable
FAIL: c-c++-common/asan/swapcontext-test-1.c  -O0  execution test
FAIL: c-c++-common/asan/swapcontext-test-1.c  -O1  execution test
FAIL: c-c++-common/asan/swapcontext-test-1.c  -O2  execution test
FAIL: c-c++-common/asan/swapcontext-test-1.c  -O3 -fomit-frame-pointer  execution test
FAIL: c-c++-common/asan/swapcontext-test-1.c  -O3 -fomit-frame-pointer -funroll-loops  execution test
FAIL: c-c++-common/asan/swapcontext-test-1.c  -O3 -fomit-frame-pointer -funroll-all-loops -finline-functions  execution test
FAIL: c-c++-common/asan/swapcontext-test-1.c  -O3 -g  execution test
FAIL: c-c++-common/asan/swapcontext-test-1.c  -Os  execution test
FAIL: c-c++-common/asan/swapcontext-test-1.c  -O2 -flto -fno-use-linker-plugin -flto-partition=none  execution test
FAIL: c-c++-common/asan/swapcontext-test-1.c  -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects  execution test 

Doesn't look like the ppc32 port would be in any worse shape than the 64-bit
one.
Peter has brought a real problem, in particular the allocator now newly relying on
2 x word size atomics which is definitely non-portable, I don't see why the answer
to that should be disable your port or build a buildbot.

	Jakub

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

On Mon, May 26, 2014 at 9:57 AM, Jakub Jelinek <jakub@redhat.com> wrote:
> On Mon, May 26, 2014 at 08:57:11AM +0400, Konstantin Serebryany wrote:
>> Last time I tried, asan did not work on ppc32 for a large number of
>> different reasons.
>
> ???
> Comparing my 4.9.0 ppc/ppc64 testresults, for 32-bit I see:
>
> FAIL: c-c++-common/asan/null-deref-1.c  -O0  output pattern test, is ASAN:SIGSEGV
> FAIL: c-c++-common/asan/null-deref-1.c  -O1  output pattern test, is ASAN:SIGSEGV
> FAIL: c-c++-common/asan/null-deref-1.c  -O2  output pattern test, is ASAN:SIGSEGV
> FAIL: c-c++-common/asan/null-deref-1.c  -O3 -fomit-frame-pointer  output pattern test, is ASAN:SIGSEGV
> FAIL: c-c++-common/asan/null-deref-1.c  -O3 -g  output pattern test, is ASAN:SIGSEGV
> FAIL: c-c++-common/asan/null-deref-1.c  -Os  output pattern test, is ASAN:SIGSEGV
> FAIL: c-c++-common/asan/null-deref-1.c  -O2 -flto -fno-use-linker-plugin -flto-partition=none  output pattern test, is ASAN:SIGSEGV
> FAIL: c-c++-common/asan/null-deref-1.c  -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects  output pattern test, is ASAN:SIGSEGV
> FAIL: c-c++-common/asan/pr59063-2.c  -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects  (test for excess errors)
> UNRESOLVED: c-c++-common/asan/pr59063-2.c  -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects  compilation failed to produce executable
>
> and for 64-bit:
>
> FAIL: c-c++-common/asan/null-deref-1.c  -O2  output pattern test, is ASAN:SIGSEGV
> FAIL: c-c++-common/asan/null-deref-1.c  -O3 -fomit-frame-pointer  output pattern test, is ASAN:SIGSEGV
> FAIL: c-c++-common/asan/null-deref-1.c  -O3 -g  output pattern test, is ASAN:SIGSEGV
> FAIL: c-c++-common/asan/null-deref-1.c  -O2 -flto -fno-use-linker-plugin -flto-partition=none  output pattern test, is ASAN:SIGSEGV
> FAIL: c-c++-common/asan/null-deref-1.c  -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects  output pattern test, is ASAN:SIGSEGV
> FAIL: c-c++-common/asan/pr59063-2.c  -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects  (test for excess errors)
> UNRESOLVED: c-c++-common/asan/pr59063-2.c  -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects  compilation failed to produce executable
> FAIL: c-c++-common/asan/swapcontext-test-1.c  -O0  execution test
> FAIL: c-c++-common/asan/swapcontext-test-1.c  -O1  execution test
> FAIL: c-c++-common/asan/swapcontext-test-1.c  -O2  execution test
> FAIL: c-c++-common/asan/swapcontext-test-1.c  -O3 -fomit-frame-pointer  execution test
> FAIL: c-c++-common/asan/swapcontext-test-1.c  -O3 -fomit-frame-pointer -funroll-loops  execution test
> FAIL: c-c++-common/asan/swapcontext-test-1.c  -O3 -fomit-frame-pointer -funroll-all-loops -finline-functions  execution test
> FAIL: c-c++-common/asan/swapcontext-test-1.c  -O3 -g  execution test
> FAIL: c-c++-common/asan/swapcontext-test-1.c  -Os  execution test
> FAIL: c-c++-common/asan/swapcontext-test-1.c  -O2 -flto -fno-use-linker-plugin -flto-partition=none  execution test
> FAIL: c-c++-common/asan/swapcontext-test-1.c  -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects  execution test
>
> Doesn't look like the ppc32 port would be in any worse shape than the 64-bit
> one.
> Peter has brought a real problem, in particular the allocator now newly relying on
> 2 x word size atomics which is definitely non-portable, I don't see why the answer
> to that should be disable your port or build a buildbot.

Because this is my default reply to any such case. :)





>
>         Jakub

Re: libsanitizer merge from upstream r208536

[Peter Bergner]

> On Mon, May 26, 2014 at 9:57 AM, Jakub Jelinek <jakub@redhat.com> wrote:
> > Doesn't look like the ppc32 port would be in any worse shape than the 64-bit
> > one.
> > Peter has brought a real problem, in particular the allocator now newly relying on
> > 2 x word size atomics which is definitely non-portable, I don't see why the answer
> > to that should be disable your port or build a buildbot.

Right, the ppc32 results definitely show it's on par with the ppc64 results.


On Mon, 2014-05-26 at 10:36 +0400, Konstantin Serebryany wrote:
> Because this is my default reply to any such case. :)

I hope that is a humorous reply and not a serious one.
In one of my other posts, I asked should 32-bit ports even attempt
to use the 2 * word_size atomics.  What is the code doing such that
it wants to use a 2 * word_size atomic?  Is it as simple as commenting
that code out for 32-bit builds of the library or do we really have
to support that?

Peter

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

On Tue, May 27, 2014 at 6:25 AM, Peter Bergner <bergner@vnet.ibm.com> wrote:
>> On Mon, May 26, 2014 at 9:57 AM, Jakub Jelinek <jakub@redhat.com> wrote:
>> > Doesn't look like the ppc32 port would be in any worse shape than the 64-bit
>> > one.
>> > Peter has brought a real problem, in particular the allocator now newly relying on
>> > 2 x word size atomics which is definitely non-portable, I don't see why the answer
>> > to that should be disable your port or build a buildbot.
>
> Right, the ppc32 results definitely show it's on par with the ppc64 results.
>
>
> On Mon, 2014-05-26 at 10:36 +0400, Konstantin Serebryany wrote:
>> Because this is my default reply to any such case. :)
>
> I hope that is a humorous reply and not a serious one.

Not really humorous. Our position is and always was:
"If you are adding support for a new architecture/platform, we
encourage you to set up a public build bot, otherwise we can not
guarantee that we will keep your code in working conditions."
https://code.google.com/p/address-sanitizer/wiki/HowToContribute


> In one of my other posts, I asked should 32-bit ports even attempt
> to use the 2 * word_size atomics.  What is the code doing such that
> it wants to use a 2 * word_size atomic?  Is it as simple as commenting
> that code out for 32-bit builds of the library or do we really have
> to support that?

Frankly, I don't remember where asan can use 2 * word_size atomic.
This might be some mistake in new code indeed.
Do you see what function calls these atomics?

--kcc

>
> Peter
>
>

Re: libsanitizer merge from upstream r208536

[Mike Stump]

On May 26, 2014, at 10:13 PM, Konstantin Serebryany <konstantin.s.serebryany@gmail.com> wrote:
>> On Mon, 2014-05-26 at 10:36 +0400, Konstantin Serebryany wrote:
>>> Because this is my default reply to any such case. :)
>> 
>> I hope that is a humorous reply and not a serious one.
> 
> Not really humorous. Our position is and always was

We don’t expect a guarantee that you keep code working.  Only that when _you_ break things that you try and help out as you can, and if you cannot, merely to ask others for help.  Doesn’t seem to me to be an unreasonable position and seems to have worked fairly well for the past 27 years.

So, the right way to treat a regression that you hear about from the gcc side, is exactly the same way you handle a green to red transition on a build bot.

So, let me ask, when you break a build bot, is your first response to want to disable the port the regression is found with?  If not, then why treat the regression found on the gcc side any different?

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

On Tue, May 27, 2014 at 9:53 PM, Mike Stump <mikestump@comcast.net> wrote:
>
> On May 26, 2014, at 10:13 PM, Konstantin Serebryany <konstantin.s.serebryany@gmail.com> wrote:
>>> On Mon, 2014-05-26 at 10:36 +0400, Konstantin Serebryany wrote:
>>>> Because this is my default reply to any such case. :)
>>>
>>> I hope that is a humorous reply and not a serious one.
>>
>> Not really humorous. Our position is and always was
>
> We don’t expect a guarantee that you keep code working.  Only that when _you_ break things that you try and help out as you can, and if you cannot, merely to ask others for help.  Doesn’t seem to me to be an unreasonable position and seems to have worked fairly well for the past 27 years.
>
> So, the right way to treat a regression that you hear about from the gcc side, is exactly the same way you handle a green to red transition on a build bot.
>
> So, let me ask, when you break a build bot, is your first response to want to disable the port the regression is found with?  If not, then why treat the regression found on the gcc side any different?

If a bot breaks, we know about it within tens of minutes.
The context is still fresh, fixing such failure is typically a matter
of minutes and costs us nothing.
Of course, the bot should be accompanied with a person who supports it
and can help us if we don't understand the failure.
I am extremely happy to see the FreeBSD build bot, which became green recently.
http://lab.llvm.org:8011/builders/sanitizer_x86_64-freeBSD9.2

If a problem is discovered 5 months after it has been introduced, the
cost for fixing it is much higher
as it involves more testing, more code reviews, etc.
Every time a platform owner sends us a patch here, we need to remind
that the patch has to be sent upstream,
and every second time we need to explain why.
Our code is unusual in many ways and we have to edit half of the
patches we receive.
We have our preferred way of sending patches which some of the
contributors tend to ignore
(I have strong opinions about the rules for sending patches in GCC,
but I keep these opinions to myself and try to follow the rules;
some of the GCC community members neglect to follow *our* rules).

Moreover, if we know that something breaks on a given platform, we may
change our design decision quickly.
If we learn about breakage months later when design is finalized,
changing the design is much more work.
I remember Jakub asked to redesign a part of code to make it more portable.
The idea was reasonable, but I would not even attempt it w/o having
bots for all supported platforms.

Finally, there is a psychological aspect -- I get sad when I learn
that I broke someone's code, even
when that someone is very polite (not all of someones are polite
though). I want to know about a breakage first -- feels better.

--kcc

Re: libsanitizer merge from upstream r208536

[Mike Stump]

On May 27, 2014, at 11:16 AM, Konstantin Serebryany <konstantin.s.serebryany@gmail.com> wrote:
> On Tue, May 27, 2014 at 9:53 PM, Mike Stump <mikestump@comcast.net> wrote:
>> 
>> On May 26, 2014, at 10:13 PM, Konstantin Serebryany <konstantin.s.serebryany@gmail.com> wrote:
>>>> On Mon, 2014-05-26 at 10:36 +0400, Konstantin Serebryany wrote:
>>>>> Because this is my default reply to any such case. :)
>>>> 
>>>> I hope that is a humorous reply and not a serious one.
>>> 
>>> Not really humorous. Our position is and always was
>> 
>> We don’t expect a guarantee that you keep code working.  Only that when _you_ break things that you try and help out as you can, and if you cannot, merely to ask others for help.  Doesn’t seem to me to be an unreasonable position and seems to have worked fairly well for the past 27 years.
>> 
>> So, the right way to treat a regression that you hear about from the gcc side, is exactly the same way you handle a green to red transition on a build bot.
>> 
>> So, let me ask, when you break a build bot, is your first response to want to disable the port the regression is found with?  If not, then why treat the regression found on the gcc side any different?
> 
> If a bot breaks, we know about it within tens of minutes.

I appreciate that.  I also appreciate that if all bugs in a checkin can be pointed out in 2 minutes, that is very nice indeed.  All I can say is, we welcome your contribution to make that happen…  but, despite that, it is generally impolite to put in bugs and say let’s remove the port that exposes the bug.  If you can do no better, then at least ignore it, this is more polite.  There will always be bugs, and you will always put some in.  Life goes on.

> If we learn about breakage months later when design is finalized,

Software is never finalized, neither are designs.

> changing the design is much more work.

Yes, software is hard.  :-)  Just wait til you try and solve a problem that autoconf was design to solve, then you will hard truly turned to the dark side.

> Finally, there is a psychological aspect -- I get sad when I learn
> that I broke someone's code

Don’t be sad.  Treat them as you would a slow running build bot.  Try and make them green.  :-)

Re: libsanitizer merge from upstream r208536

[Jakub Jelinek]

On Mon, May 26, 2014 at 09:25:37PM -0500, Peter Bergner wrote:
> In one of my other posts, I asked should 32-bit ports even attempt
> to use the 2 * word_size atomics.  What is the code doing such that
> it wants to use a 2 * word_size atomic?  Is it as simple as commenting
> that code out for 32-bit builds of the library or do we really have
> to support that?

BTW, just checked and I don't see any 2 * word_size atomics in i686
libasan.so.1 - no cmpxchg8b insns anywhere.  Thus I think it would
be really nice if you could point out where exactly is the 64-bit
atomic really needed (or gcc options + preprocessed source so that
it can be investigated in a cross-compiler).

	Jakub

Re: libsanitizer merge from upstream r208536

[Peter Bergner]

On Tue, 2014-05-27 at 20:07 +0200, Jakub Jelinek wrote:
> On Mon, May 26, 2014 at 09:25:37PM -0500, Peter Bergner wrote:
> > In one of my other posts, I asked should 32-bit ports even attempt
> > to use the 2 * word_size atomics.  What is the code doing such that
> > it wants to use a 2 * word_size atomic?  Is it as simple as commenting
> > that code out for 32-bit builds of the library or do we really have
> > to support that?
> 
> BTW, just checked and I don't see any 2 * word_size atomics in i686
> libasan.so.1 - no cmpxchg8b insns anywhere.  Thus I think it would
> be really nice if you could point out where exactly is the 64-bit
> atomic really needed (or gcc options + preprocessed source so that
> it can be investigated in a cross-compiler).

It's being called form basically two files:

[bergner@makalu-lp1 gcc-fsf-mainline-asan-debug]$ find . -name '*.o' | xargs nm -AC | grep sync_fetch_and_add_8
./powerpc64-linux/32/libsanitizer/sanitizer_common/.libs/sanitizer_allocator.o:         U __sync_fetch_and_add_8
./powerpc64-linux/32/libsanitizer/sanitizer_common/sanitizer_allocator.o:         U __sync_fetch_and_add_8
./powerpc64-linux/32/libsanitizer/asan/.libs/asan_allocator2.o:         U __sync_fetch_and_add_8
./powerpc64-linux/32/libsanitizer/asan/asan_allocator2.o:         U __sync_fetch_and_add_8


It seems to be an expansion of the atomic_load template in both object files:

00000000 <__sanitizer::atomic_uint64_t::Type __sanitizer::atomic_load<__sanitizer::atomic_uint64_t>(__sanitize
r::atomic_uint64_t const volatile*, __sanitizer::memory_order)>:
   0:   94 21 ff d0     stwu    r1,-48(r1)
   4:   7c 08 02 a6     mflr    r0
   8:   90 01 00 34     stw     r0,52(r1)
   c:   93 c1 00 28     stw     r30,40(r1)
  10:   42 9f 00 05     bcl     20,4*cr7+so,14 <__sanitizer::atomic_uint64_t::Type __sanitizer::atomic_load<__
sanitizer::atomic_uint64_t>(__sanitizer::atomic_uint64_t const volatile*, __sanitizer::memory_order)+0x14>
  14:   7f c8 02 a6     mflr    r30
  18:   3f de 00 00     addis   r30,r30,0
                        1a: R_PPC_REL16_HA      .got2+0x8006
  1c:   3b de 00 00     addi    r30,r30,0
                        1e: R_PPC_REL16_LO      .got2+0x800a
  20:   90 61 00 18     stw     r3,24(r1)
  24:   90 81 00 1c     stw     r4,28(r1)
  28:   81 21 00 18     lwz     r9,24(r1)
  2c:   38 a0 00 00     li      r5,0
  30:   38 c0 00 00     li      r6,0
  34:   7d 23 4b 78     mr      r3,r9
  38:   48 00 00 01     bl      38 <__sanitizer::atomic_uint64_t::Type __sanitizer::atomic_load<__sanitizer::a
tomic_uint64_t>(__sanitizer::atomic_uint64_t const volatile*, __sanitizer::memory_order)+0x38>
                        38: R_PPC_PLTREL24      __sync_fetch_and_add_8+0x8000
  3c:   90 61 00 20     stw     r3,32(r1)
  40:   90 81 00 24     stw     r4,36(r1)
  44:   c8 01 00 20     lfd     f0,32(r1)
  48:   d8 01 00 08     stfd    f0,8(r1)
  4c:   81 21 00 08     lwz     r9,8(r1)
  50:   81 41 00 0c     lwz     r10,12(r1)
  54:   7d 23 4b 78     mr      r3,r9
  58:   7d 44 53 78     mr      r4,r10
  5c:   80 01 00 34     lwz     r0,52(r1)
  60:   7c 08 03 a6     mtlr    r0
  64:   83 c1 00 28     lwz     r30,40(r1)
  68:   38 21 00 30     addi    r1,r1,48
  6c:   4e 80 00 20     blr


This template comes from the ./sanitizer_common/sanitizer_atomic_clang_other.h header file:

template<typename T>
INLINE typename T::Type atomic_load(
    const volatile T *a, memory_order mo) {
  DCHECK(mo & (memory_order_relaxed | memory_order_consume
      | memory_order_acquire | memory_order_seq_cst));
  DCHECK(!((uptr)a % sizeof(*a)));
  typename T::Type v;

  if (sizeof(*a) < 8 || sizeof(void*) == 8) {
    // Assume that aligned loads are atomic.
    if (mo == memory_order_relaxed) {
      v = a->val_dont_use;
    } else if (mo == memory_order_consume) {
      // Assume that processor respects data dependencies
      // (and that compiler won't break them).
      __asm__ __volatile__("" ::: "memory");
      v = a->val_dont_use;
      __asm__ __volatile__("" ::: "memory");
    } else if (mo == memory_order_acquire) {
      __asm__ __volatile__("" ::: "memory");
      v = a->val_dont_use;
      __sync_synchronize();
    } else {  // seq_cst
      // E.g. on POWER we need a hw fence even before the store.
      __sync_synchronize();
      v = a->val_dont_use;
      __sync_synchronize();
    }
  } else {
    // 64-bit load on 32-bit platform.
    // Gross, but simple and reliable.
    // Assume that it is not in read-only memory.
    v = __sync_fetch_and_add(
        const_cast<typename T::Type volatile *>(&a->val_dont_use), 0);
  }
  return v;
}

It seems x86 has it's own version in sanitizer_atomic_clang_x86.h which
maybe explains why i686 doesn't see this call?  It does:

  ...
  } else {
    // 64-bit load on 32-bit platform.
    __asm__ __volatile__(
        "movq %1, %%mm0;"  // Use mmx reg for 64-bit atomic moves
        "movq %%mm0, %0;"  // (ptr could be read-only)
        "emms;"            // Empty mmx state/Reset FP regs
        : "=m" (v)
        : "m" (a->val_dont_use)
        : // mark the FP stack and mmx registers as clobbered
          "st", "st(1)", "st(2)", "st(3)", "st(4)", "st(5)", "st(6)", "st(7)",
#ifdef __MMX__
          "mm0", "mm1", "mm2", "mm3", "mm4", "mm5", "mm6", "mm7",
#endif  // #ifdef __MMX__
          "memory");


Peter

Re: libsanitizer merge from upstream r208536

[Jakub Jelinek]

On Tue, May 27, 2014 at 04:02:08PM -0500, Peter Bergner wrote:
> It's being called form basically two files:
> 
> [bergner@makalu-lp1 gcc-fsf-mainline-asan-debug]$ find . -name '*.o' | xargs nm -AC | grep sync_fetch_and_add_8
> ./powerpc64-linux/32/libsanitizer/sanitizer_common/.libs/sanitizer_allocator.o:         U __sync_fetch_and_add_8
> ./powerpc64-linux/32/libsanitizer/sanitizer_common/sanitizer_allocator.o:         U __sync_fetch_and_add_8
> ./powerpc64-linux/32/libsanitizer/asan/.libs/asan_allocator2.o:         U __sync_fetch_and_add_8
> ./powerpc64-linux/32/libsanitizer/asan/asan_allocator2.o:         U __sync_fetch_and_add_8

Does ppc32 have any atomic 64-bit loads/stores (in the sense that the aligned
64 bits are written as one memory transaction, not each 32-bit word
separately)?
In any case, atomic_uint64_t there seems to be used only for some statistic
counter and not really atomic anyway, as additions aren't performed using
atomic instructions, but just atomic load, followed by normal arithmetics,
followed by atomic store.
Can't 32-bit counters be used instead on 32-bit arches?

I see there is another spot with atomic_uint64_t in sanitizer_lfstack.h,
but for some reason it isn't used now at all (there it would want to use
64-bit compare and exchange).

Also note that the use of MMX (ugh) means that it is expensive (emms) and
will not work on pre-MMX chips.

	Jakub

Re: libsanitizer merge from upstream r208536

[Jakub Jelinek]

On Tue, May 27, 2014 at 11:50:33PM +0200, Jakub Jelinek wrote:
> On Tue, May 27, 2014 at 04:02:08PM -0500, Peter Bergner wrote:
> > It's being called form basically two files:
> > 
> > [bergner@makalu-lp1 gcc-fsf-mainline-asan-debug]$ find . -name '*.o' | xargs nm -AC | grep sync_fetch_and_add_8
> > ./powerpc64-linux/32/libsanitizer/sanitizer_common/.libs/sanitizer_allocator.o:         U __sync_fetch_and_add_8
> > ./powerpc64-linux/32/libsanitizer/sanitizer_common/sanitizer_allocator.o:         U __sync_fetch_and_add_8
> > ./powerpc64-linux/32/libsanitizer/asan/.libs/asan_allocator2.o:         U __sync_fetch_and_add_8
> > ./powerpc64-linux/32/libsanitizer/asan/asan_allocator2.o:         U __sync_fetch_and_add_8
> 
> Does ppc32 have any atomic 64-bit loads/stores (in the sense that the aligned
> 64 bits are written as one memory transaction, not each 32-bit word
> separately)?
> In any case, atomic_uint64_t there seems to be used only for some statistic
> counter and not really atomic anyway, as additions aren't performed using
> atomic instructions, but just atomic load, followed by normal arithmetics,
> followed by atomic store.
> Can't 32-bit counters be used instead on 32-bit arches?
> 
> I see there is another spot with atomic_uint64_t in sanitizer_lfstack.h,
> but for some reason it isn't used now at all (there it would want to use
> 64-bit compare and exchange).

On ARM, while it supposedly links, because __sync_compare_and_exchange_8
is defined in libgcc.a, it will only work with post-2011 kernels and is
going to be very slow (because you do a separate compare and exchange
to load and separate compare and exchange to store, i.e. twice as much
effort to actually not achieve atomicity, sometimes even 2 syscalls).

	Jakub

Re: libsanitizer merge from upstream r208536

[Ramana Radhakrishnan]

On Tue, May 27, 2014 at 11:00 PM, Jakub Jelinek <jakub@redhat.com> wrote:
> On Tue, May 27, 2014 at 11:50:33PM +0200, Jakub Jelinek wrote:
>> On Tue, May 27, 2014 at 04:02:08PM -0500, Peter Bergner wrote:
>> > It's being called form basically two files:
>> >
>> > [bergner@makalu-lp1 gcc-fsf-mainline-asan-debug]$ find . -name '*.o' | xargs nm -AC | grep sync_fetch_and_add_8
>> > ./powerpc64-linux/32/libsanitizer/sanitizer_common/.libs/sanitizer_allocator.o:         U __sync_fetch_and_add_8
>> > ./powerpc64-linux/32/libsanitizer/sanitizer_common/sanitizer_allocator.o:         U __sync_fetch_and_add_8
>> > ./powerpc64-linux/32/libsanitizer/asan/.libs/asan_allocator2.o:         U __sync_fetch_and_add_8
>> > ./powerpc64-linux/32/libsanitizer/asan/asan_allocator2.o:         U __sync_fetch_and_add_8
>>
>> Does ppc32 have any atomic 64-bit loads/stores (in the sense that the aligned
>> 64 bits are written as one memory transaction, not each 32-bit word
>> separately)?
>> In any case, atomic_uint64_t there seems to be used only for some statistic
>> counter and not really atomic anyway, as additions aren't performed using
>> atomic instructions, but just atomic load, followed by normal arithmetics,
>> followed by atomic store.
>> Can't 32-bit counters be used instead on 32-bit arches?
>>
>> I see there is another spot with atomic_uint64_t in sanitizer_lfstack.h,
>> but for some reason it isn't used now at all (there it would want to use
>> 64-bit compare and exchange).
>
> On ARM, while it supposedly links, because __sync_compare_and_exchange_8
> is defined in libgcc.a, it will only work with post-2011 kernels and is
> going to be very slow (because you do a separate compare and exchange

FTR, this call down to the library function should only be for legacy
architectures.

On ARM we have a 64 bit atomic compare and exchange which can be done
with the ldrexd / strexd instructions at the right architecture level
(v6k and above IIRC).

Ramana

Re: libsanitizer merge from upstream r208536

[Konstantin Serebryany]

Dmitry,
You've introduced   atomic_uint64_t stats_[AllocatorStatCount]; in
http://llvm.org/viewvc/llvm-project?view=revision&revision=173332
Do you mind to change it to atomic_uintptr_t?
There is of course a chance of overflow on 32-bit arch (the number of
mallocs in a process may easily go over 2^32 in a long run),
but this is just stats, I think we can tolerate it.

--kcc

On Wed, May 28, 2014 at 2:41 AM, Ramana Radhakrishnan
<ramana.gcc@googlemail.com> wrote:
> On Tue, May 27, 2014 at 11:00 PM, Jakub Jelinek <jakub@redhat.com> wrote:
>> On Tue, May 27, 2014 at 11:50:33PM +0200, Jakub Jelinek wrote:
>>> On Tue, May 27, 2014 at 04:02:08PM -0500, Peter Bergner wrote:
>>> > It's being called form basically two files:
>>> >
>>> > [bergner@makalu-lp1 gcc-fsf-mainline-asan-debug]$ find . -name '*.o' | xargs nm -AC | grep sync_fetch_and_add_8
>>> > ./powerpc64-linux/32/libsanitizer/sanitizer_common/.libs/sanitizer_allocator.o:         U __sync_fetch_and_add_8
>>> > ./powerpc64-linux/32/libsanitizer/sanitizer_common/sanitizer_allocator.o:         U __sync_fetch_and_add_8
>>> > ./powerpc64-linux/32/libsanitizer/asan/.libs/asan_allocator2.o:         U __sync_fetch_and_add_8
>>> > ./powerpc64-linux/32/libsanitizer/asan/asan_allocator2.o:         U __sync_fetch_and_add_8
>>>
>>> Does ppc32 have any atomic 64-bit loads/stores (in the sense that the aligned
>>> 64 bits are written as one memory transaction, not each 32-bit word
>>> separately)?
>>> In any case, atomic_uint64_t there seems to be used only for some statistic
>>> counter and not really atomic anyway, as additions aren't performed using
>>> atomic instructions, but just atomic load, followed by normal arithmetics,
>>> followed by atomic store.
>>> Can't 32-bit counters be used instead on 32-bit arches?
>>>
>>> I see there is another spot with atomic_uint64_t in sanitizer_lfstack.h,
>>> but for some reason it isn't used now at all (there it would want to use
>>> 64-bit compare and exchange).
>>
>> On ARM, while it supposedly links, because __sync_compare_and_exchange_8
>> is defined in libgcc.a, it will only work with post-2011 kernels and is
>> going to be very slow (because you do a separate compare and exchange
>
> FTR, this call down to the library function should only be for legacy
> architectures.
>
> On ARM we have a 64 bit atomic compare and exchange which can be done
> with the ldrexd / strexd instructions at the right architecture level
> (v6k and above IIRC).
>
> Ramana

Re: libsanitizer merge from upstream r208536

[Dmitry Vyukov]

On Wed, May 28, 2014 at 8:36 AM, Konstantin Serebryany
<konstantin.s.serebryany@gmail.com> wrote:
> Dmitry,
> You've introduced   atomic_uint64_t stats_[AllocatorStatCount]; in
> http://llvm.org/viewvc/llvm-project?view=revision&revision=173332
> Do you mind to change it to atomic_uintptr_t?
> There is of course a chance of overflow on 32-bit arch (the number of
> mallocs in a process may easily go over 2^32 in a long run),
> but this is just stats, I think we can tolerate it.

I removed 64-bit atomics from stats in llvm r209744.




> On Wed, May 28, 2014 at 2:41 AM, Ramana Radhakrishnan
> <ramana.gcc@googlemail.com> wrote:
>> On Tue, May 27, 2014 at 11:00 PM, Jakub Jelinek <jakub@redhat.com> wrote:
>>> On Tue, May 27, 2014 at 11:50:33PM +0200, Jakub Jelinek wrote:
>>>> On Tue, May 27, 2014 at 04:02:08PM -0500, Peter Bergner wrote:
>>>> > It's being called form basically two files:
>>>> >
>>>> > [bergner@makalu-lp1 gcc-fsf-mainline-asan-debug]$ find . -name '*.o' | xargs nm -AC | grep sync_fetch_and_add_8
>>>> > ./powerpc64-linux/32/libsanitizer/sanitizer_common/.libs/sanitizer_allocator.o:         U __sync_fetch_and_add_8
>>>> > ./powerpc64-linux/32/libsanitizer/sanitizer_common/sanitizer_allocator.o:         U __sync_fetch_and_add_8
>>>> > ./powerpc64-linux/32/libsanitizer/asan/.libs/asan_allocator2.o:         U __sync_fetch_and_add_8
>>>> > ./powerpc64-linux/32/libsanitizer/asan/asan_allocator2.o:         U __sync_fetch_and_add_8
>>>>
>>>> Does ppc32 have any atomic 64-bit loads/stores (in the sense that the aligned
>>>> 64 bits are written as one memory transaction, not each 32-bit word
>>>> separately)?
>>>> In any case, atomic_uint64_t there seems to be used only for some statistic
>>>> counter and not really atomic anyway, as additions aren't performed using
>>>> atomic instructions, but just atomic load, followed by normal arithmetics,
>>>> followed by atomic store.
>>>> Can't 32-bit counters be used instead on 32-bit arches?
>>>>
>>>> I see there is another spot with atomic_uint64_t in sanitizer_lfstack.h,
>>>> but for some reason it isn't used now at all (there it would want to use
>>>> 64-bit compare and exchange).
>>>
>>> On ARM, while it supposedly links, because __sync_compare_and_exchange_8
>>> is defined in libgcc.a, it will only work with post-2011 kernels and is
>>> going to be very slow (because you do a separate compare and exchange
>>
>> FTR, this call down to the library function should only be for legacy
>> architectures.
>>
>> On ARM we have a 64 bit atomic compare and exchange which can be done
>> with the ldrexd / strexd instructions at the right architecture level
>> (v6k and above IIRC).
>>
>> Ramana

Re: libsanitizer merge from upstream r208536

[Peter Bergner]

On Tue, 2014-05-27 at 23:50 +0200, Jakub Jelinek wrote:
> On Tue, May 27, 2014 at 04:02:08PM -0500, Peter Bergner wrote:
> > It's being called form basically two files:
> > 
> > [bergner@makalu-lp1 gcc-fsf-mainline-asan-debug]$ find . -name '*.o' | xargs nm -AC | grep sync_fetch_and_add_8
> > ./powerpc64-linux/32/libsanitizer/sanitizer_common/.libs/sanitizer_allocator.o:         U __sync_fetch_and_add_8
> > ./powerpc64-linux/32/libsanitizer/sanitizer_common/sanitizer_allocator.o:         U __sync_fetch_and_add_8
> > ./powerpc64-linux/32/libsanitizer/asan/.libs/asan_allocator2.o:         U __sync_fetch_and_add_8
> > ./powerpc64-linux/32/libsanitizer/asan/asan_allocator2.o:         U __sync_fetch_and_add_8
> 
> Does ppc32 have any atomic 64-bit loads/stores (in the sense that the aligned
> 64 bits are written as one memory transaction, not each 32-bit word
> separately)?

The only option I can think of would be using the floating point
load/store instructions lfd/stfd.  Of course if we're going to
operate on them, then we'd need to copy them back to memory and
then into the integer registers again....before copying them
back to the FP registers (thru memory again) so we can store
them with the stfd.


> In any case, atomic_uint64_t there seems to be used only for some statistic
> counter and not really atomic anyway, as additions aren't performed using
> atomic instructions, but just atomic load, followed by normal arithmetics,
> followed by atomic store.
> Can't 32-bit counters be used instead on 32-bit arches?

Using 32-bit counters would be easiest if we can, but Kostya will have
to answer that.

Peter

Re: libsanitizer merge from upstream r208536

[Jakub Jelinek]

On Tue, May 27, 2014 at 05:04:52PM -0500, Peter Bergner wrote:
> On Tue, 2014-05-27 at 23:50 +0200, Jakub Jelinek wrote:
> > On Tue, May 27, 2014 at 04:02:08PM -0500, Peter Bergner wrote:
> > > It's being called form basically two files:
> > > 
> > > [bergner@makalu-lp1 gcc-fsf-mainline-asan-debug]$ find . -name '*.o' | xargs nm -AC | grep sync_fetch_and_add_8
> > > ./powerpc64-linux/32/libsanitizer/sanitizer_common/.libs/sanitizer_allocator.o:         U __sync_fetch_and_add_8
> > > ./powerpc64-linux/32/libsanitizer/sanitizer_common/sanitizer_allocator.o:         U __sync_fetch_and_add_8
> > > ./powerpc64-linux/32/libsanitizer/asan/.libs/asan_allocator2.o:         U __sync_fetch_and_add_8
> > > ./powerpc64-linux/32/libsanitizer/asan/asan_allocator2.o:         U __sync_fetch_and_add_8
> > 
> > Does ppc32 have any atomic 64-bit loads/stores (in the sense that the aligned
> > 64 bits are written as one memory transaction, not each 32-bit word
> > separately)?
> 
> The only option I can think of would be using the floating point
> load/store instructions lfd/stfd.  Of course if we're going to
> operate on them, then we'd need to copy them back to memory and
> then into the integer registers again....before copying them
> back to the FP registers (thru memory again) so we can store
> them with the stfd.

I think that is pretty much comparable to the use of MMX in i?86 sanitizer.
That will not help on soft-fp ppc, though not sure if anybody with soft-fp
builds libsanitizer.

> > In any case, atomic_uint64_t there seems to be used only for some statistic
> > counter and not really atomic anyway, as additions aren't performed using
> > atomic instructions, but just atomic load, followed by normal arithmetics,
> > followed by atomic store.
> > Can't 32-bit counters be used instead on 32-bit arches?
> 
> Using 32-bit counters would be easiest if we can, but Kostya will have
> to answer that.

I agree with that.  atomic_uintptr_t would be the type to use I think.

	Jakub

Re: libsanitizer merge from upstream r208536

[Peter Bergner]

On Tue, 2014-05-27 at 17:04 -0500, Peter Bergner wrote:
> On Tue, 2014-05-27 at 23:50 +0200, Jakub Jelinek wrote:
> > Does ppc32 have any atomic 64-bit loads/stores (in the sense that the aligned
> > 64 bits are written as one memory transaction, not each 32-bit word
> > separately)?
> 
> The only option I can think of would be using the floating point
> load/store instructions lfd/stfd.  Of course if we're going to
> operate on them, then we'd need to copy them back to memory and
> then into the integer registers again....before copying them
> back to the FP registers (thru memory again) so we can store
> them with the stfd.

Looking at the atomic_store template expansion for atomic_uint64_t
which calls __sync_val_compare_and_swap, it does use lfd/stfd.

Peter

Re: libsanitizer merge from upstream r208536

[Jakub Jelinek]

On Tue, May 27, 2014 at 05:04:52PM -0500, Peter Bergner wrote:
> On Tue, 2014-05-27 at 23:50 +0200, Jakub Jelinek wrote:
> > On Tue, May 27, 2014 at 04:02:08PM -0500, Peter Bergner wrote:
> > > It's being called form basically two files:
> > > 
> > > [bergner@makalu-lp1 gcc-fsf-mainline-asan-debug]$ find . -name '*.o' | xargs nm -AC | grep sync_fetch_and_add_8
> > > ./powerpc64-linux/32/libsanitizer/sanitizer_common/.libs/sanitizer_allocator.o:         U __sync_fetch_and_add_8
> > > ./powerpc64-linux/32/libsanitizer/sanitizer_common/sanitizer_allocator.o:         U __sync_fetch_and_add_8
> > > ./powerpc64-linux/32/libsanitizer/asan/.libs/asan_allocator2.o:         U __sync_fetch_and_add_8
> > > ./powerpc64-linux/32/libsanitizer/asan/asan_allocator2.o:         U __sync_fetch_and_add_8
> > 
> > Does ppc32 have any atomic 64-bit loads/stores (in the sense that the aligned
> > 64 bits are written as one memory transaction, not each 32-bit word
> > separately)?
> 
> The only option I can think of would be using the floating point
> load/store instructions lfd/stfd.  Of course if we're going to
> operate on them, then we'd need to copy them back to memory and
> then into the integer registers again....before copying them
> back to the FP registers (thru memory again) so we can store
> them with the stfd.

OT, if lfd/stfd is atomic (perhaps on a subset of CPUs), why doesn't
config/rs6000/sync.md implement 32-bit atomic_{load,store}di using
that insn?

Though, e.g.
http://lists.apple.com/archives/perfoptimization-dev/2008/Nov/msg00026.html
suggests that it might not be atomic on some CPUs.

	Jakub