[Bug boehm-gc/66848] New: boehm-gc fails test suite on x86_64-apple-darwin15

[Bug boehm-gc/66848] New: boehm-gc fails test suite on x86_64-apple-darwin15

[howarth.at.gcc at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66848

            Bug ID: 66848
           Summary: boehm-gc fails test suite on x86_64-apple-darwin15
           Product: gcc
           Version: 5.2.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: boehm-gc
          Assignee: unassigned at gcc dot gnu.org
          Reporter: howarth.at.gcc at gmail dot com
  Target Milestone: ---

The boehm-gc test suite in FSF gcc shows regressions at -m32/-m64 when run on
x86_64-apple-darwin15...

WARNING: program timed out.
FAIL: boehm-gc.c/gctest.c -O2 execution test
FAIL: boehm-gc.c/thread_leak_test.c -O2 execution test

FAIL: boehm-gc.lib/staticrootstest.c -O2 execution test

The same regressions are seen when running the boehm-gc test suite binaries and
libraries, built entirely on x86_64-apple-darwin14, on x86_64-apple-darwin15.
Replacing the libunwind.dylib on x86_64-apple-darwin15 with one from
x86_64-apple-darwin14 eliminates the boehm-gc test suite regressions.

This issue has been opened as radr://21372179, "the FSF boehm-gc library built
on
10.10 fails to pass its tests on 10.11". The Apple programmers claim the
sources for libunwind.dylib are unchanged between darwin14 and darwin15. The
only difference is the use of the darwin15 SDK and the Xcode 7 clang compilers.
They believe the optimizations applied by the newer clang compiler are safe
(which seems to be supported by the fact that upstream boehm-gc from
http://www.hboehm.info/gc/ shows no test suite failures when built and run
under x86_64-apple-darwin15 with Xcode 7). Rather, they suspect we are tripping
over a latent bug in the FSF gcc boehm-gc sources.

[Bug boehm-gc/66848] boehm-gc fails test suite on x86_64-apple-darwin15

[howarth.at.gcc at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66848

--- Comment #1 from Jack Howarth <howarth.at.gcc at gmail dot com> ---
Created attachment 35957
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=35957&action=edit
x86_64-apple-darwin14 binaries that reproduce regressions on
x86_64-apple-darwin15

[Bug boehm-gc/66848] boehm-gc fails test suite on x86_64-apple-darwin15

[howarth.at.gcc at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66848

--- Comment #3 from Jack Howarth <howarth.at.gcc at gmail dot com> ---
The seg fault in staticrootstexst back traces as...

# lldb ./staticrootstest
(lldb) target create "./staticrootstest"
Current executable set to './staticrootstest' (x86_64).
(lldb) r
Process 35905 launched: './staticrootstest' (x86_64)
dyld: Library not loaded: /nowhere/libstaticrootslib.1.dylib
  Referenced from:
/sw/src/fink.build/gcc5-5.1.1-1/darwin_objdir/x86_64-apple-darwin14.4.0/boehm-gc/testsuite/.libs/./staticrootstest
  Reason: image not found
Process 35905 stopped
* thread #1: tid = 0x20cf25, 0x00007fff5fc01075 dyld`dyld_fatal_error + 1, stop
reason = EXC_BREAKPOINT (code=EXC_I386_BPT, subcode=0x0)
    frame #0: 0x00007fff5fc01075 dyld`dyld_fatal_error + 1
dyld`dyld_fatal_error:
->  0x7fff5fc01075 <+1>: nop    

dyld`dyldbootstrap::start:
    0x7fff5fc01076 <+0>: pushq  %rbp
    0x7fff5fc01077 <+1>: movq   %rsp, %rbp
    0x7fff5fc0107a <+4>: pushq  %r15
(lldb) bt
* thread #1: tid = 0x20cf25, 0x00007fff5fc01075 dyld`dyld_fatal_error + 1, stop
reason = EXC_BREAKPOINT (code=EXC_I386_BPT, subcode=0x0)
  * frame #0: 0x00007fff5fc01075 dyld`dyld_fatal_error + 1
    frame #1: 0x00007fff5fc040d1 dyld`dyld::halt(char const*) + 77
    frame #2: 0x00007fff5fc060e6 dyld`dyld::_main(macho_header const*, unsigned
long, int, char const**, char const**, char const**, unsigned long*) + 4086
    frame #3: 0x00007fff5fc01276 dyld`dyldbootstrap::start(macho_header const*,
int, char const**, long, macho_header const*, unsigned long*) + 512
    frame #4: 0x00007fff5fc01036 dyld`_dyld_start + 54
(lldb)

[Bug boehm-gc/66848] boehm-gc fails test suite on x86_64-apple-darwin15

[howarth.at.gcc at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66848

--- Comment #4 from Jack Howarth <howarth.at.gcc at gmail dot com> ---
The gctest hang backtraces as...

# lldb ./gctest
(lldb) target create "./gctest"
Current executable set to './gctest' (x86_64).
(lldb) r
Process 35911 launched: './gctest' (x86_64)
Switched to incremental mode
Emulating dirty bits with mprotect/signals
thread_get_state failed in forward_exception
Process 35911 stopped
* thread #2: tid = 0x20d116, 0x00007fff8e6c4076
libsystem_kernel.dylib`__pthread_kill + 10, stop reason = signal SIGABRT
    frame #0: 0x00007fff8e6c4076 libsystem_kernel.dylib`__pthread_kill + 10
libsystem_kernel.dylib`__pthread_kill:
->  0x7fff8e6c4076 <+10>: jae    0x7fff8e6c4080            ; <+20>
    0x7fff8e6c4078 <+12>: movq   %rax, %rdi
    0x7fff8e6c407b <+15>: jmp    0x7fff8e6bf0df            ; cerror_nocancel
    0x7fff8e6c4080 <+20>: retq   
(lldb) bt
* thread #2: tid = 0x20d116, 0x00007fff8e6c4076
libsystem_kernel.dylib`__pthread_kill + 10, stop reason = signal SIGABRT
  * frame #0: 0x00007fff8e6c4076 libsystem_kernel.dylib`__pthread_kill + 10
    frame #1: 0x00007fff9e61163d libsystem_pthread.dylib`pthread_kill + 90
    frame #2: 0x00007fff95d5783b libsystem_c.dylib`abort + 129
    frame #3: 0x0000000100022e29 libgcjgc.1.dylib`GC_abort(msg=<unavailable>) +
57 at misc.c:1081
(lldb)

[Bug boehm-gc/66848] boehm-gc fails test suite on x86_64-apple-darwin15

[howarth.at.gcc at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66848

--- Comment #5 from Jack Howarth <howarth.at.gcc at gmail dot com> ---
Created attachment 35958
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=35958&action=edit
bzip2 compressed log of gctest walk in lldb from main

[Bug boehm-gc/66848] boehm-gc fails test suite on x86_64-apple-darwin15

[howarth.at.gcc at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66848

--- Comment #6 from Jack Howarth <howarth.at.gcc at gmail dot com> ---
Created attachment 35959
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=35959&action=edit
bzip2 compressed log of thread_leak_test walk in lldb from main

[Bug boehm-gc/66848] boehm-gc fails test suite on x86_64-apple-darwin15

[howarth.at.gcc at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66848

--- Comment #7 from Jack Howarth <howarth.at.gcc at gmail dot com> ---
Created attachment 35960
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=35960&action=edit
bzip2 compressed log of staticrootstest walk in lldb from main

[Bug boehm-gc/66848] boehm-gc fails test suite on x86_64-apple-darwin15

[howarth.at.gcc at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66848

--- Comment #9 from Jack Howarth <howarth.at.gcc at gmail dot com> ---
Note that the earliest upstream boehm-gc release which builds and passes make
check on 10.11 is gc-7.2.tar.gz from http://www.hboehm.info/gc/gc_source/.
Diffing the current boehm-gc sources in gcc trunk suggests that the current FSF
boehm-gc is based on 7.1 (for at least mach_dep.c, etc).

[Bug boehm-gc/66848] boehm-gc fails test suite on x86_64-apple-darwin15

[howarth.at.gcc at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66848

--- Comment #10 from Jack Howarth <howarth.at.gcc at gmail dot com> ---
FYI, the earliest upstream boehm-gc which builds and passes make check on 10.11
under the Apple clang 7.0 compiled system unwinder is gc-7.2alpha6.

[Bug boehm-gc/66848] boehm-gc fails test suite on x86_64-apple-darwin15

[howarth.at.gcc at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66848

--- Comment #11 from Jack Howarth <howarth.at.gcc at gmail dot com> ---
Changing...

--- gcc-5.2.0.orig/boehm-gc/include/private/gcconfig.h  2013-12-21
15:42:39.000000000 -0500
+++ gcc-5.2.0/boehm-gc/include/private/gcconfig.h       2015-10-11
15:41:26.000000000 -0400
@@ -1041,10 +1041,10 @@
 #   define MACH_TYPE "I386"
 #   if defined(__LP64__) || defined(_WIN64)
 #     define CPP_WORDSZ 64
-#     define ALIGNMENT 8
+#     define ALIGNMENT 2
 #   else
 #     define CPP_WORDSZ 32
-#     define ALIGNMENT 4
+#     define ALIGNMENT 2
                        /* Appears to hold for all "32 bit" compilers   */
                        /* except Borland.  The -a4 option fixes        */
                        /* Borland.                                     */
@@ -2005,10 +2005,10 @@
 # ifdef X86_64
 #   define MACH_TYPE "X86_64"
 #   ifdef __ILP32__
-#     define ALIGNMENT 4
+#     define ALIGNMENT 2
 #     define CPP_WORDSZ 32
 #   else
-#     define ALIGNMENT 8
+#     define ALIGNMENT 2
 #     define CPP_WORDSZ 64
 #   endif
 #   ifndef HBLKSIZE

suppress the failures in the boehm-gc test suite on darwin15's unwinder (as
well as using ALIGNMENT 1). Note that ALIGNMENT 4 on 64-bit doesn't suppress
the boehm-gc test suite failures.

[Bug boehm-gc/66848] boehm-gc fails test suite on x86_64-apple-darwin15

[mrs at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66848

mrs at gcc dot gnu.org <mrs at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |mrs at gcc dot gnu.org

--- Comment #12 from mrs at gcc dot gnu.org <mrs at gcc dot gnu.org> ---
Is this just a partial import from upstream?  If so, I think we should just
check it in and call the issue solved.

[Bug boehm-gc/66848] boehm-gc fails test suite on x86_64-apple-darwin15

[howarth.at.gcc at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66848

--- Comment #13 from Jack Howarth <howarth.at.gcc at gmail dot com> ---
(In reply to mrs@gcc.gnu.org from comment #12)
> Is this just a partial import from upstream?  If so, I think we should just
> check it in and call the issue solved.

   No, the patch shown is an ugly hack of reducing the alignment to 2 (or 1)
which merely suppresses the breakage of boehm-gc (and thus gcj) under the Apple
Clang 7.0 compiled libunwind.dylib on darwin15. The newer upstream boehm-gc
uses the same alignments as gcc's boehm-gc but doesn't suffer the test suite
failures.
   All I can say at the moment is that the bug seems to be alignment related.
However it is unclear to me if the bug actually lies in gcc's current boehm-gc
or whether the breakage really lies in the Apple Clang 7.0 optimizations to the
system unwinder and that the newer upstream boehm-gc merely makes the bug go
latent. Apple claims these new optimizations to the unwinder are valid but I
really doubt that the bug has been truly examined in depth to validate that .

[Bug boehm-gc/66848] boehm-gc fails test suite on x86_64-apple-darwin15

[howarth.at.gcc at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66848

--- Comment #14 from Jack Howarth <howarth.at.gcc at gmail dot com> ---
I finally got around to rebuilding the Apple open source release of
libunwind-35.3 from 10.10.5 under Xcode 7 on 10.10.5. The results are rather
interesting as the default build is a Debug one compiled at -O0. The debug
build of libunwind.dylib produces a binary which exhibits the same breakage in
the boehm-gc test suite binaries built on darwin14 as is seen on darwin15 with
the optimized system libunwind.dylib. This makes it much more likely that the
issue isn't an optimization bug in Apple Clang 7.0 but rather a linker bug in
Xcode 7. Unfortunately, it is impossible to test that by linking the Xcode 7
build under the Xcode 6 linker because the Xcode 7 build uses the 10.11 SDK on
10.10 which needs a linkage on libc++abi.tbd and thus requires the new linker
with the .tbd support.

FYI, The .tbd files are new "text-based stub libraries", that provide a much
more compact version of the stub libraries for use in the SDK, and help to
significantly reduce its download size.

[Bug boehm-gc/66848] boehm-gc fails test suite on x86_64-apple-darwin15

[howarth.at.gcc at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66848

--- Comment #15 from Jack Howarth <howarth.at.gcc at gmail dot com> ---
(In reply to Jack Howarth from comment #14)
> I finally got around to rebuilding the Apple open source release of
> libunwind-35.3 from 10.10.5 under Xcode 7 on 10.10.5. The results are rather
> interesting as the default build is a Debug one compiled at -O0. The debug
> build of libunwind.dylib produces a binary which exhibits the same breakage
> in the boehm-gc test suite binaries built on darwin14 as is seen on darwin15
> with the optimized system libunwind.dylib. This makes it much more likely
> that the issue isn't an optimization bug in Apple Clang 7.0 but rather a
> linker bug in Xcode 7. Unfortunately, it is impossible to test that by
> linking the Xcode 7 build under the Xcode 6 linker because the Xcode 7 build
> uses the 10.11 SDK on 10.10 which needs a linkage on libc++abi.tbd and thus
> requires the new linker with the .tbd support.
> 
> FYI, The .tbd files are new "text-based stub libraries", that provide a much
> more compact version of the stub libraries for use in the SDK, and help to
> significantly reduce its download size.

Nick,
     Are there any changes in the default behavior of the linker from Xcode 6.4
to 7.0 which I can revert in my Xcode 7 build of libunwind-35.3 on 10.10.5 with
linker flags?
          Jack

[Bug boehm-gc/66848] boehm-gc fails test suite on x86_64-apple-darwin15

[jeremyhu at macports dot org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66848

--- Comment #16 from Jeremy Huddleston Sequoia <jeremyhu at macports dot org> ---
(In reply to Jack Howarth from comment #14)
> I finally got around to rebuilding the Apple open source release of
> libunwind-35.3 from 10.10.5 under Xcode 7 on 10.10.5. The results are rather
> interesting as the default build is a Debug one compiled at -O0. The debug
> build of libunwind.dylib produces a binary which exhibits the same breakage
> in the boehm-gc test suite binaries built on darwin14 as is seen on darwin15
> with the optimized system libunwind.dylib. This makes it much more likely
> that the issue isn't an optimization bug in Apple Clang 7.0 but rather a
> linker bug in Xcode 7.

I don't see how you come to that conclusion.  All I see are these data points:

libunwind-35.3 built against the 10.10 SDK with Xcode6-era clang and Xcode6-era
linker produces a libunwind.dylib that does not exhibit this problem.

libunwind-35.3 built against the 10.11 SDK with Xcode7-era clang and Xcode7-era
linker produces a libunwind.dylib that exhibits this problem.

I suggest you try using the Xcode 7 linker with the Xcode 6 compiler and 10.10
SDK.  If you hypothesis is correct, it should fail.  You can do that by just
copying Xcode7's linker to
Xcode6.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld
(make sure you backup the old one of course).


> Unfortunately, it is impossible to test that by
> linking the Xcode 7 build under the Xcode 6 linker because the Xcode 7 build
> uses the 10.11 SDK on 10.10 which needs a linkage on libc++abi.tbd and thus
> requires the new linker with the .tbd support.

Well then I suggest you similarly test using the Xcode 7 compiler with the
Xcode 7 linker and the 10.10 SDK to rule out the SDK as a factor and then test
using the Xcode 7 compiler with the Xcode 6 linker and the 10.10 SDK.

> FYI, The .tbd files are new "text-based stub libraries", that provide a much
> more compact version of the stub libraries for use in the SDK, and help to
> significantly reduce its download size.

[Bug boehm-gc/66848] boehm-gc fails test suite on x86_64-apple-darwin15

[howarth.at.gcc at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66848

--- Comment #17 from Jack Howarth <howarth.at.gcc at gmail dot com> ---
Okay, I have verified on 10.10 that llvm.org clang 3.6 builds a libunwind.dylib
which doesn't show the boehm-gc test suite regressions but when libunwind.dylib
is built with llvm.org clang 3.7, it does. According to Jeremy, the commit in
llvm which tickled this regression in boehm-gc is...

http://llvm.org/viewvc/llvm-project?view=revision&revision=226751

which caused the symbol ordering to change and resulted in the linker moving
stuff out of __bss into __data.

[Bug boehm-gc/66848] boehm-gc fails test suite on x86_64-apple-darwin15

[howarth.at.gcc at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66848

--- Comment #18 from Jack Howarth <howarth.at.gcc at gmail dot com> ---
The upstream commit...

https://github.com/ivmai/bdwgc/commit/faef04e7cb3741163dfdf65900ef5d2a0530be0f

2011-02-09 Ivan Maidanski <ivmai@mail.ru>
        * src/atomic_ops.c (AO_USE_NO_SIGNALS, AO_USE_NANOSLEEP): New
        macros.
        * src/atomic_ops.c (AO_USE_WIN32_PTHREADS): Imply
        AO_USE_NO_SIGNALS.
        * src/atomic_ops.c: Don't include signal.h if AO_USE_NO_SIGNALS.
        * src/atomic_ops.c: Include time.h if AO_USE_NANOSLEEP.
        * src/atomic_ops.c (AO_locks, AO_pause): Reformat the code.
        * src/atomic_ops.c (AO_pause): Use nanosleep() if
        AO_USE_NANOSLEEP.
        * src/atomic_ops.c (all_sigs, initialized,
        AO_compare_and_swap_emulation,
        AO_compare_double_and_swap_double_emulation): Use
        AO_USE_NO_SIGNALS instead of AO_USE_WIN32_PTHREADS.

potentially could contain useful change for darwin. This is the specific commit
in between the 7.2alpha4 and 7.2alpha6 releases which eliminates the test suite
failures on darwin. The caveat is that these failures are for all darwin and
not just darwin15.

[Bug boehm-gc/66848] boehm-gc fails test suite on x86_64-apple-darwin15

[howarth.at.gcc at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66848

--- Comment #19 from Jack Howarth <howarth.at.gcc at gmail dot com> ---
This thread...

http://www.hpl.hp.com/hosted/linux/mail-archives/gc/2011-April/004472.html

suggests that the boehm-gc is older than I suspected (as it claims gcc's copy
is based on the 6.6 release). As far as I can tell, the proposal to update
gcc's boehm-gc to 7.2 never progressed past initial discussions.

http://www.hpl.hp.com/hosted/linux/mail-archives/gc/2011-April/004516.html

The thread pretty much dies off at that point.

[Bug boehm-gc/66848] boehm-gc fails test suite on x86_64-apple-darwin15

[howarth.at.gcc at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66848

--- Comment #20 from Jack Howarth <howarth.at.gcc at gmail dot com> ---
Also, the one commonality in all of the boehm-gc regressions on darwin15
(against the Apple Clang 7.0 compiled libunwind.dylib) is instances of...

* thread #1: tid = 0x20dbb8, 0x00007fff93f37148 libdyld.dylib`dyld_stub_binder,
queue = 'com.apple.main-thread', stop reason = instruction step into
    frame #0: 0x00007fff93f37148 libdyld.dylib`dyld_stub_binder
libdyld.dylib`dyld_stub_binder:
->  0x7fff93f37148 <+0>:  pushq  %rbp
    0x7fff93f37149 <+1>:  testq  $0xf, %rsp
    0x7fff93f37150 <+8>:  jne    0x7fff93f372da            ;
stack_not_16_byte_aligned_error


errors. The fact that hacking the ALIGNMENT setting in
boehm-gc/include/private/gcconfig.h to use 2 eliminates the regressions would
seem to make sense then as this would be forcing everything to be 16-bit
aligned.