* RE: libgccjit 5.2 on Mac OSX Yosemite
@ 2015-01-01 0:00 Dibyendu Majumdar
2015-01-01 0:00 ` Dibyendu Majumdar
0 siblings, 1 reply; 10+ messages in thread
From: Dibyendu Majumdar @ 2015-01-01 0:00 UTC (permalink / raw)
To: David Malcolm; +Cc: jit
Hi Dave,
You may want to submit the patch also to the homebrew folks once I have
confirmed it works as without this the official 5.2 distro isn't going
to work on Mac OSX.
Regards
From: David Malcolm
Sent: 11/08/2015 17:21
To: Dibyendu Majumdar
Cc: jit@gcc.gnu.org
Subject: Re: libgccjit 5.2 on Mac OSX Yosemite
On Mon, 2015-08-10 at 23:52 +0100, Dibyendu Majumdar wrote:
[CCing the list]
> On 10 August 2015 at 22:35, David Malcolm <dmalcolm@redhat.com> wrote:
> > Please can you try enabling logging using:
> > https://gcc.gnu.org/onlinedocs/jit/topics/contexts.html#gcc_jit_context_set_logfile
> >
> > to capture the driver invocation line (have a look for invoke_driver in
> > the log).
> >
> > You'll also need to set the bool option
> > GCC_JIT_BOOL_OPTION_KEEP_INTERMEDIATES to stop it deleting everything.
> >
>
> JIT: entering: void
> gcc::jit::playback::context::invoke_driver(const char*, const char*,
> const char*, timevar_id_t, bool, bool)
>
> JIT: argv[0]: x86_64-apple-darwin14.4.0-gcc-5.2.0
>
> JIT: argv[1]: -shared
>
> JIT: argv[2]:
> /var/folders/lm/ygx0frjs7h30b03r14k7jfhh0000gn/T//libgccjit-BmebwW/fake.s
>
> JIT: argv[3]: -o
>
> JIT: argv[4]:
> /var/folders/lm/ygx0frjs7h30b03r14k7jfhh0000gn/T//libgccjit-BmebwW/fake.so
>
> JIT: argv[5]: -fno-use-linker-plugin
>
> JIT: argv[6]: (null)
>
> Undefined symbols for architecture x86_64:
>
> "_luaD_poscall", referenced from:
>
> _ravif1 in ccouygd9.o
>
> "_luaF_close", referenced from:
>
> _ravif1 in ccouygd9.o
>
> ld: symbol(s) not found for architecture x86_64
>
>
>
> > With that, you can try to re-run the driver invocation directly from the
> > command-line, to see if you can reproduce the problem there, and if the
> > extra option above
> > -Wl,-undefined,dynamic_lookup
> > fixes it.
> >
>
> Appears to fix the error.
>
> x86_64-apple-darwin14.4.0-gcc-5.2.0 -shared
> /var/folders/lm/ygx0frjs7h30b03r14k7jfhh0000gn/T//libgccjit-BmebwW/fake.s
> -o /var/folders/lm/ygx0frjs7h30b03r14k7jfhh0000gn/T//libgccjit-BmebwW/fake.so
> -fno-use-linker-plugin -Wl,-undefined,dynamic_lookup
>
> ls -l /var/folders/lm/ygx0frjs7h30b03r14k7jfhh0000gn/T//libgccjit-BmebwW/fake.so
>
> -rwxr-xr-x 1 dibyendumajumdar staff 8504 10 Aug 23:51
> /var/folders/lm/ygx0frjs7h30b03r14k7jfhh0000gn/T//libgccjit-BmebwW/fake.so
Thanks.
Here's a patch which adds that argument to the driver invocation (it's
against trunk, but it ought to apply to gcc-5 also).
Does it fix things?
Dave
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: libgccjit 5.2 on Mac OSX Yosemite 2015-01-01 0:00 libgccjit 5.2 on Mac OSX Yosemite Dibyendu Majumdar @ 2015-01-01 0:00 ` Dibyendu Majumdar 2015-01-01 0:00 ` [PATCH, committed] jit: on OS X, add -Wl,-undefined,dynamic_lookup to driver invocation David Malcolm 2015-01-01 0:00 ` libgccjit 5.2 on Mac OSX Yosemite David Malcolm 0 siblings, 2 replies; 10+ messages in thread From: Dibyendu Majumdar @ 2015-01-01 0:00 UTC (permalink / raw) To: David Malcolm; +Cc: jit [-- Attachment #1: Type: text/plain, Size: 1014 bytes --] Dave, I was able to get the JIT compiler working on Mac OS X Yosemite using your patch. Attached is the patched home-brew script I used. When I reported the JIT compilation speed I reported earlier I forgot to mention that I was running on SSD disks. With laptop hard disk (5400 rpm) the JIT compiler is obviously even slower - the Lua Tests that ran in 1 second in interpreted mode took 1648 seconds in JIT mode. Timings below. I think that your work on getting rid of the process invocations should be given highest priority as JIT compilation times need to be improved significantly. Interpreted mode: test done on 13/08/2015, at 21:23:43 Lua 5.3 64-bit integers, 64-bit floats cleaning all!!!! ...... ---- total memory: 60.0K, max memory: 2.7M ---- total time: 0.57s (wall time: 1s) JIT mode: testing date/time test done on 13/08/2015, at 21:51:11 Lua 5.3 64-bit integers, 64-bit floats cleaning all!!!! ...... ---- total memory: 59.9K, max memory: 2.7M ---- total time: 659.17s (wall time: 1648s) [-- Attachment #2: gcc.rb --] [-- Type: text/x-ruby-script, Size: 9454 bytes --] class Gcc < Formula def arch if Hardware::CPU.type == :intel if MacOS.prefer_64_bit? "x86_64" else "i686" end elsif Hardware::CPU.type == :ppc if MacOS.prefer_64_bit? "powerpc64" else "powerpc" end end end def osmajor `uname -r`.chomp end desc "GNU compiler collection" homepage "https://gcc.gnu.org" url "http://ftpmirror.gnu.org/gcc/gcc-5.2.0/gcc-5.2.0.tar.bz2" mirror "https://ftp.gnu.org/gnu/gcc/gcc-5.2.0/gcc-5.2.0.tar.bz2" sha256 "5f835b04b5f7dd4f4d2dc96190ec1621b8d89f2dc6f638f9f8bc1b1014ba8cad" bottle do sha256 "b9a41591f99a25a5d57d818953def46c4c654eb5ec8738b5eb975b366068abc3" => :yosemite sha256 "c70b4f9c65930c0f127cb8a0b220d7b1494edea11218f5ba519b758c3eb7a2cd" => :mavericks sha256 "131566719249be8795422d86fff0c4b2ee12bb58ad5320dcbff0af2c40d3ece7" => :mountain_lion end option "with-java", "Build the gcj compiler" option "with-all-languages", "Enable all compilers and languages, except Ada" option "with-nls", "Build with native language support (localization)" option "with-jit", "Build the jit compiler" option "without-fortran", "Build without the gfortran compiler" # enabling multilib on a host that can't run 64-bit results in build failures option "without-multilib", "Build without multilib support" if MacOS.prefer_64_bit? depends_on "gmp" depends_on "libmpc" depends_on "mpfr" depends_on "isl" depends_on "ecj" if build.with?("java") || build.with?("all-languages") if MacOS.version < :leopard # The as that comes with Tiger isn't capable of dealing with the # PPC asm that comes in libitm depends_on "cctools" => :build end fails_with :gcc_4_0 fails_with :llvm # GCC bootstraps itself, so it is OK to have an incompatible C++ stdlib cxxstdlib_check :skip # The bottles are built on systems with the CLT installed, and do not work # out of the box on Xcode-only systems due to an incorrect sysroot. def pour_bottle? MacOS::CLT.installed? end def version_suffix version.to_s.slice(/\d/) end # Fix for libgccjit.so linkage on Darwin # https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64089 patch :DATA def install # GCC will suffer build errors if forced to use a particular linker. ENV.delete "LD" if MacOS.version < :leopard ENV["AS"] = ENV["AS_FOR_TARGET"] = "#{Formula["cctools"].bin}/as" end if build.with? "all-languages" # Everything but Ada, which requires a pre-existing GCC Ada compiler # (gnat) to bootstrap. GCC 4.6.0 adds go as a language option, but it is # currently only compilable on Linux. languages = %w[c c++ objc obj-c++ fortran java jit] else # C, C++, ObjC compilers are always built languages = %w[c c++ objc obj-c++] languages << "fortran" if build.with? "fortran" languages << "java" if build.with? "java" languages << "jit" if build.with? "jit" end args = [ "--build=#{arch}-apple-darwin#{osmajor}", "--prefix=#{prefix}", "--libdir=#{lib}/gcc/#{version_suffix}", "--enable-languages=#{languages.join(",")}", # Make most executables versioned to avoid conflicts. "--program-suffix=-#{version_suffix}", "--with-gmp=#{Formula["gmp"].opt_prefix}", "--with-mpfr=#{Formula["mpfr"].opt_prefix}", "--with-mpc=#{Formula["libmpc"].opt_prefix}", "--with-isl=#{Formula["isl"].opt_prefix}", "--with-system-zlib", "--enable-libstdcxx-time=yes", "--enable-stage1-checking", "--enable-checking=release", "--enable-lto", # Use 'bootstrap-debug' build configuration to force stripping of object # files prior to comparison during bootstrap (broken by Xcode 6.3). "--with-build-config=bootstrap-debug", "--disable-werror", "--with-pkgversion=Homebrew #{name} #{pkg_version} #{build.used_options*" "}".strip, "--with-bugurl=https://github.com/Homebrew/homebrew/issues" ] # "Building GCC with plugin support requires a host that supports # -fPIC, -shared, -ldl and -rdynamic." args << "--enable-plugin" if MacOS.version > :tiger # Otherwise make fails during comparison at stage 3 # See: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=45248 args << "--with-dwarf2" if MacOS.version < :leopard args << "--disable-nls" if build.without? "nls" if build.with?("java") || build.with?("all-languages") args << "--with-ecj-jar=#{Formula["ecj"].opt_share}/java/ecj.jar" end if build.without?("multilib") || !MacOS.prefer_64_bit? args << "--disable-multilib" else args << "--enable-multilib" end args << "--enable-host-shared" if build.with?("jit") || build.with?("all-languages") # Ensure correct install names when linking against libgcc_s; # see discussion in https://github.com/Homebrew/homebrew/pull/34303 inreplace "libgcc/config/t-slibgcc-darwin", "@shlib_slibdir@", "#{HOMEBREW_PREFIX}/lib/gcc/#{version_suffix}" mkdir "build" do unless MacOS::CLT.installed? # For Xcode-only systems, we need to tell the sysroot path. # "native-system-headers" will be appended args << "--with-native-system-header-dir=/usr/include" args << "--with-sysroot=#{MacOS.sdk_path}" end system "../configure", *args system "make", "bootstrap" system "make", "install" if build.with?("fortran") || build.with?("all-languages") bin.install_symlink bin/"gfortran-#{version_suffix}" => "gfortran" end end # Handle conflicts between GCC formulae and avoid interfering # with system compilers. # Since GCC 4.8 libffi stuff are no longer shipped. # Rename man7. Dir.glob(man7/"*.7") { |file| add_suffix file, version_suffix } # Even when suffixes are appended, the info pages conflict when # install-info is run. TODO fix this. info.rmtree # Rename java properties if build.with?("java") || build.with?("all-languages") config_files = [ "#{lib}/gcc/#{version_suffix}/logging.properties", "#{lib}/gcc/#{version_suffix}/security/classpath.security", "#{lib}/gcc/#{version_suffix}/i386/logging.properties", "#{lib}/gcc/#{version_suffix}/i386/security/classpath.security" ] config_files.each do |file| add_suffix file, version_suffix if File.exist? file end end end def add_suffix(file, suffix) dir = File.dirname(file) ext = File.extname(file) base = File.basename(file, ext) File.rename file, "#{dir}/#{base}-#{suffix}#{ext}" end def caveats if build.with?("multilib") then <<-EOS.undent GCC has been built with multilib support. Notably, OpenMP may not work: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=60670 If you need OpenMP support you may want to brew reinstall gcc --without-multilib EOS end end test do (testpath/"hello-c.c").write <<-EOS.undent #include <stdio.h> int main() { puts("Hello, world!"); return 0; } EOS system "#{bin}/gcc-#{version_suffix}", "-o", "hello-c", "hello-c.c" assert_equal "Hello, world!\n", `./hello-c` (testpath/"hello-cc.cc").write <<-EOS.undent #include <iostream> int main() { std::cout << "Hello, world!" << std::endl; return 0; } EOS system "#{bin}/g++-#{version_suffix}", "-o", "hello-cc", "hello-cc.cc" assert_equal "Hello, world!\n", `./hello-cc` if build.with?("fortran") || build.with?("all-languages") fixture = <<-EOS.undent integer,parameter::m=10000 real::a(m), b(m) real::fact=0.5 do concurrent (i=1:m) a(i) = a(i) + fact*b(i) end do print *, "done" end EOS (testpath/"in.f90").write(fixture) system "#{bin}/gfortran", "-c", "in.f90" system "#{bin}/gfortran", "-o", "test", "in.o" assert_equal "done", `./test`.strip end end end __END__ diff --git a/gcc/jit/Make-lang.in b/gcc/jit/Make-lang.in index 44d0750..4df2a9c 100644 --- a/gcc/jit/Make-lang.in +++ b/gcc/jit/Make-lang.in @@ -85,8 +85,7 @@ $(LIBGCCJIT_FILENAME): $(jit_OBJS) \ $(jit_OBJS) libbackend.a libcommon-target.a libcommon.a \ $(CPPLIB) $(LIBDECNUMBER) $(LIBS) $(BACKENDLIBS) \ $(EXTRA_GCC_OBJS) \ - -Wl,--version-script=$(srcdir)/jit/libgccjit.map \ - -Wl,-soname,$(LIBGCCJIT_SONAME) + -Wl,-install_name,$(LIBGCCJIT_SONAME) $(LIBGCCJIT_SONAME_SYMLINK): $(LIBGCCJIT_FILENAME) ln -sf $(LIBGCCJIT_FILENAME) $(LIBGCCJIT_SONAME_SYMLINK) diff --git a/gcc/jit/jit-playback.c b/gcc/jit/jit-playback.c index 925fa86..01cfd4b 100644 --- a/gcc/jit/jit-playback.c +++ b/gcc/jit/jit-playback.c @@ -2416,6 +2416,15 @@ invoke_driver (const char *ctxt_progname, time. */ ADD_ARG ("-fno-use-linker-plugin"); +#if defined (DARWIN_X86) || defined (DARWIN_PPC) + /* OS X's linker defaults to treating undefined symbols as errors. + If the context has any imported functions or globals they will be + undefined until the .so is dynamically-linked into the process. + Ensure that the driver passes in "-undefined dynamic_lookup" to the + linker. */ + ADD_ARG ("-Wl,-undefined,dynamic_lookup"); +#endif + /* pex argv arrays are NULL-terminated. */ argvec.safe_push (NULL); ^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH, committed] jit: on OS X, add -Wl,-undefined,dynamic_lookup to driver invocation 2015-01-01 0:00 ` Dibyendu Majumdar @ 2015-01-01 0:00 ` David Malcolm 2015-01-01 0:00 ` libgccjit 5.2 on Mac OSX Yosemite David Malcolm 1 sibling, 0 replies; 10+ messages in thread From: David Malcolm @ 2015-01-01 0:00 UTC (permalink / raw) To: jit, gcc-patches; +Cc: David Malcolm Reported as fixing the library on OS X Yosemite: https://gcc.gnu.org/ml/jit/2015-q3/msg00147.html Committed to trunk as r226882. Committed to gcc-5-branch as r226883. gcc/jit/ChangeLog: * jit-playback.c (invoke_driver): On OS X, add "-Wl,-undefined,dynamic_lookup" to the driver arguments. --- gcc/jit/jit-playback.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/gcc/jit/jit-playback.c b/gcc/jit/jit-playback.c index 925fa86..01cfd4b 100644 --- a/gcc/jit/jit-playback.c +++ b/gcc/jit/jit-playback.c @@ -2416,6 +2416,15 @@ invoke_driver (const char *ctxt_progname, time. */ ADD_ARG ("-fno-use-linker-plugin"); +#if defined (DARWIN_X86) || defined (DARWIN_PPC) + /* OS X's linker defaults to treating undefined symbols as errors. + If the context has any imported functions or globals they will be + undefined until the .so is dynamically-linked into the process. + Ensure that the driver passes in "-undefined dynamic_lookup" to the + linker. */ + ADD_ARG ("-Wl,-undefined,dynamic_lookup"); +#endif + /* pex argv arrays are NULL-terminated. */ argvec.safe_push (NULL); -- 1.8.5.3 ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: libgccjit 5.2 on Mac OSX Yosemite 2015-01-01 0:00 ` Dibyendu Majumdar 2015-01-01 0:00 ` [PATCH, committed] jit: on OS X, add -Wl,-undefined,dynamic_lookup to driver invocation David Malcolm @ 2015-01-01 0:00 ` David Malcolm 2015-01-01 0:00 ` Dibyendu Majumdar 1 sibling, 1 reply; 10+ messages in thread From: David Malcolm @ 2015-01-01 0:00 UTC (permalink / raw) To: Dibyendu Majumdar; +Cc: jit On Thu, 2015-08-13 at 22:04 +0100, Dibyendu Majumdar wrote: > Dave, > > I was able to get the JIT compiler working on Mac OS X Yosemite using > your patch. Attached is the patched home-brew script I used. Thanks; glad it's working. As noted in the followup I posted, I've now committed the patch to both branches. I've also created this pull request: https://github.com/Homebrew/homebrew/pull/42924 requesting that they bring in your change to the homebrew gcc.rb recipe. > When I reported the JIT compilation speed I reported earlier I forgot > to mention that I was running on SSD disks. With laptop hard disk > (5400 rpm) the JIT compiler is obviously even slower - the Lua Tests > that ran in 1 second in interpreted mode took 1648 seconds in JIT > mode. Timings below. I think that your work on getting rid of the > process invocations should be given highest priority as JIT > compilation times need to be improved significantly. (nods) If you get a chance to try gcc trunk, it might be interesting to try the profiling hooks I added: https://gcc.gnu.org/ml/jit/2015-q3/msg00134.html to see the libgccjit profile for Ravi. In particular, I'm hoping that the Ravi profile is sufficiently similar to that from test-benchmark.c so that my optimizations will benefit Ravi (or maybe there are other unexpected things going on?) Out of interest, how does this compare to LLVM. IIRC you had said libgccjit was taking about 5x longer than LLVM; is that still the case? > Interpreted mode: > > test done on 13/08/2015, at 21:23:43 > Lua 5.3 > 64-bit integers, 64-bit floats > cleaning all!!!! > ...... ---- total memory: 60.0K, max memory: 2.7M ---- > total time: 0.57s (wall time: 1s) > > > JIT mode: > testing date/time > test done on 13/08/2015, at 21:51:11 > Lua 5.3 > 64-bit integers, 64-bit floats > cleaning all!!!! > ...... ---- total memory: 59.9K, max memory: 2.7M ---- > total time: 659.17s (wall time: 1648s) ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: libgccjit 5.2 on Mac OSX Yosemite 2015-01-01 0:00 ` libgccjit 5.2 on Mac OSX Yosemite David Malcolm @ 2015-01-01 0:00 ` Dibyendu Majumdar 0 siblings, 0 replies; 10+ messages in thread From: Dibyendu Majumdar @ 2015-01-01 0:00 UTC (permalink / raw) To: David Malcolm; +Cc: jit On 14 August 2015 at 02:38, David Malcolm <dmalcolm@redhat.com> wrote: > Out of interest, how does this compare to LLVM. IIRC you had said > libgccjit was taking about 5x longer than LLVM; is that still the case? > >> Interpreted mode: >> >> test done on 13/08/2015, at 21:23:43 >> Lua 5.3 >> 64-bit integers, 64-bit floats >> cleaning all!!!! >> ...... ---- total memory: 60.0K, max memory: 2.7M ---- >> total time: 0.57s (wall time: 1s) >> >> >> JIT mode: >> testing date/time >> test done on 13/08/2015, at 21:51:11 >> Lua 5.3 >> 64-bit integers, 64-bit floats >> cleaning all!!!! >> ...... ---- total memory: 59.9K, max memory: 2.7M ---- >> total time: 659.17s (wall time: 1648s) > LLVM 3.7 branch on same hardware and OS: test done on 14/08/2015, at 21:49:47 Lua 5.3 64-bit integers, 64-bit floats cleaning all!!!! ...... ---- total memory: 59.9K, max memory: 2.2M ---- total time: 149.18s (wall time: 151s) Note that the LLVM version handles a few more byte codes so this would be a disadvantage to LLVM as it will compile more functions. With libgccjit you can literally watch the processes being invoked - as, collect2, ld, etc. - as the OS X terminal shows any subprocesses launched in the title bar. On my other environment which is running Ubuntu as a VM under Windows, the difference is less as that environment has SSD disks so that helps libgccjit. Regards Dibyendu ^ permalink raw reply [flat|nested] 10+ messages in thread
* libgccjit 5.2 on Mac OSX Yosemite @ 2015-01-01 0:00 Dibyendu Majumdar 2015-01-01 0:00 ` David Malcolm 0 siblings, 1 reply; 10+ messages in thread From: Dibyendu Majumdar @ 2015-01-01 0:00 UTC (permalink / raw) To: jit Hi I tried building gcc 5.2 on Mac OSX but could not get the linking to work. Then fortunately found that homebrew has a formula for deploy gcc 5.2 with jit enabled. All installed and then built Ravi successfully. But when I try to run Ravi any attempt to invoke the JIT fails with: Lua 5.3.1 Copyright (C) 1994-2015 Lua.org, PUC-Rio > function x() return end > ravi.compile(x) Undefined symbols for architecture x86_64: "_luaD_poscall", referenced from: _ravif1 in cczhNq92.o "_luaF_close", referenced from: _ravif1 in cczhNq92.o ld: symbol(s) not found for architecture x86_64 collect2: error: ld returned 1 exit status libgccjit.so: error: error invoking gcc driver: exit_status: 256 err: 0 libgccjit.so: error: whilst attempting to run a driver named: x86_64-apple-darwin14.4.0-gcc-5.2.0 libgccjit.so: error: PATH was: /Users/username/github/ravi/build:/usr/local/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/git/bin:/Library/TeX/texbin aborting due to JIT error: error invoking gcc driver: exit_status: 256 err: 0 Abort trap: 6 Any clues? I had set PATH and LD_LIBRARY_PATH. Regards ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: libgccjit 5.2 on Mac OSX Yosemite 2015-01-01 0:00 Dibyendu Majumdar @ 2015-01-01 0:00 ` David Malcolm 2015-01-01 0:00 ` David Malcolm 0 siblings, 1 reply; 10+ messages in thread From: David Malcolm @ 2015-01-01 0:00 UTC (permalink / raw) To: Dibyendu Majumdar; +Cc: jit On Mon, 2015-08-10 at 21:36 +0100, Dibyendu Majumdar wrote: > Hi > > I tried building gcc 5.2 on Mac OSX but could not get the linking to > work. Then fortunately found that homebrew has a formula for deploy > gcc 5.2 with jit enabled. Aha. Thanks. This would seem to be: https://github.com/Homebrew/homebrew/blob/master/Library/Formula/gcc.rb which I see slightly patches gcc/jit/Make-lang.in to avoid an assumption that we're building using the GNU linker. > All installed and then built Ravi successfully. But when I try to run > Ravi any attempt to invoke the JIT fails with: > > Lua 5.3.1 Copyright (C) 1994-2015 Lua.org, PUC-Rio > > > function x() return end > > > ravi.compile(x) > > Undefined symbols for architecture x86_64: > > "_luaD_poscall", referenced from: > > _ravif1 in cczhNq92.o > > "_luaF_close", referenced from: > > _ravif1 in cczhNq92.o > > ld: symbol(s) not found for architecture x86_64 > > collect2: error: ld returned 1 exit status > > libgccjit.so: error: error invoking gcc driver: exit_status: 256 err: 0 > > libgccjit.so: error: whilst attempting to run a driver named: > x86_64-apple-darwin14.4.0-gcc-5.2.0 > > libgccjit.so: error: PATH was: > /Users/username/github/ravi/build:/usr/local/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/git/bin:/Library/TeX/texbin > > aborting due to JIT error: error invoking gcc driver: exit_status: 256 err: 0 > > Abort trap: 6 > > Any clues? > > I had set PATH and LD_LIBRARY_PATH. Presumably "luaD_poscall" and "luaF_close" are symbols that you're exporting from ravi for use by the JIT-generated code, via: https://github.com/dibyendumajumdar/ravi/blob/master/src/ravi_gcctypes.c#L802 t->luaD_poscallT = gcc_jit_context_new_function( ravi->context, NULL, GCC_JIT_FUNCTION_IMPORTED, t->C_intT, "luaD_poscall", 3, params, 0); I'm not quite sure exactly where the leading underscore came from. This appears to be a linker error from the static link that creates the jit-generated .so file. If I'm reading things right, it's complaining that the code is using luaD_poscall, but it doesn't have a definition for it. My guess is that "ld" on OS X has slightly different behavior from the GNU "ld" I've been developing against, and that it defaults to a stricter linkage model where it wants to see those symbols upfront, when we want to defer them until the later dynamic link phase (when we dlopen the generated .so file). This is a guess though. What does "ld --version" print? It may be easiest to try to work from a minimal reproducer for this. To that end, what happens if you try building and running gcc/testsuite/jit.dg/test-calling-external-function.c from the jit testsuite, using the homebrew build? Thanks Dave ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: libgccjit 5.2 on Mac OSX Yosemite 2015-01-01 0:00 ` David Malcolm @ 2015-01-01 0:00 ` David Malcolm [not found] ` <CACXZuxcMbdLjzv01WyNReKCub=jZW4Qow1iZKz3juYfo-vEk=w@mail.gmail.com> 0 siblings, 1 reply; 10+ messages in thread From: David Malcolm @ 2015-01-01 0:00 UTC (permalink / raw) To: Dibyendu Majumdar; +Cc: jit On Mon, 2015-08-10 at 17:23 -0400, David Malcolm wrote: > On Mon, 2015-08-10 at 21:36 +0100, Dibyendu Majumdar wrote: > > Hi > > > > I tried building gcc 5.2 on Mac OSX but could not get the linking to > > work. Then fortunately found that homebrew has a formula for deploy > > gcc 5.2 with jit enabled. > > > Aha. Thanks. This would seem to be: > https://github.com/Homebrew/homebrew/blob/master/Library/Formula/gcc.rb > > which I see slightly patches > gcc/jit/Make-lang.in > to avoid an assumption that we're building using the GNU linker. > > > > > All installed and then built Ravi successfully. But when I try to run > > Ravi any attempt to invoke the JIT fails with: > > > > Lua 5.3.1 Copyright (C) 1994-2015 Lua.org, PUC-Rio > > > > > function x() return end > > > > > ravi.compile(x) > > > > Undefined symbols for architecture x86_64: > > > > "_luaD_poscall", referenced from: > > > > _ravif1 in cczhNq92.o > > > > "_luaF_close", referenced from: > > > > _ravif1 in cczhNq92.o > > > > ld: symbol(s) not found for architecture x86_64 > > > > collect2: error: ld returned 1 exit status > > > > libgccjit.so: error: error invoking gcc driver: exit_status: 256 err: 0 > > > > libgccjit.so: error: whilst attempting to run a driver named: > > x86_64-apple-darwin14.4.0-gcc-5.2.0 > > > > libgccjit.so: error: PATH was: > > /Users/username/github/ravi/build:/usr/local/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/git/bin:/Library/TeX/texbin > > > > aborting due to JIT error: error invoking gcc driver: exit_status: 256 err: 0 > > > > Abort trap: 6 > > > > Any clues? > > > > I had set PATH and LD_LIBRARY_PATH. > > Presumably "luaD_poscall" and "luaF_close" are symbols that you're > exporting from ravi for use by the JIT-generated code, via: > https://github.com/dibyendumajumdar/ravi/blob/master/src/ravi_gcctypes.c#L802 > > t->luaD_poscallT = gcc_jit_context_new_function( > ravi->context, NULL, GCC_JIT_FUNCTION_IMPORTED, t->C_intT, > "luaD_poscall", > 3, params, 0); > > I'm not quite sure exactly where the leading underscore came from. > > This appears to be a linker error from the static link that creates the > jit-generated .so file. If I'm reading things right, it's complaining > that the code is using luaD_poscall, but it doesn't have a definition > for it. > > My guess is that "ld" on OS X has slightly different behavior from the > GNU "ld" I've been developing against, and that it defaults to a > stricter linkage model where it wants to see those symbols upfront, when > we want to defer them until the later dynamic link phase (when we dlopen > the generated .so file). This is a guess though. It looks like we might need to pass an "-undefined" option to the OS X linker; http://www.manpages.info/macosx/ld.1.html says: (begin quote) -undefined treatment Specifies how undefined symbols are to be treated. treatment can be: error, warning, or suppress. Which cause the treatment of undefined symbols as either, errors, warnings, or suppresses the checking of undefined symbols. The default is to treat undefined symbols as errors. When the environment variable MACOSX_DEPLOYMENT_TARGET is set to 10.3 then -undefined dynamic_lookup can also be used to allow any undefined symbols to be looked up dynamically at runtime. Use of a binary built with this flag requires a system with a dynamic linker from Mac OS X 10.3 or later. The flag -undefined define_a_way can also be used to cause the static linker to create a private definition for all undefined symbols. This flag should only be used if it is known that the undefined symbols are not referenced as any use of them may cause a crash. (end of quote) Note that it defaults to treating undefined symbols as errors, which looks like what you're seeing. Hence I *think* we need to pass: -undefined dynamic_lookup to the linker. Hence I think we need something like this when invoking the driver: -Wl,-undefined,dynamic_lookup so that it passes the option to the linker. > What does "ld --version" print? > > It may be easiest to try to work from a minimal reproducer for this. > To that end, what happens if you try building and running > gcc/testsuite/jit.dg/test-calling-external-function.c > from the jit testsuite, using the homebrew build? Please can you try enabling logging using: https://gcc.gnu.org/onlinedocs/jit/topics/contexts.html#gcc_jit_context_set_logfile to capture the driver invocation line (have a look for invoke_driver in the log). You'll also need to set the bool option GCC_JIT_BOOL_OPTION_KEEP_INTERMEDIATES to stop it deleting everything. With that, you can try to re-run the driver invocation directly from the command-line, to see if you can reproduce the problem there, and if the extra option above -Wl,-undefined,dynamic_lookup fixes it. Thanks Dave ^ permalink raw reply [flat|nested] 10+ messages in thread
[parent not found: <CACXZuxcMbdLjzv01WyNReKCub=jZW4Qow1iZKz3juYfo-vEk=w@mail.gmail.com>]
* Re: libgccjit 5.2 on Mac OSX Yosemite [not found] ` <CACXZuxcMbdLjzv01WyNReKCub=jZW4Qow1iZKz3juYfo-vEk=w@mail.gmail.com> @ 2015-01-01 0:00 ` David Malcolm 2015-01-01 0:00 ` Dibyendu Majumdar 0 siblings, 1 reply; 10+ messages in thread From: David Malcolm @ 2015-01-01 0:00 UTC (permalink / raw) To: Dibyendu Majumdar; +Cc: jit [-- Attachment #1: Type: text/plain, Size: 2219 bytes --] On Mon, 2015-08-10 at 23:52 +0100, Dibyendu Majumdar wrote: [CCing the list] > On 10 August 2015 at 22:35, David Malcolm <dmalcolm@redhat.com> wrote: > > Please can you try enabling logging using: > > https://gcc.gnu.org/onlinedocs/jit/topics/contexts.html#gcc_jit_context_set_logfile > > > > to capture the driver invocation line (have a look for invoke_driver in > > the log). > > > > You'll also need to set the bool option > > GCC_JIT_BOOL_OPTION_KEEP_INTERMEDIATES to stop it deleting everything. > > > > JIT: entering: void > gcc::jit::playback::context::invoke_driver(const char*, const char*, > const char*, timevar_id_t, bool, bool) > > JIT: argv[0]: x86_64-apple-darwin14.4.0-gcc-5.2.0 > > JIT: argv[1]: -shared > > JIT: argv[2]: > /var/folders/lm/ygx0frjs7h30b03r14k7jfhh0000gn/T//libgccjit-BmebwW/fake.s > > JIT: argv[3]: -o > > JIT: argv[4]: > /var/folders/lm/ygx0frjs7h30b03r14k7jfhh0000gn/T//libgccjit-BmebwW/fake.so > > JIT: argv[5]: -fno-use-linker-plugin > > JIT: argv[6]: (null) > > Undefined symbols for architecture x86_64: > > "_luaD_poscall", referenced from: > > _ravif1 in ccouygd9.o > > "_luaF_close", referenced from: > > _ravif1 in ccouygd9.o > > ld: symbol(s) not found for architecture x86_64 > > > > > With that, you can try to re-run the driver invocation directly from the > > command-line, to see if you can reproduce the problem there, and if the > > extra option above > > -Wl,-undefined,dynamic_lookup > > fixes it. > > > > Appears to fix the error. > > x86_64-apple-darwin14.4.0-gcc-5.2.0 -shared > /var/folders/lm/ygx0frjs7h30b03r14k7jfhh0000gn/T//libgccjit-BmebwW/fake.s > -o /var/folders/lm/ygx0frjs7h30b03r14k7jfhh0000gn/T//libgccjit-BmebwW/fake.so > -fno-use-linker-plugin -Wl,-undefined,dynamic_lookup > > ls -l /var/folders/lm/ygx0frjs7h30b03r14k7jfhh0000gn/T//libgccjit-BmebwW/fake.so > > -rwxr-xr-x 1 dibyendumajumdar staff 8504 10 Aug 23:51 > /var/folders/lm/ygx0frjs7h30b03r14k7jfhh0000gn/T//libgccjit-BmebwW/fake.so Thanks. Here's a patch which adds that argument to the driver invocation (it's against trunk, but it ought to apply to gcc-5 also). Does it fix things? Dave [-- Attachment #2: 0001-jit-on-OS-X-add-Wl-undefined-dynamic_lookup-to-drive.patch --] [-- Type: text/x-patch, Size: 1211 bytes --] From 9971d11fd5e151ae00d214492e14d8d3347bf471 Mon Sep 17 00:00:00 2001 From: David Malcolm <dmalcolm@redhat.com> Date: Tue, 11 Aug 2015 11:48:46 -0400 Subject: [PATCH] jit: on OS X, add -Wl,-undefined,dynamic_lookup to driver invocation gcc/jit/ChangeLog: * jit-playback.c (invoke_driver): On OS X, add "-Wl,-undefined,dynamic_lookup" to the driver arguments. --- gcc/jit/jit-playback.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/gcc/jit/jit-playback.c b/gcc/jit/jit-playback.c index 925fa86..01cfd4b 100644 --- a/gcc/jit/jit-playback.c +++ b/gcc/jit/jit-playback.c @@ -2416,6 +2416,15 @@ invoke_driver (const char *ctxt_progname, time. */ ADD_ARG ("-fno-use-linker-plugin"); +#if defined (DARWIN_X86) || defined (DARWIN_PPC) + /* OS X's linker defaults to treating undefined symbols as errors. + If the context has any imported functions or globals they will be + undefined until the .so is dynamically-linked into the process. + Ensure that the driver passes in "-undefined dynamic_lookup" to the + linker. */ + ADD_ARG ("-Wl,-undefined,dynamic_lookup"); +#endif + /* pex argv arrays are NULL-terminated. */ argvec.safe_push (NULL); -- 1.8.5.3 ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: libgccjit 5.2 on Mac OSX Yosemite 2015-01-01 0:00 ` David Malcolm @ 2015-01-01 0:00 ` Dibyendu Majumdar 0 siblings, 0 replies; 10+ messages in thread From: Dibyendu Majumdar @ 2015-01-01 0:00 UTC (permalink / raw) To: David Malcolm; +Cc: jit On 11 August 2015 at 17:21, David Malcolm <dmalcolm@redhat.com> wrote: > Here's a patch which adds that argument to the driver invocation (it's > against trunk, but it ought to apply to gcc-5 also). > > Does it fix things? > Thanks Dave - unfortunately I have temporarily lost my Mac OSX environment - I recently upgraded to Yosemite, but since it was slow I thought I would do a clean install, and now am not stuck with the installer not finishing :-(. I will look at the patch once everything is back to normal. I cannot build gcc myself so I will have to create a modified homebrew formula I think. Regards Dibyendu ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2015-08-14 20:57 UTC | newest] Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-01-01 0:00 libgccjit 5.2 on Mac OSX Yosemite Dibyendu Majumdar 2015-01-01 0:00 ` Dibyendu Majumdar 2015-01-01 0:00 ` [PATCH, committed] jit: on OS X, add -Wl,-undefined,dynamic_lookup to driver invocation David Malcolm 2015-01-01 0:00 ` libgccjit 5.2 on Mac OSX Yosemite David Malcolm 2015-01-01 0:00 ` Dibyendu Majumdar -- strict thread matches above, loose matches on Subject: below -- 2015-01-01 0:00 Dibyendu Majumdar 2015-01-01 0:00 ` David Malcolm 2015-01-01 0:00 ` David Malcolm [not found] ` <CACXZuxcMbdLjzv01WyNReKCub=jZW4Qow1iZKz3juYfo-vEk=w@mail.gmail.com> 2015-01-01 0:00 ` David Malcolm 2015-01-01 0:00 ` Dibyendu Majumdar
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).