From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 18874 invoked by alias); 25 Sep 2014 20:33:22 -0000 Mailing-List: contact jit-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Post: List-Help: List-Subscribe: Sender: jit-owner@gcc.gnu.org Received: (qmail 18846 invoked by uid 89); 25 Sep 2014 20:33:21 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Checked: by ClamAV 0.98.4 on sourceware.org X-Virus-Found: No X-Spam-SWARE-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00,RP_MATCHES_RCVD,SPF_HELO_PASS autolearn=ham version=3.3.2 X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00,RP_MATCHES_RCVD,SPF_HELO_PASS autolearn=ham version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on sourceware.org X-Spam-Level: X-Spam-User: qpsmtpd, 2 recipients X-HELO: mx1.redhat.com From: David Malcolm To: jit@gcc.gnu.org, gcc-patches@gcc.gnu.org Cc: "Joseph S. Myers" , David Malcolm Subject: [jit] Avoiding hardcoding "gcc"; supporting accelerators? Date: Wed, 01 Jan 2014 00:00:00 -0000 Message-Id: <1411676975-1096-1-git-send-email-dmalcolm@redhat.com> In-Reply-To: References: X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-SW-Source: 2014-q3/txt/msg00044.txt.bz2 On Tue, 2014-09-23 at 23:27 +0000, Joseph S. Myers wrote: [...] > The code for compiling a .s file should: [...] > * use the $(target_noncanonical)-gcc-$(version) name for the driver rather > than plain "gcc", to maximise the chance that it is actually the same > compiler the JIT library was built for (I realise you may not actually > depend on it being the same compiler, but that does seem best; in > principle in future it should be possible to load multiple copies of the > JIT library to JIT for different targets, so that code for an offload > accelerator can go through the JIT). [...] Should this have the $(exeext) suffix seen in Makefile.in? $(target_noncanonical)-gcc-$(version)$(exeext) The attached patch does this, by generating a new gcc-harness-name.h and using that name in internal-api.c when invoking the harness. That said, I haven't applied this to my branch yet, since it requires the name to be findable on $PATH. Hence the user needs to (A) build and install the driver, and (B) $PATH needs to be updated accordingly. (A) is a minor extra hurdle when developing. In my prebuilt RPM packages I haven't actually been shipping the driver, just the libs, relying on a sufficiently compatible harness around. That may be just a side-effect of this not yet being in trunk. As for (B), would it make sense to "bake in" the path to the binary into the pex invocation, and hence to turn off PEX_SEARCH? If so, presumably I need to somehow expand the Makefile's value of $(bindir) into internal-api.c, right? (I tried this in configure.ac, but merely got "$(exec_prefix)/bin" iirc). A better long-term approach to this would be to extract the spec machinery from gcc.c (perhaps into a "libdriver.a"?) and run it directly from the jit library - but that's a rather involved patch, I suspect. [...] > [...] in > principle in future it should be possible to load multiple copies of the > JIT library to JIT for different targets, so that code for an offload > accelerator can go through the JIT). I had a mini-brainstorm about JIT and accelerator support; here goes... (Big caveat: the last time I hacked on GPUs was 12 years ago in my past life as a game developer, when people were publishing papers on using them for general compute purposes, iirc) I wonder if the appropriate approach here is to have a single library with multiple plugin backends e.g. one for the CPU, one for each GPU family, with the ability to load multiple "backends" at once. Unfortunately, "backend" is horribly overloaded here - I mean basically all of gcc here, everything other than the libgccjit.h API seen by client code. In this model libgccjit.h and .c become the only real part of the library, with all of the classes becoming abstract interfaces, with gcc_jit_context implemented by the plugins. The embedded copy of the bulk of gcc becomes the jit-plugin, and exposes only a very narrow API to the main library, with almost all of the names hidden from the linker. That way you could have something like (assuming x86_64-unknown-linux host cpu/os): +-----------+ +------------+ |client code|----|libgccjit.so| +-----------+ +------------+ | | +-------------------------------+ |----|jit-plugin-x86_64-unknown-linux| | +-------------------------------+ | | +---------------------------+ |----|jit-plugin-some-accelerator| | +---------------------------+ | | +---------------------------------+ |----|jit-plugin-some-other-accelerator| | +---------------------------------+ etc I believe this would allow us to have multiple copies of gcc linked into one process at once, each with their own idea of what their single target is, whilst having a single "libgccjit.so" to avoid issues where e.g. one client library want to use libgccjit.so for CPU-targetted compilation, another wants it for GPU work. Client code sends its code to a gcc_jit_context, which builds optimized code for the appropriate CPU/accelerator that it's associated with. One consistent API across CPU and all accelerators. The execution model would need refining a bit; currently a gcc_jit_result merely hands us a void* to be cast to the appropriate type. We'd need to do some extra work to get it to be runnable on the GPU presumably. (Yes I know this is feature creep, I'm thinking about gcc 6.0 or later, fwiw. But maybe it's doable without breaking the current client API). [...] Thoughts? Dave gcc/ChangeLog.jit: * configure: Regenerate. * configure.ac: Add generation of "gcc-harness-name.h". gcc/jit/ChangeLog.jit: * internal-api.c: Include "gcc-harness-name.h". (gcc::jit::playback::context::compile): Run the executable named GCC_HARNESS_NAME, rather than hardcoding "gcc". --- gcc/configure | 6 ++++++ gcc/configure.ac | 6 ++++++ gcc/jit/internal-api.c | 6 ++++-- 3 files changed, 16 insertions(+), 2 deletions(-) diff --git a/gcc/configure b/gcc/configure index c1922ac..8f2b8e7 100755 --- a/gcc/configure +++ b/gcc/configure @@ -28148,6 +28148,12 @@ _ACEOF fi +# Generate gcc-harness-name.h containing GCC_HARNESS_NAME for the benefit +# of jit/internal-api.c. +cat > gcc-harness-name.h < gcc-harness-name.h < @@ -4998,8 +4999,9 @@ compile () const char *argv[6]; int exit_status = 0; int err = 0; + const char *executable = GCC_HARNESS_NAME; - argv[0] = "gcc"; + argv[0] = executable; argv[1] = "-shared"; /* The input: assembler. */ argv[2] = m_path_s_file; @@ -5010,7 +5012,7 @@ compile () argv[5] = NULL; errmsg = pex_one (PEX_SEARCH, /* int flags, */ - "gcc", /* const char *executable */ + executable, const_cast (argv), ctxt_progname, /* const char *pname */ NULL, /* const char *outname */ -- 1.8.5.3