From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 15184 invoked by alias); 15 Jan 2015 19:57:37 -0000 Mailing-List: contact jit-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Post: List-Help: List-Subscribe: Sender: jit-owner@gcc.gnu.org Received: (qmail 15163 invoked by uid 89); 15 Jan 2015 19:57:36 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Checked: by ClamAV 0.98.5 on sourceware.org X-Virus-Found: No X-Spam-SWARE-Status: No, score=3.4 required=5.0 tests=AWL,BAYES_50,BODY_8BITS,GARBLED_BODY,SPF_HELO_PASS,T_RP_MATCHES_RCVD autolearn=no version=3.3.2 X-Spam-Status: No, score=3.4 required=5.0 tests=AWL,BAYES_50,BODY_8BITS,GARBLED_BODY,SPF_HELO_PASS,T_RP_MATCHES_RCVD autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on sourceware.org X-Spam-Level: *** X-Spam-User: qpsmtpd, 2 recipients X-HELO: mx1.redhat.com From: David Malcolm To: jit@gcc.gnu.org, gcc-patches@gcc.gnu.org, Jakub Jelinek , Richard Biener , Joseph Myers Cc: David Malcolm Subject: Stage 3 RFC: using "jit" for ahead-of-time compilation Date: Thu, 01 Jan 2015 00:00:00 -0000 Message-Id: <1421352359-26729-1-git-send-email-dmalcolm@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-SW-Source: 2015-q1/txt/msg00019.txt.bz2 Release managers: given that this only touches the jit, and that the jit is off by default, any objections if I go ahead and commit this? It's a late-breaking feature, but the jit as a whole is new, and I think the following is a big win, so I'd like to proceed with this in stage 3 (i.e. in the next 24 hours). There are docs and testcases. New jit API entrypoint: gcc_jit_context_compile_to_file This patch adds a way to use libgccjit for ahead-of-time compilation. I noticed that given the postprocessing steps the jit has to take to turn the .s file into in-memory code (invoke driver to convert to a .so and then dlopen), that it's not much of a leap to support compiling the .s file into objects, dynamic libraries, and executables. Doing so seems like a big win from a feature standpoint: people with pre-existing frontend code who want a backend can then plug in libgccjit and have a compiler, without needing to write it as a GCC frontend, or use LLVM. "jit" becomes something of a misnomer for this use-case. As an experiment, I used this technique to add a compiler for the language I'll refer to as "brainf" (ahem), and wrote this up for the libgccjit tutorial (it's all in the patch); prebuilt HTML can be seen at: https://dmalcolm.fedorapeople.org/gcc/libgccjit-api-docs-wip/intro/tutorial05.html The main things that are missing are: * specifying libraries to link against (Uli had some ideas about this) * cross-compilation support (needs some deeper work, especially the test suite, so deferrable to gcc 6, I guess) but the feature is useful with the patch as-is. The new test cases take jit.sum's # of expected passes from 7514 to 7571. gcc/jit/ChangeLog: * docs/cp/topics/results.rst: Rename to... * docs/cp/topics/compilation.rst: ...this, and add section on ahead-of-time compilation. * docs/cp/topics/index.rst: Update for renaming of results.rst to compilation.rst. * docs/examples/emit-alphabet.bf: New file, a sample "brainf" script. * docs/examples/tut05-bf.c: New file, implementing a compiler for "brainf". * docs/internals/test-hello-world.exe.log.txt: Update to reflect changes to logger output. * docs/intro/index.rst: Add tutorial05.rst * docs/intro/tutorial05.rst: New file. * docs/topics/results.rst: Rename to... * docs/topics/compilation.rst: ...this, and add section on ahead-of-time compilation. * docs/topics/index.rst: Update for renaming of results.rst to compilation.rst. * jit-playback.c (gcc::jit::playback::context::compile): Convert return type from result * to void. Move the code to convert to dso and dlopen the result to a new pure virtual "postprocess" method. (gcc::jit::playback::compile_to_memory::compile_to_memory): New function. (gcc::jit::playback::compile_to_memory::postprocess): New function, based on playback::context::compile. (gcc::jit::playback::compile_to_file::compile_to_file): New function. (gcc::jit::playback::compile_to_file::postprocess): New function. (gcc::jit::playback::compile_to_file::copy_file): New function. (gcc::jit::playback::context::convert_to_dso): Move internals to... (gcc::jit::playback::context::invoke_driver): New method. Add "-shared" and "-c" options to driver's argv as needed. * jit-playback.h: Include "timevar.h". (gcc::jit::playback::context::compile): Convert return type from result * to void. (gcc::jit::playback::context::postprocess): New pure virtual function, making this an abstract base class. (gcc::jit::playback::context::get_tempdir): New accessor. (gcc::jit::playback::context::invoke_driver): New function. (class gcc::jit::playback::compile_to_memory): New subclass of playback::context. (class gcc::jit::playback::compile_to_file): Likewise. * jit-recording.c (gcc::jit::recording::context::compile): Use a playback::compile_to_memory, and extract its result. (gcc::jit::recording::context::compile_to_file): New function. * jit-recording.h (gcc::jit::recording::context::compile_to_file): New function. * libgccjit++.h (gccjit::context::compile_to_file): New method. * libgccjit.c (gcc_jit_context_compile): Update log message to clarify that this is an in-memory compile. (gcc_jit_context_compile_to_file): New function. * libgccjit.h (gcc_jit_context): Clarify that you can compile a context more than once, and that you can compile to a file as well as to memory. (gcc_jit_result): Clarify that this is the result of an in-memory compilation. (gcc_jit_context_compile): Clarify that you can compile, and that this is an in-memory compilation. (enum gcc_jit_output_kind): New enum. (gcc_jit_context_compile_to_file): New function. (gcc_jit_context_enable_dump): Clarify comment to cover both forms of compilation. * libgccjit.map (gcc_jit_context_compile_to_file): New API entrypoint. * notes.txt: Update to show the playback::context::postprocess virtual function. gcc/testsuite/ChangeLog: * jit.dg/harness.h: Include . (CHECK_NO_ERRORS): New. (verify_code): Wrap prototype in #ifndef TEST_COMPILING_TO_FILE. (test_jit): Support new macro TEST_COMPILING_TO_FILE for exercising gcc_jit_context_compile_to_file. * jit.dg/jit.exp (fixed_host_execute): Fix the code for passing on args to the spawned executable. (jit-expand-vars): New function. (jit-exe-params): New variable. (dg-jit-set-exe-params): New function. (jit-dg-test): Detect testcases that use jit-verify-compile-to-file and call jit-setup-compile-to-file. Set arguments of spawned process to jit-exe-params. (jit-get-output-filename): New function. (jit-setup-compile-to-file): New function. (jit-verify-compile-to-file): New function. (jit-run-executable): New function. (jit-verify-executable): New function. * jit.dg/test-compile-to-assembler.c: New testcase. * jit.dg/test-compile-to-dynamic-library.c: New testcase. * jit.dg/test-compile-to-executable.c: New testcase. * jit.dg/test-compile-to-object.c: New testcase. --- gcc/jit/docs/cp/topics/compilation.rst | 58 +++ gcc/jit/docs/cp/topics/index.rst | 4 +- gcc/jit/docs/cp/topics/results.rst | 48 --- gcc/jit/docs/examples/emit-alphabet.bf | 17 + gcc/jit/docs/examples/tut05-bf.c | 446 +++++++++++++++++++++ .../docs/internals/test-hello-world.exe.log.txt | 48 ++- gcc/jit/docs/intro/index.rst | 3 +- gcc/jit/docs/intro/tutorial05.rst | 253 ++++++++++++ gcc/jit/docs/topics/compilation.rst | 199 +++++++++ gcc/jit/docs/topics/index.rst | 4 +- gcc/jit/docs/topics/results.rst | 127 ------ gcc/jit/jit-playback.c | 308 +++++++++++++- gcc/jit/jit-playback.h | 54 ++- gcc/jit/jit-recording.c | 40 +- gcc/jit/jit-recording.h | 4 + gcc/jit/libgccjit++.h | 12 + gcc/jit/libgccjit.c | 31 +- gcc/jit/libgccjit.h | 58 ++- gcc/jit/libgccjit.map | 1 + gcc/jit/notes.txt | 13 +- gcc/testsuite/jit.dg/harness.h | 29 +- gcc/testsuite/jit.dg/jit.exp | 209 +++++++++- gcc/testsuite/jit.dg/test-compile-to-assembler.c | 65 +++ .../jit.dg/test-compile-to-dynamic-library.c | 65 +++ gcc/testsuite/jit.dg/test-compile-to-executable.c | 110 +++++ gcc/testsuite/jit.dg/test-compile-to-object.c | 65 +++ 26 files changed, 2026 insertions(+), 245 deletions(-) create mode 100644 gcc/jit/docs/cp/topics/compilation.rst delete mode 100644 gcc/jit/docs/cp/topics/results.rst create mode 100644 gcc/jit/docs/examples/emit-alphabet.bf create mode 100644 gcc/jit/docs/examples/tut05-bf.c create mode 100644 gcc/jit/docs/intro/tutorial05.rst create mode 100644 gcc/jit/docs/topics/compilation.rst delete mode 100644 gcc/jit/docs/topics/results.rst create mode 100644 gcc/testsuite/jit.dg/test-compile-to-assembler.c create mode 100644 gcc/testsuite/jit.dg/test-compile-to-dynamic-library.c create mode 100644 gcc/testsuite/jit.dg/test-compile-to-executable.c create mode 100644 gcc/testsuite/jit.dg/test-compile-to-object.c diff --git a/gcc/jit/docs/cp/topics/compilation.rst b/gcc/jit/docs/cp/topics/compilation.rst new file mode 100644 index 0000000..05917e8 --- /dev/null +++ b/gcc/jit/docs/cp/topics/compilation.rst @@ -0,0 +1,58 @@ +.. Copyright (C) 2014-2015 Free Software Foundation, Inc. + Originally contributed by David Malcolm + + This is free software: you can redistribute it and/or modify it + under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, but + WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see + . + +.. default-domain:: cpp + +Compiling a context +=================== + +Once populated, a :class:`gccjit::context` can be compiled to +machine code, either in-memory via :func:`gccjit::context::compile` or +to disk via :func:`gccjit::context::compile_to_file`. + +You can compile a context multiple times (using either form of +compilation), although any errors that occur on the context will +prevent any future compilation of that context. + +In-memory compilation +********************* + +.. function:: gcc_jit_result *\ + gccjit::context::compile () + + This calls into GCC and builds the code, returning a + `gcc_jit_result *`. + + This is a thin wrapper around the + :c:func:`gcc_jit_context_compile` API entrypoint. + +Ahead-of-time compilation +************************* + +Although libgccjit is primarily aimed at just-in-time compilation, it +can also be used for implementing more traditional ahead-of-time +compilers, via the :func:`gccjit::context::compile_to_file` method. + +.. function:: void \ + gccjit::context::compile_to_file (enum gcc_jit_output_kind,\ + const char *output_path) + + Compile the :class:`gccjit::context` to a file of the given + kind. + + This is a thin wrapper around the + :c:func:`gcc_jit_context_compile_to_file` API entrypoint. diff --git a/gcc/jit/docs/cp/topics/index.rst b/gcc/jit/docs/cp/topics/index.rst index a129137..4ebb623 100644 --- a/gcc/jit/docs/cp/topics/index.rst +++ b/gcc/jit/docs/cp/topics/index.rst @@ -1,4 +1,4 @@ -.. Copyright (C) 2014 Free Software Foundation, Inc. +.. Copyright (C) 2014-2015 Free Software Foundation, Inc. Originally contributed by David Malcolm This is free software: you can redistribute it and/or modify it @@ -27,4 +27,4 @@ Topic Reference expressions.rst functions.rst locations.rst - results.rst + compilation.rst diff --git a/gcc/jit/docs/cp/topics/results.rst b/gcc/jit/docs/cp/topics/results.rst deleted file mode 100644 index 18200ac..0000000 --- a/gcc/jit/docs/cp/topics/results.rst +++ /dev/null @@ -1,48 +0,0 @@ -.. Copyright (C) 2014 Free Software Foundation, Inc. - Originally contributed by David Malcolm - - This is free software: you can redistribute it and/or modify it - under the terms of the GNU General Public License as published by - the Free Software Foundation, either version 3 of the License, or - (at your option) any later version. - - This program is distributed in the hope that it will be useful, but - WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program. If not, see - . - -.. default-domain:: cpp - -Compilation results -=================== - -.. type:: gcc_jit_result - - A `gcc_jit_result` encapsulates the result of compiling a context. - -.. function:: gcc_jit_result *\ - gccjit::context::compile () - - This calls into GCC and builds the code, returning a - `gcc_jit_result *`. - - -.. function:: void *\ - gcc_jit_result_get_code (gcc_jit_result *result,\ - const char *funcname) - - Locate a given function within the built machine code. - This will need to be cast to a function pointer of the - correct type before it can be called. - - -.. function:: void\ - gcc_jit_result_release (gcc_jit_result *result) - - Once we're done with the code, this unloads the built .so file. - This cleans up the result; after calling this, it's no longer - valid to use the result. diff --git a/gcc/jit/docs/examples/emit-alphabet.bf b/gcc/jit/docs/examples/emit-alphabet.bf new file mode 100644 index 0000000..6863273 --- /dev/null +++ b/gcc/jit/docs/examples/emit-alphabet.bf @@ -0,0 +1,17 @@ +[ + Emit the uppercase alphabet +] + +cell 0 = 26 +++++++++++++++++++++++++++ + +cell 1 = 65 +>+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++< + +while cell#0 != 0 +[ + > + . emit cell#1 + + increment cell@1 + <- decrement cell@0 +] diff --git a/gcc/jit/docs/examples/tut05-bf.c b/gcc/jit/docs/examples/tut05-bf.c new file mode 100644 index 0000000..f948ede --- /dev/null +++ b/gcc/jit/docs/examples/tut05-bf.c @@ -0,0 +1,446 @@ +/* A compiler for the "bf" language. */ + +#include +#include +#include + +#include "libgccjit.h" + +/* Make "main" function: + int + main (int argc, char **argv) + { + ... + } +*/ +static gcc_jit_function * +make_main (gcc_jit_context *ctxt) +{ + gcc_jit_type *int_type = + gcc_jit_context_get_type (ctxt, GCC_JIT_TYPE_INT); + gcc_jit_param *param_argc = + gcc_jit_context_new_param (ctxt, NULL, int_type, "argc"); + gcc_jit_type *char_ptr_ptr_type = + gcc_jit_type_get_pointer ( + gcc_jit_type_get_pointer ( + gcc_jit_context_get_type (ctxt, GCC_JIT_TYPE_CHAR))); + gcc_jit_param *param_argv = + gcc_jit_context_new_param (ctxt, NULL, char_ptr_ptr_type, "argv"); + gcc_jit_param *params[2] = {param_argc, param_argv}; + gcc_jit_function *func_main = + gcc_jit_context_new_function (ctxt, NULL, + GCC_JIT_FUNCTION_EXPORTED, + int_type, + "main", + 2, params, + 0); + return func_main; +} + +#define MAX_OPEN_PARENS 16 + +typedef struct bf_compiler +{ + const char *filename; + int line; + int column; + + gcc_jit_context *ctxt; + + gcc_jit_type *void_type; + gcc_jit_type *int_type; + gcc_jit_type *byte_type; + gcc_jit_type *array_type; + + gcc_jit_function *func_getchar; + gcc_jit_function *func_putchar; + + gcc_jit_function *func; + gcc_jit_block *curblock; + + gcc_jit_rvalue *int_zero; + gcc_jit_rvalue *int_one; + gcc_jit_rvalue *byte_zero; + gcc_jit_rvalue *byte_one; + gcc_jit_lvalue *data_cells; + gcc_jit_lvalue *idx; + + int num_open_parens; + gcc_jit_block *paren_test[MAX_OPEN_PARENS]; + gcc_jit_block *paren_body[MAX_OPEN_PARENS]; + gcc_jit_block *paren_after[MAX_OPEN_PARENS]; + +} bf_compiler; + +/* Bail out, with a message on stderr. */ + +static void +fatal_error (bf_compiler *bfc, const char *msg) +{ + fprintf (stderr, + "%s:%i:%i: %s", + bfc->filename, bfc->line, bfc->column, msg); + abort (); +} + +/* Get "data_cells[idx]" as an lvalue. */ + +static gcc_jit_lvalue * +bf_get_current_data (bf_compiler *bfc, gcc_jit_location *loc) +{ + return gcc_jit_context_new_array_access ( + bfc->ctxt, + loc, + gcc_jit_lvalue_as_rvalue (bfc->data_cells), + gcc_jit_lvalue_as_rvalue (bfc->idx)); +} + +/* Get "data_cells[idx] == 0" as a boolean rvalue. */ + +static gcc_jit_rvalue * +bf_current_data_is_zero (bf_compiler *bfc, gcc_jit_location *loc) +{ + return gcc_jit_context_new_comparison ( + bfc->ctxt, + loc, + GCC_JIT_COMPARISON_EQ, + gcc_jit_lvalue_as_rvalue (bf_get_current_data (bfc, loc)), + bfc->byte_zero); +} + +/* Compile one bf character. */ + +static void +bf_compile_char (bf_compiler *bfc, + unsigned char ch) +{ + gcc_jit_location *loc = + gcc_jit_context_new_location (bfc->ctxt, + bfc->filename, + bfc->line, + bfc->column); + + /* Turn this on to trace execution, by injecting putchar () + of each source char. */ + if (0) + { + gcc_jit_rvalue *arg = + gcc_jit_context_new_rvalue_from_int ( + bfc->ctxt, + bfc->int_type, + ch); + gcc_jit_rvalue *call = + gcc_jit_context_new_call (bfc->ctxt, + loc, + bfc->func_putchar, + 1, &arg); + gcc_jit_block_add_eval (bfc->curblock, + loc, + call); + } + + switch (ch) + { + case '>': + gcc_jit_block_add_comment (bfc->curblock, + loc, + "'>': idx += 1;"); + gcc_jit_block_add_assignment_op (bfc->curblock, + loc, + bfc->idx, + GCC_JIT_BINARY_OP_PLUS, + bfc->int_one); + break; + + case '<': + gcc_jit_block_add_comment (bfc->curblock, + loc, + "'<': idx -= 1;"); + gcc_jit_block_add_assignment_op (bfc->curblock, + loc, + bfc->idx, + GCC_JIT_BINARY_OP_MINUS, + bfc->int_one); + break; + + case '+': + gcc_jit_block_add_comment (bfc->curblock, + loc, + "'+': data[idx] += 1;"); + gcc_jit_block_add_assignment_op (bfc->curblock, + loc, + bf_get_current_data (bfc, loc), + GCC_JIT_BINARY_OP_PLUS, + bfc->byte_one); + break; + + case '-': + gcc_jit_block_add_comment (bfc->curblock, + loc, + "'-': data[idx] -= 1;"); + gcc_jit_block_add_assignment_op (bfc->curblock, + loc, + bf_get_current_data (bfc, loc), + GCC_JIT_BINARY_OP_MINUS, + bfc->byte_one); + break; + + case '.': + { + gcc_jit_rvalue *arg = + gcc_jit_context_new_cast ( + bfc->ctxt, + loc, + gcc_jit_lvalue_as_rvalue (bf_get_current_data (bfc, loc)), + bfc->int_type); + gcc_jit_rvalue *call = + gcc_jit_context_new_call (bfc->ctxt, + loc, + bfc->func_putchar, + 1, &arg); + gcc_jit_block_add_comment (bfc->curblock, + loc, + "'.': putchar ((int)data[idx]);"); + gcc_jit_block_add_eval (bfc->curblock, + loc, + call); + } + break; + + case ',': + { + gcc_jit_rvalue *call = + gcc_jit_context_new_call (bfc->ctxt, + loc, + bfc->func_getchar, + 0, NULL); + gcc_jit_block_add_comment ( + bfc->curblock, + loc, + "',': data[idx] = (unsigned char)getchar ();"); + gcc_jit_block_add_assignment (bfc->curblock, + loc, + bf_get_current_data (bfc, loc), + gcc_jit_context_new_cast ( + bfc->ctxt, + loc, + call, + bfc->byte_type)); + } + break; + + case '[': + { + gcc_jit_block *loop_test = + gcc_jit_function_new_block (bfc->func, NULL); + gcc_jit_block *on_zero = + gcc_jit_function_new_block (bfc->func, NULL); + gcc_jit_block *on_non_zero = + gcc_jit_function_new_block (bfc->func, NULL); + + if (bfc->num_open_parens == MAX_OPEN_PARENS) + fatal_error (bfc, "too many open parens"); + + gcc_jit_block_end_with_jump ( + bfc->curblock, + loc, + loop_test); + + gcc_jit_block_add_comment ( + loop_test, + loc, + "'['"); + gcc_jit_block_end_with_conditional ( + loop_test, + loc, + bf_current_data_is_zero (bfc, loc), + on_zero, + on_non_zero); + bfc->paren_test[bfc->num_open_parens] = loop_test; + bfc->paren_body[bfc->num_open_parens] = on_non_zero; + bfc->paren_after[bfc->num_open_parens] = on_zero; + bfc->num_open_parens += 1; + bfc->curblock = on_non_zero; + } + break; + + case ']': + { + gcc_jit_block_add_comment ( + bfc->curblock, + loc, + "']'"); + + if (bfc->num_open_parens == 0) + fatal_error (bfc, "mismatching parens"); + bfc->num_open_parens -= 1; + gcc_jit_block_end_with_jump ( + bfc->curblock, + loc, + bfc->paren_test[bfc->num_open_parens]); + bfc->curblock = bfc->paren_after[bfc->num_open_parens]; + } + break; + + case '\n': + bfc->line +=1; + bfc->column = 0; + break; + } + + if (ch != '\n') + bfc->column += 1; +} + +/* Compile the given .bf file into a gcc_jit_context, containing a + single "main" function suitable for compiling into an executable. */ + +gcc_jit_context * +bf_compile (const char *filename) +{ + bf_compiler bfc; + FILE *f_in; + int ch; + + memset (&bfc, 0, sizeof (bfc)); + + bfc.filename = filename; + f_in = fopen (filename, "r"); + if (!f_in) + fatal_error (&bfc, "unable to open file"); + bfc.line = 1; + + bfc.ctxt = gcc_jit_context_acquire (); + + gcc_jit_context_set_int_option ( + bfc.ctxt, + GCC_JIT_INT_OPTION_OPTIMIZATION_LEVEL, + 3); + gcc_jit_context_set_bool_option ( + bfc.ctxt, + GCC_JIT_BOOL_OPTION_DUMP_INITIAL_GIMPLE, + 0); + gcc_jit_context_set_bool_option ( + bfc.ctxt, + GCC_JIT_BOOL_OPTION_DEBUGINFO, + 1); + gcc_jit_context_set_bool_option ( + bfc.ctxt, + GCC_JIT_BOOL_OPTION_DUMP_EVERYTHING, + 0); + gcc_jit_context_set_bool_option ( + bfc.ctxt, + GCC_JIT_BOOL_OPTION_KEEP_INTERMEDIATES, + 0); + + bfc.void_type = + gcc_jit_context_get_type (bfc.ctxt, GCC_JIT_TYPE_VOID); + bfc.int_type = + gcc_jit_context_get_type (bfc.ctxt, GCC_JIT_TYPE_INT); + bfc.byte_type = + gcc_jit_context_get_type (bfc.ctxt, GCC_JIT_TYPE_UNSIGNED_CHAR); + bfc.array_type = + gcc_jit_context_new_array_type (bfc.ctxt, + NULL, + bfc.byte_type, + 30000); + + bfc.func_getchar = + gcc_jit_context_new_function (bfc.ctxt, NULL, + GCC_JIT_FUNCTION_IMPORTED, + bfc.int_type, + "getchar", + 0, NULL, + 0); + + gcc_jit_param *param_c = + gcc_jit_context_new_param (bfc.ctxt, NULL, bfc.int_type, "c"); + bfc.func_putchar = + gcc_jit_context_new_function (bfc.ctxt, NULL, + GCC_JIT_FUNCTION_IMPORTED, + bfc.void_type, + "putchar", + 1, ¶m_c, + 0); + + bfc.func = make_main (bfc.ctxt); + bfc.curblock = + gcc_jit_function_new_block (bfc.func, "initial"); + bfc.int_zero = gcc_jit_context_zero (bfc.ctxt, bfc.int_type); + bfc.int_one = gcc_jit_context_one (bfc.ctxt, bfc.int_type); + bfc.byte_zero = gcc_jit_context_zero (bfc.ctxt, bfc.byte_type); + bfc.byte_one = gcc_jit_context_one (bfc.ctxt, bfc.byte_type); + + bfc.data_cells = + gcc_jit_context_new_global (bfc.ctxt, NULL, + GCC_JIT_GLOBAL_INTERNAL, + bfc.array_type, + "data_cells"); + bfc.idx = + gcc_jit_function_new_local (bfc.func, NULL, + bfc.int_type, + "idx"); + + gcc_jit_block_add_comment (bfc.curblock, + NULL, + "idx = 0;"); + gcc_jit_block_add_assignment (bfc.curblock, + NULL, + bfc.idx, + bfc.int_zero); + + bfc.num_open_parens = 0; + + while ( EOF != (ch = fgetc (f_in))) + bf_compile_char (&bfc, (unsigned char)ch); + + gcc_jit_block_end_with_return (bfc.curblock, NULL, bfc.int_zero); + + fclose (f_in); + + return bfc.ctxt; +} + +/* Entrypoint to the compiler. */ + +int +main (int argc, char **argv) +{ + const char *input_file; + const char *output_file; + gcc_jit_context *ctxt; + const char *err; + + if (argc != 3) + { + fprintf (stderr, "%s: INPUT_FILE OUTPUT_FILE\n", argv[0]); + return 1; + } + + input_file = argv[1]; + output_file = argv[2]; + ctxt = bf_compile (input_file); + + gcc_jit_context_compile_to_file (ctxt, + GCC_JIT_OUTPUT_KIND_EXECUTABLE, + output_file); + + err = gcc_jit_context_get_first_error (ctxt); + + if (err) + { + gcc_jit_context_release (ctxt); + return 1; + } + + gcc_jit_context_release (ctxt); + return 0; +} + +/* Use the built compiler to compile the example to an executable: + + { dg-jit-set-exe-params SRCDIR/gcc/jit/docs/examples/emit-alphabet.bf emit-alphabet.bf.exe } + + Then run the executable, and verify that it emits the alphabet: + + { dg-final { jit-run-executable emit-alphabet.bf.exe "ABCDEFGHIJKLMNOPQRSTUVWXYZ" } } */ diff --git a/gcc/jit/docs/internals/test-hello-world.exe.log.txt b/gcc/jit/docs/internals/test-hello-world.exe.log.txt index 113dc35..205b6b4 100644 --- a/gcc/jit/docs/internals/test-hello-world.exe.log.txt +++ b/gcc/jit/docs/internals/test-hello-world.exe.log.txt @@ -38,14 +38,20 @@ JIT: entering: gcc_jit_block_add_eval JIT: exiting: gcc_jit_block_add_eval JIT: entering: gcc_jit_block_end_with_void_return JIT: exiting: gcc_jit_block_end_with_void_return +JIT: entering: gcc_jit_context_dump_reproducer_to_file +JIT: entering: void gcc::jit::recording::context::dump_reproducer_to_file(const char*) +JIT: exiting: void gcc::jit::recording::context::dump_reproducer_to_file(const char*) +JIT: exiting: gcc_jit_context_dump_reproducer_to_file JIT: entering: gcc_jit_context_compile -JIT: compiling ctxt: 0x1283e20 +JIT: in-memory compile of ctxt: 0x1283e20 JIT: entering: gcc::jit::result* gcc::jit::recording::context::compile() JIT: entering: void gcc::jit::recording::context::validate() JIT: exiting: void gcc::jit::recording::context::validate() JIT: entering: gcc::jit::playback::context::context(gcc::jit::recording::context*) JIT: exiting: gcc::jit::playback::context::context(gcc::jit::recording::context*) -JIT: entering: gcc::jit::result* gcc::jit::playback::context::compile() +JIT: entering: gcc::jit::playback::compile_to_memory::compile_to_memory(gcc::jit::recording::context*) +JIT: exiting: gcc::jit::playback::compile_to_memory::compile_to_memory(gcc::jit::recording::context*) +JIT: entering: void gcc::jit::playback::context::compile() JIT: entering: gcc::jit::tempdir::tempdir(gcc::jit::logger*, int) JIT: exiting: gcc::jit::tempdir::tempdir(gcc::jit::logger*, int) JIT: entering: bool gcc::jit::tempdir::create() @@ -86,29 +92,37 @@ JIT: entering: void gcc::jit::playback::function::postprocess() JIT: exiting: void gcc::jit::playback::function::postprocess() JIT: exiting: void gcc::jit::playback::context::replay() JIT: entering: void jit_langhook_write_globals() +JIT: entering: void gcc::jit::playback::context::write_global_decls_1() +JIT: exiting: void gcc::jit::playback::context::write_global_decls_1() +JIT: entering: void gcc::jit::playback::context::write_global_decls_2() +JIT: exiting: void gcc::jit::playback::context::write_global_decls_2() JIT: exiting: void jit_langhook_write_globals() JIT: exiting: toplev::main JIT: entering: void gcc::jit::playback::context::extract_any_requested_dumps(vec*) JIT: exiting: void gcc::jit::playback::context::extract_any_requested_dumps(vec*) JIT: entering: toplev::finalize JIT: exiting: toplev::finalize -JIT: entering: void gcc::jit::playback::context::convert_to_dso(const char*) -JIT: argv[0]: x86_64-unknown-linux-gnu-gcc-5.0.0 -JIT: argv[1]: -shared -JIT: argv[2]: /tmp/libgccjit-CKq1M9/fake.s -JIT: argv[3]: -o -JIT: argv[4]: /tmp/libgccjit-CKq1M9/fake.so -JIT: argv[5]: -fno-use-linker-plugin -JIT: argv[6]: (null) -JIT: exiting: void gcc::jit::playback::context::convert_to_dso(const char*) -JIT: entering: gcc::jit::result* gcc::jit::playback::context::dlopen_built_dso() -JIT: GCC_JIT_BOOL_OPTION_DEBUGINFO was set: handing over tempdir to jit::result -JIT: entering: gcc::jit::result::result(gcc::jit::logger*, void*, gcc::jit::tempdir*) -JIT: exiting: gcc::jit::result::result(gcc::jit::logger*, void*, gcc::jit::tempdir*) -JIT: exiting: gcc::jit::result* gcc::jit::playback::context::dlopen_built_dso() +JIT: entering: virtual void gcc::jit::playback::compile_to_memory::postprocess(const char*) +JIT: entering: void gcc::jit::playback::context::convert_to_dso(const char*) +JIT: entering: void gcc::jit::playback::context::invoke_driver(const char*, const char*, const char*, timevar_id_t, bool, bool) +JIT: argv[0]: x86_64-unknown-linux-gnu-gcc-5.0.0 +JIT: argv[1]: -shared +JIT: argv[2]: /tmp/libgccjit-CKq1M9/fake.s +JIT: argv[3]: -o +JIT: argv[4]: /tmp/libgccjit-CKq1M9/fake.so +JIT: argv[5]: -fno-use-linker-plugin +JIT: argv[6]: (null) +JIT: exiting: void gcc::jit::playback::context::invoke_driver(const char*, const char*, const char*, timevar_id_t, bool, bool) +JIT: exiting: void gcc::jit::playback::context::convert_to_dso(const char*) +JIT: entering: gcc::jit::result* gcc::jit::playback::context::dlopen_built_dso() +JIT: GCC_JIT_BOOL_OPTION_DEBUGINFO was set: handing over tempdir to jit::result +JIT: entering: gcc::jit::result::result(gcc::jit::logger*, void*, gcc::jit::tempdir*) +JIT: exiting: gcc::jit::result::result(gcc::jit::logger*, void*, gcc::jit::tempdir*) +JIT: exiting: gcc::jit::result* gcc::jit::playback::context::dlopen_built_dso() +JIT: exiting: virtual void gcc::jit::playback::compile_to_memory::postprocess(const char*) JIT: entering: void gcc::jit::playback::context::release_mutex() JIT: exiting: void gcc::jit::playback::context::release_mutex() -JIT: exiting: gcc::jit::result* gcc::jit::playback::context::compile() +JIT: exiting: void gcc::jit::playback::context::compile() JIT: entering: gcc::jit::playback::context::~context() JIT: exiting: gcc::jit::playback::context::~context() JIT: exiting: gcc::jit::result* gcc::jit::recording::context::compile() diff --git a/gcc/jit/docs/intro/index.rst b/gcc/jit/docs/intro/index.rst index d3bcec9..0f51777 100644 --- a/gcc/jit/docs/intro/index.rst +++ b/gcc/jit/docs/intro/index.rst @@ -1,4 +1,4 @@ -.. Copyright (C) 2014 Free Software Foundation, Inc. +.. Copyright (C) 2014-2015 Free Software Foundation, Inc. Originally contributed by David Malcolm This is free software: you can redistribute it and/or modify it @@ -25,3 +25,4 @@ Tutorial tutorial02.rst tutorial03.rst tutorial04.rst + tutorial05.rst diff --git a/gcc/jit/docs/intro/tutorial05.rst b/gcc/jit/docs/intro/tutorial05.rst new file mode 100644 index 0000000..865a550 --- /dev/null +++ b/gcc/jit/docs/intro/tutorial05.rst @@ -0,0 +1,253 @@ +.. Copyright (C) 2015 Free Software Foundation, Inc. + Originally contributed by David Malcolm + + This is free software: you can redistribute it and/or modify it + under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, but + WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see + . + +Tutorial part 5: Implementing an Ahead-of-Time compiler +------------------------------------------------------- + +If you have a pre-existing language frontend, it's possible to hook +it up to libgccjit as a backend. In the previous example we showed +how to do that for in-memory JIT-compilation, but libgccjit can also +compile code directly to a file, allowing you to implement a more +traditional ahead-of-time compiler ("JIT" is something of a misnomer +for this use-case). + +The essential difference is to compile the context using +:c:func:`gcc_jit_context_compile_to_file` rather than +:c:func:`gcc_jit_context_compile`. + +The "brainf" language +********************* + +In this example we use libgccjit to construct an ahead-of-time compiler +for an esoteric programming language that we shall refer to as "brainf". + +brainf scripts operate on an array of bytes, with a notional data pointer +within the array. + +brainf is hard for humans to read, but it's trivial to write a parser for +it, as there is no lexing; just a stream of bytes. The operations are: + +====================== ============================= +Character Meaning +====================== ============================= +``>`` ``idx += 1`` +``<`` ``idx -= 1`` +``+`` ``data[idx] += 1`` +``-`` ``data[idx] -= 1`` +``.`` ``output (data[idx])`` +``,`` ``data[idx] = input ()`` +``[`` loop until ``data[idx] == 0`` +``]`` end of loop +Anything else ignored +====================== ============================= + +Unlike the previous example, we'll implement an ahead-of-time compiler, +which reads ``.bf`` scripts and outputs executables (though it would +be trivial to have it run them JIT-compiled in-process). + +Here's what a simple ``.bf`` script looks like: + + .. literalinclude:: ../examples/emit-alphabet.bf + :lines: 1- + +.. note:: + + This example makes use of whitespace and comments for legibility, but + could have been written as:: + + ++++++++++++++++++++++++++ + >+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++< + [>.+<-] + + It's not a particularly useful language, except for providing + compiler-writers with a test case that's easy to parse. The point + is that you can use :c:func:`gcc_jit_context_compile_to_file` + to use libgccjit as a backend for a pre-existing language frontend. + +Converting a brainf script to libgccjit IR +****************************************** + +As before we write simple code to populate a :c:type:`gcc_jit_context *`. + + .. literalinclude:: ../examples/tut05-bf.c + :start-after: #define MAX_OPEN_PARENS 16 + :end-before: /* Entrypoint to the compiler. */ + :language: c + +Compiling a context to a file +***************************** + +Unlike the previous tutorial, this time we'll compile the context +directly to an executable, using :c:func:`gcc_jit_context_compile_to_file`: + +.. code-block:: c + + gcc_jit_context_compile_to_file (ctxt, + GCC_JIT_OUTPUT_KIND_EXECUTABLE, + output_file); + +Here's the top-level of the compiler, which is what actually calls into +:c:func:`gcc_jit_context_compile_to_file`: + + .. literalinclude:: ../examples/tut05-bf.c + :start-after: /* Entrypoint to the compiler. */ + :end-before: /* Use the built compiler to compile the example to an executable: + :language: c + +Note how once the context is populated you could trivially instead compile +it to memory using :c:func:`gcc_jit_context_compile` and run it in-process +as in the previous tutorial. + +To create an executable, we need to export a ``main`` function. Here's +how to create one from the JIT API: + + .. literalinclude:: ../examples/tut05-bf.c + :start-after: #include "libgccjit.h" + :end-before: #define MAX_OPEN_PARENS 16 + :language: c + +.. note:: + + The above implementation ignores ``argc`` and ``argv``, but you could + make use of them by exposing ``param_argc`` and ``param_argv`` to the + caller. + +Upon compiling this C code, we obtain a bf-to-machine-code compiler; +let's call it ``bfc``: + +.. code-block:: console + + $ gcc \ + tut05-bf.c \ + -o bfc \ + -lgccjit + +We can now use ``bfc`` to compile .bf files into machine code executables: + +.. code-block:: console + + $ ./bfc \ + emit-alphabet.bf \ + a.out + +which we can run directly: + +.. code-block:: console + + $ ./a.out + ABCDEFGHIJKLMNOPQRSTUVWXYZ + +Success! + +We can also inspect the generated executable using standard tools: + +.. code-block:: console + + $ objdump -d a.out |less + +which shows that libgccjit has managed to optimize the function +somewhat (for example, the runs of 26 and 65 increment operations +have become integer constants 0x1a and 0x41): + +.. code-block:: console + + 0000000000400620
: + 400620: 80 3d 39 0a 20 00 00 cmpb $0x0,0x200a39(%rip) # 601060 + 40062b: 0f 1f 44 00 00 nopl 0x0(%rax,%rax,1) + 400630: 48 83 ec 08 sub $0x8,%rsp + 400634: 0f b6 05 26 0a 20 00 movzbl 0x200a26(%rip),%eax # 601061 + 40063b: c6 05 1e 0a 20 00 1a movb $0x1a,0x200a1e(%rip) # 601060 + 400642: 8d 78 41 lea 0x41(%rax),%edi + 400645: 40 88 3d 15 0a 20 00 mov %dil,0x200a15(%rip) # 601061 + 40064c: 0f 1f 40 00 nopl 0x0(%rax) + 400650: 40 0f b6 ff movzbl %dil,%edi + 400654: e8 87 fe ff ff callq 4004e0 + 400659: 0f b6 05 01 0a 20 00 movzbl 0x200a01(%rip),%eax # 601061 + 400660: 80 2d f9 09 20 00 01 subb $0x1,0x2009f9(%rip) # 601060 + 400667: 8d 78 01 lea 0x1(%rax),%edi + 40066a: 40 88 3d f0 09 20 00 mov %dil,0x2009f0(%rip) # 601061 + 400671: 75 dd jne 400650 + 400673: 31 c0 xor %eax,%eax + 400675: 48 83 c4 08 add $0x8,%rsp + 400679: c3 retq + 40067a: 66 0f 1f 44 00 00 nopw 0x0(%rax,%rax,1) + +We also set up debugging information (via +:c:func:`gcc_jit_context_new_location` and +:c:macro:`GCC_JIT_BOOL_OPTION_DEBUGINFO`), so it's possible to use ``gdb`` +to singlestep through the generated binary and inspect the internal +state ``idx`` and ``data_cells``: + +.. code-block:: console + + (gdb) break main + Breakpoint 1 at 0x400790 + (gdb) run + Starting program: a.out + + Breakpoint 1, 0x0000000000400790 in main (argc=1, argv=0x7fffffffe448) + (gdb) stepi + 0x0000000000400797 in main (argc=1, argv=0x7fffffffe448) + (gdb) stepi + 0x00000000004007a0 in main (argc=1, argv=0x7fffffffe448) + (gdb) stepi + 9 >+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++< + (gdb) list + 4 + 5 cell 0 = 26 + 6 ++++++++++++++++++++++++++ + 7 + 8 cell 1 = 65 + 9 >+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++< + 10 + 11 while cell#0 != 0 + 12 [ + 13 > + (gdb) n + 6 ++++++++++++++++++++++++++ + (gdb) n + 9 >+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++< + (gdb) p idx + $1 = 1 + (gdb) p data_cells + $2 = "\032", '\000' + (gdb) p data_cells[0] + $3 = 26 '\032' + (gdb) p data_cells[1] + $4 = 0 '\000' + (gdb) list + 4 + 5 cell 0 = 26 + 6 ++++++++++++++++++++++++++ + 7 + 8 cell 1 = 65 + 9 >+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++< + 10 + 11 while cell#0 != 0 + 12 [ + 13 > + + +Other forms of ahead-of-time-compilation +**************************************** + +The above demonstrates compiling a :c:type:`gcc_jit_context *` directly +to an executable. It's also possible to compile it to an object file, +and to a dynamic library. See the documentation of +:c:func:`gcc_jit_context_compile_to_file` for more information. diff --git a/gcc/jit/docs/topics/compilation.rst b/gcc/jit/docs/topics/compilation.rst new file mode 100644 index 0000000..bf59cc2 --- /dev/null +++ b/gcc/jit/docs/topics/compilation.rst @@ -0,0 +1,199 @@ +.. Copyright (C) 2014-2015 Free Software Foundation, Inc. + Originally contributed by David Malcolm + + This is free software: you can redistribute it and/or modify it + under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, but + WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see + . + +.. default-domain:: c + +Compiling a context +=================== + +Once populated, a :c:type:`gcc_jit_context *` can be compiled to +machine code, either in-memory via :c:func:`gcc_jit_context_compile` or +to disk via :c:func:`gcc_jit_context_compile_to_file`. + +You can compile a context multiple times (using either form of +compilation), although any errors that occur on the context will +prevent any future compilation of that context. + +In-memory compilation +********************* + +.. function:: gcc_jit_result *\ + gcc_jit_context_compile (gcc_jit_context *ctxt) + + This calls into GCC and builds the code, returning a + `gcc_jit_result *`. + + If this is non-NULL, the caller becomes responsible for + calling :func:`gcc_jit_result_release` on it once they're done + with it. + +.. type:: gcc_jit_result + + A `gcc_jit_result` encapsulates the result of compiling a context + in-memory, and the lifetimes of any machine code functions or globals + that are within the resuilt. + +.. function:: void *\ + gcc_jit_result_get_code (gcc_jit_result *result,\ + const char *funcname) + + Locate a given function within the built machine code. + + Functions are looked up by name. For this to succeed, a function + with a name matching `funcname` must have been created on + `result`'s context (or a parent context) via a call to + :func:`gcc_jit_context_new_function` with `kind` + :macro:`GCC_JIT_FUNCTION_EXPORTED`: + + .. code-block:: c + + gcc_jit_context_new_function (ctxt, + any_location, /* or NULL */ + /* Required for func to be visible to + gcc_jit_result_get_code: */ + GCC_JIT_FUNCTION_EXPORTED, + any_return_type, + /* Must string-compare equal: */ + funcname, + /* etc */); + + If such a function is not found (or `result` or `funcname` are + ``NULL``), an error message will be emitted on stderr and + ``NULL`` will be returned. + + If the function is found, the result will need to be cast to a + function pointer of the correct type before it can be called. + + Note that the resulting machine code becomes invalid after + :func:`gcc_jit_result_release` is called on the + :type:`gcc_jit_result *`; attempting to call it after that may lead + to a segmentation fault. + +.. function:: void *\ + gcc_jit_result_get_global (gcc_jit_result *result,\ + const char *name) + + Locate a given global within the built machine code. + + Globals are looked up by name. For this to succeed, a global + with a name matching `name` must have been created on + `result`'s context (or a parent context) via a call to + :func:`gcc_jit_context_new_global` with `kind` + :macro:`GCC_JIT_GLOBAL_EXPORTED`. + + If the global is found, the result will need to be cast to a + pointer of the correct type before it can be called. + + This is a *pointer* to the global, so e.g. for an :c:type:`int` this is + an :c:type:`int *`. + + For example, given an ``int foo;`` created this way: + + .. code-block:: c + + gcc_jit_lvalue *exported_global = + gcc_jit_context_new_global (ctxt, + any_location, /* or NULL */ + GCC_JIT_GLOBAL_EXPORTED, + int_type, + "foo"); + + we can access it like this: + + .. code-block:: c + + int *ptr_to_foo = + (int *)gcc_jit_result_get_global (result, "foo"); + + If such a global is not found (or `result` or `name` are + ``NULL``), an error message will be emitted on stderr and + ``NULL`` will be returned. + + Note that the resulting address becomes invalid after + :func:`gcc_jit_result_release` is called on the + :type:`gcc_jit_result *`; attempting to use it after that may lead + to a segmentation fault. + +.. function:: void\ + gcc_jit_result_release (gcc_jit_result *result) + + Once we're done with the code, this unloads the built .so file. + This cleans up the result; after calling this, it's no longer + valid to use the result, or any code or globals that were obtained + by calling :func:`gcc_jit_result_get_code` or + :func:`gcc_jit_result_get_global` on it. + + +Ahead-of-time compilation +************************* + +Although libgccjit is primarily aimed at just-in-time compilation, it +can also be used for implementing more traditional ahead-of-time +compilers, via the :c:func:`gcc_jit_context_compile_to_file` +API entrypoint. + +.. function:: void \ + gcc_jit_context_compile_to_file (gcc_jit_context *ctxt, \ + enum gcc_jit_output_kind output_kind,\ + const char *output_path) + + Compile the :c:type:`gcc_jit_context *` to a file of the given + kind. + +:c:func:`gcc_jit_context_compile_to_file` ignores the suffix of +``output_path``, and insteads uses the given +:c:type:`enum gcc_jit_output_kind` to decide what to do. + +.. note:: + + This is different from the ``gcc`` program, which does make use of the + suffix of the output file when determining what to do. + +.. type:: enum gcc_jit_output_kind + +The available kinds of output are: + +============================================== ============== +Output kind Typical suffix +============================================== ============== +:c:macro:`GCC_JIT_OUTPUT_KIND_ASSEMBLER` .s +:c:macro:`GCC_JIT_OUTPUT_KIND_OBJECT_FILE` .o +:c:macro:`GCC_JIT_OUTPUT_KIND_DYNAMIC_LIBRARY` .so or .dll +:c:macro:`GCC_JIT_OUTPUT_KIND_EXECUTABLE` None, or .exe +============================================== ============== + +.. c:macro:: GCC_JIT_OUTPUT_KIND_ASSEMBLER + + Compile the context to an assembler file. + +.. c:macro:: GCC_JIT_OUTPUT_KIND_OBJECT_FILE + + Compile the context to an object file. + +.. c:macro:: GCC_JIT_OUTPUT_KIND_DYNAMIC_LIBRARY + + Compile the context to a dynamic library. + + There is currently no support for specifying other libraries to link + against. + +.. c:macro:: GCC_JIT_OUTPUT_KIND_EXECUTABLE + + Compile the context to an executable. + + There is currently no support for specifying libraries to link + against. diff --git a/gcc/jit/docs/topics/index.rst b/gcc/jit/docs/topics/index.rst index a129137..4ebb623 100644 --- a/gcc/jit/docs/topics/index.rst +++ b/gcc/jit/docs/topics/index.rst @@ -1,4 +1,4 @@ -.. Copyright (C) 2014 Free Software Foundation, Inc. +.. Copyright (C) 2014-2015 Free Software Foundation, Inc. Originally contributed by David Malcolm This is free software: you can redistribute it and/or modify it @@ -27,4 +27,4 @@ Topic Reference expressions.rst functions.rst locations.rst - results.rst + compilation.rst diff --git a/gcc/jit/docs/topics/results.rst b/gcc/jit/docs/topics/results.rst deleted file mode 100644 index aa5ea8b..0000000 --- a/gcc/jit/docs/topics/results.rst +++ /dev/null @@ -1,127 +0,0 @@ -.. Copyright (C) 2014 Free Software Foundation, Inc. - Originally contributed by David Malcolm - - This is free software: you can redistribute it and/or modify it - under the terms of the GNU General Public License as published by - the Free Software Foundation, either version 3 of the License, or - (at your option) any later version. - - This program is distributed in the hope that it will be useful, but - WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program. If not, see - . - -.. default-domain:: c - -Compilation results -=================== - -.. type:: gcc_jit_result - - A `gcc_jit_result` encapsulates the result of compiling a context, - and the lifetimes of any machine code functions or globals that are - within it. - -.. function:: gcc_jit_result *\ - gcc_jit_context_compile (gcc_jit_context *ctxt) - - This calls into GCC and builds the code, returning a - `gcc_jit_result *`. - - If this is non-NULL, the caller becomes responsible for - calling :func:`gcc_jit_result_release` on it once they're done - with it. - -.. function:: void *\ - gcc_jit_result_get_code (gcc_jit_result *result,\ - const char *funcname) - - Locate a given function within the built machine code. - - Functions are looked up by name. For this to succeed, a function - with a name matching `funcname` must have been created on - `result`'s context (or a parent context) via a call to - :func:`gcc_jit_context_new_function` with `kind` - :macro:`GCC_JIT_FUNCTION_EXPORTED`: - - .. code-block:: c - - gcc_jit_context_new_function (ctxt, - any_location, /* or NULL */ - /* Required for func to be visible to - gcc_jit_result_get_code: */ - GCC_JIT_FUNCTION_EXPORTED, - any_return_type, - /* Must string-compare equal: */ - funcname, - /* etc */); - - If such a function is not found (or `result` or `funcname` are - ``NULL``), an error message will be emitted on stderr and - ``NULL`` will be returned. - - If the function is found, the result will need to be cast to a - function pointer of the correct type before it can be called. - - Note that the resulting machine code becomes invalid after - :func:`gcc_jit_result_release` is called on the - :type:`gcc_jit_result *`; attempting to call it after that may lead - to a segmentation fault. - -.. function:: void *\ - gcc_jit_result_get_global (gcc_jit_result *result,\ - const char *name) - - Locate a given global within the built machine code. - - Globals are looked up by name. For this to succeed, a global - with a name matching `name` must have been created on - `result`'s context (or a parent context) via a call to - :func:`gcc_jit_context_new_global` with `kind` - :macro:`GCC_JIT_GLOBAL_EXPORTED`. - - If the global is found, the result will need to be cast to a - pointer of the correct type before it can be called. - - This is a *pointer* to the global, so e.g. for an :c:type:`int` this is - an :c:type:`int *`. - - For example, given an ``int foo;`` created this way: - - .. code-block:: c - - gcc_jit_lvalue *exported_global = - gcc_jit_context_new_global (ctxt, - any_location, /* or NULL */ - GCC_JIT_GLOBAL_EXPORTED, - int_type, - "foo"); - - we can access it like this: - - .. code-block:: c - - int *ptr_to_foo = - (int *)gcc_jit_result_get_global (result, "foo"); - - If such a global is not found (or `result` or `name` are - ``NULL``), an error message will be emitted on stderr and - ``NULL`` will be returned. - - Note that the resulting address becomes invalid after - :func:`gcc_jit_result_release` is called on the - :type:`gcc_jit_result *`; attempting to use it after that may lead - to a segmentation fault. - -.. function:: void\ - gcc_jit_result_release (gcc_jit_result *result) - - Once we're done with the code, this unloads the built .so file. - This cleans up the result; after calling this, it's no longer - valid to use the result, or any code or globals that were obtained - by calling :func:`gcc_jit_result_get_code` or - :func:`gcc_jit_result_get_global` on it. diff --git a/gcc/jit/jit-playback.c b/gcc/jit/jit-playback.c index ca4e112..b4f2073 100644 --- a/gcc/jit/jit-playback.c +++ b/gcc/jit/jit-playback.c @@ -1668,26 +1668,37 @@ auto_argvec::~auto_argvec () - Use the context's options to cconstruct command-line options, and call into the rest of GCC (toplev::main). - - Assuming it succeeds, we have a .s file; we want a .so file. - Invoke another gcc to convert the .s file to a .so file. - - dlopen the .so file - - Wrap the result up as a playback::result and return it. */ + - Assuming it succeeds, we have a .s file. + - We then run the "postprocess" vfunc: -result * + (A) In-memory compile ("gcc_jit_context_compile") + + For an in-memory compile we have the playback::compile_to_memory + subclass; "postprocess" will convert the .s file to a .so DSO, + and load it in memory (via dlopen), wrapping the result up as + a jit::result and returning it. + + (B) Compile to file ("gcc_jit_context_compile_to_file") + + When compiling to a file, we have the playback::compile_to_file + subclass; "postprocess" will either copy the .s file to the + destination (for GCC_JIT_OUTPUT_KIND_ASSEMBLER), or invoke + the driver to convert it as necessary, copying the result. */ + +void playback::context:: compile () { JIT_LOG_SCOPE (get_logger ()); const char *ctxt_progname; - result *result_obj = NULL; int keep_intermediates = get_bool_option (GCC_JIT_BOOL_OPTION_KEEP_INTERMEDIATES); m_tempdir = new tempdir (get_logger (), keep_intermediates); if (!m_tempdir->create ()) - return NULL; + return; /* Call into the rest of gcc. For now, we have to assemble command-line options to pass into @@ -1706,7 +1717,7 @@ compile () auto_argvec fake_args; make_fake_args (&fake_args, ctxt_progname, &requested_dumps); if (errors_occurred ()) - return NULL; + return; /* Acquire the JIT mutex and set "this" as the active playback ctxt. */ acquire_mutex (); @@ -1737,24 +1748,258 @@ compile () if (errors_occurred ()) { release_mutex (); - return NULL; + return; } if (get_bool_option (GCC_JIT_BOOL_OPTION_DUMP_GENERATED_CODE)) dump_generated_code (); + /* We now have a .s file. + + Run any postprocessing steps. This will either convert the .s file to + a .so DSO, and load it in memory (playback::compile_to_memory), or + convert the .s file to the requested output format, and copy it to a + given file (playback::compile_to_file). */ + postprocess (ctxt_progname); + + release_mutex (); +} + +/* Implementation of class gcc::jit::playback::compile_to_memory, + a subclass of gcc::jit::playback::context. */ + +/* playback::compile_to_memory's trivial constructor. */ + +playback::compile_to_memory::compile_to_memory (recording::context *ctxt) : + playback::context (ctxt), + m_result (NULL) +{ + JIT_LOG_SCOPE (get_logger ()); +} + +/* Implementation of the playback::context::process vfunc for compiling + to memory. + + Convert the .s file to a .so DSO, and load it in memory (via dlopen), + wrapping the result up as a jit::result and returning it. */ + +void +playback::compile_to_memory::postprocess (const char *ctxt_progname) +{ + JIT_LOG_SCOPE (get_logger ()); convert_to_dso (ctxt_progname); if (errors_occurred ()) + return; + m_result = dlopen_built_dso (); +} + +/* Implementation of class gcc::jit::playback::compile_to_file, + a subclass of gcc::jit::playback::context. */ + +/* playback::compile_to_file's trivial constructor. */ + +playback::compile_to_file::compile_to_file (recording::context *ctxt, + enum gcc_jit_output_kind output_kind, + const char *output_path) : + playback::context (ctxt), + m_output_kind (output_kind), + m_output_path (output_path) +{ + JIT_LOG_SCOPE (get_logger ()); +} + +/* Implementation of the playback::context::process vfunc for compiling + to a file. + + Either copy the .s file to the given destination (for + GCC_JIT_OUTPUT_KIND_ASSEMBLER), or invoke the driver to convert it + as necessary, copying the result. */ + +void +playback::compile_to_file::postprocess (const char *ctxt_progname) +{ + JIT_LOG_SCOPE (get_logger ()); + + /* The driver takes different actions based on the filename, so + we provide a filename with an appropriate suffix for the + output kind, and then copy it up to the user-provided path, + rather than directly compiling it to the requested output path. */ + + switch (m_output_kind) { - release_mutex (); - return NULL; + default: + gcc_unreachable (); + + case GCC_JIT_OUTPUT_KIND_ASSEMBLER: + copy_file (get_tempdir ()->get_path_s_file (), + m_output_path); + break; + + case GCC_JIT_OUTPUT_KIND_OBJECT_FILE: + { + char *tmp_o_path = ::concat (get_tempdir ()->get_path (), + "/fake.o", + NULL); + invoke_driver (ctxt_progname, + get_tempdir ()->get_path_s_file (), + tmp_o_path, + TV_ASSEMBLE, + false, /* bool shared, */ + false);/* bool run_linker */ + if (!errors_occurred ()) + copy_file (tmp_o_path, + m_output_path); + free (tmp_o_path); + } + break; + + case GCC_JIT_OUTPUT_KIND_DYNAMIC_LIBRARY: + invoke_driver (ctxt_progname, + get_tempdir ()->get_path_s_file (), + get_tempdir ()->get_path_so_file (), + TV_ASSEMBLE, + true, /* bool shared, */ + true);/* bool run_linker */ + if (!errors_occurred ()) + copy_file (get_tempdir ()->get_path_so_file (), + m_output_path); + break; + + case GCC_JIT_OUTPUT_KIND_EXECUTABLE: + { + char *tmp_exe_path = ::concat (get_tempdir ()->get_path (), + "/fake.exe", + NULL); + invoke_driver (ctxt_progname, + get_tempdir ()->get_path_s_file (), + tmp_exe_path, + TV_ASSEMBLE, + false, /* bool shared, */ + true);/* bool run_linker */ + if (!errors_occurred ()) + copy_file (tmp_exe_path, + m_output_path); + free (tmp_exe_path); + } + break; + } - result_obj = dlopen_built_dso (); +} + +/* Copy SRC_PATH to DST_PATH, preserving permission bits (in particular, + the "executable" bits). - release_mutex (); + Any errors that occur are reported on the context and hence count as + a failure of the compile. - return result_obj; + We can't in general hardlink or use "rename" from the tempdir since + it might be on a different filesystem to the destination. For example, + I get EXDEV: "Invalid cross-device link". */ + +void +playback::compile_to_file::copy_file (const char *src_path, + const char *dst_path) +{ + JIT_LOG_SCOPE (get_logger ()); + if (get_logger ()) + { + get_logger ()->log ("src_path: %s", src_path); + get_logger ()->log ("dst_path: %s", dst_path); + } + + FILE *f_in = NULL; + FILE *f_out = NULL; + size_t total_sz_in = 0; + size_t total_sz_out = 0; + char buf[4096]; + size_t sz_in; + struct stat stat_buf; + + f_in = fopen (src_path, "rb"); + if (!f_in) + { + add_error (NULL, + "unable to open %s for reading: %s", + src_path, + xstrerror (errno)); + return; + } + + /* Use stat on the filedescriptor to get the mode, + so that we can copy it over (in particular, the + "executable" bits). */ + if (-1 == fstat (fileno (f_in), &stat_buf)) + { + add_error (NULL, + "unable to fstat %s: %s", + src_path, + xstrerror (errno)); + fclose (f_in); + return; + } + + f_out = fopen (dst_path, "wb"); + if (!f_out) + { + add_error (NULL, + "unable to open %s for writing: %s", + dst_path, + xstrerror (errno)); + fclose (f_in); + return; + } + + while ( (sz_in = fread (buf, 1, sizeof (buf), f_in)) ) + { + total_sz_in += sz_in; + size_t sz_out_remaining = sz_in; + size_t sz_out_so_far = 0; + while (sz_out_remaining) + { + size_t sz_out = fwrite (buf + sz_out_so_far, + 1, + sz_out_remaining, + f_out); + gcc_assert (sz_out <= sz_out_remaining); + if (!sz_out) + { + add_error (NULL, + "error writing to %s: %s", + dst_path, + xstrerror (errno)); + fclose (f_in); + fclose (f_out); + return; + } + total_sz_out += sz_out; + sz_out_so_far += sz_out; + sz_out_remaining -= sz_out; + } + gcc_assert (sz_out_so_far == sz_in); + } + + if (!feof (f_in)) + add_error (NULL, + "error reading from %s: %s", + src_path, + xstrerror (errno)); + + fclose (f_in); + + gcc_assert (total_sz_in == total_sz_out); + if (get_logger ()) + get_logger ()->log ("total bytes copied: %ld", total_sz_out); + + /* Set the permissions of the copy to those of the original file, + in particular the "executable" bits. */ + if (-1 == fchmod (fileno (f_out), stat_buf.st_mode)) + add_error (NULL, + "error setting mode of %s: %s", + dst_path, + xstrerror (errno)); + + fclose (f_out); } /* Helper functions for gcc::jit::playback::context::compile. */ @@ -1975,9 +2220,28 @@ playback::context:: convert_to_dso (const char *ctxt_progname) { JIT_LOG_SCOPE (get_logger ()); + + invoke_driver (ctxt_progname, + m_tempdir->get_path_s_file (), + m_tempdir->get_path_so_file (), + TV_ASSEMBLE, + true, /* bool shared, */ + true);/* bool run_linker */ +} + +void +playback::context:: +invoke_driver (const char *ctxt_progname, + const char *input_file, + const char *output_file, + timevar_id_t tv_id, + bool shared, + bool run_linker) +{ + JIT_LOG_SCOPE (get_logger ()); /* Currently this lumps together both assembling and linking into TV_ASSEMBLE. */ - auto_timevar assemble_timevar (TV_ASSEMBLE); + auto_timevar assemble_timevar (tv_id); const char *errmsg; auto_vec argvec; #define ADD_ARG(arg) argvec.safe_push (arg) @@ -1986,12 +2250,16 @@ convert_to_dso (const char *ctxt_progname) const char *gcc_driver_name = GCC_DRIVER_NAME; ADD_ARG (gcc_driver_name); - ADD_ARG ("-shared"); - /* The input: assembler. */ - ADD_ARG (m_tempdir->get_path_s_file ()); - /* The output: shared library. */ + + if (shared) + ADD_ARG ("-shared"); + + if (!run_linker) + ADD_ARG ("-c"); + + ADD_ARG (input_file); ADD_ARG ("-o"); - ADD_ARG (m_tempdir->get_path_so_file ()); + ADD_ARG (output_file); /* Don't use the linker plugin. If running with just a "make" and not a "make install", then we'd diff --git a/gcc/jit/jit-playback.h b/gcc/jit/jit-playback.h index 8efd506..e9832f0 100644 --- a/gcc/jit/jit-playback.h +++ b/gcc/jit/jit-playback.h @@ -23,6 +23,8 @@ along with GCC; see the file COPYING3. If not see #include // for std::pair +#include "timevar.h" + #include "jit-recording.h" namespace gcc { @@ -35,6 +37,12 @@ namespace jit { namespace playback { +/* playback::context is an abstract base class. + + The two concrete subclasses are: + - playback::compile_to_memory + - playback::compile_to_file. */ + class context : public log_user { public: @@ -174,7 +182,7 @@ public: return m_recording_ctxt->get_builtins_manager (); } - result * + void compile (); void @@ -252,9 +260,22 @@ private: char * read_dump_file (const char *path); + virtual void postprocess (const char *ctxt_progname) = 0; + +protected: + tempdir *get_tempdir () { return m_tempdir; } + void convert_to_dso (const char *ctxt_progname); + void + invoke_driver (const char *ctxt_progname, + const char *input_file, + const char *output_file, + timevar_id_t tv_id, + bool shared, + bool run_linker); + result * dlopen_built_dso (); @@ -274,6 +295,37 @@ private: auto_vec > m_cached_locations; }; +class compile_to_memory : public context +{ + public: + compile_to_memory (recording::context *ctxt); + void postprocess (const char *ctxt_progname); + + result *get_result_obj () const { return m_result; } + + private: + result *m_result; +}; + +class compile_to_file : public context +{ + public: + compile_to_file (recording::context *ctxt, + enum gcc_jit_output_kind output_kind, + const char *output_path); + void postprocess (const char *ctxt_progname); + + private: + void + copy_file (const char *src_path, + const char *dst_path); + + private: + enum gcc_jit_output_kind m_output_kind; + const char *m_output_path; +}; + + /* A temporary wrapper object. These objects are (mostly) only valid during replay. We allocate them on the GC heap, so that they will be cleaned diff --git a/gcc/jit/jit-recording.c b/gcc/jit/jit-recording.c index 76eabbd..2b8af1f 100644 --- a/gcc/jit/jit-recording.c +++ b/gcc/jit/jit-recording.c @@ -1152,8 +1152,8 @@ recording::context::enable_dump (const char *dumpname, m_requested_dumps.safe_push (d); } -/* Validate this context, and if it passes, compile it within a - mutex. +/* Validate this context, and if it passes, compile it to memory + (within a mutex). Implements the post-error-checking part of gcc_jit_context_compile. */ @@ -1168,13 +1168,41 @@ recording::context::compile () if (errors_occurred ()) return NULL; - /* Set up a playback context. */ - ::gcc::jit::playback::context replayer (this); + /* Set up a compile_to_memory playback context. */ + ::gcc::jit::playback::compile_to_memory replayer (this); /* Use it. */ - result *result_obj = replayer.compile (); + replayer.compile (); - return result_obj; + /* Get the jit::result (or NULL) from the + compile_to_memory playback context. */ + return replayer.get_result_obj (); +} + +/* Validate this context, and if it passes, compile it to a file + (within a mutex). + + Implements the post-error-checking part of + gcc_jit_context_compile_to_file. */ + +void +recording::context::compile_to_file (enum gcc_jit_output_kind output_kind, + const char *output_path) +{ + JIT_LOG_SCOPE (get_logger ()); + + validate (); + + if (errors_occurred ()) + return; + + /* Set up a compile_to_file playback context. */ + ::gcc::jit::playback::compile_to_file replayer (this, + output_kind, + output_path); + + /* Use it. */ + replayer.compile (); } /* Format the given error using printf's conventions, print diff --git a/gcc/jit/jit-recording.h b/gcc/jit/jit-recording.h index 57e7167..0dd3164 100644 --- a/gcc/jit/jit-recording.h +++ b/gcc/jit/jit-recording.h @@ -221,6 +221,10 @@ public: compile (); void + compile_to_file (enum gcc_jit_output_kind output_kind, + const char *output_path); + + void add_error (location *loc, const char *fmt, ...) GNU_PRINTF(3, 4); diff --git a/gcc/jit/libgccjit++.h b/gcc/jit/libgccjit++.h index 9b55c91..62ef6a4 100644 --- a/gcc/jit/libgccjit++.h +++ b/gcc/jit/libgccjit++.h @@ -99,6 +99,9 @@ namespace gccjit gcc_jit_result *compile (); + void compile_to_file (enum gcc_jit_output_kind output_kind, + const char *output_path); + void dump_to_file (const std::string &path, bool update_locations); @@ -541,6 +544,15 @@ context::compile () } inline void +context::compile_to_file (enum gcc_jit_output_kind output_kind, + const char *output_path) +{ + gcc_jit_context_compile_to_file (m_inner_ctxt, + output_kind, + output_path); +} + +inline void context::dump_to_file (const std::string &path, bool update_locations) { diff --git a/gcc/jit/libgccjit.c b/gcc/jit/libgccjit.c index 0faf0f9..7eb66bd 100644 --- a/gcc/jit/libgccjit.c +++ b/gcc/jit/libgccjit.c @@ -2196,7 +2196,7 @@ gcc_jit_context_compile (gcc_jit_context *ctxt) JIT_LOG_FUNC (ctxt->get_logger ()); - ctxt->log ("compiling ctxt: %p", (void *)ctxt); + ctxt->log ("in-memory compile of ctxt: %p", (void *)ctxt); gcc_jit_result *result = (gcc_jit_result *)ctxt->compile (); @@ -2209,6 +2209,35 @@ gcc_jit_context_compile (gcc_jit_context *ctxt) /* Public entrypoint. See description in libgccjit.h. After error-checking, the real work is done by the + gcc::jit::recording::context::compile_to_file method in + jit-recording.c. */ + +void +gcc_jit_context_compile_to_file (gcc_jit_context *ctxt, + enum gcc_jit_output_kind output_kind, + const char *output_path) +{ + RETURN_IF_FAIL (ctxt, NULL, NULL, "NULL context"); + JIT_LOG_FUNC (ctxt->get_logger ()); + RETURN_IF_FAIL_PRINTF1 ( + ((output_kind >= GCC_JIT_OUTPUT_KIND_ASSEMBLER) + && (output_kind <= GCC_JIT_OUTPUT_KIND_EXECUTABLE)), + ctxt, NULL, + "unrecognized output_kind: %i", + output_kind); + RETURN_IF_FAIL (output_path, ctxt, NULL, "NULL output_path"); + + ctxt->log ("compile_to_file of ctxt: %p", (void *)ctxt); + ctxt->log ("output_kind: %i", output_kind); + ctxt->log ("output_path: %s", output_path); + + ctxt->compile_to_file (output_kind, output_path); +} + + +/* Public entrypoint. See description in libgccjit.h. + + After error-checking, the real work is done by the gcc::jit::recording::context::dump_to_file method in jit-recording.c. */ diff --git a/gcc/jit/libgccjit.h b/gcc/jit/libgccjit.h index 2f41087..12514ba 100644 --- a/gcc/jit/libgccjit.h +++ b/gcc/jit/libgccjit.h @@ -36,17 +36,20 @@ extern "C" { the API below. Invoking gcc_jit_context_compile on it gives you a gcc_jit_result * - (or NULL). + (or NULL), representing in-memory machine code. You can call gcc_jit_context_compile repeatedly on one context, giving multiple independent results. + Similarly, you can call gcc_jit_context_compile_to_file on a context + to compile to disk. + Eventually you can call gcc_jit_context_release to clean up the - context; any results created from it are still usable, and should be - cleaned up via gcc_jit_result_release. */ + context; any in-memory results created from it are still usable, and + should be cleaned up via gcc_jit_result_release. */ typedef struct gcc_jit_context gcc_jit_context; -/* A gcc_jit_result encapsulates the result of a compilation. */ +/* A gcc_jit_result encapsulates the result of an in-memory compilation. */ typedef struct gcc_jit_result gcc_jit_result; /* An object created within a context. Such objects are automatically @@ -240,12 +243,42 @@ gcc_jit_context_set_bool_option (gcc_jit_context *ctxt, enum gcc_jit_bool_option opt, int value); -/* This actually calls into GCC and runs the build, all - in a mutex for now. The result is a wrapper around a .so file. - It can only be called once on a given context. */ +/* Compile the context to in-memory machine code. + + This can be called more that once on a given context, + although any errors that occur will block further compilation. */ + extern gcc_jit_result * gcc_jit_context_compile (gcc_jit_context *ctxt); +/* Kinds of ahead-of-time compilation, for use with + gcc_jit_context_compile_to_file. */ + +enum gcc_jit_output_kind +{ + /* Compile the context to an assembler file. */ + GCC_JIT_OUTPUT_KIND_ASSEMBLER, + + /* Compile the context to an object file. */ + GCC_JIT_OUTPUT_KIND_OBJECT_FILE, + + /* Compile the context to a dynamic library. */ + GCC_JIT_OUTPUT_KIND_DYNAMIC_LIBRARY, + + /* Compile the context to an executable. */ + GCC_JIT_OUTPUT_KIND_EXECUTABLE +}; + +/* Compile the context to a file of the given kind. + + This can be called more that once on a given context, + although any errors that occur will block further compilation. */ + +extern void +gcc_jit_context_compile_to_file (gcc_jit_context *ctxt, + enum gcc_jit_output_kind output_kind, + const char *output_path); + /* To help with debugging: dump a C-like representation to the given path, describing what's been set up on the context. @@ -1079,14 +1112,15 @@ gcc_jit_context_dump_reproducer_to_file (gcc_jit_context *ctxt, The context directly stores the dumpname as a (const char *), so the passed string must outlive the context. - gcc_jit_context_compile will capture the dump as a - dynamically-allocated buffer, writing it to ``*out_ptr``. + gcc_jit_context_compile and gcc_jit_context_to_file + will capture the dump as a dynamically-allocated buffer, writing + it to ``*out_ptr``. The caller becomes responsible for calling free (*out_ptr) - each time that gcc_jit_context_compile is called. *out_ptr will be - written to, either with the address of a buffer, or with NULL if an - error occurred. + each time that gcc_jit_context_compile or gcc_jit_context_to_file + are called. *out_ptr will be written to, either with the address of a + buffer, or with NULL if an error occurred. This API entrypoint is likely to be less stable than the others. In particular, both the precise dumpnames, and the format and content diff --git a/gcc/jit/libgccjit.map b/gcc/jit/libgccjit.map index 93d5c26..89bd57be4 100644 --- a/gcc/jit/libgccjit.map +++ b/gcc/jit/libgccjit.map @@ -32,6 +32,7 @@ gcc_jit_block_get_function; gcc_jit_context_acquire; gcc_jit_context_compile; + gcc_jit_context_compile_to_file; gcc_jit_context_dump_to_file; gcc_jit_context_dump_reproducer_to_file; gcc_jit_context_enable_dump; diff --git a/gcc/jit/notes.txt b/gcc/jit/notes.txt index 7df4a7b..e92c665 100644 --- a/gcc/jit/notes.txt +++ b/gcc/jit/notes.txt @@ -74,9 +74,16 @@ Client Code . Generated . libgccjit.so . . . . │ (purge internal state) . . <──────────────────────── end of toplev::finalize . . │ . . - . . │ Convert assembler to DSO ("fake.so") - . . │ . . - . . │ Load DSO (dlopen "fake.so") + . . V─> playback::context::postprocess: + . . │ . . + . . │ (assuming an in-memory compile): + . . │ . . + . . │ . Convert assembler to DSO ("fake.so") + . . │ . . + . . │ . Load DSO (dlopen "fake.so") + . . │ . . + . . │ . Bundle it up in a jit::result + . . <── . . . . │ . . . . │ RELEASE MUTEX . . . │ . . diff --git a/gcc/testsuite/jit.dg/harness.h b/gcc/testsuite/jit.dg/harness.h index 96fc5da..db25637 100644 --- a/gcc/testsuite/jit.dg/harness.h +++ b/gcc/testsuite/jit.dg/harness.h @@ -7,12 +7,14 @@ extern void create_code (gcc_jit_context *ctxt, void * user_data); + and, #ifndef TEST_COMPILING_TO_FILE, + extern void verify_code (gcc_jit_context *ctxt, gcc_jit_result *result); - */ #include #include +#include /* test-threads.c use threads, but dejagnu.h isn't thread-safe; there's a shared "buffer", and the counts of passed/failed etc are globals. @@ -106,12 +108,23 @@ static char test[1024]; } \ } while (0) +#define CHECK_NO_ERRORS(CTXT) \ + do { \ + const char *err = gcc_jit_context_get_first_error (CTXT); \ + if (err) \ + fail ("error unexpectedly occurred: %s", err); \ + else \ + pass ("no errors occurred"); \ + } while (0) + /* Hooks that testcases should provide. */ extern void create_code (gcc_jit_context *ctxt, void * user_data); +#ifndef TEST_COMPILING_TO_FILE extern void verify_code (gcc_jit_context *ctxt, gcc_jit_result *result); +#endif extern void check_string_value (const char *funcname, const char *actual, const char *expected); @@ -322,7 +335,13 @@ test_jit (const char *argv0, void *user_data) { gcc_jit_context *ctxt; FILE *logfile; +#ifndef TEST_COMPILING_TO_FILE gcc_jit_result *result; +#endif + +#ifdef TEST_COMPILING_TO_FILE + unlink (OUTPUT_FILENAME); +#endif ctxt = gcc_jit_context_acquire (); if (!ctxt) @@ -339,16 +358,24 @@ test_jit (const char *argv0, void *user_data) dump_reproducer (ctxt, argv0); +#ifdef TEST_COMPILING_TO_FILE + gcc_jit_context_compile_to_file (ctxt, + (OUTPUT_KIND), + (OUTPUT_FILENAME)); +#else /* #ifdef TEST_COMPILING_TO_FILE */ /* This actually calls into GCC and runs the build, all in a mutex for now. */ result = gcc_jit_context_compile (ctxt); verify_code (ctxt, result); +#endif gcc_jit_context_release (ctxt); +#ifndef TEST_COMPILING_TO_FILE /* Once we're done with the code, this unloads the built .so file: */ gcc_jit_result_release (result); +#endif if (logfile) fclose (logfile); diff --git a/gcc/testsuite/jit.dg/jit.exp b/gcc/testsuite/jit.dg/jit.exp index 3caccce..c853f22 100644 --- a/gcc/testsuite/jit.dg/jit.exp +++ b/gcc/testsuite/jit.dg/jit.exp @@ -139,6 +139,8 @@ proc fixed_host_execute {args} { global text global spawn_id + verbose "fixed_host_execute: $args" + set timeoutmsg "Timed out: Never got started, " set timeout 100 set file all @@ -148,9 +150,8 @@ proc fixed_host_execute {args} { if { [llength $args] == 0} { set executable $args } else { - set executable [string trimleft [lindex [split $args " "] 0] "\{"] - set params [string trimleft [lindex [split $args " "] 1] "\{"] - set params [string trimright $params "\}"] + set executable [lindex $args 0] + set params [lindex $args 1] } verbose "The executable is $executable" 2 @@ -159,6 +160,8 @@ proc fixed_host_execute {args} { return "No source file found" } + verbose "params: $params" 2 + # spawn the executable and look for the DejaGnu output messages from the # test case. # spawn -noecho -open [open "|./${executable}" "r"] @@ -328,6 +331,39 @@ proc get_path_of_driver {} { return [file dirname $binary] } +# Expand "SRCDIR" within ARG to the location of the top-level +# src directory + +proc jit-expand-vars {arg} { + verbose "jit-expand-vars: $arg" + global srcdir + verbose " srcdir: $srcdir" + # "srcdir" is that of the gcc/testsuite directory, so + # we need to go up two levels. + set arg [string map [list "SRCDIR" $srcdir/../..] $arg] + verbose " new arg: $arg" + return $arg +} + +# Parameters used when invoking the executables built from the test cases. + +global jit-exe-params +set jit-exe-params {} + +# Set "jit-exe-params", expanding "SRCDIR" in each arg to the location of +# the top-level srcdir. + +proc dg-jit-set-exe-params { args } { + verbose "dg-jit-set-exe-params: $args" + + global jit-exe-params + set jit-exe-params {} + # Skip initial arg (line number) + foreach arg [lrange $args 1 [llength $args] ] { + lappend jit-exe-params [jit-expand-vars $arg] + } +} + proc jit-dg-test { prog do_what extra_tool_flags } { verbose "within jit-dg-test..." verbose " prog: $prog" @@ -339,6 +375,15 @@ proc jit-dg-test { prog do_what extra_tool_flags } { append extra_tool_flags " -lpthread" } + # Any test case that uses + # { dg-final { jit-verify-compile-to-file FOO } } + # needs to call jit-setup-compile-to-file here. + # (is there a better way to handle setup/finish pairs in dg?) + set tmp [grep $prog "jit-verify-compile-to-file"] + if {![string match "" $tmp]} { + jit-setup-compile-to-file $prog + } + # Determine what to name the built executable. # # We simply append .exe to the filename, e.g. @@ -464,7 +509,12 @@ proc jit-dg-test { prog do_what extra_tool_flags } { # http://lists.gnu.org/archive/html/dejagnu/2014-10/msg00000.html # We instead call a patched local copy, "fixed_host_execute", defined # above. - set result [fixed_host_execute $output_file] + + global jit-exe-params + set args ${jit-exe-params} + set jit-exe-params {} + + set result [fixed_host_execute $output_file $args ] verbose "result: $result" # Restore PATH @@ -530,6 +580,157 @@ proc jit-dg-test { prog do_what extra_tool_flags } { return [list $comp_output $output_file] } +# Given source file PROG, scrape out the value of +# #define OUTPUT_FILENAME +# failing if it's not found. + +proc jit-get-output-filename {prog} { + set tmp [grep $prog "#define OUTPUT_FILENAME (.*)"] + if {![string match "" $tmp]} { + foreach i $tmp { + verbose "i: $i" + if {[regexp "^\#define OUTPUT_FILENAME\[ \t\]\+\"(.*)\"$" $i i group] } { + verbose "group: '$group'" + return $group + } else { + fail "Unable to parse line: $i" + } + } + } + fail "Unable to locate OUTPUT_FILENAME" + return "" +} + +# For testcases that use jit-verify-compile-to-file, +# delete OUTPUT_FILENAME beforehand, to ensure that the +# testcase is indeed creating it. + +proc jit-setup-compile-to-file { prog } { + verbose "jit-setup-compile-to-file: $prog" + set output_filename [jit-get-output-filename $prog] + verbose " output_filename: $output_filename" + if {![string match "" $output_filename]} { + catch "exec rm -f $output_filename" + } +} + +# Locate OUTPUT_FILENAME within the testcase. Verify +# that a file with that name was created, and that +# the output of running the "file" utility on it +# matches the given regex. +# For use by the various test-compile-to-*.c testcases. + +proc jit-verify-compile-to-file {args} { + verbose "jit-verify-compile-to-file: $args" + + set file_regex [lindex $args 0] + verbose "file_regex: $file_regex" + + upvar 2 prog prog + verbose "prog: $prog" + set output_filename [jit-get-output-filename $prog] + verbose " output_filename: $output_filename" + + # Verify that the expected file was written out + if { [file exists $output_filename] == 1} { + pass "$output_filename exists" + } else { + fail "$output_filename does not exist" + return + } + + # Run "file" on OUTPUT_FILENAME, and verify that the output + # matches $file_regex. + spawn -noecho "file" $output_filename + set file_id $spawn_id + expect "\n" { + verbose "got newline: $expect_out(buffer)" + set classification $expect_out(buffer) + verbose "classification: $classification" + if { [regexp $file_regex $classification] } { + pass "'file' output on $output_filename matched: $file_regex" + } else { + fail "'file' output on $output_filename did not match: $file_regex" + } + } + set $spawn_id $file_id + close +} + +# Verify that the given file exists, and is executable. +# Attempt to execute it, and verify that its stdout matches +# the given regex. + +proc jit-run-executable { args } { + verbose "jit-run-executable: $args" + + set executable-name [lindex $args 0] + verbose "executable-name: ${executable-name}" + + set dg-output-text [lindex $args 1] + verbose "dg-output-text: ${dg-output-text}" + + if { [file executable ${executable-name}] } { + pass "${executable-name} has executable bit set" + } else { + fail "${executable-name} does not have executable bit set" + } + + # Attempt to run the executable; adapted from dg.exp's dg-test + set status -1 + set result [jit_load ./${executable-name}] + set status [lindex $result 0] + set output [lindex $result 1] + verbose " status: $status" + verbose " output: $output" + # send_user "After exec, status: $status\n" + if { "$status" == "pass" } { + pass "${executable-name} execution test" + verbose "Exec succeeded." 3 + set texttmp ${dg-output-text} + if { ![regexp $texttmp ${output}] } { + fail "${executable-name} output pattern test, is ${output}, should match $texttmp" + verbose "Failed test for output pattern $texttmp" 3 + } else { + pass "${executable-name} output pattern test, $texttmp" + verbose "Passed test for output pattern $texttmp" 3 + } + unset texttmp + } elseif { "$status" == "fail" } { + # It would be nice to get some info out of errorCode. + if {[info exists errorCode]} { + verbose "Exec failed, errorCode: $errorCode" 3 + } else { + verbose "Exec failed, errorCode not defined!" 3 + } + fail "${executable-name} execution test" + } else { + $status "${executable-name} execution test" + } +} + +# A way to invoke "jit-run-executable" with the given regex, +# using OUTPUT_FILENAME within the testcase to determine +# the name of the executable to run. +# For use by the test-compile-to-executable.c testcase. + +proc jit-verify-executable { args } { + verbose "jit-verify-executable: $args" + + set dg-output-text [lindex $args 0] + verbose "dg-output-text: ${dg-output-text}" + + upvar 2 name name + verbose "name: $name" + + upvar 2 prog prog + verbose "prog: $prog" + set output_filename [jit-get-output-filename $prog] + verbose " output_filename: $output_filename" + + jit-run-executable $output_filename ${dg-output-text} +} + # We need to link with --export-dynamic for test-calling-external-function.c # so that the JIT-built code can call into functions from the main program. set DEFAULT_CFLAGS "-I$srcdir/../jit -lgccjit -g -Wall -Werror -Wl,--export-dynamic" diff --git a/gcc/testsuite/jit.dg/test-compile-to-assembler.c b/gcc/testsuite/jit.dg/test-compile-to-assembler.c new file mode 100644 index 0000000..c5b282c --- /dev/null +++ b/gcc/testsuite/jit.dg/test-compile-to-assembler.c @@ -0,0 +1,65 @@ +#include +#include + +#include "libgccjit.h" + +#define TEST_COMPILING_TO_FILE +#define OUTPUT_KIND GCC_JIT_OUTPUT_KIND_ASSEMBLER +#define OUTPUT_FILENAME "output-of-test-compile-to-assembler.c.s" +#include "harness.h" + +void +create_code (gcc_jit_context *ctxt, void *user_data) +{ + /* Let's try to inject the equivalent of: + void + hello_world (const char *name) + { + // a test comment + printf ("hello %s\n", name); + } + */ + gcc_jit_type *void_type = + gcc_jit_context_get_type (ctxt, GCC_JIT_TYPE_VOID); + gcc_jit_type *const_char_ptr_type = + gcc_jit_context_get_type (ctxt, GCC_JIT_TYPE_CONST_CHAR_PTR); + gcc_jit_param *param_name = + gcc_jit_context_new_param (ctxt, NULL, const_char_ptr_type, "name"); + gcc_jit_function *func = + gcc_jit_context_new_function (ctxt, NULL, + GCC_JIT_FUNCTION_EXPORTED, + void_type, + "hello_world", + 1, ¶m_name, + 0); + + gcc_jit_param *param_format = + gcc_jit_context_new_param (ctxt, NULL, const_char_ptr_type, "format"); + gcc_jit_function *printf_func = + gcc_jit_context_new_function (ctxt, NULL, + GCC_JIT_FUNCTION_IMPORTED, + gcc_jit_context_get_type ( + ctxt, GCC_JIT_TYPE_INT), + "printf", + 1, ¶m_format, + 1); + gcc_jit_rvalue *args[2]; + args[0] = gcc_jit_context_new_string_literal (ctxt, "hello %s\n"); + args[1] = gcc_jit_param_as_rvalue (param_name); + + gcc_jit_block *block = gcc_jit_function_new_block (func, NULL); + + gcc_jit_block_add_comment ( + block, NULL, + "a test comment"); + + gcc_jit_block_add_eval ( + block, NULL, + gcc_jit_context_new_call (ctxt, + NULL, + printf_func, + 2, args)); + gcc_jit_block_end_with_void_return (block, NULL); +} + +/* { dg-final { jit-verify-compile-to-file "assembler source text" } } */ diff --git a/gcc/testsuite/jit.dg/test-compile-to-dynamic-library.c b/gcc/testsuite/jit.dg/test-compile-to-dynamic-library.c new file mode 100644 index 0000000..095f751 --- /dev/null +++ b/gcc/testsuite/jit.dg/test-compile-to-dynamic-library.c @@ -0,0 +1,65 @@ +#include +#include + +#include "libgccjit.h" + +#define TEST_COMPILING_TO_FILE +#define OUTPUT_KIND GCC_JIT_OUTPUT_KIND_DYNAMIC_LIBRARY +#define OUTPUT_FILENAME "output-of-test-compile-to-dynamic-library.c.so" +#include "harness.h" + +void +create_code (gcc_jit_context *ctxt, void *user_data) +{ + /* Let's try to inject the equivalent of: + void + hello_world (const char *name) + { + // a test comment + printf ("hello %s\n", name); + } + */ + gcc_jit_type *void_type = + gcc_jit_context_get_type (ctxt, GCC_JIT_TYPE_VOID); + gcc_jit_type *const_char_ptr_type = + gcc_jit_context_get_type (ctxt, GCC_JIT_TYPE_CONST_CHAR_PTR); + gcc_jit_param *param_name = + gcc_jit_context_new_param (ctxt, NULL, const_char_ptr_type, "name"); + gcc_jit_function *func = + gcc_jit_context_new_function (ctxt, NULL, + GCC_JIT_FUNCTION_EXPORTED, + void_type, + "hello_world", + 1, ¶m_name, + 0); + + gcc_jit_param *param_format = + gcc_jit_context_new_param (ctxt, NULL, const_char_ptr_type, "format"); + gcc_jit_function *printf_func = + gcc_jit_context_new_function (ctxt, NULL, + GCC_JIT_FUNCTION_IMPORTED, + gcc_jit_context_get_type ( + ctxt, GCC_JIT_TYPE_INT), + "printf", + 1, ¶m_format, + 1); + gcc_jit_rvalue *args[2]; + args[0] = gcc_jit_context_new_string_literal (ctxt, "hello %s\n"); + args[1] = gcc_jit_param_as_rvalue (param_name); + + gcc_jit_block *block = gcc_jit_function_new_block (func, NULL); + + gcc_jit_block_add_comment ( + block, NULL, + "a test comment"); + + gcc_jit_block_add_eval ( + block, NULL, + gcc_jit_context_new_call (ctxt, + NULL, + printf_func, + 2, args)); + gcc_jit_block_end_with_void_return (block, NULL); +} + +/* { dg-final { jit-verify-compile-to-file "shared object.+dynamically linked" } } */ diff --git a/gcc/testsuite/jit.dg/test-compile-to-executable.c b/gcc/testsuite/jit.dg/test-compile-to-executable.c new file mode 100644 index 0000000..8d7b428 --- /dev/null +++ b/gcc/testsuite/jit.dg/test-compile-to-executable.c @@ -0,0 +1,110 @@ +#include +#include + +#include "libgccjit.h" + +#define TEST_COMPILING_TO_FILE +#define OUTPUT_KIND GCC_JIT_OUTPUT_KIND_EXECUTABLE +#define OUTPUT_FILENAME "output-of-test-compile-to-executable.c.exe" +#include "harness.h" + +void +create_code (gcc_jit_context *ctxt, void *user_data) +{ + /* Let's try to inject the equivalent of: + static void + hello_world (const char *name) + { + // a test comment + printf ("hello %s\n", name); + } + + extern int + main (int argc, char **argv) + { + hello_world (argv[0]); + return 0; + } + */ + gcc_jit_type *void_type = + gcc_jit_context_get_type (ctxt, GCC_JIT_TYPE_VOID); + gcc_jit_type *const_char_ptr_type = + gcc_jit_context_get_type (ctxt, GCC_JIT_TYPE_CONST_CHAR_PTR); + gcc_jit_param *param_name = + gcc_jit_context_new_param (ctxt, NULL, const_char_ptr_type, "name"); + gcc_jit_function *func = + gcc_jit_context_new_function (ctxt, NULL, + GCC_JIT_FUNCTION_INTERNAL, + void_type, + "hello_world", + 1, ¶m_name, + 0); + + gcc_jit_param *param_format = + gcc_jit_context_new_param (ctxt, NULL, const_char_ptr_type, "format"); + gcc_jit_function *printf_func = + gcc_jit_context_new_function (ctxt, NULL, + GCC_JIT_FUNCTION_IMPORTED, + gcc_jit_context_get_type ( + ctxt, GCC_JIT_TYPE_INT), + "printf", + 1, ¶m_format, + 1); + gcc_jit_rvalue *args[2]; + args[0] = gcc_jit_context_new_string_literal (ctxt, "hello %s\n"); + args[1] = gcc_jit_param_as_rvalue (param_name); + + gcc_jit_block *block = gcc_jit_function_new_block (func, NULL); + + gcc_jit_block_add_comment ( + block, NULL, + "a test comment"); + + gcc_jit_block_add_eval ( + block, NULL, + gcc_jit_context_new_call (ctxt, + NULL, + printf_func, + 2, args)); + gcc_jit_block_end_with_void_return (block, NULL); + + gcc_jit_type *int_type = gcc_jit_context_get_type (ctxt, GCC_JIT_TYPE_INT); + gcc_jit_param *param_argc = + gcc_jit_context_new_param (ctxt, NULL, int_type, "argc"); + gcc_jit_type *char_ptr_ptr_type = + gcc_jit_type_get_pointer ( + gcc_jit_type_get_pointer ( + gcc_jit_context_get_type (ctxt, GCC_JIT_TYPE_CHAR))); + gcc_jit_param *param_argv = + gcc_jit_context_new_param (ctxt, NULL, char_ptr_ptr_type, "argv"); + gcc_jit_param *params[2] = {param_argc, param_argv}; + gcc_jit_function *func_main = + gcc_jit_context_new_function (ctxt, NULL, + GCC_JIT_FUNCTION_EXPORTED, + int_type, + "main", + 2, params, + 0); + block = gcc_jit_function_new_block (func_main, NULL); + gcc_jit_rvalue *zero = gcc_jit_context_zero (ctxt, int_type); + args[0] = gcc_jit_context_new_cast ( + ctxt, + NULL, + gcc_jit_lvalue_as_rvalue ( + gcc_jit_context_new_array_access ( + ctxt, + NULL, + gcc_jit_param_as_rvalue (param_argv), + zero)), + const_char_ptr_type); + gcc_jit_block_add_eval ( + block, NULL, + gcc_jit_context_new_call (ctxt, + NULL, + func, + 1, args)); + gcc_jit_block_end_with_return (block, NULL, zero); +} + +/* { dg-final { jit-verify-compile-to-file "executable" } } */ +/* { dg-final { jit-verify-executable "hello .*" } } */ diff --git a/gcc/testsuite/jit.dg/test-compile-to-object.c b/gcc/testsuite/jit.dg/test-compile-to-object.c new file mode 100644 index 0000000..1f7fcc6 --- /dev/null +++ b/gcc/testsuite/jit.dg/test-compile-to-object.c @@ -0,0 +1,65 @@ +#include +#include + +#include "libgccjit.h" + +#define TEST_COMPILING_TO_FILE +#define OUTPUT_KIND GCC_JIT_OUTPUT_KIND_OBJECT_FILE +#define OUTPUT_FILENAME "output-of-test-compile-to-object.c.o" +#include "harness.h" + +void +create_code (gcc_jit_context *ctxt, void *user_data) +{ + /* Let's try to inject the equivalent of: + void + hello_world (const char *name) + { + // a test comment + printf ("hello %s\n", name); + } + */ + gcc_jit_type *void_type = + gcc_jit_context_get_type (ctxt, GCC_JIT_TYPE_VOID); + gcc_jit_type *const_char_ptr_type = + gcc_jit_context_get_type (ctxt, GCC_JIT_TYPE_CONST_CHAR_PTR); + gcc_jit_param *param_name = + gcc_jit_context_new_param (ctxt, NULL, const_char_ptr_type, "name"); + gcc_jit_function *func = + gcc_jit_context_new_function (ctxt, NULL, + GCC_JIT_FUNCTION_EXPORTED, + void_type, + "hello_world", + 1, ¶m_name, + 0); + + gcc_jit_param *param_format = + gcc_jit_context_new_param (ctxt, NULL, const_char_ptr_type, "format"); + gcc_jit_function *printf_func = + gcc_jit_context_new_function (ctxt, NULL, + GCC_JIT_FUNCTION_IMPORTED, + gcc_jit_context_get_type ( + ctxt, GCC_JIT_TYPE_INT), + "printf", + 1, ¶m_format, + 1); + gcc_jit_rvalue *args[2]; + args[0] = gcc_jit_context_new_string_literal (ctxt, "hello %s\n"); + args[1] = gcc_jit_param_as_rvalue (param_name); + + gcc_jit_block *block = gcc_jit_function_new_block (func, NULL); + + gcc_jit_block_add_comment ( + block, NULL, + "a test comment"); + + gcc_jit_block_add_eval ( + block, NULL, + gcc_jit_context_new_call (ctxt, + NULL, + printf_func, + 2, args)); + gcc_jit_block_end_with_void_return (block, NULL); +} + +/* { dg-final { jit-verify-compile-to-file "relocatable" } } */ -- 1.8.5.3