[PATCH 0/5] Merger of jit branch (v2)

[PATCH 0/5] Merger of jit branch (v2)

[David Malcolm]

I'd like to merge the JIT branch into trunk:
  https://gcc.gnu.org/wiki/JIT

This is "v2" since it incorporates fixes for the various issues
identified by Joseph in an earlier submission:
  https://gcc.gnu.org/ml/gcc-patches/2014-09/msg02056.html

I've split up the current diff between trunk and the branch into 5
areas for ease of review (and to allow for early merger of the
supporting work, if it's deemed ready):

patch 1: exposes an entrypoint in libiberty that I need
patch 2: configure and Makefile changes in "gcc"
patch 3: timevar.h: Add an auto_timevar class
patch 4: State cleanups in "gcc"
patch 5: Add the "jit" code itself

[this is a diff of trunk r215958 aka
e012cdc775868e9922f5fef9068a764546876d93 which is from 2014-10-06,
vs jit branch version 75b3ee7acdc6de55354d65bb7d619386463e50a1].

I've successfully bootstrapped and regression-tested the cumulative
result of all of the patches against a control build, building them
both with --enable-host-shared, and with
  --enable-languages=c,c++,objc,obj-c++,java,fortran,ada,go,lto
adding ",jit" to the test build (both on x86_64-unknown-linux-gnu;
Fedora 20).

There were no regressions vs the control build, and the patched build
gains a jit.sum, with 4663 passes (and no failures).

OK for trunk?

Overall diffstat follows:

 ChangeLog.jit                                      |   23 +
 MAINTAINERS                                        |    1 +
 contrib/ChangeLog.jit                              |   14 +
 contrib/jit-coverage-report.py                     |   67 +
 gcc/ChangeLog.jit                                  |  323 +
 gcc/Makefile.in                                    |    9 +
 gcc/cgraph.c                                       |   14 +
 gcc/cgraph.h                                       |    6 +
 gcc/cgraphunit.c                                   |   20 +-
 gcc/configure                                      |   54 +-
 gcc/configure.ac                                   |   11 +
 gcc/diagnostic.c                                   |   11 +-
 gcc/doc/install.texi                               |    2 +-
 gcc/dwarf2out.c                                    |   87 +
 gcc/dwarf2out.h                                    |    2 +
 gcc/gcse.c                                         |    9 +
 gcc/gcse.h                                         |    2 +
 gcc/ggc-page.c                                     |    5 +
 gcc/input.c                                        |    2 +-
 gcc/ipa-cp.c                                       |   12 +
 gcc/ipa-pure-const.c                               |  113 +-
 gcc/ipa-reference.c                                |   17 +-
 gcc/ipa-reference.h                                |    1 +
 gcc/java/ChangeLog.jit                             |   14 +
 gcc/jit/ChangeLog                                  |    9 +
 gcc/jit/ChangeLog.jit                              | 3252 ++++++++++
 gcc/jit/Make-lang.in                               |  297 +
 gcc/jit/TODO.rst                                   |  119 +
 gcc/jit/config-lang.in                             |   42 +
 gcc/jit/docs/Makefile                              |  153 +
 gcc/jit/docs/_build/texinfo/Makefile               |   50 +
 gcc/jit/docs/_build/texinfo/factorial.png          |  Bin 0 -> 183838 bytes
 gcc/jit/docs/_build/texinfo/libgccjit.texi         | 6724 ++++++++++++++++++++
 gcc/jit/docs/_build/texinfo/sum-of-squares.png     |  Bin 0 -> 22839 bytes
 gcc/jit/docs/conf.py                               |  258 +
 gcc/jit/docs/examples/install-hello-world.c        |  123 +
 gcc/jit/docs/examples/tut01-square.c               |  107 +
 gcc/jit/docs/examples/tut02-sum-of-squares.c       |  172 +
 gcc/jit/docs/examples/tut03-toyvm/Makefile         |   11 +
 gcc/jit/docs/examples/tut03-toyvm/factorial.toy    |   50 +
 gcc/jit/docs/examples/tut03-toyvm/fibonacci.toy    |   66 +
 gcc/jit/docs/examples/tut03-toyvm/toyvm.c          |  861 +++
 gcc/jit/docs/index.rst                             |   50 +
 gcc/jit/docs/internals/index.rst                   |  157 +
 gcc/jit/docs/intro/factorial.png                   |  Bin 0 -> 183838 bytes
 gcc/jit/docs/intro/index.rst                       |   27 +
 gcc/jit/docs/intro/install.rst                     |  227 +
 gcc/jit/docs/intro/sum-of-squares.png              |  Bin 0 -> 22839 bytes
 gcc/jit/docs/intro/tutorial01.rst                  |  349 +
 gcc/jit/docs/intro/tutorial02.rst                  |  378 ++
 gcc/jit/docs/intro/tutorial03.rst                  | 1108 ++++
 gcc/jit/docs/topics/contexts.rst                   |  315 +
 gcc/jit/docs/topics/expressions.rst                |  524 ++
 gcc/jit/docs/topics/functions.rst                  |  311 +
 gcc/jit/docs/topics/index.rst                      |   30 +
 gcc/jit/docs/topics/locations.rst                  |   69 +
 gcc/jit/docs/topics/objects.rst                    |   86 +
 gcc/jit/docs/topics/results.rst                    |   48 +
 gcc/jit/docs/topics/types.rst                      |  217 +
 gcc/jit/dummy-frontend.c                           |  248 +
 gcc/jit/jit-builtins.c                             |  424 ++
 gcc/jit/jit-builtins.h                             |  114 +
 gcc/jit/jit-common.h                               |  182 +
 gcc/jit/jit-playback.c                             | 2098 ++++++
 gcc/jit/jit-playback.h                             |  564 ++
 gcc/jit/jit-recording.c                            | 3415 ++++++++++
 gcc/jit/jit-recording.h                            | 1593 +++++
 gcc/jit/libgccjit++.h                              | 1574 +++++
 gcc/jit/libgccjit.c                                | 1506 +++++
 gcc/jit/libgccjit.h                                |  977 +++
 gcc/jit/libgccjit.map                              |  100 +
 gcc/jit/libgccjit.pc.in                            |   11 +
 gcc/jit/notes.txt                                  |   84 +
 gcc/main.c                                         |    6 +-
 gcc/params.c                                       |   14 +
 gcc/params.h                                       |    4 +
 gcc/passes.c                                       |    6 +
 gcc/stringpool.c                                   |    5 +
 gcc/testsuite/ChangeLog.jit                        |  601 ++
 gcc/testsuite/jit.dg/all-non-failing-tests.h       |  166 +
 gcc/testsuite/jit.dg/harness.h                     |  242 +
 gcc/testsuite/jit.dg/jit.exp                       |  259 +
 gcc/testsuite/jit.dg/test-accessing-struct.c       |  112 +
 gcc/testsuite/jit.dg/test-accessing-union.c        |   97 +
 gcc/testsuite/jit.dg/test-array-as-pointer.c       |  101 +
 gcc/testsuite/jit.dg/test-arrays.c                 |  165 +
 .../jit.dg/test-calling-external-function.c        |  118 +
 gcc/testsuite/jit.dg/test-calling-function-ptr.c   |  118 +
 gcc/testsuite/jit.dg/test-combination.c            |   67 +
 gcc/testsuite/jit.dg/test-dot-product.c            |  129 +
 gcc/testsuite/jit.dg/test-empty.c                  |   20 +
 .../test-error-accessing-field-in-other-struct.c   |  114 +
 .../jit.dg/test-error-adding-to-terminated-block.c |   48 +
 gcc/testsuite/jit.dg/test-error-array-as-pointer.c |   99 +
 gcc/testsuite/jit.dg/test-error-bad-cast.c         |   63 +
 .../jit.dg/test-error-block-in-wrong-function.c    |   65 +
 ...-error-call-through-ptr-with-mismatching-args.c |   74 +
 ...test-error-call-through-ptr-with-non-function.c |   65 +
 .../test-error-call-through-ptr-with-non-pointer.c |   62 +
 ...t-error-call-through-ptr-with-not-enough-args.c |   70 +
 ...est-error-call-through-ptr-with-too-many-args.c |   87 +
 .../jit.dg/test-error-call-with-mismatching-args.c |   87 +
 .../jit.dg/test-error-call-with-not-enough-args.c  |   87 +
 .../jit.dg/test-error-call-with-too-many-args.c    |   89 +
 .../test-error-dereference-field-of-non-pointer.c  |   95 +
 .../test-error-dereference-read-of-non-pointer.c   |   55 +
 .../jit.dg/test-error-index-not-a-numeric-type.c   |   34 +
 .../test-error-mismatching-types-in-assignment.c   |   61 +
 .../jit.dg/test-error-mismatching-types-in-call.c  |   80 +
 gcc/testsuite/jit.dg/test-error-missing-return.c   |   40 +
 .../jit.dg/test-error-null-passed-to-api.c         |   31 +
 .../test-error-return-within-void-function.c       |   54 +
 .../jit.dg/test-error-unreachable-block.c          |   50 +
 .../jit.dg/test-error-unterminated-block.c         |   42 +
 .../jit.dg/test-error-value-not-a-numeric-type.c   |   29 +
 gcc/testsuite/jit.dg/test-expressions.c            |  896 +++
 gcc/testsuite/jit.dg/test-factorial.c              |  103 +
 gcc/testsuite/jit.dg/test-fibonacci.c              |  136 +
 gcc/testsuite/jit.dg/test-functions.c              |  356 ++
 gcc/testsuite/jit.dg/test-fuzzer.c                 |  462 ++
 gcc/testsuite/jit.dg/test-hello-world.c            |   72 +
 gcc/testsuite/jit.dg/test-linked-list.c            |  141 +
 gcc/testsuite/jit.dg/test-long-names.c             |  112 +
 gcc/testsuite/jit.dg/test-nested-contexts.c        |  641 ++
 gcc/testsuite/jit.dg/test-nested-loops.c           |  179 +
 gcc/testsuite/jit.dg/test-operator-overloading.cc  |  310 +
 gcc/testsuite/jit.dg/test-quadratic.c              |  488 ++
 gcc/testsuite/jit.dg/test-quadratic.cc             |  366 ++
 gcc/testsuite/jit.dg/test-reading-struct.c         |  135 +
 gcc/testsuite/jit.dg/test-string-literal.c         |   52 +
 gcc/testsuite/jit.dg/test-sum-of-squares.c         |  126 +
 gcc/testsuite/jit.dg/test-threads.c                |  252 +
 gcc/testsuite/jit.dg/test-types.c                  |  361 ++
 gcc/testsuite/jit.dg/test-using-global.c           |   73 +
 gcc/testsuite/jit.dg/test-volatile.c               |   66 +
 gcc/timevar.c                                      |    3 +
 gcc/timevar.def                                    |    6 +
 gcc/timevar.h                                      |   24 +
 gcc/toplev.c                                       |   67 +-
 gcc/toplev.h                                       |   19 +-
 include/ChangeLog.jit                              |   11 +
 include/libiberty.h                                |    5 +
 libbacktrace/ChangeLog.jit                         |   14 +
 libcpp/ChangeLog.jit                               |   22 +
 libdecnumber/ChangeLog.jit                         |   22 +
 libiberty/ChangeLog.jit                            |   23 +
 libiberty/choose-temp.c                            |    1 -
 libiberty/functions.texi                           |   13 +-
 libiberty/make-temp-file.c                         |    4 +-
 zlib/ChangeLog.jit                                 |   22 +
 150 files changed, 39206 insertions(+), 87 deletions(-)
 create mode 100644 ChangeLog.jit
 create mode 100644 contrib/ChangeLog.jit
 create mode 100644 contrib/jit-coverage-report.py
 create mode 100644 gcc/ChangeLog.jit
 create mode 100644 gcc/java/ChangeLog.jit
 create mode 100644 gcc/jit/ChangeLog
 create mode 100644 gcc/jit/ChangeLog.jit
 create mode 100644 gcc/jit/Make-lang.in
 create mode 100644 gcc/jit/TODO.rst
 create mode 100644 gcc/jit/config-lang.in
 create mode 100644 gcc/jit/docs/Makefile
 create mode 100644 gcc/jit/docs/_build/texinfo/Makefile
 create mode 100644 gcc/jit/docs/_build/texinfo/factorial.png
 create mode 100644 gcc/jit/docs/_build/texinfo/libgccjit.texi
 create mode 100644 gcc/jit/docs/_build/texinfo/sum-of-squares.png
 create mode 100644 gcc/jit/docs/conf.py
 create mode 100644 gcc/jit/docs/examples/install-hello-world.c
 create mode 100644 gcc/jit/docs/examples/tut01-square.c
 create mode 100644 gcc/jit/docs/examples/tut02-sum-of-squares.c
 create mode 100644 gcc/jit/docs/examples/tut03-toyvm/Makefile
 create mode 100644 gcc/jit/docs/examples/tut03-toyvm/factorial.toy
 create mode 100644 gcc/jit/docs/examples/tut03-toyvm/fibonacci.toy
 create mode 100644 gcc/jit/docs/examples/tut03-toyvm/toyvm.c
 create mode 100644 gcc/jit/docs/index.rst
 create mode 100644 gcc/jit/docs/internals/index.rst
 create mode 100644 gcc/jit/docs/intro/factorial.png
 create mode 100644 gcc/jit/docs/intro/index.rst
 create mode 100644 gcc/jit/docs/intro/install.rst
 create mode 100644 gcc/jit/docs/intro/sum-of-squares.png
 create mode 100644 gcc/jit/docs/intro/tutorial01.rst
 create mode 100644 gcc/jit/docs/intro/tutorial02.rst
 create mode 100644 gcc/jit/docs/intro/tutorial03.rst
 create mode 100644 gcc/jit/docs/topics/contexts.rst
 create mode 100644 gcc/jit/docs/topics/expressions.rst
 create mode 100644 gcc/jit/docs/topics/functions.rst
 create mode 100644 gcc/jit/docs/topics/index.rst
 create mode 100644 gcc/jit/docs/topics/locations.rst
 create mode 100644 gcc/jit/docs/topics/objects.rst
 create mode 100644 gcc/jit/docs/topics/results.rst
 create mode 100644 gcc/jit/docs/topics/types.rst
 create mode 100644 gcc/jit/dummy-frontend.c
 create mode 100644 gcc/jit/jit-builtins.c
 create mode 100644 gcc/jit/jit-builtins.h
 create mode 100644 gcc/jit/jit-common.h
 create mode 100644 gcc/jit/jit-playback.c
 create mode 100644 gcc/jit/jit-playback.h
 create mode 100644 gcc/jit/jit-recording.c
 create mode 100644 gcc/jit/jit-recording.h
 create mode 100644 gcc/jit/libgccjit++.h
 create mode 100644 gcc/jit/libgccjit.c
 create mode 100644 gcc/jit/libgccjit.h
 create mode 100644 gcc/jit/libgccjit.map
 create mode 100644 gcc/jit/libgccjit.pc.in
 create mode 100644 gcc/jit/notes.txt
 create mode 100644 gcc/testsuite/ChangeLog.jit
 create mode 100644 gcc/testsuite/jit.dg/all-non-failing-tests.h
 create mode 100644 gcc/testsuite/jit.dg/harness.h
 create mode 100644 gcc/testsuite/jit.dg/jit.exp
 create mode 100644 gcc/testsuite/jit.dg/test-accessing-struct.c
 create mode 100644 gcc/testsuite/jit.dg/test-accessing-union.c
 create mode 100644 gcc/testsuite/jit.dg/test-array-as-pointer.c
 create mode 100644 gcc/testsuite/jit.dg/test-arrays.c
 create mode 100644 gcc/testsuite/jit.dg/test-calling-external-function.c
 create mode 100644 gcc/testsuite/jit.dg/test-calling-function-ptr.c
 create mode 100644 gcc/testsuite/jit.dg/test-combination.c
 create mode 100644 gcc/testsuite/jit.dg/test-dot-product.c
 create mode 100644 gcc/testsuite/jit.dg/test-empty.c
 create mode 100644 gcc/testsuite/jit.dg/test-error-accessing-field-in-other-struct.c
 create mode 100644 gcc/testsuite/jit.dg/test-error-adding-to-terminated-block.c
 create mode 100644 gcc/testsuite/jit.dg/test-error-array-as-pointer.c
 create mode 100644 gcc/testsuite/jit.dg/test-error-bad-cast.c
 create mode 100644 gcc/testsuite/jit.dg/test-error-block-in-wrong-function.c
 create mode 100644 gcc/testsuite/jit.dg/test-error-call-through-ptr-with-mismatching-args.c
 create mode 100644 gcc/testsuite/jit.dg/test-error-call-through-ptr-with-non-function.c
 create mode 100644 gcc/testsuite/jit.dg/test-error-call-through-ptr-with-non-pointer.c
 create mode 100644 gcc/testsuite/jit.dg/test-error-call-through-ptr-with-not-enough-args.c
 create mode 100644 gcc/testsuite/jit.dg/test-error-call-through-ptr-with-too-many-args.c
 create mode 100644 gcc/testsuite/jit.dg/test-error-call-with-mismatching-args.c
 create mode 100644 gcc/testsuite/jit.dg/test-error-call-with-not-enough-args.c
 create mode 100644 gcc/testsuite/jit.dg/test-error-call-with-too-many-args.c
 create mode 100644 gcc/testsuite/jit.dg/test-error-dereference-field-of-non-pointer.c
 create mode 100644 gcc/testsuite/jit.dg/test-error-dereference-read-of-non-pointer.c
 create mode 100644 gcc/testsuite/jit.dg/test-error-index-not-a-numeric-type.c
 create mode 100644 gcc/testsuite/jit.dg/test-error-mismatching-types-in-assignment.c
 create mode 100644 gcc/testsuite/jit.dg/test-error-mismatching-types-in-call.c
 create mode 100644 gcc/testsuite/jit.dg/test-error-missing-return.c
 create mode 100644 gcc/testsuite/jit.dg/test-error-null-passed-to-api.c
 create mode 100644 gcc/testsuite/jit.dg/test-error-return-within-void-function.c
 create mode 100644 gcc/testsuite/jit.dg/test-error-unreachable-block.c
 create mode 100644 gcc/testsuite/jit.dg/test-error-unterminated-block.c
 create mode 100644 gcc/testsuite/jit.dg/test-error-value-not-a-numeric-type.c
 create mode 100644 gcc/testsuite/jit.dg/test-expressions.c
 create mode 100644 gcc/testsuite/jit.dg/test-factorial.c
 create mode 100644 gcc/testsuite/jit.dg/test-fibonacci.c
 create mode 100644 gcc/testsuite/jit.dg/test-functions.c
 create mode 100644 gcc/testsuite/jit.dg/test-fuzzer.c
 create mode 100644 gcc/testsuite/jit.dg/test-hello-world.c
 create mode 100644 gcc/testsuite/jit.dg/test-linked-list.c
 create mode 100644 gcc/testsuite/jit.dg/test-long-names.c
 create mode 100644 gcc/testsuite/jit.dg/test-nested-contexts.c
 create mode 100644 gcc/testsuite/jit.dg/test-nested-loops.c
 create mode 100644 gcc/testsuite/jit.dg/test-operator-overloading.cc
 create mode 100644 gcc/testsuite/jit.dg/test-quadratic.c
 create mode 100644 gcc/testsuite/jit.dg/test-quadratic.cc
 create mode 100644 gcc/testsuite/jit.dg/test-reading-struct.c
 create mode 100644 gcc/testsuite/jit.dg/test-string-literal.c
 create mode 100644 gcc/testsuite/jit.dg/test-sum-of-squares.c
 create mode 100644 gcc/testsuite/jit.dg/test-threads.c
 create mode 100644 gcc/testsuite/jit.dg/test-types.c
 create mode 100644 gcc/testsuite/jit.dg/test-using-global.c
 create mode 100644 gcc/testsuite/jit.dg/test-volatile.c
 create mode 100644 include/ChangeLog.jit
 create mode 100644 libbacktrace/ChangeLog.jit
 create mode 100644 libcpp/ChangeLog.jit
 create mode 100644 libdecnumber/ChangeLog.jit
 create mode 100644 libiberty/ChangeLog.jit
 create mode 100644 zlib/ChangeLog.jit

-- 
1.8.5.3

[PATCH 3/5] timevar.h: Add an auto_timevar class

[David Malcolm]

This is used in a couple of places in jit/jit-playback.c to ensure
that we pop the timevar on every exit path from a function.

I could rewrite them if need be, but it does simplify things.

Written by Tom Tromey.

gcc/ChangeLog:
	* timevar.h (class auto_timevar): New class.
---
 gcc/timevar.h | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/gcc/timevar.h b/gcc/timevar.h
index 6703cc9..f018e39 100644
--- a/gcc/timevar.h
+++ b/gcc/timevar.h
@@ -110,6 +110,30 @@ timevar_pop (timevar_id_t tv)
     timevar_pop_1 (tv);
 }
 
+// This is a simple timevar wrapper class that pushes a timevar in its
+// constructor and pops the timevar in its destructor.
+class auto_timevar
+{
+ public:
+  auto_timevar (timevar_id_t tv)
+    : m_tv (tv)
+  {
+    timevar_push (m_tv);
+  }
+
+  ~auto_timevar ()
+  {
+    timevar_pop (m_tv);
+  }
+
+ private:
+
+  // Private to disallow copies.
+  auto_timevar (const auto_timevar &);
+
+  timevar_id_t m_tv;
+};
+
 extern void print_time (const char *, long);
 
 #endif /* ! GCC_TIMEVAR_H */
-- 
1.8.5.3

Re: [PATCH 3/5] timevar.h: Add an auto_timevar class

[Richard Biener]

On Mon, Oct 13, 2014 at 7:45 PM, David Malcolm <dmalcolm@redhat.com> wrote:
> This is used in a couple of places in jit/jit-playback.c to ensure
> that we pop the timevar on every exit path from a function.
>
> I could rewrite them if need be, but it does simplify things.

Sorry to be bikeshedding but auto_timevar sounds odd - this is
just a one-element timevar stack.

Don't have a real better name though :/  Maybe timevar_pushpop ?

Otherwise this looks ok.

Thanks,
Richard.

> Written by Tom Tromey.
>
> gcc/ChangeLog:
>         * timevar.h (class auto_timevar): New class.
> ---
>  gcc/timevar.h | 24 ++++++++++++++++++++++++
>  1 file changed, 24 insertions(+)
>
> diff --git a/gcc/timevar.h b/gcc/timevar.h
> index 6703cc9..f018e39 100644
> --- a/gcc/timevar.h
> +++ b/gcc/timevar.h
> @@ -110,6 +110,30 @@ timevar_pop (timevar_id_t tv)
>      timevar_pop_1 (tv);
>  }
>
> +// This is a simple timevar wrapper class that pushes a timevar in its
> +// constructor and pops the timevar in its destructor.
> +class auto_timevar
> +{
> + public:
> +  auto_timevar (timevar_id_t tv)
> +    : m_tv (tv)
> +  {
> +    timevar_push (m_tv);
> +  }
> +
> +  ~auto_timevar ()
> +  {
> +    timevar_pop (m_tv);
> +  }
> +
> + private:
> +
> +  // Private to disallow copies.
> +  auto_timevar (const auto_timevar &);
> +
> +  timevar_id_t m_tv;
> +};
> +
>  extern void print_time (const char *, long);
>
>  #endif /* ! GCC_TIMEVAR_H */
> --
> 1.8.5.3
>

Re: [PATCH 3/5] timevar.h: Add an auto_timevar class

[David Malcolm]

On Tue, 2014-10-14 at 11:03 +0200, Richard Biener wrote:
> On Mon, Oct 13, 2014 at 7:45 PM, David Malcolm <dmalcolm@redhat.com> wrote:
> > This is used in a couple of places in jit/jit-playback.c to ensure
> > that we pop the timevar on every exit path from a function.
> >
> > I could rewrite them if need be, but it does simplify things.
> 
> Sorry to be bikeshedding but auto_timevar sounds odd - this is
> just a one-element timevar stack.

Sorry that the usage examples didn't make it through in my original
email; these are in patch 06/10 in gcc/jit/jit-playback.c and look like
this:

playback::context::
compile ()
{
  ... lots of code...

  {
    auto_timevar assemble_timevar (TV_ASSEMBLE);

    ... lots of code, with multiple return paths...

  }

}

the idea being that the timevar_pop happens implicitly at the exit from
the scope (e.g. via one of the error-handling returns).

FWIW I rather like the current name: I think of it as an RAII-style way
of not having to manually call timevar_pop.

The "auto_" prefix to me evokes both such RAII types as "auto_ptr" and
"auto_vec", and the fact that it's intended to be on the stack i.e. have
"auto" storage class.

> Don't have a real better name though :/  Maybe timevar_pushpop ?
> 
> Otherwise this looks ok.
> 
> Thanks,
> Richard.
> 
> > Written by Tom Tromey.
> >
> > gcc/ChangeLog:
> >         * timevar.h (class auto_timevar): New class.
> > ---
> >  gcc/timevar.h | 24 ++++++++++++++++++++++++
> >  1 file changed, 24 insertions(+)
> >
> > diff --git a/gcc/timevar.h b/gcc/timevar.h
> > index 6703cc9..f018e39 100644
> > --- a/gcc/timevar.h
> > +++ b/gcc/timevar.h
> > @@ -110,6 +110,30 @@ timevar_pop (timevar_id_t tv)
> >      timevar_pop_1 (tv);
> >  }
> >
> > +// This is a simple timevar wrapper class that pushes a timevar in its
> > +// constructor and pops the timevar in its destructor.
> > +class auto_timevar
> > +{
> > + public:
> > +  auto_timevar (timevar_id_t tv)
> > +    : m_tv (tv)
> > +  {
> > +    timevar_push (m_tv);
> > +  }
> > +
> > +  ~auto_timevar ()
> > +  {
> > +    timevar_pop (m_tv);
> > +  }
> > +
> > + private:
> > +
> > +  // Private to disallow copies.
> > +  auto_timevar (const auto_timevar &);
> > +
> > +  timevar_id_t m_tv;
> > +};
> > +
> >  extern void print_time (const char *, long);
> >
> >  #endif /* ! GCC_TIMEVAR_H */
> > --
> > 1.8.5.3
> >

Re: [PATCH 3/5] timevar.h: Add an auto_timevar class

[Richard Biener]

On Tue, Oct 14, 2014 at 5:51 PM, David Malcolm <dmalcolm@redhat.com> wrote:
> On Tue, 2014-10-14 at 11:03 +0200, Richard Biener wrote:
>> On Mon, Oct 13, 2014 at 7:45 PM, David Malcolm <dmalcolm@redhat.com> wrote:
>> > This is used in a couple of places in jit/jit-playback.c to ensure
>> > that we pop the timevar on every exit path from a function.
>> >
>> > I could rewrite them if need be, but it does simplify things.
>>
>> Sorry to be bikeshedding but auto_timevar sounds odd - this is
>> just a one-element timevar stack.
>
> Sorry that the usage examples didn't make it through in my original
> email; these are in patch 06/10 in gcc/jit/jit-playback.c and look like
> this:
>
> playback::context::
> compile ()
> {
>   ... lots of code...
>
>   {
>     auto_timevar assemble_timevar (TV_ASSEMBLE);
>
>     ... lots of code, with multiple return paths...
>
>   }
>
> }
>
> the idea being that the timevar_pop happens implicitly at the exit from
> the scope (e.g. via one of the error-handling returns).
>
> FWIW I rather like the current name: I think of it as an RAII-style way
> of not having to manually call timevar_pop.
>
> The "auto_" prefix to me evokes both such RAII types as "auto_ptr" and
> "auto_vec", and the fact that it's intended to be on the stack i.e. have
> "auto" storage class.

Yeah, I got that.  Anyway, objection withdrawn (ok, it really wasn't an
objection).

The patch is ok.

Thanks,
Richard.

>> Don't have a real better name though :/  Maybe timevar_pushpop ?
>>
>> Otherwise this looks ok.
>>
>> Thanks,
>> Richard.
>>
>> > Written by Tom Tromey.
>> >
>> > gcc/ChangeLog:
>> >         * timevar.h (class auto_timevar): New class.
>> > ---
>> >  gcc/timevar.h | 24 ++++++++++++++++++++++++
>> >  1 file changed, 24 insertions(+)
>> >
>> > diff --git a/gcc/timevar.h b/gcc/timevar.h
>> > index 6703cc9..f018e39 100644
>> > --- a/gcc/timevar.h
>> > +++ b/gcc/timevar.h
>> > @@ -110,6 +110,30 @@ timevar_pop (timevar_id_t tv)
>> >      timevar_pop_1 (tv);
>> >  }
>> >
>> > +// This is a simple timevar wrapper class that pushes a timevar in its
>> > +// constructor and pops the timevar in its destructor.
>> > +class auto_timevar
>> > +{
>> > + public:
>> > +  auto_timevar (timevar_id_t tv)
>> > +    : m_tv (tv)
>> > +  {
>> > +    timevar_push (m_tv);
>> > +  }
>> > +
>> > +  ~auto_timevar ()
>> > +  {
>> > +    timevar_pop (m_tv);
>> > +  }
>> > +
>> > + private:
>> > +
>> > +  // Private to disallow copies.
>> > +  auto_timevar (const auto_timevar &);
>> > +
>> > +  timevar_id_t m_tv;
>> > +};
>> > +
>> >  extern void print_time (const char *, long);
>> >
>> >  #endif /* ! GCC_TIMEVAR_H */
>> > --
>> > 1.8.5.3
>> >
>
>

[PATCH 1/5] libiberty: Expose choose_tmpdir, and fix constness of return type

[David Malcolm]

The jit needs to create a temporary directory, so I need to expose
libiberty's choose_tmpdir as a public entrypoint.

include/ChangeLog:
	* libiberty.h (choose_tmpdir): New prototype.

libiberty/ChangeLog:
	* choose-temp.c (choose_tmpdir): Remove now-redundant local
	copy of prototype.
	* functions.texi: Regenerate.
	* make-temp-file.c (choose_tmpdir): Convert return type from
	char * to const char * - given that this returns a pointer to
	a memoized allocation, the caller must not touch it.
---
 include/libiberty.h        |  5 +++++
 libiberty/choose-temp.c    |  1 -
 libiberty/functions.texi   | 13 ++++++-------
 libiberty/make-temp-file.c |  4 ++--
 4 files changed, 13 insertions(+), 10 deletions(-)

diff --git a/include/libiberty.h b/include/libiberty.h
index bcc1f9a..d09c9a5 100644
--- a/include/libiberty.h
+++ b/include/libiberty.h
@@ -227,6 +227,11 @@ extern char *make_relative_prefix (const char *, const char *,
 extern char *make_relative_prefix_ignore_links (const char *, const char *,
 						const char *) ATTRIBUTE_MALLOC;
 
+/* Returns a pointer to a directory path suitable for creating temporary
+   files in.  */
+
+extern const char *choose_tmpdir (void) ATTRIBUTE_RETURNS_NONNULL;
+
 /* Choose a temporary directory to use for scratch files.  */
 
 extern char *choose_temp_base (void) ATTRIBUTE_MALLOC ATTRIBUTE_RETURNS_NONNULL;
diff --git a/libiberty/choose-temp.c b/libiberty/choose-temp.c
index 0a454cf..8e1e84b 100644
--- a/libiberty/choose-temp.c
+++ b/libiberty/choose-temp.c
@@ -34,7 +34,6 @@ Boston, MA 02110-1301, USA.  */
 #endif
 
 #include "libiberty.h"
-extern char *choose_tmpdir (void);
 
 /* Name of temporary file.
    mktemp requires 6 trailing X's.  */
diff --git a/libiberty/functions.texi b/libiberty/functions.texi
index 9323ff9..387aee0 100644
--- a/libiberty/functions.texi
+++ b/libiberty/functions.texi
@@ -125,7 +125,7 @@ Uses @code{malloc} to allocate storage for @var{nelem} objects of
 
 @end deftypefn
 
-@c choose-temp.c:46
+@c choose-temp.c:45
 @deftypefn Extension char* choose_temp_base (void)
 
 Return a prefix for temporary file names or @code{NULL} if unable to
@@ -139,7 +139,7 @@ not recommended.
 @end deftypefn
 
 @c make-temp-file.c:96
-@deftypefn Replacement char* choose_tmpdir ()
+@deftypefn Replacement const char* choose_tmpdir ()
 
 Returns a pointer to a directory path suitable for creating temporary
 files in.
@@ -160,9 +160,8 @@ number of seconds used.
   @dots{}, @code{NULL})
 
 Concatenate zero or more of strings and return the result in freshly
-@code{xmalloc}ed memory.  Returns @code{NULL} if insufficient memory is
-available.  The argument list is terminated by the first @code{NULL}
-pointer encountered.  Pointers to empty strings are ignored.
+@code{xmalloc}ed memory.  The argument list is terminated by the first
+@code{NULL} pointer encountered.  Pointers to empty strings are ignored.
 
 @end deftypefn
 
@@ -528,7 +527,7 @@ nineteen EBCDIC varying characters is tested; exercise caution.)
 @end ftable
 @end defvr
 
-@c hashtab.c:336
+@c hashtab.c:328
 @deftypefn Supplemental htab_t htab_create_typed_alloc (size_t @var{size}, @
 htab_hash @var{hash_f}, htab_eq @var{eq_f}, htab_del @var{del_f}, @
 htab_alloc @var{alloc_tab_f}, htab_alloc @var{alloc_f}, @
@@ -1163,7 +1162,7 @@ control over the state of the random number generator.
 
 @end deftypefn
 
-@c concat.c:174
+@c concat.c:160
 @deftypefn Extension char* reconcat (char *@var{optr}, const char *@var{s1}, @
   @dots{}, @code{NULL})
 
diff --git a/libiberty/make-temp-file.c b/libiberty/make-temp-file.c
index 7b74f81..244cc23 100644
--- a/libiberty/make-temp-file.c
+++ b/libiberty/make-temp-file.c
@@ -93,7 +93,7 @@ static char *memoized_tmpdir;
 
 /*
 
-@deftypefn Replacement char* choose_tmpdir ()
+@deftypefn Replacement const char* choose_tmpdir ()
 
 Returns a pointer to a directory path suitable for creating temporary
 files in.
@@ -102,7 +102,7 @@ files in.
 
 */
 
-char *
+const char *
 choose_tmpdir (void)
 {
   if (!memoized_tmpdir)
-- 
1.8.5.3

Re: [PATCH 1/5] libiberty: Expose choose_tmpdir, and fix constness of return type

[Jeff Law]

On 10/13/14 11:45, David Malcolm wrote:
> The jit needs to create a temporary directory, so I need to expose
> libiberty's choose_tmpdir as a public entrypoint.
>
> include/ChangeLog:
> 	* libiberty.h (choose_tmpdir): New prototype.
>
> libiberty/ChangeLog:
> 	* choose-temp.c (choose_tmpdir): Remove now-redundant local
> 	copy of prototype.
> 	* functions.texi: Regenerate.
> 	* make-temp-file.c (choose_tmpdir): Convert return type from
> 	char * to const char * - given that this returns a pointer to
> 	a memoized allocation, the caller must not touch it.
OK.
Jeff

Re: [PATCH 1/5] libiberty: Expose choose_tmpdir, and fix constness of return type

[David Malcolm]

On Wed, 2014-10-15 at 11:33 -0600, Jeff Law wrote:
> On 10/13/14 11:45, David Malcolm wrote:
> > The jit needs to create a temporary directory, so I need to expose
> > libiberty's choose_tmpdir as a public entrypoint.
> >
> > include/ChangeLog:
> > 	* libiberty.h (choose_tmpdir): New prototype.
> >
> > libiberty/ChangeLog:
> > 	* choose-temp.c (choose_tmpdir): Remove now-redundant local
> > 	copy of prototype.
> > 	* functions.texi: Regenerate.
> > 	* make-temp-file.c (choose_tmpdir): Convert return type from
> > 	char * to const char * - given that this returns a pointer to
> > 	a memoized allocation, the caller must not touch it.
> OK.

Thanks.  Possibly a dumb question, but are there any special procedures
concerning commits to libiberty, given that it appears to be shared with
other gnu projects?  (I couldn't find any notes on the website about
this)

Re: [PATCH 1/5] libiberty: Expose choose_tmpdir, and fix constness of return type

[DJ Delorie]

As long as you commit to gcc first, or at the same time as other
commits, you'll be fine.  If you commit elsewhere first or only, it
may be reverted by a script.

[PATCH 2/5] gcc: configure and Makefile changes needed by jit

[David Malcolm]

gcc/ChangeLog:
	* configure.ac (gcc_version): Expose this value for use via
	AC_SUBST, since the jit code needs it within the new file
	libgccjit.pc.in.
	(doc_build_sys): New variable, set to "sphinx" if
	sphinx is installed, falling back to "texinfo" otherwise.
	(gcc-driver-name.h): Generate a gcc-driver-name.h file containing
	GCC_DRIVER_NAME for the benefit of jit/jit-playback.c.
	* configure: Regenerate.
	* Makefile.in (doc_build_sys): New.
	(bindir): New.
	(pkgconfigdir): New.
	(installdirs): Add creation of $(DESTDIR)$(pkgconfigdir).
	(site.exp): When constructing site.exp, add a line to set "bindir".
---
 gcc/Makefile.in  |  9 +++++++++
 gcc/configure    | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++++--
 gcc/configure.ac | 11 +++++++++++
 3 files changed, 72 insertions(+), 2 deletions(-)

diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index 1dba76f..f5e3d4c 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -314,6 +314,11 @@ write_entries_to_file = $(shell rm -f $(2) || :) $(shell touch $(2)) \
 			  $(shell expr $(range) + $(write_entries_to_file_split) - 1), $(1))" \
 	     | tr ' ' '\012' >> $(2)))
 
+# The jit documentation looks better if built with sphinx, but can be
+# built with texinfo if sphinx is not available.
+# configure sets "doc_build_sys" to "sphinx" or "texinfo" accordingly
+doc_build_sys=@doc_build_sys@
+
 # --------
 # UNSORTED
 # --------
@@ -565,6 +570,8 @@ bindir = @bindir@
 libdir = @libdir@
 # Directory in which GCC puts its executables.
 libexecdir = @libexecdir@
+# Directory in which to install .pc files for pkgconfig
+pkgconfigdir = @libdir@/pkgconfig
 
 # --------
 # UNSORTED
@@ -3125,6 +3132,7 @@ installdirs:
 	$(mkinstalldirs) $(DESTDIR)$(infodir)
 	$(mkinstalldirs) $(DESTDIR)$(man1dir)
 	$(mkinstalldirs) $(DESTDIR)$(man7dir)
+	$(mkinstalldirs) $(DESTDIR)$(pkgconfigdir)
 
 PLUGIN_HEADERS = $(TREE_H) $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) \
   toplev.h $(DIAGNOSTIC_CORE_H) $(BASIC_BLOCK_H) $(HASH_TABLE_H) \
@@ -3496,6 +3504,7 @@ site.exp: ./config.status Makefile
 	@echo "# add them to the last section" >> ./site.tmp
 	@echo "set rootme \"`${PWD_COMMAND}`\"" >> ./site.tmp
 	@echo "set srcdir \"`cd ${srcdir}; ${PWD_COMMAND}`\"" >> ./site.tmp
+	@echo "set bindir \"`cd ${bindir}; ${PWD_COMMAND}`\"" >> ./site.tmp
 	@echo "set host_triplet $(host)" >> ./site.tmp
 	@echo "set build_triplet $(build)" >> ./site.tmp
 	@echo "set target_triplet $(target)" >> ./site.tmp
diff --git a/gcc/configure b/gcc/configure
index 380a235..81634f2 100755
--- a/gcc/configure
+++ b/gcc/configure
@@ -743,6 +743,7 @@ CXXDEPMODE
 DEPDIR
 am__leading_dot
 CXXCPP
+doc_build_sys
 AR
 NM
 BISON
@@ -824,6 +825,7 @@ build_os
 build_vendor
 build_cpu
 build
+gcc_version
 target_alias
 host_alias
 build_alias
@@ -3040,6 +3042,7 @@ ac_config_headers="$ac_config_headers auto-host.h:config.in"
 
 gcc_version=`cat $srcdir/BASE-VER`
 
+
 # Determine the host, build, and target systems
 ac_aux_dir=
 for ac_dir in "$srcdir" "$srcdir/.." "$srcdir/../.."; do
@@ -8060,6 +8063,47 @@ fi
 
 fi
 
+# The jit documentation looks better if built with sphinx, but can be
+# built with texinfo if sphinx is not available.
+# Set "doc_build_sys" to "sphinx" or "texinfo" accordingly.
+# Extract the first word of "sphinx-build", so it can be a program name with args.
+set dummy sphinx-build; ac_word=$2
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5
+$as_echo_n "checking for $ac_word... " >&6; }
+if test "${ac_cv_prog_doc_build_sys+set}" = set; then :
+  $as_echo_n "(cached) " >&6
+else
+  if test -n "$doc_build_sys"; then
+  ac_cv_prog_doc_build_sys="$doc_build_sys" # Let the user override the test.
+else
+as_save_IFS=$IFS; IFS=$PATH_SEPARATOR
+for as_dir in $PATH
+do
+  IFS=$as_save_IFS
+  test -z "$as_dir" && as_dir=.
+    for ac_exec_ext in '' $ac_executable_extensions; do
+  if { test -f "$as_dir/$ac_word$ac_exec_ext" && $as_test_x "$as_dir/$ac_word$ac_exec_ext"; }; then
+    ac_cv_prog_doc_build_sys="sphinx"
+    $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5
+    break 2
+  fi
+done
+  done
+IFS=$as_save_IFS
+
+  test -z "$ac_cv_prog_doc_build_sys" && ac_cv_prog_doc_build_sys="texinfo"
+fi
+fi
+doc_build_sys=$ac_cv_prog_doc_build_sys
+if test -n "$doc_build_sys"; then
+  { $as_echo "$as_me:${as_lineno-$LINENO}: result: $doc_build_sys" >&5
+$as_echo "$doc_build_sys" >&6; }
+else
+  { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+fi
+
+
 
 # --------------------
 # Checks for C headers
@@ -18049,7 +18093,7 @@ else
   lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
   lt_status=$lt_dlunknown
   cat > conftest.$ac_ext <<_LT_EOF
-#line 18052 "configure"
+#line 18096 "configure"
 #include "confdefs.h"
 
 #if HAVE_DLFCN_H
@@ -18155,7 +18199,7 @@ else
   lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
   lt_status=$lt_dlunknown
   cat > conftest.$ac_ext <<_LT_EOF
-#line 18158 "configure"
+#line 18202 "configure"
 #include "confdefs.h"
 
 #if HAVE_DLFCN_H
@@ -28146,6 +28190,12 @@ _ACEOF
 
 fi
 
+# Generate gcc-driver-name.h containing GCC_DRIVER_NAME for the benefit
+# of jit/jit-playback.c.
+cat > gcc-driver-name.h <<EOF
+#define GCC_DRIVER_NAME "${target_noncanonical}-gcc-${gcc_BASEVER}${exeext}"
+EOF
+
 # Configure the subdirectories
 # AC_CONFIG_SUBDIRS($subdirs)
 
diff --git a/gcc/configure.ac b/gcc/configure.ac
index eb480de..0af7a77 100644
--- a/gcc/configure.ac
+++ b/gcc/configure.ac
@@ -29,6 +29,7 @@ AC_CONFIG_SRCDIR(tree.c)
 AC_CONFIG_HEADER(auto-host.h:config.in)
 
 gcc_version=`cat $srcdir/BASE-VER`
+AC_SUBST(gcc_version)
 
 # Determine the host, build, and target systems
 AC_CANONICAL_BUILD
@@ -971,6 +972,10 @@ else
   AC_CHECK_PROG(AR, ar, ar, ${CONFIG_SHELL-/bin/sh} ${srcdir}/../missing ar)
 fi
 
+# The jit documentation looks better if built with sphinx, but can be
+# built with texinfo if sphinx is not available.
+# Set "doc_build_sys" to "sphinx" or "texinfo" accordingly.
+AC_CHECK_PROG(doc_build_sys, sphinx-build, sphinx, texinfo)
 
 # --------------------
 # Checks for C headers
@@ -5674,6 +5679,12 @@ if test x"${LINKER_HASH_STYLE}" != x; then
                                          [The linker hash style])
 fi
 
+# Generate gcc-driver-name.h containing GCC_DRIVER_NAME for the benefit
+# of jit/jit-playback.c.
+cat > gcc-driver-name.h <<EOF
+#define GCC_DRIVER_NAME "${target_noncanonical}-gcc-${gcc_BASEVER}${exeext}"
+EOF
+
 # Configure the subdirectories
 # AC_CONFIG_SUBDIRS($subdirs)
 
-- 
1.8.5.3

Re: [PATCH 2/5] gcc: configure and Makefile changes needed by jit

[Jeff Law]

On 10/13/14 11:45, David Malcolm wrote:
> gcc/ChangeLog:
> 	* configure.ac (gcc_version): Expose this value for use via
> 	AC_SUBST, since the jit code needs it within the new file
> 	libgccjit.pc.in.
> 	(doc_build_sys): New variable, set to "sphinx" if
> 	sphinx is installed, falling back to "texinfo" otherwise.
> 	(gcc-driver-name.h): Generate a gcc-driver-name.h file containing
> 	GCC_DRIVER_NAME for the benefit of jit/jit-playback.c.
> 	* configure: Regenerate.
> 	* Makefile.in (doc_build_sys): New.
> 	(bindir): New.
> 	(pkgconfigdir): New.
> 	(installdirs): Add creation of $(DESTDIR)$(pkgconfigdir).
> 	(site.exp): When constructing site.exp, add a line to set "bindir".
Mostly OK.  Though a couple questions.

Why do we need pkgconfig and why do you need a bindir in site.exp?

jeff

Re: [PATCH 2/5] gcc: configure and Makefile changes needed by jit

[David Malcolm]

On Wed, 2014-10-15 at 11:36 -0600, Jeff Law wrote:
> On 10/13/14 11:45, David Malcolm wrote:
> > gcc/ChangeLog:
> > 	* configure.ac (gcc_version): Expose this value for use via
> > 	AC_SUBST, since the jit code needs it within the new file
> > 	libgccjit.pc.in.
> > 	(doc_build_sys): New variable, set to "sphinx" if
> > 	sphinx is installed, falling back to "texinfo" otherwise.
> > 	(gcc-driver-name.h): Generate a gcc-driver-name.h file containing
> > 	GCC_DRIVER_NAME for the benefit of jit/jit-playback.c.
> > 	* configure: Regenerate.
> > 	* Makefile.in (doc_build_sys): New.
> > 	(bindir): New.
> > 	(pkgconfigdir): New.
> > 	(installdirs): Add creation of $(DESTDIR)$(pkgconfigdir).
> > 	(site.exp): When constructing site.exp, add a line to set "bindir".
> Mostly OK.  Though a couple questions.
> 
> Why do we need pkgconfig and why do you need a bindir in site.exp?

I chose to provide a libgccjit.pc file in the install, so that client
code has the option of compiling and linking against the library like
this:

$ gcc jit-hello-world.c $(pkg-config libgccjit --cflags)
$ gcc jit-hello-world.o $(pkg-config libgccjit --libs)

relying on pkg-config to automatically supply the relevant flags for the
correct paths (for those people who want to use pkg-config).

To do this, we need to install the libgccjit.pc file into the correct
location for pkg-config to find it; hence we need to create
pkgconfigdir, so that the jit/Make-lang.in has somewhere to install it
to.

See https://gcc.gnu.org/ml/jit/2014-q3/msg00024.html for the commit that
added this.


As for the "bindir" in site.exp, Joseph asked me when the library
invokes a driver to convert from .s to .so to:

On Tue, 2014-09-23 at 23:27 +0000, Joseph S. Myers wrote:
> * use the $(target_noncanonical)-gcc-$(version) name for the
> driver rather than plain "gcc", to maximise the chance that it
> is actually the same compiler the JIT library was built for (I
> realise you may not actually depend on it being the same
> compiler, but that does seem best; in principle in future it
> should be possible to load multiple copies of the JIT library
> to JIT for different targets, so that code for an offload
> accelerator can go through the JIT).
( https://gcc.gnu.org/ml/jit/2014-q3/msg00033.html )

This full name is used when *installing* the driver, but doesn't exist
within the build directory.
Hence when running the library, the installation bindir needs to be in
the PATH.  In particular, (in
https://gcc.gnu.org/ml/jit/2014-q4/msg00005.html ) when running the jit
testsuite we rely on the driver having been installed, and in jit.exp we
need to temporarily prepend the installation bindir onto the front of
PATH when running test programs linked against libgccjit.so.  Hence we
need to know what bindir is from expect, hence we add it to site.exp.

Hope the above sounds sane
Dave

Re: [PATCH 2/5] gcc: configure and Makefile changes needed by jit

[Joseph S. Myers]

On Wed, 15 Oct 2014, David Malcolm wrote:

> As for the "bindir" in site.exp, Joseph asked me when the library
> invokes a driver to convert from .s to .so to:
> 
> On Tue, 2014-09-23 at 23:27 +0000, Joseph S. Myers wrote:
> > * use the $(target_noncanonical)-gcc-$(version) name for the
> > driver rather than plain "gcc", to maximise the chance that it
> > is actually the same compiler the JIT library was built for (I
> > realise you may not actually depend on it being the same
> > compiler, but that does seem best; in principle in future it
> > should be possible to load multiple copies of the JIT library
> > to JIT for different targets, so that code for an offload
> > accelerator can go through the JIT).
> ( https://gcc.gnu.org/ml/jit/2014-q3/msg00033.html )
> 
> This full name is used when *installing* the driver, but doesn't exist
> within the build directory.
> Hence when running the library, the installation bindir needs to be in
> the PATH.  In particular, (in
> https://gcc.gnu.org/ml/jit/2014-q4/msg00005.html ) when running the jit
> testsuite we rely on the driver having been installed, and in jit.exp we
> need to temporarily prepend the installation bindir onto the front of
> PATH when running test programs linked against libgccjit.so.  Hence we
> need to know what bindir is from expect, hence we add it to site.exp.

Even if the driver's been installed, it might not be in the configured 
bindir but in some other DESTDIR.

Really, the need for an installed driver for testing should be avoided. 
The ideal way to do that is for "make check" to install to a staging 
directory within the build directory (generally, the DejaGnu approach of 
passing lots of -B etc. options to tell bits of the toolchain how to find 
each other, with lots of relevant logic hardcoded inside DejaGnu itself, 
is problematic and the staging directory approach would be better, 
although it has various other complications given that GCC needs to find 
separately built / installed pieces such as binutils and runtime 
libraries).  Of course that's a much more general issue - I suppose 
someone with an installation in a DESTDIR can still test as-is by 
specifying a different value of bindir on the "make" command line that 
builds site.exp?

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: [PATCH 2/5] gcc: configure and Makefile changes needed by jit

[Jeff Law]

On 10/15/14 12:25, David Malcolm wrote:
> On Wed, 2014-10-15 at 11:36 -0600, Jeff Law wrote:
>> On 10/13/14 11:45, David Malcolm wrote:
>>> gcc/ChangeLog:
>>> 	* configure.ac (gcc_version): Expose this value for use via
>>> 	AC_SUBST, since the jit code needs it within the new file
>>> 	libgccjit.pc.in.
>>> 	(doc_build_sys): New variable, set to "sphinx" if
>>> 	sphinx is installed, falling back to "texinfo" otherwise.
>>> 	(gcc-driver-name.h): Generate a gcc-driver-name.h file containing
>>> 	GCC_DRIVER_NAME for the benefit of jit/jit-playback.c.
>>> 	* configure: Regenerate.
>>> 	* Makefile.in (doc_build_sys): New.
>>> 	(bindir): New.
>>> 	(pkgconfigdir): New.
>>> 	(installdirs): Add creation of $(DESTDIR)$(pkgconfigdir).
>>> 	(site.exp): When constructing site.exp, add a line to set "bindir".
>> Mostly OK.  Though a couple questions.
>>
>> Why do we need pkgconfig and why do you need a bindir in site.exp?
>
> I chose to provide a libgccjit.pc file in the install, so that client
> code has the option of compiling and linking against the library like
> this:
>
> $ gcc jit-hello-world.c $(pkg-config libgccjit --cflags)
> $ gcc jit-hello-world.o $(pkg-config libgccjit --libs)
WIthout a general consensus to add pkg-config, I'd rather not go this 
direction.  Right now I'd much rather see us adding the appropriate 
flags automatically.


>
> relying on pkg-config to automatically supply the relevant flags for the
> correct paths (for those people who want to use pkg-config).
But I think that is a completely independent discussion and if we go 
that direction we should make it pervasive in GCC and not just for the 
JIT bits.

>
> As for the "bindir" in site.exp, Joseph asked me when the library
> invokes a driver to convert from .s to .so to:
>
> On Tue, 2014-09-23 at 23:27 +0000, Joseph S. Myers wrote:
>> * use the $(target_noncanonical)-gcc-$(version) name for the
>> driver rather than plain "gcc", to maximise the chance that it
>> is actually the same compiler the JIT library was built for (I
>> realise you may not actually depend on it being the same
>> compiler, but that does seem best; in principle in future it
>> should be possible to load multiple copies of the JIT library
>> to JIT for different targets, so that code for an offload
>> accelerator can go through the JIT).
> ( https://gcc.gnu.org/ml/jit/2014-q3/msg00033.html )
>
> This full name is used when *installing* the driver, but doesn't exist
> within the build directory.
> Hence when running the library, the installation bindir needs to be in
> the PATH.  In particular, (in
> https://gcc.gnu.org/ml/jit/2014-q4/msg00005.html ) when running the jit
> testsuite we rely on the driver having been installed, and in jit.exp we
> need to temporarily prepend the installation bindir onto the front of
> PATH when running test programs linked against libgccjit.so.  Hence we
> need to know what bindir is from expect, hence we add it to site.exp.
So if I'm reading all this correctly, what's being implied is that 
testing is done using the installed bits rather than the in-build-tree 
bits?  We really need this to run without having been installed.

jeff

Re: [PATCH 2/5] gcc: configure and Makefile changes needed by jit

[David Malcolm]

On Wed, 2014-10-15 at 14:51 -0600, Jeff Law wrote:
> On 10/15/14 12:25, David Malcolm wrote:
> > On Wed, 2014-10-15 at 11:36 -0600, Jeff Law wrote:
> >> On 10/13/14 11:45, David Malcolm wrote:
> >>> gcc/ChangeLog:
> >>> 	* configure.ac (gcc_version): Expose this value for use via
> >>> 	AC_SUBST, since the jit code needs it within the new file
> >>> 	libgccjit.pc.in.
> >>> 	(doc_build_sys): New variable, set to "sphinx" if
> >>> 	sphinx is installed, falling back to "texinfo" otherwise.
> >>> 	(gcc-driver-name.h): Generate a gcc-driver-name.h file containing
> >>> 	GCC_DRIVER_NAME for the benefit of jit/jit-playback.c.
> >>> 	* configure: Regenerate.
> >>> 	* Makefile.in (doc_build_sys): New.
> >>> 	(bindir): New.
> >>> 	(pkgconfigdir): New.
> >>> 	(installdirs): Add creation of $(DESTDIR)$(pkgconfigdir).
> >>> 	(site.exp): When constructing site.exp, add a line to set "bindir".
> >> Mostly OK.  Though a couple questions.
> >>
> >> Why do we need pkgconfig and why do you need a bindir in site.exp?
> >
> > I chose to provide a libgccjit.pc file in the install, so that client
> > code has the option of compiling and linking against the library like
> > this:
> >
> > $ gcc jit-hello-world.c $(pkg-config libgccjit --cflags)
> > $ gcc jit-hello-world.o $(pkg-config libgccjit --libs)
> WIthout a general consensus to add pkg-config, I'd rather not go this 
> direction.  Right now I'd much rather see us adding the appropriate 
> flags automatically.

How could we do that automatically?  The reason I wanted to use
pkg-config here is not for us, it's for the users of the jit library,
e.g. GNU Octave, my "coconut" jit for CPython etc.  I don't want to
require clients of this code to use the GNU autotools (for example, the
Python bindings don't).

How do *they* select the correct include paths and linker flags for
building against libgccjit?  It's easy if the relevant files got put
into /usr/include and /usr/lib, but there could be multiple versions
installed in different prefixes for different development stacks.
pkg-config solves this by having the user set up PKG_CONFIG_PATH to pick
up the relevant .pc files for each library.

[well, it's also for *me* when I'm using the library, in that I've made
use of the pkg-config approach quite a bit since adding the support]


> > relying on pkg-config to automatically supply the relevant flags for the
> > correct paths (for those people who want to use pkg-config).
> But I think that is a completely independent discussion and if we go 
> that direction we should make it pervasive in GCC and not just for the 
> JIT bits.

I think that the jit is a special case here: I don't know of any other
shared libraries built from the gcc codebase that are intended to run on
the host, and are for use by 3rd-party libraries/binaries.  (though to
be fair, the jit rather blurs the host/build line).

But if this is going to block acceptance of the branch I can drop it;
client code can always just manually specify the -I and -l/-L
accordingly.

> > As for the "bindir" in site.exp, Joseph asked me when the library
> > invokes a driver to convert from .s to .so to:
> >
> > On Tue, 2014-09-23 at 23:27 +0000, Joseph S. Myers wrote:
> >> * use the $(target_noncanonical)-gcc-$(version) name for the
> >> driver rather than plain "gcc", to maximise the chance that it
> >> is actually the same compiler the JIT library was built for (I
> >> realise you may not actually depend on it being the same
> >> compiler, but that does seem best; in principle in future it
> >> should be possible to load multiple copies of the JIT library
> >> to JIT for different targets, so that code for an offload
> >> accelerator can go through the JIT).
> > ( https://gcc.gnu.org/ml/jit/2014-q3/msg00033.html )
> >
> > This full name is used when *installing* the driver, but doesn't exist
> > within the build directory.
> > Hence when running the library, the installation bindir needs to be in
> > the PATH.  In particular, (in
> > https://gcc.gnu.org/ml/jit/2014-q4/msg00005.html ) when running the jit
> > testsuite we rely on the driver having been installed, and in jit.exp we
> > need to temporarily prepend the installation bindir onto the front of
> > PATH when running test programs linked against libgccjit.so.  Hence we
> > need to know what bindir is from expect, hence we add it to site.exp.
> So if I'm reading all this correctly, what's being implied is that 
> testing is done using the installed bits rather than the in-build-tree 
> bits?  We really need this to run without having been installed.

One approach here might be to do make a copy of the driver binary with
the final name within the *build* dir (e.g.
"x86_64-unknown-linux-gnu-gcc-5.0.0").

Another might be the "run the driver code in-process" approach I dabbled
with here:
  https://gcc.gnu.org/ml/jit/2014-q3/msg00049.html
though that's some way from working, and is more invasive (no-one
replied to that email)

[PATCH] Avoid the need to install when running the jit testsuite

[David Malcolm]

On Wed, 2014-10-15 at 17:29 -0400, David Malcolm wrote:
> On Wed, 2014-10-15 at 14:51 -0600, Jeff Law wrote:
> > On 10/15/14 12:25, David Malcolm wrote:
> > > On Wed, 2014-10-15 at 11:36 -0600, Jeff Law wrote:
> > >> On 10/13/14 11:45, David Malcolm wrote:
> > >>> gcc/ChangeLog:
> > >>>         * configure.ac (gcc_version): Expose this value for use via
> > >>>         AC_SUBST, since the jit code needs it within the new file
> > >>>         libgccjit.pc.in.
> > >>>         (doc_build_sys): New variable, set to "sphinx" if
> > >>>         sphinx is installed, falling back to "texinfo" otherwise.
> > >>>         (gcc-driver-name.h): Generate a gcc-driver-name.h file containing
> > >>>         GCC_DRIVER_NAME for the benefit of jit/jit-playback.c.
> > >>>         * configure: Regenerate.
> > >>>         * Makefile.in (doc_build_sys): New.
> > >>>         (bindir): New.
> > >>>         (pkgconfigdir): New.
> > >>>         (installdirs): Add creation of $(DESTDIR)$(pkgconfigdir).
> > >>>         (site.exp): When constructing site.exp, add a line to set "bindir".
> > >> Mostly OK.  Though a couple questions.
> > >>
> > >> Why do we need pkgconfig and why do you need a bindir in site.exp?
> 
[...snip pkg-config discussion...]

> > As for the "bindir" in site.exp, Joseph asked me when the library
> > > invokes a driver to convert from .s to .so to:
> > >
> > > On Tue, 2014-09-23 at 23:27 +0000, Joseph S. Myers wrote:
> > >> * use the $(target_noncanonical)-gcc-$(version) name for the
> > >> driver rather than plain "gcc", to maximise the chance that it
> > >> is actually the same compiler the JIT library was built for (I
> > >> realise you may not actually depend on it being the same
> > >> compiler, but that does seem best; in principle in future it
> > >> should be possible to load multiple copies of the JIT library
> > >> to JIT for different targets, so that code for an offload
> > >> accelerator can go through the JIT).
> > > ( https://gcc.gnu.org/ml/jit/2014-q3/msg00033.html )
> > >
> > > This full name is used when *installing* the driver, but doesn't exist
> > > within the build directory.
> > > Hence when running the library, the installation bindir needs to be in
> > > the PATH.  In particular, (in
> > > https://gcc.gnu.org/ml/jit/2014-q4/msg00005.html ) when running the jit
> > > testsuite we rely on the driver having been installed, and in jit.exp we
> > > need to temporarily prepend the installation bindir onto the front of
> > > PATH when running test programs linked against libgccjit.so.  Hence we
> > > need to know what bindir is from expect, hence we add it to site.exp.
> > So if I'm reading all this correctly, what's being implied is that 
> > testing is done using the installed bits rather than the in-build-tree 
> > bits?  We really need this to run without having been installed.
> 
> One approach here might be to do make a copy of the driver binary with
> the final name within the *build* dir (e.g.
> "x86_64-unknown-linux-gnu-gcc-5.0.0").
> 
> Another might be the "run the driver code in-process" approach I dabbled
> with here:
>   https://gcc.gnu.org/ml/jit/2014-q3/msg00049.html
> though that's some way from working, and is more invasive (no-one
> replied to that email)

The first approach seems to work well, and avoids the need to
"make install" before running the testsuite.

Jeff, Joseph: how does the following look?  (not yet committed to the
branch)

Avoid discrepancy between the installed name of the driver as used
by libgccjit (e.g. "x86_64-unknown-linux-gnu-gcc-5.0.0") and that
within the builddir by adding a simple symlink to xgcc within the
builddir, with the install-time name.

When running the testsuite, set PATH and LIBRARY_PATH when invoking
built binaries that use libgccjit so that it can find the driver, and so
that when the driver invokes the linker, the linker can find libgcc,
libgcc_s, crtbeginS.o, etc, finding them all in the builddir via "xgcc"
via libgloss.

This avoids the need for installation when running the testsuite, whilst
avoiding "baking in" any paths into the built library.

This patch doesn't touch the documentation (I plan to do that as a
followup).

gcc/ChangeLog.jit:
	* Makefile.in (FULL_DRIVER_NAME): New variable, adapted from the
	install-driver target.  New target, a symlink within the builddir,
	linked to "xgcc", for use when running the JIT library from the
	builddir.
	(MOSTLYCLEANFILES): Add FULL_DRIVER_NAME.
	(install-driver): Use $(FULL_DRIVER_NAME) rather than spelling it
	out.
	(site.exp): Don't set "bindir", as this is no longer needed when
	running the jit testsuite.

gcc/jit/ChangeLog.jit:
	* Make-lang.in (jit): Add $(FULL_DRIVER_NAME) as a dependency, so
	that the symlink is created for testing.

	* jit-playback.c (gcc::jit::playback::context::compile): Add
	"-fno-use-linker-plugin" when invoking the driver.  Update error
	messages to talk about the "gcc driver" rather than the
	"gcc harness".  To ease troubleshooting, add error messages giving
	the driver name and PATH to the error-handling code that fires
	when the driver can't be found.

gcc/testsuite/ChangeLog.jit:
	* jit.dg/jit.exp (get_path_of_driver): New procedure.
	(jit-dg-test): Don't unsetenv GCC_EXEC_PREFIX, since jit-playback.c
	now adds -fno-use-linker-plugin to the driver cmdline sidestepping
	the builddir/installdir libtto_plugin naming issue.
	When setting up PATH so that the JIT library can invoke the driver
	by installation name, don't use the installation "bindir".
	Instead, simply use the location of xgcc as detected
	get_path_of_driver.  In addition, set up LIBRARY_PATH so that the
	linker run from inside the JIT library can locate libgcc etc when
	building the .so, pointing it at the same directory.
---
 gcc/Makefile.in              | 16 +++++++---
 gcc/jit/Make-lang.in         |  5 +++-
 gcc/jit/jit-playback.c       | 24 ++++++++++++---
 gcc/testsuite/jit.dg/jit.exp | 69 +++++++++++++++++++++++++++++++-------------
 4 files changed, 85 insertions(+), 29 deletions(-)

diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index f5e3d4c..523d1db 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -1504,6 +1504,9 @@ BACKEND = libbackend.a main.o @TREEBROWSER@ libcommon-target.a libcommon.a \
 # front-end checking.
 TREECHECKING = @TREECHECKING@
 
+# The full name of the driver on installation
+FULL_DRIVER_NAME=$(target_noncanonical)-gcc-$(version)$(exeext)
+
 MOSTLYCLEANFILES = insn-flags.h insn-config.h insn-codes.h \
  insn-output.c insn-recog.c insn-emit.c insn-extract.c insn-peep.c \
  insn-attr.h insn-attr-common.h insn-attrtab.c insn-dfatab.c \
@@ -1511,7 +1514,7 @@ MOSTLYCLEANFILES = insn-flags.h insn-config.h insn-codes.h \
  tm-preds.h tm-constrs.h checksum-options \
  tree-check.h min-insn-modes.c insn-modes.c insn-modes.h \
  genrtl.h gt-*.h gtype-*.h gtype-desc.c gtyp-input.list \
- xgcc$(exeext) cpp$(exeext) \
+ xgcc$(exeext) cpp$(exeext) $(FULL_DRIVER_NAME) \
  $(EXTRA_PROGRAMS) gcc-cross$(exeext) \
  $(SPECS) collect2$(exeext) gcc-ar$(exeext) gcc-nm$(exeext) \
  gcc-ranlib$(exeext) \
@@ -1520,6 +1523,12 @@ MOSTLYCLEANFILES = insn-flags.h insn-config.h insn-codes.h \
  gengtype$(exeext) *.[0-9][0-9].* *.[si] *-checksum.c libbackend.a \
  libcommon-target.a libcommon.a libgcc.mk
 
+# This symlink makes the full installation name of the driver be available
+# from within the *build* directory, for use when running the JIT library
+# from there (e.g. when running its testsuite).
+$(FULL_DRIVER_NAME): ./xgcc
+	$(LN) -s $< $@
+
 #\f
 # Language makefile fragments.
 
@@ -3248,9 +3257,9 @@ install-driver: installdirs xgcc$(exeext)
 	-rm -f $(DESTDIR)$(bindir)/$(GCC_INSTALL_NAME)$(exeext)
 	-$(INSTALL_PROGRAM) xgcc$(exeext) $(DESTDIR)$(bindir)/$(GCC_INSTALL_NAME)$(exeext)
 	-if [ "$(GCC_INSTALL_NAME)" != "$(target_noncanonical)-gcc-$(version)" ]; then \
-	  rm -f $(DESTDIR)$(bindir)/$(target_noncanonical)-gcc-$(version)$(exeext); \
+	  rm -f $(DESTDIR)$(bindir)/$(FULL_DRIVER_NAME); \
 	  ( cd $(DESTDIR)$(bindir) && \
-	    $(LN) $(GCC_INSTALL_NAME)$(exeext) $(target_noncanonical)-gcc-$(version)$(exeext) ); \
+	    $(LN) $(GCC_INSTALL_NAME)$(exeext) $(FULL_DRIVER_NAME) ); \
 	fi
 	-if [ ! -f gcc-cross$(exeext) ] \
 	    && [ "$(GCC_INSTALL_NAME)" != "$(GCC_TARGET_INSTALL_NAME)" ]; then \
@@ -3504,7 +3513,6 @@ site.exp: ./config.status Makefile
 	@echo "# add them to the last section" >> ./site.tmp
 	@echo "set rootme \"`${PWD_COMMAND}`\"" >> ./site.tmp
 	@echo "set srcdir \"`cd ${srcdir}; ${PWD_COMMAND}`\"" >> ./site.tmp
-	@echo "set bindir \"`cd ${bindir}; ${PWD_COMMAND}`\"" >> ./site.tmp
 	@echo "set host_triplet $(host)" >> ./site.tmp
 	@echo "set build_triplet $(build)" >> ./site.tmp
 	@echo "set target_triplet $(target)" >> ./site.tmp
diff --git a/gcc/jit/Make-lang.in b/gcc/jit/Make-lang.in
index 3115b1d..ac179f4 100644
--- a/gcc/jit/Make-lang.in
+++ b/gcc/jit/Make-lang.in
@@ -51,7 +51,10 @@ LIBGCCJIT_FILENAME = \
 LIBGCCJIT_LINKER_NAME_SYMLINK = $(LIBGCCJIT_LINKER_NAME)
 LIBGCCJIT_SONAME_SYMLINK = $(LIBGCCJIT_SONAME)
 
-jit: $(LIBGCCJIT_FILENAME) $(LIBGCCJIT_SYMLINK) $(LIBGCCJIT_LINKER_NAME_SYMLINK)
+jit: $(LIBGCCJIT_FILENAME) \
+	$(LIBGCCJIT_SYMLINK) \
+	$(LIBGCCJIT_LINKER_NAME_SYMLINK) \
+	$(FULL_DRIVER_NAME)
 
 # Tell GNU make to ignore these if they exist.
 .PHONY: jit
diff --git a/gcc/jit/jit-playback.c b/gcc/jit/jit-playback.c
index 6937408..74ddccc 100644
--- a/gcc/jit/jit-playback.c
+++ b/gcc/jit/jit-playback.c
@@ -1631,7 +1631,7 @@ compile ()
   {
     auto_timevar assemble_timevar (TV_ASSEMBLE);
     const char *errmsg;
-    const char *argv[6];
+    const char *argv[7];
     int exit_status = 0;
     int err = 0;
     const char *gcc_driver_name = GCC_DRIVER_NAME;
@@ -1643,8 +1643,18 @@ compile ()
     /* The output: shared library.  */
     argv[3] = "-o";
     argv[4] = m_path_so_file;
+
+    /* Don't use the linker plugin.
+       If running with just a "make" and not a "make install", then we'd
+       run into
+          "fatal error: -fuse-linker-plugin, but liblto_plugin.so not found"
+       libto_plugin is a .la at build time, with it becoming installed with
+       ".so" suffix: i.e. it doesn't exist with a .so suffix until install
+       time.  */
+    argv[5] = "-fno-use-linker-plugin";
+
     /* pex argv arrays are NULL-terminated.  */
-    argv[5] = NULL;
+    argv[6] = NULL;
 
     errmsg = pex_one (PEX_SEARCH, /* int flags, */
 		      gcc_driver_name,
@@ -1656,7 +1666,7 @@ compile ()
 		      &err); /* int *err*/
     if (errmsg)
       {
-	add_error (NULL, "error invoking gcc harness: %s", errmsg);
+	add_error (NULL, "error invoking gcc driver: %s", errmsg);
 	return NULL;
       }
 
@@ -1665,8 +1675,14 @@ compile ()
     if (exit_status || err)
       {
 	add_error (NULL,
-		   "error invoking gcc harness: exit_status: %i err: %i",
+		   "error invoking gcc driver: exit_status: %i err: %i",
 		   exit_status, err);
+	add_error (NULL,
+		   "whilst attempting to run a driver named: %s",
+		   gcc_driver_name);
+	add_error (NULL,
+		   "PATH was: %s",
+		   getenv ("PATH"));
 	return NULL;
       }
   }
diff --git a/gcc/testsuite/jit.dg/jit.exp b/gcc/testsuite/jit.dg/jit.exp
index 76a1d9d..c85e2a5 100644
--- a/gcc/testsuite/jit.dg/jit.exp
+++ b/gcc/testsuite/jit.dg/jit.exp
@@ -172,6 +172,18 @@ set tests [lsort [concat $tests [find $srcdir/../jit/docs/examples *.c]]]
 
 verbose "tests: $tests"
 
+# libgloss has found the driver (as "xgcc" or "gcc) and stored
+# its full path as GCC_UNDER_TEST.
+proc get_path_of_driver {} {
+    global GCC_UNDER_TEST
+
+    verbose "GCC_UNDER_TEST: $GCC_UNDER_TEST"
+    set binary [lindex $GCC_UNDER_TEST 0]
+    verbose "binary: $binary"
+
+    return [file dirname $binary]
+}
+
 proc jit-dg-test { prog do_what extra_tool_flags } {
     verbose "within jit-dg-test..."
     verbose "  prog: $prog"
@@ -206,31 +218,41 @@ proc jit-dg-test { prog do_what extra_tool_flags } {
     set ld_library_path "$base_dir/../../"
     set_ld_library_path_env_vars
 
-    # If running with just a "make" and not a "make install", then I was
-    # running into
-    #   "fatal error: -fuse-linker-plugin, but liblto_plugin.so not found"
-    # emitted by the inner gcc invoked to convert the .s into .so
-    # This appears to be due to not installing the built compiler;
-    # libto_plugin is a .la at build time, with the .so becoming installed
-    # at install time; the "set_ld_library_path_env_vars" function from
-    # target-libpath.exp that I'm using to set LD_LIBRARY_PATH to find
-    # the library under test, libgccjit.so, was setting GCC_EXEC_PREFIX to
-    # the builddir, thus picking up the built-but-not-installed toolchain.
-    # Hacking in an "unsetenv GCC_EXEC_PREFIX" here fixes the issue,
-    # allowing quick running of testsuite without needing a full install.
-    #
-    unsetenv GCC_EXEC_PREFIX
-
     # libgccjit uses the driver to convert .s files to .so libraries
-    # via its *installed* name, the expansion of:
+    # via its *installed* name, FULL_DRIVER_NAME
     #   ${target_noncanonical}-gcc-${gcc_BASEVER}${exeext}
     # e.g. "x86_64-unknown-linux-gnu-gcc-5.0.0"
-    # looking for it on PATH.  Hence we need to prepend the installation
-    # bindir to PATH when running the tests
+    # looking for it on PATH.  Hence we need to prepend the location of
+    # that executable to PATH when running the tests
+    set dir_containing_driver [get_path_of_driver ]
+    verbose "dir_containing_driver: $dir_containing_driver"
     global env
-    global bindir
     set old_path $env(PATH)
-    setenv "PATH" $bindir:$env(PATH)
+    setenv "PATH" $dir_containing_driver:$old_path
+    verbose -log "PATH=[getenv PATH]"
+
+    # We have:
+    #   test-executables
+    #     linked to -> libgccjit.so
+    #                    -> invokes driver:
+    #                         -> invokes the assembler
+    #                         -> invokes the linker
+    # We want to be able to run this from the builddir without installing
+    # but the linker needs to be able to locate various libraries, or we
+    # get:
+    #   ld: cannot find crtbeginS.o: No such file or directory
+    #   ld: cannot find -lgcc
+    #   ld: cannot find -lgcc_s
+    # These can be found in the "gcc" subdir of the build.
+    # Hence to be able to run the testsuite without installing, we need
+    # to set or prepend the "gcc" subdir of the build to LIBRARY_PATH:
+    if { [info exists env(LIBRARY_PATH) ] } {
+	set old_library_path $env(LIBRARY_PATH)
+	setenv "LIBRARY_PATH" $dir_containing_driver:$old_library_path
+    } else {
+	setenv "LIBRARY_PATH" $dir_containing_driver
+    }
+    verbose -log "LIBRARY_PATH=[getenv LIBRARY_PATH]"
 
     # dejagnu.exp's host_execute has code to scrape out test results
     # from the DejaGnu C API and bring back into the tcl world, so we
@@ -245,6 +267,13 @@ proc jit-dg-test { prog do_what extra_tool_flags } {
     # Restore PATH
     setenv "PATH" $old_path
 
+    # Restore LIBRARY_PATH
+    if { [info exists old_library_path] } {
+	setenv "LIBRARY_PATH" $old_library_path
+    } else {
+	unsetenv "LIBRARY_PATH"
+    }
+
     restore_ld_library_path_env_vars
 }
 
-- 
1.8.5.3

Re: [PATCH] Avoid the need to install when running the jit testsuite

[Joseph S. Myers]

On Fri, 17 Oct 2014, David Malcolm wrote:

> +# This symlink makes the full installation name of the driver be available
> +# from within the *build* directory, for use when running the JIT library
> +# from there (e.g. when running its testsuite).
> +$(FULL_DRIVER_NAME): ./xgcc
> +	$(LN) -s $< $@

I believe $(LN_S) would be normal, though (a) I don't see it being used 
anywhere, despite the definition, and (b) while the GNU Coding Standards 
still say "If you use symbolic links, you should implement a fallback for 
systems that don't have symbolic links." I'm doubtful that's of practical 
relevance to systems people are building GCC on any more.

I don't have any comments on the other parts of this patch.

-- 
Joseph S. Myers
joseph@codesourcery.com

[jit] Drop libgccjit.pc

[David Malcolm]

Committed to branch dmalcolm/jit:

pkg-config appears to be controversial, so don't provide a .pc file.

gcc/ChangeLog.jit:
	* Makefile.in (pkgconfigdir): Drop this.
	(installdirs): Likewise.
	* configure.ac (gcc_version): Don't AC_SUBST this.
	* configure: Regenerate.

gcc/jit/ChangeLog.jit:
	* Make-lang.in (jit.install-common): Drop installation of
	libgccjit.pc.
	* config-lang.in (outputs): Drop jit/libgccjit.pc.
	* libgccjit.pc.in: Delete.
---
 gcc/ChangeLog.jit       |  7 +++++++
 gcc/Makefile.in         |  3 ---
 gcc/configure           |  6 ++----
 gcc/configure.ac        |  1 -
 gcc/jit/ChangeLog.jit   |  7 +++++++
 gcc/jit/Make-lang.in    |  2 --
 gcc/jit/config-lang.in  |  4 ----
 gcc/jit/libgccjit.pc.in | 11 -----------
 8 files changed, 16 insertions(+), 25 deletions(-)
 delete mode 100644 gcc/jit/libgccjit.pc.in

diff --git a/gcc/ChangeLog.jit b/gcc/ChangeLog.jit
index bf2d6d2..8dec312 100644
--- a/gcc/ChangeLog.jit
+++ b/gcc/ChangeLog.jit
@@ -1,3 +1,10 @@
+2014-10-20  David Malcolm  <dmalcolm@redhat.com>
+
+	* Makefile.in (pkgconfigdir): Drop this.
+	(installdirs): Likewise.
+	* configure.ac (gcc_version): Don't AC_SUBST this.
+	* configure: Regenerate.
+
 2014-10-17  David Malcolm  <dmalcolm@redhat.com>
 
 	* Makefile.in (FULL_DRIVER_NAME): New variable, adapted from the
diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index 523d1db..954a1eb 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -570,8 +570,6 @@ bindir = @bindir@
 libdir = @libdir@
 # Directory in which GCC puts its executables.
 libexecdir = @libexecdir@
-# Directory in which to install .pc files for pkgconfig
-pkgconfigdir = @libdir@/pkgconfig
 
 # --------
 # UNSORTED
@@ -3141,7 +3139,6 @@ installdirs:
 	$(mkinstalldirs) $(DESTDIR)$(infodir)
 	$(mkinstalldirs) $(DESTDIR)$(man1dir)
 	$(mkinstalldirs) $(DESTDIR)$(man7dir)
-	$(mkinstalldirs) $(DESTDIR)$(pkgconfigdir)
 
 PLUGIN_HEADERS = $(TREE_H) $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) \
   toplev.h $(DIAGNOSTIC_CORE_H) $(BASIC_BLOCK_H) $(HASH_TABLE_H) \
diff --git a/gcc/configure b/gcc/configure
index 81634f2..0024ece 100755
--- a/gcc/configure
+++ b/gcc/configure
@@ -825,7 +825,6 @@ build_os
 build_vendor
 build_cpu
 build
-gcc_version
 target_alias
 host_alias
 build_alias
@@ -3042,7 +3041,6 @@ ac_config_headers="$ac_config_headers auto-host.h:config.in"
 
 gcc_version=`cat $srcdir/BASE-VER`
 
-
 # Determine the host, build, and target systems
 ac_aux_dir=
 for ac_dir in "$srcdir" "$srcdir/.." "$srcdir/../.."; do
@@ -18093,7 +18091,7 @@ else
   lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
   lt_status=$lt_dlunknown
   cat > conftest.$ac_ext <<_LT_EOF
-#line 18096 "configure"
+#line 18094 "configure"
 #include "confdefs.h"
 
 #if HAVE_DLFCN_H
@@ -18199,7 +18197,7 @@ else
   lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
   lt_status=$lt_dlunknown
   cat > conftest.$ac_ext <<_LT_EOF
-#line 18202 "configure"
+#line 18200 "configure"
 #include "confdefs.h"
 
 #if HAVE_DLFCN_H
diff --git a/gcc/configure.ac b/gcc/configure.ac
index 0af7a77..37db6ab 100644
--- a/gcc/configure.ac
+++ b/gcc/configure.ac
@@ -29,7 +29,6 @@ AC_CONFIG_SRCDIR(tree.c)
 AC_CONFIG_HEADER(auto-host.h:config.in)
 
 gcc_version=`cat $srcdir/BASE-VER`
-AC_SUBST(gcc_version)
 
 # Determine the host, build, and target systems
 AC_CANONICAL_BUILD
diff --git a/gcc/jit/ChangeLog.jit b/gcc/jit/ChangeLog.jit
index 0c55258..9a36dfd 100644
--- a/gcc/jit/ChangeLog.jit
+++ b/gcc/jit/ChangeLog.jit
@@ -1,3 +1,10 @@
+2014-10-20  David Malcolm  <dmalcolm@redhat.com>
+
+	* Make-lang.in (jit.install-common): Drop installation of
+	libgccjit.pc.
+	* config-lang.in (outputs): Drop jit/libgccjit.pc.
+	* libgccjit.pc.in: Delete.
+
 2014-10-17  David Malcolm  <dmalcolm@redhat.com>
 
 	* Make-lang.in (jit): Add $(FULL_DRIVER_NAME) as a dependency, so
diff --git a/gcc/jit/Make-lang.in b/gcc/jit/Make-lang.in
index ac179f4..167fcad 100644
--- a/gcc/jit/Make-lang.in
+++ b/gcc/jit/Make-lang.in
@@ -260,8 +260,6 @@ jit.install-common: installdirs
 	  $(DESTDIR)/$(includedir)/libgccjit.h
 	$(INSTALL_PROGRAM) $(srcdir)/jit/libgccjit++.h \
 	  $(DESTDIR)/$(includedir)/libgccjit++.h
-	$(INSTALL_PROGRAM) jit/libgccjit.pc \
-	  $(DESTDIR)/$(libdir)/pkgconfig/libgccjit.pc
 
 jit.install-man:
 
diff --git a/gcc/jit/config-lang.in b/gcc/jit/config-lang.in
index b22a5ee..7a32afe 100644
--- a/gcc/jit/config-lang.in
+++ b/gcc/jit/config-lang.in
@@ -36,7 +36,3 @@ gtfiles="\$(srcdir)/jit/dummy-frontend.c"
 # Hence to get the jit, one must configure with:
 #   --enable-host-shared --enable-languages=jit
 build_by_default="no"
-
-# Ensure that libgccjit.pc is built from libgccjit.pc.in
-# via AC_CONFIG_FILES in gcc/configure.ac
-outputs="jit/libgccjit.pc"
diff --git a/gcc/jit/libgccjit.pc.in b/gcc/jit/libgccjit.pc.in
deleted file mode 100644
index faafea5..0000000
--- a/gcc/jit/libgccjit.pc.in
+++ /dev/null
@@ -1,11 +0,0 @@
-prefix=@prefix@
-exec_prefix=@exec_prefix@
-libdir=@libdir@
-includedir=@includedir@
-
-Name: libgccjit
-Description: Library for embedding GCC for Just-In-Time compilation
-URL: https://gcc.gnu.org/wiki/JIT
-Version: @gcc_version@
-Cflags: -I${includedir}
-Libs: -lgccjit -L${libdir}
-- 
1.7.11.7

Re: [jit] Drop libgccjit.pc

[Basile Starynkevitch]

On Mon, 2014-10-20 at 13:54 -0400, David Malcolm wrote:
> Committed to branch dmalcolm/jit:
> 
> pkg-config appears to be controversial, so don't provide a .pc file.


I would put it under contrib/; it is controversial, but some would like
to have it.

Cheers.

-- 
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mine, sont seulement les miennes} ***

Re: [jit] Drop libgccjit.pc

[Matthias Klose]

Am 20.10.2014 um 22:11 schrieb Basile Starynkevitch:
> On Mon, 2014-10-20 at 13:54 -0400, David Malcolm wrote:
>> Committed to branch dmalcolm/jit:
>>
>> pkg-config appears to be controversial, so don't provide a .pc file.
> 
> 
> I would put it under contrib/; it is controversial, but some would like
> to have it.

please don't. if it's not always installed, then you have to check for a fall
back in any case.

Matthias

Re: [jit] Drop libgccjit.pc

[David Malcolm]

On Mon, 2014-10-20 at 22:11 +0200, Basile Starynkevitch wrote:
> On Mon, 2014-10-20 at 13:54 -0400, David Malcolm wrote:
> > Committed to branch dmalcolm/jit:
> > 
> > pkg-config appears to be controversial, so don't provide a .pc file.
> 
> 
> I would put it under contrib/; it is controversial, but some would like
> to have it.

It was generated, via some Makefile.in and configure.ac hooks, so I
don't think putting it under contrib/ allows us to sidestep that.

(FWIW, I'm one of the ones who'd like to have a .pc file, but I'd rather
focus on getting the jit in before stage1 closes)

Thanks
Dave

[PATCH 4/5] State cleanups

[David Malcolm]

This patch contains various cleanups needed by the jit, so that we can
rerun the compiler code repeatedly within the same process:

* Introduce per-file finalization routines to clean up file-static
  variables.

* Make various initializations idempotent (init_ggc), or cleanup their
  previous state (init_stringpool).

* Introduce a "class toplev", so that we can easy control whether
  timevars are reset each time, or run in a "cumulative" mode.

gcc/ChangeLog:
	* cgraph.c (cgraph_c_finalize): New function.
	* cgraph.h (cgraph_c_finalize): New prototype.
	(cgraphunit_c_finalize): New prototype.
	(ipa_cp_c_finalize): New prototype.
	* cgraphunit.c (first_analyzed): Move from analyze_functions
	to file-scope.
	(first_analyzed_var): Likewise.
	(analyze_functions): Move static variables into file-scope.
	(cgraphunit_c_finalize): New function.
	* diagnostic.c (diagnostic_finish): Free the memory for
	context->classify_diagnostic and context->printer, running the
	destructor for the latter.
	(bt_stop): Use toplev::main.
	* dwarf2out.c (dwarf2out_finalize): New function.
	* dwarf2out.h (dwarf2out_c_finalize): New prototype.
	* gcse.c (gcse_c_finalize): New function.
	* gcse.h (gcse_c_finalize): New prototype.
	* ggc-page.c (init_ggc): Make idempotent.
	* input.c (input_location): Initialize to UNKNOWN_LOCATION.
	* ipa-cp.c (ipa_cp_c_finalize): New function.
	* ipa-pure-const.c (function_insertion_hook_holder): Move to be
	a field of class pass_ipa_pure_const.
	(node_duplication_hook_holder): Likewise.
	(node_removal_hook_holder): Likewise.
	(register_hooks): Convert to method...
	(pass_ipa_pure_const::register_hooks): ...here, converting
	static variable init_p into...
	(pass_ipa_pure_const::init_p): ...new field.
	(pure_const_generate_summary): Update invocation of
	register_hooks to invoke as a method of current_pass.
	(pure_const_read_summary): Likewise.
	(propagate): Convert to...
	(pass_ipa_pure_const::execute): ...method.
	* ipa-reference.c (ipa_init): Move static bool init_p from here
	to...
	(ipa_init_p): New file-scope variable, so that it can be reset
	when repeatedly invoking the compiler within one process by...
	(ipa_reference_c_finalize): New function.
	* ipa-reference.h (ipa_reference_c_finalize): New.
	* main.c (main): Replace invocation of toplev_main with
	construction of a toplev instance, and call its "main" method.
	* params.c (global_init_params): Add an assert that params_finished is
	false.
	(params_c_finalize): New.
	* params.h (params_c_finalize): New.
	* passes.c (execute_ipa_summary_passes): Set "current_pass" before
	invoking generate_summary, for the benefit of pass_ipa_pure_const.
	(ipa_write_summaries_2): Assign "pass" to "current_pass" global
	before calling write_summary hook.
	(ipa_write_optimization_summaries_1): Likewise when calling
	write_optimization_summary hook.
	(ipa_read_summaries_1): Likewise for read_summary hook.
	(ipa_read_optimization_summaries_1): Likewise for
	read_optimization_summary hook.
	(execute_ipa_stmt_fixups): Likewise.
	* stringpool.c (init_stringpool): Clean up if we're called more
	than once.
	* timevar.c (timevar_init): Ignore repeated calls.
	* toplev.c: Include "dwarf2out.h", "ipa-reference.h", "gcse.h".
	(general_init): Reset "input_location" to UNKNOWN_LOCATION.
	(initialize_rtl): Move static local "initialized_once"
	into file scope, and rename to...
	(rtl_initialized): New variable.
	(do_compile): Move timevar initialization from here to
	toplev::start_timevars.
	(toplev::toplev, toplev::~toplev, toplev::start_timevars,
	toplev::finalize): New functions.
	(toplev_main): Rename to...
	(toplev::main): ...this.
	* toplev.h (class toplev): New class.
---
 gcc/cgraph.c         |  14 +++++++
 gcc/cgraph.h         |   6 +++
 gcc/cgraphunit.c     |  20 ++++++++-
 gcc/diagnostic.c     |  11 ++++-
 gcc/dwarf2out.c      |  87 +++++++++++++++++++++++++++++++++++++++
 gcc/dwarf2out.h      |   2 +
 gcc/gcse.c           |   9 ++++
 gcc/gcse.h           |   2 +
 gcc/ggc-page.c       |   5 +++
 gcc/input.c          |   2 +-
 gcc/ipa-cp.c         |  12 ++++++
 gcc/ipa-pure-const.c | 113 +++++++++++++++++++++++++++++----------------------
 gcc/ipa-reference.c  |  17 ++++++--
 gcc/ipa-reference.h  |   1 +
 gcc/main.c           |   6 ++-
 gcc/params.c         |  14 +++++++
 gcc/params.h         |   4 ++
 gcc/passes.c         |   6 +++
 gcc/stringpool.c     |   5 +++
 gcc/timevar.c        |   3 ++
 gcc/toplev.c         |  67 +++++++++++++++++++++++-------
 gcc/toplev.h         |  19 ++++++++-
 22 files changed, 351 insertions(+), 74 deletions(-)

diff --git a/gcc/cgraph.c b/gcc/cgraph.c
index 38dc7e6..0de6593 100644
--- a/gcc/cgraph.c
+++ b/gcc/cgraph.c
@@ -3088,4 +3088,18 @@ gimple_check_call_matching_types (gimple call_stmt, tree callee,
   return true;
 }
 
+/* Reset all state within cgraph.c so that we can rerun the compiler
+   within the same process.  For use by toplev::finalize.  */
+
+void
+cgraph_c_finalize (void)
+{
+  symtab = NULL;
+
+  x_cgraph_nodes_queue = NULL;
+
+  cgraph_fnver_htab = NULL;
+  version_info_node = NULL;
+}
+
 #include "gt-cgraph.h"
diff --git a/gcc/cgraph.h b/gcc/cgraph.h
index 20b5c4e..794403d 100644
--- a/gcc/cgraph.h
+++ b/gcc/cgraph.h
@@ -2029,6 +2029,7 @@ extern GTY(()) symbol_table *symtab;
 extern vec<cgraph_node *> cgraph_new_nodes;
 
 /* In cgraph.c  */
+void cgraph_c_finalize (void);
 void release_function_body (tree);
 cgraph_indirect_call_info *cgraph_allocate_init_indirect_info (void);
 
@@ -2042,6 +2043,8 @@ bool resolution_used_from_other_file_p (enum ld_plugin_symbol_resolution);
 extern bool gimple_check_call_matching_types (gimple, tree, bool);
 
 /* In cgraphunit.c  */
+void cgraphunit_c_finalize (void);
+
 /*  Initialize datastructures so DECL is a function in lowered gimple form.
     IN_SSA is true if the gimple is in SSA.  */
 basic_block init_lowered_empty_function (tree, bool);
@@ -2061,6 +2064,9 @@ void record_references_in_initializer (tree, bool);
 void cgraph_build_static_cdtor (char which, tree body, int priority);
 void ipa_discover_readonly_nonaddressable_vars (void);
 
+/* In ipa-cp.c  */
+void ipa_cp_c_finalize (void);
+
 /* In varpool.c  */
 tree ctor_for_folding (tree);
 
diff --git a/gcc/cgraphunit.c b/gcc/cgraphunit.c
index d463505..7fbf8f8 100644
--- a/gcc/cgraphunit.c
+++ b/gcc/cgraphunit.c
@@ -877,15 +877,15 @@ walk_polymorphic_call_targets (hash_set<void *> *reachable_call_targets,
 
 /* Discover all functions and variables that are trivially needed, analyze
    them as well as all functions and variables referred by them  */
+static cgraph_node *first_analyzed;
+static varpool_node *first_analyzed_var;
 
 static void
 analyze_functions (void)
 {
   /* Keep track of already processed nodes when called multiple times for
      intermodule optimization.  */
-  static cgraph_node *first_analyzed;
   cgraph_node *first_handled = first_analyzed;
-  static varpool_node *first_analyzed_var;
   varpool_node *first_handled_var = first_analyzed_var;
   hash_set<void *> reachable_call_targets;
 
@@ -2285,6 +2285,22 @@ symbol_table::finalize_compilation_unit (void)
   timevar_pop (TV_CGRAPH);
 }
 
+/* Reset all state within cgraphunit.c so that we can rerun the compiler
+   within the same process.  For use by toplev::finalize.  */
+
+void
+cgraphunit_c_finalize (void)
+{
+  gcc_assert (cgraph_new_nodes.length () == 0);
+  cgraph_new_nodes.truncate (0);
+
+  vtable_entry_type = NULL;
+  queued_nodes = &symtab_terminator;
+
+  first_analyzed = NULL;
+  first_analyzed_var = NULL;
+}
+
 /* Creates a wrapper from cgraph_node to TARGET node. Thunk is used for this
    kind of wrapper method.  */
 
diff --git a/gcc/diagnostic.c b/gcc/diagnostic.c
index 881da0b..642cbe3 100644
--- a/gcc/diagnostic.c
+++ b/gcc/diagnostic.c
@@ -177,6 +177,15 @@ diagnostic_finish (diagnostic_context *context)
     }
 
   diagnostic_file_cache_fini ();
+
+  XDELETEVEC (context->classify_diagnostic);
+  context->classify_diagnostic = NULL;
+
+  /* diagnostic_initialize allocates context->printer using XNEW
+     and placement-new.  */
+  context->printer->~pretty_printer ();
+  XDELETE (context->printer);
+  context->printer = NULL;
 }
 
 /* Initialize DIAGNOSTIC, where the message MSG has already been
@@ -342,7 +351,7 @@ diagnostic_show_locus (diagnostic_context * context,
 static const char * const bt_stop[] =
 {
   "main",
-  "toplev_main",
+  "toplev::main",
   "execute_one_pass",
   "compile_file",
 };
diff --git a/gcc/dwarf2out.c b/gcc/dwarf2out.c
index 59c05ed..a085e0a 100644
--- a/gcc/dwarf2out.c
+++ b/gcc/dwarf2out.c
@@ -24614,4 +24614,91 @@ dwarf2out_finish (const char *filename)
     output_indirect_strings ();
 }
 
+/* Reset all state within dwarf2out.c so that we can rerun the compiler
+   within the same process.  For use by toplev::finalize.  */
+
+void
+dwarf2out_c_finalize (void)
+{
+  last_var_location_insn = NULL;
+  cached_next_real_insn = NULL;
+  used_rtx_array = NULL;
+  incomplete_types = NULL;
+  decl_scope_table = NULL;
+  debug_info_section = NULL;
+  debug_skeleton_info_section = NULL;
+  debug_abbrev_section = NULL;
+  debug_skeleton_abbrev_section = NULL;
+  debug_aranges_section = NULL;
+  debug_addr_section = NULL;
+  debug_macinfo_section = NULL;
+  debug_line_section = NULL;
+  debug_skeleton_line_section = NULL;
+  debug_loc_section = NULL;
+  debug_pubnames_section = NULL;
+  debug_pubtypes_section = NULL;
+  debug_str_section = NULL;
+  debug_str_dwo_section = NULL;
+  debug_str_offsets_section = NULL;
+  debug_ranges_section = NULL;
+  debug_frame_section = NULL;
+  fde_vec = NULL;
+  debug_str_hash = NULL;
+  skeleton_debug_str_hash = NULL;
+  dw2_string_counter = 0;
+  have_multiple_function_sections = false;
+  text_section_used = false;
+  cold_text_section_used = false;
+  cold_text_section = NULL;
+  current_unit_personality = NULL;
+
+  deferred_locations_list = NULL;
+
+  next_die_offset = 0;
+  single_comp_unit_die = NULL;
+  comdat_type_list = NULL;
+  limbo_die_list = NULL;
+  deferred_asm_name = NULL;
+  file_table = NULL;
+  decl_die_table = NULL;
+  common_block_die_table = NULL;
+  decl_loc_table = NULL;
+  call_arg_locations = NULL;
+  call_arg_loc_last = NULL;
+  call_site_count = -1;
+  tail_call_site_count = -1;
+  //block_map = NULL;
+  cached_dw_loc_list_table = NULL;
+  abbrev_die_table = NULL;
+  abbrev_die_table_allocated = 0;
+  abbrev_die_table_in_use = 0;
+  line_info_label_num = 0;
+  cur_line_info_table = NULL;
+  text_section_line_info = NULL;
+  cold_text_section_line_info = NULL;
+  separate_line_info = NULL;
+  info_section_emitted = false;
+  pubname_table = NULL;
+  pubtype_table = NULL;
+  macinfo_table = NULL;
+  ranges_table = NULL;
+  ranges_table_allocated = 0;
+  ranges_table_in_use = 0;
+  ranges_by_label = 0;
+  ranges_by_label_allocated = 0;
+  ranges_by_label_in_use = 0;
+  have_location_lists = false;
+  loclabel_num = 0;
+  poc_label_num = 0;
+  current_function_has_inlines = 0;
+  last_emitted_file = NULL;
+  label_num = 0;
+  file_table_last_lookup = NULL;
+  tmpl_value_parm_die_table = NULL;
+  generic_type_instances = NULL;
+  frame_pointer_fb_offset = 0;
+  frame_pointer_fb_offset_valid = false;
+  base_types.release ();
+}
+
 #include "gt-dwarf2out.h"
diff --git a/gcc/dwarf2out.h b/gcc/dwarf2out.h
index 7843e0a..c30d81f 100644
--- a/gcc/dwarf2out.h
+++ b/gcc/dwarf2out.h
@@ -277,4 +277,6 @@ struct array_descr_info
     } dimen[10];
 };
 
+void dwarf2out_c_finalize (void);
+
 #endif /* GCC_DWARF2OUT_H */
diff --git a/gcc/gcse.c b/gcc/gcse.c
index 7c62941..210f425 100644
--- a/gcc/gcse.c
+++ b/gcc/gcse.c
@@ -4288,4 +4288,13 @@ make_pass_rtl_hoist (gcc::context *ctxt)
   return new pass_rtl_hoist (ctxt);
 }
 
+/* Reset all state within gcse.c so that we can rerun the compiler
+   within the same process.  For use by toplev::finalize.  */
+
+void
+gcse_c_finalize (void)
+{
+  test_insn = NULL;
+}
+
 #include "gt-gcse.h"
diff --git a/gcc/gcse.h b/gcc/gcse.h
index 1b8c1c6..8e6820f 100644
--- a/gcc/gcse.h
+++ b/gcc/gcse.h
@@ -39,4 +39,6 @@ extern struct target_gcse *this_target_gcse;
 #define this_target_gcse (&default_target_gcse)
 #endif
 
+void gcse_c_finalize (void);
+
 #endif
diff --git a/gcc/ggc-page.c b/gcc/ggc-page.c
index 85b1ce9..2236a3a 100644
--- a/gcc/ggc-page.c
+++ b/gcc/ggc-page.c
@@ -1697,8 +1697,13 @@ compute_inverse (unsigned order)
 void
 init_ggc (void)
 {
+  static bool init_p = false;
   unsigned order;
 
+  if (init_p)
+    return;
+  init_p = true;
+
   G.pagesize = getpagesize ();
   G.lg_pagesize = exact_log2 (G.pagesize);
 
diff --git a/gcc/input.c b/gcc/input.c
index 7a88e2e..8d6356a 100644
--- a/gcc/input.c
+++ b/gcc/input.c
@@ -105,7 +105,7 @@ struct fcache
 
 /* Current position in real source file.  */
 
-location_t input_location;
+location_t input_location = UNKNOWN_LOCATION;
 
 struct line_maps *line_table;
 
diff --git a/gcc/ipa-cp.c b/gcc/ipa-cp.c
index a3be16f..a626975 100644
--- a/gcc/ipa-cp.c
+++ b/gcc/ipa-cp.c
@@ -3827,3 +3827,15 @@ make_pass_ipa_cp (gcc::context *ctxt)
 {
   return new pass_ipa_cp (ctxt);
 }
+
+/* Reset all state within ipa-cp.c so that we can rerun the compiler
+   within the same process.  For use by toplev::finalize.  */
+
+void
+ipa_cp_c_finalize (void)
+{
+  max_count = 0;
+  overall_size = 0;
+  max_new_size = 0;
+  values_topo = NULL;
+}
diff --git a/gcc/ipa-pure-const.c b/gcc/ipa-pure-const.c
index b5ded3e..c221cd0 100644
--- a/gcc/ipa-pure-const.c
+++ b/gcc/ipa-pure-const.c
@@ -115,10 +115,45 @@ typedef struct funct_state_d * funct_state;
 
 static vec<funct_state> funct_state_vec;
 
-/* Holders of ipa cgraph hooks: */
-static struct cgraph_node_hook_list *function_insertion_hook_holder;
-static struct cgraph_2node_hook_list *node_duplication_hook_holder;
-static struct cgraph_node_hook_list *node_removal_hook_holder;
+static bool gate_pure_const (void);
+
+namespace {
+
+const pass_data pass_data_ipa_pure_const =
+{
+  IPA_PASS, /* type */
+  "pure-const", /* name */
+  OPTGROUP_NONE, /* optinfo_flags */
+  TV_IPA_PURE_CONST, /* tv_id */
+  0, /* properties_required */
+  0, /* properties_provided */
+  0, /* properties_destroyed */
+  0, /* todo_flags_start */
+  0, /* todo_flags_finish */
+};
+
+class pass_ipa_pure_const : public ipa_opt_pass_d
+{
+public:
+  pass_ipa_pure_const(gcc::context *ctxt);
+
+  /* opt_pass methods: */
+  bool gate (function *) { return gate_pure_const (); }
+  unsigned int execute (function *fun);
+
+  void register_hooks (void);
+
+private:
+  bool init_p;
+
+  /* Holders of ipa cgraph hooks: */
+  struct cgraph_node_hook_list *function_insertion_hook_holder;
+  struct cgraph_2node_hook_list *node_duplication_hook_holder;
+  struct cgraph_node_hook_list *node_removal_hook_holder;
+
+}; // class pass_ipa_pure_const
+
+} // anon namespace
 
 /* Try to guess if function body will always be visible to compiler
    when compiling the call and whether compiler will be able
@@ -881,11 +916,10 @@ remove_node_data (struct cgraph_node *node, void *data ATTRIBUTE_UNUSED)
 }
 
 \f
-static void
+void
+pass_ipa_pure_const::
 register_hooks (void)
 {
-  static bool init_p = false;
-
   if (init_p)
     return;
 
@@ -908,7 +942,8 @@ pure_const_generate_summary (void)
 {
   struct cgraph_node *node;
 
-  register_hooks ();
+  pass_ipa_pure_const *pass = static_cast <pass_ipa_pure_const *> (current_pass);
+  pass->register_hooks ();
 
   /* Process all of the functions.
 
@@ -989,7 +1024,9 @@ pure_const_read_summary (void)
   struct lto_file_decl_data *file_data;
   unsigned int j = 0;
 
-  register_hooks ();
+  pass_ipa_pure_const *pass = static_cast <pass_ipa_pure_const *> (current_pass);
+  pass->register_hooks ();
+
   while ((file_data = file_data_vec[j++]))
     {
       const char *data;
@@ -1470,8 +1507,9 @@ propagate_nothrow (void)
 /* Produce the global information by preforming a transitive closure
    on the local information that was produced by generate_summary.  */
 
-static unsigned int
-propagate (void)
+unsigned int
+pass_ipa_pure_const::
+execute (function *)
 {
   struct cgraph_node *node;
 
@@ -1500,44 +1538,23 @@ gate_pure_const (void)
 	  && !seen_error ());
 }
 
-namespace {
-
-const pass_data pass_data_ipa_pure_const =
+pass_ipa_pure_const::pass_ipa_pure_const(gcc::context *ctxt)
+    : ipa_opt_pass_d(pass_data_ipa_pure_const, ctxt,
+		     pure_const_generate_summary, /* generate_summary */
+		     pure_const_write_summary, /* write_summary */
+		     pure_const_read_summary, /* read_summary */
+		     NULL, /* write_optimization_summary */
+		     NULL, /* read_optimization_summary */
+		     NULL, /* stmt_fixup */
+		     0, /* function_transform_todo_flags_start */
+		     NULL, /* function_transform */
+		     NULL), /* variable_transform */
+  init_p(false),
+  function_insertion_hook_holder(NULL),
+  node_duplication_hook_holder(NULL),
+  node_removal_hook_holder(NULL)
 {
-  IPA_PASS, /* type */
-  "pure-const", /* name */
-  OPTGROUP_NONE, /* optinfo_flags */
-  TV_IPA_PURE_CONST, /* tv_id */
-  0, /* properties_required */
-  0, /* properties_provided */
-  0, /* properties_destroyed */
-  0, /* todo_flags_start */
-  0, /* todo_flags_finish */
-};
-
-class pass_ipa_pure_const : public ipa_opt_pass_d
-{
-public:
-  pass_ipa_pure_const (gcc::context *ctxt)
-    : ipa_opt_pass_d (pass_data_ipa_pure_const, ctxt,
-		      pure_const_generate_summary, /* generate_summary */
-		      pure_const_write_summary, /* write_summary */
-		      pure_const_read_summary, /* read_summary */
-		      NULL, /* write_optimization_summary */
-		      NULL, /* read_optimization_summary */
-		      NULL, /* stmt_fixup */
-		      0, /* function_transform_todo_flags_start */
-		      NULL, /* function_transform */
-		      NULL) /* variable_transform */
-  {}
-
-  /* opt_pass methods: */
-  virtual bool gate (function *) { return gate_pure_const (); }
-  virtual unsigned int execute (function *) { return propagate (); }
-
-}; // class pass_ipa_pure_const
-
-} // anon namespace
+}
 
 ipa_opt_pass_d *
 make_pass_ipa_pure_const (gcc::context *ctxt)
diff --git a/gcc/ipa-reference.c b/gcc/ipa-reference.c
index 9767eab..f3af47a 100644
--- a/gcc/ipa-reference.c
+++ b/gcc/ipa-reference.c
@@ -399,17 +399,17 @@ propagate_bits (ipa_reference_global_vars_info_t x_global, struct cgraph_node *x
     }
 }
 
+static bool ipa_init_p = false;
+
 /* The init routine for analyzing global static variable usage.  See
    comments at top for description.  */
 static void
 ipa_init (void)
 {
-  static bool init_p = false;
-
-  if (init_p)
+  if (ipa_init_p)
     return;
 
-  init_p = true;
+  ipa_init_p = true;
 
   if (dump_file)
     reference_vars_to_consider = splay_tree_new (splay_tree_compare_ints, 0, 0);
@@ -1173,3 +1173,12 @@ make_pass_ipa_reference (gcc::context *ctxt)
 {
   return new pass_ipa_reference (ctxt);
 }
+
+/* Reset all state within ipa-reference.c so that we can rerun the compiler
+   within the same process.  For use by toplev::finalize.  */
+
+void
+ipa_reference_c_finalize (void)
+{
+  ipa_init_p = false;
+}
diff --git a/gcc/ipa-reference.h b/gcc/ipa-reference.h
index c840024..3e7dc73 100644
--- a/gcc/ipa-reference.h
+++ b/gcc/ipa-reference.h
@@ -26,6 +26,7 @@ along with GCC; see the file COPYING3.  If not see
 /* In ipa-reference.c  */
 bitmap ipa_reference_get_not_read_global (struct cgraph_node *fn);
 bitmap ipa_reference_get_not_written_global (struct cgraph_node *fn);
+void ipa_reference_c_finalize (void);
 
 #endif  /* GCC_IPA_REFERENCE_H  */
 
diff --git a/gcc/main.c b/gcc/main.c
index 241d151..4bba041 100644
--- a/gcc/main.c
+++ b/gcc/main.c
@@ -26,12 +26,14 @@ along with GCC; see the file COPYING3.  If not see
 
 int main (int argc, char **argv);
 
-/* We define main() to call toplev_main(), which is defined in toplev.c.
+/* We define main() to call toplev::main(), which is defined in toplev.c.
    We do this in a separate file in order to allow the language front-end
    to define a different main(), if it so desires.  */
 
 int
 main (int argc, char **argv)
 {
-  return toplev_main (argc, argv);
+  toplev toplev (true);
+
+  return toplev.main (argc, argv);
 }
diff --git a/gcc/params.c b/gcc/params.c
index 3ae5ccd..7aa96e2 100644
--- a/gcc/params.c
+++ b/gcc/params.c
@@ -69,6 +69,8 @@ add_params (const param_info params[], size_t n)
 void
 global_init_params (void)
 {
+  gcc_assert (!params_finished);
+
   add_params (lang_independent_params, LAST_PARAM);
   targetm_common.option_default_params ();
 }
@@ -82,6 +84,18 @@ finish_params (void)
   params_finished = true;
 }
 
+/* Reset all state within params.c so that we can rerun the compiler
+   within the same process.  For use by toplev::finalize.  */
+
+void
+params_c_finalize (void)
+{
+  XDELETEVEC (compiler_params);
+  compiler_params = NULL;
+  num_compiler_params = 0;
+  params_finished = false;
+}
+
 /* Set the value of the parameter given by NUM to VALUE in PARAMS and
    PARAMS_SET.  If EXPLICIT_P, this is being set by the user;
    otherwise it is being set implicitly by the compiler.  */
diff --git a/gcc/params.h b/gcc/params.h
index d488e32..4779e17 100644
--- a/gcc/params.h
+++ b/gcc/params.h
@@ -113,6 +113,10 @@ extern void global_init_params (void);
    set.  */
 extern void finish_params (void);
 
+/* Reset all state in params.c  */
+
+extern void params_c_finalize (void);
+
 /* Return the default value of parameter NUM.  */
 
 extern int default_param_value (compiler_param num);
diff --git a/gcc/passes.c b/gcc/passes.c
index 5001c3d..772993d 100644
--- a/gcc/passes.c
+++ b/gcc/passes.c
@@ -1944,6 +1944,7 @@ execute_ipa_summary_passes (ipa_opt_pass_d *ipa_pass)
 	  if (pass->tv_id)
 	    timevar_push (pass->tv_id);
 
+	  current_pass = pass;
 	  ipa_pass->generate_summary ();
 
 	  /* Stop timevar.  */
@@ -2255,6 +2256,7 @@ ipa_write_summaries_2 (opt_pass *pass, struct lto_out_decl_state *state)
 
           pass_init_dump_file (pass);
 
+	  current_pass = pass;
 	  ipa_pass->write_summary ();
 
           pass_fini_dump_file (pass);
@@ -2373,6 +2375,7 @@ ipa_write_optimization_summaries_1 (opt_pass *pass,
 
           pass_init_dump_file (pass);
 
+	  current_pass = pass;
 	  ipa_pass->write_optimization_summary ();
 
           pass_fini_dump_file (pass);
@@ -2453,6 +2456,7 @@ ipa_read_summaries_1 (opt_pass *pass)
 
 	      pass_init_dump_file (pass);
 
+	      current_pass = pass;
 	      ipa_pass->read_summary ();
 
 	      pass_fini_dump_file (pass);
@@ -2503,6 +2507,7 @@ ipa_read_optimization_summaries_1 (opt_pass *pass)
 
 	      pass_init_dump_file (pass);
 
+	      current_pass = pass;
 	      ipa_pass->read_optimization_summary ();
 
 	      pass_fini_dump_file (pass);
@@ -2582,6 +2587,7 @@ execute_ipa_stmt_fixups (opt_pass *pass,
 	      if (pass->tv_id)
 		timevar_push (pass->tv_id);
 
+	      current_pass = pass;
 	      ipa_pass->stmt_fixup (node, stmts);
 
 	      /* Stop timevar.  */
diff --git a/gcc/stringpool.c b/gcc/stringpool.c
index c880cb1..330df6c 100644
--- a/gcc/stringpool.c
+++ b/gcc/stringpool.c
@@ -61,6 +61,11 @@ stringpool_ggc_alloc (size_t x)
 void
 init_stringpool (void)
 {
+  /* Clean up if we're called more than once.
+     (We can't make this idempotent since identifiers contain state) */
+  if (ident_hash)
+    ht_destroy (ident_hash);
+
   /* Create with 16K (2^14) entries.  */
   ident_hash = ht_create (14);
   ident_hash->alloc_node = alloc_node;
diff --git a/gcc/timevar.c b/gcc/timevar.c
index c111e98..0e56a23 100644
--- a/gcc/timevar.c
+++ b/gcc/timevar.c
@@ -223,6 +223,9 @@ timevar_accumulate (struct timevar_time_def *timer,
 void
 timevar_init (void)
 {
+  if (timevar_enable)
+    return;
+
   timevar_enable = true;
 
   /* Zero all elapsed times.  */
diff --git a/gcc/toplev.c b/gcc/toplev.c
index 0e626b3..fe60b40 100644
--- a/gcc/toplev.c
+++ b/gcc/toplev.c
@@ -79,6 +79,9 @@ along with GCC; see the file COPYING3.  If not see
 #include "diagnostic-color.h"
 #include "context.h"
 #include "pass_manager.h"
+#include "dwarf2out.h"
+#include "ipa-reference.h"
+#include "gcse.h"
 #include "optabs.h"
 
 #if defined(DBX_DEBUGGING_INFO) || defined(XCOFF_DEBUGGING_INFO)
@@ -97,7 +100,7 @@ along with GCC; see the file COPYING3.  If not see
 #include <new>
 
 static void general_init (const char *);
-static void do_compile (void);
+static void do_compile ();
 static void process_options (void);
 static void backend_init (void);
 static int lang_dependent_init (const char *);
@@ -1157,6 +1160,7 @@ general_init (const char *argv0)
      table.  */
   init_ggc ();
   init_stringpool ();
+  input_location = UNKNOWN_LOCATION;
   line_table = ggc_alloc<line_maps> ();
   linemap_init (line_table, BUILTINS_LOCATION);
   line_table->reallocator = realloc_for_line_map;
@@ -1704,16 +1708,16 @@ lang_dependent_init_target (void)
 /* Perform initializations that are lang-dependent or target-dependent.
    but matters only for late optimizations and RTL generation.  */
 
+static int rtl_initialized;
+
 void
 initialize_rtl (void)
 {
-  static int initialized_once;
-
   /* Initialization done just once per compilation, but delayed
      till code generation.  */
-  if (!initialized_once)
+  if (!rtl_initialized)
     ira_init_once ();
-  initialized_once = true;
+  rtl_initialized = true;
 
   /* Target specific RTL backend initialization.  */
   if (!this_target_rtl->target_specific_initialized)
@@ -1903,14 +1907,8 @@ finalize (bool no_backend)
 
 /* Initialize the compiler, and compile the input file.  */
 static void
-do_compile (void)
+do_compile ()
 {
-  /* Initialize timing first.  The C front ends read the main file in
-     the post_options hook, and C++ does file timings.  */
-  if (time_report || !quiet_flag  || flag_detailed_statistics)
-    timevar_init ();
-  timevar_start (TV_TOTAL);
-
   process_options ();
 
   /* Don't do any more if an error has already occurred.  */
@@ -1956,12 +1954,30 @@ do_compile (void)
 
       timevar_stop (TV_PHASE_FINALIZE);
     }
+}
 
-  /* Stop timing and print the times.  */
+toplev::toplev (bool use_TV_TOTAL)
+  : m_use_TV_TOTAL (use_TV_TOTAL)
+{
+  if (!m_use_TV_TOTAL)
+    start_timevars ();
+}
+
+toplev::~toplev ()
+{
   timevar_stop (TV_TOTAL);
   timevar_print (stderr);
 }
 
+void
+toplev::start_timevars ()
+{
+  if (time_report || !quiet_flag  || flag_detailed_statistics)
+    timevar_init ();
+
+  timevar_start (TV_TOTAL);
+}
+
 /* Entry point of cc1, cc1plus, jc1, f771, etc.
    Exit code is FATAL_EXIT_CODE if can't open files or if there were
    any errors, or SUCCESS_EXIT_CODE if compilation succeeded.
@@ -1969,7 +1985,7 @@ do_compile (void)
    It is not safe to call this function more than once.  */
 
 int
-toplev_main (int argc, char **argv)
+toplev::main (int argc, char **argv)
 {
   /* Parsing and gimplification sometimes need quite large stack.
      Increase stack size limits if possible.  */
@@ -2019,7 +2035,11 @@ toplev_main (int argc, char **argv)
 
   /* Exit early if we can (e.g. -help).  */
   if (!exit_after_options)
-    do_compile ();
+    {
+      if (m_use_TV_TOTAL)
+	start_timevars ();
+      do_compile ();
+    }
 
   if (warningcount || errorcount || werrorcount)
     print_ignored_options ();
@@ -2037,3 +2057,20 @@ toplev_main (int argc, char **argv)
 
   return (SUCCESS_EXIT_CODE);
 }
+
+/* For those that want to, this function aims to clean up enough state that
+   you can call toplev::main again. */
+void
+toplev::finalize (void)
+{
+  rtl_initialized = false;
+  this_target_rtl->target_specific_initialized = false;
+
+  cgraph_c_finalize ();
+  cgraphunit_c_finalize ();
+  dwarf2out_c_finalize ();
+  gcse_c_finalize ();
+  ipa_cp_c_finalize ();
+  ipa_reference_c_finalize ();
+  params_c_finalize ();
+}
diff --git a/gcc/toplev.h b/gcc/toplev.h
index 1b54578..b845843 100644
--- a/gcc/toplev.h
+++ b/gcc/toplev.h
@@ -24,7 +24,24 @@ along with GCC; see the file COPYING3.  If not see
 extern struct cl_decoded_option *save_decoded_options;
 extern unsigned int save_decoded_options_count;
 
-extern int toplev_main (int, char **);
+/* Invoking the compiler.  */
+class toplev
+{
+public:
+  toplev (bool use_TV_TOTAL);
+  ~toplev ();
+
+  int main (int argc, char **argv);
+
+  void finalize ();
+
+private:
+
+  void start_timevars ();
+
+  bool m_use_TV_TOTAL;
+};
+
 extern void rest_of_decl_compilation (tree, int, int);
 extern void rest_of_type_compilation (tree, int);
 extern void init_optimization_passes (void);
-- 
1.8.5.3

Re: [PATCH 4/5] State cleanups -- also note for MPX work

[Jeff Law]

On 10/13/14 11:45, David Malcolm wrote:
> This patch contains various cleanups needed by the jit, so that we can
> rerun the compiler code repeatedly within the same process:
>
> * Introduce per-file finalization routines to clean up file-static
>    variables.
>
> * Make various initializations idempotent (init_ggc), or cleanup their
>    previous state (init_stringpool).
>
> * Introduce a "class toplev", so that we can easy control whether
>    timevars are reset each time, or run in a "cumulative" mode.
>
> gcc/ChangeLog:
> 	* cgraph.c (cgraph_c_finalize): New function.
> 	* cgraph.h (cgraph_c_finalize): New prototype.
> 	(cgraphunit_c_finalize): New prototype.
> 	(ipa_cp_c_finalize): New prototype.
> 	* cgraphunit.c (first_analyzed): Move from analyze_functions
> 	to file-scope.
> 	(first_analyzed_var): Likewise.
> 	(analyze_functions): Move static variables into file-scope.
> 	(cgraphunit_c_finalize): New function.
> 	* diagnostic.c (diagnostic_finish): Free the memory for
> 	context->classify_diagnostic and context->printer, running the
> 	destructor for the latter.
> 	(bt_stop): Use toplev::main.
> 	* dwarf2out.c (dwarf2out_finalize): New function.
> 	* dwarf2out.h (dwarf2out_c_finalize): New prototype.
> 	* gcse.c (gcse_c_finalize): New function.
> 	* gcse.h (gcse_c_finalize): New prototype.
> 	* ggc-page.c (init_ggc): Make idempotent.
> 	* input.c (input_location): Initialize to UNKNOWN_LOCATION.
> 	* ipa-cp.c (ipa_cp_c_finalize): New function.
> 	* ipa-pure-const.c (function_insertion_hook_holder): Move to be
> 	a field of class pass_ipa_pure_const.
> 	(node_duplication_hook_holder): Likewise.
> 	(node_removal_hook_holder): Likewise.
> 	(register_hooks): Convert to method...
> 	(pass_ipa_pure_const::register_hooks): ...here, converting
> 	static variable init_p into...
> 	(pass_ipa_pure_const::init_p): ...new field.
> 	(pure_const_generate_summary): Update invocation of
> 	register_hooks to invoke as a method of current_pass.
> 	(pure_const_read_summary): Likewise.
> 	(propagate): Convert to...
> 	(pass_ipa_pure_const::execute): ...method.
> 	* ipa-reference.c (ipa_init): Move static bool init_p from here
> 	to...
> 	(ipa_init_p): New file-scope variable, so that it can be reset
> 	when repeatedly invoking the compiler within one process by...
> 	(ipa_reference_c_finalize): New function.
> 	* ipa-reference.h (ipa_reference_c_finalize): New.
> 	* main.c (main): Replace invocation of toplev_main with
> 	construction of a toplev instance, and call its "main" method.
> 	* params.c (global_init_params): Add an assert that params_finished is
> 	false.
> 	(params_c_finalize): New.
> 	* params.h (params_c_finalize): New.
> 	* passes.c (execute_ipa_summary_passes): Set "current_pass" before
> 	invoking generate_summary, for the benefit of pass_ipa_pure_const.
> 	(ipa_write_summaries_2): Assign "pass" to "current_pass" global
> 	before calling write_summary hook.
> 	(ipa_write_optimization_summaries_1): Likewise when calling
> 	write_optimization_summary hook.
> 	(ipa_read_summaries_1): Likewise for read_summary hook.
> 	(ipa_read_optimization_summaries_1): Likewise for
> 	read_optimization_summary hook.
> 	(execute_ipa_stmt_fixups): Likewise.
> 	* stringpool.c (init_stringpool): Clean up if we're called more
> 	than once.
> 	* timevar.c (timevar_init): Ignore repeated calls.
> 	* toplev.c: Include "dwarf2out.h", "ipa-reference.h", "gcse.h".
> 	(general_init): Reset "input_location" to UNKNOWN_LOCATION.
> 	(initialize_rtl): Move static local "initialized_once"
> 	into file scope, and rename to...
> 	(rtl_initialized): New variable.
> 	(do_compile): Move timevar initialization from here to
> 	toplev::start_timevars.
> 	(toplev::toplev, toplev::~toplev, toplev::start_timevars,
> 	toplev::finalize): New functions.
> 	(toplev_main): Rename to...
> 	(toplev::main): ...this.
> 	* toplev.h (class toplev): New class.
> ---
>   gcc/cgraph.c         |  14 +++++++
>   gcc/cgraph.h         |   6 +++
>   gcc/cgraphunit.c     |  20 ++++++++-
>   gcc/diagnostic.c     |  11 ++++-
>   gcc/dwarf2out.c      |  87 +++++++++++++++++++++++++++++++++++++++
>   gcc/dwarf2out.h      |   2 +
>   gcc/gcse.c           |   9 ++++
>   gcc/gcse.h           |   2 +
>   gcc/ggc-page.c       |   5 +++
>   gcc/input.c          |   2 +-
>   gcc/ipa-cp.c         |  12 ++++++
>   gcc/ipa-pure-const.c | 113 +++++++++++++++++++++++++++++----------------------
>   gcc/ipa-reference.c  |  17 ++++++--
>   gcc/ipa-reference.h  |   1 +
>   gcc/main.c           |   6 ++-
>   gcc/params.c         |  14 +++++++
>   gcc/params.h         |   4 ++
>   gcc/passes.c         |   6 +++
>   gcc/stringpool.c     |   5 +++
>   gcc/timevar.c        |   3 ++
>   gcc/toplev.c         |  67 +++++++++++++++++++++++-------
>   gcc/toplev.h         |  19 ++++++++-
>   22 files changed, 351 insertions(+), 74 deletions(-)
General note, I suspect there's going to be file scoped statics in some 
of the MPX work.  It was hard to say "you have to do this a different 
way" and make it a precondition for acceptance with your patch still in 
progress.

So that means if the MPX work goes in first, you'll need to adjust this 
patch.  If it happens the other way, the MPX work will need adjustment.

Anyway, just thought it was worth explicitly pointing out that these two 
hunks of work, while they're tackling totally different issues may 
conflict because of an implementation of the MPX bits.

>
> diff --git a/gcc/cgraph.h b/gcc/cgraph.h
> index 20b5c4e..794403d 100644
> --- a/gcc/cgraph.h
> +++ b/gcc/cgraph.h
> @@ -2042,6 +2043,8 @@ bool resolution_used_from_other_file_p (enum ld_plugin_symbol_resolution);
>   extern bool gimple_check_call_matching_types (gimple, tree, bool);
>
>   /* In cgraphunit.c  */
> +void cgraphunit_c_finalize (void);
Combine this with the declaration of init_lowered_empty_function so that 
the references to cgraphunit's functions are together.

> +
>   /*  Initialize datastructures so DECL is a function in lowered gimple form.
>       IN_SSA is true if the gimple is in SSA.  */
>   basic_block init_lowered_empty_function (tree, bool);
> @@ -2061,6 +2064,9 @@ void record_references_in_initializer (tree, bool);
>   void cgraph_build_static_cdtor (char which, tree body, int priority);
>   void ipa_discover_readonly_nonaddressable_vars (void);
>
> +/* In ipa-cp.c  */
> +void ipa_cp_c_finalize (void);
Is there a better place for this?  ipa-prop.h?

> diff --git a/gcc/dwarf2out.c b/gcc/dwarf2out.c
> index 59c05ed..a085e0a 100644
> --- a/gcc/dwarf2out.c
> +++ b/gcc/dwarf2out.c
> @@ -24614,4 +24614,91 @@ dwarf2out_finish (const char *filename)
>       output_indirect_strings ();
>   }
>
> +/* Reset all state within dwarf2out.c so that we can rerun the compiler
> +   within the same process.  For use by toplev::finalize.  */
> +
> +void
> +dwarf2out_c_finalize (void)
> +{
> +  last_var_location_insn = NULL;
[ ... ]
Makes me wonder if some of this stuff belongs in a structure.  And if 
some does, then obviously memset becomes an option for finalization. 
But that's not something I think you need to own for this to go forward.

> diff --git a/gcc/passes.c b/gcc/passes.c
> index 5001c3d..772993d 100644
> --- a/gcc/passes.c
> +++ b/gcc/passes.c
> @@ -1944,6 +1944,7 @@ execute_ipa_summary_passes (ipa_opt_pass_d *ipa_pass)
>   	  if (pass->tv_id)
>   	    timevar_push (pass->tv_id);
>
> +	  current_pass = pass;
>   	  ipa_pass->generate_summary ();
Presumably it doesn't make sense to push this down into 
ipa_pass->generate_summary?


OK with the nits above addressed.

Jeff

Re: [PATCH 4/5] State cleanups -- also note for MPX work

[David Malcolm]

On Thu, 2014-10-16 at 16:06 -0600, Jeff Law wrote:
> On 10/13/14 11:45, David Malcolm wrote:
> > This patch contains various cleanups needed by the jit, so that we can
> > rerun the compiler code repeatedly within the same process:
> >
> > * Introduce per-file finalization routines to clean up file-static
> >    variables.
> >
> > * Make various initializations idempotent (init_ggc), or cleanup their
> >    previous state (init_stringpool).
> >
> > * Introduce a "class toplev", so that we can easy control whether
> >    timevars are reset each time, or run in a "cumulative" mode.
> >
> > gcc/ChangeLog:
> > 	* cgraph.c (cgraph_c_finalize): New function.
> > 	* cgraph.h (cgraph_c_finalize): New prototype.
> > 	(cgraphunit_c_finalize): New prototype.
> > 	(ipa_cp_c_finalize): New prototype.
> > 	* cgraphunit.c (first_analyzed): Move from analyze_functions
> > 	to file-scope.
> > 	(first_analyzed_var): Likewise.
> > 	(analyze_functions): Move static variables into file-scope.
> > 	(cgraphunit_c_finalize): New function.
> > 	* diagnostic.c (diagnostic_finish): Free the memory for
> > 	context->classify_diagnostic and context->printer, running the
> > 	destructor for the latter.
> > 	(bt_stop): Use toplev::main.
> > 	* dwarf2out.c (dwarf2out_finalize): New function.
> > 	* dwarf2out.h (dwarf2out_c_finalize): New prototype.
> > 	* gcse.c (gcse_c_finalize): New function.
> > 	* gcse.h (gcse_c_finalize): New prototype.
> > 	* ggc-page.c (init_ggc): Make idempotent.
> > 	* input.c (input_location): Initialize to UNKNOWN_LOCATION.
> > 	* ipa-cp.c (ipa_cp_c_finalize): New function.
> > 	* ipa-pure-const.c (function_insertion_hook_holder): Move to be
> > 	a field of class pass_ipa_pure_const.
> > 	(node_duplication_hook_holder): Likewise.
> > 	(node_removal_hook_holder): Likewise.
> > 	(register_hooks): Convert to method...
> > 	(pass_ipa_pure_const::register_hooks): ...here, converting
> > 	static variable init_p into...
> > 	(pass_ipa_pure_const::init_p): ...new field.
> > 	(pure_const_generate_summary): Update invocation of
> > 	register_hooks to invoke as a method of current_pass.
> > 	(pure_const_read_summary): Likewise.
> > 	(propagate): Convert to...
> > 	(pass_ipa_pure_const::execute): ...method.
> > 	* ipa-reference.c (ipa_init): Move static bool init_p from here
> > 	to...
> > 	(ipa_init_p): New file-scope variable, so that it can be reset
> > 	when repeatedly invoking the compiler within one process by...
> > 	(ipa_reference_c_finalize): New function.
> > 	* ipa-reference.h (ipa_reference_c_finalize): New.
> > 	* main.c (main): Replace invocation of toplev_main with
> > 	construction of a toplev instance, and call its "main" method.
> > 	* params.c (global_init_params): Add an assert that params_finished is
> > 	false.
> > 	(params_c_finalize): New.
> > 	* params.h (params_c_finalize): New.
> > 	* passes.c (execute_ipa_summary_passes): Set "current_pass" before
> > 	invoking generate_summary, for the benefit of pass_ipa_pure_const.
> > 	(ipa_write_summaries_2): Assign "pass" to "current_pass" global
> > 	before calling write_summary hook.
> > 	(ipa_write_optimization_summaries_1): Likewise when calling
> > 	write_optimization_summary hook.
> > 	(ipa_read_summaries_1): Likewise for read_summary hook.
> > 	(ipa_read_optimization_summaries_1): Likewise for
> > 	read_optimization_summary hook.
> > 	(execute_ipa_stmt_fixups): Likewise.
> > 	* stringpool.c (init_stringpool): Clean up if we're called more
> > 	than once.
> > 	* timevar.c (timevar_init): Ignore repeated calls.
> > 	* toplev.c: Include "dwarf2out.h", "ipa-reference.h", "gcse.h".
> > 	(general_init): Reset "input_location" to UNKNOWN_LOCATION.
> > 	(initialize_rtl): Move static local "initialized_once"
> > 	into file scope, and rename to...
> > 	(rtl_initialized): New variable.
> > 	(do_compile): Move timevar initialization from here to
> > 	toplev::start_timevars.
> > 	(toplev::toplev, toplev::~toplev, toplev::start_timevars,
> > 	toplev::finalize): New functions.
> > 	(toplev_main): Rename to...
> > 	(toplev::main): ...this.
> > 	* toplev.h (class toplev): New class.
> > ---
> >   gcc/cgraph.c         |  14 +++++++
> >   gcc/cgraph.h         |   6 +++
> >   gcc/cgraphunit.c     |  20 ++++++++-
> >   gcc/diagnostic.c     |  11 ++++-
> >   gcc/dwarf2out.c      |  87 +++++++++++++++++++++++++++++++++++++++
> >   gcc/dwarf2out.h      |   2 +
> >   gcc/gcse.c           |   9 ++++
> >   gcc/gcse.h           |   2 +
> >   gcc/ggc-page.c       |   5 +++
> >   gcc/input.c          |   2 +-
> >   gcc/ipa-cp.c         |  12 ++++++
> >   gcc/ipa-pure-const.c | 113 +++++++++++++++++++++++++++++----------------------
> >   gcc/ipa-reference.c  |  17 ++++++--
> >   gcc/ipa-reference.h  |   1 +
> >   gcc/main.c           |   6 ++-
> >   gcc/params.c         |  14 +++++++
> >   gcc/params.h         |   4 ++
> >   gcc/passes.c         |   6 +++
> >   gcc/stringpool.c     |   5 +++
> >   gcc/timevar.c        |   3 ++
> >   gcc/toplev.c         |  67 +++++++++++++++++++++++-------
> >   gcc/toplev.h         |  19 ++++++++-
> >   22 files changed, 351 insertions(+), 74 deletions(-)
> General note, I suspect there's going to be file scoped statics in some 
> of the MPX work.  It was hard to say "you have to do this a different 
> way" and make it a precondition for acceptance with your patch still in 
> progress.
> 
> So that means if the MPX work goes in first, you'll need to adjust this 
> patch.  If it happens the other way, the MPX work will need adjustment.
> 
> Anyway, just thought it was worth explicitly pointing out that these two 
> hunks of work, while they're tackling totally different issues may 
> conflict because of an implementation of the MPX bits.

Presumably my state cleanup patch isn't going to break the MPX bits
though?  I haven't been following them in detail.

I should spell out that I haven't exhaustively hunted down every
file-scoped static in the source tree; this work merely fixes enough
such state to ensure that the testsuite passes: every test is run 5
times in-process, with the equivalent of -O3 -g, both individually, and
within a combined context in test-combination.c, to try to shake out
state issues.  There's also a multithreaded variant of
test-combination.c, test-threads.c, though that's mostly just to ensure
that the big mutex is working and in the right place (it runs every test
case in a separate thread all within one process, each thread running
its case 5 times in a row).

I've got some ideas for automatically tracking down additional
file-scoped static state by analysis of source or of the .o files.  Am I
right in assuming I can regard that as followup bugfixing?


> > diff --git a/gcc/cgraph.h b/gcc/cgraph.h
> > index 20b5c4e..794403d 100644
> > --- a/gcc/cgraph.h
> > +++ b/gcc/cgraph.h
> > @@ -2042,6 +2043,8 @@ bool resolution_used_from_other_file_p (enum ld_plugin_symbol_resolution);
> >   extern bool gimple_check_call_matching_types (gimple, tree, bool);
> >
> >   /* In cgraphunit.c  */
> > +void cgraphunit_c_finalize (void);
> Combine this with the declaration of init_lowered_empty_function so that 
> the references to cgraphunit's functions are together.

I believe they are, since it's immediately followed by...
> > +
> >   /*  Initialize datastructures so DECL is a function in lowered gimple form.
> >       IN_SSA is true if the gimple is in SSA.  */
> >   basic_block init_lowered_empty_function (tree, bool);

^^^ it's here.

Am I missing something?

> > @@ -2061,6 +2064,9 @@ void record_references_in_initializer (tree, bool);
> >   void cgraph_build_static_cdtor (char which, tree body, int priority);
> >   void ipa_discover_readonly_nonaddressable_vars (void);
> >
> > +/* In ipa-cp.c  */
> > +void ipa_cp_c_finalize (void);
> Is there a better place for this?  ipa-prop.h?

OK.

> > diff --git a/gcc/dwarf2out.c b/gcc/dwarf2out.c
> > index 59c05ed..a085e0a 100644
> > --- a/gcc/dwarf2out.c
> > +++ b/gcc/dwarf2out.c
> > @@ -24614,4 +24614,91 @@ dwarf2out_finish (const char *filename)
> >       output_indirect_strings ();
> >   }
> >
> > +/* Reset all state within dwarf2out.c so that we can rerun the compiler
> > +   within the same process.  For use by toplev::finalize.  */
> > +
> > +void
> > +dwarf2out_c_finalize (void)
> > +{
> > +  last_var_location_insn = NULL;
> [ ... ]
> Makes me wonder if some of this stuff belongs in a structure.  And if 
> some does, then obviously memset becomes an option for finalization. 
> But that's not something I think you need to own for this to go forward.

FWIW I've been experimenting with making the debugging structs be
classes, with the hooks becoming virtual functions.  Then all of this
debugging state could be turned into member data of the singleton
instance of the appropriate debugging class, created when the
gcc::context is built in toplev initialization.

That would be a much cleaner approach (assuming vfuncs would be
acceptable for the debug hooks) - though I don't yet have a working
patch for this.

So for now I went with the less elegant but equally effective approach.

> > diff --git a/gcc/passes.c b/gcc/passes.c
> > index 5001c3d..772993d 100644
> > --- a/gcc/passes.c
> > +++ b/gcc/passes.c
> > @@ -1944,6 +1944,7 @@ execute_ipa_summary_passes (ipa_opt_pass_d *ipa_pass)
> >   	  if (pass->tv_id)
> >   	    timevar_push (pass->tv_id);
> >
> > +	  current_pass = pass;
> >   	  ipa_pass->generate_summary ();
> Presumably it doesn't make sense to push this down into 
> ipa_pass->generate_summary?

The issue here is that ipa-pure-const.c has some state that ought to be
member data of the pass_ipa_pure_const instance.  The patch makes that
change, the issue is that there's no way to get at the relevant pass
instance just from the IPA pass hooks.  They're not vfuncs, they're just
tables of function pointers.  

(I skipped this part of the conversion of passes to C++ classes since
there wasn't a good way to handle the equivalent of checking to see if a
function ptr is NULL when turning it into a vfunc).

The easy fix is to simply ensure that "current_pass" is set  before
jumping through the IPA hook, so that pass_ipa_pure_const's
implementation can find its instance and get at the data.

> OK with the nits above addressed.

Thanks for the review.

Dave

Re: [PATCH 4/5] State cleanups -- also note for MPX work

[Jeff Law]

On 10/16/14 18:41, David Malcolm wrote:
>>
>> Anyway, just thought it was worth explicitly pointing out that these two
>> hunks of work, while they're tackling totally different issues may
>> conflict because of an implementation of the MPX bits.
>
> Presumably my state cleanup patch isn't going to break the MPX bits
> though?  I haven't been following them in detail.
You won't break MPX, but they might break you due to state being saved 
in file-scoped statics.  I haven't really looked at the state they've 
got and whether or not it needs to be cleared in a JIT world.


>
> I should spell out that I haven't exhaustively hunted down every
> file-scoped static in the source tree; this work merely fixes enough
> such state to ensure that the testsuite passes:
Noted.  We probably need to be more systematic in the future though. 
What you've done is probably good enough for now though.  Particularly 
since this just affects JIT and we can iterate on the file statics.


>
> I've got some ideas for automatically tracking down additional
> file-scoped static state by analysis of source or of the .o files.  Am I
> right in assuming I can regard that as followup bugfixing?
Yes, this would be fine for follow-up fixing.
>
>
>>> diff --git a/gcc/cgraph.h b/gcc/cgraph.h
>>> index 20b5c4e..794403d 100644
>>> --- a/gcc/cgraph.h
>>> +++ b/gcc/cgraph.h
>>> @@ -2042,6 +2043,8 @@ bool resolution_used_from_other_file_p (enum ld_plugin_symbol_resolution);
>>>    extern bool gimple_check_call_matching_types (gimple, tree, bool);
>>>
>>>    /* In cgraphunit.c  */
>>> +void cgraphunit_c_finalize (void);
>> Combine this with the declaration of init_lowered_empty_function so that
>> the references to cgraphunit's functions are together.
>
> I believe they are, since it's immediately followed by...
Nevermind, I'm an idiot.  Mentally had a "+" in front of the "In 
cgraphunit" line and thought you'd added another block of stuff for 
cgraphunit rather than just adding stuff to the existing block.

> FWIW I've been experimenting with making the debugging structs be
> classes, with the hooks becoming virtual functions.  Then all of this
> debugging state could be turned into member data of the singleton
> instance of the appropriate debugging class, created when the
> gcc::context is built in toplev initialization.
>
> That would be a much cleaner approach (assuming vfuncs would be
> acceptable for the debug hooks) - though I don't yet have a working
> patch for this.
>
> So for now I went with the less elegant but equally effective approach.
Understood.


>
>>> diff --git a/gcc/passes.c b/gcc/passes.c
>>> index 5001c3d..772993d 100644
>>> --- a/gcc/passes.c
>>> +++ b/gcc/passes.c
>>> @@ -1944,6 +1944,7 @@ execute_ipa_summary_passes (ipa_opt_pass_d *ipa_pass)
>>>    	  if (pass->tv_id)
>>>    	    timevar_push (pass->tv_id);
>>>
>>> +	  current_pass = pass;
>>>    	  ipa_pass->generate_summary ();
>> Presumably it doesn't make sense to push this down into
>> ipa_pass->generate_summary?
>
> The issue here is that ipa-pure-const.c has some state that ought to be
> member data of the pass_ipa_pure_const instance.  The patch makes that
> change, the issue is that there's no way to get at the relevant pass
> instance just from the IPA pass hooks.  They're not vfuncs, they're just
> tables of function pointers.
>
> (I skipped this part of the conversion of passes to C++ classes since
> there wasn't a good way to handle the equivalent of checking to see if a
> function ptr is NULL when turning it into a vfunc).
>
> The easy fix is to simply ensure that "current_pass" is set  before
> jumping through the IPA hook, so that pass_ipa_pure_const's
> implementation can find its instance and get at the data.
Ok.  Thanks for the explanation.
jeff

Patches 5-10 of jit merger (was: Re: [PATCH 0/5] Merger of jit branch (v2))

[David Malcolm]

On Mon, 2014-10-13 at 13:45 -0400, David Malcolm wrote:
> I'd like to merge the JIT branch into trunk:
>   https://gcc.gnu.org/wiki/JIT
> 
> This is "v2" since it incorporates fixes for the various issues
> identified by Joseph in an earlier submission:
>   https://gcc.gnu.org/ml/gcc-patches/2014-09/msg02056.html
> 
> I've split up the current diff between trunk and the branch into 5
> areas for ease of review (and to allow for early merger of the
> supporting work, if it's deemed ready):
> 
> patch 1: exposes an entrypoint in libiberty that I need
> patch 2: configure and Makefile changes in "gcc"
> patch 3: timevar.h: Add an auto_timevar class
> patch 4: State cleanups in "gcc"
> patch 5: Add the "jit" code itself
> 
> [this is a diff of trunk r215958 aka
> e012cdc775868e9922f5fef9068a764546876d93 which is from 2014-10-06,
> vs jit branch version 75b3ee7acdc6de55354d65bb7d619386463e50a1].
> 
> I've successfully bootstrapped and regression-tested the cumulative
> result of all of the patches against a control build, building them
> both with --enable-host-shared, and with
>   --enable-languages=c,c++,objc,obj-c++,java,fortran,ada,go,lto
> adding ",jit" to the test build (both on x86_64-unknown-linux-gnu;
> Fedora 20).
> 
> There were no regressions vs the control build, and the patched build
> gains a jit.sum, with 4663 passes (and no failures).
> 
> OK for trunk?

Patch 5 seems to have been too large, even compressed, so I'm breaking
it up into separate pieces and compressing, giving 10 patches in total

Patches 1-4 are as above.

Patch 5: remaining JIT-related changes outside of the gcc/jit/ subdir

Patch 6: the core of the JIT implementation: the gcc/jit subdir

Patch 7: the testsuite: gcc/testsuite/jit.dg

Patch 8: sphinx-based documentation: the gcc/jit/docs subdir

Patch 9: texinfo documentation autogenerated from the sphinx sources.

Patch 10: the ChangeLog.jit logs from the branch.

[PATCH 05/10] JIT-related changes outside of jit subdir

[David Malcolm]

ChangeLog:
	* MAINTAINERS (Various Maintainers): Add myself as jit maintainer.

contrib/ChangeLog:
	* jit-coverage-report.py: New file: a script to print crude
	code-coverage information for the libgccjit API.

gcc/ChangeLog:
	* doc/install.texi (--enable-host-shared): Specify that this is
	required when building libgccjit.
	* timevar.def (TV_JIT_REPLAY): New.
	(TV_ASSEMBLE): New.
	(TV_LINK): New.
	(TV_LOAD): New.
---
 MAINTAINERS                    |  1 +
 contrib/jit-coverage-report.py | 67 ++++++++++++++++++++++++++++++++++++++++++
 gcc/doc/install.texi           |  2 +-
 gcc/timevar.def                |  6 ++++
 4 files changed, 75 insertions(+), 1 deletion(-)
 create mode 100644 contrib/jit-coverage-report.py

diff --git a/MAINTAINERS b/MAINTAINERS
index 5dca84e..1fa679e 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -260,6 +260,7 @@ testsuite		Janis Johnson		<janisjo@codesourcery.com>
 register allocation	Vladimir Makarov	<vmakarov@redhat.com>
 gdbhooks.py		David Malcolm		<dmalcolm@redhat.com>
 SLSR			Bill Schmidt		<wschmidt@linux.vnet.ibm.com>
+jit			David Malcolm		<dmalcolm@redhat.com>
 
 Note that individuals who maintain parts of the compiler need approval to
 check in changes outside of the parts of the compiler they maintain.
diff --git a/contrib/jit-coverage-report.py b/contrib/jit-coverage-report.py
new file mode 100644
index 0000000..529336f
--- /dev/null
+++ b/contrib/jit-coverage-report.py
@@ -0,0 +1,67 @@
+#! /usr/bin/python
+#
+# Print a report on which libgccjit.so symbols are used in which test
+# cases, and which lack test coverage.  Tested with Python 2.7 and 3.2
+# To be run from the root directory of the source tree.
+#
+# Copyright (C) 2014 Free Software Foundation, Inc.
+# Written by David Malcolm <dmalcolm@redhat.com>.
+#
+# This script is Free Software, and it can be copied, distributed and
+# modified as defined in the GNU General Public License.  A copy of
+# its license can be downloaded from http://www.gnu.org/copyleft/gpl.html
+
+from collections import Counter
+import glob
+import re
+import sys
+
+def parse_map_file(path):
+    """
+    Parse libgccjit.map, returning the symbols in the API as a list of str.
+    """
+    syms = []
+    with open(path) as f:
+        for line in f:
+            m = re.match('^\s+([a-z_]+);$', line)
+            if m:
+                syms.append(m.group(1))
+    return syms
+
+def parse_test_case(path):
+    """
+    Locate all symbol-like things in a C test case, yielding
+    them as a sequence of str.
+    """
+    with open(path) as f:
+        for line in f:
+            for m in re.finditer('([_A-Za-z][_A-Za-z0-9]*)', line):
+                yield m.group(1)
+
+def find_test_cases():
+    for path in glob.glob('gcc/testsuite/jit.dg/*.[ch]'):
+        yield path
+
+api_syms = parse_map_file('gcc/jit/libgccjit.map')
+
+syms_in_test_cases = {}
+for path in find_test_cases():
+    syms_in_test_cases[path] = list(parse_test_case(path))
+
+uses = Counter()
+for sym in sorted(api_syms):
+    print('symbol: %s' % sym)
+    uses[sym] = 0
+    for path in syms_in_test_cases:
+        count = syms_in_test_cases[path].count(sym)
+        uses[sym] += count
+        if count:
+            print('  uses in %s: %i' % (path, count))
+    if uses[sym] == 0:
+        print('  NEVER USED')
+    sys.stdout.write('\n')
+
+layout = '%40s  %5s  %s'
+print(layout % ('SYMBOL', 'USES', 'HISTOGRAM'))
+for sym, count in uses.most_common():
+    print(layout % (sym, count, '*' * count if count else 'UNUSED'))
diff --git a/gcc/doc/install.texi b/gcc/doc/install.texi
index 75ac9a6..c92de28 100644
--- a/gcc/doc/install.texi
+++ b/gcc/doc/install.texi
@@ -954,7 +954,7 @@ Specify that the @emph{host} code should be built into position-independent
 machine code (with -fPIC), allowing it to be used within shared libraries,
 but yielding a slightly slower compiler.
 
-Currently this option is only of use to people developing GCC itself.
+This option is required when building the libgccjit.so library.
 
 Contrast with @option{--enable-shared}, which affects @emph{target}
 libraries.
diff --git a/gcc/timevar.def b/gcc/timevar.def
index a04d05c..b406c16 100644
--- a/gcc/timevar.def
+++ b/gcc/timevar.def
@@ -277,3 +277,9 @@ DEFTIMEVAR (TV_VERIFY_LOOP_CLOSED    , "verify loop closed")
 DEFTIMEVAR (TV_VERIFY_RTL_SHARING    , "verify RTL sharing")
 DEFTIMEVAR (TV_REBUILD_FREQUENCIES   , "rebuild frequencies")
 DEFTIMEVAR (TV_REPAIR_LOOPS	     , "repair loop structures")
+
+/* Stuff used by libgccjit.so.  */
+DEFTIMEVAR (TV_JIT_REPLAY	     , "replay of JIT client activity")
+DEFTIMEVAR (TV_ASSEMBLE	     , "assemble JIT code")
+DEFTIMEVAR (TV_LINK		     , "link JIT code")
+DEFTIMEVAR (TV_LOAD		     , "load JIT result")
-- 
1.8.5.3

Re: [PATCH 05/10] JIT-related changes outside of jit subdir

[Jeff Law]

On 10/14/14 09:10, David Malcolm wrote:
> ChangeLog:
> 	* MAINTAINERS (Various Maintainers): Add myself as jit maintainer.
>
> contrib/ChangeLog:
> 	* jit-coverage-report.py: New file: a script to print crude
> 	code-coverage information for the libgccjit API.
>
> gcc/ChangeLog:
> 	* doc/install.texi (--enable-host-shared): Specify that this is
> 	required when building libgccjit.
> 	* timevar.def (TV_JIT_REPLAY): New.
> 	(TV_ASSEMBLE): New.
> 	(TV_LINK): New.
> 	(TV_LOAD): New.
OK if/when rest of JIT bits are approved.

jeff

Re: [PATCH 05/10] JIT-related changes outside of jit subdir

[Joseph S. Myers]

Although Sphinx isn't a build dependency, as a dependency for 
regenerating checked-in files I think it should be documented in 
install.texi (like autoconf, gettext, etc.).

-- 
Joseph S. Myers
joseph@codesourcery.com

[jit] Add Sphinx to install.texi

[David Malcolm]

On Fri, 2014-10-17 at 21:25 +0000, Joseph S. Myers wrote:
> Although Sphinx isn't a build dependency, as a dependency for 
> regenerating checked-in files I think it should be documented in 
> install.texi (like autoconf, gettext, etc.).

Does this look OK?  (Committed to branch dmalcolm/jit for now)

gcc/ChangeLog.jit:
	* doc/install.texi (Tools/packages necessary for modifying GCC):
	Add Sphinx.
---
 gcc/ChangeLog.jit    | 5 +++++
 gcc/doc/install.texi | 5 +++++
 2 files changed, 10 insertions(+)

diff --git a/gcc/ChangeLog.jit b/gcc/ChangeLog.jit
index 8dec312..bcd72a4 100644
--- a/gcc/ChangeLog.jit
+++ b/gcc/ChangeLog.jit
@@ -1,5 +1,10 @@
 2014-10-20  David Malcolm  <dmalcolm@redhat.com>
 
+	* doc/install.texi (Tools/packages necessary for modifying GCC):
+	Add Sphinx.
+
+2014-10-20  David Malcolm  <dmalcolm@redhat.com>
+
 	* Makefile.in (pkgconfigdir): Drop this.
 	(installdirs): Likewise.
 	* configure.ac (gcc_version): Don't AC_SUBST this.
diff --git a/gcc/doc/install.texi b/gcc/doc/install.texi
index c92de28..b4027ea 100644
--- a/gcc/doc/install.texi
+++ b/gcc/doc/install.texi
@@ -491,6 +491,11 @@ Necessary for running @command{texi2dvi} and @command{texi2pdf}, which
 are used when running @command{make dvi} or @command{make pdf} to create
 DVI or PDF files, respectively.
 
+@item Sphinx (any working version)
+
+Necessary to regenerate @file{jit/docs/_build/texinfo} from the .rst
+files in the directories below @file{jit/docs}.
+
 @item SVN (any version)
 @itemx SSH (any version)
 
-- 
1.7.11.7

Re: [jit] Add Sphinx to install.texi

[Joseph S. Myers]

On Mon, 20 Oct 2014, David Malcolm wrote:

> +Necessary to regenerate @file{jit/docs/_build/texinfo} from the .rst

I'd say @file{.rst}, but otherwise looks OK to me.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: [jit] Add Sphinx to install.texi

[Gerald Pfeifer]

On Monday 2014-10-20 15:19, David Malcolm wrote:
> +@item Sphinx (any working version)

As opposed to "any non-working version"? ;-)  I'd just omit 
"working" from this.

Otherwise this looks good to me, except...

> +Necessary to regenerate @file{jit/docs/_build/texinfo} from the .rst
> +files in the directories below @file{jit/docs}.

...that it probably should read @file{.rst}?

Gerald

Re: [jit] Add Sphinx to install.texi

[David Malcolm]

On Tue, 2014-10-21 at 18:15 +0200, Gerald Pfeifer wrote:
> On Monday 2014-10-20 15:19, David Malcolm wrote:
> > +@item Sphinx (any working version)
> 
> As opposed to "any non-working version"? ;-)  I'd just omit 
> "working" from this.

FWIW, all my development has been with Sphinx 1.1.3, which is from
2012-03-10.  My guess is that it work with earlier versions: looking at
Sphinx's own docs, the most recently-added feature that I'm using is
domains, which is marked as 1.0 or later.

> Otherwise this looks good to me, except...
> 
> > +Necessary to regenerate @file{jit/docs/_build/texinfo} from the .rst
> > +files in the directories below @file{jit/docs}.
> 
> ...that it probably should read @file{.rst}?
> 
> Gerald

[jit] Tweaks to install.texi

[David Malcolm]

On Tue, 2014-10-21 at 15:24 -0400, David Malcolm wrote:
> On Tue, 2014-10-21 at 18:15 +0200, Gerald Pfeifer wrote:
> > On Monday 2014-10-20 15:19, David Malcolm wrote:
> > > +@item Sphinx (any working version)
> > 
> > As opposed to "any non-working version"? ;-)  I'd just omit 
> > "working" from this.
> 
> FWIW, all my development has been with Sphinx 1.1.3, which is from
> 2012-03-10.  My guess is that it work with earlier versions: looking at
> Sphinx's own docs, the most recently-added feature that I'm using is
> domains, which is marked as 1.0 or later.
> 
> > Otherwise this looks good to me, except...
> > 
> > > +Necessary to regenerate @file{jit/docs/_build/texinfo} from the .rst
> > > +files in the directories below @file{jit/docs}.
> > 
> > ...that it probably should read @file{.rst}?

Committed to branch dmalcolm/jit:

gcc/ChangeLog.jit:
	* doc/install.texi (Tools/packages necessary for modifying GCC):
	Specify that sphinx version 1.0 or later is required.  Wrap .rst
	inside an @file command.
---
 gcc/ChangeLog.jit    | 6 ++++++
 gcc/doc/install.texi | 4 ++--
 2 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/gcc/ChangeLog.jit b/gcc/ChangeLog.jit
index 1c7f585..613084e 100644
--- a/gcc/ChangeLog.jit
+++ b/gcc/ChangeLog.jit
@@ -1,3 +1,9 @@
+2014-10-29  David Malcolm  <dmalcolm@redhat.com>
+
+	* doc/install.texi (Tools/packages necessary for modifying GCC):
+	Specify that Sphinx version 1.0 or later is required.  Wrap .rst
+	inside an @file command.
+
 2014-10-22  David Malcolm  <dmalcolm@redhat.com>
 
 	* cgraph.h: Drop now-redundant prototype of ipa_cp_c_finalize,
diff --git a/gcc/doc/install.texi b/gcc/doc/install.texi
index 5add9f6..ef7656c 100644
--- a/gcc/doc/install.texi
+++ b/gcc/doc/install.texi
@@ -491,9 +491,9 @@ Necessary for running @command{texi2dvi} and @command{texi2pdf}, which
 are used when running @command{make dvi} or @command{make pdf} to create
 DVI or PDF files, respectively.
 
-@item Sphinx (any working version)
+@item Sphinx version 1.0 (or later)
 
-Necessary to regenerate @file{jit/docs/_build/texinfo} from the .rst
+Necessary to regenerate @file{jit/docs/_build/texinfo} from the @file{.rst}
 files in the directories below @file{jit/docs}.
 
 @item SVN (any version)
-- 
1.7.11.7

[PATCH 06/10] Heart of the JIT implementation (was: Re: [PATCH 0/5] Merger of jit branch (v2))

[David Malcolm]

[-- Attachment #1: Type: text/plain, Size: 2763 bytes --]

On Tue, 2014-10-14 at 11:09 -0400, David Malcolm wrote:
> On Mon, 2014-10-13 at 13:45 -0400, David Malcolm wrote:
> > I'd like to merge the JIT branch into trunk:
> >   https://gcc.gnu.org/wiki/JIT
> > 
> > This is "v2" since it incorporates fixes for the various issues
> > identified by Joseph in an earlier submission:
> >   https://gcc.gnu.org/ml/gcc-patches/2014-09/msg02056.html
> > 
> > I've split up the current diff between trunk and the branch into 5
> > areas for ease of review (and to allow for early merger of the
> > supporting work, if it's deemed ready):
> > 
> > patch 1: exposes an entrypoint in libiberty that I need
> > patch 2: configure and Makefile changes in "gcc"
> > patch 3: timevar.h: Add an auto_timevar class
> > patch 4: State cleanups in "gcc"
> > patch 5: Add the "jit" code itself
> > 
> > [this is a diff of trunk r215958 aka
> > e012cdc775868e9922f5fef9068a764546876d93 which is from 2014-10-06,
> > vs jit branch version 75b3ee7acdc6de55354d65bb7d619386463e50a1].
> > 
> > I've successfully bootstrapped and regression-tested the cumulative
> > result of all of the patches against a control build, building them
> > both with --enable-host-shared, and with
> >   --enable-languages=c,c++,objc,obj-c++,java,fortran,ada,go,lto
> > adding ",jit" to the test build (both on x86_64-unknown-linux-gnu;
> > Fedora 20).
> > 
> > There were no regressions vs the control build, and the patched build
> > gains a jit.sum, with 4663 passes (and no failures).
> > 
> > OK for trunk?
> 
> Patch 5 seems to have been too large, even compressed, so I'm breaking
> it up into separate pieces and compressing, giving 10 patches in total
> 
> Patches 1-4 are as above.
> 
> Patch 5: remaining JIT-related changes outside of the gcc/jit/ subdir
> 
> Patch 6: the core of the JIT implementation: the gcc/jit subdir
> 
> Patch 7: the testsuite: gcc/testsuite/jit.dg
> 
> Patch 8: sphinx-based documentation: the gcc/jit/docs subdir
> 
> Patch 9: texinfo documentation autogenerated from the sphinx sources.
> 
> Patch 10: the ChangeLog.jit logs from the branch.

This commit adds the gcc/jit subdirectory, implementing the library,
which looks like a frontend named "jit" from the POV of the rest of the
gcc code.

gcc/jit/ChangeLog:

        * Make-lang.in: New.
        * TODO.rst: New.
        * config-lang.in: New.
        * dummy-frontend.c: New.
        * jit-builtins.c: New.
        * jit-builtins.h: New.
        * jit-common.h: New.
        * jit-playback.c: New.
        * jit-playback.h: New.
        * jit-recording.c: New.
        * jit-recording.h: New.
        * libgccjit++.h: New.
        * libgccjit.c: New.
        * libgccjit.h: New.
        * libgccjit.map: New.
        * libgccjit.pc.in: New.
        * notes.txt: New.


[-- Attachment #2: 0006-Heart-of-the-JIT-implementation.patch.gz --]
[-- Type: application/x-gzip, Size: 70517 bytes --]

Re: [PATCH 06/10] Heart of the JIT implementation (was: Re: [PATCH 0/5] Merger of jit branch (v2))

[Joseph S. Myers]

Does libgccjit.so end up getting linked with -static-libstdc++ 
-static-libgcc?  If so, that could be problematic (are static libstdc++ 
and libgcc necessarily built as PIC so it's even possible to embed them 
into a shared library?).  It's certainly not clear that the 
-static-libstdc++ -static-libgcc default for building the compiler 
executables is the right one for building libgccjit.so.

The dump file handling appears to have no I/O error checking (no checking 
for error on fopen, nothing obvious to prevent fwrite to a NULL m_file if 
fopen did have an error, no checking for error on fclose (or fwrite)).

-- 
Joseph S. Myers
joseph@codesourcery.com

[jit] Error-handling within gcc::jit::dump

[David Malcolm]

On Fri, 2014-10-17 at 21:52 +0000, Joseph S. Myers wrote:
[...snip static linkage discussion...]

> The dump file handling appears to have no I/O error checking (no checking 
> for error on fopen, nothing obvious to prevent fwrite to a NULL m_file if 
> fopen did have an error, no checking for error on fclose (or fwrite)).

Thanks.

Does the following look OK?  (I've committed it to branch
dmalcolm/jit)

gcc/jit/ChangeLog.jit:
	* jit-recording.c (gcc::jit::dump::dump): Handle fopen failures
	by emitting an error on the context.
	(gcc::jit::dump::~dump): Likewise for fclose failures.
	(gcc::jit::dump::write): Don't attempt further work if the fopen
	failed.  Handle fwrite failures by emitting an error on the
	context.
---
 gcc/jit/ChangeLog.jit   |  9 +++++++++
 gcc/jit/jit-recording.c | 25 ++++++++++++++++++++++---
 2 files changed, 31 insertions(+), 3 deletions(-)

diff --git a/gcc/jit/ChangeLog.jit b/gcc/jit/ChangeLog.jit
index 9a36dfd..02664f0 100644
--- a/gcc/jit/ChangeLog.jit
+++ b/gcc/jit/ChangeLog.jit
@@ -1,5 +1,14 @@
 2014-10-20  David Malcolm  <dmalcolm@redhat.com>
 
+	* jit-recording.c (gcc::jit::dump::dump): Handle fopen failures
+	by emitting an error on the context.
+	(gcc::jit::dump::~dump): Likewise for fclose failures.
+	(gcc::jit::dump::write): Don't attempt further work if the fopen
+	failed.  Handle fwrite failures by emitting an error on the
+	context.
+
+2014-10-20  David Malcolm  <dmalcolm@redhat.com>
+
 	* Make-lang.in (jit.install-common): Drop installation of
 	libgccjit.pc.
 	* config-lang.in (outputs): Drop jit/libgccjit.pc.
diff --git a/gcc/jit/jit-recording.c b/gcc/jit/jit-recording.c
index 32ce49b..5a97f23 100644
--- a/gcc/jit/jit-recording.c
+++ b/gcc/jit/jit-recording.c
@@ -47,13 +47,25 @@ dump::dump (recording::context &ctxt,
   m_line (0),
   m_column (0)
 {
-  m_file  = fopen (filename, "w");
+  m_file = fopen (filename, "w");
+  if (!m_file)
+    ctxt.add_error (NULL,
+		    "error opening dump file %s for writing: %s",
+		    filename,
+		    xstrerror (errno));
 }
 
 dump::~dump ()
 {
   if (m_file)
-    fclose (m_file);
+    {
+      int err = fclose (m_file);
+      if (err)
+	m_ctxt.add_error (NULL,
+			  "error closing dump file %s: %s",
+			  m_filename,
+			  xstrerror (errno));
+    }
 }
 
 /* Write the given message to the dump, using printf-formatting
@@ -67,6 +79,11 @@ dump::write (const char *fmt, ...)
   va_list ap;
   char *buf = NULL;
 
+  /* If there was an error opening the file, we've already reported it.
+     Don't attempt further work.  */
+  if (!m_file)
+    return;
+
   va_start (ap, fmt);
   vasprintf (&buf, fmt, ap);
   va_end (ap);
@@ -78,7 +95,9 @@ dump::write (const char *fmt, ...)
       return;
     }
 
-  fwrite (buf, strlen (buf), 1, m_file);
+  if (fwrite (buf, strlen (buf), 1, m_file) != 1)
+    m_ctxt.add_error (NULL, "error writing to dump file %s",
+		      m_filename);
 
   /* Update line/column: */
   for (const char *ptr = buf; *ptr; ptr++)
-- 
1.7.11.7

Re: [jit] Error-handling within gcc::jit::dump

[Joseph S. Myers]

On Mon, 20 Oct 2014, David Malcolm wrote:

> On Fri, 2014-10-17 at 21:52 +0000, Joseph S. Myers wrote:
> [...snip static linkage discussion...]
> 
> > The dump file handling appears to have no I/O error checking (no checking 
> > for error on fopen, nothing obvious to prevent fwrite to a NULL m_file if 
> > fopen did have an error, no checking for error on fclose (or fwrite)).
> 
> Thanks.
> 
> Does the following look OK?  (I've committed it to branch
> dmalcolm/jit)

Seems fine with me.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: [PATCH 06/10] Heart of the JIT implementation (was: Re: [PATCH 0/5] Merger of jit branch (v2))

[David Malcolm]

On Fri, 2014-10-17 at 21:52 +0000, Joseph S. Myers wrote:
> Does libgccjit.so end up getting linked with -static-libstdc++ 
> -static-libgcc? 
(sorry for belated reply)

It does when built with a bootstrap, but doesn't with
--disable-bootstrap.  In the former case, the builddir's gcc/Makefile in
the final stage has:
  LDFLAGS = -static-libstdc++ -static-libgcc
and we use $(LDFLAGS) in the src's gcc/jit/Make-lang.in in the rule for
$(LIBGCCJIT_FILENAME).

As far as I can tell, this comes from these lines:
   if test "$poststage1_libs" = ""; then
     poststage1_ldflags="-static-libstdc++ -static-libgcc"
   fi])
in the top-level configure.ac's code for:
   AC_ARG_WITH(boot-ldflags,

> If so, that could be problematic (are static libstdc++ 
> and libgcc necessarily built as PIC so it's even possible to embed them 
> into a shared library?).

It works (on this machine at least); the built libgccjit.so library
successfully runs the test suite.

Looking at the build logs, I see:
  -fPIC
within the xgcc args in the libgcc build logs, and
  -prefer-pic
within the libtool args in the libstdc++ build, which according to the
libtool docs means "try to build only PIC objects".

Hence I believe although we're currently statically-linking libgcc and
libstdc++ into libgccjit.so, they're position-independent.

> It's certainly not clear that the 
> -static-libstdc++ -static-libgcc default for building the compiler 
> executables is the right one for building libgccjit.so.

Agreed, but it's unclear to me what the default should be, and how to go
about fixing it.

That said, it appears that people who want the libgccjit.so to
dynamically-link against libgcc and libstdc++ can already do so, by
selecting appropriate configuration flags (though it's not quite clear
to me what the incantation is; presumably $poststage1_libs needs to be
non-empty to avoid triggering this clause:
   if test "$poststage1_libs" = ""; then
     poststage1_ldflags="-static-libstdc++ -static-libgcc"
   fi
, right ?)

Or maybe I could turn off that clause if the "--enable-host-shared" has
been specified, so it defaults to shared linkage for that setting?

Do you have thoughts on how I should address this?  Also, given that the
code works as-is, is resolving this a blocker for merging the jit
branch?  (I've been rebasing, and plan to repost the fixed-up patches
for review shortly)

Thanks
Dave


> The dump file handling appears to have no I/O error checking (no
> checking 
> for error on fopen, nothing obvious to prevent fwrite to a NULL m_file
> if 
> fopen did have an error, no checking for error on fclose (or fwrite)).

(already addressed in a different reply)

Re: [PATCH 06/10] Heart of the JIT implementation (was: Re: [PATCH 0/5] Merger of jit branch (v2))

[Joseph S. Myers]

On Thu, 30 Oct 2014, David Malcolm wrote:

> Looking at the build logs, I see:
>   -fPIC
> within the xgcc args in the libgcc build logs, and

That seems to depend on t-libgcc-pic, but that appears to cover most 
likely hosts (including any where I can be confident PIC is actually 
needed for shared libraries).

> > It's certainly not clear that the 
> > -static-libstdc++ -static-libgcc default for building the compiler 
> > executables is the right one for building libgccjit.so.
> 
> Agreed, but it's unclear to me what the default should be, and how to go
> about fixing it.
> 
> That said, it appears that people who want the libgccjit.so to
> dynamically-link against libgcc and libstdc++ can already do so, by

Can do so for libgccjit.so but not the compiler executables?

(There are no doubt cases where it makes sense for the compiler 
executables to be dynamically linked with the shared libraries, but also I 
think cases for linking only libgccjit.so with the shared libraries.)

> Do you have thoughts on how I should address this?  Also, given that the

No.

> code works as-is, is resolving this a blocker for merging the jit
> branch?  (I've been rebasing, and plan to repost the fixed-up patches
> for review shortly)

I don't see it as a blocker, but I would not be surprised if having the 
libraries statically linked into libgccjit.so causes problems (is it safe 
to have two completely separate copies of libstdc++ in the same process?  
I don't know.).

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: [PATCH 06/10] Heart of the JIT implementation

[Jeff Law]

On 10/30/14 18:50, Joseph S. Myers wrote:

>
> I don't see it as a blocker, but I would not be surprised if having the
> libraries statically linked into libgccjit.so causes problems (is it safe
> to have two completely separate copies of libstdc++ in the same process?
> I don't know.).
Jason, Jakub, Aldy, Ben & others looked at this a few years ago for Red 
Hat and our conclusion was that having separate copies of libstdc++ in 
the same process might work sometimes, but ultimately wasn't anything we 
would be willing to stand behind and support at any level.

jeff

[PATCH 07/10] Testsuite for the JIT (Re: Patches 5-10 of jit merger (was: Re: [PATCH 0/5] Merger of jit branch (v2)))

[David Malcolm]

[-- Attachment #1: Type: text/plain, Size: 2111 bytes --]

On Tue, 2014-10-14 at 11:09 -0400, David Malcolm wrote:
> On Mon, 2014-10-13 at 13:45 -0400, David Malcolm wrote:
> > I'd like to merge the JIT branch into trunk:
> >   https://gcc.gnu.org/wiki/JIT
> > 
> > This is "v2" since it incorporates fixes for the various issues
> > identified by Joseph in an earlier submission:
> >   https://gcc.gnu.org/ml/gcc-patches/2014-09/msg02056.html
> > 
> > I've split up the current diff between trunk and the branch into 5
> > areas for ease of review (and to allow for early merger of the
> > supporting work, if it's deemed ready):
> > 
> > patch 1: exposes an entrypoint in libiberty that I need
> > patch 2: configure and Makefile changes in "gcc"
> > patch 3: timevar.h: Add an auto_timevar class
> > patch 4: State cleanups in "gcc"
> > patch 5: Add the "jit" code itself
> > 
> > [this is a diff of trunk r215958 aka
> > e012cdc775868e9922f5fef9068a764546876d93 which is from 2014-10-06,
> > vs jit branch version 75b3ee7acdc6de55354d65bb7d619386463e50a1].
> > 
> > I've successfully bootstrapped and regression-tested the cumulative
> > result of all of the patches against a control build, building them
> > both with --enable-host-shared, and with
> >   --enable-languages=c,c++,objc,obj-c++,java,fortran,ada,go,lto
> > adding ",jit" to the test build (both on x86_64-unknown-linux-gnu;
> > Fedora 20).
> > 
> > There were no regressions vs the control build, and the patched build
> > gains a jit.sum, with 4663 passes (and no failures).
> > 
> > OK for trunk?
> 
> Patch 5 seems to have been too large, even compressed, so I'm breaking
> it up into separate pieces and compressing, giving 10 patches in total
> 
> Patches 1-4 are as above.
> 
> Patch 5: remaining JIT-related changes outside of the gcc/jit/ subdir
> 
> Patch 6: the core of the JIT implementation: the gcc/jit subdir
> 
> Patch 7: the testsuite: gcc/testsuite/jit.dg

> Patch 8: sphinx-based documentation: the gcc/jit/docs subdir
> 
> Patch 9: texinfo documentation autogenerated from the sphinx sources.
> 
> Patch 10: the ChangeLog.jit logs from the branch.

Here's patch 7, the testsuite.


[-- Attachment #2: 0007-Testsuite-for-the-JIT.patch.gz --]
[-- Type: application/x-gzip, Size: 37628 bytes --]

Re: [PATCH 07/10] Testsuite for the JIT (Re: Patches 5-10 of jit merger

[Jeff Law]

On 10/14/14 09:18, David Malcolm wrote:
> On Tue, 2014-10-14 at 11:09 -0400, David Malcolm wrote:
>> On Mon, 2014-10-13 at 13:45 -0400, David Malcolm wrote:
>>> I'd like to merge the JIT branch into trunk:
>>>    https://gcc.gnu.org/wiki/JIT
>>>
>>> This is "v2" since it incorporates fixes for the various issues
>>> identified by Joseph in an earlier submission:
>>>    https://gcc.gnu.org/ml/gcc-patches/2014-09/msg02056.html
>>>
>>> I've split up the current diff between trunk and the branch into 5
>>> areas for ease of review (and to allow for early merger of the
>>> supporting work, if it's deemed ready):
>>>
>>> patch 1: exposes an entrypoint in libiberty that I need
>>> patch 2: configure and Makefile changes in "gcc"
>>> patch 3: timevar.h: Add an auto_timevar class
>>> patch 4: State cleanups in "gcc"
>>> patch 5: Add the "jit" code itself
>>>
>>> [this is a diff of trunk r215958 aka
>>> e012cdc775868e9922f5fef9068a764546876d93 which is from 2014-10-06,
>>> vs jit branch version 75b3ee7acdc6de55354d65bb7d619386463e50a1].
>>>
>>> I've successfully bootstrapped and regression-tested the cumulative
>>> result of all of the patches against a control build, building them
>>> both with --enable-host-shared, and with
>>>    --enable-languages=c,c++,objc,obj-c++,java,fortran,ada,go,lto
>>> adding ",jit" to the test build (both on x86_64-unknown-linux-gnu;
>>> Fedora 20).
>>>
>>> There were no regressions vs the control build, and the patched build
>>> gains a jit.sum, with 4663 passes (and no failures).
>>>
>>> OK for trunk?
>>
>> Patch 5 seems to have been too large, even compressed, so I'm breaking
>> it up into separate pieces and compressing, giving 10 patches in total
>>
>> Patches 1-4 are as above.
>>
>> Patch 5: remaining JIT-related changes outside of the gcc/jit/ subdir
>>
>> Patch 6: the core of the JIT implementation: the gcc/jit subdir
>>
>> Patch 7: the testsuite: gcc/testsuite/jit.dg
>
>> Patch 8: sphinx-based documentation: the gcc/jit/docs subdir
>>
>> Patch 9: texinfo documentation autogenerated from the sphinx sources.
>>
>> Patch 10: the ChangeLog.jit logs from the branch.
>
> Here's patch 7, the testsuite.
OK when rest of JIT bits are approved.

I just did some very rough sanity checking -- the details of the 
testsuite I'm totally leaving in your hands :-)

jeff

Re: [PATCH 07/10] Testsuite for the JIT (Re: Patches 5-10 of jit merger

[Mike Stump]

On Oct 15, 2014, at 10:49 AM, Jeff Law <law@redhat.com> wrote:
> I just did some very rough sanity checking -- the details of the testsuite I'm totally leaving in your hands :-)

Likewise, I’m likely to leave it totally in your hands.  If you have any specific questions or concerns, asking would be the way to go.  For example, would be nice if a cross tester found any issues before it went in, would be nice if a canadian cross tester found any issues.  But even for these, if it went in totally broken, not a big deal, as you can put in a target != host return into the .exp file and just avoid everything with 2 lines of code.  People that care, can then test it out and fix any issues found, and then remove those two lines.

[PATCH 10/10] ChangeLog files (Re: Patches 5-10 of jit merger (was: Re: [PATCH 0/5] Merger of jit branch (v2)))

[David Malcolm]

[-- Attachment #1: Type: text/plain, Size: 2123 bytes --]

On Tue, 2014-10-14 at 11:09 -0400, David Malcolm wrote:
> On Mon, 2014-10-13 at 13:45 -0400, David Malcolm wrote:
> > I'd like to merge the JIT branch into trunk:
> >   https://gcc.gnu.org/wiki/JIT
> > 
> > This is "v2" since it incorporates fixes for the various issues
> > identified by Joseph in an earlier submission:
> >   https://gcc.gnu.org/ml/gcc-patches/2014-09/msg02056.html
> > 
> > I've split up the current diff between trunk and the branch into 5
> > areas for ease of review (and to allow for early merger of the
> > supporting work, if it's deemed ready):
> > 
> > patch 1: exposes an entrypoint in libiberty that I need
> > patch 2: configure and Makefile changes in "gcc"
> > patch 3: timevar.h: Add an auto_timevar class
> > patch 4: State cleanups in "gcc"
> > patch 5: Add the "jit" code itself
> > 
> > [this is a diff of trunk r215958 aka
> > e012cdc775868e9922f5fef9068a764546876d93 which is from 2014-10-06,
> > vs jit branch version 75b3ee7acdc6de55354d65bb7d619386463e50a1].
> > 
> > I've successfully bootstrapped and regression-tested the cumulative
> > result of all of the patches against a control build, building them
> > both with --enable-host-shared, and with
> >   --enable-languages=c,c++,objc,obj-c++,java,fortran,ada,go,lto
> > adding ",jit" to the test build (both on x86_64-unknown-linux-gnu;
> > Fedora 20).
> > 
> > There were no regressions vs the control build, and the patched build
> > gains a jit.sum, with 4663 passes (and no failures).
> > 
> > OK for trunk?
> 
> Patch 5 seems to have been too large, even compressed, so I'm breaking
> it up into separate pieces and compressing, giving 10 patches in total
> 
> Patches 1-4 are as above.
> 
> Patch 5: remaining JIT-related changes outside of the gcc/jit/ subdir
> 
> Patch 6: the core of the JIT implementation: the gcc/jit subdir
> 
> Patch 7: the testsuite: gcc/testsuite/jit.dg
> 
> Patch 8: sphinx-based documentation: the gcc/jit/docs subdir
> 
> Patch 9: texinfo documentation autogenerated from the sphinx sources.
> 
> Patch 10: the ChangeLog.jit logs from the branch.

Finally, patch 10, the ChangeLog files.



[-- Attachment #2: 0010-ChangeLog-files.patch.gz --]
[-- Type: application/x-gzip, Size: 31890 bytes --]

Re: [PATCH 10/10] ChangeLog files (Re: Patches 5-10 of jit merger

[Jeff Law]

On 10/14/14 09:30, David Malcolm wrote:
> On Tue, 2014-10-14 at 11:09 -0400, David Malcolm wrote:
>> On Mon, 2014-10-13 at 13:45 -0400, David Malcolm wrote:
>>> I'd like to merge the JIT branch into trunk:
>>>    https://gcc.gnu.org/wiki/JIT
>>>
>>> This is "v2" since it incorporates fixes for the various issues
>>> identified by Joseph in an earlier submission:
>>>    https://gcc.gnu.org/ml/gcc-patches/2014-09/msg02056.html
>>>
>>> I've split up the current diff between trunk and the branch into 5
>>> areas for ease of review (and to allow for early merger of the
>>> supporting work, if it's deemed ready):
>>>
>>> patch 1: exposes an entrypoint in libiberty that I need
>>> patch 2: configure and Makefile changes in "gcc"
>>> patch 3: timevar.h: Add an auto_timevar class
>>> patch 4: State cleanups in "gcc"
>>> patch 5: Add the "jit" code itself
>>>
>>> [this is a diff of trunk r215958 aka
>>> e012cdc775868e9922f5fef9068a764546876d93 which is from 2014-10-06,
>>> vs jit branch version 75b3ee7acdc6de55354d65bb7d619386463e50a1].
>>>
>>> I've successfully bootstrapped and regression-tested the cumulative
>>> result of all of the patches against a control build, building them
>>> both with --enable-host-shared, and with
>>>    --enable-languages=c,c++,objc,obj-c++,java,fortran,ada,go,lto
>>> adding ",jit" to the test build (both on x86_64-unknown-linux-gnu;
>>> Fedora 20).
>>>
>>> There were no regressions vs the control build, and the patched build
>>> gains a jit.sum, with 4663 passes (and no failures).
>>>
>>> OK for trunk?
>>
>> Patch 5 seems to have been too large, even compressed, so I'm breaking
>> it up into separate pieces and compressing, giving 10 patches in total
>>
>> Patches 1-4 are as above.
>>
>> Patch 5: remaining JIT-related changes outside of the gcc/jit/ subdir
>>
>> Patch 6: the core of the JIT implementation: the gcc/jit subdir
>>
>> Patch 7: the testsuite: gcc/testsuite/jit.dg
>>
>> Patch 8: sphinx-based documentation: the gcc/jit/docs subdir
>>
>> Patch 9: texinfo documentation autogenerated from the sphinx sources.
>>
>> Patch 10: the ChangeLog.jit logs from the branch.
>
> Finally, patch 10, the ChangeLog files.
OK once the rest of the JIT stuff is approved.  Not going to look at 
these in any detail.

jeff

[PATCH 08/10] Documentation for the JIT library (Re: Patches 5-10 of jit merger)

[David Malcolm]

On Tue, 2014-10-14 at 11:09 -0400, David Malcolm wrote:
> On Mon, 2014-10-13 at 13:45 -0400, David Malcolm wrote:
> > I'd like to merge the JIT branch into trunk:
> >   https://gcc.gnu.org/wiki/JIT
> > 
> > This is "v2" since it incorporates fixes for the various issues
> > identified by Joseph in an earlier submission:
> >   https://gcc.gnu.org/ml/gcc-patches/2014-09/msg02056.html
> > 
> > I've split up the current diff between trunk and the branch into 5
> > areas for ease of review (and to allow for early merger of the
> > supporting work, if it's deemed ready):
> > 
> > patch 1: exposes an entrypoint in libiberty that I need
> > patch 2: configure and Makefile changes in "gcc"
> > patch 3: timevar.h: Add an auto_timevar class
> > patch 4: State cleanups in "gcc"
> > patch 5: Add the "jit" code itself
> > 
> > [this is a diff of trunk r215958 aka
> > e012cdc775868e9922f5fef9068a764546876d93 which is from 2014-10-06,
> > vs jit branch version 75b3ee7acdc6de55354d65bb7d619386463e50a1].
> > 
> > I've successfully bootstrapped and regression-tested the cumulative
> > result of all of the patches against a control build, building them
> > both with --enable-host-shared, and with
> >   --enable-languages=c,c++,objc,obj-c++,java,fortran,ada,go,lto
> > adding ",jit" to the test build (both on x86_64-unknown-linux-gnu;
> > Fedora 20).
> > 
> > There were no regressions vs the control build, and the patched build
> > gains a jit.sum, with 4663 passes (and no failures).
> > 
> > OK for trunk?
> 
> Patch 5 seems to have been too large, even compressed, so I'm breaking
> it up into separate pieces and compressing, giving 10 patches in total
> 
> Patches 1-4 are as above.
> 
> Patch 5: remaining JIT-related changes outside of the gcc/jit/ subdir
> 
> Patch 6: the core of the JIT implementation: the gcc/jit subdir
> 
> Patch 7: the testsuite: gcc/testsuite/jit.dg
> 
> Patch 8: sphinx-based documentation: the gcc/jit/docs subdir
> 
> Patch 9: texinfo documentation autogenerated from the sphinx sources.
> 
> Patch 10: the ChangeLog.jit logs from the branch.

For some reason, patches 8 and 9 don't seem to have made it through to
the list, even after a couple of attempts.

Here's the ChangeLog for patch 8:

gcc/jit/ChangeLog:

        * docs/Makefile: New.
        * docs/conf.py: New.
        * docs/examples/install-hello-world.c: New.
        * docs/examples/tut01-square.c: New.
        * docs/examples/tut02-sum-of-squares.c: New.
        * docs/examples/tut03-toyvm/Makefile: New.
        * docs/examples/tut03-toyvm/factorial.toy: New.
        * docs/examples/tut03-toyvm/fibonacci.toy: New.
        * docs/examples/tut03-toyvm/toyvm.c: New.
        * docs/index.rst: New.
        * docs/internals/index.rst: New.
        * docs/intro/factorial.png: New.
        * docs/intro/index.rst: New.
        * docs/intro/install.rst: New.
        * docs/intro/sum-of-squares.png: New.
        * docs/intro/tutorial01.rst: New.
        * docs/intro/tutorial02.rst: New.
        * docs/intro/tutorial03.rst: New.
        * docs/topics/contexts.rst: New.
        * docs/topics/expressions.rst: New.
        * docs/topics/functions.rst: New.
        * docs/topics/index.rst: New.
        * docs/topics/locations.rst: New.
        * docs/topics/objects.rst: New.
        * docs/topics/results.rst: New.
        * docs/topics/types.rst: New.

The new files can be seen at:
https://gcc.gnu.org/git/?p=gcc.git;a=tree;f=gcc/jit/docs;h=a31ffa5d5068f088a314bad3f904887f6dfa3098

HTML built from these (via sphinx) can be seen at:
https://dmalcolm.fedorapeople.org/gcc/libgccjit-api-docs/index.html

Re: [PATCH 08/10] Documentation for the JIT library (Re: Patches 5-10 of jit merger)

[Jeff Law]

On 10/15/14 10:56, David Malcolm wrote:
> On Tue, 2014-10-14 at 11:09 -0400, David Malcolm wrote:
>> On Mon, 2014-10-13 at 13:45 -0400, David Malcolm wrote:
>>> I'd like to merge the JIT branch into trunk:
>>>    https://gcc.gnu.org/wiki/JIT
>>>
>>> This is "v2" since it incorporates fixes for the various issues
>>> identified by Joseph in an earlier submission:
>>>    https://gcc.gnu.org/ml/gcc-patches/2014-09/msg02056.html
>>>
>>> I've split up the current diff between trunk and the branch into 5
>>> areas for ease of review (and to allow for early merger of the
>>> supporting work, if it's deemed ready):
>>>
>>> patch 1: exposes an entrypoint in libiberty that I need
>>> patch 2: configure and Makefile changes in "gcc"
>>> patch 3: timevar.h: Add an auto_timevar class
>>> patch 4: State cleanups in "gcc"
>>> patch 5: Add the "jit" code itself
>>>
>>> [this is a diff of trunk r215958 aka
>>> e012cdc775868e9922f5fef9068a764546876d93 which is from 2014-10-06,
>>> vs jit branch version 75b3ee7acdc6de55354d65bb7d619386463e50a1].
>>>
>>> I've successfully bootstrapped and regression-tested the cumulative
>>> result of all of the patches against a control build, building them
>>> both with --enable-host-shared, and with
>>>    --enable-languages=c,c++,objc,obj-c++,java,fortran,ada,go,lto
>>> adding ",jit" to the test build (both on x86_64-unknown-linux-gnu;
>>> Fedora 20).
>>>
>>> There were no regressions vs the control build, and the patched build
>>> gains a jit.sum, with 4663 passes (and no failures).
>>>
>>> OK for trunk?
>>
>> Patch 5 seems to have been too large, even compressed, so I'm breaking
>> it up into separate pieces and compressing, giving 10 patches in total
>>
>> Patches 1-4 are as above.
>>
>> Patch 5: remaining JIT-related changes outside of the gcc/jit/ subdir
>>
>> Patch 6: the core of the JIT implementation: the gcc/jit subdir
>>
>> Patch 7: the testsuite: gcc/testsuite/jit.dg
>>
>> Patch 8: sphinx-based documentation: the gcc/jit/docs subdir
>>
>> Patch 9: texinfo documentation autogenerated from the sphinx sources.
>>
>> Patch 10: the ChangeLog.jit logs from the branch.
>
> For some reason, patches 8 and 9 don't seem to have made it through to
> the list, even after a couple of attempts.
>
> Here's the ChangeLog for patch 8:
>
> gcc/jit/ChangeLog:
>
>          * docs/Makefile: New.
>          * docs/conf.py: New.
>          * docs/examples/install-hello-world.c: New.
>          * docs/examples/tut01-square.c: New.
>          * docs/examples/tut02-sum-of-squares.c: New.
>          * docs/examples/tut03-toyvm/Makefile: New.
>          * docs/examples/tut03-toyvm/factorial.toy: New.
>          * docs/examples/tut03-toyvm/fibonacci.toy: New.
>          * docs/examples/tut03-toyvm/toyvm.c: New.
>          * docs/index.rst: New.
>          * docs/internals/index.rst: New.
>          * docs/intro/factorial.png: New.
>          * docs/intro/index.rst: New.
>          * docs/intro/install.rst: New.
>          * docs/intro/sum-of-squares.png: New.
>          * docs/intro/tutorial01.rst: New.
>          * docs/intro/tutorial02.rst: New.
>          * docs/intro/tutorial03.rst: New.
>          * docs/topics/contexts.rst: New.
>          * docs/topics/expressions.rst: New.
>          * docs/topics/functions.rst: New.
>          * docs/topics/index.rst: New.
>          * docs/topics/locations.rst: New.
>          * docs/topics/objects.rst: New.
>          * docs/topics/results.rst: New.
>          * docs/topics/types.rst: New.
>
> The new files can be seen at:
> https://gcc.gnu.org/git/?p=gcc.git;a=tree;f=gcc/jit/docs;h=a31ffa5d5068f088a314bad3f904887f6dfa3098
>
> HTML built from these (via sphinx) can be seen at:
> https://dmalcolm.fedorapeople.org/gcc/libgccjit-api-docs/index.html
So I'm reading this and the first thing that comes to mind is *way* too 
much on installation via packages.  That documentation may be 
appropriate elsewhere, but I don't think it makes sense in GCC.

As far as installation from source, assuming the bits are integrated 
onto the trunk, this section should be dramatically simplified and in 
fact ought to be removed.

For the examples, assume folks have built from source and installed into 
whatever prefix they selected.  ie, I thik that section becomes

gcc jit-hello-world.c -o jit-hello-world -lgccjit

All the other stuff is unwanted/unneeded.  No need to talk about running 
from different prefixes, locally built trees.  Remove the pkg-config 
stuff, that's not a way we do things in the gcc-world.  If you want to 
make an argument for pkg-config, then that's independent of this work 
and should be made pervasive in GCC if we were to decide to go that 
route.  Having it just for the plugin is inconsistent and confusing.

Similarly for the stuff for how to use a working copy without 
installing.  Might be useful to document somewhere, but for GCC, we 
expect folks to build & install.   Running the testsuite should just be 
make check, which is no different than other parts of GCC and I don't 
think you need a page on how to do that.


So I think the general comment is assume the normal build/install 
scenario, removing all the stuff that becomes extraneous.  Assume your 
target audience isn't hacking GCC itself and simplify appropriately.

If you want to pull those developer-oriented docs somewhere on the GCC 
web site, I think we can find a way to make that happen, but I don't 
think all that stuf needs to be in the GCC manual.

Jeff






>
>

[jit] Update the docs

[David Malcolm]

On Wed, 2014-10-15 at 14:44 -0600, Jeff Law wrote:
> On 10/15/14 10:56, David Malcolm wrote:
> > On Tue, 2014-10-14 at 11:09 -0400, David Malcolm wrote:
> >> On Mon, 2014-10-13 at 13:45 -0400, David Malcolm wrote:
> >>> I'd like to merge the JIT branch into trunk:
> >>>    https://gcc.gnu.org/wiki/JIT
> >>>
> >>> This is "v2" since it incorporates fixes for the various issues
> >>> identified by Joseph in an earlier submission:
> >>>    https://gcc.gnu.org/ml/gcc-patches/2014-09/msg02056.html
> >>>
> >>> I've split up the current diff between trunk and the branch into 5
> >>> areas for ease of review (and to allow for early merger of the
> >>> supporting work, if it's deemed ready):
> >>>
> >>> patch 1: exposes an entrypoint in libiberty that I need
> >>> patch 2: configure and Makefile changes in "gcc"
> >>> patch 3: timevar.h: Add an auto_timevar class
> >>> patch 4: State cleanups in "gcc"
> >>> patch 5: Add the "jit" code itself
> >>>
> >>> [this is a diff of trunk r215958 aka
> >>> e012cdc775868e9922f5fef9068a764546876d93 which is from 2014-10-06,
> >>> vs jit branch version 75b3ee7acdc6de55354d65bb7d619386463e50a1].
> >>>
> >>> I've successfully bootstrapped and regression-tested the cumulative
> >>> result of all of the patches against a control build, building them
> >>> both with --enable-host-shared, and with
> >>>    --enable-languages=c,c++,objc,obj-c++,java,fortran,ada,go,lto
> >>> adding ",jit" to the test build (both on x86_64-unknown-linux-gnu;
> >>> Fedora 20).
> >>>
> >>> There were no regressions vs the control build, and the patched build
> >>> gains a jit.sum, with 4663 passes (and no failures).
> >>>
> >>> OK for trunk?
> >>
> >> Patch 5 seems to have been too large, even compressed, so I'm breaking
> >> it up into separate pieces and compressing, giving 10 patches in total
> >>
> >> Patches 1-4 are as above.
> >>
> >> Patch 5: remaining JIT-related changes outside of the gcc/jit/ subdir
> >>
> >> Patch 6: the core of the JIT implementation: the gcc/jit subdir
> >>
> >> Patch 7: the testsuite: gcc/testsuite/jit.dg
> >>
> >> Patch 8: sphinx-based documentation: the gcc/jit/docs subdir
> >>
> >> Patch 9: texinfo documentation autogenerated from the sphinx sources.
> >>
> >> Patch 10: the ChangeLog.jit logs from the branch.
> >
> > For some reason, patches 8 and 9 don't seem to have made it through to
> > the list, even after a couple of attempts.
> >
> > Here's the ChangeLog for patch 8:
> >
> > gcc/jit/ChangeLog:
> >
> >          * docs/Makefile: New.
> >          * docs/conf.py: New.
> >          * docs/examples/install-hello-world.c: New.
> >          * docs/examples/tut01-square.c: New.
> >          * docs/examples/tut02-sum-of-squares.c: New.
> >          * docs/examples/tut03-toyvm/Makefile: New.
> >          * docs/examples/tut03-toyvm/factorial.toy: New.
> >          * docs/examples/tut03-toyvm/fibonacci.toy: New.
> >          * docs/examples/tut03-toyvm/toyvm.c: New.
> >          * docs/index.rst: New.
> >          * docs/internals/index.rst: New.
> >          * docs/intro/factorial.png: New.
> >          * docs/intro/index.rst: New.
> >          * docs/intro/install.rst: New.
> >          * docs/intro/sum-of-squares.png: New.
> >          * docs/intro/tutorial01.rst: New.
> >          * docs/intro/tutorial02.rst: New.
> >          * docs/intro/tutorial03.rst: New.
> >          * docs/topics/contexts.rst: New.
> >          * docs/topics/expressions.rst: New.
> >          * docs/topics/functions.rst: New.
> >          * docs/topics/index.rst: New.
> >          * docs/topics/locations.rst: New.
> >          * docs/topics/objects.rst: New.
> >          * docs/topics/results.rst: New.
> >          * docs/topics/types.rst: New.
> >
> > The new files can be seen at:
> > https://gcc.gnu.org/git/?p=gcc.git;a=tree;f=gcc/jit/docs;h=a31ffa5d5068f088a314bad3f904887f6dfa3098
> >
> > HTML built from these (via sphinx) can be seen at:
> > https://dmalcolm.fedorapeople.org/gcc/libgccjit-api-docs/index.html
> So I'm reading this and the first thing that comes to mind is *way* too 
> much on installation via packages.  That documentation may be 
> appropriate elsewhere, but I don't think it makes sense in GCC.

I've moved all info on installation via packages to the wiki.

> As far as installation from source, assuming the bits are integrated 
> onto the trunk, this section should be dramatically simplified and in 
> fact ought to be removed.

I've eliminated much of this, moving what remains to the final page of
the docs, which is aimed at contributors (and, in particular, myself).

> For the examples, assume folks have built from source and installed into 
> whatever prefix they selected.  ie, I thik that section becomes
> 
> gcc jit-hello-world.c -o jit-hello-world -lgccjit
> 
> All the other stuff is unwanted/unneeded.  No need to talk about running 
> from different prefixes, locally built trees.  Remove the pkg-config 
> stuff, that's not a way we do things in the gcc-world.  If you want to 
> make an argument for pkg-config, then that's independent of this work 
> and should be made pervasive in GCC if we were to decide to go that 
> route.  Having it just for the plugin is inconsistent and confusing.

I'll make that argument another time.  For now, I've removed it for the
sake of getting the code into trunk.  In the patch below I've removed
mentions of it from the docs.

> Similarly for the stuff for how to use a working copy without 
> installing.  Might be useful to document somewhere, but for GCC, we 
> expect folks to build & install.   Running the testsuite should just be 
> make check, which is no different than other parts of GCC

I've updated the testsuite to no longer need an install, so I've updated
the final page of the docs to reflect that.

> and I don't  think you need a page on how to do that.

Sadly, *I* do need such a page :)

This may turn into a rant; sorry...

IIRC, it took me several hours way back when to figure out how to just
run one specific testcase (as opposed to the whole suite), and I can
never remember the right incantation, so I wrote it up on the wiki page,
and now just go there when I find myself wondering the right incantation.
I can never find things in the .texi files or the HTML built from them.

So right now there *is* a page that I use to tell me how to do that, but
it's on the wiki...

Having vital documentation live on a *wiki* disturbs me.

There's an excellent talk from Jacob Kaplan-Moss:
"Writing Great Documentation" (PyCon Atlanta 2011):
  https://www.youtube.com/watch?v=z3fRu9pkuXE
with the quote:
  "a wiki is where good documentation goes to die"

I recommend watching the whole talk (e.g. for the material about how
good documentation is fractal in nature [1]).

I have a pet theory that any time that I have to use a wiki can be
seen as a symptom of dysfunction in a project - why am I editing a
wiki, rather that a file in a source tree?  Some of the reasons might
be (not singling out gcc here):
* a project has chosen an overly cumbersome source representation for
  its docs, and is using a wiki to avoid e.g. having to type DocBook
  (I speak as a reformed former DocBook enthusiast).
* a project doesn't have a good place to put said documentation (e.g. we
  have a user manual and an installation manual; do we have a contributor
  manual?  Though why make such distinctions?  Maybe we can build a
  smoother on-ramp for turning users into contributors?)
...etc

So I need somewhere to put this documentation, and I'd rather take the
best parts of the wiki page and put them in a contributor guide.  Given
that this is Sphinx-based, there's already a degree of separation
between these docs and the existing GCC docs - this isn't going to
be a part of the GCC manual at this time, it's going to be a sibling
document.  Hence I think it's reasonable for this guide to cover both
users of *and* contributors to the JIT library, with the
contributor-focussed content isolated from the user-focussed content
as a appendix.

There's also what I call the "man page disease", where documentation
is written as if only saying anything once is a virtue.  I believe
most developers only read small parts of documentation based on skimming
and on the pages they see in internet search results, hence I believe
redundancy is a virtue: it's OK to repeat ourselves, to restate things
with examples for clarity.  This may be a generation-gap thing.

A use-case for the contributor-focussed content is for things like
"I'm debugging a local build of GNU Octave, running against a local
(not-installed) build of libgccjit, and need to make a one-line change
to libgccjit.  What exactly needs to be set in the environment when
invoking gdb?"  It's painful figuring this out by hand, and it needs
to be captured in the docs somewhere.

OK, rant over :)

> So I think the general comment is assume the normal build/install 
> scenario, removing all the stuff that becomes extraneous.  Assume your 
> target audience isn't hacking GCC itself and simplify appropriately.
> 
> If you want to pull those developer-oriented docs somewhere on the GCC 
> web site, I think we can find a way to make that happen, but I don't 
> think all that stuf needs to be in the GCC manual.

The following patch:

  * eliminates all mentions of pkg-config

  * eliminates the installation section and the discussion of packages

  * converts the final page (docs/internals/index.rst) into a short
    guide for contributors to the project.  I think having it next
    to the rest of the docs is reasonable.

I've committed it to the dmalcolm/jit branch, and prebuilt HTML can be seen
at:
  https://dmalcolm.fedorapeople.org/gcc/libgccjit-api-docs/index.html
as before.  The significant changes can be seen on the first page of the
tutorial:
  https://dmalcolm.fedorapeople.org/gcc/libgccjit-api-docs/intro/tutorial01.html
and in the final page of the docs:
  https://dmalcolm.fedorapeople.org/gcc/libgccjit-api-docs/internals/index.html
which as I said above is a compendium of contributor docs, of which
I'm the primary consumer.

Is this more acceptable?

After this patch, the "intro/install.rst" becomes a poor name.  I've
renamed it in a followup patch to "intro/tutorial01.rst".

[1] 21 minutes in fwiw, but really, watch it all, it's a great talk.

Committed to branch dmalcolm/jit:

gcc/jit/ChangeLog.jit:
	* docs/intro/install.rst ("Installation via packages"): Drop
	this section.
	("Installation from source"): Drop this section, moving parts
	of it to https://gcc.gnu.org/wiki/JIT and some others to
	docs/internals/index.rst.
	("Hello world"): This section becomes the only remaining part
	of this file.  Eliminate references to pkg-config.

	* docs/internals/index.rst
	("Using a working copy without installing every time"): Rewrite
	as...
	("Working on the JIT library"): ...new section, aimed at
	contributors (and myself) working from a build directory,
	eliminating references to installation.  Add description
	of pertinent configuration options.
	("Running the test suite"): Add setting of LIBRARY_PATH to
	description of how to run a built binary outside of the test
	suite.
	("Environment variables"): New section, describing pertinent
	environment variables.
---
 gcc/jit/ChangeLog.jit            |  23 +++++
 gcc/jit/docs/internals/index.rst | 133 +++++++++++++++++++-------
 gcc/jit/docs/intro/install.rst   | 201 +++------------------------------------
 3 files changed, 132 insertions(+), 225 deletions(-)

diff --git a/gcc/jit/ChangeLog.jit b/gcc/jit/ChangeLog.jit
index 02664f0..ab09b9d 100644
--- a/gcc/jit/ChangeLog.jit
+++ b/gcc/jit/ChangeLog.jit
@@ -1,3 +1,26 @@
+2014-10-21  David Malcolm  <dmalcolm@redhat.com>
+
+	* docs/intro/install.rst ("Installation via packages"): Drop
+	this section.
+	("Installation from source"): Drop this section, moving parts
+	of it to https://gcc.gnu.org/wiki/JIT and some others to
+	docs/internals/index.rst.
+	("Hello world"): This section becomes the only remaining part
+	of this file.  Eliminate references to pkg-config.
+
+	* docs/internals/index.rst
+	("Using a working copy without installing every time"): Rewrite
+	as...
+	("Working on the JIT library"): ...new section, aimed at
+	contributors (and myself) working from a build directory,
+	eliminating references to installation.  Add description
+	of pertinent configuration options.
+	("Running the test suite"): Add setting of LIBRARY_PATH to
+	description of how to run a built binary outside of the test
+	suite.
+	("Environment variables"): New section, describing pertinent
+	environment variables.
+
 2014-10-20  David Malcolm  <dmalcolm@redhat.com>
 
 	* jit-recording.c (gcc::jit::dump::dump): Handle fopen failures
diff --git a/gcc/jit/docs/internals/index.rst b/gcc/jit/docs/internals/index.rst
index 1e3952c..80626e4 100644
--- a/gcc/jit/docs/internals/index.rst
+++ b/gcc/jit/docs/internals/index.rst
@@ -18,20 +18,10 @@
 Internals
 =========
 
-Using a working copy without installing every time
---------------------------------------------------
-When directly working on the library you can avoid needing to install to
-test every change.
-
-You need to do a ``make install`` of the ``gcc`` subdirectory to install
-the driver binary (the top-level ``gcc`` binary).  This is used internally
-by the library for converting from .s assembler files to .so shared
-libraries.  Specifically, it looks for an executable on the ``$PATH`` with
-a name expanded by the ``configure`` script from
-``${target_noncanonical}-gcc-${gcc_BASEVER}${exeext}``,
-such as ``x86_64-unknown-linux-gnu-gcc-5.0.0``.
-
-For example, if you configured with a prefix of ``$PREFIX`` like this:
+Working on the JIT library
+--------------------------
+Having checked out the source code (to "src"), you can configure and build
+the JIT library like this:
 
 .. code-block:: bash
 
@@ -45,39 +35,44 @@ For example, if you configured with a prefix of ``$PREFIX`` like this:
      --disable-bootstrap \
      --enable-checking=release \
      --prefix=$PREFIX
+  nice make -j4 # altering the "4" to however many cores you have
 
-then you can install (once) to ensure that ``$PREFIX/bin/`` is populated:
+This should build a libgccjit.so within jit/build/gcc:
 
 .. code-block:: console
 
-  [build]$ ll ../install/bin/*gcc*
-  -rwxr-xr-x. 3 david david 2733458 Oct  6 14:25 ../install/bin/gcc
-  -rwxr-xr-x. 2 david david  136921 Oct  6 14:25 ../install/bin/gcc-ar
-  -rwxr-xr-x. 2 david david  136857 Oct  6 14:25 ../install/bin/gcc-nm
-  -rwxr-xr-x. 2 david david  136869 Oct  6 14:25 ../install/bin/gcc-ranlib
-  -rwxr-xr-x. 3 david david 2733458 Oct  6 14:25 ../install/bin/x86_64-unknown-linux-gnu-gcc
-  -rwxr-xr-x. 3 david david 2733458 Oct  6 14:25 ../install/bin/x86_64-unknown-linux-gnu-gcc-5.0.0
-  -rwxr-xr-x. 2 david david  136921 Oct  6 14:25 ../install/bin/x86_64-unknown-linux-gnu-gcc-ar
-  -rwxr-xr-x. 2 david david  136857 Oct  6 14:25 ../install/bin/x86_64-unknown-linux-gnu-gcc-nm
-  -rwxr-xr-x. 2 david david  136869 Oct  6 14:25 ../install/bin/x86_64-unknown-linux-gnu-gcc-ranlib
+ [build] $ file gcc/libgccjit.so*
+ gcc/libgccjit.so:       symbolic link to `libgccjit.so.0'
+ gcc/libgccjit.so.0:     symbolic link to `libgccjit.so.0.0.1'
+ gcc/libgccjit.so.0.0.1: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
 
-Note the presence above of ``../install/bin/x86_64-unknown-linux-gnu-gcc``.
+Here's what those configuration options mean:
 
-When building code using the API you need to ensure that ``-I`` points to
-the directory containing ``libgccjit.h`` and ``-L`` points to the
-directory containing the built library.
+.. option:: --enable-host-shared
 
-When running binaries (or debugging them), you'll need to manually set
-``LD_LIBRARY_PATH`` to the directory containing ``libgccjit.so``, and
-``PATH`` needs to contain the path to the installed binaries.
+  Configuring with this option means that the compiler is built as
+  position-independent code, which incurs a slight performance hit,
+  but it necessary for a shared library.
 
-and then you can run from a built (but not installed) copy:
+.. option:: --enable-languages=jit
 
-.. code-block:: console
+  This specifies which frontends to build.  The JIT library looks like
+  a frontend to the rest of the code.
+
+.. option:: --disable-bootstrap
+
+  For hacking on the "jit" subdirectory, performing a full
+  bootstrap can be overkill, since it's unused by a bootstrap.  However,
+  when submitting patches, you should remove this option, to ensure that
+  the compiler can still bootstrap itself.
 
-  [gcc]$ PATH=../../install/bin:$PATH LD_LIBRARY_PATH=. ./testsuite/jit/test-factorial.exe
+.. option:: --enable-checking=release
 
-without needing to reinstall everything for every tweak to the library.
+  The compile can perform extensive self-checking as it runs, useful when
+  debugging, but slowing things down.
+
+  For maximum speed, configure with ``--enable-checking=release`` to
+  disable this self-checking.
 
 Running the test suite
 ----------------------
@@ -117,11 +112,75 @@ and once a test has been compiled, you can debug it directly:
 
 .. code-block:: console
 
-   [gcc] $ PATH=../../install/bin:$PATH \
+   [gcc] $ PATH=.:$PATH \
            LD_LIBRARY_PATH=. \
+           LIBRARY_PATH=. \
              gdb --args \
                testsuite/jit/test-factorial.exe
 
+Environment variables
+---------------------
+When running client code against a locally-built libgccjit, three
+environment variables need to be set up:
+
+.. envvar:: LD_LIBRARY_PATH
+
+   `libgccjit.so` is dynamically linked into client code, so if running
+   against a locally-built library, ``LD_LIBRARY_PATH`` needs to be set
+   up appropriately.  The library can be found within the "gcc"
+   subdirectory of the build tree:
+
+  .. code-block:: console
+
+    $ file libgccjit.so*
+    libgccjit.so:       symbolic link to `libgccjit.so.0'
+    libgccjit.so.0:     symbolic link to `libgccjit.so.0.0.1'
+    libgccjit.so.0.0.1: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, not stripped
+
+.. envvar:: PATH
+
+  The library uses a driver executable for converting from .s assembler
+  files to .so shared libraries.  Specifically, it looks for a name
+  expanded from
+  ``${target_noncanonical}-gcc-${gcc_BASEVER}${exeext}``
+  such as ``x86_64-unknown-linux-gnu-gcc-5.0.0``.
+
+  Hence ``PATH`` needs to include a directory where the library can
+  locate this executable.
+
+  The executable is normally installed to the installation bindir
+  (e.g. /usr/bin), but a copy is also created within the "gcc"
+  subdirectory of the build tree for running the testsuite, and for ease
+  of development.
+
+.. envvar:: LIBRARY_PATH
+
+  The driver executable invokes the linker, and the latter needs to locate
+  support libraries needed by the generated code, or you will see errors
+  like:
+
+  .. code-block:: console
+
+    ld: cannot find crtbeginS.o: No such file or directory
+    ld: cannot find -lgcc
+    ld: cannot find -lgcc_s
+
+  Hence if running directly from a locally-built copy (without installing),
+  ``LIBRARY_PATH`` needs to contain the "gcc" subdirectory of the build
+  tree.
+
+For example, to run a binary that uses the library against a non-installed
+build of the library in LIBGCCJIT_BUILD_DIR you need an invocation of the
+client code like this, to preprend the dir to each of the environment
+variables:
+
+.. code-block:: console
+
+  $ LD_LIBRARY_PATH=$(LIBGCCJIT_BUILD_DIR):$(LD_LIBRARY_PATH) \
+    PATH=$(LIBGCCJIT_BUILD_DIR):$(PATH) \
+    LIBRARY_PATH=$(LIBGCCJIT_BUILD_DIR):$(LIBRARY_PATH) \
+      ./jit-hello-world
+  hello world
 
 Overview of code structure
 --------------------------
diff --git a/gcc/jit/docs/intro/install.rst b/gcc/jit/docs/intro/install.rst
index a0641b7..3f61684 100644
--- a/gcc/jit/docs/intro/install.rst
+++ b/gcc/jit/docs/intro/install.rst
@@ -17,153 +17,25 @@
 
 .. default-domain:: c
 
-Getting Started
----------------
+Getting Started: "Hello world"
+==============================
 
-Installation via packages
-=========================
-
-It's easiest to use pre-built packages of the library.
-
-Fedora and RHEL
-***************
-RPM packages of libgccjit (and its Python 2 and 3 bindings) are
-available for Fedora and RHEL, for i386 and x86_64.
-
-These currently should be treated as experimental.
-
-See https://copr.fedoraproject.org/coprs/dmalcolm/libgccjit/
-for information on subscribing to the appropriate repository for
-your system.  Having done this,
-
-.. code-block:: bash
-
-  sudo yum install libgccjit-devel
-
-should give you both the JIT library (`libgccjit`) and the header files
-needed to develop against it (`libgccjit-devel`):
-
-.. code-block:: console
-
-  $ rpm -qlv libgccjit
-  lrwxrwxrwx    1 root    root       18 Aug 12 07:56 /usr/lib64/libgccjit.so.0 -> libgccjit.so.0.0.1
-  -rwxr-xr-x    1 root    root 14463448 Aug 12 07:57 /usr/lib64/libgccjit.so.0.0.1
-
-  $ rpm -qlv libgccjit-devel
-  -rwxr-xr-x    1 root    root    37654 Aug 12 07:56 /usr/include/libgccjit++.h
-  -rwxr-xr-x    1 root    root    28967 Aug 12 07:56 /usr/include/libgccjit.h
-  lrwxrwxrwx    1 root    root       14 Aug 12 07:56 /usr/lib64/libgccjit.so -> libgccjit.so.0
-
-
-Other distributions
-*******************
-
-Prebuilt packages for other distributions would be most welcome; please
-contact the `jit mailing list`_.
-
-
-Installation from source
-========================
-If pre-built packages are not available, you can built the library from
-source.  Doing so currently requires about 4.2G of drive space (for
-the combination of the source tree, the build directory, and the
-installation path).
-
-The code can currently be seen within the git branch "dmalcolm/jit" here:
-  http://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/heads/dmalcolm/jit
-
-The following will check out a copy of the appropriate branch, to the
-"jit/src" subdirectory:
-
-.. code-block:: bash
-
-  mkdir jit
-  cd jit
-  git clone \
-      -b dmalcolm/jit \
-       git://gcc.gnu.org/git/gcc.git \
-       src
-
-The source tree currently occupies about 2.8G of disk space.
-
-To build it (within the "jit/build" subdirectory, installing to
-"jit/install"):
-
-.. code-block:: bash
-
-  mkdir build
-  mkdir install
-  PREFIX=$(pwd)/install
-  cd build
-  ../src/configure \
-     --enable-host-shared \
-     --enable-languages=jit \
-     --disable-bootstrap \
-     --enable-checking=release \
-     --prefix=$PREFIX
-  nice make -j4 # altering the "4" to however many cores you have
-
-On my 4-core laptop this takes 17 minutes and 1.1G of disk space
-(it's much faster with many cores and a corresponding -j setting).
-
-This should build a libgccjit.so within jit/build/gcc:
-
-.. code-block:: console
-
- [build] $ file gcc/libgccjit.so*
- gcc/libgccjit.so:       symbolic link to `libgccjit.so.0'
- gcc/libgccjit.so.0:     symbolic link to `libgccjit.so.0.0.1'
- gcc/libgccjit.so.0.0.1: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
-
-Note that this is a branch of GCC, so if it fails to build, you might want
-to consult the `general GCC FAQ <https://gcc.gnu.org/wiki/FAQ>`_  for some
-common issues, before checking on the `jit mailing list`_.
-
-.. _jit mailing list: https://gcc.gnu.org/ml/jit/
-
-You should then be able to install it (to the `--prefix` specified
-earlier) via:
-
-.. code-block:: bash
-
-  make install
-
-On my laptop this uses a further 0.4G of disk space.
-
-You should be able to see the header files within the `include`
-subdirectory of the installation prefix:
-
-.. code-block:: console
-
-  $ find $PREFIX/include
-  /home/david/gcc-jit/install/include
-  /home/david/gcc-jit/install/include/libgccjit.h
-  /home/david/gcc-jit/install/include/libgccjit++.h
-
-and the library within the `lib` subdirectory:
-
-.. code-block:: console
-
-  $ find $PREFIX/lib/libgccjit.*
-  /home/david/gcc-jit/install/lib/libgccjit.so
-  /home/david/gcc-jit/install/lib/libgccjit.so.0
-  /home/david/gcc-jit/install/lib/libgccjit.so.0.0.1
-
-
-"Hello world"
-=============
-
-Let's look at how to build and run programs that use the library.
+Before we look at the details of the API, let's look at building and
+running programs that use the library.
 
 Here's a toy "hello world" program that uses the library to synthesize
-a call to `printf` and use it to write a message to stdout.
+a call to `printf` and uses it to write a message to stdout.
+
+Don't worry about the content of the program for now; we'll cover
+the details in later parts of this tutorial.
 
    .. literalinclude:: ../examples/install-hello-world.c
     :language: c
 
-Copy it to `jit-hello-world.c`.
+Copy the above to `jit-hello-world.c`.
 
-To build it with prebuilt packages, use:
+Assuming you have the jit library installed, build the test program
+using:
 
 .. code-block:: console
 
@@ -172,56 +44,9 @@ To build it with prebuilt packages, use:
       -o jit-hello-world \
       -lgccjit
 
-  # Run the built program:
-  $ ./jit-hello-world
-  hello world
-
-
-If building against an locally-built install (to `$PREFIX`), you can use
-`pkg-config <http://www.freedesktop.org/wiki/Software/pkg-config/>`_ to
-specify the compilation and linkage flags:
+You should then be able to run the built program:
 
 .. code-block:: console
 
-  $ export PKG_CONFIG_PATH=$PREFIX/lib/pkgconfig
-  $ gcc \
-      jit-hello-world.c \
-      -o jit-hello-world \
-      $(pkg-config libgccjit --cflags --libs)
-
-This is equivalent to handcoding the include and library paths with `-I`
-and `-L` and specifying `-lgccjit` (i.e. linkage against libgccjit):
-
-.. code-block:: console
-
-  $ gcc \
-      jit-hello-world.c \
-      -o jit-hello-world \
-      -lgccjit \
-      -I$PREFIX/include -L$PREFIX/lib
-
-When running the built test program against a locally-built tree, two
-environment variables need to be set up:
-
-* `LD_LIBRARY_PATH` needs to be set up appropriately so that the dynamic
-  linker can locate the `libgccjit.so`
-
-* `PATH` needs to include the `bin` subdirectory below the installation
-  prefix, so that the library can locate a driver binary.  This is used
-  internally by the library for converting from .s assembler files to
-  .so shared libraries.
-
-  ..
-     Specifically, it looks for a name expanded from
-     ``${target_noncanonical}-gcc-${gcc_BASEVER}${exeext}`` on the
-     ``$PATH``, such as ``x86_64-unknown-linux-gnu-gcc-5.0.0``).
-
-For example, if you configured with a prefix of ``$PREFIX`` like above,
-you need an invocation like this:
-
-.. code-block:: console
-
-  $ LD_LIBRARY_PATH=$PREFIX/lib:$LD_LIBRARY_PATH \
-    PATH=$PREFIX/bin:$PATH \
-    ./jit-hello-world
+  $ ./jit-hello-world
   hello world
-- 
1.7.11.7

[PATCH 09/10] Prebuilt texinfo documentation for the JIT library (Re: Patches 5-10 of jit merger)

[David Malcolm]

On Tue, 2014-10-14 at 11:09 -0400, David Malcolm wrote:
> On Mon, 2014-10-13 at 13:45 -0400, David Malcolm wrote:
> > I'd like to merge the JIT branch into trunk:
> >   https://gcc.gnu.org/wiki/JIT
> > 
> > This is "v2" since it incorporates fixes for the various issues
> > identified by Joseph in an earlier submission:
> >   https://gcc.gnu.org/ml/gcc-patches/2014-09/msg02056.html
> > 
> > I've split up the current diff between trunk and the branch into 5
> > areas for ease of review (and to allow for early merger of the
> > supporting work, if it's deemed ready):
> > 
> > patch 1: exposes an entrypoint in libiberty that I need
> > patch 2: configure and Makefile changes in "gcc"
> > patch 3: timevar.h: Add an auto_timevar class
> > patch 4: State cleanups in "gcc"
> > patch 5: Add the "jit" code itself
> > 
> > [this is a diff of trunk r215958 aka
> > e012cdc775868e9922f5fef9068a764546876d93 which is from 2014-10-06,
> > vs jit branch version 75b3ee7acdc6de55354d65bb7d619386463e50a1].
> > 
> > I've successfully bootstrapped and regression-tested the cumulative
> > result of all of the patches against a control build, building them
> > both with --enable-host-shared, and with
> >   --enable-languages=c,c++,objc,obj-c++,java,fortran,ada,go,lto
> > adding ",jit" to the test build (both on x86_64-unknown-linux-gnu;
> > Fedora 20).
> > 
> > There were no regressions vs the control build, and the patched build
> > gains a jit.sum, with 4663 passes (and no failures).
> > 
> > OK for trunk?
> 
> Patch 5 seems to have been too large, even compressed, so I'm breaking
> it up into separate pieces and compressing, giving 10 patches in total
> 
> Patches 1-4 are as above.
> 
> Patch 5: remaining JIT-related changes outside of the gcc/jit/ subdir
> 
> Patch 6: the core of the JIT implementation: the gcc/jit subdir
> 
> Patch 7: the testsuite: gcc/testsuite/jit.dg
> 
> Patch 8: sphinx-based documentation: the gcc/jit/docs subdir
> 
> Patch 9: texinfo documentation autogenerated from the sphinx sources.
> 
> Patch 10: the ChangeLog.jit logs from the branch.

For some reason, patches 8 and 9 don't seem to have made it through to
the list, even after a couple of attempts.

Here's the ChangeLog for patch 9:

This is for the benefit of those without Sphinx installed, and is
autogenerated by running "make texinfo" in the gcc/jit/docs
subdirectory.

gcc/jit/ChangeLog:

        * docs/_build/texinfo/Makefile: New.
        * docs/_build/texinfo/factorial.png: New.
        * docs/_build/texinfo/libgccjit.texi: New.
        * docs/_build/texinfo/sum-of-squares.png: New.

The new files can be seen at:
https://gcc.gnu.org/git/?p=gcc.git;a=tree;f=gcc/jit/docs/_build/texinfo;h=4bfa8d323708ba7189fdf74532c1aa160f13f4b9

Re: [PATCH 09/10] Prebuilt texinfo documentation for the JIT library (Re: Patches 5-10 of jit merger)

[Jeff Law]

On 10/15/14 10:59, David Malcolm wrote:
> On Tue, 2014-10-14 at 11:09 -0400, David Malcolm wrote:
>> On Mon, 2014-10-13 at 13:45 -0400, David Malcolm wrote:
>>> I'd like to merge the JIT branch into trunk:
>>>    https://gcc.gnu.org/wiki/JIT
>>>
>>> This is "v2" since it incorporates fixes for the various issues
>>> identified by Joseph in an earlier submission:
>>>    https://gcc.gnu.org/ml/gcc-patches/2014-09/msg02056.html
>>>
>>> I've split up the current diff between trunk and the branch into 5
>>> areas for ease of review (and to allow for early merger of the
>>> supporting work, if it's deemed ready):
>>>
>>> patch 1: exposes an entrypoint in libiberty that I need
>>> patch 2: configure and Makefile changes in "gcc"
>>> patch 3: timevar.h: Add an auto_timevar class
>>> patch 4: State cleanups in "gcc"
>>> patch 5: Add the "jit" code itself
>>>
>>> [this is a diff of trunk r215958 aka
>>> e012cdc775868e9922f5fef9068a764546876d93 which is from 2014-10-06,
>>> vs jit branch version 75b3ee7acdc6de55354d65bb7d619386463e50a1].
>>>
>>> I've successfully bootstrapped and regression-tested the cumulative
>>> result of all of the patches against a control build, building them
>>> both with --enable-host-shared, and with
>>>    --enable-languages=c,c++,objc,obj-c++,java,fortran,ada,go,lto
>>> adding ",jit" to the test build (both on x86_64-unknown-linux-gnu;
>>> Fedora 20).
>>>
>>> There were no regressions vs the control build, and the patched build
>>> gains a jit.sum, with 4663 passes (and no failures).
>>>
>>> OK for trunk?
>>
>> Patch 5 seems to have been too large, even compressed, so I'm breaking
>> it up into separate pieces and compressing, giving 10 patches in total
>>
>> Patches 1-4 are as above.
>>
>> Patch 5: remaining JIT-related changes outside of the gcc/jit/ subdir
>>
>> Patch 6: the core of the JIT implementation: the gcc/jit subdir
>>
>> Patch 7: the testsuite: gcc/testsuite/jit.dg
>>
>> Patch 8: sphinx-based documentation: the gcc/jit/docs subdir
>>
>> Patch 9: texinfo documentation autogenerated from the sphinx sources.
>>
>> Patch 10: the ChangeLog.jit logs from the branch.
>
> For some reason, patches 8 and 9 don't seem to have made it through to
> the list, even after a couple of attempts.
>
> Here's the ChangeLog for patch 9:
>
> This is for the benefit of those without Sphinx installed, and is
> autogenerated by running "make texinfo" in the gcc/jit/docs
> subdirectory.
>
> gcc/jit/ChangeLog:
>
>          * docs/_build/texinfo/Makefile: New.
>          * docs/_build/texinfo/factorial.png: New.
>          * docs/_build/texinfo/libgccjit.texi: New.
>          * docs/_build/texinfo/sum-of-squares.png: New.
>
> The new files can be seen at:
> https://gcc.gnu.org/git/?p=gcc.git;a=tree;f=gcc/jit/docs/_build/texinfo;h=4bfa8d323708ba7189fdf74532c1aa160f13f4b9
Assuming these are built from the sphinx stuff, these are fine once the 
rest of the JIT stuff is approved -- as it updating these things if 
there's any updates necessary to the source sphinx documentation.

jeff

Re: [jit] Update the docs

[Manuel López-Ibáñez]

> IIRC, it took me several hours way back when to figure out how to just
> run one specific testcase (as opposed to the whole suite), and I can
> never remember the right incantation, so I wrote it up on the wiki page,
> and now just go there when I find myself wondering the right incantation.

This is documented in several places:

https://gcc.gnu.org/install/test.html
https://gcc.gnu.org/wiki/Testing_GCC
https://gcc.gnu.org/wiki/HowToPrepareATestcase?highlight=%28RUNTESTFLAGS%29
https://gcc.gnu.org/wiki/TestCaseWriting?highlight=%28RUNTESTFLAGS%29

I'm not saying that you should have found it, but that the information
is not easy to find even if it is there. The question is why?

> I can never find things in the .texi files or the HTML built from them.

Or the ones that we do not build from them, since a lot of important
information is not on the *.texi files but in other HTML pages...

> Having vital documentation live on a *wiki* disturbs me.

Well, it is better than having really outdated documentation or not
having any documentation at all, which is often the case in GCC. I
would like to think of the wiki as a prototyping/staging area.
Unfortunately, right now it is more like a landfill.

> I have a pet theory that any time that I have to use a wiki can be
> seen as a symptom of dysfunction in a project - why am I editing a
> wiki, rather that a file in a source tree?  Some of the reasons might
> be (not singling out gcc here):
> * a project has chosen an overly cumbersome source representation for
>   its docs, and is using a wiki to avoid e.g. having to type DocBook
>   (I speak as a reformed former DocBook enthusiast).
> * a project doesn't have a good place to put said documentation (e.g. we
>   have a user manual and an installation manual; do we have a contributor
>   manual?  Though why make such distinctions?  Maybe we can build a
>   smoother on-ramp for turning users into contributors?)
> ...etc

and I would add (singling out GCC here):

* Documentation spread between webpages, html pages generated from
*.texi and the wiki.

* From all the above, the wiki is in fact the easiest to edit because...

* texi requires review, reviews require someone to review, and if
there are too few people reviewing patches, there are even fewer
people reviewing doc patches. If I find something wrong or that should
be better documented, I need: to write a patch, to write a changelog,
submit it, wait from days to weeks (if ever reviewed), get feedback,
write another patch, perhaps another changelog, submit it, wait from
days to weeks (if ever reviewed)...

* html pages: CVS. Do I need to say more?

* Plus all this: https://gcc.gnu.org/wiki/DocumentationOverviewIssues

Example: if someone says for the nth time "I know my code is undefined
but...". I could fight with CVS, submit a patch for
https://gcc.gnu.org/faq.html, wait wait wait wait wait, get it
reviewed, then CVS, ups I did some html mistake, so it was not
applied, fix it, then CVS, then send the link to the user. Or, I could
directly edit https://gcc.gnu.org/wiki/FAQ#undefinedbut, then send the
link to the user.

And IMHO, the result in the wiki, despite all the shortcomings of
MoinMoin (which are plenty), looks nicer than both the CVS html
webpages and the HTML generated from texi.

I'm not arguing that the reviewers are not doing their job. I fully
understand that they are doing as much as they can and that GCC is
lacking contributors and the situation is not getting better. My
argument would go in the other direction:

* Do not require any pre-commit review, except for major rewrites and additions.

* Do not require changelogs for documentation patches: a commit
message and 'svn log --verbose' should be enough.

* Get rid of CVS.

* Put all HTML pages in SVN under trunk/ and add a commit-hook to
upload it, so there is the shortest delay possible between commit and
showing up online.

* Anything that looks remotely like documentation should come from the
same sources as the real documentation (FAQs, Howtos, etc.), and
hyperlink the hell out of it.

* Add a Getting Started/Contributor manual. Add one per piece of GCC:
Ada, libstdc++, etc...

* Aggressively move things that are nice from the wiki to the
documentation and hyperlink to it.

* Aggressively cleanup the wiki from outdated and incoherent stuff.

In summary, I totally agree with your rant.

>   "a wiki is where good documentation goes to die"

Indeed: https://gcc.gnu.org/wiki/GettingStarted
Indeed2: https://gcc.gnu.org/wiki/Visibility
Indeed3: https://gcc.gnu.org/wiki/LinkTimeOptimizationFAQ
Indeed4: https://gcc.gnu.org/wiki/Math_Optimization_Flags

Cheers,

    Manuel.

Bonus: Try to see how long can you go answering every question about
using/developing GCC with a link to the documentation or the wiki.

Bonusx2: Mentor someone to become a new contributor: Try to answer
each of their questions with a link to the documentation or the wiki.
See how long until they give up.